Running head: QUANTITATIVE RESEARCH STUDY CRITIQUE PAPERS 1 & 2
Quantitative Research Study Critique
Samdaye Neale (Student ID#: 15251790)
NUR 3130 Nursing Research
New York City College of Technology (CUNY)
Abstract
This paper is a student critique of a study article written by Marcia Kirwan, Anne Mathews & P. Anne Scott (2012) titled: The impact of the work environment of nurses on patient safety outcomes: A multi-level modeling approach. The authors undertook a cross-sectional quantitative study that sought to explore the relationship between the ward environment in which nurses practice and specific patient safety outcomes, using ward level variables as well as nurse level variables. The study acknowledged that prior researchers have yielded the result that a positive practice environment aggregated to hospital level has been linked to enhanced safety outcomes; this study was specific to variables at ward level that would reflect a better safety environment of patients.
Problem and Purpose
In their research, Kirwan et al. (2012) focus on patient safety at ward level and is clearly stated under the sub-heading 1.1 Research question. The research question is: “what is the impact of nurse and organizational characteristics aggregated to the ward-level on patient safety outcomes?” There has been many reports on the inadequacy and negligence in providing patient safety in healthcare organizations worldwide and the country of Ireland, where this study was conducted, the healthcare organizations face this challenge with an acknowledged background of very public failures in healthcare provision. The researches contend that central to patient safety strategy internationally is the system approach to safety. The systems approach is dependent on full and open reporting of adverse events, to maximize organizational learning around the incident, and to prevent its occurrence. While this approach to safety in other high risk industries has been widely acknowledged as a success, its suitability to healthcare is often debated. Critique Paper 1.docx
This is followed by sub-heading 1.2 Aim, which gives the purpose of their research – to identify ward level factors in Irish hospitals which impact on nurse-reported patient safety outcomes. The study was funded under the 7th Framework Programme of the European Commission (F7). The 12 country comparative study aimed to examine nursing and the nurse workforce in Europe (Sermeus et al., 2011). A number of investigations and inquiries carried out in Ireland identified common deficiencies in patient safety structures such as, poor communication structures, leadership and teamwork, along with a lack of reporting systems and analysis of adverse events. Insufficient staff knowledge around safety processes, and an acknowledged unsupportive safety culture in healthcare , have been identified as areas to be addressed in the advancement of patient safety (Department of Health and Children, 2008).
The independent variables in the study are the numbers of adverse event reports submitted by nurses, and the nurse work environment in the ward. This is dependent on ward-level nurse factors such as the proportion of nurses educated to degree level and the impact of the nurse work environment on the ward. The researchers clearly stated these variables under the third sub-heading namely, 1.3 Objectives. In this study the nurse wotk environment was measured by the 32 item Practice Environment Scale of Nursing Work Index (PES-NWI). This is the most internationally validated tool for measuring the practice environment of nurses using a number of items on scales and subscales that were scored. However, the nurse work environment has usually been calculated at hospital level (Aiken et al., 1994). Also, previous research has identified a relationship between certain unit or ward level characteristics and levels of adverse event occurrence (Mulvey Boyle, 2004), and between ward variation in the practice environment of nurses and quality of care (Van Bogaert et al., 2010). This research explores this further, in particular looking at the impact of safety outcomes of the aggregated ward-level mean for PES-NWI scores. In international studies, higher education levels have been linked to improved patient outcomes and in this study the proportion of nurses with degrees in the wards varied considerably between wards, from 10% to 100%. Therefore, the researchers created a variable which indicated the proportion of nurses per ward who were educated to degree level, and the impact of this variable on safety outcomes is examined.
The purpose of this research has much significance for Nursing because data were collected and analyzed on a per unit or ward level (as opposed to hospital level as in previous researches). In the United States of America where the original Magnet Hospital studies were conducted, (McClure et al., 1983), little attention was paid to the ward work environment and the researchers of this article contend that patterns at ward level may have been concealed or not acknowledged. The unit or ward is the operational level for nursing groups and the place where patient care is delivered. Characteristics at the ward level that were examined included potential differences across wards in both nurse factors, such as education levels or nurse burnout levels, and in features of the work environment on the ward such as managerial support, team spirit and functioning, and staffing levels. The study shows that higher numbers of reports submitted reflect a more positive attitude to patient safety and a greater understanding of the systems approach to safety and organizational learning.
Ethical approval for the study was obtained from the authors’ institution and in all of the participating hospitals, which comprised 31out of the 32 large acute general hospitals in Ireland and access to wards and nurses was negotiated through direct contact with relevant nurse management. Furthermore, the authors thanked the participants and those who facilitated the research in the study hospitals.
Method
Given the purpose of the research, the most rigorous possible design was used by including all of the large acute general hospitals in Ireland. There was a total of 31 hospitals that participated in the study; two of the hospitals under common management were analyzed as one. Therefore, there were a total of 30 study hospitals. The authors were diligent in their comparison of ward analysis of the practice environment as opposed to the organizational studies that were carried out and reported on by previous studies. A convenience sample of between two and four wards from each hospital was included from 112 wards. All nurses in direct patient care in these wards were eligible to participate. The nurses were surveyed between September 2009 and May 2010. Questionnaires were distributed to wards by the research team and designated link nurses were recruited in hospitals to oversee data collection locally. A researcher visited each hospital, and ward area, at least once during the data collection period and maintained regular telephone contact with ward managers and the designated link nurses at hospital level. This allowed for appropriate comparisons to be made to enhance interpretability of the findings. The results were that 1406 nurses in general medical and surgical wards in 30 hospitals completed questionnaires. In the final analysis for this study, data from 1397 nurses in 108 wards in 30 hospitals was used. The researchers used the maximum number of data collection points available, which resulted in a more accurate reporting on the outcome of the study. By including all nurses in direct patient care in these wards, biases and threats to the internal and external validity of the study was minimized. The data collected in Ireland was part of a European wide cross sectional study entitled Nurse Forecasting: Human Resource Planning in Nursing (RN4CAST), of which I made mention earlier. There was no blinding used, however an additional section was added in the Irish version of the nurse questionnaire which aimed to examine the adverse event reporting patterns of nurses, and in-service safety training attendance. This study was based on a research design and clearly stated what the researchers did and how it was measured, allowing the reader to evaluate the methods used, the consistency, the reliability of the study, its validity and whether it could be replicated. The study data were analyzed using the SPSS version 18 and multi-level modeling (MLM) was the analytic method of choice.
Population and Sample
The population was identified as 1397 nurses in 108 wards in 30 hospitals who were representative of all nurses in direct patient care in large acute general hospitals in Ireland. The sample was described as a convenience sample of between two and four wards from each hospital. In my opinion, this was not enough information as the reader was left in suspense as to how many wards and nurses were actually analyzed from each hospital. Convenience sampling, used in this study is most widely used by quantitative researchers, however it is the sampling method that is most vulnerable to sampling biases. I think a stratified random sampling would have yielded more accurate representation of the two hypotheses that were proposed at the beginning of the study. However, biases would have been minimized because there was no blinding done and all nurses in direct patient care on those wards were included. The sample size was adequate for the country of Ireland. With regards to the European region as a population, this sample would be inadequate and in terms of a global level the error in reporting would be even greater.
Data collection and measurement
The researchers collected data from all but one of the large acute general hospitals in Ireland that agreed to participate in this study. In total, there were 30 study hospitals with a convenience sample of between two and four wards from each hospital. In the final analysis, a total of 1397 nurses in 108 wards in 30 hospitals were surveyed using questionnaires distributed by the research team. The questionnaires included several composite scales, including the 32 item Practice Environment Scale of the Nursing Work Index (PES-NWI). This is the most internationally validated tool for measuring the practice environment of nurses. Five subscales are included in the overall tool, namely nurse participation in hospital affairs, nurse leadership and support, nurse-physician relationships, and nurse foundations for quality of care. The subscales were aptly described in the article and reference was made to the reliability of its use as was demonstrated by Lake (2002).
In previous studies, for example, in the original Magnet Hospital studies in the United States of America (McClure et al., 1983), the nurse work environment was calculated at hospital level with little attention being paid to the ward or unit level work environment. This research on the Irish nurses work environment acknowledged that previous studies identified characteristics of hospitals which made them good places for nurses to work when compared to other US hospitals and explores this subject further, in particular looking at the impact on safety outcomes of the aggregated ward-level mean for PES-NWI. The study data were analyzed using SPSS version 18 and multi-level modeling (MLM) was the analytic method of choice. The ward mean scores for the nurse work environment was calculated with a result of 72% of wards scoring a mixed work environment; 13% in a poor environment and 15% in a better environment. With regards to adverse event reporting rates over the previous year, one third of nurses stated that they did not report adverse events, and one quarter stated that they reported more than 5. A mean nurse level score was compared to the ward mean safety score to show that this experience can be impacted by local management and circumstances.
The report provides evidence that the data collection methods yielded data that were high on reliability and validity. Using calculations and statistical tables the researchers were able to show that a higher proportion of nurses with degree level education on a ward, and a more positive mean work environment in that ward, resulted in higher levels of nurse reported patient safety in the ward. A third ward variable – the ward mean for emotional exhaustion levels was also calculated, however there was not much significance to nurse reported patient safety in the ward. In a previous report from the Department of Health and Human Services in the USA (Levinson, 2012), which looked at 195 hospitals suggested that 86% of adverse events which occur to patients in hospitals go unreported. An increase in adverse events reporting by nurses demonstrates a greater understanding of the systems approach to safety, the worth of full and open investigations of incidents, and a greater willingness to facilitate organizational learning. This calls for an investment in the work environment of nurses. This does not necessarily mean higher financial costs to the institution but a “re-focus” by both general management and nurse management on effective nurse leadership and other modifiable aspects of the work environment, both within wards and throughout the organization.
Procedures and results
There were no interventions documented in this article because this was a study of patient safety outcomes. The only source of data that was utilized was the questionnaires that were distributed, retrieved and analyzed using a number of statistical tools and models. While the results of the study clearly justify the intent and purpose, the study only solidified the findings of previous works investigating the relationships between nurse characteristics, the work environment and patient safety outcomes. Had the researchers investigated hospital records and documentation or electronic records their study results may have been enhanced.
Data was collected in a manner that eliminated bias in that all nurses in direct patient care in the study wards were invited to participate, however the study made no mention of any prior training for the staff who collected the data.
Data Analysis
The study followed similar lines of previous published studies, (Aiken et al., 2011) with a small deviation from the others. A two level study was done in this study with findings at the organizational level as well as the ward level with comparisons with similar studies that were previously reported. Multilevel study was possible in this study because the number of hospitals surveyed was small (30). However, there were no confounding variables that probably may have set this study aside from previous studies. Also, the number of exclusions that took place in procuring the study sample may have hindered the process of supporting the hypotheses fully. Therefore, the researchers had to report that this study is an advancement on previous works investigating the relationships between nurse characteristics, the work environment and patient safety outcomes.
Findings
Statistical significance was presented in the form of null models and multilevel models. The ward mean scores for the nurse work environment was calculated using the PES-NWI. In the multilevel model the null model examined the variance in nurse graded patient safety between units. The mean overall and the variance between wards were used to calculate the Intra-Class Correlation (ICC). The size of the ICC determines whether MLM techniques should be used in a study. In this study the ICC cut-off point was applied and confirmed that multilevel modeling approach was appropriate. The model in this study confirms that ward factors impact on levels of patient safety in a ward. MLM techniques enabled identification of ward factors which impact safety, and to which, modifications can be achieved with minimal cost implications for hospitals.
Conclusion
This study validated the findings already documented by previous researches on a hospital basis. However, I think more should be done to get precise information on the topic at hand and to show actual statistics on areas in nursing care with shortcomings when it comes to adverse event reporting and the impact on the patient. Specifically, nurses should feel comfortable speaking to management on the issue with the assurance that reporting adverse events can be used as a tool in educating the care team as a unit and in eliminating the chances of repetition of the same incidents, without fear of retribution or attrition. This is actually one of the 7 Nurses’ Bill of Rights. In an article in The American Nurses Association’s online journal, the author wrote that “Registered Nurses promote and restore health, prevent illness, and protect people entrusted to their care. To maximize the contributions nurses make to society, it is necessary to protect the dignity and autonomy of nurses in the workplace. To that end the Nurses’ Bill of Rights must be afforded.” www.nursingworld.org/Workplace.
References
Department of Health and Children, 2008. Building a culture of patient Safety. Report of
the Commission on Patient Safety and Quality Assurance. The Stationery Office, Dublin.
McClure, M.L., Poulin, M.A., Sovie, M.D., Wandelt, M.A., 1983. Magnet Hospitals:
Attraction and Retention of Professional Nurses. American Academy of Nursing, Kansas City.
Mulvey Boyle, S., 2004. Nursing unit characteristics and patient outcomes. Nursing
Economics 22 (3), 111-123.
Sermeus, W., Aiken, L.H., Van den Heede., Rafferty, A.M., Griffiths, P., Moreno-Casbas,
M.T., Busse, R., Lindqvist, R., Scott, P.A., Bruynee, L., Brzostek, T., Kinnunen.J.,
Schubert, M., Schoonhoven, L., Zikos, D., 2011. Nurse forecasting in Europe
(RN4CAST): rationale, design, and methodology. BMC Bursing 10 (6), http://dx.doi.org/
10.1186/1472-6955-10-6.
Van Bogaert, P., Clarke, S., Roelant, E., Meulemans, H., Van de Heyning, P., 2010.
Impacts of unit-level nurse practice environment and burnout on nurse reported outcomes:
a multilevel modeling approach. Journal of Clinical Nursing 19, 1664-1674.
www.nursingworld.org/Workplace Safety/Work Environment/Nurses Bill of Rights.
Extracted on 10/15/2016.
Lake, E.T., 2002. Development of the Practice Environment scale of the Nursing Work
Index. Research in Nursing and Health 176-188.
Levinson, D.R., 2012. Hospital Incident Reporting Systems Do Not Capture Most Patient
Harm. Department of Health and human Services, Office of Inspector General. Available
Online at: http://oig.hhs.gov/oei/reports/oei-06-09-0091.pdf (accessed 11th November,
2016).
Aiken, L.H., Cimiotti, J.P., Sloane, D.M., Smith, H.L., Flynn, L., Neff, D.F., 2011. Effects
of nurse staffing and nurse education on patient deaths in hospitals with different work environments. Medical Care 49 (12).
