Measuring patient safety culture in pediatric long-term care

Abstract

PURPOSE

The purpose of this study was to test the reliability, feasibility and utility of a modified patient safety survey for use in pediatric long term care (pLTC) settings and describe patient safety culture in a sample of providers from pLTC facilities.

METHODS

A survey was adapted from the Agency for Healthcare Research and Quality Nursing Home Survey on Patient Safety Culture (PSC-pLTC) and administered to a convenience sample of providers who work in pLTC during an educational workshop in November 2015.

RESULTS

Forty-nine respondents from 32 facilities across all 4 U.S. census regions completed the survey. The adapted survey demonstrated excellent face validity, usability, feasibility and internal consistency reliability (Cronbach alpha = 0.94). Highest ratings were given to overall perceptions of safety, feedback and incident communication, supervisors’ expectations and actions and management support. Lower ratings were given to dimensions of teamwork, communication, handoffs and transitions, with the lowest ratings given to staffing and organizational learning. Ratings were associated with population and geographic region served.

CONCLUSION

This survey to measure patient safety culture adapted for pLTC demonstrated components of reliability and validity, was useable and group discussants were eager for such a measure.

1. Introduction

Healthcare, particularly acute care settings, has increasingly been focusing on patient safety culture (PSC) to improve provider performance, safe and reliable care, and ultimately patient outcomes [1]. Key features of PSC attributed to enhanced performance include: a) acknowledgment of the high-risk nature of an organization’s activities, b) determination to achieve consistently safe operations, c) a blame-free environment to report errors or near misses without fear of reprimand or punishment, d) encouragement of collaboration across ranks and disciplines to seek solutions to safety problems, and e) organizational commitment of resources to address safety concerns [1–4]. Evidence exists that a better PSC is associated with better patient outcomes, such as decreased falls, infections, post-operative hip fractures, sepsis, and pressure ulcers. Such evidence, however, is primarily from acute care settings and adult populations [5,6].

PSC is also likely to be to be relevant to pediatric long-term care (pLTC) settings, particularly given the increase in number and complexity of care needs for children with complex medical conditions [7–9]. Pediatric LTC settings are unique and have dynamic, complex care delivery systems and provider, family and child interactions. Despite the myriad of providers and importance of inter-professional communication and teamwork, we know little of the PSC in pLTC. Improving our understanding of the PSC in this unique population could provide a possible key to understanding strategies to prevent errors and provide safer care. Several standardized survey tools exist to measure PSC in acute hospitals and nursing homes, however, no tool has been developed or tested for use in pLTC [10,11,13]. Therefore, a substantial gap exists in our understanding of factors that may influence optimal outcomes for children with complex medical conditions. The purpose of this study was twofold: (1) test and evaluate the utility of a modified PSC survey for use in pLTC and (2): conduct a group discussion to describe the perspectives of providers from LTC facilities regarding the state of PSC in such facilities.

2. Methods

2.1. Sample and setting

This descriptive study of the perspectives of a national sample of providers who work in pLTC regarding PSC occurred during an educational workshop at the Pediatric Complex Care Association Annual Conference in New Brunswick, New Jersey in November 2015. The Pediatric Complex Care Association is a national, non-profit organization whose stated mission is “to create opportunities for organizations to collaborate, network, share innovations, advocate, and promote excellence in the continuum of care for children with medical complexity and their families.” Membership primarily includes organizations that provide residential care for children with complex medical conditions and includes professionals from nursing, medicine, administration, respiratory therapy, social work and education, and academia. Approximately 200 individuals attended the 2015 conference of whom 60 attended the educational workshop during which the survey was administered and the group discussion was held. Institutional review board approval was obtained from Columbia University Medical Center.

2.2. The patient safety survey

The Agency for Healthcare Research and Quality Nursing Home Survey on Patient Safety Culture (NHSPSC) survey measures perceptions of safety culture in the following dimensions: supervisor expectations/actions, organizational learning, teamwork, communication openness, error feedback/communication, non-punitive responses to incidents, staffing, hospital management support, handoffs and transitions, overall perceptions of safety [14]. The 12-dimension, 42-item, 5-point Likert scale (rated ‘strongly disagree’ to ‘strongly agree’ or ‘never’ to ‘always’) has demonstrated adequate psychometric properties [14].

We previously adapted this survey so items were relevant to the pLTC setting. For example, the term ‘this nursing home’ was replaced by ‘this facility’ and ‘resident’ was changed to ‘child’, and named the revised survey the PSC-pLTC [15]. The modified instrument demonstrated sufficient reliability (internal consistency Cronbach alpha 0.55–0.85 for each subscale) and face, content and construct validity in psychometric testing in a sample of three facilities [15]. In the current study, the educational workshop participants were asked to complete the pen and paper survey.

2.3. Structured group discussion

To further assess the workshop participants’ perspectives of PSC in pLTC, the research team asked seven pre-tested questions from the survey that participants answered using an anonymous interactive audience response system that displayed aggregated responses in real-time. Questions included the geographic region of facility, and safety issues including: overall perceptions of child safety, feedback and communication about incidents, supervisor expectations and actions promoting child safety, management support for child safety, teamwork, communication openness, handoffs and transitions, training and skills, and non-punitive response to mistakes. Questions were also asked about the survey’s usability and feasibility. Participants were seated in tables of up to eight and a member of the research team circulated the room with a microphone to facilitate discussion. The research team recorded the participants’ responses captured by the audience response system. Key discussion points were clarified and confirmed with the group as notes were taken. Following the session, the research team debriefed to agree upon what was heard and recorded.

2.4. Statistical analysis

To analyze the survey responses, after reverse coding necessary items, categories of “disagree” and “strongly disagree” were collapsed and considered negative responses whereas “agree” and “positively agree” were collapsed and considered positive responses, all items had neutral responses and remained a separate category, in accordance with AHRQ survey procedures [2]. To test the association between PSC and U.S. census region or population served (pediatric only or pediatric and adult) Fisher’s exact test was used due to small counts. All analyses were completed using SAS version 9.3 (SAS Institute Inc., Cary, NC) with level of significance p < 0.05. We performed descriptive statistics of proportions and percentages to analyze participants’ anonymous responses to the questions posed during the group discussion.

3. Results

Forty-nine of the 59 participants who attended the session completed surveys (83% response rate), these 49 respondents represented 32 facilities from all four U.S. census regions. The characteristics of the survey respondents and their facilities are shown in Table 1. Most respondents were administrators/managers (71.4%). Most had worked in their facility 11 years or more (42.9%) and most did not provide direct care. Nearly half (46.8%) of the facilities were licensed for 51–100 beds. Most facilities (63.3%) only cared for pediatrics.

Table 1.

Characteristics of Survey Respondents (n = 49) and their facilities (n = 32)

Characteristic	Number respondents (percentage of responses)
Position
Administrator/Manager	35 (71.4%)
Physician	1 (2%)
Licensed Nurse	6 (12.2%)
Nursing Assistant/Aide	4 (8.2%)
Administrative support staff	4 (8.2%)
Provide direct care to children
Yes	20 (40.8%)
No	28 (57.1%)
Time worked in facility
Less than 2 years	6 (12.5%)
3 to 5 years	13 (26.5%)
6 to 10 years	8 (16.3%)
11 years or more	21 (42.9%)
Hours worked per week
Less than 24 hours	5 (10.4%)
25 to 40 hours	16 (32.7%)
More than 40 hours	27 (55.1%)
Time of day worked
Days	43 (87.8%)
Evenings/Nights	5 (10.4%)
Facility Region*
Northeast	31 (63.3%)
Midwest	3 (6.1%)
South	6 (12.2%)
West	7 (14.3%)
Bed Size
0–25 beds	3 (8.6%)
26–50 beds	15 (31.9%)
51–100 beds	22 (46.8%)
> 100 beds	7 (14.9%)
Population served
Pediatrics only	31 (63.3%)
Pediatrics and Adults	17 (34.7%)

Notes:

^*U.S. census regions are defined as: (1) Northeast: Connecticut, Maine, Massachusetts, New Hampshire, Rhode Island, Vermont, New Jersey, New York, Pennsylvania; (2) Midwest: Illinois, Indiana, Michigan, Ohio, Wisconsin, Iowa, Kansas, Minnesota, Missouri, Nebraska, North Dakota, South Dakota; (3) South: Delaware, District of Columbia, Florida, Georgia, Maryland, North Carolina, South Carolina, Virginia, West Virginia, Alabama, Kentucky, Mississippi, Tennessee, Arkansas, Louisiana, Oklahoma, Texas; and (4) West: Arizona, Colorado, Idaho, Montana, Nevada, New Mexico, Utah, Wyoming, Alaska, California, Hawaii, Oregon, Washington.

^**May not equal 100% due to missing data.

3.1. Patient safety survey

Internal reliability testing by Cronbach alpha was deemed “excellent” for all dimensions combined (α = 0.94) and acceptable to excellent for each individual subscale [16]. The distribution of PSC scores by dimension is shown in Table 2. The highest ratings were given to overall perceptions of safety, feedback and incident communication, supervisors’ expectations and actions, and management support. Lower ratings were given to dimensions of teamwork, communication, handoffs and transitions, with the lowest ratings given to staffing and organizational learning.

Table 2.

Patient safety culture dimension ratings and internal reliability

Dimension	Internal reliability Cronbach alpha coefficient (α) (n = 49)	Overall mean score (SD) (n = 49)	Overall proportion rated positively (n = 49)	Pediatric only proportion rated positively (n = 31)	Pediatric & adults proportion rated positively (n = 17)
Overall perceptions of child safety	0.76	4.61 (0.41)	0.99	0.99	0.98
Feedback and communication about incidents	0.81	4.48 (0.56)	0.91	0.93	0.87
Supervisor expectations & actions promoting child safety	0.81	4.33 (0.58)	0.89	0.91	0.84
Management support for child safety	0.79	4.06 (0.79)	0.79	0.84	0.70
Teamwork	0.64	3.98 (0.51)	0.78	0.79	0.76
Communication openness	0.75	3.95 (0.67)	0.73	0.80	0.61
Handoffs and transitions	0.72	3.82 (0.54)	0.68	0.70	0.65
Training and skills	0.71	3.56 (0.79)	0.65	0.23	0.67
Non-punitive response to mistakes	0.78	3.49 (0.89)	0.59	0.60	0.56
Compliance with procedures	0.75	3.41 (0.80)	0.59	0.66	0.51
Staffing	0.65	3.38 (0.79)	0.57	0.66	0.44
Organizational learning	0.71	3.30 (0.43)	0.52	0.53	0.50
Overall ratings	N/A	4.21 (0.90)	0.81	0.90	0.65

Notes: Possible range 1–5 on Likert scale (1–2 considered negative, 3 considered neutral, and 4–5 considered positive). Population served missing from one survey. Overall rating includes one item only therefore Cronbach alpha coefficient not applicable.

PSC ratings were associated with population and geographic region served. Those working with pediatric only populations consistently rated each dimension higher than those working with mixed adult and pediatric populations, with the exception of training and skills. Marked differences in overall facility safety ratings were also noted in that 90% of pediatric-only facilities were rated very good to excellent compared to 65% of the pediatric-adult facilities though these were not statistically significant (Table 2). Participants from facilities located in the Northeast rated the dimension of organizational learning higher than those from other regions (Midwest, West, and South, (Fisher’s exact p = 0.01)).

3.2. Themes from structured group discussion

As indicated by responses captured by the audience response system, most (56%) respondents reported they completed the survey in less than 10 minutes 35% completed it within 10–15 minutes (35%), and 8% did not finish. About half (53%) reported completing a previous safety survey in their facility and most (63%) respondents rated their facility as “better” than most facilities. The vast majority (93%) reported that the PSC-pLTC survey was easy to use and 91% responded they would like to use the survey in their facility.

Facilitated group discussion provided additional insight into perceptions of PSC in pLTC. Four primary themes were identified: 1) communication, 2) benchmarking, 2) internal structure and hierarchy and 4) training/continuous learning. The need for ongoing and robust vertical and horizontal communication was expressed. “Huddles” with short directed discussions were identified as an important internal communication method to facilitate informing all staff in a timely fashion. The value of providing the PSC survey results to staff was identified as a method to open internal communication channels. The need to enhance communication within and between settings, was identified as a gap in care transitions, and use of interoperable electronic medical records was suggested as possible solution. Benchmarking was identified as important, by example submitting data and receiving external feedback from a benchmarking organization, such as the Agency for Healthcare Research and Quality. Participants discussed the potential for broader use of a PSC survey and opportunities for collaboration.

Internal structure issues of hierarchy and workflow were identified as problematic. Participants expressed the importance of understanding the spectrum and variations in patient safety perceptions among diverse staff, managers and parents. Creating a non-punitive environment that is “self-correcting” was described as a “work in progress.” Participants discussed the need to have “staff as part of the solution” and the value of shared governance. For example, certified nursing assistants and registered nurses report to different supervisors and shift schedules differ creating silos and delays in communication. Staffing and workflow issues were concerns in that staff is less busy when the children “go to school” and when they return there is a “rush when they are back home.”

The need for continuous training was also identified. Participants reported that policies and procedures directly impact PSC, yet are often learned only during orientation and education is not repeated throughout ongoing employment. Participants emphasized that to increase staff awareness policies and procedures must be made more accessible to staff, particularly as new policies are implemented. The need for “high reliability training” for staff empowerment where “no is an option” was also identified. One site provided an example of the value of such training; a reduction in ventilator associated pneumonia was attributed to training certified nursing assistants on procedures to safely transfer children to respiratory therapy appointments.

4. Discussion

In this sample, the PSC-pLTC demonstrated face validity, usability, feasibility and internal consistency reliability. While other tools exist to measure PSC in long term care settings, to our knowledge this is the first such tool tested among providers from a sample of pLTC settings across all U.S. census regions.

4.1. Safety perceptions in pLTC

The safety movement in healthcare began in acute care settings and is now moving beyond hospitals. In prior work we found that the PSC-pLTC survey demonstrated some properties of reliability and validity for the pLTC settings in a small one state sample [15].This study allowed us to add to that knowledge and quantify the perceptions of PSC among respondents from pLTC facilities across U.S. census regions. Areas of strength were identified, including dimensions of leadership and management support. Areas of opportunity were identified including interpersonal working conditions such as teamwork, transitions, handoffs and feedback, workflow and staffing. These areas of opportunity are worrisome as teamwork and good communication are essential to the provision of safe high quality care, and poor staffing may exacerbate these deficiencies [17].

The difference in perceptions of PSC by the population served (pediatrics-only or pediatrics and adults) may reflect underlying organizational differences that are important to providers. Additional resources may be needed in settings in which mixed populations receive care; our findings suggest these resources should target staffing, organizational learning, compliance with procedures, and non-punitive responses to mistakes. Respondents working in pediatric-only settings generally indicated that insufficient training and skills were issues. As facilities navigate complex licensing, regulatory and reimbursement parameters to allow children to age in-situ these differences in the PSC dimensions may be increasingly relevant [17–19], We also found a regional difference in perceptions of PSC, most notably regarding organizational learning, which was identified as a substantive gap. Despite the limitation of a small sample size from several regions requiring aggregation for these analyses, this finding warrants further study.

In an era of rapid health care reform and an emphasis on becoming a learning health system, our findings indicate that pLTC settings are particularly challenged. Standardized outcome measurements do not exist for pLTC, making internal and external benchmarking impossible and hampering quality improvement efforts [20]. Indeed, children with medically complex chronic conditions have been identified as a priority population for improvement efforts [19,21]. Several initiatives are underway, including payment reform through the Centers for Medicare and Medicaid Innovation Awards to incentivize providers to improve care, reduce adverse outcomes and improve transitions of care [19,21]. Additionally, underscoring the importance of standardization to guide improvement, the Pediatric Complex Care Association is undertaking a project to develop National Quality Indicators and create national benchmarks on areas such as staffing, infection control, ventilator weaning, hospital transfers and medication safety.

4.2. Limitations

This study has several limitations. All study data were collected at a voluntary professional development session and therefore self-selection bias may be problematic. Though all U.S. census regions were represented this is a small, convenience sample. The views, though representative of an interdisciplinary care team, may also reflect those more likely to respond positively given their leadership positions and this sample had over representation of administrators and mangers. Though variability in responses was found, the majority did not have direct care responsibilities and that may alter their perspectives. Further, some facilities may have been over-represented as multiple respondents from one facility may have answered the survey.

5. Conclusion

This study has demonstrated that an existing survey tool, previously tested in adult nursing homes, can be adapted for use in the pLTC setting. The adapted survey, PSC-pLTC proved easy and quick to use, reliable and valid. Discussants verbalized they were eager for such a measure to be used in their respective facilities. Respondents perceived that the survey provided actionable information which could be used to improve organizational performance and outcomes. Future work should examine the depth and breadth of PSC in individual pLTC facilities, and explore the relationship of PSC with patient outcomes.