How do we PI? Results of an EAST quality, patient safety, and outcomes survey

Abstract

Background

Quality improvement is a cornerstone for any verified trauma center. Conducting effective quality and performance improvement, however, remains a challenge. In this study, we sought to better explore the landscape and challenges facing the members of the Eastern Association for the Surgery of Trauma (EAST) through a survey.

Methods

A survey was designed by the EAST Quality Patient Safety and Outcomes Committee. It was reviewed by the EAST Research and Scholarship Committee and then distributed to 2511 EAST members. The questions were designed to understand the frequency, content, and perceptions surrounding quality improvement processes.

Results

There were 151 respondents of the 2511 surveys sent (6.0%). The majority were trauma faculty (55%) or trauma medical directors (TMDs) (37%) at American College of Surgeons level I (62%) or II (17%) trauma centers. We found a wide variety of resources being used across hospitals with the majority of cases being identified by a TMD or attending (81%) for a multidisciplinary peer review (70.2%). There was a statistically significant difference in the perception of the effectiveness of the quality improvement process with TMDs being more likely to describe their process as moderately or very effective compared with their peers (77.5% vs. 57.7%, p=0.026). The ‘Just Culture’ model appeared to have a positive effect on the process improvement environment, with providers less likely to report a non-conducive environment (10.9% vs. 27.6%, p=0.012) and less feelings of assigning blame (3.1% vs. 13.8%, p=0.026).

Conclusion

Case review remains an essential but challenging process. Our survey reveals a need to continue to advocate for appropriate time and resources to conduct strong quality improvement processes.

Level of evidence

Epidemiological study, level III.

WHAT IS ALREADY KNOWN ON THIS TOPIC

• Process improvement remains a difficult and uncomfortable process for trauma providers

WHAT THIS STUDY ADDS

• There is a wide amount of variability and perspective present in process improvement across trauma centers.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

• Trauma surgeons and trauma medical directors should be aware of the challenges their team and institutions face when engaging in process improvement such as time commitment and non-conducive environment.

Introduction

Process and quality improvement in trauma care remain core principles for every American College of Surgeons (ACS) verified trauma center. The newest edition of Resources for Optimal Care of the Injured Patient calls for ‘resources allocation (such as equipment, personnel, and administrative support), a commitment to patient safety, and an enduring focus on continuous PI’ to be prepared to care of the breadth and depth of pathology in the traumatically injured patient.¹ Conducting effective performance improvement (PI) is time-consuming but remains paramount to success in trauma care. The vast amount of resources available to help guide trauma PI leads to diverse implementation strategies across verified programs. In a survey of trauma quality improvement by Zetlen et al in 2017, only 66% of providers in high-income countries identified systems improvement as the perceived objective of morbidity and mortality conferences, with 49% of providers identifying lack of time and 17% identifying lack of interest as a barrier to increased use of conferences and trauma registries.² However, when adopted, quality improvement processes are effective. Hemmila et al examined the Michigan Trauma Quality Improvement Program and presented data to suggest that formalized quality improvement programs improve patient outcomes and decrease resource use.³ As PI is currently non-standardized and carried out in various forms, we sought to better understand how quality and PI is undertaken among trauma clinicians. The objective of this study was to survey members to understand how they execute performance improvement patient safety (PIPS) plans, resources required, frequency of meetings, participants in the process, and perceived effectiveness of the program. We hypothesized that there would be significant center-level variability in PI programs among verified trauma centers.

Methods

A REDCap survey tool was created by the Eastern Association for the Surgery of Trauma (EAST) Quality, Patient Safety, and Outcomes Committee. The survey creation team consisted of two trauma medical directors (TMDs), a trauma program manager, and the EAST Quality, Patient Safety, and Outcomes Committee chair. The questions were based on several resources standardly used in PIPS across institutions. This survey was then internally validated by circulating it among committee members to test for readability, relevance, and performance. Feedback from the committee was used to finalize the survey (see online supplemental appendix A). This survey was sent to 2511 EAST members. Members received an initial email inviting them to complete the survey and two subsequent reminders. No incentives were offered for completion of the survey. Membership categories that were surveyed included active, senior, provisional, and associate EAST members. Residents and fellows were included in the survey (approximately 3% of responses).

The survey was designed to identify the setting in which the respondents practiced as well as the methods of case review and quality improvement used. It was also designed to better understand how institutions identify cases for quality improvement, the perception of the quality improvement process, and any perceived barriers to conducting effective PI. We also sought to evaluate the use of adjunctive PI measures such as trauma video review (TVR), participation in Trauma Quality Improvement Project (TQIP)/Collaboratives, and the ‘Just Culture’ program. Data analysis was performed using SPSS (version 25) using χ² and Fisher’s exact test to analyze the categorical responses of the survey where appropriate.

Results

The survey was returned by 151 respondents of the 2511 surveys sent (6.0% response rate). There were no incomplete responses. As seen in table 1, most of the respondents were either trauma faculty (83 of 151, 55%) or TMDs (37%) at either ACS-verified level I (62%) or level II (26 of 151, 17%) trauma centers. Most respondents practiced at academic (62%) hospitals with varying distributions of trauma volume.

Table 1.

Survey response data

Respondent role (N=151)	n (%)
Trauma medical director	56 (37.1)
Trauma faculty	83 (55.0)
Fellow/resident	5 (3.3)
Other	7 (4.6)
Trauma center level (n=151)
ACS level I	94 (62.3)
ACS level II	26 (17.2)
ACS level III or IV	6 (4.0)
Non-ACS verified	25 (16.6)
Type of institution (N=151)
Academic or university affiliated	94 (62.3)
Community teaching hospital	38 (25.2)
Community private hospital	16 (10.6)
Military facility	3 (2.0)
Trauma center region (N=151)*
Northeast	37 (24.5)
Midwest	32 (21.2)
South	57 (37.7)
West	23 (15.2)
Outside the USA	2 (1.3)
Trauma center annual admissions (N=151)
<1200	19 (12.6)
1200–2400	59 (39.1)
2400–3600	38 (25.2)
>3600	35 (23.2)

*Northeast: Connecticut, Maine, Massachusetts, New Hampshire, New York, Pennsylvania, Rhode Island, and Vermont; Midwest: Iowa, Illinois, Indiana, Kansas, Michigan, Minnesota, Missouri, Nebraska, North Dakota, Ohio, South Dakota, and Wisconsin; South: Alabama, Arkansas, Delaware, District of Columbia, Florida, Georgia, Kentucky, Louisiana, Maryland, Mississippi, North Carolina, Oklahoma, South Carolina, Tennessee, Texas, Virginia, and West Virginia; West: Alaska, Arizona, California, Colorado, Hawaii, Idaho, New Mexico, Nevada, Oregon, Washington, and Wyoming.

ACS, American College of Surgeons.

Other than the uniform use of Resources for Optimal Care of the Injured Patient manual (‘Orange Book’) and high participation in the Trauma Outcomes and Performance Improvement Course (TOPIC), there was a wide variety in the resources used to assist in PI (table 2).¹ Academic or level I trauma centers had a greater percentage of respondents that participated in two or more courses, though this was not statistically significant (p=0.090 for academic centers, p=0.322 for level I trauma centers). The majority of respondents indicated that PIPS is performed monthly (51%) or weekly (45%). Level III (multidisciplinary peer review, 70%) is the most common level of review that is performed at the respondent institutions. Most institutions (70) identified a multidisciplinary peer review team consisting of emergency medicine (91%), orthopedics (87%), anesthesia (85%), and critical care (70%) as the most common attendees. The types of cases selected for PIPS and how they are identified are also displayed in table 2. Survey respondents indicated that there are a wide variety of indications for quality review and that they are reported by a range of healthcare providers. Approximately one-third of trauma center respondents reviewed deaths only at peer review.

Table 2.

PIPS plan characteristics

Frequency of PIPS (N=151)	n (%)
Weekly	68 (45)
Monthly	77 (51)
Other	6 (4)
Types of cases reviewed (N=151)
Near misses or potential for harm	107 (70.9)
All complications irrespective of level of harm	106 (70.2)
Only complications resulting in severe harm	43 (28.5)
All deaths only	51 (33.8)
Only deaths that are unanticipated or anticipated with opportunities for improvement	33 (21.9)
Interesting cases	89 (58.9)
Other	8 (5.3)
How cases are identified (N=151)
Cases reported by house staff or other clinical staff	106 (70.2)
Cases reported by attending staff	111 (73.5)
Trauma program manager review	120 (79.5)
TMD or trauma attending review	123 (81.4)
Performance improvement coordinator review	117 (77.5)
Other	9 (6.0)
Highest level of PIPS performed (N=151)
Level I, trauma divisional review	13 (8.6)
Level II, subspecialty review with TMD	9 (6.0)
Level III, multidisciplinary peer review	106 (70.2)
Level IV, multicenter peer review	23 (15.2)
Other services which routinely attend PIPS (n=151)
Anesthesia	129 (85.4)
Case management/social work	33 (21.9)
Surgical critical care	106 (70.2)
Emergency medicine	138 (91.4)
Neurosurgery	130 (86.1)
Nursing administration	80 (53)
Orthopedics	132 (87.4)
Physiatry/rehab	32 (21.2)
Other surgical services	55 (36.4)
Other	31 (20.5)
Resources contributing to PIPS (N=56)
Resources for Optimal Care of the Injured Patient	56 (100)
STN TOPIC course	39 (69.6)
STN Optimal course	18 (32.1)
Trauma Center Association of American Trauma Medical Director Course	19 (33.9)
‘Just Culture’ certification	7 (12.5)
Lean Six Sigma Certification	8 (14.3)
Other	2 (3.6)

PIPS, performance improvement patient safety; STN, Society of Trauma Nurses; TMD, trauma medical director; TOPIC, Trauma Outcomes and Performance Improvement Course.

The provider views on the PIPS process are displayed in table 3. Most of the respondents (92%) felt that correcting and preventing errors were major objectives of their conferences with (84%) education/literature review, obtaining different viewpoints/perspectives, and loop closure being other important outcomes. Only (9%) felt that assigning blame was an objective of the conference. There were several PIPS barriers that respondents identified. These included time constraints (53%), lack of participant engagement (42%), and limited institutional resources (41%). Twenty-one percent of respondents also stated that the PI format was not conducive to case review. When asked if corrective actions leading to change are effective after opportunities identified during PIPS conferences, 92% said yes (either fairly, moderately, or very effective) and 8% said no. Individual reflections after PIPS conferences were also assessed, with the respondents being queried on how they felt after presentation of a case they were involved in. Seventy-six percent felt they learned from potential errors or a system error was identified, while (24%) felt that the errors were inevitable, felt guilt and uncertainty about the outcome, or that blame was assigned. When stratifying PIPS effectiveness by role of respondents, we noticed some interesting findings (figure 1). TMDs were more likely than their colleagues to identify their PIPS conferences as either moderately effective or very effective compared with not being effective or fairly effective (77.5% vs. 57.7%, p=0.026).

Table 3.

Provider views on PIPs process

Which outcomes are part of the PIPS process? (N=151)	n (%)
Correction/prevention of errors	139 (92.1)
Education/review of literature	127 (84.1)
Obtaining different viewpoints	132 (87.4)
Assigning blame	14 (9.3)
Closing the loop with surgeon/provider	125 (82.8)
How effective are corrective actions in leading to change? (N=151)
Very effective	43 (28.5)
Moderately effective	68 (45)
Fairly effective	28 (18.5)
Not effective	12 (7.9)
PIPS process impairments (N=151)
Physician time commitment	80 (53.0)
Limited institutional resources	62 (41.1)
Environment not conducive to case review	31 (20.5)
Legal concerns	7 (4.6)
Participant engagement	64 (42.4)
Other	21 (13.9)
Provider feelings after PIPS review (N=151)
Embarrassment or guilt	5 (3.3)
Learned from potential errors	57 (37.7)
System errors were identified	58 (38.4)
Errors are inevitable	25 (16.6)
Colleagues blamed me	6 (4.0)

PIPS, performance improvement patient safety.

Figure 1.

Performance improvement patient safety effectiveness by role. TMD, trauma medical director.

Adjunctive PIPS processes were also evaluated (table 4). Participation in TQIP was the most common adjunctive measure in 95% of respondents, and 59% participated in a TQIP collaborative. TQIP was most frequently used to drill down on areas of weakness and create PIPS projects (76%). The majority of respondents also indicated that these data were shared in their PIPS conferences (70%) as opposed to only being viewed only by the trauma program leadership. Incorporation of the Just Culture philosophy was used by 14% of respondent programs, with just over half (56%) of the TMDs feeling that this improved the quality of their PI process. A quarter of respondents stated that they were in a stage too early to tell. There was great variety in how the Just Culture algorithm was implemented into the PIPS process. Programs with Just Culture feedback models were less likely to identify their environments as not conducive to constructive case review (27.6% vs. 10.9%, p=0.012) and to report feelings of assigning blame (3.1% vs. 13.8%, p=0.026). TVR was reported to be used by 20% of respondents, with the majority of institutions using it to record all activations. Videos that were selected for TVR were primarily used to focus on team communication, management decisions, and clinical performance. This was typically done in small group conferences (61%). The videos were selected for review by TVR faculty coordinator (32%) or any faculty (29%), trauma program managers (16%), and TMD (13%). The videos were primarily reviewed in a quality improvement conference (97%) or small group format (55%) with the goal of identifying communication errors (93.5%) and patient management errors (87%) and evaluating clinical performance (84%). The majority of respondents indicated that cases were referred to the medical examiner (ME) for autopsy. ME reports were most frequently shared in a PI conference, though in over 40% of respondents, these were reviewed only by trauma administration.

Table 4.

Adjunctive PIPs measures

Does your institution participate in the TQIP? (N=56)	n (%)
Yes	54 (96.4)
Describe how you use TQIP reports (N=54)
Reviewed by PI coordinator	29 (53.7)
Reviewed by TMD	20 (37)
Summary of reports presented at PIPs conference	38 (70.4)
Reports used to ‘drill down’ on problematic areas or create PIPS team projects	41 (75.9)
Does your institution participate in any TQIP collaboratives? (N=56)
Yes	33 (58.9)
Describe how you use your TQIP collaborative reports. (N=33)
Regional collaborative PIPS projects	14 (42.4)
Standardization of regional best practices	8 (24.2)
Collaborative research projects	2 (6.1)
Data validation/troubleshooting	9 (27.3)
Is TVR used? (N=151)
Yes	31 (20.5)
Which best describes your TVR? (N=31)
All activations recorded	19 (61.3)
Only highest tier activations recorded	7 (22.6)
System turned on manually	5 (16.1)
How are cases selected? (N=31)
TMD	4 (12.9)
Trauma program manager	5 (16.1)
TVR director	10 (32.3)
Any faculty	9 (29.0)
Other	3 (9.7)
How is TVR feedback provided? (N=31)
Education/QI conference	30 (96.8)
Individual or small group	17 (54.8)
No formal feedback, video available	3 (9.7)
No formal feedback, video not available	0 (0)
How is TVR used in PIPS? (N=31)
Identify communication errors	29 (93.5)
Identify management errors	27 (87.1)
Evaluate clinical performance	26 (83.9)
Does your Institution Incorporate the ‘Just Culture’ philosophy? (N=151)
Yes	64 (42.4)
Is the full Just Culture algorithm incorporate into the PIPS process? (n=64)
Fully incorporated	24 (37.5)
Only applied to cases with deviations in care	14 (21.9)
Selectively applied case by case	15 (23.4)
Unsure	10 (15.6)
Other	1 (1.6)
Has Just Culture improved the quality of peer review? (N=64)
Yes	36 (56.3)
No	12 (18.8)
Too early to tell	16 (25)
Are autopsy reviews incorporated into your PIPS process? (N=151)
All deaths referred for consideration	70 (46.4)
Selected cases referred	55 (36.4)
No	26 (17.2)

PI, performance improvement; PIPS, performance improvement patient safety; TMD, trauma medical director; TQIP, Trauma Quality Improvement Program; TVR, trauma video review.

Discussion

The ACS Committee on Trauma has set the standards of improving the quality of care for injured patients for decades. Since the introduction of the Resources for Optimal Care of the Injured Patient was first published in 1979, this has served as the metric by which centers benchmarked.⁴ A central component of this document is multidisciplinary PI focused on structure and process of care while monitoring patient outcomes and is recognized as one of the most frequent criterion deficiencies (CDs) that cause trauma centers to fail verification.^{4 5 6} As such, this document was uniformly cited by the respondents of this survey as a key basis for PIPS conferences. Since the inception of the resources document, a host of other resources and courses from a variety of professional societies (ie, TOPIC and Optimal) and industrial sectors (Just Culture and Six Sigma) have also shaped the current conduct of the PIPS process in trauma centers across the USA.

While it is evident that most centers participate in multidisciplinary PIPS efforts, there is variability regarding participants in these efforts. Subspecialty liaison presence is mandated through the CDs sent forth through the resources document, and those representatives are overwhelmingly represented in our survey. However, as trauma care transcends the entire hospital structure, it is surprising that consistent administrative presence is lacking in the majority of institutions. PIPS case selection method also appears to be variable, with up to a 30% of institutions focusing only on deaths and severe complications. Such methods miss the opportunity to identify systems issues until they reach the level of severe patient harm. Errors generally involve competent providers with the best intentions that are practicing in complex sociotechnical systems.⁷ High-reliability institutions and organizations that value safety anticipate imperfection and are committed to designing systems that minimize it. A key component of improving system design is a system of just accountability in which individuals are encouraged to report and discuss errors or near misses and are supported by the Healthcare Quality Improvement Act.⁸

Of the EAST members who responded to the survey, the majority viewed their quality improvement process in a positive light and as a tool for preventing errors, reviewing best practices, and discussing complex patient care cases with colleagues. Yet, our survey data reveal the tough reality of process improvement in trauma centers in the USA. Only 29% of those surveyed felt that their process was very effective, and our analysis suggests that this sentiment is skewed toward the TMDs who oversee it. More concerning still was the proportion of respondents who found their quality improvement programs to be ineffective, with a focus on assigning blame or instilling a sense of guilt over poor outcomes. Staff that perceive that they are working in a punitive or biased environment are less likely to report errors and can lead to deficiencies in the care process.^{7 9} Centers with impartial, well-structured peer review have positive impacts on clinical safety and performance.¹⁰ The Just Culture algorithm introduced by Outcome Engenuity¹¹ focuses on an evidence-based peer review process that recognizes that human error is omnipresent and concomitantly evaluates system design in a fair and impartial manner.¹² Adopting this algorithm requires training and supportive backing to succeed, further underscoring the importance of hospital leadership participation in quality review. Implementation of Just Culture requires time along with the willingness to allow clinical departments flexibility to move away from the traditional ‘blame’ culture of medicine. However, facilities that have undertaken this endeavor have had successful results.⁷ Our survey data appear to support these observations, with institutions participating in the Just Culture algorithm having a more conducive environment to support peer review dialog. Obtaining closure of peer review cases was also noted to be a challenge in our survey results. These findings mirror that of Hamad et al in a recent TQIP analysis using the anonymous Mortality Reporting System. They found that in 7.3% of the reported deaths, there was no mitigation strategy identified to prevent future occurrences from happening.¹³ This suggests that we are not identifying all human-level and systems-level issues in our PIPS process. These same authors also found that we tend to focus on corrective actions on the provider or care level (ie, education, counseling, and guidelines) as opposed to more effective measures of change such as process simplification and standardization of built-in redundancies, barriers, or fail safes that have more durable effects.¹⁴

Participation in other adjunctive PI measures appeared to mirror previously reported trends. As TQIP participation is a requirement for ACS verified trauma centers, it is without surprise that 95% of the respondents are involved in this endeavor. Almost 60% are also members of a TQIP collaborative which has been linked to reductions in complications and resources at a rate that is greater than using individual TQIP results alone.¹⁵ TVR use also appeared to be in line with what has been reported nationally (20% to 29%),^{16 17} with recording practices that appear to vary nationally. As this is an evolving tool with an ongoing EAST multicenter study, its optimal use is as of yet to be determined. Postmortem exams also continue to be used nationally and were well represented in our survey despite the criticisms that these are resource-intensive procedures and rely on local ME availability.¹⁸ Review of autopsy data as part of the peer review process has been linked with improved outcomes in the assessment of the trauma system function.¹⁹ Our survey responses appear to support this notion as the majority of reports are reviewed formally at PIPS meetings.

Our findings contain several limitations. As with any survey, ours is limited by the overall low response rate, so it offers just a small glimpse into the PI process at predominately academic level I trauma centers. The survey links were also not personalized to each participant, so it is possible that a respondent filled out the survey multiple times. It also offers little insight into why respondents may find their PIPS ineffective or judgmental as the answers were standardized by the survey instead of allowing for free responses. The current survey also is likely to be heavily based on the interpretation of the Resources for Optimal Care of the Injured Patient.¹ With a new resources manual forthcoming, the applicability of these results may change as the PI section will now be more standardized with expectations more clearly delineated.¹ Additionally, EAST members have increased exposure to evolving methods, such as Just Culture, through programming including the Short Course on Trauma Quality, TOPIC, and other programming at the annual scientific assembly. Access to many of the PIPS resources available appears to be influenced by the respondents’ institution, and the reasons for this are unknown. Many of the challenges identified exist on a predominately institutional level, and we should strive to continue to recognize and tailor these processes to meet those needs.

Conclusion

Case review can be a difficult and uncomfortable process. Our survey data appear to reflect that reality and poses a challenge to physicians and staff of trauma centers nationwide. Several resources are available to enhance the PIPS process, but their incorporation is far from standardized. Trauma leaders should advocate for the appropriate resources and time to conduct just, impartial process improvement activities and to share their collective experiences to advance our knowledge on this crucial component of our trauma system.

Data availability statement

Data are available upon reasonable request. All data relevant to the study are included in the article or uploaded as supplementary information. All of the responses relevant to the study have been included in the manuscript. The raw response data are available upon reasonable request.

Ethics statements

Patient consent for publication

Not applicable.

Ethics approval

The final version of the survey was reviewed and approved for distribution by the Eastern Association for the Surgery of Trauma Research and Scholarship Committee and was exempted from institutional review board (IRB) review. This study was considered exempt by the IRB as it was a quality improvement study. The IRB determined that no consent or consent statement was required, given the nature of the survey, as it was considered a non-human subject research.