Broadening the definition of patient-safety events: lessons from a multicentre learning health system collaborative

Abstract

Background

Improving safety in healthcare has been paramount for decades, yet despite major attention and investment, improvement has remained incremental. Patient safety is a major concern in US healthcare, leading to significant harm and economic losses annually. Accurately identifying safety events remains difficult due to methodological discrepancies and lack of standardisation. This study evaluated the feasibility of implementing a standardised case-review methodology and safety-event taxonomy across diverse hospital settings to assess opportunities for improvement (OFIs) and compare findings with traditional definitions.

Methods

This multicentre retrospective cohort study reports data from 103 hospitals across the USA and Canada between 2016 and 2023. A multivariable logistic regression was performed to test case reviews for differences in the presence of one or more OFIs across several hospital types (bed size, academic status, urban setting, trauma level and Centres for Medicare and Medicaid Services overall star rating) and patient characteristics (age, gender, length of stay, admission and discharge code status and mortality).

Results

19 181 cases were reviewed across the Learning Health System Collaborative, with a median of 107 reviews per hospital. Mortality was the most common cohort selection, studied by 91 hospitals (88%). At least one OFI was identified in 12 714 cases (66.3%). The logistic regression analysis found that all hospital characteristics and patient age, length of stay, code status and discharge disposition were significantly associated with at least one OFI. Of the 46 444 OFIs identified, 41 439 (89%) were from categories focused on omissions of care. The categories of end-of-life, documentation and treatment/care alone accounted for 25 980 OFIs (56%).

Conclusion

The highest volumes of safety-related OFIs were associated with omissions of care, as opposed to the traditional definition of patient safety, which primarily includes outcomes from acts of commission.

WHAT IS ALREADY KNOWN ON THIS TOPIC

• The field of patient safety emphasises the proactive approach needed to ensure patient well-being and minimise risks associated with healthcare delivery. However, traditional efforts to improve patient safety have incrementally advanced clinical outcomes. Measuring improvements, however, is challenging, given the various definitions and methods applied in published studies.

WHAT THIS STUDY ADDS

• Improving how patient-safety events are identified requires expanding how patient safety is defined. This study reimagines patient safety to address the quality of care with a mindset focused on family-centred care. To accomplish this, case reviews were performed by frontline clinical care team members with current experience to recognise faulty processes of care that create vulnerabilities and undesirable clinical experiences. As a result, many, otherwise, overlooked opportunities for improvement (OFIs) were identified. Through this novel process, over 80% of patient-safety opportunities identified were omissions of care compared with traditionally identified commissions.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

• This expanded philosophy and definition of patient safety could prompt future work to include a broader scope of patient-safety metrics and accreditation requirements. The identification and subsequent targeting to address the highest volume of OFIs offer meaningful and measurable improvement in patient outcomes, unlike the continued allocation of resources to address the diminishing number of hospital-acquired conditions. This shift towards a patient-centred method values the clinical expertise and opinions of frontline care team members to identify the most important quality improvement initiatives needed to enhance patient outcomes.

Introduction

Patient safety is a critical issue in US healthcare systems. Patients experience significant harm and die each year due to unsafe, inappropriate or inadequate healthcare delivery.^{1 2} These errors impart a substantial national economic burden of over $20 billion annually.³ Since the National Academy of Medicine report ‘To Err is Human’ elevated awareness of patient safety, many initiatives and strategies have been employed to improve healthcare processes and reduce the rate of harm.⁴ One of the primary barriers to creating safer healthcare systems has been the challenge of accurately identifying and quantifying safety events.⁵

The National Health System defines patient-safety incidents as ‘any unintended or unexpected incident, which could have, or did, lead to harm for one or more patients receiving healthcare’.⁶ The complexities inherent to healthcare make it difficult to develop a method that satisfactorily addresses all aspects of patient safety and can be successfully applied across disparate health systems. Unsurprisingly, there is a lack of standardisation for detecting and defining safety events. Inadequate data exacerbate the challenges of generating meaningful action plans for addressing the uncovered issues.⁷ These methodological and contextual variations have resulted in highly variable event rates among hospitals, making it difficult to compare and measure progress in patient safety.⁸ Traditional patient-safety approaches, often referred to as Safety-I, focus on identifying and mitigating errors and adverse events. However, recent advances in safety science, particularly the Safety-II perspective, emphasise understanding why things go right and fostering system resilience. From a safety management perspective, a proposed paradigm shift acknowledges that incidents and accidents do not only happen linearly, explores events based on frequency rather than severity and includes often overlooked processes that make things go right to encompass a more resilient health system.⁹

Patient safety has changed considerably since the seminal Harvard Medical Practice Study (HMPS); however, documenting the extent of change remains challenging. The HMPS focused on litigation and mitigating risk, which set the tone for the entire field.^{10 11} Retrospective chart review emerged as an early standard, with automated screening tools and triggers for known adverse event types later providing improved efficiency. Additionally, voluntary reporting, frequently via institutional patient-safety-event reporting systems available to medical staff, has long been vital to patient safety, providing a way to record events as they occur.¹⁰¹²,²³ Yet, these approaches limit capturing the breadth of patient-safety events. Retrospective review relies on the completeness of documentation and the reviewer’s clinical knowledge. Automated screening accuracy depends on a small, clearly defined set of event types. Voluntary reporting suffers from underreporting with very low physician participation²⁴ and is confounded by local safety culture, making it difficult to measure progress over time²⁵,²⁷ or make meaningful comparisons between health systems. A study in oncology showed that both the Global Trigger Tool and voluntary reporting were insufficient to comprehensively detect safety events, with little overlap between the two methods and a focus largely centred on commission-related events.¹⁹,²⁹ Root cause analysis has been applied to event reports to address underlying causes. Still, this method of safety review tends to overlook interconnected causes and focus on identifying one linear cause.^{30 31} While patient-safety-event data collection methods have evolved, without an associated evolution of broadened definition of safety events, the lessons learnt regarding the types of events have not substantially changed over the past 20 years.³²

Incremental adaptations to current measurement practices have failed to improve healthcare delivery substantially. While strategies to improve the capture rate of patient-safety events have been suggested, these fail to evolve how patient safety is defined. In 2015, the Learning Health System Collaborative (LHSC) was established to expand safety-event definitions to encompass care omissions and commissions impacting the patient. This was achieved by incorporating both Safety-I and Safety-II concepts and drawing from high-reliability organisation (HRO) principles, resulting in a more comprehensive approach to identifying patient-safety events.^{33 34}

Using a principle-driven implementation of consensus-based organisational learning, this study evaluates the generalisability of a standardised case-review methodology and safety-event taxonomy developed at a single academic medical centre.

1. To determine the feasibility of implementing a standardised case-review methodology and taxonomy across hospitals with varying sizes, geographic locations and patient populations.

2. To quantify the categories of opportunities identified in this methodology and compare them with traditional event definitions.

Methods

Multicentre LHSC

This multicentre retrospective cohort study reported all case-review data from 103 hospitals across the USA and Canada between 2016 and 2023. Participating hospitals ranged in size from 8 to over 2000 beds. Each hospital contributed case-review results to evaluate clinical experiences antecedent to an undesired outcome. The LHSC had a rolling enrollment that began in 2016, with the most recent hospitals beginning reviews during 2023. The Mayo Clinic Institutional Review Board reviewed the study protocol and determined it to be exempt from the requirement for IRB approval (IRB #24–0 07 340).

Implementation

A standardised implementation approach was used for every member of the LHSC and did not vary through time. There were no organisational or hospital-specific exclusion criteria for participating in the LHSC. Before implementation, leadership and key personnel from each participating institution agreed on principles rooted in the HRO framework, including (1) reviews focused on processes and systems of care, not peers; (2) deference to expertise—case reviews performed by frontline care team members; (3) reluctance to simplify—identification of every opportunity regardless of attribution or causation of harm; (4) shared taxonomy of opportunities for benchmarking; (5) 100% multidisciplinary, multispecialty consensus agreement for all identified OFIs and (6) site-specific responsibility to tailor case-review workflow to local resource constraints. A standardised and phased implementation plan was used at every site to support these foundational principles, regardless of size or affiliation. This included the following phases: (1) socialisation and alignment with the organisational strategic and operational plan; (2) onsite training for administrators and case reviewers; (3) data analysis and development of quality improvement objectives; (4) communication plan to share lessons learnt across the organisation and LHSC broadly.

Cohort and case selection

Cohort inclusion and exclusion criteria depended on the individual hospital’s choice of cohort definitions. Each LHSC site was provided training on cohort selection that included the following criteria: the cohort had an outcome that aligned with an organisational strategic goal (eg, improving sepsis), the quality office had baseline metrics indicating a need for improved outcomes and the cohort was meaningful to the frontline care team members. Cohorts were defined by an outcome, event or disease (eg, mortality, transfer to intensive care unit (ICU) or sepsis, respectively). There was no a priori screening to minimise selection bias, and consecutive or randomly selected cases within each cohort were reviewed regardless of age, gender, race, comorbidities, severity of illness or code status.

Case-review methodology

A standardised case-review process was implemented at all participating hospitals using the previously reported methodology developed at Mayo Clinic for mortality reviews.³⁵ This methodology used a population-based approach to enumerate the quality-of-care opportunities for a chosen cohort. Reviewers were active frontline care team members with current clinical expertise to recognise faulty care processes.³⁶

Methodology training

All participants (reviewers and committee members) were trained by a single individual using in-person and/or live online training sessions. Standardised training materials were provided for on-demand courses in a learning management system for sustainability and ongoing growth to scale local case-based learning.

Heuristic for OFI identification

The historical mental model used in case reviews is peer review and often the identification of only those considered preventable events. The LHSC chose to identify process and system failures regardless of causation, attribution or preventability.

The LHSC defined a shared mental model used in the implementation and reviewer training as a heuristic framework for interpreting clinical experiences in each case review. Participants were instructed in two tasks prior to commencing a case review or attending committee case discussions. First, mindfully pause and recall that this process is separate and distinct from peer review in which it does not seek to identify fault. Second, envision the clinical case as if a dearest loved one underwent the clinical experience described by the medical record. OFIs were identified as any clinical experience the reviewer or committee member would not desire for their dearest loved one. At each training session and subsequent committee meetings, the following question was asked: ‘is there anything you would have wanted to go differently or better if this was your loved one?’

The definitions and examples of the categories available for selection in the data registry are listed in table 1.

Table 1. OFI definitions and examples.

OFIs	Opportunity definitions	Examples
Gratitude	When reviewing cases, great care is also discovered. This is defined as any care discovered that the reviewer would wish their most loved family member could receive if ill.	Goals of care conversation occurs in the first 5 min of an emergency department visit with a very frail, old, chronically ill patient.
End-of-life	Issues that occur near the end-of-life that do not maximise the patient’s goals of care preferences, comfort and/or dignity.	A patient who is very short of breath at the end-of-life without symptom management by the care team. A very old patient (>95) is intubated and placed on a ventilator for respiratory failure without consulting the patient or family regarding wishes for artificial life support.
Treatment	The absence, delay or intensity of treatment delivered to patient indicated for degree of illness.	Critical treatment needed for serious illness is ordered but not delivered until the following day as opposed to the day it was ordered.
Documentation	The absence of information in a patient’s medical record (required or otherwise) that prevents a clear understanding of the patient’s current clinical condition.	Examples include required documentation completely lacking, conflicting information within the medical record, copy/paste of prior notes without updates to reflect the current situation and missing vital signs.
Delayed diagnosis	Delay in clinical diagnoses measured in days to weeks (or missed altogether) from onset of new symptoms to time of diagnosis.	Patients with more than 3 days of progressive mental status changes in the hospital do not have mention of a delirium evaluation nor a diagnosis of cognitive impairment anywhere in the record.
Communication	Lapses in sound communication practices needed for effective multidisciplinary care.	The nurse calls the physician regarding concerns about a patient’s change in clinical status, and calls/pages are not returned.
Triage	Mismatch of clinical setting and patient’s severity of physical illness.	Patients with severe hypotension admitted to the general care unit instead of the ICU.
Diagnostic	Missing or delayed tests, assessments, specialty input, clinical decision-making, etc.	Patient documentation of persistent gastric symptoms does not have an abdominal physical exam laboratory or appropriate laboratory studies ordered to evaluate.
Transitions of care	Loss of key information with movement of patient among housing areas, hospitals or with changes of therapists/doctors.	Key portions or patient’s history (eg, code status) not included in transfer documentation.
Surgical/procedural	Complications of surgeries or procedures.	Examples include patients with extravasation of intravenous medication, small bowel laceration during a gynaecology case or pneumothorax during central line insertion.
HAI	Nosocomial infection that is not present on admission.	A patient develops a wound infection following surgery.
Prophylaxis	Failure to provide appropriate preventative treatment.	Fall or aspiration precautions not implemented.
Deterioration	The clinical team does not recognise, evaluate and/or treat key signs and/or symptoms that signal impending clinical deterioration (falling blood pressure, increasing respiratory rate or pulse rate, etc)	The patient is breathing faster and faster over a 12-hour period and ultimately requires emergent intubation for respiratory failure. During the 12-hour decline, there was no documentation of a bedside evaluation, no orders and no treatment implemented.
Technology/equipment	Deficit of any technology or equipment during the provision of patient care.	Electronic health records unexpectedly stop functioning during a patient care encounter.
Medication/blood	Adverse effects of administered medication or transfusion of blood products.	Respiratory failure following the administration of opioid.
HAC	Medical condition developed by a patient in the hospital that was not present on admission.	The patient falls when hypotensive during hospitalisation and develops acute kidney injury requiring dialysis following CT scan with contrast needed for disease evaluation.
Care team	Uncoordinated multidisciplinary care or lack of continuity of care during hospitalisation.	Patient has multiple specialty providers giving input but no one ordering or coordinating care process.
Resources in the care continuum	Lack of needed resources recommended for patient is not available in the following care environment.	Patient with stage 4 chronic kidney disease without access to a nephrologist or potential future dialysis in the community.

HAChealthcare-associated conditionsHAIhealthcare-associated infectionsICUintensive care unitOFIopportunity for improvement

Case-review protocol

Through the high-level case-review life cycle (figure 1), cases were assigned to two or more independent clinical reviewers of different disciplines (eg, nurse, pharmacist, physical therapist and doctor). Reviewers at each site received instruction on a standardised seven-step case-review methodology, which efficiently directed them through different portions of the electronic medical record. Reviewers were advised to spend no more than 45 min in the medical record and 15 min documenting their findings in the electronic registry. The registry contained drop-down options for the LHSC-developed taxonomy of opportunities for improvement (OFIs). Reviewers could also assign contributing human factors to any OFI (eg, communication, teamwork, organisational and leadership issues or individual provider issues) as adapted from the Department of Defence.³⁷

Figure 1. High-level case-review life cycle. Workflow used by all members of the LHSC to perform case reviews. Sequential or randomly selected cases within a cohort are assigned to at least two case reviewers from different disciplines (eg, a nurse and a physician). Each independent review is shared among a multidisciplinary, multispecialty committee where a facilitated discussion occurs to reach consensus regarding whether the issues identified by the reviewers are counted as OFIs. Selected cases are shared with clinical areas to ‘CTL’ and promote the generalisation of knowledge learnt from that case via case-based education opportunity. After the chosen case reviews are completed, the cohort data are aggregated, analysed and shared broadly across clinical practice, quality and leadership stakeholders. CTL, close the loop; LHSC, Learning Health System Collaborative; OFIs, opportunities for improvement.

Case-review reliability

The requirement of multidisciplinary reviews necessitated the reconciliation of findings between reviewers. The medical record review process was intentionally designed to yield as much learning as possible from a multidisciplinary, multispecialty perspective. As such, inter-rater reliability was not an explicit goal. Each discipline was expected to have different findings, given they are clinically trained under various rubrics, have different scopes of clinical practice and frequently use different portions of the electronic health record.

Reconciling the multidisciplinary case reviewers, findings occurred during monthly to bimonthly clinical case discussion committee meetings, each lasting 60–90 min. At a minimum, the case-review committees were attended by the other multispecialty clinical reviewers (nurses and providers) and members of the organisation’s quality department. Other clinical staff (social work, physical therapy, pharmacy, chaplain, etc) participated in the discussion to broaden the perspectives in the dialogue. Reviewers presented the patient’s clinical experience and identified OFIs to the committee, where clinical details were clarified and differences discussed and reconciled through a thorough consensus-building process of all reviewers (and additional subspecialty clinical findings with other reviewers through dialogue with the multidisciplinary, multispecialty group of clinical experts). A complete consensus among this larger clinical group was required to finalise which issues found during the individual reviews were recorded in the data as OFIs. These meetings were facilitated under the Chatham House Rule.³⁸

Statistical analysis

The aggregate dataset for the LHSC was deidentified before receipt for analysis. The research registry anonymised each case and assigned it a unique identification number. Due to restrictions in collaborator contractual agreements, analyses did not include the names of individual hospitals, participating health systems or geographical locations.

In addition to patient demographics (age and gender) and high-level hospital descriptors (eg, number of beds), admission and discharge type of hospital floor (eg, general care, ICU, etc), clinical service line, code status and diagnoses were collected for all patients. Reviewers manually entered OFIs and corresponding patient locations, clinical service lines and contributing human factors. The category of OFI ‘opportunity for gratitude’ was excluded from all analyses because it represented a recognition of excellent care rather than an omission or commission of care.

Hospitals participating in the LHSC were compared based on several hospital characteristics, which included the number of staffed beds, academic status (defined as any medical school affiliation), designated trauma level, urban versus rural setting and the overall hospital quality star rating from the Centres for Medicare and Medicaid Services (CMS) for the same quarter as the most recent case review. For each category, statistics were reported for the number of case reviews per hospital, the number of OFIs per case and the proportion of cases with OFIs. Median and IQR were reported for continuous variables to capture their non-normal distributions. Categorical variables were reported as N (%).

To test for differences in the identification of OFIs among hospitals and patients, a logistic regression was performed using the presence of one or more OFIs as the dependent variable. Independent variables included all hospital characteristics (number of staffed beds, academic status, designated trauma level, urban setting and star rating) and the patient characteristics of age, gender, length of stay, admission code status and discharge code status, and an indicator for mortality (including in-hospital mortality and discharge to hospice). Missing values were present for age (n=5 (0.03%)), length of stay (n=96 (0.5%)), admission code status (n=1241 (6.5%)) and discharge code status (n=1212 (6.3%)). Multiple imputations by chained equations using predictive mean matching on five iterations and imputations were performed to replace missing values before running the logistic regression. Statistical analyses were conducted using R version 4.2.2.

Results

Description of participating sites

Hospitals participating in the LHSC represent a wide spectrum of sizes, types, services and qualities. The LHSC had a rolling enrollment that began in 2016, with the most recent hospitals (8) beginning reviews in 2023 for 103 hospitals. This included 13 hospitals that enrolled for a predetermined pilot duration (1 year) or a specific number of case reviews for research; 9 hospitals that were initially included when their entire health system joined subsequently chose not to continue following the training, citing competing priorities and resource constraints; and 1 hospital that withdrew when the sponsoring leader left the organisation. Of the remaining 80 hospitals that implemented as a part of ongoing quality and safety work, they performed reviews for an average of 34 months (4–88 months, depending on the timing of LHSC enrollment).

Description of reviewed cases

A total of 19 181 cases were reviewed across all the hospitals participating in the LHSC. Patients were primarily ageing adults (median age 71, IQR 60–81) and more likely to be male (n=10 469 (54.6%)). Length of stay varied but was typically several days (median 6, IQR 3–11). Code status shifted from primarily full code at admission (n=14 013 (78.1%)) towards do not resuscitate/do not intubate (DNR/DNI) (n=8121 (45.2%)) and comfort care (n=6224 (34.6%)) at discharge. Most patients were either deceased or transferred to hospice at discharge (n=18 321 (95.7%)) given that the vast majority of sites reviewed mortality cases as their first cohort.

Selected study cohorts

Most hospitals (n=91 (88%)) chose mortalities as their study cohort. However, many selected other cohorts as their local review process matured. For this analysis, 119 unique hospital-named cohorts were identified. Figure 2 shows the most common categories of cohorts ordered by the percentage of hospitals studying each cohort. The specific cohorts selected by hospitals were sometimes more targeted (ie, 30-day sepsis mortality or pneumonia readmissions) and could be included in more than one of the broader categories shown.

Figure 2. Percentage of hospitals studying common cohorts by category. Cohorts were grouped into common categories. The percentage of all hospitals participating in the LHSC with at least one case review from each cohort category is presented here. The specific cohorts studied could be more targeted (ie, 30-day sepsis mortality or pneumonia readmissions) and could be included in more than one of the broader categories. ICU, intensive care unit; LHSC, Learning Health System Collaborative.

Results by hospital type

The median number of reviews per hospital was 107, with 12 714 cases (66.3%) identifying one or more OFIs and a median number of OFIs per case of 1. LHSC hospitals were primarily located in urban settings (n=83 (80.6%)), with 50 (48.5%) being academic centres. A majority (n=66 (64%)) of hospitals had a trauma designation, with 53 (51.5%) scoring a 4- or 5-star CMS rating (table 2). Every category was represented by multiple hospitals, with only CMS star ratings of 1 or 2 having fewer than ten hospitals. Large, urban, teaching and comprehensive trauma hospitals typically completed the most case reviews. The number of OFIs per case and the proportion of cases with OFIs varied between hospitals, but each hospital category consistently had OFIs identified in at least 56% of cases.

Table 2. Hospital characteristics.

Variable	HospitalsN (%)	Number of case reviews per hospitalMedian (IQR)	Number of OFIs per caseMedian (IQR)	Percentage of cases with OFIsN (%)
Total	103	107 (25–222)	1 (0–3)	46 444 (66.3%)
Number of staffed beds
< 100	34 (33%)	25 (4–89)	1 (0–2)	1352 (56.6%)
100–200	24 (23%)	126 (85–215)	1 (0–3)	2625 (62.2%)
200–400	19 (18%)	138 (80–252)	2 (0–4)	2327 (70.5%)
> 400	26 (25%)	210 (100–372)	2 (0–4)	6383 (69.1%)
Urban setting
Urban	83 (81%)	126 (43–264)	1 (0–3)	11 355 (65.7%)
Rural	20 (19%)	37 (16–107)	2 (0–4)	1332 (70.9%)
Academic status
Teaching	50 (49%)	137 (49–334)	1 (0–4)	8540 (67.8%)
Non-teaching	53 (51%)	85 (22–169)	1 (0–3)	4147 (63.3%)
Designated trauma level
1	16 (16%)	210 (100–323)	2 (1–5)	3408 (77.7%)
2	12 (12%)	328 (88–391)	1 (0–3)	2855 (56.8%)
3	26 (25%)	114 (48–146)	2 (0–4)	2346 (69.3%)
4	12 (12%)	14 (4–53)	1 (0–3)	393 (64.2%)
Not designated	37 (36%)	88 (16–212)	1 (0–3)	3685 (64.1%)
CMS overall star rating
1	3 (3%)	133 (117–176)	2 (0–4)	310 (68.6%)
2	8 (8%)	224 (76–345)	1 (0–3)	1140 (64.3%)
3	22 (21%)	118 (71–234)	1 (0–3)	3594 (61.1%)
4	30 (29%)	118 (31–268)	1 (0–3)	3320 (65.3%)
5	23 (22%)	127 (40–211)	2 (0–4)	3400 (70.0%)
Not designated	17 (17%)	14 (2–79)	4 (1–8)	923 (84.0%)

CMSCentres for Medicare and Medicaid ServicesOFIsopportunities for improvement

Logistic regression findings

Following multivariable logistic regression analysis, it was found that all hospital and patient characteristics except for female sex were significantly associated with the presence of at least one OFI when controlling for all other variables in the model (p<0.0001) (table 3). Larger hospitals were associated with increased odds of identifying OFIs (OR 2.01>400 beds vs reference category<100 beds, p<0.001). Patients in urban (OR 0.58, p<0.0001) and academic (OR 0.75, p<0.0001) hospitals had lower odds of at least one OFI compared with rural and non-teaching hospitals. Patients from level 1 designated trauma centres were more likely to have OFIs identified compared with both all other trauma centres and undesignated hospitals (p<0.0001). While CMS’s overall star ratings showed less differentiation between ratings, patients in hospitals without a designated star rating had the highest odds of at least one OFI compared with a star rating of 1 (OR 3.61, p<0.0001). Patient characteristics of older age (OR 1.01, p<0.0001), longer length of stay (OR 1.03, p<0.0001), code statuses of DNR/DNI (OR 1.63, p<0.0001) or full code (OR 2.23, p<0.0001) compared with comfort care were all associated with higher odds of at least one OFI. Patients who died during the hospitalisation or were discharged to hospice were less likely to have OFIs identified compared with those who survived (OR 0.49, p<0.0001).

Table 3. Multivariable logistic regression for presence of OFIs.

Variable	OR (95% CI)	P value
Number of staffed beds
< 100	Reference category	–
100–200	1.59 (1.41 to 1.80)	< 0.0001
200–400	1.86 (1.63 to 2.12)	< 0.0001
> 400	2.01 (1.73 to 2.33)	< 0.0001
Urban setting
Rural	Reference category	–
Urban	0.58 (0.51 to 0.66)	< 0.0001
Academic status
Non-teaching	Reference category	–
Teaching	0.75 (0.68 to 0.84)	< 0.0001
Designated trauma level
1	Reference category	–
2	0.44 (0.40 to 0.49)	< 0.0001
3	0.67 (0.58 to 0.76)	< 0.0001
4	0.44 (0.36 to 0.54)	< 0.0001
Not designated	0.50 (0.45 to 0.56)	< 0.0001
CMS overall star rating		< 0.0001
1	Reference category	–
2	1.29 (1.02 to 1.63)	0.0368
3	1.03 (0.83 to 1.29)	0.7644
4	1.26 (1.01 to 1.58)	0.0394
5	1.46 (1.17 to 1.82)	0.0009
Not designated	3.61 (2.73 to 4.78)	< 0.0001
Age	1.01 (1.00 to 1.01)	< 0.0001
Female	1.06 (1.00 to 1.13)	0.0554
Length of stay	1.03 (1.03 to 1.03)	< 0.0001
Admission code status
Comfort care	Reference category	–
DNR/DNI	1.63 (1.34 to 1.98)	< 0.0001
Full code	2.23 (1.84 to 2.70)	< 0.0001
Discharge code status
Comfort care	Reference category	–
DNR/DNI	1.36 (1.26 to 1.47)	< 0.0001
Full code	1.29 (1.17 to 1.43)	< 0.0001
Mortality	0.49 (0.40 to 0.60)	< 0.0001

CMSCentres for Medicare and Medicaid ServicesDNR/DNIdo not resuscitate/do not intubateOFIsopportunities for improvement

Identified OFI categories

The categories of OFIs identified across all case reviews are displayed in figure 3. Most OFIs identified are non-traditional events, with the categories of end-of-life, documentation and treatment/care alone accounting for 25 980 OFIs (56%) (figure 3). When other omissions of care (eg, communication, recognition of deteriorating patients, delayed or missed diagnosis and diagnostic opportunities—light blue bars in chart) were added to the top three categories, these totalled 41 439 OFIs (89%). A total of 6386 opportunities for gratitude were identified, primarily consisting of general gratitude, a well-documented clinical course and proactive end-of-life discussion.

Figure 3. Pareto of OFI categories. The categories of OFIs identified across all case reviews are visualised in this Pareto chart with bars for category counts and the green line for the cumulative percentage. To reflect our expanded definition of patient safety, category bars were coloured by whether they were included in the traditional definition of patient-safety events using the IHI global trigger tool as a reference.³¹ The names of the OFI categories here have been shortened for visual clarity. For full names, definitions and examples, refer to table 1. OFI, opportunity for improvement.

Discussion

A multidisciplinary, multispecialty case-based organisational learning framework developed in one academic medical centre was broadly applied to more than 100 hospitals of varying settings, sizes and types. Whether in a small community hospital or a large academic medical centre, the principles underlying this approach remain inherently scalable and adaptable. The LHSC members committed to a broadened definition of patient-safety events, including traditional commission events and omissions of care leading to harm. By leveraging a standardised methodology and taxonomy, health systems can effectively name, define, quantify and analyse clinical care and patient-safety OFIs, irrespective of organisational scale or resource constraints.

The differences in definitions and approaches for identifying and measuring adverse events used in published literature make comparisons challenging. However, the significantly different findings pique curiosity and drive the desire for further exploration. Overall, rates of adverse events are approximately 1 in 3 hospital admissions or less, with adverse drug events and surgical or procedural complications being the most common.^{8 11 24 32 39} In contrast, every hospital in the LHSC reported occurrences of failures in care in more than 1 in 2 hospitalisations, with an average of 66% of reviewed cases having one or more OFI. The most common care opportunities were in end-of-life care (insufficient palliation), documentation and treatment (delayed treatment or clinical protocols not followed). There is published recognition of the impact of changes in medical practice on patient safety and the need to expand the definition of patient-safety events.³² Exploring these high-volume omission categories could reveal opportunities for systemic improvements, impacting a proportionally higher number of patients with these less-than-ideal clinical experiences.

Previous studies reported several factors that led to increased rates of adverse events: larger hospital size,⁴⁰ negative patient-safety culture⁴¹ and the lack of accreditation status or event reporting system.⁴² There was an increase in rates of OFIs from larger, rural and level 1 trauma hospitals. In addition, hospitals without a designated CMS star rating were more likely to have OFIs. While teaching hospitals had a higher percentage of OFIs, the relationship was reversed when controlling for other variables in the logistic regression, with non-teaching hospitals having higher odds of OFIs. This indicates the complexity of the interacting factors affecting the identification and presence of OFIs and is consistent with previous research demonstrating the inconsistent relationship between hospital teaching status and adverse events.⁴³ While the differing definitions of events and populations of patients reviewed do not allow for direct comparisons with previously reported associations, the findings in this study provide evidence for a relationship between hospital characteristics and the occurrence of OFIs. Exploring these correlations further could provide insights into targeted improvement strategies tailored to specific hospital contexts.

The relationship between patient characteristics and OFIs explored in the regression analysis shed light on potential associations between patient demographics and OFIs. Patients with OFIs trended older with longer hospital stays, which has also been the case in studies of adverse event rates.^{42 44} The relationship between a patient’s discharge code status and adverse event rates is unclear. The ‘comfort care only’ code status was less likely among patients who experienced OFIs. This complex issue may reflect decreased invasive and medical management when comfort is prioritised over cure. However, a lack of clear goals of care conversations early in the hospitalisation may result in an increased amount of care as terminally ill patients continue aggressive care and do not change their code status until late in their hospital stay, if at all.^{45 46} These and other issues around end-of-life were the most common opportunities identified by the LHSC.

This study has implications for evolving a definition and a growth in understanding patient safety from a frontline care team member perspective, further aligning patient-safety work with high-reliability principles. Using the LHSC’s heuristic to identify opportunities also increased the human centredness of this framework. Such changes in definition and perspective will require organisations to assess patient-safety practices and identify their areas for evolution.⁴⁷,⁴⁹ Quality and safety leadership will be challenged to allocate resources to address the high-volume events while remaining accountable to federal and state mandates and ongoing assessments of voluntary adverse event reporting. An ongoing analysis found that the LHSC identified nearly seven times as many OFIs as events found through voluntary reporting, with most OFIs representing omissions of care. Prioritising a portion of quality improvement resources to address some of the omission-related OFIs strategically may have significant benefits. Given the relatively high volume of these compared with the traditional adverse events, directed improvement could lead to a measurable improvement in outcomes and cost savings by reducing the frequency of these events. An exploratory analysis revealed a significant association between OFIs and longer length of hospital stay. Additionally, some hospitals self-reported improved outcomes following their involvement in the LHSC.

Strengths and limitations

This study has several strengths that contribute to advancing patient-safety research and practice. It is a multicentre study that includes hospitals of varying settings, sizes and types, enhancing the generalisability of the findings within the USA and Canada. The case reviews are performed by active frontline clinical staff with working knowledge of existing care processes, protocols and guidelines. The LHSC implemented a standardised training curriculum, OFI identification heuristic and taxonomy for all case reviewers to minimise variability and increase consistency in OFI identification and classification. This novel perspective shifts the conversation from the provider’s action (eg, lapse in medication order) to the patient’s clinical experience (eg, delay in receiving medication). A tone shift can increase the threshold for identifying OFIs revealing the opportunities to improve care throughout the patient’s journey in the healthcare system.

This study has the known limitations of a descriptive, retrospective cohort study, including confounding and case selection biases and statistical limitations. A more prominent issue is the inability to compare findings to the existing patient-safety literature because the LHSC used an expanded definition of patient-safety events. More work is needed to reconcile and integrate these definitions and philosophies. While leveraging clinical acumen is a strength of this study, it also introduces weaknesses in the study design. More than 80% of the OFIs identified across the LHSC and within each hospital were characterised as omissions of care, which required the reviewer to determine what was missing in the medical record. This introduces the potential for multiple cognitive biases, including, but not limited to, confirmation and anchoring bias, as well as an availability heuristic and the framing effect. These are not unique to this study and can occur in any clinical chart review. The LHSC combatted this by implementing standardised training, case-review methodology and taxonomy and requiring 100% multidisciplinary, multispecialty clinical case-review committee consensus for all OFIs included in the finalised dataset. While care team staff participating in training and case reviews may benefit from additional knowledge and skills or behavioural change, this was not analysed because it is not currently assessed as a part of implementation. Future studies should consider including an assessment of the educational impact. Finally, this work is laborious and requires LHSC organisational investment to approve frontline care team involvement to complete case reviews and attend consensus committee meetings. To meet the person-power demand, some sites decreased the number of reviews to create a feasible workload for the size of their team, other sites used innovative recruitment strategies to increase the number of reviewers available. For instance, adding nurse practitioners, physician assistants, and residents or fellows to the physician review group and worker’s comp nurses/injured nurses, ‘return to work’ nurses, rapid response team (RRT) nurses between RRT calls, clinical nursing specialists, and nurse managers to supplement the nursing reviewer pool. While many examples of creative solutions exist, they are not ubiquitously feasible. With the completion of this study, hospitals may locally modify the case-review process under the agreement that a multidisciplinary, multispecialty consensus process is used to finalise data for LHSC benchmarking.

Conclusion

This study demonstrated the feasibility of implementing a standardised case-review learning and analysis methodology across hospitals of differing sizes and affiliations. To our knowledge, this study is the first to describe the potential implications of broadening the definition of patient safety. Compared with the traditional definition of patient safety, which largely includes outcomes from unintended acts of commission, the highest volumes of safety-related undesirable patient experiences are associated with omissions of care. This illustrates the impact of a no-blame, patient-centred approach to identifying OFIs.

Data availability statement

Data are available on reasonable request.