What Safety Events Are Reported For Ambulatory Care? Analysis of Incident Reports from a Patient Safety Organization

Abstract

Introduction:

Healthcare staff document patient safety events using incident reporting systems which are compiled within Patient Safety Organization databases. We sought to describe the characteristics of incident reporting behaviors for ambulatory care from in-situ reporting systems from the United States.

Methods:

We analyzed safety reports in ambulatory settings collected from a patient safety organization comprising 400 hospital members in 10 states, from May 2012 – October 2018. We included all events involving moderate harm, severe harm, and death. We randomly sampled 200 events with no harm, 200 events with missing harm and 600 events with mild harm. We deductively coded incident types and if patient or caregiver challenges were involved. We conducted a multivariate logistic regression to identify predictors of higher harm (severe harm and death) among safety events reported. We conducted sensitivity analyses, assessing a broader harm cutoff and proportional weighting of the sub-sampled lower harm events.

Results:

Of 2,701 events, there were 41 (1.5%) unsafe conditions, 76 (2.8%) near miss, 267 (9.9) no harm, 926 (34.3%) minor harm, 1,180 (43.7) moderate harm, 159 (5.9) severe harm, and 51 (1.9%) death. Most were from outpatient subspecialty care; 5% from the home/community and 2% from primary care. Medication-related events were most common (45%). In multivariate analysis, diagnostic errors (aOR 11.5), patient/caregiver challenges (aOR 2.2), and psychiatric settings (aOR 5.0) were associated with higher harm.

Conclusions:

Outpatient reporting systems are limited for primary care and home/community settings, but ambulatory care systems report more harmful events related to diagnosis and patient and caregiver challenges. Improved standardization of reporting, focus on diagnosis, and novel approaches of safety reporting that engage patients will improve capture of events affecting patients and enable development of system-level solutions.

INTRODUCTION

Ambulatory patient safety, defined as harm occurring to patients in primary care, outpatient specialty care, and the home and community, is poorly understood in the United States (US). From what is known, ambulatory adverse events are estimated to cause up to 2,587 deaths, 75,000 hospitalizations, and 4 million additional office visits in the US.¹ Prevalence estimates vary widely; a systematic review concluded that there are between <1 and 24 patient safety incidents per 100 primary care consultations, and that between <1% and 44% of primary care adverse events are associated with severe harm.² Given that ambulatory care comprises the bulk of medical care in the US, even a small percentage of harmful incidents can affect a large number of people.

Improved understanding of ambulatory safety is a priority for healthcare organizations such as Agency for Healthcare Research and Quality, the National Academy of Medicine, and the World Health Organization.^3–5 However, patient safety research has traditionally focused on inpatient care, where there is a large degree of control over clinical processes. There is also a perception that ambulatory care is safer or low-risk.⁶ However, the dynamics of ambulatory safety are more complex, involving multi-professional teams, patient and caregiver self-management of complex conditions, and transitions in care across multiple specialties.⁷ Given these differences, ambulatory care safety requires a unique focus apart from inpatient safety initiatives.⁸

Adverse event reporting systems enable exploration of ambulatory safety. These systems are typically electronic and collect voluntary incident descriptions from healthcare personnel.⁹ Many hospitals participate in safety incident data-sharing through Patient Safety Organizations (PSO). PSOs are legally protected entities that store patient safety data from reporting systems for the purposes of improving patient safety and healthcare delivery.¹⁰ PSOs were established through The Patient Safety and Quality Improvement Act of 2005, prompted by the Institute of Medicine report To Err is Human.¹¹ Participation in a PSO is a requirement for healthcare networks desiring accreditation by the Joint Commission. PSOs have traditionally focused on inpatient-related safety, but also contain outpatient data from member hospitals’ ambulatory clinics.

We sought to explore the database of CHPSO, a PSO located in California, to analyze the characteristics of events reported that occurred in the ambulatory setting. While these incidents do not comprise all outpatient events, our goal was to understand the reporting behaviors for healthcare staff working in ambulatory care settings by describing the characteristics of adverse events that are actually entered into reporting systems. Given the team-based nature of outpatient care as well as the impactful role of patients and family caregivers, we utilized a stakeholder-engaged approach with an advisory council collaborating in the research process.

METHODS

We conducted a retrospective, cross-sectional study combining a deductive data coding process and descriptive statistical analysis of patient safety incident reports from a multi-state PSO database. Throughout our study, we worked with a Stakeholder Advisory Council of 10–12 members, including patients, caregivers, primary care clinicians, nurse, medical assistant, and pharmacist; the Stakeholder Council provided input on data abstraction, analysis, interpretation and dissemination of results. The Stakeholder Council met three times over the course of the study year with ad-hoc email communications between meetings.

Setting

The CHPSO Database (formerly known as the California Hospital Patient Safety Organization) is located in Sacramento, California. CHPSO is a federally-designated Patient Safety Organization (PSO) and comprised 400 member hospitals across 10 states in the US, with 1.5 million total safety incidents at the time of this analysis.¹² CHPSO member sites submit incident reports to the CHPSO database from their safety reporting systems according to their own individual facility-specific policies.

Data Selection

We imported all de-identified incident reports that were labeled as “Outpatient,” that occurred between the dates of May 2012-October 2018. We included events in primary care, outpatient specialty care, dialysis, home/community, behavioral facilities, and residential nursing facilities. We excluded events occurring the Emergency Department (ED). We excluded events that were too confusing to analyze and events clearly related to inpatient care.

Variables

We included variables available through the CHPSO data form; these included patient gender, age, hospital size, hospital ownership and academic status, geographic location (i.e. large metropolitan area, small metropolitan, rural, etc.), event reporter role, and practice setting of incident. CHPSO had numerous categories for event reporter. For our analysis, we grouped physicians and Allied health practitioners as “Clinician.” We grouped RN, LVN and LPN titles under the “Nurse” category. Other categories included “Pharmacist,” “Administrator,” and “Other.”

As race/ethnicity data was only present 0.006% of all events, we did not include it. CHPSO events have a designated incident type and harm level in accordance with AHRQ Common Formats.¹³ We included all events designated with the harm levels death, severe harm, and moderate harm. Using a random number generator, we selected a random 600 examples of “mild harm” events, 200 “no harm,” and 200 of “missing harm.” We saturated our sample with the higher harm-level events, but included some events from lower harm levels according to Heinrich’s Law, which theorizes that addressing causes to minor accidents will reduce accidents causing major harm.¹⁴ We defined our primary outcome as the dichotomous value of “clinically significant harm,” including events involving “severe harm” (involving a life-threatening or critical condition) and death.

We defined incident type based on synthesis of precedent taxonomies of ambulatory safety events.^15–17 In collaboration with our Stakeholder Council, we developed a variable to capture if patient or caregiver challenges were mentioned that could have inadvertently contributed to the event in question. The Stakeholder Council co-created the name and definition of this variable, as well as the sub-categories for specific challenges, with the goal that this variable would capture a range of factors that may or may not be within the patient’s direct control and does not imply direct patient fault for the event mentioned. For example, if a patient experienced hyperglycemia due to being unable to afford insulin, we would respond “Yes” to “Were there specific challenges experienced by the patient, family and/or caregiver that could have accidentally contributed to this event?.” We would then code the categories of “patient/caregiver challenges” as including “self-administration of medication” and “socioeconomic barriers.”

Data Abstraction

We adapted a precedent incident report analysis protocol from the UK¹⁸ in conjunction with input from the Stakeholder Advisory Council. We developed our abstraction form using Research Electronic Data Capture (REDCap), a secure web-based tool for collecting and storing data.¹⁹ Four coders (Authors 1–4) independently reviewed and coded events. Coders dual-coded a sample set until their concordance achieved a kappa for of greater than an a priori cutoff set at 0.7 for patient sex and harm to ensure quality control. Events in the dataset were randomly assigned to each coder; 20% of all events coded were dual-coded with another member of the study team to verify a kappa > 0.7 for patient sex and harm level to prevent coder drift.

Using REDCap, our team abstracted specific variables available in the CHPSO dataset from the structured fields and event narratives, including individual patient demographics, practice setting, hospital characteristics, and geographic characteristics. During the coding process, the study team met frequently to discuss discrepancies and determine definitions for unusual events. If the assigned harm level did not correspond to objective details in the event narrative, team members reclassified the harm level based on the World Health Organization’s International Classification for Patient Safety.²⁰

Sample Size

Presuming a binomial distribution; based on estimates from a random sample of 100, we estimated that a final sample size of 2500 events will provide a >99% probability of detecting at least 10 of the rarest event types and >80% probability of detecting 20 of the rarest event types.

Data Analysis

We first conducted crosswise descriptive statistics to assess frequencies of incident type, harm level, and patient/caregiver involvement. We then conducted a stepwise logistic regression assessing for association with higher harm events (severe harm or death); first including individual patient characteristics, then practice characteristics, then geographic/hospital characteristics. We excluded variables that were collinear and variables that resulted in overfitting of the model. We excluded equipment-related events as there were zero higher-harm events of this type. For sensitivity analyses, we assessed if a different cutoff point in our linear outcome of “higher harm” changed our associations. As additional sensitivity analysis, we reran our analysis applying proportional weighting to variables with “no harm” and “mild harm” levels based on the proportion we had subsampled out of the original dataset.

RESULTS

The CHPSO database contained 37,132 events tagged as occurring in the “outpatient” setting. Of the 400 CHPSO members, 165 submitted outpatient events over the 6-year time period, ranging from 1–3917 reports per site (median 40 events, interquartile range 159). Of these, based on our sampling protocol we abstracted 2,824 events. Of these, we excluded 123 events that were unclear or based on description clearly occurring in the inpatient setting. Our final analysis comprised 2,701 events (Figure 1). The frequencies of harm levels of events were: Unsafe conditions: 41(1.5%), Near miss: 76 (2.8), No harm: 267 (9.9), Minor harm: 926 (34.3), Moderate harm: 1,180 (43.7), Severe harm: 159 (5.9), Death: 51 (1.9), Insufficient detail/missing: 1 (0.04).

Figure 1.

CONSORT diagram of included incident reports

Full demographic characteristics of the sample are listed in Table 1. Female patients were involved in 53% of events and the most common age group was adults ages 18–64. Events were predominantly from medium-large metropolitan areas. The most common practice setting was outpatient surgical, followed by outpatient medical specialty. Only 2% were clearly from a primary care setting and only 5% were in the home or community setting. The predominant characteristics of included hospitals were those with 0–200 beds, government-affiliated, academic-affiliated. Nurses were the most common event reporter (31% of known event reporter roles), followed by pharmacists and then clinicians.

Table 1.

Demographics of included events

Sample Characteristics	Frequency (%); n=2,701
Patient gender
Female	1,435 (53.1)
Male	1,062 (39.3)
Transgender	1 (<1)
Unknown	203 (7.6)
Patient age
0–17	56 (2.1)
18–65	487 (18.0)
>65	353 (13.1)
Unknown	1,805 (66.8)
Geographic location
Small metropolitan area/rural	114 (11.0)
Medium/large metropolitan area	2,289 (84.8)
Unknown	298 (4.2)
Hospital size
<200 beds	1,788 (66.2)
200+beds	905 (33.5)
Unknown	8 (0.3)
Hospital ownership
Public/Government-affiliated	1,818 (67.3)
Private	711 (26.3)
Other/Unknown	172 (6.4)
Academic Affiliation	1,845 (68.3)
Event reporter
Nurse	808 (29.9)
Pharmacist	106 (3.9)
Clinician	87 (3.2)
Administrator	21 (0.8)
Other/Undisclosed	1,679 (62.2)
Event setting/specialty
Medical/Surgical specialties	1,576 (58.3)
Home	141 (5.2)
Physical/Occupational therapy	68 (2.5)
Primary Care	56 (2.1)
Dialysis	53 (2.0)
Laboratory/Imaging	49 (1.8)
Psych	16 (0.6)
Other/Unknown	742 (27.5)
Event harm severity
Unsafe conditions	41(1.5)
Near miss	76 (2.8)
No harm	267 (9.9)
Minor harm	926 (34.3)
Moderate harm	1,180 (43.7)
Severe harm:	159 (5.9)
Death	51 (1.9)
Insufficient detail/missing	1 (0.04)
Incident type
Medication	1,224 (45.3)
Clinical deterioration	674 (25.0)
Fall	400 (14.8)
Treatment/Procedures (including dialysis)	359 (13.3)
Patient behavioral complaint	310 (11.5)
Administrative/Documentation/Referrals	291 (10.8)
Communication	218 (8.1)
Laboratory/Imaging	153 (5.7)
Equipment failure	77 (2.9)
Diagnosis	56 (2.1)
Transport	34 (1.2)
Other	67 (2.5)
Patient/caregiver challenges mentioned in event	499 (18.5)

Medication adverse events were the most common ambulatory incident type; the majority of these events described chemotherapy or immunologic infusion reactions. Other less-common medications were anticoagulants (101 events), antibiotics (52 events), and opioids (31 events). Other notable incident types were clinical deterioration (patient had unanticipated clinical decline, triggering an event report); falls; treatment events; administrative/documentation errors; reports about patient complaints or patient behaviors, equipment breakdowns, and diagnostic errors (e.g. missed, delayed or incorrect diagnosis leading to patient harm). Exemplar quotes for each incident type are listed in Table 2.

Table 2.

Examples of incident categories from ambulatory incident reports

Event type	Example Quote
Medication	Patient with history of thrombus, started on warfarin. Developed bleeding with supratherapeutic INR and dose was lowered. Thrombus resolved, warfarin stopped. New thrombus diagnosed, MD restarted prior high dose warfarin. Follow-up anticoagulation clinic provider did not review prior bleeding history and continued higher dose. Patient missed a visit for INR lab monitoring. 7 days later was hospitalized for supratherapeutic INR.
Clinical deterioration	Patient came in for x-rays of spine. Patient with known seizure disorder and other co-morbidities. Patient had a seizure when transferred from x-ray table to gurney. Patient stabilized and sent to ED.
Fall	Patient fell at home, noted a bruise. Next day, had more difficulty with ambulation. Following day, patient could not get out of bed. Home nurse evaluated and recommended going to ED. Patient admitted with compression fracture.
Treatment/Procedures	Patient underwent excisional surgery for lesion that had been biopsied and found to be carcinoma. Patient pointed to area to biopsy, underwent biopsy but didn’t find carcinoma. After further review, MD found that other uploaded photos didn’t match site that patient had pointed to. Patient contacted for repeat biopsy of original location.
Patient behavioral complaint	RN was bringing patient in to clinic and asked about patient symptoms. Patient smacked RN on buttocks and made inappropriate comments. RN notified physician. Patient apologized. Patient noted to have made other inappropriate comments in the past.
Administrative/Documentation/Referrals	Patient ran out of catheter supplies (chronically needing self-catheterization) which were not sent by vendor, due to missing paperwork and MD signature. MD had not received any form. Patient developed urinary tract infection and had to go to ER for treatment.
Communication	Telemedicine phone was left unplugged during a personnel change, where person who was responsible for checking messages did not check messages. More than two dozen patient messages were unchecked. One patient was without insulin for multiple weeks. On safety committee review; protocol notes that patients are supposed to request refills through their pharmacy.
Laboratory/Imaging	Patient had x-ray ordered, did not receive results. Patient presented later to another facility and had fracture. Imaging company did not contact ordering facility regarding initial x-ray results.
Equipment failure	Patient’s chemotherapy port line disengaged at home during bath. Patient presented to clinic with bleeding from port site. Clinic was unable to reinsert after several tries, was forwarded to another clinic where port line was replaced successfully.
Diagnosis	Positive FOBT. Barium enema showed colon lesion. Referred to colonoscopy. Approved by GI but not scheduled. Over one year later, colonoscopy found obstructing colon cancer. Referral was not put in, apparently scheduler unable to reach patient by phone x 2, letter sent to patient as well and message sent to provider. Scheduler was temp and did not understand urgency of the clinical issue.
Transport	Patient presented to clinic with hypoxia to 70’s, improved to 80’s on supplemental oxygen. No rapid response initiated in clinic. Resident sent patient to ED but did not follow workflow in place; transferred to ED by wheelchair without nursing. No report given to RN. This transport could have gone badly, thankfully it did not.
Other	MD arrived one hour late to clinic, patients were waiting.

Note: quoted reports are altered or paraphrased to minimize identifiability. INR= International Normalized Ratio. ER= Emergency Department. MD = Medical Doctor. RN=Registered Nurse. FOBT = Fecal Occult Blood Test. GI=Gastroenterology.

Of coded events, 18.4% included reports referencing patient and caregiver challenges. These included behavioral factors, such as following clinical advice or recommendations, timely reporting of symptoms, self-administration of medications, and caregiver behaviors. These also included intrinsic barriers, such as substance use/misuse, socioeconomic limitations, patient education, co-morbid conditions, and disability.

In stepwise multivariate logistic regression, the incident types associated with higher harm in this sample (severe harm and death) were diagnostic adverse events (aOR 11.5, 95% confidence interval (CI) 5.70–23.2) and clinical deterioration (aOR 3.53, 95% CI 2.44–5.12). These event types were consistently associated with higher harm in all versions of our stepwise model. Notably, events occurring in the home (aOR 1.97, 95%CI 1.04–3.80) and events with patient or caregiver challenges implicated in the event (aOR 2.15, 95% CI 1.47–3.16) were significantly associated with higher harm. Events in psychiatric or mental health facilities were also associated with higher harm (aOR 5.04, 95% CI 1.48–17.14). Patient gender, age, event reporter, geographic location, and hospital characteristics were not associated with higher harm. Results of the logistic regression are in Table 3.

Table 3.

Variables associated with higher harm in incident reports

Incident characteristic	aOR * of Severe Harm or Death (%95 CI)	p-value
Patient gender (ref: female)
Male	0.90 (0.65, 1.24)	0.52
Patient age (ref: adult 18–65)
Child/adolescent (0–17)	1.23 (0.48, 3.15)	0.67
Elderly age > 65	0.80 (.49, 1.33)	0.39
Patient/caregiver challenges involved in incident	2.15 (1.47, 3.16)	<0.0001
Event reporter (ref: administrator)
Nurse	0.55 (0.10, 3.01)	0.49
Pharmacist	0.43 (0.06, 2.81)	0.38
Clinician	0.55 (0.09, 3.35)	0.52
Unspecified/other	0.43 (0.08, 2.35)	0.33
Event setting (ref: medical/surgical specialty)
Primary Care	0.83 (0.26, 2.69)	0.76
Home/Community	1.97 (1.04, 3.80) **	0.04
Psych/Mental health	5.04 (1.48, 17.14)	0.01
Physical/Occupational Therapy	0.84 (0.30, 2.34)	0.74
Laboratory/Imaging	0.42 (0.11, 1.54)	0.19
Dialysis	0.32 (0.07, 1.45)	0.14
Other/Unspecified	0.86 (0.54, 1.36)	0.51
Incident type
Medication	0.74 (0.48, 1.14)	0.17
Diagnosis	11.5 (5.7, 23.2)	<0.0001
Administrative/Documentation/Referral	0.67 (0.33, 1.35)	0.27
Communication	0.82 (0.43, 1.57)	0.55
Treatment/Procedure	0.95 (0.60, 1.49)	0.81
Transport	2.39 (0.87, 6.58)	0.09
Fall	0.60 (0.35, 1.02)	0.06
Patient behavioral complaint	0.48 (0.28, 0.85)	0.01
Clinical deterioration	3.53 (2.44, 5.12)	<0.0001

^*Note: Adjusted for variables shown as well as geographic region, hospital ownership, academic status, and affiliated hospital size.

^**Home/community setting was no longer statistically significant in sensitivity analysis.

In sensitivity analyses, when we applied a broader cutoff point for “higher harm” by setting our outcome as including moderate harm, severe harm, and death rather than just severe harm and death, there was minor attenuations in odds ratios for associations in the same direction. When providing proportional weights to the lower harm levels based on our sampling strata, there were increases in adjusted odds ratios for the same variables identified in our initial analysis in the same direction, with some broadening of the confidence intervals. In the sensitivity analyses, “home/community” setting was no longer statistically significant in association with clinical harm.

DISCUSSION

This analysis of outpatient adverse events is the first of our knowledge to systematically assess what types of events are collected within PSO databases for ambulatory care in the peer-reviewed literature. Many healthcare organizations have extensive databases of incident reports that largely go unanalyzed; this analysis provides a glimpse into what may be learned from them. Much research has focused on inpatient care safety reports,^21–23 is not from peer-reviewed literature,^24,25 and/or utilized electronic health record incident reports that were not compiled in PSOs.^9,26,27 Precedent work to assess primary care safety in incident reports has been done in the United Kingdom, but focused on pediatric populations and included a focus on contributing factors.¹⁵ The PSO ECRI issued an executive brief analyzing 4,355 ambulatory reports from 2017–2018. They found diagnostic testing issues comprised 47% of the sample and medication adverse events comprised 27%; event types were not compared by harm severity.²⁵ In our sample, medication related adverse events were most commonly reported. This matches much of the outpatient safety literature which has focused on medication safety as a high priority.^28–30 However, reports of medication-related events were not associated with clinically significant harm. Rather, among ambulatory reports, diagnostic errors were consistently associated with significant harm.

It is noteworthy that out of 1.5 million total reports from one PSO, only 2.5% were from the outpatient setting. Less than half of participating member sites shared any outpatient events over the 6-year period, demonstrating high variability in outpatient monitoring and data-sharing practices. In our coded sample of 2701 events, only 2.1% were from primary care. Safety reporting from ambulatory and primary care is known to be low; barriers to primary care event reporting have been well-documented.^31,32 Current primary care event reporting is ad-hoc and rarely tied to meaningful or transparent clinic review, despite numerous calls to increase primary care adverse event reporting.^3,33,34 From the data capture side, improved “mandatory” categories and a clear option to distinguish primary care clinics in PSO data would be a straightforward way to improve future monitoring and data review. There are clear means to improve the data quality of in-situ adverse event reporting systems in outpatient and primary care in the current environment. Improved education and training for primary care clinic staff would improve real-time primary care safety monitoring in general. Primary care team-focused trainings to expand safety monitoring results in a 20-fold increase in identified events.³⁵ However, individual and clinic-level barriers to increasing outpatient care reporting are numerous.³¹

Diagnostic errors were rare in this dataset, with only 56 events from a 6-year reporting period. Despite being relatively rarely reported, they had significantly higher odds of involving severe harm or death. The diagnostic errors identified typically described consequences of missing a serious diagnosis such as cancer or a serious infection; one explanation is that minor diagnostic adverse events are less likely to be discovered or acknowledged by healthcare staff. Therefore, the proportion of identified diagnostic adverse events with severe harm or death was high in our sample. Our findings validate the consensus that diagnostic safety is an underexamined yet paramount priority for outpatient care safety. Diagnostic errors are the main source of malpractice claims in primary care³⁶ and associated with patient experience and care utilization.³⁷ In one study of closed outpatient malpractice claims related to diagnosis, 59% caused serious harm and 30% caused death.³⁸ Efforts to increase detection and reporting of diagnostic errors to PSOs, and intentional, directed analysis of incidents involving diagnostic errors from existing data sources such as PSOs are priorities from our analysis.³⁹

We did not expect that medication-related events would have no association with higher harm. One interpretation may be there are higher-harm medication-related events in ambulatory care that are simply not being captured. The medication-related events reported in this dataset were related to infusions of medications; infusion centers likely have protocolized safety reporting standards that capture even minimal infusion adverse events and are not representative of the bulk of outpatient care. A takeaway may be that we need improved strategies to detect higher-harm events from ambulatory medication errors.

A surprising finding was that among reports, patient or caregiver challenges were independently associated with events with odds of higher harm. Outpatient care is highly dependent on patient and caregiver health-promoting behaviors, including self-management, following clinical advice, and navigating social and structural barriers.⁴⁰ Indeed, a landmark study of outpatient closed malpractice claims demonstrated a patient contribution in nearly half of cases.³⁸ We recognize that these events are entered from the healthcare professionals’ point of view, and may reflect reporting biases that emphasize patient culpability;⁴¹ one interpretation of this analysis is that there is an association between higher-harm ambulatory adverse events being reported and the tendency to document patient or caregiver factors. Aside from these considerations, it is plausible that patients and caregivers need additional supports in order to promote safe medical self-management in ambulatory care. Forthcoming work will include a qualitative analysis of themes identified in the incidents that mentioned patient and caregiver challenges. Patient and caregiver participation in adverse event reporting would likely reveal different contributing factors and enrich ambulatory safety monitoring, which has been shown in inpatient settings.^43,44

Events related to clinical deterioration were significantly associated with higher harm. While this association is not surprising, it suggests a deficit among outpatient care settings to be prepared for acutely decompensating patients which may require calling a code or urgent transport to emergency services. Drills or operationalized outpatient code teams may be a necessity in ambulatory care settings that care for frail or complex patients with multiple co-morbidities.

Events from behavioral or psychiatric care locations were more associated with high harm. Patient self-harm, including completed suicide or drug overdose, were the predominant safety report in these settings. Notably, the dataset contained only 31 events that clearly involved opioids. This may be due to the fact that such events occurred in the community and were not observed by healthcare, were noted but not entered into incident reporting systems in primary care/outpatient care settings, that some CHPSO incident reports did not mention the specific overdose substance, or that our sample time period from 2016–2018 was slightly ahead of the latest peak of the wave of the US opioid epidemic.⁴⁵

Our findings have multiple implications. First, we acknowledge the academic consensus that current incident reporting systems, and follow-up of reported events, are inadequate.^46,47 Leading experts recommend targeted analysis of included reports and meaningful engagement of physicians.⁴⁸ Given the major differences between inpatient and ambulatory care,⁶ it remains to be seen if the data from outpatient incident reports can provide actionable lessons, particularly for primary care.⁴⁶ Our focus on ambulatory events is one example of targeted use of PSO data. Our findings suggest that there may be means to elicit meaningful patterns even from datasets with large “signal to noise” ratios. Directed, manual analysis of subsamples of data, such as ours, identified signals to hone in on diagnostic safety and patient and caregiver factors. Novel data processing techniques such as natural language processing (NLP) has been proposed to more efficiently scan the large datasets available in PSOs or other incident reporting systems.⁴⁹ Automated techniques such as NLP can help us prioritize incidents related to clinically significant harm and unearth more expansive themes, and could be filtered for care setting such as outpatient care or primary care.

Limitations

Our study has limitations, many of which are recognized limitations in incident reporting systems worldwide.^50–54 Many outpatient safety events go unobserved, and those that are observed are not necessarily reported. Participating CHPSO members have variable submission policies, and may not export all reported events to the CHPSO database. Given this systematic selection bias in PSO reports, there exist many uncaptured events in outpatient care and the home/community setting over this time period that are not represented. The CHPSO sample in no way represents the “true universe” of adverse events in ambulatory care and no conclusions about the prevalence of harmful events in ambulatory care can be drawn from this kind of data. The presence of a report reflects both true prevalence and the propensity of an event to be entered by healthcare staff and submitted to CHPSO; these probabilities are unknown. The associations we found with clinically significant harm are calculated only within this sample of observed events, and as such our analysis describes associations in reporting behaviors in ambulatory care, rather than predictors of harm.

We had high levels of missing data in reports. Some event categories were relatively rare; rare events can bias results in logistic regression;⁵⁵ however our study was sufficiently powered to detect differences among event categories. Patient race/ethnicity data was available for less than 1% of events, preventing a racial/ethnic disparities analysis. It was often unclear what the specific practice setting was, so we may have underestimated the proportion of events relevant to primary care. This database represents outpatient care linked to hospital networks, and is not generalizable to patient safety incident reports for standalone or non-hospital-affiliated clinics. However, we represent 10 states and 400 member sites, so we presume our findings are generalizable to other PSOs. Other strengths of this study include a coding process involving stakeholders including patients and caregivers, the novel focus on ambulatory care settings, the large sample of events, and the fact that our categorizations and approach are based on precedent literature from the UK.

CONCLUSION

In one of the only analyses of outpatient PSO data from the United States focusing on identified adverse events in ambulatory care, we found that diagnosis-related events, events involving patient and caregiver challenges, as well as events at behavioral settings were associated with clinically significant harm. These are likely only the “tip of the iceberg” for adverse events in ambulatory care, particularly for primary care and events observed by patients and caregivers in the home and community. Ambulatory safety will gain meaningful directions for how to reduce preventable harm through improved use of current reporting systems, novel strategic means of capturing primary care-related events, and by patients and caregiver engagement in ambulatory safety initiatives.

Figure 2.

Frequencies of ambulatory adverse event incident types in CHPSO database