Why Identifying Adverse Events in Pediatric Emergency Care Matters

The emergency department (ED) abounds with hazards that might lead to adverse events. Clinicians must make frequent, high-consequence decisions with high diagnostic uncertainty, as part of ad hoc teams taking care of patients not generally known to them.¹ This occurs in the context of a sometimes-chaotic environment with frequent interruptions, handoffs, and shift changes. As a result, the ED tends to magnify cognitive biases that can result in AEs – particularly diagnostic AEs.² This is especially problematic for children, who account for 23 million ED visits in the US annually and are particularly vulnerable to AEs. Children are developmentally less able to communicate specific symptoms and are evaluated in EDs with variable readiness to care for them.³ Yet, little is known about the occurrence and nature of AEs in pediatric ED patients.

In this issue of BMJ Quality and Safety, Plint and colleagues sought to address this gap. They conducted a prospective observational study of children visiting nine Canadian pediatric EDs to determine whether an AE occurred within 3 weeks of a visit.⁴ AEs were defined as “any event resulting in unintended patient harm that was related to the care provided rather than to an underlying medical condition.” They found that 3% of visits were associated with an AE, of which 77% were judged to be preventable. The most common AEs related to management, diagnosis and medication. Although the large majority of AEs were minor, half prompted the need for subsequent care. Risk factors associated with a preventable AE were increasing age, history of chronic condition, increasing time to initial ED assessment, and evaluation in the acute care area (rather than the ambulatory area).

This study illuminates an underexplored area in pediatric emergency care: given its inherent hazards, how often do AEs happen? Plint and colleagues improved upon their prior single-center study⁵, providing more precise and stable estimates of AEs in this population while bolstering external validity, since estimates were consistent across the nine centers. Given that most AEs were preventable, this study also provides a clear basis for robust quality improvement efforts that have potential to reduce such AEs.

The key strengths of this study are in its multicenter coordination with high enrollment numbers and impressive and meticulous efforts to follow patients forward from their ED visit, resulting in a robust identification of AEs. Investigators selected an unbiased sample of ED shifts, examined associated hospital records of admitted patients using the Canadian Pediatric Trigger Tool and made every effort to contact all patients to identify AEs. While it would be impossible to avoid recall bias when interviewing families about their past experiences, investigators minimized this risk through a multipronged, preplanned strategy of AE identification and vetting. This study’s findings therefore represent an important advance in our understanding of AEs occurring in pediatric emergency care.

As in any study, there were a number of limitations, including some that are common to studies of adverse events. Although the data are from 2014–2015, this is unlikely to affect the types of AEs detected or diminish its significance. The types of conditions seen and potential events that occurred in the ED (such as peripheral intravenous catheter infiltrations, bleeding events, delayed diagnoses) are unlikely to have significantly changed since these data were collected. The main limitations involve three key issues: 1) methodological approaches affecting AE rates, 2) inherent subjectivity in AE determination and 3) generalizability of pediatric ED AE rates to all EDs where children receive care.

Methodological approaches affecting estimates of AE rates

This study estimated the pediatric ED AE rate to be 3% of visits. In adult cohorts, AE reviews based on a broader trigger methodology have identified AE rates of 8%.⁶ One metanalysis of mixed pediatric and adult populations by Stang et al found the range of AEs attributable to ED care to be 0.2 – 6.0%.⁷ The wide range of AE rates in earlier studies highlights how decisions about the method of detection of AEs influences reported rates. For example, the lower bound of 0.2% reported in Stang’s analysis was based on voluntary error reports, which are known to severely underestimate AEs.

There are reasons to believe that the true prevalence of AEs among children in the ED may be higher than this study reports. First, events under consideration had to be judged as at least 50% likely to be AEs to be included as such. This is a higher standard than the possible AE standard, in which determination of an AE hinges on whether some action or omission “may have contributed to” an AE. This standard is used in detecting all-cause harm, in part to avoid subjectivity.⁸ The higher 50% standard requires judgments as to whether, for example, hypotension in a septic patient who received morphine was possibly an AE due to the morphine or was attributable to their underlying condition. Second, because subsequent care was a key means of identifying AEs in Plint and colleagues’ study, events that did not prompt subsequent care would only have been included if families were aware of and reported them. Thus, minor AEs could have been missed.

Conversely, other methodological decisions in this study might have increased AE estimates. In contrast to common practice, this study did not require that in order to be considered an AE, an event must prompt an intervention.⁹ As an example, the approach used in many studies would not include asymptomatic hypoglycemia without an intervention as an AE, but instead consider it as a near-miss. In Plint and colleagues’ study, 18% of AEs did not require an intervention, which could have led to overestimation of AE rates.

Subjectivity of adverse event reviews and preventability assessments

Determining the presence of an AE generally has an interrater reliability ranging from a Cohen’s kappa of 0.4–0.7, representing fair to good agreement.^10,11 Issues that can affect determination of whether an event is considered an AE include the confidence standard used in making this decision, the types of AEs that a reviewer screens for (for example, negligent AEs versus all AEs), and the types of cases evaluated. Although retrospective record review remains a gold standard for AE detection, it is far from perfect. Reviewers are subject to hindsight and outcome biases. Events that look clear and linear in retrospect are highly uncertain and probabilistic in real time. Medical decision making is rarely documented in a way that provides retrospective insight and large parts of narratives related to events in the ED may be missing from medical documentation.

Determining AE preventability is likely to be even less objective, with kappas from 0.2–0.4, representing poor to fair agreement.^10,11 Preventability determinations may be more subject to opinion, reviewer characteristics, or even more capricious factors.

Factors that improve agreement in all AE reviews include having a smaller team of reviewers (preferable less than 5) and to a lesser extent, reviewer experience and training.¹² Clear written guidelines for reference during review meetings may also be helpful. Thus, both overall and preventable AE rates are highly dependent on both the review strategy and the threshold for calling an event an AE. Demonstration of only fair agreement in the determination as to whether an AE was preventable underscores how it is critical to have multiple rather than a single reviewer in evaluating preventability, as was the case in Plint and colleagues’ study. Future work should ensure multiple reviewers for preventability determinations and might include sensitivity analyses on AE rates using stricter and more liberal standards of AEs to probe the potential for variability.

Generalizability

In the US, about 90% of children’s ED visits are to general EDs, with only 10% taking place in dedicated pediatric EDs such as those studied here.¹³ This is likely to be similar in other developed countries. Pediatric EDs have the highest readiness to care for children, with staff, supplies, resources, protocols, and culture geared toward children’s emergency care.¹⁴ They also have the highest capability to care for children, indicating that they are by far the most likely to provide definitive emergency care and are therefore equipped with expertise across a range of both common and uncommon conditions.¹⁵ While general (i.e. non-pediatric) EDs’ pediatric readiness has been increasing in the US, hospitals’ capability to provide definitive pediatric care has been decreasing.¹⁵ This trend has been due in part to inpatient unit closures and has led to increased ED transfers of children, even for common, basic conditions such as asthma and gastroenteritis.^16,17

Most EDs evaluate few children. Indeed, a third of US EDs evaluate fewer than 5 children per day.¹⁸ Much of the management of pediatric emergency conditions can be outsourced by transferring children to EDs with more pediatric experience, but this only occurs 5% of pediatric visits.. What this means is that every ED owns most of its pediatric emergency care, whether experienced and resourced or not. How this translates to pediatric AE rates in general EDs is not known. AEs may be fewer (because of lower complexity pediatric patients), the same, or higher (because of lower readiness and capability). As general hospitals decrease the provision of definitive pediatric care¹⁵, it will be increasingly important to monitor the effectiveness and performance of emergency care systems that treat children. In particular, diagnostic safety has the potential to suffer from a decline in expertise, and attention to systematically measuring diagnostic safety in children is needed.

A final limitation related to generalizability is that care may differ between Canada and other countries. For example, Canadian EDs are known to be more parsimonious than US EDs in use of diagnostic tests for children, which could affect AE rates in either direction.¹⁹ Similarly, country-specific differences in expectations around follow up and care coordination, the role of consultants during ED encounters, and ED structure may also result in differences in AE rates.

Conclusion

With this study, Plint and co-authors have provided critical first steps toward a clearer understanding of the safety of pediatric emergency care. We believe key future steps include investigation of AE rates among children in general EDs is needed and study of the types of AEs that are sufficiently common and harmful that intervention is warranted. Given that most children do not visit pediatric EDs, evaluating quality and performance in all ED settings is a pressing concern, and is likely to become only more so in the future.