Trigger Tools for Adverse Event Detection in the Emergency Department

Dear Editor –

We read with interest Lee et al’s “Comparing the Outcomes of Reporting and Trigger Tool Methods to Capture Adverse Events in the Emergency Department.” While we applaud research into methods for detecting adverse events (AEs) in the emergency department (ED), some additional context and clarifications are needed to inform the discussion around trigger tools for this purpose. Though we agree with the authors’ findings that the “triggers” used in this study are of low value, we respectfully disagree that what is evaluated in this study was in fact a trigger tool.

Trigger tools derive from the 2-tiered review methodology introduced in the Harvard Medical Practice Study¹, further modified and popularized by the Institute for Healthcare Improvement’s Global Trigger Tool.^2,3 A main feature of trigger tools, which makes them more efficient than traditional approaches, is that rather than reviewing all cases meeting a particular screening criterion (e.g. 72-hour returns with admission), a random sample of records is reviewed (usually by a nurse) first for the presence of a trigger – from among a set of triggers – and if a trigger is present, this record is then reviewed for the presence of an AE. Only those records in which a putative AE is found then undergoes a 2^nd level confirmatory physician review. Triggers are findings that increase the likelihood of an AE. For example, the trigger “INR > 5” prompts a reviewer to look for a bleeding event. If no trigger is present, the review is complete. Nearly all trigger tools are explicitly derived through robust expert consensus processes to identify a set of triggers. A quick review of several published trigger tools reveals an average of 31 triggers with a range from 11 to 53.^4–10 While there is no hard minimum number of triggers defined for a trigger tool, it stands to reason that a tool consisting of very few triggers or a collection of narrowly-defined or low-prevalence triggers is unlikely to detect many AEs.

In the present study, the “triggers” consist of only 5 items, which are versions of blunt screening criteria that have been used by ED’s for decades with questionable utility.¹¹ Some of these (“unexpected CPR,” “in-hospital cardiac arrest”) appear highly colinear, directed at detecting the same group of rare AEs. In a study we conducted of 10 academic EDs, all sites reported using similar versions of these same screening criteria, with a collective yield of just 1.9% for detecting AEs.¹² An item such as “72-hour return admissions” would only be considered a trigger in the commonly understood sense if applied to a random sample of records, screening for its presence to guide further review, as opposed to pulling up and reviewing all cases of this criterion. There is certainly some confusion around what is meant by the term ‘trigger tool,’ and this is not the first study to report on performance of the trigger tool approach when the trigger tool in question in fact consisted of a single trigger and/or did not use trigger tool methodology. While there is no definitive statement as to what comprises a trigger tool, at a minimum trigger tool studies include: 1) a sampling process for record review; 2) screening of the sample for the presence of triggers; 3) reviewing (only) records with triggers for AEs and a 2-tiered system for confirming harm. Though this study includes a 2-tiered approach for review of their reports and record review, the other elements above do not appear to be part of the process.

Strengths of this study include of a high number of events reviewed and a strong interrater reliability. However, the definition used for the outcome, considers both non-harm errors (near misses) and harm as “adverse events,” which is a term typically applied only to events with harm. The focus for improvement efforts should be on preventable harm rather than on errors, which are common (and by definition have the ‘potential for harm’) but rarely actually result in harm. While there is certainly variability across hospitals and health systems, incident reporting systems have a poor track record in identifying AEs, with reports that these systems fail to capture 90% of events and that a high proportion of events reported are not harm.¹³

Taken together, this study compares open-ended event reporting mechanisms against what appears to be a set of screening criteria known to be poor predictors of AEs using a traditional retrospective review. With the additional confounder that the outcome of the study combined near misses and harm, it is not surprising that the reporting system outperformed these screening criteria. We the authors may have found utility in their reporting system, this should decidedly not be considered an evaluation of nor an indictment of trigger tool methods for AE detection in the ED.

Sincerely,