Improving patient safety: moving beyond the “hype” of medical errors

The recent Canadian Adverse Event Study found that 7.5% of hospitalizations were associated with adverse events and that 1 in 6 patients with an adverse event died.¹ These findings, which are not unique to Canada,² have resulted in some laudable initiatives, including creation of the Canadian Patient Safety Institute. We are nonetheless concerned that health care administrators, patient safety advocates and even researchers believe that solutions to most safety problems already exist, and that the only problem is educating health care providers to use them. We disagree.

For example, the Institute of Healthcare Improvement, a US-based nonprofit organization, launched the “100K Lives” campaign, in which it recommends several interventions to tackle 6 “safety” problems.^3,4 Although some of the recommended interventions (e.g., ASA therapy to reduce mortality after myocardial infarction) are supported by high levels of evidence, several are not. For an example of the latter, as a means to minimize preventable deaths the campaign suggests implementing a rapid-response team of clinicians to bring critical expertise quickly to the bedside of patients with dangerously abnormal vital signs. In theory, such an intervention could be effective, and some observational data do support its use;^4,5 however, the only well-designed clinical trial evaluating rapid-response teams found that the intervention had no effect on important clinical outcomes.⁶ Accepting recommendations not based on evidence may lead to no measurable improvement in clinical outcomes, while resources are wasted and the credibility of all patient safety efforts is reduced.

Even before we institute specific programs to reduce error, we need to reduce gaps in our knowledge, including our limited understanding of how to measure adverse events and their clinical significance. Evaluation of adverse events requires first a robust method for detecting them and second a means to rank their clinical importance. Only then can we be sure that we will be able to reach the third step: programs to reduce the frequency of clinically serious adverse events.

How can the presence of an adverse event be determined? Investigators usually ask trained physicians to assess the clinical information on patients who experience poor outcomes and use implicit criteria to rate whether the outcome resulted from the quality or nature of the medical care or the progress of the underlying illness. The subjective nature of this process leads to disagreement between physician reviewers and to wide variations in the reported prevalence of adverse events.^7,8,9

To illustrate this problem, consider data from a recent study¹⁰ where we determined the proportion of 328 medical patients who experienced an adverse event after hospital discharge. The proportion of patients judged to have undergone an adverse event varied not only among the 3 reviewers (23% v. 27% v. 21%) but also by how their responses were aggregated (unanimous agreement 15% v. a 2-out-of-3 majority 23%). Thus, depending on how adverse events were defined, prevalence varied from 15% to 27%. In most clinical research, a 12% absolute change in the major outcome of interest would hold substantial interest. For patient safety, these data show that such variations could simply be related to who reviewed the case and how agreement was achieved. For 4^1,11,12,13 of 6 major studies, adverse-event rates reflected judgments by 1 reviewer per case, whereas for the other 2,^14,15 only 2 physicians were used. Our data on the variation among reviewers suggest that these previous studies may have inaccurately defined the extent of safety problems.

The second issue is whether or not adverse events are clinically important. Although adverse events are known to lead at times to permanent disability or even death, most cause only temporary symptoms.^{1,11,12,13,14,15} One of the barriers to measuring the clinical impact of adverse events is distinguishing the adverse event from other causes of poor outcomes. Most studies have merely associated the occurrence of adverse events with outcomes, rather than tried to establish the more relevant causal relation. Consider a critically ill patient who develops a pneumothorax as a result of an erroneously inserted subclavian line. If the patient dies, her death is most likely the result of her critical illness rather than the pneumothorax, which is rarely life-threatening. However, if standard patient-safety research methodology were applied to this case, the patient would have had a “preventable adverse event” (the pneumothorax, caused by an error in line insertion) associated with the outcome “death.” In other words, there is no assessment of the degree to which the adverse event contributed to the patient's eventual death. This determination has not been adequately addressed in any of the major studies of adverse events.^{1,11,12,13,14,15}

To the best of our knowledge, only 1 study⁹ has assessed the effect of adverse events on patients' ultimate outcomes by asking reviewers to estimate the probability that patients would have lived longer had the adverse event not occurred. The reviewers deemed that only 6% of patients who died would have been expected to live an additional 3 months had the identified care problems not occurred.

Lastly, there are major gaps in knowledge about patient-safety practices. In a systematic review of 80 patient-safety interventions,¹⁶ those with reasonable supporting evidence consisted largely of clinical therapies such as prophylaxis of venous thromboembolism. Frequently espoused “system-based” interventions, such as improving facility design and work flow with technology and robotics, adding forcing functions to procedures (e.g., automatic stop orders for Foley catheters¹⁷ or precautions against the storage of concentrated potassium solutions with ward stock)¹⁸ and improving product labels have only modest supporting evidence. For example, computerized practitioner order entry has decreased medication errors in several studies but has not significantly reduced actual adverse drug events.¹⁹ The benefits of incident reporting, root-cause analyses and promoting a “culture of safety” are even less clear.

Even the evidence supporting apparently established safety practices is in a state of flux. For example, although perioperative β-blockers received a high evidence rating in the patient safety review,¹⁶ a recent systematic review²⁰ that included data from several trials published subsequently concluded that the evidence was insufficient to support their use; in fact, the dominant clinical outcome found was bradycardia requiring treatment. Thus, the most recent data on this safety intervention suggest that not only does it fail to improve outcomes, it may in fact worsen them.

Without more robust methods to determine when an adverse event has taken place and a better understanding of the clinical importance of adverse events, it will be impossible to set priorities for evaluating intervention strategies to reduce the chances of such events.

Given the magnitude and severity of the problem, can we afford to wait for the high-quality evidence of randomized clinical trials? Or should we just, as some have argued, get on with it?²¹ In situations where an important problem exists and evidence for a solution does not, decision-makers could use an approach analogous to that used by clinicians faced with imperfect evidence. A framework for informing such decisions might include the importance of the targeted problem (its combined prevalence and severity), the strength of the available evidence supporting the intervention, and the potential for adverse collateral effects from the intervention. Reasonable interventions should be piloted in accordance with our understanding of the problem's causes and after careful risk assessment. A thoughtful evaluation of the implementation process should be undertaken and the results published so that others can learn.

Our recommendations may appear overly cautious to some, but we are reminded of the words of Henry Louis Mencken when he stated “For every complex problem, there is a solution that is simple, neat, and wrong.” Rushing to implement poorly tested interventions that target problems of unclear significance may do little to help and ultimately may even discredit the endeavour, an effect that all of us would hope to avoid.