Medical Reversal: Why We Must Raise the Bar Before Adopting New Technologies

Abstract

Medical reversal occurs when a new clinical trial — superior to predecessors by virtue of better controls, design, size, or endpoints — contradicts current clinical practice. In recent years, we have witnessed several instances of medical reversal. Famous examples include the class 1C anti-arrhythmics post-myocardial infarction (contradicted by the CAST trial) or routine stenting for stable coronary disease (contradicted by the COURAGE trial). In this paper, we explore the phenomenon of medical reversal. The causes and consequences are discussed. Conflicts of interest among researchers and an unyielding faith in basic science are explored as root causes of reversal. Reversal harms patients who undergo the contradicted therapy during the years it was in favor and those patients who undergo the therapy in the lag time before a change in medical practice. Most importantly, it creates a loss of faith in the medical system by physicians and patients. The solution to reversal is upfront, randomized clinical trials for new clinical practices and a systematic method to evaluate practices already in existence.

In medicine, therapies as well as diagnostic and screening tests decline in popularity for two reasons. The first is the phenomenon of replacement: A practice is supplanted by one that works better. Recently, the low molecular weight heparins have replaced coumadin in the treatment and secondary prevention of deep vein thrombosis among cancer patients [1] and proton pump inhibitors have replaced histamine H2-receptor antagonists in the treatment of most patients with gastroesophageal reflux disease [2].

The second phenomenon is reversal: A medical practice falls out of favor not by being surpassed, but when we discover that it did not work all along, either failing to achieve its intended goal or carrying harms that outweighed the benefits. Although this phenomenon should be rare in the age of evidence-based medicine, it is ubiquitous. Common use of avandia [3], ezetimibe [4], atenolol [5], hormone replacement therapy [6], and the class 1C antiarrhythmic agents [7] all stopped when trials showed they were either ineffective or harmful. Reversal not only affects medications. Previously accepted indications for surgical and medical procedures also have been abandoned. In 2009, stenting for renal artery stenosis was shown to be ineffective for many patients by the Angioplasty and Stenting for Renal Artery Lesions (ASTRAL) trial [8], and in 2007, the Clinical Outcomes Utilizing Revascularization and Aggressive Drug Evaluation (COURAGE) [9] trial found no benefit to support percutaneous coronary intervention (PCI) (versus optimal medical therapy) in most patients with stable coronary artery disease. In these cases, reversal does not mean that for every indication and purpose the therapy in question was shown not to work, but simply that it was contradicted for key indications.

A comparison of replacement and reversal invites several questions. While the former represents a logical progression in medical care, the latter reveals frequent missteps. Atenolol, a popular antihypertensive and trial standard, dominated medical practice for many years. In 2004, the Losartan Intervention for Endpoint reduction (LIFE) study suggested that not all antihypertensives were the same [10]. Losartan, the angiotensin receptor blocker, significantly outperformed atenolol for those things that mattered: cardiovascular endpoints and mortality. Curiously, both drugs had the same effect on 24-hour blood pressure [11]. Whether the results were due to a benefit of losartan or a weak effect of atenolol was debated [12]. A meta-analysis to resolve the dispute showed that treatment with atenolol carried equivalent mortality as placebo [5]. Atenolol has subsequently fallen out of favor.

In our previous work, we coined the term “medical reversal,” defining it as “the phenomenon of a new trial — superior to predecessors because of better design, increased power, or more appropriate controls — contradicting current clinical practice” [13]. In this essay, we provide the first sustained account of the key issues surrounding reversal. We argue that the phenomenon of reversal does exist, that it is different from replacement, and that its consequences are serious. We will outline what we believe the current philosophy is toward adopting new technologies and suggest that it be reconsidered. A certain amount of reversal is unavoidable in medicine, as with any statistically driven science, but there are real ways reversal can and should be lessened.

Reversal Exists

The phenomenon of reversal is real, and examples abound in recent years. A few striking ones follow. In the late 20th century, sudden cardiac death, particularly during the vulnerable period after myocardial infarction, was deemed a “world wide public health problem” [14]. A type of heart rhythm, premature ventricular contractions (PVCs), was thought to contribute to such deaths [15]. A new generation of antiarrhythmic therapy was developed with the ability to suppress PVCs up to 85 percent of the time [16]. Cardiologists began using these medications in widespread fashion. In the late 1980s, the Cardiac Antiarrhythmic Suppression Trial (CAST) was conducted to assess the safety of what was then commonplace [7]. Interestingly, recruitment for the trial was hindered by physicians who refused to let patients undergo randomization with a 50 percent chance of not receiving these medications [17]. Fortunately, the trial was completed and showed that these drugs (encainide, flecainide, and later, moricizine) conferred greater mortality than placebo, and their use was curtailed for this indication.

Vertebroplasty, the injection of medical cement into fractured bone, achieved widespread use without good evidence that it worked. First described in the late 1990s [18], vertebroplasty quickly gained popularity. In 2005, it was performed more than 27,000 times in the United States [19]. A pair of articles published in the New England Journal of Medicine in 2009 conclusively showed that the procedure was no better than placebo by analyzing the outcomes of patients randomized to vertebroplasty or a sham procedure [20,21].

Finally, in what remains a contentious issue, routine mammography screening for women in their 40s was questioned in 2009 [22]. The U.S. Preventive Services Task Force (USPSTF) “recommends against routine screening mammography in women aged 40 to 49 years.” That change from 2002 guidelines was in large part based on a randomized controlled trial (RCT) of mammography that appeared in The Lancet in 2006 [23]. It compared 54,000 women who were offered mammograms starting at age 39 with 107,000 women who were not offered them. It was large, well-done, and likely the best study to address this issue. It showed only a small decline in the breast cancer death rate after 10 years, which failed to meet significance. Overall, mortality of women in this age group did not change.

Each of these examples represents a medical practice not surpassed by an alternative (replaced) but instituted in error (reversed). Atenolol may lower blood pressure, but is no better than placebo in increasingly survival. Class 1-C antiarrhythmics, used as described, increase mortality. Vertebroplasty is no better than sham-vertebroplasty in diminishing pain or promoting spine stability. And, because mounting data suggest that mammographic screening does not benefit women in their 40s, screening guidelines are changing. While one may disagree with the portrayal of any particular example, it seems implausible that one can disagree with every example.

Others have tried to quantify the rate of contradiction in medical literature. Ioannidis [24] has shown that 16 percent of highly cited articles were contradicted by future studies. In our previous work, we examined a large collection of high-impact literature and found that among articles making a claim regarding a medical practice, 13 percent were medical reversals [13]. Reversal is not a rare occurrence.

Why is Reversal Dangerous?

Reversal differs from replacement in that it produces three perils. First, reversal implies mistake or harm to patients cared for under the old model. The abandoned practices were ineffective or harmful. The cases of CAST and Avandia demonstrate harms, while COURAGE and Atenolol suggest only the harm of misplaced financial and social resources. This cannot be said about replacement. Patients who received an ultimately replaced practice were given the best care of the time, an improvement over the prior era. It is not a mistake that they did not receive what was yet to be developed. When it comes to replacement, harm occurs only if novel, more effective treatments are subject to unnecessary delay.

Second, removing a once-commonplace practice can be more difficult than imagined. Adherence to the contradicted claim furthers malfeasance. The idea that beta-carotene could diminish cancer gained popularity in the early 1980s [25]. By the mid-1990s, however, three randomized controlled trials overturned the claim [26,27,28]. However, nearly a decade passed before counterarguments were uncommon in the literature [29]. The use of routine PCI in the population contradicted by the COURAGE trial continues. Finally, routine use of pulmonary artery catheterization continues, despite being seriously challenged in 1996 [30] and further discredited in 2005 [31,32].

There are several reasons why discredited practices remain in place. Financial rewards certainly play a role. One group tried to understand the characteristics of papers that disagreed with the findings of COURAGE. They made the observation that among articles expressing reservations about the results, they were more likely to have an interventional cardiologist as corresponding author than those that were unreserved [33]. While it is easy to attribute a portion of blame to financial conflicts of interest, even in those cases in which proponents have little to gain monetarily (as with beta carotene), they remain steadfast. Siontis et al. make a similar observation, noting that “the mere wish to defend one’s practice, procedures, and scientific beliefs” may be sufficient for continuing to support a discredited practice [33].

Third, reversal undermines trust in the medical system. In the case of hormone replacement therapy (HRT) — once thought to be beneficial for reducing a woman’s risk of heart disease while treating menopausal symptoms and contradicted by the Womens’ Health Initiative — patients report feeling “furious” with doctors who “pushed” therapy upon them [34]. The pharmaceutical company Wyeth, maker of Prempro, has been sued for overstating the benefits of HRT and understating its risks, and court documents reveal questionable marketing practices by the drug maker [35]. Loss of trust in the institution occurs not only among patients, however, but among doctors as well. In the wake of the breast cancer screening controversy, the American College of Radiology and the American Cancer Society criticized the USPSTF. Patients and doctors report that they plan to continue to screen the population that does not benefit. Loss of trust is an immeasurable harm, whose effects are multifaceted and enduring.

Where does reversal come from?

When it comes to new medical practices, the lower the standard for a therapy’s acceptance, the greater the chance for future reversal. There are several reasons why we do not perform large randomized controlled trials (RCTs) powered for hard endpoints before every therapy is adopted. Chief among them are cost, the desire not to delay potentially beneficial therapy, and an unyielding, and perhaps unjustified, confidence in basic science models and surrogate outcomes. Often, therapies are promoted because they should work (the pathophysiologic model is compelling) or because a surrogate marker (used instead of a clinical endpoint because it makes a trial easier to run and cheaper) shows improvement. The examples preceding argue that such data does not always hold up.

Of course, there is a second and more cynical interpretation. Financial incentives are strongly aligned to promote new technologies. From a research standpoint, conflict of interests among trialists, industry-sponsored studies (utilizing favorable, but flawed methodology), and industry-sponsored economic analyses (with favorably biased results) all encourage wrongful optimism, facilitating approval [36-53]. Litigation that has arisen out of reversal has enhanced our understanding of this phenomenon. Such proceedings have uncovered withheld safety data, misleading marking practices, and lapses in regulatory mechanisms [54-55].

Historically, reversal can be seen as an emerging threat since the early 1990s and a consequence of the success of empiricism. Prior to the 1940s and the advent of the randomized trial [56], pathophysiologic approaches to clinical problems dominated allopathic medicine. Good scientific theories needed to be both consistent and comprehensive. Consistent in that the theories reconciled real world observations and comprehensive in that they made sense of a diverse collection of data. The best clinical medicine could achieve was congruity with the leading models of the human body in health and disease. The dominance of practice was not shaken until the early 1990s, when trials such as CAST [7] showed that the very best rationale could yield treatment that harmed patients. A mechanistic understanding of science, no matter how robust, does not guarantee empiric verification. This principle has served as precondition for the era of medical reversal.

The conventional view and a new standard

Despite this historical shift, in current practice we continue to adopt new technologies not because they are supported by the strongest evidence base, but based on a common sense appeal that they should work [57]. We can extend “common sense” to signify any set of surrogate data trials, basic science rationale, or observational results. There is direct evidence that this permissive attitude is true of approval processes. Redberg and colleagues note that only 27 percent of new cardiac devices were tested in randomized fashion prior to U.S. Food and Drug Administration (FDA) approval [58].

In light of theses considerations, the prevailing attitude must be reconsidered. A common sense standard that a treatment will work can no longer justify its adoption. Twenty years into the era of evidence-based medicine [59], we must recommit to practicing based on good evidence. In general, this means that well-done RCTs should be done before new technologies are adopted. Well-done means that in addition to strong methodology, adequate power, and blinding, such trials are appropriately controlled (in certain cases, sham-controlled) and address proper endpoints. What counts as appropriate control and proper endpoints is beyond the scope of this paper, and a subject that can be debated, but it almost certainly involves outcomes that are important in and of themselves. A hemoglobin A1c level is not something that is in itself meaningful; diabetic mortality (largely from cardiovascular causes and stroke) and diabetic end-organ damage (retinopathy, nephropathy, neuropathy) are. Hypertension is a silent killer. Silent in that patients don’t so much care about it, as its consequences. Thus, the popularity of Atenolol was particularly shameful: the treatment of a silent surrogate marker that never achieved its intended goal of helping patients live longer.

Some reversal will be inevitable

A recommitment to evidence-based medicine will not eliminate reversal. Large, well-done RCTs do represent the strongest truth claim in all of the sciences [60], but they are not beyond the reach of refutation. Reliance on strong evidence, however, will greatly diminish the frequency of reversal, and this itself would be an incredible feat.

We propose raising the bar for the adoption of new medical practices. Others have made similar appeals [58], and one author advises physicians to practice irrespective of FDA approval, demanding a higher standard in their practices [61]. However, to our knowledge, we are the first who have defined the consequences of reversal as incentive for this change. It will likely require both a strong professional ethic and centralized regulation to achieve meaningful results.

Currently, the standards for device approval remain below that of pharmaceutical drugs. Medical devices are less likely to have demonstrated safety prior to approval [62], and very few have efficacy shown in large randomized controlled trials [58]. For these reasons, one may speculate that reversal occurs more frequently among medical devices and that the reforms we have suggested will affect that industry more deeply. Such a view is plausible; however, approval for medications often also includes a sea of uncertainty — the reliance of surrogate endpoint studies, placebo lead in periods, and a shift to controversial endpoints such as “progression free survival.” Thus, it is hard to say whether devices or drugs harbor more uncertainly and where reversal might be deterred more frequently.

Instead of raising the bar for new technologies, one might contend that we simply become better at managing reversal. Contradicted practices should be more rapidly removed, and physicians should be careful to advise patients of the uncertainty in a practice. Proponents of this view understand that it requires continual reassessment [57]. “[A]s evidence subsequently accumulates, physicians must be prepared to reevaluate even a long-standing clinical practice” [57]. However, as we have argued, the inertia of a contradicted therapy extends beyond alterable considerations such as finances. Inertia makes this position untenable.

The cost of performing upfront studies may be seen as another barrier to our proposal. However, looked at a different way, higher upfront standards would significantly cut costs. Sixty-four patients and 40 control patients were required to demonstrate that vertebroplasty is not a useful therapy. Over the course of the decade preceding these trials, thousands of patients underwent the procedure paid for by Medicare alone [63]. The cost of conducting the New England Journal of Medicine studies was trivial compared to the cost of the procedure across the United States in the preceding years. While large payers clearly have the biggest incentive in funding such trials, makers of vertebroplasty equipment (and cement) and administering practitioners (interventional radiologists) have the deepest obligation.

Our discussion has centered on new technologies, but a related issue is how to deal with existing, unverified practices. A systematic and stepwise method of trials is required to uncover as-of-yet unknown reversals. A systematic way would be to prioritize interventions by cost burden. To understand the idea of “stepwise,” consider the case of minimally invasive laminectomy for chronic low back pain, which is widely performed in the United States today [64]. Initially, trials may compare surgery with sham-surgery among patients with pain, but without paraesthesia or other neurological sequellae. However, if such trials result in reversal, further studies expanding the potential territory of contradiction would be warranted.

Medicine has a moral obligation to hold itself accountable to the highest methodological standards of the time that are reasonably feasible, practical, and ethical prior to widespread implementation of new therapies. Reversal serves as a reminder that failing to do so, risks deep and lasting damage. Amid the many important topics of health care reform, we must revisit fundamental questions: How do we want the practice of medicine to advance? Do we want a profession that incrementally moves toward the good and helpful? Or one that stutters and stops, goes back and forth, moving steadfastly toward the expensive and new? A sustainable, reasonable, and honest medicine must be the former. Early upfront testing would be a boon to both patients and doctors alike.

Glossary

ASTRAL

Angioplasty and Stenting for Renal Artery Lesions

COURAGE

Clinical Outcomes Utilizing Revascularization and Aggressive Drug Evaluation

PCI

percutaneous coronary intervention

LIFE

Losartan Intervention for Endpoint reduction

PVCs

premature ventricular contractions

CAST

Cardiac Antiarrhythmic Suppression Trial

USPSTF

U.S. Preventive Services Task Force

RCT

randomized controlled trial

HRT

hormone replacement therapy

FDA

U.S. Food and Drug Administration