Why were the benefits of tPA exaggerated?

News magazine US News and World Report began an article aboutthrombolytic treatment of acute stroke with the following anecdote:

One Wednesday morning in January, 36-year-old Laurie Lucas was rushingabout, dressing her daughters for school and talking on the phone, when astrange confusion swept over her. She felt woozy. Her right arm flailed aboutand her right leg went weak. Paramedics arriving at Lucas's Sanford, Florida,home thought she was having epileptic seizures, but brain scans revealed ablockage of an artery that supplies the brain with blood. Lucas, a physicallyfit former professional cheerleader, had suffered a massive stroke... If Lucashad been stricken a year ago, before a new treatment was developed, she almostcertainly would havedied.¹(p62)

This story is interesting but misleading. It is not plausible thatphysicians could accurately conclude that Laurie Lucas “almost certainlywould have died” without treatment. Nor is it true that“massive” strokes are likely to benefit from thrombolytictreatment.² Inaddition to several anecdotes, the US News and World Report articleincludes “expert” testimony in which prominent physiciansexaggerate the efficacy and effectiveness of the new treatment.

Such exaggerations also appear in medical journals and other professionalvenues. Specialists have announced “the decade of thebrain,”³ a“new era of proactive rather than reactive stroketherapy,”⁴ and“transformation from a realm of fatalism to a field of therapeuticopportunities.”⁵Landis and associates write, “With the tools in hand, recognition ofischemic stroke as a medical emergency and application of prudent thrombolytictechniques will have a major impact on stroke morbidity andmorality.”⁶(p226)Some commentators even warn of lawsuits if thrombolytic agents are not givento eligible strokepatients.⁷ Theseclaims are without scientificbasis.^8,9,10Yet, robust exaggerations and effusive sentiments of this sort appear notmerely in the literature concerning acute stroke but on other topics as well.They are a serious problem because they exhibit interpretation bias, which maylead to errors in clinical judgment and to unrealistic patientexpectations.¹¹

SOURCES OF INTERPRETATION BIAS

Interpretation bias occurs when systematic distortions arise in theinterpretation of data from scientific studies. Ideally, interpretation biaswould be eliminated directly at the clinical level. Each physician would be anexpert at interpreting scientific data and would have unlimited time to reviewthe medical literature. Indeed, some progress has been made toward this idealwith the emergence of evidence-based medicine. Still, given the astoundingvolume of medical data and the complexities of statistical analysis, mostclinicians need help to integrate the available information into justifiableclinicalstrategies.¹²

Interpretation bias usually occurs at intermediary stages, as data sifttheir way through various medical and nonmedical interpreters on their way toclinicians and patients. At least two forces outside the medical professionpertain. First is the need for the news and entertainment industry to discoverinteresting and important events. Second is the imperative of drug and devicemanufacturers to sell their products.

In media coverage of health research, the tendency is to focus on dramaticurgent interventions and to highlight the most startling or momentous claimsabout possible medical benefits. Inaccurate or stilted reporting frequentlyfollows, often with powerful effects on patient and clinicianperceptions.^13,14Media demands or pressure from political support groups may produce a feelingof urgency among medical researchers to announce something memorable andimportant.¹²

Drug and device manufacturers target individual clinicians and patientswith advertisements, gifts, andgrants.¹⁵ Althoughmany clinicians maintain that their practice patterns are unaffected bypropaganda from pharmaceutical corporations, the companies themselves clearlybelievedifferently.¹⁶Sponsorship plays an important role in cultivating: expectation bias,“bandwagon bias,” and protreatmentbias.¹² Expectationbias can occur when companies aggressively hype their products before theproducts are actually tested. Subsequently, medical researchers andinterpreters may exhibit bias in finding what they expect to find. Bandwagonbias, which is closely related, occurs when companies enlist a critical massof support for their products. As McCormack and Greenhalgh note, an unstatedpresupposition seems to be that “if enough people say it, it becomestrue.”¹²Finally, drug and device manufacturers benefit from an implicit protreatmentbias. When few proven or well-publicized alternatives are available,clinicians and patients will use their products even when little or no dataexist that demonstrate clinical benefits.

The deleterious influence of corporate sponsorship has led to a review ofresearchers' financial conflicts ofinterest.^17,18Medical journals and other reputable sources of clinical interpretation nowfrequently require authors' disclosure of possible financial conflicts ofinterest.¹⁹ Whethersuch measures are sufficient to counterbalance bias remains a topic fordebate. To eradicate corrupting influences, it may be tempting to severaltogether the relationship between medical inquiry and for-profit industry.However, such measures would threaten the vitality of clinical research byundermining resources.

FALLACY OF SELECTIVE EMPHASIS

Perhaps the focus on financial conflicts of interest has diverted attentionfrom another source of bias in clinical interpretation: the fallacy ofselective emphasis. This fallacy (originally described by philosopher JohnDewey) occurs when intense focus on a narrow topic of interest manifestsitself in an unintentional but systematic exaggeration of the importance ofthe particular point ofinterest.²⁰(pp31-41)In epidemiologic parlance, we might equate this fallacy with“preoccupation” bias. When people spend much time and energyscrutinizing something, they often assume that the object of scrutiny must beimportant. Two important sources of the fallacy of selective emphasis seem toapply when medical researchers interpret their own work or related work intheir field.

First, most researchers are “superspecialists”—that is,medical specialists with particular research interests that occupy a largeproportion of their time and energy. Because of their narrow focus,superspecialists may be prone to develop partially skewed medical worldviews.But accurate interpretation requires someone who can estimate the effects ofnew ideas within a wide range of clinical scenarios and who is adroit atconsidering a gamut of competing possibilities. A competent generalist,specially qualified in interpretive methods and thoroughly familiar with thepertinent literature, seems best for this role.

Second, researchers have a natural interest in promoting their chosenspecialties and areas of research. This interest is not merely a matter ofmoney and greed but may involve the enhancement of prestige, professionalself-image, and the sense of a personal calling. Researchers would not choosetheir specialties if they did not think that they were interesting, promising,and important. These sources of optimism are a credit for the medicalresearcher but a liability for anyone with the task of interpreting themedical literature—especially for those involved in deciphering theeffects of their own work.

TOWARD EFFECTIVE CLINICAL INTERPRETATION

McCormack and Greenhalgh have suggested that journal editors“encourage authors to present their results initially with a minimum ofdiscussion so as to invite a range of comments and perspectives fromreaders.”¹²This is excellent advice but may fall short of an adequate remedy for thefallacy of selective emphasis (because most clinicians read the literatureselectively). If journal editors and regulators could turn to a cadre ofrelatively neutral, uninvested clinicians with special methodologic expertise,then perhaps the threat of preoccupation bias would be mitigated evenfurther.

To effect such a scenario, two strategies suggest themselves. First,medical centers and universities could distinguish between two types ofclinical role or faculty appointment: clinician-researcher andclinician-interpreter. Second, medical journals, medical societies, and/orgovernment regulators could assemble boards of expert clinician-interpretersto review scientific studies and make recommendations for practice or publicpolicy. Britain's National Institute for Clinical Excellence—whichprovides national guidance on individual technologies, the management ofspecific conditions, and clinical audit—might exemplify this sort ofproject, if it were to recruit effective clinician-interpreters to developguidelines and review the literature.

Clinician-researchers would continue with their current responsibilitiesand would be free (or even encouraged) to collaborate with private, for-profitindustry. Probably most clinician-researchers would continue to besuperspecialists. Clinician-interpreters, on the other hand, would becommitted to attitudes of economic impartiality, balanced skepticism, andmethodologic rigor. The best clinician-interpreters would be experiencedgeneralists skilled in evaluating study design and addressing questions ofexternal validity. (The external validity of a study is the degree to whichresults of the study apply in actual practice settings.) Faculty in internalmedicine, family practice, general pediatrics, and emergency medicine could beideally suited. To guarantee familiarity with a wide range of practicesettings, the recruitment of clinician-interpreters from rural regions orother settings outside the academy might also be desirable.

When government agencies such as the Food and Drug Administration (FDA)seek expert advice, they could turn to such panels of clinician-interpreters.This policy would be a significant departure from current practice. When thePeripheral and Central Nervous System Drugs Advisory Committee (of the FDA)considered the use of tissue plasminogen activator (tPA) for acute stroke, itheard mostly from clinical investigators involved in the NINDS trial (the onlymajor trial in which tPA showed significant benefit over nontreatment). Mostof these investigators were also employed by Genentech (which markets tPA).The composition of the advisory committee was itself skewed, consistingentirely of neurologic specialists without a single emergency physician orother generalist (transcript of Advisory Committee meeting, June 6, 1996, opencommittee discussion on Product License Application 96-0350 for Activase[alteplase], Genentech, for management of acute stroke. Obtained from FDA,Bethesda, MD).

Interestingly, the FDA's advisory committee invited public participation,but only Karen Putney of the National Stroke Association spoke up. Hercomments nicely illustrate the threat of publicity bias and protreatmentbias:

[National Stroke Association] understands that finding a treatment foracute ischemic stroke for which there is yet no approved therapy will do morethan anything else to improve public awareness and understanding, compel themedical community to treat stroke emergently, and improve patient outcome.This study heralds a new approach to stroke and carries the hope oftransforming the hopelessness surrounding stoke into active, aggressive, andeffective intervention to salvage brain tissue, reduce disability, and savelives.

We see here the usual false dogmas about thrombolysis for stroke: first,that improving public awareness about stroke is so important that it should bea factor in determining whether the use of a dangerous drug such as tPA isapproved; second, that tPA saves lives; and finally, that the medicalcommunity is remiss if it does not incorporate the new technology quickly intoits standard of care. These are the sort of errors that clinician-interpreterswould be trained to avoid.

The FDA is not the only organization that would benefit from a panel ofclinician-interpreters. When the American Heart Association examined the useof thrombolytic therapy for acute stroke in October 1999, they recruited anemergency physician—Jerome R Hoffman—but then dropped his namefrom their list of panel members when he refused to sign the resultingrecommendations (with which hedisagreed).²¹Initially, Hoffman was asked to submit a rebuttal essay, but his submissionwas never published. Later, he asked if the American Heart Association wouldpublish a shorter one- to two-paragraph dissent alongside the recommendations.When this offer was refused, he requested that the following statement beappended to the document: “Jerome R Hoffman also participated in thepanel but disagrees with the recommendations contained in this paper.”This request was also denied.

CONCLUSION

The fallacy of selective emphasis is an important source of interpretationbias that might be ameliorated by distinguishing between professionalclinician-interpreters and clinician-researchers and by recruitinginterpreters who are generalists with expertise in methodology. This strategywould integrate nicely with efforts to reduce other sources of interpretationbias, because nonresearchers are less apt to have financial conflicts ofinterest and may be less prone to expectation bias.

Figure 1.

Does this education campaign show protreatment bias?