Therapeutic Innovations and Diminishing Returns

Acceptance of therapeutic innovations into practice often requires demonstrating and quantifying a “treatment-effect”, typically measured as the difference in adverse outcome rates between the experimental and control arm of a randomized trial. This mathematical dependency of the measured treatment-effect on the control event rate creates a dilemma for medical innovation: While declining control rates signal therapeutic progress, sustained innovation theoretically requires an inexhaustible control rate. For industries dependent on therapeutic innovations, reducing outcome rates becomes both a primary goal and an existential threat.

This article discusses the fundamental challenge of diminishing control rates and how industry and trialists have responded, using examples from cardiovascular disease.

The Problem of Diminishing Returns

In the Second International Study of Infarct Survival (ISIS-2), ¹ aspirin and streptokinase each conferred a relative risk reduction in mortality of greater than 20% in acute myocardial infarction (MI); in combination, they reduced the 35-day mortality rate from 13.2% to 8.0%, an absolute reduction exceeding 5%. Subsequent innovations in reperfusion therapy have included genetically-engineered tPA (better than streptokinase²), and percutaneous coronary intervention (PCI) (better than tPA³). To PCI, stents have been added, first bare metal stents (better than balloon angioplasty⁴) and then drug-eluting stents (better than bare metal stents⁵), as well as glycoprotein inhibitors and clopidogrel. Meta-analyses of clinical trials in the current era show mortality rates approximating 4%.^3–5 Hence, two-decades-worth of trials since ISIS-2 achieved an absolute reduction in acute mortality comparable to that in ISIS-2 alone. While it is impossible to foresee what novel technologies in acute MI therapy are forthcoming, it is a mathematical truism that, given the diminished control rate, future innovations can never match the benefits already realized (at least by the important measure of case fatality).

The Figure shows that the benefits of successive rounds of innovation could be described by a declining exponential function. Assuming as a baseline the post-ISIS-2 mortality rate of 8%, if 5 sequentially-tested therapies each reduce mortality risk by 25%, the first will reduce absolute mortality rates by 2% (number needed to treat [NNT]=50), while the last by only 0.6% (NNT=167). All five therapies would produce approximately the same cumulative benefit as just the preceding two rounds of innovation.

Figure. Marginal Benefit and Unit Cost for Successive Rounds of Innovation.

Shown are the calculated absolute benefit (B, black line) and associated unit cost (U, number of trial enrollees for each incremental 1% of benefit, red line) for 5 hypothetical therapies tested in sequence, assuming a baseline mortality rate of 8% and a constant relative risk reduction (i.e. efficacy, E) of 25% with each therapy. The appendix shows the derivation of the formulae used to calculate B and U at the i-th generation of treatments based on their initial values (B₀ and U₀, at generation 0). Note the rate of change of U is sensitive to E. Over 5 generations, U increases more than 6-fold when E is 20%, more than 10-fold when E is 25% (as shown) and more than 25-fold when E is 33%.

While benefits decline exponentially, required trial sample size (and presumably the cost of both trials and therapies) approximates an increasing exponential function. The trial testing the first of the 5 therapies requires 7,036 patients, that testing the last 23,356 (power [1-β] =0.9, α=0.05). More than 3 times as many enrollees are needed for less than one-third the gain. The efficiency across these trials—as measured by the number of enrollees required per percentage point benefit—will have decreased more than ten-fold (i.e., unit costs will increase more than 10-fold, Figure ). If the minimally important clinical difference for a new therapy in acute MI is an absolute mortality benefit of 1%,² only 3 successful therapies can be brought to market before the control rate would no longer support further innovation, a process that might be described as innovation to extinction.

Control Rate Preservation

While an obvious response to this general trend would be a shift towards non-inferiority trials, resourceful trialists seeking superiority claims have sought other means of preserving control rates.

Composite Outcomes

When mortality becomes rare, composite outcomes bundling several component endpoints together (such as “major adverse cardiac events” [MACE]) are typically used. If the treatment has similar effects on each component endpoint, and each component has similar importance, the interpretation of results is straightforward. However, this is rarely the case. A study of 114 cardiovascular trials using composite outcomes⁶ found that the component endpoints of greatestimportance to patients systematically had far lower event rates and were associated with much smaller relative treatmenteffects than the less important component endpoints.

For example, in non-acute coronary artery disease, multiple trials have demonstrated the sequential superiority of balloon angioplasty to medical therapy, bare metal stents to balloon angioplasty, and drug-eluting stents to bare metal stents, on the composite outcome of MACE (defined as death, MI or target vessel revascularization [TVR]). However, meta-analysis has shown that these sequential improvements had little if any effect on either mortality or recurrent MI.⁷ The composite benefits reflected changes in TVR alone, an outcome of much less importance to patients, especially in the context of trials using protocol-driven angiography, which substantially inflates revascularization rates.

Indeed, apparent benefit in the composite outcome might even disguise undetected net harm, since the pivotal trials are grossly underpowered to examine the most important component endpoints. Preliminary, longer-term studies raised the possibility that drug-eluting stents may actually increase the risk of MI and mortality through late in-stent thrombosis.⁸ Though not subsequently confirmed, the principle remains that use of composite outcomes can amount to a kind of false advertising, whereby a treatment’s effect on the primary outcome (e.g. stents reduce the composite outcome death, MI and TVR) do not accurately reflect the treatment’s effect on its most important components (e.g. stents may not reduce death or MI).⁷

Surrogate Outcomes

The use of composite outcomes, however, is only a temporary fix to an inexorable problem; eventually even the less important endpoints become rare. Having dramatically reduced the rate of TVR with stents, trialists have recently proposed using a surrogate for TVR, continuous angiographic measures of stent patency⁹. Though correlated with revascularization rates across clinical trials, it is well appreciated that improvement in such surrogate outcomes does not necessarily reflect clinical benefit. The evolution of outcomes from hard clinical outcomes (like death) to composite outcomes (including softer, less important clinical endpoints) to surrogate outcomes (of possible, but not definite, clinical import) may signal a maturation in technological development toward innovation to extinction.

Change in Outcome Definition

More sensitive diagnostic thresholds can help preserve control rates through higher diagnostic yield, but shift the severity spectrum toward milder cases--leading to overdiagnosis and misperceptions of therapeutic effectiveness. For example, in cardiology, use of the new troponin-based definition may double the diagnosis of infarction¹⁰ among patients presenting with non-ST elevation acute coronary syndrome, illustrating how new, more sensitive diagnostic technologies might countervail the “deleterious” effects of therapeutics advances.

Globalization

The era of globalization may also offer new opportunities for control rate preservation. Multinational studies have shown substantial regional variation in outcome rates and in the application of evidence-based therapies¹¹. The first MI mega-trial to find a nominally significant reduction in mortality for medical reperfusion therapy in more than a decade (the recent ClOpidogrel and Metoprolol in Myocardial Infarction Trial [COMMIT]) was conducted exclusively in China, and had a 30-day control mortality rate of 8.1%¹². For comparison, a contemporaneous study testing clopidogrel in AMI at centers largely in North America and Europe had a control mortality rate of 2.2%¹³. Whether due to regional differences in the application of co-treatments, in supportive care or in the underlying risk of enrollees, the higher control rate in COMMIT likely permitted the detection of mortality reduction for clopidogrel—suggesting yet another potential incentive for the “reverse migration” of clinical trials to emerging economies.

Idiopathic “Control Rate Inflation”

Finally, even without these adaptations, the control rate does not always yield to increasingly effective therapies, as mathematically expected. In the aforementioned advance in interventional treatment of coronary disease, bare metal stents were first the “experimental” therapy (when compared to balloon angioplasty) and then the “control” therapy (when compared to drug-eluting stents). It was noted that the TVR control rates were generally higher in the subsequent trials testing drug-eluting stents than they had been in the experimental arms of earlier trials testing bare metal stents. This was particularly true in smaller trials, where rates of this outcome were roughly 50% higher with “control” bare metal stents compared to (historical) “experimental” bare metal stents.¹⁴ While the reason for this “control rate inflation” is unclear, for softer outcomes dependent on physician decision-making or adjudication, subtle changes between trials may be influential but not explicitly captured in the study protocols, and outcomes may also be sensitive to investigator bias.

Summary

Many aspects of therapeutics development, not least the exorbitant costs, incentivize risk-averse approaches modeled on previously successful therapies that, coupled with sophisticated strategies for “market differentiation”, might offer the best return on investment for industry, and sometimes also lead to meaningful improvements in public health. Even for these narrowly pragmatic aims, such strategies may have a natural life cycle. The Government Accountability Office on New Drug Development recently summarized empirical evidence for a rising failure rate in phase III clinical testing of new agents¹⁵. While other factors surely play a role, declining control rates may be an essential emblem of a broader process of maturation. Assuming theoretical limits on optimal human health and lifespan, the Malthusian relationship between declining control rates, therapeutic benefits and sample size is fundamental and necessarily impacts the marginal efficiency of therapeutic innovation. Though adaptations may preserve control rates permitting innovations that are undoubtedly beneficial, better understanding this relationship may permit wiser allocation of research resources.