Composite and surrogate outcomes in randomised controlled trials

Abstract

Composite end points may mislead—and regulators allow it to happen

In this week's BMJ, Ferreira-González and colleagues report that clinical trials may mislead if they use composite end points.¹ For example, a statement that an intervention reduces a composite end point of cardiovascular mortality, myocardial infarction, and revascularisation procedures is misleading if revascularisation procedures were more common outcomes than death or infarction, or if the intervention had a large apparent treatment effect on revascularisation but not on death or infarction.¹ It is not enough for people who use the research—doctors and patients—to be aware of such potential to mislead: pharmaceutical regulators should also examine their role.

Pharmaceutical regulation has provided benefit to society by harnessing the innovation of industry towards improving health. Pharmaceutical regulation helps to ensure that drugs are safe and achieve clinically relevant benefits for patients. Regulation also governs the manner in which drugs may be marketed to prescribers and to patients. It allows only claims that can be supported by trial evidence to be used as a basis for promotional activities. Ensuring the evidence base of information to prescribers is a laudable aim, but as Alfred North Whitehead said, “We think in generalities, but we live in detail.” The implementation of regulation has led to innovation in trial design, specifically in the choice of primary outcome measures, such as composite outcome or surrogate measures, which can lead to major challenges for people trying to interpret and use results of research, who may not be adequately served by the process.

A key aspect of drug regulation is embodied in the principles of α spending—that is, the allocation of type 1 error (the error of rejecting a null hypothesis when it is actually true) in a manner designed to avoid spurious positive conclusions.² Because the likelihood of observing a statistically significant result by chance alone increases with the number of tests, it is important to restrict the number of tests undertaken and limit the type 1 error to preserve the overall error rate for the trial. To do this, the type 1 error is allocated to different outcomes, most simply through the specification of a single primary outcome measure (where a one sided P value of <0.025 is conventionally regarded as statistically significant). Alternatively, the available type 1 error may be split between different primary outcomes, or indeed outcomes may be placed in a predefined hierarchical list, with type 1 error “spent” down the list until the conventional one sided 2.5% α level is reached (equivalent to a two sided 5% level—but no drug is ever licensed for being significantly worse than the comparator).

In terms of whether a trial has a positive or negative result, the choice of primary outcome may be of central importance. For example, the recent ADOPT trial³ compared time to failure of monotherapy (defined as a confirmed concentration of fasting plasma glucose of >180 mg/dl) in 4360 newly diagnosed patients with type 2 diabetes treated with rosiglitazone, metformin, or glyburide who were followed for a median of four years. Although this surrogate primary outcome was highly significantly in favour of rosiglitazone, no difference was seen in death rate or rate of hospital admission between the groups, and no other indication of clinical benefit was seen for patients in the rosiglitazone group. The positive primary outcome reported by the study is expected to boost sales of the product.⁴ However, after surveying the list of adverse events in the trial, it is hard to conclude that patients benefited from a longer time to failure of monotherapy.

An additional challenge in the market authorisation for pharmaceutical products is the use of composite outcomes, which potentially provide an opportunity for sponsors to “game” their trials.¹ Composite outcomes bring together two or more events that are considered as a single outcome.^{5 6} They have statistical advantages because in time to event analysis the statistical power of a study is driven by the number of events that accrue, rather than the number of patients randomised. Composite outcomes can help in avoiding arbitrary decisions between different candidate outcomes when prespecifying the primary outcome, and they have several advantages. However, a positive result for a composite outcome applies only to the cluster of events included in the composite and not to the individual components.

Regulatory behaviour may have led to the addition of “death” to many composite primary end points used in trials, and it is our experience that the Food and Drug Administration has actively promoted the use of such composite outcome measures in heart failure trials. The DREAM trial, of rosiglitazone in the prevention of diabetes in patients with impaired fasting glucose or glucose tolerance (or both), had the composite primary outcome measure of diabetes or death.⁷ The primary outcome was highly statistically significant, although there was no difference in the rate of death between the groups (30/2635 (1.1%) in the rosiglitazone group and 33/2634 (1.3%) in the placebo group). If the FDA follows standard practice, it will react to an application for extension of the marketing authorisation by granting authorisation for the composite outcome. To do so would wrongly endorse the idea that mortality was reduced.

The European licensing process seems to follow the FDA lead. Indeed, as the US represents about 60% of the world market for drugs, FDA policy drives the design of regulatory trials. Two areas of concern require attention. Firstly, the regulators should ensure that primary outcome variables in regulatory trials really do “provide a valid and reliable measure of some clinically relevant and important treatment benefit in the patient population . . .,”⁸ as required in the regulators' own guidance on the design and analysis of clinical trials. Secondly, while it is our opinion that composite outcome measures do have a useful role in the evaluation of health technologies, the difficult problem of the appropriate interpretation of composite outcome measures in regulatory policy should be dealt with. This might be achieved by using a corollary (such as a health warning), which makes it clear that on their own the individual components of a composite have not been shown to be affected by the experimental treatment.