Outcomes, Outcomes, Every where, nor any Stop to Think?

When judging clinical evidence from the medical literature, it is not just the results that require evaluation. The outcomes assessed in a study must also be scrutinized. Although ideally outcome selection would not affect study results and their interpretation, this cannot be assumed.^1,2 Hochman and McCormick³ add to this literature by raising questions about two types of outcomes, surrogate and composite endpoints, reported in recent high-profile randomized trials. They also draw attention to the potentially misleading reporting of relative risks without companion absolute risks, which for uncommon outcomes may lead to an overly optimistic sense of an intervention’s benefit or alternatively an overly negative portrayal of harm.

The authors extend previous observations that surrogate and composite outcomes appear commonly in the literature.^2,4–6 In addition to reporting the ongoing frequency of these outcome reporting issues, Hochman and McCormick found potentially worrisome associations between surrogate endpoints and both commercial funding and positive results. Surrogate endpoints were more common among commercially funded trials in the present study, in contrast to a previous report limited to cardiovascular trials.² Furthermore, trials using surrogate endpoints were more likely to report positive findings.

It must be noted that these observations should not be viewed as an indictment of commercially sponsored research. It is possible that results from such studies are more likely to be positive because they stem from more plausible study hypotheses,⁷ for example, although literature providing evidence in support of this possibility is lacking. These results are further complicated by the potential subjectivity in existing definitions of surrogate endpoints, and it is possible that there would be disagreement on some of the determinations of surrogate versus clinical endpoints in this study. Therefore, it will be important to validate the current results using a reproducible consensus definition of a surrogate endpoint, a goal perhaps within reach.⁸ Even if confirmed, the appropriateness of surrogate endpoints and the accuracy of study conclusions based on surrogate endpoints will need to be further evaluated to determine if positive results from such studies reflect trustworthy findings or provide evidence of systematic bias.

Hochman and McCormick also found numerous trials applying composite endpoints comprised of elements of disparate clinical importance, rendering them difficult to interpret in a clinically meaningful manner for patients. This problem has been reported previously,^5,9 but unfortunately appears to persist in the general medical literature. Interestingly, the authors found that composite endpoints were unexpectedly associated with negative study findings, although this result was of borderline statistical significance. The authors note that confounding of composite endpoints with other contributors to negative study outcomes may explain this finding, so further work is needed to clarify the relationship between composite endpoints and the direction of study results. Regardless of the ultimate resolution of this question, inconsistent or even inappropriate construction of composite endpoints is a common and completely avoidable threat to appropriate understanding and interpretation of trial results.

Finally, the authors confirm previous reports finding that absolute risk reductions are not detailed in many clinical trials with positive results.^10,11 This limitation of the literature continues despite the fact that reporting absolute risk measures is recommended by current guidelines endorsed by most prominent medical journals, including the CONSORT¹² and STROBE¹³ statements for clinical trials and observational studies, respectively. Of note, Hochman and McCormick assessed reporting of absolute risks only in the abstracts of the included trials, so it remains possible that these trials may have reported absolute risk measures in the body of their reports. However, the abstract is the most visible and widely distributed component of a paper, so complementing relative risk measures with absolute risk reporting in the abstract is particularly desirable.

If problematic use of surrogate and composite endpoints and inadequate reporting of absolute risks remain common in the face of existing and widely accepted guidelines, what can be done to improve the situation? First, consumers of the medical literature should demand results reported and discussed in a manner not intended to convey a misleading sense of the impact of a trial’s findings. Because this expectation is regularly not met,¹⁴ alternatives must be sought. One proposal has been to simply not read discussion sections,¹⁴ although requiring an appropriately balanced discussion section would certainly seem preferable to acting as if there were no discussion at all.

Second, journals should extend current recommendations by mandating and enforcing report of both absolute and relative risks where applicable (ideally in the abstract as well as the main text of an article). If authors feel this is inappropriate for a given study, it should be their responsibility to argue the point rather than a journal’s responsibility to support the necessity of this reporting.

Third, journals should exercise greater caution when publishing trials reporting surrogate or composite endpoints, to ensure that the text of such papers remains consistent with the composition of these endpoints. In this regard, existing guides to the appropriate application of surrogate¹⁵ and composite endpoints¹⁶ may be particularly useful. For composite endpoints, for example, the components should be of similar importance to patients, occur with similar frequencies, and reflect similar treatment effects.¹⁶ This is analogous to deciding if it is appropriate to pool study results from a systematic review in a meta-analysis: at some point, differences become extreme enough that the results can only sensibly be reported separately, and pooling in fact serves only to present a misleading summary of the data. At the very least, results for each component of a composite endpoint should be presented, and it does not seem too much to ask that composite endpoints be interpreted with thoughtful consideration of the relative importance of their components.

Finally, secondary literature providing evidence-based appraisal of current studies (for example, ACP Journal Club) may efficiently connect readers with accurate assessments and interpretations of study results. These appraisals typically focus on accepted standards for minimizing bias affecting common study designs, and can clarify where a study is particularly strong or weak in adhering to these standards. Of course, even these distillations of primary work may be shaded by bias, so readers must remain alert. In the end, it is incumbent on us as readers and reviewers to provide our own interpretations of study findings and not simply accept authors’ presented conclusions.

Surrogate and composite endpoints can be useful in well-defined situations, but the appropriateness of their application in clinical trials appears to be assessed infrequently. Similarly, absolute risks should be required elements of result reporting for clinical trials, but in fact are not uniformly reported. These are resolvable limitations of the current medical literature, and one must ask why we continue to accept these flaws in reports of scientific discovery. There is truly an ocean of evidence surrounding us, but if we imbibe indiscriminately it is our patients who suffer. “Enlightened skepticism”¹⁷ (in keeping with the theme, taking published findings with a grain of salt) seems especially prudent if we are to successfully navigate these seas.