The perils of surrogate endpoints

Introduction

Our goals in medicine are (i) to improve the quality of patients' lives, (ii) help them to live longer, and (iii) to do so at a reasonable cost. These are our true endpoints: health status, survival, and cost. It is thus entirely consistent with this point of view that these are the fundamental concepts that can be united in a formal cost–utility analysis.¹ These endpoints remain the best measures of efficacy in clinical trials comparing a new therapy to placebo or to an active control. All other measures may then be seen as surrogate endpoints or surrogates. Thus, even serious events such as myocardial infarction and stroke may be considered surrogates, as their effect is to adversely affect the critical endpoints of health status, survival, and cost. However, in common usage a surrogate is a relatively easy to measure endpoint, available over a relatively short timeframe that is used in place of the true endpoints.

Generally, surrogates are not events, but rather measurements (physiological, laboratory, or test results, e.g. biomarkers) that predict events. Thus, surrogates are most commonly measures that we can record, often with much shorter timescales than is necessary for events.² Surrogates are usually continuous variables, often but not necessarily with reasonable approximations of a normal distribution. Continuous variables, especially if approximating a normal distribution, will allow for much smaller sample sizes than dichotomous variables as well as shorter periods of follow-up and lower costs. Thus, compared with clinical outcome trials, studies with surrogate endpoints can be conducted rapidly and with much less resource use and expense than endpoint studies. Surrogates can be used in observational studies as well as in randomized trials. For instance, low-density lipoprotein cholesterol could be used as a surrogate for cardiovascular events in a non-interventional observational study. However, the most common and perhaps most critical issue is the use of surrogate endpoints in randomized trials comparing different therapies.

Potential surrogates

Any surrogate should be consistently measurable and sensitive to the intervention.³ In Table 1, we reproduce and modify a list of potential surrogates including physical exam as well as haemodynamic, blood, imaging, and other testing.² This list is not meant to be exhaustive, but rather to reflect potential surrogates in different areas of cardiovascular medicine: Hypertension, lipid disorders, diabetes, coronary artery disease, acute coronary syndromes, heart failure, and ECG abnormalities. We give an approximate, perhaps somewhat arbitrary rank to each variable's validity as a true surrogate. A surrogate is most useful when it (i) consistently predicts events in the future and (ii) if the response of the surrogate to an intervention predicts the response to the intervention in an endpoints trial.

Table 1.

Potential surrogate endpoints in clinical trials

Disease area	Potential surrogate endpoint	Validity as a true surrogate
Hypertension/vascular physiology	Blood pressure8	++++
	Carotid intima-media thickness8	++
	Microalbuminuria8,15	++/?
	Flow-mediated dilatation58,59	++
	Left ventricular hypertrophy8	++
Lipid disorders/atherosclerosis	Low-density lipoprotein cholesterol9,10,15,57	++++
	High-density lipoprotein cholesterol11,12	−
	Carotid magnetic resonance imaging60	++
	Intravascular ultrasound20	++
	Coronary computed tomography61	?
	Optical coherence tomography62	?
Diabetes	Serum glucose	++
	Haemoglobin A1c	++
	Microalbuminuria	++
Obstructive coronary artery disease	Quantitative coronary angiography63	++
	Intravascular ultrasound20	++
	Coronary computed tomography61	?
	Optical coherence tomography62	?
	Restenosis after PCI28–30	++
	Angiographic variables to predict restenosis31,32	+
Acute coronary syndromes	Troponins64	++
	Brain natriuretic peptide64	++
	Infarct size65	?
	Return of TIMI flow66,67	?
	Resolution of ST elevation67	?
Heart failure	Exercise capacity68,69	−
	Haemodynamics (e.g. cardiac output)69	−
	Ejection fraction33	++
	Remodelling (e.g. LV volume)70	++
	Brain natriuretic peptide69,71	−
Electrophysiology	Premature ventricular beats36	−
	Late potentials72	−
	Non-sustained ventricular tachycardia36 TTaTachycardia	−

The symbols ‘−‘ and ‘+’ to ‘+ ++ +’ indicate the reliability of the potential surrogate; the symbol ‘?’ indicates currently unknown. A valid surrogate or true surrogate will be on the causal path to and have a strong, consistent statistical relationship with the clinical endpoint.

Hypertension

The variable perhaps most often thought of as a consistently useful surrogate in interventional trials is the simple measurement of blood pressure. Lowering blood pressure using different therapies has consistently resulted in reduced events, in particular stroke.⁴ However, even this relationship is not straight forward. For instance, while blood pressure is related to event rates to pressures <120 mmHg systolic, there is insufficient evidence that lowering blood pressure with pharmaceutical in patients with hypertension to <140 mmHg systolic will reduce event rates.⁵ Furthermore, in some trials similar blood pressure reduction led to different effects on hard endpoints such as mortality and stroke.^6,7 Other measures of vascular physiology are less reliable. Despite its limitations, blood pressure remains a useful surrogate as recognized in the current guidelines from the Joint National Committee as well as the European Society of Cardiology; blood pressure level remains a therapeutic goal.^4,8 However, blood pressure may not be useful in all clinical situations. For instance, blood pressure will fall in patients with a history of hypertension who develop heart failure. In this common clinical scenario, blood pressure will no longer be a useful surrogate.

Lipids

Low-density lipoprotein (LDL) cholesterol is the most useful surrogate lipid measure, given the many trials with statins that demonstrated both reduced LDL cholesterol as well as cardiovascular events. While, the efficacy of other agents to reduce event rates by reducing LDL cholesterol is not as well established, recent data have shown both LDL and event rate reduction with ezetimibe as well as early data with PCSK9 antibodies.^9–12 Nonetheless, it is possible that the mechanism by which statins reduce event rates may not be entirely by their effects on LDL cholesterol alone; statins also exert pleiotrophic effects on small G proteins among other effects.¹³ Of importance, there remains some uncertainty about target levels for LDL as the endpoint trials did not include target levels for LDL; leading to target levels no longer being recognized in the 2013 Guidelines from the American College of Cardiology and the American Heart Association.¹⁴ However, the European Society of Cardiology guidelines have maintained such an approach.^15,16

High-density lipoprotein (HDL) cholesterol plasma levels have been shown to predict events in epidemiological studies, but the event outcome data on pharmaceuticals to raise HDL have been unconvincing. Indeed, it appears that the biological activity of HDL cholesterol changes in patients with diabetes, coronary artery disease, and acute coronary syndromes, most likely due to a change on its proteome composition.^17–19 Surrogates related to atherosclerosis, including imaging such as calcium scoring and carotid ultrasound are also at best uncertain.^20,21

Diabetes

Diabetes is clearly a risk factor for cardiovascular events.^22,23 However, there is little evidence that tight control of diabetes will reduce cardiovascular events.²⁴ Indeed, recent trials showed that lowering haemoglobin 1Ac with blood sugar lowering drugs such as DDP-4 inhibitors or strict lifestyle measures did not reduce cardiovascular events.^25–27

Coronary disease

Measures of the extent and severity of coronary artery disease noted in Table 1 are also mixed as surrogates. While the extent of coronary artery disease noted on angiograms has been shown to positively correlate with cardiovascular events, the angiogram is more useful as a guide to therapy than as a surrogate outcome. Other measures of coronary disease are also uncertain surrogates. Measures of acute coronary syndromes, such as troponins are useful diagnostically to guide therapy, but have not proven useful as surrogate endpoint for events in randomized trials. Restenosis after PCI is a surrogate for quality of life and myocardial infarction.^28–30 Various angiographic measures have also been used as surrogates for restenosis or target lesion revascularization.^31,32

Heart failure

There are inadequate potential surrogates in heart failure.³³ In particular, changes in exercise capacity, haemodynamic variables, and ejection fraction have failed to predict clinical outcome. Thus, although the ejection fraction predicts outcome, changes thereof under treatment with inotropes may even be associated with an increase in mortality.³⁴ Somewhat paradoxically, β-blockers, in spite of their negative inotropic effects, result in a slightly improved ejection fraction over time, and improved survival.³⁵ Thus, we cannot use ejection fraction or any other measure of improved left ventricular performance as a reliable surrogate in heart failure trials.

Electrocardiogram

Findings on the ECG may also predict events, but have not been reliable as guides to effectiveness of therapy. Perhaps the best known failure is this endpoint is that suppression of premature ventricular contractions (PVCs) post-MI by class I anti-arrhythmic agents will not reduce events, but rather enhanced mortality.³⁶

Why surrogates fail

A potential surrogate is often considered as an intermediate endpoint in a clinical trial because it is found to predict outcome in observational studies. Thus, blood pressure and LDL cholesterol are well known (and in fact accepted by regulatory agencies such as the Federal Drug Administration or the European Medical Agency) to predict outcome, and have been used as surrogate endpoints in clinical trials. Indeed, clinical outcome studies did show that interventions which favourably affected these surrogates did in general reduce the incidence of cardiovascular events.

However, there are also many examples where a therapy was shown to favourably affect a surrogate, but was not found to reduce cardiovascular events. Thus, serum HDL cholesterol level has an inverse relationship with cardiovascular events. Furthermore, both niacin and cholesterol ester transfer protein blockers have been shown to increase serum HDL cholesterol.^11,12,37 Nonetheless, recent trials with these agents have not shown efficacy in reducing cardiovascular events; indeed, one of them even increased mortality in spite of marked increases in HDL cholesterol.¹²

As noted above, while PVCs noted on the ECG post myocardial infarction predict events, a trial of anti-arrhythmic agents which reduce PVCs failed to show efficacy in preventing events.³⁶

As a third example, hormone replacement therapy in post-menopausal women will favourably affect the serum lipid profile. However, randomized trials of hormone replacement therapy have not been shown to reduce cardiovascular events.³⁸ It is also possible that the problem with a therapy that works on a surrogate but failed in an outcomes trial was due to inadequate dosing, too short a time period, an inappropriate patient population or too small a population.

Causality

Why do some potential surrogates seem to work well, while others fail? We can gain insight by considering the nature of causality.^39,40 A true surrogate should be in the causal path of a true endpoint. Thus, LDL cholesterol is a good candidate true surrogate, as high LDL causes more events by directly augmenting atherosclerotic plaque formation, a prime culprit for myocardial infarction and cardiovascular death. On the other hand, lowering LDL causes the event rate to be lower as it reduces the lipid content of plaques and hence the vulnerability of plaques. To establish causation requires a deep understanding of the pathophysiology of the disease process and hence is a stronger criterion than just noting an association. If there is association but not causation, then the relationship between a surrogate and outcome events may be confounded. A confounder is a variable that predicts outcomes and has higher prevalence in the group with the potential surrogate of interest.³⁹ Thus, PVCs may be confounded by left ventricular function, whereby patients with left ventricular dysfunction have more PVCs, and it may be the left ventricular dysfunction which causes increased mortality. If this is the case, a drug which decreases PVCs may have no effect on left ventricular function and thus may not exert any effect on subsequent mortality.³⁶

A true surrogate will always be affected before the clinical endpoint as it often precedes late disease states that lead to myocardial infarction, stroke, or death. However, it is wrong to think that a temporal relationship is all that is necessary to establish causality. This has been recognized since ancient times as noted by the famous humorous statement – ‘post hoc ergo propter hoc’, which is Latin for ‘after this, therefore because of this’. Temporality is just one component of what is necessary to consider causal relationship.

This was considered by pioneering British epidemiologist and statistician Austin Bradford Hill. In the 1950s Doll and Hill^41,42 published a pair of sentinel papers showing the association between cigarette smoking and lung cancer. These papers, plus data from Framingham on the impact of cigarette smoking on cardiovascular disease risk,⁴³ helped move the US Federal Government in 1964 to issue the first Surgeon General's Report on Smoking and Health.^44,45 Nonetheless, before the 1960s there was controversy over whether cigarette smoking was causally related to lung cancer as opposed to just being an association. Hill considered the steps necessary to establish a causal relationship between any risk factor, i.e. surrogate, and future events.⁴⁶ The Bradford Hill criteria are: (i) temporal relationship, the cause must always come before the effect, (ii) strength of association, (iii) dose–response relationship, (iv) consistency of the relationship, (v) biological plausibility, (vi) consideration of alternatives, (vii) experimental verification, (viii) specificity, that is a specific cause for a specific effect, and (ix) coherence, that is compatible with existing knowledge.^39,46 Specificity is often omitted as it is generally not fulfilled for diseases which may have multiple causes. Establishing causality requires consideration of these criteria, and then general acceptance by the scientific community and society at large. Now a surrogate need not be the root cause of the endpoint, but it should be on the causal path. The Bradford Hill criteria can provide useful guidance in this respect. For instance, if we consider whether LDL cholesterol, we may note that it fulfils these criteria, perhaps with the exception of specificity.

In Figure 1, we offer a conceptual model of the various ways in which the relationship between a potential surrogate and a clinical endpoint on a causal path can occur. In Figure 1A, we have a true surrogate which is in the causal path, with the intervention occurring earlier in the causal path, and thus eliciting a response in both the surrogate and endpoint. This is the only type of relationship in which the surrogate will be reliable. This is perhaps most clearly demonstrated in therapy for hypertension, where pharmaceutical therapy reduces blood pressure and reduces the incidence of strokes. The Bradford Hill criteria would also suggest that a blood pressure response to treatment is in the causal path to cardiovascular events. In Figure 1B, the potential surrogate is in the causal path, but the intervention is downstream of the surrogate. In this case, the intervention affects the outcome but not the surrogate. For instance, coronary atherosclerosis is in causal path leading to cardiovascular death. However, an intervention such as aspirin may occur downstream, preventing the outcome but not affecting the surrogate. In Figure 1C, the potential surrogate is in the causal path, but the intervention only affects the surrogate. This could explain why ventricular tachycardia is in the causal path to sudden death, but anti-arrhythmic drugs may only affect the surrogate and not prevent the endpoint. In Figure 1D, the intervention elicits a response in both the surrogate and the endpoint, but the surrogate is not in the causal path. This type of situation may be unstable, and not consistently reproducible. In Figure 1E and F, the potential surrogate is not in the causal. In Figure 1E, the intervention only affects the surrogate and not the endpoint, and in Figure 1F, the intervention only affects the endpoint and not the surrogate. In these latter two circumstances, the potential surrogate is not really a surrogate at all. Figure 1G shows a more complex scenario: a potential surrogate is in the causal pathway, but there are other pathways to the endpoint that are independent of the surrogate. For instance, while statins will lower LDL cholesterol and decrease cardiovascular mortality secondary to atherosclerotic disease, cardiovascular mortality may occur independently of therapy with statins and independently of LDL cholesterol. Similarly, blood pressure elevation is on the pathway to myocardial infarction, but there are other pathways with myocardial infarction occurring independently of hypertension. In this setting, blood pressure control may decrease the incidence of myocardial infarction without eliminating it. Empiric data from clinical trials can show whether an intervention affects both the surrogate and the clinical endpoint. However, without clear understanding of the biologic mechanisms involved it cannot be certain if the surrogate is in the causal path. This framework around causality is conceptually helpful, but not necessarily practical as it is often not clear for any potential surrogate whether it is in the causal path. Thus, a more practical empirical approach is needed.

Figure 1.

Conceptual model of the various possible relationships between a potential surrogate and a clinical endpoint. Only (A) is a true surrogate. See text for explanation of the panels. Reprinted with permission from Elsevier (The Lancet 2010;376:1673).

The relationship of surrogates to clinical endpoints

The statistical relationship of surrogates to clinical endpoints is multi-faceted and needs careful explanation. Surrogates, such as biomarkers, are usually continuous variables while events are generally binary or categorical. The relationship of a surrogate to the clinical endpoint is more complex than that of a risk factor to an endpoint in that a therapy's value is based on its effect on the clinical endpoint. If a surrogate was considered only as a risk factor, then standard methods of discrimination (e.g. the c statistic or R²), calibration and validation could be used. However, the key issue is whether therapeutic efficacy as assessed by the surrogate captures therapeutic efficacy on the clinical endpoint. The mathematical expression of this relationship has several aspects. Prentice proposed that a surrogate must capture any statistical relationship between the treatment and the true endpoint.⁴⁷ This would be consistent with Figure 1A, where the surrogate and the clinical endpoint are clearly on the causal path after the intervention. However, it is recognized that a surrogate may not explain all of the relationship between the intervention and the endpoint.⁴⁸ This more complex relationship would be consistent with Figure 1G.³ How can this relationship be explained? An early proposal was the proportion of treatment effect (PTE) of a clinical endpoint explained by a surrogate endpoint. Proportion of treatment effect may be defined as (β-βa)/β where β and βa are, respectively, the differences between treatments in a trial without and with adjustment for the potential surrogate endpoint.⁴⁹ However, PTE is not a valid proportion as it can have values outside of the range 0–1 (e.g. where βa is negative).^49–51 Buyse and Molenberghs⁵² built on these ideas to develop a pair of metrics, RE (relative effect), which is the effect of treatment on the true endpoint relative to that of treatment on the surrogate, and γ_a, which is the association between the surrogate and true endpoint after adjustment for treatment. A surrogate is perfect at the individual level if there is perfect association between the surrogate and the true endpoint after adjustment for treatment. A surrogate is said to be perfect at the population level if RE = 1. Buyse and Molenberghs⁵² initially considered the case where the outcomes of both surrogate and true endpoint were jointly normal or jointly binary in a single trial. Buyse et al.⁵³ extended this to meta-analyses and then Molenberghs et al.⁵⁴ extended this to mixed discrete and continuous outcomes. The metrics which follow from these analyses are $R_{\mathrm{trial}}^2$, which is the quality of a surrogate at the trial level, and $R_{\mathrm{indiv}}^2$, which is the individual patient-level association between the surrogate and true endpoint after adjustment for trial and treatment effects. These R² values range from 0 (a useless surrogate) to 1 (a perfect surrogate). Furthermore, these values are independent of the strength of the effect of treatment on either endpoint. A practical issue is that it is necessary to have large datasets from randomized trials to achieve good precision for these values, which is critical to permit reliable prediction of the treatment effect on the true endpoint given the treatment effect on the surrogate.³² Finally, estimates of these values should be validated across a number of trials to ensure consistency of the relationship.^32,50,54 Such mathematical expressions of the relationship between a surrogate and true endpoint have been rarely developed in practice.³² Further insights concerning the statistical relationship between surrogate and clinical endpoint are offered by Weir and Walley⁵⁵ for a mathematical review and by Hughes⁴⁹ for a narrative review.

How can surrogates be used?

Given the complex relationship between interventions, surrogates, and endpoints, how can surrogates best be used? It is unusual for a surrogate to be so reliable that it can replace clinical endpoints for regulatory approval and medical decision-making purposes. Studies with surrogate endpoints will generally be much less expensive and much more rapid to conduct than studies with clinical endpoints. However, it is very difficult to be confident about the relationship between the surrogate and the clinical endpoint. Studies with surrogates will also generally have a smaller number of patients and a shorter time span. This limits such studies for evaluation of safety, where safety endpoints may have no pathophysiologic relationship with the surrogate. Thus clinical endpoint trials will remain essential, although even endpoint trials may not have adequate power for safety. Given problems that have been noted in the past with regulatory bodies approving therapies based on surrogates, it is likely that the demand for endpoint studies for regulatory approval will be the norm, and use of surrogates for licensing new therapies will be the exception. Trials with surrogate endpoints that do not show efficacy may obviate the need for endpoint trials, saving time, expense, and avoiding unnecessary patient risk. Surrogates will remain of interest in developing new therapies and providing pathophysiologic insight and guiding the development of clinical endpoint trials.

For surrogates to be useful, even if only as a phase II trial rather than a pivotal trial for registration purposes, there are several criteria to consider (Table 2).⁵⁶ Thus, the potential surrogate should be considered in specific disease states, for specific therapies and for specific clinical outcomes. Thus, LDL cholesterol can be used as a surrogate for statins in patients with hyperlipidaemia and the endpoint is cardiovascular death. Next, there should be epidemiologic evidence of association between the surrogate and events. While blood pressure is an excellent example of this, HDL cholesterol, perhaps unfortunately, is as well. The surrogate should also respond to therapy consistently. Thus, LDL cholesterol is consistently reduced by statins. Next there should be a strong statistical relationship in the reduction of clinical events explained by the surrogate. Although, realistically, surrogates will also generally be developed without the mathematical rigor reviewed above. Finally, the reduction of events across trials is seen to be consistent.³² Thus, statins consistently reduce LDL cholesterol, and consistently reduce events (Figure 2, with permission).⁵⁷

Table 2.

Criteria for validating a surrogate

Define patients, treatments, and clinical endpoints for which the potential surrogate applies. A strong statistical association between the surrogate and the clinical outcome of interest. Strong, consistent evidence of treatment differences in the surrogate for each trial. Treatment difference in clinical outcome within each trial is statistically explained by the surrogate. Across trials, magnitudes of treatment difference in the surrogate and in the clinical outcome are closely linked.

Figure 2.

The relationship between low-density lipoprotein lowering and cardiovascular events in 26 clinical trials of statins.

Conclusions

The use of surrogates is complex, and there is no single criterion or standard that can readily be applied. An understanding of causality and consideration of the relevant practical criteria is important, but the adoption of a surrogate must always be considered on a case by case basis. The place of surrogates in phase II trials may be reasonable as a guide to pivotal phase three trials. However, the uncertainty of surrogates must limit their use in phase III trials, where the unreliability of surrogates alone for registration is recognized, so as to avoid potential risk to public health.

Funding

Work supported by an Institutional Development Award (IDeA) from the National Institute of General Medical Sciences of the National Institutes of Health under grant number U54-GM104941 (PI: Binder-Macleod).