The (Mis)interpretation of Hazard Ratios in Clinical Trials

Introduction

The last decade has brought a renaissance of clinical trials in nephrology. Measures of effect from these trials should accurately convey benefits and harms of treatment to facilitate shared decision making. Although trials can report multiple effect measures, the hazard ratio (HR), estimated using a Cox proportional hazards model, is commonly reported. Despite their ubiquity, HRs are frequently misinterpreted, including in major kidney guidelines.¹ Previous authors have also questioned the utility of the HR when interpreting trial findings.^2–5

We review how and why HRs can lead to misinterpretations of trial findings and outline four key limitations of HRs: (1) frequent misinterpretation as a risk ratio, (2) violations of the proportional hazards assumption, (3) inherent selection bias in period-specific HRs, and (4) the noncollapsibility of HRs.

Hazard Ratios Are Not Risk Ratios

Risk is the probability of experiencing the incidence of the outcome by a specified time. It ranges from zero to one and cannot decrease as follow-up time accrues. With complete follow-up, risk is estimated by the number of individuals who develop the outcome divided by the number of individuals at the start of follow-up. However, follow-up ends before the event of interest occurs for some patients, referred to as right censoring. With right censoring, the risk is no longer a simple proportion. Instead, risk is commonly estimated using the Kaplan–Meier estimator. The Kaplan–Meier estimator works by calculating conditional probabilities at each event time (probability of the event occurring among event-free patients at that time) and then aggregates these probabilities across time. Group comparisons can then be made using risk differences or risk ratios at different times.

In contrast, the hazard is the instantaneous rate of events among individuals who have not yet had the event before that time. Hazards can increase or decrease over time and can be any non-negative number (i.e., ≥0). To use a simple analogy, if risk is the proportion of patients who have progressed to kidney failure (the destination) by time t, hazard is the instantaneous speed (the velocity) at which kidney failure is approaching for those individuals who are still event-free. The HR is essentially an average of these relative velocities across follow-up, whereas the risk ratio compares the proportion who have reached the destination at one specific time point. Thus, the HR and risk ratio are fundamentally different measures of effect: the magnitude of the HR cannot stand in for the magnitude of the risk ratio.⁶

A constant HR over a study period can correspond to a changing risk ratio.⁷ To illustrate, consider a trial in advanced CKD. In the control group, the monthly hazard (velocity) of progressing to kidney failure is 1% (i.e., each month 99% will not progress). The 1-, 3-, and 5-year risk of kidney failure will be 11.4% (1–0.99¹²), approximately 30%, and approximately 45%, respectively. Now consider a treatment with a constant HR of 0.6 reducing the hazard to 0.6% (1%×0.6) per month. The risks in the treatment group at 1, 3, and 5 years are approximately 7% (1–0.994¹²), approximately 19%, and approximately 30%, respectively. The risk ratio rises from 0.61 (7%/11.4%) at 1 year to 0.67 at 5 years. Although treatment maintains a constant 40% reduction in hazard of progression (HR, 0.6), the risk ratio shrinks as the study progresses.

Violations of Proportional Hazards Assumption Are Common in Clinical Trials

The Cox proportional hazards model relates treatment assignment to hazard. A key feature of the Cox model is the proportional hazards assumption, i.e., hazards in the two groups are proportional over time so the ratio of these hazards (HR) is constant over the trial's duration. In our previous CKD example, this means the velocity of progression to kidney failure in the treatment groups is always 60% of the control group. Irrespective of how the absolute pace of progression in the cohort changes over time, the ratio between these two velocities must remain constant.

The proportional hazards assumption can be untenable when immediate and delayed effects are in opposite directions.^5,8 In PEXIVAS, a trial of therapies in severe vasculitis, the cumulative incidence curves diverged, converged, and crossed during the 6-year follow-up period.⁹ Comparing reduced- with standard-dose glucocorticoids, these curves suggested early benefit followed by potential harm in the long term. As the ratio of the hazard functions changes over time, a single HR is not an informative summary of the treatment effect across the entire follow-up period.

The proportional hazards assumption is also unlikely to hold in many trials, if there is no immediate effect of treatment.^3,8 In the FIDELITY analysis of finerenone in diabetic kidney disease, the outcome of major adverse cardiovascular events (MACE) differed little during the first year.¹⁰ The overall HR of 0.86 represents a weighted average of the HR over the entire follow-up period. This weighted average is dependent on the pattern of censoring. If many FIDELITY participants were censored early (had short follow-up), then later effects contribute very little and the overall HR moves toward 1.0. If more participants remained under long follow-up, where effect was more apparent, the overall HR shifts further from 1.0.

This sensitivity to the censoring distribution is a property specific to the HR. Measures based on cumulative risk (risk ratio, risk difference) are not affected by the pattern of censoring in this way.⁵

When using the Cox proportional hazards model, whether the proportional hazards assumption holds should be carefully considered, particularly as formal tests may be underpowered in small- and moderate-sized trials.⁸

The Period-Specific Hazard Ratio Suffers from In-Built Selection Bias

To address the issues raised in the previous section, it is tempting to divide follow-up into intervals when the proportional hazards holds and report HRs within each period. However, these period-specific HRs cannot be meaningfully interpreted, as we are no longer comparing the original randomized populations.⁴

Randomization guarantees comparability at baseline; difference in outcomes between the originally randomized groups can be ascribed to the causal effect of treatment. As a trial progresses, the disease severity of participants who remain event-free changes: Participants at the highest risk may have already experienced the outcome in one arm but not the other. Comparing outcomes between survivors at the start of each interval “breaks” randomization and introduces selection bias.

Thus, period-specific HRs cannot be interpreted as changes in treatment effect over time. They likely reflect differential loss of high-risk participants resulting in different populations being compared at the start of each period.³

The Hazard Ratio Is Not Collapsible

Risk differences and risk ratios are collapsible measures. This means the overall risk ratio will always be between the largest and smallest risk ratios of any subgroups at a given time. For example, the overall trial risk ratio will always lie between the separate risk ratios for male and female participants. The HR is noncollapsible: The overall HR can be larger (or smaller) than all subgroup HRs at the same time point.³ This mathematical property complicates interpretation of the HR.

In PISCES, a trial of fish oil supplementation in patients undergoing hemodialysis, a variation of the Cox model was used to analyze recurrent MACE.¹¹ The overall HR of 0.57 is not a weighted average of the subgroup HRs: 0.50 (prior cardiovascular events) and 0.55 (no prior cardiovascular events). This cannot be due to confounding because history of cardiovascular events was balanced by randomization. This phenomenon also makes it tricky to establish whether subgroup HRs differ because of treatment heterogeneity or a mathematical consequence of noncollapsibility.

Moreover, adjusting for a covariate predictive of the outcome in a Cox model can change the magnitude of the HR, even when the covariate is balanced by randomization. In contrast, the risk difference and risk ratio would remain unchanged. Adjustment also shifts the HR's interpretation from a marginal effect (effect had everyone received one therapy versus had everyone received the comparator) to a conditional (covariate-specific) effect. This complicates efforts to meta-analyze HRs across trials that used different covariate adjustment sets.

Alternative Measures of Effect in Time-to-Event Data

Given the complexities of interpreting the HR and its frequently untenable assumptions, HRs should rarely be used as the sole summary of trial findings. Risk differences and risk ratios at clinically relevant, prespecified time points avoid many of the interpretation problems posed by HRs.

When a single summary measure across the entirety of follow-up is desired, one alternative is the restricted mean survival time (RMST). RMST is the average amount of event-free time a participant experiences and corresponds to the area under the survival curve.^2,6 The difference in RMST between trial arms represents the average event-free time a participant would gain (or lose) because of treatment at a specific time horizon. For example, SPRINT found an intensive systolic BP target (<120 mm Hg) resulted in an overall HR of 0.74 for composite MACE. A more patient-centered interpretation of SPRINT data is that targeting a systolic BP of <120 mm Hg delayed MACE by an average of 15 days over 4 years compared with a standard BP target.²

The predominance of the HR likely reflects statistical convenience and long-standing convention. The evolving landscape of trials in nephrology provides an opportunity to adopt effect measures (e.g., risk difference/risk ratio/RMST) that convey findings in ways that are more interpretable, meaningful, and relevant for decision making.

Supplementary Material

jasn-37-2319-s001.pdf ^{(1.4MB, pdf)}

Disclosures

Disclosure forms, as provided by each author, are available with the online version of the article at http://links.lww.com/JSN/F890.

Author Contributions

Conceptualization: Amarnath Marthi, Paul N. Zivich.

Project administration: Amarnath Marthi.

Supervision: Amy K. Mottl, Paul N. Zivich.

Writing – original draft: Amarnath Marthi.

Writing – review & editing: Amarnath Marthi, Amy K. Mottl, Paul N. Zivich.

Funding

None.