Making Sense of Hierarchical Composite End Points in Randomized Clinical Trials—A Primer for Infectious Diseases Clinicians and Researchers

Abstract

Hierarchical composite end points (HCEs), combining features of simple composite end points and conventional ordinal end points, are increasingly being used in infectious diseases (ID) research. However, many clinicians may be unfamiliar with these novel end points, including the variety of different target parameters that may be of interest and the methods that can be used to estimate them. In this review, we provide a conceptual overview of HCEs by defining them and providing examples from the ID literature. We explain different methods for analyzing HCEs, including (1) the Wilcoxon rank sum approach (often used in studies with a desirability of outcome ranking [DOOR] end point), (2) generalized pairwise comparisons (used to estimate a win ratio or win odds), (3) proportional odds model (and the relevance of the proportional odds assumption), and, (4) the probabilistic index model. This review will help ID clinicians and healthcare providers interpret current and future research using such end points.

Hierarchical composite end points (HCEs) are increasingly being used in infectious diseases research. In this review aimed at educating the clinical reader, we provide a conceptual overview of HCEs and explain the different methods for analyzing them.

Hierarchical composite end points (HCEs) are increasingly being used in clinical trials [1, 2]. Recent examples from the infectious diseases (ID) literature include desirability of outcome ranking (DOOR) and post hoc analyses estimating a win ratio [2–4]. However, clinicians may be unfamiliar with these novel end points, which are analyzed differently from more familiar binary or continuous end points. In addition, HCE nomenclature is often inconsistent, and related methodologic concepts are named differently depending on the field that uses them. As these end points gain popularity in clinical research, a clear framework of how to understand and interpret their analyses is important for ID clinicians.

In this review, we aim to (1) give a conceptual overview of HCEs, (2) explain different methods for analyzing HCEs, and (3) clarify the nomenclature used in the literature and propose a consistent terminology for common use. To aid the reader, we have provided a glossary in Table 1 that provides definitions of terms or concepts. We illustrate these concepts in a paired article, describing how we applied different methods of analyzing HCEs in post hoc analyses of 2 clinical trials, BALANCE and CAMERA2 [5, 6].

Table 1.

Glossary of Terminology

Term	Explanation
End point/outcome types
Binary end point/outcome	An end point/outcome whereby patients can only have 1 of 2 potential states: either they have or do not have the outcome. The most common example is mortality—patients are either alive or dead at the end of the follow-up period.
Composite end point/outcome	An end point combining multiple different events of interest, whereby a patient experiencing any of the component events is considered as having the composite end point of interest.
Continuous end point/outcome	An end point for which the outcome is a continuous variable that can take any value within a specific range—for example, HIV viral load, or LDL cholesterol.
DOOR	A special case of an ordinal end point, whereby multiple separate clinical events are combined into a single outcome and ranked in order from best to worst outcome. Patients can experience >1 component outcome but are assigned the “worst” ranking based on the prespecified ranking order. Often used in infectious diseases clinical trials. Multiple consensus DOORs have been developed for various infectious syndromes.
HCE	An end point type combining features of both ordinal and composite end points; they are composite, as they combine multiple clinical events, but they are also ordinal, as these different events or component outcomes are ranked in order of importance (eg, death ranked above disease complication or treatment-related adverse events); synonyms in the literature include prioritized outcomes, ordinal composite outcomes, and ordered composite end points.
Ordinal end point/outcome	An end point/outcome for which a clinical event of interest is categorized into multiple categories, with a clear ranked order of the direction of superiority; in a classic ordinal outcome, this is typically in a single clinical domain (eg, pain, neurologic outcome, respiratory status), and categories are mutually exclusive.
Time-to-event end point/outcome	An end point/outcome that accounts for not just the occurrence of an event of interest but also the time to its first occurrence; the event can be binary (eg, death), composite (among several component outcomes, eg, major adverse cardiovascular events), or recurrent (eg, hospitalization or emergency department attendance).
Analytic methods or features
Generalized pairwise comparisons	An analytic method whereby all possible patient pair combinations are made between 2 treatment groups and each patient pair is evaluated to determine which patient has a superior outcome; this can be used for any type of outcome, including HCEs, where each pair is evaluated based on the prespecified hierarchy of outcomes.
Mann–Whitney U test	A nonparametric statistical test, often used to compare the distributions of 2 nonnormally distributed variables; statistically, it tests the null hypothesis that, for randomly selected values x and y from 2 treatment groups, the probability that x > y is equal to the probability that y > x (ie, that the distributions are identical), and it is the same as the Wilcoxon rank sum test.
PI model	A flexible semiparametric model that uses the generalized pairwise comparisons framework and can incorporate multiple different outcome types to estimate a covariate-adjusted PI.
PO assumption	The assumption in a PO model that the treatment exerts a consistent effect across all levels of the ordinal scale; there is debate in the statistical community as to how to test for this assumption (where statistical tests tend to have less power), as well as the degree to which it needs to hold.
PO model	A parametric model used to analyze ordinal outcomes, in which the distribution of the ordinal outcome is used to estimate an odds ratio representing the odds of going up ≥1 step in the ordinal scale; also known as the cumulative logistic model assuming PO or ordered/ordinal logistic regression.
Proportional win fractions model	A flexible semiparametric model used to analyze HCEs, which can include multiple covariates to estimate an adjusted win ratio.
Wilcoxon rank sum test	See Mann–Whitney U test.
Target parameters
Better DOOR probability	The commonly used target parameter of a DOOR analysis, ie, the probability that a randomly selected patient in the treatment group has a DOOR outcome superior to that of a randomly selected patient in the control (reference or comparator) group (see Concordance probability and PI).
Concordance probability	Often represented by the abbreviation c; the probability that a randomly chosen observation from the treatment group is greater than one from the control group (see Better DOOR probability and PI).
Net treatment benefit	A target parameter used in generalized pairwise comparisons analysis, which represents the difference between wins and losses for the experimental intervention; it is a positive or negative proportion (on a scale from −1.0 to 1.0), for which a net treatment benefit >0 is considered superior, and it can be interpreted as the percentage of occasions in which the treatment is superior to the control.
Number needed to treat	The number of patients needed to be treated with the experimental intervention of interest for 1 patient to receive the benefit in terms of the outcome of interest; using a more familiar binary outcome of mortality it is the number needed to be treated to prevent 1 death, and using an HCS it is the number needed to be treated to achieve a superior global outcome in 1 patient, considering all component clinical events assessed in the hierarchy.
PI	The probability that any randomly selected patient in 1 of 2 groups has a superior outcome to a randomly selected patient in the other group (mathematically written as P(A > B), where P represents probability; A, a patient in group A; and B, a patient in group B (synonyms are better DOOR probability and concordance probability).
Win odds	A target parameter that can be used to summarize analysis of an HCE, which represents the ratio of the number of wins (and half the number of ties) vs the number of losses (and half the number of ties). This differs from the win ratio as it splits the ties (ie, patients with the same outcomes across all components of the hierarchy) across both groups, and it may be preferred in trials with a large proportion of tied outcomes. A win odds takes values between 0 and infinity, and a value >1.0 represents superiority of the experimental treatment group.
Win ratio	A target parameter that can be used to summarize analysis of an HCE, which represents the ratio of the number of wins to the number of losses for the experimental treatment group; a win ratio ranges from 0 to infinity, with values >1.0 representing superiority of the experimental treatment group.
Others
RADAR	A “tiebreaker” used in some DOOR analyses, whereby antibiotic duration is used to further distinguish and rank patients within the same level on the DOOR scale (patients receiving a shorter antibiotic duration are ranked as superior to those receiving a longer duration); this can also be seen as an HCE in which antibiotic duration is the second step in the hierarchy after the initial DOOR.

Abbreviations: DOOR, desirability of outcome ranking; HCE, hierarchical composite end point; HIV, human immunodeficiency; LDL, low-density lipoprotein; PI, probabilistic index; PO, proportional odds; RADAR, response adjusted for duration of antibiotic risk.

REVIEW

Conventional Composite and Ordinal Outcomes

HCEs combine features of composite outcomes and conventional ordinal outcomes (Figure 1). Composite outcomes combine multiple binary end points into a single outcome—eg, the POET trial combined death, unplanned cardiac surgery, embolic events, or bacteremia relapse as a composite outcome, such that a patient who had any of the component outcomes was considered as having an event [7]. This increases frequency of the outcome, increasing statistical power and reducing required sample size [8]. This may be especially useful in conditions where death is infrequent or a truncating event (competing risk) that prevents observation of other end points [9]. However, one problem with conventional composite outcomes is that component events are treated equally even if they have varying clinical importance [10]. Larger changes in less important components may thus obscure smaller changes in more important outcomes.

Figure 1.

Examples of ordinal end points, composite end points, and hierarchical composite end points. Italicized acronyms indicate example trials that use the stated example end point. Abbreviations: COVID-19, coronavirus disease 2019; WHO, World Health Organization.

Ordinal outcomes are a special case of categorical outcomes, where mutually exclusive categories are rank ordered in a clinically meaningful manner, though the difference between contiguous categories may not necessarily be equidistant [11]. A common example of an ordinal scale is the modified Rankin scale for poststroke functional recovery [12]. The World Health Organization has developed a core ordinal outcome for coronavirus disease 2019 (COVID-19) randomized clinical trials, the World Health Organization clinical progression scale, reflecting a range of disease states in COVID-19 from asymptomatic (best) to death (worst) [13]. Ordinal outcomes can also provide greater statistical efficiency, by permitting a spread of event outcomes, capturing more information than a binary outcome [14, 15]. From a clinician's perspective, ordinal outcomes are appealing as they reflect different gradations of severity encountered in clinical practice. However, they may be challenging to analyze and interpret [1].

HCEs Defined

HCEs combine the 2 concepts of composite and ordinal outcomes: they are composite as they combine multiple events, and they are also hierarchical as they establish a rank order or hierarchy of clinical importance across the different component events. While a conventional ordinal outcome reflects gradations of severity of the same clinical attribute (eg, pain, function, or respiratory status), an HCE may combine multiple different clinical attributes and, at the same time, establish a rank order among them. They can thus combine multiple different health states related to efficacy and/or safety into a single end point, which may be more reflective of clinical decision making. Different component ranks of a conventional ordinal outcome are mutually exclusive (as one cannot have 2 different levels of the same clinical attribute), whereas different components of an HCE may overlap—but the “worst” component is counted, based on the stated rank order. The rank order prioritizing different outcome components addresses the limitation of conventional composite end points where components are weighted equally. This is especially beneficial when the different components are of varying severity or frequency. Table 2 summarizes key differences between HCEs and traditional composite or ordinal end points.

Table 2.

Comparison Between Composite, Ordinal, and Hierarchical Composite End Points

Feature	Composite End Point	Ordinal End Point	Hierarchical Composite End Point
Development and selection of end point	Some degree of subjectivity in terms of component selection; few widely validated composite end points in infectious diseases	Less subjectivity in terms of end point construction, unless there is variation within how different levels in the ordinal scale are defined; few widely validated ordinal end points in infectious diseases	Greatest degree of subjectivity present, since choices have to be made both in terms of component selection as well as order of prioritization; several consensus DOOR outcomes developed for different infectious syndromes; more flexible due to ability to combine multiple end point types (eg, binary, time-to-event, and continuous end points)
Clinical relevance	May be less clinically meaningful, especially if outcome components are not of equal clinical importance; larger changes in less clinically important components may obscure smaller changes in more clinically important components	More clinically meaningful as different levels may reflect different gradations of severity encountered in clinical practice	Reflects real-world clinical decision making by combining competing risks and trade-offs into a single end point, permitting a more comprehensive risk-benefit assessment and providing a “global” assessment of the patient's condition
Statistical efficiency (power)	More efficient than conventional binary end points as the event rate is increased but less efficient than ordinal end points	More efficient than composite end points by capturing a spread of event outcomes, thus capturing more information	May be more or less efficient than composite or ordinal outcomes, depending on the selection of components, relative ranking, and whether treatments exert effects in different directions among different components
Sample size determination	Straightforward, easily conducted using simple online calculators with minimal data inputs (expected event rate, hypothesized effect size, α level, and power)	More complicated than for composite end points but still relatively straightforward with available calculators and minimal data inputs (expected distribution across different levels, hypothesized effect size, α level, and power)	Dependent on the method of analysis and target parameter; most complicated, especially when considering mix of different outcome types; may require extensive simulations
Commonly used analytic methods	Calculation of risk difference, risk ratio, or OR (for binary end point); logistic regression model to estimate OR (for binary end point); log-rank test with Kaplan-Meier curve (for time-to-event end point); Cox proportional hazards model to estimate hazard ratio (for time-to-event end point)	Dichotomization at cutoff point and analysis as binary end point (considered inappropriate and statistically inefficient); Mann–Whitney U tests to compare distributions; PO model to estimate common OR	Mann–Whitney U tests to compare distributions and calculate PI; generalized pairwise comparisons to estimate win statistics; PO model to estimate common OR; PI model to estimate PI
Interpretability of target parameters	Target parameters are commonly used and familiar to clinicians: risk differences, risk ratios, or ORs (for binary composite end points) or hazard ratios (for time-to-event composite end points)	Target parameter is often the OR from a PO model, which may be less interpretable to clinicians who are less familiar with this than with more common target parameters	Target parameters (PI, win ratio, win odds) are less familiar to clinical audiences

Abbreviations: DOOR, desirability of outcome ranking; OR, odds ratio; PI, probabilistic index; PO, proportional odds.

An example of an HCE is the DOOR outcome, which is increasingly used in ID clinical research [2, 3]. Multiple consensus DOOR outcomes have been developed for a variety of infectious disease syndromes [16–19]. For example, a consensus DOOR outcome for Staphylococcus aureus bacteremia combines death, treatment failure, infectious complications, and drug adverse events, and it has been used in the DOTS trial evaluating dalbavancin [20, 21].

DOOR is an example of an HCE in which, as in conventional composite outcomes, each component is binary (ie, yes/no for each constituent component). However, HCEs can also have greater flexibility by combining different types of component outcomes (eg, time to event, continuous, binary, or ordinal). Finkelstein and Schoenfeld [22] demonstrated this in post hoc analyses of human immunodeficiency virus clinical trials where they combined 3 component outcomes in an HCE: (1) death (time-to-event outcome), (2) opportunistic infection (time-to-event outcome), and (3) quality of life (ordinal outcome), ranked in order from most to least important.

An extension of DOOR incorporating antibiotic duration, DOOR/RADAR (response adjusted for duration of antibiotic risk), where receipt of shorter antibiotic duration (a continuous outcome) is considered superior to longer duration [3 , 23], can also be conceptualized as belonging to this HCE framework—patients are first evaluated by the DOOR ordinal scale, followed by antibiotic duration as the second rung in the HCE. DOOR/RADAR has been proposed as an alternative to the more traditional noninferiority design to study antibiotic durations, sidestepping some of the pitfalls of noninferiority trials, such as bias in the setting of treatment nonadherence or difficulties with establishing a noninferiority margin [3]. The use of other “tiebreakers” together with DOOR, such as quality of life or functional outcome, are other examples of HCEs [21, 24, 25].

Analysis of HCEs

Statistical analytic methods can be categorized into parametric or nonparametric approaches. Nonparametric approaches are based only on counting processes of the raw data, and a statistical model (with model parameters, hence the term parametric) is not created to summarize the data and estimate coefficients that correspond to the treatment effect. With nonparametric approaches, covariate adjustment is limited, so if one desires an adjusted estimate, a parametric model is usually required. For those new to these concepts, an analogous comparison is between nonparametric versus parametric approaches for a binary outcome (eg, death)—the nonparametric approach is a χ² test to test the association between 2 variables, while the parametric approach is a logistic regression model where one can adjust for other variables (eg, age or sex) to derive an adjusted odds ratio (OR). In clinical trials, covariate adjustment is recommended when a variable is used to stratify randomization; adjusting for covariates that are predictive of the outcome can also improve power by providing more precise estimates (with narrower confidence intervals [CIs]) [26].

Nonparametric Approaches

Studies that use DOORs can be analyzed using a nonparametric approach, the Mann–Whitney U test (also known as the Wilcoxon rank sum test). The Mann–Whitney U test is commonly used in biomedical research to compare medians of nonnormally distributed continuous outcomes, by comparing the relative distributions of the range of values in 2 groups. The way this is used to analyze DOORs is similar: the distributions of ranks are compared across 2 groups, with the null hypothesis being that the distributions of both groups are the same. The primary effect estimated in this application is the probability that any randomly selected patient in one group has a superior outcome to a randomly selected patient in the other group—mathematically written as $P(A>B)$, where P represents probability; A, the outcome of a patient in group A; and B, the outcome of a patient in group B.

This effect estimate has been variably termed the better DOOR probability, probabilistic index, or concordance probability c [2 , 27, 28]. A probability of 50% represents no difference between groups, and a result where the entire CI is >50% represents superiority. For example, the SCOUT-CAP trial, which compared 5 versus 10 days of antibiotics for community-acquired pneumonia in young children, used an HCE that ranked the following in order of priority: clinical response, resolution of symptoms, antibiotic-associated adverse effects, and duration of antibiotics received. The 5-day group had a 69% probability (95% CI: 63%–75%) of a more desirable outcome (DOOR-RADAR) compared with the 10-day group, indicating superiority of the short-course treatment (since the lower bound of the CI exceeded 50%) [23]. The first 3 outcome components were similar between treatment groups, but inclusion of the antibiotic treatment duration as an additional tiebreaker resulted in the shorter treatment group being superior. This assumes that shorter durations are preferable when other clinical outcomes are similar.

More generally, HCEs can be analyzed using the framework of generalized pairwise comparisons [29]. In this method, all possible patient pair combinations are made between the 2 treatment groups, and each pair is evaluated based on the hierarchy of outcomes to determine which treatment group is favored within each pair (Figure 2). The hierarchical ordering is such that the most important outcome is considered first, and if that is tied (eg, both individuals in the pair did not have the outcome), then the second most important outcome is considered, and so forth. This results in 3 summary numbers: total number of wins, losses, and ties. Different treatment effect estimates (win ratio, win odds, net treatment benefit, number needed to treat, and probabilistic index) can be obtained from these 3 summary numbers using different formulas (Supplementary Appendix) [30–32].

Figure 2.

Example of a generalized pairwise comparisons analysis in a hypothetical clinical trial. In this hypothetical example clinical trial, 20 patients are randomized to 2 treatment groups—groups A and B. The simplified hierarchical composite end point depicted here consists of 2 outcomes in order of priority: (1) death and (2) complication. Each patient in one group is compared with each patient in the other (total 10 × 10 = 100 comparisons) and determined to be a win, loss, or tie depending on the outcomes. The total number of wins, losses, and ties are added up, which permits calculation of the win ratio, win odds, and net treatment benefit. Abbreviation: SOC, standard of care.

Using a simple binary outcome, the win ratio is the simple inverse of an OR and can be interpreted in a similar way (Supplementary Figure 1). With the inclusion of multiple outcomes in an HCE, the interpretation extends to being the ratio of having an overall superior outcome (a “win”) in one group versus the other group. The MERINO trial compared piperacillin-tazobactam versus meropenem for patients with ceftriaxone-resistant Escherichia coli or Klebsiella pneumoniae bloodstream infection, and it demonstrated a higher 30-day mortality rate in the piperacillin-tazobactam arm, failing to demonstrate noninferiority of piperacillin-tazobactam [33]. However, there were concerns that much of this mortality rate was due to underlying comorbid conditions rather than being infection related.

To address this limitation, Hardy and colleagues [4] constructed a post hoc HCE also including microbiologic relapse and secondary infection and then applied the generalized pairwise comparisons method to estimate the win ratio and win odds of piperacillin-tazobactam versus meropenem, thus considering clinically important outcomes that may be more directly relevant to the underlying infection The win ratio was 0.40 (95% CI: .22–.71) and win odds was 0.79 (.68–.92); that is, there were fewer wins in the piperacillin-tazobactam group, whether or not ties were considered. The upper bound of the CIs of both these estimates were <1.0, indicating the inferiority of piperacillin-tazobactam compared with meropenem, aligning with the conclusion of the primary analysis. The win ratio here can also be interpreted as excluding patients with equivalent outcomes, patients in the piperacillin-tazobactam group having a 60% lower chance of a more favorable outcome compared with those in the meropenem group. The win odds is interpreted as the odds of a patient in the piperacillin-tazobactam group having a more favorable outcome being 21% lower than for a patient in the meropenem group.

Parametric Approaches

Analogous parametric analytic approaches are also available for HCEs. Ordinal outcomes are commonly analyzed using a proportional odds (PO) model (also known as the cumulative logistic model assuming PO) [1]. Similar to how the logistic regression model can be conceptualized as a multivariable or model-based extension of the χ² test, the PO model can be conceptualized as a multivariable extension of the Mann–Whitney U test [11, 28, 34]. The target parameter estimated using a PO model is a common (or proportional) OR. This common OR can be interpreted as the odds a patient in the treatment group has of going up ≥1 step on the ordinal scale (ie, having a more favorable outcome) compared with a patient in the control group.

As an example, the REMAP-CAP platform trial evaluated different treatment strategies for COVID-19 pneumonia and used an ordinal scale combining survival and organ support–free days, analyzed using a PO model. In the domain evaluating tocilizumab, the common OR for a favorable outcome was 1.64 (95% CI, 1.25–2.14), indicating the superiority of tocilizumab versus control, since the entirety of the CI was >1.0 [35]. This can be interpreted as tocilizumab-treated patients having higher odds of improved survival and/or more organ support–free days.

The PO model relies on the PO assumption that the treatment exerts the same effect across the distribution of the ordinal scale. This implies that the ORs for each binary split (ie, above and below each level) of the ordinal scale are the same, which may not be a realistic assumption in practice. There is debate as to the importance of this assumption and optimal solutions when it is violated [36–38], which is beyond the scope of this review. As long as the direction of effect is consistent across all levels of the ordinal scale (ie, there is a positive effect throughout the scale, even if the actual value of the OR at each binary split is different), the model is still interpretable—in this case, the common OR would be interpreted as a summary OR representing the odds of having a superior outcome in one group compared with the other (Figure 3A). In contrast, if there is a discordant effect at different ends of the scale (eg, the treatment exerts a beneficial effect on one end but a detrimental effect on the other end of the scale), the common OR is no longer clinically interpretable because it may contradict the treatment effect at different levels (Figure 3B).

Figure 3.

Illustration of the proportional odds model (PO), in settings with a consistent effect across the ordinal scale (A) and a discordant effect at different ends of the ordinal scale (B). Abbreviation: OR, odds ratio.

The PO model can also be used for HCEs—with a hierarchy of different outcome components, the number of “steps” on the ordinal scale increases: for example, if we had a 2-step hierarchy where the first step is a 4-point ordinal scale, and the second step is antibiotic duration (ranging from 1 to 7 days), there would be a total of 4 × 7 = 28 possible ranks or steps on the hierarchical composite scale. It is less likely for the OR to be the same at each of these steps, but as long as the effect is consistent in direction across the entire combined scale, the common OR is still a clinically valid interpretation.

Alternative modeling approaches have also been proposed for HCEs. Mao and Wang [39] developed a semiparametric model, the proportional win fractions model, that permits multivariate adjustment of generalized pairwise comparison analyses, providing an adjusted win ratio. Thas and colleagues [40] developed the probabilistic index model, which allows modeling of a variety of combination of outcomes (continuous, ordinal, binary, and time-to-event outcomes), providing a summary adjusted probabilistic index [41]. This has also been applied to calculate adjusted win odds for HCEs using the generalized pairwise comparisons framework [42].

These alternative modeling approaches have not been commonly used in the ID literature. A previous review identified 12 RCTs using DOOR analyses as primary, secondary, or post hoc analyses, but all analyzed only DOOR outcomes using the nonparametric Mann–Whitney U approach [2]. This may be in part due to the confusing nomenclature for these concepts, with different terms being used despite their being related.

Clarification of Terminology and the Relationship Between Different Methods

One reason behind this confusion is the lack of clarity behind how to distinguish outcomes, target parameters, and analytic methods. An outcome is the metric used to measure the effect of treatment on the patient (at the individual level), the target parameter is the summary measure that quantifies the treatment effect of clinical interest, and the analytic method is the statistical method used to estimate the target parameter by comparing outcomes in one group versus those in the other group. Returning to an example of a simple binary outcome, the outcome is the 90-day mortality status, the target parameter could be the OR, and the analytic method could be a logistic regression model.

A DOOR is strictly an outcome, not an analytic method or target parameter. In contrast, a win ratio is a target parameter and not an outcome or analytic method. Thus, “DOOR analysis” or “win ratio analysis” are technically inaccurate and misleading terms. Confusingly, the relationships among outcome, target parameter, and analytic method are not one-to-one relationships; most outcomes can be summarized using multiple target parameters, each of which can analyzed in different ways (nonparametric and parametric) (Figure 4).

Figure 4.

Relationships among choice of end points, target parameters, and analytic methods. Arrows between end points and target parameters represent the possible target parameters that can be calculated or estimated from each end point, and arrows between analytic methods and target parameters represent the output target parameters from each analytic method. Arrows between target parameters show that the different parameters can be estimated or approximated from each other.

The different target parameters are related and can be calculated (or estimated) from each other (see the Supplementary Appendix). The different analytic methods described in this review will often give similar results if the constituent components of the HCE are the same. Ultimately, the target parameter of interest should be discussed and chosen in the planning stage of a trial—what makes sense for the given context, how it is going to be interpreted to represent better or worse outcomes for patients, and how it is going to be used to aid decision making in patient care. We encourage investigators to use clear and consistent terminology and follow research best practices, including prespecifying primary and secondary outcomes along with a statistical analysis plan that outlines the estimand and target parameter of interest [43]. This could also include details of sensitivity analyses or a “backup” plan when there are significant departures from key model assumptions (eg, PO).

DISCUSSION

In this review, we have explained what an HCE is, outlined common methods used to analyze HCEs, and clarified use of terminology present in the literature. The use of these end points is gaining popularity, and we hope that this review can help clinical readers become familiar with how to appraise research studies using these end points.

From the researcher's perspective, when to use these end points is beyond the scope of this review and is a question that requires further study. One clinical rationale supporting the use of HCEs is that they can capture a range of clinical outcomes that may be important to patients [20, 44, 45]. This can allow a more nuanced assessment of benefits versus harms of any intervention, as well as competing risks and trade-offs, as opposed to making a conclusion based on a singular clinical component (eg, mortality). This may be especially relevant in ID trials, particularly those evaluating antimicrobial use, where there are inherent competing tensions: as clinicians we want to treat patients to optimize their outcomes, but as antimicrobial stewards we also want to minimize collateral harms or the development of antimicrobial resistance.

We have also not covered how to construct these end points. Consensus DOORs have been developed for a few infectious syndromes, but for most clinical questions new HCEs will have to be constructed. This process has a degree of subjectivity, and results may be sensitive to the choice of components and how the hierarchy is defined. Ideally, development of end points should involve a consultative process with multiple stakeholders, a consensus-generating mechanism, and validation across different settings [2, 45, 46].

From a statistical perspective, a stated advantage of HCEs is that they may be more statistically efficient than simple binary outcomes (ie, may have more power and hence require a smaller sample) [45]. However, this is not necessarily true, especially if treatments exert effects in different directions among different components of the hierarchy (with a resultant net effect near zero). More methodologic work is required to compare HCEs with more conventional end points, to identify the optimal clinical questions and trial settings where they should be used. Methods for determining sample size when the target parameter is a common OR are well described [47, 48], but determining sample sizes with other target parameters or analytic methods can be more complex, often requiring extensive simulation work [49, 50].

Further research should be conducted to extend the applications of the different analytic methods described above to trial designs beyond simple 2-arm parallel group trials, such as cluster-randomized trials, multiarm trials, or adaptive platform trials using a bayesian analytic framework. Beyond this methodologic work, research should also be conducted as to the acceptability of these outcomes and the interpretability of their target parameters, which may be less familiar than conventional effect measures such as ORs or risk ratios. This research should involve all stakeholders in the clinical research ecosystem, including patients, clinicians, guideline writers, and regulators, to ensure adequate knowledge translation of results arising from trials using these outcomes.

Supplementary Data

Supplementary materials are available at Clinical Infectious Diseases online. Consisting of data provided by the authors to benefit the reader, the posted materials are not copyedited and are the sole responsibility of the authors, so questions or comments should be addressed to the corresponding author.

Notes

Acknowledgments. We thank Stephanie Sutjipto, MBBS and Pei Hua Lee, MBBS, who assisted with vetting the manuscript.

Financial support. This work was supported by funding to S. W. X. O., who conducted this research as part of his PhD studies, from the Melbourne Research Scholarship (University of Melbourne); the Emerging & Pandemic Infections Consortium and Connaught International Scholarship (University of Toronto); and the Queen Elizabeth II Graduate Scholarship in Science and Technology (Government of Ontario, Canada).