Abstract
Background
Clinical trials of treatments for serious infections commonly use the primary endpoint of all-cause mortality. However, many trial participants survive their infection and this endpoint may not truly reflect important benefits and risks of therapy. The win ratio uses a hierarchical composite endpoint that can incorporate and prioritize outcome measures by relative clinical importance.
Methods
The win ratio methodology was applied post hoc to outcomes observed in the MERINO trial, which compared piperacillin-tazobactam with meropenem. We quantified the win ratio with a primary hierarchical composite endpoint, including all-cause mortality, microbiological relapse, and secondary infection. A win ratio of 1 would correspond to no difference between the 2 antibiotics, while a ratio <1 favors meropenem. Further analyses were performed to calculate the win odds and to introduce a continuous outcome variable in order to reduce ties.
Results
With the hierarchy of all-cause mortality, microbiological relapse, and secondary infection, the win ratio estimate was 0.40 (95% confidence interval [CI], .22–.71]; P = .002), favoring meropenem over piperacillin-tazobactam. However, 73.4% of the pairs were tied due to the small proportion of events. The win odds, a modification of the win ratio accounting for ties, was 0.79 (95% CI, .68–.92). The addition of length of stay to the primary composite greatly minimized the number of ties (4.6%) with a win ratio estimate of 0.77 (95% CI, .60–.99; P = .04).
Conclusions
The application of the win ratio methodology to the MERINO trial data illustrates its utility and feasibility for use in antimicrobial trials.
Keywords: win ratio, hierarchical composite outcome, randomized controlled trial, antimicrobial resistance, bloodstream infections
The win ratio is a method for assessing trial outcome using a hierarchical composite endpoint. Using its application post hoc to the MERINO trial provided further detail on clinically important outcomes, demonstrating its potential utility in infectious diseases trials.
Randomized controlled trials (RCTs) of antibiotic strategies for multidrug-resistant organisms (MDROs) are difficult because of slow patient accrual, few standard-of-care options, exclusion of patients with underlying conditions, and a lack of well-accepted endpoints. Two recent Delphi processes in bloodstream infections (BSIs) and hospital-acquired pneumonia proposed the use of composite endpoints in future trials [1, 2]. Composite endpoints can capture multiple clinically significant events, providing a comprehensive evaluation of the treatment effect, and increase statistical efficiency by increasing the overall event rate. However, traditionally composite endpoints regard all component outcomes as having equivalent weighting and clinical importance. For example, 1 recent trial had a primary composite endpoint including all-cause mortality (ACM) at 90 days and persistent bacteremia at day 5, yet these are unlikely to be viewed as having equal clinical importance [3].
Rank-based or time-dependent composite endpoints were recently recommended in a white paper as a novel approach that could provide more meaningful data in RCTs for infections with MDROs [4]. Several rank-based methods have been proposed to address the clinical priority of individual components. The desirability of outcome ranking (DOOR) has been increasingly used in infectious disease trials [5–9]. In recent years, another rank-based method, the win ratio, has been adopted as a primary composite endpoint in cardiovascular trials and in their post hoc analysis [10–14]. The win ratio can incorporate any clinically relevant outcome (eg, survival, clinical and microbiologic success, and selection for resistant organisms) into an endpoint, prioritizes components by clinical importance, and can include binary, time to occurrence or recurrent events. The application of the win ratio approach in antimicrobial trials is yet to be fully explored.
The “Meropenem versus piperacillin-tazobactam for definitive treatment of BSIs due to ceftriaxone nonsusceptible Escherichia coli and Klebsiella spp” (MERINO) trial used ACM at 30 days as the primary outcome, with a noninferiority margin of 5% [15]. The trial was ceased early due to higher 30-day mortality in patients randomized to piperacillin-tazobactam compared to meropenem and failed to demonstrate noninferiority. However, there were concerns that ACM may not best reflect infection-related outcomes in a diverse patient population that has risks of dying from other underlying diseases.
The objectives of this study were to demonstrate the use, and potential added value, of the win ratio method in antimicrobial trials for MDROs using a post hoc hierarchical composite endpoint analysis of the MERINO trial as an example. We analyzed a selection of clinically relevant outcomes and demonstrate how the win ratio performs in comparison with a single binary mortality outcome. In addition, we evaluated a stratified win ratio analysis with subgroup analyses and compared win ratio estimates with a conventional risk ratio analysis. From these analyses, we provide insights and illustrate how the win ratio approach can be adopted in future antimicrobial trials for MDRO infections.
METHODS
Win Ratio Approach
The win ratio approach can incorporate benefits and risks into a hierarchy of outcomes based on clinical importance to assess superiority between treatment groups [10, 14]. Using the unmatched approach, all possible pairs are formed with every patient from the investigational treatment and control treatment groups and compared with respect to the most important outcome. A patient from the treatment group is classified as a “win” or “loss” dependent on who has the event of interest. If an event does not occur, or an event occurs for each of the pair at the same time or end of follow-up, the pair is considered a tie and the comparison moves to the next outcome in the hierarchy until all components are exhausted. The win ratio is the proportion of the total number of pairs where the investigational treatment arm was a win to the total number of losses. A value of 1 corresponds to no difference between the 2 groups, while a value >1 favors the investigational treatment and a ratio <1 favors the control treatment. An illustration of an example of the win ratio approach is provided in Figure 1. To account for remaining ties, the “win odds,” a modification of the win ratio that includes half of all ties added to the treatment and control groups, was also calculated [16].
Figure 1.
The win ratio approach: Classifying pairs as wins, losses, and ties, in this example prioritizing time to all-cause mortality before relapse and secondary infection in the hierarchical composite endpoint. Every patient in the investigational treatment group (A) is compared with every patient in the control group (B). If a patient in group B dies before a patient in group A, it is a win to group A. Or if a patient in group A dies before the patient in group B, it is a loss to group A. If none of the patients in the pair die, or if both patients die at the same time, it becomes a tie and the comparison moves to the next outcome in the hierarchy. This comparison continues with each outcome until all outcomes are exhausted. The win ratio is the number of wins for group A divided by the number of losses for group A.
Application of the Win Ratio Approach
We analyzed the MERINO trial data using the unmatched win ratio approach post hoc [10]. The primary hierarchical composite endpoint included (1) 30-day ACM, (2) microbiological relapse, or (3) secondary infection (Table 1). This composite prioritized mortality as the worst possible outcome, followed by a microbiologic outcome (relapse), and finally secondary infections (due to concerns of ecological side effects from carbapenems vs carbapenem-sparing alternatives). This analysis was repeated with the addition of a continuous endpoint, length of stay, as a fourth component. We acknowledge that endpoints should be determined a priori. Further exploratory analyses were conducted with substitution of 30-day ACM with 14-day ACM or time to death, and the addition of clinical and microbiological failure at day 4. These components were selected to examine if the inclusion of earlier measures of failure would impact on the estimates.
Table 1.
Primary Hierarchical Composite Endpoint for the Win Ratio Approach With the MERINO Trial
| Priority | Outcome |
|---|---|
| 1 (most important) | All-cause mortality at 30 d |
| 2 | Microbiological relapse |
| 3 (least important) | Secondary infection with multidrug-resistant organisms or Clostridioides difficile |
Abbreviation: MERINO, Meropenem versus piperacillin-tazobactam for definitive treatment of BSIs due to ceftriaxone nonsusceptible Escherichia coli and Klebsiella spp.
Comparison With Risk Ratio
Conventionally, binary outcomes are compared between groups using a risk difference, risk ratio, or odds ratio where risk is the probability of a bad outcome. With the win ratio, the benefit of a treatment (ie, where the treatment has more wins than losses) is expressed with a ratio >1. To compare the win ratio with the risk ratio analysis of a nonranked composite endpoint, we also reported the inverse of the risk ratio (1/RR) and the respective 95% confidence interval (CI). Similar to the win ratio, a 1/RR >1 will indicate the benefit of the investigational treatment.
Stratified Win Ratio
A stratified win ratio approach was also applied to the primary composite endpoint according to infection acquisition being healthcare-associated/hospital-acquired (HAI) (vs community-acquired) and urinary tract infection (UTI) (vs other sources) [17, 18] (Table 1). With the stratified win ratio, participants were divided into strata and the pairwise comparison was conducted in each stratum. The stratum-specific win ratios were then combined across the strata and weighted according to numbers in each stratum to estimate an overall win ratio for both strata.
Statistical Analyses
All analyses were performed using the primary analysis population, defined as any correctly randomized patient receiving at least 1 dose of allocated study drug. The proportions of wins, losses, and ties for patients assigned to piperacillin-tazobactam were obtained with the hierarchies described. Calculation of the win ratio was according to the method of Pocock et al [10], and the corresponding variances were based on the U-statistic method proposed by Bebu and Lachin [19] using the “WinRatio” R package [20]. To account for remaining ties, the “win odds” was calculated based on the method of Dong et al [16] using the “WINS” R package [21]. Multiple imputation was applied for missing data (as was performed in the original MERINO trial analysis) with multivariate imputation by chained equations using the “mice” package in R [22]. The win ratio was calculated using 5 imputed data sets. The logs of the win ratio estimates and variances from each imputed dataset were pooled using Rubin's rules to give a single pooled estimate and variance for each composite across all imputations [23]. All analyses were conducted in R version 4.2.2 (R Foundation for Statistical Computing, Vienna, Austria) or Stata version 15.1 (StataCorp, College Station, Texas). The R code is publicly available (https://github.com/MelissaHardy/Winratio.git).
Ethical Approval
The MERINO trial was approved by ethics committees at each site and written informed consent was obtained from all patients. This post hoc analysis was granted review exemption by The University of Queensland Human Ethics Committee (2022/HE002470).
RESULTS
In the MERINO study, 378 participants were evaluated in the primary analysis population for the primary outcome of 30-day ACM. Of these participants, 14.3% (54/378) experienced at least 1 event in the primary composite: 7.9% (30/378) died, 3.4% (13/378) relapsed, and 6.1% (23/378) developed a secondary infection (Supplementary Figure 2).
Win Ratio Analyses
Using the unmatched win ratio approach, each participant randomized to piperacillin-tazobactam (n = 187) was paired with each participant randomized to meropenem (n = 191), resulting in 35 717 paired comparisons (Figure 2). The analysis with the primary hierarchical composite endpoint of ACM at 30 days, microbiological relapse, and secondary infection demonstrated a win ratio of 0.40 (95% CI, .22–.71; P = .002). Thus, the result favors meropenem.
Figure 2.
The win ratio analysis of the MERINO trial using the unmatched approach. This figure illustrates the proportion of wins (blue), ties (gray), and losses (red) for each of the 35 717 piperacillin-tazobactam versus meropenem paired comparisons for the hierarchical composite endpoint. A “win” for 30-d all-cause mortality means that in 3.2% of the 35 717 pairs this favored piperacillin-tazobactam, whereas in 11.8% of pairs this favored meropenem (constituting a “loss” for piperacillin-tazobactam in this outcome measure) with 73.4% of ties remaining. The win odds calculation incorporates half the ties to the wins and losses. Abbreviations: ACM, all-cause mortality; CI, confidence interval; MERINO, Meropenem versus piperacillin-tazobactam for definitive treatment of BSIs due to ceftriaxone nonsusceptible Escherichia coli and Klebsiella spp.
Due to the small proportion of events for each outcome, 73.4% of the pairs remained tied. We accounted for these ties by first calculating the “win odds,” which was 0.79 (95% CI, .68–.92; P = .002). Second, we added a continuous outcome (length of stay) to the primary composite as a final component to reduce the remaining ties (Figure 3, row 2). Its addition minimized the number of ties to 4.6% and contributed the majority of the wins and losses to the estimate, in comparison to the other components of the composite. The resulting win ratio and “win odds” estimates were 0.77 (95% CI, .60–.99; P = .04) and 0.78 (95% CI, .62–.99; P = .04), respectively. All of these results favor meropenem.
Figure 3.
Primary analyses of the win ratio approach for the MERINO trial with different components in the hierarchical composite endpoint and the proportion of piperacillin-tazobactam wins, losses, and ties compared with meropenem. The win ratio and win odds and associated 95% confidence intervals and P values are shown. Abbreviations: CI, confidence interval; MER, meropenem; MERINO, Meropenem versus piperacillin-tazobactam for definitive treatment of BSIs due to ceftriaxone nonsusceptible Escherichia coli and Klebsiella spp; PTZ, piperacillin-tazobactam.
The win ratio estimates for the alternative composite endpoints where 14-day ACM or time to death replaced 30-day ACM were similar to the primary composite (Supplementary Figure 1, rows 1 and 2). When clinical and microbiological failure at day 4 was added to the primary composite as a second component (Supplementary Figure 1, row 3), it resulted in a higher win ratio estimate of 0.60 (95% CI, .41–.88; P = .008) with a smaller proportion of ties. The “win odds” estimates for all alternative composites were similar.
Comparison With Risk Ratio
The comparisons with the 1/RR with the primary composite endpoint of ACM, relapse, and secondary infection are shown in Table 2. The 1/RR for piperacillin-tazobactam versus meropenem was 0.45 (95% CI, .26–.77; P = .003) or a risk ratio of 2.22 (95% CI, 1.3–3.8; P = .003) and the win ratio and “win odds” was 0.40 (95% CI, .22–.77; P = .002) and 0.79 (95% CI, .68–.92; P = .002), respectively. A slightly greater difference in the treatment effect was observed with an alternative composite, which included clinical and microbiological failure, when the 1/RR was compared to the win ratio estimate (Table 2). However, for this composite endpoint the 1/RR was similar to the “win odds.”
Table 2.
Comparison of Risk Ratio Analyses (1/RR) and the Win Ratio With 2 Different Composite Endpoints
| Outcomes | Hierarchy 1 | ||||
|---|---|---|---|---|---|
| PTZ (n = 187) |
MER (n = 191) |
1/RR (95% CI) | P Value | ||
| ACM (30 d) | 23 (12.3) | 7 (3.7) | … | … | |
| Relapse | 9 (4.8) | 4 (1.9) | … | … | |
| Secondary infection | 15 (8.0) | 8 (4.2) | … | … | |
| Composite outcome | 37 (19.8) | 17 (8.9) | 0.45 (.26–.77)a | .003 | … |
| Wins/losses | Winsb | Lossesb | WR (95% CI) | WO (95% CI) | |
| 30-d ACM | 1148 (3.2) | 4232 (11.8) | … | … | |
| Relapse | 628 (1.8) | 1274 (3.6) | … | … | |
| Secondary infection | 920 (2.6) | 1286 (3.6) | … | … | |
| Composite outcome | 2696 (7.5) | 6793 (19.0) | 0.40 (.22–.77) | .002 | 0.79 (.68–.92) |
| Outcomes | Hierarchy 2 | ||||
|---|---|---|---|---|---|
| PTZ (n = 187) |
MER (n = 191) |
1/RR (95% CI) | P Value | ||
| ACM (30 d) | 23 (12.3) | 7 (3.7) | … | … | |
| Clinical/microbiological failure | 65 (34.2) | 50 (26.2) | … | … | |
| Relapse | 9 (4.8) | 4 (1.9) | … | … | |
| Secondary infection | 15 (8.0) | 8 (4.2) | … | … | |
| Composite outcome | 79 (42.2) | 62 (32.5) | 0.77 (.59–1.00)c | .05 | … |
| Wins/losses | Winsb | Lossesb | WR (95% CI) | WO (95% CI) | |
| 30-d ACM | 1148 (3.2) | 4232 (11.8) | … | … | |
| Clinical/microbiological failure | 5273 (14.8) | 6968 (19.5) | … | … | |
| Relapse | 628 (1.8) | 1274 (3.6) | … | … | |
| Secondary infection | 920 (2.6) | 1286 (3.6) | … | … | |
| Composite outcome | 20.9 (7458) | 34.7 (12 382) | 0.60 (.41–.88) | .008 | 0.76 (.62–.93) |
Data are presented as No. (%) unless otherwise indicated.
Abbreviations: ACM, all-cause mortality; CI, confidence interval; MER, meropenem; PTZ, piperacillin-tazobactam; 1/RR, inverse risk ratio; WO, win odds; WR, win ratio.
aRisk ratio, 2.22 (95% CI, 1.3–3.8); P = .003.
bUnmatched win ratio analysis from 35 717 paired comparisons.
cRisk ratio, 1.3 (95% CI, 1.0–1.7); P = .05.
Stratified Win Ratio Analyses
Using the primary composite endpoint, the overall combined stratified win ratio estimates accounting for acquisition and source of infection were similar to the unstratified estimates with 0.40 (95% CI, .21–.76; P = .005) and 0.43 (95% CI, .23–.80; P = .008), respectively (Figure 4). The overall combined stratified “win odds” for both groups were also similar to the unstratified estimates. However, the subgroup win ratio and “win odds” estimates for the HAI versus non-HAI and UTI versus non-UTI source infections were different (Figure 4). The win ratio estimates for the non-HAIs and UTI source had wider CIs due to small sample sizes and were not statistically significant compared to their alternative strata.
Figure 4.
The stratified win ratio analysis of the primary composite endpoint all-cause mortality at 30 d, relapse, and secondary infection calculated within subgroup strata into an overall win ratio and win odds estimate. The win ratio and win odds estimates and 95% confidence interval (CI) in each stratum and the overall stratified win ratio and 95% CI and P values are shown. Bold values denoted the overall estimates for the subgroups/strata combined. Abbreviations: CI, confidence interval; HAI, healthcare-associated infection; MER, meropenem; PTZ, piperacillin-tazobactam; UT, urinary tract.
DISCUSSION
In this post hoc analysis of the MERINO trial, the win ratio and “win odds” estimates with a hierarchical composite endpoint favored meropenem, supporting the findings of the original study, which used a single binary outcome (30-day ACM). The application of a hierarchical composite endpoint has expanded the understanding of the trial results by considering microbiological and ecological outcomes and can clarify concerns with relying on a single primary endpoint in a diverse patient population with other underlying diseases. Further exploratory analyses with composites including earlier measures of failure also had similar findings between the treatment groups. “Win odds,” which utilized all pairs of comparisons including ties, when compared to win ratio, showed that ignoring ties can lead to an overestimation of the treatment effect [16, 24]. It has been suggested that the win ratio should only be used when the amount of tied data is negligible, in which case the win ratio approximates the “win odds” [24, 25].
Our stratified analyses demonstrated that win ratio can be used to analyze subgroups to identify groups with a more favorable treatment outcome. This is appropriate in studies where baseline characteristics are considered in the randomization (ie, stratified randomization). While the higher “win odds” estimates in the non-HAI and UTI source subgroups could potentially suggest less of a difference in treatment effect within these subgroups, caution is warranted. Small sample sizes are inadequately powered to detect true differences and fail to control the type I error rate [26].
More recently, rank-based composite endpoints that combine patient-oriented and disease-related endpoints have been recommended for trials with infections caused by MDROs [4]. They allow for combining safety and efficacy and prioritize clinically important events. The ranked-based method, DOOR, has been increasingly utilized in antimicrobial trials. The Antimicrobial Resistance Leadership Group has largely proposed a 5-rank DOOR, depending on survival status, absence of clinical response, occurrence of infectious complications, and serious adverse events (SAEs) [8]. Death from any cause is the least desirable outcome and ranks worst, with survival with clinical response and without infectious complications or SAEs ranked best. Intermediate status is ranked according to the presence of 1, 2, or 3 “events.” The win ratio is being increasingly adopted as either a primary or secondary outcome in cardiovascular and coronavirus disease 2019 trials. It was accepted by regulatory authorities in the drug approval of tafamidis on the basis of the ATTR-ACT (Safety and Efficacy of Tafamidis in Patients With Transthyretin Cardiomyopathy; NCT01994889) trial and included in a number of phase 3 clinical trials (eg, NCT03860935, NCT04001504, NCT04847557, NCT04510493) [11]. To date, the use of the win ratio in antimicrobial trials includes a brief report on pneumonia in patients on mechanical ventilation; more recently, it was proposed as primary endpoint in a study of severe hospital-acquired pneumonia [27, 28].
There are similarities in the DOOR and the modified win ratio methodologies. They can both provide an overall patient assessment with the flexibility to include disease-related, drug-related, and patient-oriented outcomes that can occur throughout the course of the disease. However, their interpretation differs in that DOOR produces a probability while the win ratio and win odds are conditional odds and odds, respectively. A third win statistic, “net treatment benefit,” is the difference in the win proportions and is an absolute measure of treatment effect, and its inverse provides the number needed to treat. All 3 win statistics can be complementary and can be reported together in studies to show the strength of treatment effect [29]. DOOR can consider multiple events simultaneously, while win ratio considers one outcome at a time. The win ratio can allow the incorporation of different types of variables including continuous outcomes, timing of events, and repeated nonfatal events and can allow an analysis of an individual component's contribution to the overall treatment effect. The approach can be easily implemented with packages in R and Stata [20, 21, 30].
Selecting composite components and their hierarchy is subjective and requires careful consideration. The inclusion of a continuous endpoint such as length of stay or patient-reported outcomes can allow an outcome to be assigned to all participants and reduce the remaining ties. However, if a continuous endpoint is not measured precisely, its use could have unintended consequences—if higher-ranked outcome measures are rare, the continuous variable will to a large extent determine the outcome of the study as measured by the win ratio. There are also concerns that combining multiple clinical outcomes into a single composite may obscure important differences in a single component [6]. Treatment effect differences can be driven by less severe, nonfatal outcomes. For example, length of stay can decide winners across all pairs in contrast to rare events like mortality, and length of stay could largely influence the treatment estimate of the study. To overcome this, reporting of win ratio should provide a detailed analysis of the contribution of each component on the overall treatment effect (Figure 2). Additionally, caution is advised when any composite components have opposite treatment effects where wins from one outcome could be neutralized by losses in another, masking important conclusions from comparing single outcomes.
This study has limitations with the post hoc methodology, and the outcomes included in the hierarchical composite endpoints were limited by the initial trial design. Ideally, the ranking and selection of outcomes in the composite endpoint should be considered a priori and agreed on by an expert panel consensus. It is important that a composite endpoint is defined before unblinding and analyzing data. Furthermore, pretrial discussions regarding the viewpoint of patient representatives, the interpretation of the treatment outcomes, and the difference between treatments should be conducted. As a novel endpoint in this disease area, future work is needed to better understand how the win ratio and win statistics can be utilized prospectively in the design and analysis of antimicrobial trials.
In conclusion, our analysis illustrated the utility and flexibility of the win ratio approach using a hierarchical composite endpoint to an antimicrobial trial. The win ratio approach provides an alternative option for a rank-based composite endpoint to provide more meaningful data in the evaluation of treatments in RCTs for infections with MDROs.
Supplementary Data
Supplementary materials are available at Clinical Infectious Diseases online. Consisting of data provided by the authors to benefit the reader, the posted materials are not copyedited and are the sole responsibility of the authors, so questions or comments should be addressed to the corresponding author.
Supplementary Material
Contributor Information
Melissa Hardy, UQ Centre for Clinical Research, The University of Queensland, Brisbane, Queensland, Australia.
Patrick N A Harris, UQ Centre for Clinical Research, The University of Queensland, Brisbane, Queensland, Australia; Central Microbiology Laboratory, Pathology Queensland, Brisbane, Queensland, Australia.
David L Paterson, UQ Centre for Clinical Research, The University of Queensland, Brisbane, Queensland, Australia; ADVANCE-ID, Saw Swee Hock School of Public Health, National University of Singapore, Singapore; Infectious Diseases Translational Research Programme, Yong Loo Lin School of Medicine, National University of Singapore, Singapore.
Mark D Chatfield, UQ Centre for Clinical Research, The University of Queensland, Brisbane, Queensland, Australia.
Yin Mo, ADVANCE-ID, Saw Swee Hock School of Public Health, National University of Singapore, Singapore; Centre for Tropical Medicine and Global Health, Nuffield Department of Medicine, University of Oxford, United Kingdom; Mahidol-Oxford Tropical Medicine Research Unit, Faculty of Tropical Medicine, Mahidol University, Bangkok, Thailand; Division of Infectious Diseases, University Medicine Cluster, National University Hospital, Singapore; Yong Loo Lin School of Medicine, National University of Singapore, Singapore.
for the MERINO Trial Investigators:
Paul A Tambyah, David C Lye, Tau H Lee, Mesut Yilmaz, Thamer H Alenazi, Yaseen Arabi, Marco Falcone, Matteo Bassetti, Elda Righi, Benjamin A Rogers, Souha Kanj, Hasan Bhally, Jon Iredell, Marc Mendelson, Tom H Boyles, David Looke, Spiros Miyakis, Genevieve Walls, Mohammed Al Khamis, Ahmed Zikri, Amy Crowe, Paul Ingram, Nick Daneman, Paul Griffin, Eugene Athan, Penelope Lorenc, Peter Baker, Leah Roberts, Scott A Beatson, Tiffany Harris-Brown, and Anton Y Peleg
Notes
Author Contributions. D. L. P., Y. M., and M. H. designed the study. M. H. performed the data analysis and wrote the main version of the manuscript. P. N. A. H. and Y. M. supervised data analysis. Y. M. and M. D. C. supervised statistical analysis. D. L. P. and Y. M. supervised data interpretation and the manuscript drafting. All authors contributed to revising the manuscript and approved its final version.
Acknowledgments. The authors thank all study investigators, staff, and participants who participated in the MERINO trial. The MERINO trial investigators are Paul A. Tambyah, David C. Lye, Tau H. Lee, Mesut Yilmaz, Thamer H. Alenazi, Yaseen Arabi, Marco Falcone, Matteo Bassetti, Elda Righi, Benjamin A. Rogers, Souha Kanj, Hasan Bhally, Jon Iredell, Marc Mendelson, Tom H. Boyles, David Looke, Spiros Miyakis, Genevieve Walls, Mohammed Al Khamis, Ahmed Zikri, Amy Crowe, Paul Ingram, Nick Daneman, Paul Griffin, Eugene Athan, Penelope Lorenc, Peter Baker, Leah Roberts, Scott A. Beatson, Tiffany Harris-Brown, and Anton Y. Peleg. We also acknowledge Ms Tiffany Au for the assistance with the study submission to the Human Reasearch Ethics Commitee.
Financial support. M. H. is receiving a scholarship from the University of Queensland in support of her PhD candidacy.
References
- 1. Harris PNA, McNamara JF, Lye DC, et al. Proposed primary endpoints for use in clinical trials that compare treatment options for bloodstream infection in adults: a consensus definition. Clin Microbiol Infect 2017; 23:533–41. [DOI] [PubMed] [Google Scholar]
- 2. Weiss E, Zahar J-R, Alder J, et al. Elaboration of consensus clinical endpoints to evaluate antimicrobial treatment efficacy in future hospital-acquired/ventilator-associated bacterial pneumonia clinical trials. Clin Infect Dis 2019; 69:1912–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Tong SYC, Lye DC, Yahav D, et al. Effect of vancomycin or daptomycin with vs without an antistaphylococcal β-lactam on mortality, bacteremia, relapse, or treatment failure in patients with MRSA bacteremia: a randomized clinical trial. JAMA 2020; 323:527–37. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. de Kraker MEA, Sommer H, de Velde F, et al. Optimizing the design and analysis of clinical trials for antibacterials against multidrug-resistant organisms: a white paper from COMBACTE's STAT-Net. Clin Infect Dis 2018; 67:1922–31. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Evans SR, Rubin D, Follmann D, et al. Desirability of outcome ranking (DOOR) and response adjusted for duration of antibiotic risk (RADAR). Clin Infect Dis 2015; 61:800–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Ong SWX, Petersiel N, Loewenthal MR, Daneman N, Tong SYC, Davis JS. Unlocking the DOOR—how to design, apply, analyse, and interpret desirability of outcome ranking endpoints in infectious diseases clinical trials. Clin Microbiol Infect 2023; 29:1024–30. [DOI] [PubMed] [Google Scholar]
- 7. Howard-Anderson J, Dai W, Yahav D, et al. A desirability of outcome ranking analysis of a randomized clinical trial comparing seven versus fourteen days of antibiotics for uncomplicated gram-negative bloodstream infection. Open Forum Infect Dis 2022; 9:ofac140. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Howard-Anderson J, Hamasaki T, Dai W, et al. Improving traditional registrational trial end points: development and application of a desirability of outcome ranking end point for complicated urinary tract infection clinical trials. Clin Infect Dis 2022; 76:e1157–65. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Howard-Anderson J, Hamasaki T, Dai W, et al. Moving beyond mortality: development and application of a desirability of outcome ranking (DOOR) endpoint for hospital-acquired bacterial pneumonia and ventilator-associated bacterial pneumonia. Clin Infect Dis 2024; 78:259–68. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Pocock SJ, Ariti CA, Collier TJ, Wang D. The win ratio: a new approach to the analysis of composite endpoints in clinical trials based on clinical priorities. Eur Heart J 2012; 33:176–82. [DOI] [PubMed] [Google Scholar]
- 11. Maurer MS, Schwartz JH, Gundapaneni B, et al. Tafamidis treatment for patients with transthyretin amyloid cardiomyopathy. N Engl J Med 2018; 379:1007–16. [DOI] [PubMed] [Google Scholar]
- 12. Ferreira JP, Jhund PS, Duarte K, et al. Use of the win ratio in cardiovascular trials. JACC Heart Fail 2020; 8:441–50. [DOI] [PubMed] [Google Scholar]
- 13. Redfors B, Gregson J, Crowley A, et al. The win ratio approach for composite endpoints: practical guidance based on previous experience. Eur Heart J 2020; 41:4391–9. [DOI] [PubMed] [Google Scholar]
- 14. Pocock SJ, Ferreira JP, Collier TJ, et al. The win ratio method in heart failure trials: lessons learnt from EMPULSE. Eur J Heart Fail 2023; 25:632–41. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Harris PNA, Tambyah PA, Lye DC, et al. Effect of piperacillin-tazobactam vs meropenem on 30-day mortality for patients with E coli or Klebsiella pneumoniae bloodstream infection and ceftriaxone resistance: a randomized clinical trial. JAMA 2018; 320:984–94. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Dong G, Hoaglin DC, Qiu J, et al. The win ratio: on interpretation and handling of ties. Stat Biopharm Res 2020; 12:99–106. [Google Scholar]
- 17. Dong G, Qiu J, Wang D, Vandemeulebroecke M. The stratified win ratio. J Biopharm Stat 2018; 28:778–96. [DOI] [PubMed] [Google Scholar]
- 18. Dong G, Li D, Ballerstedt S, Vandemeulebroecke M. A generalized analytic solution to the win ratio to analyze a composite endpoint considering the clinical importance order among components. Pharm Stat 2016; 15:430–7. [DOI] [PubMed] [Google Scholar]
- 19. Bebu I, Lachin JM. Large sample inference for a win ratio analysis of a composite outcome based on prioritized components. Biostatistics 2016; 17:178–87. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Duarte K, Ferreira JP. WinRatio: win ratio for prioritized outcomes and 95% confidence interval. R package version 1.0. 2020. Available at: https://CRAN.R-project.org/package=WinRatio. Accessed 25 October 2022.
- 21. Cui Y, Huang BWINS. The R WINS package. 2022. Available at: https://cran.r-project.org/web/packages/WINS/WINS.pdf. Accessed 3 January 2024.
- 22. van Buuren S, Groothuis-Oudshoorn K. Mice: multivariate imputation by chained equations in R. J Stat Softw 2011; 45:1–67. [Google Scholar]
- 23. Heymans M, Eekhout I. Applied missing data analysis with SPSS and (R) Studio. Heymans and Eekhout: Amsterdam, The Netherlands, 2019. Available at: https://www.iriseekhout.com/publication/heymans-2019-applied/. Accessed 23 September 2023.
- 24. Brunner E, Vandemeulebroecke M, Mütze T. Win odds: an adaptation of the win ratio to include ties. Stat Med 2021; 40:3367–84. [DOI] [PubMed] [Google Scholar]
- 25. Wang D, Pocock S. A win ratio approach to comparing continuous non-normal outcomes in clinical trials. Pharm Stat 2016; 15:238–45. [DOI] [PubMed] [Google Scholar]
- 26. Verbeeck J, Ozenne B, Anderson WN. Evaluation of inferential methods for the net benefit and win ratio statistics. J Biopharm Stat 2020; 30:765–82. [DOI] [PubMed] [Google Scholar]
- 27. Albin OR, Pogue JM, Wunderink RG, Kaye KS. Which trial do we need? Combination antimicrobial therapy for hospital-acquired bacterial pneumonia caused by Pseudomonas aeruginosa. Clin Microbiol Infect 2023; 30:162–4. [DOI] [PubMed] [Google Scholar]
- 28. Montgomery A, Abuan T, Kollef M. The win ratio method: a novel hierarchical endpoint for pneumonia trials in patients on mechanical ventilation. Crit Care 2014; 18:P260. [Google Scholar]
- 29. Dong G, Huang B, Verbeeck J, et al. Win statistics (win ratio, win odds, and net benefit) can complement one another to show the strength of the treatment effect on time-to-event outcomes. Pharm Stat 2023; 22:20–33. [DOI] [PubMed] [Google Scholar]
- 30. Gregson J, Ferreira JP, Collier T. Winratiotest: a command for implementing the win ratio and stratified win ratio in Stata. Stata J 2023; 23:835–50. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.