Abstract
Objectives:
Neonatal trials have traditionally used binary composite short-term (such as death or bronchopulmonary dysplasia) or longer-term (such as death or severe neurodevelopmental impairment) outcomes. We applied the Desirability Of Outcome Ranking (DOOR) method to rank the overall patient outcome by best (no morbidities) to worst (death).
Study Design:
Using a completed large multicenter trial (Milking In Non-Vigorous Infants [MINVI]) of umbilical cord milking (UCM) vs. early cord clamping (ECC), we applied the DOOR methodology to neonatal outcomes. Six outcomes were chosen and ranked: no interventions or NICU admission (most desirable); received initial cardiorespiratory support at birth; neonatal intensive care unit (NICU) admission for predefined criteria; mild hypoxic-ischemic encephalopathy (HIE); moderate to severe HIE; and death (least desirable).
Results:
1524 non-vigorous newborns born between 35–42 weeks’ gestation had data for analysis. The DOOR distribution was different between the UCM and ECC arms, with a significantly greater probability (55.8% [95% CI 53.1–58.5%; p<0.0001]) of a randomly selected neonate having a more desirable outcome if they were in the UCM arm. DOOR probabilities of averting individual adverse outcomes such as NICU admission for predefined criteria (52.8%; 95%CI 50.5–55.1%) and cardiorespiratory support (54.0%; 95%CI 51.6–56.4%) were significantly higher among those in the UCM group.
Conclusion:
DOOR provides an overall assessment of the benefits and harms with greater insight than typical binary composite measures to clinicians and parents when evaluating an intervention. Future neonatal trials should consider the a priori use of the DOOR methodology to evaluate trial outcomes.
Keywords: Deferred Consent, Cluster Trials, Cord Management
Introduction
Neonatal trials typically evaluate interventions aimed to reduce traditional composite outcomes such as the binary combination of brain injury or death. Binary composites will also not recognize important differences within failure or success categories. Brain injury or death are importantly different but counted the same during analyses Such primary outcomes also do not provide an overall balanced assessment of risks and benefits. For example, interventions can result in both benefit and harm. Importantly, outcomes are not all similar in the degree of impact upon health or in how they are perceived by parents. Preterm newborns, for example, can experience multiple morbidities ranging from the need for treatment for a patent ductus arteriosus to necrotizing enterocolitis. Term and near-term infants may need cardiopulmonary resuscitation, require ICU admission, or develop hypoxic-ischemic encephalopathy. Despite the variability in these outcomes, researchers often choose one morbidity or a composite of morbidities as a trial’s primary outcome, even though these choices suffer from the above limitations.
An alternative approach is to evaluate multiple outcomes concurrently to analyze patient experiences as a whole rather than as individual, separate components. The Desirability of Outcome Ranking (DOOR), is an innovative statistical approach that has been used in several adult trials1–5 to address the limitations of using traditional outcomes. The DOOR approach does not rely on a single binary outcome but instead compares different strategies/treatments using ordinal endpoints according to the desirability of the overall patient-centric outcome (e.g., from the most to the least desirable: (i) patient survival without minor or major adverse events, (ii) survival with major adverse events, to (iii) death) enabling recognition of finer gradations of effects. Because the DOOR approach simultaneously considers benefits and harms, researchers can define the global superiority of an intervention in a single outcome.
However, before the DOOR approach can be used in future neonatal studies and trials, there is value in demonstrating how it can be applied to an existing trial. Therefore, our aim was to apply the DOOR methodology to an existing neonatal trial, the Milking In Non-Vigorous Infants (MINVI) trial which aimed to study cord management in non-vigorous newborns born between 35–42 weeks’ gestation. 6
Methods:
The MINVI trial study design, methods, and results have been previously reported. 6 Briefly, MINVI was a cluster-randomized, crossover trial conducted at 10 multinational sites (7 in the United States, 2 in Canada and 1 in Poland). The trial compared intact cord milking (squeezing an intact umbilical cord 4 times before clamping and cutting) with early cord clamping (clamping the umbilical cord within 60 seconds) in newborns born at 35–42 weeks’ gestation that had poor tone, color, or poor breathing efforts. The primary outcome was admission to the NICU for predefined criteria (respiratory distress, bradycardia or tachycardia, hypotonia, lethargy or difficulty arousing, hypertonia or irritability, poor feeding or emesis, hypoglycemia, oxygen desaturations or cyanosis, need for oxygen, apnea, seizures or seizure-like activity, hyperbilirubinemia, and temperature instability) that was not found to be statistically different between the two groups after accounting for the cluster-crossover trial design. For the present analysis, since the best outcome for these infants is a healthy delivery with no resuscitation, this was chosen as the best outcome, with death chosen as the worst outcome. Four additional outcomes that formed a gradient between the best and worst outcomes were also chosen and ranked: received initial cardiorespiratory support; NICU admission for predefined criteria; mild hypoxic-ischemic encephalopathy (HIE); and moderate-to-severe HIE.
In the present DOOR analysis, each newborn was assigned a mutually exclusive rank 1 through 6. Rank 1 represented the most desirable outcome and included anyone who was alive and did not experience any of the prespecified outcomes within the follow-up period, whereas rank 6 indicated the least desirable outcome and included all newborns who died before discharge. Ranks 2 through 5 included newborns who were alive but received cardiorespiratory support, were admitted to the NICU for predefined criteria, had mild HIE, or had moderate to severe HIE, respectively (Table 1). For the overall DOOR probability, newborns that met more than one category (i.e., were admitted to NICU and received cardiorespiratory support) were ranked per the worst category (i.e. higher rank). The distribution of DOOR ranks were compared between the UCM and ECC groups, and the probability that a randomly selected newborn assigned to UCM would have a more desirable outcome (i.e. a better rank) than a randomly selected newborn assigned to ECC was determined. When evaluating each of the individual components, outcomes were not mutually exclusive, and patients were included for each outcome they experienced. A DOOR probability of 50% indicates no difference between groups.
Table 1.
DOOR distribution by treatment group.
| DOOR category (ordered most desirable to least desirable)* | UCM N (%) | ECC N (%) | Gain (+) or Loss (-) per 1000 Neonates | |
|---|---|---|---|---|
| Difference | Cumulative Difference | |||
| 1. None of the outcomes below | 281 (36.6) | 198 (26.2) | +105 | +105 |
| 2. Received cardiorespiratory support | 287 (37.4) | 320 (42.3) | −49 | +56 |
| 3. NICU admission for MINVI reason | 172 (22.4) | 200 (26.4) | −40 | +16 |
| 4. Mild HIE | 15 (2.0) | 14 (1.8) | +1 | +17 |
| 5. Moderate to severe HIE | 12 (1.6) | 21 (2.8) | −12 | +5 |
| 6. Death | 0 (0.0) | 4 (0.5) | −5 | 0 |
neonate with more than one outcome were categorized per their least desirable outcome
In the present analysis, cluster assignment was ignored, and the DOOR probability was assessed across all 10 sites combined. As a sensitivity analysis, we conducted a stratified DOOR analysis for each cluster and then combined the obtained DOOR probabilities using a meta-analysis type of approach to derive a single combined DOOR probability. The ordinal outcomes included in the DOOR endpoint were also analyzed using a partial credit strategy,5 a method allowing for a non-uniform gradation between the ranks instead of an equal distance assignment. This approach is analogous to scoring an academic test, assigning 100% to the most desirable outcome, 0% to the least (e.g., death), and “partial credit” to each intermediate DOOR rank. The contrast between treatment strategies can be compared by varying the partial credit assignment of the intermediate ranks, which allows personalized clinician preferences to be incorporated. Indeed, while respecting the rank order, a clinician’s perspective may be to prioritize survival in considering the benefit of the intervention and may assign higher scores to ranks 1–5 [i.e., scores of 100, 90, 80, 70, and 60] compared to rank 6 [death, with score of 0]. On the other hand, another clinician may assign lower scores to the HIE outcomes [ranks 4–5] that are closer to the death one [i.e., 100, 90, 80, 30, 20, and 0, respectively for ranks 1–6]. Treatment comparisons are then made by comparing mean partial credit grades between treatments based on these weighted allocations. The partial credit method can alleviate concerns that a small decrement in a very important outcome component could be offset by a large advantage in an outcome component of lesser importance despite appropriate prioritization and ranking. In this study, four clinicians were asked to independently provide partial credit ranks to the outcomes based on their expertise.
Analyses were performed using SAS (version 9.4 for Windows; SAS Institute, Cary, NC) and the “DOOR Analyses: Standard Edition” online R shiny application (https://methods.bsc.gwu.edu/), which calculates the DOOR probability and confidence intervals, and performs the partial credit analyses based on user input. All tests were considered exploratory and p-values<0.05 were used to determine statistical significance.
Results:
Of the 1730 infants with primary outcome data, 260 were excluded due to missing data for HIE and/or cardiorespiratory support, leaving n=1524 infants included in this analysis. Of these, 767 (50.3%) were randomized to UCM and 757 (49.7%) were randomized to ECC. Descriptive characteristics of the study population have been previously published.6 In brief, characteristics were similar between the n=1730 vs the n=1524.
Most infants received cardiorespiratory support (n=975, 65.6%) and n=438 infants were admitted to NICU for MINVI criteria (28.7%). The distribution by treatment group for each individual component is shown in Table 2 and Figure 3. More neonates in the ECC group received cardiorespiratory support than in the UCM group (68.0% vs. 60.0%). Table 1 provides the numbers of neonates in each DOOR category when neonates with more than one outcome were categorized per their least desirable outcome. Overall, neonates in the UCM group experienced more desirable outcomes than in the ECC group. Figure 1 provides a graphical display of the DOOR distribution.
Table 2.
DOOR component distribution by treatment group.
| DOOR component | UCM N (%) | ECC N (%) | Expected Gained (+) or Loss (−) (per1000) |
|---|---|---|---|
| Received cardiorespiratory support | 460 (60.0) | 515 (68.0) | −81 |
| NICU admission for MINVI reason | 199 (25.9) | 239 (31.6) | −56 |
| Mild HIE | 15 (2.0) | 14 (1.8) | 1 |
| Moderate to severe HIE | 12 (1.6) | 24 (3.2) | −16 |
| Death | 0 (0.0) | 4 (0.5) | −5 |
Figure 3.
Bar chart of component outcomes by treatment.
Figure 1.
DOOR distribution by treatment group.
The distribution of the DOOR outcome was then compared to determine whether either intervention yielded a greater probability (i.e., > 50%) of resulting in neonates with the most desirable outcome. The DOOR distribution was statistically significant between the UCM and ECC groups, with 55.8% probability (95% CI 53.1–58.5%; p<0.0001), of a randomly selected neonate from the UCM group having a more desirable global outcome than a randomly selected neonate from the ECC group.
Forest plots demonstrating the DOOR probabilities overall, and for each individual DOOR component, are shown in Figure 2. Neonates in the UCM group were also estimated to have a greater probability of better individual outcomes than those in the ECC group. For example, the DOOR probability of averting NICU admission (52.8%; 95%CI 50.5–55.1%) and averting cardiorespiratory support (54.0%; 95%CI 51.6–56.4%) were significantly higher in the UCM group than the ECC group. The DOOR probability of averting moderate to severe HIE (50.8%; 95%CI 50.0–51.6%) and death (50.3%; 95%CI 50.0–50.6%) were borderline higher in the UCM group than the ECC group. Small numbers of death were observed in this analysis (0 in the UCM group vs. 4 in the ECC group). The DOOR probability of being in rank 1 (i.e., having none of the outcomes) relative to all other ranks was significantly higher in the UCM group than the ECC group (55.2%; 95% CI 52.9–57.5%; Figure 4).
Figure 2.
Forest plot summary of the estimated DOOR probabilities for the DOOR probability and each component outcome of having a more desirable outcome with in the UCM group compared with the ECC group. CI, confidence interval. The DOOR probability represents the probability that a randomly selected neonate will have a better DOOR (rank) if assigned to UCM compared to their DOOR (rank) had they been assigned to ECC. A DOOR probability near 50% indicates there is no difference in DOOR distributions by treatment group. Significant differences by treatment group were observed for the received cardiorespiratory support and the NICU admission for MINVI criteria components (DOOR probabilities 54.0%, 95% CI [51.6%, 56.4%] and 52.8%, 95% CI [50.5%, 55.1%], respectively).
Figure 4.
Forest plot summary of the estimated DOOR probabilities for the DOOR probability and for the cumulative DOOR probability based on sequential dichotomization of the DOOR outcome (i.e., rank 1 relative to all other outcomes, rank 1 and 2 relative to all other outcomes, etc.).
Figure 5 shows the forest plot for the DOOR probabilities in each cluster. The meta-analysis-derived overall DOOR probability was 55.7% (95% CI 53.0–58.3) between the UCM and ECC groups, providing a similar result to the DOOR probability that was obtained ignoring cluster assignment. In addition, as shown in Figure 6, neonates in the UCM group had a better DOOR ranking compared with those in the ECC group for each partial credit assigned rating.
Figures 5.
Forest plot of estimated DOOR probabilitiesfor each individual cluster (10 sites labeled site A-J) and for a meta-analysis-derived DOOR probability.
Figure 6.
Plot of the treatment difference based on partial credit grading vs. DOOR probability based upon selected grading keys with corresponding partial credit analysis summary table. Grading keys 1–4 are from the 4 independent clinician ratings, grading key 5 is the average. Based on the grading key from the first clinician, the mean partial credit score was 91.2 in the UCM group and 88.9 in the ECC group resulting in a mean difference in the treatments of 2.3, 95% CI (1.2, 3.4), and p-value <0.0001. Based on the grading key from the fourth clinician, the mean partial credit score was 74.8 in the UCM group and 69.8 in the ECC group resulting in a mean difference in the treatments of 5.0, 95% CI (2.8, 7.2), p<0.0001. The plot includes vertical and horizontal lines for each bivariate estimate displaying confidence interval estimates for the difference in means and the DOOR probability.
Discussion:
Using a cord-management trial, we have provided an example of the application of a DOOR approach and illustrated how it can provide useful insight beyond that of a traditional composite or single outcome. Specifically, the DOOR outcome provides an overall reflection of the patient experience at birth since it represents a combination of benefits and harms.
Umbilical cord milking is an ideal intervention for a DOOR analysis due to debate about its risks and benefits. Animal studies have suggested adverse cerebral cardiovascular flows in anesthetized preterm lambs who undergo cord milking,7 with human studies supporting this concern by showing an association between cord milking and severe IVH in extremely preterm infants.8 However, cord milking has also demonstrated benefits in preterm infants, including improved transition9 and cardiac function in preterm10 and term infants11 making it the ideal intervention to simultaneously study risks and benefits.
The DOOR methodology has been previously applied in several adult trials, both as part of post hoc analyses (1, 12, 13, 14) and with DOOR predefined as a primary endpoint (15, 16). Carefully defining the DOOR outcome is critical and with DOOR defined as a primary endpoint 1, 12–16. Carefully defining the DOOR outcome is critical and can be accomplished using surveys of experts and Delphi analyses17, 18and is the subject of FDA ORISE fellowships. These studies demonstrate the ability of the DOOR paradigm for obtaining a deeper understanding of the patient-centric benefits and harms of interventions that may not be observable using traditional approaches. A consideration for this study is that adult trials have the benefit of getting stakeholder input from patients themselves, whereas neonate trials must rely on parental and clinician input to determine which outcomes matter most. In our highest risk newborns, the extremely preterm infant, it is common for the “healthy preterm” to be mechanically ventilated, require oxygen, and have prolonged hospitalizations. Grading these outcomes is challenging and parents and clinicians may have differing perspectives on what outcomes are most important. Since the MINVI trial enrolled newborns born at 35–42 weeks’ gestation, it was logical to assume that the best outcome was no cardiorespiratory intervention, and that outcomes such as severe HIE are worse than mild-moderate HIE.
Other investigators have begun to consider the value of outcomes that more comprehensively account for the many events that may occur subsequent to an intervention. A recent neonatal trial used a global rank score to evaluate the effects of pre-operative steroids on post-operative outcomes.19 Hill et al. assigned hierarchical scores to different outcomes that infants experienced (e.g., heart transplant or need for dialysis) based on international consensus recommendations.19 However, this method did not allow for ranking of patients if they experience more than one outcome.
The DOOR paradigm recognizes important tradeoffs of benefits and harms providing an overall summary. Significance on component outcomes does not necessarily imply significance on the DOOR outcome, and significance on the DOOR outcome does not necessarily imply significance on the component outcomes. In this study, two of the five components (received cardiorespiratory support and NICU admission for MINVI criteria) leaned in the direction favoring UCM while the other three were borderline or not significant, yet culminating in a significant DOOR (Figure 2). Indeed, modest non-significant improvements on individual outcomes may add up to significant improvements overall. DOOR does not replace traditional separate analyses of each outcome. It builds upon it, strategically composing the component outcomes towards dyad-centric evaluation. Analyses of each component outcome is considered part of comprehensive DOOR analyses consistent with fundamentals of composite outcomes.
A limitation of our approach includes lack of an a priori DOOR analysis specified prior to the initiation of the trial. While we included our secondary outcomes from the original trial as part of the analyses, we acknowledge that choosing other outcomes could have altered our ability to detect a difference. The trial consulted parent representatives, neonatologists and NICU nurses for input to develop partial credit analyses, but we did not develop a large stakeholder consensus based guidelines as other DOOR studies have done.1–3 One of the values of the partial credit approach is evaluating the robustness of the results with varying perspectives about the value of the DOOR categories. Partial credit analyses from our group demonstrated all grading keys were significant with comparable advantages 2.3 −5 on a 100 point scale (Figure 6). As stated, these outcomes seem to be straightforward. Additional limitations include missing data (15% missing outcome data) and the inability to adequately account for the cluster-cross-over trial design. Therefore, this analysis should be considered more illustrative of the DOOR methodology than hypothesis testing. Strengths of the study include the large sample size, results which are consistent with the larger trial, and the application of this novel methodology to the neonatal population.
Conclusion
In conclusion, the use of a DOOR analysis provided an overall assessment of the benefits and harms of intact cord milking. Future neonatal trials should consider the a priori use of the DOOR outcomes in clinical trials.
Highlights.
Neonatal trials have previously used composite short-term or longer-term outcomes,
We applied the DOOR method and found that it provided an accurate assessment of the benefits and harms with greater insight than typical composite measures to clinicians and parents when evaluating an intervention.
Acknowledgments
Research reported in this publication was supported by the Eunice Kennedy Shriver National Institute of Child Health & Human Development of the National Institutes of Health under award Number R01HD096023. This trial is registered on ClinicalTrials.gov NCT# 03631940.
Abbreviations and Acronyms:
- ECC
Early Cord Clamping
- UCM
Umbilical Cord Milking
- NICU
Neonatal Intensive Care Unit
- DOOR
Desirability Of Outcome Ranking
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Data Sharing Statement:
Deidentified individual participant data (including data dictionaries) will be made available, in addition to study protocols, the statistical analysis plan, and the informed consent form upon request. The data will be made available upon publication to researchers who provide a methodologically sound proposal for use in achieving the goals of the approved proposal. Proposals should be submitted to anup.katheria@sharp.com
References:
- [1].Evans SR, Knutsson M, Amarenco P, Albers GW, Bath PM, Denison H, et al. Methodologies for pragmatic and efficient assessment of benefits and harms: Application to the SOCRATES trial. Clin Trials. 2020;17:617–26. [DOI] [PubMed] [Google Scholar]
- [2].Celestin AR, Odom SR, Angelidou K, Evans SR, Coimbra R, Guidry CA, et al. Novel Method Suggests Global Superiority of Short-Duration Antibiotics for Intra-abdominal Infections. Clinical Infectious Diseases: An Official Publication of the Infectious Diseases Society of America. 2017;65:1577–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [3].Rattan R, Allen CJ, Sawyer RG, Askari R, Banton KL, Claridge JA, et al. Patients with Complicated Intra-Abdominal Infection Presenting with Sepsis Do Not Require Longer Duration of Antimicrobial Therapy. Journal of the American College of Surgeons. 2016;222:440–6. [DOI] [PubMed] [Google Scholar]
- [4].Evans SR, Rubin D, Follmann D, Pennello G, Huskins WC, Powers JH, et al. Desirability of Outcome Ranking (DOOR) and Response Adjusted for Duration of Antibiotic Risk (RADAR). Clin Infect Dis. 2015;61:800–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [5].Evans SR, Follmann D. Using Outcomes to Analyze Patients Rather than Patients to Analyze Outcomes: A Step toward Pragmatism in Benefit:risk Evaluation. Stat Biopharm Res. 2016;8:386–93. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [6].Katheria AC, Clark E, Yoder B, Schmölzer GM, Yan Law BH, El-Naggar W, et al. Umbilical cord milking in nonvigorous infants: a cluster-randomized crossover trial. American Journal of Obstetrics and Gynecology. 2023;228:217.e1–e14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [7].Blank DA, Polglase GR, Kluckow M, Gill AW, Crossley KJ, Moxham A, et al. Haemodynamic effects of umbilical cord milking in premature sheep during the neonatal transition. Archives of disease in childhood Fetal and neonatal edition. 2017. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [8].Katheria A, Reister F, Essers J, Mendler M, Hummler H, Subramaniam A, et al. Association of Umbilical Cord Milking vs Delayed Umbilical Cord Clamping With Death or Severe Intraventricular Hemorrhage Among Preterm Infants. JAMA : the journal of the American Medical Association. 2019;322:1877–86. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [9].Katheria A, Blank D, Rich W, Finer N. Umbilical cord milking improves transition in premature infants at birth. PloS one. 2014;9:e94085. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [10].Katheria AC, Leone TA, Woelkers D, Garey DM, Rich W, Finer NN. The effects of umbilical cord milking on hemodynamics and neonatal outcomes in premature neonates. J Pediatr. 2014;164:1045–50.e1. [DOI] [PubMed] [Google Scholar]
- [11].Katheria A, Mercer J, Poeltler D, Morales A, Torres N, Lakshminrusimha S, et al. Hemodynamic Changes with Umbilical Cord Milking in Nonvigorous Newborns: A Randomized Cluster Cross-over Trial. The Journal of pediatrics. 2023;257:113383. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [12].van Duin D, Lok JJ, Earley M, Cober E, Richter SS, Perez F, et al. Colistin Versus Ceftazidime-Avibactam in the Treatment of Infections Due to Carbapenem-Resistant Enterobacteriaceae. Clin Infect Dis. 2018;66:163–71. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [13].Howard-Anderson J, Hamasaki T, Dai W, Collyar D, Rubin D, Nambiar S, et al. Improving Traditional Registrational Trial End Points: Development and Application of a Desirability of Outcome Ranking End Point for Complicated Urinary Tract Infection Clinical Trials. Clin Infect Dis. 2023;76:e1157–e65. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [14].Kinamon T, Gopinath R, Waack U, Needles M, Rubin D, Collyar D, et al. Exploration of a Potential DOOR Endpoint for Complicated Intra-Abdominal Infections Using Nine Registrational Trials for Antibacterial Drugs. Clin Infect Dis. 2023. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [15].Tamma PD, Souli M, Billard M, Campbell J, Conrad D, Ellison DW, et al. Safety and microbiological activity of phage therapy in persons with cystic fibrosis colonized with Pseudomonas aeruginosa: study protocol for a phase 1b/2, multicenter, randomized, double-blind, placebo-controlled trial. Trials. 2022;23:1057. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [16].Turner NA, Zaharoff S, King H, Evans S, Hamasaki T, Lodise T, et al. Dalbavancin as an option for treatment of S. aureus bacteremia (DOTS): study protocol for a phase 2b, multicenter, randomized, open-label clinical trial. Trials. 2022;23:407. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [17].Doernberg SB, Tran TTT, Tong SYC, Paul M, Yahav D, Davis JS, et al. Good Studies Evaluate the Disease While Great Studies Evaluate the Patient: Development and Application of a Desirability of Outcome Ranking Endpoint for Staphylococcus aureus Bloodstream Infection. Clin Infect Dis. 2019;68:1691–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [18].Johns BP, Dewar DC, Loewenthal MR, Manning LA, Atrey A, Atri N, et al. A desirability of outcome ranking (DOOR) for periprosthetic joint infection - a Delphi analysis. J Bone Jt Infect. 2022;7:221–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [19].Hill KD, Kannankeril PJ, Jacobs JP, Baldwin HS, Jacobs ML, O’Brien SM, et al. Methylprednisolone for Heart Surgery in Infants — A Randomized, Controlled Trial. New England Journal of Medicine. 2022;387:2138–49. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
Deidentified individual participant data (including data dictionaries) will be made available, in addition to study protocols, the statistical analysis plan, and the informed consent form upon request. The data will be made available upon publication to researchers who provide a methodologically sound proposal for use in achieving the goals of the approved proposal. Proposals should be submitted to anup.katheria@sharp.com