Understanding and Improving Clinical Trial Outcome Measures in Acute Respiratory Failure

The critical care community conducts significant research, with over 850 trials in the National Institutes of Health registry (1). To facilitate the synthesis and interpretation of findings across these studies, and effectively and efficiently advance clinical research toward improved patient outcomes, randomized controlled trials (RCTs) must be performed using valid and comparable outcome measures.

The Call for Standardization of Outcome Measures in RCTs

RCTs demonstrate substantial variability in outcome measures across many different clinical specialties (2). Within critical care medicine, there has been relatively little critical evaluation of outcome measures used in clinical research. This finding has led to recommendations from a number of national and international groups, including Roundtable conferences (1, 3), two workshops of the National Heart, Lung, and Blood Institute (NHLBI) (4, 5), the Society of Critical Care Medicine (6), and the Multisociety Task Force for Critical Care Research (7). These recommendations call for researchers to critically evaluate outcome measures, and use valid, appropriate, standardized measures across studies.

Outcome Measures in Trials of Patients with Acute Respiratory Failure

In this issue of the Journal, Blackwood and colleagues (pp. 886–893) (8) and Contentin and colleagues (pp. 998–1002) (9) each publish reports critically evaluating mechanical ventilation–related outcome measures from trials published in high-impact journals. Findings from these reports are consistent, demonstrating that no more than 25% of mechanical ventilation trials reported a definition for mechanical ventilation duration, and approximately 65% reported a definition of ventilator-free days. Among those reporting definitions, importantly, there was substantial variability in both the definition used and the time point of evaluation. Furthermore, the report by Contentin and colleagues (9) provides a detailed description of seven items requiring consideration when defining and calculating mechanical ventilation duration and ventilator-free days as outcome measures.

In addition to standardizing definitions, important issues remain regarding appropriate, standardized timing of outcome assessments and the associated methods for patient follow-up over this time period. This issue of timing is important, because interventions in critically ill patients may have benefits or harms well beyond hospital discharge or 28-day follow up (10, 11), and inferences regarding the effect of critical care interventions may change with longer durations of follow up (12, 13). Determining the optimal duration of follow-up will depend on the specific intervention and outcome being evaluated, and requires additional empirical research to understand patients’ typical trajectories of recovery after critical illness (14).

Even after considering issues of definitions and timing, there is a need for greater recognition that, even without an effect on mortality or mechanical ventilation duration, interventions in the intensive care unit may have important effects on survivors’ long-term functional outcomes (15). These functional outcomes fall across a wide range of domains, including aspects of physical, cognitive, and mental health (6). Recent RCTs of patients with acute respiratory failure provide examples of successful evaluation of these important functional outcomes over 6- to 12-month follow-up periods (16, 17).

In evaluating functional outcomes after hospital discharge, additional methodological considerations arise. Issues such as loss to follow up and censoring due to death may bias study results. More specifically, loss to follow up contributes to missing data and selection bias, whereas censoring due to death can bias the estimated effect of an intervention when there is differential mortality between treatment groups (18).

The Way Forward to Improving Outcome Measurement

Across clinical specialties, there is an international effort for reaching consensus on outcome measures and establishing “core outcome sets” that represent agreed-upon, standardized collections of outcome measures that will be reported in all trials within a clinical area (19). A well established example of work in this area comes from rheumatology, where, for more than 20 years, the Outcome Measures for Rheumatology Clinical Trials collaboration has been working to establish core outcome sets (20). Moreover, across clinical specialties, there is the Core Outcome Measures in Effectiveness Trials (COMET) initiative (http://www.comet-initiative.org/) that has an active database with more than 490 references of work planned, in process, or completed with respect to core outcome sets.

For trials of critically ill patients, plans for moving forward are developing (Figure 1). For instance, within the COMET initiative, there are at least three projects in the planning or execution phases that focus on core outcome sets in the areas of: (1) mechanical ventilation outcomes; (2) rehabilitation outcomes; and (3) long-term functional outcomes. In addition, the NHLBI has recently funded a new 5-year, investigator-initiated, national resource–related research project (R24HL111895) to create and disseminate resources related to: (1) establishing core outcome sets for long-term physical, cognitive, and mental health outcomes in survivors of acute respiratory failure and acute respiratory distress syndrome; (2) maximizing cohort retention in long-term, longitudinal research studies; and (3) developing statistical methods and programs for addressing censoring due to death in evaluation of long-term functional outcomes.

Figure 1.

An approach to understanding and improving clinical trial outcome measures in acute respiratory failure.

The methodological work and efforts to establish core outcome set projects have potential for international input and uptake via the existing International Forum for Acute Care Trialists (InFACT; http://www.infactglobal.org) group. InFACT is a global collaboration of more than 20 investigator-led clinical research consortia. InFACT’s Outcomes Measurement Group, with representation from these research consortia, is actively working in this area.

In summary, there are clear recommendations for greater standardization of outcome measures in clinical trials evaluating patients with acute respiratory failure, and objective data in this issue of the Journal support these recommendations. With recent initiatives, collaborations, and NHLBI funding, there are exciting new opportunities to make progress in understanding and improving clinical trial outcome measures in acute respiratory failure.