Table 3.

Challenges for critical care trial design, analysis, and interpretation: frequentist versus Bayesian approaches

Problem	Description of problem in conventional analysis of critical care trials	Explanation of how a Bayesian approach could address this problem
Quantitative use of external information in trial analyses	Conventional trials do not quantify the influence of prior information in the analysis of a trial.	A Bayesian approach, including determination of a minimum clinically important difference and a set of defensible priors combining data and expert opinion, can provide the probability of benefit based on prior information and trial data.
Insufficient sample size	Sample sizes in critical illness trials are often too small to exclude clinically important differences in mortality and other outcomes.	Communicating results as the posterior probability of exceeding different clinically-relevant thresholds provides greater insight into the meaning of results from small trials as to whether treatment benefit has been adequately ruled in or out and whether there is persistent equipoise. The Bayesian paradigm can guide sample size estimation by clarifying the amount of information (sample size) required to achieve consensus across all defensible priors.
Applying results to clinical practice	Frequentist analyses provide no formal methods to guide how results might be adapted at the bedside depending on a clinician’s informed skepticism or enthusiasm for that therapy in that patient.	Knowing the probability of exceeding a clinically relevant threshold under different priors is more aligned with day-to-day clinical problem solving.
Adopting therapies with minimal clinical benefit	No benefit is too small to generate a significant p-value, if a trial enrolls enough patients. In clinical practice, patients and clinicians may require a certain amount of benefit before adopting or using a therapy.	The Bayesian approach allows flexible calculation of the probability of benefit at different thresholds which can be selected based on the characteristics of individuals or populations.
Abandoning therapies with potential clinical benefit	Interpretations of frequentist analyses may conflate indeterminate and truly negative results, leading to abandonment of therapies where clinical benefit has not been ruled out.	The posterior probability distribution quantifies the extent to which clinical benefit has or has not been ruled out.
Controversy about small trials with extreme results	Frequentist trials are analyzed in isolation, occasionally leading to strongly positive results that generate controversy or premature adoption of a therapy.	Quantifying the extent to which results change across differing prior distributions helps quantify the degree of scientific uncertainty around results from small trials.
Stopping trial enrolment for futility or benefit	Complicated rules around p-values and interim analyses can lead to early stopping of trials that could still have contributed helpful data to a clinical question.	Results from Bayesian analyses are not contingent on what you intended to do and allow decisions about stopping or continuing a trial to be based on the probability that a therapy will be beneficial.
Nonstandard trial designs require complex statistics	Nonstandard trial designs such as adaptive trials are potentially helpful yet awkward to design and analyze with frequentist principles.	Bayesian approaches make adaptive trial designs more analytically feasible.
Gleaning insights from subgroup analyses is difficult	Trials often include several clinically relevant subgroups but frequentist analysis requires either complex multiplicity corrections or the qualifier “exploratory”; trials often lack the sample size to demonstrate a definitive answer for the entire trial population, let alone a relevant subgroup.	Posterior probability distributions can be derived for the treatment effect in any subgroup of a trial, and prior distributions can also be specified for any subgroup. Bayesian hierarchical regression is a more sophisticated technique well-suited for this problem.46
Forming clinical guidelines from trial data requires accurate synthesis of data from multiple studies and trials	Frequentist analyses are quantitatively isolated and combining results through meta-analysis may not capture the full spectrum of information available in cohort or physiologic studies.	Investigating the range of posterior probabilities according to prior information could help make clinical guideline formulation from randomized trials more intuitive and informed.
Statistical support is required for safe conduct of randomized trials	Conventional frequentist trials often retain advanced statistical support for study design and analysis.	Trials designed with a Bayesian approach may require more specialized statistical support and expertise, which may be a barrier for some investigators and trials groups.
Trial outcomes may not be important to patients or clinicians	Frequentist analyses sometimes use surrogate outcomes in order to increase statistical power.	A Bayesian approach may be more flexible with regards to surrogate outcomes because it can capture the uncertain relationship between the surrogate and the true patient-important outcome, however, a Bayesian approach will not overcome a poorly-chosen surrogate outcome.
Missing outcome data may impede trial analysis	Conventional trials may become biased through missing outcome data.	Bayesian approaches do not necessarily offer any better options with regards to missing data.
Randomization is logistically difficult and may deter participation	Randomizing patients to treatments can sometimes be a barrier for patients and clinicians to participate in trials.	Randomization remains a central tool for quantitative causal inference, whether using frequentist or Bayesian methods.
Informed consent can be burdensome and deter participation	Processes of informed consent can be burdensome for clinicians and patients.	Different analytic paradigms do not obviate the need for ethical informed consent in trials.