Agents intervening against delirium in the intensive care unit trial—Protocol for a secondary Bayesian analysis

Abstract

Background

Delirium is highly prevalent in the intensive care unit (ICU) and is associated with high morbidity and mortality. The antipsychotic haloperidol is the most frequently used agent to treat delirium although this is not supported by solid evidence. The agents intervening against delirium in the intensive care unit (AID‐ICU) trial investigates the effects of haloperidol versus placebo for the treatment of delirium in adult ICU patients.

Methods

This protocol describes the secondary, pre‐planned Bayesian analyses of the primary and secondary outcomes up to day 90 of the AID‐ICU trial. We will use Bayesian linear regression models for all count outcomes and Bayesian logistic regression models for all dichotomous outcomes. We will adjust for stratification variables (site and delirium subtype) and use weakly informative priors supplemented with sensitivity analyses using sceptical priors. We will present results as absolute differences (mean differences and risk differences) and relative differences (ratios of means and relative risks). Posteriors will be summarised using median values as point estimates and percentile‐based 95% credibility intervals. Probabilities of any benefit/harm, clinically important benefit/harm and clinically unimportant differences will be presented for all outcomes.

Discussion

The results of this secondary, pre‐planned Bayesian analysis will complement the primary frequentist analysis of the AID‐ICU trial and facilitate a nuanced and probabilistic interpretation of the trial results.

1. INTRODUCTION

Delirium affects between 30% and 50% of intensive care unit (ICU) patients ¹ , ² and is associated with detrimental short‐ and long‐term outcomes. ³ , ⁴ , ⁵ , ⁶ , ⁷ , ⁸ Haloperidol, a first‐generation antipsychotic, is the most frequently used agent to treat delirium ⁹ albeit no evidence‐based treatment currently exists for this condition. ¹⁰ , ¹¹ The clinical practice guideline on delirium (PADIS guideline) from the Society of Critical Care Medicine has not supported the use of haloperidol since 2013. ¹² , ¹³ Two recent systematic reviews ¹⁰ , ¹¹ did not find any firm evidence on the effect of haloperidol on short‐ and long‐term outcomes, but evidence is sparse as only one trial with low‐risk of bias investigating haloperidol treatment of delirium currently exists (n = 376, haloperidol vs. placebo). ¹⁴ Consequently, uncertainty remains regarding the potential beneficial or harmful effect of haloperidol for the treatment of delirium. The agents intervening against delirium in the intensive care unit (AID‐ICU) trial is a large, pragmatic, placebo‐controlled, randomised trial that will provide high‐quality data on the use of haloperidol for the treatment of delirium in adult, acutely admitted ICU patients. ¹⁵ , ¹⁶

The present protocol and statistical analysis plan outline the rationale and methodology for a secondary, pre‐planned Bayesian analysis of all outcomes registered up to Day 90 in the AID‐ICU trial. The Bayesian analysis will supplement the primary, frequentist analysis and provide easily interpretable effect estimates that may help clinicians, researchers and policymakers interpret the findings of the trial.

2. METHODS

2.1. Study design and conduct

This protocol and statistical analysis plan describe a pre‐planned, secondary Bayesian analysis of the AID‐ICU trial. The protocol has been finalised after enrolment of the last patient, but before the end of follow‐up and database closure.

The AID‐ICU trial is an investigator‐initiated, multicentre, parallel‐group, blinded, centrally randomised and stratified (for site and delirium subtype) trial of haloperidol versus placebo in 1000 ICU patients with delirium. The aim of the AID‐ICU trial is to assess the benefit and harms of haloperidol for the treatment of delirium in adult ICU patients.

The trial started enrolment in June 2018, recruited the last patient April 11 2022; the day of 90‐day follow‐up for the last patient is thus July 11, 2022. The protocol ¹⁵ and statistical analysis plan ¹⁶ have been published elsewhere, and information on trial, including protocol and detailed variable definitions, is available at the trial website (www.cric.nu/aid-icu).

2.2. Approvals, registrations and reporting

The trial is approved by the regulatory authorities in the participating countries and registered at the European Union Clinical Trials Register (EudraCT no. 2017‐003829‐15) and at ClinicalTrials.gov (NCT03392376). Further details are available in the primary protocol. ¹⁵

This secondary study protocol and the final study report adhere to or will adhere to the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement ¹⁷ (completed checklist included in the Supporting Information Material S1) with the Bayesian analyses planned and conducted according to the Reporting Of Bayes Used in clinical STudies (ROBUST) guideline. ¹⁸ Results will be reported regardless of findings.

2.3. Enrolment criteria

Eligible patients are adults who are acutely admitted to the ICU and diagnosed with delirium using a validated screening tool. ¹⁵ We exclude patients with contraindications to haloperidol; those with habitual use of any antipsychotic medication or antipsychotics in the ICU prior to screening; alcohol‐induced delirium (delirium tremens); where assessment of delirium is non‐applicable (language barriers, serious auditory or visual disabilities); who are fertile (<50 years) with positive urine or plasma human chorionic gonadotropin and those in whom consent cannot be obtained according to national regulations.

2.4. Interventions

Trial participants are randomised in a 1:1 ratio to receive either intravenous 2.5‐mg haloperidol or matching placebo three times daily as long as patients are delirious. If needed, additional trial medication may be administered up to a maximum dose of 20‐mg haloperidol/placebo per day. In case of incontrollable delirium, trial participants may receive escape medication (propofol, benzodiazepines or alpha2‐agonists) as decided by the clinical team. Further details are available in the primary protocol. ¹⁵

2.5. Outcomes

In this secondary analysis of the AID‐ICU trial, we will assess outcomes registered within the 90‐day intervention period. One‐year follow‐up data will thus not be included in this secondary analysis. The secondary outcome ‘total number of SARs to haloperidol’ will not be analysed in this secondary analysis as it is expected to be identical to the binary outcome ‘Number of patients with one or more SARs to haloperidol to day 90’. The outcomes analysed are listed below.

2.5.1. Primary outcome

1. Days alive and out of hospital from randomisation to Day 90 (DAOH).

2.5.2. Secondary outcomes

• 2All‐cause mortality at Day 90.
• 3Days alive without delirium or coma in the ICU to Day 90.
• 4Days alive without mechanical ventilation in the ICU to Day 90.
• 5Days alive with escape medicine per patient to Day 90.
• 6Number of patients using escape medicine to Day 90.
• 7Number of patients with one or more Serious Adverse Reactions (SARs) to haloperidol to Day 90.

Death within the 90‐day intervention period is not penalised to zero in the count outcomes. The numbers reflect the actual days alive irrespective of survival status later in the intervention period.

2.6. Sample size

The AID‐ICU trial will enrol 1000 trial participants. This sample size estimation was based on conventional frequentist analyses of data from the observational AID‐ICU cohort study. ⁹ Justification of sample size and power calculations was presented in the primary statistical analysis plan. ¹⁶ In this secondary analysis, we will include all participants in the intention‐to‐treat population.

2.7. Statistical analysis

All analyses will be conducted in the statistical software R (R Core Team, R Foundation for Statistical Computing) using the Tidyverse ¹⁹ packages and Stan ²⁰ through the brms R package. ²¹ All analyses will be adjusted for the stratification variables; site and delirium motor subtype (hyperactive vs. hypoactive delirium).

2.7.1. Principles of Bayesian analyses

Bayesian analyses consist of three important concepts: prior probability distributions, data likelihoods and posterior probability distributions. Prior probability distributions commonly referred to as priors, describe prior knowledge or beliefs about an effect estimate before data analysis. Data are incorporated via a likelihood function, and the prior probability distribution is updated using Bayes' theorem resulting in a posterior probability distribution also known as the posterior. The posterior is thus the result of a Bayesian analysis and encodes an updated probability distribution of the effect estimate. ²² , ²³

2.7.2. Priors

We will primarily use weakly informative priors encompassing all plausible effect sizes. These priors will have minimal influence on the results, as the posteriors will be dominated by the data of the AID‐ICU trial due to the large sample size.

Sensitivity analyses will be conducted using sceptical priors for the intervention effects. These priors are sceptical of large effects sizes and will ‘shrink’ effect estimates towards no difference. This is based on the current knowledge that many interventional trials in critical care have shown either small, clinically unimportant or statistically insignificant differences. ²⁴ , ²⁵ Currently, no high‐quality data exist to form prior beliefs about the effect of haloperidol on our primary and secondary outcomes; while some data from other trials exist, these trials either contain different or differently defined outcomes. ¹⁰ If relevant external evidence becomes available before trial completion, we will consider additional sensitivity analyses incorporating these data in evidence‐based priors. Complete prior details are presented in the supplement.

2.7.3. Summarisation and presentation of results

For count outcomes, we will present absolute differences as mean differences (MD) and relative differences as ratios of means (RoM). For binary outcomes, we will present absolute differences as risk differences (RD) and relative differences as relative risks (RR). All effect measures will be presented as average (marginal) treatment effects, calculated as previously described. ²⁶ Posteriors will be summarised using median values as point estimates and percentile‐based 95% credibility intervals (CrIs).

The cumulated posterior distributions for the parameters of primary interest (MD and RoM or RD and RR for the treatment effects) for each outcome will be visualised as outlined in previous work performed by the group. ²⁷ This enables the visualisation of probabilities of all possible effect sizes.

Further, to ease the translation of the results into clinical practice, we will calculate probabilities of any benefit/harm, clinically important benefit/harm (defined as an MD ≤−1 or ≥1 day for count outcomes and RD of ≤−2 or ≥2 percentage points for binary outcomes) or clinically unimportant differences. Results will be presented as outlined in Table 1.

TABLE 1.

Effect estimates and probabilities (mock table)

Outcome	Effect estimates		Probability of effects with haloperidol
	Relative difference	Absolute difference	Any benefit	Any harm	Clinically important benefit	Clinically important harm	No clinically important difference
Count outcomes example: days alive and out of hospital	RoM #.## (CrI #.## ‐ #.##)	MD #.## days (CrI #.##‐#.## days)	##.#%	##.#%	##.#%	##.#%	##.#%
Binary outcomes example: 90‐day mortality	RR #.## (CrI #.## ‐ #.##)	RD #.## (CrI #.## ‐ #.##)	##.#%	##.#%	##.#%	##.#%	##.#%

Note: Mock table illustrating how results will be presented. ‘#’ will be replaced with actual numbers being the results of the analyses. Effect estimates are presented as ratio of means (RoM), mean difference (MD), relative risk (RR) and risk difference (RD) with percentile‐based 95% credibility intervals (CrI). RoM >1 and MD >0 favours haloperidol, while RR <1 and RD <0 percentage points favours haloperidol. All effect estimates are adjusted for stratification variables being site and delirium motor subtype. Any benefit is the probability of an RoM >1/MD >0 days or RR <1/RD <0% and any harm is the probability of an RoM <1/MD <0 days or an RR >1/RD >0%. Clinically important benefit is the probability of MD ≥1 days for count outcomes or RD ≤−2 percentage points for binary outcomes. Clinically important harm is the probability of MD ≤−1 days for count outcomes or RD ≥2 percentage points for binary outcomes. No clinically important difference is the probability of effect between clinically important benefit/harm and no clinically important difference.

2.7.4. Analysis of the count outcomes

Distributions of data for the primary count outcome and secondary count outcomes are expected to be non‐normal as a consequence of a high in‐hospital mortality and thereby zero‐inflation of the data. This was also confirmed by the interim analysis conducted after randomisation of 500 patients (the full population was assessed, not the two treatment groups). Despite the expected non‐normal distributions, we will analyse all count outcomes using adjusted Bayesian linear regression models. Linear models are robust for non‐normal distributions when estimating between‐group differences in large samples, ²⁸ , ²⁹ and are easy to interpret and to specify sensible, interpretable priors for.

2.7.5. Analysis of binary outcomes

All binary outcomes will be analysed using adjusted Bayesian logistic regression models.

2.7.6. Missing data handling

We expect the amount of missing data to be low and will report data completeness. If less than 5% of patients have missing data for one or more variables included in an analysis, complete case analysis without imputation will be conducted. If ≥5% of patients have missing data in any of the specified analyses, we will impute data following the same strategy as for the primary (frequentist) analyses of the trial. ¹⁶ If multiple imputation is used, models will be fit to each imputed dataset separately, followed by stacking posterior draws from all model fits and using the stacked posteriors for subsequent calculations.

2.7.7. Model diagnostics

Models will in general be assessed as described in the groups' previous work. ³⁰ , ³¹ , ³²

For modelling, we will use Stan's default dynamic Hamiltonian Monte Carlo sampler with four chains, at least 10,000 total post‐warm‐up samples and bulk/tail effective sample sizes for the parameters of primary interest of at least 5000. Sampler settings will be tuned to avoid divergent transitions and chain convergence will be assessed visually by the inspection of overlain density and trace plots. Rhat statistics ≤1.01 for all parameters will be required. ³³ , ³⁴

Model fits will be assessed by graphical posterior predictive checks of the group means/proportions ³⁵ and Pareto‐smoothed importance sampling leave‐one‐out cross‐validation focusing on the number of effective parameters. ³⁶ , ³⁷

If multiple imputations are used, models will be fitted and assessed separately in each imputed dataset before posteriors are stacked, with the requirements for the number of post‐warm‐up samples and effective sample sizes applying to the pooled samples.

3. DISCUSSION

This secondary Bayesian analysis will supplement the primary frequentist analysis of the AID‐ICU trial and provide additional information on the effects of haloperidol on delirium in critically ill patients.

In the Bayesian framework, the probability of any effect size may be calculated and the analysis can thereby inform clinicians on, for example, any benefit/harm, clinical important benefit/harm using preferred thresholds and other effect sizes of interest. The calculated effect size is supplied with a 95% CrIs that represents the 95% most probable values. Within this framework, trial results can be interpreted as direct probabilities of a given effect size avoiding the conventional dichotomisation of trial results as statistically significant or not. Statistical significance is based on an arbitrary threshold (e.g., p values <.05), which has been criticised by statisticians and clinicians as they do not provide any information on effect sizes or clinical relevance. ³⁸ , ³⁹

3.1. Strength and limitations

The primary protocol outlines the basic strength and limitations of the AID‐ICU trial, which include stringent methodology, central randomisation with concealed allocation, blinding of all stakeholders and a large sample size based on power estimations from a large international cohort of delirious ICU patients. ¹⁵

Conducting a secondary Bayesian analysis of the trial will aid interpretation of the trial. The analyses will be conducted in accordance with this pre‐planned analysis plan that will ensure transparency and validity of the work. The limitations include the exact priors and thresholds for clinically important benefit and harm. No general consensus exists for thresholds of clinically important benefit and harm and other priors and thresholds could reasonably have been chosen. However, the priors used in this study are weakly informative which means they will have little influence on the results, and they will easily be overwhelmed by the data. The clinically important benefit or harm may be defined differently by others; however, these thresholds have been used in previous work performed by the group and all effect sizes will be available in plots of posterior distribution for the parameters of primary interest. ²⁶ , ⁴⁰

4. CONCLUSIONS

This pre‐planned secondary Bayesian analysis of the AID‐ICU trial will provide additional information on the probable effects of haloperidol for the treatment of delirium in adult, acutely admitted ICU patients. The proposed study will contribute with a nuanced interpretation of trial data with assessment of direct probabilities and uncertainties for pre‐defined clinically relevant effect sizes. These results may help clinicians, researchers and policymakers interpret trial results and guide future care and research.

Supporting information

Appendix S1 Supporting Information.

Andersen‐Ranberg N, Poulsen LM, Perner A, et al. Agents intervening against delirium in the intensive care unit trial—Protocol for a secondary Bayesian analysis. Acta Anaesthesiol Scand. 2022;66(7):898‐903. doi: 10.1111/aas.14091