Rationale, design and methodology of a trial evaluating three strategies designed to improve sedation quality in intensive care units (DESIST study)

Abstract

Objectives

To describe the rationale, design and methodology for a trial of three novel interventions developed to improve sedation-analgesia quality in adult intensive care units (ICUs).

Participants and Setting

8 clusters, each a Scottish ICU. All mechanically ventilated sedated patients were potentially eligible for inclusion in data analysis.

Design

Cluster randomised design in 8 ICUs, with ICUs randomised after 45 weeks baseline data collection to implement one of four intervention combinations: a web-based educational programme (2 ICUs); education plus regular sedation quality feedback using process control charts (2 ICUs); education plus a novel sedation monitoring technology (2 ICUs); or all three interventions. ICUs measured sedation-analgesia quality, relevant drug use and clinical outcomes, during a 45-week preintervention and 45-week postintervention period separated by an 8-week implementation period. The intended sample size was >100 patients per site per study period.

Main Outcome measures

The primary outcome was the proportion of 12 h care periods with optimum sedation-analgesia, defined as the absence of agitation, unnecessary deep sedation, poor relaxation and poor ventilator synchronisation. Secondary outcomes were proportions of care periods with each of these four components of optimum sedation and rates of sedation-related adverse events. Sedative and analgesic drug use, and ICU and hospital outcomes were also measured.

Analytic approach

Multilevel generalised linear regression mixed models will explore the effects of each intervention taking clustering into account, and adjusting for age, gender and APACHE II score. Sedation-analgesia quality outcomes will be explored at ICU level and individual patient level. A process evaluation using mixed methods including quantitative description of intervention implementation, focus groups and direct observation will provide explanatory information regarding any effects observed.

Conclusions

The DESIST study uses a novel design to provide system-level evaluation of three contrasting complex interventions on sedation-analgesia quality. Recruitment is complete and analysis ongoing.

Trial registration number

NCT01634451.

Strengths and limitations of this study.

• This detailed description of the development and design of the DESIST trial will facilitate better understanding of the study results.

• The modified cluster design will efficiently enable an exploration of the system level effects of three contrasting interventions on sedation quality in mechanically ventilated patients.

• The use of multilevel generalised linear regression mixed models will enable adjustment for important intensive care unit and patient level factors in the analysis to explore treatment effects at system level.

• A detailed mixed methods process evaluation of the complex interventions will enable a better understanding of each intervention and any clinical effects observed.

• Weaknesses include the potential for variability in uptake and use of the interventions, and limited study power for secondary outcomes such as drug use and length of stay. The importance of participation bias is also difficult to assess in system-level quality improvement research.

Introduction

Most mechanically ventilated critically ill patients require sedation and analgesia. Avoidance of unnecessary deep sedation is a priority in intensive care units (ICUs), because this is associated with adverse outcomes such as longer ICU stay, more ICU-acquired infections and, possibly, higher mortality.^1–3 Strategies that promote lighter sedation can improve these outcomes,⁴ but patient agitation can also compromise patient safety, and increase staff workload and stress.^5–7 Lighter sedation also potentially exposes patients to pain and discomfort, which are widely reported by ICU survivors.^8–11 Optimum sedation is patient specific, but the avoidance of deep sedation can be considered while ensuring adequate management of pain and agitation. The most effective system level approaches for optimising all aspects of sedation within ICUs is uncertain. Implementing and sustaining improvements in ICU sedation quality is challenging.

We describe the rationale, design and methodology of a quality improvement trial designed to evaluate the effectiveness of three contrasting complex intervention strategies. Each was developed to improve sedation quality for ICU patients, but with differing underpinning rationale. This trial is the central project in the Development and Evaluation of Strategies to Improve Sedation practice in InTensive care (DESIST) research programme (ClinicalTrials.gov NCT01634451). Underpinning research completed during the early part of the research developed and validated the trial outcome measures, and one of the interventions used in the trial, and has been reported in detail separately.¹² The trial has completed recruitment (December 2014), and analysis and reporting is ongoing. This manuscript provides a detailed description of the development of the trial design and the methodological approach to analysis, which will complement the reporting of the study findings. The DESIST research programme is funded by the Chief Scientists Office, Scotland; and through unrestricted research sponsorship from GE Healthcare.

Study objectives

The primary objective of DESIST is to explore the effectiveness of three system-wide interventions on sedation-analgesia quality in mechanically ventilated ICU patients, and any interaction between the interventions. The three interventions studied are:

1. A web-based modular educational resource, targeted primarily at nursing staff, termed ‘DESIST education’.

2. Feedback of sedation quality at regular intervals using process control methodology, termed ‘DESIST process feedback’.

3. The introduction of a novel sedation monitoring system into the ICU, the Responsiveness Index (RI), which continuously monitors patient arousals by facial electromyograph (fEMG) analysis, with the intention of alerting staff to ‘unresponsive’ patients who are at highest risk of deep sedation. This is termed ‘DESIST responsiveness monitoring’.

Secondary objectives are to evaluate the effect of the three interventions on: (1) the incidence of predefined sedation-related adverse events; (2) the use of intravenous sedative analgesic and antipsychotic drugs; (3) the duration of mechanical ventilation, ICU and hospital stay; and (4) ICU and hospital mortality. In addition, data will be collected to explore measures of patient experience, memories and traumatic experiences in the ICU.

A mixed methods design was considered most suited to describing the trial outcomes. Specifically, mixed methods for process evaluation of the intervention study will be used to supplement the main quantitative outcomes.

Study overview

The DESIST trial is the final phase of a programme of quality improvement work, undertaken in several stages.

Development and implementation of sedation quality measurement tools

A systematic review found no single simple clinical tool that could be systematically used to measure all aspects of patient status in relation to sedation-analgesia, or to define optimum sedation.¹³ Specifically, an outcome measure was required that incorporated unnecessary deep sedation, agitation and pain/discomfort, and included information required to define a consistent denominator for prevalence calculations.

In stage 1, a Sedation Quality Assessment Tool (SQAT) was developed and validated for measuring sedation-analgesia quality for each 12 h care period (typically an ICU nursing ‘shift’). This was termed a ‘DESIST care period’. Other data recorded in the SQAT included the concurrent use of mechanical ventilation; sedative, analgesic and neuromuscular blocking drugs; the presence of clinical conditions that justify deep sedation; and the requirement for therapies that may require deep sedation. The validity and reliability of the SQAT tool are reported separately.¹² The SQAT is shown in figure 1.

Figure 1.

The Sedation Quality Assessment Tool (SQAT) used to capture data for each 12 h period of nursing care during ICU admission. ICU, intensive care unit.

In stage 2, the SQAT was progressively implemented into routine practice in all the eight participating ICUs. Once established with high completion rates by nursing staff, we started enrolling patients into a preintervention data collection period (see below). During this period we used an iterative process with multiple members of the research team to develop algorithms that used SQAT data to classify patient status for each DESIST care period in the following domains to a binary yes/no state: (1) unnecessary deep sedation; (2) agitated; (3) poor limb relaxation (a measure of pain/discomfort); and (4) poor ventilator synchronisation. These four measures were the most valid and reliable overall as metrics for describing sedation quality.¹² Importantly, we used SQAT fields to censor episodes of deep sedation that were potentially clinically indicated (for example: advanced ventilation strategies, therapeutic hypothermia, brain injury) and capture only unnecessary deep sedation counts. SQAT fields also enabled relevant denominator data to be captured, for example, ventilation status and presence of coma despite not receiving sedatives, to censure patients where appropriate during calculation of sedation quality metrics. The outcome of this part of the project was a list of quality measures that demonstrated reliability, construct and face validity. The final measures chosen were:

1. Proportion of DESIST care periods with patient agitation

2. Proportion of DESIST care periods with excessive sedation

3. Proportion of DESIST care periods with poor relaxation

4. Proportion of DESIST care periods with poor ventilator synchronisation

5. Proportion of DESIST care periods with optimum sedation (optimum sedation was a DESIST care period where neither patient agitation, excessive sedation, poor relaxation, nor poor ventilator synchronisation were present, after censoring for relevant clinical information).

This part of the DESIST research programme has been reported separately.¹²

Assessing the impact of the quality improvement interventions: general design

DESIST is a quality improvement trial. Our hypothesis is that the interventions we developed will be used within the ICUs to improve practice, which will translate into improvements to sedation quality, measured using the metrics developed for the trial. DESIST is an evaluation of complex healthcare interventions and we expect any effects observed will result from the interventions implemented plus the way these are used to change practice. We designed a novel cluster-randomised trial with an a priori plan to analyse quantitative data, using multilevel regression models to explore the effects of each intervention on the primary and secondary outcomes, with adjustment for relevant patient-level factors. The study collected preintervention data in all eight ICUs for 45 weeks. Thereafter, two ICUs were randomised to receive the DESIST education intervention, two were randomised to receive DESIST education plus DESIST process feedback, two were randomised to receive DESIST education plus DESIST responsiveness monitoring and two received all three interventions. After an 8-week implementation period, data were collected for a further 45 weeks. Figure 2 shows the general design and structure of the trial. We planned to evaluate DESIST education using a before-after approach for all eight participating ICUs; for the DESIST responsiveness monitoring and DESIST process feedback interventions, we designed a factorial-type analysis of the four ICUs receiving the intervention versus four ICUs not receiving that intervention. We chose to include all ICUs in the evaluation of DESIST education to ensure all participating units received at least one intervention; this increased ‘buy-in’, given the high workload involved in patient enrolment and data collection. Consistent with the recommendations of the Medical Research Council complex intervention framework^{14 15} and the CONSORT guidance for reporting trials of interventions involving multiple elements,¹⁶ we designed a process evaluation that used mixed qualitative and quantitative approaches to help understand whether the interventions were implemented as planned, the barriers to implementation and factors that worked well/less well. The process evaluation also captures the user experience of each intervention, which is important for success and post-study implementation.

Figure 2.

The general structure of the DESIST trial indicating the preintervention, implementation and postintervention periods in relation to the quality improvement interventions. The total duration of each period and the timings of qualitative studies to inform the process evaluation are shown. ICU, intensive care unit.

Development and description of the interventions used in the trial

DESIST education

Several studies have identified education and knowledge as a barrier to successful sedation-analgesia improvement,^{17 18} together with anxiety relating to patient wakefulness.^{5 6 19} This intervention aimed to increase knowledge and training of nursing and other staff in a comprehensive consistent manner. A bespoke modular education package was developed in collaboration with an NHS provider of web-based interactive educational materials (LearnPro NHS: http://www.learnpro.co.uk). This includes in-built assessment to ensure acquisition of core knowledge across a range of relevant domains. Core knowledge tests have to be ‘passed’ in order to complete training successfully. The education package is available on NHS computers and personal computers/devices, and typically takes 2–3 h in total to complete. The nine modules are titled: Why is it important to get sedation right? assessing sedative state; commonly used agents in sedation-analgesia; avoiding excessive sedation; assessing pain and discomfort in ICU; managing agitation; managing delirium; drug withdrawal; and, helping patients sleep in the ICU. The education package can be viewed through the following test link: http://packagemanager.learnprouk.com using the username: desisttest. The password is: welcome. In the trial, individuals use unique logins to enable tracking of training and performance on assessments.

Each ICU developed strategies aimed at achieving 100% rates of completion by ICU nursing staff during the 2–3 months implementation phase of the trial, with a minimum target of 80%. Other staff groups were encouraged to undertake training but this was not mandated. Inbuilt systems tracked training completion within each ICU and fed back data to local trainers. Individuals completed a questionnaire comprising 10 core knowledge questions prior to undertaking the education package; this was repeated >5 months after the implementation phase and an analysis prespecified to assess rates of core knowledge and changes/retention during the study. The DESIST education package was freely available to all staff throughout the postintervention period, along with strategies to use it, as an ongoing resource developed in each ICU.

DESIST process feedback

Regular visual performance feedback is a well-established mechanism for driving quality improvement, which has been effective in the critical care setting,²⁰ particularly in relation to ICU-acquired infection.^{21 22} Using an iterative development process that included engagement with clinicians in the participating ICUs, process control charts were developed to illustrate the proportion of DESIST care periods with optimum sedation, unnecessary deep sedation, agitation, poor limb relaxation and poor ventilator synchronisation over time. These were piloted using data collected during the preintervention part of the trial. To reduce random variability, all data for 2-month periods from each ICU were combined to generate process control Proportion ‘P’ charts illustrating the proportion of DESIST care periods for which each of the five sedation-related measures were present.²³ Charts included upper and lower warning and control limits, and a mean value for each ICU using the preintervention data. In addition, prespecified sedation-related adverse events were collected on a daily basis by research staff, and used to generate count ‘G’ charts documenting the numbers of patients treated without a sedation-related adverse event. Warning and control limits for these charts were calculated. For the ICUs randomised to receive sedation quality feedback, reports of sedation quality were provided using these charts, and updated every 2 months during the intervention period, to summarise trends in sedation-analgesia quality and adverse event rates. ICUs were provided with strategies to share data from the reports (including posters and slide-sets), and encouraged to integrate these into quality improvement and other activities. The development and a description of the process control charts took part in the early part of the DESIST research programme and have been described in detail separately.¹²

DESIST responsiveness monitoring

The Responsiveness monitor is a novel technology developed collaboratively between GE Healthcare and Edinburgh University. The concept is to provide a measure of patient arousal based on continuous fEMG data collected via frontal electrodes, with the intention of providing a continuous alert to the potential presence of excessive sedation. The algorithm utilises the previous 60 min of fEMG data to derive an index, the Responsiveness Index (RI), calibrated 0–100. The algorithm has been published previously,²⁴ together with initial clinical validation studies. RI data are further simplified using a ‘traffic light’ system in which Red RI values (RI 0–20) are intended to indicate patients with a higher probability of deep sedation, whereas Amber values (RI 20–40) and Green values (RI >40) indicate more frequent arousals and a lower probability of deep sedation. Validation studies indicated a non-linear correlation/concordance with intermittent clinical sedation assessments; specifically, Red values can occur during sleep and low levels of clinical stimulation, and as a result of illness-related coma. The system is not designed for use in patients receiving neuromuscular blocking drugs. In a proof of concept trial, RI monitoring was used safely to assist nurse sedation decision-making when red RI values were present, and was acceptable to staff.²⁵ Preliminary data suggested that RI monitoring could decrease the time spent with lower RI values, and may decrease sedation-analgesia use, especially when the RI was low early in the ICU stay. The ICUs randomised to RI monitoring in DESIST were supplied with sufficient monitoring systems to enable the technology to be used in all enrolled patients, and staff received training and instruction in the use and interpretation of data. The ICUs were encouraged to use RI monitoring throughout periods of sedation for all enrolled patients, and nurses were encouraged to review and reduce sedation, when appropriate, if Red RI values were present. However, no strict protocols were supplied to determine sedation-analgesia use according to RI data, given that the correlation between RI value and clinical sedation status is non-linear.

Study population

We aimed to undertake a system level evaluation, so all mechanically ventilated, intubated patients were considered potentially eligible, including patients in whom mechanical ventilation via an endotracheal tube was instituted at some time after ICU admission; these patients were eligible for inclusion from the time intubation and mechanical ventilation began.

Exclusions at the time of screening were patients: (1) not receiving mechanical ventilation via an endotracheal tube or a tracheostomy; (2) in whom mechanical ventilation had been discontinued at the time of screening for study inclusion; (3) in whom discontinuation of mechanical ventilation was anticipated in the next 4 h; (4) in whom a decision to withdraw active therapy had been made; (5) in whom consent from a relative or welfare guardian was not obtained within 48 h of the start of mechanical ventilation; and (6) in whom the clinician caring for the patient declined permission for inclusion.

According to the cluster design, the entire ICU agreed to implement any of the interventions prior to participation. This was achieved through meetings and presentations to medical and nursing leads, relevant health service managers, education teams and quality improvement teams, prior to starting preimplementation baseline data collection. However, as almost all eligible patients lacked mental capacity at the time of screening for consent to participate, the Adults with Incapacity Act (Scotland; 2000) required consent from a relative/welfare guardian. To avoid potential selection bias, an identical approach was used throughout the study phases and across all ICUs. For the focus groups and field work, ethical approval was provided within the main trial application. Individual consent from staff members was not required.

Data collection

Demographic and sedation-related data

For all participating patients age, sex, APACHE II score and admission diagnosis were recorded via the Scottish Intensive Care Society Audit database (http://www.sicsag.scot.nhs.uk). Throughout the study, a SQAT was completed for all DESIST care periods from enrolment until ICU discharge or death, by the clinical nurse caring for the patient at the end of each 12 h period of care. All SQAT forms were collected, and data entered into the trial database at the Edinburgh Clinical Trials Unit. In addition, data were collected daily by research staff in the case record file to include: cumulative dose of sedative, analgesic and antipsychotic medication; the occurrence of predefined sedation-related adverse events (unplanned removal of nasogastric tube, central line, arterial line, drain or peripheral line; unplanned extubation; staff injury; patient injury); mechanical ventilation status; timing of extubation; and ICU outcome. At the end of the study, data to describe duration of mechanical ventilation, ICU length of stay, hospital length of stay, and ICU and hospital mortality, will be extracted from the Scottish Intensive Care Society Audit database, which is collected within each ICU using the Wardwatcher system.

Patient experience data

Sedation practice may influence patient recall of their ICU stay, including experience of discomfort, pain, unpleasant dreams and delusional memories. This may contribute to longer term psychological morbidity such as anxiety, depression and post-traumatic stress. Wherever possible, patients who survived their ICU stay were approached to complete the following questionnaire-based measures once mental capacity was regained, preferably close to the time of discharge from hospital:

Impact of Events Scale Revised (IES-R):²⁶ This 22 item scale is designed to detect post-traumatic symptomatology and is suitable for delivery in the hospital period following critical care. Responses can be used to generate scores for ‘avoidance’, ‘intrusion’ and ‘hyperarousal’ subscales, as well as a total IES-R score.

Intensive Care Experience Questionnaire (ICE-Q):²⁷ This 31 item questionnaire is designed to assess the patient’s memories of intensive care and perception of the ICU experience in four domains, namely, ‘awareness of surroundings’, ‘frightening experiences’, ‘recall of experience’ and ‘satisfaction with care’.

Study phases

The exact duration of each phase was uncertain at study start-up, because the sedation-analgesia quality metrics were under development and sample size estimations relied on early recruitment data within the ICUs. The justification for the final duration of enrolment is shown below (see sample size).

Preintervention period

The preintervention period finally comprised 45 weeks for each ICU. During this period, no study interventions occurred and data collected represented baseline practice for each ICU. We used data during this period, to estimate sample size for statistical modelling and determine the duration of patient enrolment for each phase.

Implementation period

The introduction and implementation of the allocated intervention(s) aimed to be completed during a 2-month period. All ICUs nominated a local champion (a co-investigator on the project), who facilitated change within the ICU and engaged existing clinical and quality improvement teams. For DESIST education, this comprised training at least 80% of nursing staff (target 100%) through completion of the DESIST education package. For DESIST process feedback, sedation quality reports plotting data at 2 monthly intervals were supplied for the 45-week preintervention period and a multidisciplinary stakeholder meeting was organised in the ICU. Methods for sharing and utilising the sedation quality process feedback in routine ICU activities were developed. For DESIST responsiveness monitoring, RI monitoring systems were provided, with training of core staff in their use and interpretation, and support during clinical use was offered.

Postimplementation period

Data continued to be collected for all enrolled patients during a further 45-week period during which the allocated interventions were used to support sedation-analgesia practice. For the ICUs allocated to sedation quality feedback, data were entered into the trial database in real time and used to generate the sedation quality process feedback reports every 2 months, at which point charts were updated to include the preceding 2-month period. For the ICUs allocated to RI monitoring, RI data were recorded hourly during periods of monitoring, using the Red, Amber and Green colour coding. These data were used during the process evaluation to describe the use of the RI technology. All ICUs received at least one visit from the research team during the first 2–3 months of the postimplementation period and additional support as required by telephone or teleconference.

Sample size

We initially estimated that the eight ICUs would admit ≈2900 ventilated patients each year (range 250–750 between ICUs), although many patients would be excluded due to short ventilation duration. Assuming 40–60% enrolment of ventilated patients would have provided approximately 1400 patients per year. Baseline proportions of DESIST care periods with ‘optimum sedation’ were uncertain prior to the study. We estimated 70% rates for the purpose of modelling sample size.

We expected differences between the ICUs, so estimates required to take clustering into account. We initially modelled study power using ‘best case’ and ‘worst case’ recruitment scenarios based on recruitment during the early part of the preintervention period. An absolute increase in the rate of optimum sedation by 25% from a baseline rate of 70% would be clinically important and likely to translate into patient benefit. We assumed power 80%, two-sided significance level 5% and comparison of intervention versus control with a continuity-corrected χ² test. To account for clustering (participants within ICU), a sample size inflation factor 1+(m−1)×ICC was applied where m is the cluster size and ICC the intraclass correlation coefficient. The ICC was unknown prior to the study; table 1 gives the primary outcome improvements detectable for a range of ICC values with numbers of participants ranging from 250 per ICU per study period (1000 per four ICU cluster) (see table 1(A)) to 66 per ICU per study period (264 per four ICU cluster) (see table 1(B). These estimates indicated that, although the ICC was uncertain, we would have sufficient power to detect moderate to large changes in the primary outcome measure, and would likely be able to measure effect sizes for the secondary outcomes with a reasonable degree of precision. Lower numbers of patients recruited per ICU would decrease the penalty paid for the clustering in the design, which offsets the reduction in sample size, such that lower recruitment rates would have minimal effect on our study power to detect improvements in rates of optimum sedation practice.

Table 1.

‘best’ and ‘worst’ case scenarios used to estimate the optimum sample size and duration of the preintervention and postintervention periods in the eight ICUs

Participants per ICU	Power (%)	Reference process rate (%)	Mean process rate change detectable (%)	Base sample size per group	ICC	Inflation factor	Inflated sample size (4 ICUs per analysis group): participants per group
(A) ‘Best case scenario’ sample size justification
250	80	70	19.7	74	0.05	13.45	1000
250	80	70	26.2	38	0.10	25.90	1000
250	80	70	29.3	29	0.13	33.37	1000
(B) ‘Worst case scenario’ sample size justification
66	80	70	19.7	74	0.040	3.57	264
66	80	70	26.2	38	0.092	6.95	264
66	80	70	29.3	29	0.125	9.10	264

Inflation factor is 1+(m−1)×ICC, where m=cluster size.

ICU, intensive care unit.

Based on data during the early part of the preintervention period, we estimated that the smallest ICU would recruit, on average, 2–3 patients/week, and that recruitment for 45 weeks during each of the preintervention and postintervention periods would be expected to provide ≥100 patients per ICU per study period (≥400 patients per four ICU group). As the primary analysis was based on DESIST care periods, rather than individual patients, we expected substantially larger numbers of evaluable data periods for the modelling.

Randomisation

We organised the study so that participating ICUs started the study in a staggered manner to enable support by the research team at key stages, notably the implementation period. Randomised allocation was revealed to ICUs at the end of the preintervention period to ensure allocation concealment during baseline data collection. The eight ICUs were randomised according to computer-generated random permuted blocks, stratified according to the time at which the ICU started recruitment to the study into ‘early’ (first four ICUs to start recruitment) and ‘late’ (last four ICUs to start recruitment), in order to balance higher and lower recruiting sites within each intervention comparison.

Two ICUs were therefore randomised into each of the following four intervention groups:

•  Group 1: DESIST education

•  Group 2: DESIST education plus DESIST process feedback

•  Group 3: DESIST education plus DESIST Responsiveness monitoring

•  Group 4: DESIST education plus DESIST process feedback plus DESIST Responsiveness monitoring.

Process evaluation and qualitative study

A prespecified plan recorded compliance with the planned implementation strategy by local research implementation teams. The following data were captured relevant to each of the three interventions as follows:

1. ‘DESIST education’: the proportion of ICU nursing staff that completed the DESIST education package and passed all modular assessments;

2. ‘DESIST process feedback’: the number of process reports and slide sets provided to the ICUs. The maximum number of reports, including the implementation period report, was six;

3. ‘DESIST responsiveness monitoring’: whether formal training in the use of theRI monitoring occurred according to a prespecified training schedule; the number of enrolled patients who received any period of RI monitoring; the number of RI data logged by bedside nurses, based on the data recorded in the case record file. The RI data observed in the enrolled patients will also be described to explore how the technology was used.

Qualitative data were collected during all phases of the study. Multiprofessional focus groups were conducted in each ICU prior to the implementation phase, to understand the current culture of sedation practice. During the implementation and postimplementation phases, participant observation was undertaken by the same researcher (KK) in all ICUs in three distinct timelines, to understand the uptake of the interventions and changes in practice. This was at: the end of the implementation phase; midway in the postimplementation phase; at the end of the postimplementation phase. We also formed multiprofessional focus groups in the final month of the postimplementation phase, in which participants reflected on the uptake of the intervention(s) and the changes in sedation practice.

Data from field notes from participant observation and focus groups transcripts were verbatim transcribed and then checked for accuracy of transcription by the qualitative researcher and by a member of the research team in each ICU. Data were entered into NVivo V.10 for Windows software for qualitative analysis (QSR International, Ltd). For analysis, data will be organised by ICU setting for coding. An inductive thematic analysis will be conducted without a predefined theoretical framework to allow the in-depth exploration and understanding of the impact of interventions on sedation management. Constant comparison will ensure that the thematic analysis represents all perspectives, and negative cases will be sought. Validity checking of the coding includes recoding of data from four ICUs, representative of each intervention group, by an independent researcher (JH). Discordant coding and disagreement will be resolved by discussion within the wider research team. To build a valid argument for choosing the themes, the related literature will be searched to facilitate the interpretation of the data.

Primary coding will involve identifying common patterns of experiences with each intervention in each ICU. All data that relate to the already classified patterns will be explored to categorise engagers versus non-engagers with the interventions; barriers and facilitators to adopting and implementing the intervention(s) will be explored, and quality improvement strategies and changes in sedation practice that occurred will be considered for each ICU. Patterns of changes in sedation practice will be compared to the identified gaps in practice from the preintervention focus groups for each ICU. Related patterns will be combined and catalogued into subthemes. We will compare themes by characteristics of ICUs (ie, size of ICU, patient case mix, staff levels). Finally, we will compare themes by combination of interventions to identify any intervention(s) interaction patterns. Comparison of ICUs in relation to engagement and adoption of the interventions will be summarised to inform and understand the quantitative analysis.

Outcomes

Study outcomes are shown in box 1. The primary outcome, and main secondary outcomes, are measures of sedation quality. Other secondary outcomes describe sedative drug use, ICU and hospital outcomes. Exploratory outcomes will include the measures of ICU experience recorded among surviving patients after their discharge from the ICU.

Box 1. The outcomes that will be used in the DESIST trial.

Primary outcome

Proportion of DESIST care periods with optimum sedation

Secondary sedation quality outcomes

1. ICU level outcomes

   • A1: Proportion of DESIST care periods with patient agitation
   • A2: Proportion of DESIST care periods with excessive sedation
   • A3: Proportion of DESIST care periods with poor relaxation
   • A4: Proportion of DESIST care periods with poor ventilator synchronisation
2. Patient-level sedation outcomes

   • B1: Number of DESIST care periods with optimum sedation per mechanically ventilated patient
   • B2: Number of DESIST care periods with agitation per mechanically ventilated patient
   • B3: Number of DESIST care periods per patient with excessive sedation
   • B4: Number of DESIST care periods with poor relaxation per mechanically ventilated patient
   • B5: Number of DESIST care periods with poor ventilator synchronisation per mechanically ventilated patient
3. Sedation-related adverse events

   • C1: Proportion of days during mechanical ventilation on which a sedation-related adverse event occurred*
   • C2: Proportion of patients receiving mechanical ventilation in whom a sedation-related adverse event occurred

Sedation, analgesic and antipsychotic drug use

• D. Sedative and analgesic drug use

  • D1: Total use of intravenous sedative drugs per patient (propofol equivalents†)
  • D2: Proportion of ICU days on which ≥4000 mg propofol or propofol equivalents were given (an index of likely deep sedation)
  • D3: Total use of intravenous analgesic drugs per patient (alfentanil equivalents†)
  • D4: Proportion of patients receiving haloperidol

ICU outcomes

• E. Duration of mechanical ventilation during index ICU admission (days)

• F. Duration of ICU stay (days)

• G. Duration of hospital stay (days)

• H. ICU mortality

• I. Hospital mortality

Exploratory patient experience outcomes

• J. Patient experience and symptoms

  • J1: Intensive Care Experience questionnaire (ICE-Q) score for awareness of surroundings, frightening experiences, recall of experience and satisfaction with care domains
  • J2: Impact of events scale revised (IES-R) score for: avoidance, intrusion, hyperarousal and total score

Nursing staff knowledge of sedation

• K. Change in nursing staff knowledge (based on pretraining vs 5 months post-training test comparisons)

*Unplanned removal of NG tube, central line, arterial line, drain or peripheral line; unplanned extubation; staff injury; patient injury. Any combination of the listed adverse events will comprise an ‘adverse event day’.

†A conversion algorithm for all agents used in the ICUs into propofol and alfentanil equivalents is described in online supplementary material.

NG, nasogastric tube.

Analysis

The analysis will be based on the intention-to-treat principle and will include all ICUs and all consented patients with available data. The only exception will be patients admitted with a diagnosis of status epilepticus: these are rare (about 1% of admissions), require more complex and individualised sedation management and could confound the outcome measures of interest in DESIST. An overall significance level of 5% (two-sided) will be used; an emphasis on reporting 95% CIs in the estimation of effect sizes will be used given the study design, and uncertainties about event rates and recruitment. The planned analyses will be performed using STATA (StataCorp; http://www.stata.com), MLwiN (University of Bristol; http://www.bristol.ac.uk/cmm/software/mlwin) and SAS (http://www.sas.com) statistical software.

The following ICU level factors will be used to describe the patients included within each ICU (cluster): number and proportion of eligible patients enrolled in each trial phase; occurrence of each sedation quality outcome in each trial phase; age, gender, APACHE II score, admission type and ICU admission diagnosis of patients in each trial phase. Age, sex and APACHE II score will be used in the modelling to adjust for differences between ICUs and also changes within ICUs between the preintervention and postimplementation periods.

The primary analysis of the binary primary outcome will be a multilevel generalised linear regression mixed model that uses outcome as the dependent variable; ICU, time period (preintervention or postimplementation) and ICU by time period interaction as the fixed effects independent variables at the ICU level; and age, sex and APACHE II score as the fixed effects independent variables at the admission level. A 3-level multilevel random intercept model will be fitted using MLwiN, where DESIST care period is level 1, admission is level 2 and ICU is level 3. Markov Chain Monte Carlo methods will be used for parameter estimation, and the intraclass correlation coefficient (ICC) will be reported at level 2 (admission) and level 3 (ICU).

The multilevel model will not in the first instance make formal grouped comparisons between ICUs randomised/not randomised to DESIST responsiveness monitoring (R); and between ICUs randomised/not randomised to DESIST process feedback (P). Similarly, it will not make a formal grouped comparison between preintervention and postimplementation to assess the effect of DESIST education (E). Instead, an OR and 95% CI will be calculated for the preintervention to postimplementation change within each ICU. An OR value greater than one would indicate an increase in the proportion of DESIST care periods in which there was optimum sedation. In the light of the observed variability in the OR estimates across ICUs, and informed by the findings from the qualitative data, a decision will be made on whether it is appropriate to perform a pooled analysis across ICUs to summarise intervention effects for any of E, R, P or the interaction between R and P. We plan a priori to make this decision in conjunction with the members of the independent data monitoring committee, who will review the individual ICU-level quantitative data together with the initial findings of the process evaluation and qualitative study. This stage has been included because we anticipate there may be wide variability in uptake and implementation of the interventions that might make grouped comparison illogical. The process evaluation, completed by researchers blinded to quantitative analysis of each ICU (KK and JH), will inform this decision.

In the event that computational difficulties prevent the successful fitting of the complex three-level multilevel model described above, we will revert to an alternative strategy whereby a two-level multilevel model will be fitted to each ICU separately, in which DESIST care period is level 1 and admission is level 2. Time period will be included as an independent variable at ICU level; age, sex and APACHE II scores will be the independent variables at the admission level; and a random intercept will be included for level 2.

Sensitivity analyses

We anticipate that implementation and uptake of the interventions may not be complete at the end of the 2-month implementation period, and are likely to continue during the postintervention period. In addition, the QI process is intended to continue throughout the postintervention period. We will repeat the analysis including only postimplementation data recorded in the final 30 weeks of the study, which will assess the effect of the interventions after a 5–6-month total period of QI activity.

Secondary outcomes

Binary secondary outcomes (A1, A2, A3, A4; C1; D2; table 1) will be analysed in the same manner as in the primary outcome. Binary secondary outcomes measured at the patient or admission level (C2; H; I; table 1) will be analysed using a 2-level multilevel generalised linear model, including the same independent variables as used in the model for the primary outcome. Continuous secondary outcomes (D1, D3; table 1) will be analysed using a 2-level multilevel normal linear model, including the same independent variables as used in the model for the primary outcome. Secondary outcomes involving a count of a number of events (B1, B2, B3, B4, B5) will be analysed using 2-level multilevel Poisson regression, including an offset term for the number of eligible care periods in an admission. The model will contain the same independent variables as used in the primary outcome model. A rate ratio and 95% CI will be calculated for the preintervention to postimplementation change within each ICU. A rate ratio value greater than one would indicate an average increase in the outcome event count per admission. Time-to-event secondary outcomes (E, F, G) will be analysed using a 2-level multilevel Cox proportional hazards regression model. The model will be fitted using a Poisson model in MLwiN by splitting follow-up time into as many intervals as there are events, and will contain the same independent variables as used in the primary outcome analysis. Larger time intervals may be used if it becomes too computationally intensive, which would result in a close approximation of the Cox model.

A detailed analysis plan was agreed prior to locking the data base, and is included as online supplementary material.

Comment

The DESIST study uses a novel design and analysis to evaluate the effectiveness of three contrasting interventions developed to improve sedation quality. The broad inclusion criteria ensure the population studied will represent a wide range of critically ill patients, and have external validity to most settings and healthcare systems providing ICU care. This approach will decrease the risk of enrolment bias. The three interventions developed have differing underpinning rationale. DESIST education is a knowledge-based intervention using web-based learning with inbuilt assessment. We hypothesise that this intervention may act by directly improving clinical practice by individual bedside nurses. DESIST process feedback is an information-based intervention using novel quality-indicators developed in the research programme to feedback performance to the ICU over time using process control methodology, an approach that has been successful for decreasing ICU-acquired infections.^{21 22} We hypothesise that this approach may drive quality improvement through regular provision of detailed performance data, although quality improvement will rely on ICUs generating interventions and undertaking Plan-Do-Study-Act tests of change. DESIST responsiveness monitoring will use a novel technology designed to alert bedside clinical staff to the possibility of excessive sedation (Red responsiveness number). We hypothesise that this approach will support nurse decision-making and decrease the prevalence of excessive sedation, which is associated with adverse patient outcomes.

Given the absence of agreed methods for quantifying sedation quality and optimum sedation-analgesia for the purpose of clinical evaluations, we had to develop measures of sedation quality specifically for the trial.¹² We maximised efficiency by undertaking part of this work during the preintervention data collection period. Previous sedation trials have focused on metrics such as duration of mechanical ventilation and ICU stay, where the trial interventions were designed to minimise sedation use and deep sedation.^28–31 These outcomes are important, but may miss other important outcomes such as pain, agitation and sedation-related adverse events. Current guidelines emphasise the need to provide optimum pain, agitation and delirium management in addition to avoiding unnecessary deep sedation,¹ and observational studies indicate a high prevalence of recalled pain and distressing memories among ICU survivors.^{11 32} Patients also express preference for deeper sedation and get greater satisfaction when target sedation ranges are achieved.³³ Our primary outcome and main secondary outcomes specifically quantify the prevalence of overall optimum sedation, as well as measures of unnecessary deep sedation, agitation and pain/discomfort. Our analytical approach will also explore these at a system level (ICU level) and at patient level. A limitation of our design is that we will not capture the prevalence of delirium, which is associated with adverse outcomes.¹ Delirium measurement requires frequent clinical assessments by trained staff, but resource constraints meant it was not feasible to include this. We considered including delirium as an outcome measure within the trial, but our focus was on optimising sedation-analgesia management. In addition, although delirium has been associated with adverse outcomes, no intervention other than optimising sedation practice has been shown to reduce delirium prevalence.¹ By capturing rates of agitation in a systematic manner, we expect to detect cases of agitated delirium, although part of the focus of the education package is to highlight that several factors can cause agitation in the ICU patient. In addition, by recording the use of haloperidol, the most widely used drug for treating agitated delirium in our setting, we expect to capture major changes in pharmacological delirium management.

Our modified cluster design tests the three interventions simultaneously, and uses generalised linear mixed models to explore treatment effects and interactions. Advantages of this approach include its efficiency, which is relevant given the complexity and high cost of clinical trials in critically ill populations. As DESIST is a quality improvement trial aiming to evaluate three different interventions simultaneously, we considered this approach best suited to the research questions. We could have designed cluster-randomised or step-wedge trials testing each intervention separately, but this would have been significantly more complex and expensive. An individual patient parallel randomised design was not well suited to the system-level interventions. Disadvantages of our approach included uncertain study power prior to starting, and the real likelihood of differing degrees and patterns of uptake of the interventions between the participating ICUs. By using data during the early preintervention period, we were able to optimise sample size and recruitment duration, and will focus on effect sizes with 95% CIs to describe quantitative findings. Another potential weakness was our inability to blind clinicians from the trial interventions, increasing the possibility of participation bias. This is unavoidable in system-level quality improvement research, which relies on behaviour change. Our design aimed to reduce the potential importance of participation bias by the long duration of data collection, the inclusion of a sensitivity analysis including data from the final 30 weeks, and the use of a bespoke validated quality assessment tool that used offline algorithms to calculate outcomes and blinded these from staff during the trial. We also planned, a priori, to include a detailed process evaluation using mixed methods to understand the impact of the interventions on practice throughout the recruitment period. These approaches are in keeping with current recommendations for evaluating and reporting complex healthcare interventions.^{14 15} We recognise that DESIST may not provide definitive effectiveness data, however, it will inform the ongoing development of strategies to improve sedation quality. The methodological approach will also be potentially relevant to other complex intervention research both in critical care and in other challenging settings.