Psychological outcomes following a nurse-led preventative psychological intervention for critically ill patients trial: Statistical and health economic analysis plan

Abstract

The Psychological Outcomes following a nurse-led Preventative Psychological Intervention for critically ill patients trial is a cluster-randomised controlled trial of the clinical and cost-effectiveness of a complex nurse-led preventative psychological intervention compared with usual care in reducing patient-reported post-traumatic stress disorder symptom severity, and other reported psychological morbidities, at six months among Level 3 (intensive care) patients in adult general critical care units in England, Wales and Northern Ireland. This paper describes the proposed statistical and health economic analyses for the Psychological Outcomes following a nurse-led Preventative Psychological Intervention for critically ill patients trial. It is important to complete and publish this plan before inspecting and locking the trial data to ensure that post hoc and data-derived decisions are avoided.

Trial registration: ISRCTN53448131

Introduction

The Psychological Outcomes following a nurse-led Preventative Psychological Intervention for critically ill patients (POPPI) trial is a cluster-randomised controlled trial (cluster-RCT) investigating the clinical and cost-effectiveness of a complex nurse-led preventative psychological intervention compared to usual care for post-traumatic stress disorder (PTSD) symptoms, and other psychological morbidities, among Level 3 (intensive care) patients in adult, general critical care units in England, Wales and Northern Ireland (protocol available from https://www.journalslibrary.nihr.ac.uk/programmes/hsdr/1264124/). About 170,000 patients are admitted to adult, general critical care units in the National Health Service (NHS) each year. It has been estimated that around 50% of critically ill patients suffer serious emotional distress, and up to two-thirds have unusual experiences such as hallucinations and delusions, while in the unit.^1,2 Rigorous and relevant evidence is urgently needed to help reduce the burden of serious psychological morbidity on critically ill patients and their carers. Also, cost-effective strategies are needed to reduce the burden on the NHS. The purpose of this statistical and health economic analysis plan is to document the planned analyses for the clinical effectiveness and cost-effectiveness evaluations of the POPPI trial. The plan was completed before inspecting the data in order to avoid post hoc data-driven decisions.³

Trial design

For more detail of the trial design, please see the published protocol. A brief outline is presented below.

Trial sites and patients

NHS adult, general critical care units in England, Wales and Northern Ireland were eligible to participate. A total of 24 sites were randomly assigned to either the intervention or control groups with each site recruiting for between 13 and 17 months, with a staggered start to allow for roll-out of the intervention following a five-month baseline period (Figure 1).

Figure 1.

POPPI cluster-RCT schedule.

Inclusion and exclusion criteria

Inclusion criteria

• Age 18 years or greater

• Greater than 48 h in the critical care unit

• Receipt of Level 3 critical care (for any period of time) during first 48 h in the critical care unit

• Between +1 and −1 on the Richmond Agitation Sedation Scale⁴

• Glasgow Coma Scale score of 15

• English speaking

• Ability to communicate orally

Exclusion criteria

• Pre-existing cognitive impairment, such as dementia

• Pre-existing psychotic illness, such as schizophrenia

• Pre-existing PTSD

• Receiving end-of-life care

• Previously recruited to POPPI

Intervention

The POPPI trial is evaluating a complex intervention comprising three elements:

1. Creating a therapeutic environment in critical care

2. Three stress support sessions for patients screened as acutely stressed

3. Relaxation and recovery programme for patients screened as acutely stressed

Following the five-month baseline period, intervention group sites will undergo a one-month transition period, during which they will transition from delivering usual care to delivering the intervention. Each intervention group site will select three experienced critical care nurses (POPPI nurses). At the beginning of the transition period, all POPPI nurses will attend a three-day POPPI nurse training course. Following the training course, the POPPI nurses and local education/research teams will commence delivery of the intervention.

Each intervention group site will create a therapeutic environment by encouraging culture change in their unit. This will be facilitated by ensuring all clinical critical care unit staff complete the POPPI online training and by teaching good communication skills and psychological care at the bedside.

The Intensive Care Psychological Assessment Tool (IPAT) is a validated screening tool used to detect acute psychological stress and unusual experiences such as hallucinations in critically ill patients.⁵ Consenting, eligible patients will be assessed using the IPAT by a trained staff member. A patient is deemed highly stressed if they score seven or more on the IPAT and should be referred, as soon as possible, to a POPPI nurse to receive the three stress support sessions.

The aims of the stress support sessions are for nurses to develop a trusting relationship with patients, so patients can discuss concerns which they might feel embarrassed or worried about communicating, and to reduce emotional distress. The three stress support sessions are to be delivered by the same POPPI nurse ideally within one week. Each session lasts approximately 30 min. Patients will receive a tablet computer with a relaxation and recovery app to use between sessions and a self-help booklet and DVD.

Outcomes

All outcomes will be assessed and reported at the individual patient level.

Primary outcomes

The primary outcome for the clinical effectiveness evaluation will be patient-reported PTSD symptom severity at six months, measured using the PTSD Symptom Scale – Self-Report version (PSS-SR), which conforms to all Diagnostic and Statistical Manual of Mental Disorders 4th edition diagnostic criteria for PTSD and which has been validated for use in critical care unit survivors.⁶

The primary outcomes for the cost-effectiveness evaluation will be incremental costs, quality-adjusted life years (QALYs) and net monetary benefit at six months.

Secondary outcomes

• Days alive and free from sedation to day 30

• Duration of critical care unit stay

• PSS-SR greater than 18 points at six months

• Depression at six months, measured using the Hospital Anxiety and Depression Scale (HADS)⁷

• Anxiety at six months, measured using the HADS

• Health-related quality of life (HRQoL) at six months, measured by the EuroQol (EQ-5D-5L) questionnaire⁸

Power calculation

The initial power calculation for the POPPI cluster-RCT was calculated for the original grant submission and prior to conducting the POPPI feasibility study. Following completion of the POPPI feasibility study, the assumptions underlying the power calculation were reviewed using the results from the feasibility study to ensure the proposed design retained adequate power. Finally, during the early phase of recruitment to the POPPI cluster-RCT, the assumptions underlying the pre-cluster-RCT power calculation were reviewed again once outcome data were available from the baseline (pre-intervention) period for 20 of the 24 sites. Details of these three stages are provided in the online version of this article.

The final review of assumptions established that a minimum of 1378 patients were required to provide an anticipated 85% power (allowing for uncertainty in the between-site variation, between 79 and 91% power) to detect a difference of 4.2 points on the PSS-SR.

Allocation of sites

Participating sites were allocated to intervention or control groups using a restricted randomisation (full enumeration) approach to ensure balance across the treatment groups in geographical location, teaching status and size of unit.⁹ Allocation of the eight sites within each stagger was performed during the second month of recruitment in the baseline period.

Statistical analysis

Full details of the statistical analysis are provided in the online version of this article. A brief summary is provided below.

Analysis principles

All analyses will be based on the intention-to-treat (ITT) principle. All tests will be two sided with significance levels set at p < 0.05 and with no adjustment for multiplicity.

Handling of missing data

All patients who provide informed consent will be accounted for in the report of the trial. Patients that withdraw from the trial and do not give permission for data collected prior to withdrawal to be used in the final analysis, those that die before six months and those lost to follow-up for mortality will be excluded from the analysis of six-month psychological outcomes. Patients recruited during the transition period (and the equivalent months in control sites) will be excluded from the primary analysis but included in a sensitivity analysis. All other recruited patients will be included in the primary analysis, with outcomes imputed.

Loss to follow-up will be reported by treatment group. Reasons for withdrawal and loss to follow-up will be reported, when known.

Multiple imputation will be used to complete missing data for baseline covariates and resource use and non-response and partial response for the PSS-SR, HADS and EQ-5D-5L, under the assumption that responses are missing at random conditional on the observed data.¹⁰ Twenty multiply imputed datasets will be generated using Markov Chain Monte Carlo.

For the primary clinical and cost-effectiveness outcomes two sensitivity analyses will be used to address alternative assumptions regarding the missing data mechanism: missing completely at random (MCAR) and missing not at random (MNAR). To evaluate the results under the assumption of MCAR, the analyses will be repeated using complete case data (i.e. only those patients returning a completed questionnaire).¹⁰ To evaluate the results under the assumption that responses are MNAR, i.e. the probability of missing data depends on the patient’s outcome after conditioning on the observed data, a pattern-mixture model approach will be used.¹¹ Pattern-mixture models allow the outcome to be modelled differently according to whether it is observed or missing. To inform the assumptions about the parameters for the missing pattern that cannot be estimated from the data (sensitivity parameters), expert opinion about outcome differences between patients with missing versus complete data will be elicited from a representative sample of the clinical staff involved with the POPPI trial across the different trial centres and other interested experts.¹²

Trial profile

The flow of patients through the trial will be displayed in a Consolidated Standards of Reporting Trials diagram.¹³

Baseline characteristics

Baseline demographic and clinical data will be summarised for the ITT population, for each of the two treatment groups in each of the two time periods. No statistical testing will be carried out for any of the summary measures whilst comparing the baseline and treatment variables between the treatment groups.

Delivery of the intervention

Uptake of the POPPI online training will be reported for intervention sites over time as the percentage of the enumerated critical care unit staff that had completed the training course by month against a target of >80% completion.

Delivery of the intervention at a patient level will be summarised for patients in the intervention group during the intervention period.

Clinical effectiveness analysis – Primary outcome

The primary analysis for the clinical evaluation will examine if there is a significant difference in the mean PSS-SR at six months between patients recruited to the intervention group compared to the control group using a generalised linear mixed model (GLMM) at the individual patient level (patients nested within sites and within treatment group/time period) including site as a random effect variable and period as a fixed effect variable. The analysis will adjust for pre-specified baseline covariates at both patient and site level. The primary effect estimate will be the interaction (difference in difference) between treatment group and time period.

A secondary analysis will use structural mean models with an instrumental variable of randomised allocated treatment to estimate the efficacy (adherence adjusted causal effect) of the stress support sessions among those patients consenting to psychological assessment and stress support sessions, assessed as being at high risk of psychological morbidity (IPAT score ≥ 7) and receiving at least two stress support sessions.¹⁴

Clinical effectiveness analysis – Secondary outcomes

Analyses of the secondary outcomes will also be performed using GLMMs (like the primary outcome analysis), with identity link (i.e. linear regression) for continuous secondary outcomes and logit link (i.e. logistic regression) for binary secondary outcomes.

Subgroup analyses

The a priori identified subgroups that will be used for the subgroup analyses are as follows:

• Age (quartiles)

• Gender (male versus female)

• Socio-economic status (quintile of Index of Multiple Deprivation 2015¹⁵)

• Duration of delirium (no delirium versus delirium < median duration versus delirium ≥ median duration)

• Short-form State-Trait Anxiety Inventory (STAI-6) score¹⁶ (quartiles)

• Surgical status (emergency/urgent surgery versus elective/scheduled surgery versus non-surgical)

• Overall site engagement from process evaluation (low versus medium versus high)

• Heterogeneity of treatment effect (quintiles of predicted risk from a risk prediction model for the primary outcome developed using the usual care patient’s data adjusting for a priori important covariates of age, gender, socio-economic status, duration of delirium, STAI-6 and surgical status)

The evaluation of the treatment effect on the primary outcome of this study will be carried out using a formal test of interaction which will be obtained from the GLMM.¹⁷

Process evaluation

Analysis of the process evaluation will use a combination of qualitative and quantitative methods to assess and describe the variation in the delivery of the intervention across sites.¹⁸ The analysis will be conducted independent of the trial team before the outcome evaluation to avoid any bias in the interpretation of the process data and to generate hypotheses that may be subsequently tested in statistical analyses of integrated process and outcome data. The structural mean models described above will be extended to incorporate additional potential mediator variables on the causal pathway between treatment allocation and treatment effect identified by the independent process evaluation team, e.g. nurse competence following training, adherence to the therapeutic approach, adherence to therapy and overall site engagement.¹⁹

Economic evaluation

A full cost-effectiveness analysis (CEA) will be undertaken to assess the relative cost-effectiveness of the intervention compared with usual care. Resource use and outcome data collected as part of the cluster-RCT will be used to report cost-effectiveness at six months and to project the lifetime cost-effectiveness of each strategy. The cost analysis will take a health and personal health services perspective.²⁰ Cost will be calculated from patient-level resource use data on length of stay in critical care and hospital, for the index admission and any readmission before six months, use of personal health services after hospital discharge and within six-month post-randomisation, and additional staff time required to deliver the interventions. Resource use data will be combined with unit costs to report the total costs per patient at six months for intervention versus usual care.^21,22 HRQoL data from the EQ-5D-5L questionnaires at six months will be combined with survival data to report QALYs at six months. QALY will be calculated by valuing each patient’s survival time by their HRQoL at six months according to the ‘area under the curve’ approach.²³

The CEA will follow the ITT principle and report the mean (95% confidence interval) incremental costs, QALYs and net monetary benefit at six months. The CEA will use GLMMs that allow for clustering of patients including site as a random effect variable and period as a fixed effect variable.²⁴ The analysis will adjust for pre-specified baseline covariates at both patient and site level. The primary effect estimate will be the interaction (difference in difference) between treatment groups and time period. The CEA will use this model to estimate the effect of the intervention on mean cost and mean QALY (allowing for the correlation between the costs and QALYs at the individual and cluster level).

Lifetime cost-effectiveness will be projected by summarising the relative effects of alternative strategies on long-term survival and HRQoL, informed by extrapolations of patient survival data.^25,26 Adherence adjusted analysis and subgroup analysis will be undertaken for the pre-specified subgroups as per the analysis of clinical effectiveness.

Authors’ contribution

JW and ZS drafted the manuscript and will deliver the statistical and health economic analyses, respectively. DAH and RG will oversee the statistical and health economic analyses. KMR is Chief Investigator of the POPPI Trial and has oversight of the trial. PM leads the trial management. DW and DH are study co-investigators and contributed to the writing of this analysis plan. All authors read and approved the final manuscript.

Declaration of conflicting interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: The views and opinions expressed therein are those of the authors and do not necessarily reflect those of the HS&DR Programme, NIHR, NHS or the Department of Health.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This project was funded by the National Institute for Health Research (NIHR) Health Services and Delivery Research (HS&DR) Programme (project number 12/64/124).