Opportunities and challenges of clustering, crossing over, and using registry data in the PEPTIC trial

The Proton Pump Inhibitors (PPIs) versus Histamine-2 Receptor Blockers (H₂RBs) for Ulcer Prophylaxis Therapy in the Intensive Care Unit (ICU) (PEPTIC) trial is the largest randomised clinical trial ever conducted in the field of intensive care medicine.¹ The potential clinical implications of the trial have been the subject of a previous editorial.² Here we focus on the implications of the study for clinical trial science and on the opportunities the study provides for exploratory analyses that will potentially shed further light on the relative safety and efficacy of using PPIs or H₂RBs for stress ulcer prophylaxis in the critically ill.

Novel aspects of the PEPTIC trial design

The PEPTIC trial design incorporated a number of novel aspects that have important implications for the future of ICU research (Table 1). The trial used existing registry data sources predominantly, which greatly reduced the amount of data that needed to be collected from individual patients. This cluster, crossover design³ tested regimens of stress ulcer prophylaxis implemented at the level of the ICU.⁴ Each ICU used either a PPI or H₂RB for 6 months and then switched to the alternative class of drug for the subsequent 6 months. The order of treatments used in study ICUs was randomised. The cluster crossover design embedded the trial into usual clinical practice and meant a large number of patients were enrolled in a short time frame; every patient invasively mechanically ventilated within 24 hours was included in the trial by default. Arguably, for the first time in a randomised clinical trial in intensive care research, there was sufficient power to detect what might reasonably be considered a minimum clinically important difference. While the initial, pre-trial sample size calculations suggested the trial would provide 80% power to detect a 2.4% absolute risk difference,⁴ in reality, the trial provided 80% power to detect a 1.8% absolute risk difference.¹ The reason for the discrepancy was mainly because of a difference between the estimated within- and between-period cluster correlation coefficients used in the sample size calculations and the observed coefficients in the trial itself.⁵

Table 1.

Features of the PEPTIC trial design

Feature	Potential advantages	Required trade-offs
Cluster randomisation	■Simplifies recruitment because recruitment of individual patients is not required ■Can be used to embed a trial into usual clinical care ■Provides information on the real-world effect of implementing treatment strategies	■Non-adherence with assigned study medicines can occur and, when it does, the ability to draw causal inferences about the effect of those medicines (as opposed to the effect of implementation of treatment strategies at the level of the ICU) is diminished
Crossover	■Increases power compared with a parallel arm cluster randomised design	■Introduces the possibility of carry-over effects in situations where study interventions induce changes in clinician behaviour that might not easily be unlearned ■Power depends on factors including “between periods within cluster" variability, which may not be easy to predict
Use of registry data	■Reduces trial costs because the workload associated with collection of trial data is largely eliminated ■Allows sites with limited or no research coordinator workforce to participate in the study	■Registry data may contain errors ■Registries in different countries may collect information in different ways ■All data of interest may not be included in registries and so collection of some patient data from medical records may still be required
Waiver of consent	■Approval of enrolment into the trial with either a waiver of consent or with permission to enrol patients and then to provide them with the opportunity to opt out of participation once they have recovered sufficiently to provide such consent combined with cluster randomisation allows for very rapid recruitment of large numbers of trial participants	■Waiver of consent or opt-out consent models are only possible in countries where such models of consent are in line with local laws and regulations
Broad eligibility criteria	■Increases generalisability of results	■Patients with a low risk of death as well as patients with a very high risk of death and potential for modifiable mortality are included

Remarkably, the trial afforded similar power to what would have been observed with an individual randomised controlled trial with the same number of participants. This occurred principally because there was little variability in the in-hospital mortality rates of each ICU over time. Such variability is labelled as "between periods within cluster” variability, and is what remains after taking the within-ICU differences between the two interventions — namely, ICU constant factors are cancelled out, and only factors that vary over the two observation periods remain. Minimising this variability is a key component of the power of a cluster crossover design,⁵ and as indicated in the additional statistical analyses of in-hospital mortality provided in the supplementary appendix of the PEPTIC article,¹ this variability was essentially zero. This is the absolute statistical best-case scenario for a cluster crossover trial. However, this best case may not be generalisable to future cluster crossover trials in ICU because it depends on design characteristics of the trial (ICU locations, observation period lengths), outcome measures being assessed, and intrinsic variation in outcomes between patients within the same ICU. Nevertheless, based on what was observed in the 50 ICUs in five countries in the PEPTIC trial, it is possible that such variability over 6-month periods among patients who are invasively mechanically ventilated within 24 hours of ICU admission is consistently small.

Relevance to the design of future trials

Even despite these considerations, given the efficient recruitment rates we achieved, the implications of such statistical power for future trials of ubiquitous ICU interventions, such as fluid therapy, oxygen therapy, nutrition and blood pressure targets, are potentially profound. With no observed between-group difference, a 95% confidence interval in a trial similar to PEPTIC would be expected to exclude either an increase or decrease in mortality of one percentage point. Yet, with power to detect absolute mortality differences of 2%, small differences in mortality potentially attributable to idiosyncratic practice variations for ubiquitous therapies are now identifiable.⁶ While an absolute mortality difference of 2% may appear small, this equates to a number needed to treat of 50, and to 2000 lives saved or lost for every 100 000 patients treated. Even smaller differences will be detectable if multiple crossovers are performed,⁷ although such designs may pose additional logical difficulties that need careful consideration.

In the PEPTIC trial, while most of the key data were obtained from registries, some data were collected at an individual patient level. Using a combination of registry and individual patient data made it relatively easy to conduct the trial in multiple countries. Using registry data sources greatly reduced costs compared with collecting trial-specific data at an individual patient level. The trial was conducted with less than $500 000 of funding, a cost of under $20 per patient. Such comparatively low cost means that this trial technology might allow future large scale trials to be conducted in low income countries where registries are rapidly developing.⁸ Even in higher income countries, registry-embedding potentially means sites that do not have resources for research coordinators can contribute, resulting in a greater generalisability of trial results.

Trade-offs inherent in the PEPTIC trial design

One limitation of the PEPTIC trial, which has received attention in trial commentary,⁹ is the amount of non-adherence with assigned treatment. Around 20% of patients admitted when an ICU was assigned to H₂RBs received PPIs. Such non-adherence confounds interpretation of the trial results with regard to the efficacy of individual drugs and may have been reduced if more resources had been devoted to educating staff in study centres about trial protocols and procedures. On the other hand, this may simply be an unavoidable trade-off with a cluster crossover trial that means these types of trials are best considered to be about the effectiveness of implementing particular treatment strategies rather than about the efficacy of the individual medicines being compared. In future trials of this nature, efforts to incorporate process evaluation may give a clearer idea of what drives non-adherence to ICU- assigned therapy. Such process evaluation is likely to be most important for common interventions where there may potentially be strongly held beliefs that drive practice. Collecting data on drivers of non-adherence may also enable compliance-adjusted statistical analyses that address estimates of efficacy of the individual medicine in addition to those of strategies.¹⁰

Despite the degree of non-adherence to assigned therapy that occurred, there was substantial separation in the exposure to drug classes between treatment groups, and randomisation provides a sound basis for determining that observed differences in outcomes were attributable to these differences in drug exposure. One common criticism of large scale pragmatic trials is that they fail to account for the individual differences in patient characteristics that clinicians might use at the bedside to inform decision making.¹¹ However, because the PEPTIC trial was so large, it provides a unique opportunity to explore the relative safety and effectiveness of PPI and H₂RB treatment strategies in different patient groups.

The rationale for exploratory analyses using the PEPTIC trial data

We have already seen that a strategy of using PPIs rather than H₂RBs in patients with high illness acuity,¹² who are at the greatest risk of clinically important upper gastrointestinal bleeding is associated with an increased risk of death. Other groups of potential interest include sepsis, chronic liver disease, trauma and burns, coagulopathy and bleeding, abdominal surgery, brain injuries, and renal failure. The rationale for conducting future exploratory analyses in these patient subgroups are shown in Table 2.

Table 2.

Rationale for exploratory analyses comparing proton pump inhibitors (PPIs) with histamine-2 receptor blockers (H₂RBs)

Subgroup	Rationale	Hypotheses
Sepsis	PPIs have been reported to exert a range of immunosuppressive effects13, 14 that could potentially increase the risk of death in patients with sepsis	The mortality risk associated with using PPIs instead of H2RBs will be greater in patients with sepsis than in patients without sepsis
Chronic liver disease	Patients with chronic liver disease may have a high risk of upper GI bleeding. If this risk is sufficiently high, the balance of observed risks may favour PPIs over H2RBs for this patient group	The mortality risk associated with using PPIs instead of H2RBs will be less in patients with liver disease than in patients without liver disease
Trauma	Patients who have suffered major trauma, including those with traumatic brain injuries and burns, may have a high risk of upper GI bleeding. If this risk is sufficiently high, the balance of observed risks may favour PPIs over H2RBs for this patient group. On the other hand, patients with burns are at very high risk of developing infections, and if PPIs increase the risk of dying from such infections, it might be particularly important to avoid them in this patient group	The mortality risk associated with using PPIs instead of H2RBs will be less in trauma patients (with or without traumatic brain injury) than in patients without trauma, except for patients with burns, where the mortality risk will be greater in PPI-group patients
Coagulopathy and bleeding	Patients who have coagulopathy or are at risk of coagulopathy because they are admitted to the ICU with haemorrhage may have a high risk of upper GI bleeding. If this risk is sufficiently high, the balance of observed risks may favour PPIs over H2RBs for this patient group	The mortality risk associated with using PPIs instead of H2RBs will be less in patients with bleeding and/or coagulopathy than in patients without bleeding and/or coagulopathy
Brain injuries	Patients with brain injuries may have a high risk of upper GI bleeding. If this risk is sufficiently high, the balance of observed risks may favour PPIs over H2RBs for this patient group	The mortality risk associated with using PPIs instead of H2RBs will be less in patients with brain injuries than in patients without brain injuries
Abdominal surgery	Patients who have major abdominal surgery are often not fed for a period after surgery. They may have a high risk of upper GI bleeding. If this risk is sufficiently high, the balance of observed risks may favour PPIs over H2RBs, particularly in elective surgical cases where the risk of perioperative mortality is relatively low	The mortality risk observed in patients who are admitted after elective major abdominal surgery will be extremely low, but the upper GI bleeding risk will be comparatively high. Consequently, the balance of risks will favour using PPIs instead of H2RBs for stress ulcer prophylaxis in this group
Renal failure	Patients with renal failure often have high illness acuity and patients with high illness acuity appear to have an increased risk of death when assigned to PPIs. On the other hand, patients with renal failure may have platelet dysfunction that predisposes them to developing upper GI bleeding	The mortality risk associated with using PPIs instead of H2RBs will be greater in patients with sepsis than in patients without sepsis, despite the fact that this group of patients will have a high risk of upper GI bleeding

GI = gastrointestinal; ICU = intensive care unit.

Conclusions

Despite some necessary trade-offs, the unique methodology used in the PEPTIC trial has many potential advantages compared with a conventional individual patient randomised controlled trial and may provide the impetus for future research activities. Particularly for ubiquitous ICU therapies, this trial design may prove attractive in the future. There is also a strong rationale for a number of exploratory analyses using PEPTIC trial data.

Competing interests

None declared.