Indirect (herd) effects of topical antibiotic prophylaxis and oral care versus non-antimicrobial methods increase mortality among ICU patients: realigning Cochrane review data to emulate a three-tier cluster randomised trial

Abstract

Objective

This study aimed to estimate the direct effects to recipients and indirect (herd) effects to non-recipients of each of topical antibiotic prophylaxis (TAP) and oral care methods on patient mortality within randomised concurrent controlled trials (RCCT) using Cochrane review data.

Design

Control and intervention groups from 209 RCCTs of TAP (tier 3), oral care (tier 2) each versus non-antimicrobial (tier 1) ventilator-associated pneumonia (VAP) prevention interventions arranged to emulate a three-tiered cluster randomised trial (CRT). Eligible RCCTs were those including ICU patients with >50% of patients receiving >24 hours of mechanical ventilation (MV) with mortality data available as abstracted in 13 Cochrane reviews.

Exposures

Direct and indirect exposures to either TAP or oral care within RCCTs versus non-antimicrobial VAP prevention interventions.

Main outcomes and measures

The ICU mortality within control and intervention groups, respectively, within RCCTs of either TAP or oral care versus that within non-antimicrobial VAP prevention RCCTs serving as benchmark.

Results

The ICU mortality was 23.9%, 23.0% and 20.3% for intervention groups and 28.7%, 25.5% and 19.5% for control groups of RCCTs of TAP (tier 1), oral care (tier 2) and non-antimicrobial (tier 3) methods of VAP prevention, respectively. In a random effects meta-regression including late mortality data and adjusting for group mean age, year of study publication and MV proportion, the direct effect of TAP and oral care versus non-antimicrobial methods were 1.04 (95% CI 0.78 to 1.30) and 1.1 (95% CI 0.77 to 1.43) whereas the indirect effects were 1.39 (95% CI 1.03 to 1.74) and 1.26 (95% CI 0.89 to 1.62), respectively.

Conclusions

Indirect (herd) effects from TAP and oral care methods on mortality are stronger than the direct effects as made apparent by the three-tiered CRT. These indirect effects, being harmful to concurrent control groups by increasing mortality, perversely inflate the appearance of benefit within RCCTs.

STRENGTHS AND LIMITATIONS OF THIS STUDY.

• The main strength of this analysis is that it estimates the indirect (herd) effects from topical antibiotic prophylaxis use to concurrent control group patients within the ICU context. This effect would be difficult and, being potentially harmful, unethical to study otherwise .

• Another strength is that the data, derived from 13 published systematic reviews, are traceable to the 209 individual randomised concurrent controlled trials (RCCT) with triangulation of summary findings using alternate cluster randomised trial (CRT) versus RCCT realignments of the data.

• Heterogeneity of the populations, interventions and publication dates is discussed as both a strength and a limitation in the generalisability of the results.

• A possible limitation is that while the analysis in the CRT realisation can be adjusted for group-level confounders, it cannot be adjusted for patient-level confounders.

• A clear limitation is that the generalisability of these findings to those study populations generally excluded from Cochrane reviews on the basis of having lower study quality scores is uncertain.

Introduction

Patients receiving prolonged (>24 hours) Intenstive Care Unit (ICU) stay are at high risk of acquiring various infections in the ICU including ventilator-associated pneumonia (VAP). An extensive range of infection prevention methods have been studied among patients receiving, or likely to receive, mechanical ventilation (MV) towards reducing mortality incidence.^1–13

Oropharyngeal applications of topical antibiotic prophylaxis (TAP), such as selective digestive decontamination (SDD) and selective oropharyngeal decontamination (SOD), appear highly effective towards preventing both VAP and mortality, with apparent reductions of ~50% for VAP and ~10%–25% for mortality.^{12 13} By contrast, oral care and other methods achieve modest or no significant reductions in VAP, and none demonstrate consistent mortality prevention.^1–11 Moreover, some interventions might increase the risk of VAP.

Infection prevention interventions, such as vaccination, which reduce transmission of infections among intervention recipients might indirectly protect concurrent non-recipients within randomised concurrent controlled trial (RCCT) populations. Estimating this herd (indirect) protection from vaccine interventions requires cluster randomised trials (CRT) where the event rates of non-recipients within vaccine exposed cohorts are compared with that of non-recipients within cohorts exposed to ineffective or neutral interventions.^14–16 Demonstrating the population efficacy and safety intervention requires a CRT. In a novel approach, the herd effects of cholera vaccination were estimated by post hoc reanalysis of the control and interventions groups within multiple cholera vaccine RCCTs rearranged to emulate a single CRT.¹⁷

The objective here is to emulate a three-tiered CRT to estimate the direct and indirect (herd) effects of either TAP (tier 3) or oral care (tier 2) on ICU mortality among the control and intervention groups within RCCTs. The control and intervention groups of RCCTs of various non-antimicrobial-based VAP prevention interventions serve as tier 1 (benchmark) interventions. The study concept and plan are outlined in figure 1.

Figure 1.

Analysis of alternate realisations of Cochrane review data arranged as two tiers of interventions (third tier not shown). On the left, mortality incidence data as realised as individual randomised concurrent controlled trials (RCCTs) within meta-analyses presented within individual Cochrane reviews or on the right, the same data as realised as a single cluster randomised trial (CRT). On the left, are six hypothetical RCCTs, three RCCTs of blue (top, tier 1, ICUs 1–3) and three RCCTs of red (bottom, tier 2, ICUs 4–6) interventions to prevent ICU infections. Eligible patients (small ● and ○) within each RCCT (oval) are randomly assigned to receive either the study intervention (●) or standard care (○) within the ICU of each study. Each RCCT represents a single ICU. The study-specific effect size (ES) estimate (|---|, under each oval) from each RCCT is combined to provide overall summary estimates (↔) derived by meta-analysis for red (tier 2) and blue (tier 1) interventions, respectively. The group (ICU) level incidences for patients receiving the study intervention versus standard care within each RCCT might be noted but are generally immaterial in this realisation. To be valid as an estimate of the patient-level effect of the intervention of interest requires an assumption that this intervention has no indirect (herd) effect on the control group patients. This assumption is testable within a CRT. On the right are the same end points, patients and ICUs realised as a single CRT. Eligible patients within ICUs (clusters) have received either a benchmark intervention (blue ●; tier 1) or the intervention of interest (red ●; tier 2). A benchmark intervention is one that is regarded as ineffective against the end point of interest. The patients receiving standard care (○) in each ICU are contextually exposed to an ICU environment in which either red or blue interventions are in use. That the red and blue interventions have been randomly assigned in each ICU, and that each ICU might provide an estimate of effect size for the red or blue interventions, is immaterial in this realisation. The group (cluster, ICU) level incidences for patients within each ICU receiving the study intervention (mid-size ● under each oval) or standard care (mid-size ○ under each oval) are used to derive summary-level incidence estimates for populations receiving either a red or blue intervention or standard care in either a red or blue ICU, respectively. In this realisation as a CRT, the difference between the overall incidences for the intervention populations (large red vs blue ● and solid vertical lines) are used in deriving estimates of the direct effect (DE) of the intervention of interest. The difference between the overall incidences for the standard care populations (large red vs blue ○ and dotted vertical lines) are used in deriving estimates of the indirect effect (IE) of the intervention of interest. The differences between the mortality incidence for intervention versus control groups in each respective study and tier are immaterial in this realisation. To be valid as estimates of the group-level direct and indirect effects of the intervention of interest requires an assumption that the benchmark (blue, tier 1) intervention has a negligible effect in comparison to the effect of the intervention (red, tier 2) of interest (notional nett effects). This assumption is testable within an RCCT. ICU is Intensive Care Unit.

Materials and methods

Study plan

The study plan here follows that of a recent umbrella review with three modifications.¹⁸ First, the data used here are limited to that abstracted from primary studies as listed within Cochrane reviews of TAP, oral care and various other VAP prevention interventions among ICU patients for whom the majority receive MV. These Cochrane reviews are an objective and transparent source of study data abstracted according to uniform methodology from studies meeting specified criteria for inclusion and quality scoring. The Cochrane reviews provide the data arrayed in the RCCT format towards deriving all estimates here.

Second, control and intervention group data from these RCCTs are rearrayed as clusters to simulate origin from a single three-tiered CRT of direct and indirect exposures to TAP (tier 3) and oral care (tier 2) methods each versus control and intervention groups of various non-antimicrobial (tier 1) ICU infection prevention interventions serving as the benchmark. In this array, the event rates of control groups (ie, non-recipient)] ICU patient groups within tier 3 and tier 2 RCCTs are compared versus control groups of tier 1 RCCTs to enable estimation of indirect effects of TAP and oral care, respectively. Likewise, the event rates of tier 3 and tier 2 RCCT intervention groups are compared with enable estimation of the direct effects of TAP and oral care, respectively, each versus the tier 1 intervention groups (figure 1).

The third modification is that the RCCT summary effect sizes and the group mean mortality incidences in different tiers are reconciled with those estimated in the various source documents to achieve a triangulation.

Study selection

Included studies were those providing mortality data for ICU patients towards mortality effect size estimates among 13 systematic reviews.^1–13 Cochrane reviews of interventions studied in specific patient populations such as studies of interventions for patients with the acute respiratory distress syndrome were excluded. Studies without mortality data and studies with <50% patients receiving MV were excluded. Studies reporting VAP incidence proportions were assumed to have >50% patients receiving MV if this was not explicitly stated. Studies limited to paediatric populations were not excluded except in sensitivity tests.

Categorisation of studies into tiers

The 13 Cochrane reviews were classified into three tiers of intervention. Tier 1 includes RCCTs of upper gastrointestinal tract,¹ feeding,² airway,^5–8 and probiotic⁹ interventions. Tiers 2 and 3 include RCCTs of oral care^{10 11} and TAP interventions,^{12 13} respectively.

Decant of groups

The data were extracted retaining the designation of control and intervention groups as assigned by the source Cochrane review. For multiarm studies and for studies comparing two alternate treatments, the designation as control or intervention group follows the listing associated with the first effect size estimate appearing within the respective systematic review with each control or intervention group appearing only once in the analysis here. Among TAP-RCCTs are duplex studies, where the control groups receive protocolised parenteral antibiotic prophylaxis (PPAP) without TAP. The designation of these duplex groups as control groups as in both the original RCCTs and the Cochrane reviews is retained here.

Outcomes measures

The VAP or ICU mortality incidences are the proportion of patients with VAP or ICU mortality, respectively, per 100 patients as abstracted in each Cochrane review. Late mortality is here defined as mortality censored beyond 21 days or labelled as hospital mortality. The following were also extracted where available; whether <90% (an arbitrary threshold) of patients received MV, the group mean (or median) age and year of study publication.

The study quality scores, as scored in each Cochrane reviews using a variable number of score elements, were harmonised here by binary conversion to majority or minority scores of the various quality elements.

The VAP incidences and RCCT prevention effect sizes are reported here for reference and are derived as for the mortality incidences.

Mortality prevention effect estimates from RCCT array

With the data in the original RCCT array as within each Cochrane reviews, the summary effects were derived for each category and subcategory of intervention. These study specific and overall summary effect sizes and associated 95% CI were derived as caterpillar plots¹⁹ using the DerSimonian-Laird random-effect methods of meta-analysis using the ‘meta’ command in Stata V.17 (Stata).

Meta-regression and mortality prevention effect estimates from CRT array

With the data in the CRT array, the mortality incidence was derived for control and intervention groups of each of tiers 1, 2 and 3 RCCTs from the meta-regression models as adjusted and unadjusted mortality log odds. The direct and indirect effect of tiers 3 and 2 each vs tier 1 interventions were derived as ORs from non-linear combinations of log odds estimates derived in the meta-regression models using the Stata command ‘nlcom’. The Stata script is provided in online supplemental file 1.

Four sensitivity tests of the findings were undertaken. The meta-regression was repeated after excluding studies limited to paediatric populations, studies with <40 patients overall, and studies which reported zero mortality. As an additional sensitivity test, the probiotic interventions, which conceivably could have indirect effects, were reclassified as tier 2 interventions.

Adjusted analysis

In deriving a causal inference, a study design not requiring an adjusted analysis would be preferable. The issue of whether to adjust for cross-level bias (ie, adjusting for imbalances at the patient level for analyses at the group level and vice versa) confronted an earlier CRT.²⁰ These authors consulted a panel of experts who advised that such an adjusted analysis would be valid. An analysis adjusted for cross level bias, as undertaken there,²⁰ is also undertaken here as a sensitivity test with the meta-regression repeated with adjustment for proportion receiving MV, quality score, study publication year and group mean age. These study-level variables were selected as they could conceivably influence ICU mortality and were readily identifiable in the systematic reviews.

Patient and public involvement

Patients or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

Results

Characteristics of the studies

There were 317 studies listed in 13 Cochrane reviews^1–13 of which 108 studies, either without mortality data or with <50% of patients receiving MV, were excluded leaving 209 studies with mortality proportion data available as either ICU mortality data or, for 49 studies, as late mortality data. The pneumonia and mortality prevention effect sizes and incidence proportions for each study are presented as raw data (electronic supplementary material: ESM online supplemental tables S1–S6), as figures (online supplemental figures S1–S31S32) and in summary form (online supplemental tables S7 and S8). Most studies were published between 1990 and 2010 and a minority contained patient groups for which <90% received MV. Only six studies were limited to paediatric ICU populations (table 1). The groups of 10 multiarm studies, with groups variably appearing as either control or intervention groups in multiple effect size estimates, appear in the analysis here only once.

Table 1.

Characteristics of studies

Characteristics	Tier 1				Tier 2	Tier 3
	UGIT	Feeding	Airway	Probiotic	Oral care	Antibiotic
Review characteristics
No of reviews	1	3	4	1	2	2
Inclusion criteria*
RCCT	+	+	+	+	+	+
Quasi-RCCT	+	+	Excl	Excl	Excl	Excl
ICU	+	+	+	+	+	+
MV	±	±	+	+	+	+
Exclusions*	GI bleed	Pancreatitis paediatric	paediatric	paediatric	various	paediatric
Number of listed studies†	121	46	61	8	40	41
Ineligible†	48	7	38	2	11	0
Listing (ESM table)	S1	S2	S3	S4	S5	S6
Study characteristics
eligible †	73	39	23	6	29	41
MV for >48 hours for <90%‡	41	20	0	0	0	7
Majority quality score §	62	15	6	2	23	29
Paediatric ICUs	2	–	–	–	4	–
Late mortality census ¶	9	18	5	4	9	7
Study publication year (range)	1978–2017	1983–2016	1987–2012	2006–2010	1996–2019	1987–2014
Control group characteristics
No of patients	5218	5017	2639	635	2156	4198
No of groups**	84	41	25	9	32	41
Length of stay, mean, days	9.7	15.8	11.0	15.0	14.7	14.8
95% CI	4.8 to 15	11.9 to 19.7	7.4 to 14.6	10.2 to 19.8	11.6 to 17.8	12.4 to 17.1
Control group characteristics
APACHE II§§ 95% CI (n)	17.6 16.4 to 18.9 (29)	18.9 17.5 to 20.4 (23)	20.8 18.9 to 22.6 (14)	19.9 17.3 to 22.5 (7)	19.7 17.7 to 21.6 (13)	17.8 16.5 to 19.1 (28)
Group mean age, years	50.8	48.6	56	56.4	53.0	51.1
95% CI (n)	48.6 to 53.0 (89)	44.2 to 53.0 (38)	51.1 to 62.3 (23)	47.4 to 65.4 (9)	49.2 to 56.9 (28)	46.8 to 55.5 (38)
group size median (IQR)††	43 25–67	30 20–51	50 33–120	71 36–87	50 31–94	52 30–130
Intervention groups characteristics
No of patients	5482	5071	2675	592	2301	6393
No of groups**	87	42	25	9	31	44
Length of stay	9.9	15.8	11.0	15.0	14.7	14.8
95% CI	4.8 to 15	11.9 to 19.7	7.4 to 14.6	10.2 to 19.8	11.6 to 17.8	12.4 to 17.1
APACHE II§§	17.6	18.9	20.8	19.9	19.7	17.8
95% CI (n)	16.4 to 18.9 (29)	17.5 to 20.4 (23)	18.9 to 22.6 (14)	17.3 to 22.5 (7)	17.7 to 21.6 (13)	16.5 to 19.1 (28)
Group mean age, years	50.8	48.6	56	56.4	53.0	51.1
95% CI (n)	48.6 to 53.0 (89)	44.2 to 53.0 (38)	51.1 to 62.3 (23)	47.4 to 65.4 (9)	49.2 to 56.9 (28)	46.8 to 55.5 (38)
Group size median (IQR)††	50 25–71	33 19–50	54 29–123	68 26–78	50 30–105	52 31–130
VAP prevention effect‡‡ (OR; 95% CI; n)	1.16; 1.02 to 1.32 (51) Online supplemental figure S1	0.77; 0.53 to 1.12 (20) Online supplemental figure S2	0.80; 0.67 to 0.96 (21) Online supplemental figures S3 and S4	0.63; 0.36 to 1.11 (6) Online supplemental figure S5	0.44; 0.32 to 0.60 (32) Online supplemental figure S6	0.35; 0.26 to 0.45 (38) Online supplemental figure S7

*Inclusion (+) and exclusion criteria as specified in the original systematic reviews±means not listed as an inclusion or exclusion criteria.

†Number of studies listed in the original Cochrane review, the number ineligible and eligible are the numbers not meeting or meeting, respectively, inclusion criteria for this analysis.

‡Studies for which less than 90% of patients were reported to receive >48 hours of MV.

§Majority quality score derived as meeting the majority of quality score criteria in the original Cochrane review.

¶Late mortality is either hospital mortality or mortality census beyond day 21.

**Note, several studies had several control or intervention groups. Hence, the number of groups does not equal the number of studies.

††Data are median and IQR.

‡‡Summary effect size derived by meta-analysis for VAP prevention effect for studies meeting inclusion criteria aggregated across all interventions in the category. These are contrasted with the summary effect sizes for all studies as listed in the Cochrane reviews in online supplemental table S7.

§§APACHE II is Acute Physiology and Chronic Health Evaluation II.

ESM, electronic supplementary material; MV, mechanical ventilation; UGIT, upper gastrointestinal tract; VAP, ventilator-associated pneumonia.

Length of stay and APACHE II score were available for only a minority of groups whereas group mean age was available for nearly all groups. Among control groups, the group mean age among the studies of adult ICUs and the length of stay and APACHE II score were similar across the six categories of studies. The studies of TAP interventions were more commonly awarded a majority quality score than were studies in other intervention categories.

Mortality incidence proportions

The control (figure 2, online supplemental figure S31, S32) and intervention (online supplemental figure S31, S32) group ICU mortality incidence proportion benchmarks, derived from the four categories of non-antimicrobial (tier 1) studies, was 19.5% (95% CI 17.4% to 21.6%) and 20.3% (95% CI 18.2% to 22.6%), respectively. Paradoxically, the mean ICU mortality incidence proportion for the oral care (tier 2) and TAP (tier 3) intervention groups, being 23.0% (95% CI 17.9% to 28.7%) and 23.9% (95% CI 19.5% to 28.7%), respectively, were each more similar to these benchmarks than were the mean ICU mortality incidence proportion for the oral care and the TAP control groups. The latter were each more than five percentage points higher (25.5%; 20.8–31.2 and 28.7%; 24.1–33.9, respectively). The mean late mortality incidences for the control and intervention groups for the TAP intervention category showed similar discrepancies as with the respective control or intervention ICU mortality benchmarks.

Figure 2.

The ICU (left) and late (right) mortality incidence versus respective benchmarks (central vertical line) for the control groups from six categories of studies of VAP prevention methods drawn from 13 Cochrane reviews. For each category the 95% confidence limits (horizontal error bars) associated with the summary incidence are shown. In each panel, the central vertical line is the benchmark being the summary mean derived from the control groups of the four categories of tier 1 studies (UGIT, Feeding, Airway & Probiotic) other than oral care and TAP (=antibiotic). The intervention group mortality incidences are shown as online supplemental figure S30S31 & S32 and the figures are shown with the individual studies identified as online supplemental figure S31 S32. TAP, topical antibiotic prophylaxis; UGIT, upper gastrointestinal tract; VAP, ventilator-associated pneumonia. ICU, Intensive Care Unit.

The VAP incidence proportions for the control and intervention groups of all categories were each similar to the respective control or intervention VAP benchmarks (derived from tier 1 studies) with the exception of the control groups of oral care and TAP-RCCTs, which were each higher (online supplemental figure S30).

Analysis of RCCT array

The apparent mortality prevention effect of TAP was affirmed in the analysis of the data in the RCCT array (table 2).

Table 2.

Mortality effect size estimates derived from RCCT array

Intervention	ES	95% CI	n	Source (references, page numbers, listed in data supplement)
UGIT interventions
Any versus UC	1.1	0.9 to 1.34	19	(s210 p402)
	1.1	0.87 to 1.4	28	Online supplemental figure S8
H2RA versus PPI	0.96	0.78 to 1.19	12	(s210 p441)
	0.91	0.66 to 1.27	12	Online supplemental figure S9
H2RA versus antacids	1.01	0.66 to 1.55	11	(s210 p452)
	0.98	0.62 to 1.53	12	Online supplemental figure S10
H2RA versus sucralfate	1.1	0.95 to 1.24	21	(s210 p459)
	1.12	0.95 to 1.33	22	Online supplemental figure S11
Antacids versus sucralfate	1.15	0.93 to 1.4	11	(s210 p479)
	1.13	0.87 to 1.45	14	Online supplemental figure S12
Other versus other	NR
	0.97	0.7 to 1.14	16	Online supplemental figure S13
Aggregate	1.05	0.95 to 1.16	103
Feeding interventions
EN versus PN	1.19	0.8 to 1.77	6	(s211 p91)
	1.01	0.87 to 1.18	16	Online supplemental figure S14
EN versus EN+PN	0.99	0.84 to 1.16	5	(s211 p98)
	1.8	0.92 to 3.5	4	Online supplemental figure S15
EN versus delayed EN	1.0	0.16 to 6.4	1	(s212 p48)
	0.74	0.40 to 1.38	7	Online supplemental figure S16
PP versus gastric	1.03	0.83 to 1.29	11	(s213 p50)
	1.06	0.77 to 1.46	11	Online supplemental figure S17
Aggregate	1.05	0.92 to 1.19	35
Airway interventions
CTSS versus OTSS	1.02	0.84 to 1.23	5	(s214 p27)
	1.04	0.78 to 1.38	5	Online supplemental figure S18
HH versus HME	1.03	0.89 to 1.2	12	(s215 p63)
	1.05	0.86 to 1.30	12	Online supplemental figure S19
semirecumbent versus supine	NR			(s216)
	0.95	0.75 to 1.21	9	Online supplemental figure S20
Silver ETT	1.09	0.93 to 1.29	1	(s217 p22)
	1.13	0.9 to 1.42	1	Online supplemental figure S20
Aggregate	1.03	0.91 to 1.17	26
Probiotic interventions
	0.84	0.58 to 1.22	5	(s218 p30)
	0.81	0.62 to 1.06	9	Online supplemental figure S21
Intervention	ES	95% CI	n	source(ref & page number)
Oral care interventions
Chlorhexidine versus UC	1.09	0.96 to 1.23	14	(s219 & s220)
	1.13	0.93 to 1.38	12	Online supplemental figure S22 & S25
Toothbrushing	0.84	0.67 to 1.05	5	(s219 & s220)
	0.78	0.58 to 1.06	5	Online supplemental figure S23 & S25
Other versus other
	0.79	0.50 to 1.24	10	Online supplemental figure S24 & S25
Aggregate	0.97	0.83 to 1.14	33
Antibiotic interventions
TAP+PPAP versus UC	0.75	0.65 to 0.87	17	(s221 p53)
	0.84	0.73 to 0.96	18	(s222 p94)
	0.77	0.66 to 0.91	19	Online supplemental figure S25S26 & S29
TAP versus UC	0.97	0.79 to 1.2	13	(s221 p53)
	0.97	0.87 to 1.07	15	(s222 p97)
	0.99	0.78 to 1.26	15	Online supplemental figure S26S27 & S29
TAP+PPAP versus PPAP(=duplex)*	0.98	0.73 to 1.32	7	(s221 p53)
	0.92	0.72 to 1.18	7	(s222 p98)
	0.91	0.66 to 1.26	7	Online supplemental figure S27S28 & S29
Aggregate	0.86;	0.78 to 0.96	42

Source is either Cochrane review or online supplemental figures in this supplement.

The effect size estimates derived here are all ORs although in the source documents (210–222) these estimates are variously ORs or RRs. Those in bold achieved statistical significance.

The number of studies here differs from the equivalent in the Cochrane reviews as only those studies with mortality data in addition to pneumonia data have been included. The number of studies included in the generation of mortality effect sizes includes those reporting either or both of ICU and late mortality data towards generation of a single mortality effect size.

Duplex studies of antibiotic interventions were those in which the control group received PPAP.

Effect sizes significantly difference from null in bold.

Abbreviations; ICU is Intensive Care Unit, UC is Usual Care, H2RA is Histamine 2 receptor antagonists, PPI is Proton Pump Inhibitors, EN is Enteral nutrition, PN is Parenteral Nutrition, CTSS is Closed Tracheal Secretion Suction, OTSS is Open Tracheal Secretion Suction, HH is Heated Humidification, HME is Heat & Moisture Exchange, ETT is Endotracheal Tube.

*Duplex study is one where the study protocol provides PPAP to control group patients while providing both PPAP and TAP to intervention group patients.

CTSS, Closed Tracheal Secretion Suction; EN, Enteral nutrition; ES, effect size; ETT, Endotracheal Tube; HH, Heated Humidification; HME, Heat & Moisture Exchange; H2RA, Histamine 2 receptor antagonists; ICU, Intensive Care Unit; n, number of studies; NR, not reported; OTSS, Open Tracheal Secretion Suction; PN, Parenteral Nutrition; PPAP, protocolised parenteral antibiotic prophylaxis; PPI, Proton Pump Inhibitors; RR, risk ratio; TAP, topical antibiotic prophylaxis; UC, Usual Care; UGIT, upper gastrointestinal tract.

Analysis of CRT array

The higher ICU mortality (online supplemental figure S31) among control groups of TAP-RCCTs remained evident in meta-regression models (online supplemental table S9), with late mortality incidence proportion data included, either with adjustment for group-level parameters, such as majority study quality score, whether <90% of patients received MV, year of publication and group mean age (online supplemental figure S32) or without adjustment.

These meta-regression findings were robust to the exclusion of each of the following as sensitivity tests; paediatric studies, studies with fewer than 40 patients and groups with zero mortality. The findings were also robust to the reclassification of probiotic studies to tier 2 (data not shown).

Effect sizes

There was a striking difference in the effect sizes for TAP mortality prevention depending on whether the data were arrayed in the original RCCT format (table 2) or in the CRT format (table 3).

Table 3.

Mortality effect size estimates derived from simulated CRT array*

	OR	Oral care (tier 2)	OR	TAP (tier 3)
		95% CI		95% CI
CRT estimates (unadjusted)†
Direct effect	1.11	0.78 to 1.43	1.25	0.94 to 1.57
Indirect effect	1.28	0.92 to 1.64	1.64	1.23 to 2.05
CRT estimates (adjusted)‡
Direct effect	1.10	0.77 to 1.43	1.04	0.78 to 1.30
Indirect effect	1.26	0.89 to 1.62	1.39	1.03 to 1.74

*These effect size estimates are derived using the RCCTs of the tier 1 interventions as a benchmark group.

†CRT estimates (unadjusted estimates) come from online supplemental table S9 model 1. This model includes 460 groups from the 209 studies whether reporting ICU (Intensive Care Unit) or late mortality.

‡CRT estimates (adjusted estimates) come from online supplemental table S9 model 2 and are adjusted for group mean age, year of study publication, proportion receiving MV>90% and majority quality score. This model includes 428 groups from the 209 studies whether reporting ICU or late mortality.

CRT, cluster randomised trial; ICU, Intensive Care Unit; MV, mechanical ventilation; RCCT, randomised concurrent controlled trial.

With the data arrayed in the RCCT format, only the TAP-based interventions showed a significant mortality (table 2) or VAP prevention effect (online supplemental table S7).

With the mortality data arrayed in the CRT format, no mortality prevention effect was apparent for either oral care or TAP interventions as a direct effect (table 3). By contrast, there was a stronger indirect effect of TAP-based interventions evident which equates to an increase in mortality.

Triangulation

The effect sizes as derived here in the RCCT format for all six categories align with the effect sizes as derived within the respective Cochrane reviews (table 2, online supplemental table S7). The mean mortality incidences as derived here in the CRT format for all six categories (figures 2 and online supplemental figures S2 and S3S31) align with the mortality incidences as reported in the respective Cochrane reviews (online supplemental table S8).

Discussion

Data derived from 13 Cochrane reviews are arranged here in three tiers and analysed in alternate arrays (figure 1). The array emulating a CRT enables estimates of the direct and indirect effects of each of oral care (tier 2) and TAP (tier 3) interventions versus non-antibiotic interventions (tier 1) on ICU mortality incidence among MV patients in the ICU context. The findings here reaffirm and reconcile four otherwise paradoxical existing findings and raise one new finding, all within the same dataset.

First, with the data in the original RCCT array, the apparent mortality prevention effect from TAP and the lack of mortality benefit from either tier 1 or tier 2 interventions are recapitulated. The effect size estimates derived here accord with those derived within the respective Cochrane reviews (table 2, online supplemental tables S7 and S8). Of note, the interventions other than TAP include those, such as enteral feeding and gastric acid suppression, which potentially increase colonisation and VAP risk.^1–4 A borderline increased mortality from oral chlorhexidine interventions noted here (table 2) accords with that noted previously.²¹

Second, with the data arrayed to emulate a CRT, mortality prevention as direct effects of either TAP or oral care interventions are again not evident with the estimates derived here in accord with those of five large CRTs (each >3000 patients and each >6 ICUs) of TAP, as either SOD or as SDD, and topical chlorhexidine.^{20 22–26}

Of five large CRTs, one found similar overall mortality within ICUs randomised to TAP as either SOD or as SDD versus standard care with significant differences emerging after adjusting for differences in underlying patient mortality risk.²⁰ The second found lower mortality associated with SDD versus SOD but lacked standard care control groups.²² Reanalysis of this study²² revealed a strong and lasting ecological effect of the TAP intervention.²³ The third found no reduction, with any of SDD, SOD or topical chlorhexidine, in either bacteraemia or mortality even in adjusted analyses.²⁴ The fourth, and largest, found a non-significant <2 percentage point difference with TAP intervention.²⁵ The fifth, a stepped wedge CRT of oral chlorhexidine deadoption, found no mortality difference following deadoption.²⁶ Within each of these CRTs, the ICU mortality differences were each less than two percentage points in unadjusted analyses.

The high ICU mortality among control groups of TAP-RCCTs and the absence of an unusually low TAP intervention group ICU mortality is again noted and accords to findings in the umbrella review of ICU mortality proportions among VAP prevention interventions among 206 included studies,¹⁸ only 96 of which are included here. In the triangulation, these two findings are also incidentally noted among the median mortalities of the control and the interventions groups, as summarised within each Cochrane review (which here are abstracted in online supplemental table S8). The median mortality is 303 per 1000 patients among TAP concurrent control groups vs 255 per 1000 patients for intervention groups receiving prophylaxis with TAP and PPAP.¹³ By contrast, the median mortality in the Cochrane review overview summaries of control and intervention groups of all other interventions are all less than 280 per 1000 patients.^1–11 The mean ICU mortality among 14 control groups from studies of TAP with a non-concurrent design (which are included neither here nor in the Cochran reviews) is 23.5 (19.3–28.3).¹⁸

The new finding arising here is that mortality prevention as an indirect (herd) effect is not evident for either TAP or oral care. On the contrary, the ORs being greater than one, together with the unusually high ICU mortality incidence among control groups of tier 3 (TAP) RCCTs versus a benchmark derived from tier 1 control groups, which remains evident in meta-regression models (online supplemental table S9) adjusting for group-level indices of underlying patient risk such as group mean age (online supplemental figure S32), suggests that the indirect effect of TAP towards non-recipient patients within RCCT studies is harmful and increases mortality. Likewise, but less so, for oral care (tier 2) interventions.

The need to evaluate the overall population effect of TAP is widely acknowledged given the potential ecological changes from TAP use in the ICU.²³ However, none of the published CRTs have been designed to estimate the indirect effects of TAP. Moreover, indirect (herd) effects would not be observable within either individual RCCTs nor the systematic reviews which include them as they rely on an assumption of their absence in deriving the RCCT effect estimates.

Undertaking a CRT to evaluate the indirect (herd) effect of TAP would be difficult due to the logistical and ethical considerations involved. The most recent²⁷ required >15 000 patients in >19 ICUs in three countries for >80% power to detect a 3.5% absolute risk reduction as a direct effect of TAP. A CRT to prove the population safety of TAP would require >15 000 patients in >100 ICUs studied for >2 years to obtain sufficient statistical power to find a two-percentage point mortality difference as an indirect (herd) effect. It would also be ethically challenging to undertake such a CRT with eligible ICU patients randomly exposed to either TAP intervention versus possible harm. Moreover, the equipoise for TAP benefit versus risk is contested.^{28 29}

Limitations

There is considerable heterogeneity in the interventions, populations and study designs among studies published over several decades included in the analysis here. The characteristics of the patient populations of the RCCTs included within the 13 Cochrane reviews appear broadly similar across categories with respect to several parameters and the results of all 13 Cochrane reviews would presumably be generalisable to the MV patient population generally. However, the groups within the emulated CRT here have not been cluster randomised.

The true indirect effect of TAP in the ICU context may have been underestimated here. The Cochrane review of TAP includes two^{20 30} studies with control groups that, by being non-concurrent, would not have been exposed to an indirect effect of TAP, and seven within duplex studies, with control groups which, by receiving PPAP, might have experienced an attenuated indirect effect.

The impact of competing risks, such as ICU discharge, towards estimating ICU mortality, is uncertain. Hospital-acquired infections are up to 50% more common among patients discharged after receiving either SDD or SOD during ICU stay versus patients discharged following standard care.³¹ Concerns regarding rebound infections following cessation of enteral antibiotic prophylaxis led to selective decontamination being abandoned among haemato-oncology units in the 1980s.^{32 33} Further exploration of the possible influence of rebound would require more late mortality data than is available here.

Finally, the possibility of publication bias or unmeasured confounder bias remains to be excluded. However, this unmeasured confounder would need to be uniformly distributed among groups of TAP-RCCTs to account for both the unusually high control group mortality incidence together with the absence of unusually low mortality incidence among TAP intervention groups. Postulating such an unmeasured confounder would seriously undermine the credible generalisability of the RCCTs findings from tier 2 and tier 3 interventions.

Of note, the findings of CRTs, in contrast to RCCTs, inherently relate to the group rather than the patient level of analysis. Indeed, a strength of this analysis is that it enables the estimation of the indirect (herd) effects or tier 2 and tier 3 interventions within this population which are not observable at the patient level. The findings were robust to adjustment for key group-level factors such as group mean age and proportion receiving MV as indicators of group-level mortality risk. The data and the group characteristics for the clusters that underpin the meta-regression models are traceable in the ESM.

The VAP incidences here have been provided here to substantiate the relative ineffectiveness of non-antibiotic (tier 1) interventions. Of note, the differences in VAP incidence among the tier 2 and tier 3 interventions would be insufficient to account for the mortality differences as the attributable mortality of VAP is estimated to be <35%.^{34 35} It is generally presumed that the protective effects of TAP, in the various formulations, are mediated through the control of gut overgrowth, the dysbiosis that underlies invasive infections and sepsis among ICU patients.^{36 37} There are profound and paradoxical differences in control group incidences of bacteraemia and candidaemia and other infection end points within this literature, which align with the control group mortality differences seen here.^38–41

Indirect (herd) effects are to be expected within infection prevention programmes such as population vaccination programmes to prevent influenza,⁴² cholera¹⁷ and typhoid,¹⁴ and mass antibiotic distribution programmes such as used for trachoma prevention⁴³ within populations. Of note, herd effects have been estimated for these scenarios, whereas to date, not for the TAP interventions, despite the expectation stated in the first study of TAP that indirect effects would occur.⁴⁴

Addendum

One large CRT²⁵ and one updated systematic review⁴⁵ of TAP have been published in the 18 months while this manuscript has been under review. The summary result of this systematic review also found a discrepancy between the mortality prevention effect of TAP observed within studies with an RCCT design, with a summary 15% difference in mortality, versus a CRT design, with a summary 0% difference in mortality. The summary difference in bacteraemia incidence between studies with an RCCT versus a CRT design are similarly discrepant.⁴⁶ Moreover, an analysis of the secular trend in underlying risk of ICU patients in 210 studies published over three decades, of which only 90 appear in the analysis here, fails to account for the high ICU mortality among the control groups of TAP-RCCTs.⁴⁷ The rationale and design for a postulated CRT (The SHEET trial) to demonstrate the population safety of TAP has been published.⁴⁸ The postulated SHEET trial will never be undertaken for two reasons. First, SHEET would be logistically complex and second, it would be unethical to randomise patients to possible harm.⁴⁸ Length of stay behaves differently as a confounder of study effect size for the non-antimicrobial versus antimicrobial Cochrane review RCCT’s. ⁴⁹

Conclusion

Data as abstracted within 13 Cochrane reviews have been arranged to simulate a three-tier CRT to estimate indirect (herd) effects for TAP and oral care methods, each versus non-antimicrobial based methods of infection prevention, on mortality of ICU patients. The control group mortality proportions that underlie the strong summary TAP intervention effect size of TAP-RCCTs among ICU patients are unusually high whereas the intervention group mortality proportions within these TAP-RCCTs are not unusually low. These indirect effects were foreseen in the first study of SDD⁴⁴ but have never been estimated. They remain stronger than the direct effects in the adjusted analysis. These paradoxical observations raise concern that the indirect (herd) effects from TAP, and less so oral care interventions, within individual RCCTs, are stronger than the direct effects and become apparent only within the three-tiered CRT realisation of the Cochrane review data. Moreover, these indirect effects are harmful to and increase mortality among concurrent control group patients. This, perversely, inflates the appearance of overall benefit within RCCTs.

Data availability statement

The data used in this analysis are available in the online supplement as extracted from databases published in references ^(1–13). No additional data are available.

Ethics statements

Patient consent for publication

Not applicable.

Ethics approval

Ethics committee review of this study was not required as this is an analysis of published work and each individual included study had been approved by an ethics vote.