Economic evaluation alongside the Probiotics to Prevent Severe Pneumonia and Endotracheal Colonization Trial (E-PROSPECT): study protocol

Abstract

Introduction

Ventilator-associated pneumonia (VAP) is a common healthcare-associated infection in the intensive care unit (ICU). Probiotics are defined as live microorganisms that may confer health benefits when ingested. Prior randomised trials suggest that probiotics may prevent infections such as VAP and Clostridioides difficile–associated diarrhoea (CDAD). PROSPECT (Probiotics to Prevent Severe Pneumonia and Endotracheal Colonization Trial) is a multicentre, double-blinded, randomised controlled trial comparing the efficacy of the probiotic Lactobacillus rhamnosus GG with usual care versus usual care without probiotics in preventing VAP and other clinically important outcomes in critically ill patients admitted to the ICU.

Methods and analysis

The objective of E-PROSPECT is to determine the incremental cost-effectiveness of L. rhamnosus GG plus usual care versus usual care without probiotics in critically ill patients. E-PROSPECT will be performed from the public healthcare payer’s perspective over a time horizon from ICU admission to hospital discharge.

We will determine probabilities of in-ICU and in-hospital events from all patients alongside PROSPECT. We will retrieve unit costs for each resource use item using jurisdiction-specific public databases, supplemented by individual site unit costs if such databases are unavailable. Direct costs will include medications, personnel costs, radiology/laboratory testing, operative/non-operative procedures and per-day hospital ‘hoteling’ costs not otherwise encompassed. The primary outcome is the incremental cost per VAP prevented between the two treatment groups. Other clinical events such as CDAD, antibiotic-associated diarrhoea and in-hospital mortality will be included as secondary outcomes. We will perform pre-specified subgroup analyses (medical/surgical/trauma; age; frailty status; antibiotic use; prevalent vs no prevalent pneumonia) and probabilistic sensitivity analyses for VAP, then generate confidence intervals using the non-parametric bootstrapping approach.

Ethics and dissemination

Study approval for E-PROSPECT was granted by the Hamilton Integrated Research Ethics Board of McMaster University on 29 July 2019. Informed consent was obtained from the patient or substitute decision-maker in PROSPECT. The findings of this study will be published in peer-reviewed journals.

Trial registration number

NCT01782755; Pre-results.

Strengths and limitations of this study.

• A priori study protocol with prospective clinical and economic data collection with representation from international jurisdictions.

• The balance of randomisation reduces risk of bias in the cost-effectiveness analysis occurring on patient level.

• A relatively short time horizon.

• Primary outcome of incremental cost to avoid a clinical event (cost-effectiveness approach), rather than a cost-utility approach (incremental cost per quality-adjusted life year).

Background

Context

Ventilator-associated pneumonia (VAP) is the most common healthcare-associated infection in the intensive care unit (ICU), resulting in a high burden of illness.^{1 2} A 2005 systematic review found a pooled cumulative VAP incidence of 23% (95% CI 19% to 27%) in randomised controlled trials (RCTs) and 10% (95% CI 7% to 13%) in observational studies.² In addition, VAP is associated with a twofold attributable risk of dying in the ICU (OR 2.02, 95% CI 1.2 to 3.6), and the cost attributed to VAP ranges from US$10 000 to $13 000 per patient.² Thus, VAP prevention is a patient-important safety goal during critical illness.^{1 3 4}

Probiotics are defined as “live microorganisms which, when administered in adequate amounts, may confer a potential health benefit on the host”.^{5 6} They are reported to enhance gut barrier function, reduce host pathogenic bacterial load, modify gut microbiota and modulate the immune system.^7–10 Probiotics studies suggest benefits including reduced incidence of healthcare-associated infections.^11–14 A recent meta-analysis of RCTs suggests that probiotics administered to critically ill mechanically ventilated patients were associated with a 26% lower VAP rate (95% CI 10% to 39%) and 20% lower infection rates overall (95% CI 5% to 32%).¹⁵ However, these findings arose from 30 small, mostly low-quality single-centre RCTs (n=18–300, 2972 total patients in the meta-analysis), yielding imprecise estimates and results with uncertain internal and external validity.¹⁵

Further, probiotics may reduce the incidence of diarrhoea, specifically Clostridioides difficile–associated diarrhoea (CDAD), which can cause serious complications such as pseudomembranous colitis, toxic megacolon and death.¹⁶ In a recent Cochrane systematic review and meta-analysis of 31 RCTs including 8672 patients who were receiving antibiotics and concurrent probiotics, moderate certainty evidence suggested that probiotics were effective at reducing the burden of CDAD for patients and the healthcare system.¹⁶

Current Knowledge

We recently performed a systematic review of economic evaluations examining probiotics in hospitalised patients, evaluating their cost-effectiveness for reducing VAP, CDAD and antibiotic-associated diarrhoea (AAD), while also identifying variables that could drive costs.¹⁷ From 721 potentially relevant studies, 7 met the eligibility criteria. Probiotics appear to be either cost-effective or cost-saving in six of seven studies compared with other prophylactic strategies within usual care to prevent healthcare-associated infection in acutely ill hospitalised patients. However, Grading of Recommendations Assessment, Development and Evaluation (GRADE) evaluations indicated a high risk of bias and very low quality/certainty of clinical evidence, such that cost-effectiveness evidence on the use of probiotics in adult hospitalised patients was weak. Furthermore, probiotic manufacturers funded three of seven (43%) studies, all of which were reported as either cost-effective or cost-saving.¹⁷ Some probiotic economic evaluations were designed after the results of the trial were published.

Study Aims

Therefore, we have designed this economic evaluation (E-PROSPECT) alongside the multicentre PROSPECT (ClinicalTrials.gov no. NCT01782755), assessing the incremental cost-effectiveness ratio (ICER) of probiotics versus usual care for critically ill adult patients.^18–20

Methods

Overview of prospect

PROSPECT is a randomised, double-blinded multicentre controlled trial. It used a central system for concealed 1:1 ratio to randomise patients (in variable unspecified block sizes, stratified by centre and by medical, surgical or trauma admission status) to either 1×10¹⁰ colony-forming units of Lactobacillus rhamnosus GG (iHealth, Inc.) or an identical placebo suspended in tap water administered twice daily via feeding tube in the ICU.²⁰ PROSPECT has enrolled 2653 critically ill patients between October 2013 and March 2019 throughout 44 ICUs (41 in Canada, 2 in the USA and 1 in Saudi Arabia). Patients, healthcare providers, investigators and research personnel were all blinded to group allocation. Sample size calculation has been previously described.^18–20

E-PROSPECT design

The primary objective of E-PROSPECT is to estimate the incremental cost per VAP prevented arising from a prevention strategy of using probiotics with usual care (the probiotics arm) versus usual care without probiotics (the usual care arm) during hospitalisation. Our secondary analyses of ICERs include healthcare-associated complications (CDAD, AAD) and mortality.^18–20

Our economic evaluation will be performed from the public healthcare payer’s perspective,²¹ over the time horizon of the ICU admission to hospital discharge or death (table 1). Our economic evaluation protocol was developed (table 1) according to established CHEERS (Consolidated Health Economic Evaluation Reporting Standards) and international cost-effectiveness analysis guidelines.^{22 23}

Table 1.

Summary of economic evaluation framework

Question	Is the use of probiotics as compared with standard care without probiotics cost-effective for the prevention of VAP and other clinically important outcomes in critically ill medical-surgical patients in PROSPECT?
Perspective	Public payer (in-hospital costs)
Setting	Ventilated ICU patients (44 centres, 3 countries: 41 Canada, 2 USA, 1 Saudi Arabia)
Comparators	Probiotics (Lactobacillus rhamnosus GG) with usual care vs usual care without probiotics
Time horizon	From ICU participant admission to hospital discharge/death (non-fixed time span)
Discount rate	No discounting (no long-term follow-up over 1 year)
Clinical outcomes	VAP, CDAD, AAD, length of stay and mortality (ICU and hospital)
Costs	Direct medical costs associated with treatment and complications (ICU and ward costs, personnel, medications, laboratory tests, diagnostic testing and procedures/surgeries)
Evaluation	Primary outcome: incremental cost-efficacy ratios (ICERs) per in-hospital VAP event avoided Secondary outcomes: ICERs for other clinically important outcomes: Incremental cost per CDAD avoided Incremental cost per AAD avoided Incremental cost per death avoided
Currency (price date)	US dollars (2019)
Uncertainty	Non-parametric bootstrapping to produce confidence intervals Cost sampling from various hospitals (stratified by location) Sensitivity analyses to deal with structural and methodological uncertainty

AAD, antibiotic associated diarrhoea; CDAD, Clostriodiodes difficile–associated diarrhoea; ICER, incremental cost-efficacy/effectiveness ratio; ICU, intensive care unit; PROSPECT, Probiotics: Prevention of Severe Pneumonia and Endotracheal Colonization Trial; VAP, ventilator-associated pneumonia.

Clinical outcomes

Clinical outcomes that will be examined in E-PROSPECT are described with definitions in online supplementary table 1 that were previously described from PROSPECT.²⁰ Clinical events such as VAP (primary outcome), CDAD, AAD and hospital mortality (secondary outcomes) will be gleaned from PROSPECT, with a statistical analysis methodology previously described.²⁰ For the dichotomous outcomes, we will use time-to-event analyses. HRs and associated 95% CIs will be estimated using a stratified Cox proportional-hazards model. For continuous outcomes, we will report estimates of the difference between intervention and control groups, 95% CIs and associated p values.²⁰

These dichotomous outcomes with proportions and continuous outcomes with point estimates (eg, length of stay, which will be used for calculation of resource utilisation) will be used to calculate both incremental costs (resource utilisation) and effects. Incremental effects will be defined as the difference in per-patient event rates or the difference in proportion of a clinical event (eg, VAP) between groups.

Health care resource utilisation

Based on our systematic literature review¹⁷ and published evidence,^18–20 we identified a list of relevant healthcare resource items that includes medications, physician/personnel utilisation, diagnostic radiology/laboratory testing, and operative/non-operative procedures and per-day hospital ‘hoteling’ costs not otherwise encompassed. Antimicrobial use in ICU will be defined as days of therapy, defined daily dose of therapy and antimicrobial-free days.^{24 25} Only systemic antimicrobials will be captured whether prophylactic or therapeutic in intent. Topical creams, eye/ear drops and inhaled antimicrobials will be excluded. We will also document the duration of mechanical ventilation, ICU and hospital length of stay and mortality. The healthcare resource uses will be collected alongside PROSPECT. For missing resource use data, we will choose appropriate imputation methods according to the type and distribution of the missing data.^{26 27} Otherwise, we will use an appropriate ‘standard dose’ for non-titratable medications (eg, chlorhexidine) and a clinically appropriate ‘medium dose’ for titratable medications (eg, vasopressors or inotropes).

Unit costs

Unit costs for healthcare resource items will be identified through jurisdiction-specific (regions/provinces/states which manage healthcare delivery in their area) public databases (eg, pharmacy drug formularies, physician billing schedule of benefits, Medicare/Medicaid reimbursement manuals, labour department wages/salaries, manufacturer costs). When there is a small sample or distribution of unit costs (ie, a provincial jurisdiction may have the same cost for a particular procedure), we will estimate the SE if possible, or incorporate a ±25% error around the mean unit cost distribution.

For unit costs not represented in public databases, we will obtain site-specific unit costs from the participating PROSPECT sites. We will first conduct a pilot study of unit cost acquisition at a convenience sample of nine participating centres (Canadian: British Columbia, Alberta, Manitoba, Ontario, Quebec, Nova Scotia; US: Minnesota, Missouri; and Saudi Arabia) to request a list of unit costs (online supplementary table 2: E-PROSPECT unit cost data extraction table). The site investigator or research co-ordinator will then contact the most appropriate individual in each hospital’s accounting, human resources, pharmacy, radiology or laboratory departments to obtain the unit costs.²⁸ In all cases, costs will be requested (if available). If only charges are known, then we will attempt to convert to costs by the institution’s cost-to-charge estimate for that item, where it exists.²⁸

Direct costs will be presented in the pre-specified cost categories (online supplementary table 2). Assumptions regarding resource utilisation are presented in online supplementary table 3. We will assess direct unit costs for study product–related resources associated with outcomes of VAP, CDAD, AAD and mortality. If a specific line-item unit cost is not attainable for a specific jurisdiction,²⁸ we will (1) ask another site within the same jurisdiction for missing unit costs and (2) derive a cost-ratio from acquired line-items (ie, drug costs both known in two jurisdictions), then using the cost-ratio impute the missing line-item unit costs for the missing jurisdiction (by multiplying the cost-ratio against a known jurisdiction’s acquired line-item to impute the line-item unit cost for the missing jurisdiction). (3) If line-item unit costs are still missing after multiple imputation (with missing variables), a mean unit cost approach will be used for the remaining jurisdictions which did report unit costs.

The pilot phase may inform amendments to our protocol. For example, if a unit cost for a particular line-item is deemed to be small and/or has a low clinical incidence rate, then that line-item may be removed from the final analysis. Items without a difference in clinical outcome/resource utilisation between intervention and control groups but which contribute substantially to costs may still be retained (even if little to no incremental difference in costs would exist between the two arms) in order to maintain face validity and accurately reflect the magnitude of costs for hospitalisation of a critically ill patient. Once the list of line-items has been pared down to those which are deemed to be cost drivers, and clinically relevant while also feasible to obtain, the remaining line-item list will be surveyed across a sampling of individual sites from each representative jurisdiction from PROSPECT.

Unit cost data will be summarised among all sites, and by country, to explore variability across centres and countries and to improve the generalisability of results. Visible outliers will be reconfirmed with individual hospital contacts. Participating sites will be queried to determine if particular costs have changed substantially (eg, by more than 25%), beyond inflationary or deflationary changes, over the course of the study. If there are substantial changes that have occurred over time, we will use the mean unit costs adjusted for inflation over the mean duration of the trial.²⁸

Cost analysis

The cost for each resource use item will be calculated by multiplying the natural resource utilisation units by the unit cost. The total cost per patient will be the sum of the cost of items used from the time of randomisation until discharge from hospital or death. The incremental mean cost will be estimated by calculating the difference in the total per-patient costs between the two groups. All costs will be converted to 2019 US dollars, accounting for annual inflation.^29–33

We plan on using international currency conversion, instead of purchase power parity (PPP)–based conversions, because health-specific PPPs are not available for all participating countries, and non-health PPP conversion rates vary substantially over the period of the analysis.³⁰ Country-specific costs will be considered only in sensitivity analyses.

Incremental costs will be calculated using the difference in mean per-patient cost between the two treatment arms. We have developed a costing operations manual outlining this process (online supplementary table 4: E-PROSPECT costing manual).³⁰

Cost-effectiveness analyses

Means (SD) or frequency (percentage) will be used to describe effect and cost estimates wherever appropriate. Chi-square tests and two-sample t-test comparisons will be used as appropriate to compare baseline characteristics between the two arms. The primary outcome will be based on the intention-to-treat principle and will form the clinical event estimates for the economic evaluation. Regression analyses may be performed if there is residual confounding, based on previously described methodology.²⁰

The incremental cost-effectiveness ratio (ICER) is the ratio of incremental costs per VAP prevented of probiotics versus usual care during the period of hospitalisation (from ICU admission to hospital discharge or death). The incremental mean costs will be estimated from all patients in both groups based on multiplying the resource unit cost by resource utilisation as described previously. The incremental mean effects will be derived from PROSPECT, where incremental effects were defined as the difference in per-patient event rates or the difference in proportion of a clinical event (eg, VAP) between groups.^{28 34} In secondary analysis, we will also calculate ICER using other clinical outcomes (ie, CDAD, AAD, mortality). If there is dominance in cost-effectiveness (ie, one treatment is better at lower cost than the other treatment), we will present the difference in cost and effect separately, without calculating the ICER for the base-case analysis. When there is no difference in clinical outcomes, we will present incremental cost and effects separately, without calculating an ICER for the base-case analysis.

Subgroup analyses

As subgroup analyses, we will investigate specific patients who may have differential effects and costs as compared with the entire population, including diagnostic category (medical, surgical, trauma)²; age <65 years, 65–75 years and >75 years^{35 36}; frailty status (baseline Clinical Frailty Score >5 of 9 vs 5)³⁷; patients who received/did not receive antibiotics within 2 days of randomisation²⁰; and prevalent (present at the time of enrolment) versus no prevalent pneumonia.²⁰

Uncertainty analyses

Because patient characteristics and costs may differ in different jurisdictions and outside clinical trial settings, and there will be uncertainty associated in the estimation of each group’s clinical outcomes and separately in the associated group’s costs, we have prospectively planned an uncertainty analysis to explore how ICERs may change with plausible ranges in costs of probiotics.

To test the robustness of our results (and determine the uncertainty associated with cost and effects estimation), we will perform a probabilistic sensitivity analysis of pairs of known costs and effects, using non-parametric bootstrapping techniques to generate 95% CIs. We will perform 1000 bootstrap simulations in the following manner: each simulation will draw the same number of patients per group (as per intention-to-treat), with replacement (for both events and cost) in pairs. For each sample, the difference in event rate and cost was calculated, obtaining 1000 pairs of differences in cost and event rate.^{38 39} Cost-effectiveness acceptability curves will be used to present the probability of probiotics being cost-effective over a wide range of willingness-to-pay thresholds.²¹

Scenario analyses will also be performed with variations of estimates of pairs of potentially influential variables (ie, costs of probiotics, per-day cost of care in ICU and hospital wards) across plausible ranges (variation of costs: 50%–150%) to explore potential cost differences in higher-spending and lower-spending healthcare jurisdictions to determine if different estimates change the overall results.

All analyses will be undertaken using Excel (Microsoft, Redmond, Washington, USA) and SAS (Cary, North Carolina, USA).

Patient and public involvement

Patients or the public were not involved in the development of the research question, design, or conduct, or reporting, or dissemination plans of our research. The burden of the intervention was not assessed in the patients themselves.

Ethics and dissemination

Research ethics approval for E-PROSPECT was granted by the Hamilton Integrated Research Ethics Board (HIREB) of McMaster University (project identifier: REB#:15-322). Informed consent was obtained from each participant in PROSPECT, or their substitute decision-maker, in accordance with local REB approvals. We anticipate that a majority of sites participating in E-PROSPECT will consider central HIREB approval as satisfactory to obtain additional non-specific patient-based costing data from their centre. All economic data, as with trial data, will be de-identified, maintained in a password-protected and encrypted laptop or desktop, in locked offices. All de-identified datasets, technical appendices and statistical code will be published alongside the economic evaluation. Knowledge translation of the results will be disseminated to patients, public and healthcare providers through peer-reviewed journals. The CHEERS checklist has been completed (online supplementary table 5).

Discussion

PROSPECT is the largest trial undertaken of probiotic usage for VAP prophylaxis in critically ill patients. Although probiotics have been shown in prior trials to prevent VAP and CDAD, their relative effects, side effects and cost-effectiveness remain uncertain. PROSPECT will determine whether probiotics reduce the frequency of VAP and other healthcare-associated complications during critical illness.^18–20

An economic evaluation jointly considers both costs and effects between alternative treatment options. Thus, physicians, administrators and policy-makers can know whether a new treatment provides good value for the healthcare expenditure. E-PROSPECT will answer these questions and address the cost-effectiveness of probiotics for VAP prevention. The literature currently has a paucity of health economic evaluations, illustrating the importance of E-PROSPECT.⁴⁰

Strengths and limitations

Some aspects of our methodology have potential limitations. First, the time horizon is relatively short, with no outpatient follow-up (only reporting in-hospital outcomes). Other studies have used relative, non-fixed time horizons in health economic evaluations,³⁴ including those investigating probiotics.^{41 42} We will carefully interpret these cost-effectiveness ratios in context from the short time horizon. Second, our primary outcome is the incremental cost to avoid a VAP event and other clinically important outcomes, not the incremental cost per quality-adjusted life year gained in a cost-utility analysis.²¹ PROSPECT is not designed to measure long-term outcome or downstream life expectancy (hence no lifetime time horizon). However, if PROSPECT shows a difference in hospital survival due to probiotics, this will be addressed as a secondary outcome. As with all efficacy trials, the generalisability and external validity of a health economic evaluation concurrently performed with an RCT may not represent the same treatment effects and costs as in routine clinical practice.

E-PROSPECT has several advantages.⁴³ First, we reduce the potential for investigator hypothesis-driven biases by pre-specifying our parameters of analysis (subgroup and sensitivity analysis) for the health economic evaluation prior to unblinding of the trial. Second, trial randomisation can reduce bias and confounding according to different baseline characteristics between study groups. Third, the concurrent collection of clinical and economic data can reduce the costs of data collection and minimise the possible problem of missing data if attempting to obtain it retrospectively. Fourth, we have chosen to gather costs from healthcare systems from multiple countries participating in the PROSPECT trial. We anticipate a wide variability in institutional reporting patient-specific cost accounting.^{28 34} Although this has the potential to introduce variability in cost estimates, this approach will also likely enhance the generalisability of our results. Finally, timely economic data can be useful to healthcare policy-makers to aid in resource allocation decisions. There are several clinician-researchers that are advocating for the embracing the science of value in healthcare,⁴⁴ while others state that cost-effectiveness analysis should be mandatory in clinical-effectiveness research to aid in clinical guideline development and public healthcare decision policy.⁴⁵ By conducting our economic analysis concurrent with the PROSPECT trial, we take advantage of each of these strengths.²⁸