The cost-effectiveness of early goal-directed therapy: an economic evaluation alongside the ARISE trial

Abstract

Objective: To determine the cost-effectiveness of early goal-directed therapy (EGDT) for patients with early septic shock.

Design: Within-trial cost-effectiveness evaluation.

Setting: Nineteen hospitals in Australia and New Zealand.

Participants and interventions: Patients with early septic shock enrolled in the Australasian Resuscitation in Sepsis Evaluation (ARISE) trial were randomly assigned to EGDT versus usual care. A subgroup of patients participated in a nested economic evaluation study in which detailed resource use data were collected until 12 months after randomisation.

Outcome measures: Clinical outcomes included lives saved, life-years gained and quality-adjusted life-years (QALYs), with mortality collected until 12 months and health-related quality of life assessed at baseline, 6 and 12 months using the 3-level EuroQol five dimensions questionnaire (EQ-5D-3L). Economic outcomes included health care resource use, costs and cost-effectiveness from the Australian health care payer perspective.

Results: A total of 205 patients (100 EGDT, 105 usual care) participated in the nested economic evaluation study, of which 203 had complete resource use data. Unadjusted mean health care costs to 12 months were $67 223 (standard deviation [SD], $72 397) in the EGDT group and $54 179 (SD, $61 980) in the usual care group, with a mean difference of $13 044 (95% CI, –$5791 to $31 878). There was no difference between groups with regards to lives saved (EGDT, 69.4% v usual care, 68.6%; P = 1.0), life-years gained (mean EGDT, 0.746 [SD, 0.406] v usual care, 0.725 [SD, 0.417]; P = 0.72) or QALYs (mean EGDT, 0.318 [SD, 0.291] v usual care, 0.367 [SD, 0.295]; P = 0.24). EGDT was dominated (higher costs, lower effectiveness) by usual care in 80.4% of bootstrap replications. For a willingness-to-pay threshold of $50 000 per QALY, the probability of EGDT being cost-effective was only 6.4%.

Conclusions: In patients presenting to the emergency department with early septic shock, EGDT compared with usual care was not cost-effective.

Clinical trial registration:ClinicalTrials.gov number NCT00975793.

Sepsis is recognised as a global health priority,¹ affecting millions of people worldwide annually.² The economic burden of sepsis is high, with total economic costs in Australia estimated to exceed $1.5 billion annually.³ The management of sepsis continues to be challenging and controversial, and understanding the effectiveness, costs and cost-effectiveness of sepsis interventions is essential.

The Australasian Resuscitation in Sepsis Evaluation (ARISE) trial was a multicentre, randomised controlled trial (RCT) of early goal-directed therapy (EGDT) compared with usual care in patients presenting to the emergency department (ED) with early septic shock. The trial found no difference in the primary outcome of all-cause mortality at 90 days⁴ and no difference in quality of life or mortality at 12 months after randomisation.⁵ However, interventions in which there is no difference in outcomes may still be cost-effective where there are differences in resource use between groups.

A recent systematic review of health economic evaluations for sepsis interventions concluded that high quality economics evaluations are needed to increase our understanding of the cost-effectiveness of interventions in clinical practice and to inform decision makers.⁶ While economic evaluations of EGDT have been conducted in a variety of countries including the United States, Spain, Brazil and the United Kingdom,7, 8, 9, 10, 11, 12 the results of economic evaluations conducted in different health care systems may not reflect the Australian experience due to significant differences between countries, such as funding mechanisms, treatment preferences and patient characteristics. This article presents the cost-effectiveness analysis for patients enrolled in the ARISE trial to determine the cost-effectiveness of EGDT in an Australian setting, as pre-specified in the statistical analysis plan.¹³ It is reported based on the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) guidelines.¹⁴

Methods

Study design and participants

ARISE was a prospective, multinational, randomised, parallel-group trial analysing the effects of EGDT compared with usual care in 1591 patients presenting to the ED with early septic shock. The ARISE RCT design, methodology and main results have been previously published,⁴ as have the long term mortality and quality of life outcomes.⁵ In brief, patients with early septic shock were randomised in a 1:1 ratio to receive EGDT or usual care for a 6-hour period. Participants were randomised between October 2008 and April 2014.

Due to site resource and funding constraints, economic outcomes were collected in a subgroup of Australian and New Zealand sites, which collected resource use data on a subset of patients randomised between April 2013 and April 2014. Consenting trial patients were enrolled consecutively until the sample size was reached to enable the number of patients recruited to be proportional to the total number of patients recruited to the larger study at each site. Ethics approval was obtained at all participating hospitals and written informed consent was obtained from the patient or their legal representative. The cost-effectiveness analysis was conducted from the Australian public health care system perspective using a within-trial time horizon of one year.

Clinical outcomes

Effectiveness was measured in terms of lives saved, life-years gained and quality-adjusted life-years (QALYs). Mortality was recorded at hospital discharge, 90 days, and 6 and 12 months using methods previously published.4, 5 Lives saved was calculated as the difference in the proportion of survivors at one year between the EGDT and usual care groups, while life-years gained was calculated as the difference in survival time to one year.

QALYs were calculated using utility scores derived from the 3-level EuroQol five dimensions questionnaire (EQ-5D-3L), administered at 6 and 12 months after randomisation, and valued using the UK time trade-off tariff.¹⁵ Patients were assumed to have a QALY score of zero at randomisation, in accordance with other studies recruiting critically ill patients.16, 17, 18 QALYs were calculated by determining the area under the curve, assuming a linear improvement/decline in quality of life from a baseline score of zero to the utility score at 6 months and a linear improvement/decline from 6 to 12 months. Patients who died while in hospital or before the 6-month follow-up were assigned 0 QALYs. QALYs for patients who died between 6 and 12 months were calculated assuming a linear decline from 6 months until the time of death. For patients missing a 6-month utility score, a linear improvement/decline from randomisation until 12 months was assumed.

Cost outcomes

Detailed daily resource use was collected for the index admission, including the number and type of pathology and radiology tests, the volume of fluids received, medications received, details of surgical and other procedures, line and catheter insertions, staffing consultations (eg, allied health), and patient location (ED, intensive care unit [ICU] or ward). Additional data were collected on post-discharge resource use to one year including readmissions, rehabilitation, long term care, and outpatient consultations.

Costs were determined by multiplying identified resources consumed by a unit cost for each resource. Where available, resource use was valued at the cost to representative hospitals participating in the ARISE RCT. For example, hospitals provided their consumable cost per item, and where not available or where not detailed enough, externally validated prices (eg, Medicare Benefits Schedule prices for tests where no internal unit cost was provided or available) were used. All costs are presented in 2014 Australian dollars (AUD). As the time horizon of the economic evaluation was one year, costs were not discounted. Full details on the methodology used to collect resource use and the unit costs are provided in the Online Appendix, appendix 2.

Cost-effectiveness

Unadjusted mean differences between the randomised groups were calculated for costs, lives saved, life-years gained and QALYs together with 95% confidence intervals (CIs). The incremental cost effectiveness ratios (ICERs) were calculated using the difference in mean total costs to one year between the EGDT and usual care groups divided by the difference in outcomes (lives saved, life-years gained and QALYs). To increase the robustness of the analysis, bootstrap procedures were conducted to estimate uncertainty around costs and effectiveness estimates, with 1000 replications, stratified by treatment group. The differences in average costs and QALYs between the randomised groups were used to calculate the incremental net benefit of EGDT versus usual care, with each QALY valued at a willingness-to-pay threshold of $50 000 per QALY, a commonly used threshold for assessing the cost-effectiveness of an intervention.¹⁹

Heterogeneity in cost-effectiveness was explored within subgroups defined by age, gender, severity of illness (Acute Physiology and Chronic Health Evaluation [APACHE] II score), and type of presentation (hypotension only, hyperlactataemia only, or both). Sensitivity analyses were conducted to assess the effect of various assumptions in the cost-effectiveness analyses. Further details on the methodology used in the sensitivity analysis are provided in the Online Appendix, appendix 2.

Statistical analysis

All data were analysed on an intention-to-treat basis and no assumptions were made for missing data. Data are presented as proportions for categorical data, and mean and standard deviation (SD) for continuous data. Cost data were presented as mean (SD) as the outcome of interest for economic assessment is the difference in the arithmetic mean cost (as the arithmetic mean cost is the important summary statistic from budgetary and social perspectives).²⁰ Comparisons between groups were conducted using χ² tests for binary outcomes and t tests for continuous data. While an assumption of parametric tests is that the data are normally distributed, and cost data may be anticipated to be skewed, in samples of similar sizes, the tests are robust to violations of normality.²⁰ Data were analysed using Stata v14.2 (StataCorp, USA). All tests were two-sided with a significance level of 5%.

Results

A total of 19 ARISE RCT sites agreed to participate in the economic evaluation substudy (Online Appendix, eTable 1). Detailed resource use data were available from 19 sites on 205 patients, of which 100 were randomly allocated to receive EGDT and 105 were randomly assigned to usual care. These patients were randomised between 16 April 2013 and 23 April 2014. Two patients had missing data for components of resource use (eg, inpatient consultations) and were excluded from the analysis.

Baseline characteristics by treatment group of patients participating in the economic analysis substudy are shown in Table 1. There were no significant differences between treatment groups for any of the baseline variables. Baseline characteristics and outcomes for patients who did and did not participate in the economic evaluation substudy are shown in the Online Appendix, eTable 2. There were no significant differences between those participating in the substudy and those not participating.

Table 1.

Baseline characteristics of costed patients by treatment group

	EGDT	Usual care	P
Total number of patients	98	105
Sex, male	59 (60.2%)	57 (54.3%)	0.48
Age (years), mean (SD)	62.7 (16.1)	64.0 (17.4)	0.58
APACHE II score, mean (SD)	15.4 (6.5)	16.3 (7.5)	0.39
Mechanical ventilation	6 (6.1%)	6 (5.7%)	1.00
Vasopressor infusion	16 (16.3%)	16 (15.2%)	0.85
Charlson comorbidity score (≥ 1)	63 (64.3%)	55 (52.4%)	0.09
Usual accommodation
Home	94 (95.9%)	100 (95.2%)	1.00
Long term facility	4 (4.1%)	5 (4.8%)

APACHE = Acute Physiology and Chronic Health Evaluation; EGDT = early goal-directed therapy; SD = standard deviation.

Resource use and costs

Table 2 shows the average resource use for the EGDT and usual care groups. While the EGDT group had higher post-discharge resource use, with higher readmission days, rehabilitation days and long term care days, none of these differences reached statistical significance (Online Appendix, eTable 3).

Table 2.

Resource use during the index admission and to one year

	EGDT	Usual care	P
Total number of patients	98	105
Index admission resource use
Duration of stay, mean (SD)
ED LOS, hours	4.38 (1.81)	5.00 (2.49)	0.04
ICU LOS, days	4.01 (4.07)	4.00 (4.66)	0.99
Ward LOS, days	12.35 (25.08)	11.55 (13.32)	0.77
Surgery	19 (19.4%)	30 (28.6%)	0.14
Invasive mechanical ventilation	26 (26.5%)	32 (30.5%)	0.64
Renal replacement therapy	11 (11.2%)	20 (19.1%)	0.17
ECMO	1 (1.0%)	0	0.48
Post-discharge resource use
Readmission	56 (57.1%)	51 (48.6%)	0.26
Readmission total LOS, mean (SD)	15.35 (25.82)	8.95 (20.28)	0.05
Rehabilitation	11 (11.2%)	13 (12.4%)	0.83
Rehabilitation LOS, mean (SD)	4.42 (16.02)	2.81 (9.08)	0.38
Long term care	10 (10.2%)	6 (5.7%)	0.30
Long term care LOS, mean (SD)	17.03 (68.63)	10.22 (54.72)	0.43

ECMO = extracorporeal membrane oxygenation; ED = emergency department; EGDT = early goal-directed therapy; ICU = intensive care unit; LOS = length of stay; SD = standard deviation.

The distribution of total costs to one year (Figure 1) had similar distributions for both treatment groups, with both groups having outliers with total costs to one year of over $200 000. Unadjusted mean costs were $67 223 (SD, $72 397) in the EGDT group and $54 179 (SD, $61 980) in the usual care group, a difference of $13 044 (95% CI, –$5791 to $31 878; P = 0.17) (Table 3). The majority of the difference in costs was due to higher post-discharge costs, but the difference was not statistically significant. There were also no significant differences in the components of cost, for both the initial hospital admission and post-discharge costs (Table 3 and Online Appendix, eTable 4). The total cost for days spent in the ICU was significantly higher than for days spent in the ward, but there was no difference between treatment groups (Online Appendix, eTable 5).

Figure 1.

Distribution of costs to one year

AUD = Australian dollars; EGDT = early goal-directed therapy.

Table 3.

Total costs to one year

	EGDT	Usual care	P
Total number of patients	98	105
Total cost to 12 months, mean (SD)	$67 223 ($72 397)	$54 179 ($61 980)	0.17
Total inpatient costs, mean (SD)	$30 821 ($23 930)	$30 731 ($25 109)	0.98
Total post-discharge costs, mean (SD)	$36 402 ($65 236)	$23 447 ($53 082)	0.12
Initial hospital stay cost components, mean (SD)
Pathology	$2440 ($2213)	$2711 ($2790)	0.45
Radiology	$1040 ($1108)	$1118 ($1118)	0.62
Drugs	$1065 ($2403)	$931 ($2738)	0.71
Fluids (includes blood products and nutrition)	$745 ($2006)	$1045 ($2060)	0.30
All procedures	$1621 ($1416)	$1406 ($1838)	0.36
Post-discharge costs components, mean (SD)
Readmission costs	$16 744 ($24 798)	$11 104 ($19 915)	0.08
Inpatient rehabilitation costs	$4019 ($14 571)	$2556 ($8261)	0.38
Long term care costs	$13 951 ($57 911)	$8574 ($46 256)	0.46
Outpatient costs to 12 months	$1687 ($3935)	$1213 ($2757)	0.32

EGDT = early goal-directed therapy; SD = standard deviation.

Outcomes

Lives saved and life-years gained were available for all patients participating in the economic substudy, with QALYS to one year available for 201 patients (99.0%). At one year, 30 patients (30.6%) in the EGDT group had died, compared with 33 (31.4%) in the usual care group. There was no difference between groups in the proportion of lives saved, life-years gained, EQ-5D-3L utility scores and QALYs to one year (Table 4).

Table 4.

Outcomes and incremental cost-effectiveness ratios

Outcome	EGDT	Usual care	Incremental costs⁎†	Incremental outcome⁎	ICER (cost/outcome)
Total number of patients	98	105
Lives saved (LS)	68 (69.4%)	72 (68.6%)	$13 044 (–$5791 to $31 878)	0.82% (–12.1% to 13.7%)	$1 597 834/LS
Life-years gained (LYG), mean (SD)	0.746 (0.406)	0.725 (0.417)		0.021 (-0.093 to 0.135)	$631 682/LYG
QALYs, mean (SD)	0.318 (0.291)	0.367 (0.295)		-0.049 (-0.131 to 0.034)	Dominated

AUD = Australian dollars; EGDT = early goal-directed therapy; ICER = incremental cost-effectiveness ratio; QALYs = quality-adjusted life-years; SD = standard deviation.

^⁎Data are presented in mean (95% CI).

^†2014 Australian dollars.

Cost-effectiveness

Table 4 shows the incremental cost, incremental QALYs, and the cost per QALY, along with the cost per life saved and cost per life-year gained of EGDT compared with usual care. As EGDT costs more than usual care and produces less QALYs, it is dominated by usual care.

The uncertainty in the incremental costs and QALYs is represented on the cost-effectiveness plane in Figure 2. EGDT resulted in an improvement in QALYs for only 11.8% of replications, while costs were lower among patients in the EGDT group for 9.2% of replications, indicating a high degree of uncertainty around estimates for cost-effectiveness. In 80.4% of replications, usual care dominated EGDT (ie, usual care is more effective and less costly). It was not possible to calculate a 95% CI for the cost per QALY, as two lines drawn through the origin of Figure 2 were unable to exclude 2.5% of the distribution of the cost-effectiveness points on either side of the lines.²⁰ However, the incremental net benefit for EGDT compared with usual care was negative at –$15 180 (95% CI, –$36 075 to $3426), suggesting lower net benefits.

Figure 2.

Cost-effectiveness plane

AUD = Australian dollars; EGDT = early goal-directed therapy; QALYs = quality-adjusted life-years.

For a willingness-to-pay threshold of $50 000 per QALY, the probability of EGDT being cost-effective was only 6.4%. This probability did not vary substantially when altering the willingness-to-pay threshold (Online Appendix, eFigure 1), with the probability of EGDT being cost-effective when compared with usual care remaining below 10% even at a willingness-to-pay of $1 000 000 per QALY.

The results were consistent across subgroups, with EGDT dominated by usual care or the resulting ICERs well above accepted thresholds (Online Appendix, eTable 6). Adjusting inputs in the sensitivity analyses did not alter conclusions, with EGDT remaining more expensive than usual care and producing less QALYs for all scenarios, other than the unadjusted analysis including all available QALYs (n = 1377), which showed that EGDT produced a small, non-significant increase in QALYs (Online Appendix, eTable 7). The resulting ICER of $1 416 342 per QALY is well above commonly accepted thresholds.

Discussion

This economic evaluation of the ARISE RCT did not find any evidence that EGDT is a cost-effective intervention compared with usual care for patients presenting to the ED with early septic shock. EGDT resulted in higher costs, both for the initial hospital stay and overall to one year, and produced less QALYs, although the differences were not statistically significant. The resulting ICERs for cost per life saved and cost per life-year gained were high and well above commonly accepted thresholds for cost-effectiveness. The cost per QALY was negative, indicating that EGDT is dominated by usual care. Results were consistent across subgroups, with no evidence of cost-effectiveness of EGDT in any subgroup. Sensitivity analyses confirmed the results are robust to alternative assumptions.

Previous economic evaluations of EGDT have found EGDT to be a cost-effective intervention compared with usual care; however, the majority of these evaluations were based on one small, single-centre trial or pre–post studies.7, 8, 10, 11, 12, 21 An economic evaluation of the original EGDT trial by Rivers and colleagues²² found that EGDT was associated with a reduction in hospital costs of 22% and an improvement in QALYs, with a probability of over 96% that EGDT was cost-effective at US$20 000 per QALY.²¹ In contrast, in a more recent economic evaluation of the ProMISe trial, conducted in the United Kingdom at a similar time to the ARISE RCT, the probability that EGDT was cost-effective (at a willingness-to-pay of £20 000 per QALY) was below 30%.¹⁷ The results of the current economic evaluation support the results of the ProMISe evaluation, although the current results indicate lower probabilities of cost-effectiveness in the Australian setting.

Economic evaluations are constrained by limited availability of high quality evidence from RCTs and limited data on long term outcomes including quality of life. Previous economic evaluations used data from non-randomised trials or from single-centre RCTs, which can result in an over-representation of the cost-effectiveness of an intervention. In addition, many of the evaluations with a stated or assumed lifetime horizon did not include any costs beyond the initial hospitalisation,⁶ despite sepsis being known to have long term consequences, including a higher risk of readmissions.²³ In the current economic evaluation, health care costs between hospital discharge and one year were significant, contributing about 50% of the total costs to one year. Given these findings, future studies of sepsis interventions should ensure that data are collected for ongoing post-discharge resource use. Future work should also include developing methods to efficiently collect resource use data in trial cohorts, with consideration of data linkage for post-hospital health care resource use and the addition of resource use data points into existing intensive care (and other) databases.

While there is often detailed evidence for the effectiveness of many interventions for sepsis, decision makers are also interested in which interventions deliver value for money in the context of limited health care resources. A recent narrative review on the cost of intensive care services in Australia indicated there continues to be a paucity of data regarding the costs of critical care.²⁴ While some work has been done internationally investigating the cost of treating patients with sepsis,25, 26, 27 such data are not easily generalisable to the Australian setting due to differences in treatment patterns, treatment availability, physician preferences, patient characteristics and funding mechanisms. The current economic evaluation provides detailed costings for septic patients which can be used in future economic evaluations.

The ARISE RCT economic evaluation has several strengths. It was prospectively designed and used data from a large, high quality, pragmatic RCT that had minimal loss to follow-up.⁵ Quality of life was measured at multiple times points allowing determination of QALYs at an individual patient level. The use of a detailed resource use case report form allowed accurate costing of trial patients, and the inclusion of post-discharge costs allowed for accurate capture of total health care costs to one year following randomisation. It also ensured the economic evaluation was able to capture differences in costs associated with any health care resource use rather than just those anticipated to be drivers of the incremental costs of EGDT over usual care.

Our economic evaluation has some limitations. Due to resource constraints, detailed data on health care resource use were only able to be collected on a sample of 205 patients enrolled in the ARISE RCT. While there were no significant differences in the main characteristics of the costed patients compared with the rest of the trial patients, it is possible that the total costs do not accurately reflect the total costs of the entire trial cohort. Data on all patients would also have enabled a more accurate costing with less variability. In determining QALYs, it was assumed that patients had a utility score of 0 at randomisation. While this is common practice in critical care economic evaluations,16, 17, 18 some patients in the ARISE RCT did not require ICU admission and this assumption may not be valid in these patients. However, the assumption is unlikely to influence incremental QALYs and therefore unlikely to influence determinations of cost-effectiveness. A further limitation is that the study does not incorporate the current Sepsis-3 definitions²⁸ as these had not been developed when the ARISE RCT was planned. A post hoc analysis of ARISE RCT participants found that 12.8% met the Sepsis-3 criteria for septic shock, with 84.7% meeting the Sepsis-3 criteria for sepsis.²⁹ As a result, the cost-effectiveness of EGDT using the Sepsis-3 definitions is unclear.

Conclusions

The current economic evaluation found that there is no evidence that EGDT is a cost-effective intervention compared with usual care for patients presenting to the ED with early septic shock. We found that the probability that EGDT is a cost-effective intervention compared with usual care is less than 10%, even at a willingness-to-pay threshold of $1 000 000 per QALY. Our analysis confirms that EGDT, as a packaged protocol of care, is not superior to usual care and is unlikely to offer value for money to the Australian health care system.

Competing interests

No relevant disclosures.