Application of patient-reported outcome measures (PROMs) data to estimate cost-effectiveness of hernia surgery in England

Abstract

Objectives

To demonstrate potential uses of nationally collected patient-reported outcome measures (PROMs) data to estimate cost-effectiveness of hernia surgery.

Design

Cost-utility model populated with national PROMs, National Reference Cost and Hospital Episodes Statistics data.

Setting

Hospitals in England that provided elective inguinal hernia repair surgery for NHS patients between 1 April 2009 and 31 March 2010.

Participants

Patients >18 years undergoing NHS-funded elective hernia surgery in English hospitals who completed PROMs questionnaires.

Main outcome measures

Change in quality-adjusted life year (QALY) following surgery; cost per QALY of surgery by acute provider hospital; health gain and cost per QALY by surgery type received (laparoscopic or open hernia repair).

Results

The casemix-adjusted, discounted (at 3.5%) and degraded (over 25 years) mean change in QALYs following elective hernia repair surgery is 0.826 (95% CI, 0.793–0.859) compared to a counterfactual of no treatment. Patients undergoing laparoscopic surgery show a significantly greater gain in health-related quality of life (EQ-5D index change, 0.0915; 95% CI, 0.0850–0.0979) with an estimated gain of 0.923 QALYS (95% CI, 0.859–0.988) compared to those having open repair (EQ-5D index change, 0.0806; 95% CI, 0.0771–0.0841) at 0.817 QALYS (95% CI, 0.782–0.852). The average cost of hernia surgery in England is £1554, representing a mean cost per QALY of £1881. The mean cost of laparoscopic and open hernia surgery is equivocal (£1421 vs. £1426 respectively) but laparoscopies appear to offer higher cost-utility at £1540 per QALY, compared to £1746 per QALY for open surgery.

Conclusions

Routine PROMs data derived from NHS patients could be usefully analyzed to estimate health outcomes and cost-effectiveness of interventions to inform decision-making. This analysis suggests elective hernia surgery offers value-for-money, and laparoscopic repair is more clinically effective and generates higher cost-utility than open surgery.

Background

Improving outcomes for committed resources is a concern across all health economies.¹ In England, the National Health Service (NHS) has to find productivity gains of £20 billion over four years to 2014/2015.² While access to NHS services is determined by clinical need,³ healthcare resources are finite and prioritization decisions have to consider need, effectiveness and cost-effectiveness.⁴ Raising elective surgery thresholds could save money, and reducing access to hernia repair surgery is one suggestion.^5,6

Cost-utility analysis using quality-adjusted life years (QALYs) as the generic measure of health benefit (utility), is often used to inform which treatments the NHS should offer, or to whom.⁷ These analyses require data about costs and benefits of interventions. The NHS generates routine information about its costs, but has historically been poor in producing information about benefits.⁸ However, there are signs this may be changing, most recently with the Government setting out its ambition to increase the number of QALYs – generally and for specific procedures – the NHS produces each year.⁹

Since April 2009, all providers of NHS-funded care in England have been required to collect preoperative and postoperative data of patients’ own assessments of their health-related quality of life for four surgical procedures, including inguinal hernia repair.¹⁰ These patient-reported outcome measures (PROMs) use validated survey tools and enable the calculation of health-related quality-of-life change following surgery. They have the potential to improve patient care and resource use, by enabling comparisons between interventions and also healthcare providers.^11,12

By combining PROMs data with provider-level costs of surgery provision, and patient record data to ascertain surgery type, it should be possible to estimate effectiveness of these operations overall and to quantify the health gain and associated cost by type of surgical intervention.

The purpose of this paper is to demonstrate this application, using elective hernia repair as an example.

Method

Data

The detailed rationale and methodology for PROMs are described elsewhere.¹⁰ Briefly: since April 2009, patients >12 years old undergoing hernia repair are invited to complete two questionnaires – one prior to, and a second at least three months after, surgery.¹³ Of those invited to participate in 2009/2010, 66.1% returned the first questionnaire, and 80.0% of these completed the follow-up one.¹⁴

The questionnaire includes a standardized, validated instrument, the EQ-5D, which asks questions across five domains (mobility, self-care, usual activity, pain, anxiety/depression).^15,16 The resulting score is combined with social value weights derived from the UK population to provide a single index value, which can range from one (perfect health) to zero (death).¹⁷

Between 1 April 2009 and 31 March 2010, the NHS funded 68,640 eligible hernia repair operations in England.¹⁸ This analysis uses the finalized data-set comprising 18,280 patient episodes (of which 80 records have been suppressed due to small numbers, which could compromise patient confidentiality).¹⁴

Change in EQ-5D after surgery

Hospitals with <30 PROMs returns (n = 47) were excluded, leaving 164 providers and 17,463 patient episodes in our sample. In addition to before-and-after EQ-5D index scores, the data-set provides casemix-adjusted (taking account of patient characteristic variations and other factors that are beyond the direct control of hospitals) average changes in the EQ-5D index score for each hospital.¹⁹

Change in QALYs

To model the estimated QALY change for each hospital we assumed two possible interventions – surgery or no surgery – with three possible resultant health states: (1) following surgery, the health outcomes remain at the postoperative level; (2) following surgery, the health-related quality of life degrades evenly over 25 years back to the pre-surgery level; (3) the counterfactual scenario, where surgery is not undertaken and there is no subsequent degradation in health (see Figure 1). We applied a 25-year time horizon to be consistent with existing analyses of hernia surgery, which seems plausible since long-term recurrence for open and laparoscopic repair is low (1.2% and 2.4% respectively, but not statistically different).^20,21 In line with HM Treasury’s recommendations we discounted all estimated future health benefits at 3.5%.²²

Figure 1.

Stylized model to calculate QALYs (discounted at 3% annually) over time (25 years) using EQ-5D index data

The change in QALYs for each hospital, assuming patients experience no degradation in their health, was calculated by subtracting its discounted preoperative from the discounted postoperative casemix-adjusted EQ-5D index score, summed over 25 years (equation (1)). A similar calculation assuming degradation in health postoperatively was also calculated (equation (2)).

$$[\sum _{i=0}^{25}(Q_h\text{'}\text{'}/1.{035}^i)]-[\sum _{i=0}^{25}(Q_h\text{'}/(1.{035}^i))]$$ (1)

$$\begin{array}{c}\multicolumn{1}{c}{[\sum _{i=0}^{25}(Q_{\mathrm{hi}}\text{'}\text{'}-(\frac{Q_h\text{'}\text{'}-Q_h\text{'}}{25}))/(1.{035}^i)]}\\ \multicolumn{1}{c}{-[\sum _{i=0}^{25}(Q_h\text{'}/(1.{035}^i))]}\end{array}$$ (2)

where: $Q_h\text{'}\text{'}$ = postoperative casemix-adjusted average EQ5D index score for hospital, h $Q_h\text{'}$ = preoperative casemix-adjusted average EQ5D index score for hospital, h i = year, and i = 0 = year in which operation was performed.

Hospital costs

Unit costs by healthcare resource group (HRG) for NHS hospitals in England are produced annually and published in the National Reference Cost (NRC) data-set.⁸ We mapped the clinical codes for each hernia procedure included in the PROMs programme to the corresponding HRG codes in the NRC data-set (Table 1).²³ The cost and volume of each HRG code was extracted from the NRC and the arithmetic mean cost of hernia surgery by HRG code calculated. Our cost calculation used NRC data for 99,658 elective NHS hernia episodes. Patients aged ≤18 years were excluded, as the burden of hernias is highest in young children, and it is not possible to disaggregate patients aged 12–18 years who are eligible to participate in PROMs, and those <12 years who are not.²⁴

Each hospital’s weighted average cost for surgery was then calculated by multiplying volumes of each HRG code activity coded by that hospital by the average cost and dividing the sum of these costs by the overall activity level. The cost per QALY for each provider was obtained by dividing this figure by the discounted casemix-adjusted change in patient QALYs for each provider.

Calculating the cost per QALY for independent providers (n = 19) was not possible as their costs are not published.

Change in QALYs by surgery type

Finally, we linked the PROMs with a restricted Hospital Episodes Statistics (HES) data-set using the ‘Episode Key’ identifier in order to classify patients according to type of surgery they received. This resulted in a slightly smaller sample (n = 17,776): three-quarters of patients had open repair (n = 13,971) in 228 hospitals, compared to 3805 who had laparoscopic surgery in 171 hospitals. Overall QALY scores were generated using the same model described above, and the average QALY change for each surgery-type calculated. We calculated weighted average costs for laparoscopic and open repairs by identifying the different operations through HRG codes in the HES records, which then allowed us to calculate an average cost per QALY for both interventions.

Results

Change in EQ-5D score

The overall adjusted average change in EQ-5D score following surgery was 0.0830 (95% CI, 0.0798–0.0862). Applying the Department of Health’s methodology for identifying outliers shows two hospitals have EQ5D index scores that fall on the 95% confidence limits, and four are extremely close to them (Figure 2).²⁵

Figure 2.

Funnel plot of adjusted EQ-5D change against number of PROMs returns (min. n = 30) received from NHS (n = 145) and independent sector (n = 19) hospitals providing hernia surgery for NHS patients in 2009–2010, with 95% and 99% upper and lower confidence limits, respectively

QALY change

The discounted crude mean difference in QALYs following hernia surgery is 1.451 (95% CI, 1.395–1.507) QALYs, reducing to 0.836 (95% CI, 0.819–0.853) QALYs assuming degrading health. The casemix-adjusted figures were not significantly different: discounted mean difference in QALYs following hernia surgery at trusts with ≥30 returns is 1.433 QALYs (95% CI, 1.376–1.490), reducing to 0.826 (95% CI, 0.793–0.859) assuming degrading health-related quality of life over 25 years.

The majority (n = 154) of providers show statistically significant QALY gains after surgery, and seven trusts have patient-reported surgical outcomes that are significantly higher than the national average (Figure 3). The range of casemix-adjusted scores is wider among NHS hospitals (min, 0.111; max, 1.370), compared to independent sector providers treating NHS-funded patients (min, 0.574; max, 1.260).

Figure 3.

Mean change in casemix-adjusted, discounted (at 3.5%) and degraded (over 25 years) QALYs following hernia surgery in NHS (n = 145) and independent sector (n = 19) hospitals with 95% confidence interval error bars by ranked provider with ≥30 PROMs returns, 2009–2010

Cost per QALY

Among hospitals with ≥30 PROMs returns, the mean cost of surgery is £1554, representing an estimated national average cost per QALY of £1881 (range, £548–£14,020). The casemix-adjusted mean cost of elective hernia surgery repair by NHS hospital varied seven-fold, ranging from £475 to £3443. Three hospitals had particularly high cost per QALY values, two due to small gains in QALYs and one due to high cost of hernia procedures. The majority (about 85%) of NHS providers generate a casemix-adjusted cost per QALY of between £1000 and £3000 for hernia surgery (Figure 4). Figure 4 also shows no significant relationship between costs and outcomes (correlation coefficient, r = 0.07).

Figure 4.

Cost-effectiveness plane showing discounted (at 3%) and degraded (over 25 years) mean QALYs change after hernia surgery, and associated average cost among NHS providers (n = 145) with ≥30 PROMs returns, 2009–2010

Surgery type

When the sample was split by type of intervention, patients undergoing laparoscopic surgery had an average preoperative EQ-5D score of 0.809 (95% CI, 0.803–0.815) compared to 0.790 (95% CI, 0.786–0.793) for those having open repair. The respective postoperative EQ-5D gains were 0.0915 (95% CI, 0.0850–0.0979) and 0.0806 (95% CI, 0.0771–0.0841).

The average crude change (i.e. no degradation) in QALYs for patients undergoing open repair was 1.417 (95% CI, 1.355–1.479), and those having laparoscopies was 1.601 (95% CI, 1.489–1.713). Assuming degrading health and discounting benefits at 3.5% annually, laparoscopic repair generates a significantly greater average health gain of 0.923 (95% CI, 0.859–0.988) QALYs compared to 0.817 (95% CI, 0.782–0.852) QALYs for open repairs (Figure 5). The confidence interval around the mean QALY change for each procedure shows the estimated difference could range from 0.007 QALYs to 0.206 QALYs.

Figure 5.

Mean change in discounted (at 3.5%) and degraded (over 25 years) QALYs with 95% confidence intervals by type of surgery

The national average unit cost of elective laparoscopic and open surgery is similar (£1421 vs. £1426, respectively). In terms of cost-effectiveness, open repair costs £1746 per QALY, compared to £1540 per QALY for laparoscopic surgery.

Discussion

This paper describes how PROMs data could be used to generate a cost-effectiveness estimate of hernia surgery, and compare the health gain by type of surgery received, using routine data from patients being treated by the NHS. As the NHS seeks to make improvements in productivity, there is increasing interest in offering treatments to patients most likely to benefit, thereby maximizing health outcomes, and also ensuring that interventions are cost-effective.^2,5,9

The National Institute for Health and Clinical Excellence (NICE) does not usually recommend that the NHS provides treatments with an incremental cost-effectiveness ratio of over £20,000 to £30,000 per QALY, a figure that aligns with the marginal valuation of a life-year based on healthcare commissioner spending patterns.^7,26 At £1881 per QALY, hernia surgery appears to offer good value for money.

This contrasts with recent guidance that suggests elective hernia repair is of ‘low clinical value’ and that consequently commissioners should consider restricting access to such procedures.⁶ But commissioners need to be aware that they are required to demonstrate clearly whether a procedure is genuinely of low clinical value to justify any access restrictions they may wish to impose, thereby underscoring the need for the development of a reproducible methodology such as this.²⁷

Our analysis suggests patients undergoing laparoscopies report significantly larger gains in outcomes (0.923 QALYs) than those having open repairs (0.817 QALYs). The findings support existing evidence that laparoscopies are generally associated with faster recovery and less persisting pain.²⁰ However, serious complications are more likely and operations take longer.²⁰ This might explain greater variation in outcomes and hence the wider confidence intervals around QALY gain following laparoscopic repair, though we hypothesize this is primarily driven by the smaller sample size.

Laparoscopies can cost more than open procedures however – an additional cost of between £300 and £350 (2005 prices).²⁰ By contrast, the NRC data used in this analysis suggest national average costs are similar. Based on this and the calculated changes in QALYs, we conclude that at £1540 per QALY, laparoscopic surgery is more cost-effective than open repair (£1746 per QALY). This echoes research from America which concludes laparoscopic repair is the dominant treatment option.²⁸

We note that the overall average cost for hernia surgery (£1554) is higher than the average cost for either laparoscopic or open repair separately. We hypothesize these differences may have arisen if those patients that were lost from the sample the data-sets were linked (n = 504), were disproportionately from more expensive HRG codes. However, because we were unable to link them, we cannot check the missing patients’ procedures codes to identify any systematic differences between this ‘lost’ and the main sample.

Although almost no outliers in terms of overall EQ-5D change were identified, there was large variation in cost per QALY generated by individual providers. Healthcare commissioners could use analyses such as these as the starting point for undertaking qualitative investigations with providers to understand what is driving local differences. Providers might also find this a useful exercise to benchmark against their peers, and to use this as an opportunity to improve practice. However, it would also be important to include those trusts with <30 PROMs returns as low surgery volumes can impact outcomes, but also to understand why return rates are so low.²⁹

Strengths and limitations

There are several limitations to our study.

First, we used NRC data to estimate costs of providing hernia surgery. Data quality concerns have been raised, though audits suggest inpatient activity and total costs data are reliable, though improvements are possible.^30,31 In terms of overall average cost our sample seems representative, with average costs of hernia repair from our sample similar to average tariff price (Table 1), which is based on NRC data from all NHS hospitals and (most) patients undergoing this procedure.

Table 1.

Hernia procedure healthcare resource group (HRG) codes

HRG code	HRG description	Included in analysis?	Arithmetic mean cost of surgery (£)	Elective spell tariff (£)
FZ17A	Abdominal Hernia Procedures 19 years and over with Major CC	Yes	3495	2838
FZ17B	Abdominal Hernia Procedures 19 years and over with Intermediate CC	Yes	2692	2869
FZ17C	Abdominal Hernia Procedures 19 years and over without CC	Yes	1989	2453
FZ17D	Abdominal Hernia Procedures 18 years and under	No*	–	–
FZ18A	Inguinal Umbilical or Femoral Hernia Repairs 19 years and over with Major CC	Yes	1903	1570
FZ18B	Inguinal Umbilical or Femoral Hernia Repairs 19 years and over with Intermediate CC	Yes	1637	1539
FZ18C	Inguinal Umbilical or Femoral Hernia Repairs 19 years and over without CC	Yes	1376	1370
FZ18D	Inguinal Umbilical or Femoral Hernia Repairs 18 years and under	No*	–	–
FZ19Z	Herniotomy Procedures	Yes	1287	1357

^*Patients < 18 years old were excluded from the analysis as the burden of paediatric hernias tends to be highest in infants and young children and it is not possible to disaggregate PROMs returns by age

However, as NRC data do not differentiate between laparoscopic and open hernia procedures, calculating costs for these procedures was only possible on a weighted average basis. We may therefore have underestimated laparoscopy costs. It would take an increase of around 19% (£268) in the national average unit cost of a laparoscopic procedure to eliminate the cost per QALY difference between laparoscopic and open repair. Some caution should be exercised when interpreting the hospital-level cost-effectiveness estimates. This also highlights a need for costing data that enables precise identification of costs for different procedures. This is particularly relevant for laparoscopic hernia repair as costs are highly dependable on whether disposable or reusable equipment is used.³² In future, it may also be possible to use alternative cost data such as Patient Level Information and Costing System information.

Second, our model is relatively unsophisticated and assumes constant degradation of health following either type of surgery, but no degradation for the counterfactual health state. The latter assumption is unlikely as the rationale for offering surgery is to reduce the risk of emergency surgery due to strangulation, which has poor outcomes. By assuming no degradation in the counterfactual, our model is conservative so that any benefit is underestimated: existing research estimates a lifetime gain of 9.04 QALYs following laparoscopic surgery.²⁸ Our model is also applied uniformly to all hospitals and both treatment options: alternative assumptions, such as average hernia recurrence rate, or health status changes over time, are possible. However, any assumptions if applied to all hospitals uniformly should not alter their relative QALY changes or cost-utility values.

Furthermore, although inter-group patient characteristics are unlikely to differ as the contraindications for laparoscopy are close to those of open repair, surgical skill-mix could affect which surgery is offered, and hence health outcome.²¹ Minimizing such potential confounding requires the development of sophisticated risk-adjustment models.

Third, the nature of PROMs data means our model infers long-term effectiveness by comparing two cross-sectional health status measures. This may undermine the model if one procedure is not definitively superior to the other over time.¹¹ For example, it is unclear whether laparoscopic surgery itself is associated with increased recurrence rates, or if this is confounded by surgical skill level. If recurrence is more likely with laparoscopies, then quantifying the health advantage is difficult if patients are not followed up for extended time periods. More research is needed to develop sophisticated estimates of health status trajectories, so they can be included in models such as ours.

Fourth, despite being a mandatory requirement, PROMs return rates are variable, the data cover only about 30% of eligible hernia operations and biases may exist within the data.^14,33 We reduced the risk of small numbers bias by excluding providers where <30 patients had completed both questionnaires.¹³ Nevertheless, patients with worse outcomes, men, non-white groups, the most deprived, those living alone and patients with three or more co-morbidities are less likely to return the second questionnaire.^33,34 It is therefore possible that non-response, at provider and patient-level, has biased our results. Further investigation is needed to establish how analyses can be adjusted to account for biases. Although the PROMs data-set has flaws, it is the only national sample providing patient-assessed quality of life information for the majority of patients undergoing surgery commissioned by the NHS in England.¹³ The data-set currently provides the only way of undertaking such a piece of work.

Conclusion

Using hernia repair as an example, we have demonstrated how routinely collected national PROMs information could enable health outcome and cost-effectiveness comparisons between interventions, and, could ultimately be used by commissioners to inform healthcare purchasing decisions. Using current data and methodology, we show hernia repair seems to offer value for money, that laparoscopic repair is more effective and may be more cost effective than open surgery. With improvements to existing data sources it should also be possible to use this methodology to compare individual healthcare providers.

DECLARATIONS

Competing interests

JA was an independent member of the Department of Health PROMs Stakeholder Group (2010–2012)

Funding

This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors. SCC’s position with the King’s Fund is a specialty training placement funded by London Deanery, the regional body responsible for providing higher specialty medical and dental training programmes

Ethical approval

Ethical approval was not required for this work, as the research was a secondary analysis of two existing, restricted data-sets which do not contain patient identifiable information

Guarantor

JA

Contributorship

JA conceived the work and with SCC designed the study. JA supervised SCC and JT conducting the analyses, and SCC drafted the paper. All authors helped interpret the results and provided additional drafting and editorial comment to, and approved, the final version

Acknowledgements

The authors would like to thank Celia Ingham-Clark and Susan Consterdine for their insightful advice; both the Choice and PROMs team, Department of Health, and Health and Social Care Information Centre for their respective guidance on analysing the various data-sets; Gordon Buchanan for his specialist clinical input, and Virginia Warren for editorial comments

Provenance

Submitted; peer reviewed by Karen Bloor