Is it worth treating patients with COVID‐19 in intensive care? Utility, choice, costs and value

Was all the effort put in by healthcare staff during the COVID‐19 pandemic worth it? In this issue of Anaesthesia, Schallner et al. attempt to answer this, at least in part for intensive care units (ICU) from an economic perspective [1]. They conclude that the cost of critical care for COVID‐19 patients was worth it (with the possible exception of extracorporeal membrane oxygenation) because their estimated cost per quality‐adjusted life‐years (QALY) of £7511 is far below the thresholds conventionally set (e.g. around £30,000 by the National Institute of Health and Care Excellence, NICE).

Clinicians are generally used to research studies that test hypotheses, or to clinical trials that ascertain the effectiveness of one intervention vs. another. Economic analyses are different because they are designed to inform decisions, especially in situations where healthcare funding is constrained. Here, choices have to be made to deliver one treatment over another or preferentially treat one group of patients vs. another. On one side of this equation is ‘cost’, which, as I discuss below, is not always straightforward to measure. On the other side of the equation is ‘outcome’ and studies are variously classified as ‘cost–benefit’, ‘cost minimisation’, ‘cost‐effectiveness’ or ‘cost‐utility’ [2]. Colloquially, the words ‘benefit’, ‘effectiveness’ and ‘utility’ are used interchangeably (and even Schallner et al. fall into this trap in sections of their article [1]), but in fact, as Table 1 shows, they have precise meaning. Cost–benefit applies when the common metric on both sides of the equation is monetary (e.g. £). In cost‐effectiveness, outcomes are measured in a single natural unit, such as years of survival or blood pressure in mmHg (the better treatment being one that say, reduces blood pressure by 10 mmHg for less cost). In a cost‐utility study, like that by Schallner et al., outcomes are expressed by a measure that considers both the effect on mortality and morbidity (QALYs). Mortality (survival) is a hard endpoint, but quality of life is less so and this is where discussions around limitations of the methodology are focused.

Table 1.

Summary of different types of economic evaluations for healthcare interventions, with their relative strengths and weaknesses.

Type of study	Description	Strengths/weakness
Cost‐effectiveness analysis	Outcomes measured in a single natural unit (e.g. life‐years gained, mmHg blood pressure).	Appropriate in comparing interventions with same outcome measure; pragmatic but can miss benefits not encompassed by the metric.
Cost–benefit analysis	Both costs and outcomes are expressed in monetary units (£). Can include ‘willingness to pay’ studies.	Units of measurement easily understood, but monetary outcomes can be difficult to estimate (assumes all benefits can be expressed in monetary terms).
Cost‐utility analysis	Outcomes are expressed by a generic measure of health taking into account mortality and morbidity (e.g. quality‐adjusted life‐years, QALYs).	Permits comparison of different interventions in different fields; complicated and requires accepting QALYs as valid measure.
Cost minimisation analysis	Comparing costs of interventions which have equivalent health effects.	Simple; but few interventions have precisely the same health effects.
Cost consequence analysis	Outcomes are not summarised in a single measure, but multiple outcomes of interest are reported.	Captures a range of outcomes, but can be complex and too context‐specific.

Even before considering the cost‐utility aspects of the study there are some clinical elements that may be confounders. The study spanned a wide range of the pandemic, from close to its onset in April 2020 until April 2021. We know that overall mortality improved dramatically and that new treatments ranged from the cheap (e.g. dexamethasone) to the very expensive (e.g. sotrovimab); and also of course some treatments proved ineffective (e.g. remdesivir, hydroxychloroquine) [3]. Schallner et al. considered ‘ICU care’ as a single package of care, yet this must have been a heterogeneous and dynamic mix of specific interventions over this period. Moreover, the criteria for ICU admission changed, and the impact of vaccination may have plausibly influenced the characteristics of admitted patients. The sample size of the study (n = 49) was too small to capture all these nuances.

Measuring the quality of life

Theoretically, if health‐related quality of life (HRQoL) could be perfectly measured, then it would decline gradually until point of death (except, of course, in sudden death). Beneficial treatment should not only extend life, but also maintain a higher HRQoL for longer. Thus, the area under the curve between two treatments (or treatment vs. no treatment) represents the number of QALYs gained by the intervention (Fig. 1).

Figure 1.

Relative decline in HRQoL with two treatments (where a could be no treatment or control); the yellow area under the curve is the QALYs gained by treatment b.

Ultimately, ‘quality of life’ is a subjective measure, assessed by individuals themselves, with several methods used to estimate it [4]. Broadly, each involves first defining the relevant health states of interest and then assigning a value to these and combining the scores. So‐called ‘direct’ methods include the visual analogue scale; ‘time trade‐off’; and ‘standard gamble’. The first is perhaps simplest to understand, where the top of the scale (e.g. 100) indicates ‘best imaginable health’ and the bottom (e.g. 0) the worst. A limitation is that as a ‘rating task’ rather than a ‘choice’ there can be scaling bias: the difference between 10 and 20 may not mean the same as that between 80 and 90. Respondents appear reluctant to score at extreme ends of the scale [5].

In time trade‐off, individuals are presented with the choice between living for x unhealthy years vs. living in full health for fewer years (y years; y < x). The ‘point of indifference’ is the ratio x:y (Fig. 2). Thus, if people respond that they would live for 5 years with motor neurone disease vs. 2 years of good health, then the HRQoL weight is 0.4.

Figure 2.

Illustration of concept of time trade‐off. The respondent is offered x fixed years of life at some poor quality of life in ill health (red). They are then asked to say whether y years at full health is preferred or not (black), with point x varied. In this way, the point of equivalence (indifference) can be estimated. Further visual aids may be needed to convey the concept of ill health, and one problem with this approach is that rarely is quality of life in ill health (or full health) constant over time in the way illustrated.

Choice is also presented in the standard gamble but in terms of risk (Fig. 3). On the one hand is the certainty of the impaired health state and on the other hand is a gamble; this gamble being between full health (risk x%) and death (risk 100‐x%). This is quite a complex question: what risk of death coupled with the potential for benefit of a healthy life equates to the certainty of ill‐health? The probability of death is varied in the question until the individual is indifferent between the certainty option and the gamble option [6].

Figure 3.

Illustration of standard gamble. The respondent is invited to ‘gamble’ or ‘not gamble’. If they do not gamble, they have the certainty of a certain poor health state for a defined number of years. If they do gamble, then they have x% chance of full health or 100‐x% chance of death. The value of x is varied until the group's preferences are equal in the choice of gambling or not. Even this simplified diagram indicates the difficulty some respondents have in understanding the choices being presented.

What all these approaches aim to do is offer people choice between the inevitable trade‐off between length of life and its quality. The standard gamble seems an elegant extension of the time trade‐off but suffers from the irrationality of human beings. This was exposed by the Allais paradox, simplified by Kahneman (see online Supporting Information; Appendix S1) [7], and has been applied to many walks of life, including healthcare. These human frailties are exploited widely in advertising [8], and even closer to home, by taxation systems [9]. People would often choose to work harder/longer to earn higher gross incomes, even if that entails equal or even lower net income despite their hard work [10]. Therefore, the utility of choice methods like standard gamble or time trade‐off methods accurately to assess HQoLs has been questioned, because people do not always express rational choices [11].

In addition to these limitations, such direct methods are time consuming and raise ethical concerns (e.g. should a dying patient be asked to value choices concerning good health, which they can never have?). Alternative, indirect methods involve using pre‐scored, generic preference‐based measures as a reference, with the study group asked only a set of more neutral questions that describe their perceptions of their health. These responses generate a score validated against the reference. Schallner et al. used the EQ‐5D but other tools are available (including those termed SF‐36, SF‐6D and HUI 1–3). Debates revolve around which to use; one difference for example being the health states encompassed; ranging from 245 for EQ‐5D to an incredible 972,000 for HUI 3 [4, 5].

Regardless of whether there are really almost a million different types of health conditions, the concept of QALYs raises some interesting ethical dilemmas. Generally, QALYs are used to increase the total health of the population; yet, there may be some groups that gain or lose more in QALYs with the given intervention. To some, this seems fair (‘QALY egalitarianism’) [12]. However, a contrary argument is to weight QALYs. One reason to do this (based on equity) is to redress existing health inequalities: a QALY gained by lower socio‐economic, disadvantaged groups may be judged more valuable. Another (based on economic efficiency) is almost the opposite: to weight the QALYs of those who can contribute most to economic productivity, with the implicit notion that this gain will permeate down to the disadvantaged or economically unproductive [4]. The logic of this can be harsh, as witnessed by the public television exchange in which the retired Supreme Court judge Lord Sumption told a stage‐4 cancer patient that her life was ‘less valuable’ than that of others [13]. Schallner et al. did not weight their QALY measure but with a relatively small sample of 45, subgroup weighting may have been difficult [1].

Two related, but contrasting, viewpoints underline the subjective nature of QALYs. Some very disabled patients, well adjusted to their disability, will score their own health status as ‘perfect’ [14]. Other patients who contemplate assisted dying may score their current health status as negative (worse than death) [15, 16]. Both these extreme perceptions, genuinely and strongly held, can skew the HRQoL data.

Measuring cost

This discussion on the limitations of measuring HRQoLs and QALYs is not comprehensive but gives a flavour of the critiques offered in the literature. Readers might think that measuring the other side of the equation, cost, must be so much simpler. Unfortunately, it is not [17]. Understanding costs inevitably involves accounting and there are two broad types: financial and management [18].

Financial accounting is used to present the financial health of a company to external stakeholders, including boards, shareholders, investors, creditors and regulators, in prescribed ways according to international accounting standards. Financial accounting is not relevant to studies like that of Schallner et al.

In contrast, management accounting concerns how budgets are apportioned within an organisation and is used to make daily operational decisions (e.g. how to price the product or service). Almost every large organisation structures itself into smaller ‘business units’ or ‘cost centres’ which in hospitals generally correspond to clinical departments. Each will have its direct costs; drugs or other consumables used, and these are relatively readily estimated. Staff costs can also be straightforward, unless they have duties spread across departments, in which case their relative activities need deconstructing. However, there will always be indirect costs (overheads) which are shared across departments, such as electricity; water; porters; or shared equipment. The costs of these overheads can be apportioned in different ways, e.g. by floor space occupied, by staff numbers or by activity (patients treated). There is no fixed regulatory requirement to apportion these in a certain way, and the total costs of a department can be greatly sensitive to which method is used.

An alternative to traditional accounting of indirect costs in this way is ‘activity‐based costing’, of which ‘service line reporting’ in healthcare is a version [19]. This method seeks more precisely to assign overhead costs to the specific processes that generate or use them. However, in complex organisations there may be hundreds or thousands of processes contributing to overheads, and disentangling these can be so time‐consuming that the effort may not be worth it. Careful reading of the supplementary materials in the article by Schallner et al. suggests that ‘hours spent in ICU’ was the metric used to handle overhead costs. This matters because using alternative methods of activity‐based costing, or weighting by different factors, may yield a different result. A further general problem with any cost analysis taken at any point in time is the subsequent variation in prices due to market forces or, latterly, inflation which can differentially affect elements of the overall cost. If the price of one good or service rises disproportionately to others, the conclusions may not apply in future.

One point of consensus is that official metrics do not reflect true costs. In Germany, this is based on diagnosis‐related groups. Schallner et al. state that “In this system, inpatient hospital treatment is billed and reimbursed…[in]…a very inaccurate estimate of true treatment costs” [1]. They studiously avoided simply using diagnosis‐related groups, but this framework is akin to previous observations that NHS tariffs are inaccurate [20], and that few if anyone in the NHS knows what the true costs of care actually are [21]. In other words, if the prices being set for goods, services and consumables are not their true market values, but instead some arbitrary tariff set by government, then the cost side of the equation will not be meaningful.

Do costs matter?

A useful exercise in reading any paper is to imagine what the reaction would be if the conclusion were the opposite of that being proposed. How would we respond if Schallner et al. concluded that it was not worthwhile (cost utility) to treat COVID‐19 in ICU? It would be unlikely – if not unimaginable – that society would then recommend that COVID‐19 patients should be limited to ward care, and that beyond this, patients should perish rather than enter ICU. In other words, while Schallner et al. reassure us that the cost of ICU care for COVID‐19 patients was worth paying, their study does not help answer the wider question of what value we place as a society on treating COVID‐19 [22]. There are some societal questions that transcend monetary cost: the abolition of slavery or equal rights for women. It literally does not matter what the financial equation is to achieve any of these things: we are unashamedly prepared to pay any price and the threshold cost‐per‐QALY for each of these is infinite [23]. This may apply to COVID‐19 too: the then Chancellor of the UK Exchequer Rishi Sunak committed, as a matter of economic policy, to do “whatever it takes” financially to overcome the COVID‐19 pandemic [24]. In the context of pre‐pandemic analysis, the NICE cut‐off for cost‐effective (utility) interventions was approximately £30,000 per QALY; against the background of the pandemic this is, by definition, irrelevant as Sunak's policy placed no limit.

This willingness to pay for things that we value as a society underpins the philosophy of the NHS. Schallner et al. undertook their study in an economic framework in Germany in which hospitals, being private financial entities, must balance their budgets. Failure to balance budgets leads to bankruptcy and around 12% are at risk of closure [25]. In contrast, governments and the public sector services they directly fund and underwrite, are not bound by the same fiscal rules because governments create (print) the currency used in the economy. This is the basis of ‘modern monetary theory’ which equates government spending commitment to the printing of money. By definition, entities in the public sector cannot become bankrupt unless by choice, which should free them to focus on real value and not just value‐for‐money [26].

In summary, Schallner et al. have undertaken a difficult study and offered extremely reassuring results [1]. When set against the economic analyses of the pre‐pandemic world, ICU care for COVID‐19 (notwithstanding all the limitations and flaws in the metrics of QALYs and measuring costs) represents excellent cost utility. However, given that this is a post‐hoc study, and that there was in reality little or no choice at the time as to whether to treat or not – and probably even less choice should another pandemic wave arise again – it is not clear that the data will or should inform public policy about whether or not to offer critically ill COVID‐19 patients ICU support. We will almost always choose to do whatever it takes.

Supporting information

Appendix S1. The irrationality of human choice.