Abstract
Aims
Health economic assessments increasingly contribute to funding decisions on new treatments. Treatments for many poor prognosis conditions perform badly in such assessments because of high costs and modest effects on survival. We aimed to determine whether underlying shortness of prognosis should also be considered as a modifier in such assessments.
Methods
Two hundred and eighty-three doctors and 201 oncology patients were asked to allocate treatment resource between hypothetical patients with unspecified life-shortening diseases. The prognoses with and without treatment were varied such that consistent use of one of four potential allocation strategies could be deduced: life years gained (LYGs) – which did not incorporate prognosis without treatment information; percentage increase in life years (PILY); life expectancy with treatment (LEWT) or immediate risk of death (IRD).
Results
Random choices were rare; 47% and 64% of doctors and patients, respectively, used prognosis without treatment in their strategies; while 50% and 32%, respectively, used pure LYG-based strategies. Ranking orders were LYG > PILY > IRD > LEWT (doctors) and LEWT > LYG > IRD > PILY (patients). When LYG information alone could not be used, 76% of doctors prioritized shorter prognoses, compared with 45% of patients.
Conclusions
Information on prognosis without treatment is used within the resource allocation strategies of many doctors and most patients, and should be considered as a qualitative modifier during the health economic assessments of new treatments for life-shortening diseases. A single dominant strategy incorporating this information for any quantitative modification of health units is not apparent.
What is already known about this subject
The dominant health economic units upon which new treatment funding decisions are made are the incremental cost per life year gained (LYG) or the cost per quality-adjusted life year (QALY) gained.
Neither of these units modifies the amount of health gained, by the amount of health patients would have had if they had not been given the treatment under consideration, which may unfairly undervalue the treatments for poor prognosis conditions.
How certain patients make decisions about their own treatment has previously been explored, but not how they, or doctors, would allocate hypothetical resource within a healthcare system given information on disease-treatment scenarios' prognoses with and without treatment.
What this study adds
Information on prognosis without treatment is used within the resource allocation strategies of many doctors and most patients.
Individuals use this information in a variety of different ways and a single dominant strategy for quantitative modification of health units is not apparent.
Information on prognosis without treatment, or prognosis with standard treatment, is available from the control arm of randomized controlled clinical trials and should be used qualitatively to facilitate decision-making around the second inflexion point on cost per QALY/LYG acceptability curves.
Keywords: cancer, NICE, prognosis, resource allocation, Scottish Medicines Consortium
Introduction
Within recent years cost-effectiveness has been introduced as an additional hurdle for new drugs to overcome, before they are recommended for resource allocation within a growing number of healthcare systems [1, 2]. Within the UK, the two predominant organizations making these recommendations are the National Institute for Clinical Excellence (NICE) in England and Wales, and the Scottish Medicines Consortium (SMC) in Scotland. The unit values upon which such organizations base their health economic assessments, for each indication of a drug, are usually either the incremental cost per life year gained (LYG) for life-shortening conditions or, where available, the cost per quality-adjusted life year (QALY) gained. As small additional increases in survival for life-shortening conditions conceivably could be very important to those who have a short time left to live, it is noteworthy that neither of these units modifies the amount of health gained, by the amount of health the patients would have had if they had not been given the treatment [3]. Consequently, treatments producing only modest improvements in the survival of short prognosis conditions, such as advanced cancer or severe heart failure, may be undervalued by these measures. Although absolute cost per QALY thresholds for acceptance/rejection are not employed, a sigmoidal relationship between increasing cost per QALY and the probability of a technology being rejected is recognized [4]. The upper inflexion point of such a curve, beyond which a technology is unlikely to be approved in the UK currently lies within the £25 000–£35 000 per QALY range. Within this range NICE has stated that a number of additional factors beyond the cost per QALY are (or should be) considered to facilitate the decision-making process. These factors include the ‘innovative nature of the technology’ and the ‘particular features of the condition and population using the technology’. ‘Particular features’ explicitly incorporates ‘special considerations of equity’ with many of NICE's judgements underpinned by the assumption that a ‘QALY is a QALY is a QALY’, i.e. that the weight given to a QALY should be the same regardless of the beneficiary's age, sex, social role or deprivation, how many QALYs they have ‘previously enjoyed’ or are in ‘prospect of enjoying’[4]. It is the last of these points that we address here, with regard to life-shortening conditions. Prognosis without treatment seemed to inform the choices made in an earlier pilot study in which respondents were asked to choose which one of three hypothetical patients with life-shortening conditions to treat [3]. To explore in more detail the importance of underlying prognosis without treatment, the three commonest resource allocation strategies incorporating information on prognosis without treatment, described by the respondents in the pilot study, were compared quantitatively to the current health economic standard of LYGs (which does not incorporate this information). Decision patterns of doctors and oncology patients were captured, when they were placed in the position of resource-allocators and asked to distribute treatment resource between a series of hypothetical patients whose prognoses from life-shortening diseases and gains from a novel treatment varied. The hypothetical patients and their benefits from treatment, in theory, being representative of the results of phase III clinical studies of different novel treatments on different disease types used as the basis for NICE and SMC decision-making [3].
Methods
A previous qualitative pilot study of doctors, nurses and medical secretaries determined the dominant resource-allocation strategies involving prognosis without treatment to compare against the standard LYG-based strategy (Table 1) [3]. A deductive questionnaire was developed for assessing quantitatively between these four potential resource-allocation strategies (Figure 1). It was explained to respondents, both verbally and in writing that:
Table 1.
Potential resource allocation strategies when information is provided on individuals' prognosis without treatment and gain in lifetime from treatment
| Life years gained (LYGs) – treatment allocation to the individual/group that gains the most extra lifetime from being treated |
| Immediate risk of death (IRD) – treatment allocation to the individual/group that would die the soonest if not treated |
| Percentage increase in life years (PILY) – treatment allocation to the individual/group that gains the greatest percentage increase in their life expectancy from being treated |
| Life expectancy with treatment (LEWT) – treatment allocation to the individual/group that has the greatest total life expectancy when prognosis without treatment and gain in lifetime from treatment are added together |
Figure 1.
Resource allocation questionnaire. The clinical scenario on which the questions are based is described in the methods section
‘There are two hypothetical patients each with a disease that they will eventually die from, but they will be well until they die. You are not told what the disease is, but it could be something like AIDS, or a type of cancer. There is a treatment for the disease that is not a cure, but whoever you give it to will live a period of time longer and they will be well during this extra time. There are no side-effects and there is no inconvenience associated with the treatment, but there is only enough of it to give it to one of the two patients in each question and you have to decide which one. You are not told anything about the patients' personal circumstances, such as their age, or sex, or whether they are married or have children. Whatever you imagine about the two patients, it is the same for both’.
Four questions then followed, each one asking the respondent to choose between treating patients A and B, where the parameters of gain from treatment and prognosis without treatment differed. The answers given to the first three questions separated out the four main resource allocation strategies as follows:
| Potential strategies used: | if chose A | if chose B |
|---|---|---|
| In Q1: | IRD | LYG, LEWT, PILY |
| In Q2: | LEWT | LYG, IRD, PILY |
| In Q3: | IRD, PILY | LYG, LEWT |
Consequently, assuming constancy of method across a narrow range of prognoses without treatment and gains in lifetime from treatment, and that no other relevant strategies existed and cosegregated, an overall resource-allocation strategy for the respondent could be deduced by looking for the one strategy that followed all choices. The answers to Q1, Q2 and Q3, in order, offered eight possible outcomes and deductions:
AAA = no discernible strategy
AAB = no discernible strategy
ABB = no discernible strategy
BBB = lYG-based strategy
ABA = IRD-based strategy
BAB = LEWT-based strategy
BAA = no discernible strategy
BBA = PILY-based strategy
A fourth question, in which the gain in lifetime from treatment was constant for both patients, was added soon after study commencement, to address the role of prognosis without treatment as a potential second-tier resource allocation strategy when LYG-based approaches were not tenable as the primary strategy.
The questionnaire was administered to both doctors, as the potential allocators of resource, and patients attending the Edinburgh Cancer Centre (ECC), as the potential recipients of resource allocation decisions pertaining to life-shortening conditions.
Using a two-group continuity-corrected chi-squared test, a sample size of 175 per group was calculated as being sufficient to detect a 20% difference in strategy preferences, within and between groups, with 80% power at the 5% significance level (two-sided test). A separate subgroup of medical and clinical (radiation) oncologists, aiming for at least 50 individuals, was also recruited to assess whether there was any evidence that doctors with day-to-day experience of life-shortening diseases would give different responses from doctors in general. No randomization steps were involved. Postal returns were permitted. Because of delay in some postal returns, final numbers of completed questionnaires exceeded the original recruitment targets.
On completion, groups (excluding responses with no discernible strategy) were initially compared using chi-squared or Fisher's exact tests (depending on the number of events in each cell). For those that showed a statistically significant difference (P < 0.05), each strategy choice was then compared with the same strategy choice between groups using a binomial test of proportions.
Results
Three hundred and thirteen doctors were approached, of whom 283 completed the questionnaire (33 questionnaires containing only questions 1–3, 250 questionnaires containing questions 1–4). Of these, 84 were oncologists (medical or clinical) of specialist registrar/staff grade or higher, 57 of whom completed the questionnaire (all containing questions 1–4). Two hundred and ninety-eight patients were approached, of whom 201 completed the questionnaire (26 questionnaires containing only questions 1–3, 175 containing questions 1–4).
There was no evidence of a framing effect from question 4, with the proportions of doctors and patients answering A or B to each of questions 1–3 being similar regardless of whether the fourth question was present or absent (data not shown). Data from all questions 1–3, regardless of the presence or absence of question 4 within the same questionnaire, were combined in the analysis of each respondent group. The resource allocation strategies deduced from the answers given to questions 1–3 for doctors and patients, and their answers to question 4, are shown in Table 2. Oncologists and nononcology doctors produced identical rankings, but differed in terms of the proportions choosing LYG-based (64% vs. 49%, respectively, P = 0.031) and LEWT-based strategies (0% vs. 9%, respectively, P < 0.001). The patients differed from the doctors (all doctors combined) for both the strategies deduced from questions 1–3 (chi-squared P < 0.001) and for question 4 (chi-squared P < 0.001). Using a binomial test of proportions, there was a statistically significant difference between doctors and patients for each specific resource allocation strategy (P < 0.001), except IRD (P = 0.102).
Table 2.
Resource allocation strategies of doctors and oncology patients within the questionnaire
| Doctors (total) | Doctors (nononcologists) | Doctors (oncologists) | Oncology patients | |
|---|---|---|---|---|
| Deduced resource allocation strategy (questions 1–3): | ||||
| Total = 283 | Total = 226 | Total = 57 | Total = 201 | |
| LYG | 142 (50%) | 106 (47%) | 36 (63%) | 65 (32%) |
| IRD | 35 (12%) | 27 (12%) | 8 (14%) | 35 (17%) |
| PILY | 77 (27%) | 65 (29%) | 12 (21%) | 21 (10%) |
| LEWT | 20 (7%) | 20 (9%) | 0 (0%) | 73 (36%) |
| No discernible strategy | 9 (3%) | 8 (4%) | 1 (2%) | 7 (4%) |
| Allocation when LYGs equal between patients (question 4): | ||||
| Total = 175 | Total = 249 | Total = 193 | Total = 56 | |
| A – to shorter prognosis | 188 (76%) | 142 (74%) | 46 (82%) | 79 (45%) |
| B – to longer prognosis | 61 (24%) | 51 (26%) | 10 (18%) | 96 (55%) |
Discussion
Healthcare resource allocation bodies, commonly base their health economic decision-making on the cost per LYG or, per QALY, of each specific therapeutic interventional scenario. Prognosis without treatment (or ‘prognosis with standard treatment’, as opposed to that with the new therapeutic intervention under consideration) seemed to inform respondent treatment-allocation choices made in an earlier pilot study [3]. The same broad concept, expressed as ‘urgency of need’ has also been argued as a relevant factor in the allocation of a limited supply of donor organs to potential transplant recipients, although there are certain differences, e.g. the assumption that treatment equates to ‘cure’ in many transplant cases [5, 6].
Within the confines of the deductive questionnaire employed in the present study, for both doctors and patients the number of respondents without discernible strategies was proportionally very low (<4% in all groups), implying that the choices made were rarely random. Random choice would have produced responses without a discernible strategy in 50% of cases. Non-oncology doctors and oncologists produced identical rankings and only minor differences in their proportional splits in terms of their preferred resource allocation strategy: LYG > PILY > IRD > LEWT (Table 2), suggesting that precise medical background or particular exposure to life-shortening conditions has little influence on the strategy employed by doctors. Within the study costs were not mentioned, and it does not seem unreasonable to assume that the costs from the perspective of the decision-maker were therefore equivalent, or equally irrelevant, for each choice. Since quality of life both on and off treatment was explicitly stated as good (i.e. fixed) within the study, these results provide the first approximate validation by doctors of cost per LYG or cost per QALY as the dominant resource allocation strategies employed by bodies such as NICE and the SMC.
Of the other strategies explored in this study, PILY- and IRD-based strategies are similar in that they both emphasize value in relation to the shortness of the prognosis without treatment (in contrast to a LEWT-based strategy, which places greater value on longer prognosis without treatment). While the predominance of LYG-based strategies among doctors may initially appear to negate the earlier suggestion that prognosis without treatment is relevant to resource allocation, when the proportions choosing PILY- or IRD-based strategies were combined, a good case for providing information on shortness of prognosis regardless of LYG was apparent with 40% (112/283) preferring a strategy employing this information compared with 50% (142/283) preferring a strategy that did not. In addition, in question 4, when the LYGs associated with treatment were the same for each choice, there was a clear preference for allocating resource to the individual with the shortest prognosis without treatment (Table 2). The very low percentage of doctors favouring LEWT-based resource allocation strategies, particularly among oncologists, also supports this view.
Although doctors and other healthcare professionals are traditionally the individuals most associated with healthcare resource allocation, there is a move to involve the consumers of healthcare in policy making [7, 8]. To this end we solicited the views of patients attending the ECC as examples of those with experience of life-shortening diseases. The patients' resource allocation decision-making was very different from that of the doctors, both in terms of ranking (LEWT > LYG > IRD > PILY) and proportional splits. The most notable difference between the doctor and patient groups was the prioritization by 36% of patients of a strategy valuing length of overall survival with treatment (LEWT-based strategy), apparently rating longer prognoses without treatment above shorter ones when allocating resource (Table 2). It is important to note that decision-making about resource allocation to others is potentially very different from personal choices about one's own treatment, an area that has previously been explored for cancer patients to some extent [9]. Patients with advanced cancer have been noted to be more likely to accept aggressive treatment aimed at extending life if they believed they had a high chance of surviving 6 months on standard therapy, compared with those who believed they were unlikely to survive 6 months, broadly in line with the principles of LEWT-based resource allocation [9]. In question 4, however, the patient group was fairly evenly split as to whether they emphasized length or shortness of prognosis without treatment when LYGs were the same. We have previously argued that a universally applied LEWT-based strategy would not be tenable as a resource-allocation strategy in health economic assessments, because of the potential to underemphasize or ignore the impact of the treatment under consideration [3]. Perhaps, despite the questionnaire being worded in a non disease-specific manner about allocating resource to others, rather than to the respondents, some patients may have been voting for the clinical state that they themselves would prefer, valuing total prognosis absolutely, regardless of how the different elements contributed to the overall figure.
Discrepancies between the decisions made by doctors and by the public (rather than specifically by patients) have been noted with respect to other resource allocation issues, including donor organs for transplant and ‘do not resuscitate’ orders [10, 11]. These discrepancies initially raise issues about ensuring that all sides are fully informed and the consequences of decisions made clear [10, 12]. Although the questionnaire employed in both the pilot study and the present study did not constrain the diagnoses of the hypothetical patients to any particular condition, questions also arise as to what extent the responses from any patient group (in this example, oncology patients) are generalizable to other patient groups (e.g. those with other life-shortening diseases) or the general public. Assuming that these issues can be adequately addressed, the present study strongly suggests that prognosis without treatment is important in many doctors' and most patients' resource allocation decision-making (PILY + IRD + LEWT = 47% and 64%, respectively). On this basis we reject the notion that a ‘QALY is a QALY is a QALY’ under the life-shortening conditions explored within this study. However, a single dominant resource allocation strategy, incorporating prognosis without treatment information, suitable for quantitative use within health economic assessments is not apparent and the continued use of LYG-based strategies as the quantitative standard is supported [3]. Even when individuals use prognostic information in a comparable manner (e.g. valuing shortness rather than length of prognosis without treatment), they seem to do so in subtly different ways. This suggests that prognosis without treatment should instead be supplied primarily as a qualitative facilitator of decision-making around the second inflexion point on the cost per QALY acceptability curve, in a manner akin to the way that information on the ‘innovative nature’ of new technologies is already employed [4]. Information on prognosis without treatment, or prognosis with standard treatment, is freely available from the control arm of the randomized controlled clinical trials already used as source data within existing health economic assessment procedures. We would support the consideration of data on prognosis without treatment during the assessment of all new treatments for life-shortening conditions, allowing the clinical and lay members of resource allocation bodies to then weigh this information alongside other available quantitative and qualitative health economic modifiers as they see fit.
Acknowledgments
We thank Catriona Graham (Wellcome Trust Clinical Research Facility, Western General, Edinburgh) for help with the statistical analysis.
Funding: Chief Scientistís Office Cancer Programme Support Grant 0/0361/01/173.
Competing Interests: There are no conflicts of interest for any of the authors. All authors have approved the manuscript.
Ethics approval: Lothian Research Ethics Committee.
References
- 1.Taylor RS, Drummond MF, Salkeld G, Sullivan SD. Inclusion of cost-effectiveness in licensing requirements of new drugs: the fourth hurdle. BMJ. 2004;329:972–5. doi: 10.1136/bmj.329.7472.972. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Cairns J. Providing guidance to the NHS. The Scottish Medicines Consortium and the National Institute for Clinical Excellence compared. Health Policy. 2006;76:134–43. doi: 10.1016/j.healthpol.2005.05.006. [DOI] [PubMed] [Google Scholar]
- 3.Camidge R, Walker A, Oliver JJ, Nussey F, Maxwell S, Jodrell D, Webb DJ. Prognosis without treatment as a modifer in health economic assessments. BMJ. 2005;330:1382–4. doi: 10.1136/bmj.330.7504.1382. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Rawlins MD, Culyer AJ. National Institute for Clinical Excellence and its value judgments. BMJ. 2004;329:224–7. doi: 10.1136/bmj.329.7459.224. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Eghtesad B, Bronsther O, Irish W, Casavilla A, Abu-Elmagd K, van Thiel D, Tzakis A, Fung JF, Starzi TE. Disease gravity and urgency of need as guidelines for liver allocation. Hepatology. 1994;20:56S–62S. doi: 10.1016/0270-9139(94)90274-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Council on Ethical and Judicial Affairs. American Medical Association. Ethical considerations in the allocation of organs and other scarce medical resources among patients. Arch Intern Med. 1995;155:29–40. [PubMed] [Google Scholar]
- 7.Florin D, Dixon J. Public involvement in health care. BMJ. 2004;328:159–61. doi: 10.1136/bmj.328.7432.159. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Rawlins MD. Pharmacopolitics and deliberative democracy. Clin Med. 2005;5:471–5. doi: 10.7861/clinmedicine.5-5-471. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Weeks JC, Cook EF, OíDay SJ, Peterson LM, Wenger N, Reding D, Harrell FE, Kussin P, Dawson NV, Connors AF, Lynn J, Phillips RS. Relationship between cancer patients' predictions of prognosis and their treatment preferences. JAMA. 1998;279:1709–14. doi: 10.1001/jama.279.21.1709. [DOI] [PubMed] [Google Scholar]
- 10.Neuberger J, Adams D, MacMaster P, Maidment A, Speed M. Assessing priorities for allocation of donor liver grafts: survey of public and clinicians. BMJ. 1998;317:172–5. doi: 10.1136/bmj.317.7152.172. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Manisty C, Waxman J. Doctors should not discuss resuscitation with terminally ill patients. BMJ. 2003;327:614–5. doi: 10.1136/bmj.327.7415.614. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Murphy DJ, Burrows D, Santilli S, Kemp AW, Tenner S, Kreling B, Teno J. The influence of the probability of survival on patients' preferences regarding cardiopulmonary resuscitation. N Engl J Med. 1994;330:545–9. doi: 10.1056/NEJM199402243300807. [DOI] [PubMed] [Google Scholar]