Public involvement in health care priority setting: an economic perspective

Abstract

Background Public involvement in health care decision making and priority setting in the UK is being promoted by recent policy initiatives. In 1993, the British Medical Association called for public consultation where rationing of services was to be undertaken. The approach to priority setting advocated by many health economists is the maximization of quality adjusted life years (QALYs). Typically, for a particular health care programme, the QALY calculation takes account of four features: (1) the number of patients receiving the programme, (2) the survival gain, (3) the gain in quality of life and, (4) the probability of treatment success. Only one feature, that relating to quality of life, is based upon public preferences. If the QALY is to be used as a tool for health care resource allocation at a societal level then it should incorporate broader societal preferences.

Methods This study used an interview‐based survey of 91 members of the general public to explore whether the traditional QALY maximization model is a good predictor of public responses to health care priority setting choices.

Results and conclusions Many respondents did not choose consistently in line with a QALY maximization objective and were most influenced by quality of life concerns. There was little support for health care programmes that provided a prognostic improvement but left patients in relatively poor states of health. The level of respondent engagement in the survey exercise was not sensitive to the provision of supporting clinical information.

Introduction

The National Health Service (NHS) reforms in the early 1990s emphasized increased patient choice in health care decision making. ¹ National Health Service documents, such as Local Voices, called for increased dialogue between health authorities and local communities about the future of local services which reinforced this message. ² This theme runs through the 1997 white paper which emphasizes public participation in health care decision making. ³ Thus, many of the key proposed changes in the organizational structure of the NHS have implications for greater public and/or patient consultation. Health authorities, for example, are expected to involve the public in developing health improvement programmes; health action zones are described as offering opportunities to explore new approaches to involving local people in decisions about the delivery of health care; and primary care groups (PCGs) are expected to have effective arrangements for public consultation.

In 1993 the British Medical Association (BMA), called for public consultation in health care decision making where rationing of services was to be undertaken. ⁴ The rationing debate was epitomized in 1995 by the case of Child B, a young leukaemia sufferer, whose local health authority had refused to fund a second bone marrow transplant. The child’s father challenged the decision of the health authority in the High Court, which ruled that the decision should be reconsidered. However, the Appeal Court upheld the health authority’s right to refuse treatment. The details of the court cases and whether or not the decision was based on clinical opinion or was an example of explicit rationing, are discussed in detail elsewhere. ^{5 , 6}

The debate that followed the Child B case highlighted two principal views: first, that some form of rationing is inevitable in a cash limited NHS, and second, that there should be more explicit public debate about the principles and issues concerned in health care resource allocation decisions. ^{7 , 8} Incorporating the public’s views as an element in the decision‐making process might help decision makers tackle the most difficult areas of resource allocation and ensure they can present appropriate justification for these decisions. However, the most appropriate methods for involving the public have not been established. Citizens’ juries are being tested but the information requirements of jurors and the dominance by more articulate and confident members are issues to be resolved. Other studies have used a simple market research technique, such as an opinion poll, but this approach has limitations from an economist’s perspective in that it fails to adequately measure a respondent’s intensity of preference and does not incorporate some notion of sacrifice. ⁹ The approach adopted in the empirical work reported in this paper attempts to overcome some of the weaknesses of these other studies in that the nature and level of the opportunity cost, or sacrifice, associated with resource allocation decisions is made explicit.

The approach to priority setting in health care promoted by many health economists is the maximization of quality adjusted life years (QALYs). ^{10 , 11} This approach is not without its critics but represents one set of criteria that might be considered when discussing rationing issues, and is the model at the core of the arguments outlined in this study.

The traditional QALY maximization approach

The traditional QALY maximization approach combines two forms of outcomes that flow from health care interventions, namely quality of life and survival. These two outcomes are combined into a single index number (life years adjusted for quality) which is used to compare levels of benefit between interventions. The methods used in the calculation of QALYs are described elsewhere. ¹²

For a particular health care intervention or programme, the typical QALY calculation would take account of four features:

• the number of patients receiving the programme;

• the survival gain;

• the gain in quality of life and;

• the probability of the treatment being successful.

The intervention or programme that produces the largest number of additional QALYs, for a given cost, is considered the most efficient. However, the traditional QALY measure has only one dimension based upon public preferences (i.e. quality of life). ¹³ In general, there has been little consideration of distributional preferences and preferences relating to risk in the health sector. The issue of risk was highlighted in the Child B case when the question was raised about the appropriateness of using resources in cases where the chance of success was very low. ⁸ Distributional considerations are obviously relevant if QALY maximization is to be viewed as an attempt to maximize health‐related social value, in the sense of society’s preferences for different health‐related outcomes. ¹⁴ For example, a distribution of benefits that resulted in one individual receiving additional QALYs might not be viewed as having the same social value as a larger number of individuals receiving an equivalent QALY gain.

Key questions

If QALY is to be used as a tool for informing judgements in health care resource allocation at a societal level, it is essential that preferences not just for quality of life but also for the other components, notably risk and distribution of benefits, are incorporated. Two sets of distributional concerns can be distinguished:

(1) those relating solely to the number of people who receive treatment (i.e. preferences for more rather than fewer beneficiaries in terms of a wider distribution of health benefits, regardless of who receives them);

(2) those relating to the personal characteristics of those who receive treatment (i.e. preferences to be given to some groups in society over others).

Williams ^{15 , 16} has advocated that consideration should be given to personal characteristics in resource allocation decisions. He argues that:

…the best way to integrate efficiency and equity considerations in the provision of health care would be to attach equity‐weights to QALYs. QALYs measure benefits to health care in standard units, and equity‐weights allow benefit valuation to become person‐specific to the extent that it is policy‐relevant. ¹⁵

Williams’ view on equity in resource allocation stems from a belief in the ‘fair innings’ argument: if certain groups in society are not achieving what society views as a ‘fair innings’, either in terms of life years or quality‐adjusted life years, then they deserve to be given favour in terms of access to health care. ¹⁶ The first set of distributional concerns, dealing with numbers receiving treatment, were incorporated in the design of this study. The second set of distributional concerns, whilst important and worthy of investigation, were viewed as beyond the scope of the study reported here.

Public preferences: an empirical investigation

An investigation of public preferences in the context of health care resource allocation decisions was undertaken. The study had two core objectives. First, it sought to explore whether the traditional QALY maximization model is a good predictor of public responses when the components of the QALY calculation are made explicit in terms of distributional issues (i.e. the number of people being treated) and risk (i.e. the chance of treatment success). Second, the exercise was designed to investigate alternative approaches, in terms of questionnaire design and content, for conducting public surveys on health care priority setting issues. Specifically, the level of variation in respondent engagement with the inclusion or exclusion of clinical information was explored.

Methods

QALY maximization and public choices

Conjoint analysis methods were employed as a means of generating scenarios for presentation. Conjoint methods are widely used in other fields of economics, notably transport economics, and were employed here to ensure that the choices presented covered a broad range of trade‐offs between all components of the QALY calculation. Previous conjoint analysis studies in health care have focused on specific clinical questions, such as the management of miscarriage ¹⁷ or diagnostic strategies for knee injuries. ¹⁸ There have been no similar studies which have adopted a conjoint analysis framework for eliciting broad public preferences in health care decision making.

Four attributes were selected to allow investigation of the core components of the QALY model (as defined above):

(1) the chance of success of the intervention (four levels: 0.1%, 1%, 10%, 50%);

(2) the number of people receiving treatment (three levels: 1, 10, 100);

(3) survival gains if treatment is successful (two levels: 1 year, 5 years);

(4) quality of life defined using EuroQol EQ‐5D health states (four levels) ¹⁹ ;

• full health (tariff 1.00);

• usual activities: some problems/depression: moderate (tariff 0.893);

• usual activities: unable to perform/depression: moderate (tariff 0.566);

• usual activities: unable to perform/depression: extreme (tariff 0.401).

The levels were selected to allow investigation of a reasonable range within each attribute whilst being aware of the need to present scenarios that respondents would see as plausible. However, to limit the number of variables presented to respondents, a decision was made to focus only on two components of EuroQol EQ‐5D, namely usual activities and depression/anxiety. Given that the background to the study was the case of Child B, the range of attribute levels was selected to include characteristics associated with such cases. This dictated the inclusion of low chances of success, relatively small numbers of patients, short survival periods but a wide range of possible quality of life states. The range of quality of life descriptions reflects the fact that some treatments for people with serious conditions are associated with ‘harms’ that represent important adverse consequences in terms of quality of life.

From the full range of 96 combinations of attributes and levels, a subset of 16 were identified for use in the survey, using an orthogonal main effects design. ²⁰ In general terms, the methods used for the conjoint design were similar to those reported in other health conjoint analysis exercises (for example, see Bryan et al. ¹⁸ ).

Once the scenarios had been generated they were discussed by all members of the research team who defined a suitable clinical situation for each in terms of the disease and associated intervention. This was considered important, whether or not the clinical information would be presented to the public, to allow the research team, the interviewers and the respondents to be confident that the information being presented reflected clinical situations that might require prioritization decisions. The 16 scenarios (defined as A to P) were randomly paired to form eight choices and this procedure was carried out three times to generate three different sets of choices and three separate questionnaires. Summary details of the choices presented are given in Table 1. The pairing process led to some choices with a dominant scenario (that is, better or no worse on all attributes). These dominant choices were included to allow internal consistency to be assessed. Examples of the scenarios presented are given in Fig. 1.

Table 1.

 Summary details of scenarios

Figure 1.

 Example scenarios.

An assessment was made of the proportion of respondents who chose in line with QALY maximization, and where a large proportion (arbitrarily defined as 30% or greater) chose against a QALY maximization line, the relevant choices were examined to establish whether any particular attribute appeared to be driving the choice against QALY maximization.

Provision of clinical information

There was uncertainty amongst the research team about whether or not the clinical details associated with each scenario should be presented to respondents. The balance to be struck was between ensuring that respondents were engaged by the questions and viewed the scenarios as plausible (which would tend to favour providing details of the clinical information), and avoiding the inevitable biases in responses that would be introduced by reference to specific diseases, such as cancer or HIV/AIDS.

The effect of presenting different levels of clinical information to respondents was therefore investigated by varying the information available to three subgroups in the following manner:

• respondents from Group 1 were presented with all the clinical information relevant to the scenario;

• respondents from Group 2 were not initially presented with the clinical information but if the interviewer perceived a lack of engagement or the respondent asked for further information then it was provided, and this was recorded;

• respondents from Group 3 were presented with no clinical information. If further information was requested then this was recorded but the interviewer did not reveal it to the respondent.

Refusal by respondents to provide a response to all choices was used as a proxy for poor engagement and the three panels were compared in this respect. The prior expectation was that groups 2 or 3 would have more refusals than group 1. Such a result would indicate that the lack of relevant clinical information resulted in poorer respondent engagement.

Data collection

Data collection was undertaken using structured face‐to‐face interviews conducted by a public survey company (MORI), using experienced interviewers. Information was elicited from respondents by the presentation of the series of choices involving two alternative health care scenarios. Each respondent was asked to express a preference for one of the two options for every choice. The interviewer read out the scenario descriptions (see Fig. 1) to respondents who were able to view a show‐card that contained only the summary information about the choice. The context described to respondents was one of a Health Authority facing a situation where it is unable to fund all it would wish to and so a series of difficult choices have to be made. There were 91 randomly selected interviewees. Although a number of conjoint analysis studies have been conducted in health care, they have tended to focus on specific clinical questions, such as the management of miscarriage or diagnostic strategies for knee injuries. ^{17 , 18} Thus, the nature of the preferences and the level of variation in responses expected in this survey was difficult to predict. Therefore, an arbitrary judgement on sample size was made since a formal calculation was not possible.

The 91 respondents were divided into three groups, each receiving one of the alternative designs of questionnaires; two groups had 31 respondents each and the other had 29. Age, sex, ethnicity and socio‐economic status were similar between groups and, broadly reflected the population of a health district in south‐east England.

Results

Of the 91 respondents in this survey, 46 were female. The mean age of the whole group was 44 years (range: 16–78). Most (73/91) indicated that their current health was either ‘very good’ or ‘good’, and most (61/91) did not have a long‐standing illness, disability or infirmity. One respondent failed to provide data on all eight choices, providing answers for just five of the eight choices included in the questionnaire. The maximum number of data‐points, in terms of responses to choices, which would have been available from the study was 728 (i.e. if all 91 respondents had completed all 8 choices); the total number provided by respondents was 725.

For each of the choices posed to respondents, the expected QALY gain was calculated for each scenario and so the choice that would be made by respondents adopting a strict QALY maximization line was established. In both groups 1 and 2 one of the eight choices had scenarios with the same QALY score and so it was not possible to identify a QALY maximizing option. This is shown in Table 2 along with the actual choices made for each question. It is clear that for some of the choices, many of the respondents did not choose in line with a QALY maximization approach. For all choices, the range is from as low as only 23% following a traditional QALY line to as high as 93%. Where the QALY difference between the scenarios was very large (i.e. in excess of 200 QALYs) then nearly all respondents (93%) in all panels followed the QALY maximization line. However, even for those choices where the QALY difference was still quite large (i.e. between five and 10 QALYs), almost 30% of respondents did not choose in the way predicted by a QALY maximizing line. Thus, in general terms, the data indicate that a substantial proportion of respondents did not view QALY maximization as the appropriate objective in the resource allocation exercise.

Table 2.

 QALY maximization and public choices: results

The level of support for a QALY maximization line was compared between the three groups. For Group 1 where clinical information was provided, in only two (of the eight) choices did 70% or more of respondents choose in line with QALY maximization. However, in groups 2 and 3 where limited or no clinical information was provided, for four (of the eight) choices, 70% or more of respondents chose in line with QALY maximization. This finding provides some support for the hypothesis that respondents were likely to be highly influenced by the provision of clinical information and so chose against a QALY maximization line. This hypothesis was supported by anecdotal evidence from the interviewers, who explained that the clinical information appeared to induce an emotional and not a considered response. For virtually all of the choices in groups 2 and 3 where fewer than 70% of respondents chose in line with QALY maximization, the scenario that was chosen was superior in terms of quality of life. Thus, it appears that the principal driver of choices against QALY maximization in groups 2 and 3 was the level of the quality of life attribute. Respondents were, in general, reluctant to support programmes that left patients in poor states of health, even where the other attributes were more favourable.

The level of engagement of respondents in this survey did not appear sensitive to variation in the levels of clinical information provided. All three groups had high completion rates and, thus, we assume high levels of engagement. Again, this was strongly supported by anecdotal evidence from the interviewers during the interview debrief meeting. Of the 31 respondents in group 2, only one was given additional clinical information. Thus, from the evidence of this study there seems little to be gained from providing supporting clinical information to respondents in such surveys. Four dominant choices were included in the survey (at least one in each version of the questionnaire). In total, six respondents ‘failed’ the consistency test and chose an alternative that was by definition ‘inferior’. The six were distributed evenly between the three groups. If such ‘failures’ can be taken as a proxy for poor responder engagement then again it would appear that the failure to provide additional clinical information did not lead to poor engagement and is a further argument for not providing such information.

Discussion

The first main objective of our study was to explore the traditional QALY maximization model and to establish whether or not it was a good predictor of public responses when the components of the QALY were made explicit in terms of distributional issues and risk. The main message to emerge from the empirical work reported in this paper is that, in general terms, many of the respondents did not choose in line with QALY maximization. Whilst the level of this inconsistency with a QALY maximization line varied to some degree with the magnitude of the difference in QALYs between the options, it would not be reasonable to attribute inconsistencies to respondent error. The findings provide corroboration for the similar results found in other surveys conducted by Nord and colleagues. ²¹ It, thus, confirms the need for further empirical work which would provide a test of the hypotheses generated in this study relating to why respondents were not choosing in line with QALY maximization. For example, since quality of life issues appeared to be a driving factor in choices against QALY maximization, the robustness and implications of this finding need to be explored in further work. It would appear that the level of quality of life that patients might be expected to achieve following treatment is viewed as an important issue in its own right. Thus, there was little support for health care programmes that left patients in relatively poor health states, even though this represented a prognostic improvement. In general, this study highlights the importance of the constituent components of the QALY model being more fully based on public preferences.

The second main objective of our study was to investigate alternative approaches to questionnaire design and content for conducting public surveys on health care priority setting issues. Empirical work along the lines of that reported here, which investigates public preferences for health‐related outcomes, is faced with a difficulty relating to respondent engagement. A choice has to be made between providing respondents only with information on the key attributes of interest, such as survival and quality of life, and providing additional supporting details in terms of clinical conditions, age of patients, nature of the interventions, etc. The advantage of providing only the bare facts is that the pure respondent preferences for the key attributes are elicited and they are not confused by the extraneous supporting information. However, this is only achieved if respondents are highly engaged in the exercise and, thus, their stated responses to the hypothetical questions truly reflect their underlying preferences that would be revealed if they were in a position to make such decisions. The concern is that the artificial nature of the exercise negates the revelation of true preferences, and so perhaps it is better to provide supporting details which will ‘muddy the waters’ but provide a truer reflection of preferences. This study provides no support for the provision of additional clinical information; the respondents appeared to be highly engaged in the exercise when no clinical information was provided.

A weakness of the survey approach reported in this paper is the reliance upon quantitative methods and approaches in data collection and analysis. Others have suggested that public preferences regarding health outcome and distributional issues are not immediately accessible and are open to manipulation by the way the questions are framed. ²² If this is the case, there exists an important role for qualitative work in this area at three levels. Firstly, at the questionnaire design stage, in‐depth discussions with potential respondents would help to explore the extent to which there is misunderstanding in the way the questions are framed, and such discussions could investigate whether true preferences are being accessed by the questions posed. Secondly, qualitative work at the preference elicitation stage might allow more considered and well‐informed preferences to be revealed, especially if that were to happen as part of a group exercise. Thirdly, qualitative methods have an important role in the interpretation of the results of questionnaire‐based studies, in terms of helping to understand the meaning of the results and to begin to explore the factors that are driving the results. However, the resource constraints within which health research has to be conducted inevitably implies striking a balance between eliciting considered preferences from a small well‐informed group and eliciting less well considered preferences from a larger, more representative, sample.

Acknowledgements

We are grateful to the members of the project’s advisory panel who have provided comments on an earlier draft of this paper (Chris Ham, Mandy Ryan and Rob Sheldon) and our other colleagues at HSMC who have discussed this project with us, notably Shirley McIver and Darrin Baines. Particular thanks goes to Alan Williams who discussed an earlier draft of the paper at the Health Economists’ Study Group meeting in Galway, 1998. Source of funding: East and North Hertfordshire Health Authority.