Harnessing the potential to quantify public preferences for healthcare priorities through citizens’ juries

Abstract

Despite progress towards greater public engagement, questions about the optimal approach to access public preferences remain unanswered. We review two increasingly popular methods for engaging the public in healthcare priority-setting and determining their preferences; the Citizens’ Jury (CJ) and Discrete Choice Experiment (DCE). We discuss the theoretical framework from which each method is derived, its application in healthcare, and critique the information it can provide for decision-makers. We conclude that combining deliberation of an informed public via CJs and quantification of preferences using DCE methods, whilst it remains to be tested as an approach to engaging the public in priority-setting, could potentially achieve much richer information than the application of either method in isolation

Background

Public participation can inform decision-making about the distribution and delivery of healthcare, and can be especially valuable in relation to difficult decisions about priority-setting and resource allocation (1). Despite progress towards greater public engagement (2), many questions remain about the optimal approach to adopt (3). The public engagement literature is diverse and spans many disciplines. Due to the influence of differing philosophical paradigms, distinct approaches and conceptualisations of engagement have emerged (4). While most engagement methods rely on qualitative approaches, a distinct body of work has emerged that is dominated by quantitative approaches and focused upon the measurement of public preferences. This paper conjectures that qualitative engagement approaches, and especially approaches that encourage deliberation, can be employed synergistically with quantitative preference-based methods to maximise the value of public input to health policy decision-making processes.

Mechanisms for public engagement

High quality public engagement has been identified as comprising a number of salient characteristics including inclusivity, transparency, responsiveness, and authenticity (5). However, implementation of public engagement does not always achieve this ideal. Given the variable quality of public engagement activities in healthcare, it is not surprising to see an evolving interest in quantifying public and patient preferences to inform priority-setting (6–8). The potential for public preferences to inform decision-making is clear; decision-makers have indicated a desire to consider public views and the public have expressed a desire for their preferences to inform priority-setting decisions (9,10). Public preferences for health-related quality of life are already considered by policy-makers in some jurisdictions in so far as they are incorporated into the comparative benefits gained from healthcare interventions (via a cost-utility framework) (11). However, it has been recognised that patients and society gain social value from the way that healthcare is distributed beyond the health outcome achieved (12,13). Approaches to capturing this value and incorporating it within the evaluation of healthcare have been recognised as important but have not yet been realised in practice (8,14).

There is a substantial literature on the array of methods available to engage the public in healthcare decision-making [(see for example (2,4,15)]. Ryan and colleagues provide a comprehensive systematic review and comparative assessment of the methods that can be used to elicit public preferences for healthcare (9), concluding that “there is no single, best method to gain public opinion”. Nevertheless, they do make recommendations around the appropriateness of selected qualitative and quantitative techniques. Two of their preferred methods, the Citizens’ Jury (CJ) and discrete choice methods, have been gaining prominence in the health literature in recent years [see for example (13,14,16–19)]. Each is associated with a number of features that make them particularly attractive for public engagement (Table 1), making them worthy of further consideration.

Table 1 . Summary of the CJ and DCE approaches .

	Citizens’ Jury (CJ)	Discrete Choice Experiment (DCE)
Theoretical framework	Theories of deliberative democracy	Consumer welfare and random utility theories
Approach	Deliberative; output is from an informed sample.	Stated preference survey; output is from a potentially uninformed sample.
Sampling strategy (selection, sample size)	Random selection across a stratified sample to reflect the diversity of the population. Optimum sample size unknown; generally between 12–24 (20); although up to 30 have been used (22,23).	Ideally a large, randomly selected sample, representative of the population. Minimum sample size of 20 suggested for precision (21); but contentious. Ideally a larger sample.
Data and analysis	Qualitative. Jury discussions and interactions can be recorded, transcribed and analysed thematically.	Quantitative. Choice observations are analysed using regression methods.
Output with information to be used for policy	Juror recommendations following deliberation. These may or may not be unanimous. Output does not capture strength of preference (without use of an additional method).	Captures relative strength of preference. Has the potential to derive preference weights. Trade-offs between desirable characteristics and willingness to pay can be estimated. Weights will represent the average preference across the sample. Has the potential to explore heterogeneity within the sample if a large sample is used.

Citizens’ Juries

For many political theorists and philosophers, the greatest challenge for those seeking to extend and improve the practices of democratic politics has been how to transfer what worked in the relatively small scale of 5^th century BC Athenian politics to contemporary mass societies. Many of those practices are not able to meet the demands of larger populations. CJs offer a way of seeking informed public views using a democratic, deliberative process, while avoiding the problem of demands from large populations by limiting participation to a small number of people.

CJs combine a deliberative process with inclusion of diversity through random selection across a stratified sample (24). Fishkin argued that traditional measures of public opinion, such as polling, simply model the ignorance of the general public (25). When asked for their views on most matters of concern to policy-makers, uninformed members of the general public will offer their views and reveal their preferences, even if they know or care little about these matters. Drawing on extensive survey data from the American electorate, Somin demonstrated the extent of this ignorance, which he describes not as a moral failing but as a rational response to the inability of individual citizens to influence large scale political outcomes (26). CJs seek to address both these limitations, by educating participants through the deliberative process and by offering the prospect of a direct and tangible influence on policy decisions. Thus, this approach offers a potentially powerful contribution to modern policy-making.

In a CJ, a small sample of the public is presented with a policy dilemma for deliberation. They review evidence about its nature and possible resolution, presented by experts who can be cross examined. The initial views of participants are potentially transformed by hearing the reasoning of others and through the requirement for each participant to justify their views. It is assumed that the deliberations of this sub-sample of the population can be taken to fairly represent the conscience and intelligence of the general public (23,27). Allowing a purposive representation of the public to become more informed and then to deliberate about priorities for healthcare offers a potentially more reliable voice than opinion polls, surveys, focus groups or patient representative groups which are likely to present self-interested views and may reflect varying degrees of ignorance about the matter in question (28). There is evidence that jurors in these settings become more actively engaged in debates, express their views, are able to recall fine details about the information presented and, subsequently, develop a greater sense of coherence (16,18,23,27,29).

There have been a number of published applications of the CJ and related deliberative approaches in healthcare priority-setting (16,18,29,30), some of which have been commissioned by health authorities. One of the most prominent examples is the routine use of a variant of CJs by the UK’s National Institute for Health and Care Excellence (NICE) for priority-setting guidance (23). It could be argued that the NICE Citizens’ Council, which is essentially a standing body, is not really a model of a CJ, which is usually constituted to address a particular topic. The former is an institutionalised engagement model which over time could become part of the establishment itself, rather than an independent group of citizens (3). Nevertheless, NICE limit the term served by members of the Council to three years, which might counteract this tendency. The definition of a CJ in the literature is somewhat subjective, and does not deal with nuances of the method such as when a citizen becomes “institutionalised”. If the CJ method is to be scientifically acceptable, there will need to be some clarification defining the methodological boundaries of what constitutes a CJ. The UK is not the only jurisdiction to use this approach for public engagement in priority-setting, with interest in the approach in Canada, Australia and New Zealand (16,19,30,31). However, to our knowledge the UK is the only country so far to routinely embrace a CJ approach, rather than investigate their role through more ad hoc research projects.

What information do CJs provide for decision-makers?

CJs potentially promise a number of tangible benefits to policy and decision-makers. If implemented well with unbiased selection of participants, robust deliberation supported by mutual trust, and appropriate facilitation and expertise, CJs offer an opportunity to address the challenges facing those who seek to promote public participation in decision-making. Unlike many contemporary forms of public debate (especially those conducted online) that can degenerate rapidly into unpleasant exercises in abuse and ridicule, the more structured process of the CJ better guarantees the traditional Millsian right to free expression and an obligation to disagree respectfully (32). It is expected that the group of lay people who form a Jury will have different views and preferences, but the process of deliberation respects these differences whilst also striving for common ground. There is a belief that decisions made by a wider group of people (i.e. as in a CJ) are better in some way than those made by smaller, self-selecting groups of experts (33). It is assumed that the wider group is better placed to identify practical and logical flaws and provides a greater degree of political legitimacy to decisions.

Despite these potential benefits, there are shortcomings associated with CJs, and their uptake into priority-setting has been variable. Methodological uncertainties associated with the CJ approach need addressing before it can become a method routinely relied on by decision-makers. These include the optimal number of people to participate in a CJ, how to select and recruit witnesses, and the extent to which the recommendations of the Jury are representative of broader public preferences. CJs are designed to deliver a consensus among the group and to achieve this in a descriptive way, but typically they do little to quantify preferences surrounding healthcare decisions. Thus, there is a need to capture the relative strength of preferences among the members of the group as well as their final consensus opinion.

Discrete choice methods

Prominent among the approaches for quantifying public preferences for priority-setting are choice-based methods, such as the DCE (9,14,34). DCEs typically involve the presentation of a series of choices in which respondents are asked to choose one of two or more alternative scenarios, each representing a unique combination of specified attributes and levels of the treatment or service, under consideration (14,21). Statistical analysis of the individuals’ choices identifies the relative importance of the attributes and the trade-offs individuals make when choosing one scenario over another (i.e. the amount of one attribute they are prepared to forgo to gain more of another).

The DCE is grounded in random utility theory (35–37) which views the utility of a health program or priority-setting alternative as consisting of a systematic or observable component and a random component that is not observed. DCEs also draw on Lancaster’s theory of consumer demand which views the utility of the program or alternative as being composed of the utility of each of its distinct parts or characteristics (38). Thus, the DCE method relies on some key assumptions, most notably that participants make decisions in such a way as to maximise their own utility (39). In measuring social preferences for priority-setting (40–42), the DCE method makes the implicit assumption that individuals are altruistic in their decision-making and that their preferences can be aggregated into a meaningful social whole. This presumes that for an individual, making choices so as to maximise the utility of society equates to the same decision rule as maximising their own utility. The firm theoretical basis in consumer welfare theories has led to the increasing popularity of the DCE method to assess preferences (14).

The DCE has become a popular instrument for quantifying healthcare preferences and its specific use in quantifying preferences related to priority-setting is also increasing (13,14,40–43). It has been used to measure preferences not only around the desirable characteristics of a healthcare service per se, but also around the distribution of healthcare within a population (44). It has also been used to assess preferences for the funding of healthcare, including the additional level of taxation people are willing to accept for a health or healthcare improvement (45,46). Although the uptake of the method by policy-makers is largely unknown, there have been prominent applications of the method to elicit social preferences backed by substantial public funding [e.g. the UK “Social Value of a QALY” project (47)].

What information do DCEs provide for decision-makers?

The DCE can be administered relatively easily to a large, randomly selected representative sample of the population (9). It is arguably a less resource intensive method of community engagement than many other approaches; although, resource and costs would likely be high for large sample sizes. It measures not only the direction of preferences around a topic (e.g. should health gain attributed to young children be weighted more highly than those attributed to older people?), but also the relative strength of preference for one alternative policy choice compared to another (e.g. how much extra weight should be attributed to young children), and the trade-offs that respondents would be willing to make between different characteristics of that choice.

The usefulness of most preference-based approaches (including DCEs) may be limited when the respondents represent what might be called a naïve sample of the general public, that is they lack personal knowledge or experience on the issue and, thus, little weight can be given to the results (9,48). This is exacerbated, since respondents to a DCE have generally not had the opportunity to deliberate an issue before their preferences are elicited. Indeed, there is some indication that deliberation might affect preferences (49), but the extent of the impact and, most importantly, the implications for decision-making have not been widely evaluated. Nevertheless, use of a general public sample who do not have specific experience of a particular issue may be helpful in avoiding the “veil of experience” (tendency to prefer that which is familiar) which has been reported when eliciting patient preferences (50). When eliciting preferences for priority setting (or indeed deliberating issues in a CJ), it is important to also consider the extent to which the individuals providing their preferences have had experience with the issue at hand.

The potential for a combined methods approach

Studies on public input into priority-setting typically adopt either a qualitative or quantitative approach (9). There is clear potential for combining qualitative and quantitative methods to address priority-setting dilemmas. Some previous engagement methods, such as the Choosing Health Plans All Together (CHAT) methodology developed in the United States (51,52), have combined qualitative and quantitative approaches to public participation. However, these approaches have generally reported descriptive measures of preference (e.g. proportion choosing a certain option) rather than relative strength of preference by eliciting preference weights for the characteristics of each option based on trade-offs and opportunity cost.

A mixed methods approach combining the CJ deliberative method and a method capable of weighting preferences for priority setting has not to date been trialled. This combined approach would potentially allow the in-depth deliberation of topical issues in healthcare priority-setting via the CJ approach, with quantification of the preferences of the Jurors via a DCE. Providing at least twenty jurors participate, this could provide both comprehensive guidance on the opinions and the relative strength of preference of informed jurors around a priority-setting topic. Preferences could be measured before and after the CJ, to measure not only the strength of preferences around the issue, but also any impact that deliberation might have on preferences. Sampling theory does not provide clear guidance for DCE sample sizes that are required to give precise preference estimates; sample sizes are generally estimated according to rules of thumb, with a suggested minimum of 20 required to estimate a preference model (21). However, others consider a larger sample size is required (34,48), and most DCE studies have used samples greater than 20 participants (14). For a combined approach to be viable, the feasibility of estimating a DCE based on a sample of 20 participants (which is also close to the maximum sample that would be recommended to participate in a CJ), requires empirical confirmation.

One further potential benefit for the combined approach is the possibility that completion of a DCE before a CJ could allow jurors to become more familiar with the priority-setting context and possible trade-offs that might be involved. Whether this phenomenon occurs, and if so whether it is beneficial to the quality and extent of engagement, would require further exploration. The same DCE could also be used to assess the preferences of a larger statistically representative but uninformed public sample on the same topic, enabling an assessment of representative views and the extent to which they differ from the views of the Jurors before they undertook their deliberations. Intuitively this combined approach might be expected to offer benefits over a single approach alone; however, empirical investigation is required to test the impact of a combined approach on decision-making processes and outcomes.

Conclusions

To encourage a greater and appropriate uptake of public engagement by health policy-makers, it is imperative that researchers address the methodological uncertainties outlined in this paper. Apathy and ignorance constitute two serious challenges for contemporary democratic politics. There is evidence of growing political disengagement among the citizens of many OECD countries and of increasing lack of trust in political leaders and representatives at all levels of government. However, there is also evidence that some citizens are willing to participate in more thoughtful and intensive forms of political debate, especially if these are seen to make a tangible contribution to policy developments. CJs offer the prospect of just such an informed and thoughtful process, but they are necessarily limited to a small number of participants. By using a combination of random and stratified selection techniques to engage a large sample of participants and by imposing the rigour of DCEs during the course of Juries with a statistically representative population sample alongside the Juries, the outcome of these deliberations has the potential to be more realistic (in policy terms) and more representative (in political terms). Empirical research on the feasibility and outcomes of this unique combination of CJs and DCEs would reveal whether or not these putative benefits are realised in practice.

Acknowledgements

This manuscript was prepared with the support of funding from an Australian Research Council Linkage Grant (#LP100200446). Jennifer Whitty was supported by a research fellowship from the Queensland Government Department of Employment, Economic Development and Innovation, Queensland Health and Griffith University.

Ethical issues

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Authors’ contributions

All authors conceived and contributed their perspectives to the concept underpinning this manuscript. JW drafted the manuscript; all authors contributed to the manuscript, reviewed and revised the manuscript for academic content, and approved the final version.

Authors’ affiliations

¹School of Pharmacy, Faculty of Health and Behavioural Sciences, The University of Queensland, Brisbane, Queensland, Australia; and Population and Social Health Research Program, Griffith Health Institute, Griffith University, Logan campus, University Drive, Meadowbrook, Queensland, Australia. ²Urban Research Program, Griffith School of Environment, Griffith University, Gold Coast campus, Southport, Queensland, Australia. ³Centre of National Research on Disability and Rehabilitation, Population and Social Health Research Program, Griffith Health Institute, Griffith University, Logan campus, University Drive, Meadowbrook, Queensland, Australia. ⁴Flinders Health Economics Group, School of Medicine, Flinders University, Adelaide, Australia. ⁵Menzies Centre for Health Policy, School of Public Health, University of Sydney, New South Wales, Australia. ⁶Division of Health and Social Care Research, King’s College School of Medicine, London, UK. ⁷Centre for Applied Health Economics, Population and Social Health Research Program, Griffith Health Institute, Griffith University, Logan campus, University Drive, Meadowbrook, Queensland, Australia.

Citation: Whitty JA, Burton P, Kendall E, Ratcliffe J, Wilson A, Littlejohns P, et al. Harnessing the potential to quantify public preferences for healthcare priorities through citizens’ juries. Int J Health Policy Manag 2014; 3: 57–62. doi: 10.15171/ijhpm.2014.61