What is the evidence base for public involvement in health‐care policy?: results of a systematic scoping review

Abstract

Background

Public involvement in health‐care policy has been advocated as a means to enhance health system responsiveness, yet evidence for its impact has been difficult to ascertain.

Objectives

To review the peer‐reviewed empirical evidence on outcomes of public involvement in health‐care policy.

Methods

We systematically searched PsychINFO and PubMed from November 2000 to April 2010 for empirical studies that reported on original research only; studies in languages other than English, German or French were excluded. Data were extracted using a standardized evidence table with a priori determined headings.

Main results

Nineteen studies were identified as eligible for inclusion in our review. We found that sound empirical evidence of the outcomes of public involvement activities in health care remains underdeveloped. The concept and the indicators used to examine and determine outcomes remain poorly specified and inconsistent, as does the reporting of the evidence. There was some evidence for the developmental role of public involvement, such as enhancing awareness, understanding and competencies among lay participants. Evidence for instrumental benefits of public involvement initiatives was less well documented.

Conclusions

Despite the growing body of work on public involvement in health‐care policy, evidence of its impact remains scarce; thus, firm conclusions about involvement activities that are appropriate and effective for policy development are difficult to draw. However, focus on outcomes risks missing the normative argument that involving the public in the health‐care policy process may be seen to be of intrinsic value.

Introduction

Public involvement policies have been advocated as a means of enhancing the responsiveness of health‐care systems.1 Despite its obvious appeal, the concept of public involvement remains poorly defined and its rationale and objectives are rarely specified when applied to the health‐care sector.1, 2, 3 Florin and Dixon (2004) define public involvement as ‘the involvement of members of the public in strategic decisions about health services and policy at local or national level’, thus distinct from patient involvement, which refers more specifically to ‘the involvement of individual patients, together with health professionals, in making decisions about their own health care’.4

Yet, this distinction between public and patient involvement is often not clear cut, with the term ‘involvement’ often used synonymously with ‘engagement’ or ‘participation’, while ‘public’ is frequently used interchangeably with ‘citizen’, ‘consumer’, ‘lay (person)’, ‘(service) user’ or ‘patient’. This largely reflects the different perspectives researchers have adapted to conceptualize public involvement. Coulter (2002) suggested that the twenty‐first‐century health service user is at once ‘a decision‐maker, a care manager, a coproducer of health, an evaluator, a potential change agent, a taxpayer and an active citizen whose voice must be heard by decision‐makers’.5 Given the range of terms applied to this concept, it is important to take a broad approach to identifying the empirical evidence on the effects of public involvement to track trends, similarities and differences in this field of research in health policy.

Involvement of the public has been sought in various fields of health‐care policy, including health service planning and delivery, health research and priority‐setting. Accordingly, public involvement policies have taken several forms, such as regional/local health councils, as for example in the UK, Australia and Canada6, 7; citizen juries8; public consultation9; and regional health conferences.10 Yet, the rationale behind introducing similar forms of public involvement has differed across settings such as grassroots active citizen movement or top‐down government approaches.11

It may be that some areas of health‐care policy are more amenable than others to the use of public participation initiatives, which would require drawing on an evidence base for their impact.12 A systematic review by Crawford et al. (2002) on the effects of involving patients in the development and planning of health services found a minority of studies describing the impact of involvement.13 These tended to show that involving service users did contribute to changes in care provision, such as the commissioning of new health services. However, the review focused on patient involvement in the area of service delivery rather than on broader aspects of public involvement in health‐care policies such as priority‐setting and health service planning. Furthermore, the review by Crawford et al. (2002) examined work that was published until 2000 only, so not capturing initiatives and policies initiated during the past decade.13

Within this context, it is timely to revisit the concept of public involvement in health‐care policy and examine systematically the recent peer‐reviewed evidence for the impact of related initiatives and policies from a broader perspective of health‐care decision making. Unlike other reviews of public participation which scoped evidence from other sectors,14 we focused on health literature explicitly so as to both update and extend previous work in this sector, and also ensure that results were specific to health care and not other decision making settings. Moreover, we were less interested in what is generally known, or not, and more concerned to learn about what was reported to ‘work’, with what effect(s) and the nature of this evidence. This would be a first step towards understanding which types of initiatives may be more amenable to different areas of health‐care policy and identifying the key contextual factors that call for, promote or hinder the implementation of relevant forms, and thus provide a framework for advancing public involvement in the health‐care policy process.

Methods

We carried out a systematic search of the peer‐reviewed literature on the evidence of outcomes of public involvement in health‐care policy using bibliometric databases PsychINFO and PubMed. Our concept of ‘outcome’ was deliberately not pre‐specified so as not to unduly limit papers that might consider for example, positive/negative effects on participating actors as well as changes to, or influences on, a health policy decision under consideration. As a scoping review, we were also interested in understanding what authors in the field identified, or understood, as an outcome. Thus, we included any indicator of effect measured in an evaluation of a public involvement initiative which could be described (however appropriate) as an ‘outcome’ or ‘impact’.

We applied broad search terms, using combinations of ‘public/consumer/user/civic/citizen/lay/client’, ‘engagement/involvement/participation/representation’, ‘planning/priority‐setting/decision’ and ‘health’ (‘/’ indicating ‘or’). The search was limited to studies published from November 2000 to April 2010 and included empirical studies that reported on original research only, particularly evaluation studies. We excluded surveys of beliefs/attitudes about participation and review articles except as a source for hand‐searching additional original studies. We further excluded editorials, letters and commentaries. Studies in languages other than English, German or French were also excluded.

Our review concerned evidence of outcomes of ‘public’ involvement related to health‐care decision making, priority‐setting, resource allocation and/or health service planning at the macro‐ and meso‐level. We conceptualized involvement as taking a broader societal perspective and included studies of public involvement where the public appeared to be (i) lay citizens or community members, (ii) representatives of organized social interest groups, including coalitions, partnerships, etc., or (iii) organizations of staff members and/or patients/customers/clients only if their input was provided from a broader perspective of the group (rather than from a singular person's interests). We therefore excluded studies that examined involvement in terms of service development, shared decision making of individual treatment, clinical decisions, participatory action research, therapeutic and service delivery decisions and clinical guidelines. We also excluded studies examining preferences for participation among health‐care professionals or users as these concerned the structure and process of involvement strategies rather than their outcomes.

Titles and abstracts were screened for eligibility for inclusion. Duplicate eligible studies were removed and abstracts examined further for full‐text retrieval. Retrieved papers were read in full and references followed up for final inclusion of studies meeting the eligibility criteria. Studies were analysed using a standardized evidence table with a priori determined headings. Data were extracted on stated study objective, study type, design, year, population(s) studied, geographical setting, stated definition of ‘public’, reported form of ‘involvement’, stated goal of the initiative (if relevant), outcome measure(s), key findings and source. Our systematic scoping review aimed to explore the evidence from evaluation research and was therefore not designed to rate the included studies as we knew from previous work1 that the quality of evaluation evidence would likely be varied.

Results

We identified 137 eligible studies, of which 19 met the inclusion criteria (Figure 1). These mainly reported on work undertaken in Anglo‐American contexts (England,15, 16, 17, 18, 19, 20 Canada,21, 22, 23, 24, 25 the USA26, 27, 28 and Northern Ireland29), with a smaller body of work examining public involvement in countries such as France,10 the Netherlands,30 Italy31 and Israel.32 Studies largely reported on work undertaken between the mid‐1990s and the mid‐2000s, with length of observations ranging from one year to seven years; four studies had not reported on the period of study.17, 20, 24, 30

Figure 1.

Flow diagram of literature search and selection of papers. *Exclusion criteria: service development (micro‐level); participatory action research; planning health research priorities; public health programme design/planning; treatment decision making; clinical decision; shared decision making; care/discharge planning; therapeutic and service delivery decisions; preferences for/views on participation; guidance/framework/concept; dissertations; book/chapters; grey literature; editorials/opinion papers.

Study design

Studies considered eligible for inclusion were reported to be of two types: descriptive case studies (N = 9) and empirical studies (N = 10), each with wide variation in study design and methodologies used. Among empirical studies, one described a cross‐sectional design17, and three reported a comparative pre–post design with subgroup analysis21, 22, 26; two also used stratified random sampling to recruit local citizens for participation in the deliberation method being studied.21, 22 Three studies used qualitative methods,27, 28, 30 and another four employed mixed methods involving qualitative and quantitative approaches.10, 15, 22, 24 Where reported, data collection was through participant surveys (cross‐sectional or longitudinal),15, 17, 22, 24 interviews,10, 15, 30 direct observation/ethnographic fieldwork22, 24, 29 and document review.10, 30

The design of studies defined as descriptive case studies was often reported simply as ‘qualitative’,16, 19, 23, 25, 29 although one reported details on the method used to analyse the qualitative data.18 One case study in Italy was reported as employing ‘quantitative’ methods,31 while another provided no information about their case study design or methodology.32 Mixed methods were described in one case study, collecting data through surveys, interviews and focus groups.20

The reported methods of included studies rarely specified a timeframe of an evaluation in its entirety. The majority of studies either had not reported the period of observation for the whole evaluation19, 23, 30; or provided timeframes, often covering twelve months or less, of parts of the data collection, usually in relation to when a survey was mailed or interviews conducted.15, 16, 17, 18, 26, 29, 31, 32 Few studies described a longer evaluation timeframe. Examples included a 30‐month period of study,20 as well as three,22 five27, 28 and up to seven10, 25 years of data.

Defining ‘the public’ and approaches to involvement

While all studies reviewed reported on the goal of the public involvement initiative analysed, definitions of what constituted ‘the public’, varied and were generally unclear. The vast majority of studies had not provided an explicit definition or statement of how the public was operationalized for the analysis in question.10, 15, 16, 17, 18, 20, 22, 25, 26, 27, 28, 29, 31

Where the public was defined and/or operationalized, definitions included some of the following: representatives of patient organizations31; ordinary citizens,32 further operationalized as ‘individuals with no particular axe to grind and whose voices might not otherwise be heard’24; individuals who lived in [area]19; citizen members of a community/local residents21; and community.23 Similarly, definitions and operationalization of ‘public involvement’ varied among studies. Thus, different approaches to public involvement were frequently used interchangeably (e.g. representation, participation, involvement) even within studies, despite formal definition of the form of involvement being analysed (e.g. consultation) (Table 1)

Table 1.

Definitions and terminology of public involvement

Stated form of public involvement under study	Terms used to describe form of involvement under study	Example
Consultation	Involve, engage, lay representation, deliberation, participation	15, 17, 19, 21, 22, 32
Participation	Involvement, representative, deliberation, engagement	26, 27, 28
Engagement	Involvement (community), partnership, empowering, collaboration	16
Partnership	Participation (public), involvement (community/consumer), representative, collaboration, empowerment, engagement (community)	25, 29, 31
Community development	Involvement, participation	18
Representation	Consultation, involvement, participation	20

Stated goals of public involvement initiatives varied widely, which was reflected in the form of public involvement (e.g. consultation) and approaches to involving, through, for example, surveys, conference and website, community health councils, public meetings, local patient groups, with considerable overlap between concepts and methods. (Table 2).33

Table 2.

Summary of public involvement type, methods used and goals

Stated form of public involvement under study	Involvement method(s) used	Stated goal(s) of public involvement initiative	Source
Consultation	Round tables, town hall meetings, open houses + population‐specific events + issue‐specific groups	To engage, enable and empower people for health through resource allocation function of local health integration networks	23
	Citizen panel, using mail survey, telephone survey or face‐to‐face group meetings	To obtain citizen input to inform a community‐wide health goal‐setting process	21
	Meeting (6 hours, plenary and small‐group sessions)	To provide the sponsoring regional health authority (RHA) with public input on an issue of importance	22
	Community health councils (CHCs) + local patient groups + public meetings	To involve and engage all stakeholders in shaping policies and decisions about primary care provision and delivery, including consulting and engaging local communities	15
	Surveys + service audit + group meetings + delivering staff training + board membership	To involve service users in the planning and delivery of psychiatric services	17
	Citizens’ jury (5 days)	To understand better the needs of all sections of the community so that appropriate primary care services can be provided	19
	Regional meetings (6) + survey	To elicit ordinary citizens’ views on pertinent health‐care priorities	32
Participation	Various: e.g. consultative role to government, guideline or indicator development groups, negotiation of collective contracts, lobbying activities	To contribute the patient perspective	30
	Client representation at council meetings	To develop a structure for gathering client information that will be useful for improving organizational performance	28
	Group simulation game using a roulette wheel	To promote participatory decision making in health‐care priority‐setting that is inclusive, deliberative and accessible for a diverse lay audience, especially low‐income, low‐education groups	26
	Consumer council + policy review process + hospital's performance improvement system	Varied by involvement method	27
Engagement	Citizens’ jury (2.5 days)	To engage the public in priority‐setting for health technology assessment	24
	Partnership‐based collaboration among a collection of agencies, groups and individuals	To explore mechanisms for breaking through current organizational boundaries to tackle inequalities and deliver better services and better health care	16
Partnership	Representation from community groups in collaborations	To reduce health inequalities through co‐ordinated activity of different agencies	29
	Various: for example, training courses, collaborative initiatives, website	To involve lay people, patients’ associations and scientific‐medical representatives in the health debate	31
	Various: for example, Women's Health Express Advisory Council, Salvation Army Health Council	To establish a formal mechanism for public participation in health policy decisions regarding the delivery of women's health services	25
Community development	Inclusive process of networked representation, especially for marginalized community members	To promote community participation in decision making about local health services	18
Representation	Various: for example, consumer groups participating in regional health conferences	To serve as a mechanism for rendering debates more transparent and concrete	10
	Various: for example, user representation on NHS Trust board, on locality reference groups, on staff training groups and on Joint Commission Board	To enable listening to and consulting users; to enable and enhance users’ and carers’ voices	20

Outcomes of public involvement

It was often unclear what effect(s) of public involvement a study aimed to measure, although most appeared to focus on assessing the (perceived) impact of the public (however defined) on power‐sharing or on decision outcomes. Where reported, details of specific and measurable information about the indicators of effect were generally lacking. This lack of clarity on indicators for public involvement effect or outcome meant that systematic comparison of approaches to measurement and choice of indicators of outcome was not feasible. Instead relevant information had to be extracted from study authors’ discussions of reported results (Table 3).

Table 3.

Summary of public involvement outcome measures

Outcome measure(s)		Source
Reported study type
Case studies	Emergent themes	25
	Participant views The ways in which local health service planning and delivery changed	18
Empirical studies	Effects on:  Participating citizens  Sponsoring decision‐makers  The decision making it [local citizen meeting] was designed to inform	22
	Extent of CHC [involvement method] in primary care group work Nature of public consultation Perceived impact	15
	Specific changes to policy or practice	17
	Effects on:	30
	Influence on policy making  Patient organizations
	Participant knowledge/learning Effect of participation exercise on changing individual health benefit selections between rounds towards more ‘public‐spirited orientations’	26
Study type not explicitly stated
Case studies	Participant views and satisfaction Frequency of project website hits	31
	Participant views Compatibility of policy recommendations and impact (defined as decisions about actions that will be taken)	32
	Proximal ‘benefits’ identified as: ‘individual empowerment’ ‘bridging social ties’ ‘synergy’ (defined as ‘creative solutions to intractable problems’)	29
	Extent of realization of goal of ‘involving and empowering local communities to achieve sustainable development’ Factors that affected this process	16
Empirical studies	Content of jury deliberations Participant views	24
	Participant views and satisfaction	21
	Realization of council meeting action decisions	28
	Changes to policy achieved	27

A common outcome measure across studies of public involvement was ‘participant views’, whether stated as an outcome or not. Some studies used ‘change’ as a measure for the influence of public involvement on decisions, policies or practice. A small number of studies measured participant knowledge, empowerment and social ties created as an outcome of public involvement. It should be noted that, in certain settings, the identification of specific outcome measures was not relevant because the study effectively presented a process evaluation as an account of the nature and development of a public involvement initiative, although authors reported finding some positive impact.10, 19, 20, 23

Several studies found the impact or influence of public involvement on shaping strategic decisions in health‐care policy to be minimal,15, 16, 20 geographically variable10, 22 or mixed.30 For example, van de Bovenkamp et al. (2009) reported mixed results of ‘successful influence’ on policy outcomes: on the one hand, interviews with representatives from patient organizations suggested there were multiple (n = 18) examples of unsuccessful attempts to influence organizational decision making, such as collective contract negotiations with insurers; but on the other hand, they also found several examples of influence and self‐reported positive effect on the well‐being of participants.30 The effects of public involvement on, for example, the uptake of a recommended decision or priority, might occur at a regional level more than a national level because involved parties are more associated with that territory10; although the effects of the same initiative can also differ across regions in the same country.22 Even at a local level, minimal effects on shaping strategic decisions were found for public involvement initiatives aiming to engage all stakeholders15 (i.e. a collection of diverse actors) and which were relatively unbalanced or had constraints on the involvement of the public, such as formal rules16 or no direct means of setting the agenda.20

Others noted a direct effect of public involvement by means of either improving lay participants’ learning or knowledge of the consultation topics or the process of health‐care priority‐setting26, 31, 32 or changing service providers’ perceptions and ways of operating.18 There may also be a benefit of deliberative procedures as a means for building consensus among diverse views and for changing the orientation of individual choices towards a more societal perspective.25, 26, 32 One study also highlighted the (potential) negative aspects of the deliberative process, noting the opportunity cost related to financing public involvement and the unintended consequences for relationships among participating stakeholders when government actors were challenged by members of the public.29

There was some evidence that public involvement initiatives can change policy or practice.34 Several studies provided specific examples of how public involvement had influenced decision making, reporting that recommendations were accepted and implemented22; practical changes/improvement was achieved17, 27, 28; and priorities, identified through a public involvement activity, became the object of a regional programme10 or resulted in new financial means being leveraged for new services.18, 19, 29 Another study reported a number of practical changes in community development to demonstrate the positive impact of public involvement such as various community projects targeting disadvantaged groups (e.g. youth, isolated families, ethnic minorities) to increase local access to care services.18 Finally, the replication of a deliberative procedure by two other organizations was interpreted in another study as evidence of the positive impact of the initiative studied.32

Discussion

This review has shown how, despite the growing body of work on public involvement in health‐care policy, robust evidence of the impact of public involvement remains scarce. We found examples of a poor fit between evaluation aims and the study design and indicators employed. Although several studies aimed to assess the influence of public involvement on decisions, policies or practice using ‘change’ as an indicator of effect, few were designed to be comparative or had sufficiently long observation periods to adequately measure such impact on policy and practice in the longer term. Furthermore, the concept of public involvement that was under study and the indicators used to evaluate and determine any resultant effect/impact were poorly specified and inconsistent. This finding is perhaps not surprising given the continuing absence of a consensus on the definition of public involvement, and the variation in purpose of and approaches to involvement, either of which are often not made explicit.2, 3 Nearly half of the included studies had not explicitly stated what a priori outcomes/effects they aimed to measure. The need to infer measurable indicators from authors’ discussions is consistent with other work, which has noted that the nature of effectiveness of public involvement is poorly defined,33 as are other potential outcomes.35 Thus, there is scope for developing robust evaluation research methods and measures for complex health initiatives such as public involvement.

The absence of clear definitions, operationalization and reporting of specific outcome measures is problematic for purposes of evaluation as well as for the development of policy informed by evidence on the effects of public involvement (on process, output and outcome). These observations contributed a level of conceptual ambiguity that raised concerns about: (i) the form(s) of involvement reported to have an effect and (ii) the outcomes (or other effects) that may be reasonably expected for a given type and method of public involvement studied. Thus, there is uncertainty about the reliability of those impacts that are reported in the current literature, and about the nature and scope of reasonable outcome(s) of public involvement that may be measured given stated objectives. For example, stated objectives of a given initiative could be broad and vague (e.g. to elicit ‘ordinary’ citizens’ view on specific priorities or to obtain citizen input to inform community health goal setting), but the indicators of effect described (post hoc) being narrow and specific (e.g. participant views and satisfaction, or compatibility of policy recommendations and decisions acted upon).21, 32 Against the background of a general lack of consensus of who is ‘the public’, our study therefore reinforces the on‐going challenges of determining the evidence for positive or negative effects of public involvement. Addressing these challenges is a key step towards answering the question of what type of public involvement makes a difference in what context(s) and for which policy areas.

The observed scarcity of formal evaluations of public involvement efforts is not new,14 although there may be an expectation that a growing body of empirical work might have advanced the evidence base. It has been noted that there is a lack of evidence about when involvement is effective.13, 14, 36, 37, 38 Mitton et al. (2009), in their review of public involvement in health‐care priority‐setting, showed that conclusions about the ‘success’ of a given public involvement initiative appear to be independent of any formal evaluation.14 Specifically, they found that studies in which the engagement process (however defined) was intended to influence or affect an actual decision, 60 per cent concluded that an impact had been achieved, while only 10 per cent stated that this had not been the case. We also found that authors reported a positive impact and yet presented a review of an initiative's process only. This might reflect the normative argument that ‘good deliberative democratic processes are of value in and of themselves’,26 and so related studies may have used quality of deliberative procedures as the criterion for tacitly defining ‘impact’. Thus, there are real difficulties in drawing firm conclusions from the existing evidence base, which appeared patchy and poorly reported, that would inform policy development of appropriate and effective involvement activities in health‐care decision making.

Yet, while existing empirical work has some limitations, what is available does seem to support the notion that public involvement initiatives or activities can affect the health‐care policy process by influencing strategic decisions on service delivery or priority‐setting, for example. In particular, the evidence we found supported what has been defined as the developmental role of public involvement; that is, benefits incurred relate to improving lay participants’ knowledge of subject areas and/or decision making processes, and increasing awareness among decision‐makers and/or service providers about ways of operating in the health‐care sector. In contrast, empirical evidence for instrumental benefits from public involvement initiatives was less well documented, with the possible exception of strategies contributing to consensus building where the issues under deliberation were more concrete and locally specific. Overall, our finding of the relatively limited evidence of longer‐term impact of public involvement is consistent with other work concluding that involvement policies have had little impact34, 39, 40, 41 or little independent impact. That is, where participants’ preferences ‘match’ policy or service preferences, the public's requests are more likely to be met and thereby demonstrate the impact of involvement on policy or practice.42, 43

It is worth noting, however, that the emphasis placed on assessing outcomes/impact of public involvement risks missing the normative value of public involvement as intrinsically good because it is a deliberative democratic process.26 As Thurston et al. (2005) suggested, ‘success should not be limited to whether high‐level governance decisions are made’ but instead may be determined in terms of the establishment of a formal partnership capable of informing the development of health policy by challenging the status quo and adding priorities to the agenda.25 Thus, it seems important that any evaluation of the outcomes of public involvement also includes explicit criteria to assess the quality of deliberative procedures. Additionally, research in this area may benefit from a more ‘realistic evaluation’ approach which takes into account the contextual influences on the measured effect(s) of complex health initiatives such as public involvement44; the common use of qualitative methods among the study designs reviewed here could be further exploited to develop this issue in the current evidence base. One study illustrating the variable success of an initiative implemented in different regions in Canada22 is a starting point for more research needed here. Some possible indicators for the role of context in modifying or mediating the effect(s) of engagement might be informed by research on perceptions of barriers or facilitators, including the level of investment and commitment to the goals of a given initiative, or the nature of the decisions expected from public involvement;22 the attitudes of different stakeholders towards the impact of engagement;17 how the public are invited to be involved and by whom, on what set of issues, controlled by whom and at what level of the system.30

In summary, there is still a need for more rigorous evaluation research in this area, with specific and measurable indicators of effect that are clearly reported, to enable better understanding of which forms of public involvement are more appropriate and effective to improve decision making and policy in terms of not only processes but particularly outcomes. Our review, however, did not consider literature from other sectors where there is a history of public engagement efforts such as environmental sciences and urban planning because we deliberately aimed to focus on health‐care. Indeed, it might be expected that a greater amount and/or quality of evidence would be found in the non‐health literature, which we excluded or in the broader social science databases which were not searched. While the restrictions on our search could be considered a limitation, we believe such a narrow focus strengthens our review as it enables a clearer understanding of the empirical evidence within the health field by not assuming public involvement will be the same, nor its effects generalizable across different sectors with varied histories and levels of development of public involvement initiatives. This narrow focus also had the utility of limiting the extent of differences in eligible studies to be compared and of enabling greater coherence between the information described and what we aimed to find. Furthermore, while our review may have been limited by the small number of databases searched, it was more comprehensive of potential studies within a given database as we examined results in three common languages. Nevertheless, as we included the published peer‐reviewed literature only, our review may have missed a wider body of evidence reported in the grey literature that may have provided further insights into the evidence of outcomes of public involvement; some of which has been reviewed elsewhere.14

Conclusion

Public involvement in health‐care policy has received considerable attention over past decades. Yet, the scope, objectives and desired outcomes of public involvement policies remain poorly defined, with patchy evidence of impact. There remains concern about the lack of clarity on who the public is and what involvement is intended to achieve,45, 46 although there is an expectation that participatory techniques will become more salient in complex areas such as genetics than in health service planning and prioritization.12

From a research perspective, there is a need to further our understanding of whether and how individuals assume the different roles of active citizen, user and potential user and how individual expectations and motivations for involvement may be influenced by the structure of the health‐care system and by social and political values. From a policy perspective, there is a need to build research capacity through incentives for more robust evaluations in this area and to synthesize a better evidence base that consistently takes a common approach. In so doing, a greater step can be made towards a stronger evidence base for whether public involvement improves processes and/or outcomes of decision making and policy. Such evidence is a minimum requirement for comparatively assessing which areas of health‐care policy are the most amenable to the use of public participation and then within a given area, what type of public involvement makes a difference in what context(s).

Source of funding

This work was funded by Bertelsmann Stiftung, Gütersloh, Germany. It was part of a larger project previously reported: Conklin A, Morris Z, Nolte E. Involving the public in health‐care policy: An update of the research evidence and proposed evaluation framework. Santa Monica/Cambridge: RAND Corporation, 2010.

Conflict of interest

None declared.

Supporting information

Table S1. Details of included studies on the impact of public involvment in health care policy.