Methods of consumer involvement in developing healthcare policy and research, clinical practice guidelines and patient information material

Abstract

Background

The importance of consumer involvement in health care is widely recognised. Consumers can be involved in developing healthcare policy and research, clinical practice guidelines and patient information material, through consultations to elicit their views or through collaborative processes. Consultations can be single events, or repeated events, large or small scale. They can involve individuals or groups of consumers to allow debate; the groups may be convened especially for the consultation or be established consumer organisations. They can be organised in different forums and through different media.

Objectives

To assess the effects of consumer involvement and compare different methods of involvement in developing healthcare policy and research, clinical practice guidelines, and patient information material.

Search methods

For the 2006 version of this review (Nilsen 2006) we searched: the Cochrane Consumers and Communication Review Group's Specialised Register (4 May 2006); the Cochrane Controlled Trials Register (CENTRAL) (The Cochrane Library, Issue 1 2006), MEDLINE (1966 to January Week 2 2006); EMBASE (1980 to Week 03 2006); CINAHL (1982 to December Week 2 2005), PsycINFO (1806 to January Week 3 2006); Sociological Abstracts (1952 to 24 January 2006); and SIGLE (System for Information on Grey Literature in Europe) (1980 to 2003/1). We scanned reference lists from relevant articles and contacted authors.

For the 2009 update we revised the previous search strategies and searched: the Cochrane Central Register of Controlled Trials (CENTRAL), including the Cochrane Consumers and Communication Review Group's Specialised Register (The Cochrane Library, Issue 2 2009), MEDLINE (1950 to May Week 1 2009); EMBASE (1980 to Week 19 2009); CINAHL (1981 to 8 July 2009), PsycINFO (1806 to May Week 1 2009); Sociological Abstracts (1952 to 11 May 2009). We did not search OpenSIGLE for the review update. We scanned reference lists from relevant articles and searched the Science Citation Index Expanded and the Social Sciences Citation Index (1975 to 9 September 2009) for studies citing the included studies in this review.

Selection criteria

Randomised controlled trials assessing methods for involving consumers in developing healthcare policy and research, clinical practice guidelines or patient information material. The outcome measures were: participation or response rates of consumers; consumer views elicited; consumer influence on decisions, healthcare outcomes or resource utilisation; consumers' or professionals' satisfaction with the involvement process or resulting products; impact on the participating consumers; costs.

Data collection and analysis

Two review authors independently selected trials for inclusion, assessed their quality and extracted data. We contacted trial authors for clarification and to seek missing data. We presented results in a narrative summary and pooled data as appropriate.

Main results

We included six randomised controlled trials with moderate or high risk of bias, involving 2123 participants. There is moderate quality evidence that involving consumers in the development of patient information material results in material that is more relevant, readable and understandable to patients, without affecting their anxiety. This 'consumer‐informed' material can also improve patients' knowledge. There is low quality evidence that using consumer interviewers instead of staff interviewers in satisfaction surveys can have a small influence on the survey results. There is low quality evidence that an informed consent document developed with consumer input (potential trial participants) may have little if any impact on understanding compared to a consent document developed by trial investigators only. There is very low quality evidence that telephone discussions and face‐to‐face group meetings engage consumers better than mailed surveys in order to set priorities for community health goals. They also result in different priorities being set for these goals.

Authors' conclusions

There is little evidence from randomised controlled trials of the effects of consumer involvement in healthcare decisions at the population level. The trials included in this review demonstrate that randomised controlled trials are feasible for providing evidence about the effects of involving consumers in these decisions.

Plain language summary

Consumer involvement in the development of healthcare policy and research, clinical practice guidelines and information for patients

The importance of consumer involvement at all levels of the health services is widely recognised. This review shows that little research has been done to find the best ways of involving consumers in healthcare decisions at the population level. Most of the included trials compared consultations with consumers with no consultations with consumers. There is moderate quality evidence from two trials that involving consumers in the development of patient information material results in material that is more relevant, readable and understandable, without affecting anxiety. This 'consumer‐informed' material can also improve knowledge. Two trials, which compared using consumer interviewers with staff interviewers as data collectors for patient satisfaction surveys, found small differences in satisfaction survey results, with less favourable results obtained when consumers were the interviewers. One trial comparing two informed consent documents, one developed with consumer input and the other developed by the trial investigators, showed that consumer input may have little if any impact on understanding of the trial described in the consent document. One trial, comparing two different methods for involving the public (telephone discussion and a face‐to‐face group meeting), showed that a face‐to‐face meeting is most likely to engage consumers and may result in different community health priorities.

Summary of findings

for the main comparison.

Mental health patients compared with mental health staff used as interviewers of mental health patients (Clark 1999; Polowczyk 1993).
Patient or population: Mental health patients Settings: Mental health outpatient facilities in Toronto, Canada and Suffolk County New York, USA Intervention: Mental health patient interviewers Comparison: Mental health staff interviewers
Outcomes	Absolute effect	No of Participants (studies)	Quality of the evidence (GRADE)	Comments
Satisfaction with mental health services (consumer influence on resource utilisation)	MD ‐ 0.14 (‐0.23 to ‐ 0.06)	650 (2)	++OO low$	Based on these two trials there is low quality evidence of small differences in satisfaction survey results when consumer interviewers are used instead of staff interviewers.
MD: Mean difference
GRADE Working Group grades of evidence High quality: Further research is very unlikely to change our confidence in the estimate of effect (++++) Moderate quality: Further research is likely to have an important impact on our confidence in the estimate of effect and may change the estimate (+++O) Low quality: Further research is very likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate (++OO) Very low quality: We are very uncertain about the estimate (+OOO)

$ Serious limitation due to concealment of allocation and blinded assessment of primary outcome(s) not clear. Some uncertainty about directness.

2.

Face to face meetings compared with telephone meetings for obtaining change of views on health issues (Abelson 2003).
Patient or population: Consumers of a community organization Settings: Local community in Ontario, Canada Intervention: Face to face meetings Comparison: Telephone meetings
Outcomes	No of Participants (studies)	Quality of the evidence (GRADE)	Comments
Healthcare priorities (consumer influence on decisions)	29 (1)	+OOO very low$	Based on this trial there is very low quality evidence of telephone discussions compared with face‐to‐face meetings changing consumer priorities for community health goals.
GRADE Working Group grades of evidence High quality: Further research is very unlikely to change our confidence in the estimate of effect (++++) Moderate quality: Further research is likely to have an important impact on our confidence in the estimate of effect and may change the estimate (+++O) Low quality: Further research is very likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate (++OO) Very low quality: We are very uncertain about the estimate (+OOO)

$ Very serious limitations due to concealment of allocation and blinded assessment of primary outcome(s) not clear, and follow‐up of patients not done (results from control group (mail group) were excluded because of low response rate). Sparse data due to small number of participants (46 divided into three trial groups).

3.

Leaflets written by patients and professionals together compared with leaflets written by professionals alone used in patients undergoing endoscopy or patients who receive patient‐controlled analgesia (PCA) (Aabakken 1997; Chumbley 2002).
Patient or population: patients undergoing endoscopy or patients who receive PCA Settings: Hospitals in [London?] UK and in Oslo, Norway Intervention: Leaflets written by patients and professionals together Comparison: leaflets written by professionals alone
Outcomes	No of Participants (studies)	Quality of the evidence (GRADE)	Comments
Anxiety (consumer influence on healthcare outcomes)	335 (2)	+++O moderate$	Based on these two trials, there is moderate quality evidence that there may be little or no difference in worries or anxiety associated with procedures for patients receiving information material developed following consumer consultation, compared with patients receiving material developed without consumer consultation.
Satisfaction with information material (consumers' satisfaction with products resulting from consumer involvement)	335 (2)	+++O moderate$	Based on these two trials there is moderate quality evidence that consumer consultation prior to developing patient information material probably results in material that is more relevant, readable and understandable to patients.
Knowledge of patient‐controlled analgesia (consumer influence on healthcare outcomes)	100 (1)	+++O moderate$	Based on this trial there is moderate quality evidence that consumer consultation before developing patient information material probably can improve the knowledge of patients who read the material.
GRADE Working Group grades of evidence High quality: Further research is very unlikely to change our confidence in the estimate of effect (++++) Moderate quality: Further research is likely to have an important impact on our confidence in the estimate of effect and may change the estimate (+++O) Low quality: Further research is very likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate (++OO) Very low quality: We are very uncertain about the estimate (+OOO)

$ Serious limitation due to blinded assessment of primary outcome(s) and baseline measurement not clear.

4.

Informed consent document developed with input from a consumer group compared with investigator‐developed consent document for Gulf War veterans' illness (Guarino 2006).
Patient or population: Patients with Gulf War veterans' illness Settings: Clinical research at 20 US medical centers Intervention: Consumer‐developed consent document Comparison: Investigator‐developed consent document
Outcomes	Illustrative comparative risks* (95% CI)		No of Participants (studies)	Quality of the evidence (GRADE)	Comments
	Assumed risk	Corresponding risk
	Investigator‐developed consent document	Consumer‐developed consent document
Understanding (consumer influence on healthcare outcomes) Informed Consent Questionnaire‐4. Scale from: 0 to 1. (Follow‐up: 12 months)	The mean understanding in the control groups was 0.728	The mean understanding in the intervention groups was 0.006 higher (0.029 lower to 0.04 higher)	1092 (1)	++OO low§#	Based on this trial there is low quality evidence that consumer consultation in the development of consent documents may have little if any impact on participant’s self‐reported understanding of the trial described in the consent document.
Satisfaction (consumer satisfaction resulting from consumer involvement)			1092 (1)	++OO low§#	Based on this trial there is low quality evidence that consumer consultation in the development of consent documents may have little if any impact on satisfaction with study participation.
Adherence (participation rates of consumers)			1092 (1)	++OO low§#	Based on this trial there is low quality evidence that consumer consultation in the development of consent documents may have little if any impact on adherence to the protocol.
Refusal to participate (participation rates of consumers)			1092 (1)	++OO low§#	Based on this trial there is low quality evidence that consumer consultation in the development of consent documents may have little if any impact on refusal to participate.
*The basis for the assumed risk (e.g. the median control group risk across studies) is provided in footnotes. The corresponding risk (and its 95% confidence interval) is based on the assumed risk in the comparison group and the relative effect of the intervention (and its 95% CI). CI: Confidence interval
GRADE Working Group grades of evidence High quality: Further research is very unlikely to change our confidence in the estimate of effect (++++) Moderate quality: Further research is likely to have an important impact on our confidence in the estimate of effect and may change the estimate (+++O) Low quality: Further research is very likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate (++OO) Very low quality: We are very uncertain about the estimate (+OOO)

§ Cluster randomized trial. Unclear allocation concealment and blinding. Drop out less than 20%, however only 71% of the participants completed primary outcome measure at all four visits.

# Not validated questionnaire prior to the trial. Only one trial with 1092 participants.

Background

The importance of consumer involvement at all levels of the health services is widely recognised. Our review focuses on the effects of promoting and organising consumer involvement to inform or participate in decisions about health care for populations, including decisions about healthcare policies and planning (eg. inequalities in health care); clinical policies (eg. clinical practice guidelines); patient information materials (eg. that aim to inform patients about personal healthcare decisions); and healthcare research (eg. design of clinical or epidemiological studies, identification of relevant outcomes, priority setting).

Potential benefits of consumer involvement in health care

The potential benefits of consumer involvement in health care include: policy, research, practice and patient information that includes consumers' ideas or addresses their concerns; improved implementation of research findings; better care; and better health. Consumer participation can be viewed as a goal in itself by encouraging participative democracy, public accountability and transparency. For example, the World Health Organization's Declaration of Alma Ata states that "The people have the right and duty to participate individually and collectively in the planning and implementation of their health care" (WHO 1978). Consumers may offer different and complementary perspectives to those of professionals. Also, they may not have the same conflicts of interest and loyalties as professionals.

It is assumed that input from consumers in planning of health care can lead to more accessible and acceptable health services, and improve health and quality of life (Crawford 2002). Consumer involvement is also thought to lead to health research of greater quality and clinical relevance (Boote 2002), and greater uptake of findings (Whitstock 2003). There is a lack of research that reliably investigates whether consumer involvement achieves these intentions and, if so, which methods of consumer involvement are most effective.

Potential barriers to consumer involvement in health care

Consumer involvement in health care is also an idea that faces considerable resistance. Although most health professionals are dedicated, they face many challenging demands. They are hierarchically socialised and organised to view themselves as authorities. It is also claimed, for instance, that consumer involvement can make research projects costlier and longer than some researchers and grant‐awarding bodies would expect, and that consumers may have biased views on certain health issues, which may threaten the traditional academic impartiality of knowledge development (Boote 2002).

Consumers may not find it meaningful to function as consumer representatives because their opportunities for input and influence are minimised. There seems to be a lack of research that reliably investigates methods for overcoming such barriers to consumer participation.

Framework for evaluating methods of consumer involvement

Consumer involvement varies according to its purpose, the consumers involved, the degree of involvement, the methods employed to support this involvement, and the context. An early attempt to characterise consumer involvement, initiated largely by metropolitan institutions in high income countries, proposed a 'ladder of participation.' Rungs on the ladder represented increasing degrees of participation, from: non‐participation or manipulation and therapy; through the tokenism of informing, consulting and placating; to citizen power through partnership, delegated power and citizen control (Arnstein 1969). A similar scale has been described, drawing on participatory processes for research in low and middle income countries (Cornwall 1996). This acknowledges the tokenism that occurs at one end of the scale and the co‐learning or independent collective action at the other.

Besides the degree of involvement, ways of supporting this involvement vary widely. Methods may involve individuals or groups of consumers, and the latter may be pre‐existing or convened for the purpose. A two‐dimensional representation of models of involvement combining degrees of involvement (from information to control), and distinguishing individual and group involvement, has been employed to describe consumer involvement in mental health services in England (Glasby 2003). How consumer involvement develops and how it is viewed by health professionals and by different sectors of society is also influenced by who initiated specific encounters: the consumers or the professional services (Mullen 1984).

We chose a framework for describing consumer involvement constructed for a systematic review of consumer involvement in setting research agendas (Oliver 2004; Table 5). When employed in Oliver and colleagues' review, the framework accommodated the diverse methods spanning all these dimensions (degree of involvement, individuals or groups of consumers, and initiated by consumers or professionals) across low, middle and high income countries.

1. Methods of the Review: Comparisons.

Intervention	Comparison
DEGREE OF CONSUMER INVOLVEMENT (1) Consultation (2) Collaboration (3) Collaboration	DEGREE OF CONSUMER INVOLVEMENT (1) No involvement (2) No involvement (3) Consultation
FORUM FOR COMMUNICATION (1) Consultative fora, eg. town meeting, written consultation, interviews, focus groups (2) Collaborative, eg. committee membership, permanent consumer panels (3) Collaborative forum	FORUM FOR COMMUNICATION (1) No consumers in consultative forum ‐ different/multiple fora for consultation (2) No consumers in collaborative forum ‐ different/multiple fora for collaboration (3) Consultative forum (eg. invitation to a single committee meeting)
INVOLVEMENT IN DECISION MAKING (1) Involvement in decision‐making implied, eg. committee membership (2) Involvement in decision‐making explicit, eg. voting, ranking (3) Explicit involvement	INVOLVEMENT IN DECISION MAKING (1) No involvement in decisions (2) No involvement in decisions (3) Implicit involvement
RECRUITMENT OF PROFESSIONALS/CONSUMERS (1) Targeted, personal invitations (2) Wide advertisement (a) Mass media (b) Telephone (c) Mail (d) E‐mail	RECRUITMENT OF PROFESSIONALS/CONSUMERS (1) Target, personal invitations versus wide advertisement (2) Different/multiple methods of wide advertisement
TRAINING AND SUPPORT FOR PROFESSIONALS/CONSUMERS (1) Education (2) Counselling (3) Introduction day	TRAINING AND SUPPORT FOR PROFESSIONALS/CONSUMERS (1) Training versus no training (2) Different/multiple methods of training (3) Different/multiple trainers (4) Different timing of training (introductory, on‐going)
FINANCIAL SUPPORT (1) Funding/staffing specifically to support consumer involvement enterprises (2) Reimbursement of consumer expenses (3) Fee or honoraria (4) No financial support	FINANCIAL SUPPORT (1) Financial support versus no financial support (2) Different/multiple policies for financial support
PRACTICAL SUPPORT (1) For example, administrative support for consumer groups	PRACTICAL SUPPORT (1) Practical support versus no practical support (2) Different/multiple types of practical support

For this framework, Arnstein's ladder of participation was simplified to the three steps employed by the Consumers in NHS Research Support Unit (since renamed INVOLVE): consultation, collaboration and consumer control (Hanley 2004). 'Consultation' was defined as asking consumers for their views and using these views to inform decision‐making. For example, funders of research have held one‐off meetings with consumers to ask them about their priorities for research, or written to consumers in accessible terms to invite their views. Consumers' views were not necessarily adopted, although they may have informed decisions. 'Collaboration' was described as active, on‐going partnership with consumers. For example, consumers have been committee members or collaborated less formally to complete a task. Again, there is no guarantee that consumers' views will influence decisions, but there is more opportunity for them to be heard than in consultations. 'Consumer‐controlled' research was described as consumers designing, undertaking and disseminating the results of a research project. Professionals were only involved at the invitation of the consumers.

Within this framework, methods were further distinguished by descriptions of the forum for communication (such as one‐to‐one interviews, focus groups, citizens' juries, town meetings, committee meetings, working groups) and methods for decision making (such as informal committee consensus, voting, ranking, scoring, visual scales and Delphi surveys). The presence or absence of transparent descriptions of methods for decision making can distinguish implied involvement in decisions (such as in committee meetings), some examples of which are misleading, and explicit involvement in decisions. This can inform interpretations of tokenism, which is widespread in consumer involvement.

All of these interventions are worthy of review. Although our framework for consumer involvement helped us characterise the interventions, care needs to be taken with interpretation, considering the diverse purposes of consumer involvement and the different cultural and political contexts.

Need for comparative evaluations of consumer involvement

Relevant reviews (Boote 2002; Crawford 2002; Oliver 2004) revealed a lack of comparative evaluations of promoting or organising consumer involvement in health care. Therefore, in the previous version of this review, Nilsen 2006, we searched for quasi‐randomised trials, interrupted time series analyses, and controlled before‐after studies in addition to randomised controlled trials. As we identified and included five randomised controlled trials, we decided to search for randomised controlled trials only for the updated review (2009).

Objectives

To determine the effects of consumer involvement and compare different methods of involvement in developing healthcare policy and research, clinical practice guidelines, and patient information material.

Methods

Criteria for considering studies for this review

Types of studies

Randomised controlled trials.

Types of participants

Healthcare consumers or professionals involved in decisions about health care at the population level, or evaluating the effects of consumer involvement.

Healthcare consumers could include: patients; unpaid carers; parents/guardians; users of health services; disabled people; members of the public who are the potential recipients of health promotion/public health programmes; groups asking for research because they believe they have been exposed to potentially harmful circumstances, products or services; groups asking for research because they believe they have been denied products or services from which they believe they could have benefited; and organisations that represent service users and carers (Hanley 2004). Depending on the context, they could be described with any of the following terms: 'lay', 'non‐expert', 'service user', 'survivor' or 'member of the general public'.

Types of interventions

Ways of involving consumers to inform, or participate in, decisions about healthcare policy and research, clinical practice guidelines or patient information material.

• Healthcare policy was defined as laws, rules, financial and administrative orders made either by governments, non‐government organisations or private organisations, that are intended to directly affect the provision and use of health services.

• Healthcare research included clinical research, epidemiological research and health services research (investigating need, demand, supply, use, and outcome of health services).

• Clinical practice guidelines were defined as "systematically developed statements to assist both practitioner and patient decisions in specific circumstances" (Field 1992).

• Patient information material included printed, audio‐visual and electronic information that is intended to help patients to make informed decisions about healthcare.

We used a framework for describing consumer involvement that distinguishes methods and comparisons in terms of the degree of consumer involvement, the forum for communication, how consumers are involved in actual decision‐making, how consumers are recruited, and how consumers/professionals are trained and supported (Oliver 2004; Table 5). We give examples of these categories in the Background section.

Relevant comparisons included: the development of healthcare policy, clinical practice guidelines and patient information material with or without consumer involvement; and different methods of consumer involvement, for example involving consumers as members of the steering group of a research project, versus involving them as participants of focus groups. For training and support interventions, comparisons include both providing versus not providing training or support for consumer involvement, and providing two or more different forms of training or support. We did not include interventions where consumers are involved as lay health workers or teachers only.

Evaluations of consumer involvement in healthcare research does not include individuals participating in trials as sources of data alone (ordinary trial participants). Evaluations of consumer involvement in healthcare policy, clinical practice guidelines and patient information material could include individuals participating in trials as sources of data alone if they are needed to provide data for evaluating products (eg. information material or guidelines) developed with consumer input.

Types of outcome measures

To be included a trial must have had a quantitative measure, requiring the use of validated instruments, of at least one of the following outcomes: participation or response rates of consumers; consumer views elicited; consumer influence on decisions, healthcare outcomes or resource utilisation; consumers' or professionals' satisfaction with the involvement process or resulting products; impact on the participating consumers; costs.

Search methods for identification of studies

For the 2006 version of this review (Nilsen 2006) we searched the following databases without language restriction:

• The Cochrane Consumers and Communication Review Group's Specialised Register (Searched 4 May 2006)

• The Cochrane Central Register of Controlled Trials (CENTRAL), (The Cochrane Library Issue 1 2006)

• MEDLINE (Ovid) (1966 to January Week 2 2006)

• EMBASE (Ovid) (1980 to 2006 Week 03)

• CINAHL (Ovid) (1982 to December Week 2 2005)

• PsycINFO (Ovid) (1806 to January Week 3 2006)

• Sociological Abstracts (CSA) (1952 to 24 January 2006)

• SIGLE (System for Information on Grey Literature in Europe) (WebSpirs) (1980 to 2003/1)

We ran a test search in the following databases:

• CSA Worldwide Political Science Abstracts

• ERIC

• International Political Science Abstracts

• NTIS (the USA government's National Technical Information Service)

• PAIS (Public Affairs Information Service)

As no relevant records were retrieved, we conducted no further searches in these databases.

We screened the reference lists of all of the relevant reports retrieved, and searched the Science Citation Index for articles citing key references that were identified using the above search strategies. We contacted authors of relevant papers, relevant organisations, and discussion lists to identify additional studies, including unpublished and ongoing studies.

For the 2009 update of this review we revised the previous search strategies and searched the following databases without language restriction:

• The Cochrane Central Register of Controlled Trials (CENTRAL) including the Cochrane Consumers and Communication Review Group's specialised register, (The Cochrane Library Issue 2 2009) (searched 22 April 2009)

• MEDLINE (Ovid) (1950 to May Week 1 2009) (searched 11 May 2009)

• EMBASE (Ovid) (1980 to 2009 Week 19) (searched 11 May 2009)

• CINAHL (EBSCO) (1981 to 8 July 2009)

• PsycINFO (Ovid) (1806 to May Week 1 2009) (searched 11 May 2009)

• Sociological Abstracts (CSA) (1952 to 11 May 2009)

We screened the reference lists of all of the relevant reports retrieved, and searched the Science Citation Index Expanded and the Social Sciences Citation Index (1975 to 9 September 2009) for studies citing the included studies in this review.

We did not search OpenSIGLE for the review update.

We present all revised and updated search strategies in Appendix 1.

Data collection and analysis

Two review authors reviewed all of the search results and reference lists of relevant reports (two of ESN, HTM and MJ). We retrieved the full text of potentially relevant reports and two (of the same) review authors assessed the relevance of those studies, assessed the quality of included studies and extracted data from included studies independently. We resolved disagreements by discussion, including another review author when necessary.

We used standard criteria to assess the methodological quality of studies (protection against bias) (EPOC 2003; Table 6; Table 7) and extracted information about concealment of allocation, follow‐up of professionals, follow‐up of patients or episodes of care, blinded assessment of primary outcomes, baseline measurement, reliable primary outcome measures and protection against contamination. The data extractors independently assessed overall quality (risk of bias) for each main outcome within each trial, using the following guidelines:

2. Cochrane EPOC Group: Quality assessment.

Standard criteria are used to assess the methodological quality of studies included in EPOC reviews (protection against bias). Each criterion is scored as DONE, NOT CLEAR, or NOT DONE. Details regarding the application of these criteria are available from the editorial base. Seven standard criteria are used to assess the methodological quality of randomised controlled trials (RCTs) and controlled clinical trials (CCTs): 1. Concealment of allocation (protection against selection bias). This is scored as DONE if the unit of allocation was by institution, team or professional and any random process was described explicitly; or if the unit of allocation was by patient or episode of care and there was some form of centralised randomisation scheme, an on‐site computer system or sealed opaque envelopes were used. 2. Follow‐up of professionals (protection against exclusion bias). This is scored as DONE if outcome measures were obtained for 80% to 100% of subjects randomised. 3. Follow‐up of patients. This is scored as DONE if outcome measures were obtained for 80% to 100% of patients randomised, or for patients who entered the trial. 4. Blinded assessment of primary outcome(s) (protection against detection bias). This is scored as DONE if the authors state explicitly that the primary outcome variables were assessed blindly, or the outcome variables are objective, eg. length of hospital stay, drug levels as assessed by a standardised test. Primary outcome(s) are those variables that correspond to the primary hypothesis or question as defined by the authors. In the event that some of the primary outcome variables were assessed in a blind fashion and others were not, each is scored separately. 5. Baseline measurement. This is scored as DONE if performance or patient outcomes were measured prior to the intervention, and no substantial differences were present across trial groups. 6. Reliable primary outcome measure(s). This is scored as DONE if there were two or more raters with at least 90% agreement or kappa greater than or equal to 0.8 OR the outcome data were obtained from some automated system, eg. length of hospital stay, drug levels as assessed by a standardised test. 7. Protection against contamination. This is scored as DONE if allocation was by community, institution or practice and it is unlikely that the control group received the intervention.

3. Risk of bias assessment.

Trial	Concealment of allocation	Follow‐up of professionals	Follow‐up of patients or episodes of care	Blinded assessment of primary outcome(s)	Baseline measurement	Reliable primary outcome measure(s)	Protection against contamination	Overall risk of bias
Abelson 2003	Not clear	Not relevant	Not done	Not clear	Done	Done	Done	High
Chumbley 2002	Done	Not relevant	Done	Not clear	Not clear	Done	Done	Moderate
Clark 1999	Not clear	Not relevant	Done	Not clear	Done	Done	Done	Moderate
Aabakken 1997	Done	Not relevant	Done	Not clear	Not clear	Done	Done	Moderate
Polowczyk 1993	Not clear	Not relevant	Done	Not clear	Done	Done	Done	Moderate
Guarino 2006	Not clear	Not relevant	Not done	Not clear	Done	Not clear	Done	High

• Low risk of bias: all seven criteria scored as 'done';

• Moderate risk of bias: one or two criteria scored as 'not clear' or 'not done';

• High risk of bias: more than two criteria scored as 'not clear' or 'not done'.

In the table Characteristics of included studies we used the following convention for rating allocation concealment:

• Grade A: Adequate concealment;

• Grade B: Uncertain;

• Grade C: Clearly inadequate concealment;

• Grade D: Not used (no attempt at concealment).

We extracted the following additional information from included trials using a standardised data extraction form:

• Type of study (randomised controlled trial);

• Type of process (development of healthcare policy, clinical practice guidelines, patient information material, or healthcare research);

• Purpose and scope for the activity in which consumers are involved;

• Trial setting (country, key features of the system in which the process is undertaken);

• Characteristics of the consumer participants (individual patients or members of the public; or members of organised groups);

• Characteristics of the professionals (professional status, former experience of collaborating with consumers);

• Characteristics of the interventions that are compared (degree of consumer involvement, forum for communication, methods of decision‐making, recruitment, training and support);

• Main outcome measures and trial duration; and

• The results for the main outcome measures.

We calculated a mean difference for one outcome reported by Clark 1999 and Polowczyk 1993 using a fixed effect model. The remaining results of the included trials were not pooled because of heterogeneity in interventions and outcome measures.

We had postulated the following potential explanatory factors for variation in effects: differences in the characteristics of the interventions, settings, types of process, trial quality. However, the interventions and outcome measures in the included trials were so diverse, that this was not relevant.

We have summarised what is known about the effects of various ways of involving consumers, noting important methods of involvement for which no evaluations were found. Quality of the evidence (the extent to which one can be confident that an estimate of effect is correct) was graded using the approach recommended by the GRADE Working Group (Guyatt 2008). This approach distinguishes between four grades of evidence: high, moderate, low and very low quality evidence.

We aimed to identify important factors that should be taken into consideration by anyone contemplating using any of the identified means of promoting consumer involvement, including: possible trade‐offs (of the expected benefits versus harms, if any, and costs), the quality of the available evidence, possible differences in baseline levels of consumer involvement, and other important factors that might have affected the applicability of the available evidence to practice in specific settings. As we did not find much evidence, we have provided a framework for future evaluations and guidance for evaluating interventions to involve consumers in healthcare decisions at the population level (Table 5).

Consumer participation

For the previous version of this review, Nilsen 2006, we established a consumer panel (e‐mail discussion list), consisting of members of the Cochrane Consumer Network. The consumer panel was asked to undertake tasks, such as making review authors aware of unpublished studies that could be considered for inclusion and commenting on drafts of the protocol and review. We did not involve consumers in the work of updating this review in 2009, as we did not make changes to the objectives or selection criteria, apart from the decision to include randomised controlled trials only.

Results

Description of studies

For the previous version of this review, Nilsen 2006, electronic searching yielded a total of 9529 citations after duplicates were removed. We obtained full text copies of 118 articles for further assessment. Five trials met our inclusion criteria. Two trials were excluded.

Revised and updated searches for the 2009 version of the review retrieved an additional 5011 citations. We obtained full text copies of 22 articles for further assessment. One trial met our inclusion criteria, resulting in a total of six included trials.

We identified one trial about consumer involvement in healthcare policy (Abelson 2003), three trials about consumer involvement in healthcare research (Clark 1999; Guarino 2006; Polowczyk 1993) and two trials about consumer involvement in the development of patient information material (Aabakken 1997; Chumbley 2002). All of the identified trials evaluated consumer consultation to inform the development of healthcare policy, research or patient information. None of them evaluated consumer involvement in decision making during this development.

Consumers in Guarino 2006 were involved in the development of an informed consent document. Consumers in Abelson 2003 were trial participants and were also involved in healthcare decisions at the population level (that is, involved in determining healthcare priorities). Consumers in Aabakken 1997 and Chumbley 2002 were involved in the preparation of patient information leaflets. Consumers in Clark 1999 and Polowczyk 1993 were involved as data collectors. In Clark 1999 consumers were also involved in the preparation of a satisfaction survey instrument.

We identified no randomised controlled trials about consumer involvement in the development of clinical practice guidelines.

Healthcare policy (one included trial)

Abelson 2003 compared three different methods of consulting consumers. Participants were members of community organisations, and their task was to inform health service priorities by completing a survey. All participants in the telephone discussion and face‐to‐face meeting groups, and 8 of the 17 participants in the mail survey group completed the same survey twice. The two intervention groups participated in either telephone discussion or a community face‐to‐face meeting before completing the survey for the second time. The trial investigated if two‐way discussion (telephone discussion or face‐to‐face meeting) resulted in different healthcare priorities being identified by consumers (and these outcome measures were classified as consumer influence on decisions). In the framework for describing consumer involvement (Table 5) the methods used in Abelson 2003 were categorised as repeated consultations using different forums for communication (face‐to‐face meeting, telephone discussion, mail survey).

Healthcare research (three included trials)

Both Clark 1999 and Polowczyk 1993 organised patient satisfaction surveys among participants recruited from mental health outpatient clinics. Participants were interviewed about their satisfaction with mental health services. The trial investigators collaborated with consumers (former patients) in an ongoing working relationship by involving them in this research as data collectors (interviewers). The intervention group was interviewed by consumers, while the control group was interviewed by health professionals. Consumer interviewers and health professionals used the same survey instrument. The trials investigated whether data collected by consumer interviewers differed from data collected by health professionals (outcome measures classified as consumer influence on resource utilization). In the framework for describing consumer involvement (Table 5) the authors of Clark 1999 and Polowczyk 1993 worked collaboratively with the interviewers. Despite the collaborative working relationship, there is no evidence that the consumer interviewers participated in decision‐making.

The third trial in this category (Guarino 2006) evaluated an informed consent document for use in a cluster randomised controlled trial (parent trial) investigating the effect of cognitive behavioural therapy and exercise on Gulf War veterans' illnesses. Participants were Gulf War veterans eligible for the parent trial. The intervention group received a consent document developed with input from a focus group of consumers (Gulf War veterans). The control group received a consent document developed by professionals (the trial investigators). The primary outcome measure was participants' self‐rated understanding of the parent trial (classified as consumer influence on healthcare outcomes). Secondary outcomes were satisfaction with study participation, adherence to the protocol and refusal to participate. Consumers in the focus group used the investigator‐developed consent document as a starting point for developing their consent document. They only made minor changes to the background, procedures and benefits sections of the document, and lowered the reading level slightly. In terms of the framework for describing consumer involvement (Table 5), Guarino 2006 consulted consumers face‐to‐face in three focus group sessions.

Patient information material (two included trials and two excluded trials)

Two included trials (Aabakken 1997;Chumbley 2002) and one excluded trial (Roberts 2002) assessed the effects of patient information material that was developed with consumer consultation. One excluded trial (Angell 2003) assessed the effects of patient information material that was developed with consumer collaboration

Chumbley 2002 tested an information leaflet about patient‐controlled analgesia (PCA). Participants were patients preparing to undergo surgery the next day, with no prior experience of PCA. The intervention group received a leaflet developed by health professionals and consumers. Consumers informed the design of the leaflet as participants of focus groups. The control group received a leaflet developed by health professionals alone. The outcome measures were: patients' worries about using PCA; knowledge of PCA (classified as consumer influence on healthcare outcomes); and rating of the quality of the leaflet (classified as consumers' satisfaction with products resulting from consumer involvement). The leaflet developed following consumer consultation included more information than the leaflet developed by health professionals alone. It included information about side effects of PCA, that PCA contains morphine, that overdosing or addiction was not possible and details about how to use PCA. In addition the leaflet developed following consumer consultation had more illustrations and scored better in a readability test.

Aabakken 1997 tested information material about endoscopic procedures. Participants were patients who were recruited on the day of their endoscopic examination. The intervention group received material developed by health professionals after consulting consumers. Individual interviews with consumers were used to inform the development of the material. The control group received material developed by health professionals alone. Outcome measures were patient satisfaction with the material (classified as consumers' satisfaction with products resulting from consumer involvement) and anxiety related to the endoscopic procedure (classified as consumer influence on healthcare outcomes). The authors reported that the leaflets developed following consumer consultation had improved layout and illustrations, simpler language and included more detailed practical information than the leaflet developed by professionals alone. Consumer consultation resulted in the development of three leaflets, each describing one specific endoscopic procedure, as having descriptions of three different procedures in the same leaflet (as in that developed by professionals) could confuse readers. In terms of the framework for describing consumer involvement (Table 5) Aabakken 1997 and Chumbley 2002 consulted consumers, each on a single occasion, face‐to‐face in focus groups and individual interviews respectively.

We excluded two trials (Angell 2003; Roberts 2002); see table Characteristics of excluded studies. These trials both compared information material prepared with consumer involvement to usual care (that is, no information material) which could not serve as fair 'no consumer involvement' comparisons. In these trials, the effects of consumer involvement could not be separated from the effects of the educational material itself.

Risk of bias in included studies

Most of the included trials had some methodological limitations (Table 7). We contacted authors when information about risk of bias was missing, in most cases without success. However, for Aabakken 1997 we received information about allocation concealment and blinding. We assessed the overall risk of bias as high in Abelson 2003, and Guarino 2006, and moderate in the four other trials. The criteria for assessing risk of bias are described in the Methods section.

Effects of interventions

See: Table 1; Table 2; Table 3; Table 4

Five of the six included trials were comparisons of consumer consultation versus no involvement (Aabakken 1997; Chumbley 2002; Clark 1999; Guarino 2006; Polowczyk 1993). Clark 1999 and Polowczyk 1993 also worked collaboratively with consumers to organise these consultations. The sixth trial was a comparison of three different methods of consumer consultation (Abelson 2003). We did not find any trials reporting any of the other comparisons described in Table 5, including involvement in decision‐making, different methods of recruitment, and different ways of providing training and support.

Different communication forums for involvement in health policy

Abelson 2003 compared two forms of deliberative consumer involvement (telephone discussion and a group face‐to‐face meeting) and a mailed survey in eliciting priorities for community health goals. Participants were members of community organisations. Due to a low response rate in the mailed survey group, Abelson and colleagues excluded this group from the analysis. There is no indication as to whether or how the consumer priorities elicited did indeed inform community health goals.

A statistically significant difference was found for one of seven reported health‐related community strengths (an improving local economy) (P < 0.05), with the proportion of people indicating this as very important to health increasing by 7% in the phone group and decreasing by 31% in the face‐to‐face meeting group. There were no statistically significant before and after changes in rankings of five health concerns in the telephone group. There was one statistically significant change in the face‐to‐face meeting group, where mental illness went from an average score of 3.62 to 3.00 on a scale from one to five, with one being the most important (P < 0.05). Based on this trial there is very low quality evidence (Table 2) of telephone discussions compared with face‐to‐face meetings changing consumer priorities for community health goals. Both appeared to achieve more involvement than a mailed survey, based on the low response rate to the mailed survey, and both resulted in changes in the views of participants.

Consumer involvement versus no consumer involvement in research

Two trials compared consumers (patients) with professionals as data collectors in patient satisfaction surveys (Clark 1999; Polowczyk 1993). They compared the data collected, to investigate if responses given to consumer interviewers differed from responses given to staff (professional) interviewers. Neither of the trials reported whether or how these data informed subsequent service development.

Clark 1999 found that those who were surveyed by consumer interviewers gave significantly more 'extreme negative' responses, defined as a score of zero on any question (on a scale from 0 to 4), compared to those surveyed by staff interviewers (P < 0.02). There was no significant difference between the two groups in the number of 'extreme positive' responses, defined as a score of four on any question. Overall, participants reported high levels of satisfaction with mental health outpatient services regardless of whether the interviewer was a consumer or staff member.

Polowczyk 1993 also found that participants reported high levels of satisfaction with mental health outpatient services regardless of whether the interviewer was a consumer. In this trial the average satisfaction score was a little lower in the consumer (client) interviewed group than it was in the staff interviewed group (0.16 on a scale from 1 to 4, P < 0.05).

When we pooled the results of these two trials, the overall difference was similar (0.14 on a scale from 0 to 4, P = 0.001), see Analysis 1.1. Clark 1999 asked for patients' views on satisfaction with case management services and with physicians' services. We have reported results from satisfaction with case management services only. The conclusion would not have differed if scores from satisfaction with physicians' services had been used. Based on these two trials there is low quality evidence (Table 1) of small differences in satisfaction survey results when consumer interviewers are used instead of staff interviewers. There is no evidence as to whether subsequent decisions were influenced.

1.1. Analysis.

Comparison 1 Patient versus staff interviewers, Outcome 1 Satisfaction with care (Clark: 0 = low, 4 = high; Polowczyk: 1 = low, 4 = high).

Guarino 2006 compared an informed consent document developed with consumer input (potential trial participants) to a consent document developed by professionals (trial investigators). The primary outcome in this trial, understanding of the trial described in the consent document (parent trial), was measured by a four‐item informed consent questionnaire (ICQ‐4). Data were collected four times, at trial entry and at three follow‐up visits. Overall, there were no statistically significant differences between trial groups for understanding of the parent trial. This was the case at all four time points. Mean (95% CI) group differences ranged from +0.020 (‐0.015 to 0.055) (better understanding) at entry to ‐0.021 (‐0.054 to 0.012) (worse understanding) at three‐months for the participant versus the investigator document group. In both groups level of understanding was generally high. For example, approximately one‐third of participants answered that they completely understood the trial at the time when they decided to participate (highest level on a four‐point scale) compared to less than 5% who said they did not understand the trial at all (lowest level on the scale). Based on this trial there is low quality evidence that consumer consultation in the development of consent documents may have little if any impact on participant’s self‐reported understanding of the trial described in the consent document, satisfaction with study participation, adherence to the protocol or refusal to participate (Table 4).

Consumer involvement versus no consumer involvement in preparing patient information

Two trials evaluated products (patient information leaflets) which were developed following consumer consultation (Aabakken 1997; Chumbley 2002). The leaflets were compared with patient information developed without consumer consultation.

As the leaflets developed following consumer consultation contained more detailed information than the leaflets developed without consumers, both trials referred to previous research that suggests that giving patients sufficient health information may reduce their anxiety, and assessed anxiety (worries) as outcomes. The trials did not, however, find that patient information material developed following consumer consultation reduced worries or anxiety related to patient‐controlled analgesia (PCA) (Chumbley 2002) or endoscopy (Aabakken 1997). Chumbley 2002, using a five‐point Likert scale, found small differences in the proportions of patients who were worried about becoming addicted (P = 0.68), about getting too much drug (P = 0.21), or about giving themselves too much drug (P = 0.26) with PCA. Between 80% and 90% of participants were not at all or only slightly worried in both groups for all three questions evaluating worries. Aabakken 1997 used a five‐point Likert scale to measure anxiety, ranging from 'completely calm' to 'terrified', for various aspects of endoscopic procedures. For anxiety level, there was a small, statistically significant difference in favour of the leaflet developed with consumers (P = 0.04). Based on these two trials, there is moderate quality evidence (Table 3) that there may be little or no difference in worries or anxiety associated with procedures for patients receiving information material developed following consumer consultation, compared with patients receiving material developed without consumer consultation. Other outcomes reported were the quality of patient information material and patients' knowledge.

In Chumbley 2002 more patients rated the information given in leaflets developed following consumer consultation as being very or extremely clear (84%), compared with patients who received leaflets which had no consumer consultation in their preparation (48%, P < 0.001). Thirty per cent of those who read the leaflet developed following consumer consultation required no more information about the 'painkiller', compared with 8% of those who read the leaflet developed without consumer consultation (P = 0.002). Aabakken 1997 found that patients were significantly more satisfied with leaflets developed following consumer consultation compared with leaflets developed without consumer consultation (P = 0.04).

Based on these two trials there is moderate quality evidence (Table 3) that consumer consultation prior to developing patient information material probably results in material that is more relevant, readable and understandable to patients.

Chumbley 2002 used six multiple choice questions about PCA to measure knowledge obtained from reading leaflets. Participants who read the leaflet developed following consumer consultation were significantly better informed. There was a statistically significant difference in the proportion of correct responses for five of the six questions. For example, 58% of those who read the leaflet developed following consumer consultation recognised that all the side effects listed could be caused by PCA, whereas none of those who read the leaflet developed without consumer consultation gave the correct answer (P < 0.001); and 49 of those who read the leaflet developed following consumer consultation knew that morphine was used in PCA compared with seven of those who read the leaflet developed without consumer consultation (P < 0.001). Information about side effects and morphine use was provided only in the leaflet developed follow ing consumer consultation. Based on this trial there is moderate quality evidence (Table 3) that consumer consultation before developing patient information material probably can improve the knowledge of patients who read the material.

Discussion

A summary of all of the included comparisons, the outcomes reported in these and the type of process (guidelines development, research, development of patient information material and health policy) is shown in a matrix (Table 8). The most striking feature of the matrix is the preponderance of empty cells, where we were unable to identify any trials despite our extensive literature searches. There was only one trial of consumer priorities for healthcare policy (but without evidence of consumer views informing policy decisions), three trials in healthcare research (again, without evidence of consumer views informing decisions about research or services), and two trials of consumer involvement in developing patient information material. We did not find any trials of consumer involvement in developing practice guidelines.

4. Matrix.

Outcome

Degree of involvement

Forum for communication

Involvement in decision making

Implementing involvement

G

R

I

P

G

R

I

P

G

R

I

P

G

R

I

P

Response rates

Decisions

B

Prioritising

B

Rating

B

Ranking

B

Health care

A C

Quality of info

A C

Knowledge

C

Resource use

D E F

Research results

D E F

Worries

A C

Cost

G: Guidelines; R: Research; I: Information material; P: Policy

A: Aabakken 1997

B: Abelson 2003

C: Chumbley 2002

D: Clark 1999

E: Guarino 2006

F: Polowczyk 1993

We found only one trial of consumers collaborating in an on‐going working relationship, and no trials of consumer control. We found no trials of consumer involvement in decision making. In the trials included in this review, consumers only collected or provided data to inform decisions.

The evidence appears to be strongest for the benefits of consumer involvement compared with no consumer involvement in developing patient information materials (Table 3). Few conclusions can be drawn about the benefits, adverse effects or costs of consumer involvement in any of the other areas that we have considered in this review. Little can be concluded about the benefits, adverse effects and costs of different forums for communication, different degrees of involvement, different ways of recruiting consumers, different ways of providing training and support, or different degrees of financial support (Table 5). There is also a paucity of evidence for most outcomes, including participation or response rates, decisions, healthcare outcomes, satisfaction, impacts on participating consumers, and costs.

None of the included trials addressed possible adverse effects of consumer involvement, such as tokenism or consumer involvement slowing the process down and making it costlier.

The validity of the outcomes knowledge (Chumbley 2002) and anxiety (Aabakken 1997; Chumbley 2002) is probably limited to evaluations of material providing more detailed or additional information (as a result of consumer involvement) than the control material. The impact of information on anxiety level may also depend on the topic and the setting in which the information is provided. Other important outcomes not considered in Aabakken 1997 or Chumbley 2002 are whether consumer involvement in developing patient information material can increase the recipients' ability to participate in decision‐making about their own health care, and their level of satisfaction with the healthcare decisions made after having read the material.

Clark 1999 and Polowczyk 1993 have methodological limitations (no information given about allocation concealment or blinding). In addition, when investigating if consumers instead of staff members should be used as interviewers in patient satisfaction surveys, it might be helpful to consider whether there are factors other than the distinction between consumer and staff that could influence the responses; for example, the personality of the interviewer, or how well the interviewer and the interviewed previously knew each other.

In Guarino 2006 there was little or no difference in participants’ self‐reported understanding of the parent trial between a consent document developed with and without consumer involvement. This could be due to the two documents being quite similar, as the focus group of consumers used the investigator‐developed document as a starting point for their revised document. Other reasons for similar results in the two trial groups could be that the participants (Gulf War veterans) were all trained in reading complicated documents, and that the parent trial was relatively easy to understand. It might have been helpful to measure both perceived and actual understanding in trial participants. In addition, Guarino 2006 did not provide information on allocation concealment and blinding, and used an unvalidated questionnaire prior to the trial.

We assessed the quality of evidence for the findings from Abelson 2003 to be very low because of methodological limitations (no information given about allocation concealment or blinding and more than 20% loss to follow‐up) and a small sample (3 trial arms with a total of 46 participants).

Authors' conclusions

Implications for practice.

There is a huge gap in the evidence from randomised controlled trials about desirable and adverse effects of consumer involvement in healthcare decisions at the population level, or how to achieve effective consumer involvement.

We found evidence, from two trials that provide moderate quality evidence, that consumer involvement in developing patient information material probably improves the clarity of the information and the knowledge of people who read the material.

We found evidence, from two trials that provide low quality evidence, that using consumer interviewers instead of staff interviewers might result in small differences in satisfaction surveys.

We found low quality evidence from one trial that consumer involvement in the development of an informed consent document might have little or no impact on participants' perceived understanding of the trial described in the consent document.

There was very low quality evidence from one trial of differences in the views of participants towards priorities for community health goals when telephone discussions were used compared with face‐to‐face meetings to involve the public.

As we discussed in the Background, there are good arguments for attempting to achieve effective consumer involvement. In light of the paucity of evidence we have demonstrated in this review, people making decisions about how best to involve consumers may wish to rely on advice based on practical experience and common sense, such as the principles of successful consumer involvement in NHS research (Telford 2004). Finally, they should consider evaluating options about which they are uncertain, in well‐designed and reported randomised trials if possible.

Implications for research.

The effects of involving consumers in developing healthcare policy and research, clinical practice guidelines and patient information material remain largely unevaluated. The six trials included in this review demonstrate that randomised controlled trials of consumer involvement are feasible. Variation in practice and uncertainty about how best to achieve effective consumer involvement suggest that there is a need for trials to reduce this uncertainty. Trials are needed to evaluate the effects of different:

• methods for recruiting consumers;

• degree of involvement (relationship between consumers and professionals);

• forums for communication;

• degrees of consumer involvement in decision‐making;

• ways of providing training and support; and

• degrees of financial support.

Given the few randomised controlled trials identified, updates of this review will continue to have a broad focus.

History

Protocol first published: Issue 1, 2004
 Review first published: Issue 3, 2006

Date	Event	Description
20 October 2009	New search has been performed	We revised search strategies and ran new searches. We limited the types of studies included to RCTs only. One study (Guarino 2006) has been added to the review; this had no substantial impact upon the review's conclusions. This updated review was first published on issue 1 2010 of The Cochrane Library.
20 October 2008	Amended	Converted to new review format.

Appendices

Appendix 1. Search strategies

Consumers and Communication Review Group specialised register search strategy  

Strategy 1: 
 ("consumer participation" or "patient participation" or "patient advocacy" or "consumer advocacy" or "consumer organisation*" or "consumer organization*" ) and ("health policy" or "health planning" or "health care rationing" or "health care reform" or "health priorities" or "community health planning" or "state health plan*" or "health polic*" or "health reform*" or "health services research" or "peer review research" or "health research" or "research agenda" or "research priorit*" or "research program*" or guideline* or "information services" or "health information" or "patient* information" or "consumer* information") 
 
 Strategy 2: 
 CCCRG coding scheme codes. 
 S3 or Cb‐c or W2 or S3 
 
 Strategy 3: 
 (consumer* or stakeholder* or patient* or user* or lay* or client* or disab* or citizen* or communit* or public or advoca* or carer* or caregiver* or parent* or relative*) and ("health policy" or "health planning" or "health care rationing" or "health care reform" or "health priorities" or "community health planning" or "state health plan*" or "health polic*" or "health reform*" or "health services research" or "peer review research" or "health research" or "research agenda" or "research priorit*" or "research program*" or guideline* or "information services" or "health information" or "patient* information" or "consumer* information")

CENTRAL (including the Cochrane Consumers and Communication Review Group's Specialised Register)

#1        MeSH descriptor Consumer Participation, this term only

#2        MeSH descriptor Patient Participation, this term only

#3        MeSH descriptor Consumer Advocacy, this term only

#4        MeSH descriptor Consumer Organizations, this term only

#5        MeSH descriptor Public Opinion, this term only

#6        (consumer? or patient? or stakeholder? or user? or lay or citizen? or public or client?) NEAR/2 (particip* or involv* or represent* or collaborat* or consult* or contribut* or engagement or opinion? or deliberat* or dialogue):ti,ab

#7        citizen* NEXT (council? or jury or juries or panel?):ti,ab

#8        public NEXT (meeting? or forum?):ti,ab

#9        (participatory NEXT intervention?):ti,ab

#10      (consumer? or patient?) NEAR/2 (organisation? or organization?):ti,ab

#11      MeSH descriptor Health Policy, this term only

#12      MeSH descriptor Health Planning, this term only

#13      MeSH descriptor Health Priorities, this term only

#14      MeSH descriptor Policy Making, this term only

#15      MeSH descriptor Decision Making, this term only

#16      MeSH descriptor Decision Making, Organizational, this term only

#17      health* NEAR/3 (policy or policies or planning or priorit*):ti,ab

#18      (decision NEXT making):ti,ab

#19      MeSH descriptor Health Services Research, this term only

#20      MeSH descriptor Health Care Surveys, this term only

#21      MeSH descriptor Research, this term only

#22      (participatory NEXT research):ti,ab

#23      (health* NEAR/3 research):ti,ab

#24      research NEAR/3 (agenda? or priorit* or program*):ti,ab

#25      (design* or (recruit* NEAR/3 subject?) or (data NEAR/3 collect*) or (analysis NEAR/3 data) or (dissemination NEAR/3 finding?) or (dissemination NEAR/3 result?) or interviewer?):ti,ab

#26      research:ti,ab

#27      (#25 AND #26)

#28      MeSH descriptor Guidelines as Topic, this term only

#29      MeSH descriptor Practice Guidelines as Topic, this term only

#30      guideline?:ti,ab

#31      MeSH descriptor Pamphlets, this term only

#32      ((health NEXT information) or (information NEXT material?) or (patient NEXT information) or (consumer NEXT information) or pamphlet? or booklet? or leaflet? or brochure?) NEAR/3 (develop* or produc* or evaluat* or design* or "feed back" or feedback or input or "in put" or comment*):ti,ab

#33      (#1 OR #2 OR #3 OR #4 OR #5 OR #6 OR #7 OR #8 OR #9 OR #10)

#34      (#11 OR #12 OR #13 OR #14 OR #15 OR #16 OR #17 OR #18 OR #19 OR #20 OR #21 OR #22 OR #23 OR #24 OR #27 OR #28 OR #29 OR #30 OR #31 OR #32)

#35      (#33 AND #34)

MEDLINE

1. Consumer Participation/

2. Patient Participation/

3. Consumer Advocacy/

4. Consumer Organizations/

5. Public Opinion/

6. ((consumer? or patient? or stakeholder? or user? or lay or citizen? or public or client?) adj (particip$ or involv$ or represent$ or collaborat$ or consult$ or contribut$ or engagement or deliberat$ or dialogue or opinion?)).tw.

7. (citizen$ adj (council? or jury or juries or panel?)).tw.

8. (public adj (meeting? or forum?)).tw.

9. participatory intervention?.tw.

10. ((consumer? or patient?) adj organi#ation?).tw.

11. or/1‐10

12. Health Policy/

13. Health Planning/

14. Health Priorities/

15. Policy Making/

16. Decision Making/

17. Decision Making, Organizational/

18. (health$ adj3 (policy or policies or planning or priorit$)).tw.

19. decision making.tw.

20. Health Services Research/

21. Health Care Surveys/

22. Research/

23. participatory research.tw.

24. ((health or health care or healthcare) adj research).tw.

25. (research adj3 (agenda? or priorit$ or program$)).tw.

26. ((design$ or (recruit$ adj3 subject?) or (data adj3 collect$) or (analysis adj3 data) or dissemination) adj3 (finding? or result? or interviewer?)).tw.

27. research.tw.

28. 26 and 27

29. Guidelines as Topic/

30. Practice Guidelines as Topic/

31. guideline?.tw.

32. Pamphlets/

33. ((health information or information material? or patient information or consumer information or pamphlet? or booklet? or leaflet? or brochure?) adj3 (develop$ or produc$ or evaluat$ or design$ or feed back or feedback or input or in put or comment$)).tw.

34. or/12‐25,28‐33

35. randomized controlled trial.pt.

36. controlled clinical trial.pt.

37. random$.tw.

38. placebo.ab.

39. trial.ab.

40. groups.ab.

41. or/35‐40

42. Animals/

43. Humans/

44. 42 not (42 and 43)

45. comment.pt.

46. editorial.pt.

47. or/45‐46

48. 41 not (44 or 47)

49. 11 and 34 and 48

EMBASE

1. Patient Participation/

2. Consumer/

3. Consumer Advocacy/

4. Public Opinion/

5. ((consumer? or patient? or stakeholder? or user? or lay or citizen? or public or client?) adj (particip$ or involv$ or represent$ or collaborat$ or consult$ or contribut$ or engagement or deliberat$ or dialogue or opinion?)).tw.

6. (citizen$ adj (council? or jury or juries or panel?)).tw.

7. (public adj (meeting? or forum?)).tw.

8. participatory intervention?.tw.

9. ((consumer? or patient?) adj organi#ation?).tw.

10. or/1‐9

11. Health Care Policy/

12. Health Care Planning/

13. Decision Making/

14. (health$ adj3 (policy or policies or planning or priorit$)).tw.

15. decision making.tw.

16. Health Services Research/

17. Medical Research/

18. Research/

19. participatory research.tw.

20. ((health or health care or healthcare) adj research).tw.

21. (research adj3 (agenda? or priorit$ or program$)).tw.

22. ((design$ or (recruit$ adj3 subject?) or (data adj3 collect$) or (analysis adj3 data) or dissemination) adj3 (finding? or result? or interviewer?)).tw.

23. research.tw.

24. 22 and 23

25. Practice Guideline/

26. guideline?.tw.

27. Consumer Health Information/

28. Patient Information/

29. Medical Information/

30. ((health information or information material? or patient information or consumer information or pamphlet? or booklet? or leaflet? or brochure?) adj3 (develop$ or produc$ or evaluat$ or design$ or feed back or feedback or input or in put or comment$)).tw.

31. or/11‐21,24‐30

32. Randomized Controlled Trial/

33. Controlled Study/

34. Major Clinical Study/

35. (randomi$ or randomly).tw.

36. (controlled adj (study or trial or design)).tw.

37. or/32‐36

38. Nonhuman/

39. Animal/

40. Animal Experiment/

41. or/38‐40

42. Human/

43. 41 not (41 and 42)

44. 37 not 43

45. 10 and 31 and 44

CINAHL

#	Query
S32	S3 and S26 and S31
S31	S27 or S28 or S29 or S30
S30	TI ( randomi* or randomly or controlled N2 trial or controlled N2 study or controlled N2 design ) or AB ( randomi* or randomly or controlled N2 trial or controlled N2 study or controlled N2 design )
S29	PT Clinical Trial
S28	(MH "Random Assignment")
S27	(MH "Clinical Trials")
S26	S4 or S5 or S6 or S7 or S8 or S9 or S10 or S11 or S12 or S13 or S14 or S15 or S16 or S17 or S18 or S19 or S20 or S21 or S22 or S23 or S24 or S25
S25	TI ( health* N2 policy or health N2 priorit* or health* N2 planning ) or AB ( health* N2 policy or health N2 priorit* or health* N2 planning )
S24	TI "decision making" or AB "decision making"
S23	TI ( health* N2 research or clinical N2 research ) or AB ( health* N2 research or clinical N2 research )
S22	TI guideline* or AB guideline*
S21	TI ( health* N2 information or "information material" or "patient information" or "consumer information" or booklet* or pamphlet* or leaflet* ) or AB ( health* N2 information or "information material" or "patient information" or "consumer information" or booklet* or pamphlet* or leaflet* )
S20	(MH "Public Opinion")
S19	(MH "Research Priorities")
S18	(MH "Health Services Research")
S17	(MH "Research, Mental Health")
S16	(MH "Clinical Research")
S15	(MH "Research")
S14	(MH "Health Policy")
S13	(MH "Health and Welfare Planning")
S12	(MH "Health Resource Allocation")
S11	(MH "Decision Making")
S10	(MH "Decision Making, Organizational")
S9	(MH "Surveys")
S8	(MH "Practice Guidelines")
S7	(MH "Consumer Health Information")
S6	(MH "Print Materials")
S5	(MH "Pamphlets")
S4	(MH "Health Information")
S3	S1 or S2
S2	TI ( consumer N2 particip* or patient* N2 particip* or client* N2 particip* or citizen* N2 particip* or consumer N2 involve* or patient* N2 involve* or client* N2 involve* or citizen* N2 involve* ) or AB ( consumer N2 particip* or patient* N2 particip* or client* N2 particip* or citizen* N2 particip* or consumer N2 involve* or patient* N2 involve* or client* N2 involve* or citizen* N2 involve* )
S1	(MH "Consumer Participation")

PsycINFO

1. Client Participation/

2. Clients/

3. Advocacy/

4. Public Opinion/

5. ((consumer? or patient? or stakeholder? or user? or lay or citizen? or public or client?) adj (particip$ or involv$ or represent$ or collaborat$ or consult$ or contribut$ or engagement or deliberat$ or dialogue or opinion?)).ti,ab.

6. (citizen$ adj (council? or jury or juries or panel?)).ti,ab.

7. (public adj (meeting? or forum?)).ti,ab.

8. participatory intervention?.ti,ab.

9. ((consumer? or patient?) adj organi#ation?).ti,ab.

10. or/1‐9

11. Policy Making/

12. Health Care Policy/

13. Government Policy Making/

14. Decision Making/

15. (health$ adj3 (policy or policies or planning or priorit$)).ti,ab.

16. decision making.ti,ab.

17. Experimentation/

18. Surveys/

19. participatory research.ti,ab.

20. ((health or health care or healthcare) adj research).ti,ab.

21. (research adj3 (agenda? or priorit$ or program$)).ti,ab.

22. ((design$ or (recruit$ adj3 subject?) or (data adj3 collect$) or (analysis adj3 data) or dissemination) adj3 (finding? or result? or interviewer?)).ti,ab.

23. research.ti,ab.

24. 22 and 23

25. Treatment Guidelines/

26. guideline?.ti,ab.

27. Written Communication/

28. Reading Materials/

29. ((health information or information material? or patient information or consumer information or pamphlet? or booklet? or leaflet? or brochure?) adj3 (develop$ or produc$ or evaluat$ or design$ or feed back or feedback or input or in put or comment$)).ti,ab.

30. or/11‐21,24‐29

31. (randomi$ or randomly).ti,ab.

32. control$.ti,ab.

33. groups.ab.

34. "2000".md.

35. or/31‐34

36. animal.po.

37. human.po.

38. 36 not (36 and 37)

39. 35 not 38

40. 10 and 30 and 39

Sociological Abstracts

(KW=(consumer* within 2 particip*) or (consumer* within 2 involv*) or (consumer* within 2 represent*) or (consumer organisation*) or (consumer organization*) or (patient organisation*) or (patient organization*) or (patient* within 2 particip*) or (patient* within 2 involv*) or (patient* within 2 represent*) or (client* within 2 particip*) or (client* within 2 involv*) or (client* within 2 represent*) or (citizen* within 2 particip*) or (citizen* within 2 involv*) or (citizen* within 2 represent*) or (stakeholder* within 2 particip*) or (stakeholder* within 2 involv*) or (stakeholder* within 2 represent*) or (consumer advocacy) or (patient advocacy) or (client advocacy))

And

(KW=(health* within 2 policy) or (health* within 2 planning) or (health* within 2 priorit*) or (decision making) or (health* within 2 research) or (medical within 2 research) or (health survey*) or (research within 2 agenda) or (research within 2 priorit*) or (research program*) or (guideline*) or (health* within 2 information) or (patient information) or (consumer information) or (information material*) or (pamphlet*) or (booklet*) or (leaflet*) or (brochure*))

And

(KW=(randomi* or randomly or control* or compare* or effect*))

SIGLE  

(((consumer* or stakeholder? or patient? or user? or lay* or disab* or citizen? or communit* or public or advoca* or carer? or caregiver? or parent? or relative? or client?) adj (particip* or involv* or represent* or collaborat* or consult* or contribut*)) or ((consumer* or stakeholder? or patient? or user? or lay* or disab* or citizen? or communit* or public or advoca* or carer? or caregiver? or parent? or relative? or client?) near6 (particip* or involv* or represent* or collaborat* or consult* or contribut*))) and (health* or clinical)

Data and analyses

Comparison 1. Patient versus staff interviewers.

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
1 Satisfaction with care (Clark: 0 = low, 4 = high; Polowczyk: 1 = low, 4 = high)	2	650	Mean Difference (IV, Fixed, 95% CI)	‐0.14 [‐0.23, ‐0.06]

Characteristics of studies

Characteristics of included studies [ordered by study ID]

Aabakken 1997.

Methods	RCT Risk of Bias: Moderate Trial aim: To incorporate some patient feedback into patient information brochures about endoscopic procedures, and to compare the three new brochures with the old brochure in terms of procedure‐related anxiety and general patient satisfaction.
Participants	235 patients who were summoned for endoscopic examinations were randomised to evaluate a patient information brochure developed with consumer input (n =115) compared to a brochure developed by health professionals only (n = 120). Patients were 130 women and 105 men, mean age 56 years. 124 patients (54%) had previously undergone an endoscopic procedure. Setting: Ullevaal University Hospital. Country: Norway.
Interventions	Type of process: Consumer involvement in development of patient information material. A set of three new patient information brochures about endoscopic procedures developed with input from patients (intervention) was compared to an old patient information brochure developed by health professionals (control). Patient information which was included in the new brochures was obtained via patient questionnaires.
Outcomes	Level of anxiety related to endoscopy, patient satisfaction with brochure.
Notes	This trial consisted of two parts. Consumer involvement in development of information took place in the first part, as 136 patients were surveyed about their information needs.
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Low risk	A ‐ Adequate

Abelson 2003.

Methods	RCT Risk of Bias: High Trial aim: 'To examine the effects of introducing different opportunities for deliberation into a process for obtaining public input into a community health goals priority setting project'.
Participants	46 participants were identified and recruited from a list of 176 community organisations. They were randomly assigned to three trial arms: mail survey (17), telephone discussion (16), face‐to‐face group meeting (13). Their average age was 47.5 years. 80% female. 93% with at least college or university‐level degree. 80% employed. Setting: Small community of approximately 90,000 residents. Country: Canada
Interventions	Type of process: Consumer involvement in healthcare policy. All participants were asked to prioritise local health concerns by completing a survey. Then two of the trial groups were given the opportunity to deliberate (telephone discussion and a 2.5 hour long face‐to‐face community meeting respectively) before completing the same survey again. The mail survey group was given no opportunity to deliberate, and 8 of its participants completed the survey for the second time. Priorities were compared both before and after deliberation and between trial groups.
Outcomes	Impact of deliberative methods on: prioritising local health concerns for action; rating the importance of local strengths for improving community health; and ranking of health determinants.
Notes	Due to low response rate in the mail survey group, trial authors excluded this group's results from their analysis.
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Unclear risk	B ‐ Unclear

Chumbley 2002.

Methods	RCT Risk of Bias: Moderate Trial aim: To formulate and evaluate an information leaflet for patients using patient‐controlled analgesia (PCA).
Participants	100 patients with no former experience of using PCA were recruited on the day before having surgery. They were randomised to evaluate a patient information leaflet developed with consumer input (n = 50) compared to a leaflet developed by health professionals only (n = 50). Mean age: 48 years (21 to 85). The patients had undergone different types of surgery. Setting: 10 surgical wards. Country: UK.
Interventions	Type of process: Consumer involvement in developing of patient information material. A new patient information leaflet about PCA developed with input from patients (intervention) was compared to an old patient information leaflet developed by health professionals (control). The patient information which was included in the new leaflet was obtained via focus groups.
Outcomes	Main outcomes: Worries about using PCA; the clarify of information regarding PCA; knowledge of PCA.
Notes	This trial consisted of two parts. Consumer involvement in development of information took place in the first part (qualitative study). Seven focus groups were established to obtain consumer views on content and design of the leaflet.
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Low risk	A ‐ Adequate

Clark 1999.

Methods	RCT Risk of Bias: Moderate Allocation concealment: Clients interviewed by the client interviewers had been ill for a longer period than those interviewed by the staff members. Small number of interviewers (four at each of the two facilities). An interviewer effect could have been related to the personality of the interviewers rather than to the distinction client interviewer vs staff interviewer. Trial aim: To look for differences in data collected by consumers compared to data collected by health professionals in a patient satisfaction survey.
Participants	120 outpatient clients, aged 18 to 65 years who had a diagnosis of a major mental illness were randomly assigned to be interviewed by either a consumer or a professional about their satisfaction with services . Setting: Clarke Institute of Psychiatry and Queen Street Mental Health Centre in Toronto. Country: Canada.
Interventions	Type of process: Consumer involvement in research. The intervention group (60) was interviewed by consumers. The control group (60) was interviewed by professionals.
Outcomes	Interviewer effect measured by between‐group difference in level of patient satisfaction with case management and physicians' services, and between‐group difference in the number of extremely negative or positive responses.
Notes	Consumers were involved in research (patient satisfaction survey) as interviewers (data collectors).
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Unclear risk	B ‐ Unclear

Guarino 2006.

Methods	Cluster RCT Risk of Bias: High Trial aim: To compare an informed consent document developed with input from a consumer group of Gulf War veterans to a consent document developed by trial investigators in terms of self‐reported participant understanding, satisfaction with trial participation, trial refusal and adherence to the parent trial protocol.
Participants	20 medical centers that previously had recruited participants with Gulf War veterans’ illnesses into the parent trial of the current trial were randomised to receive either the consumer‐developed consent form or the investigator‐developed consent form. Setting: 18 Veterans Affairs and two Department of Defence medical centers Country: USA
Interventions	Type of process: Consumer involvement in research. The intervention group (10 medical centers/505 veterans) received a consumer‐developed consent form. The control group (10 centers/514 veterans) received a investigator‐developed consent form.
Outcomes	Self‐reported participant understanding, satisfaction with study participation, adherence to the protocol and refusal to participate.
Notes	Overall mean participant education was 14.1 years. Years of education were significantly higher in the intervention group than in the control group.
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Unclear risk	B ‐ Unclear

Polowczyk 1993.

Methods	RCT Risk of Bias: Moderate Allocation concealment: Number of participants in control group: 305. Number of participants in intervention group: 225. This difference in the group size indicates that the randomisation procedure has not been satisfactory. Trial aim: To look for differences in data collected by consumers compared to data collected by health professionals in a patient satisfaction survey.
Participants	530 patients with serious and persistent mental illness who attended ten outpatient clinics were randomly assigned to be interviewed about their satisfaction with services by either a consumer or a professional. Setting: Ten outpatient clinics, three continuing treatment centres, and a psychosocial club, operated by the Kings Park Psychiatric Center, New York state. Country: USA.
Interventions	Type of process: Consumer involvement in research. The intervention group (225) was interviewed by consumers. The control group (305) was interviewed by professionals.
Outcomes	Interviewer effect measured by between‐group differences in level of patient satisfaction with mental health services.
Notes	Consumers were involved in research (patient satisfaction survey) as interviewers (data collectors).
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Unclear risk	B ‐ Unclear

Characteristics of excluded studies [ordered by study ID]

Study	Reason for exclusion
Angell 2003	Compared information material prepared with consumer involvement to usual care, which could not serve as a fair 'no consumer involvement' comparison. The effects of consumer involvement could not be separated from the effects of the educational material itself.
Roberts 2002	Compared information material prepared with consumer involvement to usual care, which could not serve as a fair 'no consumer involvement' comparison. The effects of consumer involvement could not be separated from the effects of the educational material itself.

Differences between protocol and review

in the protocol and the previous version of this review, Nilsen 2006, we searched for quasi‐randomised trials, interrupted time series analyses, and controlled before‐after studies in addition to randomised controlled trials. As we identified and included five randomised controlled trials in Nilsen 2006, we decided to search for randomised controlled trials only for the updated review.

Contributions of authors

ESN, HTM and MJ drafted the protocol and developed the search strategy with input from SO and ADO. SO constructed the framework for examining reports of consumer involvement. MJ conducted the searches. ESN, HTM and MJ applied the selection criteria, collected and analysed the data and drafted the review. SO and ADO commented on drafts of the review.

The lead author, ESN, passed away in January 2013.

Sources of support

Internal sources

• Norwegian Knowledge Centre for Health Services, Norway.

• Norwegian Directorate for Health and Social Affairs, Norway.

External sources

• No sources of support supplied

Declarations of interest

None known

Edited (no change to conclusions)