Patient, Family, and Community Advisory Councils in Health Care and Research: a Systematic Review

Abstract

Background

Patient-centeredness is a characteristic of high-quality medical care and requires engaging community members in health systems’ decision-making. One key patient engagement strategy is patient, family, and community advisory boards/councils (PFACs), yet the evidence to guide PFACs is lacking. Systematic reviews on patient engagement may benefit from patient input, but feasibility is unclear.

Methods

A team of physicians, researchers, and a PFAC member conducted a systematic review to examine the impact of PFACs on health systems and describe optimal strategies for PFAC conduct. We searched MEDLINE, Embase, PsycINFO, CINAHL, Scopus, and Social Science Citation Index from inception through September 2016, as well as pre-identified websites. Two reviewers independently screened and abstracted data from studies, then assessed randomized studies for risk of bias and observational studies for quality using standardized measures. We performed a realist synthesis—which asks what works, for whom, under what circumstances—of abstracted data via 12 monthly meetings between investigators and two feedback sessions with a hospital-based PFAC.

Results

Eighteen articles describing 16 studies met study criteria. Randomized studies demonstrated moderate to high risk of bias and observational studies demonstrated poor to fair quality. Studies engaged patients at multiple levels of the health care system and suggested that in-person deliberation with health system leadership was most effective. Studies involving patient engagement in research focused on increasing study participation. PFAC recruitment was by nomination (n = 11) or not described (n = 5). No common measure of patient, family, or community engagement was identified. Realist synthesis was enriched by feedback from PFAC members.

Discussion

PFACs engage communities through individual projects but evidence of their impact on outcomes is lacking. A paucity of randomized controlled trials or high-quality observational studies guide strategies for engagement through PFACs. Standardized measurement tools for engagement are needed. Strategies for PFAC recruitment should be investigated and reported. PFAC members can feasibly contribute to systematic reviews.

Registration and Funding Source

A protocol for record eligibility was developed a priori and was registered in the PROSPERO database of systematic reviews (registration number CRD42016052817). The Department of Veterans Affairs’ Office of Academic Affiliations, through the National Clinician Scholars Program, funded this study.

Electronic supplementary material

The online version of this article (10.1007/s11606-018-4565-9) contains supplementary material, which is available to authorized users.

INTRODUCTION

Patient-centeredness is an important characteristic of high-quality medical care^1–4 and is associated with decreased health care utilization,^{5, 6} improved continuity of care,⁷ and improved clinical outcomes.⁸ The delivery of patient-centered care requires engaging patients, their families, and their communities in decision-making across levels of health systems: at the level of direct care, organizational design, policy-making, and research.^{9, 10} Mechanisms by which patient engagement produces patient-centered care and better health include policy, research, and practice strategies that incorporate patients’ ideas or address their concerns, as well as improved implementation of research findings.^11–14

One approach for engaging patients in health systems, endorsed by several national health care quality agencies, is the formation of patient/family advisory boards or councils (PFACs).^15–18 These are groups of individuals who use services in that health care system (or family members of those that do). They collectively become involved in processes in that system that pertain to education and orientation for staff, ethics, quality improvement, research initiatives, or patient safety.¹⁸ Evidence for the effectiveness of PFACs, however, is lacking.^{19, 20} A 2006 systematic review on mechanisms of patient engagement found little evidence from randomized controlled trials on the impact of PFACs and similar organizations on health care decisions at the population level other than in the development of patient information material, which is more relevant and understandable when PFACs were involved.¹¹ Reviews of the effects of PFACs on ambulatory adult clinics,²¹ engagement in research only,^22–24 or the impact of such programs on the people involved²⁵ have also shown that transparency of process, compensation of patients, stakeholder engagement, and contextual factors impact patient engagement, but that gaps in research on impact on health systems persist.

As patient engagement has become more compulsory for reimbursement and accreditation purposes, PFACs are increasingly prevalent.¹² Yet, evidence-based approaches to PFACs, such as recruitment methods, organizational makeup, and implementation strategies, have not been systematically aggregated and analyzed. We therefore conducted a systematic review with a primary aim of characterizing the impact of PFACs on health systems. We used the framework of Carman, et al., which defines three levels where patient engagement is needed: direct care, organizational design, and policy-making,¹⁰ and due to the growing recognition of the value of community-engaged research, we added a fourth level, engagement in research.^26–29 As earlier systematic reviews have not engaged patients and/or PFAC members in their analysis teams,^21–25 a lack of understanding of the feasibility or impact of engaging the public in systematic reviews persists.³⁰ Given this gap, and because multiple stakeholders are often needed for an ecological perspective that attends to a complex problem in health care,³¹ a secondary aim of this review was to test the feasibility of having a PFAC member participate in the conduct of a systematic review.

METHODS

We followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) standards of quality for reporting systematic reviews.³² We developed a protocol for study eligibility a priori and registered it in the PROSPERO database of systematic reviews (registration number CRD42016052817). Our data synthesis process drew from realist synthesis, an analytic approach driven by realist theory that considers the interaction between context, mechanism, and outcome in evaluating an intervention.^{33, 34} This study was not considered human subjects research by the Yale School of Medicine Human Investigation Committee.

Analysis Team

We crafted a research team of diverse stakeholders because our topic involves collaborations among patients, families, community members, clinicians, and policymakers, and due to recent calls to involve the public in systematic review processes.^35–37 Our team included clinician-investigators who conduct community-engaged research (BO, MR),³¹ a clinician-administrator who organized a PFAC in an academic clinic (IG), a community research liaison for a research fellowship (AG), a medical librarian with experience in systematic reviews (JG), a patient with leadership roles on several PFACs in an academic hospital (MEP), and a post-baccalaureate student (MH).

Study Selection

We defined our intervention of interest using the Agency for Healthcare Research and Quality’s classification of a PFAC: a group that engages “patients and families directly into the planning, delivery, and evaluation of care.”³⁸ We limited our search to those that involved an organization of patients, family members, or community members (possibly including clinicians or hospital staff) that advised a health care system on the level of direct care, organizational design and quality improvement, policy-making, or research, but did not require that the term PFAC be used (eTable 1 in Supplement).¹⁰ Because we anticipated heterogeneity of outcomes, we did not limit our search to particular outcome measures. We sought studies that included a comparison group, including a randomized control, a parallel cohort, or a historical control (i.e., before-and-after evaluations).

Data Sources and Searches

We searched MEDLINE (Ovid MEDLINE 1946 to September Week 3 2016), Embase (Ovid Embase 1974 to 2016 September 30), PsycINFO (Ovid PsycINFO 1967 to September Week 4 2016), CINAHL (Ebsco), Scopus (Elsevier), and Social Science Citation Index (Web of Science, Thomson Reuters). We identified additional studies by scanning other systematic reviews. We also searched the websites of the following pre-identified organizations for appropriate studies: Institute for Patient and Family-Centered Care, RAND, Patient-Centered Outcomes Research Institute, Agency for Healthcare Research and Quality, and the Robert Wood Johnson Foundation. We conducted all searches on October 3, 2016.

To produce relevant controlled vocabulary and keyword terms, we analyzed 12 previously identified key articles using the Yale MeSH Analyzer (http://mesh.med.yale.edu/). In each database, we ran scoping searches and used an iterative process to translate and refine the search strategies. We used the previously identified articles to validate the success of our searches (eSearch 1 in Supplement). We limited our search to English-language articles but not to any time period.

Data Screening and Abstraction

Two authors independently screened titles and abstracts, and conflicts were resolved by consensus. For each screened article, two authors independently abstracted information about the context, intervention, and outcomes into a standardized form. If desired information was not published, we contacted the first author of the article by email to inquire. To pool the varied expertise on our team, at least two team members read all studies, one of whom had health services research experience. Disagreements were resolved by consensus with the input of the first author.

Intervention and outcome heterogeneity precluded meta-analyses. Instead, we drew from a realist synthesis strategy in which reviewers delineated the contextual influences (C) that are hypothesized to have triggered the relevant mechanisms (R) to generate the outcomes (O) of interest.^{34, 39} We arrived at consensus for C-R-O sequences during iterative consultations among the research team, with monthly meetings between the lead investigator (BO) and the PFAC member (MEP), and semi-monthly meetings among the entire team from December 2016 through December 2017. We consulted the PFAC of Yale-New Haven Hospital twice (December 2016 and January 2018) for in-person feedback on study design, terminology for search strategies, and interpretation of the three realist synthesis elements.

Methodological Quality Assessment

Two reviewers independently completed the quality assessment of each study using the Cochrane Risk of Bias Tool for randomized trials⁴⁰ and the Newcastle-Ottawa Scales for observational studies.⁴¹

PFAC Classification

We classified PFACs as those that either informed direct care, organizational design, policy-making, or research, and assessed the outcomes at each of these levels in the health care system separately. We adapted the classification of Carman, et al.,¹⁰ adding engagement in research because of growing recognition of the value of community-engaged research.^26–29 As themes emerged during the abstraction process, we considered alternative ways to classify programs, but found these categories were the most effective and consistent with frameworks that ground other studies.^{42, 43}

RESULTS

Our search yielded 4389 articles, with 3718 remaining after removal of duplicates. We screened 940 abstracts and identified 18 articles that met criteria for inclusion (Fig. 1). On two occasions, we identified two articles that present data on the same study^44–47, for a total of 16 unique studies (Table 1); these included 13 interventional studies (Table 2)^44–58 and three observational studies (Table 3).^59–61 Studies addressed the four levels of the health care system and engaged patients (n = 7), community members (n = 8), and family members (n = 1). They employed heterogeneous terminology to describe organizations and their components: “patient,” “consumer,” or “community” to describe those engaged in organizations called “boards,” “councils,” “committees,” “patient groups,” or “voting panels.” For interventional studies, recruitment to PFACs or similar organizations occurred by nomination (n = 9) or recruitment was not described (n = 4); none involved election processes.

Figure 1.

PRISMA flow diagram of screened articles.

Table 1.

Characteristics of the 16 Reviewed Studies

Characteristic	n (%)
Level of the healthcare system addressed (among all 16 included studies)
Direct care	4 (25)
Organizational design	4 (25)
Policy-making	6 (38)
Research	5 (31)
Patient/family organization recruitment (among 13 interventional studies)
Recruited by clinical/administrative staff	1 (8)
Recruited by community outreach	8 (62)
Elected	0 (0)
Recruitment not described	4 (31)
Outcomes of interest (among all 16 included studies)
Organizational priority setting	5 (32)
Implementation of quality improvement	4 (25)
Recruitment/retention of research subjects	4 (25)
Conduct of research	1 (6)
Direct care services	1 (6)
Clinical outcomes	1 (6)

Table 2.

Organizational Type, Systems Addressed, and Key Findings from Interventional Studies (n = 13)

Author, year of publication, country, study design	Study year(s)	Organization type and recruitment strategy	Level(s) of health care system addressed				Data collection method and key findings
			Direct care	Organizational design	Policy-making	Research
Abelson 2003 Canada RCT	2001	Citizens’ panel; recruited by inviting community organizations to identify one organizational representative each			•		Panels randomized to one of three methods of obtaining input: mail survey, telephone discussion, or 2.5 h community meeting, then surveyed before and after. Ranking of health concerns and health-related community strengths more likely to change and to reach consensus in face to face group Rankings of determinants of health did not change regardless of methods of obtaining input
Boivin 2014a, Boivin 2014b Canada RCT	2010–2011	Deliberation meetings between patients, health professionals; recruited through advertising and snowballing followed by stratified random selection			•		Health and Social Services Centers’ prioritization efforts randomized to those that included patients and professionals or professionals only. All participants surveyed; meetings were video recorded and analyzed. Priorities identified by patients and professionals placed more emphasis on access, self-care support, and partnership with community organizations. Patient involvement increases cost of prioritization process by 17% and time to agreement by 10%. Legitimacy, credibility, and power explain variations in patients’ influence.
Carlson 1990 Sweden RCT	1985–1986	Patient-staff groups in primary health care centers; recruitment not described		•	•		Problems pertinent to diabetes care as identified by patient-staff groups in meetings underwent content analysis, and staff memberswere interviewed. Problems identified were related to the organization of care more than staff-patient relationships or the social environment. After 18 months, plans had been successfully implemented for 70% of the problems.
Chadiha 2011 USA Cohort	2003–2009	Community advisory board; recruitment not described				•	A logic model was used to analyze the process the board used to develop a research volunteer registry of older African Americans. During the 7 years of the study, the registry increased from 102 to 1273 enrollees.
Crowley 2002 United Kingdom Cohort	1997–1998	Community Action on Health committee (PFAC); recruitment by nomination from community organizations followed by elections		•			Before-and-after evaluation of various stakeholders via interviews, mailed questionnaires, and participant-observation. PFAC meetings were most effective when they occurred in community settings and when they were accountable to, or addressed by, community organizations. Providers perceived the committee to be most useful in providing a community perspective and opportunities for dialog with patients
Cupertino 2015 USA Cohort	2012–2013	Community advisory board (CAB); recruited by 2 “key community leaders”				•	Evaluation of one project performed by CAB (training of promotores de salud) via before-and-after interviews and surveys of the promotores. Of 27 promotores enrolled, 22 (82%) completed at least 2 of 3 trainings. Compared to pretraining, after training the promotores were more likely to correctly define cancer, identify biopsies, describe cancer stages, and report familiarity with cancer research studies. They were not more likely to report interest in participating in cancer research.
DeCamp 2015 USA Cohort	2011–2012	Latino Family Advisory Board; families recruited based on recommendations of providers and staff	•	•			Structured observations of meetings, member “check-ins,” group reflection, and multiple interviews of each participant. Clinic changes attributable to board included improvements to the waiting room and enhanced educational materials. Family board members reported satisfaction and a sense of achievement.
Edgren 2005 USA Cohort		Steering Committee composed of community members, representatives from community-based organizations, and local health systems; recruitment not described				•	Minutes from meetings, documentation records, interviews with Steering Committee members. Steering Committee effectively piloted comparative study examining relative effectiveness of incentives for research participants. Steering Committee effectively hired and trained community members as interviewers to collect qualitative data. 78% of research participants remained with the project at first-year follow-up data collection.
Fouad 2014 USA RCT	2013	Community Health Advisors; recruited from churches, civic organizations, and through referrals made by community leaders				•	Participants enrolled in a multicenter clinical trial were randomized to receive community advisor support or not; adherence rates for clinical research visits were measured. Adherence rates for scheduled clinical visits significantly higher in those working with community advisors (80%) compared to the control group (65%).
Fraenkel 2016 USA Cohort	2014	Patient voting panel; recruited from an online arthritis support community	•		•		Recommendations for clinical practice guidelines were compared between a group that contained a patient voting panel and 1 that did not. For 13 of 16 recommendations, the patient panel recommended the same course of action as did the physician-dominated panel. Differences were due to how the two panels valued the balance between harms and benefits.
Houweling 2013, Tripathy 2010 India RCT	2005–2008	Women’s groups; open to all who could attend in tribal communities and facilitated by local women	•				Neonatal mortality rate and maternal depression. Neonatal mortality rate was significantly lower in communities randomized to establish women’s groups; more socioeconomically marginalized communities demonstrated the greatest effect size. Maternal depression was also decreased but not to a significant effect.
Nutt 1976 USA Cohort	1975–1976	Health planning groups; recruited members of community organizations’ boards or by home care agency directors			•		Retrospective review of groups’ plans, analyzed and assigned quality score Expert (provider) planning groups produced plans of significantly higher quality than patient groups. Patient groups focusing on the topic of home care produced plans of significantly higher quality than those focusing on primary health care
Wyatt 2008 UK case study*	2004–2005	Research teams in a primary care study program; recruitment not described				•	Case study via retrospective review of records, interviews, focus groups, surveys, and participant-observation of research project processes All projects that included patients met all aims and objectives. Patients were perceived to have impacted study design, recruitment, data collection, analysis, and dissemination of findings

RCT randomized controlled trial (including cluster randomized trials)

*This study was included because it involved data collection at multiple time periods during a project, including pre- and post-evaluations, and was therefore assessed for quality using the Newcastle-Ottawa Scale for cohort studies

Table 3.

Health Systems Studied, Measures of Participation, and Key Findings from Cross-sectional Studies (n = 3)

Author, year of publication, country	Study year(s)	Health systems studied	Measure of public participation used	Key findings
Checkoway 1984 USA	1983	United States Health Systems Agencies (HSAs)	Community participation index (CPI)	Factors associated with higher CPI included: Strong patient leadership Provider dominance of policy and planning Patient board member deference to providers Executive director commitment Patient board member knowledge of issues and planning Staff training and experience in involving the public
Groene 2014 7 European countries	2011–2012	Hospitals	Score on the involvement of patients and their representatives in quality management functions	Involving patients and their representatives in quality management functions: Is more likely at the hospital than departmental levels More frequently involves implementation of quality improvement projects than the design of such projects Is not statistically associated with the implementation of patient-centered care strategies
Hiller 1997 USA	1995	State newborn screening programs	Questionnaire addressing the screening tests performed, level of public participation in newborn screening policy and practice, and involvement and rights of parents in screening programs	Advisory committees pertaining to newborn screening programs: Were established in 36 US states, and 26 of these had patient representation The public was consulted: In 33 states for consideration of additions to the newborn screening panel In 4 states on issue of laboratory methods and quality control

Risk of bias among randomized trials was moderate to high (Table 4) and observational studies were of poor or fair quality (Table 5). For randomized trials, the most common sources of biases were that participants and study personnel were not blinded nor was outcome assessment blinded (four of five studies). For observational trials, the most common reasons for low-quality assessment were a low-quality selection of the non-exposed cohort (10 of 11 studies) and limited demonstration that the outcome of interest was not present at the start of the study (11 of 11 studies).

Table 4.

Risk of Bias for Randomized Studies (Cochrane Risk of Bias Tool)

Study	Random sequence generation	Allocation concealment	Blinding of participants and personnel	Blinding of outcome assessment	Incomplete outcome data	Selective reporting	Other sources of bias
Abelson54	Low risk	High risk	High risk	Low risk	Low risk	Low risk	High risk
Boivin46, 47	Low risk	Low risk	Low risk	High risk	Low risk	Low risk	High risk
Carlson53	Low risk	Unclear risk	High risk	High risk	Low risk	Low risk	High risk
Fouad50	Low risk	High risk	High risk	High risk	Low risk	Low risk	Low risk
Houweling44 and Tripathy45	Low risk	High risk	High risk	High risk	Low risk	Low risk	Low risk

Table 5.

Quality Assessment for Observational Studies (Newcastle-Ottawa Scale)

Study	Representative exposed cohort	Selection of the non- exposed cohort	Ascertainment of exposure	Demonstration that outcome of interest was not present at start of study	Comparability of cohorts on the basis of the design or analysis	Assessment of outcome	Follow-up long enough for outcomes to occur	Adequacy of follow-up of cohorts
Chadiha52	*	*	*		*		*	*
Checkoway61	*					*	*
Crowley56			*		*	*	*	*
Cupertino57			*		*		*	*
DeCamp58			*		*		*	*
Edgren55	*		*		*		*	*
Fraenkel49	*		*		*		*	*
Groene60	*					*	*
Hiller59	*						*
Nutt48					*		*	*
Wyatt51					*		*

Organizational Makeup, Recruitment Strategies, and Levels of the Health Care System Addressed

Engagement took the form of groups of patients, family members, or community members only^{44, 50, 52, 56, 57}; or groups that also involved staff or clinicians^{53, 55, 58}; in four studies, these two strategies were compared.^{46–49, 51} Two studies engaged groups that speak non-dominant languages: one involved indigenous communities in India⁴⁴ and one involved Spanish-speaking immigrants in the USA.⁵⁸

PFACs consisted of collective engagement of patients or community members and in one case focused on engaging family members (parents of pediatric patients).⁵⁸ Recruitment of individuals tended to occur through nomination, such as asking for representatives from local organizations. No studies used democratic election for recruiting, and five studies did not report on recruitment strategies.^{51–53, 59, 60}

Engagement that Informs Direct Care Practices

Three studies evaluated interventions that inform direct care practices.^{44, 49, 58} These included two in which patient or parent groups developed their own care strategies or educational materials^{44, 58} and one in which patient groups made recommendations for the development of disease-specific practice guidelines.⁴⁹ These were considered effective strategies that enhanced the care experience⁵⁸ or equitably improved clinical outcomes by having greater effect in more socioeconomically marginalized communities.⁴⁴ The patient-derived clinical practice guidelines were similar to those derived by expert groups, with differences predominately due to patients having a different conceptualization of the balance between risks and benefits than physicians.⁴⁹

Engagement that Informs Organizational Design

Three studies evaluated interventions that inform organizational design of health systems.^{53, 56, 58} They targeted organizations that are disease-specific (diabetes centers)⁵³ or population-specific (pediatrics clinic for families with limited English proficiency)⁵⁸ as well as primary care clinics.⁵⁶ All studies used multiple forms of data collection, including interviews, surveys, and participant observations, to triangulate perceptions and project-based outcomes. When comparing strategies for engaging community members, the Community Action on Health committee in the UK found greater satisfaction among participants and greater effectiveness of deliberation when meetings were held in community settings and facilitated by existing community organizations than when they occurred on the grounds of clinics or health systems.⁵⁶

Engagement that Informs Policy-Making

Five studies (six articles) evaluated interventions that inform policy-making.^{46–49, 53, 54} Three studies in this group compared the recommendations made by organizations that contained patients with those that contained only “experts,” such as clinicians or organizational leadership.^46–49 Those that contained patients produced similar recommendations to those containing clinicians and organizational leadership only,⁴⁹ but the process of producing recommendations was costlier and achieving consensus took longer when patients were involved.^{46, 47} Also, when patients were involved, policy priorities and plans were deemed to be of lower quality when scored by content experts.⁴⁸ Two studies in this group compared mechanisms of engaging patients and community members, and found that in-person, collective deliberation is more effective than using mailed surveys or telephone calls,⁵⁴ and that patients with greater community credibility (such as leaders in community organizations) were more effective participants in PFACs than those without.^{46, 47}

Engagement that Informs Health-Related Research

Five studies evaluated interventions that inform health-related research.^{50–52, 55, 57} In four of these studies, patient groups assisted in the recruitment and retention of research participants among marginalized populations, and produced greater recruitment numbers and retention.^{50, 52, 55, 57} Only one study suggested that these individuals provided guidance on other dimensions of research besides participant recruitment and retention, such as study design or dissemination of findings, but the nature of involvement was not elaborated upon.⁵¹

Cross-sectional Surveys of Patient, Family, and Community Engagement Across Systems

Three cross-sectional studies examined engagement across systems (Table 3).^59–61 All three used different measurement tools for assessing levels of engagement, only one⁶⁰ of which tools had been validated in previous research.⁶² Higher levels of engagement were associated both with strong patient leadership and with patient deference to providers.⁶¹ Engagement more often occurred at the hospital level than at departmental levels, and more often focused on implementation of quality improvement projects than project design.⁶⁰

Feasibility of PFAC Member Engagement in the Review Process

Concerning our secondary aim of assessing the feasibility of engaging a PFAC member in the review process, the PFAC member contributed to the study’s search terms (as she had networked with PFACs at other institutions that use other terminology, such as “voting panel” or “community advisory board”) and to the interpretation of findings, particularly gaps in the studies they found to be relevant yet missing, such as descriptions of recruitment and training processes for PFAC members. The PFAC member did not have formal research experience and was unable to perform certain tasks required of the systematic review: identification of study design or comparison groups and assessments of bias or quality of studies. Interpreting the abstracted data in a realist synthesis framework—delineating context, mechanisms, and outcomes of individual studies—was feasible for the PFAC member and formed the basis our approach in analysis team meetings.

DISCUSSION

We identified 16 studies that addressed engaging patients, family members, and community members in various levels of the health care system, including health-related research. In-person deliberation with health system leadership was most effective, and studies that involved patient engagement in research focused on increasing study participation but not in research design or the dissemination of research findings. Programs varied in their structural makeup, the terminology employed to describe organizations and their components, strategies of recruitment of individuals, and types of outcomes measured. All studies suggested important benefits of community engagement to health systems and research, but some described drawbacks, including greater costs, longer time to achieve consensus among groups, and the generation of lower quality plans when patients are involved.^46–48

While there has been increasing interest in patient engagement^{1, 18, 38, 63, 64} and our search strategy captured 3718 documents, only 16 met our inclusion or study design criteria. Most were limited by study design. We excluded many studies that described programs but did not evaluate them with a control group, including a historical control, but instead narrated the formation of a project and summarized the lessons learned. The preponderance of articles structured this way represents a challenge in measuring the impact of patient engagement in projects that may not be well defined at the initiation of a partnership between patients and clinicians, staff, or researchers. Experts in community-engaged research have proposed various mechanisms for evaluating these sorts of partnerships, which highlight qualitative methods,³¹ “contextual” variables that describe historical context or the cohesion among community members,^{24, 65} or the realist synthesis framework we employed.^{33, 34, 39}

This review builds on the work of previous reviews^{11–13, 23–25, 66–68} by including studies across various levels of health care systems, focusing on the organizational makeup of PFACs and similar organizations, assessing standardization in measurement tools of patient engagement, and describing recruitment strategies. We found that studies that compared different mechanisms of patient and community engagement suggested three guiding principles. First, in-person and collective deliberation, particularly in community-based settings (such as community organizations), is more effective than using mailed surveys or telephone calls, or one-on-one meetings.^{54, 56} Second, patients with greater community credibility, such as leaders of community organizations, are more effective participants in PFACs than those without.^{46, 47} And third, while organizational or policy recommendations made by groups that included patients were similar to those made by experts,⁴⁹ this process takes more time and resources^{46, 47} and may produce plans of lower quality.⁴⁸

The heterogeneity of outcomes and recruitment processes described in our studies of interest, as well as the three guiding principles, suggest three opportunities for improvement in the conduct and evaluation of PFACs and related organizations. First, we identify a need for standardized measurement tools for patient, family, or community engagement, such as the patient-centeredness tool used by Groene, et al., the only tool we identified that is used in multiple studies and that is validated.^{60, 62} Second, the process of recruiting members to these organizations should be transparent and evaluated. Third, because community members who already had leadership experience tended to be more effective PFAC members, trainings should be available for those members without such experience to allow their voices to be raised effectively on PFACs. Specifically, those who experience health care inequities, including those from traditionally marginalized groups, may offer insights in PFACs that others do not.^{18, 58, 69} Because priorities and plans developed by patient-containing groups can be of poorer quality and may take more time, PFACs should evaluate and refine their processes of training new members, particularly those with little organizational leadership experience. Programs for training members of the public to effectively participate in other civic organizations and democratic processes, such as “Democracy Schools” to prepare citizens for participation in city governments, could serve as models.⁷⁰

Engaging those who are not trained in medicine or research, such as PFAC members, in systematic reviews is feasible and produces important dividends to project design (in the case of this project, search terminology) and to the realist synthesis of abstracted information (in this project, identifying gaps in the literature regarding recruitment and training of PFAC members). Manualized strategies for involvement of stakeholders such as PFAC members in systematic reviews are needed, particularly for the assessment of studies’ risk of bias or quality. This warrants further exploration as leaders in this methodology such as the Cochrane Collaboration have considered patient involvement as an integral organizational attribute since inception³⁵ and yet there are few examples of active patient involvement in the review process.⁷¹ An ongoing systematic review is aiming to further characterize optimal strategies for involving the public and other stakeholders in the conduct of systematic reviews.³⁰

Our systematic review has several limitations. Abstracted data from observational studies, which made up the bulk of our screened studies, may lead to conclusions that would differ from data obtained via randomized studies.⁷² The bias and low quality of the studies selected decrease the generalizability of the conclusions of each and heterogeneity of outcomes prohibits meta-analyses. The exclusion of many appropriate interventions due to weak study design might have excluded those interventions that engaged patients and community -members in ways that made the development of a comparison group difficult or impossible. Our search required an iterative process of identifying articles of interest, analyzing their content and indexing, and building a strategy constituted a process by which important studies may have not been included. Finally, we captured studies across four decades and several continents, and such variability in time and space complicates the ability of our findings to inform practice in any one health system.

Despite these limitations, we have identified three guiding principles and opportunities for improvement for the conduct of and evaluation of mechanisms of collective patient engagement. In performing this review, which engaged multiple stakeholders in local PFACs, we demonstrated that individuals with little to no research experience, but with important perspectives on the topic, can be successfully engaged in performing a systematic review on a complex intervention when using a realist synthesis to analyze abstracted data.^{36, 37} More data and metrics are needed to guide the development of PFACs, particularly focused on standardizing evaluation measures as well as evaluating training and recruitment strategies. Future systematic reviews on patient engagement or patient-centered outcomes should involve patients, family members, and/or community members in the analysis processes.

Funders

This work was supported by the United States Department of Veterans Affairs’ Office of Academic Affiliations through the National Clinician Scholars Program.

Compliance with ethical standards

Conflict of interest

The authors declare that they do not have a conflict of interest.