PCOR, CER, and CBPR: Alphabet Soup or Complementary Fields of Health Research?

Abstract

Comparative effectiveness research (CER) and community‐ based participatory research (CBPR) are two fields of research that do not have a history of strong collaboration. However, CER and CBPR researchers could benefit from interdisciplinary collaboration to design and implement relevant, timely, action‐oriented research. This commentary explores field‐specific definitions of stakeholders and then outlines various roles stakeholders might play within grant‐funded research. Questions such as “What stakeholders should be involved?” and “How are stakeholders involved?” are addressed. The goal of this commentary is to highlight how the expertise and experiences of CBPR investigators can enhance the field of CER and to describe strategies for encouraging stakeholder involvement in CER research through the lens of CBPR. It is recommended that a team‐based approach to conducting stakeholder‐engaged CER encourages multiple stakeholders and “end users” to contribute their diverse expertise to the research process and contributes to the development of research with an increased likelihood of improving patient health and healthcare.

Introduction

Recent calls for comparative effectiveness research (CER) contain a specific requirement that stakeholders be involved effectively throughout the research process. CER investigators should look to the approaches and experiences of colleagues conducting community‐based participatory research (CBPR). While these two fields of research do not have a history of strong collaboration, CER and CBPR researchers could benefit from interdisciplinary collaboration to design and implement relevant, timely, action‐oriented research.

Background

The Patient Protection and Affordable Care Act of 2010 created the Patient‐Centered Outcomes Research Institute (PCORI) to promote clinical effectiveness research designed to provide information about the best available evidence to help patients and their providers make more informed healthcare decisions.1 According to PCORI's recent 2012 request for applications “the engagement of patients and stakeholders in the design and conduct of research is a key strategy for ensuring that the research question and the conduct of the research remain patient‐centered.” 2 The PCORI announcement requires that a plan for engaging patients and other key stakeholders be included as a necessary component of the applications. For example, applicants are asked to provide plans for how the research team will meaningfully engage and include relevant patients and other key stakeholders and how they will be compensated for their time and participation.

CER is focused on informing healthcare decisions by providing evidence on the effectiveness, benefits, and harms of different treatment options, and such evidence is primarily quantitative. Using sophisticated research methods and data analytic techniques, CER researchers conduct studies which compare drugs, medical devices, tests, surgeries, healthcare delivery methods, and behavioral and preventive interventions. The CER field has few examples of partnering with nontraditional researcher partners including stakeholders and community members. As noted by Selby et al., the PCORI Board of Governors sought to place patients at the “center of CER” to “ensure that the PCORI research agenda stays focused on practical questions, relevant outcomes and study populations.”1 In fact, PCORI is focused on an expansion of CER to patient‐centered outcomes research (PCOR). The goal of PCORI is to generate information that will answer four patient‐centered questions at the core of PCOR:2

1. “Given my personal characteristics, conditions and preferences, what should I expect will happen to me?”

2. “What are my options and what are the potential benefits and harms of those options?”

3. “What can I do to improve the outcomes that are most important to me?”

4. “How can clinicians and the care delivery systems they work in help me make the best decisions about my health and healthcare?”

As such, PCOR includes research not focused solely on comparisons of treatments, but also on broader aspects of healthcare delivery such as quality of care and dissemination of health information.

Concannon et al.’s recent assertion that no taxonomy or guidelines exist to help researchers interested in stakeholder‐ engaged CER is incorrect.3 The field of CBPR in fact has a long history of research based on partnerships with stakeholders. While types of questions and research purposes may differ, CER, PCOR, and CBPR all have a shared focus on effective engagement of stakeholders.

CBPR, by definition, is research that seeks to equitably involve community members, academic researchers, and others (such as healthcare providers) in all phases of the research process: all partners contribute expertise and share decision making and ownership of the project.4, 5, 6 The definition of “community” as adopted by many researchers engaged in CBPR is “those who have a shared unit of identity.” This definition is not limited to those residing in a geographic locale; rather community is an expansive and inclusive concept, and can include patients with shared health experiences (e.g., HIV positive women or children living with asthma). While often strongly associated with more qualitative research methods, CPBR is an approach to research, not a methodology, that can also be incorporated into quantitative research, such as those approaches often employed in CER.7

This commentary explores field specific definitions of stakeholders and then outlines various roles stakeholders might play within grant‐funded research. The goal of this commentary is to highlight how the expertise and experiences of CBPR investigators can enhance the field of CER and to describe strategies for encouraging stakeholder involvement in CER research through the lens of CBPR. The following tables and content for the paper were created as the result of a collaboration between CBPR and CER investigators within the Clinical and Translational Science Institute at the University of Pittsburgh (co‐authors on this paper) who worked together to develop training materials for academic researchers interested in responding to the PCORI calls for proposals. The resulting Stakeholder Engagement Toolkit provides researchers with information and techniques needed to identify and engage relevant stakeholders in their research. More information about the toolkit and the tables included below can be found at http://www.ctsi.pitt.edu/cercserv.shtml.

Key Ingredients for Stakeholder‐Engaged Research

Defining stakeholder: Who should be involved?

The process of identifying and engaging stakeholders in the research process can be daunting and overwhelming for researchers with little experience in stakeholder‐ or community‐engaged research. Such an approach to research is not consistent with the researcher‐driven approach that is most often taught in graduate school and utilized in traditional public health research.8 An important first step of the engagement process is to identify exactly who should have a voice and be included in the research—that is, for whom is this research relevant and useful? Unfortunately, defining stakeholders is not easily accomplished as definitions vary widely. A careful examination of stakeholder definitions used by key federal agencies funding health research shows that the term is generally used to refer to those who have interest in or who are impacted by the proposed research (Table 1). However, one common definition is not employed; the specific groups identified vary from patients, advocacy groups, community members, public health professionals, to even funding agencies themselves. The Agency for Healthcare Research and Quality (AHRQ) refers to all of these groups within their definition of stakeholder presented in The Effective Health Care Program Stakeholder Guide, “Persons or groups who have a vested interest in the clinical decision and the evidence that supports that decision. Each has a unique and valuable perspective. Stakeholders include: Patients, caregivers, patient advocacy organizations, clinicians, etc…”9 PCORI only specifically names patients, advocacy groups, caregivers, community groups, healthcare providers, professional associations and then notes that stakeholders can include “others who can bring insights on the patient perspective.” This wide range of groups included under the rubric of “stakeholders” does little to help clarify exactly who should be included in stakeholder‐engaged research. Instead, this diffuse definition suggests that almost any group can be considered a stakeholder as long as strong justification is provided regarding how and why they have an interest in the proposed research. These broad definitions of stakeholders are consistent with the definition of community used in CBPR, a field which stresses as a key principle the articulation of how “community” is defined and operationalized to determine who should be included in the research process.

Table 1.

Stakeholder definition and types by agency

	Patients	Caregivers	Providers	Community	Advocacy groups	Public health professionals	Academics/researchers	Professional associations	Policymakers/government	Funding agencies
AGENCY
Patient‐Centered Outcomes Research Institute (PCORI)	X	X	X	X	X			X
National Institutes of Health (NIH)	X			X	X		X
Agency for Healthcare Research and Quality (AHRQ)	X	X	X	X	X	X	X	X	X	X
Centers for Disease Control and Prevention (CDC)	X			X	X					X
Centers for Medicare and Medicaid Services (CMS)	X	X	X		X				X

Tables adopted from the Stakeholder Engagement Toolkit (2012); Comparative Effectiveness Research Core, University of Pittsburgh Clinical and Translational Sciences Institute (editor: Monica Costlow).

Determining levels of stakeholder‐engagement: How are stakeholders involved?

Another critical decision point in stakeholder‐engaged research is determining what role stakeholders will play in the proposed research. Table 2 outlines six possible levels of stakeholder involvement; the associated roles and responsibilities to the project are described and specific proposal requirements are presented. Stakeholder involvement can range from shared project leadership to general project support. Each level of stakeholder involvement has specific proposal and budget requirements when applying to places such as PCORI or the National Institutes of Health. For example, a shared Principal Investigator (PI) must have an established subcontract that contains specific information regarding salary, fringe, travel and indirect costs. If the PI is working on behalf of a community agency or organization, the budget justification should include an intention to obtain a Federal‐wide Assurance (FWA) as part of the subcontract. The FWA is a way for an institution, such as a community center, to say they agree to follow governmental research rules. An FWA ensures research participant safety and confidentiality. CBPR investigators with experience sharing project leadership with community partners should be viewed as a valuable resource for CER investigators exploring NIH community‐engaged and PCORI funding opportunities given their extensive experiences addressing proposal and budget requirements associated with stakeholder engagement in the research process.

Table 2.

Stakeholder/community involvement in research: Levels of engagement and associated application requirements

Level of involvement	Roles and responsibilities	Proposal requirements	Budget requirements
Shared‐Principal Investigator (PI)	• Assumes leadership role in project. • Shared authority and responsibility for leading and directing the project, intellectually and logistically. • Involved in all phases of the research process. • Has expertise essential to the project. • When multiple PIs are named, each is responsible and accountable to the applicant organization.	NIH biosketch – includes NIH eRA Commons ID and personal statement should reflect leadership role in the project. Not necessary, but recommended: Individual Investigator Agreement. If funded, this will be required by IRB.	Subcontract with specific budget line items for salary, fringe, travel costs, indirects, etc. If PI is working on behalf of a community agency, budget justification should include intention to obtain a FWA as part of the subcontract.
Co‐Investigator (co‐I)	• Works with the PI in the scientific development or execution of a project. • Has expertise essential to the project. • May be employed by, or be affiliated with, the applicant/grantee organization or another organization participating in the project under a consortium agreement. • Devotes a specified percentage of time to the project.	NIH biosketch – includes NIH eRA Commons ID and personal statement should reflect leadership role in the project. Not necessary, but recommended: Individual Investigator Agreement. If funded, this will be required by IRB.	Subcontract with specific budget line items for salary, fringe, travel costs, indirects, etc. If co‐I is working on behalf of a community agency, budget justification should include intention to obtain a FWA as part of the subcontract.
Individual Consultant	• Provides professional advice or services for a fee, but typically not as an employee of the engaging party. • Has specific expertise in an area essential for the project.	Letter of support indicating clear expectations regarding specific role for consulting, amount of time, and reimbursement. To prevent apparent conflicts of interest, written guidelines must be established indicating the conditions of payment of consulting fees. Research ethics training also required if the consultant will be engaged in research Not necessary, but recommended: NIH biosketch.	Specific budget line item included under consultants. Budget justification should include clear statement about what the consultant will be doing, how often and for how much. If this individual is working on behalf of a community agency, budget justification should include intention to obtain a FWA as part of the subcontract.
Advisory Board Member	Provides consultative and other expert advice to the project investigators as part of a group process. • Has expertise to provide feedback on specific aspects of the project. • May include assistance the following: ○ Development of surveys. ○ Suggestions regarding recruitment. ○ Interpretation of findings. ○ Dissemination activities.	Letter of support indicating purpose of the advisory board, how often they will meet, what kind of input they will provide, and their specific expertise that they bring to the board (most advisory boards do not participate in the data collection process, thus are not considered engaged in research).	Budget justification should include payment to Advisory Board Members as a specific research expense.
Nonprofit/Community Organization Support	• Supportive of project. • May be able to help with recruitment. • May be able to assist with interpretation of findings and dissemination.	Letter of support indicating history of the community organization and their role in the project (if assisting with recruitment, etc.)	In some instances, PIs may wish to formally include stipends for the community organization in the budget as a specific research expense.
Other Stakeholder Engagement: many ways in which to engage stakeholders depending on the project and research questions	• Other methods for engaging stakeholder include, but not limited to: ○ Identifying a group of patients through the research participant registry to provide specific individual feedback related to the project. ○ Focus group discussions with community members or healthcare providers. ○ Open community forum to discuss aspects of the project. ○ Interviews with key informants (could be community leaders, consumers, caregivers of patients, etc).	Other methods for engaging stakeholders should be included as part of the Research Strategy – specifics regarding recruitment strategies should be addressed.	Include in Budget Justification as a Research Expense.

Tables adopted from the Stakeholder Engagement Toolkit (2012); Comparative Effectiveness Research Core, University of Pittsburgh Clinical and Translational Sciences Institute (editor: Monica Costlow).

Conclusions

A team‐based approach to conducting stakeholder‐engaged CER encourages multiple stakeholders and “end users” to contribute their diverse expertise to the research process and contributes to the development of research with an increased likelihood of improving patient health and healthcare. CER investigators should invite CBPR researchers with experience conducting community‐engaged research to guide the identification and engagement of appropriate stakeholders. CBPR investigators and engaged stakeholders can participate in design of research with relevance for the intended populations and planning for involvement of stakeholders in not only the conduct of the research itself, but translation of research findings to practice and policy. This paradigm shift, driven in large part by the guidelines of the Patient Protection and Affordable Care Act and the establishment of PCORI, requires both CER and CBPR investigators to find common ground and ensure that patients and the communities in which they are embedded are the central focus of the research enterprise.