A Multi-Level Approach to Stakeholder Engagement in the formulation of a Clinical Data Research Network

Abstract

Objectives

To ensure meaningful engagement of stakeholders (patients, clinicians, and communities) in developing the Mid-South Clinical Data Research Network (MS-CDRN), we implemented a comprehensive, multi-level approach: 1) identify barriers to involving stakeholders in governance, network design, and implementation; 2) engage stakeholders in priority setting and research topic generation; 3) develop strategies to fully integrate stakeholders in CDRN governance and oversight; and 4) solicit guidance on patient-centered tools and strategies for recruiting research participants.

Methods

We engaged stakeholders 1) as integral research team members; 2) on oversight and advisory committees; 3) as consultants (using Community Engagement Studios); and 4) through interviews and surveys. We recruited stakeholders from community health centers, churches, barbershops, health fairs, a volunteer registry, and a patient portal. We prioritized recruitment from populations often underrepresented in research.

Results

During the first 18 months, we engaged 5,670 stakeholders in developing the MS-CDRN. These were research team members and on governance committees (N=10), consultants (N=58), survey respondents (N=5,543), and interviewees (N=59). Stakeholders identified important barriers and facilitators to engagement, developed stakeholder-informed policies, provided feedback on priority topics and research questions, and developed an intake process for data requests and interventional studies that included reviewing for appropriate patient-centeredness, patient engagement, and dissemination.

Discussion

Multi-level stakeholder engagement is a novel systematic approach to developing a meaningful patient-centered and patient-engaged research program. This approach allows ongoing input from highly engaged stakeholders while leveraging focused input from larger, more diverse groups to enhance the patient-centeredness of research and increase relevance to broader audiences.

INTRODUCTION

Engaging patients, consumers, and other community stakeholders in research has emerged as an important mechanism to accelerate the processes used to translate research into practice pragmatically. A 2013 Institute of Medicine report emphasized that stakeholders should be engaged in all phases of clinical and translational research, thus resulting in an increased number of funding announcements requiring stakeholder engagement.¹ Engaging stakeholders is expected to generate research that is more relevant, increasing public trust and confidence in research while enhancing public participation in research – a central challenge facing clinical research enterprises.

Involving patients and consumers in research is complex, and many investigators struggle to identify approaches for engaging non-researchers in both timely and meaningful ways.^2–4 Substantial progress has been made in engaging stakeholders in individual research studies and research infrastructure programs like the Clinical and Translational Science Awards.⁵ Yet, little is known about methods of involving patients and other stakeholders in large patient-centered research programs like the National Patient-Centered Clinical Research Network (PCORnet). The Mid-South Clinical Data Research Network (MS-CDRN),⁶ led by Vanderbilt University Medical Center, is one of 11 CDRNs funded by the Patient-Centered Outcomes Research Institute in Phase 1 of PCORnet and encompasses three large health systems with more than 20 million patients nationally. The stakeholder engagement methods required to create a patient-centered research program of this magnitude that is also geographically dispersed has not previously been described.

This paper presents our approach to implementing a comprehensive engagement plan for the MS-CDRN and provides a guiding framework for developing a stakeholder-engaged and patient-centered research network. We describe our methods of stakeholder engagement, the varied backgrounds and experiences of our stakeholders, and the stakeholder specific input they provided to enrich the formulation of the MS-CDRN.

METHODS

Our approach is guided by a multi-level stakeholder engagement model developed by our team (Figure 1).⁷ The model incorporates the following key concepts: 1) a number of approaches have successfully involved stakeholders in research, 2) the number of stakeholders engaged and extent of engagement should reflect the goals and aims of the research, 3) stakeholder roles span a continuum from providing brief, targeted input to highly involved, leadership roles and, 4) the training and experience required varies based on stakeholders’ roles. This Continuum of Community Stakeholder Engagement in Research Model was applied to convene the stakeholder members as described below.

Figure 1. Continuum of Community (Stakeholder) Engagement in Research.

A general model by which to build a framework for stakeholder engagement in health research and promote patient-centeredness.

Research Team: Our research team includes a patient investigator (NW) who participated in all research team activities. The patient investigator provides critical ongoing input to shape the structure and practices of the CDRN by identifying and prioritizing populations, selecting content areas for surveys, suggesting approaches for participant recruitment, providing input on the look and functionality of the website, developing policies and procedures for stakeholder engagement and proposal review and problem-solving challenges to implementation. The patient investigator was selected based on prior experience as a research advocate and community advisory board member for several research programs.

Oversight Committee: Two stakeholders representing a local community health center and the state department of public health are members of the Oversight Committee, which is comprised of key leaders who provide informed guidance for overall CDRN performance and offer critical input into problem areas. Advisory Council: The 8-member Stakeholder Advisory Council is comprised of patients with experience in advisory or leadership roles. Research experience was not a requirement for membership. The patients on the Council represent the health conditions related to the initial CDRN cohorts: Sickle Cell Disease, cardiovascular disease and obesity.

The Stakeholder Advisory Council met regularly throughout the 18-month start-up period and has provided guidance in the development of policies and operating processes. Orientation was provided to the members to introduce the purpose, goals, and mission of the network. Members received lists of definitions, commonly used acronyms, and rosters of the researchers involved. Community Engagement Studios (CE Studios)⁸ are a consultative method of engagement to facilitate project-specific input from a panel of health care consumers tailored to the investigator’s population of interest. Panelists share their knowledge and opinions based on their lived experience and provide feedback to the investigator in efforts to improve research processes and implementation. This approach, pioneered by our team, can be used at any stage of research. Six CE Studios were convened with 58 stakeholders to develop the stakeholder engagement plan and provide guidance to the three initial cohorts.

Survey Respondents and Interviewees: Our team, including the patient investigator, identified priority populations consisting of racial/ethnic minorities, individuals with multiple chronic conditions, low-income groups, rural and urban residents, and older adults. Our inclusion criteria were broad allowing anyone 18 years or older with capacity to consent to participate in surveys. We used multiple recruitment strategies including in-person engagement at community health centers, minority-owned barbershops, and community health fairs as well as online recruitment using the Vanderbilt patient portal and ResearchMatch⁹, a volunteer registry.

All surveys and interviews were approved by the Vanderbilt University Medical Center Institutional Review Board.

We surveyed 5,063 patients and health care consumers to understand attitudes towards research, barriers to participating in research, and research priorities. Approximately 70% of respondents were women, 23% racial/ethnic minorities and the mean age was 48.1 years (range 20–94 years; SD 17.3). Additional demographics are in Table 3.

Table 3.

Consumer Survey Demographics

	Total (n=5,063)
	N (%)
AGE (n=4,219)
18–39	1,325 (31.4)
40–59	1,720 (40.8)
60+	1,174 (27.8)
GENDER (n=4,522)
Male	1,378 (30.5)
Female	3,144 (69.5)
RACE (n =4,668)
Caucasian	3,552 (76.1)
Racial/Ethnic Minorities*	1,069 (22.9)
EDUCATION (n=4,498)
Some High School or less	186 (4.1)
High School Graduate/Some college	1,674 (37.2)
College Graduate	2,638 (58.6)
EMPLOYMENT (n= 4,244)
Full-time	2,288 (53.9)
Part-time	396 (9.3)
Unemployed	253 (6.0)
Non-salary/Unpaid	1,307 (30.8)
HOUSEHOLD INCOME (n=3,965)
<$25,000	755 (19.0)
$25,000–$49,999	898 (22.6)
$50,000–$74,999	770 (19.4)
$75,000–$99,000	571 (14.4)
>$100,000	971 (24.5)
HEALTH INSURANCE (n=4,291)
Insured	3,283 (76.5)
Medicare	655 (15.3)
Medicaid	177 (4.1)
Uninsured	176 (4.1)
HEALTH CONDITIONS (n=4,604)
Cardiovascular	1,993 (43.3)
Overweight/Obesity	1,478 (32.1)
Diabetes	608 (13.2)
Asthma	508 (11.0)

^*Includes Asian, Black, Hispanic, Middle Eastern/North African, American Indian/Alaskan Native, Native Hawaiian, and Other.

The tables do not compare their respondents to the population they were intended to represent, or note the implications of any differences? (e.g., 24.5% of those responding had a household income of >$100,000; only 4.1% were uninsured). Neither do they give us much information about what they said, and how (if at all) this varied by group.

We surveyed 480 health care providers including – primary care and specialty physicians, pharmacists, dentists, PT/OT/RT, psychologists, and licensed social workers) to assess attitudes and interests in participating in a research network and to identify barriers and incentives to participation. Seventy-two percent of health care providers surveyed were women and 29% were racial/ethnic minorities. Additional demographics are in table 4. Fifty-nine providers were also interviewed to obtain more detailed feedback.

Table 4.

Provider Survey Demographics

	Total (n=480)
	N (%)
GENDER (n=413)
Male	158 (38.3)
Female	255 (61.7)
RACE (n=401)
Caucasian	285 (71.1)
African American	80 (20.0)
Asian	25 (6.2)
Hispanic/Latino	11 (2.7)
DISCIPLINE (n=416)
Physicians	188 (45.2)
Nurses/Nurse Practitioners	105 (25.2)
Dentists	6 (1.4)
Physician Assistants	19 (4.6)
Pharmacists	7 (1.7)
Other*	95 (22.8)
PRACTICE (n=404)
Hospital-based	61 (15.1)
Solo	47 (11.6)
Single Specialty	98 (24.3)
Multi-Specialty Group	43 (10.6)
Academic Medical	63 (15.6)
Community Health Center	60 (14.9)
Other	32 (7.9)

^*Includes Nutritionist, Physical/Occupational/Respiratory Therapists, Psychologist, and Social Workers

To support co-learning opportunities for the stakeholder groups described above and enhance communication across levels of engagement (research team, Oversight Committee, Advisory Council, and overall leadership), we created an online shared workspace using Vanderbilt’s Research Organization Collaboration and Knowledge Exchange Toolkit (ROCKET).¹⁰ This continues to be an ideal platform for document sharing (leadership meeting presentations, Memoranda of Understanding (MOU), meeting minutes, templates, and planned activities), proposal review, patient-centered resources, glossary of commonly used PCOR/CEnR and MS-CDRN specific terms and acronyms, and comment boards for feedback and topic generation. This portal does not require Vanderbilt credentials to log in, allowing for easy access to our stakeholders.

RESULTS

We engaged 5,670 patients, clinicians and community members in developing the MS-CDRN using the multi-level stakeholder engagement model. The vast contributions of these stakeholder groups to enhance our approach to the MS-CDRN infrastructure are reported below.

The stakeholders on the Oversight Committee influenced key decisions for MS-CDRN including Phase 2 network expansion, building partnerships with other health care systems, researchers, and organizations, and activity prioritization. A detailed description is available in Table 1.

Table 1.

Description of Stakeholder Specific Input and Outcomes on Development of the MS-CDRN

Method	Method Description	Number	Compensation	Outcomes and Impact
Patient Investigator	Integral part of the research team	2	≥$50 overall	Shaped overall network design and implementation Memoranda of Understanding established Leadership at national levels Stakeholder review process Creation of web-based collaborative workspace
Governance	Oversight Committee - 2, Stakeholder Engagement Council (Advisory Council) – 8	10	$100–200Per meeting
CE Studios	Patients from VUMC, VHAN, Meharry, MWHC, Greenway PRIME network	58	$50	Feedback and revisions to recruitment and survey design, demographic modules
Clinician Interviews	Semi-structured	59	$100	Priorities for research Important factors for participating in research Barriers to Research participation
Clinician Surveys	Targeting areas less engaged with research	480	$25
Patient and Community Surveys	Web-based and in-person	5,063	$10	Facilitators/barriers to research participation Issues of privacy, trust, confidentiality Communication tools to collect Protected Health Information Attitudes towards research

Engagement activities occurred between March 2014–September 2015.

Objectives:

• –
  Understand barriers to patients’ and clinicians’ involvement in governance
• –
  Engage patients and clinicians in priority setting and research topic
• –
  Develop patient-centered strategies and policies to ensure patients and clinicians are involved in the governance;
• –
  Solicit patient and clinician input on the development and use of patient-centered tools;
• –
  Provide guidance regarding appropriate strategies for recruiting patients in research

The Stakeholder Advisory Council provided input to the MS-CDRN leadership and investigators to help generate research questions, review research proposals, provide input to researchers on patient engagement and patient-centered outcomes, monitor progress, and help disseminate MS-CDRN information to the broader public. During the early stages, the Stakeholder Advisory Council helped develop policies and processes for integrating stakeholder input into the network, identifying priority populations, and overcoming barriers to engagement. The Advisory Council worked closely with the Research Team to develop a process to vet all data and research requests made to the MS-CDRN. This process includes a quarterly review of all queries, a review of observational study requests by two rotating committee members, and a full committee review of intervention study requests. The intervention study review includes a patient-centeredness plan template for researchers to submit to the committee and patient-centered review criteria.

The group helped develop and subsequently received training on ROCKET, a collaborative communication platform.

Each MS-CDRN cohort research team (Healthy Weight, coronary heart disease, and Sickle Cell) as well as our Stakeholder Engagement team requested stakeholder feedback using the CE Studio process. Six studios were conducted with feedback from 58 stakeholder experts. Feedback to investigators included revisions of recruitment letters to patients, changes to surveys (less technical/medical jargon, more patient-centered questions, more culturally relevant questions), enhanced/tailored recruitment strategies, and changes to protocols (see Table 2).

Table 2.

Selected examples of changes to structure of MS-CDRN^*

Changes to survey structure	Add Questions: trust & behaviors Remove Questions: appear similar, less words, more visuals
Increase patient-centeredness	Options added for assessing pain Add “Health impacted ability to make appointment” Address other factors that impact weight (i.e. transportation, income, access)
Reduce technical/medical jargon	“Proximal” for “closer to the body” “Idiopathic” for “unknown cause”
Increase cultural relevance of questions	Changed phrase “unmarried couple” to “Living with a Partner” Race and ethnicity questions broadened
Enhance recruitment strategies	Use patient portal and social media Limit survey to 15 minutes Reduce length of invitation letter
Changes in protocol	Change in payment options Methods to collect data Provide different incentives for teenagers

^*Data from 58 stakeholders participation in 6 Community Engagement Studios

The feedback provided by the 5,063 patients and health care consumers informed MS-CDRN policies and procedures related to trust, privacy, and confidentiality of patient data, overall interactions with medical research and researchers, and best methods for communication of health information to specific populations. All of the survey items were optional, which was also a suggestion from the stakeholders on our research team and Advisory Council (see Table 3).

The 480 health care providers that participated in the survey research identified likely barriers to actively engaging in the CDRN including lack of time, potential negative impact on clinic flow, and additional paperwork. The health care providers’ top priorities for CDRN research were improving management of chronic conditions and preventative health services (See Table 4).

DISCUSSION

We successfully implemented our multi-level stakeholder engagement plan, resulting in substantial input from patients, community members, and health care providers with varied backgrounds and experiences. To our knowledge, stakeholder engagement during the development of a research program has not been previously implemented at this scale. Stakeholders have been engaged in all levels of governance, planning, and implementation of the CDRN including the research team, Stakeholder Advisory Council, and Oversight Committee.

More than 5,000 health care consumers gave targeted feedback to determine facilitators and barriers to participation in CDRN research. These surveys captured patients’ concerns surrounding issues of privacy, trust, and confidentiality of clinical data.⁶ This informed MS-CDRN policies and procedures including developing additional measures to protect privacy while fostering transparency in the work of the CDRN. Stakeholder input also helped shape communication between the MS-CDRN and the general public by raising the investigators’ awareness of cultural considerations when addressing or recruiting distinct patient groups and informing the overall interactions between the MS-CDRN and patients/community members. These engagement activities have fostered co-learning for researchers and patients and have strengthened communication and linkages among patients, community stakeholders, and the MS-CDRN.

During the 18-month Phase I period, we learned a series of effective implementation approaches that will strengthen relationships with the community and health care providers. We identified the following effective approaches: 1) engage stakeholders early in the planning process; 2) provide stakeholders with clear expectations (Memoranda of Understanding were ideal for our approach); 3) provide adequate preparation (orientation, training, resources) for both the stakeholders and the academic team members to ensure meaningful engagement; 4) prioritize effective communication with regular updates and provide explanation of acronyms and research/medical terminology; 5) use established networks such as stakeholder groups affiliated with CTSAs and Prevention Research Centers; 6) actively engage leaders of patient and health advocacy groups; and 7) leverage providers as trusted agents to facilitate patient engagement.

A comprehensive approach to engagement can be implemented promptly and broadly to elicit the preferences and needs of a broad range of stakeholders. This input can result in important changes to the research infrastructure and process to enhance its relevance and usefulness. Long-term metrics will need to be tracked to determine other impacts such as efficiency of the network and patient-centeredness of the research conducted.