Collaboration and Context in the Design of Community-Engaged Research Training

Abstract

Collaboration between academic researchers and community members, clinicians, and organizations is valued at all levels of the program development process in community-engaged health research (CEnR). This descriptive study examined a convenience sample of 30 projects addressing training in CEnR methods and strategies within the Clinical and Translational Science Awards (CTSA) consortium. Projects were selected from among posters presented at an annual community engagement conference over a 3-year period. Study goals were to learn more about how community participation in the design process affected selection of training topics, how distinct community settings influenced the selection of training formats, and the role of evaluation in preparing training participants to pursue future health research programming. Results indicated (1) a modest increase in training topics that reflected community health priorities as a result of community (as well as academic) participation at the program design stage, (2) a wide range of community-based settings for CEnR training programs, and (3) the majority of respondents conducted evaluations, which led in turn to revisions in the curricula for future training sessions. Practice and research implications are that the collaboration displayed by academic community teams around CEnR training should be traced to see if this participatory practice transfers to the design of health promotion programs. Second, collaborative training design tenets, community formats and settings, and evaluation strategies should be disseminated throughout the CTSA network and beyond. Third, common evaluative metrics and indicators of success for CEnR training programs should be identified across CTSA institutions.

BACKGROUND

The challenge for the [CTSA] consortium is not only to build infrastructure that connects researchers to communities but equally to build community infrastructure to support the implementation and dissemination of research findings in order to best foster health within each community context.

(Eder, Carter-Edwards, Hurd, Rumala, & Wallerstein, 2013, p. 4)

While the role of training programs for biomedical and translational researchers in academic medical centers—from students to senior faculty—has received growing attention (McGaghie, 2010; Meyers, Begg, Fleming, & Merchant, 2012; Zerhouni, 2005), little has been published about community-engaged research (CEnR) training. In CEnR, the values of collaboration and participation characterize both the design and the outcomes of health research projects that bring together university-based academics, clinicians, and community members. CEnR has been an important component of the Clinical and Translational Science Award (CTSA) program since its inception in 2006 (Ahmed & Palermo, 2010; Leshner, Terry, Schultz, & Liverman, 2013; McGaghie, 2010; Michener et al., 2012; Woolf, 2008; Zerhouni, 2003, 2005). The role of CEnR in translational science is to engage multiple voices in the research that will inform all phases of health care promotion and delivery and ultimately improve societal health.

This descriptive study examines design decisions of CEnR academic and community partners as they build programs to prepare training participants to better understand and promote health research (Ahmed & Palermo, 2010; CTSA Consortium, 2011; Leshner et al., 2013; Michener et al., 2012; U.S. Department of Health and Human Services, National Institutes of Health, National Center for Advancing Translational Science, n.d.). The learning strategies employed in the design of programs that teach the collaborative, scientific, and ethical aspects of CEnR to academic and non-scientific communities build on the place-based educational possibilities inherent in community settings. Generally, training in academic medical settings relies on standard pedagogical approaches, situating education in the clinical environment and employing strategies such as lectures and slide presentations delivered by medical school faculty (Ramani, 2006; Vaughn & Baker, 2001). In contrast, learning rooted in the real-world contexts of individuals and communities is more likely to reflect participants’ relationships, experiences, and culture (Fenwick, 2003; Kolb, 1984) as they relate to health.

RECENT STUDIES ON CENR TRAINING PROGRAM DESIGN

While collaboration and community participation are highly valued in CEnR literature and practice (CTSA Consortium, 2011; Leshner et al., 2013; U.S. Department of Health and Human Services, National Institutes of Health, National Center for Advancing Translational Science, n.d.), the educational strategies best suited for combined community and university-based researcher training are only beginning to be examined (Boyer et al., 2018; Cunningham-Erves et al., 2018; Stewart, Spencer, Davis, Hart, & Boateng, 2018; Ziegahn et al., 2018). An earlier article by our team reported a descriptive study of CTSA CEnR training programs representing 30 CTSA academic medical institutions (Ziegahn et al., 2018). This article examined how translational workforce training projects incorporated the basic principles of collaboration and bidirectionality in their design and expected outcomes. The majority of the projects were designed to train both community members and academics to jointly conduct research and community health improvement activities. Results revealed research teams representing multiple academic disciplines, health care providers, and diverse communities. Training topics addressed research on prevention and treatment of specific diseases, knowledge and skills necessary for building CEnR infrastructure, ethical research conduct, and/or bidirectional communication skills for community members and academics. Perceived training outcomes included capacity-building skills to form and implement research partnerships, prepare CTSA institutions to better support and disseminate CEnR training curricula and teaching strategies, and strengthen institutional commitment to integrate community into translational science.

Other CTSA studies on CEnR training as a vehicle to make research concepts more available to communities describe the participation of community members, practitioners, community-based organizations, and academics as trainees and codesigners (Balls-Berry et al., 2017; Battaglia, Pamphile, Bak, Spencer, & Gunn, 2019; Boyer et al., 2018; Cunningham-Erves et al., 2018; Stewart et al., 2018). Codesigners sought community input on learning formats outside of the standard academic classroom-based lecture and slide presentation formats. Boyer et al. (2018) developed a continuum of stakeholder research engagement to prepare community members for different research roles (see Figure 1), ranging from more passive, short-term participation to full collaboration in leading research initiatives.

FIGURE 1. Continuum of Community (Stakeholder) Engagement in Research.

NOTE: This is a general model by which to build a framework for stakeholder engagement in health research and promote patient-centeredness. Co-PI = co–principal investigator; PI = principal investigator. From Boyer et al. (2018). Reprinted with permission.

Full community participation is necessary to move beyond the often superficial collaboration and participation in the design and implementation of programs aimed at eliminating disparities (Shaw-Ridley & Ridley, 2010). For example, online surveys of preferred formats and topics for training design led to development of separate curricular tracks for individuals desiring research advisory roles and to community-based organizations interested in either conducting their own research or partnering with health researchers to study community-identified health issues (Cunningham-Erves et al., 2018).

Similarly, while evaluation is generally present in some form in health research education efforts, the extent to which CEnR training programs assess collaborative aspects of the design process or analyze program results across participating CTSA institutions is unclear. Academic medical centers are currently searching for measures that will allow them to evaluate the effectiveness and return on investment of CEnR (Szilagyi et al., 2014). Toward that end, collaborative health research conceptual models have been proposed that allow for measurement of variables related to structure and context, process and relational dynamics, interventions, outcomes, and impact (Jagosh et al., 2012; Minkler & Wallerstein, 2008; Sandoval et al., 2012; Szilagyi et al., 2014). The community engagement (CE) logic model, which looks at metrics for measurement of CEnR effectiveness and impact within the CTSA translational science domain, requires that we differentiate inputs from short, intermediate, and long-term results (Eder et al., 2013). The model recognizes different training inputs for both community members and academics learning about CE. Combined trainings would “ultimately bring together the distinct yet complementary knowledge bases of scientists and community members” (Ziegahn et al., 2018, p. 2), increasing, in the short term, the capacity among CTSAs to conduct CEnR.

PURPOSE

Our goals for this follow-up study of CEnR training programs were to (1) look at how community participation in the design process influenced selection of training topics, (2) explore more deeply the dynamics between design and delivery formats (e.g., didactic vs. interactive, in-person vs. webinars, community based vs. academic campus based) and how they reinforce participation and collaboration, and (3) examine how evaluation processes and results lead to indicators of best practices for design and implementation of CEnR training.

METHOD

Between 2012 and 2014, CEnR researchers from CTSA institutions convened annually in Bethesda, Maryland, to share best practices and discuss the science of CEnR. Approximately 200 participants representing 62 academic medical centers and their community partners attended each meeting. From digital meeting archives, we drew a convenience or opportunity sample of posters highlighting CEnR education or training from the 175 posters presented. Posters that described deliberate attempts to educate researchers and/or communities about CEnR principles, goals, methods, or activities were included; posters where an educational program was only an incidental rather than primary focus were excluded. Forty-four percent (77/175) of posters qualified for study inclusion, and 36 of the 77 lead authors (47%) agreed to participate. Forty-one out of 77 nonparticipant authors did not respond to repeated e-mail or phone invitations to participate. Six projects initially deemed appropriate for study inclusion based on references to CE, research, or training in poster titles were excluded when lead contacts indicated an educational role unrelated to CEnR. Thus, the final sample was 30 posters/projects, sponsored by 24 CTSA institutions.

Over the 3-year study time span, a core group of seven staff representing seven different CTSA institutions determined study goals and criteria for poster inclusion, developed an interview guide and procedures for contacting poster authors, conducted interviews, and analyzed data. Institutional review board approval was not required because interview respondents provided data on projects rather than on themselves as human subjects (U.S. Department of Health and Human Services, Office for Human Research Protections, 2016).

The study team designed a 17-item semistructured telephone survey about CEnR training focus, audiences, and goals; educational formats and evaluation measures; and desired outcomes. The survey included forced-choice and open-ended questions targeting five major categories—the translational science spectrum, audiences, educational formats, evaluation, and outcomes. Team members accessed the interview guide and survey instrument through Vanderbilt University’s REDCap (Research Electronic Data Capture) database.

Study team interviewers were five of the study authors experienced in open-ended interviewing. The lead author led a 1-week practice process whereby interviewers gained common understanding of the questions and experience with qualitative probing techniques by interviewing CE colleagues at their respective CTSA institutions. Once the team was assured that variation in question posing had been minimized and questions were understood as intended, study interviews began. Interviewers then met through biweekly phone conferences throughout the data collection phase to discuss preliminary findings and emerging themes.

Telephone interview data were either typed directly into REDCap or transferred from interviewers’ notes. Data were then transferred to an Excel spreadsheet, and descriptive statistics were calculated using SPSS 21 software. For open-ended responses, a subsequent spreadsheet contained descriptive information in five categories derived from primary interview categories: translational science, audience, educational format, evaluation, and outcomes. Three members of the team evaluated data assigned to these categories for consistency, consolidation, and clarity. Smaller teams of three and four then analyzed data for themes—the attaching of codes derived from theory to a set of texts, followed by analysis of the code distribution (Bernard & Ryan, 2010). For example, seven subcategories emerged under outcomes (capacity building, barriers, systems change, sustainability, dissemination, tools and resources, and preventive health topics). All the themes were cross-validated by two team members to ensure congruence.

RESULTS

Training Topics

CEnR education and training topics exemplified the practice and policy side of the translational science spectrum. Most topics fell into the Type 4 (Scaling and Policy Reform) and Type 5 (Globalization and Public Opinion) stages of dissemination and CE, public health, and disease prevention (Graham et al., 2016). Table 1 displays the training topics identified by design teams both with and without community partner involvement.

TABLE 1.

Participants and Topics in CEnR Program Development

Community Partners Not Involved in Development (12 Programs)	Community Partners Involved in Development (20 Programs)
CEnR training for academic researchers Research skills training for community Build academic–community partnerships Increase minority participation in clinical trials Improve health communication Train community health providers	CEnR training for academic researchers Research skills training for community Build academic–community partnerships Increase minority participation in clinical trials Improve health communication Train community health providers Engage community members in research advisory roles Raise community awareness about health Address health disparities Develop community-based intervention

NOTE: CEnR = community-engaged research.

The training programs that were developed jointly by academic and community partners encompassed all 10 issues. However, programs that were developed exclusively by academicians did not include topics addressing community member involvement in research advisory roles, raising community awareness about health, addressing health disparities, and developing a community-based intervention. These additional topics are arguably more community specific than broader topics aimed at introducing researchers from both academe and communities to basic CEnR concepts and skills.

Training Formats and Settings

Responses about project format from the 30 respondents were grouped into four primary categories used commonly in adult education and training settings: small groups, community-based learning, online learning, and large groups (Caffarella & Daffron, 2013). Projects frequently combined modalities depending on project goals, audience characteristics, cost, and social, cultural, and organizational context.

Small Groups.

There were three distinct modalities:

• Workshops (16, 53%): Traditional leader-led classes focusing on CEnR were frequently supplemented with small peer-led workshops, face-to-face meetings to discuss CEnR-related modules, or health-related plays followed by small community workshops.

• One-on-one settings (3, 10%): Individualized coaching or advice on CEnR was often embedded in larger workshops or classes. As examples, facilitators provided feedback on draft proposals after grant-writing workshops or counseled high school students on collaborative science projects designed by students, families, and teachers.

• Other small-group formats (3, 10%): These included community research seminars for new board members and field trips to study sites.

Community-Based Learning.

There were 13 projects (43%) within this format. Science cafes and other projects met in libraries, civic clubs such as Rotary, disease support groups, and community health centers. Churches were cited as important settings for CEnR training and education, particularly in African American and Latino communities. Examples include (1) “train-the-trainer” programs for community health advisors coaching fellow church members, (2) church-based education on research protection and ethics in clinical trials, and (3) promotora training.

In a project addressing social determinants of health, a cultural competence committee and community center conducted a mapping and assessment exercise via a walking tour of neighborhoods and then presented results to the community. A field-based project initiated in response to the 2010 Deepwater Horizon Gulf oil spill trained Gulf fisherfolk as “citizen scientists” who, along with academic researchers and other stakeholders, conducted water testing. In addition, this project aimed to increase science literacy, build trust and participation in environmental health and science communities, and strengthen CEnR capacity.

Online Learning.

Twelve (40%) projects used distance learning formats for community and researcher training, particularly webinars (8/12). For community members, webinar topics included colon cancer prevention, how to conduct a literature review for proposals, CEnR teaching methodologies, and research participant protection issues. Webinars for researchers focused on health literacy and community-based participatory research. Other distance technologies included online instructional platforms to train wellness coaches, photovoice for children and parents in a program for asthma reduction, and a YouTube video on evaluation methods for promotoras.

Large Groups.

The single project in this category used a health conference to disseminate information on reducing cardiometabolic risk and increasing clinical trial recruitment through community-based participatory research practices initially presented in workshops and one-on-one meetings.

Evaluation

Twenty-five of 30 projects conducted an evaluation (see Table 2). Evaluation strategies were formative in nature, meaning they were implemented to inform future CEnR training design efforts.

TABLE 2.

Community-Engaged Research Projects Conducting Evaluations and Utilizing Results

Evaluation Strategy	Number	Percentage
Conducted project evaluation	25	83 (n = 30)
Satisfaction surveys	14	56 (n = 25)a
Pre–post tests	10	40 (n = 25)
Multiple-choice assessments	7	28 (n = 25)
Other	11	44 (n = 25)

^aBased on a subpopulation of the 25 projects that conducted an evaluation.

Most of the methods employed, either singly or in combination, are used frequently in program evaluation. Less common evaluation strategies (Other; 11) included preparation of a product (e.g., a peer-reviewed article), receiving a grant from a CTSA institution for evaluation purposes, and monitoring and/or proxy measures from related CE activities such as returned test kits, graded papers, and qualitative methods. Participant learning was evaluated independently via pre–post tests in almost half of the projects.

A majority (86%) of respondents also reported changes to their training programs based on participant feedback, most frequently to the content and curricula (see Table 3).

TABLE 3.

Outcomes of Community-Engaged Research Training Evaluations

Change	Number	Percentage
Made changes to program	25	86 (n = 29)a
Changed course content	9	36 (n = 25)b
Changed course format	8	32 (n = 25)
Modified larger community initiatives	8	32 (n = 25)
Broadened audience, updated materials	6	24 (n = 25)
Interested in sustaining evaluation efforts	3	12 (n = 25)

^aOne of the 30 respondents did not answer this question.

^bBased on a subpopulation of the 25 projects that conducted an evaluation.

Of the reported changes to training postevaluation, nine were related to content and curricula that addressed participant needs and feedback. Four (4/9) gave specific examples of alterations: changing case examples in training to include individuals of importance to local communities and adopting healthy recipes suggested by community members. Eight respondents reported one or more changes to course format: adapting the pace and delivery of instruction to the audience’s level of training and experience (5/8), and changing training format, adjusting location, and duration (3/8). In two cases (2/8), training formats evolved from a structured didactic approach to a more interactive and hands-on one, based on community preferences.

A third of respondents (8/25) also reported that feedback resulted in modifications to larger community initiatives. In half of these cases, project organizers updated materials such as assessment tools and consent forms to address comprehension and cultural relevance issues. Less frequent changes included adjustments to grant submissions (2/8), generation of a new research topic (1/8), and creation of a new research collaboration (1/8). In six cases, evaluation resulted in either broadening the intended audience (3/6) or creating or updating course materials (3/6). Only three projects expressed interest in sustaining evaluation efforts over time. Two of these mentioned evaluations planned to prepare for another CEnR training proposal, and one cited lack of funding as curtailing further evaluation plans.

DISCUSSION AND LIMITATIONS

Three structural components informed by values of collaboration and bidirectionality supported the CEnR training projects in this study. First, the data suggest that design teams that involved both university and community-based researchers resulted in greater sensitivity to community and societal research roles and interests. Similarly, the selection of formats that made use of existing community organizations and settings allowed for increased learning about community health priorities for both trainers and participants. Finally, most projects reported evaluation measures that informed further CEnR training efforts as well as related community health initiatives. These components are reflected in Principle 7 of the Principles of Community Engagement (CTSA Consortium, 2011), which states that CEnR can only be sustained by identifying community assets and developing community and academic capacity and resources to make decisions and take action.

The CTSA logic model, developed to guide CE efforts, is reflected in the findings of this study at the level of CE actions and foundations for success (Eder et al., 2013). Infrastructural inputs believed to be crucial to establish CE programs include structures to support CE, education of both community members and academics around CE, dissemination of research findings, and resources for seed grants and CEnR infrastructure development. Short-term markers of success included community–university trust, reduced barriers to communication and collaboration, community research capacity, stronger relationships with community health programs, and novel methods.

In our study, the formation of diverse training design teams reflected community-specific situations that informed training topics and selection of relevant local settings. This suggests that an early focus on collaboration can increase the likelihood that short- and intermediate-term results related to trust, collaboration, relationship building, and the structural capacity to conduct CEnR will hasten the institutionalization of collaborative training norms and outcomes.

Although the numbers on involvement in CEnR code-sign efforts were small, the participation of community partners on most research design teams suggests that the codesigner role increases the extent of engagement in health research over time (Boyer et al., 2018). Programs designed by both community members and academics provided more opportunities to apply newly learned skills in research informed by community perspectives, raise community awareness, address relevant health disparities, and develop community-based interventions. Such ongoing involvement, along with earlier findings revealing that a majority (77%) of CEnR projects were designed to train both community members and health researchers so that trainings could be jointly conducted (Ziegahn et al., 2018), raises the hope that (1) CEnR training will reveal authentic community interests and needs (Shaw-Ridley & Ridley, 2010) and that (2) both health research education and outcomes can be sustained (Boyer et al., 2018).

The creativity displayed in choosing learning formats and settings illustrates sensitivity to diversity in audiences, health topics, and cultural and other contextual considerations (Ziegahn et al., 2018). While CEnR training within CTSA institutions may indeed be somewhat informal, as suggested by the Institute of Medicine report (Leshner et al., 2013), this informality may be better characterized as flexibility, critical to successful CE. The mixture of classroom-based formats with complementary community- and field-based settings, along with distance learning, illustrated the innovation in experiential learning, team science, and CE and dissemination called for by the Institute of Medicine (Leshner et al., 2013) and experiential learning theorists (Fenwick, 2003; Kolb, 1984).

Most CEnR programs evaluated their training programs, taking advantage of data from related CE efforts and utilizing feedback to modify curriculum or training design based on audience needs and preferences. This supports Fenwick’s (2003) position on the evaluation phase of training in more experiential and participative educational models, which allow a community to “refine(s) its practices, develop(s) new ones, or discard(s) and change(s) practices that are harmful or dysfunctional” (p. 27). These applications of data correspond to the largest category of perceived outcomes—capacity building (Ziegahn et al., 2018)—and reflect the structural or context domains cited in research on collaborative research (Jagosh et al., 2012; Minkler & Wallerstein, 2008; Sandoval et al., 2012; Szilagyi et al., 2014).

Limitations of this study included the exploratory nature of our project, which yielded a variety of training topics, audiences, and formats, making comparisons difficult. In addition, we did not conduct a comprehensive inventory of all CTSA CEnR training projects; our sample was limited to those poster projects presented at national CTSA meetings whose authors agreed to participate. A further limitation was the multifaceted nature of CEnR partnership training, shared by academic medical and nonmedical institutions as well as nonprofit organizations. Finally, we did not collect data on the degree of community involvement in designing evaluation strategies.

IMPLICATIONS FOR PRACTICE AND RESEARCH

While we cannot state that including community partners in design and implementation led to training projects and outcomes that are more sensitive to community health concerns or better health promotion practices, we do see trends that merit further investigation.

• Collaborative academic–community teams who work together at the research phase may be more likely to continue partnerships at the level of practice as health promotion and treatment options are designed and implemented. It would be useful to collect data on the degree to which CEnR training participants continue working jointly when they have opportunities to participate in the development and promotion of clinical-based interventions.

• CEnR training design tenets, successes, and challenges, from program inception to evaluation, should be disseminated to curriculum designers inside and outside of the CTSA domain. These opportunities and sources could be made available to community-engaged researchers open to sharing and learning from one another.

• Novel metrics, identified in the 2013 CTSA logic model of CE and by other scholars of CEnR evaluation and metrics, form the critical foundations of bidirectional partnerships to enhance community research capacity and infrastructure, at least in theory (Eder et al., 2013; Jagosh et al., 2012; Minkler & Wallerstein, 2008; Sandoval et al., 2012; Szilagyi et al., 2014). However, willingness to utilize creative formats and methods depends partly on the larger institutional culture and the degree to which innovation is valued and accessible. CEnR program planners should continue to design common evaluative metrics, including indicators of success, for use across all phases of CEnR design, from identification of purpose and audience through project evaluation. Future research efforts might also trace the links between specific skills learned as a result of CEnR training and their community applications.