Characterizing the Use of Research-Community Partnerships in Studies of Evidence-Based Interventions in Children’s Community Services

Abstract

This study characterized the use of research community partnerships (RCPs) to tailor evidence-based intervention, training, and implementation models for delivery across different childhood problems and service contexts using a survey completed by project principal investigators and community partners. To build on previous RCP research and to explicate the tacit knowledge gained through collaborative efforts, the following were examined: (1) characteristics of studies using RCP models; (2) RCP functioning, processes, and products; (3) processes of tailoring evidence-based practices (EBPs) for community implementation ; and (4) perceptions of the benefits and challenges of collaborating with community providers and consumers. Results indicated that researchers were solely or jointly involved in the formation of almost all of the RCPs; interpersonal and operational processes were perceived as primary challenges; community partners’ roles included greater involvement in implementation and participant recruitment than more traditional research activities; and the partnership process was perceived to increase the relevance and “fit” of interventions and research.

There is considerable awareness of the gap between evidence-based practices (EBPs) and community-based services (Costello, He, Sampson, Kessler, & Merikangas, 2014; Garland Haine-Schlagel, Brookman-Frazee, Baker-Ericzen, Trask, & Fawley-King, 2013). The growing field of implementation science focuses on increasing studies and improving methods to promote the systematic uptake of research findings and EBPs into community service settings (Eccles & Mittman, 2006; Rabin & Brownson, 2012). There are a growing number of implementation frameworks and models (Damschroder, Aron, Keith, Kirsh, Alexander, & Lowery, 2009; Greenhalgh, Robert, Macfarlane, Bate, & Kyriakidou, 2004). A number of these models highlight the critical role of research-community partnerships (RCPs) to support the relevance and organizational “fit” of interventions to maximize uptake and to build organizational infrastructures to support intervention sustainability (Aarons, Hurlburt, & Horwitz, 2011; Greenhalgh et al., 2004; Mendel, Meredith, Schoenbaum, Sherbourne, & Wells, 2008; Olsen, McGinnis, Roundtable on Value & Science-Driven Health Care, & Institute of Medicine, 2010; Smith, Williams, Owen, Rubenstein, & Chaney, 2008). National policy directives from the National Institute of Mental Health, Centers for Disease Control and Prevention, and the Institute of Medicine call for improved collaboration between researchers and other community stakeholders to enhance the translation of research results into community-based care. Most recently, The Patient-Centered Outcomes Research Institute (PCORI) was authorized by the Patient Protection and Affordable Care Act of 2010. This institute was established to “help people make better-informed healthcare decisions and improve healthcare delivery and outcomes by producing and promoting high-integrity, evidence-based information that comes from research guided by patients, caregivers, and the broader health care community” (PCORI Methodology Committee, 2013). Given the increasing explicit attention to and funding for research conducted in collaboration with community stakeholders, research on the processes of collaboration are highly relevant and timely.

Evidence-based treatments will have the greatest public health impact when they are broadly used by community stakeholders (including providers and their clients); stakeholder use depends in large part on the “fit” of the treatment within the community-care context (Aarons et al., 2011). Researchers who are developing and testing treatment approaches can partner with community stakeholders (including providers and clients) to capitalize on complementary knowledge and expertise, aimed at maximizing the research rigor and contextual fit for targeted practices. Compared to traditional researcher-driven models of research-to-practice translation, partnered research has the potential to improve the utility of interventions developed/adapted, the success and efficiency of uptake, the sustainability of interventions in targeted service settings, and the ultimate clinical effectiveness with target populations.

Examples of Research-Community Partnerships

We use the term research-community partnership (RCP) to refer to formal collaborative relationships between researchers and community stakeholders regardless of the purpose of the collaborative activities and theoretical model used to define the partnership.

There are growing numbers of examples of RCPs in the field of mental health services, often influenced by theoretical models of partnerships (e.g., community-based participatory research (Israel, Schulz, Parker, & Becker, 1998); community-partnered participatory research (Jones & Wells, 2007)). To date, partnerships have been employed to: conduct practice-based research to characterize community-based mental health care (Garland, Plemmons, & Koontz, 2006), promote knowledge exchange between researchers, practitioners, and service organizations (Sullivan, Duan, Mukherjee, Kirchner, Perry, & Henderson, 2005), conduct research that is relevant to practice (McMillen, Lenze, Hawley, & Osborne, 2009), and improve the quality of community-based mental health care overall (Lindamer et al., 2008; Lindamer et al., 2009; Wells, Miranda, Bruce, Alegria, & Wallerstein, 2004).

In this paper we specifically focus on RCPs developed to select and adapt EBP clinical intervention, training, and broader implementation models for child service systems (Mental Health, Education, Early Intervention, Child Welfare). In addition to the RCPs mentioned above and used in community-based research broadly, there are also examples of RCPs specifically targeting implementation of evidence-based practices in community-based mental health services (Chorpita, & Mueller, 2008; Chorpita, et al., 2002; Southam-Gerow, Hourigan, & Allin, 2009; Wells et al., 2004). Community-partnered participatory research (CPPR) is an RCP model specifically designed to support efforts to implement evidence-based practices in community-based mental health services (Jones et al., 2007; Wells et al., 2013). This model was designed to support the use of EBP in ways that are consistent with community needs and values (Wells et al., 2004) and has been applied to examine the participatory process in collaboratively selecting and testing interventions (Bluthenthal et al., 2006). Taken together, the existing RCP research provides important information about factors that facilitate (or inhibit) collaboration (e.g., trust, shared goals), and anticipated outcomes of collaborative research (e.g., practice relevant research). It also highlights the developmental nature of partnerships (from initial formation to sustainable infrastructure). These studies represent important developments in the field in response to calls for improved partnership between researchers and community-stakeholders.

Our research group has recently focused on addressing the need to explicate the collaborative process in child mental health services by integrating the extant conceptual and empirical literature to develop a framework for RCPs in child mental health (Brookman-Frazee, Stahmer, Lewis, Feder, & Reed, 2012; Garland & Brookman-Frazee, 2013). Figure 1 outlines our RCP framework adapted for the current study based on the conceptual and theoretical literature as well as “lessons learned” from case studies (Brookman-Frazee et al., 2012). The framework illustrates the iterative and dynamic process of RCP development and the potential outcomes of these efforts. It highlights the multiple dynamic phases of RCPs and the collaborative processes that occur in the community context of the RCP. It posits that RCP functioning (including both interpersonal and operational functioning) can lead to partnership synergy (proximal outcome), which can then lead to a variety of potential distal outcomes, including benefits to the individuals involved in the RCP, organizations and communities.

Figure 1.

Model of research community partnerships. Adapted from Brookman-Frazee et al. (2012)

The purpose of the current study is to build on previous RCP case studies, theory, and research to make more explicit much of the tacit knowledge that many researchers have gained through collaborative efforts. Specifically, we surveyed Principal Investigators (PIs) and Community Partners who were directly involved in an array of systematically identified studies that used RCP approaches to develop, adapt or implement evidence-based practices addressing childhood developmental or mental health problems. We apply the Brookman-Frazee, Stahmer, and colleagues (2012) framework to: 1) characterize projects using an RCP approach and collaborative processes, and 2) identify themes regarding perceived challenges, benefits and lessons learned associated with research-community collaboration.

Methods

Procedures

Project and project contact identification process

A literature and grants search was conducted to identify potential projects eligible for inclusion. The following literature and grants databases were used: ERIC, PsycINFO, PubMed, Web of Knowledge, NIH Reporter, the Institute of Education Science grants database, the Department of Defense grants database, and the Autism Speaks grants database. Combinations of the following search terms were used to identify articles or grants published or funding initiated between 2002–2012: “youth/child,” “evidence-based practice/intervention," "CBPR/community-based participatory research/ community partnership," “mental health/development,” “community,” and “implementation.”

The initial search yielded 64 published studies or grant-funded projects. A consensus coding process was used to determine inclusion. First, three members of the research team independently reviewed the abstracts of each study or grant to determine initial eligibility based on the following eligibility criteria:

1. The project included implementation of a specific EBP(s) or an adapted EBP intervention targeting mental health or developmental problems in youth (ages 0–22 years).

2. The focus of the project was to adapt the clinical intervention, training model, or implementation strategy in a targeted community setting(s).

3. The intervention was delivered by community providers in a community service setting (school, mental health clinic, specialty developmental clinic/early intervention, primary care, child welfare).

4. The project involved collaboration (exchange of ideas, knowledge or information, shared tasks) between researchers and community stakeholders.

Using these criteria, each of the three reviewers made one of the following inclusion recommendations: meets criteria, does not meet criteria, unable to determine (i.e., more information is needed to make a determination). All three of the reviewers agreed on the initial eligibility determination: Include (n = 22), Exclude (n =13) and Unable to determine – More Information needed (n = 29). Additional information was then obtained for the 29 projects. This included reviewing published papers or publicly available online information regarding grants projects. The three reviewers then reviewed and discussed the additional information as a group and made a final inclusion decision. In total, 38 of the 64 projects met all four eligibility criteria and were included. Email addresses were subsequently identified for the study PI or director through publicly available sources (e.g., grants databases, publications, university profiles).

Survey administration process

The web-based survey was emailed to a total of 35 PIs who represented the 38 identified projects. Three PIs were involved in more than one identified project so they were asked to report on the project in which they were most actively involved. Eighteen of the 35 PIs responded about 18 unique projects. At the end of the survey, PIs were asked to identify community stakeholders who were involved in the project. Thirteen of the 18 PIs provided emails for a total of 29 community partners. The survey was ultimately emailed to a total of 26 community partners (three community partners were excluded due to an invalid email address). Ten of the 26 community partners responded about 7 projects. One of the community partners responded about a different project than the PI. Therefore, there were 6 projects with linked community partner and PI respondents. All survey respondents were offered a $10 electronic gift card. The Institutional Review Boards of the University of California, San Diego and Rady Children’s Hospital, San Diego approved survey content (described below) and distribution procedures. Figure 2 depicts the process of final sample selection.

Figure 2.

Participant sample selection.

Participants

The total sample included 28 survey respondents that comprised 18 of the 35 eligible PIs (51%) and 10 of the 26 eligible community partners (40%). Participants were 75% female and 86% White/Caucasian, per self-report. Table 1 reports the demographic information collected about participants. Note that there were no statistically significant differences between PIs and community partners on demographic variables.

Table 1.

Participant Demographics

	PI (n = 18)	CP (n = 10)	Total (n = 28)
Age: M (SD); Range	52.06 (9.22); 36–66	54.70 (8.11); 42–66	53 (8.78); 36–66
Gender (%)
Female	13 (72%)	8 (80%)	21 (75%)
Race/Ethnicity (%)
White	16 (89%)	8 (80%)	24 (86%)
Latino/Hispanic	1 (6%)	--	1 (4%)
African American	1 (6%)	1 (10%)	2 (7%)
Asian/Pacific Islander	1 (6%)	1 (10%)	2 (7%)

Measures

Web-based survey

An online software program was used to create and distribute the web-based survey consisting of 25 items (including both multiple choice and open-ended items) organized by three primary Sections:

• Section 1: Project Characteristics. Six items including: project aims, project funding, service systems (e.g., Mental Health, Child Welfare) involved in the project, age and primarily clinical problem of target population, name and primary target of the clinical intervention.

• Section 2: Collaborative Process. Fourteen items including: who initiated the research-community partnership, community stakeholders involved in the project, brief description of how research and community partners were identified to participate in the collaborative group, use of a theoretical model to structure, and implement, or evaluate the research-community partnership, methods are/ were used to communicate and execute project activities, level of involvement ratings of any community partner on 20 different project activities, role of community partners in identifying the clinical intervention, role of community partners in adapting the EBP, compensation for community partners for project participation, how partner funding was determined, continued community partner participation after completion of the grant/project/funding, percentage of community partners have continued participation with the project since the beginning, impact of staff turnover on ongoing participation, anticipated/ actual “products” of the research-community partnership.

• Section 3: Benefits and Challenges of Research-Community Collaboration. Three open-ended items including: top four challenges of research-community collaboration, top four benefits of research-community collaboration, list a key “lesson learned” from your experience with research-community collaboration.

Multiple choice items inquired about characteristics of the projects, RCP functioning, processes, and products, and processes of tailoring EBPs for implementation in community settings. Open-ended items inquired about perceptions of the benefits and challenges of research-community collaboration and lessons learned from the RCP experience. The survey took approximately 20 minutes to complete.

Data Analytic Plan

Descriptive statistics were performed for multiple choice items to describe participant demographics, characteristics of projects, and RCP functioning and processes. Open-ended items were examined using qualitative data analytic methods to describe participants’ perceptions of the benefits and challenges of using RCP approaches and lessons learned from their RCP experience. Since only a small number of projects had both PI and CP responses (6 of 18), PI responses were used to report characteristics of the projects and CP involvement (Survey Sections 1 and 2). We conducted secondary, exploratory analyses (independent sample t-tests) to examine difference in perceptions of involvement ratings. Further, exploratory analyses (chi-square and independent sample t-tests) were also conducted to compare certain project characteristics and CP involvement ratings for those projects with linked CP respondents (n = 6 projects) and those without linked CP respondents (n = 12 projects).

Qualitative data (i.e. brief responses to open-ended Section 3 questions) were analyzed using a coding, consensus, and comparison methodology (Willms, Best, Taylor, Gilbert, Wilson, Lindsay, & Singer, 1990), which followed an iterative approach rooted in grounded theory (Glaser & Strauss, 1967). Qualitative responses were brief, ranging between 1–2 sentences. These responses were assigned codes divided into two levels of analysis: 1) general (codes at the broadest level of themes), and 2) subthemes (codes at the second level of themes and subsumed within a general theme). Two members of the research team each coded all qualitative responses independently after the final coding list was established. Once the coding list was established, the coders and PI reviewed of all of the codes for each survey until members of the research team reached consensus as to which codes should be applied to specific segments of text. Quantitative and qualitative data were then integrated through triangulation and guided by the RCP framework of Brookman-Frazee and colleagues (2012) in order to examine convergence, expansion, and complementarity as suggested by Creswell and Plano Cark (2006).

Results

Results are organized according to the Brookman-Frazee, Stahmer et al. (2012) framework specifying the Formation Phase, collaborative processes, and proximal and distal outcomes. PI responses were used to characterize the quantitative data gathered for the Project Characteristics (survey Section 1) and Collaborative Process (survey Section 2) questions. Exploratory analyses comparing community partner and PI responses regarding collaborative process and differences between projects with and without linked community partner respondents are integrated in the respective sections. Themes that emerged from qualitative coding of both PI and community partner responses to questions related to the Collaborative Process (survey Section 2) and Benefits and Challenges of Research-Community Collaboration (survey Section 3) items are integrated to complement and expand the findings from PI responses to restricted response items.

Phase 1: Formation (Initiation) of RCPs

PIs’ responses describing characteristics of the projects provides information relevant to the formation of RCPs (e.g., purpose of initiating the RCP, members), are summarized in Table 2, outlining project characteristics. As noted, the RCPs were primarily initiated by researchers (50%) or jointly initiated (44%). Agency leaders/administrators and caregiver consumers represented the highest proportions (89% and 78% of projects, respectively) of stakeholder group. When asked to describe how initial partners were selected, most PIs indicated that partners were at least partially included based on existing relationships (e.g., “Existing relationships first, then leaders serving the population”). Others indicated that relevant stakeholders were selected based on interest or relevance (e.g., “Researchers and community leaders were identified based upon interest and expertise”). Exploratory analyses comparing characteristics of projects with and without community partner respondents revealed that a significantly higher proportion with linked community partner respondents targeted autism spectrum disorder as the primary clinical population (50% vs. 0%) and included clinicians as community stakeholders (100% vs. 17%).

Table 2.

Project Characteristics (based on PI report)

	With Linked CP Respondent (n=6)	Without Linked CP Respondent (n=12)	Total (n=18)
RCP Initiation
Researcher-initiated	33% (2)	58% (7)	50% (9)
Jointly Initiated	50% (3)	42% (5)	44% (8)
Community Stakeholder-initiated	17% (1)	8% (1)	11% (2)
Funder	0% (0)	8% (1)	6% (1)
Community Stakeholder Participants
Funding Agency/System Representatives	50% (3)	17% (2)	28% (5)
Provider Agency Leaders/Administrators	100% (6)	83% (10)	89% (16)
Clinicians	100% (6)	17% (2)	44% (8)
Caregiver Consumers	100% (6)	67% (8)	78% (14)
Youth Consumers	17% (1)	42% (5)	33% (6)
Other (Community Block Captains, Community	0% (0)	42% (5)	28% (5)
Teams, Head Start, Teachers, School District and Partnership)
Project Aim(s)
Examine intervention effectiveness	67% (4)	75% (9)	72% (13)
Examine implementation process/outcomes	83% (5)	67% (8)	72% (13)
Adapt clinical protocol/materials	67% (4)	50% (6)	56% (10)
Adapt training protocol/materials	50% (3)	33% (4)	39% (7)
Develop/test a partnership model	50% (3)	33% (4)	39% (7)
Examine collaborative process	33% (2)	25% (3)	28% (5)
Other	17% (1)	8% (1)	11% (2)
Funding
Federal research grant	100% (6)	83% (10)	89%(16)
Federal service contract	0% (0)	25% (3)	17% (3)
Local service contract	0% (0)	8% (1)	6% (1)
Project Service System
School/Special Education	50% (3)	42% (5)	44% (8)
Mental Health	50% (3)	42% (5)	44% (8)
Child Welfare	0% (0)	25% (3)	17% (3)
Early Intervention	17% (1)	8 % (1)	11% (2)
Primary Care	0% (0)	8% (1)	6% (1)
Other	0% (0)	25% (3)	17% (3)
Primary Clinical Problem
ASD	50% (3)	0% (0)	17% (3)
Disruptive Behavior Disorders	0% (0)	25% (3)	17% (3)
At risk	0% (0)	17% (2)	11% (2)
Substance Use	0% (0)	17% (2)	11% (2)
Trauma	17% (1)	8% (1)	11% (2)
ADHD	0% (0)	8% (1)	6% (1)
Obesity	17% (1)	0% (0)	6% (1)
Depression	0% (0)	8% (1)	6% (1)
Other	0% (0)	8% (1)	6% (1)
N/A	17% (1)	8% (1)	11% (2)
RCP Model
CPPR	67% (4)	25% (3)	39% (7)
CBPR	17% (1)	42% (5)	33% (6)
Other: PROSPER	0% (0)	8% (1)	6% (1)
No Model Used	17% (1)	25% (3)	22% (4)

Note: Characteristics are not mutually exclusive

The most commonly-reported aims of the 18 projects were to examine the effectiveness of a specified intervention (72%) and to examine the implementation process and outcomes of an intervention (72%), with a vast majority of projects receiving federal research grant funding (89%). Of those 16 projects funded by federal research grants, ten received funding from the National Institute of Mental Health (NIMH). Other projects funded by federal research grants received funding from the National Institute of Minority Health and Health Disparities (NIMHD, n = 2), Centers for Disease Control (CDC, n = 1), National Institute on Drug Abuse (NIDA, n = 1), the Substance Abuse and Mental Health Services Administration (SAMHSA, n = 1), and Health Resources and Services Administration (HRSA, n = 1).

Projects were conducted in a range of community service systems with the school/special education and mental health systems representing the majority of projects (44% each). A variety of youth clinical problems were the focus of interventions (e.g., autism spectrum disorders, disruptive behavior disorders, substance use).

Of those projects that endorsed utilizing an RCP theoretical model, the majority of PIs reported applying a CBPR or CPPR theoretical model (33% and 39%, respectively).

Research Community Partnership (RCP) Functioning: Managing Interpersonal Processes

Establishing relationships/trust

As mentioned above, initial partnerships were often initiated based on existing relationships or the leadership role of the individual in the community or research environment. Community partners and PIs indicated that relationships and interaction styles of members provided challenges associated with collaboration. For example, community partners reported that “not being heard,” “possibility of having practice methods criticized,” and “lack of engagement” were interpersonal challenges to the RCP. Although less salient for PIs, they commented, “Time for sufficient communication and relationship development,” and “building trust” were challenging. Multiple PIs commented on the importance of taking time to nurture the collaborative relationships. For example, one PI suggested, “Investment in the relationship is key, critical to all other aspects of the project. If you skimp in this area - problems will emerge in some other aspect of the project.” Additionally, PIs commented that language and knowledge differences between researchers and community stakeholders (e.g., “Different language/meaning of terms,” “Lack of foundational knowledge regarding research/ evaluation”) as challenging. Although this was not as salient a theme for community partners, one commented that “acronyms” were challenging.

Roles/responsibilities

When asked to rate the level of community partner involvement in variety of activities on a four-point likert scale (1 = Not at all involved; 4 = Very involved), PIs reported that, overall (based on a composite of 20 project activities) community partners were moderately involved in project activities (M = 2.7; SD = .75; range 1.5 to 4.0). As illustrated in Figure 2, the activities with the highest levels of community partner involved related to implementation planning (M = 3.56; SD = .78; range 1 to 4) and participant recruitment (M = 3.44; SD = .62; range 2 to 4). Not surprisingly, PIs reported that community partners were less involved in more traditional research activities such as manuscript writing (M = 2.06; SD = 1.03; range 1 to 4) and data analysis (M = 1.53; SD =. 62; range 1 to 4).

Exploratory analyses were conducted to examine difference in Community Partner Involvement ratings based on community partner and PI report for the subset of projects with linked community partner and PI ratings (n = 6 projects). Although not statistically significant, the average of Community Partner Involvement items based on PI report was slightly higher than community partners reported (PI: M = 3.03; SD = .68; CP: M = 2.72; SD = .79). Similarly, there were no statistically significant differences between community partner -rated and PI-rated community partner involvement on individual items. Additionally, there was generally agreement between community partners and PIs in activities with the highest and lowest involvement. Specifically, the following items were rated the highest for both community partners and PIs: Providing feedback on the intervention protocol, Providing feedback on the clinical training protocol, Participant recruitment, Implementation planning, Selecting an intervention; and Revising intervention materials. Likewise, both community partners and PIs rated the following activities with the lowest involvement ratings: Data analysis, Obtaining funding for intervention, Writing papers, and Preparing grant applications.

Exploratory analyses were also conducted to examine differences in PI-rated Community Partner Involvement ratings for those projects with (n=6) and without (n=12) linked community partner ratings. Although not statistically significant, PIs with a linked community partner respondent for their project had higher average ratings of community partner involvement than PIs without a linked community partner respondent (With Linked CP: M = 3.03; SD = .68; Without Linked CP: M = 2.50; SD = .71). Analyses of individual items revealed significantly higher ratings for two individual activities. Specifically, PIs with linked community partner respondents rated Feedback on intervention protocol (M = 4.00; SD = 0.00) higher than PIs without linked community partner respondents (M = 2.83; SD = 1.19). Similarly, PIs with linked community partner respondents rated Revising intervention Materials (M = 3.67; SD = 0.62) higher than PIs without linked community partner respondents (M = 2.33; SD = 1.37).

Comments from PIs revealed specific challenges related to roles and responsibilities regarding community partners. For example, PIs described “Task delegation to community investigators,” “Involvement of community stakeholders in data analysis,” and “Equal commitment of effort” as challenges to the collaborative process. One community partner also identified the “Requirement of ‘additional’ work” as a challenge.

Research Community Partnership (RCP) Functioning: Managing Operational Processes

Leadership and administrative support

The importance of leadership and resources was captured by the open ended comment from one PI that “Having a consistent leader and shared admin support is key, as is food at meetings!”

Structure and communication

PIs reported on structures used to facilitate communication and execute project activities. All PIs reported that telephone or in-person meetings were used for this purpose and many reported holding frequent meetings (56% weekly, 44% monthly). A few PIs added comments that the frequency of meetings varied over time. Further, the vast majority of PIs (94%) also reported that email was used to facilitate communication and execute project activities. Other methods included newsletters (22%) and web-based groups (6%). Open end-responses indicate that challenges regarding scheduling meetings (e.g., “Finding mutually convenient meeting times”) and finding time to develop and maintain collaborative relationships (e.g., “Time for sufficient communication and relationship development”) were salient for PIs. When asked about lessons learned, multiple PIs mentioned that collaboration takes time.

Phase 2: Execution of Activities

Proximal (Process) Outcomes

Partnership synergy and collaborative relationships

For both PIs and community partners, significant perceived benefits of the RCP were the partnership synergy and collaborative relationships that developed. For example, PIs reported that “establishing partnership towards shared goals,” “synergy in ideas,” and “enthusiasm, interest, and good ideas provided by community partners” were benefits of the RCP. Similarly, community partners reported that “[a] rich feedback loop between practitioners and researchers,” “building and participating in a supportive and reflective community,” and “collaboration toward common goals” were benefits of the RCP. Multiple PIs also commented on enjoying the collaborative process (e.g., “Stimulating and fun (never a dull moment)”).

Knowledge exchange

Both PI and community respondents also described the value of the knowledge exchange that occurred between partners. For example, PIs described “Learning about the demands of community settings,” “Improved awareness of challenges faced for achieving shared goals” and “Researchers learn about the context of community-based care” as benefits of collaboration. Community partners similarly reported that “Researchers have a better understanding of how the model actually works in community clinics,” “Sharing of expertise,” and “Informs communities about research.”

Tangible products

As indicated in Table 3, PIs reported that a significant majority of RCP products were publications and intervention or training materials. Importantly, PIs reported that a significant minority of publications and targeted a community, rather than research, audience. Conference presentations were more equally distributed across academic and community audiences.

Table 3.

PI report of anticipated/actual “products”

Products	% (n) PIs
Publication Focus	Research audience	Community audience
Collaborative process	78% (14)	39% (7)
Intervention development process	78% (14)	44% (8)
Intervention effectiveness	78% (14)	39% (7)
Intervention implementation	67% (12)	28% (5)
Conference presentation	94% (17)	83% (15)
Intervention materials	78% (14)
Training materials	78% (14)
Intervention training workshops/lectures	72% (13)

In response to items about the benefits of collaboration and lessons learned, both PIs and community partners commented on the impact of collaboration on RCP products. Both respondent groups described how the RCP increased the relevance and “fit” of the intervention or materials. PIs’ comments included, “Development of relevant interventions,” “Feasible interventions,” “Sustainable intervention,” “Better fit between intervention and site,” “Ultimate product is richer and more sustainable.” Similarly, community partners described, “Meeting the needs of the community in an outstanding example of evidence-based practice” and “Model combines principles (strong community mindset) and theory (research perspective).” In addition, multiple PIs reported that there was enhanced community buy-in for the intervention (e.g., “Buy-in from target stakeholders,” “Facilitate provider buy-in”).

Also related to the impact of the RCP on products, a strong theme from PIs was the positive impact of collaboration on the research. Specifically, there were a number of comments about the relevance of research to practice: “Ground research in realities of real practice,” “Relevant research questions,” “Research findings are richer, more complex,” and “Increase feasibility of practice-based re search.”

Phase 3: Sustainability

Distal outcomes

Both groups of respondents discussed the enhanced quality of service delivery, improved outcomes, and the capacity to sustain the intervention resulting from the RCP.

Improved system capacity through EBP implementation

In response to open-ended questions about perceived benefits and lessons learned, PIs and community partners commented about implementation of the intervention and improved capacity of agencies or communities. For example, a community partner commented that a benefit of the RCP was “contributing to the advancement of intervention locally and beyond.” Another commented that “community agencies stay current on best practices and innovative treatment modes.” Similarly, PIs commented on the potential for sustained implementation of EBPs (e.g., “Ability to work with many cohorts over time to more fully impact community outcomes,” “Increased likelihood of sustaining the program over time”). Both community partners and PIs also commented specifically on improved child or family outcomes.

Sustained capacity of collaboration

A majority of PIs (67%) indicated that community partners will or did continue to participate at the completion of the grant/project funding. Twenty-two percent indicated that they were “not sure” of community partner continued participation. Only 11% indicated that community partners would not continue to participate in the RCP. A majority of PIs (61%) indicated that 80% or more of the community partners have continued participation with the project since the beginning. Further PIs commented in open ended responses that a benefit of collaboration was a “relationship built for future studies.”

Additional Themes Related to Conducting Community Research

In addition to the themes related to the process of collaboration between researchers and community partners, PI comments also reflected tension between research and community contexts and challenges related to conducting research in the community. Multiple PIs commented on challenges related to methodological rigor (e.g., “balancing methodological rigor with community feedback”) and organizational factors impacting research and service delivery (e.g., funding, staff turnover). For example, one PI listed “Lack of foundational knowledge regarding research/evaluation” of the community partners as a challenge and one noted concern that the “control group gets intervention prematurely.”

Discussion

Overall, this study adds to our understanding of the use of RCPs in child-focused community based intervention/implementation research and begins to provide a data-based understanding of the benefits and challenges of conducting research using RCPs. Projects involving collaboration between researchers and community stakeholders to develop, adapt, or implement evidence-based practices addressing childhood developmental or mental health problems were examined to identify themes related to challenges and benefits of using RCPs from the formation to execution of activities and sustainability phases.

It is of note that researchers were either solely or jointly involved in the formation of almost all of the RCPs. This may be a reflection of a recent move by funding agencies to require inclusion of community partners in many funding mechanisms and the practical necessity of community partnerships for successful implementation of EBPs. Researchers, then, have clear motivation to seek out relationships with community stakeholders in the implementation of these studies. Community members reported value in the RCP participation, however they may not have the motivation to seek out researchers for such purposes. It is also possible that community members seeking to implement evidence-based services may not think of researcher partners as helpful in such endeavors. This perception may be accurate as federal research funding, for example, does not have as its primary mission to create sustained changes to service delivery. Additionally, community members may have limited access to researchers with specific interest in studying interventions in community settings.

Primary challenges reported related to difficulties with interpersonal and operational processes related to RCP functioning, including communication, task delegation and time/logistics issues. Communication difficulties often included use of technical terms and community members’ variable understanding of research methodology. These challenges may reflect the broader differences between the contexts of research and community services (approaches to quality improvement, language, relative emphasis on individual vs. group differences). These differences may indicate the need for education regarding research methods (especially those relevant to the community project), current community practice and defining common terms used by the research team and those used in community practice at the start of a collaborative effort. This type of exercise might also help the group to build trust through joint learning and working towards a common goal.

PI reports of task division in RCPs indicated that community partners were more likely to be involved in tasks relating to implementation or participant recruitment, while PIs reported involvement in more traditional research activities such as manuscript writing and data analysis. This task division may reflect a logical distribution of roles based on respective areas of expertise. However, examining ways to cross these boundaries to provide direct experience with implementation for researchers and data analysis and participation in publication for community providers may improve collaboration, trust and understanding across disciplines and knowledge exchange. In addition, there may be ways to include community partners in the development of publications for clinical audiences, thereby increasing the reach of the project and providing additional education to community providers.

Community partner responses were obtained for approximately one third of the projects. Exploratory analyses revealed that PIs with a linked community partner respondent for their project rated community partner involvement higher than PIs without a linked community partner respondent. In particular, PIs rated community partner involvement in providing feedback on intervention protocols and revising intervention materials higher for projects with linked partner responses. These findings suggest that projects with greater partnership may rely on community partners for involvement in the development or adaptation of interventions for community delivery. Future research should examine whether partner involvement in development of an intervention increases the fit of the intervention with practice settings.

Primary benefits were often reported in relation to proximal outcomes in which both PIs and community partners reported positive perceptions of the strong collaborative relationships built through the RCP. In fact, the knowledge exchange and relationships appear to be the highlight of the process for both groups and may be an important element of sustainment over time. Importantly, the RCPs were quite productive in terms of publication of partnership and intervention results, development of intervention and training materials and presenting to both academic and community organizations. This productivity is likely to be very important to the motivation of both research and community partners seeking further career development, be it through academic publication or intervention training.

In addition, PIs clearly saw the value of partnership synergy on distal outcomes such as increased intervention relevance and fit, improved services/outcomes, and ongoing capacity for continued collaboration. Community partners also made positive comments about increased community capacity to provide quality services and reported improved service outcomes leading directly from RCP work. The sustainment of a majority of RCPs beyond funding periods highlights the value of such partnerships for both community and research participants. Future research is needed to determine whether sustained partnership infrastructure is associated with sustained improvements in the capacity of communities to serve high need populations.

Limitations

There are several limitations that may affect the generalizability of the results. It would be preferable to have a larger number of participants and a higher response rate especially from community partners. It is possible that community responses were more likely from those with a positive experience, especially given that PIs provided contact information for community respondents. Further, analyses comparing projects with and without community partner respondents suggest that partner involvement may differ for these projects. As such, community partner experiences regarding the challenges and benefits of collaboration may not generalize to other partners on other projects that have less involvement. Also, the use of a web-based survey facilitated data collection, particularly for a busy, professional participant sample was necessary; however, less text was generated from open response items than would have been ideal. The same questions in an interview format might have resulted in richer responses, but interviews would have added considerable participant burden. It is recommended that future research include use of in-depth qualitative methods to obtain richer responses. Another concern is that the sample was derived from a search of databases. In order to be included, partnerships had to meet a high threshold of productivity (e.g., publication or successful grant application) so the sample might not be fully representative however, may provide insight into processes for the most productive groups. The search method likely influenced the high reliance on federal funding seen in the participants. There likely are other RCPs that did not meet our inclusion criteria. Future research is needed that includes alternative methods of recruitment to capture a more diverse group of RCP projects given the potential biases in the types of projects examined in the current study. Despite these limitations, this study provides novel results which indicate that use of RCP approaches are promising in tailoring evidence-based practices for delivery in a range of community service system settings.

Additionally, collaboration between researchers and community stakeholders is a feasible, productive, and valuable method to support or drive research on the effectiveness and implementation of interventions in community settings. Future research might focus on the development of clear strategies to build trust, identify community and research partners and to address challenges in communication and role distribution. More attention to generating resources for partnership work is also needed.

These results provide preliminary evidence for the value of RCPs in developing/adapting EBPs for implementation as well as for improving the ongoing capacity of the research team to access community settings and participants.

Figure 3.

PI report of community partner involvement in project activities