Abstract
Participating in community-engaged dissemination and implementation (CEDI) research is challenging for a variety of reasons. Currently, there is not specific guidance or a tool available for researchers to assess their readiness to conduct CEDI research. We propose a conceptual framework that identifies detailed competencies for researchers participating in CEDI and maps these competencies to domains. The framework is a necessary step toward developing a CEDI research readiness survey that measures a researcher’s attitudes, willingness, and self-reported ability for acquiring the knowledge and performing the behaviors necessary for effective community engagement. The conceptual framework for CEDI competencies was developed by a team of eight faculty and staff affiliated with a university’s Clinical and Translational Science Award (CTSA). The authors developed CEDI competencies by identifying the attitudes, knowledge, and behaviors necessary for carrying out commonly accepted CE principles. After collectively developing an initial list of competencies, team members individually mapped each competency to a single domain that provided the best fit. Following the individual mapping, the group held two sessions in which the sorting preferences were shared and discrepancies were discussed until consensus was reached. During this discussion, modifications to wording of competencies and domains were made as needed. The team then engaged five community stakeholders to review and modify the competencies and domains. The CEDI framework consists of 40 competencies organized into nine domains: perceived value of CE in D&I research, introspection and openness, knowledge of community characteristics, appreciation for stakeholder’s experience with and attitudes toward research, preparing the partnership for collaborative decision-making, collaborative planning for the research design and goals, communication effectiveness, equitable distribution of resources and credit, and sustaining the partnership. Delineation of CEDI competencies advances the broader CE principles and D&I research goals found in the literature and facilitates development of readiness assessments tied to specific training resources for researchers interested in conducting CEDI research.
Keywords: Dissemination and implementation, Implementation science, Community engagement, Stakeholder engagement, Education and training
Background
Dissemination and implementation (D&I) research requires thoughtful partnerships between researchers, practitioners, policy makers, and other key stakeholders [1–6]. As such, D&I differs substantially from many other types of research, including some health services research, which may never involve direct interaction with members of the participating organizations [7]. For example, D&I research commonly involves assessing evidence in the context of community stakeholder needs [8–10], assessing barriers and facilitators to disseminating and implementing the evidence within a specific setting [11], and adapting and testing dissemination and implementation strategies within the setting [12–14]. Stakeholder engagement can be thought of as a continuum ranging from stakeholders solely as participants in a research study to stakeholders exercising control over aspects of the research process. Some of the most extensive engagement approaches in the continuum—Community-Based Participatory Research (CBPR), for example—may not be appropriate for all studies [15]. However, achieving meaningful involvement from the perspective of key stakeholders (e.g., patients, families, healthcare providers, health and community-based organizations) who are influencing the conduct of research enhances the likelihood of success of research activities and, therefore, the overall D&I project [1]. In particular, a high level of stakeholder engagement can help ensure that the research focus is relevant and important for patients, providers, communities, and/or policy makers; help increase practicality and validity of data collection and analysis; and improve the likelihood that study results are disseminated widely and used effectively to create positive impacts on health [15].
As evidenced by such initiatives as the Patient-Centered Outcomes Research Institute (PCORI) [16] and Clinical and Translational Science Awards (CTSA) [17], there is increasing recognition of the importance of participatory approaches that foster meaningful involvement of community stakeholders in research. However, collaboration with stakeholders is challenging to achieve for a variety of reasons. For example, differences between stakeholders and researchers may exist in terms of which problems should be addressed by research [18, 19]. Stakeholders might be reluctant to participate in research that they believe will not directly benefit patients and the community or lead to changes in programs or policies [20]. Similarly, stakeholders may distrust researchers and the research process due to past failures to disseminate research findings within communities and organizations who have participated in research [21]. Partnerships between researchers and stakeholders can also be strained by differential access to knowledge and resources, which creates an inequitable distribution of power [22]. Investigators may have an inflated sense of how implementation “should” be done, and community members may believe that their concerns will be dismissed or that they do not have the necessary skills to participate in research [23]. In addition to such interpersonal factors, structural barriers also may exist, such as uncompensated time for stakeholders and/or burdensome processes required for transferring funds between universities and stakeholders [24]. These challenges may cause researchers to avoid stakeholder engagement altogether. For those researchers who do attempt to engage stakeholders, the challenges may contribute to ineffective stakeholder engagement, which may do more harm than good by perpetuating distrust between community stakeholders and researchers. Ineffective stakeholder engagement can also result in wasted time for both the researcher and community stakeholders. Therefore, it is important for a researcher to know whether he/she is ready to conduct community-engaged D&I (CEDI) research prior to beginning a specific CEDI project. This begs the question, “How do researchers know whether they are ready to conduct CEDI research?”
A pragmatic assessment of a researcher’s readiness for CEDI research would indicate whether a researcher has (1) positive attitudes toward community engagement as well as (2) willingness and (3) ability to acquire knowledge about the community and conduct research in a collaborative way. Researchers with these necessary CEDI competencies could feel confident in pursuing their research endeavors, whereas those who are lacking one or more competencies could pursue training appropriate for remediating identified gaps. An important step toward developing a pragmatic assessment of a researcher’s CEDI readiness is development of a comprehensive framework of specified community engagement competencies for D&I research. Such a framework would outline the researcher’s attitudes as well as the knowledge and behaviors that they would need to be willing and able to acquire and perform to conduct CEDI research.
Although some frameworks and surveys are available related to building and/or assessing partnerships between researchers and community stakeholders [25–28], we have found no assessment that assesses specific aspects of an individual researcher’s readiness to conduct CEDI research prior to engaging in a partnership. The CE and D&I literatures do offer important foundational resources that synthesize and categorize theories and frameworks in translational science [29–31]; identify principles for CE [32] and community-based participatory research (CBPR), specifically [33–35]; identify broad competencies for conducting D&I research [36, 37]; and discuss employing CBPR in D&I [1, 15, 38]. Although the available CE and CBPR principles identify important processes for CE, and the available D&I competencies identify research processes that require community engagement, neither delineates a set of attitudes, knowledge, and behaviors required to carry out integrated CEDI processes. For example, recent guidance on D&I training offered the following competency: “Identify and apply techniques for stakeholder analysis and engagement when implementing evidence-based practices [36].” Articulating this high-level competency is clearly important. However, a necessary next step is to describe in detail what a researcher must believe, know, and do in order to perform this high-level competency. Without such detail, development of a pragmatic assessment of D&I researcher readiness for conducting CEDI will be hindered.
To fill this gap in the current literature, we propose a conceptual framework that identifies detailed competencies for researchers participating in CEDI and maps these competencies to domains. This CEDI framework is not a process model, per se, in that it does not describe a process for translating research into practice [31]. Instead, it outlines the attitudes, knowledge, and behaviors that researchers should have and/or be willing to acquire and perform before beginning a process that involves engaging stakeholders in D&I research. Our work builds upon the CE and D&I literatures—specifically the principles of CE and CE-related competencies for D&I research training—and capitalizes on the CE and D&I expertise of members of our CTSA. The proposed framework of competencies should not be considered the final version but, instead, a thorough product ready for further assessment by a broader group of stakeholders using various methods to assess validity. After such processes have been completed, the framework will serve as the basis for developing a survey-based assessment for individual researchers that will help determine a researcher’s readiness to conduct CEDI research, identify specific training needs (if any) for the researcher, and ultimately facilitate growth in the number of D&I researchers who are skilled in CE approaches.
Conceptual framework
What is community engagement?
Community engagement (CE) is an umbrella term that has encompassed many concepts, models, and definitions over time, ranging from Saul Alinsky’s community organizing model utilizing confrontational strategies for social change [39] to participatory strategies advocated by the World Health Organization, which stress the importance of community members’ participation in improving health [40]. Community engagement has also included concepts of empowerment and co-learning [41, 42], community capacity-building [43], and community-based participatory research [43–45].
What are the principles of community engagement?
In an effort to consolidate multiple CE concepts, the Centers for Disease Control and Prevention (CDC) published the first edition of the Principles of Community Engagement in 1997 to provide researchers, community members, and health professionals with guiding principles for how to effectively involve and mobilize stakeholders in health improvement initiatives [46]. According to the Principles, community engagement is the “process of working collaboratively with groups of people who are affiliated by geographic proximity, special interests, or similar situations with respect to issues affecting their well-being [46].” The Principles were updated by the National Institutes of Health’s CTSA consortium in 2011 and are considered a well-respected and integrated framework appropriate for training a new generation of translational and clinical scientists who are interested in practicing CE research [32].
The nine principles embody a host of attitudes and behaviors that are necessary for successful CE efforts (see Table 1). Each principle is implemented according to the circumstances of the project, community, partnership, and/or organization. The principles are organized under three overarching themes. The first theme includes principles that elucidate needed actions prior to initiating CE (e.g., be clear and become knowledgeable about the community). The second theme contains principles that suggest necessary actions for CE to occur (e.g., establish relationships and build trust, and understand the community empowers itself). The final theme consists of principles that are comprised of ideas and acts for ensuring the engagement endeavor succeeds (e.g., understand partnering is necessary, respect the community, use community assets, release control, and be committed) [32]. CE can be employed in various ways. Collaborators may include organized groups, agencies, institutions, or individuals, and the Principles may be implemented in health promotion, research, or policy related fields [32].
Table 1.
Principles | Abbreviated principles |
---|---|
Prior to initiating the CE effort… | |
1. Be clear about the purposes or goals of the engagement effort and the populations and/or communities you want to engage | Be clear |
2. Become knowledgeable about the community’s culture, economic conditions, social networks, political and power structures, norms and values, demographic trends, history, and experience with efforts by outside groups to engage it in various programs. Learn about the community’s perceptions of those initiating the engagement activities | Be knowledgeable |
For engagement to occur, you must… | |
3. Go to the community, establish relationships, build trust, work with the formal and informal leadership, and seek commitment from community organizations and leaders to create processes for mobilizing the community | Establish trust |
4. Remember and accept that collective self-determination is the responsibility and right of all people in a community. No external entity should assume it can bestow on a community the power to act in its own self-interest | The community empowers itself |
For successful engagement… | |
5. Partnering with the community is necessary to create change and improve health | Partnering is necessary |
6. All aspects of community engagement must recognize and respect the diversity of the community. Awareness of the various cultures of a community and other factors affecting diversity must be paramount in planning, designing, and implementing approaches to engaging a community | Respect the community |
7. Community engagement can only be sustained by identifying and mobilizing community assets and strengths and by developing the community’s capacity and resources to make decisions and take action | Utilize community assets |
8. Organizations that wish to engage a community as well as individuals seeking to effect change must be prepared to release control of actions or interventions to the community and be flexible enough to meet its changing needs | Release control |
9. Community collaboration requires long-term commitment by the engaging organization and its partners | Be committed |
Adapted from Principles of Community Engagement. 2. Washington: US Department of Health and Human Services, 2011
Why integrate community engagement and D&I research?
Benefits associated with implementing CE principles include improved health outcomes [47, 48] and research capacity for communities and organizations, as well as professional development and expanded networks for researchers [49–51]. Policy makers, funding agencies, and community leaders are increasingly demanding the use of CE to facilitate partnerships throughout the research process [50], and CE offers particular benefits to D&I research. Sustained engagement with community partners stands to enhance the rigor and relevance of all phases of D&I research, including the identification of practice gaps, the assessment of barriers and facilitators to change, the selection and tailoring of implementation strategies, the selection and execution of an appropriate study design, the collection and analysis of data, and the dissemination of results. In addition, CE principles provide partners with opportunities to develop processes for research findings to be used in alternative ways (i.e., to improve services and change policy or practices) [32, 49]. Considering CE’s benefits to D&I research and the petition from funders, policy makers, and community leaders for its use across the research continuum [50], it is imperative that D&I researchers understand and acknowledge CE’s relevance, and appropriately integrate its principles into their scholarship.
Methods
The conceptual framework for CEDI competencies was developed by a team of eight faculty and staff affiliated with a university’s CTSA. The CTSA core with which the team is affiliated aims to transform partnerships between academic investigators and community members by providing services such as facilitating stakeholder engagement with community members, health care providers, and investigators; delivering technical assistance for implementation science studies; training academic and clinical audiences on best practices for community-engaged research; and assisting community-based organizations with capacity building for participation in research studies [52]. Thus, the team involved in developing the CEDI competencies has combined expertise in community engagement, dissemination, and implementation science. Specifically, the team is comprised of the CTSA’s (1) director, (2) associate director, and (3) data and evaluation coordinator for the community engagement program; (4) the faculty lead, (5) a faculty expert, (6) a senior investigator, and (7) the communication and dissemination specialist from the D&I methods unit; and (8) a postdoctoral fellow from a health equity research center. Collectively the team has nearly 100 years of experience in the fields of community engagement and D&I research and/or practice. Team members hold degrees in various disciplines—including aging studies, medicine, public administration, public health, and social work—with research and experience focusing on a range of topics that span the life course and are pertinent to health disparities, such as maternal and child health, behavioral health, tobacco prevention and control, cancer, HIV/AIDS, and cardiovascular disease.
We used the Principles of Community Engagement [32] as the primary guide for CE principles because of its prominence in the field and its application to a wide range of community engagement efforts rather than the narrower focus of CBPR. However, we reviewed additional resources of CE principles [27, 33–35, 53] to determine whether important aspects of CE may not be reflected in the Principles of Community Engagement resource. We also considered relevant health behavior theories that could inform the development of competencies based on the principles. Of notable importance are the Theory of Reasoned Action and Theory of Planned Behavior, which suggest that an individual’s attitude toward a behavior is influenced by the individual’s belief that the behavior will lead to a desired outcome [54]. Furthermore, we drew upon Social Cognitive Theory (SCT), acknowledging that for researchers to learn new skills, they are influenced by social and environmental cues from their peers, mentors, community partners, and institutions, which can model and/or encourage community-engaged research practices [55]. Also notable from SCT is the concept of self-efficacy or belief that one has the capability to engage in a particular action.
In defining competencies that would reflect best practices in conducting community-engaged research, we developed a table that included the following information: the community engagement (CE) principles as defined by NIH [32], potential domains, and potential competencies. Our approach to developing this table was adapted from nominal group technique (NGT), which has been used in various contexts, including the development and evaluation of educational curricula [56, 57]. We believed this approach would be useful to answering the key question, “What are the competencies for conducting CEDI?” Because we did not believe it would be practical for each team member to identify a complete list of competencies, as a first step, two authors (CS and TY) developed an initial list of competencies derived from the CE principles by identifying the attitudes, knowledge, and behaviors necessary for carrying out each principle. The intent was to word each competency with enough specificity so that it could be operationalized as one or more survey items in future work that would develop a pragmatic survey measure to assess CEDI research readiness. One author (CS) then developed an initial set of content areas (domains) to which the initial list of competencies could be mapped. Each domain was provided a working definition. For example, two competencies that reflect the domain communication effectiveness were use plain language in discussions with community partners and engage in active listening during discussions. At this point, four additional authors (BP, CR, ZE, JS) were brought into the process to review and revise the initial draft. The four new members were then asked to individually identify questions or concerns about the competencies (e.g., unclear wording, potential missing competencies). The six team members then met to collectively add, delete, or re-word the competencies based on the individual questions/concerns raised in the previous step. At the conclusion of this step, the team had a revised list of competencies. Next, the team members individually mapped each competency to the single domain that seemed to provide the best fit. Following the individual mapping, the group was convened for two sessions in which the sorting preferences were shared, discrepancies were discussed, and modifications to competencies and domains (e.g., wording, definitions, ordering of competencies within domains) were made until consensus was reached. Consensus was defined as all members of the team agreeing to the placement of each competency within one domain.
After the team reached consensus on the draft competencies and domains, we solicited feedback from five community stakeholders with experience participating in research. Specifically, the stakeholders consisted of two CTSA community engagement advocates, a faith-based health promotion specialist, and a director and an operational specialist of a behavioral health organization. Together, these stakeholders have approximately 50 years of community engagement experience. These stakeholders were asked to review the competencies and domains individually and to identify domains that were potentially missing, inappropriate, or duplicative as well as to assess whether the competencies were mapped to domains appropriately. The competencies and domains were distributed via email to the stakeholders. Approximately 1 week later, a research team member (TY) followed up with each stakeholder via telephone to discuss and document their feedback on the competencies and domains. TY then summarized the feedback in a table with a row for each stakeholder and a column for each domain and for summary comments.
Another research team member (CS) then incorporated the stakeholders’ feedback into the draft CEDI domains and competencies. Examples of modifications based on the stakeholder feedback include splitting the “Knowledge of Community Characteristics” domain into two by adding the “Appreciation for stakeholder’s experience with and attitudes toward research” domain, adding three new competencies (e.g., “Incorporate capacity building into the partnership so that stakeholders learn new skills and develop new capabilities for the future”), and re-ordering domains (i.e., moving “Introspection and openness” so that it immediately follows “Perceived value of CE in D&I research”). The revised domains and competencies were then shared with the other team members and the five stakeholders for final review and revision. At this final stage of review, only minimal edits were made to the wording of three competencies.
Recommended CEDI competencies
We identified 40 competencies categorized into nine CEDI domains reflecting attitudes, knowledge, and behaviors for researchers conducting CEDI research (see Table 2). These competencies and domains represent a necessary first step toward future validation of a CEDI readiness assessment. Because readiness involves willingness and ability to perform specific activities, the CEDI competencies include constructs similar to some models that aim to describe the process for engaging stakeholders and conducting D&I research, for example, the Knowledge to Action Framework [58], the Interactive Systems Framework for Dissemination and Implementation [59], and CBPR for clinical trials [60], which emphasize including all parties collaboratively across the knowledge translation continuum [29]. However, there are also key differences between the CEDI constructs and such process models, as some CEDI competencies are precursors to the process. These precursors share similarities to some determinant frameworks, such as PARIHS [61], Understanding-User-Context Framework [62], and the Implementation of Change model [63], which identify barriers and enablers of implementation outcomes [31]. Finally, some of the CEDI competencies share similarities with the “group dynamics characteristics of effective partnerships” that Schulz and colleagues [27] identified, such as “two-way open communication” and “agreed upon problem-solving processes.” However, the CEDI competencies are specified at a more granular level, providing actions for open communication as well as attitudes and behaviors that support consensus building for problem-solving processes. This level of specificity will facilitate development of survey items to assess readiness. The CEDI domains are described below.
Table 2.
Domain | Competencies |
---|---|
Perceived value of CE in D&I research: The researcher’s attitude toward the potential for enhancing D&I research processes and outcomes through community engagement |
1. Believe that partnership with the community can help to effectively address barriers to implementation and generate strategies to implement effective services 2. Believe that partnership with the community better enables answers to clinical questions and improves the impact of the research through policy change and bridging the gap between research and practice 3. Believe the partnership can produce valuable non-research benefits to the community (e.g., workforce development) |
Introspection and openness: The researcher’s willingness and/or ability to examine their own preconceptions and to be receptive of others’ beliefs and opinions |
1. Engage in self-reflection about one’s own cultural background and how it shapes one’s views of health and health care 2. Examine one’s own preconceived notions about specific cultures and populations represented in the community (e.g., notions formed through life experiences, previous research findings) 3. Recognize cultural differences between oneself and community representatives and avoid making assumptions about similarities in culture, experiences, and values 4. Practice cultural humility |
Knowledge of community characteristics: The researcher’s willingness and/or ability to learn about the community’s characteristics and prior experiences |
1. Define the stakeholder community (or communities) one wants to engage 2. Map stakeholders within the community (e.g., identify stakeholders, analyze relationships, identify communication preferences) 3. Identify representatives of each stakeholder group and investigate the rules of engagement among these organizations/groups 4. Meet with community stakeholders in the community setting 5. Examine demographics and aspects of diversity within the community 6. Examine the social determinants of health in the community (i.e., economic stability, education, health and healthcare, neighborhood and built environment) and how they influence perceptions of problems, priorities, and solutions related to health and health systems 7. Examine key historical events, customs, and power dynamics in the community, in the context of social determinants 8. Perform a needs assessment to identify community needs and priorities 9. Identify strengths within the community that can support and sustain change intended to improve health |
Appreciation for stakeholder’s experience with and attitudes toward research: The researcher’s willingness and/or ability to assess how the community’s research attitudes and experiences may affect the partnership |
1. Examine the community’s attitudes toward research—including the process of research, the types of evidence that they value, and their views of the products of research, such as evidence-based practices and guidelines 2. Investigate the community’s past and current research efforts to implement change(s) that would improve its circumstances 3. Examine the community’s perceptions of the partnering researcher and/or their academic institution |
Preparing the partnership for collaborative decision-making: The researcher’s willingness and/or ability to organize the partnership in a way that facilitate dialogues, collective decision-making, and coordinated action |
1. Acknowledge the expertise in facilitation and mobilization that community stakeholders have 2. Observe the partner’s formal and informal process for decision-making 3. Obtain commitment from community organizations and leaders to co-create processes for decision-making and mobilizing the community 4. Collaboratively outline the responsibilities of both community members and researchers |
Collaborative planning for the research design and goals: The researcher’s willingness and/or ability to adapt to the attitudes and needs of community stakeholders when defining the research process |
1. Collaboratively select an implementation framework or theory and decide how to use it to guide specific intervention, evaluation, and dissemination activities within the research 2. Collaboratively adapt interventions and implementation outcomes to meet the needs and preferences of the community 3. Collaboratively select the implementation outcomes and health outcomes of interest as well as indicators to assess progress toward the outcomes 4. Co-create a timeline with meaningful benchmarks for both the community and academic partner 5. Work with stakeholders to integrate research needs within current processes and minimize data collection burden for stakeholders |
Communication effectiveness: The researchers’ willingness and/or ability to clearly present ideas, listen to community partners, and work through issues |
1. Use plain language in discussions with community partners 2. Use language that is culturally sensitive 3. Engage in active listening during discussions 4. Engage in productive conflict resolution techniques 5. Clarify misunderstandings respectfully |
Equitable distribution of resources and credit: The researcher’s willingness and/or ability to share resources for conducting the research and credit for outcomes of the research |
1. Provide the community with financial resources needed to engage effectively in the research 2. Promote equity in resource distribution across all partners, including between the community partner and the researcher’s institution as well as between community stakeholder groups participating in the partnership 3. Share credit for successes by inviting community partners to participate in presentations of the research and acknowledging the community’s role in publications and media coverage |
Sustaining the partnership: The researcher’s willingness and/or ability to invest in a long-term relationship with community stakeholders |
1. Incorporate capacity building into the partnership so that stakeholders learn new skills and develop new capabilities for the future 2. Seek external funding for the partnership to become self-sustaining 3. Collaboratively plan for future research projects 4. Commit time and effort to addressing stakeholder needs beyond the scope of the research agenda by volunteering in the community and connecting stakeholders with other individuals and resources |
Perceived value of CE in D&I research refers to the researcher’s attitude toward the potential for enhancing research processes and outcomes through stakeholder engagement. More specifically, this domain points to the researcher’s understanding and positive attitude that CE can improve such activities as identifying implementation barriers and strategies, and, ultimately, bridge the gap between research and practice within the community [64]. This domain is important because a researcher who does not hold positive beliefs about the value of CE and self-efficacy in their ability to implement CE strategies likely will fall short on the competencies in the other domains.
Introspection and openness refers to the researcher’s willingness to examine their own preconceptions and to be receptive of others’ beliefs and opinions. It highlights the need for researchers to be both inward looking—self-aware and willing to examine their own preconceived notions and biases—and receptive to the beliefs of others in order to work effectively across cultures [64]. This self-reflection and openness is a precursor to effective communication, collaborative planning, and effective facilitation of new decision-making processes.
Knowledge of community characteristics refers to the researcher’s willingness and ability to understand important characteristics of the community, practice, or setting. This robust domain includes a broad range of stakeholder characteristics, including general characteristics (e.g., demographics, culture, economics) and health and healthcare characteristics (e.g., prevalence of disease, access to health care services, organizational context) [65]. Some of these characteristics may be understood through quantitative analysis (e.g., demographics); however, the majority requires an understanding developed through qualitative approaches (e.g., focus groups, informal conversations) and is iteratively refined through long-term engagement with stakeholders who bring their expert knowledge of a community and organization to bear on framing the research. Knowledge of the community is a precursor to the researcher’s ability to understand whether a particular topic will likely be of concern to community stakeholders, whether there are opinion leaders and potential champions who might support a change effort [66, 67], and ultimately whether implementation of an effort focused on the topic is feasible [68]. This knowledge is also a vital precursor to collaborative research planning, which incorporates bidirectional feedback from community partners with their own skill sets and expertise, including identifying a process for adapting the intervention and suggesting modifications while also maintaining fidelity of the intervention [36].
Appreciation for stakeholder’s experience with and attitudes toward research focuses on the community’s perceptions of research in general and the researcher and/or their institution specifically, as well as prior experience with implementing change efforts to improve the community’s circumstances [69, 70]. Although this domain represents, in part, a type of knowledge about the community, it is important to differentiate from the “Knowledge of community characteristics domain” because it also includes perceptions that stakeholders form about the researcher and/or their institution, which may be based on such factors as direct experience, second-hand information, reputation, or initial impressions. The researcher’s willingness and ability to explore, understand, and account for stakeholder experiences and perceptions are important because these factors could prove to be barriers or facilitators to engagement.
Preparing the partnership for collaborative decision-making refers to the researcher’s willingness and ability to organize the partnership in a way that facilitates dialogue, collective decision-making, and coordinated action within the community to improve health. Doing so requires the researcher to acknowledge the expertise in facilitation and mobilization that stakeholders bring, observe the stakeholders formal and informal decision-making processes, obtain permission from stakeholders to co-create decision-making processes, and define stakeholder and researcher roles. Through this collaborative structure, the team can best understand status quo-decision-making processes, secure commitment from all stakeholders to redesign these processes as needed, and engage in productive conflict resolution techniques rather than counterproductive techniques such as domination or coercion [71].
Collaborative planning for the research design and goals refers to the D&I researcher’s willingness and ability to adapt to the attitudes and needs of the stakeholders when defining the research process and research goals. Many researchers are accustomed to defining the aims, methods, and measures of success independently. However, a key component of stakeholder-engaged research is flexibility and adaptation based on strengths, interests, needs, and desires of stakeholders [72]. This domain reflects the need for researchers to work collaboratively with partners develop a brand for the project; adapt interventions; select implementation strategies; select implementation and health outcomes that are patient, family, and community centered; identify ways to minimize data collection burden for stakeholders; and develop project timelines. For example, stakeholder-engaged development of a project name, logo, and color scheme contributes to a project brand that resonates within the community and enhances effectiveness of promotional materials (e.g., for recruitment) [73]. Another example of collaborative decision-making is building consensus around appropriate indicators of progress that will enable meaningful and reasonable targets for desired outcomes [74]. Collaborative planning helps ensure that all stakeholders see the potential benefit of the research, see the effort required as equitable and appropriate, and ultimately remain committed to the research.
Communication effectiveness refers to the researcher’s ability to discuss issues, share ideas, and listen to stakeholders. Therefore, effective communication is bidirectional, involving both information exchange (i.e., transactional) and relational (i.e., transformational) processes [75]. Effective communication requires understanding the target audience, roles in the communications process, barriers to effective communication, and careful selection of communication approaches [76]. Plain, conversational language (as opposed to scientific jargon) and clear, concise messages are important [77] as are active listening to the ideas of others and the nurturing of interpersonal relationships [75]. Communication effectiveness is important because the extent to which a researcher is able to communicate effectively likely influences their ability to engage in productive discussions about the community (e.g., prior experience with implementing change), share information about the research topic and process (e.g., possible implementation strategies), facilitate decision-making and action within the community (e.g., identification of meaningful measures of progress toward desired outcomes), and resolve conflicts that might arise.
Equitable distribution of resources and credit refers to the researcher’s willingness and ability to share resources with stakeholders for their contributions to conducting the research and credit for outcomes of the research. Specifically, this reflects the researcher’s willingness to secure adequate financial resources for the CEDI effort and to promote equitable distribution of the resources across their institution and the external partners. Also important is inviting stakeholders to participate in presentations and manuscripts on the research and ensuring stakeholders receive due credit for their role in the research outcomes and products. This domain relates to the facilitation of decision-making and mobilization domain in the context of promoting equitable distribution of resources within the community. Furthermore, this domain represents issues that are critical for developing trust within the partnership.
Sustaining the partnership reflects the researcher’s willingness and ability to invest the time and effort to develop and sustain a long-term relationship with the stakeholders. This domain integrates activities required to build trust and plan for a partnership beyond the life of funding for a single research project. Specifically, these activities include ensuring that the partnership facilitates capacity building for the stakeholder community, seeking funding to support the ongoing partnership, planning for future projects, and committing time to support efforts outside of the research agenda (e.g., volunteering in the community, facilitating collaboration with other external partners to address such issues as infrastructure development and educational opportunities).
Discussion
In this article, we have identified competencies for conducting community-engaged D&I research and have mapped them to nine domains. These competencies build upon well-described and accepted principles of community engagement [32, 46] and competencies for D&I research training [36, 37, 78]. An important distinction between these recommended competencies and previous literature is in the level of granularity of the competencies and the distinct focus on CE in D&I research. The CEDI competencies describe specific attitudes, knowledge, and behaviors that will enable a D&I researcher to achieve broader goals identified in previous literature, for example, to “Build relationships with community members and community-based organizations, in order to engage multiple perspectives on the problem” [37]. We believe that this level of granularity and the categorization of the CEDI competencies is necessary for future work to develop a readiness assessment for D&I researchers prior to their engagement in a partnership with community stakeholders.
Development of the CEDI competencies is consistent with the increasing emphasis on stakeholder engagement in D&I research [1, 7, 79, 80]. Specifically, CE can help ensure that the research topic and interventions are relevant and responsive to stakeholder needs and interests; data collection tools are appropriate and yield valid data; findings are fully understood and interpreted meaningfully; and findings are disseminated effectively to inform practice, programs, and policies [15]. These issues clearly relate to the potential impact of D&I research; however, they also have implications for the challenges D&I researchers face conducting research. For example, recruitment for D&I research can be difficult for various reasons, such as stakeholder skepticism about the usefulness of research, the time-commitment required of participants, competing demands of time and effort, and discrepancies between the researcher’s and stakeholders’ expected timeline for the project [81]. CEDI competencies point to communication approaches and other activities that can help clarify roles and expectations in terms of time involved with project participation, minimize the effort of data collection imposed on study participants, and ensure that any effort contributes to clear and valuable benefits to the community.
The recommended CEDI competencies are a first step toward development of a readiness assessment for researchers interested in conducting CEDI. This assessment will be comprised of a scale that measures a researcher’s attitudes, willingness, and self-reported ability for acquiring the knowledge and performing the behaviors necessary for effective community engagement. It will determine specific gaps in competencies that can be addressed with training opportunities prior to engaging in a partnership. Future research could build upon our current version of CEDI competencies to conduct content validity assessment with a larger group of researchers and community stakeholders prior to development and validation of the CEDI readiness survey assessment. An ultimate goal could be to disseminate the assessment through CTSAs and other research institutions interested in promoting community-engaged D&I research. We believe this trajectory would accelerate development and spread of CE-related training for D&I researchers and, ultimately, increase the effective use of CE principles and practices in D&I research. Furthermore, this line of research and survey development would complement current tools and measures useful for assessing partnership capacity [26] and organizational readiness [82], by assessing researcher readiness prior to partnership formation and guiding the researcher to additional training as needed. In other words, CEDI assessment would enable researchers to enter into partnerships well prepared to serve as a collaborative member of the research team and to support capacity building and readiness development among partnering stakeholders. In addition, to identifying specific gaps in CEDI readiness, the assessment also could be administered pre and post training to assess acceptability and impact of training opportunities completed. Finally, we sought to advance competencies that are specific to both community-engaged and dissemination and implementation research. As evidenced by this Special Issue, these areas are highly complementary and in some cases so tightly connected that it is difficult to parse out competencies that are specific only to community engagement or dissemination and implementation. Some of the competencies (e.g., “Believe that partnership with the community can help to effectively address barriers to implementation and generate strategies to implement effective services” and “Collaboratively select an implementation framework or theory and decide how to use it to guide specific intervention, evaluation, and dissemination activities within the research”) may be more easily identifiable as specific to dissemination and implementation given their inclusion of terminology that is often associated with that field. Others appear to be much more generic and easily applicable to a wide range of research pursuits (e.g., competencies in the “Communication Effectiveness” domain). Since the field of dissemination and implementation draws from a rich tapestry of theories, conceptual frameworks, methods, and outcomes that have their origins elsewhere, it is common to have difficulty articulating what is “uniquely D&I.” We believe that this integrated set of CEDI competencies will be broadly applicable to areas beyond dissemination and implementation research given the breadth and utility of both areas, although we acknowledge that further research is needed to assess the relevance of the CEDI competencies for other fields.
Limitations
Our present version of CEDI domains and competencies has a few limitations. First, the domains and competencies may have been different had we used a different framework for CE principles as a guide. However, one of the strengths of the Principles of Community Engagement [32] is that it was developed based on consensus building among a large group of experts and stakeholders. Second, although our team is comprised of individuals with substantial CE and D&I expertise, we do not represent all perspectives of CE and D&I experts nationally and internationally. Similarly, the five stakeholders that participated in our development process may not be representative of all stakeholders who participate in community-engaged research. Further testing of the framework could benefit from broader input on whether our competencies adequately represent CE principles for D&I research.
Conclusions
D&I research requires effective partnerships between researchers and stakeholders who represent the communities and organizations participating in the study. However, researchers may not receive sufficient training on how to establish and maintain these effective partnerships. Delineating CEDI competencies advances the broader CE principles and D&I research goals found in the literature by serving as an important step toward developing a readiness assessment for CEDI research. Such an assessment can point researchers toward specific training resources needed prior to conducting CEDI research.
Acknowledgements
The authors would like to thank William (Bill) Kearney; Ginny Lewis, MSW; Lisa Quarles; Ronnie Rubin, PhD; and Shawna Weaver, LCSW for their insightful and comprehensive feedback on the CEDI competencies and domains. The authors also thank Kea Turner for her assistance with literature searches for this project. The authors are all supported by the National Institutes of Health (NIH) through the UNC Clinical Translation Science Award (1UL1TR001111). BJP also was supported in part by NIH through R25 MH080916; L30 MH108060; P30 AI50410; R01 MH106510 and The Duke Endowment (1945-SP). GCS also was supported by NIH through Grant Award Number K24 HL105493. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.
Compliance with ethical standards
Conflict of interest
The authors declare that they have no competing interests.
Ethical approval
This project did not include human and animal subjects or secondary data sources about human subjects and therefore was not reviewed by an IRB.
Footnotes
Implications
Research: Future research should focus on development and validation of a CEDI readiness assessment tool comprised of survey measures, with an ultimate goal of disseminating the tool through CTSAs and other research institutions interested in promoting community-engaged D&I research.
Practice: Researchers should use the CEDI competencies to assess their readiness for conducting community-engaged D&I research, and community stakeholders could use the competencies to clarify their understanding of the characteristics and processes of effective community-researcher partnerships.
Policy: Policy makers should use this framework to guide resource allocation and policies that promote effective researcher-community partnerships and interinstitutional collaboration.
Data from this project were presented as a poster at the 9th Annual Conference on the Science of Dissemination and Implementation. The authors have full control of all primary data and agree to allow the journal to review their data if requested. The findings reported have not been previously published, and the manuscript is not being simultaneously submitted elsewhere.
References
- 1.Chambers DA, Azrin ST. Partnership: a fundamental component of dissemination and implementation research. Psychiatric Services. 2013;64(16):509–511. doi: 10.1176/appi.ps.201300032. [DOI] [PubMed] [Google Scholar]
- 2.Spoth R, Clair S, Greenberg M, Redmond C, Shin C. Toward dissemination of evidence-based family interventions: maintenance of community-based partnership recruitment results and associated factors. Journal of Family Psychology. 2007;21(2):137–146. doi: 10.1037/0893-3200.21.2.137. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Research, N. A. M. H. C. S., & Workgroup, C. E . The road ahead: research partnerships to transform services. Bethesda: National Institute of Mental Health; 2006. [Google Scholar]
- 4.Palinkas LA, Short C, Wong M. Research-practice-policy partnerships for implementation of evidence-based practices in child welfare and child mental health. New York: William T. Grant Foundation; 2015. [Google Scholar]
- 5.Inkelas M, Brown AF, Vassar SD, Sankaré IC, Martinez AB, Kubicek K, et al. Enhancing dissemination, implementation, and improvement science in CTSAs through regional partnerships. Clinical and Translational Science. 2015;8(6):800–806. doi: 10.1111/cts.12348. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Bledsoe-Mansori SE, Bellamy JL, Wike T, Grady M, Grady M, Dinata E, et al. Agency-university partnerships for evidence-based practice: a national survey of schools of social work. Social Work Research. 2013;37(3):179–193. doi: 10.1093/swr/svt015. [DOI] [Google Scholar]
- 7.Esposito D, Heeringa J, Bradley K, Croake S, Kimmey L. PCORI dissemination & implementation framework. Washington, D.C.: Patient-Centered Outcomes Research Institute; 2015. [Google Scholar]
- 8.Powell BJ, Bosk EA, Wilen JS, Danko CM, Van Scoyoc A, Banman A. Evidence-based programs in “real world” settings: Finding the best fit. In: Daro D, Cohn Donnelly A, Huang L, editors. Advances in child abuse prevention knoweldge: the perspective of new leadership. New York: Springer; 2015. pp. 145–177. [Google Scholar]
- 9.McCracken SG, Marsh JC. Practitioner expertise in evidence-based practice decision making. Research on Social Work Practice. 2008;18(4):301–310. doi: 10.1177/1049731507308143. [DOI] [Google Scholar]
- 10.Walker SC, Lyon AR, Trupin EW. The consistencies and vagaries of the Washington State Inventory of Evidence-Based Practice: the definition of “evidence-based” in a policy context. Adm Policy Ment Health Ment Health Serv Res. 2015; 1–13. doi:10.1007/s10488-015-0652-y. [DOI] [PubMed]
- 11.Flottorp SA, Oxman AD, Krause J, Musila NR, Wensing M, Godycki-Cwirko M, et al. A checklist for identifying determinants of practice: a systematic review and synthesis of frameworks and taxonomies of factors that prevent or enable improvements in healthcare professional practice. Implementation Science. 2013;8(35):1–11. doi: 10.1186/1748-5908-8-35. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implementation Science. 2015;10(21):1–14. doi: 10.1186/s13012-015-0209-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Baker R, Cammosso-Stefinovic J, Gillies C, Shaw E J, Cheater F, Flottorp S, Robertson N. Tailored interventions to overcome identified barriers to change: effects on professional practice and health care outcomes. Cochrane Database of Syst Rev. 2010; 3 Art. No.: CD005470: 1–77. doi:10.1002/14651858.CD005470.pub2. [DOI] [PMC free article] [PubMed]
- 14.Powell BJ, Beidas RS, Lewis CC, Aarons GA, McMillen JC, Proctor EK, Mandell DS. Methods to improve the selection and tailoring of implementation strategies. Journal of Behavioral Health Services & Research. 2015 doi: 10.1007/s11414-015-9475-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Minkler M, Salvatore AL, Brownson RC, Colditz GA, Proctor EK. Dissemination and implementation research in. New York: Oxford University Press; 2012. Participatory approaches for study design and analysis in dissemination and implementation research; pp. 192–212. [Google Scholar]
- 16.Fleurence R, Selby JV, Odom-Walker K, Hunt G, Meltzer D, Slutsky JR, Yancy C. How the Patient-Centered Outcomes Research Institute is engaging patients and others in shaping its research agenda. Health Affairs. 2013;32(2):393–400. doi: 10.1377/hlthaff.2012.1176. [DOI] [PubMed] [Google Scholar]
- 17.Zerhouni EA, Alving B. Clinical and Translational Science Awards: a framework for a national research agenda. Translational Research. 2006;148(1):4–5. doi: 10.1016/j.lab.2006.05.001. [DOI] [PubMed] [Google Scholar]
- 18.Buchanan DR. Building academic-community linkages for health promotion: a case study in Massachusetts. American Journal of Health Promotion. 1996;10(4):262–269. doi: 10.4278/0890-1171-10.4.262. [DOI] [PubMed] [Google Scholar]
- 19.Taylor M. Community participation in the real world: opportunities and pitfalls in new governance spaces. Urban Studies. 2007;44(2):297–317. doi: 10.1080/00420980601074987. [DOI] [Google Scholar]
- 20.Wang CC. Photovoice: a participatory action research strategy applied to women’s health. Journal of Women’s Health. 1999;8(2):185–192. doi: 10.1089/jwh.1999.8.185. [DOI] [PubMed] [Google Scholar]
- 21.Schulz AJ, Parker EA, Israel BA, Becker AB, Maciak BJ, Hollis R. Conducting a participatory community-based survey for a community health intervention on Detroit’s east side. Journal of Public Health Management and Practice. 1998;4(2):10–24. doi: 10.1097/00124784-199803000-00004. [DOI] [PubMed] [Google Scholar]
- 22.Alegria M, Wong Y, Mulvaney-Day N, Nillni A, Proctor EK, Nickel M, et al. Community-based partnered research: new directions in mental health services research. Ethnicity & Disease. 2012;21:S1–8–S1–16. [PMC free article] [PubMed] [Google Scholar]
- 23.Wallerstein NB, Duran B. Using community-based participatory research to address health disparities. Health Promotion Practice. 2006;7(3):312–323. doi: 10.1177/1524839906289376. [DOI] [PubMed] [Google Scholar]
- 24.Black KZ, Hardy CY, De Marco M, Ammerman AS, Corbie-Smith G, Council, B et al. Beyond incentives for involvement to compensation for consultants: increasing equity in CBPR approaches. Progress in Community Health Partnerships. 2013;7(3):263–270. doi: 10.1353/cpr.2013.0040. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.MacLellan-Wright MF, Anderson D, Barber S, Smith N, Cantin B, Felix R, Raine K. The development of measures of community capacity for community-based funding programs in Canada. Health Promotion International. 2007;22(4):299–306. doi: 10.1093/heapro/dam024. [DOI] [PubMed] [Google Scholar]
- 26.Andrews JO, Newman SD, Cox MJ, Meadows O. Are we ready? A toolkit for academic-community partnerships in preparation for community-based participatory research. Charleston: Medical University of South Carolina; 2011. [Google Scholar]
- 27.Schulz AJ, Israel BA, Lantz P. Instrument for evaluating dimensions of group dynamics within community-based participatory research partnerships. Evaluation and Program Planning. 2003;26:249–262. doi: 10.1016/S0149-7189(03)00029-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Israel BA, Lantz PM, McGranaghan RJ, Kerr DL, Guzman JR. Methods in community-based participatory research for health. San Francisco: Jossey-Bass, Inc.; 2005. Documentation and evaluation of CBPR partnerships: in-depth interviews and closed-ended questionnaires; pp. 255–277. [Google Scholar]
- 29.Mitchell SA, Fisher CA, Hastings CE, Silverman LB, Wallen GR. A thematic analysis of theoretical models for translational science in nursing: mapping the field. Nursing Outlook. 2010;58(6):287–300. doi: 10.1016/j.outlook.2010.07.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Tabak RG, Khoong EC, Chambers DA, Brownson RC. Bridging research and practice: models for dissemination and implementation research. American Journal of Preventive Medicine. 2012;43(3):337–350. doi: 10.1016/j.amepre.2012.05.024. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Nilsen P. Making sense of implementation theories, models and frameworks. Implementation Science. 2015;10(53):1–13. doi: 10.1186/s13012-015-0242-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Clinical and Translational Science Awards Consortium, Community Engagement Key Function Committee, Task Force on the Principles of Community Engagement . Principles of community engagement (no. 11-7782) Washington, D. C: Department of Health and Human Services, National Institues of Health, Centers for Disease Control and Prevention; 2011. [Google Scholar]
- 33.Minkler M, Wallerstein N. Community-based participatory research for health: from process to outcomes. New York: Wiley; 2011. [Google Scholar]
- 34.Israel BA. Methods for community-based participatory research for health. 2. San Francisco: Jossey-Bass, Inc.; 2013. [Google Scholar]
- 35.Minkler M. Community based participatory research for health. San Francisco: Jossey-Bass, Inc.; 2003. [Google Scholar]
- 36.Padek M, Colditz G, Dobbins M, Koscielniak N, Proctor EK, Sales AE, Brownson RC. Developing educational competencies for dissemination and implementation research training programs: an exploratory analysis using card sorts. Implementation Science. 2015;10(114):1–9. doi: 10.1186/s13012-015-0304-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Gonzales R, Handley MA, Ackerman S, O’Sullivan PS. A framework for training health professionals in implementation and dissemination science. Academic Medicine. 2012;87(3):271–280. doi: 10.1097/ACM.0b013e3182449d33. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Palinkas LA, Soydan H. Translation and implementation of evidence-based practice. New York: Oxford University Press; 2012. [Google Scholar]
- 39.Alinksky S. Rules for radicals: a pragmatic primer for realistic radicals. New York: Random House; 1971. [Google Scholar]
- 40.Wallerstein N, Minkler M, Carter-Edwards L, Avila M, Sánchez V. Improving health through community engagement, community organization, and community building. In: Glanz K, Rimer BK, Viswanath K, editors. Health behavior: theory, research, and practice. 5th. Hoboken: Wiley; 2015. pp. 277–300. [Google Scholar]
- 41.Freire P. Pedagogy of the oppressed. New York: Continuum; 2000. [Google Scholar]
- 42.Wallerstein N, Bernstein E. Empowerment education: Freire’s ideas adapted to health education. Education Quarterly. 1988;15(4):379–394. doi: 10.1177/109019818801500402. [DOI] [PubMed] [Google Scholar]
- 43.Goodman RM, Speers MA, McLeroy K, Fawcett S, Kegler M, Parker E, et al. Identifying and defining the dimensions of community capacity to provide a basis for measurement. Health Education & Behavior. 1998;25(3):258–278. doi: 10.1177/109019819802500303. [DOI] [PubMed] [Google Scholar]
- 44.Freire P. Pedagogy of the oppressed. New York: Herder and Herder; 1970. [Google Scholar]
- 45.Israel BA, Schulz AJ, Parker EA, Becker AB. Review of community-based research: assessing partnership approaches to improve public health. Annual Review of Public Health. 1998;19:173–202. doi: 10.1146/annurev.publhealth.19.1.173. [DOI] [PubMed] [Google Scholar]
- 46.Centers for Disease Control and Prevention . Principles of community engagement. Atlanta: Public Health Practice Program Office; 1997. [Google Scholar]
- 47.Attree P, French B, Milton B, Povall S, Whitehead M, Popay J. The experience of community engagement for individuals: a rapid review of the evidence. Health & Social Care in the Community. 2011;19(3):250–260. doi: 10.1111/j.1365-2524.2010.00976.x. [DOI] [PubMed] [Google Scholar]
- 48.Milton B, Attree P, French B, Povall S, Whitehead M, Popay J. The impact of community engagement on health and social outcomes: a systematic review. Community Development Journal. 2012;47(3):316–334. doi: 10.1093/cdj/bsr043. [DOI] [Google Scholar]
- 49.Roman Isler M, Corbie-Smith G. Practical steps to community engaged research: from inputs to outcomes. The Journal of Law, Medicine & Ethics. 2012;40:904–914. doi: 10.1111/j.1748-720X.2012.00719.x. [DOI] [PubMed] [Google Scholar]
- 50.Ahmed SM, Palermo AGS. Community engagement in research: frameworks for education and peer review. American Journal of Public Health. 2010;100(8):1380–1387. doi: 10.2105/AJPH.2009.178137. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.Viswanathan M, Ammerman A, Eng E, Gartlehner G, Lohr KN, Griffith D, et al. Community-based participatory research: assessing the evidence (evidence report/technology assessment no. 99 no. 04-E022–2) Rockville: Agency for Healthcare Research and Quality; 2004. [PMC free article] [PubMed] [Google Scholar]
- 52.The North Carolina Translational & Clinical Science Institute: About CARES. (2016). Retrieved July 14, 2016, from https://tracs.unc.edu/index.php/services/cares.
- 53.Israel BA, Eng E, Schulz AJ, Parker EA. Methods for community-based participatory research for health. San Francisco: Jossey-Bass, Inc.; 2013. [Google Scholar]
- 54.Ajzen I. The theory of planned behavior. Organizational Behavior and Decision Processes. 1991;50:179–211. doi: 10.1016/0749-5978(91)90020-T. [DOI] [Google Scholar]
- 55.Bandura A. Social foundation of thought and action: a social cognitive theory. New York: Prentice-Hall; 1986. [Google Scholar]
- 56.O’Neil MJ, Jackson L. Nominal group technique: a process for initiating curriculum development in higher education. Studies in Higher Education. 1983;8(2):129–138. doi: 10.1080/03075078312331378994. [DOI] [Google Scholar]
- 57.Chapple M, Murphy R. The nominal group technique: extending the evaluation of students’ teaching and learning experiences. Assessment & Evaluation in Higher Education. 1996;21(2):147–160. doi: 10.1080/0260293960210204. [DOI] [Google Scholar]
- 58.Graham ID, Logan J, Harrison MB, Straus SE, Tetroe J, Caswell W, Robinson N. Lost in knowledge translation: time for a map? The Journal of Continuing Education in the Health Professions. 2006;26(1):13–24. doi: 10.1002/chp.47. [DOI] [PubMed] [Google Scholar]
- 59.Wandersman A, Duffy J, Flaspohler P, Noonan R, Lubell K, Stillman L, et al. Bridging the gap between prevention research and practice: the interactive systems framework for dissemination and implementation. American Journal of Community Psychology. 2008;41:171–181. doi: 10.1007/s10464-008-9174-z. [DOI] [PubMed] [Google Scholar]
- 60.Leykum LK, Pugh JA, Lanham HJ, Harmon J, McDaniel RR. Implementation research design: integrating participatory action research into randomized controlled trials. Implementation Science. 2009;4(69):1–8. doi: 10.1186/1748-5908-4-69. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 61.Rycroft-Malone J. The PARIHS framework: a framework for guiding the implementation of evidence-based practice. Journal of Nursing Care Quality. 2004;19(4):297–304. doi: 10.1097/00001786-200410000-00002. [DOI] [PubMed] [Google Scholar]
- 62.Jacobson N, Butterill D, Goering P. Development of a framework for knowledge translation: understanding user context. Journal of Health Services Research & Policy. 2003;8(2):94–99. doi: 10.1258/135581903321466067. [DOI] [PubMed] [Google Scholar]
- 63.Grol R, Wensing M, Eccles M, Davis D, editors. Improving patient care: the implementation of change in health care. 2. Chichester: Wiley; 2013. [Google Scholar]
- 64.Axtell SA, Avery MD, Westra B. Incorporating cultural competence content into graduate nursing curricula through community-university collaboration. Journal of Transcultural Nursing. 2010;21(2):183–191. doi: 10.1177/1043659609357633. [DOI] [PubMed] [Google Scholar]
- 65.Aarons GA, Glisson C, Green PD, Hoagwood K, Kelleher KJ, Landsverk JA, The Research Network on Youth Mental Health The organizational social context of mental health services and clinician attitudes toward evidence-based practice: a United States national study. Implementation Science. 2012;7(56):1–15. doi: 10.1186/1748-5908-7-56. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 66.Rogers EM. Diffusion of innovations. 5. New York: Free Press; 2003. [Google Scholar]
- 67.Shea CM, Belden CM. What is the extent of research on the characteristics, behaviors, and impacts of health information technology champions? A scoping review. BMC Medical Informatics and Decision Making. 2016;16(2):1–17. doi: 10.1186/s12911-016-0240-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 68.Straus S. The action cycle. In: Straus S, Tetroe J, Graham I, editors. Knowledge translation in healthcare. Hoboken: Wiley-Blackwell; 2009. p. 59. [Google Scholar]
- 69.Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implementation Science. 2009;4(50):1–15. doi: 10.1186/1748-5908-4-50. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 70.Weiner BJ. A theory of organizational readiness for change. Implementation Science. 2009;4(67):1–9. doi: 10.1186/1748-5908-4-67. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 71.Mohr J, Spekman R. Characteristics of partnership success: partnership attributes, communication behavior, and conflict resolution techniques. Strategic Management Journal. 1994;15(2):135–152. doi: 10.1002/smj.4250150205. [DOI] [Google Scholar]
- 72.Deverka PA, Lavallee DC, Desai PJ, Esmail LC, Ramsey SD, Veenstra DL, Tunis SR. Stakeholder participation in comparative effectiveness research: defining a framework for effective engagement. Journal of Comparative Effectiveness Research. 2012;1(2):181–194. doi: 10.2217/cer.12.7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 73.Holt, C. L., Tagai, E. K., Scheirer, M. A., Santos, S. L. Z., Bowie, J., Haider, M., et al. Translating evidence-based interventions for implementation: experiences from Project HEAL in African American churches. Implementation Science, 9(66). doi:10.1186/1748-5908-9-66. [DOI] [PMC free article] [PubMed]
- 74.Braspenning J, Hermens R, Calsbeek H, Westert G, Campbell S. Improving patient care: the implementation of change in health care. 2nd. Chichester: Wiley; 2013. Quality and safety of care: the role of indicators; pp. 115–136. [Google Scholar]
- 75.Manojlovich M, Squires JE, Davies B, Graham ID. Hiding in plain sight: communication theory in implementation science. Implementation Science. 2015;10(58):1–11. doi: 10.1186/s13012-015-0244-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 76.Fraser SW. Accelerating the spread of good practice: a workbook for health care. West Sussex: Kingsham Press; 2002. [Google Scholar]
- 77.Norman N, Cowart S, Felzien M, Haynes C, Hernandez M, Rodriquez MP, et al. Testing to prevent colon cancer: how rural community members took on a community-based intervention. Annals of Family Medicine. 2013;11(6):568–570. doi: 10.1370/afm.1586. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 78.Straus SE, Brouwers M, Johnson D, Lavis JN, Légaré F, Majumdar SR, et al. Core competencies in the science and practice of knowledge translation: description of a Canadian strategic training initiative. Implementation Science. 2011;6(127):1–7. doi: 10.1186/1748-5908-6-127. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 79.National Institutes of Health . Dissemination and implementation research in health (R01) Bethesda: National Institutes of Health; 2016. [Google Scholar]
- 80.Powell BJ, Beidas RS. Advancing implementation research and practice in behavioral health systems: introduction to the special issue. Administration and Policy in Mental Health and Mental Health Services Research. 2016 doi: 10.1007/s10488-016-0762-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 81.Lal S, Urquhart R, Cornelissen E, Newman K, Van Eerd D, Powell BJ, Chan V. Trainees’ self-reported challenges in knowledge translation research and practice. Worldviews on Evidence-Based Nursing. 2015;12(6):348–354. doi: 10.1111/wvn.12118. [DOI] [PubMed] [Google Scholar]
- 82.Shea CM, Jacobs SR, Esserman DA, Bruce K, Weiner BJ. Organizational readiness for implementing change: a psychosocial assessment of a new measure. Implementation Science. 2014;9(7):1–15. doi: 10.1186/1748-5908-9-7. [DOI] [PMC free article] [PubMed] [Google Scholar]