Abstract
Background:
Community-engaged research (CEnR) is an approach to conducting research that actively involves both academic and community partners. Yet many academic researchers have limited knowledge of emerging science and processes for effectively engaging communities and community members are often subjects of research with limited knowledge and participation in the development and implementation of research.
Objectives:
The purpose of this article is to explore two CEnR research training programs, both funded by National Institutes of Health for the explicit purpose of facilitating translational science. South Carolina developed the initial program which served as a model for the Delaware program.
Methods:
Information is presented about how these two programs recruit, develop, and support academic and community partnerships, as well as how each utilizes mentorship, funding and structured training programs for successful CEnR with an emphasis on CBPR. The development of each program, the funding source, selection process, team requirements and expectations, educational content, evaluation and outcomes are described.
Results:
Both programs have increased the number and quality of community-engaged researchers, with 40 academic and community dyad partnerships participating in the training and successfully completing pilot projects. Evaluations reveal the development of effective academic-community partnerships for research with successful dissemination and return on investment ranging from $9.72 to $41.59 for each dollar invested in the projects.
Conclusions:
Research teams have demonstrated improvements in developing and utilizing CEnR and CBPR approaches. These intermediate measures of success demonstrate the need for similar programs that provide training, preparation, and support to those interested in CEnR.
Keywords: Community health partnerships, Education, Nonprofessional, Education, Sociology and Social Phenomena, Education, Professional, Education, Sociology and Social Phenomena, Teaching, Education, Sociology and Social Phenomena, Community health research
BACKGROUND
Need for education in community-engaged research
Community-engaged research (CEnR) is an approach for conducting research that requires partnership development, cooperation and negotiation, and usually a commitment to addressing health issues that are of interest to the impacted community. Ideally, community input is incorporated in the development of the research question, implementation of the research project, analysis of the results and/or dissemination of the findings to community stakeholders. Community engagement is an important element of the successful translation of research from bench to bedside and community1. CEnR is a framework or approach for conducting research, not a methodology in and of itself. The framework, referred to as CEnR, is defined as
“…the process of working collaboratively with and through groups of people affiliated by geographic proximity, special interest, or similar situations to address issues affecting the well-being of those people. It is a powerful vehicle for bringing about environmental and behavioral changes that will improve the health of the community and its members. It often involves partnerships and coalitions that help mobilize resources and influence systems, change relationships among partners, and serve as catalysts for changing policies, programs, and practices2.”
This approach is characterized by applying principles of community engagement and forming partnerships between communities. At the heart of all CEnR is the understanding that community members will be involved as partners in some meaningful ways in the research process3. When the voices of communities affected by the health issue are incorporated into research, the potential for developing and implementing successful interventions can be increased and the potential for dissemination within the community increases. CEnR opens the door to the insider perspective too often lacking in more traditional research structures, and training is needed to prepare most academic researchers to work effectively with communities and to solicit and incorporate community input in a culturally sensitive, respectful, and useful manner.
The National Institute of Health (NIH) has created a new impetus toward CEnR through an increase in funding mechanisms that require or at least support community partnerships and participation through its current focus on “translation” (i.e. turning research into practice by taking it from “the bench to the bedside and into the community”). The purpose of this article is to: 1) Describe current NIH infrastructure requirements to assist universities and their communities to improve CEnR; 2) To describe CEnR and community-based participatory research (CBPR) educational programs at the Medical University of South Carolina (MUSC) (South Carolina Clinical and Translational Research Institute) (SCTR)and University of Delaware Clinical and Translational Research (ACCEL Program); 3) Program accomplishments; and 4) Discuss collaborative process for building joint efforts to enhance training and support individuals to become independent community-engaged researchers.
NIH Infrastructure to Assist Universities and their Communities to Improve CEnR
Increasingly, community partnerships and participation are recognized as necessary for translating existing research to implement and sustain new health promotion programs, change clinical practice, improve population health, and reduce health disparities3. The NIH promotes clinical and translational research through numerous approaches and this article focuses on two programs currently building and improving the infrastructure to support research for improving health.
The Clinical and Translational Science Award (CTSA) from the National Center for Advancing Translational Sciences (NCATS) at the NIH “catalyzes the generation of innovative methods and technologies that will enhance the development, testing and implementation of diagnostics and therapeutics across a wide range of human diseases and conditions4.” The initiative is the primary example of an NIH-funded mechanism requiring a translational approach to the clinical research enterprise5. The purpose of the CTSA award is to assist academic institutions to create a uniquely transformative, novel, and integrative academic home for Clinical and Translational Science that has the resources to train and advance a cadre of well-trained multi- and interdisciplinary investigators and research teams with access to innovative research tools and information technologies to promote the application of new knowledge and techniques to patient care6. CTSA Program support enables research teams including scientists, patient advocacy organizations and community members to tackle system-wide scientific and operational problems in clinical and translational research that no one team can overcome. Program goals of the CTSA awards are to:
Train and cultivate the translational science workforce;
Engage patients and communities in every phase of the translational process;
Promote the integration of special and underserved populations in translational research across the human lifespan;
Innovate processes to increase the quality and efficiency of translational research, particularly of multisite trials; and
Advance the use of cutting-edge informatics6.
To ensure community engagement in the research process, research institutions must collaborate with community organizations to identify and understand public health needs. Through the CTSA Program, NCATS supports a broad range of activities that engage communities in health initiatives and clinical research. Working with federal and nonprofit agencies, CTSA Program hubs collaborate with public health professionals, health care providers, researchers and community-based groups to:
Develop methods of effective community dialogue and research.
Ensure that updated health information is widely available.
Provide information and access to clinical trials and studies.
Promote participation in clinical trials7.
MUSC received a CTSA award to establish the South Carolina Clinical and Translational Research Institute (SCTR). SCTR was the first identified CTSA program identified in the scientific literature to provide an organized program for jointly training dyads of academic and community partners to conduct CEnR and then providing grant funding to implement the proposed grant. MUSC first received a five-year award in 2009 and again in 2015 to further develop and support SCTR.
The other NIH award program focused on creating infrastructure for research is the Institutional Development Award (IDeA) Clinical and Translational Research (CTR) Program. Based on recent guidelines, the CTR is required to include community engagement in their activities. Specifically, the CTR “fosters health-related research and enhances the competitiveness of investigators at institutions located in states in which the aggregate success rate for applications to NIH has been historically low. The program also serves unique populations—such as rural and medically underserved communities—in these states8.” Specifically, the research strategy for the Community Engagement and Outreach Core must:
Identify priority health issues and concerns of communities/populations within the participating state(s).
Involve the community in setting research priorities that directly affect targeted communities/populations.
- Provide services and resources that will support investigators in conducting community engagement and outreach activities, as well as expertise and assistance on, but not limited to, the following:
- planning, implementing, evaluating, and disseminating effective preventions and interventions
- cultural sensitivity training for institutional clinical and translational researchers
- community and health care provider education and outreach
- establishing community advisory boards
- software development for facilitating collaboration with community practitioners
- strategies for communicating with and promoting participation of diverse populations and community groups
- recruitment and retention of research participants in clinical and translational research
- develop two-way communication with relevant community groups
- develop best practices for engaging various community members
Plans for integrating community engagement into leadership, research, and communication strategies of the CTR that includes clinicians, advocacy groups, and other community stakeholders. Plans for community engagement and participation must be made clear and significant beyond the mere inclusion of human subjects in research.
Provide educational and mentoring activities that will be available to investigators9.
Delaware was awarded a CTR known as Delaware Clinical and Translational Research (DE-CTR ACCEL) program in 2013 and partners include the University of Delaware, Nemours A.I. du Pont Children’s Hospital, Christiana Care Health System as well as MUSC. DE-CTR ACCEL was one of the first CTR programs to include an organized training program and grant funding for CEnR for academic and community researchers through their ACE Program. Funding was renewed in 2018 for an additional 5 years.
The focus of both the SCTR and DE-CTR ACCEL is creating an infrastructure at the participating institutions for developing and applying the science of translational research to improve research related to health care and health. Each program developed dedicated training and funding mechanisms for CEnR to specifically promote community-based participatory research (CBPR) by jointly training dyads of community and academic researchers on CEnR and specifically CBPR. CBPR is defined as:
“ a collaborative research approach that is designed to ensure and establish structures for participation by communities affected by the issue being studied, representatives of organizations, and researchers in all aspects of the research process to improve health and well-being through taking action, including social change…….CBPR involves:
Co-learning and reciprocal transfer of expertise, by all research partners, with particular emphasis on the issues that can be studied with CBPR methods.
Shared decision-making power.
Mutual ownership of the processes and products of the research enterprise10.”
CBPR was specifically emphasized in our trainings as it builds trust in communities, and increases shared knowledge and experiences between researchers and community leaders that lead to more culturally appropriate measurement, frameworks, and interventions10. And CBPR is reported by researchers as increasing research participation rates which is a challenge for many researchers10.
A review of peer-reviewed literature did not identify any similar programs for jointly training dyads of community and academic partners to work together as principal investigators on specific funded CBPR projects. However, the Community Campus Partnership for Health (CCPH) organization has consistently worked with both community and academic programs to promote CEnR and specifically CBPR11. Examples of other programs were identified that provided CEnR training to specific groups such as some that trained academic researchers (but not community members) to implement CEnR or CBPR programs in communities12 or graduate medical students to focus on CEnR13 or community members to participate as researchers14 while others funded CEnR projects and provided separate trainings for community and academic members interested in CEnR.15 One example of a program that published a description of extensive campus and community trainings and development of CEnR across campus (with a focus on faculty development) is the University of California16. Several of the academic and community leaders for the CES-P had previously participated in trainings offered by CCPH and that learning helped to design and improve CES-P. Additionally, the guidelines for partnership from CCPH was integrated into both CES-P and ACCEL ACE programs11. Informal discussions with other groups helped to increase our confidence but no specific information from the programs was integrated into CES-P. However, the CES-P program leaders at MUSC, along with their community partners, have consulted with several other universities, CTSAs and CTRs for development of programs for joint training of academic and community partners including a program at the University of Galway in Ireland that is currently adapting the CES-P program to meet their needs related to training dyads of patient and academic partners to improve care and health in specific groups of patients. Below we provide an overview of both programs focused on joint CBPR training of teams of community and academic partners who will implement a project focused on a jointly developed research project in a selected community, as well as program accomplishments, methods for collaboration across programs and future plans.
PROGRAM OVERVIEWS
MUSC’s Community Engaged Scholars Program (CES-P)
In 2009, MUSC developed and has maintained a Community Engaged Scholars Program with the overall goal of co-training and guiding academic and community members to collaboratively develop and implement research to ultimately improve health within and across South Carolina (SC) communities. The CES-P is one of the first CTSA-funded programs that developed a joint training and mentoring research program for academic faculty and community leaders and then provided funding for developing, implementing, and evaluating community-engaged research6, 7. It was highlighted in the Institute of Medicine Report on CTSAs as a novel and promising initiative17. The program was collaboratively developed by academic nurse leaders with strong ties to the community and community partners18 and included a) “intensive charrette training sponsored by the Campus Community Partnerships for Health18;” b) an analysis of the strengths, weaknesses, opportunities, and threats (SWOT analysis) across the institution and communities served by SCTR; c) a comprehensive review of existing programs and curricula for training in CEnR; d) evaluation of local needs; e) development of the curriculum; f) approval by community and academic boards; and g) funding and support from U.S. Army Medical Research and Material Command (USAMRMC) and the Telemedicine & Advanced Technology Research Center (TATRC), SCTR and the College of Nursing’s Center for Community Health Partnerships18. Both Community and Academic Advisory Boards of the University provided additional input and approved the program. The process for development, implementation and evaluation is further described by Andrews and colleagues19 and is based on CBPR principles and the local community and academic evidence of the gaps in research focused on communities. MUSC’s CES-P has been effectively funding and educating researchers since 2009. Each year the process includes a) A call for applications to the CES-P that is disseminated to both community organizations and colleges across the state; b) sharing and telephone conference call to review and answer questions about the application process (which includes i. title of project; ii. academic and community organizations and specific partners who serve as multiple principal investigators; iii. proposed research aims or questions and their long-term goals; iv. strategies for implementing research aims or answering the questions and how the application will be evaluated; v. description of partnerships ability to complete the study as well as “who will do what;” vi. a description of the environments, community, and resources; vii. references; viii. budget; ix. resumes or biosketches; x. memorandum of understanding including signatures and contact information of the principal investigators (PIs) and collaborating partners; and xi. supervisor consent form for each of the PIs.
Once applications are received by the CES-P leaders, the applications are jointly scored by academic and community leaders (from the SCTR Community Advisory Board and previous CES-P awardees) using a NIH type review. Those selected are notified and dates/times for the training of the teams are set through collaborative decisions of the awardees and the CES-P leaders. One of the first steps in the curriculum is to identify the educational needs of these awardees and then integrate these needs into the curriculum.
Awardee Learners and their needs:
The awardee learners are community and academic members who have identified a problem of interest to the community and are working together to identify a research question and formulate a research process for addressing the research question. Most have identified the need for both training in CEnR (and specifically CBPR) and pilot/feasibility funding. The curriculum is built on adult learning theory, specifically transformative learning20, a theory of learning that is uniquely adult, and grounded in the nature of developmental process and human communication. “Learning is understood as the process of using a prior interpretation to construe a new or revised interpretation of the meaning of one’s experience in order to guide future action20.” If selected as a CES-P scholar, the learning needs of the scholars are explored by qualitative discussions and quantitative survey by the team and learning needs are incorporated into the learning objectives. Each session incorporates the identified learning needs and a discussion of how to apply the content by the academic and community participants, as well as feedback from the participants. Based on ongoing discussions with the awardees, content and discussion are adapted or expanded to meet identified needs during the training program.
Current Process and Learning Objectives for Education and Training:
The program currently has broad-based input from many disciplines (nursing, public health, medicine, statistics and university and community leaders) and emphasizes team science and leadership for community engaged research at all levels of the socioecological model. The focus is on developing processes and tools that facilitate the education and partnering of community investigators with academic and clinical researchers in ways that enhance project effectiveness in fund solicitation, recruiting, and mentoring through the research process including dissemination of the project findings to the lay and scientific communities. Each year, both academic and community PIs return to share results and challenges with the new cohort of participants. The learning objectives are outlined in Table 1.
Table 1:
Community Engaged-Scholars Program (CES-P) and ACCEL Community Engaged Research (ACE) Awards Learning Objectives
| South Carolina Clinical and Translational Research Institute Community Engaged-Scholars Program (CES-P) Skill Development and Learning Objectives | Delaware Clinical and Translational Research ACCEL Community Engaged Research (ACE) Awards Skill Development and Learning Objectives |
|---|---|
By the end of the training, the Community and Academic Partners will be able to:
|
By the end of the training, the Community and Academic Partners will be able to:
|
| Number of Training Hours in Curriculum: 15 | Number of Training Hours in Curriculum: 15–26 |
The development of the evidence-based toolkit “Are We Ready” – A Toolkit for Academic-Community Partnerships in Preparation for Community-based Participatory Research (CBPR) – has guided Community Engaged Scholars through the process of developing and implementing effective partnerships for research and action21, funding, and pursuing research projects involving community investigators paired with academic and/or clinical researchers throughout the MUSC program’s history. The toolkit components guide the investigators in exploring “goodness of fit” of relationship, capacity of the partnership and project, and guidelines for partnership operations and research implementation. “Are We Ready” was utilized by the ACCEL Community Engagement (ACE) program (see below) in its instruction process as well. The toolkit may be downloaded from the SCTR website by completing a short form to describe potential use: https://research.musc.edu/resources/sctr/programs/community-engagement/engaged-scholars.
The CES Program at Medical University of South Carolina currently offers:
Competitive pilot funding for approved projects up to $10,000
Training program, required of dyads of academic and community investigators, intended to facilitate the development of academic/community partnerships to foster grant preparation and implementation skills for furthering additional future funding
Mentorship by both community and academic researchers/leaders
Linkages to all SCTR and community resources
The CES-P application process begins with an Announcement/Call for Proposals (CFP), and includes active recruitment and a phone conference for all interested/potential proposers. A Letter of Intent and a standard research proposal are then required. These submitted proposals are reviewed and scored by community and academic reviewers using guidelines for review of NIH grants. Scored proposals are further reviewed by both SCTR Community Advisory Board members and CEnR academic leaders as the proposals go through a final selection process. All project teams receive review comments and recommendations of their proposal. For those that are funded, contracts are exchanged between SCTR and project team and the 15-week (meeting 2 hours per week) training program begins. The sessions are led by both academic and community members. The academic members bring the science of community-based participatory research while community members bring the practical application of the science in the community. Currently teams may participate through our online learning system, but we are developing the CES-P as a mixed synchronous and asynchronous learning experience. The specific learning objectives for the CES-P have evolved over time with input from community and academic awardees, as well as SCTR’s Community Advisory Board members. An overview of the current learning objectives for CES-P dyads is shown in Table 2. During the training, the awardees identify their mentoring needs and are matched with both community and academic members who volunteer to help with learning as well as implementation and evaluation of their research. Some mentors may join the team (if invited) to help with future research. Also, SCTR provides free consultations related to research needs for the team, and all team members receive an ID that allows them to use all SCTR resources. The most commonly used resources are the library, statistical and IRB consultations and REDCap system for data capture.
Table 2.
Overview of CES-P Awards by Health Issue and Communities
| CES-P Awards funded by SCTR | ||||
|---|---|---|---|---|
| Health Issues | Population Group(s) | Community Setting(s) | Academic Investigator(s) | Community Organization Investigator(s) |
| Chronic disease prevention & management (n=15): | Church members | `Faith-based communities | Cancer Center Epidemiologist | Ministers |
| Cancer (n=2); HIV-AIDS (n=2); Obesity (n=7); Quality improvement; Dementia; Alpha-1; Diabetes & hypertension (n=2) Kidney dialysis |
Prisoners | Prisons | Biostatistician | HIV Minister |
| Children & Teachers | Title 1 schools | OB/Gyn MD | DAE Foundation Leader | |
| Providers & patients (n=2) | Rural health clinic | Family Medicine MD | Clinic Nurse | |
| Veterans | VA clinic and community | Nurse PhD | LowCountry AIDS Director | |
| Patients with dementia & caregivers | Respite care | Nurse PhD | Respite Care Director | |
| Alpha-1 persons | Online | Nurse and MD Pulmonologist | Alpha-1 Association leader | |
| Food bank leaders and users(n=2) | Community food banks | RD Public Health Nurse | Community Organization Leader | |
| Kidney dialysis patients | Dialysis clinic | Exercise physiologist | Nephrologist MD | |
| Underserved communities | Schools & food bank | Pediatrician MD | Food Bank RD | |
| Adolescents (n=2) | Pediatric primary care | Pediatrician MD | Community Pediatrician MD | |
| Patients & community residents | Hospital, university, community | Biologist PhD | Wellness Coordinator | |
| Wellness (n=8): | Teachers, staff & students | Charleston County schools | Pediatrician MD | School District Special Projects Leader |
| Overall wellness(n=2); Sexual Assault prevention; General prevention; Nutrition education healthy eating; Building trust; Obesity prevention (n=2) |
Youth | After school program | Psychologist | WINGS for Kids After School Leader |
| College students | Citadel | Psychologist | People Against Rape Leader | |
| Community members | Food Bank Senior Center | Pediatrician MD | Food Bank RD | |
| Older adults | Community | PhD RD | Council of Governments staff | |
| Hispanic-Latino adults | Childcare facilities | Psychologist | PASOs Leader | |
| Families | Churches & community groups | Health science professor | LiveWell Greenville Staff | |
| Rural adolescent community members | Community | PhD nurse | Town planner, health department health educator, AHEC leader | |
| Disabilities (n=3): | 3 County area | All | Nurse PhD Physical Therapist | Disability Resource Center leader |
| Physical | Children and adolescents (n=2) | Schools (n=2) | Teachers | |
| MD & Education faculty | New Beginnings leader | |||
| Pregnancy (n=1) | Patients | Clinic | Pediatrician | Federally Qualified Health Center Physician |
| Dental health (n=1) | Residents | Rural Community | Dentist | Town Mayor |
| Environmental health (n=1) | Lupus patients | Low-income community | MD | Lowcountry Alliance for Model Community Leader |
| ACE Awards funded by ACCEL | ||||
| Health Issues | Population Group(s) | Community Setting(s) | Academic Investigator(s) | Community Organization Investigator(s) |
| Wellness (n=3): | Adolescents | Schools | Sociologist Professor | Delaware Office of Education leader |
| Tobacco use; Pregnancy prevention; Healthy food |
Parents of adolescents | Clinics | Nemours Pediatrician MD | Children & Families staff |
| Persons visiting Zoo | Zoo | Social Policy PhD Professor | Delaware Zoological Society leader | |
| Chronic disease prevention and/or management (n=3): | Cancer survivors | Clinic | Behavioral Science Professor | Delaware Support Network staff |
| Physical activity and cancer; Resources for CHWs; Palliative care |
Community Health Workers (CHWs) | Primary Care Clinic and outreach to communities | Christiana Care Family Medicine Professor | Helen F. Graham Cancer Center Community Outreach & Education staff |
| Homebound patients | Christiana Care patients living in communities | Geriatrician Christiana Care | Visiting Nurse Association nurse | |
| Disabilities (n=3): | Young adults | Community | Behavioral Health Professor | Endless Possibilities in the Community leader |
| Intellectual; Autism; Sensory motor; |
Chldren with Autism | Clinic | Psychologist | Delaware Guidance leader |
| Children with Cerebral Palsy (CP) | School | Physical Therapy Professor | HMS School for Children with CP | |
| Environmental Health (n=1) | Community members | Low Resource Neighborhoods | Nurse PhD | Low Country Alliance for Model Communities leader |
Following the training, the team updates their grants to incorporate feedback from the reviews, learning from the training, adjusts their budgets to share funding based on workload, and obtains approval by their Institutional Review Board (IRB). Once their application is updated and approved by both SCTR’s Community Advisory and their Executive Boards and IRB, the funded projects begin. Over the course of the award year, academic and community mentors as well as CES-P leaders continue guiding the researchers through regular required telephone or onsite meetings to give a sounding board for challenges encountered and solutions to resolve them. All teams are expected to disseminate findings to both communities participating in the research as well as to professional groups either through posters, presentations or scientific publications. Evaluation of progress is reported at the midpoint of the award year and then annually to track publications and other grants over the 5-year period.
ACCEL’s Community Engaged Scholars Program
Delaware’s ACCEL Program worked collaboratively with MUSC’s CES-P leaders to develop the ACE Awards Program. The ACE Award Program was designed with the CES-P model, as a base to promote efforts to accelerate clinical and translational research goals. This unique partnership of the two groups has led to further collaborations, innovations and the growth of both training programs over the past few years.
Awardee Learners and their needs:
The learners and their needs are essentially the same as those for MUSC: community-academic partners/teams across Delaware with a shared interest and a mutual CEnR question.
History and Learning Objectives for Training and Education:
DE-CTR ACCEL developed the ACE awards model in 2014 under the leadership of a physician, educator, and researcher with a passion for clinical research aimed at addressing health care needs in a way to translate study results into policy and/or practices that benefit members of Delaware communities. Because of the many roles held by CEnR advocates such as the lead of the Community Engagement and Outreach (CEO) core of DE-CTR ACCEL and a member of multiple Institutional Review Boards, leadership had a unique perspective on community engaged research and firsthand knowledge of how critical it is for community engaged research to be conducted by both academicians and community stakeholders working hand in hand. An overview of the learning objectives is shown in Table 1.
To forward this community-based research in a successful manner, with sensitivity for the community studied and affected, the ACE Program requires the academic and community partners to attend a training curriculum designed to maximize their chance for quality interaction in the community throughout the study. Throughout the four years of developing community researcher teams and three years of funding projects, the ACE curriculum has been flexible and growing in response to feedback from MUSC CES-P leader, course instructors and attendees. Pre and post surveys were distributed to attendees of the curriculum to ensure feedback is obtained and incorporated in future years.
The ACE program offers:
Competitive pilot funding up to $20,000 per year
Training program curriculum (offered both in-person and through video-streaming) based on CES-P
Guidance through the IRB approval process, as necessary
Mentorship
Access to all resources across the DE-CTR ACCEL program
The application process is very similar to the CES-P and follows the standard announcement/call for proposals, and includes active recruitment and a meeting for all interested/potential proposers. A letter of intent and a standard research proposal are required. These proposals are all reviewed and scored individually by at least one experienced community reviewer and one experienced researcher reviewer. Scored proposals are gathered and discussed in a formal meeting of all reviewers as they together go through a final selection process. Much like CES-P at MUSC, all project teams receive a notification whether selected for award or not which includes review comments and recommendations involving their proposal.
Once selected, contracts are exchanged between the program and project team and the training begins. The ACE curriculum included between five and eight weekly classes around 2–3 hours each. In the initial year, all materials were delivered in a live format i.e. mandatory in-person (or live via video streaming) lectures for both academic and community investigators. In response to feedback, didactics were condensed into webinars that can be viewed remotely in preparation for live sessions, to minimize live attendance requirements. In-person classes review supplemental webinars and focus on applying the lessons on the research projects awarded a tailored format e.g. teams may watch a brief webinar in advance on study design and then receive feedback in class on their specific proposal from epidemiologists and statistician. About half of the live training is team building based on “Are We Ready” toolkit21. The ACE curriculum added a focus on grant writing skills, how to write specific aims, and the specific aims page. The ACE curriculum has live consultations with institutional review boards.
Like the CES-Program, projects are submitted to the IRB for approval by the end of the course and the projects begin soon afterwards. Also similar to CES-P, required 6-month and 1-year progress reports capture updated research work including preliminary results, planned presentations and publications, and budget spending.
The ACE Program has been successful thus far in launching intriguing and important projects that address the needs of many different communities affected. Investigator teams have fielded ten research studies managed by twenty-two trained investigators including twelve community members. Table 2 summarizes the health conditions addressed, the involved participants and their communities, the academic and community principal investigators, as well as community organizations. The top three priorities for both CES-P and ACE centered around prevention and management of chronic conditions, wellness, and disabilities while the most common sites were geographic communities and organizations in the communities, various clinics and their patients, schools/youth facilities and students and their parents. However, the health priorities were diverse ranging from prevention to palliative care, and the sites and participants included all age groups, and community sites ranged from online communities across the United States to local faith-based communities. Physician and nurse faculty were the more common academic principal investigators, while community principal investigators were from government, schools, clinics, faith-based groups, service organizations and even the local zoo. Approximately half of the studies focused on developing and testing interventions while others focused on assessment and planning for future intervention development.
Evaluation and Results of CES-P and ACE Programs
Program evaluation includes evaluation of each of the sessions and evaluation of the overall program using the RE-AIM model22 (Reach, Effectiveness, Adoption, Implementation, and Maintenance) and an analysis of return on investment (ROI). Some components of the RE-AIM are evaluated through a more formal evaluation process while others still need improvements to capture data. One of the goals for the recently received CTSA and CTR grants is to improve the evaluation component for CES-P and ACE.
Table 3 describes the number of awards and funding by year for the two programs (Reach). CES-P, with a longer 9-year history had a greater number of awards, 30 compared with 11 awards (over 3 years) in ACE program. In addition, the smaller award amount for CES-P ($5-$10K) also allowed for a greater volume of awards. However, CES-P received feedback that smaller awards limited community participation and after year 1, set their award amount to $10K (as opposed to $5K). Aware of this challenge in the CES-P program, the ACE awards began at a higher funding level, $20K from the start and has plans to further increase funding during the new grant award.
Table 3:
Number of Awards per Year and Total Funding for South Carolina Clinical and Translational Research Institute (SCTR) Community Engaged-Scholars Program (CES-P) and Delaware-CTR ACCEL Community Engaged Research (ACE) Awards
| Year of Funding | MUSC/SCTR CES-P Awards | Delaware-CTR ACCEL ACE Awards | ||
|---|---|---|---|---|
| # Team Awards | Total Award $ | # Team Awards | Total Award $ | |
| 2009–10 | 6 | $30,000 | ||
| 2010–11 | 5 | $50,000 | ||
| 2011–12 | 3 | $30,000 | ||
| 2013–14 | 4 | $40,000 | ||
| 2014–15 | 4 | $40,000 | 4 | $80,000 |
| 2015–16 | 2 | $20,000 | 5* | $100,000 |
| 2016–17 | 4 | $40,000 | 2 | $40,000 |
| 2017–18 | 2 | $20,000 | ||
| 30 | $270,000 | 11 | $220,000 | |
Competitive Renewal and Funding for 1 awardee from 2014–15 to continue in 2015–16
Effectiveness of each educational session is evaluated on a 5-pont Likert scale by participants and most report above average scores for effectively sharing information that facilitates learning related to CEnR and specifically their CBPR projects. Additionally, the participants qualitatively report suggestions for improvements of the educational sessions. Some of the suggestions include: video of each session so if unable to make the class, they can watch the video; identifying mentors early so that the mentor and research team can work together to refine the grant; more time for discussion of material; class review and feedback on all grants prior to refining the grant during the curriculum; assistance and consultation related to IRB and analysis of quantitative data. All of these suggestions have been addressed. Not all participants choose to evaluate each session, but of those who evaluate the sessions, the average scores are above average to excellent for at least 90% of the sessions. Of those with low scores, the lecturer and the content are carefully reviewed and improvements implemented.
Adoption and Implementation of educational content is captured as each of the PIs use the information from the educational sessions to improve their grants and apply the principles of CBPR during implementation of their research projects. Additionally, other PIs from the community and the academic settings review the grants and make suggestions for improvements. Most participants work to adopt the content in their grant proposals and report plans to adopt and integrate their learning while working to collect, analyze and report their data and outcomes in both community and academic settings. The implementation is evaluated in reports, posters, and sharing of processes and outcomes.
Maintenance of their research with their communities is largely assessed through ROI or reports of continuing research funding. Table 4 shows the crude data used to calculate return on investment (ROI) for each award program. Both ROI, $41.59 for CES-P and $9.72 for ACE are favorable and again differences seem reasonable given the differences in award amounts and the relative “newness” of the ACE program.
Table 4:
Community Engaged-Scholars Program (CES-P) and Delaware-CTR ACCEL Community Engaged Research (ACE) Awards Summary Program Outcomes to-date
| Award Type | Total # of Awards | Total Amount of External Funding Received | Return on Investment (ROI) (in dollars)* | Total # of Publications | Total # of Presentations |
|---|---|---|---|---|---|
| CES-P | 24* | $11,500,000 | $41.59* | 12 | 25 |
| ACE | 9* | $1,749,170 | $9.72* | 2 | 34 |
2016–2018 Awardees are not included in program outcomes ROI analysis as these teams are just now completing initial funding awards and applying for additional funding. ROI = (grant funding after the initial grant by SCTR CES-P or ACCEL ACE - initial grant funding by SCTR CES-P or ACCEL ACE) / initial grant funding by SCTR CES-P or ACCEL ACE
Figure 1 shows CES-P and ACE’s dissemination outcomes to the scientific communities: presentations and publications for each program. ACE awardees had highest levels of local/regional and national presentations and relatively fewer peer-reviewed publications, commensurate with a new program. The CES-P, a well-developed program, has impressive number of presentations (local, national, international) and publications. Additionally, the dual PIs share their findings with their respective communities
Figure 1: Dissemination Comparison of Academic Presentations and Publication CES-P and ACE Programs*.
* Evaluation done in 2017
Each funded project is asked or encouraged to disseminate outcomes to the participating communities. Many outcomes reflect the integrated use of the RE-AIM framework for evaluation of their results. The dissemination processes that were informally reported by the PIs varied. Some examples include presentations summarizing their research finding for their communities and particularly their participants, newspaper and online articles and photos focused on their research findings, and community photovoice exhibit. One group developed an indepth photovoice book that was taken by the participants to the SC Legislature as a tool for assisting the participants to persuade the Legislature to change laws related to parking for persons with disabilities. Their actions led to a change in SC law that set and enforced high fines for those who use handicapped parking spaces but do not have a handicapped tag for their car. Another photovoice project showed the community members how to improve healthy eating habits and another provided input on how to improve foods distributed by food banks for hypertension and diabetes. These led to improvements in healthy foods served in churches and healthier food packages for those with reported health issues by the food banks. In addition to the photovoice projects, many visited community groups that participated in the project and did oral presentations related to findings or shared handouts on findings. Findings were informally reported across projects. One project resulted in the community and the academic institution setting up a dental clinic offering free services to those unable to pay for dental care, while others resulted in practice changes in clinics, colleges, service organizations, food banks and healthier menus at daycare and churches. Only four projects have reported no outcomes and one is working on issues with the Institutional Review Board and may not be able to use the data that have been collected due to community issues with data collection.
DISCUSSION
Both educational programs implemented a well-reviewed and effective education program and success is likely due to characteristics which were shared and consistent between the program courses, as well as the highly motivated PIs. Both programs were highly tailored to the community and academic learners needs and focused on the projects proposed. Both CES-P and ACE were responsive to feedback to optimize and streamline face-to-face group learning, thereby reducing burden of time for participants. Both programs required attendance and IRB submission as incentive for funding release at the end of education. Both strongly emphasized the team-building skills required for CBPR throughout the use of the “Are We Ready” toolkit21. Both CES-P and ACE were effective as evidenced by products of dissemination and return on investment. The difference in dissemination products and ROI likely reflect the differences between an early program and a more developed program. So, early programs should expect ROI outcomes more compatible with ACE while aiming for long term productivity of CES-P.
Both programs had good dissemination results with CES-P showing excellent outcomes consistent with a long-standing, mature grant educational program. ACE results are very promising, as well. Funding levels were different, deliberately higher in ACE to respond to barriers discovered in CES-P and to attract participation of healthcare providers. ACE was conceived within a CTR without a medical school and led by a physician researcher. Health care providers are only able to participate in research with the support of clinical leadership. To garner support for the amount of time to complete this type of curriculum and project, the value of the work itself and dollar support must be sufficiently high.
We measured ROI as a surrogate for accomplishment of the intended outcome of external funding. Although this is not a formal cost analysis, the data trends are positive and promising. One limitation is that the measurements are cross-sectional over time rather than truly prospective; thus, funding obtained is not necessarily casually linked to participation. However, this is a meaningful measure to report especially when considering the need to always justify programs as sustainable both to internal and external stakeholders. True outcomes, for example, new academic-community connections that lead to products of dissemination and are sustained over time, are harder to measure. But we have created web plots of collaborative connections and can show increasing connectivity across our programs and will in future work begin to categorize and code these connections in terms of community partners. Most importantly, the PIs have shared self-reported outcomes across multiple communities, and sustainability and continued collaboration across multiple projects.
Future work of both CES-P and ACE include a more indepth evaluation of the programs and their contributions to improving the health and well-being of communities. Although self-reported outcomes by the PIs are reported, a continuing more formal ongoing evaluation is indicated including:
Identification of a more formalized process for reporting community priorities, conveying these priorities to academic researchers, and recruitment of both community and academic members to jointly work together as PIs to improve community outcomes.
Methods for working with under-resourced communities to more fully participate in the application development.
Assessment of each educational session and the ongoing suggestions and integration of improvements, as well as identifying the barriers and facilitators to learning and specifically to collaboration, applying the learning in selected communities and integrating this learning into programs.
A formalized process for both evaluation through the educational program that is tied to outcomes and identification of research findings and results from each of the funded projects as well as their continued work with communities and application of learning with other communities.
Identification of ongoing institutional support and sustainability of funding for both programs.
Other creative approaches to linking community members and priorities with academic researchers.
ACE, the younger CTR-based program, will be part of the larger ACCEL pilot grants program ($80K) with the express goals of continuing CEnR and CBPR, attracting health care providers and promoting more clinical science. Based on knowledge that these projects consistently need more time, the spending period was increased to 2 years. Collectively, the CES-P and ACE programs leaders recognize a need to integrate CBPR and CEnR more ubiquitously throughout the CTSA and CTR. Therefore, leaders are working collectively to provide this education remotely by creating online toolkits and developing online learning platforms. These tools for implementing the educational trainings are currently available and can be disseminated to other CTR’s and CTSAs since many programs across the United States are implementing community engagement, particularly as this has become a requirement in CTSAs and IDeA CTRs6,7,9. Program leaders are also actively developing competencies for CBPR and working with other CTRs and CTSAs to disseminate knowledge regarding how these programs have been and can be an effective tool for engaging communities to advance translational science and improve health, especially in vulnerable populations.
FUNDING:
Funding has been received for the CES-P and ACE Awards from:
CTSA from the NIH National Center for Advancing Translational Sciences (NCATS) through Grant Number UL1 TR001450 (Jenkins, Burshell) to the Medical University of South Carolina
Institutional Development Award (IDeA) from the National Institute of General Medical Sciences of the National Institutes of Health under grant number U54-GM104941 (Bittner-Fagin, Harrington, Pasarella, Jenkins) to the University of Delaware
Footnotes
RESEARCH ETHICAL APPROVALS:
Institutional Review Boards: Each of the participants in the CES-P and ACE Awards have received Institutional Review Board review and approval prior to initiating their research study.
REFERENCES
- 1.Handley M, Pasick R, Potter M, Oliva G, Goldstein E, Nguyen T. (2010). Community-Engaged Research: A Quick-Start Guide for Researchers. From the Series: UCSF Clinical and Translational Science Institute (CTSI) Resource Manuals and Guides to Community-Engaged Research, Fleisher P, ed. Published byClinical Translational Science Institute Community Engagement Program, University of California San Francisco. http://ctsi.ucsf.edu/files/CE/guide_for_researchers.pdf [Google Scholar]
- 2.Center for Disease Control and Prevention (CDC). (1997). Community engagement: Definitions and organizing concepts from the literature. http://www.cdc.gov/phppo/pce/
- 3.Center for Disease Control and Prevention (CDC). (June2011). Principles of community engagement. Second edition. http://www.atsdr.cdc.gov/communityengagement/pdf/PCE_Report_508_FINAL.pdf
- 4.Collins FS (2011). Reengineering Translational Science: The Time is Right. Science Translational Medicine, 3(90). doi: 10.1126/scitranslmed.3002747 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Horowitz C, Robinson M, Seifer S. (2009). Community-based participatory research from the margin to the mainstream: are researchers prepared? Circulation, 119(19), 2633–2642. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Department of Health and Human Services. National Institutes of Health. Institutional Clinical and Translational Science Award. Request for Applications: RFA-RM-07-007. https://grants.nih.gov/grants/guide/rfa-files/RFA-RM-07-007.html
- 7.Department of Health and Human Services. National Institutes of Health. Institutional Clinical and Translational Science Award. Request for Applications: PAR-18-940. https://grants.nih.gov/grants/guide/pa-files/PAR-18-940.html
- 8.Department of Health and Human Services. National Institute of General Medical Sciences. Institutional Development Award. Retrieved from https://www.nigms.nih.gov/Research/DRCB/IDeA/Pages/default.aspx
- 9.Department of Health and Human Services. National Institutes of Health. Institutional Development Award (IDeA) Program Infrastructure for Clinical and Translational Research (IDeA-CTR)(U54). PAR-17-304. Retrieved from https://grants.nih.gov/grants/guide/pa-files/PAR-17-304.html
- 10.Viswanathan M, Ammerman A, Eng E, Garlehner G, Lohr KN, Griffith D, … & Webb L (2004). Community-based participatory research: Assessing the evidence: Summary. In AHRQ evidence report summaries. Agency for Healthcare Research and Quality (US). [PMC free article] [PubMed] [Google Scholar]
- 11.Community Campus Partnerships for Health (ND). https://www.ccph.info [PubMed]
- 12.Coffey J, Huff-Davis A, Lindsey C, Norman O, Curtis H, Criner C, & Stewart MK (2017). The development of a community engagement workshop: a community-led approach for building researcher capacity. Progress in community health partnerships: research, education, and action, 11(3), 321–329. [DOI] [PubMed] [Google Scholar]
- 13.DeHaven MJ, Gimpel NE, Dallo FJ, & Billmeier TM (2011). Reaching the underserved through community-based participatory research and service learning: description and evaluation of a unique medical student training program. Journal of Public Health Management and Practice, 17(4), 363–368. [DOI] [PubMed] [Google Scholar]
- 14.Rubin CL, Martinez LS, Chu J, Hacker K, Brugge D, Pirie A, … & Leslie LK (2012). Community-engaged pedagogy: a strengths-based approach to involving diverse stakeholders in research partnerships. Progress in community health partnerships: research, education, and action, 6(4), 481. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Allen ML, Culhane-Pera KA, Pergament S, & Call KT (2011). A capacity building program to promote CBPR partnerships between academic researchers and community members. Clinical and translational science, 4(6), 428–433. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.DeLugan RM, Roussos S, & Skram G (2014). Linking academic and community guidelines for community-engaged scholarship. Journal of Higher Education Outreach and Engagement, 18(1), 155–168. [Google Scholar]
- 17.Institute of Medicine. (2013). The CTSA Program at NIH: Opportunities for Advancing Clinical and Translational Research. Washington, DC: The National Academies Press. doi: 10.17226/18323. [DOI] [PubMed] [Google Scholar]
- 18.Andrews JO, Cox MJ, Newman SD, Gillenwater G, Warner G, Winkler JA, & Slaughter S (2013). Training partnership dyads for community-based participatory research: Strategies and lessons learned from the community engaged scholars program. Health promotion practice, 14(4), 524–533. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Andrews JO, Newman SD, Meadows O, Cox MJ, & Bunting S (2010). Partnership readiness for community-based participatory research. Health education research, 27(4), 555–571. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Mezirow J (1996). Contemporary paradigms of learning. Adult Education Quarterly, 50, 5–23. [Google Scholar]
- 21.Andrews JO, Cox MJ, Newman SD, & Meadows O (2011). Development and evaluation of a toolkit to assess partnership readiness for community-based participatory research. Progress in Community Health Partnerships, 5(2), 183. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Sweet SN, Ginis KAM, Estabrooks PA, & Latimer-Cheung AE (2014). Operationalizing the RE-AIM framework to evaluate the impact of multi-sector partnerships. Implementation Science, 9(1), 74. [DOI] [PMC free article] [PubMed] [Google Scholar]