Abstract
Introduction:
The extent to which Clinical and Translational Science Award (CTSA) programs offer publicly accessible online resources for training in community-engaged research (CEnR) core competencies is unknown. This study cataloged publicly accessible online CEnR resources from CTSAs and mapped resources to CEnR core competency domains.
Methods:
Following a search and review of the current literature regarding CEnR competencies, CEnR core competency domains were identified and defined. A systematic review of publicly accessible online CEnR resources from all 64 current CTSAs was conducted between July 2018 and May 2019. Resource content was independently reviewed by two reviewers and scored for the inclusion of each CEnR core competency domain. Domain scores across all resources were assessed using descriptive statistics.
Results:
Eight CEnR core competency domains were identified. Overall, 214 CEnR resources publicly accessible online from 35 CTSAs were eligible for review. Scoring discrepancies for at least one domain within a resource initially occurred in 51% of resources. “CEnR methods” (50.5%) and “Knowledge and relationships with communities” (40.2%) were the most frequently addressed domains, while “CEnR program evaluation” (12.1%) and “Dissemination and advocacy” (11.2%) were the least frequently addressed domains. Additionally, challenges were noted in navigating CTSA websites to access CEnR resources, and CEnR competency nomenclature was not standardized.
Conclusions:
Our findings guide CEnR stakeholders to identify publicly accessible online resources and gaps to address in CEnR resource development. Standardized nomenclature for CEnR competency is needed for effective CEnR resource classification. Uniform organization of CTSA websites may maximize navigability.
Keywords: Community-engaged research, community engagement, stakeholder engagement, training, education
Introduction
Community-engaged research (CEnR) involves community stakeholders and researchers collaborating to co-design, implement, evaluate, and disseminate clinical and translational research that is culturally appropriate, valued, efficient, and effective [1]. CEnR is an important avenue for both researchers and community stakeholders to generate and disseminate knowledge in order to improve the health and well-being of patients and communities [2, 3]. There has been increased interest among academic institutions and community stakeholders to conduct CEnR. Supported by the National Institutes of Health, the Clinical and Translational Science Award (CTSA) program currently funds 64 institutions across the USA to further the progress of translational research [4]. One of the primary goals of the CTSA program is to foster CEnR by engaging patients and communities in every phase of the translational science and research processes [5, 6].
Establishing successful community–academic partnerships in research requires training, skills, and resources. With community engagement as a core component of all CTSA institutions, much of the work done across CTSAs in CEnR involves the development, testing, and dissemination of trainings and resources to enhance CEnR skills and knowledge among researcher and stakeholder populations, with the end goal of ensuring more successful community–academic partnerships. For more communities and researchers to effectively collaborate in CEnR, increasing access to quality CEnR education and training is crucial [1]. Therefore, the availability of resources online that are readily accessible to researchers and communities and contain accurate, up-to-date content is an essential part of the continued success of CEnR [7]. To this point, there is a focus by the National Center for Advancing Translational Sciences (NCATS) on shared resources and information to help advance CEnR [8]. However, the extent to which existing CEnR resources are publicly accessible online through CTSAs, and how these resources address CEnR core competencies is unclear.
The purpose of this study was to understand the extent to which CTSA programs offer publicly accessible online resources for training in CEnR core competencies. In this study, the term CEnR was used to refer to all forms of related research (e.g., community-based participatory research) [1]. The study purpose was addressed by examining the following aims: (1) identify all CEnR resources currently offered by CTSAs, with a focus on publicly accessible online resources and (2) systematically review publicly accessible online CEnR resources for whether resource content addressed identified CEnR core competency domains.
Materials and Methods
Study Design
Literature specific to CEnR education and training competencies was used to identify CEnR core competency domains. In April 2018, a search of PubMed was performed using the following search terms: community-based participatory research, community engagement, and competency. We also searched references cited in selected articles. An article was included if the primary focus was discussing competencies for training and performing CEnR. A total of eight articles were included for the final review. Common competencies and key attributes discussed in these articles were synthesized to identify core domains based on the literature [2, 6, 8–13]. Definitions and key aspects of domains were then agreed upon by research team members. The final eight CEnR core competency domains included: knowledge and perceptions of CEnR; personal traits necessary for CEnR; knowledge and relationships with communities; training of those involved in CEnR; CEnR methods; CEnR program evaluation; resource sharing and communication; and dissemination and advocacy (Table 1).
Table 1.
CEnR core competency domain definitions and characteristics
| Knowledge and perceptions of CEnR |
Basic principles and concepts integral to understanding and performing CEnR, including:
|
| Personal traits necessary for CEnR |
Personal attributes essential to successful CEnR participation, including:
|
| Knowledge and relationships with communities |
Stakeholder relationships and their involvement in CEnR, including:
|
| Training of those involved in CEnR |
Ability of academic–community partners to successfully incorporate new stakeholders in CEnR project and processes, including:
|
| CEnR methods |
Technical components and skills required to perform CEnR, including:
|
| CEnR program evaluation |
|
| Resource sharing and communication |
Facilitating equitable sharing of funding, resources, and credit involved in CEnR projects, including:
|
| Dissemination and advocacy |
Methods for effectively communicating CEnR results to leverage positive change for key stakeholders are the following:
|
CEnR, community-engaged research.
We then conducted a systematic review of CEnR resources publicly accessible online between July 2018 and May 2019 from the websites of all 64 currently funded CTSAs to identify CEnR-specific resources. Resource characteristics were recorded including institution name, link to resource, ability to download resource, intended audience for resource, and target population of interest for a resource. Resources were included for review and analysis if all components of the resource were fully available online; they were primarily intended for education or training in performing CEnR; they were directly developed, co-developed, or sponsored by a CTSA; and they were directly available through a CTSA website. A CEnR resource was determined to be publicly accessible online if all components of the resource were available online. Resources were excluded if the website used to access the resource required the study team to request information or permission from the host CTSA in order to access the resource, and that access was not received within 5 days; the resource required in-person training, or blended online components with off-line formats, such as in-person training; or, the resource was not intended for CEnR education and training.
Overall, 647 CEnR resources available between July 2018 and May 2019 from 64 CTSAs were identified. Of those, 376 resources were identified as being publicly accessible online. A total of 214 of 647 (33.1%) publicly accessible online CEnR resources from 35 of 64 (54.7%) CTSAs met the criteria for inclusion in review (Fig. 1) [14-48]. Most resources reviewed were in the text (i.e., downloadable literature, PowerPoints, etc.), webinar, or recorded lecture format.
Fig. 1.
Flowchart for the inclusion of publicly accessible online resources for CEnR.
CEnR, community-engaged research; CTSA, Clinical and Translational Science Award.
For each resource, the CEnR core competency domains addressed were evaluated. For each resource included in the final sample, the content was independently reviewed by two reviewers and subsequently scored by the extent to which it addressed each CEnR core competency domain using a dichotomous score as follows: (1) Addressed – the domain was the main topic discussed in the resource, or was substantively discussed, but was not the main focus of the resource or (2) Not addressed – minimally discussed (i.e., only discussed in a few sentences or bullet points) or not discussed at all. Reviewers’ domain scores for the same resource across reviewers were then compared. To resolve scoring discrepancies, reviewers of the resource convened to reach an agreement on the final domain score.
Finally, reviewers were asked to keep field notes regarding observations made in accessing and reviewing resources. These notes were then compared and discussed by the study team to determine what, if any, common themes were noted among reviewers.
IRB approval for this study was not required.
Statistical Analysis
Domain scores for study sample resources were aggregated and analyzed with descriptive statistics (frequencies and percentages) to assess the extent to which each of the eight identified CEnR core competency domains was addressed across resources.
Results
Of the 214 resources included for review, 60% (n = 130) were directly downloadable documents. The remaining 40% required additional steps to access the resource (i.e., a video played on YouTube or an interactive module that could not be downloaded) and the resource itself was “housed” online.
The intended audiences for the resources reviewed included community partners (33%; n = 71), academic partners (62%; n = 134), and clinician partners (8%; n = 18). For 13% (n = 29) of resources, the intended audience was unclear. About 12% (n = 27) of resources discussed the conduct of CEnR with specific communities or populations, including Latinx communities, African-Americans, and military veterans.
The CEnR core competency domains most commonly addressed in the study sample of resources were “CEnR methods” (50.5%; n = 108) and “Knowledge and relationships with communities” (40.2%; n = 86). The CEnR core competency domains least commonly addressed in the study sample of resources were “Dissemination and advocacy” (11.2%; n = 24) and “CEnR program evaluation” (12.1%; n = 26). Further details regarding the percentage of core domains addressed across all sample resources are provided in Fig. 2.
Fig. 2.
Percentage of core domains addressed across all sample resources by domain.
CEnR, community-engaged research.
Comparison of reviewers’ field notes revealed that a lack of standardized CEnR nomenclature became problematic when reviewing and scoring resources. Scoring discrepancies for at least one domain within a resource initially occurred in 50.9% (n = 109) of resources. All scoring discrepancies were resolved. Of the resources in which scoring discrepancies initially occurred, 71.6% (n = 78) were ultimately determined to address the domain in question, while 28.4% (n = 31) were ultimately determined to not address the domain in question. The core domains with the most observed initial scoring discrepancies included “Knowledge and perceptions of CEnR” (20.2%; n = 22) and “Personal traits necessary for CEnR” (18.3%; n = 20). The core domains with the fewest initial scoring discrepancies were “CEnR program evaluation” (3.7%; n = 4) and “Dissemination and advocacy” (9.2%; n = 10). Further details regarding initial reviewer discrepancies in scoring the extent to which domains were addressed within resources are provided in Fig. 3.
Fig. 3.
Initial reviewer discrepancies scoring the extent to which domains were addressed within resources by percentage.
CEnR, community-engaged research.
Additionally, the comparison of reviewers’ field notes revealed CTSA website navigability played a major role in excluding potential resources from the analysis. Issues related to the navigability of CTSA websites that were noted by reviewers included: inconsistent organization and nomenclature related to labeling CEnR resources, convoluted hyperlinks and website organization to access CEnR resources, and restricted access to CEnR resources (e.g., needed faculty/staff credentials to access resources).
Discussion
Engagement of communities across the continuum of translational research is vital in ensuring that such research meaningfully improves the health and well-being of those communities [49]. To accomplish this, community and academic partners must understand all aspects of CEnR so that community partners are able to understand and actively contribute to all phases of the research process. Education and training in every domain of CEnR core competency are then essential for both communities and researchers to ensure that all stakeholders are competent and comfortable contributing to CEnR.
Through this study, we generated a comprehensive inventory of CEnR resources that are publicly accessible online to researchers and other stakeholders interested in learning about and conducting CEnR. Our comprehensive review of these resources elucidated the CEnR core competency domains that are addressed in currently available resources. Results suggest that CTSAs provide numerous resources specific to CEnR through online formats. “CEnR methods” and “Knowledge and relationships with communities” were the most commonly addressed domains by these online resources. Our findings also indicate that major gaps exist in resources addressing the domains of “CEnR program evaluation” and “Dissemination and advocacy.” Furthermore, the domains of “Knowledge and perceptions of CEnR” and “Personal traits necessary for CEnR” were difficult to initially score with consistency. Finally, considering that only 33.1% of CEnR resources reviewed were publicly accessible online, and those resources only came from 54.7% of CTSAs, this demonstrates that there is a need for CTSAs to substantially improve public access to online CEnR resources.
Several challenges pertaining to accessing and classifying publicly accessible online CEnR resources were noted. First, the original source of the materials was not always clear to the reviewer. While many tools were created by the institution hosting them on their website, some resources were shared from other sources and thus redundant between sites. In the future, it will be useful for tools to be identified with the date of creation and source institution(s) so stakeholders can identify content developers and content modifications to facilitate adaptation and dissemination of CEnR resources. Second, we encountered challenges in successfully navigating CTSA websites to access CEnR resources. Marked differences were observed in how CTSA websites were structured to house and allow access to CEnR resources which led to challenges in navigation. Third, several CTSA websites offered CEnR resources by providing “hyperlinks to hyperlinks,” where instead of the resource being directly available through the CTSA website, one would have to follow several hyperlinks in order to locate the original resource of interest. Even among our trained reviewers and study team, efforts to systematically follow these hyperlinks became time-consuming and confusing. In order to maximize the efficient use of online CEnR resources, clear guidelines for formatting access to materials on CTSA sites to be shared by multiple stakeholders are recommended to ensure minimal navigability issues for all potential stakeholders. Areas of website navigability have been previously identified, and include: “the clarity with which the target of a navigational element is described by that element (clarity of target), the clarity with which a navigational element conveys the underlying structure of site information (clarity of structure), and the degree to which the site content is appropriately subdivided or hierarchically organized with respect to the relationships between the content sections (logic of structure).” [50].
Finally, as our team reviewed resources, we found that the CEnR core competency domains we identified based on our review of CEnR competency literature did not adequately cover resource content concerning social determinants of health, cultural competency, and other aspects of community diversity essential to successful CEnR. The potential for CEnR to meaningfully account for the important roles of social determinants of health and cultural competency in healthcare research has long been recognized [51, 52]. The NCATS strategic plan emphasizes that translational research efforts should include an additional study of clinical implementation and social determinants of health [8]. Therefore, future CEnR resources need to include standardized nomenclature and training materials with a domain and associated competencies specific to social determinants of health and cultural competency.
Inconsistencies in the language used to define and describe various aspects and competencies related to CEnR was a limiting feature of resources included in this study. More than half of the resources had at least one domain scoring discrepancy between reviewers after the initial review, which suggested some challenges related to domain nomenclature. Due to the lack of standardized nomenclature to describe key CEnR concepts, it was challenging for reviewers to compare CEnR resources between CTSAs to ensure all similar resources were being captured in the study sample. This also created difficulty in identifying the CEnR core competency domains addressed within each resource, leading to many of the initial scoring discrepancies observed. Inconsistent language makes it difficult for stakeholders to search for relevant resources and difficult for novice CEnR stakeholders to understand key terminology and concepts. For example, we found that a variety of terms were used to refer to community partners (e.g., community stakeholders, patient partners, experience ambassadors, community advisors, etc.). Such a variety of terms and descriptions could be confusing to academic and community partners seeking to understand how community partners should be educated and participate CEnR projects. In a recent study, Eder and colleagues (2018) surveyed key informants at CTSAs in an effort to understand the consistency of CEnR definitions used among participating institutions. Their findings indicate there is a distinct need in CEnR for clearly delineated, specialized nomenclature in order to accurately describe and perform CEnR [53]. A standardized set of terminology and curricula of the core skills and resources are needed for CEnR to facilitate addressing gaps in CEnR and improving this body of knowledge [9]. Our findings also indicate that efforts to harmonize language in this area of research will have a high yield for the efficiency and effectiveness of future CEnR. As CTSAs attempt to leverage the use of existing resources and develop new ones, the standardization of CEnR language and concepts will be essential, and will align with the strategic plan of the NCATS to streamline the development of best practices, and facilitate more sharing and collaboration in translational science [8].
Current CEnR literature also does not describe how institutional factors, such as mentorship and resource availability to foster community outreach and engagement, can affect the development of community engagement. This is relevant to the creation of CEnR resources because universities that have a strong foundation of commitment to CEnR might emphasize competencies differently or develop curriculum differently than institutions with a weaker foundation [11]. The competencies needed for successful CEnR should describe specific attitudes, knowledge, and behaviors that a researcher must have to achieve both the goals of CEnR and of the partnership between the researcher and the community. Determining gaps in competency coverage of available resources can help address training opportunities for researchers and community partners prior to entering a partnership [54]. Much of the literature from our review recommends building a conceptual framework that identifies key competencies of CEnR that can be mapped to specific domains of knowledge [13]. To the co-authors’ knowledge, this is the first study to synthesize recognized components of CEnR competency into such defined domains. Identifying and defining core competency domains from the relevant literature will help in standardizing CEnR terminology for common use. There are additional efforts underway to identify core domains of CEnR competency using differing methodology, including a recent multi-institutional report from the Joint Working Group [6, 9, 11, 55]. Future CEnR research must synthesize such reports in a concerted effort to standardize CEnR competencies.
Some CTSAs provide a variety of outstanding CEnR education and training resources for community, academic, and clinical stakeholders. For example, the University of California San Francisco provided access to a variety of lectures and written materials intended for wide dissemination and use in education and training in CEnR [34]. The University of Illinois at Chicago also provided a variety of written training materials for CEnR, as well as several resources for working with specific vulnerable populations [36]. However, such quality was not uniform across resources, and there was great variation observed in the content, quality, and formats of the CEnR resources reviewed. Such variation presents stakeholders with difficulties in determining whether CEnR resources from CTSAs could be adapted for use in education and training in their potential CEnR projects. Identifying standard expectations for CEnR education and training resources to achieve would aid stakeholders in determining whether a resource could be utilized for their purposes. Therefore, there would be a benefit in developing grading criteria to help stakeholders assess the practical utility and adaptability of available CEnR resources. Future research should develop tools that help stakeholders evaluate the utility and adaptability of CEnR resources and should incorporate key aspects of CEnR resources observed in this study: content as it relates to core domains of CEnR competencies, intended audience, format, and accessibility.
The findings of this study should be interpreted while considering the following limitations. While the CEnR core competency domains used in this study were identified and defined using the expert consensus of the study team members, other CEnR experts may have identified and defined domains differently. Despite the interest in performing CEnR and the wealth of CEnR resources available from CTSAs, there is limited data and published literature related to the core competencies necessary for a successful scholarship of CEnR, as well as extreme variation in the methods and quality of literature concerning CEnR [6, 7]. Additional CEnR research is needed to establish universal CEnR core competency domains and their definitions. Since this study focused on the resources developed and provided by CTSAs, resources developed or provided by non-CTSA organizations that support CEnR education and training were not assessed. Finally, this study focused on the content of CEnR resources, and not the effectiveness of resources in providing adequate education and training. Future studies should examine the extent to which the intended audiences of these resources are able to successfully perform CEnR to fully assess the value of resources for use in CEnR education and training.
Conclusion
This study suggests that CEnR resources are abundantly available online from CTSAs. Many available resources addressed at least one of the CEnR core competency domains identified by our study team. However, the lack of clear organizational structure within CTSA websites and standardized CEnR competency nomenclature render those resource platforms difficult to navigate, even for CEnR experts. Variability in the language used to describe essential components of CEnR is likely to be a major barrier to standardizing education and training for community stakeholders and researchers participating in CEnR. Future work should focus on consensus building toward a standardized nomenclature and guidelines to described CEnR competencies, and on incorporating competencies related to social determinants of health into CEnR education and training.
In order to build and sustain abilities to share resources for training and education in CEnR across CTSAs, these efforts are necessary. This study elucidated gaps in CEnR education and training that must be addressed. Gaps were identified not only in the content related to competencies in publicly accessible online CEnR resources, but also in the alignment of online resource access to ensure ready to access these invaluable resources generated by CTSAs. This study, combined with the knowledge generated through expert consensus panels, may be a platform on which to develop future resources, create standardized CEnR nomenclature, and bring forth the potential development of a comprehensive, systematically organized repository of tools specific to CEnR.
Acknowledgments
R.J. Piasecki received support from the Predoctoral Fellowship in Interdisciplinary Training in Cardiovascular Health Research (T32 NR012704), the Philip D. Raso Scholarship provided by Nurses Educational Funds, Inc., and the Ruth L. Kirschstein Predoctoral Individual National Research Service Award (1F31NR018362-01A1).
E. Quarles, A. Bilheimer, and L. Carter-Edwards received support through the North Carolina Translational and Clinical Science Institute (NC TraCS). NC TraCS at the University of North Carolina at Chapel Hill is funded by the National Center for Advancing Translational Sciences through the Clinical and Translational Science Awards (CTSA) program (UL1TR002489).
C.R. Dennison Himmelfarb received support through the Johns Hopkins Institute for Clinical and Translational Research (ICTR). The Johns Hopkins ICTR is funded by the National Center for Advancing Translational Sciences through the Clinical and Translational Science Awards program (UL1TR003098 and UL1TR001079).
Disclosures
The authors have no conflicts of interest to declare.
References
- 1.CDC. Principles of Community Engagement (2nd ed). NIH publication no. 11-7782. 2011. [cited June 30, 2019]. (http://www.atsdr.cdc.gov/communityengagement/pdf/pce_report_508_final.pdf)
- 2. Andrews JO, Newman SD, Meadows O, et al. Partnership readiness for community-based participatory research. Health Education Research 2012; 27(4): 555–571. doi: 10.1093/her/cyq050. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Rubio DM, Schoenbaum EE, Lee LS, et al. Defining translational research: implications for training. Academic Medicine 2010; 85(3): 470–475. doi: 10.1097/ACM.0b013e3181ccd618. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. CTSA Program Hubs. Clinical and Translational Science Awards (CTSA) Program [Internet], 2019 [cited June 30, 2019]. (https://ncats.nih.gov/ctsa/about/hubs)
- 5. About the CTSA Program. Clinical and Translational Science Awards (CTSA) Program. (https://ncats.nih.gov/ctsa/about). Updated 25 October 2018. Accessed 30 June 2019.
- 6. DiGirolamo A, Geller AC, Tendulkar SA, et al. Community-Based Participatory Research Skills and Training Needs in a Sample of Academic Researchers from a Clinical and Translational Science Center in the Northeast. Clinical and Translational Science 2012; 5(3): 301–305. doi: 10.1111/j.1752-8062.2012.00406.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. D’Alonzo KT. Getting started in CBPR: lessons in building community partnerships for new researchers. Nursing Inquiry 2010; 17(4): 282–288. doi: 10.1111/j.1440-1800.2010.00510.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. NCATS strategic plan. National Institutes of Health, National Center for Advancing Translational Sciences; 2016.
- 9. Faridi Z, Grunbaum JA, Gray BS, et al. Community-based participatory research: necessary next steps. Preventing Chronic Disease 2007; 4(3): 1–5. doi:A70 [pii]. [PMC free article] [PubMed] [Google Scholar]
- 10. Carter-Edwards L, Cook JL, McDonald MA, et al. Report on CTSA Consortium Use of the Community Engagement Consulting Service. Clinical and Translational Science 2013; 6(1): 34–39. doi: 10.1111/cts.12006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Parker E, Margolis LH, Eng E, Henríquez-Roldán C. Assessing the capacity of health departments to engage in community-based participatory public health. American Journal of Public Health 2003; 93(3): 472–476. doi: 10.2105/ajph.93.3.472. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Doberneck DM, Bargerstock BA, McNall M, et al. Community engagement competencies for graduate and professional students: Michigan State University’s approach to professional development. Michigan Journal of Community Service Learning 2017; 24(1): 122–142. 10.3998/mjcsloa.3239521.0024.111. [DOI] [Google Scholar]
- 13. Shea CM, Young TL, Powell BJ, et al. Researcher readiness for participating in community-engaged dissemination and implementation research: a conceptual framework of core competencies. Translational Behavioral Medicine 2017; 7(3): 393–404. doi: 10.1007/s13142-017-0486-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Harold and Muriel Block Institute for Clinical and Translational Research at Einstein and Montefiore. Albert Einstein College of Medicine. (http://www.einstein.yu.edu/centers/ictr/)
- 15.Clinical and Translational Science Institute. Boston University. (http://bu.edu/ctsi/)
- 16.Clinical and Translational Science Collaborative. Case Western University. (http://case.edu/medicine/ctsc/)
- 17.Chicagoland Clinical and Translational Science Award. Northwestern University, University of Chicago, Rush University, Advocate Health Care, Illinois Institute of Technology, Loyola University Chicago, NorthShore University HealthSystem, and University of Illinois Chicago. (https://www.nucats.northwestern.edu/resources/chicago-ctsa-shared-resources.html)
- 18.Clinical and Translational Science Institute at Children’s National Hospital. Children’s National Hospital and George Washington University. (https://www.ctsicn.org/)
- 19.SYNERGY Clinical and Translational Science Institute. Dartmouth College. (https://synergy.dartmouth.edu/)
- 20.Clinical and Translational Science Institute. Duke University. (https://www.ctsi.duke.edu/)
- 21.Georgetown-Howard Universities Center for Clinical and Translational Science. Georgetown University and Howard University. (http://www.georgetownhowardctsa.org/)
- 22.Georgia Clinical and Translational Science Alliance. Emory University, Morehouse School of Medicine, Georgia Institute of Technology, and University of Georgia. (https://georgiactsa.org/)
- 23.Harvard Catalyst. Harvard University. (https://catalyst.harvard.edu/)
- 24.Center for Clinical and Translational Science. Mayo Clinic. (https://www.mayo.edu/research/centers-programs/center-clinical-translational-science)
- 25.Clinical and Translational Science Institute of Southeast Wisconsin. Medical College of Wisconsin, Children’s Wisconsin, Froedtert Hospital, Marquette University, Milwaukee School of Engineering, University of Wisconsin-Milwaukee, Milwaukee VA Medical Center, and Versiti. (https://ctsi.mcw.edu/)
- 26.Clinical and Translational Science Institute. New York University. (https://med.nyu.edu/departments-institutes/clinical-translational-science/)
- 27.Center for Clinical and Translational Science. The Ohio State University, The Ohio State University Wexner Medical Center, and Nationwide Children’s Hospital. (https://ccts.osu.edu/)
- 28.Center for Clinical and Translational Science. The Rockefeller University. (http://www2.rockefeller.edu/ccts/)
- 29.Center for Clinical and Translational Science. University of Alabama at Birmingham. (https://www.uab.edu/ccts/)
- 30.Translational Research Institute. University of Arkansas Medical Sciences. (https://tri.uams.edu/)
- 31.Institute for Clinical and Translational Science. University of California, Irvine. (https://www.icts.uci.edu/index.php)
- 32.University of California, Los Angeles Clinical and Translational Science Institute. University of California, Los Angeles. (https://ctsi.ucla.edu/)
- 33.Altman Clinical and Translational Research Institute. University of California, San Diego. (https://medschool.ucsd.edu/research/actri/Pages/default.aspx)
- 34.Clinical and Translational Science Institute. University of California, San Francisco. (https://ctsi.ucsf.edu/)
- 35.Colorado Clinical and Translational Sciences Institute. University of Colorado Denver. (https://cctsi.cuanschutz.edu/)
- 36.Center for Clinical and Translational Science. University of Illinois at Chicago. (https://ccts.uic.edu/)
- 37.Clinical and Translational Science Institute. University of Minnesota. (https://www.ctsi.umn.edu/)
- 38.Clinical and Translational Science Center. University of New Mexico. (https://hsc.unm.edu/research/ctsc/)
- 39.North Carolina Translational and Clinical Sciences Institute. University of North Carolina at Chapel Hill. (https://tracs.unc.edu/)
- 40.Clinical and Translational Science Institute. University of Pittsburgh. (https://ctsi.pitt.edu/)
- 41.University of Rochester Clinical & Translational Science Institute. University of Rochester. (https://www.urmc.rochester.edu/clinical-translational-science-institute.aspx)
- 42.Southern California Clinical and Translational Science Institute. University of Southern California, Children’s Hospital Los Angeles, and Los Angeles County Department of Health Services. (https://sc-ctsi.org/)
- 43.Institute of Translational Health Sciences. University of Washington, Fred Hutchinson Cancer Research Center, and Seattle Children’s Hospital. (https://www.iths.org/)
- 44.Institute for Clinical and Translational Research. University of Wisconsin-Madison. (https://ictr.wisc.edu/)
- 45.Vanderbilt Institute for Clinical and Translational Research. Vanderbilt University. (https://victr.vumc.org/)
- 46.C. Kenneth and Dianne Wright Center for Clinical and Translational Research. Virginia Commonwealth University. (https://cctr.vcu.edu/)
- 47.Institute of Clinical and Translational Sciences. Washington University in St. Louis. (https://icts.wustl.edu/) [DOI] [PMC free article] [PubMed]
- 48.Yale Center for Clinical Investigation. Yale University. (https://medicine.yale.edu/ycci/)
- 49. Institute of Medicine The CTSA Program at NIH: Opportunities for Advancing Clinical and Translational Research. Washington, DC: National Academic Press, 2013. [PubMed] [Google Scholar]
- 50. Wojdynski BW, Kalyanaraman S. The three dimensions of website navigability: explication and effects. Journal of the Association for Information Science and Technology 2015; 67(2): 454–464. doi: 10.1002/asi.23395. [DOI] [Google Scholar]
- 51. Seifer SD, Sisco S. Mining the challenges of CBPR for improvements in urban health. Journal of Urban Health 2006; 83(6): 981–984. doi: 10.1007/s11524-006-9112-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52. Wallerstein NB, Duran B. Using community-based participatory research to address health disparities. Health Promotion Practice 2006; 7(3): 312–323. Epub 2006 Jun 7. doi: 10.1177/1524839906289376. [DOI] [PubMed] [Google Scholar]
- 53. Eder MM, Evans E, Funes M, et al. Defining and measuring community engagement and community-engaged research: clinical and translational science institutional practices. Progress in Community Health Partnersh 2018; 12(2): 145–156. doi: 10.1353/cpr.2018.0034. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 54. Chari R, Matthews LJ, Blumenthal MS, et al. The Promise of Community Citizen Science. Santa Monica, CA: RAND Corporation, 2017. (https://www.rand.org/pubs/perspectives/PE256.html) [Google Scholar]
- 55. Richmond A, Strelnick H, Tilson H, et al. Collaboration/engagement and workforce development domain task forces: researcher training and education and community capacity building workgroup. Unpublished data. December 2018.