The Community Research Scholars Initiative: A Mid‐Project Assessment

Abstract

Community organizations addressing health and human service needs generally have minimal capacity for research and evaluation. As a result, they are often inadequately equipped to independently carry out activities that can be critical for their own success, such as conducting needs assessments, identifying best practices, and evaluating outcomes. Moreover, they are unable to develop equitable partnerships with academic researchers to conduct community‐based research. This paper reports on the progress of the Community Research Scholar Initiative (CRSI), a program that aims to enhance community research and evaluation capacity through training of selected employees from Greater Cleveland community organizations. The intensive 2‐year CRSI program includes didactic instruction, fieldwork, multiple levels of community and academic engagement, leadership training, and a mentored research project. The first cohort of CRSI Scholars, their community organizations, and other community stakeholders have incorporated program lessons into their practices and operations. The CRSI program evaluation indicates: the importance of careful Scholar selection; the need to engage executive leadership from Scholar organizations; the value of a curriculum integrating classwork, fieldwork, and community engagement; and the need for continual scholar skill and knowledge assessment. These findings and lessons learned guide other efforts to enhance community organization research and evaluation capacity.

Introduction

Health and human service community organizations frequently have limited capacity for research and evaluation.1 As a result, they are ill‐equipped to independently carry out many evaluation activities, and limited in their ability to develop equitable partnerships with academic researchers to conduct research. In contrast, academic institutions may be adept in these activities, but often lack the frontline knowledge of the communities they seek to serve. The limitations of both the community organizations and the academic institutions can be problematic since meaningful community‐academic partnerships can provide a mechanism for translating research findings into improved health outcomes.2, 3, 4 The Community Research Scholar Initiative (CRSI) was developed to build upon the Community‐Academic partnership paradigm in a way that fosters bi‐directional learning and collaboration. CRSI's design includes an intensive 2‐year training program for health and human service professionals, covering a broad array of research and evaluation methods. The logic model in Figure 1, developed by an independent evaluator, shows the overarching goals of the CRSI project.5 It is based on a common logic model format, defining inputs, processes, outputs, Scholar outcomes, organizational outcomes, and longer term impacts.6 Of particular concern to CRSI is the identified outcomes, and discernible impacts on the local health and human services nonprofit sector.7 Midway through the grant period, we examine how these larger goals have been realized, and lessons learned from the first cohort.

Figure 1.

Logic model for the community research scholars initiative.

Methods

CRSI will complete two rounds of research training and capacity building by recruiting 2 cohorts of health and human service professionals into an intensive 2‐year training program. The first cohort included three individuals (Scholars), and six Scholars from different community organizations will be recruited for the second cohort (Table 1). Scholars are compensated for 40% of their salaried time to participate in CRSI 16 hours weekly. CRSI is supported by a National Institutes of Health grant, and is maintained through a memorandum of understanding between the Scholar's home organization and MetroHealth Medical Center.

Table 1.

Time line of community research scholar initiative deliverables for 5‐year grant period

Year	1	1	1	1	2	2	2	2	3	3	3	3	4	4	4	4	5	5	5	5
Quarter	1	2	3	4	1	2	3	4	1	2	3	4	1	2	3	4	1	2	3	4
Development and recruitment
Develop training curriculum	x	x
Recruit scholars and supervisors	x	x							x	x
Scholar training: first cohort
Course work			x	x	x	x	x	x	x	x
Dialogues with academic and community experts			x	x	x	x	x	x	x	x
Research projects					x	x	x	x	x	x
Educating communities about research & evaluation									x	x	x	x	x	x	x	x	x	x
Scholar training: second cohort
Course work											x	x	x	x	x	x	x
Dialogues with academic and community experts											x	x	x	x	x	x	x
Research projects													x	x	x	x	x
Educating communities about research & evaluation																	x	x	x	x
Community‐based research network (CBRN)
Recruit community organizations	x	x							x	x
Community research and evaluation seminar			x	x	x	x	x	x	x	x	x	x	x	x	x	x	x	x
Advise new cohort of scholars											x	x	x	x	x	x	x	x
CBRN planning meetings									x	x	x	x	x	x	x	x	x	x
Collaborative research or evaluation project											x	x	x	x	x	x	x	x
Dissemination of findings to communities																			x	x
Evaluation and manuscripts
Description and lessons learned of CRSI								x	x
Outcomes of CRSI																			x	x
Individual scholar projects										x	x								x	x
CBRN joint project																			x	x

Selection of the first cohort of Scholars was based on a rigorous application process, with input from an advisory board composed of Center for Reducing Health Disparities (CRHD) staff and community leaders. CRSI was advertised via CRHD email lists and website, the local United Way, social media, and personal referrals. All applications were reviewed and ranked by the advisory board, based on components, such as Evidence of motivation and commitment to the research scholars program; Capacity to implement research scholar training and project findings within organization; Evidence of CEO commitment; Ability of applicant to articulate the research needs of organization and/or the community.8

The first three Scholars selected for CRSI include a data analyst from Beech Brook (a children and family mental health and foster care agency), a referral and information specialist from Frontline Services (a mental health agency), and a community outreach coordinator from Susan G. Komen NorthEast Ohio (a breast cancer funding and education agency). These agencies were reviewed for a variety of factors including size, ability to yield organizational impact, financial stability, and commitment.

Using a combination of adult learning theories and community/organizational change strategies, the CRSI curriculum draws upon individual experience, critical reflection, and professional development. The 2‐year cycle culminates in a research project jointly developed by the Scholar, his or her supervisor, and staff mentors. The didactic curriculum includes four areas: (1) research overview, (2) quantitative methods, (3) qualitative and mixed methods, and (4) translation and organizational applications of skills and knowledge. Each area consists of 15–20 content or skill modules organized around three concepts: engaging ideas, engaging academia, and engaging community (Table 2). Additional weekly assignments consist of journal reflections, blog writing, and peer teaching. Through these mechanisms, research and evaluation topics are further integrated into Scholars’ understanding while also sharpening the presentation and leadership skills.

Table 2.

Evaluation of community research scholar initiative (CRSI) by participants. Based on a 5‐point Likert scale, where 1 = strongly disagree and 5 = strongly agree

Since beginning CRSI …
Research	Mean (SD)
I feel more comfortable thinking about research	4.0 (0.0)
I know more about research	4.0 (0.0)
I have a strong grasp of various research methods	3.7 (0.6)
I feel more engaged in research issues/topics	4.0 (0.0)
Professional confidence
My presentation skills have improved	4.0 (0.0)
I am more of a leader	4.0 (1.0)
I work better on a team	3.7 (1.5)
I have expanded my career options	4.0 (0.0)
Client impact
I can make a difference in my community through research	4.0 (0.0)
I have a better sense about the impact research can have on communities	4.0 (1.0)

Using concepts and skills acquired during the program, Scholars designed an research project with a budget $4,500 each. Scholars created a list of at least five possible research ideas, conducted literature reviews, and presented this information to their organizations’ Executive Directors and the CRSI Co‐Directors for guidance.

Two external evaluators were contracted to determine the impact of CRSI on individual Scholars, and on their organizations. One evaluator has conducted surveys with the Scholars every 6 months and conducted group interviews to gauge the Scholars’ evaluation of the program. The other evaluator meets with CRSI staff, Scholars, and the Scholars’ executive directors to examine organizational impact and long‐term sustainability of enhanced research capacity.9, 10

Results

The three Scholars reported having overall positive experiences with the program. Specifically, Scholars have expressed increased knowledge of the research process, more confidence in speaking and interacting in academic settings, and belief that the work they are doing with CRSI will have an impact in helping their clients individually and their organization as a whole.

Networking with both academic and community members in Cleveland has been noted as a beneficial and rewarding aspect of the program, and Scholars say they feel comfortable reaching out to those experts when seeking advice (Table  3).

Table 3.

List of curriculum topics

Introduction and overview of research

Office set up and orientation [delete this line]

Broad overview of issues and dilemmas in research

Identifying and developing the research question

Introduction to research methods

Introduction to research ethics

Conducting a literature review

Basic interviewing and report writing

Blog and other writing

Health disparities: the general research

Health disparities: applications and translation

Health disparities: northeast Ohio

Cultures of academia, nonprofits, and “the community”

Community/academic partnerships

Northeast Ohio community tools and resources: overview

Northeast Ohio academic tools and resources: an overview

IRBs and the formal research process

Quantitative methods

The nature of quantitative research: uses and limitations

Key issues: measurability/causality; validity/reliability

Key issues: replicability/generalization; sampling/design

Basic statistics

Survey research

Structured interviews

Structured observation

Secondary analysis

Using large databases and database tools

Content analysis

Data storage and management

Quantitative data analysis and interpretation

Quantitative analysis tools and instruments

Quantitative review and community research boot camp

Quantitative final and research project update

Qualitative and mixed methods

The nature of qualitative research: uses and limitations

Key issues: trustworthiness/credibility; ethics and reflexivity

Key issues: the use of theory; language and setting

Ethnography

Statistical issues in qualitative versus quantitative

Statistical application in qualitative research

Focus groups

Interviewing in qualitative research

Conversation/discourse analysis

Qualitative analysis tools and instruments

Qualitative data analysis and interpretation

Mixed methods: an overview

Mixed methods: an end goal

Qualitative review and community research boot camp

Qualitative final and research project update

Research project implementation, translation, and organizational change

Overall research design

Power and politics: community forces and institutions

Power and politics: community change

Organizational culture

Power and change

Strategic leadership

Needs assessment: part 1

Needs assessment: part 2

Evaluation part 1

Evaluation part 2

Researching and adopting evidence based practice

Quality improvement

Performance improvement

Coalitions and partnerships: part 1

Coalitions and partnerships: part 2

Community‐based participatory research

Meeting management

Conflict negotiation

Working with media

Using new media

Organization strategic planning

Community change management

Writing the research paper

Writing grant proposals

Graphic displays/posters

The three Scholars reported challenges surrounding how they would incorporate their new knowledge in their work, as well as how they would share acquired knowledge and skills with peers and supervisors. These challenges included questions about the roles they can and will play in persuading their agencies to integrate expanded research capacity into organizational practice and culture.

It is unclear whether the Scholars’ uncertainty about large scale organizational change is a result of insufficient time (organizational change requiring sustained efforts over time), insufficient skill‐building, or weaknesses in the overall program design of CRSI (e.g., selecting insufficiently motivated organizations). These questions have led to fruitful questions about organizational “research readiness,” and how to assess for such readiness to maximize efforts such as CRSI.

As a result of the training to date, each Scholar has developed a personally and organizationally relevant research project, and has submitted their research protocol to the relevant IRB for approval.

The Beech Brook Scholar has developed a descriptive study examining the entire population of Beech Brook children with mental health diagnoses (n = 500) and comorbid physical health diagnoses, designed with the awareness that while such research has been conducted with adult populations, there is scant evidence of comorbid physical and mental health diagnoses among children and adolescents. It is hoped that obtaining a better understanding of such comorbidity will lead to new treatment modalities that can seamlessly integrate mental and physical health needs of the client population.

The Frontline Services Scholar is planning a randomized control trial of guided imagery to reduce stress reduction among Frontline staff. For an agency that confronts multiple forms of trauma among vulnerable populations, vicarious stress, and professional burnout is a concern, both in terms of human resources management and effectiveness of services. The Scholar hypothesizes that introducing a nondisruptive, staff‐initiated strategy for reducing stress while on the job will contribute to employee wellbeing and better outcomes for clients, as compared to staff who do not employ such a strategy.

The Susan G. Komen Scholar is examining causes of Community Health Worker (CHW) attrition among breast health CHWs. While current literature supports the effectiveness of CHWs in promoting health‐seeking behavior, there is widespread anecdotal concern about the high turnover of trained CHWs. Examining causes of volunteer and paid CHW workforce instability may guide training and selection for future CHW programs.

Discussion

Initial evaluation data confirm CRSI effectiveness in improving research‐related knowledge and skills among Scholars. These evaluation findings, reviews by CRSI program among staff at the CRHD, CRSI scholars, evaluators, and community partners suggest a number of possible mid‐initiative “lessons learned.” They fall into three categories: Curriculum Content and Design, Program Administration, and Culture and Third‐Party Relationships.

In terms of Curriculum Content and Design, Scholar feedback and evaluator input suggest that the multipronged approach to each content module, which involves engaging with critical texts, academic professionals, and community experts on each subject, reinforces Scholars’ acquisition of critical knowledge and skills. This approach also appears effective in helping Scholars translate research findings into community settings.

Scholars report that the requirement that they regularly “teach back” to each other on their scheduled readings not only reinforces learning, but nurtures content organization and presentation skills that will be useful not only for research implementation but also dissemination. In a similar vein, the CRSI's curriculum, which includes content on and practice with communications skills, group dynamics, leadership, and organizational development appears to be useful as Scholars design and carry out their research within dynamic, diverse, and complex systems.

In terms of Program Administration, at least one critical lesson is apparent. While the CRSI co‐directors placed strong emphasis on the commitment of applicant organizations and their executive directors, investment of those key executives has been uneven. Thus, in the next cohort of Scholars, the program will seek to develop more effective methods for determining directors’ levels of investment in developing their organizations’ research capacity and/or their readiness to do so.

On the purely administrative or management side, the CRSI co‐directors have grappled with several questions. Despite the fact that the current Scholar cohort is small, with only three individuals, the initiative has proven to be labor intensive. Knowing that the next cohort will expand to six Scholars, with the attendant additional administrative, supervisory, and coaching requirements thereof, raises concerns about future workload for the co‐directors.

We have also begun to pose the following question: Would the CRSI program's evolving into an academic certificate‐granting program offer new opportunities for diffusion of innovations and approaches, and sustainability of program funding? Creation of such a program has the potential to generate ongoing tuition‐based revenue to support the CRSI training model, and could support larger scale adoption of research methods and knowledge as more students enroll in the program. We will continue to explore questions of diffusion and sustainability as the CRSI program moves forward.

In terms of Culture and Third‐Party Relationships, we can make a number of observations. The first is that the successful training of a community‐based researcher is dependent on a responsive and engaged organizational culture. Briefly, such a culture would be marked by strong support of CRSI efforts by executive and clinical leadership; support of the CRSI scholar to carry out necessary research tasks at the organization; and a willingness on the part of the organization to more effectively utilize research. This observation has led to discussion about how to assess whether an organization is “research ready,” and the specific features, attributes, and internal capacities that signal research readiness.

A second observation is that scalability of successful research within organizations may be dependent on organization size and funding resources. Smaller organizations, in which staff is often required to “wear many hats,” may have a more difficult time developing research capacity.

Third, while it appears that CRSI will have meaningful impact on individual Scholars, and is likely to improve research capacity at their host organizations, it is not yet clear whether larger scale, sector‐wide of research capacity, within the local community, can be achieved.

Finally, we have begun to pose the question of whether structural changes, particularly in terms of foundation and government funding, can facilitate enhancement of research capacity within health and human service agencies. This could take the form of small percentage set‐asides for research and evaluation on all foundation grants; funding support for staff training in research and evaluation; or other approaches.

These findings are preliminary. Our sample size is small (three), and as of this writing, Scholars have eight more months to go in the program, during which time they will undergo additional training in translational research and community/organizational change, and complete their research projects. In January of 2015, six new Scholars will begin the program for a 2‐year cycle; the full complement of nine participating individuals and organizations won't complete their work until 2017.

Conclusion

In conclusion, CRSI has provided a positive learning experience for the current cohort of Scholars, and has had some impact on their host organizations. Outcomes to date appear significant, and many of CRSI's features—the content and style of its curriculum, its approach to the nature of research, its community outreach and networking elements, and its emphasis on communication and leadership—have combined to foster notable impact.

In the future, CRSI will need to more intensively investigate the questions of impact diffusion and project sustainability, and make participant‐informed decisions about how trained Scholars and their participating organizations can continue their efforts after 2017. Some early explorations, including developing a community‐based institutional review board, new Practice‐Based Research Networks, or designing a community‐managed research ethics consultation service, have begun.

Sources of Funding

J. Theurer was supported by grants MD002265 and UL1TR000439 from the National Institutes of Health. E. Pike was supported by grants MD002265 and UL1TR000439 from the National Institutes of Health. A. R. Sehgal was supported by grants MD002265 and UL1TR000439 from the National Institutes of Health. R. L. Fischer was supported by grant MD002265 from the National Institutes of Health. C. Collins was supported by grant MD002265 from the National Institutes of Health.