Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2023 Sep 1.
Published in final edited form as: J Community Psychol. 2022 Mar 28:10.1002/jcop.22846. doi: 10.1002/jcop.22846

Testing a Digitally Distributed Method to Recruit a Network of Community Organizations to Fight the Consequences of the Drug Epidemic: A Study in Thirteen American States

Haesung Jung 1,*, Aashna Sunderrajan 2,*, Marta Durantini 1, Edgardo Sanchez 3, Liliane Windsor 3, Sally Chan 1, Thomas O’Brien 3, Bita Fayaz Farkhad 1, Alex Karan 3, Carol A Lee 3, Soonhyung Kwon 3, Dolores Albarracín 1
PMCID: PMC9464661  NIHMSID: NIHMS1788742  PMID: 35344609

Abstract

Objective:

To mitigate the opioid epidemic, a concerted effort to educate, prevent, diagnose, treat, and engage residents is required. In this paper, a digitally distributed method to form a large network of organizations was tested with 99 counties in regions with high vulnerability to HCV (Hepatitis C Virus).

Methods:

The method involved a cascade of contacts going from email to phone calls to videoconferencing and measuring the number of contacts required, amount of time taken, and the proportion of success at recruiting at least one community organization per county.

Results:

A recruitment period of five months and 2,118 contact attempts led to the recruitment of organizations from 73 out of our 99 target counties. Organizations belonging to health departments required more attempts and time to recruit but ultimately enrolled at higher rates than did other organizations such as coalitions and agencies. Organizations from counties more (vs. less) vulnerable to HCV outbreaks required more attempts to recruit, and using multiple recruitment methods (e.g., emails, phone calls, and Zoom meetings) improved enrollment success.

Conclusions:

Overall, this method proved to be successful at remotely engaging a large-scale network of communities with different levels of risk within a large geographic region.

Keywords: recruitment, community engagement, opioid use, HCV, rural health

Over the past two decades, the rural regions of Appalachia, the Midwest, and the South in the United States have had rates of opioid use exceeding the national average (Hoots et al., 2018) and rates of overdose mortality that are 65% higher than in other regions (48.3 vs. 29.2 deaths per 100,000; Meit et al., 2019). These regions have also had increasing rates of infectious disease outbreaks (Zibbell et al., 2015), with rates of HCV associated deaths that are 32% higher than in other regions (4.9 vs. 3.7 deaths per 100,000; Centers for Disease Control and Prevention [CDC], 2018). Although the origins of these crises are complex, social determinants of health, misconceptions about opioids, and a culture of isolation and despair are among the most critical causes.

One possible solution to address these problems is to mobilize communities, combining the knowledge, skills, and resources of a network of organizations to develop effective and locally feasible answers (Cunningham et al., 2015; Pratt et al., 2020; Richardson & Allegrante, 2000). Particularly, a network of community partners may be able to coordinate efforts and work to improve local conditions, including proposing and testing solutions to reduce the harm of opioid use and HCV infections. In fact, the CDC (Centers for Disease Control and Prevention) has recognized the leading role community networks play in preventing drug use (https://www.cdc.gov/drugoverdose/featured-topics/drug-free-communities.html)) and hence, scholars have increasingly engaged community partners to develop interventions addressing drug use. For example, Zimmerman et al. (2020) established a partnership with community members, patients, policymakers, and service providers in a rural community of Virginia to identify and prioritize strategies for combating the opioid epidemic. Martinez et al. (2020) partnered with 16 counties across four states to implement practices that reduce opioid overdose deaths. Despite these efforts, community partnerships in these projects have stayed relatively localized and have primarily involved a small number of communities or states, although America’s opioid epidemic impacts a wide region. Hence, in this paper, we describe our efforts to implement a systematic and digitally distributed method to remotely recruit communities from a sample of 99 at-risk counties across thirteen states surrounding the Appalachian region. We then compare our recruitment data against existing benchmarks, identify predictors of recruitment, and estimate the degree of bias in our recruitment as a function of the HCV risk of each county.

Digitally Distributed Community Recruitment

Principles of community engaged research have been employed worldwide to guide the work of researchers, organizations, and community members (Belone et al., 2016) in areas including mental health (e.g., Fortuna et al., 2019), cardiovascular health (e.g., Yingling et al., 2016), substance use (e.g., Windsor et al., 2018), and HIV (e.g., Rhodes et al., 2018), with the premise that community engagement not only increases participation from diverse sectors but also makes interventions more sustainable (Albert et al., 2011). The last few decades have shown high levels of interest in engaging communities, as well as increased success (Pinto et al., 2015; Viswanathan et al., 2004; Windsor et al., 2018).

Collaborative work with community partners has typically involved a small number of people and organizations working in geographic proximity to each other (National Institutes of Health [NIH], 2011). This approach is consistent with the need to develop deeper and trusting relationships on projects that require significant time and resource commitments (Lucero et al., 2018). However, many of the problems that affect the health and wellbeing of community members, including opioid use and HCV infections, are complex and span across large geographical areas. These problems cannot be solved by any person or organization working alone (Mitchell & Shortell, 2000) but rather, require a geographically distributed network of communities whose organizations are mobilized to improve their community problems.

Moreover, recruitment efforts should not only reach communities with favorable pre-existing attitudes toward research partnerships but also those that are less favorable. For example, some communities may be reluctant to join a community network or a research partnership because they underestimate the severity of health issues in their community or fear exposing their vulnerability to outsiders. Such reluctant communities are often underrepresented in research partnerships. According to Festinger (1964), people often tend to seek information that confirm their points of view because they feel comfortable in these situations (see also Fetterman & Hart, 2020; Hart et al., 2009). Likewise, individuals who are already in compliance with the health recommendations of an intervention are the ones most likely to participate (Earl et al., 2009; Noguchi et al, 2007; Wilson & Albarracín, 2015). Similarly, organizations already involved in research are the ones most likely to participate in other research (Kaiser et al., 2017). Therefore, communities that are better equipped to address the problem may be the ones most willing to participate, whereas those that are less equipped may be less willing.

In recent years, researchers have turned to technology-based recruitment approaches, utilizing digitally mediated communication tools such as mobile phones, email, social media, and online conferencing to help tackle some of these obstacles (Dalessandro, 2018). Unlike in-person approaches, technology-based recruitment has the advantage of reaching underrepresented populations (Ryan, 2013), including those living in difficult-to-reach geographic locations (Rhodes et al., 2003), is more cost effective for researchers (Graham et al., 2008; Gordon et al., 2006; Ryan, 2013), and more convenient for communities (O’Connor et al., 2016). In the past, technology-based recruitment approaches have demonstrated their utility in health-related research (Ramo et al., 2010), including with surveys (Temple & Brown, 2011) and interventions to improve physical health (Ramo et al., 2014), and have become increasingly more feasible given the exponential increase in access to the internet (Pew Research Center, 2019). With its potential for wider reach and greater engagement, technology-based recruitment approaches may provide the solution to recruit varied communities across geographic boundaries.

Overview of our Project

In this paper, we examined the efficacy of a technology-based recruitment method to engage a large network of community partners. The general purpose of the network was to engage communities to study their perceptions and beliefs about drug use as well as their health behavior, and to develop interventions that are tailored to community culture and needs. A detailed description and activities of the community network appears in Figure 1.

Figure 1.

Figure 1.

Open in a new tab

An example of the recruitment email initially sent to all 99 counties.

First, we identified the counties most vulnerable to HCV outbreaks associated with injection drug use based on Van Handel and colleagues (2016), resulting in our target of 99 counties across thirteen states. Then, we used a cascaded effort of emails, phone calls, and Zoom meetings to recruit contacts, tracking all contacts, including the date, form of communication, and outcome. We measured the number of attempts we made, the time taken (in days) from our initial invitation to enrollment, our enrollment success, and our overall recruitment rate. We assessed a period of five months of recruitment before the onset of the COVID-19 pandemic.

Method

County Selection and Randomization

At the onset of the project, we identified the counties ranked in the top 5% for vulnerability to the rapid dissemination of HCV associated with injection drug use, based on findings by Van Handel and colleagues (2016). In their work, Van Handel and colleagues used a multi-step approach that identified a set of six indicators (i.e., drug overdose deaths, prescription opioid sales, per capita income, white non-Hispanic race/ethnicity, unemployment, and buprenorphine prescribing potential by waiver) associated with higher county rates of acute HCV infection, a proxy outcome for injection drug use. Using these indicators, the authors calculated a composite index score to rank each county’s vulnerability, identifying 220 counties in 26 states within the 95th percentile of most vulnerable.

We then identified the counties within this 95th percentile located in Appalachia, the Midwest, and the South. This resulted in a sample of 198 counties. From this, we used a random number generator to select half of these counties, for a final sample of 99 counties distributed across thirteen states, including Georgia, Illinois, Indiana, Kansas, Kentucky, Michigan, Missouri, North Carolina, Ohio, Pennsylvania, Tennessee, Virginia, and West Virginia. These counties appear in the Appendix.

Identification of Target Organizations

As part of our recruitment efforts, we first brainstormed the types of organizations we should target. This included local health departments; coalitions related to substance use, HIV, or other health outcomes; and agencies that represented aspects of community life, including hospitals, law enforcement, prison/parole/drug courts, family services, and religious institutions. We then used a combination of online sources, as well as referrals from the health departments we contacted, to identify specific organizations that fell into these categories. For each organization, we also identified an individual to serve as a point of contact (e.g., the director of the health department). With these efforts, we were able to compile an initial list of 3,150 organizations across the 99 counties. During recruitment, this list was updated when we identified other relevant organizations to contact.

Distributed Recruitment Methods

To further aid our recruitment efforts, we developed a Distributed Recruitment Method (DRM) involving a cascaded effort of emails, phone calls, and Zoom meetings to recruit contacts. First, we sent out an initial email to organizations to introduce our project. After the initial email, we sent a follow-up email and/or made a phone call to each organization to request a meeting, in line with evidence showing that strong communication is best achieved through scheduled meetings (Pinto et al., 2014). When organizations obliged, we set up informational meetings to discuss our project, as well as any concerns or challenges of participating in our project. During these meetings, we also answered questions, provided clarifications, and enrolled interested parties. Sample scripts for these recruitment methods appear in Figures 1 and 2. When an organization agreed to join our board, we sent them additional materials, including an informed consent form, a questionnaire to assess community needs, and a $200 gift card for their participation in the board over a one-year period. The gift card was sent to the individual who signed the consent form and completed the questionnaire, to be used however they saw fit. Reminder emails and phone calls were also made to sustain communication between our research team and the organization. Throughout this process, we tried to establish one member of our research team as the primary contact for each organization, as consistent staffing has been shown to be important in developing trust, long-term relationships, and more honest conversations (Kaiser et al., 2017).

Figure 2.

Figure 2.

Figure 2.

Open in a new tab

The call guide used when contacting organizations in all 99 counties.

Recruitment Recording System

To track our communication efforts, we developed the Recruitment Recording System (RRS), which tracked all contacts with organizations, including the date, form of communication, and outcome. Specifically, we recorded information, including the state, county, and the Federal Information Processing Standard (FIPS) of the organization; the organization’s classification (whether it was a health department, coalition, or other agency) and type (whether it was a hospital, law enforcement, prison/parole/drug court, family service, religious institution, or something else); and finally, the organization’s name, physical address, phone number, and email address. We also recorded the name of the research team member who made the contact attempt, the attempt number, the date of the contact attempt, the recruitment method used (e.g., email, phone, mobile messaging, zoom), and the status of the target organization (enrolled, in progress).

Data Analytic Plan

Our primary goals in this manuscript were to evaluate our recruitment efforts against existing benchmarks, determine predictors of successful recruitment, and identify potential biases in our recruitment. To evaluate our recruitment efforts, we compared the number of attempts made, the time taken to recruit, and our recruitment rate with benchmarks from the existing literature. The number of attempts needed to successfully enroll an organization involved a count of each discrete recruitment attempt made. Although best practices advocate for the measurement of recruitment attempts (Khodyakov et al., 2018), we were unable to find reports on this. When looking at retention of community organizations, however, prior work has shown that sending three or more follow-up emails increases participation, especially when combined with phone calls (Horvath et al., 2012). We therefore compared the number of attempts we made with this benchmark. Time taken to recruit was measured as the number of days between the first and last contact attempt between November 2019 (the start of our recruitment efforts) and March 2020. Prior work shows that the time taken to complete half the recruitment goal for a multi-site trial can take between 4.4 months (134 days) and 5.8 months (176 days) to achieve (Monaghan et al., 2007). Thus, we compared the time it took us to recruit with this benchmark. Recruitment rate was calculated by dividing the number of enrollees by the number of people who were offered participation. Meta-analyzed reported rates of enrollment have estimated average success at 53% (Noguchi et al., 2007), and this is the benchmark we used.

To determine which aspects of our recruitment method predicted success, we conducted a series of multilevel models, including the type of organization (health department, coalition, agency), the number of recruiters involved in each attempt, and the number of methods we used to recruit agencies (including email, phone, and Zoom), as well as county (Level 2) and state (Level 3), as predictors of the number of attempts we made, the time taken to recruit, and whether we were successfully able to enroll an organization or not (treated as a binary variable). To estimate any bias in our recruitment as a function of the HCV risk of each county, we also included the vulnerability rank of each county in our models. We reverse-coded the original vulnerability rank provided by Van Handel and colleagues (2016) so that higher scores represented greater vulnerability.

Results

The purpose of our paper was to examine the efficacy of a digitally distributed recruitment method to engage a large, cross-regional network of community partners. We first describe any observable differences in our recruitment method. We then compare our recruitment data with existing benchmarks, determine predictors of this recruitment success, and, finally, identify possible biases in our recruitment.

Descriptive Results

We first analyzed the average number of recruiters and methods we used, as well as the type of organizations we contacted. On average our recruitment involved between one and two recruiters (M = 1.35, SD = 0.54). Because we had little variability, however, it was not possible to determine whether differences in recruiter characteristics (including their recruitment experience and academic position) moderated success. Our recruitment involved more than one method of recruitment (M = 1.51, SD = 0.64), most frequently combining emails with phone calls and informational meetings over Zoom. We made the most contact attempts to recruit health departments (M = 8.33, SD = 5.44), which took an average 68.46 days to recruit and had an enrollment success of 48%. In contrast, both coalitions and agencies took fewer contact attempts (coalition: M = 4.78, SD = 3.23; agency: M = 2.81, SD = 2.69), required less time (coalition: 24.84 days; agency: 42.50 days), but had lower enrollment success (coalition: 39%; agency: 2%). Therefore, health departments were difficult to recruit (requiring more attempts and more time) but had a high rate of enrollment success.

Benchmarking Number of Contact Attempts, Time to Recruit, and Recruitment Rate

We were interested in the success of our recruitment method, operationalized by the number of attempts made, time to recruit, and overall recruitment rate, and assessed vis-à-vis existing benchmarks.

Table 1 presents these variables by region and state, with some states collapsed to protect the identity of the counties. We made 2,118 contact attempts, with an average of 4.05 attempts per county. It took us an average of 54.59 days to successfully enroll an organization. During this period, we were able to achieve 74% of our recruitment goal (89 community organizations from 73 out of our 99 target counties). In fact, our overall recruitment rate was 59%. All in all, our findings met or exceeded all existing benchmarks in the literature (e.g., Horvath et al., 2012; Monaghan et al., 2007; Noguchi et al., 2007), suggesting that our method to recruit a geographically dispersed network of organizations was successful.

Table 1.

Summary Recruitment Data

Region, State, and Organization Number of Recruiters Used Number of Approaches Used Number of Attempts Made Time Taken to Recruit Recruitment Rate
Appalachia 1.33 1.60 5.09 58.06 17.17%
 VA 1.14 1.57 5.52 67.00 14.29%
  Health Department 1.00 1.83 8.67 64.00 33.33%
  Coalition 1.00 2.00 7.50 73.00 50.00%
  Agency 1.23 1.38 3.77 0.00%
 WV 1.38 1.60 4.97 56.14 17.95%
  Health Department 1.43 2.29 8.64 79.86 50.00%
  Coalition 1.11 1.67 4.89 24.80 55.56%
  Agency 1.42 1.42 4.05 51.50 3.64%
Great Plains 1.64 1.71 5.00 59.50 14.29%
 KS 1.64 1.71 5.00 59.50 14.29%
  Health Department 2.00 2.33 9.00 77.00 33.33%
  Coalition 2.50 2.50 6.50 42.00 50.00%
  Agency 1.33 1.33 3.33 0.00%
Midwest 1.21 1.43 3.93 52.56 10.67%
 IN 1.23 1.64 6.05 23.00 9.09%
  Health Department 1.60 2.60 11.40 31.00 20.00%
  Coalition 1.50 2.00 8.00 0.00%
  Agency 1.07 1.27 4.00 15.00 6.67%
 MI 1.04 1.25 2.42 36.00 8.33%
  Health Department 1.17 1.50 2.67 36.00 33.33%
  Agency 1.00 1.17 2.33 0.00%
 MO 1.25 1.41 3.82 73.00 9.09%
  Health Department 1.89 2.22 10.89 73.00 44.44%
  Coalition 1.25 1.25 1.25 0.00%
  Agency 1.06 1.19 2.10 0.00%
 OH 1.25 1.43 3.83 53.88 13.33%
  Health Department 1.90 2.30 12.00 51.00 50.00%
  Coalition 1.33 1.67 4.33 69.00 33.33%
  Agency 1.11 1.23 2.06 53.50 4.26%
South 1.42 1.52 3.66 53.58 14.40%
 GA 1.29 1.43 5.29 47.67 14.29%
  Health Department 1.50 2.25 11.50 47.67 75.00%
  Coalition 3.00 1.00 5.00 0.00%
  Agency 1.13 1.25 3.75 0.00%
 KY 1.43 1.50 3.34 55.35 14.78%
  Health Department 1.63 2.00 7.19 61.93 87.50%
  Coalition 1.43 1.57 4.14 22.00 28.57%
  Agency 1.39 1.40 2.61 30.00 1.09%
 NC 1.35 1.55 4.65 79.75 12.90%
  Health Department 2.00 2.33 9.00 98.00 50.00%
  Coalition 1.67 1.67 4.00 25.00 33.33%
  Agency 1.14 1.32 3.55 0.00%
 TN 1.46 1.57 3.36 43.83 14.44%
  Health Department 1.85 1.90 5.95 107.75 25.00%
  Coalition 2.13 2.13 5.13 11.88 50.00%
  Agency 1.11 1.28 1.87 0.00%
Open in a new tab

Note. Due to suppression, details about enrollment from IL and PA have been aggregated with states whose vulnerability scores were similar to those of IL and PA. Unless otherwise specified, all values reported are averages. Blanks in the Time till Enrollment reflect unsuccessful recruitment attempts.

Table 1 also shows the regional and state variability in number of attempts, time to recruit, and recruitment rate. For example, we made the fewest contact attempts in the South (M = 3.66) and the most in Appalachia (M = 5.09). We took the shortest time to recruit in Indiana (M = 30.67) and the longest in Missouri (M = 82). Our recruitment success was lowest in Michigan (8%) and highest in West Virginia (18%). This variability was important to allow for the analyses of predictors of number of attempts, time to recruit, and enrollment success, which we conducted next.

Predictors of Recruitment

Overall, the number of attempts made was positively associated with time to recruit (r = .68, p = .01). However, the number of attempts and time to recruit were not associated with enrollment success (r = .27, p = .36 and r = .28, p = .36, respectively). We were therefore interested in determining whether aspects of our recruitment method predicted these indices of recruitment outcomes. In doing so, we conducted a series of multilevel models, including the vulnerability score of each state, the type of organization (health department, coalition, agency), the number of recruiters we used in each attempt, and the number of methods we used to recruit agencies (including email, phone, and Zoom), as well as county (Level 2) and state (Level 3), to examine whether these methodological variables predicted recruitment success. See Table 2 for the model details. Not surprisingly, number of attempts increased with number of recruiters and methods used, as well as when health departments were targeted as opposed to coalitions or agencies. Similarly, the time taken to recruit was longer, but enrollment success was higher, when more methods were used and when targeting health departments relative to other coalitions or agencies. These findings were homogeneous across county and state.

Table 2.

Predictors of Number of Attempts, Time Taken to Recruit, and Enrollment Success

Attempts Made Time to Recruit Enrollment Success
β SE p-value β SE p-value β SE p-value
Fixed Effects
Intercept 1.66 0.09 <0.0001 −6.57 0.35 <0.0001 −1.50 0.43 <0.01
Organization-Agency −0.63 0.07 <0.0001 −2.63 0.57 <0.001 −2.93 0.64 <0.001
Organization-Coalition −0.47 0.10 <0.0001 0.61 0.46 0.20 0.07 0.60 0.90
Recruiters 0.40 0.07 <0.001 0.24 0.33 0.47 0.43 0.38 0.26
Method 0.41 0.07 <0.0001 1.57 0.30 <0.0001 2.06 0.43 <0.0001
Vulnerability Rank 0.00 0.00 0.01 0.00 0.00 0.97 0.00 0.00 0.83
Error Variance
Level-2 (County/FIPS)
Intercept 0.03 0.02 0.03 0.18 0.36 0.58 0.00 . 1.00
Organization 0.04 0.02 0.01
Recruiters 0.06 0.04 0.03 0.85 0.63 0.08
Method 0.06 0.03 <0.01
Vulnerability Rank
Level-3 (State)
Intercept 0.04 0.03 <0.0001 0.00 . 1.00 0.00 . 1.00
Organization 0.39 0.26 0.04 0.64 0.43 0.04
Recruiters
Method
Vulnerability Rank 0.00 . 1.00 0.00 0.00 1.00
Model Fit
AIC 2195.10
BIC 2201.30
−2 Res Log Pseudo L 3572.09 3257.53
Open in a new tab

Note. The vulnerability rank was obtained from Van Handel and colleagues (2016) and reverse-coded so that higher scores represented greater vulnerability. For the model predicting time to enroll, the model with a random slope for Rank at Level 2 did not converge, so no results are presented here

Bias Assessment

Finally, we were interested in assessing whether the potential for our method to succeed differed by the vulnerability of a county. Therefore, Van Handel and colleague’s (2016) vulnerability rank was also included in our analysis in Table 2. For ease of interpretation, we reverse-coded the original vulnerability score so that higher scores represented greater vulnerability. Consistent with this possibility, more contact attempts were required when a county had a higher vulnerability index (β = 0.0017, SE = 0.0006, p = .01). However, this effect was small, and neither time to recruit nor enrollment success differed between counties of different vulnerabilities.

Discussion

We investigated the feasibility of recruiting for a large network of communities in multiple regions to address a common problem and the results from our recruitment efforts led to several important conclusions. First, we found that it took us an average of 4.05 recruitment attempts per county to meet 74% of our recruitment goal in only 54.59 days, meeting or exceeding all existing benchmarks in the literature (e.g., Horvath et al., 2012; Monaghan et al., 2007; Noguchi et al., 2007). Second, we found that community partners from health departments required a higher number of attempts, and took longer to recruit, compared to partners from either coalitions or agencies, but their enrollment probability was highest. In contrast, community partners from agencies required a lower number of attempts, and took less time to recruit, but their enrollment probability was lower.

This difference in recruitment outcomes among partners from health departments and those from coalitions or agencies could potentially reflect the differences in organizational structure and their decision-making process. Specifically, given that local health departments tend to be larger, more structured, and funded by government entities, engaging partners from health departments required more attempts and time to reach a decision-maker who could approve the partnership and often required approval from several stakeholders (e.g., county-level director, state-level director), compared to partners from coalitions and agencies. This longer time did not necessarily reflect lack of interest on the part of health departments, however, as their enrollment rate was higher than that of community partners from coalitions and agencies. Therefore, forming partnerships with local health departments is vital in generating the resources and perspectives to craft community solutions (Lasker & Weiss, 2003; Minkler & Wallerstein, 2005), and our results find that our approach is viable, even when done largely virtually.

We also found that, although using multiple recruitment methods did not reduce the time it took to enroll a community partner, it did improve enrollment success. Other recruitment and retention studies have similarly found that using multiple modes of contact with participants is a key factor to maintaining high levels of engagement (Horvath et al., 2012). It is important to note that our recruitment efforts were largely costless given that we relied on digitally mediated communication tools and there were numerous options (e.g., Gmail, outlook, Google voice, Google hangout, zoom) that offered free services for making phone calls, sending emails, and hosting online conference meetings.

We further found that community partners from counties that were more vulnerable to HCV outbreaks required more attempts to recruit but did not vary in the time needed to recruit or the enrollment success. These findings suggest that, although the probability between recruiting a higher or lower risk county did not differ, higher risk counties are more difficult to reach, consistent with prior work (Earl et al., 2009; Noguchi et al, 2007; Wilson & Albarracín, 2015). Finally, although we contacted various types of coalitions and agencies that represent aspects of community well-being, we had more success in recruiting coalitions and agencies that directly dealt with substance use and associated health problems, which constituted 88% of the coalitions/agencies that joined our network. Other coalitions/agencies we recruited included educational and religious institutions (22%). These findings suggest that the alignment between the agenda of the network and the agenda of the community organizations facilitates recruitment outcomes, such that the more the network’s goals fit with pre-existing goals of community organizations, the more likely the organizations are to join.

Many collaborations often struggle to find ways to enable diverse participants to work together productively and to sustain their collaborative efforts over time (Okubo & Weidman, 2000). As our advisory board members are geographically distributed across thirteen states, we need a system to keep everyone abreast of the project and facilitate interactions. We have thus developed the Board Interaction System (BIS), a virtual meeting space for the research team and enrolled members, including leaders and community members, to work together and advance health solutions. The BIS will allow members to stay updated on information and facilitate community engagement, by enabling board members to remain abreast of the project and provide their advice on features of the development of the project. The BIS will also include training materials1 and videos 2 to provide members with a better understanding of the project, advisory board, and platform, as well as skills useful for engaging in research activities. As our project progresses, we will be able to evaluate whether the use of our BIS enhances retention and collaboration during our project.

Although this paper analyzed the outcomes of using a digitally distributed method to recruit a geographically dispersed network of community organizations, our analyses are not without limitations. Specifically, our data do not provide a conclusive answer as to why some community organizations (i.e., health departments) took longer and higher number of attempts to recruit than did others (i.e., coalitions/agencies). Several organization-level characteristics, such as the size, structure, and the availability of funding could systematically influence organizations’ interests and likelihood of joining a research partnership. Future research could pay more granular attention to these characteristics and identify optimal strategies to recruit different types of organizations. Likewise, organizational-level characteristics could also affect retention in the activities of the community network. For example, partners from health departments may be less likely to follow through the activities compared to partners from coalitions and agencies, as their time is distributed across multiple health issues (some of which are unanticipated) within communities. We plan on examining this possibility as we continue our collaborative partnership with community organizations. Lastly, despite our success in engaging a relatively large network of community partners surrounding the Appalachian region, we look forward to future attempts in utilizing the digitally distributed method to recruit communities in other regions. Community norms and culture can indeed influence their receptivity to different methods, such that some regions may be more open to digitally mediated communications whereas others may prefer more traditional methods including in-person visits and meetings.

Concluding Remarks

The opioid crisis poses a significant health threat in the United States. Despite a sense of urgency among researchers, policy makers, and communities, the implementation of effective evidence-based practices to reduce infections and overdose within communities remains suboptimal. Given the potential for community partnerships to bridge the gap between research and practice, it is imperative that investments be made to foster the inclusion of community members in the development of priorities that might affect social and public health services. In this paper, we described an innovative model to support sustainable, meaningful recruitment of geographically dispersed community partners for research planning and activities. In a period of five months, we were able to recruit organizations from 73 of out 99 counties across thirteen states, showing that a digitally distributed recruitment method can be successful. Findings from this study have the potential to advance multi-state collaborative research and develop an intervention model that other communities can use to address the opioid epidemic, the current COVID-19 crisis, and other health issues affecting vulnerable communities.

Supplementary Material

Appendix

Acknowledgments

Funding

This study was funding by Grant DP1DA048570 (awarded to Dolores Albarracín) from the National Institute of Health.

Footnotes

Ethics approval

This study was performed in line with the principles of the Declaration of Helsinki. Approval was granted by the Ethics Committee of the University of Pennsylvania (No. 849203).

Consent to participate

Informed consent was obtained from all individual participants included in the study.

Author Declarations

Conflicts of interest/Competing interests

The authors have no relevant financial or non-financial interests to disclose.

1

These materials include short and long versions of self-paced introduction materials that cover topics including participatory action research, research ethics, compliance training, and using Zoom.

2

These videos were created using an iterative process. First, we created a list of topics that should be covered. For each topic, we selected pre-existing videos that could be useful. We then contacted the owners of these videos to obtain permission to use specific clips. This step typically involved an internal review by the owner's institution to ensure that there would be no violation of third-party copyright laws. If we failed to obtain permission, we used an alternative video and repeated the process. We then combined these videos to generate one video for each topic.

References

  1. Albert S, Brason FW, Sanford CK, Dasgupta N, Graham J, & Lovette B. (2011). Project Lazarus: community-based overdose prevention in rural North Carolina. Pain Medicine, 12(suppl_2), S77–S85. doi: 10.1111/j.1526-4637.2011.01128.x [DOI] [PubMed] [Google Scholar]
  2. Atkins A, McClung RP, Kilkenny M, Bernstein K, Willenburg K, Edwards A, & Oster AM (2020). Notes from the field: outbreak of human immunodeficiency virus infection among persons who inject drugs—Cabell County, West Virginia, 2018–2019. Morbidity and Mortality Weekly Report, 69(16), 499–500. doi: 10.15585/mmwr.mm6916a2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Belone L, Lucero JE, Duran B, Tafoya G, Baker EA, Chan D, & Wallerstein N (2016). Community-based participatory research conceptual model: Community partner consultation and face validity. Qualitative Health Research, 26(1), 117–135. doi: 10.1177/1049732314557084 [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Centers for Disease Control and Prevention. (2018). Viral Hepatitis Surveillance Report 2018 — Hepatitis C [Data set]. Centers for Disease Control and Prevention. https://www.cdc.gov/hepatitis/statistics/2018surveillance/HepC.htm#Table3.1 [Google Scholar]
  5. Centers for Disease Control and Prevention. (n.d.) Communities Are Leading the Way to Prevent Youth Substance Use. Retrieved January 14, 2022, from https://www.cdc.gov/drugoverdose/featured-topics/drug-free-communities.html
  6. Conrad C, Bradley HM, Broz D, Buddha S, Chapman EL, Galang RR, & Duwve JM (2015). Community outbreak of HIV infection linked to injection drug use of oxymorphone—Indiana, 2015. Morbidity and Mortality Weekly Report, 64(16), 443–444. [PMC free article] [PubMed] [Google Scholar]
  7. Cunningham J, Miller ST, Joosten Y, Elzey JD, Israel T, King C, & Wilkins CH. (2015). Community‐Engaged Strategies to Promote Relevance of Research Capacity‐Building Efforts Targeting Community Organizations. Clinical and Translational Science, 8(5), 513–517. doi: 10.1111/cts.12274 [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Dalessandro C. (2018). Recruitment Tools for Reaching Millennials: The Digital Difference. International Journal of Qualitative Methods, 17(1), 1609406918774446. [Google Scholar]
  9. Drahota AMY, Meza RD, Brikho B, Naaf M, Estabillo JA, Gomez ED, & Aarons GA (2016). Community‐academic partnerships: A systematic review of the state of the literature and recommendations for future research. The Milbank Quarterly, 94(1), 163–214. doi: 10.1111/1468-0009.12184 [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Earl A, Albarracín D, Durantini MR, Gunnoe JB, Leeper J, & Levitt JH (2009). Participation in counseling programs: High-risk participants are reluctant to accept HIV-prevention counseling. Journal of Consulting and Clinical Psychology, 77(4), 668–679. doi: 10.1037/a0015763 [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Festinger L. (1964). Conflict, decision, and dissonance. Stanford, CA: Stanford University Press. [Google Scholar]
  12. Fetterman Z, & Hart W. (2020). Selective exposure in the domain of health. In Sweeny K, Robbins ML, & Cohen LM (Eds.), The Wiley Encyclopedia of Health Psychology (pp. 579–585). Hoboken, NJ: John Wiley & Sons Ltd. [Google Scholar]
  13. Fortuna K, Barr P, Goldstein C, Walker R, Brewer L, Zagaria A, & Bartels S. (2019). Application of community-engaged research to inform the development and implementation of a peer-delivered mobile health intervention for adults with serious mental illness. Journal of Participatory Medicine, 11(1), e12380. doi: 10.2196/12380 [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Gordon JS, Akers L, Severson HH, Danaher BG, & Boles SM (2006). Successful participant recruitment strategies for an online smokeless tobacco cessation program. Nicotine & Tobacco Research, 8(Suppl_1), S35–S41. doi: 10.1080/14622200601039014 [DOI] [PubMed] [Google Scholar]
  15. Graham A, Milner P, Saul J, & Pfaff L. (2008). Online advertising as a public health and recruitment tool: comparison of different media campaigns to increase demand for smoking cessation interventions. Journal of Medical Internet Research, 10(5), e50. doi: 10.2196/jmir.1001 [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Hart W, Albarracín D, Eagly AH, Brechan I, Lindberg MJ, & Merrill L. (2009). Feeling validated versus being correct: A meta-analysis of selective exposure to information. Psychological Bulletin, 135(4), 555–588. doi: 10.1037/a0015701 [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Hoots BE, Xu L, Kariisa M, Wilson NO, Rudd RA, Scholl L, & Seth P (2018). 2018 Annual surveillance report of drug-related risks and outcomes--United States [Data set]. Centers for Disease Control and Prevention. https://stacks.cdc.gov/view/cdc/58547 [Google Scholar]
  18. Horvath KJ, Nygaard K, Danilenko GP, Goknur S, Oakes JM, & Rosser BS (2012). Strategies to retain participants in a long-term HIV prevention randomized controlled trial: Lessons from the MINTS-II study. AIDS and Behavior, 16(2), 469–479. doi: 10.1007/s10461-011-9957-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Kaiser BL, Thomas GR, & Bowers BJ (2017). A case study of engaging hard‐to‐reach participants in the research process: Community advisors on research design and strategies (CARDS)®. Research in Nursing & Health, 40(1), 70–79. doi: 10.1002/nur.21753 [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Khodyakov D, Bromley E, Evans SK, & Sieck K. (2018). Best practices for participant and stakeholder engagement in the All of Us Research Program. RAND. doi: 10.7249/RR2578 [DOI]
  21. Lasker RD, & Weiss ES (2003). Broadening participation in community problem solving: A multidisciplinary model to support collaborative practice and research. Journal of Urban Health, 80(1), 14–60. doi: 10.1093/jurban/jtg014 [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Lucero J, Wallerstein N, Duran B, Alegria M, Greene-Moton E, Israel B, Kastelic S, Magarati M, Oetzel J, Pearson C, Schulz A, Villegas M, & White Hat ER (2018). Development of a mixed methods investigation of process and outcomes of community-based participatory research. Journal of Mixed Methods Research, 12(1), 55–74. doi: 10.1177/1558689816633309 [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Martinez LS, Rapkin BD, Young A, Freisthler B, Glasgow LS, Hunt T, & Battaglia T (2020). Community engagement to implement evidence-based practices in the HEALing communities study. Drug and Alcohol Dependence, 217, 108326. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Meit M, Heffernan M, & Tanenbaum E. (2019). Investigating the impact of the diseases of despair in Appalachia. Journal of Appalachian Health, 1(2), 7–18. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Minkler M, & Wallerstein N. (2005). Improving health through community organization and community building: A health perspective. In Minkler M. (Ed.), Community Organizing and Community Building for Health (pp. 26–50). Piscataway, NJ: Rutgers University Press. [Google Scholar]
  26. Mitchell SM, & Shortell SM (2000). The governance and management of effective community health partnerships: A typology for research, policy, and practice. The Milbank Quarterly, 78(2), 241–289. doi: 10.1111/1468-0009.00170 [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Monaghan H, Richens A, Colman S, Currie R, Girgis S, Jayne K, & Patel A (2007). A randomised trial of the effects of an additional communication strategy on recruitment into a large-scale, multi-centre trial. Contemporary Clinical Trials, 28(1), 1–5. doi: 10.1016/j.cct.2006.06.004 [DOI] [PubMed] [Google Scholar]
  28. National Institute of Health. (2011). Principles of Community Engagement [Data set]. National Institute of Health. https://www.atsdr.cdc.gov/communityengagement/pdf/PCE_Report_508_FINAL.pdf
  29. Noguchi K, Albarracín D, Durantini MR, & Glasman LR (2007). Who participates in which health promotion programs? A meta-analysis of motivations underlying enrollment and retention in HIV-prevention interventions. Psychological Bulletin, 133(6), 955–975. doi: 10.1037/0033-2909.133.6.955 [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. O’Connor S, Hanlon P, O’Donnell CA, Garcia S, Glanville J, & Mair FS (2016). Understanding factors affecting patient and public engagement and recruitment to digital health interventions: a systematic review of qualitative studies. BMC Medical Informatics and Decision Making, 16(1), 1–15. doi: 10.1186/s12911-016-0359-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Okubo D, & Weidman K. (2000). Engaging the community in core public health functions. National Civic Review, 89(4), 309–326. doi: 10.1002/ncr.89403 [DOI] [Google Scholar]
  32. Pew Research Center. (2019). Internet/Broadband Fact Sheet [Data set]. Pew Research Center. https://www.pewresearch.org/internet/fact-sheet/internet-broadband/
  33. Pinto RM, Spector AY, Rahman R, Gastolomendo JD, K L, L S, & A PE. (2015). Research advisory board members’ contributions and expectations in the USA. Health Promotion International, 30(2), 328–338. doi: 10.1093/heapro/dat042 [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Pinto RM, Wall MM, & Spector AY (2014). Modeling the structure of partnership between researchers and front-line service providers: Strengthening collaborative public health research. Journal of Mixed Methods Research, 8(1), 83–106. doi: 10.1177/1558689813490835 [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Pratt B, Cheah PY, & Marsh V. (2020). Solidarity and community engagement in global health research. The American Journal of Bioethics, 20(5), 43–56. doi: 10.1080/15265161.2020.1745930 [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Ramo DE, Hall SM, & Prochaska JJ (2010). Reaching young adult smokers through the internet: comparison of three recruitment mechanisms. Nicotine & Tobacco Research, 12(7), 768–775. doi: 10.1093/ntr/ntq086 [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Ramo DE, Rodriguez TM, Chavez K, Sommer MJ, & Prochaska JJ (2014). Facebook recruitment of young adult smokers for a cessation trial: Methods, metrics, and lessons learned. Internet Interventions, 1(2), 58–64. doi: 10.1016/j.invent.2014.05.001 [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Rhodes SD, Bowie DA, & Hergenrather KC (2003). Collecting behavioural data using the world wide web: considerations for researchers. Journal of Epidemiology & Community Health, 57(1), 68–73. doi: 10.1136/jech.57.1.68 [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Rhodes SD, Tanner AE, Mann-Jackson L, Alonzo J, Horridge DN, Van Dam CN, & Andrade M (2018). Community-engaged research as an approach to expedite advances in HIV prevention, care, and treatment: A call to action. AIDS Education and Prevention, 30(3), 243–253. doi: 10.1521/aeap.2018.30.3.243 [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Richardson WC, & Allegrante JP (2000). Shaping the future of health through global partnerships. In Koop CE, Pearson CE, & Schwarz MR (Eds.), Critical Issues in Global Health (pp. 375–383). San Francisco, CA: Jossey-Bass. [Google Scholar]
  41. Ryan GS (2013). Online social networks for patient involvement and recruitment in clinical research. Nurse Researcher, 21(1), 35–39. doi: 10.7748/nr2013.09.21.1.35.e302 [DOI] [PubMed] [Google Scholar]
  42. Temple EC, & Brown RF (2011). A comparison of internet-based participant recruitment methods: Engaging the hidden population of cannabis users in research. Journal of Research Practice, 7(2), D2–D2. [Google Scholar]
  43. Van Handel MM, Rose CE, Hallisey EJ, Kolling JL, Zibbell JE, Lewis B, & Brooks JT. (2016). County-level vulnerability assessment for rapid dissemination of HIV or HCV infections among persons who inject drugs, United States. Journal of Acquired Immune Deficiency Syndromes, 73(3), 323–331. doi: 10.1097/QAI.0000000000001098 [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Viswanathan M, Ammerman A, Eng E, Gartlehner G, Lohr KN, Griffith D, Rhodes SD, Samuel-Hodge C, Maty S, Lux L, Webb L, Sutton SF, Swinson T, Jackman A, & Whitener L. (2004). Community-based participatory research: Assessing the evidence. AHRQ Evidence Report Summaries, 99. [PMC free article] [PubMed] [Google Scholar]
  45. Wilson K, & Albarracín D. (2015). Barriers to accessing HIV-prevention in clinic settings: Higher alcohol use and more sex partners predict decreased exposure to HIV-prevention counseling. Psychology, Health & Medicine, 20(1), 87–96. doi: 10.1080/13548506.2014.902484 [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Windsor LC, Benoit E, Smith D, Pinto RM, & Kugler KC (2018). Optimizing a community-engaged multi-level group intervention to reduce substance use: an application of the multiphase optimization strategy. Trials, 19(1), 1–15. doi: 10.1186/s13063-018-2624-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Yingling LR, Brooks AT, Wallen GR, Peters-Lawrence M, McClurkin M, Cooper-McCann R, & Powell-Wiley TM (2016). Community engagement to optimize the use of web-based and wearable technology in a cardiovascular health and needs assessment study: a mixed methods approach. JMIR mHealth and uHealth, 4(2), e38. doi: 10.2196/mhealth.4489 [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Zibbell JE, Iqbal K, Patel RC, Suryaprasad A, Sanders KJ, Moore-Moravian L, & Holtzman D (2015). Increases in hepatitis C virus infection related to injection drug use among persons aged≤ 30 years—Kentucky, Tennessee, Virginia, and West Virginia, 2006–2012. Morbidity and Mortality Weekly Report, 64(17), 453–458. [PMC free article] [PubMed] [Google Scholar]
  49. Zimmerman EB, Rafie CL, Moser DE, Hargrove A, Noe T, & Mills CA (2020). Participatory Action Planning to Address the Opioid Crisis in a Rural Virginia Community Using the SEED Method. Journal of Participatory Research Methods, 1(1), 13182. [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Appendix
NIHMS1788742-supplement-Appendix.docx (20.3KB, docx)

RESOURCES