Inclusive Recruitment Strategies to Maximize Sociodemographic Diversity among Participants: A St. Louis Case Study

Abstract

Background. Sociodemographically diverse study samples are critical for research related to health decision making. However, not all researchers have the training, capacity, and funding to engage research methods that recruit the most diverse populations. Objective and Methods. We used participant-generated data, staff salary data, and participant observation to examine the effectiveness and cost of strategies that we used for screening, enrolling, and retaining a sociodemographically diverse sample for a risk communication and behavior change randomized controlled trial. Results. It took approximately 646 hours to contact 1,626 individuals and enroll 554 participants (505 of whom completed the baseline survey; 45.2% were members of a underrepresented racial/ethnic group, 19.4% had no college education, 49.5% were age 30–49 y). Retention at 90-d follow-up was 93%. The total cost was USD$19,898.50. The average cost was $35.92 per participant enrolled. In-person recruitment was most successful in identifying the largest proportion of screened and eligible participants who were members of underrepresented racial/ethnic populations (32.8% and 27.8%, respectively) and with no college experience (39.7% and 33.5%, respectively); it also had the highest total cost ($8,079.17). Existing research pools identified the largest proportion of younger participants (ages 30–49 y; 39.3% and 43.4% for screened and eligible, respectively). Existing listservs yielded the smallest proportion of individuals with no college experience and the fewest members of underrepresented racial/ethnic populations but had the lowest total cost ($290.33). Newspaper ads identified the fewest younger individuals and also had the highest cost per participant enrolled ($166.21). Word of mouth had the lowest cost per participant enrolled ($10.47). Conclusion. Results help medical decision-making researchers formulate recruitment plans that increase sociodemographic diversity in study samples. We also ask funders to accommodate increased costs required to maximize sociodemographic diversity in medical decision-making research.

Highlights

• We provide concrete strategies for recruiting, enrolling, and retaining a sociodemographically diverse study sample.

• We offer cost estimates for all stages of study recruitment and found that in-person recruitment was the most effective, but also the most expensive, way to identify Black participants and participants with no college experience.

• It is critical for investigators to have access to institutional infrastructure and resources to support conducting research that is inclusive of diverse sociodemographic groups.

• An intentionally diverse recruitment staff supports a diverse study sample.

Justice is a central principle of ethical conduct in human subjects research. ¹ Specifically, “fair subject selection requires that the scientific goals of the study, not vulnerability, privilege, or other factors unrelated to the purposes of the research, be the primary basis for determining the groups and individuals that will be recruited and enrolled.” ² Ensuring that research studies enroll adequate numbers of participants from underrepresented sociodemographic groups not only is the right thing to do but also is crucial for scientific advancement. Nevertheless, many studies do not meet recruitment goals in this area.^3–5

The gold standard toward improving diversity in study samples and developing equitable research designs is community-based participatory research (CBPR). ⁶ Through this collaborative approach, community members are actively engaged in the research process, working side by side with researchers to define the research question, design and implement the research methodology, and interpret and disseminate the findings.^7,8 Critically, an “engaged process” includes “meaningful engagement of patients (or the community/population impacted by the proposed work), including efforts for the development of processes for continued communication, decision-making, shared vision, change management, and implementation (at initiation).” ⁹ Well-executed engaged research however, “is not easy” ⁹ ; this is particularly true of racially stratified and intentionally divested regions across the United States such as the Greater St. Louis area.

The sociocultural, historical, and political landscape of St. Louis significantly influences the enrollment of sociodemographically diverse research participants. Disparities exist and continue to widen because of gaps in “education, economic status, employment, and housing” and other social factors such as discrimination and racism. ¹⁰ Within this landscape, the vast majority of the St. Louis population identifies as either non-Hispanic white (46%) or non-Hispanic Black/African American (45%). ¹¹ Racist zoning laws and discriminatory housing practices (e.g., “redlining” and “restrictive covenants”) in St. Louis and other US cities have produced and maintained residential segregation, resulting in clearly identifiable Black and white neighborhoods and social and health inequities^12,13 and “omissions.” ¹⁴ The relatively few residents in St. Louis who identify as races and ethnicities other than non-Hispanic white or non-Hispanic Black/African American (7%) typically reside in historically redlined neighborhoods.

As much as any individual researcher may recognize the value of engaged research and endorse its principles, these contextual realities make it challenging to recruit meaningful sociodemographic diversity and conduct CBPR or a fully “engaged process” research methodology. ⁹ Due to time and cost constraints, disciplinary boundaries, lack of training, ethical concerns associated with the methodology, geographic location, or underdeveloped relationships with community stakeholders, CBPR is not always feasible (or appropriate) to deploy by the principal investigator.^6,15,16 Early-career investigators may be particularly influenced by such logistical challenges—but these challenges ought not imply that researchers acquiesce and avoid promoting meaningful sociodemographic diversity.

In this spirit, this article offers concrete and actionable strategies that researchers of health decision making can use to involve sociodemographically diverse research participants in interventions when it is not possible to employ a fully engaged approach. To accomplish this, we offer a case study analysis that illustrates recruitment strategies for a randomized controlled trial (RCT) focused on risk communication and physical activity behavior change. Although every study has its own goals and constraints, this case study aims to provide researchers of health decision making valuable information to assist in developing new research projects, preparing grant proposals, and executing rigorous research that enrolls and retains a sociodemographically diverse sample of participants. Our objectives were the following:

1. to describe which processes and strategies were more and less effective in screening, enrolling, and retaining a sociodemographically diverse study sample over a 13-month period and

2. to estimate the per-participant costs (including researcher time, study supplies, and recruitment expenses) of different recruitment and retention processes and strategies for different population segments.

Methods

All methods, procedures, and materials were approved by the Washington University in St. Louis Institutional Review Board.

Description of Original RCT

The original RCT comprised 2 components, each with their own goal (see the full study descriptions and their effects on key outcomes in Waters et al. ¹⁷ and Ackermann et al. ¹⁸ ). First, we sought to help sociodemographically diverse audiences who did not meet 2008 US national physical activity guidelines ¹⁹ to obtain a more meaningful and useful picture of how important a single health behavior could be for their overall health. Therefore, we created an intervention component that tested which of 3 visual displays (i.e., bulleted list [usual care control], simple table, or risk ladder) was most effective in communicating personalized disease risk information for 5 diseases associated with insufficient physical activity (colon cancer, breast cancer, heart disease, diabetes, and stroke). ¹⁷ Key outcomes were information comprehension and intentions to increase physical activity.

The second goal was to support the participants in increasing physical activity behavior. Thus, we created a second intervention component that guided participants through mental imagery activities that helped them identify, set, achieve, and maintain a physical activity goal over 90 d. ¹⁸ The key outcome was minutes of physical activity per week at 90-d post-baseline. This component included an active control group designed to improve sleep hygiene behavior.

Brief Overview of the Original RCT Process

For the original RCT, we aimed to recruit 550 participants evenly stratified (±5%) by race/ethnicity and formal education between July 2017 and July 2018. Eligibility criteria were: 18 and 64 y of age, exercised less than 150 min in the past 7 d, and had no more than 1 comorbidity for men or no more than 2 comorbidities for women. ⁱ We stratified recruitment by race/ethnicity and education to facilitate quantitative analyses aimed at evaluating the generalizability of the study findings to people who are members of sociodemographic groups who are underrepresented in research. Our study sample was intentionally inclusive of individuals with less formal education and members of underrepresented racial/ethnic groups because these groups are disproportionately affected by the negative health effects of insufficient physical activity. ²⁰

In brief, after we identified individuals eligible to participate in the study, they provided written informed consent and were considered enrolled. Then, participants were randomized to 1 of the 6 intervention conditions (i.e., 3 conditions for intervention component 1 and 2 conditions for intervention component 2) and completed the first intervention component. ¹⁷ Specifically, they used a cellphone’s Web browser containing a personalized risk assessment for several chronic diseases. After completing the assessment, viewing their disease risk, and answering a survey, participants used the cellphone to begin the second intervention component: listening to an audio-recording of a self-regulation intervention that used mental imagery to help develop goals related to exercise or sleep hygiene. ¹⁸ After completing another survey, participants received text message reminders to complete the mental imagery for 3 weeks after the intervention, 4 weekly text message surveys, and a mailed survey 90 days after baseline. Participants were compensated $10 for each text message survey they completed and $20 for completing each longer survey.

Preparing for Recruiting Participants

The sociocultural and historical factors mentioned in the introduction influenced our decisions about the locations we identified for recruitment efforts and our strategy for recruiting participants. Because individuals from underserved racial and ethnic groups and with less formal education are less likely to participate in research,^21–23 we began by engaging the Recruitment Enhancement Core (hereafter REC; https://icts.wustl.edu/items/recruitment-enhancement-core-rec/). REC’s purpose is to engage in a variety of actions that help investigators meet or exceed their recruitment goals, with a particular emphasis on recruiting individuals who have been historically underrepresented in research due to marginalization and mistreatment by the biomedical research community. At the time of the study, the REC employed 1 full-time team member focused on recruiting people from underrepresented sociodemographic groups, 2 full-time team members focused on recruiting for clinical trials broadly, a full-time media consultant, and a database of 14,435 individuals who agreed to be contacted for research studies (hereafter Volunteers for Health Registry).

We began planning recruitment activities in May 2017 with a half-day orientation. The orientation included the principal investigator (PI) of the study (non-Hispanic white woman, St. Louis resident), the project manager (non-Hispanic white woman, St. Louis resident), the REC recruitment specialist focused on increasing representation from underrepresented sociodemographic groups (Black woman, St. Louis resident), 2 additional recruitment specialists (non-Hispanic white women), and 1 graduate research assistant (Black woman, born and raised resident of St. Louis). ⁱⁱ The PI described the purpose of the study, its eligibility and recruitment stratification requirements, and preliminary plans for recruitment. Then, the group brainstormed potential recruitment locations and discussed the logistics of recruiting and collecting data in public areas.

This and other trainings provided the opportunity to discuss the rationale and ethics for our recruitment stratification strategy (i.e., existing health inequities in the St. Louis region), ethics for participant incentives (i.e., needs to be high enough to demonstrate respect for participants’ time, yet not so high as to be coercive), controlling access to technology in the field (e.g., always locking smartphones away until they are ready to use), logistics for field data collection, and additional recruitment sources, strategies, and methods. These trainings allowed the research team to build a comprehensive, multipronged, and dynamic recruitment plan for the next year of data collection. We aimed to enroll 12 participants per week to reach our sample size goal of 500 in 12 months.

Detailed Description of Recruitment Strategy

Since our recruitment strategy was sensitive to the history and cultural context of St. Louis, the result was a collection of locations that we believed would facilitate recruitment of a diverse group of participants. We began by e-mailing members of the Volunteers for Health Registry, individuals from our lab’s previous studies (who agreed to be contacted for future research), and employees from the Department of Internal Medicine listserv. We anticipated that most individuals recruited through the above modalities would have more formal education but that they would represent many racial groups.

Then, team members began distributing recruitment flyers in locations where we had prior success recruiting underrepresented populations for other research studies. We also began field recruitment throughout the St. Louis area, which involved attending community events and businesses to pass out flyers, discuss the study in depth with potential participants, and screen potential participants for eligibility on the spot. At some locations, we recruited, screened, and even completed the in-person portion of the data collection.

We aimed for 1 field recruitment session per week; however, actual field recruitment sessions varied between 0 and 3 per week. We focused on recruiting from local libraries, health care offices, health fairs, concerts, community centers, food pantries, and technical colleges. The sociodemographic characteristics of the individuals recruited at each field location varied based on the type of location. For example, we recruited at technical colleges to identify individuals with less formal education. Often, food pantry sociodemographic features depended on the neighborhood in which the pantry was located but attracted people with both more and less formal education. If we were falling behind our monthly recruitment goal for a particular group, we would concentrate the next month’s efforts on locations that would more likely yield participation. All field recruitment sessions were done in teams of two. Researchers posted flyers opportunistically throughout St. Louis and contacted members off the Volunteers for Health Registry through the duration of the recruitment process. Team members met weekly to discuss best practices, challenges, and changes to the recruitment process.

In August 2017, we advertised in 2 weekly newspapers, the Riverfront Times, a progressive newspaper, and the St. Louis American, a newspaper focused on the city’s Black community. We also advertised on Craigslist, Facebook, and Researchmatch.org. As participants began to enroll in the RCT, we noticed that enrolled study participants were sharing the study with friends and family; thus, many participants were recruited through word of mouth.

Data Sources and Management

Lab members used REDcap to track all participant activities related to the study. The data for this case study included information obtained from our study records and information provided directly by participants during the study procedure. Participant-reported data included information from the risk assessment (i.e., age, sex, race/ethnicity), the baseline survey (i.e., highest level of formal schooling), and the eligibility screener (i.e., employment affiliation: university employee v. non–university employee). Study record data were stored in REDCap databases developed to manage study logistics, including records of all contacts or attempted contacts (e.g., voicemail) with potential participants.

To quantify instances of contact with participants, we reviewed study records to identify and code each relevant instance of contact—or attempted contact—between staff and participants. Each instance was coded as e-mail, text, phone, in-person, or other. “Other” instances were excluded from the analysis and comprised instances in which a previously enrolled participant reached out to inquire about participating in the study again or a staff person reached a phone number that was out of service. Inbound and outbound phone calls and e-mails were counted as an instance of contact, regardless of whether the potential participant responded. For example, if an individual called to inquire about the study because they saw an ad in the newspaper, this would be considered 1 contact.

Instances of contact were further coded as recruitment (instances of contact prior to enrollment) or retention (instances of contact after enrollment but prior to completing all study components). To ensure data quality, 3 coders completed iterative rounds of coding in pairs of two. Coders independently coded a set of records, then met to compare and reconcile differences. The rate of disagreement was measured for each round until it was less than 10%. The remaining records were coded independently by a single coder.

Data Analysis

We calculated frequencies, percentages, means, and standard deviations of recruitment and retention rates and the number of times participants did not arrive for scheduled appointments or canceled the day of the appointment. We categorized this information by recruitment source: social media, flyers, newspaper advertisements, existing research pools, word of mouth, in person, and university list-serve, and by participant demographic characteristics: education (no college experience v. at least some college experience), race/ethnicity (underrepresented racial/ethnic group v. white), sex (female v. male), age (30–49 y v. 50–60 y), and affiliation with the researchers’ university or medical center (yes v. no).

Time and Cost Estimation

To estimate staff time spent on calls, e-mails, and texts for recruitment and retention activities, we asked each research team member (n = 10) to provide an estimate of the most common amount of time spent making recruitment calls, sending recruitment e-mails, and sending recruitment texts. We then averaged these estimates for each method. Individual estimates were consistent across individuals and with the final estimate rounded to the nearest integer (calls = 5 min, e-mail = 5 min, text = 3 min). Using an average salary plus 30% benefits for a hypothetical full-time midlevel research coordinator at our university (at the time of the study; $20 per hour + $6 benefits = $26 per hour), we calculated a staffing cost per minute of recruitment effort. ⁱⁱⁱ We then multiplied this by the number of minutes for each communication type (i.e., calls, e-mails, texts) and the number of participants to come up with total costs, per-participant costs, and study totals subcategorized by recruitment source. We did not incorporate costs associated with retention because there was minimal variation across participants, and most activities were automated, such as text reminders.

Costs for materials and fees were calculated directly from administrative files (including graphic design fees for flyers, printed advertisements, and newspaper advertisements). Social media advertisements included paid Facebook and Craigslist ads. In-person recruitment costs were calculated based on actual time spent in the field, multiplied by the average salary described above. Costs for local travel and room reservation fees were collected from administrative files. The following example demonstrates our cost estimation process:

To derive the costs associated with social media recruitment, we calculated the labor hours spent recruiting participants via social media: 505 phone calls at 5 min each (42 labor hours), 147 e-mails at 5 min each (12 labor hours), and 75 texts at 3 min each (4 labor hours), for a total of 58 labor hours (there were no in-person recruitment costs for social media recruitment). Using the estimated staff hourly rate of $26, we calculated the total cost of labor for social media recruitment at $1,510. We also spent $515.22 on paid social media advertising (e.g., Facebook ads) for a total cost of $2,025. Finally, dividing by the number of participants contacted (n = 207) and enrolled (n = 59) provided a total cost of $9.78 per participant contacted and $34.32 per participant enrolled.

Results

Objective 1

Our study team contacted (or attempted to contact) 1,626 potential participants, screened 1,198 of them (73.7% of those contacted), and found 703 eligible for participation (43.2% of those contacted). We enrolled and randomized 554 participants ^iv (34.1% of those contacted; see CONSORT diagram in online Supplementary Figure A.1). Of all the individuals who were identified as eligible for participation, the highest proportion were recruited by existing research pools (40.1%) or in-person contact with a staff member (18.6%). The lowest proportion were recruited by newspaper advertisements (3.1%) or the university listserv (3.3%).

Figure 1a and b and online Supplementary Table A.1 show the number of screened and eligible individuals per recruitment source by race/ethnicity, education, and age. In-person recruitment identified the largest proportion of screened and eligible participants who were members of underrepresented racial/ethnic populations (32.8% and 27.8%, respectively; see diamond shading). In-person recruitment also yielded the most people with no college experience (39.7% and 33.5%, respectively; see diamond shading). The largest proportion of younger participants (ages 30–49 y) were identified through existing research pools (39.3% and 43.4% for screened and eligible, respectively; see diagonal line shading). The university listserv yielded the smallest proportion of individuals with no college experience and the fewest members of underrepresented racial/ethnic populations (see light gray shading). We saw the smallest proportion of younger participants through newspaper ads (1.2% of those screened and 1.4% of those eligible; see horizontal line shading).

Figure 1.

(a) Number of screened individuals by recruitment source and demographics. (b) Number of eligible individuals by recruitment source and demographics.

The average number of recruitment contacts per participant was 4.1 (s = 2.5), and the average number of retention contacts per participant was 0.6 (s = 0.7). Overall, 173 (26%) of individuals who scheduled an appointment did not show up or canceled on the day of the appointment at least once. For details about how the number of contacts varied by sociodemographic characteristics and enrollment by individuals who missed their initial appointments, see online Supplementary Table A.2. Of the 703 individuals who were eligible for participation, 505 completed the baseline survey (Table 1). At baseline, race/ethnicity (45.2% from underrepresented racial/ethnic group; 54.9% non-Hispanic white) and age (49.5% age 30–49 y and 50.5% age 50–64 y) were equally distributed, but there were disproportionately more participants with at least some college education (80.0%) and who were women (81.6%). The distribution of demographic characteristics was generally consistent across the duration of the study (baseline, text message surveys, 90-d follow-up). Only 37 individuals were lost to follow-up (93% completion rate).

Table 1.

Number of Individuals Screened and Who Completed Different Parts of the Intervention by Demographics ^a

	St. Louis City Population (n = 311,273)	Screened, n (%)	Enrolled and Completed Baseline Survey, n (%)	Completed 2+ Text Messaging Surveys, n (%)	Completed 90-d Follow-up Survey, n (%)
Overall sample		1,198	505	496 (94.3%)	468 (92.7%)
Education
No college	80,677 (36.7%)	285 (23.8%)	98 (19.4%)	92 (19.3%)	90 (19.2%)
At least some college	139,017 (63.3%)	912 (76.1%)	404 (80.0%)	383 (80.5%)	377 (80.6%)
Missing	0 (0%)	1 (0.1%)	3 (0.1%)	1 (0.2%)	1 (0.2%)
Race/ethnicity
Underrepresented group	176,656 (56.8%)	537 (44.8%)	228 (45.2%)	210 (44.1%)	206 (44.0%)
Non-Hispanic white	134,617 (43.3%)	648 (54.1%)	277 (54.9%)	266 (55.9%)	262 (56.0%)
Missing	0 (0%)	13 (1.1%)	0 (0%)	0 (0%)	0 (0%)
Age, y
30–49	87,260 (28.0%)	600 (50.1%)	250 (49.5%)	236 (49.6%)	228 (48.7%)
50–64	60,472 (19.4%)	598 (49.9%)	255 (50.5%)	240 (50.4%)	240 (51.3%)
Missing	0 (0%)	0 (0%)	0 (0%)	0 (0%)	0 (0%)
Gender	(Sex)
Male	150,532 (48.4%)		93 (18.4%)	80 (16.8%)	80 (17.1%)
Female	160,741 (51.6%)		412 (81.6%)	396 (83.2%)	388 (82.9%)
Missing	0 (0%)		0 (0%)	0 (0%)	0 (0%)
University affiliation
Not affiliated			381 (75.4%)	353 (74.2%)	348 (74.4%)
Affiliated			116 (23.0%)	115 (24.2%)	112 (23.9%)
Missing			8 (0.1%)	8 (1.7%)	8 (1.7%)

^aThere were 49 individuals who enrolled in the study but were later found to have been screen fails. We have removed these individuals from the calculation of the “Enrolled and Completed Baseline Survey” column. The objectives of the randomized controlled trial required that we recruit a sample that was not representative of the geographic area where the study was conducted. Nevertheless, we provide information about the sociodemographic distribution of individuals residing in St. Louis City for interested readers (American Community Survey 2014–2018 5-y estimates). ²⁴ For the education distribution of St. Louis City, the percentages are calculated from the population aged 25 y and older (n = 219,694). The percentages presented for age will not add up to 100% because this is presented as out of the total population in St. Louis City of all ages, whereas participants in this study were only of ages 30 to 64 y.

Objective 2

The approximately 646 h spent contacting and enrolling participants equated to about US $19,989.50 total spent and averaged to approximately $12.24 per participant contacted and $35.92 per participant enrolled. When exploring total cost across the entire study by recruitment method, in-person recruitment had the highest total cost at $8,079.17 (Figure 2a). Newspaper advertisements had the highest cost per participant enrolled at $166.21 per participant (Figure 2b). The university listserv had the lowest total cost at $290.33, while word of mouth had the lowest cost per participant enrolled at $10.47 per participant. We did not examine retention costs because retention processes were automated via text messaging, so there was minimal variation across participants. Detailed cost calculations and cost per participant per recruitment strategy can be found in online Supplementary Table A.3.

Figure 2.

(a) Total cost across participants. (b) Cost per enrolled participant.

Discussion

This case study describes the participant recruitment plan and associated cost estimates for a risk communication and behavior change intervention to provide a resource for investigators who wish to conduct health decision-making research with inclusive sociodemographic samples when a CPBR approach is not possible. We do not assert that the strategies described in this article are a substitute for CBPR. When compared with Goodman and Thompson’s continuum of community engagement, our study falls in the most perfunctory stage of engagement, known as non-participation. ⁹ Nevertheless, our overall recruitment plan was generally successful in ethically recruiting a large proportion of historically underrepresented research participants from Black St. Louis communities. Our inclusion of cost estimates for each recruitment strategy and population group is a key contribution to the literature; prior research is often more narrowly focused on incentives, a single recruitment strategy, or a single sociodemographic group ²⁵ (for exceptions, see Carroll et al. ²¹ and Milo Rasouly et al. ²⁶ ). Providing information about which strategies are most effective in identifying the greatest number of eligible participants (Table 2), with information about number of hours and cost per participant enrolled (Supplementary Table A.3), illustrates the importance of a multipronged recruitment strategy.

Table 2.

Number of Individuals Contacted, Screened, and Eligible by Recruitment Source ^a

Recruitment Source	Contacted, n (Column %)	Screened, n (Column %)	Eligible, n (Column %)
Existing research pools	780 (48.0)	449 (37.5)	282 (40.1)
In person	257 (15.8)	254 (21.2)	131 (18.6)
Social media	207 (12.7)	186 (15.5)	104 (14.8)
Flyers	85 (5.2)	109 (9.0)	64 (9.1)
Word of mouth	76 (4.7)	94 (7.9)	60 (8.5)
University listserv	36 (2.2)	41 (3.4)	23 (3.3)
Newspapers	27 (1.7)	33 (2.8)	22 (3.1)
Missing	158 (9.7)	32 (2.7)	17 (2.4)
Total	1626 (100)	1198 (100)	703 (99.9)

^aDue to rounding error, the total column percentages may not add to 100%. Contacted represents individuals who 1) reached out directly to study staff either in person or via phone call or e-mail or 2) received a phone call, e-mail, or in-person interaction by study staff. Screened represents individuals who were evaluated for study eligibility. Eligible represents individuals whose answers to the eligibility screener indicated they could participate in the study.

Our study contrasts with prior research, much of which examined the effectiveness of only a single recruitment strategy, rather than how to successfully leverage multiple recruitment strategies concurrently. ²⁷ Other research also often focuses on only 1 group, such as a specific racial/ethnic group, women, people with disabilities, or seniors.^5,28,29 However, all studies benefit from diverse recruitment across all these groups to maximize generalizability of outcomes, to ensure research is inclusive of all sociodemographic groups, and to produce ethically rigorous science.

Lessons Learned

We hope that the following lessons learned will encourage researchers to improve the sociodemographic diversity and inclusivity of their study samples.

1. In-person recruitment was the most effective way to identify Black participants and participants with no college experience. This is consistent with work completed by researchers who focused on building interpersonal relationships and focused on in-person recruitment.^5,30 However, it was also expensive, because of the staffing time required. But expense and efficiency are not ethical reasons for failing to recruit sufficient participants from sociodemographic groups that are underrepresented in research samples.^1,2 This is especially true when those same groups—in this case, people who are Black and/or have less formal education—are more likely to develop and/or have poor outcomes from the conditions under investigation and thus have the potential for greater benefit from the study interventions.

2. It is critical for investigators to have access to institutional infrastructure and resources to support conducting research that is inclusive of diverse sociodemographic groups. REC staff were quite successful at recruiting participants of color and participants with no more than vocational-technical training. They also made significant efforts to recruit underrepresented groups in person and made significant efforts prior to our study to ensure representation from underrepresented groups in their registry.

3. An intentionally diverse recruitment staff supports a diverse study sample. We understand that all investigators may not have access to strong institutional resources to support inclusive recruitment like REC. In addition, infrastructure support alone is not sufficient to ensure inclusive recruitment. In our study, the lead recruitment coordinator at the REC was a long-term Black resident of St. Louis and deeply engaged in her community. Her intimate knowledge of local history, current events, her meaningful relationships, and dedication to using this knowledge and relationships for supporting our study were essential to the successful recruitment of individuals from underrepresented groups. Another team member (the first author of this article) was also a Black woman and native to St. Louis with similar knowledge and community relationships. Public health literature supports the benefits of race-matched recruitment. ³¹ We strongly suggest that researchers collaborate with Black and Brown individuals trained to work with underrepresented populations.

4. We learned that frequent contact between participants and the research team is vital. Do not remove participants from the study if they miss an initial or follow-up visit (many participants who missed 1 study point still responded at the following study point). Frequent contact by text message at the beginning of our study, combined with weekly incentives to complete surveys by text message, not only maintained engagement by making it easy for participants to remember our study for the final follow-up survey but also demonstrated that we could be trusted to send them their incentives in a timely manner.

5. Finally, “diversity is not a monolith,” ³² and researchers should be explicit about what is meant by “diversity” and inclusive recruitment. Not every method worked for recruiting participants from each sociodemographic category. For example, we were unable to meet our goal of 50% ± 5% of participants with no more than vocational-technical training. Our results indicate that we could have been more successful if we had spent more staff time on in-person recruitment. Similarly, we experienced challenges with recruiting men to our study. Since sex was not an eligibility criterion or stratification variable for the original RCT, we did not ask about it in our screening process. This means we have limited information about why men were less likely to participate. It may be that women were more likely to express interest in the study and complete screening, or it may be that women were more likely to be eligible than men. It may also have been that men need different recruitment methods than we deployed in this study.

Limitations

Missing data limits some of our quantitative analyses. For example, we do not have demographic information on individuals who were not screened for eligibility; this limits our ability to draw inferences about similarities and differences between people who did and did not elect to undergo screening. Poor phone connections, missing or illegible information on paper forms, and limitations in participants’ memories produced sporadic missing data as well. Given the demographic makeup of the St. Louis region, our study sample included few participants who were not Black or white. This limits our knowledge about how these recruitment strategies might speak to people from other racial or ethnic backgrounds. Yet, Black-white segregation maintains a common denominator in other major US cities such as Detroit, Pittsburgh, and Baltimore. ³³ Case study analyses do not include a control group; therefore, it is impossible to quantify the relative gain from these recruitment efforts compared with what would have happened without such efforts. Finally, this study is vulnerable to the “healthy volunteer” bias that is present in most health behavior intervention research.

Conclusion

In 1966 at the Medical Committee for Human Rights, Dr. Martin Luther King, Jr., decreed, “of all forms of inequality, injustice in healthcare is the most shocking and inhuman.” While Dr. King was referencing the US Public Health Services’ Tuskegee Study of Untreated Syphilis in the Negro Male, the impact of racial, socioeconomic, gender, and educational oppression and their intersections continue to plague health care and thwart health equity to this day. Researchers who conduct health decision-making RCTs are well equipped to conduct meaningful research that seeks to eliminate health inequities. As scientists deeply committed to justice and inclusion, we believe that CBPR remains one of the most effective approaches to engage participants from sociodemographic groups that are underrepresented in research and that honors community members as contributors to knowledge production. ⁹ Nevertheless, we hope that the case described in this article encourages investigators to increase representation and inclusive sociodemographic sampling actively and consistently in their health decision-making research—regardless of their access to more engaged research methodologies.

Other locations in the United States struggle with similar challenges when recruiting from underrepresented groups into their large randomized controlled trials. We hope that this article provides specific strategic, budgetary, and timeline guidance for researchers who wish to screen, enroll, and retain participants from underrepresented groups for medical decision-making research. We also hope that this article encourages individuals at funding agencies to advocate for fully funding budgets that accommodate the increased time and expense required by such work and to advocate against large unilateral budget cuts prior to funding. We anticipate that researchers will be inspired to develop more inclusive and representative recruitment methods and not shy away from the important work involved in recruiting a sociodemographically diverse study sample.

Health decision-making interventions that recruit individuals from groups that are historically underrepresented in research will take more time and will be more expensive than interventions that are composed of college-educated white people. Such work is necessary to make true progress toward alleviating health disparities, identifying health omissions, and creating equitable research. To ensure that the benefits and implementation of our research reach all groups historically marginalized by society, we must commit to using all of the equitable and antiracist methodological tools in our behavioral research toolkit.