Abstract
Background/Aims
Research is needed to identify promising recruitment strategies to reach and engage diverse young adults in diabetes clinical research. The aim of this study was to examine the relative strengths and weaknesses of three recruitment strategies used in a diabetes self-management clinical trial: social media advertising (Facebook), targeted mailing, and in-person solicitation of clinic patients.
Methods
Strategies were compared in terms of: (1) cost-effectiveness (i.e., cost of recruitment / number of enrolled participants), (2) ability to yield participants who would not otherwise be reached by alternative strategies, and (3) likelihood of participants recruited through each strategy to adhere to study procedures. We further explored the appeal (overall and among age and gender subgroups) of social media advertisement features.
Results
In-person recruitment of clinic patients was overall the most cost-effective strategy. However, differences in demographic, clinical, and psychosocial characteristics of participants recruited via different strategies suggest that the combination of these approaches yielded a more diverse sample than would any one strategy alone. Once successfully enrolled, there was no difference in study completion and intervention adherence between individuals recruited by the three recruitment strategies.
Conclusions
Ultimately, the utility of a recruitment strategy is defined by its ability to effectively attract people representative of the target population who are willing to enroll in and complete the study. Leveraging a variety of recruitment strategies appears to produce a more representative sample of young adults, including those who are less engaged in diabetes care.
Keywords: Diabetes mellitus, recruitment, randomized controlled trials, social media
Introduction
Young adults (age 18–30) with diabetes face unique challenges including the transition from pediatric to adult health care, elevated prevalence of mental health issues,1,2 the onset of diabetes-related comorbidities and complications,3,4 and social and professional role changes. Considering these challenges, the transition into adulthood is often associated with decreased engagement in medical care,5–7 elevating the importance of supporting young adults with diabetes. Yet, this age group has been largely overlooked with respect to psychosocial and self-management interventions.8,9
Quality randomized controlled trials (RCTs) are needed to address the diabetes care needs of young adults. This progress is often hampered by the challenges of recruiting racially, ethnically, and socioeconomically diverse young adults into research using traditional recruitment strategies.10 The recruitment of under-represented populations is important to improve the external validity of diabetes trials considering its prevalence among non-white, low-income individuals in the United States.11 Research is needed to identify promising recruitment strategies to reach and engage these populations.12
The use of social media for study recruitment is appealing because of the speed and breadth of dissemination, the potential for reaching populations who would not otherwise participate in research, and for producing a high volume of study inquiries with relatively little staff involvement.13–15 Social media may be especially effective in recruiting young adults considering its popularity in this age group.16 Social media has undoubtedly transformed the ways in which young adults communicate, yet little is known about its effectiveness in recruiting economically, racially and ethnically diverse young adults with diabetes in clinical research.
The present study examines and compares the relative strengths and weaknesses of three recruitment strategies- Facebook advertising, targeted mailings, and in-person solicitation- in recruiting diverse young adults with type 1 or type 2 diabetes. Consistent with recent recommendations,17,18 we consider the effectiveness of these approaches in terms of: (1) cost-effectiveness (i.e., total cost of recruitment divided by the number of recruited participants); (2) ability to reach and engage participants who would not otherwise be reached by alternative recruitment strategies; and (3) participants’ adherence to study procedures.
We also examine which components of social media ads were most successful in recruiting participants. There is compelling evidence that health messages are more effective when framed in ways that match recipients’ motives,19,20 cultural values,21,22 gender and age. Extending the model of precision medicine to recruitment, we explore the idea of goodness-of-fit between ad features and individuals’ characteristics on the likelihood of individuals clicking on the ad. We further calculate the cost of ad click as a function of ad features and potential participants’ age and gender.
Methods
Study design
This ancillary study capitalizes on the Resilient, Empowered, Active Living with Diabetes (REAL Diabetes) study23 to examine the relative strengths and weaknesses of Facebook advertising, targeted mailings, and in-person solicitation in recruiting diverse young adults with diabetes. The REAL study, described in detail elsewhere, was a two-arm RCT evaluating the efficacy of an occupational therapy diabetes management intervention tailored to low-income and racial/ethnic minority young adults with type 1 or type 2 diabetes.23 Main findings revealed that the intervention improved blood glucose levels (HbA1c) and diabetes-related quality of life.24 Recruitment took place between October 2014 and December 2015. Initially, participants were recruited from partnering diabetes clinics. Targeted mailings to clinic patients and Facebook advertisements were eventually added to bolster the study sample. The study was approved by the University of Southern California Institutional Review Board (IRB), and all participants provided written informed consent prior to participation.
Recruitment methods
In-person recruitment
In-person recruitment involved solicitation of clinic patients at two local University-affiliated hospitals: one County hospital (diabetes clinic serving 18–35-year-olds) and one children’s hospital (diabetes clinic serving patients from birth through young adulthood*). At the County hospital, physicians presented the study to patients during their routine appointments from October 2014 to December 2015. Interested patients were introduced to a research assistant who provided additional details about the study, obtained informed consent and collected contact information. At the children’s hospital, research staff approached potentially eligible patients in the waiting room between October 2014 and May 2015. If interested, patients completed enrollment procedures at a later appointment due to lack of private space available.
After 5 months of recruitment, the County hospital went through administrative changes which temporarily, but dramatically, reduced the number of patients attending clinic. Due to these challenges, and to the challenges of recruiting patients at the children’s hospital, two additional recruitment strategies were introduced to ensure that recruitment targets would be met within the study’s targeted timeframe.
Targeted mailings
In June 2015, we replaced in-person solicitation at the children’s hospital with targeted mailings to current and former patients. Packets, including a study brochure and a letter inviting patients to participate in the study, were mailed to patients who had been seen within the past year and provisionally met eligibility criteria. The first mailing was sent to 1000 patients in three waves from July to September 2015. By November 2015, the team had almost reached their recruitment goal. Consequently, a follow-up mailing was only sent to a randomly selected subset of 320 patients.
Social media
In June 2015, the team implemented paid advertising on Facebook to identify prospective participants. Ad themes were brainstormed during research meetings and created using low-cost photo editing tools. Advertisements were directed to targeted audiences based on Facebook’s pre-existing parameters (age, language, geographic location, and users’ “Likes” indicative of an interest in diabetes). Individuals viewing the ad could “click through” to a recruitment website providing additional information about the study and a form to provide contact information for follow-up by research staff. Ad campaigns were monitored and assessed in real time through an advertising dashboard. Based on dashboard results, the team either kept an ad active, or temporarily or permanently replaced it with a new one. These changes were not made on a set schedule, but continuously adapted in response to changes in user engagement.
Data analysis
Our analysis compares social media, mailing, and in-person recruitment on three parameters: (1) cost-effectiveness (i.e., total cost of recruitment divided by the number of enrolled participants); (2) ability to recruit participants who would not otherwise be reached by alternative recruitment strategies; and (3) participants’ adherence to study procedures. To explore the role of message framing in social media recruitment, we examine which social media ad features were most effective in reaching participants (vis-à-vis generating link clicks to our recruitment website), and whether some ad features were more likely to appeal to age and gender subgroups. We finally calculate the cost per unique link click (cost per unique person who clicks a link) as a function of individuals’ age, gender and ad features. Statistical analyses were performed using SAS for Windows version 9.4 (SAS, Cary, NC). A two-sided α-level of 0.05 was considered statistically significant.
Cost-effectiveness (cost per recruited participant)
For each recruitment strategy, costs associated solely with that recruitment approach were calculated; costs common to all approaches (e.g. phone outreach to schedule appointments; time spent completing enrollment paperwork) were omitted from analyses. Calculations of staff time assumed a graduate student worker paid $18 an hour plus fringe benefits. In-person costs included hourly wages for time spent in clinic. Mailing costs included postage and envelopes for mailings, and hourly wages for time spent preparing mailings. Social media costs included advertising fees and hourly wages for staff time spent managing social media ad campaigns. Mailing and social media costs also included hourly wages for time spent conducting eligibility screenings for individuals who responded to ads and mailings.
Subsample characteristics
To determine differences among participants recruited by each strategy, we performed Fisher Exact and Kruskal-Wallis tests on demographic, clinical, and psychosocial characteristics collected at baseline surveys. Data collection procedures and surveys are discussed in detail elsewhere.23
Participant eligibility and engagement
Fisher Exact Test was used to evaluate between-group differences in the proportion of participants who (a) completed eligibility screening; (b) were found to be eligible for the study; (c) if eligible, chose to enroll in the study; and (d) among those who enrolled, completed the study. Among participants randomized to the intervention group, we used a Kruskal-Wallis test to evaluate between-group differences in treatment adherence.
Message framing within social media advertisements
Purchasers of Facebook ads can access summary performance measures for each individual ad version including audience and various marketing statistics (e.g., Impressions: number of times ad was viewed; Reach: number of people who saw the ad at least once; Link clicks: number of clicks on ad links; Unique link clicks: number of unique people who performed a link click; Cost: amount spent on that ad version). We marketed ads to the 18–24 years and 25–34 years age groups. Data for individuals outside of these age groups who saw our ads were excluded from further analysis.
We categorized our advertisements according to language (English or Spanish), use of diabetes-related imagery (e.g. glucometer and test strips) versus generic imagery (e.g., stopwatch), and the presence or absence of six different components in each ad’s caption: (1) time running out; (2) compensation; (3) appealing to altruism; (4) conveying the study team’s empathy; (5) providing a “call to action”; and (6) acknowledging difficult aspects of managing diabetes. Figure 1 provides example of advertisements combining these features. Overall, we created 48 unique ads comprised of various combinations of the above elements.
Figure 1.
Examples of advertisements combining different features used for recruitment purposes: Left: Bilingual, Time running out, Acknowledging difficult aspects of diabetes; Middle: Spanish, Take action steps, Acknowledging difficult aspects of diabetes, Diabetes imagery; Right: English, Help others, Take action steps, Diabetes imagery.
We analyzed unique link clicks as a measure of how compelling our audience found each ad. The variance for unique link clicks per reach (i.e., the ratio of people who clicked the link in an ad relative to the total number of people exposed to the ad) can be calculated from summary data provided by Facebook, along with p-values and confidence intervals. We used logistic regression models to determine the odds of a person who sees a given ad clicking on the link, comparing various audience and ad characteristics. A higher odds ratio indicates a greater chance that the ad would lead to a click through to the study website. We calculated univariate models by gender, age group, ad language, type of imagery and the presence or absence of the six components outlined above. The reference group was the not-present group for dichotomous features and the group with the largest reach for features with more than two categories. To evaluate whether certain ad features differentially appealed to population subgroups, we also considered these models stratified by gender and age (18–24-year-old vs. 25–34-year-old). There were 29 captions, 19 pictures and English/Spanish variations in ads, yet only 48 unique versions were delivered. Therefore, clustering of these characteristics occurred, confounding the relationship between ad characteristics and the odds of producing a click. Because simple adjustment could not correct for the clustering, we show unadjusted results with a caution that the association of any given ad feature may be due to another feature with which it was clustered, rather than an effect of the given feature.
Cost per click as a function of age group, gender and advertisement features
While the ideal measure of ad effectiveness would have been cost per enrolled individual, it is impossible to match ad views or clicks to individuals who contacted the study via our recruitment website. Therefore, we report cost per unique link click as the most proximal measure available. As we do not have individual cost data for each unique click, we could not determine if there were statistically significant differences in cost per unique link click. Thus, we present only mean costs per unique link click by various subgroups, with no conclusions regarding the implication of these means.
Results
Of all individuals (n=231) who were in contact with the study team, 218 individuals were screened and 81 were successfully enrolled in the study. Of the 81 study participants, two recruited from non-systematic methods (costs unknown) were removed from analyses.
Eligibility and engagement
Table 1 outlines retention through each study phase. All individuals solicited at clinics, and most of those reached through mailing (90%) and social media (87%) completed screening. Among all screened individuals, a greater proportion of those solicited in-person (69%) and via mailing (68%) were eligible compared to individuals reached via social media (46%), p<0.01. After determining eligibility, 53% of those solicited in-person declined enrollment, versus 15% and 18% of individuals identified via mailing and social media, respectively (p<0.01). Once enrolled, there was no difference in study completion rates (p=0.38) or intervention adherence (p=0.49 for number of sessions; p=0.77 for total treatment dose) across recruitment strategies.
Table 1.
Likelihood of eligibility, enrollment, retention and adherence by recruitment method (among individuals who were in contact with the study team).
| Data are mean (SD) or n (%) | Overall | In-person | Mailing | Social media | p-valuea |
|---|---|---|---|---|---|
| All potential participants | 231 | 127 | 21 | 83 | |
| Lost to follow-up/ declined before screening | 13 (6) | 0 (0) | 2 (10) | 11 (13) | <0.01 |
| Screened | 218 (94) | 127 (100) | 19 (90) | 72 (87) | |
| Among screened participants | |||||
| Ineligible | 85 (40) | 40 (31) | 6 (32) | 39 (54) | <0.01 |
| Eligible | 133 (60) | 87 (69) | 13 (68) | 33 (46) | |
| Among eligible participants | |||||
| Declined enrollment | 54 (41) | 46 (53) | 2 (15) | 6 (18) | <0.01 |
| Enrolled | 79 (59) | 41 (47) | 11 (85) | 27 (82) | |
| Among enrolled participants | |||||
| Completed | 70 (89) | 37 (90) | 10 (91) | 23 (85) | 0.38 |
| Withdrew/lost to follow-up | 9 (11) | 4 (10) | 1 (9) | 4 (15) | |
| Treatment dose in intervention groupb | n=40 | n=20 | n=8 | n=12 | p-valuec |
| Number of sessions | 10.0 (4.0–14.0) | 12.5 (3.8–14.0) | 7.0 (3.8–10.3) | 8.5 (3.3–14.0) | 0.49 |
| Total treatment dose (HH:MM) | 10:20 (3:30–13:49) | 11:33 (3:31–14:00) | 7:10 (3:29–11:54) | 8:25 (3:30–14:11) | 0.77 |
Fisher Exact Test for global difference between the 3 recruitment strategies
Data are median (interquartile range)
Kruskal-Wallis test for global difference between the 3 recruitment strategies
Cost effectiveness
Table 2 details costs related to each recruitment strategy. Targeted mailing had the lowest cost per participant ($217), but despite sending 1320 letters, only 11 individuals enrolled. In-person recruitment cost $290 per participant and enrolled 41 participants. Social media cost $334 per participant and enrolled 27 participants.
Table 2.
Detailed costs of recruitment strategies.
| In-persona (n=39) | Mailings (n=11) | Social media (n=27) | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Cost per Item | Units | Cost per unit | Total costb | Units | Cost per unit | Total costb | Units | Cost per unit | Total costb |
| Staff timec: in-person recruitment | 472 | 24 | 11328 | -- | -- | -- | -- | -- | -- |
| Staff time: phone eligibility screening | -- | -- | -- | 21 | 24 | 504 | 83 | 24 | 1992 |
| Staff time: in-person eligibility screening | -- | -- | -- | 3.5 | 24 | 84 | 21 | 24 | 504 |
| Staff time: managing social media advertisements | -- | -- | -- | -- | -- | -- | 140 | 24 | 3360 |
| Staff time: preparing mailings | -- | -- | -- | 48 | 24 | 1152 | -- | -- | -- |
| Social media advertisements | -- | -- | -- | -- | -- | -- | -- | -- | 3164 |
| Postage for mailings | -- | -- | -- | 1320 | 0.49 | 647 | -- | -- | -- |
| Overall Cost | |||||||||
| Total cost | 11328 | 2387 | 9020 | ||||||
| Cost per participant | 290 | 217 | 334 | ||||||
| Prospective participants reached | |||||||||
| Total number of clicks | n.a | n.a | 1202906 | ||||||
| Patients approached | 127 | n.a | n.a | ||||||
| Letters mailed | n.a | 1320 | n.a | ||||||
Excludes two people recruited in-person from the Children’s Hospital clinic, as staff time could not be calculated
In US dollars
Staff time is calculated at $24 per hour (including fringe benefits) for graduate student workers for all positions
Subsample characteristics
Participants enrolled via different recruitment strategies differed demographically and clinically (Table 3). Social media reached older participants (p<0.01), and clinic recruitment reached a greater proportion of male participants (p<0.01). The vast majority of clinic recruits (95%) received care from an endocrinologist, versus 45% of participants recruited via mailing and 48% recruited via social media (global p<0.01). Individuals recruited in clinics reported more frequent routine diabetes visits than other participants (p<0.01).
Table 3.
Participant characteristics by recruitment method
| Data are mean ± SD or n (%) | In-person (n=41) | Mailing (n=11) | Social media (n=27) | p-valuea |
|---|---|---|---|---|
| Demographic | ||||
| Age (years) | 21.4 ± 3.0 | 21.5 ± 1.5 | 24.6 ± 3.8 | <0.01 |
| Gender (n=female) | 18 (44) | 8 (73) | 23 (85) | <0.01 |
| Generation | ||||
| Participant born outside United States | 14 (34) | 0 (0) | 6 (22) | 0.06 |
| Neither parent born in United States | 18 (44) | 7 (64) | 9 (33) | |
| At least one parent born in the United States | 9 (22) | 4 (36) | 12 (44) | |
| Race/Ethnicity | ||||
| White | 1 (2) | 1 (9) | 6 (22) | 0.13 |
| Black | 4 (10) | 2 (18) | 2 (7) | |
| Hispanic/Latino | 35 (85) | 8 (73) | 18 (67) | |
| Other | 1 (2) | 0 (0) | 1 (4) | |
| Parents’ highest education level | ||||
| < high school | 16 (46) | 7 (64) | 4 (15) | 0.04 |
| High school degree/GED | 7 (20) | 3 (27) | 7 (27) | |
| Some college, trade, tech | 4 (11) | 0 (0) | 8 (31) | |
| College graduate+ | 8 (23) | 1 (9) | 7 (27) | |
| Neighborhood median household income ($K) | 39.1 ± 12.1 | 44.9 ± 19.2 | 50.8± 17.9 | 0.03 |
| Neighborhood % below Federal Poverty Level | 25.2 ± 9.6 | 25.8 ± 16.1 | 20.6 ± 11.3 | 0.13 |
| Clinical | ||||
| Type 1 Diabetes | 33 (80) | 8 (73) | 19 (70) | 0.62 |
| Type 2 Diabetes | 8 (20) | 3 (27) | 8 (30) | |
| Diabetes duration | 9.5 ± 5.6 | 11.1 ± 4.4 | 9.6 ± 6.6 | 0.61 |
| A1C | 10.6 ± 1.6 | 11.4 ± 2.3 | 10.8 ± 2.1 | 0.66 |
| Blood glucose monitoring (Summary of Diabetes Self-Care Activities, SDSCA)42 | 3.8 ± 2.4 | 2.9 ± 2.7 | 2.8 ± 3.0 | 0.25 |
| Medication adherence (SDSCA) | 6.4 ± 1.4 | 4.6 ± 2.7 | 5.6 ± 1.8 | 0.01 |
| Habit strength for glucose monitoring (Self-Report Behavioural Automaticity Index, SRBAI)43 | 16.8 ± 6.5 | 12.6 ± 7.3 | 12.7 ± 6.6 | 0.03 |
| Habit strength for taking medications (SRBAI) | 19.7 ± 5.7 | 17.4 ± 8.0 | 18.1 ± 6.7 | 0.60 |
| Source of care | ||||
| Endocrinology | 37 (95) | 5 (45) | 13 (48) | <0.01 |
| Primary care | 1 (3) | 5 (45) | 8 (30) | |
| No regular source of care | 0 (0) | 1 (9) | 4 (15) | |
| Unknownb | 1 (3) | 0 (0) | 2 (7) | |
| Healthcare utilization (12mo prior randomization) | ||||
| Number of routine diabetes visits | 3.6 ± 1.5 | 2.4 ± 1.9 | 2.7 ± 1.9 | 0.03 |
| ≥2 visits with A1C recorded ≥3 months apart | 35 (85) | 4 (40) | 9 (36) | <0.01 |
| Participants reporting ≥1 diabetes-related hospitalization | 9 (22) | 2 (18) | 7 (26) | 0.93 |
| Psychosocial | ||||
| Diabetes distress (Problem Areas in Diabetes-Short Form, PAID-SF; range: 0–20) 44 | 8.6 ± 6.0 | 8.5 ± 4.6 | 11.9 ± 5.2 | 0.06 |
| Audit of Diabetes-Dependent Quality of Life (ADDQoL; range: −3 to +9) 45 | −2.3 ± 1.7 | −1.8 ± 1.3 | −3.3 ± 1.7 | 0.02 |
| Diabetes knowledge (Diabetes knowledge Questionnaire, DKQ; range: 0–24) 46 | 17.2 ± 3.3 | 18.3 ± 3.8 | 19.4 ± 2.2 | 0.02 |
| Participation Objective, Participation Subjective (Objective range: weighted z scores −3 to 3, Subjective range −4 to +4) 47 | −0.1 ± 0.2 | −0.1 ± 0.3 | 0.1 ± 0.3 | 0.06 |
Kruskal-Wallace Test or Fisher Exact Test for global difference between the 3 recruitment strategies
Participants had a provider, but specialist vs. primary care physician status could not be determined.
Effectiveness of social media advertisement features and differential appeal by age and gender
Table 4 presents the likelihood of clicking through to the study website across gender, age, and advertisement features, and shows mean cost per unique click through to the study website by gender, age, and ad features. Younger viewers (18–24-year-olds) were more likely to click through than older viewers (25–34 year-olds), and ads in Spanish were less likely to elicit clicks than English ads. Ads mentioning altruism or the study team’s empathy were more likely to elicit clicks. Ads mentioning urgency, a call to action, or difficulty managing diabetes were less likely, to elicit clicks than ads without these components.
Table 4.
Likelihood of clicking through to study website across gender and age subgroups and social media advertisement features.
| Mean cost per unique click (USD $) | Odds ratio (95% CI) for unique clicks per reach | Odds ratio for unique link clicks per reach | ||||||
|---|---|---|---|---|---|---|---|---|
| Male | Female | |||||||
| Age 18–24 | Age 25–34 | Age 18–24 | Age 25–34 | |||||
| Population subgroup | Gendera | Male | 0.44 | 1.01 (0.96–1.06) | ||||
| Female | 0.46 | 1.00 | ||||||
| Age | 18–24 | 0.41 | 1.11 (1.06–1.17) | |||||
| 25–34 | 0.48 | 1.00 | ||||||
| Advertisement features | Language | English | 0.42 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 |
| Spanish or bilingual | 0.47 | 0.88 (0.82, 0.93) | 0.68 (0.59, 0.78) | 1.54 (1.34, 1.76) | 0.70 (0.63, 0.78) | 0.90 (0.82, 0.99) | ||
| Diabetes-related imagery | No | 0.45 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | |
| Yes | 0.44 | 0.96 (0.91–1.00) | 1.22 (1.09, 1.37) | 0.74 (0.66, 0.84) | 1.11 (1.01, 1.23) | 0.88 (0.80, 0.95) | ||
| Compensation | No | 0.43 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | |
| Yes | 0.47 | 1.04 (0.99, 1.09) | 1.50 (1.34, 1.68) | 0.52 (0.46, 0.59) | 1.78 (1.62, 1.96) | 0.89 (0.82, 0.96) | ||
| Urgency/time running out | No | 0.47 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | |
| Yes | 0.39 | 0.54 (0.50–0.57) | 0.46 (0.40, 0.53) | 0.99 | 0.36 (0.32, 0.40) | 0.62 (0.55, 0.71) | ||
| Altruism | No | 0.41 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | |
| Yes | 0.87 | 1.60 (1.49–1.73) | 1.32 (1.10, 1.58) | 2.02 (1.66, 2.47) | 1.43 (1.24, 1.66) | 1.70 (1.51, 1.90) | ||
| Study team’s empathy | No | 0.45 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | |
| Yes | 0.43 | 1.80 (1.69–1.92) | 1.98 (1.72, 2.28) | 2.44 (2.10, 2.85) | 2.08 (1.83, 2.38) | 1.43 (1.30, 1.57) | ||
| Call to action | No | 0.35 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | |
| Yes | 0.53 | 0.53 (0.50–0.56) | 0.85 (0.76, 0.97) | 0.25 (0.22, 0.28) | 1.00 | 0.40 (0.36, 0.43) | ||
| Difficult aspects of managing diabetes | No | 0.56 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | |
| Yes | 0.39 | 0.88 (0.84–0.93) | 1.12 | 0.60 (0.53, 0.68) | 1.06 | 0.85 (0.78, 0.92) | ||
Unknown were excluded from the analyses.
Bold text indicates statistical significance at p<0.05
Men ages 25–34 were more likely to click on ads in Spanish, whereas younger men and women of all ages were less likely to click on Spanish than English ads. Younger viewers were more prone to click on ads mentioning compensation or featuring diabetes imagery. Additionally, lower click-through counts for ads acknowledging difficulty in managing diabetes appear to be driven by older viewers.
Cost per click by age, gender and advertisement features
Costs of eliciting unique clicks ranged from $0.35 to $0.56, except for advertisements mentioning altruism ($0.87). Costs were similar for men ($0.44) and women ($0.46), and slightly higher for older ($0.48) than younger individuals ($0.41).
Discussion
This study examined the relative strengths and weaknesses of three recruitment strategies used in the REAL clinical trial.23 In-person recruitment was the most effective strategy overall, when taking into account both cost and yield of enrolled participants. Targeted mailing had the lowest cost per participant and attracted a greater proportion of eligible participants, but was not viable for recruiting a large sample, as enrollment (n=11) was notably low compared to previous trials using mailings for recruitment.25,26,27 Consistent with previous research,28,29 Facebook advertising was the least cost-effective strategy, likely due to a decreased ability to target potentially eligible participants relative to other recruitment strategies. However, it yielded one-third of the overall sample, utilizing less staff time per enrolled participant than in-person recruitment. Thus, for studies with sufficient budgets but limited staff availability, social media may be viable in recruiting large samples.
The high degree of heterogeneity in the reporting of recruitment costs, and study designs for which recruitment costs are reported, make it challenging to compare across studies. Nevertheless, it is notable that our social media recruitment costs (as well as our recruitment costs using other methods) greatly exceeded those of previous studies, which ranged from $1-$110 per participant.30 We attribute this to several reasons: first, our calculations were inclusive of more costs [e.g., staff time, supplies] than many studies. Second, we recruited individuals with relatively low prevalence conditions from within a defined geographic area, whereas many studies using social media target much broader populations. Third, we were recruiting for an in-person clinical trial with relatively high participant burden, whereas most studies recruiting via social media are conducting surveys, cross-sectional studies, or relatively low-burden clinical trials.
The low eligibility rate of those recruited via social media is not surprising, as patients approached in clinic and via mailing were specifically targeted based on eligibility parameters. Social media advertisements do not allow for such specificity; thus, a large number of ineligible individuals were exposed to ads. This is consistent with previous studies suggesting that social media is useful to generate a large pool of interested individuals, but less effective in yielding eligible participants.13,31–35 Of note, a greater proportion of individuals solicited in person declined study participation, relative to mailings and social media. All clinic patients who provisionally met eligibility criteria were approached, but they were not actively seeking to take part in the study. By contrast, social media and mailing respondents had proactively reached out to the study team with an interest in participating. Once enrolled, however, participants recruited via social media were as likely to complete the study than participants recruited via other strategies, contrasting with previous research suggesting that social media may reach individuals who are less likely to adhere to study procedures than those recruited via other means.31,36
Compared to in-person recruitment, social media and mailings were more effective in recruiting women than men, and participants living in higher income neighborhoods. The difference in neighborhood income is likely attributable to our in-person recruitment taking place within a safety-net setting. The observed gender differences, consistent with previous research,31 may be due to inadequate tailoring of our recruitment materials to appeal to men, or to women’s increased care-seeking relative to men.37–39 Perhaps counterintuitively, participants recruited at the young adult and pediatric clinics were similar in age, and younger than participants recruited through social media. This reflects practices in transition of care in the US generally, and at our recruitment sites specifically, where there is considerable overlap in ages served by pediatric and adult care settings.
Because Facebook pricing is based on a cost per click, developing social media ads that are more appealing will not necessarily affect the cost-effectiveness of this recruitment method. Nevertheless, ads that are appealing overall, or among particular population subgroups, may increase the pace of recruitment or allow for more targeted recruitment. Our findings indicate that ads featuring altruism or the study team’s empathy were more effective than those that did not, whereas including a call to action or urgency of study enrollment were less effective. These findings are contrary to previous research demonstrating that loss-framed messages are more effective than gain-framed messages in online study recruitment.40 These findings represent a starting point for more systematic research on the appeal of ad features to various populations.
Limitations
First, given the small sample size, these findings are preliminary and may not generalize to different populations and settings, particularly those outside the US. Second, results in Table 4 have not been adjusted for multiple comparisons. The features of the ads could not be independently analyzed, and the appeal of each ad likely depends on uncontrolled or even unrecognized features; we therefore note that significant associations may not be causal. The reader is therefore cautioned to regard these findings as exploratory. Third, we were unable to determine how many individuals saw the social media ads nor how many letters were effectively delivered and read, and thus were unable to quantify exposure to these recruitment strategies. Fourth, the lack of cost data associated with individual social media link clicks, and the inability to match link clicks with enrolled participants, limit our ability to draw meaningful inferences from this data. Finally, our social media recruitment was limited to paid advertisements on Facebook, the most popular social media site at the time of the study. Changes in social media use could conceivably produce different results today.
Conclusion
Recruitment options are rapidly evolving, and researchers have the ability to diversify and tailor their strategies to reach and engage participants. These decisions should be intentional and informed by the study objectives and the relative trade-offs of available strategies. The present study suggests that leveraging a variety of recruitment strategies produces a more diverse sample than would recruitment through one strategy alone, with similar adherence and retention. These findings are encouraging, indicating that recruitment strategies that reach beyond clinical settings can engage hard-to-reach segments of the population.41 Our exploratory findings regarding the performance of social media advertisements suggest that message framing may also be important in efficiently reaching a targeted audience, although more rigorous research is needed in this area. Finally, given the small sample size of the present study, replication with larger samples of participants, in the US and worldwide, are needed as results may not be applicable in other healthcare settings.
Acknowledgements
This work was supported by the National Institute of Diabetes and Digestive and Kidney Diseases at the National Institutes of Health (1K01 DK099202-01A1). The content is solely the responsibility of the authors and does not represent the official view of the National Institutes of Health. This study is registered with clinicaltrials.gov, clinical trial registration number NCT02214641. The datasets analyzed for the current manuscript are available from the corresponding author on reasonable request.
Footnotes
The pediatric clinic has no fixed age at which patients must transition to adult care; transitions occur based on patient readiness and/or requirements imposed by payors. The majority of patients transition in their twenties.
References
- 1.Bernstein CM, Stockwell MS, Gallagher MP, et al. Mental health issues in adolescents and young adults with type 1 diabetes: prevalence and impact on glycemic control. Clin Pediatr (Phila) 2013; 52(1): 10–15. [DOI] [PubMed] [Google Scholar]
- 2.Johnson B, Elliott J, Scott A, et al. Medical and psychological outcomes for young adults with Type 1 diabetes: no improvement despite recent advances in diabetes care. Diabet Med 2014; 31(2): 227–231. [DOI] [PubMed] [Google Scholar]
- 3.Dabelea D, Stafford JM, Mayer-Davis EJ, et al. Association of type 1 diabetes vs type 2 diabetes diagnosed during childhood and adolescence with complications during teenage years and young adulthood. JAMA 2017; 317(8): 825–835. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.James S, Gallagher R, Dunbabin J, et al. Prevalence of vascular complications and factors predictive of their development in young adults with type 1 diabetes: systematic literature review. BMC Res Notes 2014; 7: 593. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Kipps S, Bahu T, Ong K, et al. Current methods of transfer of young people with Type 1 diabetes to adult services. Diabet Med 2002; 19(8): 649–654. [DOI] [PubMed] [Google Scholar]
- 6.Bryden KS, Dunger DB, Mayou RA, et al. Poor prognosis of young adults with type 1 diabetes: a longitudinal study. Diabetes Care 2003; 26(4): 1052–1057. [DOI] [PubMed] [Google Scholar]
- 7.Northam EA, Lin A, Finch S, et al. Psychosocial well-being and functional outcomes in youth with type 1 diabetes 12 years after disease onset. Diabetes Care 2010; 33(7): 1430–1437. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Jewell RR and Gorey KM. Psychosocial interventions for emergent adults with type 1 diabetes: near-empty systematic review and exploratory meta-analysis. Diabetes Spectr 2019: 32(3): 249–256. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.O’Hara MC, Hynes L, O’Donnell M, et al. A systematic review of interventions to improve outcomes for young adults with Type 1 diabetes. Diabet Med 2017; 34(6): 753–769. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Hinshaw SP, Hoagwood K, Jensen PS, et al. AACAP 2001 Research Forum: Challenges and recommendations regarding recruitment and retention of participants in research investigations. J Am Acad Child Adolesc Psychiatry 2004; 43(8): 1037–1045. [DOI] [PubMed] [Google Scholar]
- 11.Lado JJ and Lipman TH. Racial and ethnic disparities in the incidence, treatment, and outcomes of youth with type 1 diabetes. Endocrinol Metab Clin North Am 2016; 45(2): 453–461. [DOI] [PubMed] [Google Scholar]
- 12.Danaher BG and Seeley JR. Methodological issues in research on web-based behavioral interventions. Ann Behav Med 2009; 38(1): 28–39. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Ramo DE and Prochaska JJ. Broad reach and targeted recruitment using Facebook for an online survey of young adult substance use. J Med Internet Res 2012; 14(1): e28. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Topolovec-Vranic J and Natarajan K. The use of social media in recruitment for medical research studies: a scoping review. J Med Internet Res 2016; 18(11): e286. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Chu JL and Snider CE. Use of a social networking web site for recruiting Canadian youth for medical research. J Adolesc Health 2013; 52(6): 792–794. [DOI] [PubMed] [Google Scholar]
- 16.Pew Research Center. Social Media Fact Sheet. Washington, D.C: 2018, February 5. [Google Scholar]
- 17.Whitaker C, Stevelink S and Fear N. The use of Facebook in recruiting participants for health research purposes: a systematic review. J Med Internet Res 2017; 19(8): e290. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Huynh L, Johns B, Liu S-H, et al. Cost-effectiveness of health research study participant recruitment strategies: a systematic review. Clin Trials 2014; 11(5): 576–583. [DOI] [PubMed] [Google Scholar]
- 19.Gallagher KM and Updegraff JA. Health message framing effects on attitudes, intentions, and behavior: a meta-analytic review. Ann Behav Med 2012; 43(1): 101–116. [DOI] [PubMed] [Google Scholar]
- 20.Rothman AJ and Salovey P. Shaping perceptions to motivate healthy behavior: The role of message framing. Psychol Bull 1997; 121(1): 3–19. [DOI] [PubMed] [Google Scholar]
- 21.Betsch C, Bohm R, Airhihenbuwa CO, et al. Improving medical decision making and health promotion through culture-sensitive health communication: an agenda for science and practice. Med Decis Making 2016; 36(7): 811–833. [DOI] [PubMed] [Google Scholar]
- 22.Kreuter MW and McClure SM. The role of culture in health communication. Annu Rev Public Health 2004; 25: 439–455. [DOI] [PubMed] [Google Scholar]
- 23.Pyatak EA, Carandang K, Vigen C, et al. Resilient, empowered, active living with diabetes (REAL Diabetes) study: methodology and baseline characteristics of a randomized controlled trial evaluating an occupation-based diabetes management intervention for young adults. Contemp Clin Trials 2017; 54: 8–17. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Pyatak EA, Carandang K, Vigen CLP, et al. Occupational therapy intervention improves glycemic control and quality of life among young adults with diabetes: the Resilient, Empowered, Active Living with Diabetes (REAL Diabetes) randomized controlled trial. Diabetes Care 2018; 41(4): 696–704. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Johnson EJ, Niles BL and Mori DL. Targeted recruitment of adults with type 2 diabetes for a physical activity intervention. Diabetes Spectr 2015; 28(2): 99–105. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Hinshaw LB, Jackson SA and Chen MY. Direct mailing was a successful recruitment strategy for a lung-cancer screening trial. J Clin Epidemiol 2007; 60(8): 853–857. [DOI] [PubMed] [Google Scholar]
- 27.Robinson JL, Fuerch JH, Winiewicz DD, et al. Cost effectiveness of recruitment methods in an obesity prevention trial for young children. Prev Med 2007; 44(6): 499–503. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Juraschek SP, Plante TB, Charleston J, et al. Use of online recruitment strategies in a randomized trial of cancer survivors. Clin Trials 2018; 15(2): 130–138. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Guthrie KA, Caan B, Diem S, et al. Facebook advertising for recruitment of midlife women with bothersome vaginal symptoms: a pilot study. Clin Trials 2019; 16(5): 476–480. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Thornton L, Batterham PJ, Fassnacht DB, et al. Recruiting for health, medical or psychosocial research using Facebook: systematic review. Internet Interv 2016; 4: 72–81. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Frandsen M, Thow M and Ferguson SG. The effectiveness of social media (Facebook) compared with more traditional advertising methods for recruiting eligible participants to health research studies: a randomized, controlled clinical trial. JMIR Res Protoc 2016; 5(3): e161. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Gorman JR, Roberts SC, Dominick SA, et al. A diversified recruitment approach incorporating social media leads to research participation among young adult-aged female cancer survivors. J Adolesc Young Adult Oncol 2014; 3(2): 59–65. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Moreno MA, Waite A, Pumper M, et al. Recruiting adolescent research participants: in-person compared to social media approaches. Cyberpsychol Behav Soc Netw 2017; 20(1): 64–67. [DOI] [PubMed] [Google Scholar]
- 34.Ramo DE, Hall SM and Prochaska JJ. Reaching young adult smokers through the Internet: Comparison of three recruitment mechanisms. Nicotine Tob Res 2010; 12(7): 768–775. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Shere M, Zhao XY and Koren G. The role of social media in recruiting for clinical trials in pregnancy. PLoS One 2014; 9(3): e92744. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Rait MA, Prochaska JJ and Rubinstein ML. Recruitment of adolescents for a smoking study: use of traditional strategies and social media. Transl Behav Med 2015; 5(3): 254–259. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Wang Y, Hunt K, Nazareth I, et al. Do men consult less than women? An analysis of routinely collected UK general practice data. BMJ Open 2013; 3(8): e003320. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Shalev V, Chodick G, Heymann AD, et al. Gender differences in healthcare utilization and medical indicators among patients with diabetes. Public Health 2005; 119(1): 45–49. [DOI] [PubMed] [Google Scholar]
- 39.Kraemer HU, Rueter G, Schoettker B, et al. Gender differences in healthcare utilization of patients with diabetes. Am J Manag Care 2012; 18(7): 362–369. [PubMed] [Google Scholar]
- 40.Graham AL, Fang Y, Moreno JL, et al. Online advertising to reach and recruit Latino smokers to an internet cessation program: impact and costs. J Med Internet Res 2012; 14(4): e116. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Garvey KC, Wolpert HA, Laffel LM, et al. Health care transition in young adults with type 1 diabetes: barriers to timely establishment of adult diabetes care. Endocr Pract 2013; 19(6): 946–952. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Toobert DJ, Hampson S and Glasgow RE. The summary of diabetes self-care activities measure: results from 7 studies and a revised scale. Diabetes Care 2000; 23(7): 943–950. [DOI] [PubMed] [Google Scholar]
- 43.Gardner B, Abraham C, Lally P, et al. Towards parsimony in habit measurement: testing the convergent and predictive validity of an automaticity subscale of the self-report habit index. Int J Behav Nutr Phys Act 2012; 9: 102. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.McGuire BE, Morrison TG, Hermanns N, et al. Short-form measures of diabetes-related emotional distress: the Problem Areas in Diabetes Scale (PAID)-5 and PAID-1. Diabetologia 2010; 53(1): 66–69. [DOI] [PubMed] [Google Scholar]
- 45.Bradley C, Todd C, Gorton T, et al. The development of an individualized questionnaire measure of perceived impact of diabetes on quality of life: the ADDQoL. Qual Life Res 1999; 8(1–2): 79–91. [DOI] [PubMed] [Google Scholar]
- 46.Garcia AA, Villagomez ET, Brown SA, et al. The Starr County Diabetes Education Study: development of the Spanish-language diabetes knowledge questionnaire. Diabetes Care 2001; 24(1): 16–21. [DOI] [PubMed] [Google Scholar]
- 47.Brown M, Dijkers MPJM, Gordor WA, et al. Participation objective, participation subjective - A measure of participation combining outsider and insider perspectives. J Head Trauma Rehabil 2004; 19(6): 459–481. [DOI] [PubMed] [Google Scholar]