A Long-Term Evaluation of Facebook for Recruitment and Retention in the ENDIA Type 1 Diabetes Pregnancy-Birth Cohort Study

Abstract

Background:

The Environmental Determinants of Islet Autoimmunity (ENDIA) study is an Australia-wide pregnancy-birth cohort study following children who have a first-degree relative with type 1 diabetes (ACTRN1261300794707). A dedicated ENDIA Facebook page was established in 2013 with the aim of enhancing recruitment and supporting participant retention through dissemination of study information. To measure the impact of Facebook, we evaluated the sources of referral to the study, cohort demographics, and withdrawal rates. We also investigated whether engagement with Facebook content was associated with specific post themes.

Methods:

Characteristics of Facebook versus conventional recruits were compared using linear, logistic, and multinomial logistic regression models. Logistic regression was used to determine the risk of study withdrawal. Data pertaining to 794 Facebook posts over 7.5 years were included in the analysis.

Results:

Facebook was the third largest source of referral (300/1511; 19.9%). Facebook recruits were more frequently Australian-born (P < .001) enrolling postnatally (P = .01) and withdrew from the study at a significantly lower rate compared with conventional recruits (4.7% vs 12.3%; P < .001) after a median of follow-up of 3.3 years. Facebook content featuring stories and images of participants received the highest engagement even though <20% of the 2337 Facebook followers were enrolled in the study.

Conclusions:

Facebook was a valuable recruitment tool for ENDIA. Compared with conventional recruits, Facebook recruits were three times less likely to withdraw during long-term follow-up and had different sociodemographic characteristics. Facebook content featuring participants was the most engaging. These findings inform social media strategies for future cohort and type 1 diabetes studies.

Trial Registration:

Australia New Zealand Clinical Trials Registry: ACTRN1261300794707.

Introduction

Recruitment of subjects into clinical research is a challenge, with declining participation rates reported globally. ¹ Restrictions on people’s time and social, financial, and geographical pressures limit access to research participation. In Australia, this is compounded by changes to privacy laws that restrict access to national databases by health researchers. ² To overcome this, traditional recruitment methods including advertising in newspapers, radio, and television are being replaced with social media campaigns that take advantage of a growing online presence.^3-5

Facebook dominates the social media market with over 1.79 billion daily users reported in June 2020. More than 18 million of these are in Australia, representing 71% of the Australian population. Australian users engage with social media for an average of almost two hours per day, making it the second most popular media activity behind watching television. ⁶ For clinical research, Facebook holds promise as a tool for targeting potentially eligible study participants because users are increasingly accessing social media for health information. ⁷ This facilitates engagement with specific individuals who may be difficult to connect with, ⁴ have specific and uncommon medical conditions including type 1 diabetes (T1D),^8,9 and/or have a time-limited condition such as pregnancy.^10,11 The Diabetes Online Community, in particular, is recognized as being active and rapidly growing, ¹² with high levels of willingness to engage and share information among peers and advocacy groups. ¹³

Studies have reported that Facebook is an effective tool for recruitment in clinical trials and cohort studies.^{3,5,9,10,14-17} However, none have reported its impact on long-term retention. Participant attrition may compromise the validity and integrity of clinical research due to reduced data collection and survivorship bias, thus understanding factors that enhance participant retention are critical for all trials and cohorts.^18,19

The overall aim of this analysis was to assess the characteristics and retention of participants who cited Facebook as their source of referral to the Environmental Determinants of Islet Autoimmunity (ENDIA) study. The ENDIA study is an Australian-wide pregnancy-birth cohort study of children who have a first-degree relative (FDR) with T1D. ²⁰ The ENDIA study seeks to identify environmental factors in prenatal and early postnatal life that increase the penetrance of T1D risk genes, leading to pancreatic islet autoimmunity and T1D. In this article, the long-term impact of using Facebook as a recruitment and retention tool was evaluated by investigating (1) the reported sources of referral of ENDIA participants, (2) the sociodemographic of Facebook recruits versus “conventional” study recruits, and (3) whether Facebook recruits were less likely to withdraw from the study. We also evaluated the most engaging types of Facebook content, thereby contributing to retention.

Methods

ENDIA Study Recruitment

Eligible participants were recruited to the ENDIA study from across Australia.^20,21 Eligibility criteria were pregnant women with T1D, pregnant women whose unborn baby’s father or sibling had T1D, or babies less than six months of age who had a parent or sibling with T1D. Face-to-face recruitment by research coordinators at antenatal, endocrinology, and high-risk pregnancy outpatient clinics was the primary strategy to reach pregnant women living with T1D. Both women and men with T1D were targeted through personalized mail and email invitations sent on behalf of ENDIA by the Australian National Diabetes Services Scheme ²² and JDRF Australia. Members of the study team also participated in community events held by advocacy groups, for example, guest speaker presentations or diabetes advocacy walks. Study promotional materials, brochures, and posters were displayed at hospitals and health facilities across the country. Occasional mainstream media pieces helped promote interest in the study, especially with the launch and new funding announcements. Social media was introduced with a Facebook page in 2013 and Instagram account in 2017, the latter of which was not widely followed, thus not contributing to recruitment. The intention when establishing the Facebook page was to facilitate a broader national reach, particularly to target fathers and parents of children with T1D having another baby, as well as women who were not attending lead hospital clinics. Community referrals, that is, word-of-mouth, were an unexpected source of recruitment. All sources of referral to ENDIA were categorized as: outpatient clinics, other health care professional referrals, social media (nominally Facebook), previous enrollment in ENDIA, targeted communications from diabetes organizations, brochures/posters, community referrals, mainstream media, and public events/presentations. Participants indicated at the time of consent the source of referral, and multiple sources could be nominated.

Ethical Approvals

The ENDIA study was reviewed and approved under the Australian National Mutual Acceptance Scheme by the Women’s and Children’s Hospital Network (WCHN) Human Research Ethics Committee (HREC/16/WCHN/066). The WCHN HREC reviewed all content proposed for the ENDIA Facebook page as this constituted an “advertisement” in the context of the National Statement on Ethical Conduct in Human Research. ²³ Study conduct in Western Australia was approved by the Women and Newborn Health Service Ethics Committee (RGS0000002639). The ENDIA study is registered on the Australia New Zealand Clinical Trials Registry: ACTRN1261300794707.

Facebook Data

The ENDIA Facebook page was created on February 25, 2013, registered as a “Non-profit Organization” in the “Medical & Health” category. The first follower to the page was recorded on May 1, 2013, with the first post published on June 19, 2013. Using the “Insights” function within Facebook, activity data for content posted from February 2013 to December 2020 were extracted in April 2021. This represented the complete recruitment period plus a year of retention activity. Insight data included number and date of “likes” and “unlikes” to the page, page follower characteristics, number of posts, and post engagement. All posts contained an image or a video. The 794 published posts were categorized according to nine different themes based on their content: recruitment, staff updates, study updates, participant updates, T1D information, discussion topics, competitions, significant days, and “fun.” Descriptions and example posts for each of these categories are provided in Supplemental Materials. The categorization was performed by a single individual following extraction of all data. The categories were determined prior to categorization based on the known types of content developed by the ENDIA team.

Statistical Analysis

To compare the characteristics of participants recruited via Facebook versus conventional recruits, univariable linear regression models were used to compare continuous outcome variables, logistic regression models were used to compare binary outcome variables, and multinomial logistic regression was used when the outcome variable had more than two levels. For withdrawal and inactive status, it was necessary to adjust for the effect of the same mother with more than one child participating. In addition, follow-up time needed to be adjusted for, as it was considered to be a confounding variable. Thus, for these variables, mixed-effects logistic regression models were used, where a random intercept for mother was included in the model and follow-up time was included as a covariate. Data were analyzed using the statistical computing environment R version 4.1.0. ²⁴ Models were fit using base R functions, with the exception of the multinomial regression model that was fit using the nnet package ²⁵ and the generalized linear mixed models that were fit using the lme4 package. ²⁶ Summary tables were prepared using the table1 package. ²⁷ A significance level of α = 0.05 was used for all analyses.

Table 1.

Characteristics of 1624 Identifiable ENDIA Facebook Page Followers From a Total of 2465 Followers at the End of 2020.

Characteristic	No. (%)
Sex
Female	1290 (79.4)
Male	332 (20.4)
Missing	2 (0.1)
ENDIA participation
Not parent of ENDIA participant	1323 (81.5)
Parent of ENDIA participant	301 (18.5)
Staff
Not ENDIA staff	1575 (97.0)
ENDIA Staff	48 (3.0)
Missing	1 (0.1)
T1D link
T1D link	955 (58.8)
No T1D link	612 (37.7)
Missing	57 (3.5)
Country
Australian	1332 (82.0)
International	288 (17.7)
Missing	4 (0.2)

Abbreviations: ENDIA, Environmental Determinants of Islet Autoimmunity; T1D, type 1 diabetes.

Results

Mode of Recruitment to the ENDIA Study

The source of recruitment to the ENDIA study as reported by participants at the time of consent is summarized in Figure 1. Facebook provided the third highest reported source (300/1511; 19.9%), behind outpatient clinics (500/1511; 33.1%) and health professional referrals (347/1511; 23.0%). Repeat participants with an older child enrolled in the study accounted for 228 (15.1%) of 1511 recruits. This was followed by targeted campaigns through diabetes organizations (188/1511; 12.4%), advertising using study brochures and posters (143/1511; 9.5%), community referrals (77/1511; 5.1%), mainstream media (43/1511; 2.8%), and public events or presentations (27/1511; 1.8%). Importantly, 342 (22.6%) of 1511 participants reported hearing about ENDIA through more than one approach, for example, both through Facebook and being approached by a research nurse in clinic.

Figure 1.

Participant self-reported source of referral to the ENDIA study. More than one source could be nominated. Abbreviations: ENDIA, Environmental Determinants of Islet Autoimmunity; HCP, health care professional.

Demographics of Facebook Page Followers

The ENDIA Facebook Page recorded 2334 followers from its creation in February 2013 to the end of recruitment in December 2019. Around 10% of followers had unfollowed (“unliked”) the page over this time. An additional 195 individuals followed the page during 2020 (while 61 unfollowed), resulting in 2465 followers by December 2020.

Facebook insights data indicated that of these 2465 individuals, most lived in Australia (82.5%), were women (83%), and aged between 35 and 44 years (31%). Privacy settings of 1624/2465 (65.9%) of 2465 followers allowed page moderators to access some identifying characteristics (Table 1). Most Facebook followers were not parents of study participants (81.5%). There were 301 followers that could be identified as parents of 339 children enrolled in ENDIA. These 301 followers did not completely overlap with the 300 participants citing social media as the mode of referral to the study (Table 2). This could be explained by privacy settings preventing identification of Facebook followers, thus subsequent linking to study participants and/or the use of pseudonyms making identification unreliable. Almost 60% of identifiable followers demonstrated links to T1D on their Facebook profiles. This was established in various ways including following diabetes-related Facebook pages, sharing T1D advocacy posts, providing public status updates that reference T1D, and/or posting of photos of themselves, children, or family members wearing diabetes technology. It is acknowledged that this may be an underrepresentation as the social media experience of T1D is complex and some individuals may not explicitly reveal their connection with T1D in their publicly accessible profile. Such Facebook users may still be gaining social support for their T1D in less visible ways.

Table 2.

Characteristics of Facebook Versus Convention Recruits as Recorded on Study Entry.

Characteristic	Conventional recruits (N = 1211)	Facebook recruits a (N = 300)	P value
Maternal age at recruitment
Mean (SD)	32.2 (4.80)	32.0 (4.33)	.63 b
Type of recruit
Prenatal recruit	1003 (82.8%)	228 (76.0%)	.01 c
Postnatal recruit	208 (17.2%)	72 (24.0%)
Maternal country of birth d
Born in Australia	746 (75.4%)	213 (87.7%)	<.001 c
Born overseas	200 (20.2%)	26 (10.7%)
Missing	44 (4.4%)	4 (1.6%)
T1D relationship
Mother	725 (59.9%)	193 (64.3%)	.22 e
Father	322 (26.6%)	73 (24.3%)
Sibling	124 (10.2%)	25 (8.3%)
Multiple FDR with T1D	36 (3.0%)	6 (2%)
Other genetic FDR f	3 (0.2%)	3 (1%)
Maternal education level
Bachelor degree or above	653 (53.9%)	179 (59.7%)	.07 c
Less than bachelor degree	558 (46.1%)	121 (40.3%)
Maternal employment status
Full-time	474 (39.1%)	127 (42.3%)	.15 c
Part-time or less	737 (60.9%)	173 (57.7%)
Withdrawn from study by July 2021 g
Not withdrawn	1062 (87.7%)	286 (95.3%)	<.001 c
Withdrawn	149 (12.3%)	14 (4.7%)
Inactive participant at July 2021 g
Active	678 (56.0%)	191 (63.7%)	.25 c
Inactive	387 (32.0%)	95 (31.7%)
No longer in study	146(12.1%)	14(4.7%)

Abbreviations: ENDIA, Environmental Determinants of Islet Autoimmunity; FDR, first-degree relative; T1D, type 1 diabetes.

^aRepresents individuals who nominated Facebook as the source of referral on study entry. These participants do not overlap completely with the 301 Facebook page followers who were identified as parents of ENDIA participants (Table 1).

Models used for analysis were: ^blinear regression and ^clogistic regression.

^dCombined values add to 1233 as unique mothers rather than mother-infant pairs were appropriate for this comparison.

^eMultinomial logistic regression.

^fRepresents individuals whose first-degree genetic relationship was not a traditional mother, father, or sibling.

^gModels adjusted for the effect of the same mother being in the study multiple times via a random intercept term and also adjusted for follow-up length.

Around 10% of identifiable followers likely followed the ENDIA Facebook page unintentionally. As one example, “Endia” is a Nigerian Hip-Hop musician and 42 followers with no links to diabetes were from Nigeria—the second most popular location of followers.

Characteristics of Facebook Versus Conventional Recruits

The purpose of establishing a Facebook page was to facilitate broader national reach—particularly expectant fathers living with T1D and expecting families who had an older child diagnosed with T1D, as well as women who may not be attending large hospital clinics. When comparing Facebook recruits versus those recruited using other means, there were no significant differences with respect to maternal age at recruitment, education level, or employment status (Table 2). The T1D family member relationship was also not significantly different between conventional and Facebook recruits (likelihood ratio [LR] test— ${\chi }_{\left(4,1511\right)}^2=5.77$ , P = .22). A significantly higher proportion of Facebook recruits were born in Australia (odds ratio [OR] = 2.1, 95% confidence interval [CI] = 1.4-3.1, P < .001). Unexpectedly, postnatal recruitment was significantly more common among Facebook recruits (OR = 1.5, 95% CI = 1.1-2.0, P = .01).

Association Between Recruitment Through Facebook and Retention

Retention of Facebook recruits was significantly better than those recruited through conventional means. As of July 2021, 12.3% of participants recruited by conventional means had withdrawn from the study after median [IQR] follow-up of 3.7 [2.5, 5.1] years. In contrast, 4.7% of Facebook recruits had withdrawn after median [IQR] follow-up of 3.3 [2.4, 4.9] years (OR = 0.31, 95% CI = 0.17-0.56, P < 0.001). No difference was found between the two groups for becoming inactive, defined as having missed two or more of the most recent study visits.

Facebook Activity

Post engagement

Between February 2013 and the end of December 2020, 794 posts were published on the Facebook page. Themes associated with each post are detailed in Table 3 with examples of each theme given in Supplemental Materials. Facebook defines engagement as any post click, like, share, or comment. Posts featuring participants and study updates received the highest median engagement. Although recruitment posts accounted for <10% of all posts, these obtained the highest rate of median shares. The shares are especially important from a recruitment perspective as the information reaches a wider audience than those following the Facebook page. Over the recruitment period, 3413 posts were shared 3413 times, bringing total engagements to 70 591.

Table 3.

Facebook Engagements by Number and Type of Posts During ENDIA Study Recruitment Period Until December 2019.

Post type	Number	% of all posts	Total engagements	Median engagements	Total shares	Median shares
Participant updates	229	28.8	23 542	56.0	559	2
Study updates	159	20.0	20 647	51.0	790	3
Staff updates	98	12.3	4615	25.0	199	1
Significant days	81	10.2	2821	17.0	253	1
Fun	74	9.3	2019	16.0	140	1
Recruitment calls	63	7.9	10 822	33.0	915	5
T1D info	45	5.7	3744	24.5	419	3
Discussion	31	3.9	1617	39.0	102	1
Competitions	14	1.8	764	44.5	36	1
Total	794	100	70 591	38.0	3413	2

Abbreviations: ENDIA, Environmental Determinants of Islet Autoimmunity; T1D, type 1 diabetes.

Boosted posts

While posting on Facebook is essentially free, Facebook offers a paid function to “boost” posts to a targeted audience, including to individuals who do not follow the Page. Page administrators pay a user-defined amount to target audiences with specific characteristics or interests, for example, individuals who like similar pages or relevant topics such as “type 1 diabetes,” “parenting,” or “pregnancy.” Eleven ENDIA Facebook posts were boosted at a cost of AU$100 for each post. These boosted posts consisted of eight recruitment calls, two study updates, and a competition post. The boosted posts had on average five times the reach and nine times the engagement of nonboosted posts (Table 4). However, there was insufficient evidence as to whether the increased reach and engagements on these boosted posts had any impact on recruitment. Overall, ENDIA Facebook posts reached 523 512 unique Facebook users, including 94 710 users for boosted posts, although this Facebook metric is an estimate only.

Table 4.

Comparison of Boosted and Nonboosted Facebook Posts.

Attribute	Not boosted (N = 783)	Boosted (N = 11)
Post likes (or other reactions)
Median [Min, Max]	23 [0, 201]	119 [33, 432]
Post comment
Median [Min, Max]	1 [0, 52]	10 [0, 40]
Shares
Median [Min, Max]	2 [0, 76]	58 [1, 88]
Missing	1 (0.1%)	0 (0%)
Engagements
Median [Min, Max]	37 [0, 1680]	323 [45, 4580]
Missing	13 (1.7%)	0 (0%)
Reach
Median [Min, Max]	881 [12, 169 000]	7100 [715, 33 100]
Missing	444 (56.7%)	0 (0%)

Negative activity on Facebook

It has previously been reported that selection bias from negative online comments can harm recruitment or retention. ²⁸ However, the ENDIA Facebook page only received 24 negative engagements over the 7.5-year reporting period. Examples of comments include a parent expressing distress at their child developing positive islet autoantibodies and another about blood tests on their child. Other negative engagements consisted of the use of the “angry” emoji.

Conclusions

Facebook was the third most cited source of recruitment to the ENDIA study at 19.9%. The novel finding of this analysis was that those quoting Facebook as the source of referral were three times less likely to withdraw than those recruited by other means after more than three years of average follow-up. The successful use of Facebook for clinical trials and cohort study recruitment is well described; however, the long-term retention outcomes for participants recruited via social media versus conventional recruits have not been reported previously. Social media is a relatively new phenomenon in clinical research, and ENDIA could be considered as an “early adopter” as the page was established in 2013 at the time ENDIA recruitment commenced.

We have identified three studies that compared retention between Facebook and traditional recruiting methods within the same cohort study or trial, but all were limited by duration of follow-up and size. The first measured retention at only two weeks post-consent, ⁸ the second included 27 participants at six months, ⁹ and the third included just three participants at 12-month follow-up. ²⁹ While we could not prove that those recruited to ENDIA through Facebook continued to engage with the ENDIA Facebook page due to privacy settings, this is a likely reason for their better retention.

A significant point-of-difference between Facebook and conventional recruits was country of birth, with a significantly higher proportion of Australia-born mothers among Facebook recruits versus conventional recruits (87.7% vs 75.4%). For both groups. the percentage who were born in Australia exceeds the contemporary general population; 64.0% of all women who birthed in Australia in 2019 were Australia-born. ³⁰ These findings add to the work of others,^5,31 showing that a bias toward particular sociodemographic groups may be exacerbated by Facebook recruitment. While self-reported data on ancestry were requested from all ENDIA parents, the majority provided country of birth (Australia or not) only.

Mothers with T1D were targeted for recruitment at specialist Diabetes in Pregnancy clinics, thus were the easiest group to engage. This may account for the significantly higher proportion of pregnancy recruits in the conventional group versus the Facebook group (82.8% vs 76.0%). One of the goals in using Facebook to promote recruitment was to reach more men with T1D who were expecting a baby. This was important given that the risk of T1D among children born to fathers affected by T1D is increased compared with children of affected mothers. ³² Although content targeting fathers was frequently published on the Facebook page and imagery depicting fathers was intentionally included in all different post types (refer to Supplemental Materials), the distribution of the T1D family member among Facebook versus conventional recruits was not significantly different. Indeed, Facebook recruits tended to have higher proportion of mothers with T1D at 64.3% versus 59.9% for conventional recruits (P > .05), which was not anticipated.

Taken together, using Facebook in the manner employed by the ENDIA study as a recruitment tool did not improve representation of women recruited in pregnancy, migrant populations, or fathers with T1D. Due to sociodemographic biases associated with Facebook, our findings support the use of multiple methods of cohort study recruitment as reported by others.^5,8,16

An important aspect of growing a social media community is regular posting of engaging and accurate content. In our thematic analysis from almost 800 Facebook posts representing more than 70 000 engagements, posts featuring images and stories of study participants and study updates achieved the highest median engagements. This indicated that both study participants and the broader T1D community, who were the majority of page followers, valued the community aspect of the study as well as being kept informed of the study’s progress. The use of the paid “boost” feature allowed key posts such as recruitment drives to be viewed by a larger relevant audience. Only 11 of 974 ENDIA Facebook posts were boosted, reaching 94 710 Facebook users at a total cost of AU$1100 (<AU$0.02 per person “reached”). Although not all reached individuals would be eligible to enroll in the study, and there were insufficient data to link boosting with recruitment, the investment was tiny relative to other recruitment strategies such as physical mailouts, costing AU$1.30 per targeted individual. In retrospect, ENDIA could have made greater use of paid Facebook features to support recruitment, and this should be factored into recruitment budgets for future studies. The impact of boosting on retention remains unknown. Future opportunities for the use of the ENDIA Facebook page could include the re-engagement of those lost to follow-up via their Facebook profiles as others have described.^33,34

Study Limitations

The limitations of this study were that although the ENDIA Facebook page recorded 2465 followers to the end of December 2020, we could only report on those 1624 (65.9%) identifiable followers whose privacy settings allowed it. Of those who had a public profile, the use of pseudonyms made identification and therefore linking between Facebook followers and study participants challenging. It is possible that participants who made their Facebook page private differed from those whose pages were more public, in which case the sociodemographic data of page followers may be biased.

There are anecdotal indications that the discussion and promotion of ENDIA on closed Facebook groups may also have contributed to engagement with the ENDIA Facebook page, which may in turn have influenced recruitment and retention of individuals to the ENDIA study. The content posted to closed Facebook groups is inaccessible unless one is a member. Very little information can be obtained about the number or characteristics of members, or the content of posts made within these closed groups; thus, no metrics could be collected associated with closed Facebook groups. There is little in the literature or online comparing the demographics of public versus private Facebook groups, particularly as such groups are often established with the specific intent of maintaining privacy. ⁷ Bar-Ilan et al ³⁵ reported that active participation in a closed Facebook group is associated with a high level of engagement with the same topic offline. This link between online and offline behaviors warrants further investigation.

Summary

In summary, we have acquired unique data on the impact of Facebook on recruitment and retention of an Australian pregnancy-birth cohort study over an extended timeframe spanning recruitment and follow-up. Facebook was a valuable recruitment tool, although it did not increase diversity among the cohort sociodemographics. Critically, participants recruited via Facebook were three times less likely to withdraw than those recruited by other means. Facebook content featuring study participants resulted in higher engagement and highlights the value of the community aspect of the study. Our findings are relevant for researchers who are planning a social media strategy to recruit and retain participants in cohort and type 1 diabetes studies.