Use of electronic recruitment methods in a clinical trial of adults with gout

Abstract

Background/Aims

Electronic-based recruitment methods are increasingly utilized in clinical trials to recruit and enroll research participants. The cost-effectiveness of electronic-based methods and impact on sample generalizability is unknown. We compared recruitment yields, cost-effectiveness, and demographic characteristics across several electronic and traditional recruitment methods.

Methods

We analyzed data from The Diet Gout Trial (DIGO) recruitment campaign. DIGO was a randomized, controlled, cross-over trial that examined the effects of a DASH-like diet on uric acid levels in adults with gout. We used four electronic medical record (EMR) and four non-EMR-based recruitment methods to identify and recruit potentially eligible participants. We calculated the response rate, screening visit completion rate, and randomization rate for each method. We also determined cost per response, screening, and randomization for each method. Lastly, we compared the demographic characteristics among individuals that completed screening visit by recruitment method.

Results

Of the 294 adults who responded to the recruitment campaign, 51% were identified from EMR-based methods. Patient portal messaging, an EMR-based method, resulted in the highest response rate (4%), screening visit completion rate (37%), and randomization rate (21%) among these eight methods. EMR-based methods ($60) were more cost-effective per response than non-EMR based methods ($107). Electronic-based methods, including patient portal messaging and Facebook, had the highest proportion of White individuals screened (52% and 60%). Direct mail to non-active patient portal increased enrollment of traditionally underrepresented groups, including both women and African Americans.

Conclusion

An EMR-based recruitment strategy that utilized the EMR for participant identification and postal mailing for participant outreach was cost-effective and increased participation of under-represented groups. This hybrid strategy represents a promising approach to improve the timely execution and broad generalizability of future clinical trials.

Background and Significance

Trial recruitment is one of the principal determinants of trial success. Ineffective recruitment is directly responsible for budget deficits, early determination of study futility, and inadequate sample size.^1–3 Furthermore, ineffective recruitment strategies can lead to non-representative trial populations,⁴ contributing to under-representation of women and minority racial groups in research and inability to achieve generalizable and informative trial results.^5–7

With the advent of electronic medical records (EMR) and social media, investigators have multiple new available tools for meeting enrollment targets, such as EMR patient portal messaging^8–10 and targeted Facebook advertisements.^11,12 These novel strategies have the potential to reach more targeted populations faster than traditional media, such as periodicals and physical mailed advertisements.^11,13,14 However, previous literature has indicated that the use of primarily digital methods for recruitment may result in homogenous trial samples and under-representation of minority racial groups.^8,11,15 Further exploration is warranted to understand if these electronic tools are cost effective or can be used to recruit a diverse, generalizable study population relative to traditional recruitment methods.

Thus, we analyzed data from the recruitment campaign of The Diet Gout Trial (DIGO), which incorporated EMR and non-EMR (direct mail, periodicals, social media) approaches to compare recruitment yields amongst several EMR-based and non-EMR-based recruitment methods, determine the relative cost-effectiveness of EMR-based versus non-EMR-based recruitment methods, and assess the demographic characteristics of screened populations across each recruitment method. We hypothesized that EMR-based methods would achieve higher recruitment yields at lower cost compared to non-EMR based methods. We also hypothesized that electronic based methods (i.e. patient portal messaging, Facebook) would differ in demographic composition and result in a less diverse sample than more traditional methods.

Methods

Study design

DIGO was a randomized, controlled, cross-over trial that examined the effect of a Dietary Approaches to Stop Hypertension (DASH)-like diet on serum uric acid levels in adults with gout. Details of the trial’s characteristics can be found in Table 1. DIGO involved two four-week phases: 1.) A dietitian-directed diet phase, and 2.) A self-directed usual diet phase. All participants participated in both phases of the trial. In addition to the intervention, participants were asked to complete two clinic-based follow-up visits. Participants were incentivized with $105 of foods per week in the dietitian-directed phase, $25 for each phase of trial completion (randomization, intervention, and follow-up visits), and personal lab results and health information. All trial visits were carried out at the Johns Hopkins ProHealth community-based, research clinic in Woodlawn (West Baltimore), Maryland.

Table 1.

DIGO Trial Characteristics

Trial duration	10–14 weeks
Inclusion Criteria	Adults ages 18 year of age and older
	Self-reported history of gout
	Serum uric acid level ≥ 7 mg/dL
Exclusion Criteria	Active treatment or plans to initiate urate lowering therapy
	Excessive alcohol use (>14 drinks/week)
	Chronic kidney disease (Stage 4 or 5)
	Recent or planned changes to medication use (steroids, lipid-lowering medications, antihypertensive agents) Major gastrointestinal conditions affecting food absorption*
Number of data collection visits	3 pre-randomization
	1 randomization
	2 post-randomization
Grocery activity by assignment
Self-directed	None
DASH	4 weekly grocery ordering visits, 4 trips to the study center to pick up groceries
Incentives
Food	$105 a week for 4 weeks
Monetary	$25 for completing randomization, intervention, and control follow-up visits (total $75)
Other	Lab results, health information
Recruitment goal	40
Enrollee’s demographic information	Mean age of 59.0 years
	19% Women
	49% African American/Black

^*Major conditions included: history of gastric bypass surgery, active inflammatory bowel disease, malabsorption, or major gastrointestinal resection)

Target population

DIGO had a recruitment goal of 40 participants. Trial recruitment targeted individuals who had a self-reported history of gout and were 18 years of age and older. Given the in-person visits and the requirement that participants pick up their groceries at the study center, we targeted adults living near the research clinic. The trial excluded individuals currently using or planning to initiate urate lowering therapy. Based on the demographic characteristics of individuals residing in central Maryland with gout, we anticipated that ~80% of participants would be men and ~40% would be African American. Additional information detailing the trial’s eligibility criteria are found in Table 1.

Recruitment procedures

Recruitment occurred from June 2018 to April 2019. Selection of recruitment strategies were based on our prior recruitment experience with community mailings, social media, and the electronic medical record.^11,16 Given our observation that EMR electronic portal users were predominantly white, we included a novel, hybrid approach in this study that involved mailing our study brochure to adults identified to have gout in the EMR. Methods were adjusted throughout the study based on real-time yields to optimize recruitment goals (see Supplement Table ST1 for additional details). Because of the higher prevalence of gout in males, some recruitment strategies solely targeted men.^17–19 All recruitment strategies were approved by the Johns Hopkins Institutional Review Board (IRB) which approved a HIPAA (Health Insurance Portability and Accountability Act) waiver to mail research brochures to patient addresses.

For the purposes of this paper, recruitment methods are split into two categories: EMR-based recruitment methods and non-EMR based recruitment methods. Methods that utilized the EMR to identify and/or recruit research participants were categorized as an EMR-based recruitment method.

EMR-based recruitment methods.

A series of EMR-based recruitment methods were used to identify and recruit research participants. These methods included: patient portal messaging to active MyChart users; postal mailing to active MyChart users; postal mailing to non-active MyChart users; and repeat postal mailing to the non-active MyChart users. For each of these four methods, potential participants were identified via a computable phenotype based on age, geography (proximity to the research center), and having an ICD9 or ICD10 billing definition with search word “gout” (Supplement Table ST2).²⁰ Lists of patients were organized by patient portal activity. A patient was considered to have an “active” patient portal if he or she opted in for MyChart messages, provided an email address that was not null, provided an email address in a proper format, did not opt out of research communications, logged into his or her portal within the last year, and did not have a “do not contact” flag (designated for electronic communications) in their EMR profile. All other patients were defined as non-active MyChart portal users. To contact non-active MyChart portal users via direct postal mail, the IRB required a HIPAA waiver outlining the justification for use. Of note, patients of Johns Hopkins Hospital with a non-active patient portal do receive correspondence via mail as part of their regular care.

Patient portal messaging was carried out by the Johns Hopkins University Institute for Clinical and Translational Research MyChart Recruitment Service. A working group composed of experts in recruitment methods, clinical researchers, clinicians, and data scientists oversee this Service. Specific approval from this working group is required prior to IRB submission and utilization of the Service. Service team members carry out all Service-related operations, including sending of messages. Details of the service processes are published elsewhere.^8,9 In summary, potential participants that were identified through the computable phenotype described above were sent a patient portal message. This message detailed the purpose of the trial, eligibility criteria, incentives, and study contact information. Individuals were able to access the portal message via portal website or mobile application. For details regarding the patient portal message see Supplement Figure SF1.

A total of 1,397 active MyChart users were contacted through patient portal messaging. A large subset of these patients were also contacted a second time through postal mailing to active MyChart users (n=1,188). Non-active MyChart users were contacted twice through postal mailing, with six months between mailing dates. The first set of mailings was sent to 1,895 individuals. After removing individuals that were deceased (n=3), the repeat mailing to non-Active MyChart users was sent to 1,892 individuals. All postal mailings included a pre-paid return postage for individuals to express interest in the trial. Brochures were placed in envelopes for the subsequent mailing after a patient expressed concern about privacy.

Non-EMR based recruitment methods.

In addition to EMR-based methods, the trial’s recruitment campaign implemented non-EMR based methods, including Facebook advertisements, periodical advertisements, postal mailing to prior participants of studies conducted at our clinical research center (not specifically gout patients), postal mailing to a vendor-acquired community list, word of mouth, and staff and doctor referrals. Two targeting strategies of a Facebook advertisement were utilized with simple variations based on logo and text (details of these queries are in Supplement Table ST3). Both versions of the advertisement targeted men that lived within a 25-mile radius to the trial site. Individuals that clicked on the advertisement were directly linked to the trial’s website. The website apprised the viewer about the study, furnished IRB and PI information, and included a secure, embedded Qualtrics form for enrollment.

Periodical advertisements (Supplement Figure SF2) were displayed in two local Baltimore area newspapers, The Northwest Voice and The Beacon, for two months. We also sent a letter to 676 prior ProHealth participants who were willing to be contacted regarding future studies.

Finally, we mailed our study brochure to 10,341 adults living in the zip codes within a 25-mile radius of the research center. These zip codes included both urban and suburban communities. Names were purchased through a commercial vendor that did not have gout status. One of these mailings targeted men only (N= 4,965). All brochures contained contact information for our clinic, website, and pre-paid postage return cards.

Participant response

Individuals interested in the trial were able to reply to the study invitation by phone, email, website, or postal mail. Individuals recruited via Facebook were directly linked to the website, but were able to call or email if preferred. Source of inquiry and date of inquiry were maintained by recruitment staff in a secure database. Following inquiry, staff members contacted interested individuals via phone to complete a pre-screening visit, which included verbal consent. Afterward, eligible individuals were invited to a screening visit the ProHealth Clinic, where written informed consent was obtained, along with demographic information, including age, sex and race. If individuals remained interested and eligible, they were invited to a second screening visit and later enrolled in the study.

Cost analysis

Costs for each recruitment method were tracked and kept by trial staff (invoices) and verified via account payment documentation. Postal mailing to the active MyChart users, non-active MyChart users, and the vendor-acquired community list were based on brochure printing costs, mail house labeling fees, mail house envelope fees, mail house delivery fees, USPS shipping fees, and returned postcards shipping and handling fees. Cost of postal mailings to previous participants was based on the cost of paper, ink, postage and envelope prices at a local print shop.

Annual fees associated with bulk mailing and business reply mail were applied to estimate the independent cost of using each mailing modality. Note that some of these upfront costs would be reduced by using more than one mailing strategy.

The MyChart patient portal messaging cost was based upon the query development fee and the EMR data integration fee. The cost does not increase based on number of individuals identified or contacted. Consistent with the mailing approach above, the onetime cost of query development was also added to the postal cost of mailing active and non-active MyChart users. It is important to note that the University’s Institute of Clinical and Translation Research covers the administrative costs of the MyChart Recruitment Service. This includes administrative staff explaining the service to researchers, assisting with IRB submission, and sending out messages on the team’s behalf. These could be additional costs to consider at different institutions.

The advertisements through Facebook and the periodicals were based upon each method’s aggregate fees for use.

Key Metrics and Statistics

Three yields were calculated to understand the effectiveness of the campaign’s recruitment methods: the response rate; the screening visit completion rate; and the enrollment rate.¹¹ The response rate was calculated for the EMR-based approaches, in addition to postal mailing to prior participants and the vendor-acquired community mailing list. This figure was derived by dividing the number of individuals that inquired about the trial (numerator) by the number of individuals contacted (denominator). In order to assign an individual to a recruitment source, two complementary approaches were utilized: individuals were matched to their source of recruitment at initial contact and individuals that noted postal mailing as their source of recruitment were matched to their source of origination via contact lists maintained by the study team in an encrypted and firewall protected database. Contact lists were kept for the active MyChart users, non-active MyChart users, and prior participants.

For repeat postal mailing to non-active MyChart users, an assumption was made that individuals who inquired after the second mailing date responded to the repeat, and not the initial, mailing. Individuals that noted their source of recruitment to be postal mailing but did not match a name on the contact lists were assumed to be from the vendor-acquired community mailing list. The screening visit completion rate and enrollment rate were calculated for each recruitment modality. These were derived by dividing the number of individuals that completed the screening visit or enrolled (numerator), respectively, by the number of individuals that inquired from that source (denominator).

Three separate cost-effectiveness yields (cost per response, cost per screening visit completed, and cost per enrollee) were calculated for each recruitment method, including the EMR-based methods, Facebook advertisements, periodical advertisements, and postal mailing to prior participants or the vendor-acquired community mailing list. Yields were derived by dividing the total cost of the recruitment method (numerator) by the number of individuals who completed the respective step (contact, screening, enrollment) of the recruitment funnel (denominator). We also examined the number of additional responses, screening visit completions, or enrollees needed to achieve similar costs across approaches.

Results

Overall recruitment response and comparison of yields

In total, 294 individuals responded to our recruitment campaign, 51.0% of which were recruited from EMR-based methods (Figure 1). The most utilized sources of inquiry were phone (n=126) and website (n=66) (Supplementary Figure F3). Following the pre-screening visit, 77 individuals completed the first screening visit and 43 individuals enrolled, resulting in an average screening visit completion rate of 26% and an average enrollment rate of 15% (Table 2). MyChart patient portal messaging had the highest response rate (4%), screening visit completion rate (37%), and enrollment rate (21%) among the recruitment methods. Postal mailing to non-active MyChart users also resulted in recruitment yields higher than the campaign’s average with a combined (initial and repeat mailing) response rate of 2%, a combined screening visit completion rate of 28%, and a combined enrollment rate of 15%. Periodicals and postal mailing to active MyChart users were the least effective recruitment methods, with only 2 screening visit completions and 1 enrollee each. Of note, several individuals who responded (n=26), completed the screening visit (n=6), and enrolled (n=3), but did not disclose their source of recruitment.

Figure 1.

Recruitment Funnel

Table 2.

Campaign Response by Recruitment Method

Source	Contacted	Responded	Yield (Responded/Contacted), %	Completed SV	Yield (Completed SV/Responded), %	Enrolled	Yield (Enrolled/Responded), %
EMR-Based Approaches
MyChart Patient Portal Message	1397	57	4.1	21	36.8	12	21.1
Postal Mailing: Active MyChart Users	1188	21	1.8	2	9.5	1	4.8
Postal Mailing: Non-active MyChart Users	1895	50	2.6	13	26.0	7	14.0
Postal Mailing: Repeat Non-Active MyChart Users	1892	22	1.2	7	31.8	4	18.2
Non-EMR Based Approaches
Facebook		26		5	19.2	4	15.4
Periodicals		7		2	28.5	1	14.3
Postal Mailing: Prior Participants	676	3	0.4	1	33.3	0	0.0
Postal Mailing: Vendor-acquired community mailing lists	10341	46	0.4	11	23.9	7	15.2
Word of Mouth		16		7	43.8	4	25.0
Prior Participant		1		0	0.0	0	0.0
Website		2		1	50.0	0	0.0
Staff Referral		9		2	22.2	0	0.0
Doctor		1		0	0.0	0	0.0
Other Source		7		0	0.0	0	0.0
Unknown		26		5	19.2	3	11.5
Total		294	1.8	77	26.2	43	14.6

Note: SV: Screening Visit, EMR: Electronic Medical Record

Demographic differences by recruitment source and inquiry preference

Demographic characteristics varied by recruitment strategy (Table 3). The highest number of Black/African American individuals were recruited through MyChart patient portal messaging (n=6), postal mailing to non-active MyChart users (n=12), and postal mailing to the vendor-acquired community mailing list (n=6). Proportional to the number of screening visit completions per the respective method, postal mailing to non-active MyChart users (60%) and the vendor-acquired community mailing list (54%) resulted in the highest yield of Black/African Americans. Methods that yielded predominantly White individuals were MyChart patient portal messaging (52%) and Facebook (60%). The only method that had representation among all four racial categories (White, Black/African American, American Indian, Asian/Pacific Islander) was postal mailing to non-active MyChart users.

Table 3.

Demographic Characteristics by Recruitment Method at the First Screening Visit

	EMR-Based Methods				Non-EMR Based Methods
	MyChart Patient Portal Messaging	Postal Mailing: Active MyChart Users	Postal Mailing: Non-Active MyChart Users	Postal Mailing: Community Mailing*	Facebook**	Other***	Word of Mouth	Unknown
Completed SV1 (N)	21	2	20	11	5	5	7	5
Mean age (years)	57	50	62	63	60	62	54	68
Female, %	14	0	35	18	20	80	29	40
Race
White, %	52	50	25	18	60	0	14	20
African American/Black, %	28	0	60	54	20	100	42	80
Asian/Pacific Islander, %	10	0	5	9	20	0	14	0
American Indian, %	0	0	5	0	0	0	0	0
Other/Unknown, %	10	50	5	18	0	0	29	0

Note: SV: Screening Visit; EMR: Electronic Medical Record

Six individuals had missing gender information at the first screening visit. There was one participant recruited through the website that did not provide demographic information

^*48% of the Community Mailing List was sent to men only to minimize costs (given higher prevalence of gout among men);

^**Facebook ads were targeted to men to minimize costs (given higher prevalence of gout among men);

^***Participants recruited from periodicals, postal mailing to prior participants and staff referrals were combined into “other” category in order to protect patient identity

The highest number of females were recruited from postal mailing to non-active MyChart users (n=7) and MyChart patient portal messaging (n=3), however, some methods, including Facebook and over half of the vendor-acquired community mailing lists, did not target women. The demographic characteristics of individuals from the EMR queries are reported in Supplementary Table ST4 by patient portal activity group.

The source by which individuals responded to the recruitment campaign varied by race and sex (Table 4). Contact via website was the most utilized source of inquiry for Whites (42%), Asian/Pacific Islanders and American Indians (85%), and males (41%) among adults, who completed the screening visit. Phone was the most utilized source of inquiry for Black/African Americans (49%) and females (57%) among those completing the screening visit.

Table 4.

Response Type by Race and Sex for Individuals who Completed the First Screening Visit

	Female	Male	Unknown Gender	White	African American/Black	Other*	Unknown Race
Total (N)	21	50	6	24	37	7	9
Email, %	5	16	17	21	8	14	11
Mail, %	19	10	0	4	19	0	11
Phone, %	57	30	17	29	49	14	22
Website, %	5	40	67	42	14	86	56
Other/Unknown, %	14	4	0	4	11	0	0

^*Participants that identified as Asian/Pacific Islander or American Indian were combined into “other” category in order to protect patient identity

Comparison of cost

Details of cost for each recruitment method can be found in Table 5. Mailing non-active MyChart users was the most cost-effective recruitment method by response ($33), screening visit completion ($130), and enrollment ($241). The second most cost-effective approach was MyChart patient portal messaging, which cost $52 per response, $140 per screening visit completion and $247 per enrollee. The least cost-effective methods were postal mailing to active MyChart users and periodical advertisements, which cost $1,598 and $1,828 per enrollee, respectively. Other recruitment methods, including repeat postal mailing to non-active MyChart, Facebook, and postal mailing to the vendor-acquired community mailing list cost $425, $588, and $1,235 per enrollee, respectively. The number of additional responses, screening visit completions, or enrollees needed to achieve equivalent cost across approaches may be viewed in Supplement Table ST5. For example, in order for postal mailing to the vendor-acquired community mailing list to be as cost-effective as postal mailing to non-active MyChart users, an additional 211 responses, 56 screening visit completions and 29 enrollments would be necessary. Other recruitment methods would require very minimal changes in the number of enrollees to be as cost-effective as postal mailing to non-active MyChart users, such as MyChart Patient Portal Messaging and postal mailing to previous participants, which would be need only 1 additional enrollee.

Table 5.

Cost of Recruitment by Recruitment Method

Source	Cost ($)	Responded	Cost ($)/Response	Completed SV	Cost ($)/SV Completion	Enrolled	Cost ($)/Enrollee
EMR-Based Methods
MyChart Patient Portal Messaging	2,948.00	57	51.72	21	140.38	12	245.67
Postal Mailing: Active MyChart Users	1,598.20	21	76.10	2	799.10	1	1,598.20
Postal Mailing: Non-Active MyChart	1,685.25	50	33.71	13	129.63	7	240.75
Postal Mailing: Repeat Non-Active	1,698.66	22	77.21	7	242.67	4	424.67
MyChart Users
Non-EMR Based Methods
Postal Mailing: Previous Participants	148.02	3	49.34	1	148.02	0	-
Facebook	2,353.15	26	84.15	5	470.63	4	588.29
Postal Mailing: Community Mailing	8,643.61	46	187.90	11	785.78	7	1,234.80
Periodicals	1,828.00	7	261.14	2	914.00	1	1,828.00

Note: SV: Screening Visit

Per unit cost ($0.22) of Previous Participant Mailing was based on the cost of paper, ink, postage, and envelope prices. Per unit cost of Active MyChart ($0.88), Inactive MyChart ($0.28), Repeat Inactive ($0.88) and Community Mailing ($0.68) was based on the cost of brochure printing, mail house labeling fees, mail house envelope fees, mail house delivery fees, USPS shipping fees, and returned postcards postage and handling fees. Community mailing had an added fee per name of male individuals ($0.60). Annual fees for bulk mailing and business reply through USPS applied to all unique mailing types ($540). One-time query development fee applied to Active and Inactive MyChart mailing types ($588).

Acceptability of Recruitment Campaign

One individual contacted via patient portal messaging requested to opt out of patient portal research communications following the receipt of this trial’s study invitation. An additional individual requested that brochures be sent to his/her home address in an envelope to protect patient privacy. There were three deceased individuals contacted through postal mailing to non-active MyChart users. Unfortunately, one of these individual’s family expressed distress related to this occurrence. There were 2 complaints received by the study team members regarding our recruitment campaign. Several individuals expressed their interest to participate in the study via email, selections of which are documented in Table 6.

Table 6.

Compendium of email inquiries from prospective participants

Direct Quotes
I suffer from gout and am making lifestyle changes in an attempt to reduce the need for long term drug use.
I read of your DIGO study and wondered if you have published any result yet? It’s hard to find dependable, non contradictory information on the correct changes to make whether in diet, exercise, lifestyle etc. I understand you may not be in a position to share your results. If there are any reliable sources of information you could direct me to it would be greatly appreciated. Whatever your response to this message, thank you for undertaking research into this seemingly poorly understood area of human biochemistry.*
I suffer from gout. I have always had high uric acid levels and I’ve changed my diet around 10 times. I have no idea what it is from. I do not drink alcohol and I also have chronic kidney problems. I really need this. I honestly just want to be better and not compensated. Please call or email at your convenience.
Thank you for inviting me into this trial group. I have responded through the form you provided…I am always happy to assist in research in any way.
I received a message in My Chart reference the above subject. I’m interested in more information and possibly participating.
I received the communication about the research program, Diet Gout (DiGo). From the quick assessment, it appears I may be eligible. This certainly appears to be a study that would be very beneficial.
Hello, I received an email regarding this study in my JHU MyChart and I believe I am eligible for the gout study. I would be happy to participate. Please let me know next steps.
I received a message in regard to participating in the Diet Gout research program and would be very interested.
I received information about your Gout trial in the mail recently and I was interested in participating. Please let me know what I need to do to join the trial.

Note:

^*This message was received after the study was completed, but prior to publication of study results

Discussion

In a dietary-intervention trial that enrolled adults with gout, we found that EMR-based methods resulted in higher recruitment yields and lower costs compared to non-EMR based methods. Additionally, we found that postal mailing to non-active MyChart users and to the vendor-acquired community mailing list yielded the most diverse samples of screened and enrolled individuals. Lastly, we documented that these recruitment methods were largely well-accepted by potential participants, as this recruitment campaign only generated 2 complaints and only 1 request for removal from future recruitment messages after sending a total of 1,397 patient portal messages and 15,992 postal mailings.

Use of the EMR for cohort identification and participant recruitment has increased in recent years.^10,21,22 Our study contributes to the existing and growing body of evidence that EMR based-methods are a promising tool for research recruitment and can reduce the cost and time associated with traditional recruitment methods.^8,23,24 Among enrolled participants of this trial, 56% were recruited by an EMR-based method. On average, EMR-based methods cost 44% and 31% less per response and per enrollee compared to non-EMR based approaches. Further, postal mailing to non-active MyChart users identified through the EMR cost on average $994 less per enrollee than postal mailing to the vendor-acquired community mailing list. This finding underscores the advantage of utilizing a targeted EMR-based approach to identify potential participants. In addition to the cost reduction associated with the EMR, utilizing the EMR for participant recruitment can offer a significant time advantage. A study using MyChart patient portal messaging to recruit a sample of older adults into a community-based study of vitamin D supplementation for fall prevention showed that half of the recruitment inquiries/responses were received within 2 days of the patient portal message being sent,¹³ a turnaround time that is likely much briefer than individuals being recruited through traditional methods, such as postal mailing. On the contrary, utilizing a vendor-acquired community mailing list does offer advantages that the EMR-based methods cannot, such as enrolling adults outside the Johns Hopkins Health System patients in clinical research, which may be important for generalizability. Future research should further probe the advantages and disadvantages to EMR-based methods in order to effectively apply these strategies in future recruitment campaigns.

The literature surrounding the ability of these EMR-based methods to recruit a diverse, generalizable sample to a clinical trial is limited.¹⁹ Previous research has shown that individuals who utilize the EMR patient portal are not representative of their health system’s population, in that they are more likely to be white and non-Hispanic.⁸ The downstream effects of the underrepresentation of minority groups has been noted in two studies that used patient portal messaging, where 92% and 91% of the enrolled participants were white.^9,13 In contrast, our study demonstrated that the EMR-based methods recruited a diverse sample of participants. However, this is likely because we deployed a variety of EMR-based methods. For example, the number of African Americans screened through EMR-based methods would decrease by two-thirds if direct mailing to non-active MyChart users had not been utilized. These findings emphasize the need to further understand the contributors of the disparities observed in EMR patient portal usage. as well as the need to simultaneously implement several EMR-based approaches to ensure a diverse and generalizable trial sample.

Our study evaluated the use of two electronic recruitment methods, MyChart patient portal messaging and Facebook advertisements. Overall, we found that these two forms of electronic recruitment methods (i.e. EMR and Facebook) were more cost-effective than traditional methods (i.e. periodical advertisements and general postal mailings to community). However, the use of technology may be less cost-effective when recruiting under-represented minority groups, such as African Americans. In a previous study enrolling cancer survivors, utilizing Facebook to target and enroll African Americans cost substantially more than traditional methods.¹¹ While we did not examine cost differences by race, the recruitment methods with the lowest proportion of African Americans screened were patient portal messaging to active MyChart users and Facebook. Understanding the cost-effectiveness of electronically based recruitment for under-represented populations remains an important area for future research.

The majority of active MyChart and non-active MyChart users were contacted twice in this recruitment campaign. The second contact, which was done through postal mailing for both groups, resulted in an additional 43 inquiries and 5 enrollments for these groups. It is also worthwhile to note that repeat mailing to non-active MyChart users was the third most cost-effective strategy, following patient portal messaging to active MyChart users and the first postal mailing to non-active MyChart users. These findings highlight the possible advantage repeat communication might offer in a recruitment campaign; however, we cannot confidently attribute the additional inquiries and enrollments from the second communications to repeat exposure. It may be that those individuals did not see the first outreach communication. Future studies should investigate the effectiveness of repeat communication and multi-method recruitment strategies to single individuals.

This study had limitations. First, the DIGO study team did not have access to the names purchased from the community mailing vendor. As a result, we were unable to rule out the possibility of prior participants, active MyChart users, and non-active MyChart users being contacted through more than one direct mailing method. Second, achieving accurate cost estimation was difficult. There were several mutual costs, such as the US postal service annual mailing fee, that were applied to each method. In addition, costs covered by Hopkins or institutional grants, such as the development and maintenance of the website or study staff/personnel, were not added to these estimates. Third, this study did not utilize a number of possible recruitment methods (e.g. radio, television, community outreach strategies), and therefore we do not know how EMR-based methods compare to these other platforms. These strategies, particularly community outreach strategies, such as recruitment in churches and barbershops, may provide a unique opportunity to reach and recruit underrepresented groups in research and should be compared to EMR-based methods in future research. Similarly, this study did not track the details of the inquiries received through word of mouth, or implement strategies to incentivize word of mouth recruitment, which limits our understanding of the effectiveness of this strategy in a community setting. Fifth, the sample size was small. Hence, estimates of ‘cost per person’ are likely imprecise, especially when disaggregated by recruitment methods. Sixth, several individuals selected to not report their sex and/or race, resulting in a less than fully complete demographic profile of the cohort. There is a possibility that our conclusions regarding the recruitment methods’ influence on demographic representativeness would change based on these data elements. Seventh, this study recruitment took place in a single geographic location in the U.S. and for a single disease state. Some of the response patterns observed in our study could be related specifically to gout, in that its prominent feature of pain related to gout flares may motivate patients to participate in clinical research at higher rates than those with asymptomatic disorders (e.g., hypertension). As previous literature has indicated that the effectiveness of EMR-based methods may differ on disease condition,⁸ these comparisons should be replicated in different study populations and clinical profiles. Similarly, these comparisons should also be tested in other geographic locations. Lastly, because gout disproportionately affects men,¹⁹ not all of our recruitment methods equally targeted men and women, which limits our ability to determine the effects of recruitment approaches on the sex-distribution of enrollees.

Notwithstanding the above limitations, the DIGO study also had many strengths. To our knowledge, this is one of the first studies to test the effectiveness of several EMR-based recruitment strategies concurrently with several non-EMR based approaches. We collected detailed information regarding source of recruitment, allowing for yield and cost comparisons between and within EMR-based and non-EMR based methods. Further, the secure collection of demographic data of EMR-based approaches provided the opportunity to examine if EMR-based methods have the ability to recruit a diverse, generalizable study population.

These findings have important implications for clinical research. First, this study successfully recruited participants that were identified via the EMR through patient portal messaging and direct mail. This recruitment occurred without physician involvement and/or physician consent. In a clinical environment, permission is typically obtained from the treating physician before contacting patients to participate in a research study. Although this step is justified as protectionary, it is rooted in a paternalistic paradigm of research where the physician is responsible for deciding what is best for the patient.²⁵ Such practices remove patient autonomy and may impede participant access. Second, this study utilized a total of eight recruitment methods to achieve sample yields and diversity. Our findings emphasize the need for utilizing multiple forms of participant recruitment to address disparities in clinical research. Understanding the utility of technology in the recruitment of under-represented minority groups remains an important topic in future research. Third, our study demonstrates cost-effectiveness from targeted, case-finding approaches. This is particularly relevant for less common conditions, like gout, where the community prevalence is approximately 5%.^26,27

In conclusion, we found that direct mail to patients who did not have an active patient portal increased the number of African Americans who participated in our trial. Adopting this hybrid strategy of participant identification and outreach represents a promising approach to improve the timely execution and broad generalizability of future clinical trials.