Abstract
Electronic health records (EHRs) are often used for recruitment into research studies, as they efficiently facilitate targeted outreach. While studies increasingly are reaching out to potential participants through the EHR patient portal, there is little available information about which approaches are most effective. We surveyed all investigators at one academic medical center who had used the Epic MyChart patient portal for recruitment. We found that messages were typically adapted for a large group, but not tailored further for individual subgroups. The vast majority of studies sent a message only once. Recruitment costs were modest, averaging $431/study. The results show some promise for recruiting through the patient portal but also identified ways in which messages could be optimized.
Keywords: patient portals, electronic health records, patient selection, humans
INTRODUCTION
Human observational studies and randomized clinical trials remain the gold standard for accruing new knowledge in medicine and improving health. There are numerous challenges to conducting these studies, but one of the greatest is recruiting an adequate number of participants. Study protocols require a minimum number of participants to achieve meaningful results, but too often, funded research fails to meet intended enrollment targets.1,2 Failure to recruit an adequate sample limits the scientific validity of the research, may put participants at needless risk (since the scientific benefit may be limited with an inadequate sample size), and wastes research funding.
It is also essential to enroll a diverse and representative sample. When groups are underrepresented in clinical research, it compromises the external validity and limits the extent to which the research findings can be applied to those groups.3 There has been increased attention to ensuring research participation represents groups across the lifespan, as well as diversity by race/ethnicity, sex, and other individual characteristics.4,5
Various recruitment strategies are used to achieve study enrollment targets. These include mailings, social media, newspaper, and radio advertisements and tailoring studies to defined populations to ensure cultural sensitivity and maximum feasibility. With the widespread use of the electronic health record (EHR), it is now also possible to identify and directly contact specific groups of individuals within a health care system.6 EHRs can enable identification of a population of patients based on study inclusion and exclusion criteria. Notifications about individual studies, through secure patient portals like MyChart in Epic, can inform potentially eligible patients about opportunities to participate in research. While studies have increasingly been using direct outreach through EHR patient portals, little is known about the feasibility and efficacy of this approach. We describe the experience of clinical researchers at a large academic medical center who used an EHR patient portal to inform patients about studies as part of their recruitment efforts. Our objectives were to: (1) determine the approximate cost required to reach out through the portal; and (2) collect preliminary data on efficacy of this approach.
METHODS
Participants
Using our institutional research database,7 we identified all clinical studies that used MyChart as a recruitment tool between 2018 and 2020. We sent personalized email requests to the principal investigator (PI) of each study in February, 2021, asking them to complete a questionnaire about the drafting and implementation of the MyChart invitation. Two follow-up messages were sent to non-responders over a 5-week period.
Survey design and implementation
The questionnaire was programmed in the Research Electronic Data Capture (REDCap) data management platform,8 with a link to the survey provided in the email enabling direct data collection into a secure electronic file. The questionnaire included 22 questions and took approximately 10 min to complete. It was intended to be completed by either the PI or project manager.
We asked the respondents to estimate how many hours were spent on recruitment through the patient portal by the PI, a co-investigator, the project manager, and other study staff. Responses were none, <2 h, 2–5 h, >5 but <10 h, and >10 h.
Analyses
Given the descriptive nature of this investigation, most analyses are confined to counts and proportions. To estimate the cost of recruitment through the patient portal, we assigned an average salary rate for the different roles based on NYU’s 35-h work week—PI $150/h; co-investigator $90/h; project manager $50/h; and other study staff $35/h. We used the interval midpoint as our workload estimate, so that we estimated 3.5 h for people reporting 2–5 h and 10 h for people reporting >10 h. We multiplied the hourly cost by the total number of hours to calculate an average cost per study.
RESULTS
We identified 38 PIs who had used MyChart for research recruitment across a total of 45 studies. We received completed surveys for 33 studies (73%, Table 1). Most respondents (76%) used other recruitment methods in addition to MyChart, including community-based recruitment (n = 11), advertisements (n = 10), e-mail (n = 8), social media (n = 7), and other approaches (n = 12). Recruitment messages were most commonly prepared by the research coordinator/research assistant (RC/RA) or the PI. While 76% indicated they customized the message to reflect anticipated patient characteristics, only 3% said they had different messages for different types of patients. Over three-quarters (82%) of studies reported sending the message only once.
Table 1.
N (%) | |
---|---|
Was MyChart the only tool used for participant recruitment? Yes | 8 (24%) |
Who prepared the MyChart message? | |
Principal investigator | 15 (46%) |
Co-investigator | 7 (21%) |
Project manager | 9 (27%) |
Research coordinator/research assistant | 18 (54%) |
Other | 1 (3%) |
Was the message customized based on patient characteristics? Yes | 25 (76%) |
Did you have different messages for different types of patients? Yes | 1 (3%) |
Were institutional resources (eg, CTSA) used to prepare the message and notification system? Yes | 11 (33%) |
How many times did potential participants receive messages? | |
One | 27 (82%) |
Two | 6 (18%) |
More | 0 (0%) |
Who monitored response to the MyChart message? (multiple responses allowed) | |
Principal investigator | 7 (21%) |
Co-investigator | 2 (6%) |
Project manager | 13 (39%) |
Research coordinator/research assistant | 24 (73%) |
Other | 1 (3%) |
Were you able to determine if messages had been opened? Yes | 9 (27%) |
If yes, what proportion of messages were opened? | |
<25% | 4 (44%) |
25–50% | 3 (33%) |
51–75% | 2 (22%) |
>75% | 0 (0%) |
What proportion of people who were sent messages responded in any way? | |
<25% | 19 (58%) |
25–50% | 0 (0%) |
51–75% | 3 (9%) |
>75% | 0 (0%) |
Did not monitor | 11 (33%) |
What proportion of people who were sent messages were recruited using this approach? | |
<25% | 21 (64%) |
25–50% | 4 (12%) |
51–75% | 0 (0%) |
>75% | 4 (12%) |
Did not monitor | 4 (12%) |
There was no missing data in the responses so sample size is 33 for each question, except for contingent questions (eg, “If yes, what proportion of messages were opened?”).
CTSA: Clinical Translational Science Award.
Monitoring responses to messages was much more variable, with 21% of PIs indicating responsibility for monitoring the message versus 73% of RC/RA’s. Only 27% of teams reported being able to determine if the messages had been opened. Of those that could determine if the message was opened, 77% said less than half of patients opened it. The majority of respondents (64%) reported recruiting less than one-quarter of participants who were sent messages.
The average number of hours spent on recruitment through the patient portal was as follows: PI—1.41 h, co-investigator—0.39 h, project manager—1.65 h, other study staff—2.89 h. This resulted in an average cost of $431/study for recruitment through the patient portal. Nearly all studies (31/33) reported there were no other costs involved in recruitment through the patient portal.
DISCUSSION
In this survey of studies using the Epic MyChart patient portal, we found recruitment through the portal was typically one of several methods used for recruitment. Messages were most often prepared by the PI or RC/RA, while the responses were typically monitored by the project manager or RC/RA. Messages were most often customized based on patient characteristics such as race, gender, and age group, but only one study sent different messages to different groups based on those characteristics. This suggests that the messages were general in nature and targeted large patient groups (eg, adults over age 65 with diabetes), without an attempt to adapt the message for specific subgroups. Of the studies that could determine if the message was opened, nearly all reported that less than half of patients opened the message. More than three-quarters of studies reported only sending one message.
While many research studies have used MyChart for recruitment and discussed their experience,9–12 we are only aware of one publication that looked systematically at the process of using patient portals for recruitment across multiple studies. Miller et al reported on the results at Johns Hopkins of using MyChart to identify potential participants and then reaching out through secure messaging to offer enrollment.13 Of the 13 studies they report on, over three-quarters recruited participants with a specific disease or condition and the average response rate to secure messaging was 2.9%. Similar to Miller et al, results from our survey suggest a direct outreach approach to recruitment through the EHR has some value. The procedure is perceived as requiring less effort than other recruitment outreach methods, and indeed average approximate costs ($431/study) were modest.
We believe recruitment results could be improved by applying some well-established principles of survey research.14 One simple strategy to increase response rate would be to send multiple messages, as is typically done for mailed and telephone surveys. Greater yield from the use of patient portals may be achieved through efforts to increase message visibility and ease of responding. Having the message come from the person’s primary care physician or other respected figure may also increase the response rate.
Our study has several limitations. First, this was based on self-report; we did not attempt to confirm any responses through actual recruitment data. Second, this is limited to a single institution, albeit a large institution with several different hospitals and over 1000 active research studies. Third, we only looked at use of MyChart, Epic’s patient portal, since that is the EHR used by NYU. While Epic is nationally the most commonly used EHR with 31% of the EHR market share,15 experiences may be different with other patient portals.
Further studies are needed to determine “best practices” in direct recruitment through patient portals. Studies might highlight how to frame recruitment messages in a more focused manner and how to increase both the rate of people opening these messages and responding. Given the ability to directly target populations of patients using data from the EHR, it will be important to optimize this potentially efficient approach to outreach and recruitment.
It will also be important to determine how to best implement EHR-based recruitment at the institutional level. Recruitment using the patient portal is still a novel strategy. If this approach becomes more common, we can envision patients being inundated with recruitment messages. Institutional guidelines for coordination or consolidation of recruitment may become necessary. In addition, patient acceptance should be assessed regarding inconvenience and concerns about confidentiality.
In conclusion, our findings suggest that recruiting through the patient portal is modestly helpful as an approach to recruitment. Just as the marketing of products has become increasingly personalized and targeted, so will research recruitment efforts. Use of patient portals is at the forefront of this movement. Effort will be required at the study level and institutional level to make the process more efficient and successful.
FUNDING
The effort of Drs Sherman, Chodosh, and Langford on this manuscript was supported in part by the Engagement in Longevity and Medicine grant from the National Institute of Aging (#1 R24 AG063725-01, PIs—Joshua Chodosh, Chau Trinh-Shevrin, and Scott Sherman). The effort of Drs Sherman, Langford, and Trachtman and Ms Hampp on this manuscript was supported in part by the NYU Clinical and Translational Science Institute grant from the National Center for Advancing Translational Sciences (#UL1 TR001445, PIs—Bruce Cronstein and Judith Hochman).
AUTHOR CONTRIBUTIONS
SES: conceptualization, methodology, interpretation of data, writing—original draft, writing—review and editing; ATL: conceptualization, methodology, interpretation of data, writing—review and editing; JC: conceptualization, methodology, interpretation of data, writing—review and editing; CH: interpretation of data, writing—review and editing; and HT: conceptualization, methodology, interpretation of data, writing—original draft, writing—review and editing.
ACKNOWLEDGMENTS
The authors would like to acknowledge Ms Jessica Smilowitz for her help in creating the REDCap database and extracting the data. They would also like to acknowledge Ms Rosie Ferris for her help in extracting the data.
CONFLICT OF INTEREST STATEMENT
Dr HT is a consultant for Travere Therapeutics, Goldfinch Bio, Angion, Natera, and Akebia. He is also Chair of the Data Monitoring Committee for pediatric trials of tolvaptan being conducted by Otsuka Pharmaceutical. The other authors (SES, ATL, JC, and CH) have no conflicts to report.
Contributor Information
Scott E Sherman, Department of Population Health, NYU Langone Health, NYU Grossman School of Medicine, New York, New York, USA; Department of Medicine, NYU Langone Health, NYU Grossman School of Medicine, New York, New York, USA; Department of Psychiatry, NYU Langone Health, NYU Grossman School of Medicine, New York, New York, USA; Department of Medicine, VA New York Harbor Healthcare System, New York, New York, USA; Clinical and Translational Science Institute, NYU Langone Health, NYU Grossman School of Medicine, New York, New York, USA.
Aisha T Langford, Department of Population Health, NYU Langone Health, NYU Grossman School of Medicine, New York, New York, USA; Clinical and Translational Science Institute, NYU Langone Health, NYU Grossman School of Medicine, New York, New York, USA.
Joshua Chodosh, Department of Population Health, NYU Langone Health, NYU Grossman School of Medicine, New York, New York, USA; Department of Medicine, NYU Langone Health, NYU Grossman School of Medicine, New York, New York, USA; Department of Medicine, VA New York Harbor Healthcare System, New York, New York, USA.
Carina Hampp, Clinical and Translational Science Institute, NYU Langone Health, NYU Grossman School of Medicine, New York, New York, USA.
Howard Trachtman, Clinical and Translational Science Institute, NYU Langone Health, NYU Grossman School of Medicine, New York, New York, USA; Department of Pediatrics, University of Michigan, Ann Arbor, Michigan, USA.
Data Availability
The data underlying this article will be shared on reasonable request to the corresponding author.
REFERENCES
- 1. Treweek S, Pitkethly M, Cook J, et al. Strategies to improve recruitment to randomised trials. Cochrane Database Syst Rev 2018; 2 (2): MR000013. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Fogel DB. Factors associated with clinical trials that fail and opportunities for improving the likelihood of success: a review. Contemp Clin Trials Commun 2018; 11: 156–64. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Caplan A, Friesen P.. Health disparities and clinical trial recruitment: is there a duty to tweet? PLoS Biol 2017; 15 (3): e2002040. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Chen MS Jr, Lara PN, Dang JH, Paterniti DA, Kelly K.. Twenty years post-NIH Revitalization Act: enhancing minority participation in clinical trials (EMPaCT): laying the groundwork for improving minority clinical trial accrual: renewing the case for enhancing minority participation in cancer clinical trials. Cancer 2014; 120 Suppl 7 (7): 1091–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Turner BE, Steinberg JR, Weeks BT, Rodriguez F, Cullen MR.. Race/ethnicity reporting and representation in US clinical trials: a cohort study. Lancet Reg Health Am 2022; 11: 100252. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Bougrab N, Li D, Trachtman H, Sherman S, Thornton R, Langford AT.. An electronic health record-based strategy to recruit for a Patient Advisory Council for Research: implications for inclusion. J Clin Transl Sci 2020; 4 (1): 69–72. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Langford A, Sherman S, Thornton R, et al. Profiling clinical research activity at an academic medical center by using institutional databases: content analysis. JMIR Public Health Surveill 2020; 6 (3): e12813. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Harris PA, Taylor R, Minor BL, et al. ; REDCap Consortium. The REDCap consortium: building an international community of software platform partners. J Biomed Inform 2019; 95: 103208. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Pfaff E, Lee A, Bradford R, et al. Recruiting for a pragmatic trial using the electronic health record and patient portal: successes and lessons learned. J Am Med Inform Assoc 2019; 26 (1): 44–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Miller HN, Charleston J, Wu B, et al. Use of electronic recruitment methods in a clinical trial of adults with gout. Clin Trials 2021; 18 (1): 92–103. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Plante TB, Gleason KT, Miller HN, et al. ; STURDY Collaborative Research Group. Recruitment of trial participants through electronic medical record patient portal messaging: a pilot study. Clin Trials 2020; 17 (1): 30–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Tabriz AA, Fleming PJ, Shin Y, et al. Challenges and opportunities using online portals to recruit diverse patients to behavioral trials. J Am Med Inform Assoc 2019; 26 (12): 1637–44. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Miller HN, Gleason KT, Juraschek SP, et al. Electronic medical record-based cohort selection and direct-to-patient, targeted recruitment: early efficacy and lessons learned. J Am Med Inform Assoc 2019; 26 (11): 1209–17. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Fowler FJ Jr. Survey Research Methods. 5th ed.Los Angeles, CA: Sage Publications, Inc.; 2013. [Google Scholar]
- 15. Drees J. EHR market share 2021: 10 things to know about major players Epic, Cerner, Meditech & Allscripts. Becker’s Hospital Review. https://www.beckershospitalreview.com/ehrs/ehr-market-share-2021-10-things-to-know-about-major-players-epic-cerner-meditech-allscripts.html. Accessed December 11, 2021.
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
The data underlying this article will be shared on reasonable request to the corresponding author.