Abstract
Clinical research relies on enrollment of appropriate subjects. Individuals who hear about research from their clinician are more likely to participate. We developed two strategies for research subject enrollment using the EHR and its underlying data: pop-up alerts for clinicians, and patient lists for on-site research assistants. Eleven studies used clinician alerts and most referred an adequate volume of potential subjects (range 17 to 1,162; median 324). Only a small portion of these potential subjects consented to participate (range 3% to 25%; median 11%). Two of the three studies that used EHR derived patient lists and on-site research assistants reached their enrollment goals. All three principal investigators were satisfied with this approach. These results demonstrate that EHR alerts and patient lists can facilitate research subject enrollment in primary care pediatrics. Future work will identify methods to improve the clinician EHR alert when on-site enrollment by research assistants is not feasible.
Background
Clinical research relies on the voluntary participation of appropriate subjects. Difficulty enrolling volunteers may often hinder research that would otherwise succeed. Prior studies have described barriers and strategies to improve research enrollment in a variety of clinical domains1–3. Barriers include clinician discomfort discussing research, patient or clinician lack of interest, patient fear of research, and inconvenience. Patients are more likely to participate if their clinician discusses research opportunities4,5. Enrolling subjects in pediatric research adds additional challenges related to parental or guardian consent. To date, the most common approaches to subject enrollment include mailing lists, advertising in the media, flyers in patient care areas, and direct enrollment on-site using research assistants.
With the increasing use of electronic health records (EHR) there is a natural opportunity to investigate the use of this tool for research subject enrollment. Key characteristics present in many EHRs include richness of data, large number of potential subjects whose data is available, and freshness of the data. The ability to rapidly identify potential subjects before interventions occur that would preclude study participation is also important6. Several past experiences with the EHR as a tool to facilitate research subject enrollment have been favorable7–9. However, one study using EHR based enrollment that required primary care clinicians to obtain formal consent from patients was not successful10. Data are not available regarding EHR based research enrollment in primary care pediatrics.
Concurrently with the arrival of a centralized electronic health record (EHR) in 2003, the Pediatric Research Consortium (PeRC) was created as a practice-based research network at The Children's Hospital of Philadelphia (CHOP). As part of the development of the PeRC network, clinicians were surveyed to help identify common areas of research interest. The PRINS_1 survey was modified by PeRC to solicit areas of clinical research interest11. The results identified multiple areas of interest with the most common being obesity prevention and treatment, ADHD identification and treatment and strategies for caring for patients with mental health issues (unpublished data).
There were two early projects in the PeRC network that used the EHR for research subject enrollment. One effort involved a multi-institution prospective study that required a large number of infant enrollees. PeRC supported enrollment with a pop-up prompt built into the infant well child visit documentation templates in the EHR, resulting in the near effortless enrollment of 410 infants over 12 months. This early success resulted in an explosion of requests from pediatricians and sub-specialists to use the EHR to facilitate subject enrollment.
PeRC leadership recognized that clinicians in the practices would not accept the burden of complex research subject enrollment strategies. Feedback was obtained from PeRC representatives in the primary care network, clinician EHR “super-users”, and from participants at monthly primary care provider meetings to define new centralized approaches for subject enrollment using the EHR that fit well into the office flow.
Specific Aims
Describe two methods for supporting research subject enrollment in primary care pediatrics using the EHR: automated alerts delivered to clinicians, and patient lists delivered to research assistants
Describe experience to date with the research subject enrollment process in the EHR
Methods
Based on feedback collected by PeRC In 2005 and the technical capabilities of the EHR (EpicCare®; Epic Systems Corp, Madison, WI), two methods for research subject enrollment were implemented. The first involved enrollment by clinicians at the point-of-care supported by pop-up prompts in the EHR. The second used on-site research assistants working from lists of potential subjects derived from the EHR.
Choosing the enrollment strategy
Before choosing an enrollment strategy for each study, PeRC met with the research team to review the usability of existing data in the EHR to identify potential subjects. In discussion with the research team and the institutional review board (IRB), PeRC helped determine which enrollment strategy to implement. Factors considered in the decision were volume of subjects required, anticipated accuracy of automated alerts, the resources available to the study team, and interest in the study by practice clinicians. Initially, automated alerts were often preferred for compliance reasons due to cautious interpretation of the HIPAA rules for handling identifiable patient data, which were implemented early in PeRC’s experience with facilitating research subject enrollment12. However, for selected studies the IRB did approve sending lists of potential subjects to on-site research assistants.
Enrollment by clinicians using alerts in the EHR
Enrollment pop-up alerts were programmed to appear in the assessment and plan portion of the clinician documentation (Figure 1). To avoid alert fatigue, the trigger rules were deliberately programmed to maximize the positive predictive value (favor specificity over sensitivity). The standard fields that defined the trigger rules for each study were: start date, end date, age range, gender, race/ethnicity, department, visit type (e.g. office vs. telephone), reason for visit, and diagnoses. Additional custom fields were programmed for specific studies (e.g. body mass index percentile, preterm gestational age, and positive urine culture).
Figure 1.
Pop-up research enrollment alert appears in the assessment and plan area of the documentation.
The content of each alert was standard for all studies. The top half included the purpose of the study, eligibility criteria, and the benefits or compensation for participants. Brevity was essential and approximately 50 words were recommended. The bottom half listed the allowed responses: ignore (the default), OK to contact, declined, and not eligible.
During the documentation process in the exam room, clinicians were expected to briefly review the alert with the patient and their family and inquire whether the family was interested in being contacted by the research team. If so, verbal permission to be contacted was obtained. A single click was required to document the appropriate response.
Lists of patients that were marked as “OK to contact” were sent to the research team each week on Monday. The research team was responsible for contacting the family by phone, explaining the study in detail and obtaining formal consent for participation.
Enrollment by research assistants using patient lists derived from the EHR
When approved by the IRB, research assistants were sent lists of potential subjects with upcoming appointments. The inclusion criteria were chosen to maximize the negative predictive value (favor sensitivity over specificity). This approach provided the investigators with the greatest number of potential research subjects. These lists were delivered each Friday and contained appointment data for potential subjects for the subsequent week. The age, gender, race and ethnicity of the patients were also included. This appointment information helped the study team target specific dates and times to be on-site in order to maximize their recruitment.
Results
During the two-year interval from 3/1/2005 to 3/1/2007 there were research subject enrollment efforts from 14 studies available for review (Table 1). Of these studies, 9 were aligned with PeRC interest areas, 11 utilized pop-up enrollment alerts delivered to clinicians at the point of care, and 3 utilized patient lists delivered to on-site research assistants.
Table 1.
Description of PeRC study characteristics
| Study | Description | PeRC Interest |
|---|---|---|
| BMDCS†○ | Bone mineral density in childhood | None |
| CF†○ | Lung function measures of preschool children with CF | None |
| CHD†○ | Feeding behaviors and energy cost in infants with CHD | None |
| CRANIO*†○ | Neurobehavioral correlates of craniosynostosis | None |
| DRIVERS | Traffic injury prevention | Driving Safety |
| FRESH | Maternal smoking cessation | Screening/Prevention |
| GARDEN GANG○ | Food substitution for child nutrition | Obesity |
| HHSS○ | Preventive health screening for cardiovascular disease | Screening/Prevention |
| LEAD | Philadelphia lead safe home study | Screening/Prevention |
| RDP†○ | Reference data project | None |
| SLEEP/AUTISM†○ | Impact of sleep disruption on children with Autism Spectrum Disorder and caregivers | Mental Health |
| SLEEP/QOL* | Relationship among sleep problems, obesity, and quality of life in school-aged children | Obesity |
| T2DM○ | Glucose tolerance in obese siblings of children with Type 2 Diabetes | Obesity |
| VASP* | Varicella history as a marker for immunity | Screening/Prevention |
On-site research assistants enrolled subjects
Study enrolled healthy controls only
Travel required for subjects to participate
Enrollment Alerts
Qualitative feedback from PeRC clinicians at advisory board meetings and EHR “super-user” workgroups revealed overall satisfaction with the pop-up enrollment alerts. However, many clinicians felt that the prompts were too easily overlooked due to the number of other alerts that appeared. Several recommended moving the prompt to the history of present illness section at the top of the note, which they felt was usually completed by clinicians in the exam room.
Volume of referrals ranged from 17 (BMDCS) to 1,162 (HHSS) with a median of 324. Duration of enrollment efforts ranged from 3 months (DRIVERS) to 22 months (GARDENGANG) with a median of 9 months (Figure 2). Each potential research subject was counted only once for each study. Possible end points for each patient's enrollment opportunity include being referred, declining, being ineligible, or having never been informed of the study by the clinician (ignored).
Figure 2.
Horizontal bars indicate the frequency of each response type for the pop-up enrollment prompt. The number of months of enrollment is also shown.
*Studies marked with an asterisk were only seeking a control group
Limited data was available regarding how many of the referred subjects actually consented to participate in the study. Based on available documentation of enrollment efforts captured by the principal investigators for each project, the percent of referred subjects that were successfully consented by the research team ranged from 3% (GARDENGANG) to 25% (SLEEP/AUTISM) with a median of 11% (Table 2). Principal investigators reported inability to contact the subject was the most common reason that referred patients did not consent to participate. They recommended a prompt for clinicians to enter the best phone number and contact time to reach families be added in the future.
Table 2.
Number of referred subjects that consented
| Study | Potential Subjects | |
|---|---|---|
| Referred | Consented (%) | |
| GARDENGANG | 491 | 15 (3%) |
| RDP | 427 | 52* (12%) |
| T2DM | 315 | 33 (11%) |
| SLEEP/AUTISM | 32 | 8 (25%) |
Only 37 consented subjects actually attended the RDP study visit. These data were not available for the other studies.
Data from each clinician's use of the enrollment prompts was analyzed for the 6 studies with data on ignored alerts. A clinician was considered a potential participant in the referral effort for a study if they were presented with that study's prompt at least once. The level of each clinician's participation was determined by calculating their fractional contribution of referrals out of all referrals made. By examining the referral volume from the most active clinician participants the minimum number of clinicians required to reach 95% of the cumulative referrals was determined. These subgroups of clinicians were considered active participants. The percent of active participants ranged from 19% (HHSS) to 40% (BMDCS) with a median of 30% (Table 3).
Table 3.
The numbers of potential and active clinician participants in each enrollment effort are shown.
| Study | Clinician Participants | |
|---|---|---|
| Potential | Active (%) | |
| HHSS | 78* | 15 (19%) |
| LEAD | 138* | 36 (26%) |
| CF | 9 | 3 (33%) |
| DRIVERS | 37* | 10 (27%) |
| SLEEP/AUTISM | 9 | 3 (33%) |
| BMDCS | 5 | 2 (40%) |
| OVERALL | 276 | 69 (25%) |
Included residents as potential participants
Patient Lists
The three studies that used patient lists to prompt on-site research assistants to enroll potential subjects were well received by clinicians because minimal on-site effort was required. Qualitative feedback from the principal investigators leading these studies also indicated a high degree of satisfaction with this enrollment strategy. Two of the three studies achieved their enrollment goal (Table 4). The one study that did not achieve their goal (CRANIO) faced unique challenges because a significant on-going commitment from families and precisely matched controls were required. The other two studies were either completed immediately on-site, or with a single follow-up phone call.
Table 4.
Subject enrollment using patient lists and on-site research assistants
| Study | Sites | Duration | Enrollment | |
|---|---|---|---|---|
| Goal | Actual | |||
| VASP | 5 | 22 months | 1000 | 1021 |
| SLEEP/QOL | 2 | 4 months | 100 | 100 |
| CRANIO | 1 | 7 months | 10 | 2 |
Discussion
These observational results suggest that both pop-up research enrollment alerts delivered to clinicians and lists of potential subjects delivered to on-site research assistants can facilitate research subject enrollment. The mostly positive qualitative feedback received from clinicians and principal investigators was encouraging.
For studies that used the pop-up alerts a small portion (median 11%) of the referred potential subjects actually consented to participate in the study. Inability to contact the referred subjects was the most common reason that consent was not obtained. Documenting the best phone number and time to call may help address this problem.
For all studies only a small subgroup of the clinicians who had the opportunity to participate in the referral process actually did so. Possible approaches in the future include only prompting clinicians who have expressed an interest in participating, or monitoring usage patterns to support follow-up efforts with clinicians who may not understand the study or how the enrollment prompt works.
All three studies using lists of potential subjects to on-site research assistants were favorably received by both clinicians and the principal investigators. The principal investigator of the one study that did not meet its enrollment goal was still satisfied with the process due to the unique challenges that their study faced. Unfortunately this method may not be appropriate for studies that require patients from a large geographic area, have a limited budget, or involve sensitive subject matters that may not meet IRB criteria for disclosing lists of patients to affiliated on-site research assistants.
Conclusion
The electronic health record can facilitate the research subject enrollment process using either pop-up alerts delivered to clinicians, or lists of potential subjects delivered to on-site research assistants. The use of on-site research assistants to directly enroll subjects generally resulted in successful enrollment of more subjects. Future work will attempt to improve the design of the clinician prompts.
Acknowledgments
The authors thank Dr. Trude Haecker for her leadership as the chief medical officer for the ambulatory care network.
References
- 1.Butterfield PG, Yates SM, Rogers B, Healow JM. Overcoming subject recruitment challenges: strategies for successful collaboration with novice research agencies. Appl Nurs Res. 2003;16(1):46–52. doi: 10.1053/apnr.2003.50004. [DOI] [PubMed] [Google Scholar]
- 2.Cooley ME, Sarna L, Brown JK, Williams RD, Chernecky C, Padilla G, Danao LL. Challenges of recruitment and retention in multisite clinical research. Cancer Nurs. 2003;26(5):376–84. doi: 10.1097/00002820-200310000-00006. [DOI] [PubMed] [Google Scholar]
- 3.Morrison-Beedy D, Aronowitz T, Dyne J, Mkandawire L, Murphy C, Martin J. The nurse clinician as research participant recruiter: experience from a longitudinal intervention study. J NY State Nurses Assoc. 2001;32(2):9–13. [PubMed] [Google Scholar]
- 4.Harris Interactive 2005New survey shows public perception of opportunity to participate in clinical trials has decreased slightly from last yearAvailable at www.harrisinteractive.com
- 5.Melisko ME, Hassin F, Metzroth L, Moore DH, Brown B, Patel K, Rugo HS, Tripathy D. Patient and physician attitudes toward breast cancer clinical trials: developing interventions based on understanding barriers. Clin Breast Cancer. 2005;6(1):45–54. doi: 10.3816/CBC.2005.n.008. [DOI] [PubMed] [Google Scholar]
- 6.NIH, National Center for Research Resources. McLean, VA: MITRE Center for Enterprise Modernization; 2006. Electronic health records overview. [Google Scholar]
- 7.Embi PJ, Jain A, Clark J, Bizjack S, Hornung R, Harris CM. Effect of a clinical trial alert system on physician participation in trial recruitment. Arch Intern Med. 2005;165:2272–0. doi: 10.1001/archinte.165.19.2272. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Kho A, Zafar A, Tierney W. Information Technology in PBRNs: The Indiana University Medical Group Research Network (IUMG ResNet) Experience. J Am Board Fam Med. 2007;20(2):196–203. doi: 10.3122/jabfm.2007.02.060114. [DOI] [PubMed] [Google Scholar]
- 9.Miller JL. The EHR solution to clinical trial recruitment in physician groups. Health Management Technology. 2006;27(12):22–5. [PubMed] [Google Scholar]
- 10.Mosis G, Dieleman JP, Stricker BCh, van der Lei J, Sturkenboom MC. A randomized database study in general practice yielded quality data but patient recruitment in routine consultation was not practical. Journal of Clinical Epidemiology. 2006;59(5):497–502. doi: 10.1016/j.jclinepi.2005.11.007. [DOI] [PubMed] [Google Scholar]
- 11.Agency for Healthcare Research and Quality, Primary Care Network Survey Clinician InterviewAvailable at: extranet.ahrq.gov/pbrnet
- 12.Pace WD, Staton EW, Holcomb S. Practice-based research network studies in the age of HIPAA. Ann Fam Med. 2005;3(Suppl 1):S38–45. doi: 10.1370/afm.301. [DOI] [PMC free article] [PubMed] [Google Scholar]