Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2023 Feb 27.
Published in final edited form as: Ethics Hum Res. 2023 Jan;45(1):29–38. doi: 10.1002/eahr.500154

Factors That Impact Hospital-Specific Enrollment Rates for a Neonatal Clinical Trial: An Analysis of the HEAL Study

Katherine F Guttmann 1, Sijia Li 2, Yvonne W Wu 3, Sandra E Juul 4, Benjamin S Wilfond 5, Elliott Mark Weiss 6, HEAL RECRUITMENT COLLABORATIVE
PMCID: PMC9969810  NIHMSID: NIHMS1875676  PMID: 36691692

Abstract

Inconsistent enrollment among hospitals for neonatal clinical trials may lead to study populations that are not representative of the patient population in the neonatal intensive care unit. The High-Dose Erythropoietin for Asphyxia and Encephalopathy (HEAL) trial was a multisite randomized clinical trial investigating erythropoietin as a neuroprotective treatment for term infants (those born between 37 and 42 complete weeks) with hypoxic ischemic encephalopathy. Substantial variability was noted in enrollment rate by hospital. We developed survey questions across five conceptual domains to understand systems-level issues that might contribute to variation in enrollment rate by hospital. Our study found that hospitals varied in their responses across these five domains. We propose three potential reasons that we found a lack of identifiable hospital-level factors that correlated with enrollment rates: sample-size limitations, methodological concerns, and confounding factors. Future studies with a larger sample size should be considered to evaluate contributors to hospital-level variability. This will lead to more robust recruitment strategies, improved enrollment, and decreases in the waste of research resources.

Keywords: clinical trial, clinical trials enrollment, clinical trial recruitment, neonatal clinical trials, HEAL study, human research ethics

Large randomized clinical trials (RCTs) are needed to advance evidence-based medicine. Within large multisite studies, inconsistent enrollment among hospitals is common. Inconsistent enrollment can lead to study populations that are not representative of patient populations recruited from various hospital units, which can limit a study’s external validity.1 In the context of clinical trials conducted with patients in the neonatal intensive care unit (NICU), factors that impact enrollment (see figure 1) can be categorized as parental,2 patient, hospital, and study dependent. Among these, hospital-level factors (which we define as hospital characteristics, recruitment processes, personnel, and site-specific challenges) are the least understood and studied. It is unknown which hospital-level characteristics are most important for successful enrollment.3 Researchers seldom publish details of recruitment practices, hindering evaluation of these issues.4 Prior work suggests that hospitals that dedicate more resources to research and are systematic in their approach are more likely to successfully recruit participants.5 Other hospital-level factors that may be important include the experience level, communication skills, and commitment of research coordinators and primary investigators6 as well as the relationship between research staff members and clinicians.7 Prior studies also indicate that institutional research culture and site regulatory approval processes may affect the success of enrollment at the site level.8 A better understanding of how site-level factors contribute to enrollment variability is critically needed to reduce bias and increase generalizability. Identifying key hospital-level factors that impact enrollment may allow funders and researchers to support more successful recruitment efforts at all study hospitals.

Figure 1.

Figure 1.

Open in a new tab

Conceptual Framework

The High-Dose Erythropoietin for Asphyxia and Encephalopathy (HEAL) trial is a multi-institutional RCT investigating the safety and efficacy of erythropoietin to improve neurodevelopmental outcomes in newborns receiving therapeutic hypothermia for hypoxic ischemic encephalopathy (HIE), brain injury that occurs at or near the time of birth as a result of oxygen deprivation.9 Recruitment occurred between 2017 and 2019 at 21 hospitals. Substantial variability was noted in enrollment rates by hospital. We therefore used the HEAL trial as a case study of a multicenter RCT to better characterize potential factors responsible for variability in enrollment rates among hospitals.

STUDY METHODS

We developed a survey to understand systems-level issues that might contribute to variation in enrollment rate by hospital. We worked collaboratively with representatives from each hospital, using a previously described iterative process to generate survey questions.10 The survey addressed five conceptual domains: hospital characteristics, recruitment processes, personnel, barriers to enrollment, and response to challenges. Questions were refined after discussion with an external panel of experts in neonatal clinical trials, survey development, and research ethics. Participants responded to most questions using a 5-point Likert scale ranging from “strongly disagree” to “strongly agree.” Surveys were administered over the phone. Each hospital designated the individual who was most knowledgeable about recruitment practices at that hospital for the HEAL trial to answer questions. We also created a set of questions to assess whether there was a correlation between the study personnel’s belief in the efficacy of the study drug and the site enrollment rate. This subset of questions was completed by the hospital principal investigator (PI), primary research coordinator, and the person who approached the most families for HEAL consent at each hospital. Finally, we included a limited number of questions eliciting free-text responses. The complete survey questions are available in appendix A.

Descriptive statistics were used to report findings and characterize site-level variation in enrollment rates. We included 21 of the 25 HEAL trial hospitals, excluding four that enrolled fewer than five patients into the HEAL trial. “Enrollment rate” was then defined as the total number of infants enrolled divided by the total number eligible within each hospital. We used analysis of variance (ANOVA) to examine associations between enrollment rate and site-specific factors. We also conducted a limited thematic analysis of free-text responses. This analysis of enrollment processes was approved by the Seattle Children’s Hospital institutional review board (IRB).

STUDY RESULTS

Hospital characteristics and recruitment process.

Hospitals were spread across the country geographically and were mostly academic medical centers (see table 1). Hospital-level enrollment ranged from 40.5% to 88.6% of eligible infants (see table 2), with 494 out of 772 eligible infants (64.0%) enrolling in the HEAL trial. To assess whether the number of eligible infants at each site correlated with enrollment rates, we performed a simple linear regression. This produced an R2 value of 0.03, indicating a poor “goodness of fit” and no meaningful relationship between the number of eligible patients and the enrollment rate. There was also substantial variability in the proportion of potentially eligible patients who were deemed eligible for study participation (32.7% to 74.4%). The number of parents screened at each hospital ranged from 7 to 74. Though the small number of sites preclude meaningful objective comparison, subjectively, the hospitals affiliated with the study PIs did not outperform other sites in enrollment, ranking 4th and 16th out of 21 in enrollment rates. Participants reported differences in IRB requirements by hospital (see table 3). Nearly half of respondents “strongly agreed” that IRB policies negatively affected their ability to enroll patients (see table 3). Although two thirds (14/21) of hospitals reported that their IRB allowed them to obtain consent for enrollment by phone, only a minority used this option (see table 3). There was no correlation between responses to questions regarding IRB requirements and enrollment rate. IRB policies relating to the consent process differed by site. These challenges were detailed in our open-ended question on the topic. One site stated that the requirement to obtain signatures from the people giving consent on a paper document (“wet signatures”) was a significant challenge. While some sites reported that faxing “remote signatures” was burdensome, describing it as, for example, “cumbersome” or “time consuming,” one site reported that “remote consent was helpful.”

Table 1.

Characteristics of Enrollment Hospitals

Site characteristic   N (%) Enrollment rate (SD) P-value
Geographic location 0.03
  Northeast 4/21 (14%) 0.58 (0.13)
  South 5/21 (33%) 0.73 (0.10)
  Midwest 6/21 (29%) 0.55 (0.13)
  West 6/21 (29%) 0.73 (0.11)
Academic hospital 0.22
  Yes 16/21 (76%) 0.67 (0.13) 0.22
  No 5/21 (24%) 0.58 (0.16)
Free-standing children’s hospital 0.86
  Yes 15/21 (71%) 0.64 (0.14)
  No 6/21 (29%) 0.67 (0.14)
Licensed NICU beds 0.94
  1-50 4/21 (19%) 0.61 (0.17)
  51-75 7/21 (33%) 0.71 (0.10)
  76-100 6/21 (29%) 0.57 (0.15)
  101 + 4/21 (19%) 0.71 (0.12)
NICU level 0.66
  III 4/21 (19%) 0.62 (0.19)
  IV 17/21 (81%) 0.66 (0.13)
Has a “neuro NICU” 0.28
  Yes 13/21 (62%) 0.68 (0.13)
  No 8/21 (38%) 0.61 (0.15)
Inborn/outborn 0.36
  All outborn 11/21 (52%) 0.63 (0.15)
  Mostly outborn 2/21 (10%) 0.66 (0.08)
  About equal 4/21 (19%) 0.75 (0.06)
  Mostly inborn 2/21 (10%) 0.66 (0.08)
  All inborn 2/21 (10%) 0.68 (0.22)
Open in a new tab

Table 2.

Enrollment by Hospital

Potential participants screened Potential participants eligible Eligibility rate1 Participants enrolled Enrollment rate2
66   37 56.1%   15 40.5%
83   50 60.2%   21 42.0%
71   46 64.8%   20 43.5%
50   19 38.0%   10 52.6%
119   60 50.4%   32 53.3%
98   56 57.1%   30 53.6%
40   22 55.0%   13 59.1%
86   49 57.0%   29 59.2%
16   10 62.5% 6 60.0%
116   67 57.8%   43 64.2%
40   21 52.5%   14 66.7%
20   14 70.0%   10 71.4%
13 7 53.8% 5 71.4%
28   19 67.9%   14 73.7%
85   42 49.4%   31 73.8%
106   74 69.8%   56 75.7%
102   51 50.0%   39 76.5%
107   35 32.7%   27 77.1%
72   26 36.1%   21 80.8%
43   32 74.4%   27 84.4%
65   35 53.8%   31 88.6%
Open in a new tab
1

The eligibility rate was defined as the number of eligible infants divided by the number of screened infants.

2

The enrollment rate was defined as the number of enrolled infants divided by the number of eligible infants.

Table 3.

Hospital-Specific IRB Regulations

N (%) Enrollment rate (SD) P-value
IRB approach to consent
  Allowed by phone 0.37
 Yes 14 (67%) 0.67 (0.14)
 No 7 (33%) 0.61 (0.15)
  Allowed remote signatures 0.33
 Yes 18 (86%) 0.64 (0.14)
 No 3 (14%) 0.73 (0.17)
  Required in-person signatures 0.31
 Yes 9 (43%) 0.61 (0.16)
 No 12 (57%) 0.68 (0.13)
How frequently was consent obtained over the phone? 0.78
  Never or rarely 9 (43%) 0.62 (0.13)
  Occasionally 3 (14%) 0.80 (0.09)
  Frequently or very frequently 9 (43%) 0.64 (0.14)
In what languages other than English could consent be obtained? 0.43
  Spanish
 Yes 18 (86%) 0.66 (0.14)
 No 3 (14%) 0.59 (0.11)
  Other 0.68
 Yes 7 (33%) 0.67 (0.19)
 No 14 (67%) 0.64 (0.11)
Who was able to obtain consent (general)? 0.20
  Study PI
 Yes 18 (86%) 0.67 (0.14)
 No 3 (14%) 0.55 (0.15)
  Any study physician 0.88
 Yes 15 (71%) 0.65 (0.14)
 No 6 (29%) 0.66 (0.16)
  Research coordinator 0.91
 Yes 13 (62%) 0.65 (0.14)
 No 8 (38%) 0.65 (0.15)
  Research nurse 0.07
 Yes 6 (29%) 0.74 (0.09)
 No 15 (71%) 0.62 (0.14)
Did the IRB require the clinical team to approach the family before the research team could approach? 0.93
  Yes 5 (24%) 0.66 (0.12)
  No 16 (76%) 0.65 (0.15)
IRB rules/regulations negatively impacted enrollment. 0.83
  Strongly disagree 10 (48%) 0.64 (0.13)
  All other responses 11 (52%) 0.66 (0.15)
Open in a new tab

Personnel.

Participants did not identify substantive enrollment issues related to study personnel. All participants reported “rarely” missing potentially eligible families due to personnel issues and stated that someone was “frequently” available to recruit during a personnel leave. Most participants (82%) reported that there was a person in leadership who championed the HEAL trial. Though the efficacy of erythropoietin for mitigation of neurodevelopmental impairment in patients with HIE was not known at the outset of our study, key personnel were optimistic about study-drug efficacy (see table 4). More than half agreed that it was likely that erythropoietin would prove to be effective, with 30% stating they neither agreed nor disagreed. Despite this enthusiasm, there was no statistical correlation between responses to these questions and enrollment rate.

Table 4.

Perspectives of Key Personnel

N (%) Enrollment rate (SD)
Respondent’s clinical or research role
  NICU attending 19 (47.5%) 0.63 (0.14)
  NICU fellow 0
  Pediatric resident 0
  Advanced practice provider, physician assistant, or nurse practitioner 0
  Neurology attending 1 (2.5%) 0.64
  Neurology fellow 0
  Neurology resident 0
  Research coordinator 17 (42.5%) 0.63 (0.12)
  Other 4 (10.0%) 0.72 (0.09)
Role for HEAL recruitment
  Did most consenting 13 (32.5%) 0.65 (0.14)
  Primary research coordinator 13 (32.5%) 0.64 (0.13)
  HEAL site PI 11 (27.5%) 0.62 (0.13)
  Other 2 (5%) 0.70 (0.09)
“It is likely that EPO [erythropoietin] will ultimately be shown to be effective neuroprotection for term infants with HIE.”
  Strongly disagree 2 (5%) 0.66 (0.10)
  Disagree 0 (0%)
  Neither agree nor disagree 16 (40%) 0.64 (0.14)
  Agree 19 (47.5%) 0.65 (0.13)
  Strongly agree 3 (7.5%) 0.55 (0.04)
“I would encourage a close friend or family member to enroll their infant in the HEAL study.”
  Strongly disagree 2 (5%) 0.66 (0.10)
  Disagree 0 (0%) 0.63 (0.07)
  Neither agree nor disagree 5 (12.5%) 0.64 (0.15)
  Agree 8 (20%) 0.64 (0.14)
  Strongly agree 25 (62.5%)
Experience enrolling in clinical trials prior to HEAL
  No experience 6 (15%) 0.49 (0.10)
  1-2 clinical trials 3 (7.5%) 0.65 (0.06)
  2-5 clinical trials 6 (15%) 0.49 (0.10)
  6 or more clinical trials 25 (62.5%) 0.67 (0.12)
Open in a new tab

Barriers to enrollment and response to challenges.

Survey respondents identified very few challenges in relation to recruiting for the HEAL trial. Challenges that were described in free-text responses included a very limited time window for enrollment (24 hours), which was specific to this study design (10/21 respondents); serious illness of the mom (2/21); parents being generally overwhelmed, being mistrustful of research, and/or having limited medical literacy (5/21); the unexpectedness of the diagnosis with uncertain prognosis (2/21); and language barriers (5/21). To overcome barriers, respondents reported strategies such as creating a script, checklist, or talking points to ensure relevant information was conveyed during the consent process. Respondents also described actions to support recruitment related to both personnel (e.g., increasing numbers of research staff or increasing clinical team awareness of the study) and process (e.g., adding automated texts sent to the primary investigator and research coordinator anytime a cooling blanket was ordered).

Respondents varied in their perception of parental attitudes toward research enrollment. Forty three percent of participants neither agreed nor disagreed with the statement “At our site, potential participants are hesitant to enroll in research,” while 38% disagreed and 19% agreed. However, most respondents (62%) agreed with the statement “At our site, potential participants are willing to enroll in research,” though a third (33%) neither agreed nor disagreed. There was no statistical correlation between challenges endorsed by study participants and hospital enrollment rate.

DISCUSSION

We identified substantial variation in enrollment rates among 21 hospitals in a large multisite neonatal RCT. Prior studies have also demonstrated variation and shown inefficient allocation of resources to low-enrolling hospitals in large multisite RCTs.11 Although our survey was designed to include key factors identified from the literature, such as hospital characteristics, the recruitment process, personnel, and challenges,12 none of these factors were significantly associated with enrollment rates. Three potential reasons for the lack of identifiable factors that correlated with enrollment rate are sample-size limitations, methodological concerns, and confounding factors.

Our prior work demonstrated that there are important noninstitutional factors that may impact enrollment of neonates into clinical trials. For example, we previously reported lower rates of enrollment among Black parents as well as among parents on Medicaid.13 Work by others has similarly demonstrated racial and ethnic disparities in research enrollment.14 We also found that parents who reported trusting medical researchers were more likely to enroll in research and that how parents experienced the recruitment process varied by parental enrollment status.15 Such factors are critically important, and may be even more important than institutional factors. However, institutional factors have been the least well studied. The work presented here is meant to contribute to the start of explorations of this additional category of factors.

Our results revealed that, although many in the scientific community considered the study intervention to be in clinical equipoise with the standard treatment, individual study team members were optimistic about the intervention’s potential for benefit. Most survey respondents reported that they believed that erythropoietin would ultimately be shown to be effective in providing neuroprotection for term and near-term infants with HIE. Further work must evaluate in what ways such optimism might influence recruitment practices. The existence of such beliefs should be considered when training study team members, particularly those responsible for consent discussions.

Further work is needed to understand how site-level variation influences enrollment rates. Such insights may reveal opportunities for targeted intervention to improve enrollment.16 The findings described here can provide useful background information and serve to warn future researchers who aim to study recruitment strategies for multisite RCTs of potential pitfalls. A better understanding of site-specific variation in recruitment factors may enable the development of interventions to increase enrollment,17 which may in turn result in a research population that is more representative of the larger NICU patient population, increase external validity, diminish research waste, and ultimately lead to a more robust evidence base within neonatal clinical care.

Our study has several important limitations that may explain why we were not able to identify significant associations between hospital-level factors and enrollment rate. First, our sample size of 21 hospitals may have been too small for detection of statistically meaningful differences. Much larger samples may be needed to achieve adequate power. Second, the methods that we employed may not have been appropriate to answer our primary research question. Our iterative process in survey development may have failed to ask the right questions. Qualitative methods such as interviews or focus groups may be better suited to identify unknown contributors to site-level variability that we may have missed. Third, by focusing our questions exclusively on issues related to site-level differences, we were unable to control for within-hospital differences and for relevant confounders. For example, within the research team, a single high-performing individual could skew analysis of an otherwise low-performing hospital. Fourth, and perhaps most importantly, our methods did not allow for engagement with parents about their reasons for enrolling or not enrolling their infant in the clinical trial. Future researchers attempting to elucidate site-level factors should consider obtaining more granular data, such as enrollment rates and recruitment practices for each team member.

Though we had hoped that our work might begin to explain why significant variation in enrollment by site exists, perhaps instead it revealed significant complexity. Confounders such as infant factors and parent demographics may interact in complex ways with hospital-level factors to impact enrollment. For example, parents of more severely ill infants may respond to variation in recruitment practices differently than parents of more stable infants. Parents from historically marginalized populations may also respond differently to recruitment practices. To explore these and other confounders, larger studies involving more sites and potentially enriched for key social and demographic factors are needed.

Our work suggests that analysis of factors at the institutional level alone is likely insufficient to answer the study question. Within institutions themselves exists an array of intersecting issues involving the institutions, the culture of recruitment, and the parents and patients themselves, among others. Although we began with hospital factors as listed in figure 1, perhaps an expanded conceptual model that incorporates these larger issues might better capture the complexity that we uncovered. We therefore propose figure 2 as an expanded model for future work. Future researchers should consider obtaining information on these issues concurrently to enable assessment of the contribution of each factor and potential interactions between them.

Figure 2.

Figure 2.

Open in a new tab

Revised Conceptual Framework

CONCLUSION

We found significant variability in enrollment rate by hospital for the HEAL trial. In our survey of 21 enrolling hospitals, we identified differences in enrollment practices, but no significant correlation of these practices with enrollment rate. Future methodological studies with larger sample sizes should be considered to study which factors contribute to site-level variability, which, in turn, may optimize recruitment strategies for multisite neonatal clinical trials. This could lead to more successful recruitment, larger enrollment, and a decrease in wasted research resources.

ACKNOWLEDGMENT

The HEAL Recruitment Collaborative helped guide survey creation and served as champions for this study at each site, facilitating survey completion at each.

This study was supported by the Treuman Katz Incubator Fund from the Treuman Katz Center for Pediatric Bioethics at Seattle Children’s Research Institute (with Weiss as the principal investigator [PI]). Additional support was provided by the Eunice Kennedy Shriver National Institute of Child Health and Human Development (through grant K23HD103872, with Weiss as the PI).

Appendix A.

Study Survey

Definitions

Site = Hospital in which NICU patients were recruited for HEAL

Recruitment = The process of tracking, approaching, enrolling, consenting, and providing follow-up related to study participation; includes both those who enroll and those who do not Enrollment = A portion of recruitment involving pragmatics of a participant joining research Consent = A portion of recruitment involving the regulatory required processes of informed consent for research participation

Scale to use for most questions:

B B
Strongly Disagree Disagree Neither Agree or Disagree Agree Strongly Agree
Open in a new tab

A. HEAL site characteristics

  1. Study site name:

  2. Name of person completing this form:

  3. Is this site:
    1. an academic medical center: a hospital that is organizationally and administratively integrated with a medical school
    2. a community hospital
  4. Is this site:
    1. a free-standing children’s hospital: not housed within a larger hospital that caters to adults
    2. an attached children’s hospital housed within a larger hospital that caters to adults
    3. a NICU located within an adult hospital that is not part of a children’s hospital

  5. How many NICU beds is this site licensed for?

  6. What level NICU?
    1. III
    2. IV
  7. Does this site have a “NeuroNICU” unit?
    1. Yes
    2. No

  8. If yes, please describe.

  9. Please select the proportion that best reflects the patient mix of all NICU admissions at this site.
    1. All outborn
    2. Mostly outborn
    3. About equally inborn and outborn
    4. Mostly inborn
    5. All inborn

B. Personnel

  • 10.
    At our site, we missed potential participants due to study personnel issues such as someone being out on leave or vacation:
    1. Never
    2. Very Rarely
    3. Rarely
    4. Occasionally
    5. Frequently
    6. Very Frequently
  • 11.
    At our site, when a key person performing recruitment was out on leave, on vacation, or away for another reason, there was another person easily available to recruit in their place:
    1. Never
    2. Very Rarely
    3. Rarely
    4. Occasionally
    5. Frequently
    6. Very Frequently
  • 12.
    At our site, there were formal meetings related to recruitment for HEAL led by the site PI:
    1. Never
    2. 1-3 times per year
    3. 4-6 times per year
    4. 7 or more times per year
  • 13.
    At our site, there was a person (whether or not directly involved in the study) in a leadership position (e.g., division chief, department chair, medical director) who championed the HEAL study.
    1. Strongly disagree
    2. Disagree
    3. Neither agree or disagree
    4. Agree
    5. Strongly agree
  • 14.

    Describe this person’s position and what they did for study.

  • 15.
    Who was/were the primary person/s who performed the day-to-day tasks necessary for keeping the HEAL study going? (OK to pick more than 1)
    1. Site PI
    2. Research coordinator
    3. Other (please list)

C. Process

  • 16.
    IRB approach to consent (choose all that apply):
    1. Allowed to consent by phone
    2. Allowed to obtain signatures remotely (e.g., photocopy with follow-up in-person signature was sufficient)
    3. Required in-person signature prior to enrollment
      16A. Please tell me more about how consent requirements may have affected recruitment.
  • 17.
    If chose A above: how frequently was consent obtained for study participation over the phone?
    1. Never
    2. Very Rarely
    3. Rarely
    4. Occasionally
    5. Frequently
    6. Very Frequently
  • 18.
    In what languages where you able to obtain consent? (choose all that apply):
    1. English
    2. Spanish
    3. Other—describe process
  • 19.
    Who was authorized to obtain consent for the HEAL study at this site? (can choose multiple)
    1. Study PI
    2. Any study physician
    3. Any physician
    4. Research Coordinator
    5. Research nurse
    6. Study NNP or PA
    7. Other: (please specify)
  • 20.
    Did the IRB require that a clinical team member approach the family before the research team could approach?
    1. Yes
    2. No
  • 21.

    What date was the protocol for HEAL initially submitted to the IRB at this site?

  • 22.
    At this site, IRB rules and/or restrictions negatively impacted HEAL enrollment.
    1. Strongly disagree
    2. Disagree
    3. Neither agree or disagree
    4. Agree* (*please describe below)
    5. Strongly agree* (*please describe below)
  • 23.

    Please tell me more about how IRB practices and restrictions affected study recruitment.

Tracking of potential participants

  • 24.
    At this site, how did research team become aware of potential HEAL participants? (please select as many as apply)
    1. An automated system upon NICU admission
    2. Manually identified by research team (specify whom of: research coordinator, study physician, other?)
    3. Medical team notified research team (specify whom of: admitting physician, NICU fellow, NICU resident, NICU APP, neurology attending, neurology fellow, neurology resident, neurology APP, clerk/unit coordinator, transport team, other)
    4. Any other relevant processes not described above?
  • 25.
    At this site, we had a system to identify potential participants that worked well.
    1. Strongly disagree
    2. Disagree
    3. Neither agree or disagree
    4. Agree
    5. Strongly agree
  • 26.

    Why do you think this system worked or didn’t work well?

  • 27.
    At this site, what percentage of eligible patients do you think were missed due to failure to identify a patient that would have been eligible?
    1. <1%
    2. 1-5%
    3. 5-10%
    4. 10-25%
    5. >25%

Coverage system for obtaining consent

  • 28.
    How did clinical team members know who was on call for HEAL at this site?
    1. It was always the same person
    2. There was an on-call schedule that was either posted somewhere easy to find, or communicated regularly to the team
    3. There was an on-call schedule that was posted somewhere but it was not easy to find
    4. There was an on-call schedule but it is not posted anywhere
    5. It was not necessary to know, there was just one phone number or pager to call that connected you to the appropriate person
    6. Other, please describe
  • 29.
    For this site, there was a well-functioning back-up system if on call person was unavailable for some reason.
    1. Strongly disagree
    2. Disagree
    3. Neither agree or disagree
    4. Agree
    5. Strongly agree
  • 30.
    At this site, there was always someone available to enroll for HEAL.
    1. Strongly disagree
    2. Disagree
    3. Neither agree or disagree
    4. Agree
    5. Strongly agree
  • 31.
    At this site, we had difficulty with having adequate staff to enroll on nights.
    1. Strongly disagree
    2. Disagree
    3. Neither agree or disagree
    4. Agree
    5. Strongly agree
  • 32.
    At this site, we had difficulty with having adequate staff to enroll on weekends.
    1. Strongly disagree
    2. Disagree
    3. Neither agree or disagree
    4. Agree
    5. Strongly agree
  • 33.
    At this site, we had difficulty with having adequate staff to enroll on holidays.
    1. Strongly disagree
    2. Disagree
    3. Neither agree or disagree
    4. Agree
    5. Strongly agree

First contact with family

  • 34.
    At this site, a member of the research team routinely made contact with each family before the consent conversation (including any of: speaking by phone, introducing themselves, introducing the study, explaining that they’d be approaching for consent later, or any other reason).
    1. Strongly disagree
    2. Disagree
    3. Neither agree or disagree
    4. Agree
    5. Strongly agree
  • 35.

    Please describe the typical approach for initially contacting the family (e.g., in person vs. phone; one vs. multiple conversations).

  • 36.
    How did you determine a good time to approach family for HEAL consent? (can choose multiple)
    1. Ask the bedside nurse before approaching
    2. Ask the attending physician before approaching
    3. Ask a resident or fellow before approaching
    4. Ask a nurse practitioner or physician assistant before approaching
    5. Look in the chart to get a sense of how baby was doing and what else might be going on
    6. Other, please describe
  • 37.
    At our site, the person approaching the family for consent usually spoke directly with the attending physician caring for the baby prior to approaching a family for enrollment in HEAL.
    1. Strongly disagree
    2. Disagree
    3. Neither agree or disagree
    4. Agree
    5. Strongly agree
  • 38.
    At this site, the study team often felt resistance from the clinical team regarding approaching a family for HEAL enrollment.
    1. Strongly disagree
    2. Disagree
    3. Neither agree or disagree
    4. Agree
    5. Strongly agree
  • 39.

    Please describe resistance from clinical team in approaching for HEAL, including from whom (nurse, NNP, PA, fellow, resident, attending, other).

D. Challenging situations/barriers

  • 40.

    What recruitment challenges came up during the HEAL study?

  • 41.

    How far were individuals involved in enrollment/consenting (e.g., researcher coordinators, research physicians) located from the NICU on weekdays? (examples: in the unit, in the same building, a 2 min walk down the hall, a 15 min drive away, taking call from home).

    41A. How far were individuals involved in enrollment/consenting (e.g., researcher coordinators, research physicians) located from the NICU on weekends, nights, and holidays? (examples: in the unit, in the same building, a 2 min walk down the hall, a 15 min drive away, taking call from home).

  • 42.
    At our site, distance between the location of those consenting and the NICU negatively affected recruitment for HEAL.
    1. Strongly disagree
    2. Disagree
    3. Neither agree or disagree
    4. Agree
    5. Strongly agree
  • 43.
    At our site, potential participants are hesitant to enroll in research.
    1. Strongly disagree
    2. Disagree
    3. Neither agree or disagree
    4. Agree
    5. Strongly agree
  • 44.

    Why do you think this may be true?

  • 45.
    At our site, potential participants are willing to enroll in research.
    1. Strongly disagree
    2. Disagree
    3. Neither agree or disagree
    4. Agree
    5. Strongly agree
  • 46.

    Why do you think this may be true?

  • 47.

    Please describe any specific steps or adaptations to overcome the challenges or impediments of recruitment for HEAL that were made at this site.

  • 48.

    Which elements of your site’s recruitment processes worked especially well in contributing to the success of patient recruitment at your site? Feel free to share any thoughts you have regarding what worked well.

  • 49.

    Please share 1-2 lessons that you learned during the 3 years of HEAL recruitment at your site, regarding how to optimize recruitment processes for neonatal treatment trials. We would appreciate any advice or observations you can share, whether big or small.

  • 50.

    Would you modify any of your site’s recruitment practices next time you are involved in a neonatal treatment trial? If so, how?

Research team

  • 51.
    Who was able to get consent for HEAL at your site (choose all that apply)?
    1. NICU attending
    2. NICU fellow
    3. Pediatric resident
    4. APP/PA/NP
    5. Neurology attending
    6. Neurology fellow
    7. Neurology resident
    8. Research coordinator
    9. Other, please describe
  • 52.

    How many individuals who were able to get consent for HEAL at your site? (give a number)

  • 53.

    How did this actually occur logistically (e.g., was there a single or small group of high performers? Was it evenly split among a larger group?)

  • 54.

    Describe the team involved with tracking of potential participants (e.g., numbers of individuals, their roles).

QUESTIONS FOR:

HEAL SITE PI,
PRIMARY HEAL RESEARCH COORDINATOR,
Individual who did the most consenting at each site

Please state the extent to which you agree with the following statements:

  • 63.
    It is likely that EPO will ultimately be shown to be effective as a neuroprotection for term infants with HIE.
    1. Strongly agree
    2. Agree
    3. Neither agree nor disagree
    4. Disagree
    5. Strongly disagree
  • 64.
    I would encourage a close friend or family member to enroll their infant in the HEAL study.
    1. Strongly agree
    2. Agree
    3. Neither agree nor disagree
    4. Disagree
    5. Strongly disagree
  • 65.
    Prior to HEAL, for how many clinical trials were you part of the team enrolling people?
    1. No experience
    2. 1-2 times
    3. 3-5 times
    4. 6 or more
  • 66.
    Prior to HEAL, what role did you serve enrolling people in clinical trials (choose as many as appropriate)?
    1. None
    2. Research coordinator for clinical trial
    3. Site-PI for clinical trial
    4. Lead PI for clinical trial
    5. Enrolling families for clinical trials lead by colleagues
    6. Other, please describe
  • 67.

    Describe your roles at work outside of HEAL (examples may include: medical director of NICU, NICU attending, division faculty, clinical nurse, clinical APP, NICU fellow, pediatric resident, working on other clinical trial, working on bench research, leading a research lab, serving on IRB, etc.).

Footnotes

SUPPORTING INFORMATION

The appendix is available in the “Supporting Information” section for the online version of this article and via Ethics & Human Research’s “Supporting Information” page: https://www.thehastingscenter.org/supporting-information-ehr/.

Contributor Information

Katherine F. Guttmann, assistant professor in the Department of Pediatrics at the Icahn School of Medicine at Mount Sinai

Sijia Li, doctoral student in the Department of Biostatistics at the University of Washington School of Public Health.

Yvonne W. Wu, professor of neurology and pediatrics in the Departments of Neurology and Pediatrics at the University of California San Francisco School of Medicine

Sandra E. Juul, professor in the Department of Pediatrics at the University of Washington School of Medicine

Benjamin S. Wilfond, professor in the Department of Pediatrics at the University of Washington School of Medicine and Treuman Katz Center for Pediatric Bioethics at the Seattle Children’s Research Institute

Elliott Mark Weiss, associate professor at the University of Washington School of Medicine and Treuman Katz Center for Pediatric Bioethics at the Seattle Children’s Research Institute.

REFERENCES

  • 1.Qureshi AI, et al. , “Low Patient Enrollment Sites in Multicenter Randomized Clinical Trials of Cerebrovascular Diseases: Associated Factors and Impact on Trial Outcomes,” Journal of Stroke and Cerebrovascular Diseases 21, no. 2 (2012): 131–42; [DOI] [PubMed] [Google Scholar]; Shea S, et al. , “Enrollment in Clinical Trials: Institutional Factors Affecting Enrollment in the Cardiac Arrhythmia Suppression Trial (CAST),” Controlled Clinical Trials 13, no. 6 (1992): 466–86; [DOI] [PubMed] [Google Scholar]; Ramanan M, et al. , “Does Asymmetry in Patient Recruitment in Large Critical Care Trials Follow the Pareto Principle?,” Trials 21, no. 1 (2020): 1–8; [DOI] [PMC free article] [PubMed] [Google Scholar]; Peckham E, et al. , “Successful Recruitment to Trials: Findings from the SCIMITAR+ Trial,” Trials 19, no. 1 (2018): 1–6; [DOI] [PMC free article] [PubMed] [Google Scholar]; Gazendam A, et al. , “Recruitment Patterns in a Large International Randomized Controlled Trial of Perioperative Care in Cancer Patients,” Trials 22, no. 1 (2021): 1–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Weiss EM, et al. , “Parental Factors Associated with the Decision to Participate in a Neonatal Clinical Trial,” JAMA Network Open 4, no. 1 (2021): e2032106; [DOI] [PMC free article] [PubMed] [Google Scholar]; Weiss EM, et al. , “Parental Enrollment Decision-Making for a Neonatal Clinical Trial,” Journal of Pediatrics 239 (2021): 143–149.e3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Rich W, et al. , “Antenatal Consent in the SUPPORT Trial: Challenges, Costs, and Representative Enrollment,” Pediatrics 126, no. 1 (2010): e215–21. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Toerien M, et al. , “A Review of Reporting of Participant Recruitment and Retention in RCTs in Six Major Journals,” Trials 10, no. 52 (2009): doi: 10.1186/1745-6215-10-52; [DOI] [PMC free article] [PubMed] [Google Scholar]; Hudson BF, et al. , “The Under Reporting of Recruitment Strategies in Research with Children with Life-Threatening Illnesses: A Systematic Review,” Palliative Medicine 31, no. 5 (2017): 419–36. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Levett KM, et al. , “Site-Specific Predictors of Successful Recruitment to a Perinatal Clinical Trial,” Clinical Trials 11, no. 5 (2014): 584–89. [DOI] [PubMed] [Google Scholar]
  • 6.Tooher RL, Middleton PF, and Crowther CA, “A Thematic Analysis of Factors Influencing Recruitment to Maternal and Perinatal Trials,” BMC Pregnancy and Childbirth 8, no. 36 (2008): doi: 10.1186/1471-2393-8-36. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Peckham et al., “Successful Recruitment to Trials: Findings from the SCIMITAR+ Trial.” [DOI] [PMC free article] [PubMed]
  • 8.Tooher Middleton, and Crowther, “A Thematic Analysis of Factors Influencing Recruitment to Maternal and Perinatal Trials”; [DOI] [PMC free article] [PubMed] [Google Scholar]; Bruhn H, et al. , “Estimating Site Performance (ESP): Can Trial Managers Predict Recruitment Success at Trial Sites? An Exploratory Study,” Trials 20, no. 1 (2019): 1–10. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Juul SE, et al. , “High-Dose Erythropoietin for Asphyxia and Encephalopathy (HEAL): A Randomized Controlled Trial—Background, Aims, and Study Protocol,” Neonatology 113, no. 4 (2018): 331–38; [DOI] [PMC free article] [PubMed] [Google Scholar]; Wu YW, and Juul SE, “High-Dose Erythropoietin for Asphyxia and Encephalopathy (HEAL),” ClinicalTrials.gov, accessed January 3, 2022 https://clinicaltrials.gov/ct2/show/NCT02811263. [Google Scholar]
  • 10.Porter KM, et al. , “Approaches to Carrier Testing and Results Disclosure in Translational Genomics Research: The Clinical Sequencing Exploratory Research Consortium Experience,” Molecular Genetics and Genomic Medicine 6, no. 6 (2018): 898–909. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Kitterman DR, et al. , “The Prevalence and Economic Impact of Low-Enrolling Clinical Studies at an Academic Medical Center,” Academic Medicine 86, no. 11 (2011): 1360–66. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Peckham et al. , “Successful Recruitment to Trials”; [Google Scholar]; Bhatnagar S, et al. , “Development and Impact of an Intervention to Boost Recruitment in a Multi-Center Pediatric Randomized Clinical Trial,” Clinical Pediatrics 53, no. 2 (2014): 151–57; [DOI] [PMC free article] [PubMed] [Google Scholar]; Levett et al., “Site-Specific Predictors of Successful Recruitment to a Perinatal Clinical Trial.” [DOI] [PubMed] [Google Scholar]
  • 13.Weiss et al. , “Parental Factors Associated with the Decision to Participate in a Neonatal Clinical Trial”; Weiss et al. , “Parental Enrollment Decision-Making for a Neonatal Clinical Trial.” [DOI] [PMC free article] [PubMed]
  • 14.Foglia EE, et al. , “Short-Term Outcomes of Infants Enrolled in Randomized Clinical Trials vs Those Eligible but Not Enrolled,” Journal of the American Medical Association 313, no. 23 (2015): 2377–79; [DOI] [PMC free article] [PubMed] [Google Scholar]; Kelly ML, et al. , “The Participation of Minorities in Published Pediatric Research,” Journal of the National Medical Association 97, no 6 (2005): 777–83. [PMC free article] [PubMed] [Google Scholar]
  • 15.Weiss et al., “Parental Factors Associated with the Decision to Participate in a Neonatal Clinical Trial”; [DOI] [PMC free article] [PubMed] [Google Scholar]; Weiss et al., “Parental Enrollment Decision-Making for a Neonatal Clinical Trial.” [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Fletcher B, et al. , “Improving the Recruitment Activity of Clinicians in Randomised Controlled Trials: A Systematic Review,” BMJ Open 2, no. 1 (2012): doi: 10.1136/bmjopen-2011-000496; [DOI] [PMC free article] [PubMed] [Google Scholar]; Treweek S, et al. , “Strategies to Improve Recruitment to Randomised Trials,” Cochrane Database of Systematic Reviews, no. 2 (2010): doi: 10.1002/14651858.MR000013.pub6; [DOI] [PMC free article] [PubMed] [Google Scholar]; Park CH, et al. , “Comparison of Recruitment Strategy Outcomes in the National Children’s Study,” Pediatrics 140, no. 2 (2017): doi: 10.1542/peds.2016-2822. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Stewart TM, et al. , “Comprehensive Assessment of Long-Term Effects of Reducing Intake of Energy Phase 2 (CALERIE Phase 2) Screening and Recruitment: Methods and Results,” Contemporary Clinical Trials 34, no. 1 (2013): 10–20. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES