Addressing Statistical Power and Increasing Diversity in Hospice Research: Electronic Medical Record Participant Identification Compared to Nurse Referral Approaches to Recruitment

Abstract

Context.

Recruitment of targeted samples into hospice clinical trials is often challenging. While electronic medical records (EMR) are commonly used in hospital-based research, it is uncommon in hospice research. The community setting and the variability in hospices and their medical record creates unique challenges.

Objectives.

This paper compares recruitment in two hospice randomized controlled trials, each of which had a group recruited by using the EMR identification and a group recruited by nurse referral. We sought to answer three questions: 1) What is the impact of using the EMR to identify hospice participants for clinical research? 2) How do the referral count and consent rate (referrals that ultimately result in verbal informed consent to participate in research) differ between hospice agencies using an EMR participant identification approach compared to those using a nurse referral approach? and 3) What are the challenges associated with using the EMR to identify potential research participants?

Method.

Recruitment data from two hospice clinical trials was combined into a new database. Data from hospice nurse referral agencies was compared with data from those agencies who participated in EMR-identified referrals.

Results.

The EMR identification process was feasible and efficient, resulting in more referrals and more consented participants than the nurse referral method. Of particular interest is that 8% more black caregivers were recruited using the EMR identification process than the nurse referral.

Conclusions.

The EMR-identified recruitment process is the recommended method in hospice research.

Background Challenges of Recruitment Into Clinical Trials

Randomized clinical trials (RCTs) have long been considered the gold standard for testing the efficacy of health interventions.^1–3 A key element in the planning of RCTs is the determination of a goal for a sample size. The sample size calculation must be high enough to detect statistical and/or clinical significance with confidence that the outcome is not due to chance. Recruitment into clinical trials is one of the biggest challenges for most research teams.^2,4,5 Many trials fail because they are unable to reach their recruitment goals. One study of 440 trials found that nearly half (45%) had to request an extension to meet their targets and, even with an extension, only 78% recruited 80% or more of their target sample size.⁶ Recruitment barriers include lack of access to the study population, difficulty identifying participants who meet all inclusion criteria, and gatekeeping by healthcare providers who are too busy or overprotective of potential participants.^7–10

Research in hospice and palliative care is no exception to these challenges. Boland concluded that consistent with other healthcare settings, RCTs in hospice and palliative care environments have “difficult to recruit” participants.¹¹ Common recruitment challenges include clinician gatekeeping, patients to sick to consent, and overwhelmed caregivers.^12–14 Complicating the matter further is the high attrition often observed in these settings, which raises the necessary sample size given retention-related challenges, including situational stressors and/or patient death due to advanced illness.¹⁵

Because of the challenges in recruiting participants to clinical trials, much has been written about strategies used to improve recruitment rates. One systematic review examined 15 studies of trials that tested the effect of specific recruitment strategies on accrual to clinical trials of people with organ failure and cancer.¹¹ They identified strategies that improved recruitment, including the provision of memory aids, contacting referrals before arriving to consent participants and using “opt-out” consent protocols.¹¹ A Cochrane review of recruitment strategies from 68 clinical trials found three strategies that improved recruitment and showed high-certainty evidence. Informing potential participants what treatment they would receive, using a follow-up telephone call for people who did not respond to a mailed invitation, and developing tailored recruitment materials with extensive input from members of the study population were noted as effective.¹⁶ However, the literature shows that too few clinical trials analyze and report on the effectiveness of their recruitment strategies.^6,11,16

The lack of evidence regarding effective recruitment strategies extends to hospice and palliative care clinical trials as well. One study examined recruitment challenges specific to hospice clinical trials and found that when family members better understood the possible benefits and often limited burdens involved in the study, they were less anxious and more willing to consent to participate.¹⁷

Further complicating hospice research recruitment are racial disparities in hospice utilization, which result in disproportionately fewer racial and ethnic minorities receiving hospice care.¹⁷

Thus, while a given research sample may proportionately represent hospice care recipients, it may not reflect the diversity of our population as a whole. Additionally, hospice staff members may have conscious or unconscious biases and/or skill deficits that deter them from referring racial and ethnic minorities into research. For example, they may assume that minoritized individuals are disinterested in research or that other stressors or competing demands would preclude their participation. Furthermore, non-Hispanic white hospice staff members may lack the cross-cultural communication skills necessary to competently and confidently discuss research opportunities with minoritized hospice patients and their families.¹⁸

This paper reports on recruitment into two hospice RCTs testing psychosocial interventions for family caregivers of hospice patients (one trial focused on cancer caregivers, while the other focused on dementia caregivers). Both trials were underperforming with regard to recruitment milestones until researchers implemented a protocol in which hospice electronic medical records (EMR) were utilized to identify potential participants. While specific protocol logistics differed by hospice agency, the overall strategy was to generate EMR reports to identify patients whose family caregivers were likely to meet study eligibility criteria rather than relying upon hospice nurses to refer eligible individuals. Collaboration with the Institutional Review Board to obtain a partial waiver of HIPAA authorization for recruitment purposes allowed researchers to access automated reports listing likely eligible patients and/or family caregivers, permitting the research team to conduct further EMR screening and, if deemed appropriate, contact potential participants without requiring direct clinician involvement. This is in contrast to the former method of asking hospice nurses to identify participants and forward contact information to the research team for follow-up.

This paper reports on differences in recruitment in two hospice RCTs by comparing EMR participant identification and nurse referral approaches. We sought to answer three questions: 1) What is the impact of using the EMR to identify hospice patients and family caregivers for clinical research (number of attempts to contact, number oconsents, etc)? 2) How do the referral count and consent rate (referrals that ultimately result in verbal informed consent to participate in research) differ between hospice agencies using an EMR participant identification approach compared to those using a nurse referral approach? and 3) What are the challenges associated with using the EMR to identify potential research participants?

Methods

Sample: Two Hospice Clinical Trials

Data from two similar but separate hospice RCTs were compared to answer the research questions. The two trials had some overlap in investigators, research staff, and participating hospice agencies. Both were five-year NIH-funded RCTs testing psychosocial interventions for hospice family caregivers (2018–2023; 2020–1014). However, the trials involved participants who were caring for patients with different diagnoses (cancer and dementia), employed different study designs (cluster randomized trial and individually randomized trial), and the interventions were entirely unique.

Study one (Access for Cancer Caregivers to Education and Support for Shared decision-making [ACCESS]) is a pragmatic crossover cluster RCT for family caregivers of hospice patients with cancer. A detailed protocol is published elsewhere.¹⁹ Using a three arm crossover cluster trial design that randomizes hospice agencies into three groups (two intervention and one usual care). The intervention tested the efficacy using web conferencing as a tool to improve decisionmaking for caregivers included in hospice team meetings. Inclusion criteria required family members to be English speaking, at least 18 years of age, and designated as a caregiver and decision maker for the hospice patient diagnosed with cancer and enrolled in a participating hospice. Finally, the family caregiver had to be willing to participate in the ACCESS intervention if they were receiving services from a hospice agency randomized to the intervention group at the time of the individual’s recruitment into the study. The target sample size was 468 family caregivers.

Caregiver Speaks is a recently completed two-arm, multisite, pragmatic RCT of a technologically mediated storytelling intervention designed to improve caregivers’ depression and anxiety by facilitating their ability to make meaning of their caregiving and bereavement experiences through sharing their personal stories. The intervention is designed for family caregivers over the age of 18 years caring for hospice patients living with dementia (PLWD) who express a willingness to participate in the intervention. A detailed protocol is published elsewhere.²⁰ The target sample size was the same as ACCESS, 468 family caregivers.

Each study maintained a separate database for potential participants and for those who consented to participate. Both studies created regular reports documenting referrals/potential participants, consents, reasons for declining to consent, withdrawals, discharges, and deaths (similar to Table 1). These reports, created by our research staff were termed “recruitment dashboards” and allowed continuous engagement by the research team through weekly review of just-in-time data to identify recruitment trends at each study site and problem-solve around low rates.

Table 1.

Summary of Completed Surveys and Demographics by Referral Type

	EMR (N = 743)	Nurse (N = 201)	P-valuea
Caregiver age
Mean (SD)	58.3 (11.7)	57.4 (11.8)	0.428
Median (min, max)	59.0 (22.0, 90.0)	59.0 (22.0, 90.0)
Missing	12 (1.6%)	3 (1.5%)
Caregiver race
White/Caucasian	589 (81%)	175 (89%)	0.012
Non-white	135 (19%)	21 (11%)
Missing	19 (2.6%)	5 (2.5%)
Caregiver ethnicity
Non-hispanic	713 (97%)	194 (99%)	0.295
Hispanic	19 (3%)	2 (1%)
Missing	11 (1.5%)	5 (2.5%)
Caregiver income
Under $20,000 per y	69 (11%)	15 (9%)	0.351
$20,000 to $39,999	125 (20%)	32 (19%)
$40,000 to $69,000	170 (27%)	57 (33%)
Over $70,000	276 (43%)	67 (39%)
Missing	103 (13.9%)	30 (14.9%)
Area deprivation national percentile
Mean (SD)	57.0 (24.3)	54.3 (21.8)	0.183
Median (min, max)	57.0 (1.00,100)	55.0 (2.00, 98.0)
Missing	49 (6.6%)	19 (9.5%)

^aP-values were derived from Chi-square tests for categorical variables and Wilcoxon two-sample tests for continuous variables.

Percentages for categorical variables were calculated using the nonmissing sample.

Trial Recruitment Protocols

The trials had similar recruitment protocols. When moving to the EMR participant identification process, there was no change in inclusion criteria for either study. The initial participant referral method for both studies required hospice nurses to share a flyer with families upon hospice admission and ask family caregivers if the nurse could share their contact information with the research team. If the family caregiver agreed to be contacted, hospice nurses sent the contact information to the research team for follow-up. In these situations, receipt of contact information by the research team was defined as a “referral.” Upon referral, a research staff member called the family caregiver and made a final determination that the potential participant met the inclusion criteria (defined as qualified referral). Research staff also explained the study and asked for consent to participate. Once consent was received the participant completed the baseline surveys for the study.

Covid-19 prompted us to remove hospice nurses from the referral process given that they were preoccupied with clinical responsibilties. We decided that an automated referral process using the EMR to identify potential participants would be a more efficient and effective approach to recruitment. We met with each participating hospice agency in each trial and discussed how this new process might work in their particular setting. Some agencies welcomed the change, while others were uncomfortable and preferred the existing referral approach. As a result, both studies had agencies that remained with the historical referral approach and those that welcomed the EHR participant identification model.

The first step in implementing EMR participant identification at hospice agencies that were in agreement with this approach was to obtain approval of the Institutional Review Boards (IRB) of record (each RCT had a different IRB) and the issuance of a partial waiver of HIPAA authorization for recruitment purposes. Given the diversity in hospice agencies, EMR identification protocols needed be individually outlined with each agency. Both referral/participant identification processes are shown in Fig. 1.

Fig. 1.

Comparison of two group recruitment process.

In total, there were 11 hospice agencies from which data were collected (three unique to ACCESS, 4 unique to Caregiver Speaks, and four that were utilized by both studies). Four of the sites utilized nurse referral throughout the entire duration of the study, five sites utilized EMR throughout the entirety of the study, and two sites started as either EMR or nurse and switched at some point during the study. Regardless of how contact information was received for potential participants, each RCT maintained two separate databases. One database, called the referral database, had contact information before consent and the other database (termed “consent database”) contained postconsent data. The referral database contained information on all potential participants without respect to final determination regarding whether they met the RCT inclusion or exclusion criteria or their decision to participate. Referral data were limited and included basic contact information, hospice agency, and hospice admission date. We were thus able to look at the number of potential participants in each hospice agency every week. Research staff attempted to contact all potential participants. Even those they were ultimately unable to contact via phone were entered into the referral database. Upon initial contact, the research staff member determined if the potential participant met study inclusion criteria. If the family caregiver was determined to be eligible, the referral was labeled “qualified.” For this analysis, data from both trials were combined and study participants were classified by their referral method.

Overall Study Design and Data Analysis

Data for both studies was combined into one data base for secondary analysis. We used descriptive statistics and bivariate statistical tests to examine differences between EMR participant identification and nurse referrals for both trials combined. Specifically, we examined differences between referral approaches on referral outcomes and associated variables including whether the referral was qualified, whether the person referred ultimately consented, and mean number referral calls. Within our consented sample, we also examined differences in caregiver age, caregiver race, caregiver ethnicity, caregiver household income, and the area deprivation index of the caregiver’s neighborhood.^21,22 We examined mean, standard deviation, median, and range for continuous variables, and frequencies for categorical variables. We also examined bar charts visualize differences in qualified referral counts, consent counts, and consent rates between EMR participant identification and nurse referrals. To answer our research questions, we used chi-square tests and Wilcoxon tests. We considered P < 0.05 to be statistical evidence of differences between EMR participant identification and nurse referrals.

Results

Table 1 summarizes the demographic characteristics of consented participants who entered the study via EMR participant identification and nurse referral. There was only one statistically significant difference in the demographics between the two groups: There were 8% more Black family caregivers in the group of participants who were identified via the EMR than there were in the group of participants who were referred to the study by a hospice nurse (P = 0.012).

Table 2 shows the final recruitment statistics by group (EMR participant identification versus nurse referral). Figs. 2–4 graphically illustrate the differences in qualified referrals, consents, and consent rates for the two groups. Both trials met their target enrollment and had a combined total of 946 consented participants. ACCESS consented a total of 489 participants, which exceeded its recruitment target (468), and Caregiver Speaks met 98% of the target sample size with 457 participants, however 2 (n = 2) cases were excluded from the current analysis due to being self-referrals.

Table 2.

Summary of Recruitment by Referral Type

	EMR (N = 4388)	Nurse (N = 616)	P-valuea
Referral result
Nonqualified referral	1540 (35%)	210 (34%)	0.672
Qualified referral	2825 (65%)	402 (66%)
Missing	23 (0.5%)	4 (0.6%)
Consent
Did not consent	1997 (73%)	200 (50%)	<0.001
Consented	743 (27%)	201 (50%)
Missing	1648 (37.6%)	215 (34.9%)
Referral calls
Mean (SD)	2.17 (1.90)	2.44 (1.85)	<0.001
Median (min, max)	1.00 (0, 6.00)	2.00 (0, 6.00)

^aP-values were derived from Chi-square tests for categorical variables and Wilcoxon two-sample tests for continuous variables.

Percentages for categorical variables were calculated using the nonmissing sample.

Fig. 2.

Qualified referral count by referral type.

Fig. 4.

Consent rate by referral type.

Note: Consent rate is the percent of all qualified referrals that consented to enroll in the study.

The results indicate there were statistically significant differences in recruitment outcomes between EMR participant identification and nurse referrals. While there was no statistically significant difference between qualified referrals (i.e. both methods provided a similar proportion of qualified referrals), 50% of nurse referrals agreed to participate in their respective study and 27% of EMR referrals agreed to participate (P < 0.001). Despite not being statistically significantly different by referral method, the number of qualified referrals (nurse referred or EMR identified potential participant who met inclusion criteria) to the research study was substantially higher in the EMR group (Fig. 2), which led to more consented participants (Fig. 3). However, the consent rate (percentage of potential participants who consented and enrolled in the trial) was lower in the EMR group (see Fig. 4). In other words, a larger percentage of potential participants referred to the study by a hospice nurse agreed to participate than did potential participants identified by the EHR, however the count for both qualified referrals and consents was much greater for EHR than nurse referral. Additionally, potential participants identified via EMR required fewer referral calls on average and had a lower median number of referral calls to complete the referral process (regardless of outcome). In both cases, potential participants were called at various times of day and days of the week for a maximum of 6 calls. For potential participants identified via EMR, the median participant required 1 call whereas the median participant for nurse referrals required 2 calls (P < 0.001).

Fig. 3.

Consent count by referral type.

Discussion

EHR identification of potential participants for hospice research resulted in recruitment of a greater number of family caregivers than did nurse referrals in these two trials. While using the EMR to facilitate research participation is relatively novel in the hospice research literature, it is not unusual in other healthcare settings.^11,23 A systematic review examined the impact of using the EMR for clinical research recruitment and found that in 13 studies the EMR was effective due to its capability for identification and fast processing time, especially for clinical trials in acute conditions.²³ Individuals conducting the review also found that EMR participant identification was sufficiently precise with regard to correlation to eligibility criteria, with low levels of false positives and false negatives. Furthermore, EMR participant identification often resulted in cost savings, as the primary cost involved in this approach was to cover time and resources required to set up the system for automatic screening.²³ Similarly, Miller et al. concluded that EHR-based cohort selection is a promising approach to identifying and enrolling research participants.²⁴ Likewise, EMR participant identification has been shown to be an effective approach in studies involving patients with rare diseases and cardiovascular diseases.²⁵

It is interesting to note that potential participants identified via EMR required fewer referral calls on average and had a lower median number of referral calls to complete the referral process (regardless of outcome). This means that overall the staff cost for recruitment was lower in the EMR referral group. One possible reason is that we receive the contact information sooner so perhaps caregivers are more likely to answer the phone and are less overwhelmed. It may be beneficial to explore this further in future studies.

Of particular interest in this study is the difference in the increased recruitment of minority participants (Black family caregivers) in the EHR participant identification group versus the nurse referral group. Racial disparities in access to hospice care is a long-standing issue.^26,27 Given the lower number of racial and ethnic minority individuals receiving hospice services when compared to nonminoritized individuals, achieving racially and ethnically diverse samples of hospice research participants is challenging.^26,27 The successful recruitment of 8% more Black family caregivers in the EMR participant identification group is an important finding and demonstrates an potentially unexpected benefit of EMR recruitment for hospice research. We have no data to show a causal relationship but future research into this area especially to explain bias, would be helpful.

In addition to numerous benefits, using the EMR to identify potential participants for hospice research has several unique challenges compared with utilization of this approach in other healthcare settings. First, not every hospice utilizes an EMR. EMRs in hospice have faced implementation challenges as most are modified from hospitals or home health agency-focused Information Technology (IT) systems and hospices do not generally have dedicated IT staff. Additionally, small rural hospices have found it too expensive to invest in an EMR for few patients. Finally, not every hospice that utilizes an EMR utilizes the same EMR, making the generating of automatic reports a customized experience when working across several different hospice agencies. Therefore, working with eleven hospice agencies and their agency-specific EMRs required us to have eleven different specific recruitment protocols.

Finally, the lack of reliable five year publically available, national, data on each hospice agency makes the comparison by size and by population served difficult. While one state requires this reporting, the second state did not. Nor do we know the rural/urban distribution for the same reason. Additionally, the hospice landscape is changing so fast and frequently that basic information such as ownership, tax status, and office location also changed over the five years of the study. Another challenge in doing hospice research is that what is negotiated at the beginning of a study had to be renegotiated with each administration and ownership change.

It is important to note the differing opinions of hospice leaders regarding the use of the EMR for research. All of the participating hospices had IT capacity for EMR research, the difference in participation was their understanding of a HIPAA waiver and comfort with sharing of information without stated consent. Despite having a partial waiver of HIPAA authorization for recruitment purposes, it was challenging for some hospice agency administrators to feel comfortable providing the contact information for individuals who had not consented to the release of their information when they enrolled in hospice. While hospital-based hospices may mention research in their standard consent and enrollment packets, research in other hospice agencies is rare and thus not usually a part of the clinical consent documentation. While most hospice agency concerns about privacy can be addressed with the partial HIPAA waiver and IRB approval, successful implementation of an EMR participant identification approach still requires hospice leadership to understand the process and regulations carefully and potentially contact their legal staff, as many corporate providers are unfamiliar with these waivers. There is typically little incentive for hospice administration to engage in these potentially time-intensive tasks, likely limiting uptake of this approach.

Another challenge that is unique for the two RCTs described here is that they targeted family caregiver participants rather than patient participants. Some EMRs contain limited information about family members, so this information had to be obtained through open notes or inquiring when calling a patient’s phone number. Another challenge is that, while the EMR contains some diagnostic information, in the case of hospice, a terminal diagnosis is carefully coded, but this is not always true for a secondary or comorbid diagnosis. In the Caregiver Speaks RCT, which enrolled family caregivers of patients with dementia as a terminal/primary or comorbid/secondary diagnosis, we found that dementia was not always noted in the EMR, except as a complication in the open notes. In this case, the research staff members had to screen clinical notes to assess the inclusion criteria of dementia. Thus, the EMR referral report cannot always be taken on face value and may require further research to ensure inclusion criteria are met.

Regardless of the EMR, the ability of an EMR participant identification report to be run on a daily basis, utilizing an EMR participant identification approach enabled researchers to contact hospice patients and their families sooner after hospice enrollment than does a nurse referral approach, which must follow a nursing visit and which requires time for the nurse to discuss the study with the patient and/or family caregiver and then provide contact information to the research team. The ability for the research team to follow up in a more timely manner is particularly advantageous when recruiting cancer patients or their family caregivers, as the average length of stay is very short and the need to intervene quickly is important.²⁸ However, in the case of dementia, where the average length of stay was more than twice that of the cancer patients in our two RCTs, we ultimately chose to wait a few days after the hospice enrollment before approaching families identified via EMR. We reasoned that this would allow families to adjust to hospice and thus we were not contacting them when they were potentially overwhelmed with meeting their hospice team and becoming oriented to hospice services.

While the number of potential participants increased with the EMR participant identification approach (and, thus, the number of family caregivers who had the opportunity to participate in our research increased), the percentage of family caregivers who consented after having been identified via EMR was lower than the percentage of family caregivers who consent after being referred to the research team by their hospice nurse. This was likely due in part to the perceived endorsement of the research by the nurse when they introduced the study to patients and family members.

Finally, given the limited information available to our research team via the EMR participant identification report and in light of how quickly the condition of a hospice patient can change, we established standard operating procedures to ensure that research staff members confirmed current patient status and acuity information immediately preceding their contact of the patient or family caregiver for sites that utilized the EMR participant identification approach. These processes prevented us from attempting to contact the families of deceased or actively dying patients, which both we and our clinical partners deemed important. Although at hospice sites utilizing the nurse referral approach nurses were instructed not to refer patients (or family caregivers of patients) who were actively dying, our research team was typically unable to access just-in-time information when recruiting potential participants from these sites. If a patient’s condition had deteriorated rapidly and/or unexpectedly since the nurse’s original referral, we risked contacting the patient’s family at an inconvenient and emotionally charged time, highlighting yet another benefit of the EMR participant identification approach.

Recruitment and the resulting consent not only depend on the identification of potential participants but also on several other issues. While the EMR provides additional potential participants, it is still critically important that the information provided about the study be presented clearly with a focus on the benefits the research can provide and the importance of the research in helping others. These two components of the explanation of the research project are important to participants.^17,29

Conclusion

The EMR can be an important tool for hospice researchers recruiting for clinical trials. Recruitment with the EMR is often faster, less costly, and, in our experience conducting two recent hospice RCTs, resulted in a higher number of minority participants than the more traditional nurse referral approach. However, using the EMR requires important education of hospice administrators to address privacy concerns. Additionally, there are many considerations when setting up the systems to ensure that the inclusion criteria are matched, which may require manual screening of free text which cannot be automatically extracted. Finally, it remains important to have a carefully worded recruitment script following the identification of appropriate referrals. Our team concludes that the EMR is the most successful way to identify referrals in this setting and obtain the necessary sample size for adequately powered trials.

Key Message.

Hospice clinical trials can benefit from working toward EMR-based identification of potential study participants. The process requires individual arrangements with every hospice program but the time savings, efficiency and improvement in the recruitment of minority participants make it worth the effort.