Challenges in Implementing Hospice Clinical Trials: Preserving Scientific Integrity While Facing Change

Abstract

Background/Aims:

Numerous changes can occur between the original design plans for clinical trials, the submission of funding proposals, and the implementation of the clinical trial. In the hospice setting, environmental changes can present significant obstacles, which require changes to the original plan designs, recruitment and staffing. The purpose of the paper is to share lessons and problem-solving strategies that can assist in future hospice trials.

Methods:

This paper uses one hospice clinical trial as an exemplar to demonstrate challenges for clinical trial research in this setting. Using preliminary data collected during the first months of a trial, the research team details the many ways their current protocol reflects changes from the originally proposed plans. Experiences are used as an exemplar to address the following questions: 1) How do research environments change between the initial submission of a funding proposal and the eventual award? 2) How can investigators maintain the integrity of the research and accommodate unexpected changes in the research environment?

Results:

The changing environment within the hospice setting required design, sampling, and recruitment changes within the first year. The decision-making process resulted in a stronger design with greater generalization. As a result of necessary protocol changes, the study results are positioned to be translational following the study conclusion.

Conclusion:

Researchers would do well to review their protocol and statistics early in a clinical trial. They should be prepared for adjustments to accommodate market and environmental changes outside their control. Ongoing data monitoring, specifically related to recruitment, is advised.

The randomized clinical trial (RCT) design has traditionally been regarded as the gold standard for assessing efficacy of interventions. The experimental method can control for bias, resulting in high internal validity. A limitation of this design is that the strict inclusion and exclusion criteria, the blinding of those receiving the intervention, and the controlling of the environment in which the study is conducted, may harm the external validity of the approach.(1) While clinical trials inform healthcare practice, poor study design, complexity, inadequate power, infeasibility, and an inability to translate findings into practice have dampened their impact.(2, 3) Recent literature recommends that researchers conducting clinical trials do an early assessment of the trial by comparing the assumptions of the design with the early trial experience, especially related to recruitment and attrition assumptions.(4) In strict clinical trials this would potentially constitute a “pilot period,” however in a more pragmatic trial, given its flexibility to adapt to a real world setting, this early assessment would not necessarily have to separate from the trial.(1, 5)

Pragmatic randomized trial designs have become an alternative to address the external validity concerns of the RCT. They provide flexibility and increase the likelihood of stronger external validity. Flexibility is built into the eligibility criteria, the intervention components, comparison groups or data sets, follow-up intensity, outcomes, participant compliance, and provider adherence, allowing the research to take place in conditions that most closely approximate actual clinical environments (1, 6)

Funding

Conducting a trial is expensive, and securing funding has challenges, especially in hospice and palliative medicine.(7) Study planning and design begins with an early set of assumptions based on preliminary work and the literature. One of the many challenges in conducting a trial is the time between planning and implementation, often requiring at least two years.(8) Additionally, it commonly takes five years to test an intervention and analyze results and nearly 12 months to publish results. Planning, funding, implementing, and disseminating findings of clinical trials can be a painfully slow process.(3)

When the complexities of clinical trial research are combined with the unique challenges found in end-of-life care settings, such as hospice, plans made years prior become out-of-date at the time of implementation.(8–13) Research in hospice is challenging given the short length of time individuals are enrolled (mean 72 days) and the frailty of the patients which often results in missing data.(14, 15) Additionally, hospice ownership and patient mix often change dramatically. For example, nationally, the percent of hospice patients with cancer went from 37% in 2013 to 27% in 2017.(14)

The purpose of the paper is to share lessons and problem-solving strategies that can inform future clinical trials. We share our experience as an exemplar to address the following questions: 1) How do research environments change between the initial submission of a funding proposal and the eventual award? and 2) How can investigators maintain the integrity of the research and accommodate unexpected changes in the research environment?

Funding Timeline

Access for Cancer Caregivers to Education and Support for Shared Decision-making (ACCESS) was awarded funding in January 2017 as a five-year, multi-site, three-arm, pragmatic, randomized, clinical trial. (R01CA203999; ) The original proposal was submitted in October 2013 and was resubmitted twice, the final time in March 2016. The Notice of Award of Funding from the National Cancer Institute was received on January 12, 2017, 39 months after the first submission and 10 months after the final submission. The approved study has three specific aims: 1) Evaluate the effect of the ACCESS intervention on hospice cancer caregiver anxiety, pain knowledge, and patient pain; 2) Evaluate the effect of online groups as sources of educational and emotional support for cancer family caregivers; and 3) Assess hospice staff and family caregiver satisfaction with shared decision-making processes.

Intervention

The ACCESS intervention and study design are based on preliminary work from a previous five-year clinical trial named ACTIVE (Assessing Caregivers for Team Intervention via Video Encounters) which used web conferencing to empower family caregivers of hospice patients to participate in hospice care plan meetings (R01NR011472; ). (16) The ACCESS intervention has two important modifications. First, the ACTIVE trial revealed that family caregivers of cancer patients had higher anxiety and depression and lower quality of life than caregivers of non-cancer patients; thus, ACCESS targets family caregivers of hospice patients with cancer. Second, family caregivers participating in ACTIVE care plan meetings were inadequately prepared to participate in decision making due to need for social support and education about caregiving in the context of advanced illness. As a result, care plan meetings in which family caregivers participated lasted nearly twice as long as those with only the hospice providers present, significantly limiting the likelihood of the intervention’s translation into routine practice.(16) ACCESS added an online support group to provide education and social support outside of the care plan meeting to reduce the additional time required for care plan meetings.(16, 17)

The ACCESS intervention consists of two components: 1) web conferencing to connect family caregivers to the hospice team during care plan meetings, and 2) a professionally-facilitated online support group for family caregivers, which includes educational content on pain, death and dying, caregiver self-care, social support, hospice care, and shared decision making. Based on our preliminary work, we hypothesize that family caregivers participating in the ACCESS intervention (study arm 1) will have lower anxiety and depression than those who only participate in the online support group but do not use web conferencing to join care plan meetings (study arm 2), and a control group who neither participate in the online support group nor use web conferencing to join care plan meetings (study arm 3).

Participant Recruitment

During the admission visit, the hospice nurse informs eligible family caregivers that their agency is partnering with researchers at the University to learn how to help family caregivers of hospice patients. Caregivers are given the opportunity to “opt out” of contact by the research team, and the nurse is asked to flag caregivers whose patient appears to be actively dying. A member of the research team calls all family caregivers who did not opt out and ensures eligibility, explains the study, and obtains verbal consent.

Participating family caregivers in all three study arms continue to receive usual hospice care for the duration of their involvement in the study and electronically complete standardized instruments. Following their exit from the study each caregiver is invited to participate in a semi-structured interview to provide feedback on their research experiences and the intervention components they received, if any.

Study Design Challenges

The study design, recruitment plan, and study sites mirrored the ACTIVE trial which successfully recruited more than 400 participants. Letters of support were provided from the ACTIVE study hospices, relying on the strength and experience with those agencies to yield the same results in the new trial. However, in the 10 months between the final submission of the grant proposal and the receipt of the funding, the administration of the all the proposed hospices had changed and they were no longer interested in participating in the research.. New hospice agencies were recruited that were geographically spread out across the state. While this change improved generalizability of results, it significantly increased research costs due to the long distances research staff would have to travel to every agency site for care plan meetings.

With increased research costs due to new partnerships with more geographically distant sites, it became clear that it would no longer be feasible to simultaneously deliver the full ACCESS intervention (study arm 1) at all hospice sites. Thus, before we began recruiting study participants, and with permission from the funding agency, we changed the initial individual randomization design to a randomized cluster design in which randomization is at the hospice site level and research staff need only attend care plan meetings at the sites randomly assigned to receive the full ACCESS intervention. An additional benefit of this approach was that contamination among the three arms was minimized, as all family caregivers at a given site were exposed to the same treatment condition.

Based upon the location of research staff, projected number of cancer patients served by each hospice, and diversity in geographic location, we selected six hospices as research sites. Three hospices were located in urban areas (west and central parts of the state) and three in rural locations (central and south parts of the state). We used a random number generator to assign 2 hospice sites to each study arm. This change necessitated a new power analysis, which revealed that 107 family caregivers would now be needed from each site. This increased our target sample size from 420 to 642 participants.

Under-performing Clusters Challenge

Recruitment began slowly. Several changes to the recruitment script were made as we reduced the technicality of the language we used, and we explicitly acknowledged caregiver stress and time considerations at the beginning of recruitment conversations. Additionally, we decided to wait five days after receiving a referral to contact potential participants, allowing time for the hospice staff to make their initial visits and more accurately identify family caregivers whose patients were actively dying and, thus, should not be contacted.

Following Lake and colleagues’ recommendation that the assumptions behind study recruitment should be assessed early in a clinical trial, we reviewed our enrollment and power assumptions after the first six months of recruitment.(4) Recalculating our power with the actual versus predicted attrition, we found our cluster power analysis to be appropriate. However, in examining recruitment at each site, we found the number of cancer patients admitted in one urban hospice had declined significantly since the year before. Given its current cancer enrollment, this site would have needed to enroll 100% of its cancer caregiver population to meet its enrollment target. Likewise, a rural hospice site would have needed to recruit 75% of its enrolled cancer caregivers. Neither of these targets were realistic. Complicating the issue even further, these two sites had both been randomized to the same study arm, so it would not be possible for a well-performing site to compensate for an under-performing site within the same study arm. Faced with an inadequate number of cancer patients at these hospice programs, we began searching the literature for alternatives and discovered that replacing the under-performing sites with new ones would violate the assumptions of our planned intention-to-treat analysis.(18) Moreover, adding clusters while retaining the existing ones only increased the required sample needed for adequate power.(19) We considered merging six clusters into three, but rejected this option because simulations of this strategy found that cluster merging produced a systematic reduction in study power.(20) We needed a more efficient study design that lowered our needed sample size to offset the necessary increase that would occur by adding an additional cluster.

Fowell et al suggest cluster crossover designs as a feasible and preferable design for studies involving dying patients and their families.(21) In this design, each cluster is exposed to two or more different treatment conditions at different points in time. Thus, each site essentially becomes its own control, which increases study precision as well as power. Crespi suggests that in a cluster crossover design only half the total number of clusters is needed when compared to a traditional cluster design.(22) A drawback to the cluster crossover design is that valuable months of the study must be dedicated to “wash out” periods to minimize the risk of contamination as clusters transition from one study arm to another.(22) Given that the average length of time family caregivers are enrolled in our study is 55 days and our intervention lasts a maximum of 90 days, we determined that a 90-day wash out period would allow all participants at any given site to complete their study involvement before the site transitioned to a different study arm. Reich outlines three considerations for a cluster crossover design: 1) the feasibility an intervention can be washed out, 2) the length of the study, and 3) the impact a crossover will have to efficiency and power of the study.(23) Rietbergen suggests that clusters be randomly assigned to a sequence of interventions so that each cluster receives each intervention in random order.(24) Based on this literature we changed from a cluster randomized trial to a cluster crossover randomized trial, accommodating the under-preforming cluster without violating the intention-to-treat premise or increasing research costs. Our early assessment of participant accrual and corresponding recruitment processes allowed us to identify significant challenges and address them early enough that necessary, yet substantial changes were feasible within our study timeline.

The increased power resulting from the crossover design allowed us to add an additional cluster in another urban area in the eastern part of the state. The additional urban hospice gave us a more diverse geographic sampling frame. These decisions improved the power and precision of our study and the generalizability of our results with only a modest increase in cost. Using the same computer-generated number approach to randomization, the additional cluster was assigned to study arms 1 and 2 (i.e., the full ACCESS intervention and the online group only portion of the intervention). We determined that each cluster would be exposed to two of the three treatment conditions at different points in time in an effort to be sensitive to the demands on the hospices as we changed the nature of the research with each crossover. As shown in Table 2 a crossover schedule was created where each cluster experienced one arm for 12 months, underwent a 90- day washout period, experienced the second arm for 12 months followed by another 90-day washout period, and then returned to their initial study arm for the remainder of the data collection period. The original 6 hospices had three waves or two crossovers, the additional hospice would have two waves or one crossover. This final randomization and additional site resulted in 4 clusters for the ACCESS intervention, 5 clusters for the Facebook intervention, and 5 control clusters.

Table 2:

Summary of Crossover Design

Grant Quarter	Hospice I	Hospice W	Hospice G	Hospice J	Hospice C	Hospice O	Hospice B
Q1	Planning and Start up	Planning and Start up	Planning and Start up	Planning and Start up	Planning and Start up	Planning and Start up
Q2
Q3	FB
Q4	FB	A	A	C
Q5*	FB	A	A	C	FB	C
Q6	FB	A	A	C	FB	C
Q7	WO	A	A	C	FB	C
Q8	A	WO	WO	FB	FB	C
Q9**	A	C	C	FB	WO	FB
Q10	A	C	C	FB	A	FB
Q11	A	C	C	FB	A	FB	A
Q12	WO	C	C	WO	A	FB	A
Q13	FB	A	A	C	A	WO	A
Q14	FB	A	A	C	WO	C	A
Q15	FB	A	A	C	FB	C	WO
Q16	FB	A	A	C	FB	C	FB
Q17					FB	C	FB
Q18					FB		FB
Q19							FB
Q20	Final Analysis
Total Cross overs	2	2	2	2	2	2	1

^*Q5 Preliminary analysis A=ACCESS intervention group; FB= Facebook intervention group; C=Control group;

^**Q9 Second analysis of recruitment

Using formulas suggested by Lake and Arnup for computing power in a small cluster randomized crossover trial with two arm crossover, we recalculated our power.(4, 25, 26) Based on our 32% first year attrition rate, and 90% power, our new sample size under the cluster crossover design with seven clusters was 74 per cluster for a total of 518. (See Table 1) The cluster crossover design resulted in a considerably smaller sample size for each cluster and overall. With our final design, we would need 74 participants at each site and 518 total as compared to the cluster design which needed 107 per cluster and a total of 642. The cluster crossover design decision is not only more realistic but also results in greater generalizability of results due a larger geographic and a more diverse sample.

Table 1:

Summary of Power Calculations and Assumptions by Design

Time	Primary outcome	Mean difference	Within group SD	Attrition	Alpha	Tails	Power	ICC	Per arm/cluster Sample size	Number of arms/clusters	Total sample size
Individual randomization	GAD-7	3 points	4	15%	.05	2	90%	NA	140 per arm	3 arms	420
Cluster design	GAD-7	3 points	4	20%	.05	2	90%	.25	107 per cluster	6 clusters	642
Cluster Crossover design	GAD-7	3 points	4	32%	.05	2	90%	.25	74 per cluster	7 clusters	519

Intervention Challenges

Our third major challenge was a lack of engagement with the educational videos provided as part of our online support group. Despite preliminary evaluation of video content with hospice staff and former caregivers during our planning, our preliminary analysis revealed that, on average, only 41% of each educational video was being viewed. While the video on death and dying was considerably more popular than the others (on average, caregivers viewed 52.7% of the video), its content was still not being fully seen. Additionally, our review of exit interviews found very few comments on any of the videos. Thus, we concluded that major changes were needed to increase engagement with this content. To that end, we partnered with a researcher with expertise in using telenovelas (short, dramatic videos popular in many Latino cultures) to provide education around end-of-life issues. Despite low enrollment of Latino’s in the project (1%), we hypothesize this format will be successful in other populations. The new four chapter telenovela covered the same topics as our more didactic videos, but using a more narrative (and potentially more engaging(27)) educational approach. We will be comparing this approach with our initial videos.

Implications for Future Hospice Clinical Trial Research

While the examples in this paper are specific to hospice, similar issues arise in other clinical trial settings. The ever-changing landscape of healthcare in particular creates similar challenges due to mergers, buyouts, and bankruptcies. Recruitment is a well-documented challenge in all clinical trials. The lag time between funding and a final award is a problem in all research settings. Hospice research, despite its unique population, is fairly representative of challenges found in most behavioral intervention trials.

A recent systematic review identified only 10 clinical trials that had been conducted in hospice care between 1985 and 2015.(28) The literature explains the many challenges that have been experienced in trying to conduct these trials, recruitment being the most problematic.(29) The challenges and corresponding problem solving we detail here have many important implications for future intervention research.

When planning a clinical trial, investigators should consider the fast-paced, rapidly-changing setting during the design phase. Researchers should balance the need to maintain scientific rigor and integrity with the flexibility and responsiveness required to successfully conduct large-scale, multi-site studies in diverse and geographically dispersed sites. Our pragmatic trial design and early examination of research processes allowed our team to modify our trial to accommodate the many changes that occurred between the time we wrote our grant proposal and the actual start of our study. While sometimes challenging to “sell” to scientific review groups, research needs to be implemented in a real-world setting if it is to be translational; thus more pragmatic and flexible trial designs deserve serious consideration.

A planned preliminary analysis of recruitment that includes forecasting based on actual experience following a few months of implementation is also advised. Reassessing initial assumptions based on population size, recruitment success, attrition, and power assumptions early in the trial allow opportunities for important adjustments. Finally, the efficiency of a crossover trial should be seriously considered in the planning phase. (21) Power is increased substantially with this design and thus it may be more cost efficient. If a study can withstand the washout period(s), it is worthy of consideration. (23)

Evidence-based interventions are needed in hospice care.(30) It is important that investigators remain aware of the changing environment in which they plan to conduct research and make plans that are sufficiently flexible to allow them to respond to changes that affect the feasibility of their research or the likelihood of their study findings actually affecting clinical practice. A data monitoring plan that targets known challenges and a selection of a study design appropriate for the real world can dramatically assist in supporting a successful trial.

Table 3 :

Summary of Cluster Design

	ACCESS Group	FACEBOOK Group	CONTROL Group
Hospice I	x
Hospice W	x	x	x
Hospice G	x		x
Hospice C		x	x
Hospice O		x	x
Hospice J		x	x
Hospice B	x	x
Total Clusters	4	5	5

^*Hospices (Clusters) randomized to groups and intervention order using random number generator

Key Message.

This article describes the challenges of funding and implementing a clinical trial in the hospice setting using one ongoing trial as an example. The authors advocate for pragmatic designs allowing for better translation of findings and preliminary analysis testing recruitment assumptions.