Methodological Challenges in Conducting a Multi-Site Randomized Clinical Trial of Massage Therapy in Hospice

Abstract

Researchers conducting multi-site studies of interventions for end-of-life symptom management face significant challenges with respect to obtaining an adequate sample and training and retaining on-site study teams. The purpose of this paper is to describe the strategies and responses to these challenges in a multi-site randomized clinical trial (RCT) of the efficacy of massage therapy for decreasing pain among patients with advanced cancer in palliative care/hospice settings. Over a period of 36 months, we enrolled 380 participants across 15 sites; 27% of whom withdrew prior to study completion (less than the anticipated 30% rate). We saw an average of 68% turnover amongst study staff. Three key qualities characterized successful on-site study teams: (1) organizational commitment; (2) strong leadership from on-site study coordinators; and (3) effective lines of communication between the on-site study coordinators and both their teams and the university-based research team. Issues of recruitment, retention and training should be accounted for in hospice-based research study design and budgeting.

Overview of the REST (Reducing End-of-Life Symptoms with Touch) Study

In 2003, researchers from the University of Colorado Denver Schools of Medicine and Nursing received funding from the National Institutes of Health (NCCAM-1 R01 AT01006-01A2) to conduct a multi-site randomized clinical trial (RCT) to examine the efficacy of massage therapy for decreasing pain, improving quality of life, and lessening symptom distress in people with advanced cancer. Fifteen organizations, 14 of which were hospices and members of a national palliative care practice-based research network¹ served as study sites. Patients from these sites who were eligible and consented were randomly assigned to receive massage therapy (intervention) or the control condition, simple touch. Both treatment arms followed similar protocols which included up to six 30-minute treatment sessions over a 2-week period. At each site a team of committed hospice employees or volunteers implemented the study procedures. Participants receiving massage therapy had significantly greater immediate improvement in pain and mood; differences that were not sustained over time. Participants in both study arms experienced some improvement in pain and symptom distress over the course of their study enrollment; use of pain medications remained stable.²

Methodologic Challenges

Obtaining an adequate and appropriate sample

In designing and implementing the REST study, we encountered the common challenge of obtaining an adequate sample. Participant recruitment may be difficult in any clinical trial; characteristics of the hospice/palliative care environment present unique challenges. Despite palliative care/hospice studies devoting significant time and resources to participant recruitment, sample sizes are often smaller than anticipated.^3–5 Researchers may significantly modify study designs to maintain trial viability.⁶ In the REST study, we planned for a sample size of 440, but ended the study with 380 participants, of whom 27% withdrew prematurely. We anticipated these challenges through inclusion/exclusion criteria, sampling, and recruitment and retention strategies (Table 1).

Table 1.

Methodologic Challenges and Approaches in Conducting a Multi-Site Randomized Clinical Trial of Massage Therapy in Hospice

Challenge	Approach
Obtaining an adequate and appropriate sample
Inclusion/exclusion criteria	- Reflected the parameters of the study: Adults, diagnosed with advanced cancer, had at least moderate pain, life expectancy of at least 3 weeks, cognitively able to consent to treatment and respond to study questions.
	- Ensured patient safety: Excluded patients who had a potential for bleeding or who were on anticoagulant therapy, and patients presenting with an unstable spine or who were at high risk for fractures.
	- Minimized the effect of intervening variables on the outcomes: Excluded patients who had received massage therapy during the 4 weeks prior to study enrollment.
Sample size	- Ensured sufficient statistical power to detect real differences in outcomes: Accounted for high attrition rate (estimated at 30%) in this population.
Participant recruitment and retention	- Set explicit monthly accrual goals for each site based on hospice census - Monitored accrual and distributed monthly reports tracking the progress of each site.
	- Offered incentives and rewards to those sites meeting their accrual goals and to the most productive sites.
	- Regularly communicated the intent, importance, and procedures of the study via regular conference calls, site visits, and attendance at local hospice interdisciplinary team meetings.
	- Distributed study information flyers for care providers and patients/families.
Minimizing withdrawal	- Carefully emphasized during the consent process the 50-50 chance of receiving massage therapy.
	- Offered a post-study massage session for participants randomized to the control arm.
	- Chose an active control arm.
Training and retaining study teams at each site
Training teams at a distance	- Two large training sessions held in Colorado for original study sites.
	- Additional on-site training sessions for new sites with extensive training manual and video.
	- Review and remediation of study procedures and protocol for sites with extended lag time between training and enrollment.
	- To address significant team member turnover, a train-the-trainer model was employed so that on-site team members could train new or replacement members. Consistency and reliability of these new or replacement members was maintained by verbal tests over the phone to check knowledge and competence with study protocol.
	- Touch providers often shadowed a more experienced member until reaching comfort with the protocol.
Ongoing quality assurance and communication	- Site visits conducted at least once a year to monitor adherence to study protocol, respond to any questions, encourage and thank the teams, and to collect best practices from each site.
	- Monthly conference calls with study sites to reinforce best practices in recruitment, adherence to study protocol, and collective problem-solving.
	- One research team member assigned to each study site as the primary contact.

Inclusion and exclusion criteria

Researchers develop inclusion and exclusion criteria to reflect the parameters of the study, ensure participant safety, and minimize the effect of intervening variables on outcomes. In the REST study, participants were English-speaking adults (age ≥ 18 years) diagnosed with advanced cancer, with at least moderate pain, and a minimum life expectancy of three weeks. Recruiting participants with this life expectancy from hospices can be difficult. By the time patients enroll in hospice life expectancy may prohibit participation in research; the average length of stay in hospice care is 70 days.⁷

Five hundred and nine patients were screened for REST study enrollment; 129 were not enrolled due to ineligibility or refusal. Initial screening was conducted primarily through chart review or query of clinical staff. Although massage therapy is generally safe when provided by trained professionals, persons who may be hurt by massage therapy must be excluded from participation, including those at high risk for bleeding or fracture.⁸ Exclusion criteria (anticoagulation therapy, a platelet count less than 10,000 or a known unstable spine) were assessed in the second phase of screening, primarily by patient interview. Anticoagulation therapy accounted for 2% of ineligibles, an unstable spine for 1%; no one had a known platelet count <10,000. Also primarily ascertained from patient interview, the final phase of enrollment screening included pain level, cognitive impairment and recent massage therapy. The trial was limited to those with at least moderate pain (at least 4 on a 0–10 point scale) in the week prior to study entry in order to assure that there was potential to detect a change in this primary outcome. Cognitive changes in this population are common. To assure that participants were able to provide informed consent and to fully participate, we measured cognitive awareness with the Short Portable Mental Status Questionnaire (SPMSQ)^9,10; greater than 5 errors resulted in exclusion. For a clinical trial, it is important to enroll patients only after a reasonable wash-out period of prior use of the study intervention. For the REST study, participants were ineligible if they had professional massage within 4 weeks prior to screening. Four percent of those screened were ineligible because pain levels were less than moderate, 2% due to cognitive impairment and 2% due to recent massage. Eleven percent refused consent. As is appropriate in a randomized clinical trial, we collected data from those who declined participation as well as those who agreed so that characteristics of both groups could be compared.³

Achieving racial/ethnic diversity in the study sample was challenging as 82% of 2008 hospice admissions nationally were White/Caucasian.⁷ The small number of minority patients enrolled was a concern throughout the study and was addressed regularly by the research team and the Data and Safety Monitoring Board (DSMB), although minority enrollment was just slightly less than expected (24 Hispanic patients vs expected 28; 14 African-American patients vs expected 15) based on demographics of the patients served by the study sites.

Sample size

Sample size must provide sufficient statistical power to detect effects of the intervention on selected outcomes. The power calculation must take into account expected loss of subjects due to death, cognitive changes or symptoms that may prompt a decision to withdraw from the study. This need to oversample has been reported by others^3–5,11 and is an important consideration when planning hospice-based research. Based on previous experience, sample size for the REST study was calculated to assure a sufficient number of participants despite an expected 30% study withdrawal rate.

Participant recruitment and retention

Researchers recruiting participants from a population with advanced illness face significant barriers to participant accrual, posing threats to external validity due to selective recruitment.⁵ Specific barriers to study enrollment in this population include variations in procedures to identify possible participants, tension between research and practice goals,^12,13 clinical staff acting as gatekeepers, competing priorities and, for this study, preconceptions about massage or touch therapies. To identify potential participants there must be a systematic process of locating those who meet basic study criteria.

Our target enrollment was 440. As with many studies, initial enrollment was slower than anticipated. Therefore, we modified the target enrollment to 380 in the second year of the study and increased efforts to improve accrual in the final months. The study was re-budgeted and extended via a no-cost extension in order to maximize available months for study accrual. On-site study coordinators developed processes for initial screening and approaching potentially eligible individuals about study participation. Processes varied according to local resources. For example, sites with a robust information technology system were able to generate lists of potentially eligible participants. At other sites, the study coordinator regularly attended interdisciplinary team meetings to learn about potentially eligible patients. In planning for hospice-based recruitment, researchers should understand the local context and work with sites to devise a recruitment strategy that meets needs both for research rigor and local feasibility. We obtained Institutional Review Board (IRB) approval for these varied approaches to participant identification and recruitment.

In addition to different screening approaches, hospice staff often became gatekeepers, guarding their patients from what they perceived as bothersome involvement in research during the precious time that remained in their lives and actively obstructing recruitment efforts or passively refraining from supporting them. This occurred despite evidence that patients often view participation in research as a gift or legacy that they were leaving behind for others' benefits.¹⁴ Some patients decided not to enroll because of competing priorities; many patients feel a need to balance involvement in research with a desire to spend more quality time with family and friends. Those with fewer visitors seemed more open to presence of the study team. Another barrier to recruitment was focus of the study on touch therapies. Because massage involves intimate touch, people who are less familiar with massage therapy (often the elderly) may associate it with sexual activity, which can be a barrier to participation.

We addressed these recruitment challenges in several ways. We monitored accrual monthly and found that sites with high subject accrual had a study champion, someone who saw the value of conducting research on massage therapy and thus sold participation to staff and patients by sharing enthusiasm for the study and clearly explaining the need for rigorous research. Benefits of engaging an in house person for recruitment has been previously identified.^15,16 Like others,¹⁷ we found that staff education was critical to successful recruitment in order to dispel concerns about research exploitation. The REST team developed informational flyers for hospice staff and patients/families that briefly described study goals, basic study eligibility criteria and contact information for participation. Some sites distributed these flyers directly to all enrolled patients, essentially directly marketing the study.

Accrual goals were set based on each site's average daily census. Study sites received a monthly report showing enrollment progress by site by month and quarter; progress was benchmarked against both overall and site-specific target accrual (Figure 1), permitting sites to compare their progress. These reports encouraged healthy competition and camaraderie. In order to stimulate and reward recruitment, we initiated an IRB-approved incentive program after the first several months of sluggish enrollment. These incentives were not included in the original grant budget and so were funded out of other sources – an important lesson for future grant proposals. Originally budgeted payments per patient enrolled were designed to at least marginally compensate for time invested by the on-site study team members: $25 per visit for massage (maximum 6 visits), $15 per visit for simple touch (maximum 6 visits) and $75 per study participant for both the on-site data collector and study coordinator, for a maximum of $390 per patient enrolled. Of note, these budgeted amounts were significantly less than amounts being paid by a competing pharmaceutical study that was being conducted nationally at the same time as our NIH-funded study, creating a financial disincentive for hospices that were participating in both studies to spend staff resources towards the REST study. Additional incentives that we added over the course of the study to stimulate/reward enrollment included gas cards (initially valued at $10, then at $20, to offset the rising gasoline prices) for all on-site study team members at sites that met their quarterly accrual goals, a $250 donation to the hospices that met or exceeded their total expected study enrollment, and an additional $250 donation, a recognition lunch and congratulatory letter to the hospice CEO and a feature in the practice-based research network newsletter for the highest enrolling site. The incentive program components were suggested by on-site study team members. Despite these measures, there was wide variation in enrollment rates across the 15 study sites (mean, 72%; range, 30–100%), reflecting unique factors at play at individual sites.

FIG. 1.

Actual enrollment compared with target.

Minimizing withdrawal

Minimizing study withdrawal was another challenge. Of the 103 participants who withdrew (27%), the most common reason was patient or family request (31% of all withdrawals), followed by disease progression (22%), and development of cognitive impairment (17%). Coordination between participants, treatment providers, and interviewers was essential given the number of treatments (6 over a 2 week period) and follow up interviews (4 over the study period). Most other reasons for study withdrawal were related to scheduling conflicts.

Massage therapy studies are plagued by control group withdrawals.^18,19 People may consent to participate in massage therapy trials hoping for randomization to the group receiving massage. A major success of the REST study was that there was no greater withdrawal from the control group than the treatment group. One reason for this may have been the careful explanation to participants during the consent process that agreeing to participate meant having a 50-50 chance of receiving massage therapy. Another reason for the similar withdrawal rate may have been that the control exposure of simple touch was perceived as a pleasant experience. We also offered a free massage following study completion to all participants randomized to control. Inclusion of an active study arm for massage therapy studies and offering a post-study free massage are strategies to mitigate drop-out rates in massage therapy research. (11).

Training and retaining study teams at each site

Another major challenge in conducting this multi-site RCT was training and retaining study teams at each site. Challenges were related to training teams at a distance, ongoing quality assurance and communication, and competing demands of staff whose primary job responsibilities were not research-related. During the course of the REST study, there were a total of 15 sites enrolling patients, each with their own on-site study team. Six sites started at the beginning of the funding period, 2 joined in the second study quarter, 2 more in the third study quarter, 1 during the fourth study quarter, 3 during the fifth study quarter and 1 during the sixth study quarter. Study sites participated for varying periods of time, with only 3 of the original sites enrolling participants for the entire 3-year period. The researchers were located in Colorado; study sites, part of the Population-based Palliative Care Research Network (PoPCRN), were located in Colorado, California, Florida, Massachusetts, Illinois, Arkansas, North Carolina, and Washington, DC. On-site study teams consisted of at least five members: 1) a coordinator who provided oversight, recruited, screened and enrolled participants, requested randomization and assigned therapists to provide the study treatments, and reviewed and submitted all completed instruments; 2) a data collector who administered the instruments according to the protocol; 3) at least two massage therapists who administered the massage treatments and collected data before and after each treatment; 4) at least two providers of simple touch (control condition), who were volunteers or hospice workers who administered these treatments and collected the same data as the massage therapists. On-site study teams were not employed by the study and the study was not sufficiently budgeted to buy out a significant portion of on-site staff time, an oversight that we have addressed in subsequent grant proposals.

Training teams at a distance

The original six on-site study teams were trained via two group training sessions held in Colorado at the beginning of the study. As additional sites joined over time, on-site trainings were conducted. The day-long training sessions addressed: 1) study overview; 2) the patient with advanced cancer; 3) orientation to the training manual and study materials; 4) study protocols specific to the functions of each study team member; 5) supervised practice sessions for administering treatments, consenting and collecting data; and 6) final review summary and questions. A comprehensive training manual and video demonstrating the massage therapy and simple touch protocols were developed to facilitate and reinforce training of the on-site study teams. Study manual development took several months of a full time research assistant's time, in collaboration with the principal investigators and co-investigators. The training video was developed by research team members in conjunction with the educational media department at the University of Colorado Denver, a resource that was available without cost to investigators as faculty members. Manual and training video printing and distribution costs were substantial, requiring rebudgeting of grant funds.

We were confronted by several problems related to study team training. When there was a lag time of more than a few weeks between team training and beginning study enrollment, teams needed substantial support to feel comfortable and competent with the study procedures, requiring review and remediation. We encountered significant and unexpected on-site team member turnover with rates ranging from 0-240% among study sites (average rate = 68%). Turnover was most commonly due to staff leaving the organization, personal or family illness and competing commitments. Back-up massage therapists and simple touch providers were included as part of the original on-site study team, but we did not plan for routine back-ups for on-site study coordinators or data collectors. When these essential team members were not available, the study was placed on hold from a few days to several months at that site until they were available again or new team members could be recruited and trained. As the study progressed, some study sites recruited back-up study coordinators and data collectors in order to avoid these study interruptions, an idea that was suggested by one of the study sites and shared via the monthly conference call. A train-the-trainer model was developed so that existing on-site team members trained new members. To maintain consistency and reliability, all new team members had phone conferences with the research team to test their knowledge and competence with the study protocol. Therapy providers often shadowed a more experienced member until they felt comfortable conducting the protocol independently. Toward the end of the study, some sites recruited fill-ins who were not part of the formal on-site study team, but who could either help out on an as-needed basis, or were team members that were shared across local study sites (range of fill-ins per site, 0-16). Expectations and training for these individuals were the same as for regular study team members. For example, in the Denver metropolitan area we shared massage therapists, simple touch providers and data collectors across several study sites. Over the course of the study, a total of 203 on-site study team members were trained. Based on these experiences, we realized the importance of planning and budgeting for training of larger on-site study teams with built-in redundancy and for training of replacements over the course of the study period given the inevitability of on-site staff turnover.

All hospice-based team members faced competing demands. Research was a lower priority than the responsibilities associated with their position at the hospice. This was particularly true for on-site study coordinators, who often held administrative positions. Because the research team was some distance away from most of the sites, we encountered an “out of sight out of mind” phenomenon. In order to counter this, we maintained contact with the on-site study coordinators through in-person site visits, monthly conference calls, and enrollment progress reports.

Ongoing quality assurance and communication

Assuring quality and maintaining good communication was essential to successful study completion. This was accomplished through building relationships between the on-site team and the research team, an approach that has been previously noted to be effective.^11,17 One research team member was the designated liaison for each site, assuming responsibility for regular contact with the on-site study coordinator and conducting the site visits. This liaison thus knew each site, its team, and its specific challenges well.

Site visits were conducted at least annually, more frequently if there were known or suspected problems. The purposes of the site visits were to recognize teams, monitor adherence to study procedures, coach and clarify issues, encourage progress towards enrollment goals and collect best practices to be shared among sites. A standard site visit protocol guided visits. Site visitors met with the entire on-site study team, reviewed records and verified adherence to protocols and then provided written post-visit feedback including recommendations for quality improvement. We found it helpful to have an on-site study coordinator from one of the productive sites accompany the research team liaison on visits to sites that were struggling to attain enrollment goals in order to personally share effective approaches and help less productive sites problem solve.

Monthly conference calls with on-site study team members were held throughout the study. Conference calls for the touch therapy providers were held separately from study coordinators and data collectors to ensure that treatment-specific issues could be addressed while maintaining blinding of data collectors to study outcomes. During conference calls, the research team reinforced issues such as best practices in recruitment and careful completion of forms and questionnaires. On-site study team members were encouraged to share their experiences, ideas for improvement, and questions. As study team members gained familiarity and comfort with each other, the calls permitted peer-to-peer sharing and problem solving. On-site study team members often had more creative solutions and had more credibility with their peers at other sites than did the research team. Conference calls thus provided a forum for connecting with others involved with the REST study, to problem-solve and learn from each other.

Successful on-site study teams had three common qualities. First, they had strong organizational commitment to the study, with hospice administration approving and supporting study participation. Second, strong leadership in the role of on-site study coordinator was evident. These individuals represented the study at each site and promoted it throughout the organization. When coordinators were truly invested in the study, they put time and effort into study-related activities. Third, successful sites had effective lines of communication including regular in-person informal contact, regularly scheduled study team meetings and email communication.

Summary

Rigorous research, particularly multi-site randomized clinical trials, conducted in hospice and palliative care organizations, is essential to inform end-of-life clinical care decisions. In conducting a large multi-site RCT of massage therapy in palliative care, the REST researchers faced the methodological challenges of obtaining an adequate sample and training and retaining on-site study teams. Lessons learned as we faced these challenges should inform planning for and conducting future hospice-based investigations.

Acknowledgments

The authors would like to acknowledge and thank the dedication of the hospice on-site study teams, and the generosity of the participants and their families, without whom this study would not have been possible:

• Catholic Hospice, Miami, Florida

• Circle of Life Hospice, Springdale, Arkansas

• Community Hospices of Maryland, DC and Virginia, Silver Spring, Maryland

• Hope Hospice, Ft. Myers, Florida

• Hospice & Palliative Care Center, Winston-Salem, North Carolina

• Hospice & Palliative Care, Cape Cod, Massachusetts

• Hospice at Charlotte, Charlotte, North Carolina

• Hospice of Saint John, Lakewood, Colorado

• Hospice Partners, Hillside, Illinois

• HospiceCare, Berkshires, Massachusetts

• LifePath Hospice, Tampa, Florida

• Palliative Care Center & Hospice, North Shore, Illinois

• Pathways – The Denver Hospice, Denver, Colorado

• Pikes Peak Hospice, Colorado Springs, Colorado

• San Diego Hospice, San Diego, California

• University of Colorado Cancer Center, Denver, Colorado

The study was funded by National Institutes of Health/National Center for Complementary and Alternative Medicine (1R01AT01006-01A2), Mendel/Asarch Lung Cancer Family Foundation Grants Program, Paul Beeson Physician Faculty Scholars in Aging Research Award and Robert Wood Johnson Generalist Physician Faculty Scholars Program (Kutner, PI).

Author Disclosure Statement

No competing financial interests exist.