How redesigning AD clinical trials might increase study partners’ willingness to participate

Abstract

Background:

Timely recruiting and retaining participants into Alzheimer disease (AD) clinical trials is a challenge. We used conjoint analysis to identify how alterations in attributes of clinical trial design improve willingness to participate: risk, home visits, car service, or increased chance of receiving intervention.

Method:

A total of 108 study partners of patients with very mild to severe stage AD rated willingness to allow their relative to participate in eight clinical trials that varied combinations of the four attributes.

Results:

The highest utility was for home visits (0.89) which essentially compensated for the disutility of high risk (−0.85). The combination of home visits and car service was redundant, with almost no increase in utility over home visits alone. Seventeen percent were willing to participate in a trial with no amenities; the addition of home visits increased predicted willingness to participate to 27%; low risk, home visits, and higher chance of active treatment increased predicted willingness to 60%. The value of reducing the hassles of travel correlated well with measures of AD severity (activities of daily living r = 0.41, p < 0.001; basic activities of daily living r = 0.38, p < 0.001; Neuropsychiatric Inventory severity p = 0.24, p = 0.01; Neuropsychiatric Inventory distress r = 0.23, p < 0.02). No association was found between degree of study partner burden and willingness to tolerate risk of an intervention.

Conclusion:

Clinical trials that reduce travel inconvenience may offset the disincentive of study features such as the risk of intervention and may also increase willingness to participate. Redesigning trials may also help recruit patients with more severe Alzheimer disease. Shorter recruitment periods and increased retention rates may offset costs of these changes.

GLOSSARY

AD

= Alzheimer disease;

BLUP

= best linear unbiased prediction;

RAQ

= Research Attitude Questionnaire.

Society recognizes the urgent need to develop Alzheimer disease (AD) treatments with the clinical trial essential to achieve this goal.¹ AD clinical trials face challenges common to all clinical trials, in particular, the risks and uncertainty of benefit of the intervention.²

There are also unique challenges. An AD clinical trial requires not only the commitment of the person with the disease, but a study partner as well, often referred to as a caregiver or knowledgeable informant. These persons, typically a spouse, partner, or adult child, must accompany the patient to visits, complete assessments, assure that the study drug is taken, and assist in assessing the patient’s condition. This time and effort is one of the main reasons for deciding not to enroll in an AD clinical trial.³

These barriers have substantial costs. Trials might fill slowly or not at all, which delays the time to complete the study and increases the costs of the research. The hassles of travel and time can affect the diversity of those who do enroll in the trial. Persons with fewer resources or greater functional impairment may enroll at lower rates or, if they do, may be more likely to drop out. The net result is that progress slows and the subjects may not represent the population of persons who would actually use the drug under study.

Fortunately, many of the barriers to enrollment are addressable. The risks of the intervention, the chance of getting a placebo, and the location of the study visits are all subject to the investigator team’s choices. For example, a clinical trial can involve a low risk drug, with a greater than 50% chance of randomization to the drug, and selected visits could be done in subjects’ homes. But as attractive as these options are, researchers do not know their potential value to study partners. Specifically, which features of a redesigned AD clinical trial are most likely to increase study partners’ willingness to participate?

The purpose of this study was to answer this question by assessing the value of a set of potential modifications of an AD clinical trial. In particular: Is a clinical trial that addresses the hassles of time and travel relatively more attractive to study partners who are caring for persons with more severe functional and behavioral problems? Are more burdened study partners more willing to tolerate a risky intervention? This question addresses the common, but largely untested, concern that burdened study partners are more risk tolerant. Finally, we sought to examine the relative cost of these trials compared to a trial without any amenities.

To answer these questions, we used a method for measuring preferences called conjoint analysis.^4–6 The conjoint method asks study partners of persons with AD to rate their willingness to participate in AD clinical trials that mix different features of a clinical trial. We used these ratings to calculate the value, or utility, for each of the specific features for each individual respondent. These utilities can then predict the value of any combination of the attributes. Conjoint analysis is widely used to optimize new product design in business,⁷ and is seeing increased usage in healthcare settings.^8,9

METHODS

Subjects and eligibility criteria.

Eligible participants were study partners of patients with National Institute of Neurological and Communicative Disorders and Stroke–Alzheimer’s Disease and Related Disorders Association criteria¹⁰ probable AD who were enrolled in the PENN Memory Center cohort and self-identified as the person who 1) would accompany a relative on visits for a trial and complete study forms, 2) lived within a 1.5 hour driving distance from the PENN Memory Center, 3) served as a knowledgeable informant for healthcare professionals, and 4) assisted the patient in making decisions. Patients had probable AD, did not reside in a nursing home, and were independent in ambulation and feeding.

Data gathering.

A research assistant conducted a face-to-face interview in the participant’s home or another convenient location. Subjects reviewed a two-page description of a clinical trial testing a hypothetical drug “Alzprotex.” Next, they rated their willingness to have their relative enroll in this trial, and then reviewed descriptions of eight alternative trial designs that manipulated the risk of the intervention, probability of placebo randomization, and location of and transportation to the study visits. For each trial, participants rated the likelihood of allowing their relative to participate.

The description of the Alzprotex trial, adapted from a clinical trial that was recruiting at the PENN Memory Center, included the study’s purpose, sponsor, the possible benefits of the study to both the participant and society, and the study details: a 21-month-long randomized and placebo- controlled clinical trial with 50–50 probability of drug vs placebo, all 10 study visits at the PENN Memory Center, and a 2% risk of cardiac damage.

The research assistant used the understanding subscale of the MacArthur Competency Assessment Tool for Clinical Research¹¹ to verify the participant understood the trial’s features.

Participants were then shown a two-page description of four study design features, each of which had two potential levels:

1. Location of study visits: a) initial and final visits at the PENN Memory Center with eight intermediate visits at the patient’s home, or b) all visits at PENN Memory Center.

2. Transportation to study visits: a) family member responsible, or b) an optional car service transports the patient and family member.

3. Chance of receiving Alzprotex: a) 50%–50% randomization, or b) two chances in three. The percentages were also shown visually as two pie charts. This change in chances doubles the odds of active treatment (2:1 vs 1:1).

4. Potential risks of Alzprotex: a) minimal risk level (insomnia [11%], anxiety [7%], sleepiness [5%], indigestion [6%], nausea [4%], bruising [5%], diarrhea [7%], and headache [7%]), or b) minimal risk level plus 2% risk of heart inflammation.

We used the “full profile” conjoint method which presents scenarios containing one level for each factor. The design was a fractional factorial design with four binary attributes in eight scenarios generated using SAS 9.1,¹² a design sufficient to measure both the main effects and the interaction between study location and transportation. A full factorial design would be 16 scenarios, which pilot testing revealed was too many cards for many respondents to comfortably rate.

Study partners were shown the eight scenarios on small cards and asked to rate how likely they would be to allow their relative to participate in each trial by placing the cards on a large board with seven spaces labeled from “definitely not participate” to “definitely participate.” After rating the eight scenarios, they ranked the eight scenarios from the most to least preferred.

Participant characteristics.

Participants completed the following measures.

Study partner and patient demographics.

Age, years of education, race, gender, financial burden.

Travel in number of minutes to the PENN Memory Center.

Participant estimates of the time it would take them to travel with their relative to the PENN Memory Center, and the calculated travel time using Mapquest.¹³

Patient dementia severity.

Basic activities of daily living,¹⁴ instrumental activities of daily living,¹⁵ brief form of the Neuropsychiatric Inventory Severity and Distress Subscales.¹⁶

Study partner health and well-being.

Overall health rating (poor, fair, good, very good, excellent), and short screen for caregiver burden.¹⁷

Study partner attitudes about research.

Research Attitude Questionnaire (RAQ), an 11-item scale that asks respondents a series of question about their support for and value of research along a five-point Likert scale from 1 = strongly disagree to 5 = strongly agree (Cronbach alpha = 0.75).¹⁸

Data analyses.

To compute the utilities for each respondent, we used a random effects linear model with empirical Bayes-based best linear unbiased prediction (BLUP). BLUP uses both the population averaged mean response and the respondent’s deviation from it to create a weighted average with the weights dependent on the relative variability of the two estimates and number of observations per subject and across subjects. In short, the effects show shrinkage toward the population mean when there is relatively high variability in the subject-specific means.^19,20 This type of method is now preferred to the traditional method of treating each respondent’s data as a separate regression.²¹ All analysis was conducted in SAS 9.1.²²

For each amenity, we calculated the average additional cost per subject. For the home visit amenity, we calculated the cost estimates from MapQuest mileage data for staff travel reimbursement and opportunity cost of additional staff time for transportation to a subject’s home based on the hourly salaries of relevant staff. Cost estimates for chance of receiving Alzprotex were calculated based on cost of enrolling additional subjects. Cost estimates for car services were averaged from quotes given from car services in the Philadelphia area.

To calculate how various combinations of the clinical trial attributes predicted study partners’ willingness to participate, we summed the per person utilities to compute per person willingness to participate. This generated a score between 1 and 7. We assigned a score of 5 or above as “willing to participate,” as our willingness to participate scale labeled 5, 6, and 7 as “possibly,” “probably,” and “definitely willing to participate.”

Human subjects protections.

Participants provided verbal informed consent to participate in this University of Pennsylvania Institutional Review Board approved project and received a $20 gift certificate for their time and effort.

RESULTS

Participant characteristics.

Table 1 shows the characteristics of the 108 study partners who participated (response rate = 87%) and of the patients for whom they care. The majority of study partners were female (69%), Caucasian (77%), and non-Hispanic (96%) with at least a few years of post-high school education (15.7 ± 2.9 years). Just over half (52%) were spouses or partners and the rest were mainly adult children. Table 2 shows that the patients had, on average, substantial impairments in instrumental and basic activities of daily living and in the frequency and severity of neuropsychiatric symptoms.

Table 1 Participant characteristics (n = 108)

Table 2 Patient dementia severity

Overall utility of each clinical trial attribute.

The figure shows the mean and 95% CI for the utility of each clinical trial amenity. The CIs show that all utility estimates differed from 0 (p < 0.0001). As expected, the higher risk feature had a negative utility, meaning it decreased willingness to participate by, on average, 0.85 points (CI −1.10 to −0.60) along the seven-point scale. In contrast, all of the other attributes had, as expected, a positive utility, meaning each generally increased study partners’ willingness to participate.

Figure Factor utilities

A positive value means the factor is associated with increased willingness to participate and a negative value means the factor is associated with a decreased willingness to participate.

The key findings are that a home visit increased willingness to participate by nearly one point 0.89 (CI 0.64 to 1.14) and that a home visit was, on average, more valued than a car service, whose mean utility was 0.58 (CI 0.35 to 0.82), or a greater chance at study drug, whose mean utility was 0.52 (CI 0.35 to 0.69). Finally, the combination of home visits and a car service for the two visits that were still required at PENN Memory Center for the initial and final study visits (0.94 [CI 0.70 to 1.18]) did not substantially differ from home visits alone.

Association between the utility of each feature and patient and study partner characteristics.

A full-profile conjoint analysis generates utilities on a per-person basis for each of the features studied. This permits analyses to identify market segments, that is, groups of persons who find an attribute especially attractive or unattractive.

Table 3 shows that the value of features that addressed travel issues was associated with greater dementia severity as measured by instrumental (r = 0.41, p < 0.001) and basic (r = 0.38, p < 0.001) activities of daily living. A smaller association was seen with both the presence (r = 0.24, p = 0.01) and severity (r = 0.23, p = 0.02) of patient behavioral problems. In addition, the value of reducing travel hassles was associated with the study partner’s estimate of the time to travel to the PENN Memory Center (r = 0.28, p = 0.003). Notably, no association was seen with the MapQuest calculated travel time (r = 0.14, p = 0.14). Finally, the value of reducing travel was negatively associated with lower scores in the research attitudes questionnaire (r = −0.21, p = 0.03).

Table 3 Association between patient and study partner characteristics and trial modifications

Patient dementia severity, study partner distress, and estimated travel time were not associated with the study partners’ utility for the risk of the intervention. In particular, we did not find an association between either the degree of study partners’ subjective burden or the Neuropsychiatric Inventory distress scale and the utility of risk. We did find an association between the calculated travel time to the PENN Memory Center and the utility of risk (r = 0.27, p = 0.005). We found no associations between patient dementia severity, study partner distress, and estimated or calculated travel time and the utility of reducing the chance of placebo assignment.

Potential net impact of the alternative clinical trial designs.

To further examine the value of these alternative ways to design an AD clinical trial, we examined how various combinations of features predicted that a study partner would allow the relative to participate, with “participate” operationalized as a score of five or higher on the predicted willingness to participate scale. We generated scores by adding up the utilities for each person that corresponds to the specific scenario.

Table 4 shows that 17% of the 108 study partners had a predicted willingness to participate in the usual clinical trial, that is, a trial that involved an intervention with a risk of heart inflammation and all visits at the PENN Memory Center with the study partner responsible for travel. The addition of home visits and a greater chance at the study drug increased this proportion to 42%. The most attractive design was a trial that had low risk, home visits, and a 67% chance of randomization to study drug, where nearly two-thirds (60%) were willing to participate.

Table 4 Summary of the predicted willingness to participate in selected clinical trial designs and the additional cost per subject for each design

Table 4 displays estimated additional cost per subject for each trial feature alteration. These cost data are site-specific estimates. We present them to provide context of the cost per subject over the course of the entire trial of the various alterations in a clinical trial design.

DISCUSSION

This study suggests that AD clinical trial designs that address barriers to timely and efficiently completing a clinical trial may increase a study partner’s willingness to participate and, in particular, may increase the willingness of study partners of patients with more severe AD.

First, a home visit appears the most valued. In this study, we assessed the value of a trial that required in-clinic visits for initial and final study assessments, but home visits for the other assessments. Notably, the net value of home visits and a car service was not substantially greater than the value of home visits alone, suggesting that the potentially more expensive car service is unnecessary.

Second, the value of design modifications that reduced travel was associated with greater dementia severity as measured by both instrumental and basic activities of daily living. Additionally, the value of travel was associated with greater patient behavioral problems. Together, these results suggest that addressing the hassles of travel will recruit patients with more dementia severity diversity. It is possible that home visits and a car service might also reduce dropout rates. Dropouts increase study costs by lowering statistical power and creating confounding if the dropouts are not at random.

Third, we did not find an association between study partner value of the risk of the intervention and study partner or patient characteristics. Specifically, we were not able to show that study partner burden was associated with the utility of risk. This result is reassuring as it suggests that the oft repeated concern that burdened study partners are more risk tolerant may, in fact, not be the case.

Fourth, an association existed between the value of reducing travel and lower scores on the RAQ. This 11-item questionnaire measures the degree that a person has favorable attitudes about research with questions such as “I have a positive view about medical research in general” and “Medical researchers are mainly motivated by personal gain.” The association between lower scores on this scale and a greater value for reducing the hassles of travel suggests that a trial that reduces the necessity of travel to the study site will attract a more diverse cohort of participants in terms of their views about research. Whether their subsequent experience in research would in turn change their attitudes about research is an important question that warrants further study.

Limitations include that the study partners were recruited from a single study site and that site was an AD Center. We chose this design because clinical trials are typically conducted at such Centers and we wanted to study persons who were familiar with a particular center, thereby increasing the credibility and transparency of our survey. Further research is needed to examine how study partners who do not attend an AD Center value the trial design modifications we studied. Second, the sample’s race and ethnicity were not diverse. Third, our study is based on choice for scenarios, not actual choice in a clinical setting. Thus, while the results suggest which factors are important, they do not tell us how well that importance will translate into actual participation. Likewise, our design shows respondents a set of alternative scenarios, which they rate knowing that there are better or worse scenarios. This might be analogous to a situation in which respondents choose among alternative clinical trials from a set. The situation when confronted with a single clinical trial by a trusted and personable researcher may be somewhat different and result in higher willingness to participate than in the situation we present.

Would any gains in the recruitment be worth the additional cost? Study costs will change based on methodology used and the number and type of staff needed at each visit. Although costs vary by study and are approximations, we estimate that adding eight home visits increases the base $5,000 per subject cost by about 12%. Adding the 67% chance of active treatment also increases costs about 12% due to the slightly larger sample size required to maintain power. A field trial is necessary to show whether the lower costs from less time to recruit subjects and less dropouts would offset the additional costs of amenities.

ACKNOWLEDGMENT

The authors thank the family members of patients who participated in this study. They also thank Emily Murphy and Samantha Schwartz for their work in data gathering and analyses.

Address correspondence and reprint requests to Dr. Jason Karlawish, 3615 Chestnut Street, Philadelphia, PA 19104 Jason.karlawish@uphs.upenn.edu

Supported by the Marian S. Ware Alzheimer Program, NIA grant P30-AG-10124, and a Greenwall Faculty Scholar Award to Dr. Karlawish.

Disclosure: The authors report no disclosures.

Received June 24, 2008. Accepted in final form August 26, 2008.