Abstract
Background:
Missing data are a notable problem in Alzheimer’s disease clinical trials. One cause of missing data is participant dropout. The Research Attitudes Questionnaire is a 7-item instrument that measures an individual’s attitudes toward biomedical research, with higher scores indicating more favorable attitudes. The objective of this study was to describe the performance of the Research Attitudes Questionnaire over time and to examine whether Research Attitudes Questionnaire scores predict study dropout and other participant behaviors that affect trial integrity.
Methods:
The Research Attitudes Questionnaire was collected at baseline and weeks 26 and 52 from each member of 119 participant/study partner dyads enrolled in a Phase 2, randomized, double-blind, placebo-controlled mild-to-moderate Alzheimer’s disease clinical trial. Within-subject longitudinal analyses examined change in Research Attitudes Questionnaire scores over time in each population. Logistic regression analyses that controlled for trial arm and clustering in trial sites were used to assess whether baseline Research Attitudes Questionnaire scores predicted trial completion, study medication compliance, and enrollment in optional substudies.
Results:
Participants and study partners endorsed statistically similar ratings on the Research Attitudes Questionnaire that were stable over time. Participants with baseline Research Attitudes Questionnaire scores above 28.5 were 4.7 (95%CI 1.01 to 21.95) times as likely to complete the trial compared to those with lower scores. Applying the same cutoff, baseline study partner Research Attitudes Questionnaire scores were similarly able to predict study completion (Odds Ratio [OR]=4.2, 95% CI 1.71 to 10.32). Using a score cutoff of 27.5, higher participant Research Attitudes Questionnaire scores predicted study medication compliance (OR=5.85, 95% CI 1.34 to 25.54). No relationship was observed between Research Attitudes Questionnaire score and participation in optional substudies.
Conclusions:
This brief instrument that measures research attitudes may identify participants at risk for behaviors that cause missing data.
Keywords: Dropout, retention, missingness
Background
Clinical trials provide essential evidence to advance clinical practice. Missing data are a key barrier to clinical trial success and can lead to serious methodological problems.1, 2 Participant dropout is a major source of missing data and a particular problem in Alzheimer’s disease (AD) clinical trials.3 AD is an age-related disease. Trial samples are older and suffer from comorbidities that can pose challenges to participation and increase risk for adverse events during trials.4 AD trials are often 18-months or longer duration so as to measure disease progression and detect treatment effects.5-8 AD trial sample sizes typically anticipate an attrition rate of approximately 10% per year. Unfortunately, dropout frequently exceeds anticipated rates.9 When it does, it threatens the scientific validity of the trial. Trials with greater than expected dropout may have inadequate statistical power. Because attrition is rarely completely at random, the group that remains often differs from the sample originally enrolled,10, 11 resulting in potential bias and limiting generalizability of findings.1, 12 Studies with unanticipated dropout may also place participants at risk without the benefit of advancing scientific knowledge.13, 14 Even in the setting of expected rates of dropout, increased retention could increase power to detect smaller than expected treatment effects. Minimizing dropout is therefore vital for the scientific and ethical integrity of clinical trials.
The optimal approach to address missing data in clinical trials is to prevent its occurrence.15 One way to do this is to identify participants at risk for dropout.16 Trial dropout has been shown to vary by geographic region and disease severity in AD research.17, 18 Social and cultural factors are also known to correlate with participant dropout, and are therefore candidates to be used in predictive measures. For example, informant type has been shown to correlate with trial dropout but because most participant-study partner dyads are spousal dyads, targeting spouses is of limited usefulness.18 Standardized measures that are valid and reliable in predicting dropout could therefore be useful for identifying individuals who are at risk for discontinuing participation in research and may require intervention.
The Research Attitudes Questionnaire (RAQ) was developed with the goal of measuring social and cultural factors that may influence research participation decisions, including enrollment and continued participation (dropout).19 The RAQ is a brief, validated, 7-question instrument designed to gauge a person’s attitudes toward biomedical research.19 It assesses factors known to associate with research participation such as altruism and personal responsibility.9
In three studies to date, the RAQ has been shown to predict willingness to participate in research.20-22 For example, in a study of AD caregivers, higher RAQ scores were associated with greater willingness to enroll their loved one in a clinical trial (OR=1.39), a stronger predictor than patients’ symptom severity or even caregiver burden.21 To further validate whether RAQ scores predict trial participant behaviors, we assessed whether RAQ scores were associated with other participant behaviors such as study treatment compliance and drop out. We hypothesized that lower RAQ scores would predict a higher likelihood of dropout.
Methods
Participants and data source
We performed a secondary analysis of data collected in the Resveratrol clinical trial (NCT01504854), a randomized, double-blind, placebo-controlled study.23, 24 The Resveratrol trial was a multicenter, phase 2 trial, conducted June 2012–March 2014. Participants were recruited from 26 US academic clinics affiliated with the Alzheimer’s Disease Cooperative Study. The enrollment target was 120 (60 per group). Actual enrollment was 119. All participants were randomized to placebo or resveratrol 500 mg orally once daily (with dose escalation by 500-mg increments every 13 weeks, ending with 1,000 mg twice daily). Brain magnetic resonance imaging and cerebrospinal fluid collection were performed at baseline and after completion of treatment. A subgroup of 15 participants enrolled in a randomized study for 24-hour pharmacokinetics at selected sites. For these individuals, blood samples were collected at baseline and weeks 13, 26, 39, and 52. Participants signed informed consent that was approved by a local institutional review board when enrolling in this trial. This consent included all protocol-specified collection of data, including the longitudinal collection of the RAQ. The trial protocol, including inclusion and exclusion criteria, has been described previously.23 Participants were required to meet diagnostic criteria for probable AD based on National Institute of Neurological and Communicative Disorders and Stroke/Alzheimer’s Disease and Related Disorders Association criteria, be at least 49 years of age, and score 14-26 on the Mini-Mental State Exam. Participants were required to enroll with a study partner (informant), who was able to accompany them to all study visits and who had direct contact with the patient for a minimum of two or more days per week (equivalent to 10 hours per week). The resveratrol trial did not offer open label extension to participants. Data from patients with mild to moderate AD (n=119) and their study partners (informants, n=119) were collected over one year, including the longitudinal collection of the RAQ.
Measures
The RAQ is a brief 7-item self-report assessment that measures attitudes towards biomedical research (Appendix 3 in the online supplemental material).19 Items address views about research in society, altruism, trust in investigators, optimism about research outcomes, and safety. Responses to each item are recorded on a 5-point scale from (1) “strongly disagree” to (5) “strongly agree.” Thus, the possible range of scores for the RAQ is 7-35, with higher scores indicating more favorable attitudes toward research. The 7-item version was generated based on assessments of internal validity and factor analyses of a larger previous set of items. The 7-item RAQ demonstrated good reliability (Item-total correlation alpha = 0.81) and benefit of including no reverse coded items, simplifying scoring and enhancing usability.19 Prior research has shown higher RAQ scores correlate with greater willingness to participate in research, which suggests content validity for the scale.20-22 To our knowledge, this study represents the first examination of within-subject longitudinal change in the RAQ. The RAQ was completed independently by both participants and their study partners at baseline and study weeks 26 and 52. Study procedures did not allow the scales to be completed together and so the observations can be analyzed as independent.
Baseline characteristics of participants and their study partners were recorded at screening. This included self-report sociodemographic characteristics of sex/gender, race, education level, and clinical laboratory testing for apolipoprotein E genotype and other characteristics. Three outcomes from the Resveratrol trial were included in the current analyses: The AD Assessment Scale - cognitive subscale, which was originally designed as a rating scale to assess the severity of cognitive and noncognitive dysfunction from mild to severe AD, is scored from 0 to 70 by summing the number of errors made on each task so that higher scores indicate worse performance.25 The Alzheimer’s Disease Cooperative Study’s Activities of Daily Living Scale is a 23-item scale consisting of six basic activities of daily living and 17 instrumental activities of daily living. The total score ranges from 0-78, with a lower score indicating greater severity.26 The Neuropsychiatric Inventory measures 12 sub-domains of dementia-related behavioral functioning: delusions, hallucinations, agitation/aggression, dysphoria, anxiety, euphoria, apathy, disinhibition, irritability/lability, aberrant motor activity, night-time behavioral disturbances, and appetite and eating abnormalities.27
Study partners were asked an additional question about their relationship to the participant, characterized here as spouse or other.
Study endpoints
To assess how RAQ scores associated with trial outcomes, we examined three participant behaviors: (1) completion of the trial, (2) adherence to study medications, and (3) enrollment in optional substudies. Trial non-completion or dropout was defined as the patient discontinuing the study before 52-weeks for a reason other than death. Study medication adherence was assessed via formal pill counts at each study visit. We defined compliance based on the study protocol; participants with >80% but <120% expected pill counts were deemed compliant. Post randomization, patients were encouraged to enroll in optional pharmacokinetic substudies to assess time course of absorption, distribution, and other metabolic qualities of the study drug. These substudies recruited at 9 of the 21 study sites. Patients were offered the opportunity to enroll in substudies at baseline and weeks 13, 26, 39, and 52.23
Statistical analysis
We used descriptive statistics to characterize the sample. To assess variability in RAQ scores of participants and their study partners, we used generalized linear models to estimate mean scores at each study collection point (baseline, week 26, week 52). Paired mean differences were calculated to assess change in RAQ scores from baseline to week 26 and to week 52. We used difference-in-difference analyses to estimate correspondence of RAQ scores between patients and study partners. To assess how RAQ scores associated with patient and study partner baseline clinical and social characteristics, we performed forward stepwise regression to build full multivariable models. Alpha-to-keep was 0.20. Tobit models were used in order to minimize ceiling effects by using a 2-part model right censored at the upper value of the range of scores.28 Though models of baseline characteristics infrequently failed to meet assumptions of normality, we used non-parametric models to back up parametric models. In all cases, conclusions related to statistical significance from the models that are reported are the same as those from the nonparametric models.
We calculated a Youden index, which is a summary statistic estimating the optimal cut point of the Receiver Operating Characteristic curve.29, 30 The purpose was to derive the optimal cut point of patient and study partner baseline RAQ scores for predicting between those who did and did not complete the trial, adhere to study medications, or enroll in substudies. Logistic regression was used to estimate the odds ratios of each study outcome based on the calculated RAQ cut points for each trial outcome. These analyses excluded patient-study partner dyads where patients were known to have died (n=3).
Four patients had missing responses on individual RAQ items. Patient-level mean scores were used to substitute missing responses, given ≤3 of the 7 responses were missing for each case. In one instance where a RAQ score was missing for a study partner, the dyad was excluded from the analysis. In longitudinal analyses where RAQ scores were missing for an active dyad, a change of zero was assumed. Bias corrected and accelerated 95% confidence intervals (95% CI) were estimated based on 1,000 bootstrap samples. 95% Cis that do not contain zero were considered statistically significant at p<0.05. All analyses statistically controlled for trial treatment group assignment and for participant clustering within study sites.
Results
Participant characteristics
Study participants on average were 71.3 years old (Table 1). Most were female (57.1%) and self-identified as white (90.8%). Just over half (52.1%) had completed a 4-year college degree. More than three-quarters (78.2%) of participants enrolled with a spousal study partner. An even larger majority co-resided with their study partner (86.6%). On average, participants spent 130 hours per week with their study partners.
Table 1.
Baseline Characteristics of Study Participants and Partners, Resveratrol Trial (N=119)
Characteristic | Study Patients |
---|---|
Age, mean (SD) | 71.3 (8.1) |
65+ years old, n (%) | 93 (78.2) |
Females, n (%) | 68 (57.1) |
Race / Ethnicity, n (%) | |
White, Non-Latino | 108 (90.8) |
African American, Non-Latino | 7 (5.9) |
Othera | 4 (3.2) |
Hispanic or Latino Ethnicity, n (%)) | 1 (0.8) |
Education, n (%) | |
High School/GED or Less | 31 (26.1) |
Some College or 2-year Degreeb | 26 (21.0) |
4-year College Degree or beyond | 62 (52.1) |
ADCS - Activities of Daily Living Inventory, median (IQR) | 65 (16) |
ADAS Cognitive Behavior Score, mean (SD) | 24.5 (9.5) |
NPI,c median (IQR) | 8 (12) |
APOE genotype, n (%) | |
Noncarrier | 38 (31.9) |
Heterozygote | 59 (49.6) |
Homozygote | 22 (18.5) |
Study Partners | |
Female,d n (%) | 63 (52.9) |
Spousal relationship to study patient, n (%) | 93 (78.2) |
Resides with study patient, e n (%) | 103 (86.6) |
Hours per week spent with patient, median (IQR) | 130 (98) |
Note. Column percentages may not total 100 due to rounding. SD = standard deviation. ADCS = Alzheimer’s Disease Cooperative Study, ADAS = Alzheimer’s Disease Assessment Scale, NPI = Neuropsychiatric Inventory, IQR = interquartile range.
Category includes those who identified as Asian and American Indian or Alaskan Native.
4-year college, master’s, doctorate, or professional degrees.
Includes one person whose total score was missing one domain score.
Sex/gender is unknown for 5 study partners.
Includes one person who lives half-time with participant.
Longitudinal analyses of RAQ scores
In fixed effect analyses that adjusted for trial arm, participants reported a mean RAQ score at baseline of 31.2 (95%CI 28.5 to 33.3). Scores at weeks 26 (Mean, 33.5, 95%CI 31.3 to 35.8) and 52 (Mean, 32.3, 95%CI 30.0 to 34.8) did not differ from baseline (Table 2). Mean study partner RAQ scores demonstrated similar consistency over time (both p>0.05; Table 2). At baseline, participants and study partners endorsed similar ratings on the RAQ (Figure 1). On average, the discrepancy between participants’ and their study partners’ RAQ scores were negligible and did not differ statistically over time (both p>0.05).
Table 2.
Means and Paired Mean Differences in RAQ Scores Over Time
Baseline (N=119) Mean (95%CI) |
Week 26 (N=111) Mean (95%CI) |
Week 52 (N=95) Mean (95%CI) |
Wk26 – BL (N=119) PMD (95%CI) |
Wk52 – BL (N=119) PMD (95%CI) |
|
---|---|---|---|---|---|
Participant RAQ | 30.6 (29.7 to 31.4) | 30.9 (30.0 to 31.5) | 30.2 (29.2 to 31.0) | 0.1 (−0.07 to 1.1) | −0.4 (−1.4 to 0.4) |
Adjusted Participant RAQ | 31.2 (28.5 to 33.3) | 33.5 (31.3 to 35.8) | 32.3 (30.0 to 34.8) | 0.1 (−1.0 to 1.2) | −0.4 (−1.5 to 0.7) |
Study Partner RAQ | 30.4 (29.4 to 31.1) | 31.1 (30.4 to 31.7) | 31.2 (30.2 to 31.8) | 0.6 (−0.1 to 1.6) | −0.2 (−0.9 to 0.6) |
Adjusted Study Partner RAQ | 29.9 (26.6 to 32.2) | 30.9 (29.2 to 32.7) | 32.6 (30.5 to 34.9) | 0.6 (−0.4 to 1.7) | 0.7 (−0.4 to 1.8) |
PMD (95%CI) | PMD (95%CI) | PMD (95%CI) | DID (95%CI) | DID (95%CI) | |
Study Participant – Study Partner RAQ | 0.3 (−0.6 to 1.2) | −0.3 (−1.2 to 0.6) | −1.0 (−2.2 to 0.2) | 0.5 (−0.3 to 1.6) | 0.6 (−0.4 to 1.5) |
Adjusted Study | 1.3 (−1.8 to 4.4) | 2.6 (0.2 to 5.3) | −0.3 (−3.2 to 2.4) | −0.5 (−1.8 to 0.7) | −1.3 (−2.7 to 0.1) |
Participant – Study Partner RAQ |
Note. Fixed effects analysis. Analysis assumes zero change for individuals with missing data follow up data. Adjusted analyses statistically control for study arm assignment. DID = Difference-in-difference; PMD = Paired Mean Difference; 95%CI= 95% bias corrected and accelerated confidence interval based on 1000 bootstrap samples.
Figure 1.
Scatterplot of participant and study partner baseline RAQ scores (N=119)
Baseline correlates of RAQ scores
In trivariate analyses that adjusted for trial arm, study partners of participants with higher Neuropsychiatric Inventory scores (Mean difference, 3.81, 95%CI 1.77 to 6.09) and Alzheimer's Disease Assessment Scale - Cognitive Subscale scores (Mean difference, 1.66, 95%CI 0.10 to 3.52) as well as study partners who resided with participants (Mean difference, 2.05, 95%CI 0.27 to 4.45) had higher RAQ scores at baseline (Table 3). In a full model that adjusted for study partner characteristics of residing with participant, hours spent with participant, being a spouse of participant, and the participant’s degree of cognitive (Alzheimer's Disease Assessment Scale - Cognitive Subscale) and functional (Alzheimer’s Disease Cooperative Study’s activities of daily living) impairment, only participant Neuropsychiatric Inventory scores remained significantly associated with higher study partner RAQ scores at baseline (Mean difference, 3.71, 95%CI 1.17 to 6.26).
Table 3.
Participant and Study Partner Correlates of Baseline RAQ Scores
Trivariate Model (N=119) Mean Difference (95%CI) |
Full Multivariable Model (N=119) Mean Difference (95%CI) |
|
---|---|---|
Study Participant | ||
Age, years | −0.07 (−0.31 to 0.04) | |
Female vs. Male | 0.37 (−1.41 to 2.59) | |
Education (years) | 0.11 (−0.48 to 0.64) | |
ADCS ADL scorea | 1.71 (−0.33 to 3.66) | 1.66 (−0.75 to 3.81) |
ADAS-cog scorea,b | 0.95 (−1.27 to 2.61) | |
NPI scorea,b | 1.64 (−0.65 to 4.20) | 1.51 (−1.03 to 4.37) |
APOE genotypec | ||
Heterozygote | −1.66 (−3.86 to 0.38) | −1.97 (−4.33 to 0.03) |
Homozygote | 0.65 (−1.77 to 2.57) | |
Study Partner | ||
Resides with participantd | 2.05* (0.27 to 4.45) | 0.91 (−2.77 to 3.65) |
Hours per week with participant | 0.01 (−0.0 to 0.03) | 0.02 (−0.01 to 0.05) |
Female vs malee | −0.0 (−0.05 to 0.06) | |
Spouse vs non-spouse | 1.33 (−0.60 to 3.41) | −0.34 (−3.40 to 2.57) |
Participant characteristics | ||
Age, years | −0.03 (−0.24 to 0.15) | |
Female vs. male | −0.29 (−1.70 to 1.89) | |
ADCS ADL scorea | 1.68 (−0.14 to 3.25) | 0.79 (−1.07 to 2.98) |
ADAS-cog scorea,b | 1.66* (0.10 to 3.52) | 0.82 (−1.05 to 2.69) |
NPI scorea,b | 3.81*** (1.77 to 6.09) | 3.71 (1.17 to 6.26)** |
Note. Adjusted analyses statistically control for study arm allocation and participant clustering within study site. Alpha to carry forward from trivariate analyses to full multivariable model was P<0.20. APOE = Apolipoprotein E; ADAS = Alzheimer's Disease Assessment Scale; ADCS = Alzheimer's Disease Cooperative Study; ADL = Activities of Daily Living.
Dichotomized at median value
Predicting lower value.
Noncarrier is reference category.
Does not reside with study patient is reference category. Resides with study patient includes one person who lives half-time with patient.
Sex/gender is unknown for 5 study partners.
≤0.05
≤0.01
≤0.001
Associations of RAQ scores with trial outcomes
In total, 15 of 119 participants failed to complete the trial, including three who died. In analyses that adjusted for trial arm and participant clustering within study site (and removed those who died), participants with a RAQ score of 28.5 or higher were about 4.7 (95%CI 1.01 to 21.95) times as likely to complete the trial, compared to those with lower scores (Table 4). In adjusted analyses, participants with study partners with a RAQ score of 28.5 or higher were about 4.2 (95%CI 1.71 to 10.32) times as likely to complete the trial, compared to participants with partners with lower scores.
Table 4.
Adjusted Odds Ratios (OR) of Baseline RAQ Scores Predicting Study Outcomes
Baseline OR (95% CI) |
Week 26 OR (95% CI) |
Week 52 OR (95% CI) |
|
---|---|---|---|
Study Patient | |||
Compliant based on pill check | NA | 2.74 (0.54 to 16.53) | 5.85* (1.34 to 25.54) |
Study completion vs dropout | NA | ICS | 4.71* (1.01 to 21.95) |
Enrollment in PK studiesb | 2.74 (0.88 to 8.55) | 2.99 (0.60 to 14.78) | 4.24 (0.86 to 1.76) |
Study Partner | |||
Compliant based on pill check | ICS | ICS | 1.98 (0.23 to 17.01) |
Study completion vs dropout | ICS | ICS | 4.20** (1.71 to 10.32) |
Enrollment in PK studiesa | ICS | 1.83 (0.17 to 19.38) | 1.65 (0.54 to 5.07) |
Study Patient – Study Partner (PMD) | |||
Compliant based on pill check | ICS | 5.05 (0.58 to 43.81) | 4.88 (0.43 to 55.29) |
Study Completion vs dropout | ICS | ICS | 1.48 (0.21 to 10.65) |
Enrollment in PK studiesa | 3.55 (0.32 to 39.27) | 2.10 (0.18 to 23.87) | 3.26 (0.30 to 35.11) |
Note. Trivariate analyses statistically control for study arm allocation and participant clustering within study site. Analyses exclude patients known to have died (n=3).
ICS = insufficient cell size; PK = pharmacokinetic; NA = No empirical cut point identified. PMD = Paired Mean Difference.
Subsample of participants at study sites that recruited at least one participant into PK studies (N=79). See Appendix 2.
≤0.05
≤0.01
≤0.001
In analyses that adjusted for trial arm and participant clustering within study site, participants with a RAQ score of 27.5 or higher were nearly six times as likely (OR, 5.85, 95%CI 1.34 to 25.54) to be adherent to study medications. We found no significant relationships between participation in optional substudies and either participant or study partner RAQ scores. No statistically significant associations were identified between discrepancies in RAQ scores in participant/partner dyads and trial outcomes (Table 4).
Discussion
We found that among participants in a prospective randomized controlled clinical trial for mild-to-moderate AD, research attitudes measured with the RAQ were stable over time, fairly concordant within trial participant/partner dyads, and associated with important trial outcomes including study completion and compliance with study medication. These results may be important to investigators designing AD (or other therapeutic area) trials. The RAQ can be self-administered and includes only seven single-score items (supplemental material Appendix 3), resulting in minimal burden for participants and for the investigative team.
The longitudinal stability of the RAQ appears robust. While this may suggest that the collection of RAQ scores need not be overly frequent, the current results do not provide information related to the implications of changes in RAQ scores within a study. Similarly, few participant or study partner characteristics were associated with baseline RAQ scores. This is likely due to the largely homogenous sample, which may in part reflect strict trial inclusion/exclusion criteria (e.g., restricting to specific ranges on cognitive tests and excluding individuals with comorbidities). Incorporating the RAQ into larger, less restricted, natural history and/or biomarker studies could provide added information about the scale’s performance and value, as well as potentially critical additional information related to study retention and compliance.31, 32
Baseline RAQ scores appeared related to higher psychiatric comorbidity and worse cognitive performance. The explanation for these observations is unclear. A cognitive bias could lead to differences in reporting on the RAQ. For example, the worse a person’s symptoms the more prone he or she may be to the IKEA effect,33 which is a cognitive bias in which consumers place a disproportionately high value on products they partially created. The research analogy would be more symptomatic participants placing a higher value on drug-discovery research than those with no or milder symptoms. They may have a greater desire to benefit from participation. Another possible explanation is selection bias. Individuals with relatively worse symptoms and lower functioning may need to have stronger beliefs in the value of research to overcome the barriers presented by their symptoms to join a study, compared to those with no or milder symptoms. Alternatively, the selection bias could relate to another confounding factor, such as the study partner. Trial participants with worse symptoms may be more likely to have a caregiver who can more easily step into the role of study partner, as compared to more independently functioning individuals who may not have a study partner as easily identified.
Low RAQ scores at baseline may serve as a flag to monitor and potentially identify and intervene to retain participants to study completion who are at greatest risk for loss to follow-up. Future research will be necessary to confirm the current results and further instruct optimal use of the information provided by the RAQ. For example, it may be that RAQ scores, in combination with other risk factors for trial dropout can be used to instruct increased retention tactics for particular participants.34 The current data to not address the mechanism through which the RAQ identifies differential risk for dropout. Previous studies have found that the safety item (item #5) was most strongly predictive of willingness to participate in a clinical trial among AD caregivers, for example.21 Given the low frequency of dropout, however, this study was not powered to assess associations among the individual scale items. Whether this or other constructs measured by the scale, or whether constructs may be differentially important in unique populations, will require additional research.
We note several limitations of this study. Dropout in the parent trial was low (n=15), potentially limiting the robustness and generalizability of the results to other trials, where dropout frequently exceeds 20%.9 We cannot rule out that random variation contributed to the study observations. The conceptual consistency of the study’s results with those published previously, however, suggest the findings are promising. Small sample size, nonetheless, limited other analyses, such as examination of potential associations between RAQ scores and participation in optional substudies, for which only a subset of sites in the parent trial participated. Discordance in RAQ scores between members of trial dyads was also low, limiting the opportunity to assess whether such occurrence was associated with negative trial outcomes. Scores on the RAQ were also high, introducing the potential for ceiling effects. In addition, further study and replication of validity and reliability of the RAQ cutoffs derived in this study are needed. The cutoffs in the current study predicted dropout or compliance issues for participants enrolled in this trial. These specific cutoffs, or the overall relationship between RAQ and trial outcomes, may not generalize to other studies or trials that have substantively different demands on enrollment or participation.
Other limitations based on the parent study should be noted and further limit generalizability. The trial recruited predominantly white, well-educated participants and was conducted exclusively in the United States. It also enrolled an even greater than typical proportion of spousal dyads, who may be at lower risk for dropout18 and have higher RAQ scores compared to adult children and other study partners.21
Conclusions
These results represent novel information for AD trialists and suggest that measuring research attitudes may identify participants at risk for dropout. Doing so, if supported by future research, may instruct novel practices to maintain greater proportions of trial cohorts to completion, increasing statistical power and advancing therapeutic research.
Acknowledgements.
We thank the participants and study partners in the Resveratrol clinical trial. The trial was supported by NIA U01 AG010483. JDG is supported by NIA AG016573, AG066519, and NCATS UL1 TR001414. JK is supported by NIA AG010124.
Appendix
Appendix 1:
Author affiliations and contributions
Author | Location | Contribution |
---|---|---|
Shana D. Stites | University of Pennsylvania | Designed the study, performed statistical analysis, drafted and edited the manuscript, approved the final manuscript |
R. Scott Turner | Georgetown University | Designed the study, edited the manuscript, approved the final manuscript |
Jeanine Gill | University of Pennsylvania | Assisted with statistical analysis, edited the manuscript |
Anna Gurian | University of Pennsylvania | Conducted the literature review, edited the manuscript |
Jason Karlawish | University of Pennsylvania | Designed the study, edited the manuscript, approved the final manuscript |
Joshua D. Grill | University of California, Irvine | Designed the study, edited the manuscript, approved the final manuscript |
Appendix 2:
Cut points for baseline RAQ scores for each study outcome and time point
Baseline | Week 26 | Week 52 | |
---|---|---|---|
Study Patient | |||
Compliant based on pill check | ICS | 28.5 | 27.5 |
Study completion vs dropout | ICS | ICS | 25.5 |
Enrollment in PK studiesa | 28.5 | 28.5 | 28.5 |
Study Partner | |||
Compliant based on pill check | ICS | NA | 34.5 |
Study completion vs dropout | ICS | ICS | 28.5 |
Enrollment in PK studiesa | NA | 34.5 | 32.5 |
Study Patient – Study Partner (PMD) |
|||
Compliant based on pill check | ICS | 0.5 | 1.5 |
Study Completion vs dropout | ICS | ICS | −6.5 |
Enrollment in PK studiesa | −1.5 | −1.5 | −1.5 |
Note. ICS = insufficient cell size; PK = pharmacokinetic.
Subsample of participants at study sites that recruited at least one participant into PK studies.
Appendix 3:
RAQ Items
Strongly Disagree |
Disagree | Neutral | Agree | Strongly Agree |
|
---|---|---|---|---|---|
1. I have a positive view about medical research in general. | □ | □ | □ | □ | □ |
2. Medical researchers can be trusted to protect the interests of people who take part in their research studies. | □ | □ | □ | □ | □ |
3. We all have some responsibility to help others by volunteering for medical research. | □ | □ | □ | □ | □ |
4. Society needs to devote more resources to medical research. | □ | □ | □ | □ | □ |
5. Participating in medical research is generally safe. | □ | □ | □ | □ | □ |
6. If I volunteer for medical research, I know my personal information will be kept private and confidential. | □ | □ | □ | □ | □ |
7. Medical research will find cures for many major diseases during my lifetime. | □ | □ | □ | □ | □ |
Footnotes
Clinicaltrials.gov identifier: NCT01504854
Industry-sponsorship: No industry sponsorship.
References
- 1.Angst J, Bech P, Bobon D, et al. Report on the third consensus conference on the methodology of clinical trials with antipsychotic drugs. Pharmacopsychiatry 1991; 24: 149–152. 1991/September/01. [DOI] [PubMed] [Google Scholar]
- 2.Gross RA and Johnston KC. Levels of evidence: Taking Neurology to the next level. Neurology 2009; 72: 8–10. [DOI] [PubMed] [Google Scholar]
- 3.Gardette V, Coley N, Toulza O, et al. Attrition in geriatric research: how important is it and how should it be dealt with? The journal of nutrition, health & aging 2007; 11: 265–271. [PubMed] [Google Scholar]
- 4.Bynum JPW, Rabins PV, Weller W, et al. The Relationship Between a Dementia Diagnosis, Chronic Illness, Medicare Expenditures, and Hospital Use. Journal of the American Geriatrics Society 2004; 52: 187–194. DOI: 10.1111/j.1532-5415.2004.52054.x. [DOI] [PubMed] [Google Scholar]
- 5.Schneider LS and Sano M. Current Alzheimer's disease clinical trials: methods and placebo outcomes. Alzheimers Dement 2009; 5: 388–397. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Vellas B, Andrieu S, Sampaio C, et al. Disease-modifying trials in Alzheimer's disease: a European task force consensus. Lancet neurology 2007; 6: 56–62. [DOI] [PubMed] [Google Scholar]
- 7.Henley DB, Sundell KL, Sethuraman G, et al. Safety profile of Alzheimer's disease populations in Alzheimer's Disease Neuroimaging Initiative and other 18-month studies. Alzheimers Dement 2012; 8: 407–416. [DOI] [PubMed] [Google Scholar]
- 8.Honig LS, Vellas B, Woodward M, et al. Trial of Solanezumab for Mild Dementia Due to Alzheimer's Disease. The New England journal of medicine 2018; 378: 321–330. [DOI] [PubMed] [Google Scholar]
- 9.Grill JD and Karlawish J. Addressing the challenges to successful recruitment and retention in Alzheimer's disease clinical trials. Alzheimers Res Ther 2010; 2: 34. 2010/December/22. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Hayden JA, Cote P and Bombardier C. Evaluation of the quality of prognosis studies in systematic reviews. Annals of internal medicine 2006; 144: 427–437. [DOI] [PubMed] [Google Scholar]
- 11.Greenland S. Response and follow-up bias in cohort studies. Am J Epidemiol 1977; 106: 184–187. 1977/September/01. [DOI] [PubMed] [Google Scholar]
- 12.Crane PK and Doody RS. Donepezil treatment of patients with MCI: a 48-week randomized, placebo- controlled trial. Neurology 2009; 73: 1514–1515. [DOI] [PubMed] [Google Scholar]
- 13.Halpern SD, Karlawish JH and Berlin JA. The continuing unethical conduct of underpowered clinical trials. JAMA 2002; 288: 358–362. [DOI] [PubMed] [Google Scholar]
- 14.Emanuel EJ, Wendler D and Grady C. What makes clinical research ethical? JAMA 2000; 283: 2701–2711. [DOI] [PubMed] [Google Scholar]
- 15.National Research Council (US) Panel on Handling Missing Data in Clinical Trials. The prevention and treatment of missing data in clinical trials. 2010. Washington, DC: The National Academies Press. [PubMed] [Google Scholar]
- 16.Gul RB and Ali PA. Clinical trials: the challenge of recruitment and retention of participants. Journal of Clinical Nursing 2018; 19: 227–233. [DOI] [PubMed] [Google Scholar]
- 17.Grill JD, Raman R, Ernstrom K, et al. Comparing recruitment, retention, and safety reporting among geographic regions in multinational Alzheimer’s disease clinical trials. Alzheimer's Research & Therapy 2015; 7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Grill JD, Raman R, Ernstrom K, et al. Effect of study partner on the conduct of Alzheimer disease clinical trials. Neurology 2013; 80: 282–288. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Rubright JD, Cary MS, Karlawish JH, et al. Measuring how people view biomedical research: Reliability and validity analysis of the Research Attitudes Questionnaire. J Empir Res Hum Res Ethics 2011; 6: 63–68. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Grill JD, Zhou Y, Elashoff D, et al. Disclosure of amyloid status is not a barrier to recruitment in preclinical Alzheimer's disease clinical trials. Neurobiol Aging 2016; 39: 147–153. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Cary MS, Rubright JD, Grill JD, et al. Why are spousal caregivers more prevalent than nonspousal caregivers as study partners in AD dementia clinical trials? Alzheimer Dis Assoc Disord 2015; 29: 70–74. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Neugroschl J, Sewell M, De La Fuente A, et al. Attitudes and Perceptions of Research in Aging and Dementia in an Urban Minority Population. J Alzheimers Dis 2016; 53: 69–72. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Turner RS, Thomas RG, Craft S, et al. A randomized, double-blind, placebo-controlled trial of resveratrol for Alzheimer disease. Neurology 2015; 85: 1383–1391. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Moussa C, Hebron M, Huang X, et al. Resveratrol regulates neuro-inflammation and induces adaptive immunity in Alzheimer's disease. Journal of neuroinflammation 2017; 14: 1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Rosen WG, Mohs RC and Davis KL. A new rating scale for Alzheimer's disease. The American journal of psychiatry 1984; 141: 1356–1364. [DOI] [PubMed] [Google Scholar]
- 26.Galasko D, Bennett D, Sano M, et al. An inventory to assess activities of daily living for clinical trials in Alzheimer's disease. The Alzheimer's Disease Cooperative Study. Alzheimer Dis Assoc Disord 1997; 11 Suppl 2: S33–39. [PubMed] [Google Scholar]
- 27.Cummings JL. The Neuropsychiatric Inventory: assessing psychopathology in dementia patients. Neurology 1997; 48: S10–16. [DOI] [PubMed] [Google Scholar]
- 28.McBee M Modeling Outcomes With Floor or Ceiling Effects: An Introduction to the Tobit Model. Gifted Child Quarterly 2010; 54: 7. [Google Scholar]
- 29.Martinez-Camblor P and Pardo-Fernandez JC. The Youden Index in the Generalized Receiver Operating Characteristic Curve Context. Int J Biostat 2019; 15 2019/April/04. [DOI] [PubMed] [Google Scholar]
- 30.Leeflang MM, Moons KG, Reitsma JB, et al. Bias in sensitivity and specificity caused by data-driven selection of optimal cutoff values: mechanisms, magnitude, and solutions. Clin Chem 2008; 54: 729–737. [DOI] [PubMed] [Google Scholar]
- 31.Lo RY and Jagust WJ. Predicting missing biomarker data in a longitudinal study of Alzheimer disease. Neurology 2012; 78: 1376–1382. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Grill JD, Kwon J, Teylan MA, et al. Retention of Alzheimer Disease Research Participants. Alzheimer Dis Assoc Disord 2019; 33: 299–306. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Radtke T, Liszewska N, Horodyska K, et al. Cooking together: The IKEA effect on family vegetable intake. Br J Health Psycho 2019; 24: 896–912. [DOI] [PubMed] [Google Scholar]
- 34.Robinson KA, Dinglas VD, Sukrithan V, et al. Updated systematic review identifies substantial number of retention strategies: using more strategies retains more study participants. Journal of clinical epidemiology 2015; 68: 1481–1487. [DOI] [PMC free article] [PubMed] [Google Scholar]