Retaining Participants in Longitudinal Studies of Alzheimer’s Disease

Abstract

Background:

Retention of study participants is essential to advancing Alzheimer’s disease (AD) research and developing therapeutic interventions. However, recent multi-year AD studies have lost 10% to 54% of participants.

Objective:

We surveyed a random sample of 443 participants (Clinical Dementia Rating [CDR] ≤ 1) at four Alzheimer Disease Research Centers to elucidate perceived facilitators and barriers to continued participation in longitudinal AD research.

Methods:

Reasons for participation were characterized with factor analysis. Effects of perceived fulfillment of one’s own goals and complaints on attendance and likelihood of dropout were estimated with logistic regression models. Open-ended responses suggesting study improvements were analyzed with a Latent Dirichlet Allocation topic model.

Results:

Factor analyses revealed two categories, personal benefit and altruism, as drivers of continued participation. Participants with cognitive impairment (CDR>0) emphasized personal benefits more than societal benefits. Participants with higher trust in medical researchers were more likely to emphasize broader social benefits. A minority endorsed any complaints. Higher perceived fulfillment of one’s own goals and fewer complaints were related to higher attendance and lower likelihood of dropout. Facilitators included access to medical center support and/or future treatment, learning about AD and memory concerns, and enjoying time with staff. Participants’ suggestions emphasized more feedback about individual test results and AD research.

Conclusion:

The results confirmed previously identified facilitators and barriers. Two new areas, improved communication about individual test results and greater feedback about AD research, emerged as the primary factors to improve participation.

INTRODUCTION

Over 270,000 study participant volunteers are needed for longitudinal Alzheimer’s disease (AD) studies [1].Recruitment and retention of study participants has been considered “the greatest obstacle to developing new [AD] treatments [2].” Attrition and missed visits can undermine the representativeness of a study and cause severe missing data problems, particularly in longitudinal research [3–8]. Such longitudinal studies, often conducted at the preclinical and prodromalphases of AD, are critical for the development of therapeutic interventions [9–12]. These studies involve considerable burden for participants and their families [13, 14]. As a result, multi-year clinical trials and observational studies have lost between 10% and 54% of participants [14, 15]. Optimizing retention of participants and ensuring that they regularly participate while enrolled in longitudinal AD studies are essential to increase the effectiveness, validity, and generalizability of study findings to advance AD research.

Previous studies have identified a variety of participant characteristics that may be associated with attrition including age [16–18], gender [16], education [6, 17], race [6, 17, 19], neuropsychiatric symptoms [6, 20], cognitive impairment [6, 16, 17, 20], decreased functional abilities [16], physical inactivity [6, 16], social isolation [6], questionable co-participant reliability [6], and non-spouse informants [6, 21]. As these factors are largely immutable, they provide little guidance for adjusting study design to improve retention. Relatively little is known about AD study participants’ perceived facilitators and barriers to participation and whether these affect their continued participation. Facilitators to research participation in general for older adults include altruism [22, 23], access to the latest evidence-based care [24, 25], gaining a better understanding of memory change [26], and strong relationships with research staff [22, 24]. Barriers to retention of older adult participants include illness [24] or family member death [27], institutionalization [22], ambulatory difficulties [22], driving cessation [28], and inability to identify a study partner [29]. Retention of underrepresented populations also warrants urgent effort [22, 30]. Research on barriers to recruitment of underrepresented older adults has revealed concerns including inaccessibility of research sites [22, 24], mistrust of researchers [22,24,31–34], unmet service needs [22], and researchers’ lack of familiarity with community norms, values, and cultures [22, 34–38]. Establishing trust within the community by fostering connections at multiple levels has been identified as a facilitator to recruitment and retention of racial and ethnic minority older adult participants [38–42].

To improve the retention of participants in longitudinal studies of AD, we conducted a multicenter, mixed methods study to identify perceived facilitators and barriers to longitudinal research participation and to examine their influence on retention, including how regularly individuals participate (attendance rates) and dropouts. We surveyed 443 longitudinal research participants and 212 research study partners at four Alzheimer Disease Research Centers (ADRCs) to identify perceived facilitators and barriers to ongoing research participation. We tested the hypothesis that these facilitators and barriers would differ among pre-specified subgroups (i.e., that participants from traditionally underrepresented racial and ethnic groups, younger participants, and participants without cognitive impairment may perceive more barriers).

METHODS

Participants

Research participants were recruited from active cohorts at four ADRCs funded by the National Institute on Aging: Knight Alzheimer Disease Research Center at Washington University in St. Louis (Knight ADRC), University of Pittsburgh ADRC (PITT ADRC), University of Wisconsin ADRC (Wisconsin ADRC), and University of California–Irvine ADRC (UCI ADRC). Participants and study partners were invited to participate if the participant: 1) was 45 years of age or older, 2) was currently enrolled in longitudinal studies, and 3) had a Clinical Dementia Rating (CDR^®) score of≤1 at their previous clinical assessment. Participants were excluded if they were institutionalized and/or did not reside in the geographic area of the ADRC.

Based on a random selection at each ADRC, participants were approached and invited to participate in person or via phone call between scheduled ADRC visits. If interested, participants and study partners completed informed consent. Written consent was secured at in-person visits; verbal consent was obtained if the participant and study partner were approached by phone or were unable to complete the survey during a study visit. This study was approved by the Institutional Review Boards at all four sites and was conducted in 2018–2019.

Study procedures

Participants completed a 20-minute survey about facilitators and barriers to longitudinal study retention in person or over the phone. In-person surveys were administered verbally with a study team member between scheduled visits. Surveys were completed via phone if the participant or study partner chose to complete the survey at a later date or provided consent to the study over the phone. To address any problems with comprehension, questions were repeated when requested, and participants could have someone with them to assist in understanding the questions. All participants and study partners were instructed to reflect on their most recent Uniform Data Set study visit at the ADRC prior to answering questions. They were offered a $5 gift card as remuneration for participation. Trained and certified raters administered the survey. We utilized the National Alzheimer’s Coordinating Center (NACC) Uniform Data Set [43, 44] to analyze demographic information for this sample, which included gender, race, and ethnicity based on self-identification of the participant.

Measures

We developed and piloted a participant and study partner version of the survey to address facilitators and barriers specific to longitudinal studies of AD. The survey was based on: 1) a review of the literature, 2) specific procedures and expectations of participants in longitudinal AD studies, and 3) two standardized assessments [45, 46]. The survey addressed the perceived burdens and benefits of participating in longitudinal AD studies for participants and study partners. Items were ranked on a five-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree) unless otherwise noted. The final survey was composed of 57 items, with two open-ended questions (see Supplementary Material).

We also analyzed two metrics of participation that impact retention: attendance rates and dropouts. First, we calculated participants’ attendance rates based on the number of study visits they had attended divided by the number of possible study visits for which they had been scheduled. Second, we identified participants who had dropped out of the study. We defined a dropout as a participant who was defined in the NACC database as no longer receiving follow-ups and who had not died or been removed from the study as of October 18, 2020. Such a removal might happen, for example, due to a CDR score above 2.

Statistical analyses

Study responses and scores were entered directly by participants or a study team member using Research Electronic Data Capture (REDCap) [47]. REDCap provides secure data entry with real-time validation. REDCap servers for this study were housed in a firewall-protected, limited-access data center managed by the Washington University Division of Biostatistics. Quality control programs were executed to verify identification, evaluate consistency, and monitor recruitment and retention. Secure data transfer was completed using a standardized protocol. Response frequencies were computed for each item, and open-ended responses were transcribed and thematically coded. Analyses were conducted in STATA 15.1 (StataCorp, LLC, College Station, TX).

Principal component factor analysis was performed to estimate and explain the latent structure of reasons for participation. Factors were rotated with varimax (orthogonal) and promax (oblique) techniques. Regression scoring was used to generate factor scores on the rotated dimensions. Linear regression analysis of the factor scores was by ordinary least squares (OLS) with robust (Huber/White/sandwich) standard errors. OLS with robust standard errors and the ‘medeff’ procedure were used in the mediation analysis. Analysis of attendance rates was conducted with the procedure ‘fracreg’ (fractional logistic regression), which is appropriate for proportions. Analysis of dropouts was executed with the’ firthlogit’ procedure (Firthlogistic regression), which is designed for rare binary events. The topic models were estimated with ‘ldagibbs,’ a Latent Dirichlet Allocation machine learning topic model.

RESULTS

Four hundred forty-three participants completed the survey across the four ADRCs. Demographic characteristics for participants by site and overall are included in Table 1.

Table 1.

Demographic characteristics of participants by site and overall

Participants
	Knight ADRC (n = 111)	Wisconsin ADRC (n = 112)	PITT ADRC (n = 110)	UCI ADRC (n = 110)	Total (N = 443)
Age, Mean (SD)	68.0 (6.8)	63.1 (8.7)	67.3 (7.3)	70.7 (7.8)	67.3 (8.1)
Gender
Female	61 (54%)	75 (67%)	63 (57%)	63 (57%)	262 (59%)
Male	51 (46%)	37 (33%)	47 (43%)	47 (43%)	182(41%)
Race/Ethnicity, n (%)
Black	25 (22%)	14 (12%)	8 (7%)	0 (0%)	47 (11%)
Asian	0 (0%)	0 (0%)	0 (0%)	41 (37%)	47 (9%)
White	87 (78%)	94 (84%)	101 (92%)	67 (61%)	349 (79%)
Other	0 (0%)	4 (4%)	1 (1%)	2 (2%)	7 (1%)
Hispanic, n (%)
Yes	0 (0%)	0 (0%)	1 (1%)	6 (5%)	7 (2%)
No	112(100%)	112(100%)	109 (99%)	104 (95%)	437 (98%)
Years of Education, Mean (SD)	16.1 (2.6)	16.1 (2.8)	16.4 (2.5)	17.3 (2.3)	16.5 (2.6)
APOE4, n (%)
Positive	47 (42%)	43 (38%)	38 (35%)	32 (29%)	160 (36%)
Negative	65 (58%)	68 (61%)	72 (65%)	59 (54%)	264 (59%)
Missing	0 (0%)	1 (1%)	0 (0%)	19 (17%)	20 (5%)
CDR, n (%)
CDR = 0	90 (80%)	90 (80%)	51 (46%)	77 (70%)	310 (70%)
CDR > 0	22 (20%)	22 (20%)	59 (54%)	33 (30%)	134 (30%)
Days enrolled, Mean (SD)	2,559.4 (1458.8)	1,599.8 (742.1)	1,768.8 (1207.8)	1,635.0 (1434.9)	1,857.7 (1282.1)

Goals of participation and perceived satisfaction

The participants’ general assessment of their participation was highly positive. The vast majority (>84%) agreed or strongly agreed that their participation is valuable and that they are accomplishing their goals for participation. Very few participants (<6%) agreed or strongly agreed that they regretted their decision to participate or that they had had second thoughts about participation.

Perceived facilitators

Common reasons for participation were to advance AD research and to benefit society and future generations, but many participants identified multiple goals influencing their decision to participate (Fig. 1). A factor analysis of the reasons for participation revealed that these goals were organized into two general categories: those involving personal benefits and those involving altruism (Table 2). The eigenvalues for the first three factors (2.38, 1.44, and 0.09) strongly supported a two-factor model. An oblique rotation revealed only a weak (0.18) positive correlation between the dimensions and very similar factor loadings to those reported here.

Fig. 1.

Reasons for participation in ADRC.

Table 2.

Factorⁱ and regression analyses of reasons for participation

Reasons for participating	Factor 1 (personal benefit)	Factor 2 (altruism)
To advance AD research	−0.04	0.79
To benefit society	0.09	0.78
To benefit future generations of own family	0.21	0.59
Because I have concerns about memory	0.62	–0.11
To gain access to future treatments	0.72	0.11
To learn more about AD	0.60	0.21
To enjoy time with staff	0.43	0.19
To access medical center support	0.80	0.04
Predictors of the factor scores for each category of motivation (OLS regression coefficients and robust standard errors)
CDR Score >0 (0 if CDR=0; 1 if CDR>0)	0.59** (0.08)	−0.33** (0.11)
Trust medical researchers (1, not at all, to 5, a great deal)	0.09 (0.05)	0.32** (0.06)
Male	0.10 (0.08)	−0.07 (0.09)
Black	0.48** (0.11)	0.13 (0.15)
Asian	0.28* (0.14)	−0.08 (0.14)
Other race	0.57* (0.28)	−0.22 (0.29)
Knight ADRC	−0.35** (0.11)	−0.05 (0.14)
Wisconsin ADRC	−0.23** (0.12)	0.09 (0.12)
UCI ADRC	0.14 (0.12)	0.09 (0.13)
N	439	439
Adjusted R 2	0.19	0.13

^**p<0.01;

^*p<0.05.

ⁱVarimax (orthogonal) rotation.

Participants’ goals were associated their clinical status, with personal benefits systematically emphasized more and societal benefits emphasized less by participants with documented evidence of cognitive impairment (CDR>0). Those with higher trust in medical researchers were more likely to emphasize broader social benefits. In addition, Black, and to a lesser extent, Asian, participants were more likely to emphasize goals with personal benefits than were White (the baseline category in the regressions in Table 2). To ensure that these results were not due to differences across sites in participants’ reasons for participation, the analysis included controls for ADRC site (PITT ADRC was the baseline category).

Perceived barriers

Overall, participants endorsed few challenges with their continued participation. A minority (39%) of participants endorsed any of the 19 negative factors offered on the survey. Complaints with which at least 5% of participants agreed or strongly agreed were: difficulty keeping track of procedures (14%), fatigue (12%), inconvenient travel (11%), distance (11%), visits too long (10%), emotional distress (7%), physical pain (7%), breach of privacy (5%), and side effects (5%). The frequency and type of complaint was generally consistent across age cohorts, sexes, and races. One exception was that more Black participants (15%) expressed concern with privacy thandid White participants (4%). Based on a mediation analysis, almost one-third (31%) of this effect was due to lower trust in medical researchers among Black participants (Fig. 2).

Fig. 2.

Trust in medical research mediates effect of race on privacy concerns. The values in the figure represent standardized regression coefficients. The value in parentheses is for the total effect of Black race, which was estimated in a regression without trust in medical researchers. All regressions included controls for other races and for study sites. **p<0.01, *p<0.05, two-tailed test.

The effect of perceived fulfillment of one’s own goals and complaints on retention

For 405 of the 443 participants who responded to the survey, information was available about the number of possible (scheduled) study visits and the number of completed study visits as of January 25, 2021. The number of expected visits ranged from 0 to 13. We identified 12 participants out of 443 as dropouts. Overall, the attendance rate was high. For participants with at least two possible study visits (N=377), 75% had perfect attendance, the average attendance rate was 92% (SD=16%), and the lowest performance was 20%. Lower attendance was associated with higher dropout rates (p<0.01, difference in proportions test). Participants who dropped out had an average attendance rate of 59%, while those who remained in the study had a rate of 93%.

Table 3 reports fractional logistic regression results estimating the effect of participants’ perceived fulfillment of their own goals and perceived complaints on their attendance rates and Firth logistic regression results for their likelihood of dropout. The analysis of attendance rates includes only participants with two or more possible visits and includes a control variable for the total number of possible visits. The control variable ensures that the number of possible visits does not confound the analysis because, for example, the participants most committed to the study may have both relatively high attendance rates and relatively long histories of visits. Also, these analyses included controls for the participants’ ADRC sites. This means that the estimated effects of complaints and fulfillment of one’s own goals on attendance and dropouts were based on comparisons of participants within study sites, not across sites.

Table 3.

Perceived fulfillment of one’s own goals, complaints, and retention

	Attendance rate			Dropout
Accomplishing Goals (range: 1–5)	0.38* (0.16)	-	-	−1.34** (0.39)	-
Number of Complaints (range: 0–13) Specific Complaints:	-	−0.24** (0.05)	-	-	0.56* (0.24)
Fatigue	-	-	−0.19 (0.17)	-	-
Inconvenient travel	-	-	0.10(0.20)	-	-
Distance	-	-	0.20(0.19)	-	-
Visits too long	-	-	0.10(0.17)	-	-
Emotional distress	-	-	0.16(0.18)	-	-
Difficulty keeping track	-	-	−0.33 (0.22)	-	-
Physical pain	-	-	0.18 (0.20)	-	-
Breach of privacy	-	-	0.45** (0.15)	-	-
Side effects	-	-	−0.05 (0.25)	-	-
Area deprivation index	−0.02 (0.04)	−0.01 (0.04)	−0.02 (0.04)	0.13 (0.08)	0.09 (0.08)
Age (y)	−0.01 (0.02)	−0.01 (0.02)	−0.01 (0.02)	0.09 (0.05)
CDR Score > 0 (0 if CDR = 0; 1 if CDR > 0)	−0.09 (0.27)	−0.03 (0.27)	−0.07 (0.28)	0.68 (0.74)
Years of education	0.05 (0.05)	0.05 (0.05)	0.05 (0.05)	0.00(0.14)	−0.02(0.12)
Male	0.08 (0.26)	0.03 (0.25)	0.03 (0.256)	−0.56 (0.80)	−0.24 (0.72)
Black	−0.86* (0.38)	−0.85* (0.38)	−0.63 (0.34)	0.95 (0.91)	1.55 (0.92)
Asian	0.81 (0.56)	1.25 (0.63)	1.12* (0.56)	0.50(1.99)	−3.09 (3.38)
Other race	−1.22 (0.73)	−1.31* (0.57)	−1.30 (0.65)	1.68 (1.73)	2.01 (1.79)
Number of scheduled study visits	−0.01 (0.03)	−0.02 (0.03)	−0.04 (0.03)	-	-
N	361	363	362	427	429

Fractional logistic regression results for attendance rate, excluding participants with fewer than two possible visits. Firth logistic regression results for dropout. Log odds ratios reported with standard errors in parentheses. All models included controls for ADRC site.

^**p≤0.01,

^*p < 0.05, two-tailed test.

Participants who reported that ADRC study participation fulfilled their goals had, on average, higher odds of having perfect attendance (OR=1.46, 95% CI: 1.07–1.99) and a lower risk of dropout (OR=0.26, 95% CI: 0.12–0.56). Participants with a higher number of complaints had lower odds of having perfect attendance (OR=0.79, 95% CI: 0.71–0.87) and were more likely to drop out than those with fewer complaints (OR=1.75, 95% CI: 1.10–2.79). Of the nine most common complaints (Fig. 3), the only one that achieved a statistically significant association with attendance rates and dropout risk was a breach of privacy. This privacy effect (p<0.002) remained significant after Bonferroni correction (p<0.01) for multiple hypothesis testing. Given that dropouts were rare, the data did not allow for precise estimates of the effects of specific complaints on the risk of dropout.

Fig. 3.

Topic model of participants’ suggestions to enhance their study experience. The text presented for each topic represents the beginning of the three comments with the highest share of the specific topic.

We observed no independent effects on dropout rate or attendance of age, sex, education, cognitive decline, or socioeconomic context (measured with area deprivation index). Race did affect attendance, with lower attendance rates among Black participants compared to White and Asian participants.

Open-ended comments about improving participant study experience

The survey included an open-ended question: “What is the most important thing that ADRC researchers could do or already do to enhance your experience with the study?” Three hundred seventy-two participants responded, typically with brief answers. Figure 3 presents the results of a Topic Model summarizing these comments. The model excluded words with fewer than five letters and assumed comments focused on a small number of topics. We selected eight topics based on model fit and the interpretability/coherence of the topics.

Many comments offered no suggestions or criticisms. The topics “satisfied with study,” “positive experience,” and “find a cure” accounted for 45% of comments. The two most common suggestions involved the topics “more feedback” and “learn more about AD.” The remaining 25% of comments involved concerns related to travel, study partners, and finding a cure for AD.

The participants’ topics of emphasis were consistent with their survey responses. For example, the odds of emphasizing the topic“ learn more about AD” in one’s open-ended comments increased by 1.37 times (95% CI: 1.08–1.74) as one expressed higher agreement that the reason for one’s participation was to learn about AD on the closed-ended survey question. The odds of emphasizing the topic “provide feedback” rose by 1.31 (95% CI: 1.05–1.63) as one’s evaluation of their most recent clinical assessment interview became more negative. Finally, an increase in agreement that study visits were too long was associated with 1.32 times greater odds (95% CI: 1.05–1.66) of emphasizing the topic “time problems.”

DISCUSSION

This survey of study participants at four ADRC sites reveals that study participants vary in how they view their participation, why they participate, and the barriers they encounter to continued participation. These factors are important predictors of both how often participants attend visits and whether they drop out. Thus, the survey identifies aspects of study conduct and participant experience, including facilitators and barriers, that are relevant to reducing dropouts and increasing attendance in longitudinal AD studies.

To our knowledge, this is the first multi-site systematic investigation of facilitators and barriers to retention in longitudinal AD studies. Our two-factor analysis revealed that participants’ goals for participation were for personal benefit or altruism, which are the most common reasons participants enroll in clinical research [32]. Previous studies have examined facilitators and barriers to study participation in other populations, but research is limited for participants in AD studies. Facilitators to participation in the four ADRC longitudinal AD studies included altruism [22,23], access to evidence based care (medical center support and/or future treatment) [24, 25], learning about AD and memory concerns [26], and enjoying time with staff [22, 24]. A limited number of participants identified one or more barriers, including difficulty keeping track of procedures, fatigue, inconvenient travel, distance, visits too long, emotional distress, physical pain, breach of privacy, and fear of side effects.

While our findings are broadly consistent with previous literature [22–26, 32], they have important implications for retention in longitudinal AD studies. Specifically, our results indicate that perceived fulfillment of one’s own goals and concerns about the execution of the AD studies, including lack of information about study findings and their impact on the field of AD, may play a significant role in attrition and attendance. Furthermore, the analysis suggests that limiting attrition due to participants’ characteristics—such as trust in medical research or cognitive decline—can be addressed by responding to the differences in goals associated with these characteristics. For example, those with higher trust in medical researchers who emphasized participating for societal benefit may respond to strategies that appeal to their desire to advance AD research and benefit society and future generations. Conversely, participants with CDR>0 were more likely to emphasize personal benefits from participation than were those with CDR=0. These participants may be motivated to continue participating, for example, if their concerns about memory are addressed. This is especially important as those with cognitive impairment are at higher risk for attrition [6, 16, 17, 20].

Black participants also tended to participate for personal benefit, but they had lower attendance rates (although they were not at greater risk for dropout) compared to White participants [17, 48]. Addressing Black participants’ goals for participation and evaluating barriers to participation over time, including low levels of trust in medical research, lack of access to health care, and privacy concerns, have potential to improve retention. Black participants and other underrepresented minorities are generally referred or recruited for ADRC participation through different channels than White participants, which can impact their reasons for participation and their risk for dropout [49, 50]. Strategies that may positively impact retention include providing additional information about AD and improving access to future treatments and medical center support.

The open-ended comments aligned with participants’ survey responses and identified areas of study design that could enhance participants’ senses of fulfillment of their own goals and contributions to the field of AD, and thus, retention. Participants suggested specific types of facilitators that would address their own reasons for participating, as well as their concerns. The most common suggestions revolved around increasing feedback about the tests administered at study visits and providing more general information about AD and related research. These could easily be accommodated under existing ADRC study protocols, which frequently involve sharing diagnostic test results and holding events and celebrations to thank participants [17]. Logistical considerations regarding travel, study partners, and time demands were reported less frequently but also appear to be important areas of concern.

The study has several limitations. Only current participants whom we succeeded in contacting were included in the study. These participants may be unusually compliant or committed. Additionally, due to the terms of the participants’ informed consent, we could not survey participants who had already dropped out. These former participants—as well as participants who were unwilling to participate in the survey—would likely be valuable sources of information about attrition. Open-ended questions of those former participants about their reasons for dropout would be a valuable complement to the structured survey from this study. Although our participants were randomly selected from four ADRCs, the results may not generalize to individuals at other ADRCs, where burden and retention tactics may differ, or to studies performed in other settings. Although we had good representation among Black and Asian participants, we had fewer participants from Hispanic ethnicities, further limiting generalizability. These data were collected prior to the COVID-19 pandemic. COVID-19 created additional retention challenges that should be examined in future studies. Also, we had very limited ability to evaluate the effect of remuneration on retention. Wisconsin ADRC paid participants from underrepresented groups $50 per study visit. We found no relationship between that remuneration and rates of attendance or dropouts. Finally, we did not retain information on the mode of survey, which may have influenced survey responses.

Despite these limitations, identifying and addressing participants’ perceived facilitators and barriers are crucial to enhancing longitudinal participation in AD studies. In addition to providing information about individual test results and feedback about AD research, researchers should consider using methods to identify participants’ goals over time. These may change for systematic reasons (e.g., an increase in CDR score), and tailoring strategies to those changes may increase retention in longitudinal studies of AD.