Adding Delayed Recall to the Alzheimer Disease Assessment Scale is Useful in Studies of Mild Cognitive Impairment But Not Alzheimer Disease

Abstract

Objective

To determine if the addition of delayed recall (DR) assessment adds sensitivity to the cognitive subscale of the Alzheimer Disease Assessment Scale (ADAS-cog) in clinical trials in mild cognitive impairment (MCI) and Alzheimer Disease (AD).

Background

Memory, particularly DR, is the most sensitive test for early detection of AD and MCI. However, it is not clear that assessment of DR adds benefit for measuring change over time after a diagnosis is made or in clinical trials. The ADAS-cog is the most commonly used tool to assess treatment efficacy in AD clinical trials. In an attempt to improve sensitivity to change, assessment of DR after the 3-trial, 10-word list was added to the standard 11-item ADAS-cog. We examined the added value of the DR in participants with MCI and AD followed for at least 1 year.

Design/Methods

Data from 111 subjects with AD and 259 subjects with MCI who were randomly assigned to the placebo arm of 2 clinical trials were included. Participants with AD had Mini- Mental State Examination scores of 13 to 27 and those with MCI had 24 to 30. We calculated the ADAS-cog11 score based on the original 11 items (range: best to worse, 0 to 70), the DR item score (range: 0 to 10 words not recalled), and the ADAS-cog12 (range: 0 to 80). We assessed the rate of missing items for DR over time, the change scores, the association between scores and baseline performance, and used longitudinal mixed effects regression models to examine the rate of change.

Results

At baseline AD subjects were near floor on DR (8.93 ± 1.6 SD) and showed little change over 1 year (0.12 ± 1.34); the MCI subjects baseline DR was 6.2 ± 2.2 with 1-year change of 0.20 ± 1.7. We compared standardized change (change/SD) for ADAS-cog11, and 12 in MCI and found a 10% improvement with ADAS-cog12; there was no improvement in the AD group. In a subset of MCI and AD cases with matching Mini-Mental State Examination (23 to 27), the ADAS-cog12 provided an 18% improvement in standardized change in MCI subjects, with no benefit in the AD cohort, primarily owing to increased variance.

Conclusions/Relevance

The addition of DR to the ADAS-cog score increased the ability to detect change in subjects with MCI over 1 year compared with the ADAS-cog11 but increased the variance in subjects with AD, even in those with mild impairment These findings speak to the need to tailor outcome measures to the specific study population and diagnosis for maximal efficiency and economy when conducting clinical trials.

The Alzheimer Disease Assessment Scale-cognitive subscale (ADAS-cog),¹ is the most commonly used cognitive outcome measure in clinical trials for Alzheimer disease (AD). It has been successfully used to demonstrate cognitive benefit in clinical trial of patients with mild-to-moderate AD,² but may have limited sensitivity to longitudinal decline, particularly in those with mild deficits such as those with mild cognitive impairment (MCI). In an attempt to improve the sensitivity of cognitive assessments, several additional subtests have been developed and used in some trials. One addition is the assessment of delayed recall (DR) of the word list, as DR may be important in MCI and early AD.³ This subtest consists of asking the subject to recall the 10 words used in the 3 trials of the immediate recall word list of the ADAS-cog. The addition of this task was initially supported by study of longitudinal change on word list recall among subjects with AD over a range of disease severity levels.⁴ DR was assessed after 4 learning trials, and the results supported its use in those with mild AD. The current work assesses the value of adding this subtest to the standard ADAS-cog 10-word, 3-trial learning for patients enrolled in the placebo arm of long-term clinical trials in mild-to-moderate AD and MCI.

METHODS

Data was examined from subjects with AD who were randomly assigned to the placebo arm of a year-long clinical trial of nonsteroidal anti-inflammatory drugs⁵ and from subjects with MCI who were randomly assigned to the placebo arm of a 3-year clinical trial of vitamin E or donepezil.⁶ Both of these studies were conducted by the Alzheimer’s Disease Cooperative Study Group. Previous reports describe the recruitment criteria for these trials in detail. In brief, AD patients were recruited from 1999 to 2000, were required to meet National Institute of Neurological and Communicative Diseases and Stroke-Alzheimer’s Disease and Related Disorders Association criteria for AD,⁷ were described as mild-to-moderate as indicated by an Mini-Mental State Examination (MMSE) of 13 to 26,⁸ and had no evidence of other neurological or psychiatric disease. Background use of standard of care treatments for AD (most commonly cholinesterase inhibitors) was permitted. The MCI patients were recruited from 1999 to 2004 and were described as amnestic MCI according to the criteria of Peterson.⁹ This was operationalized as an MMSE score of 24 or greater and performance on the DR of the first paragraph of the Wechsler Memory Scale approximately 1.5 SDs below an age and education adjusted mean in the presence of otherwise intact cognition, with no significant impairment in social or occupational functioning. Concomitant use of prescriptive cognitive enhancers (cholinesterase inhibitors, memantine) and vitamin E was prohibited. As described in the original publications,^5,6 informed consent was obtained from patients and family members in the AD patient trial and from patient in the MCI trial.

ADAS Scoring Procedures

The ADAS-cog11, which ranges from 0 to 70 with each point representing a performance error and lower scores reflecting better performance, was administered according to the methodology described by Rosen et al.¹ It consists of 11 subtests, including 7 performance-based assessments and 4 rater-based items. The performance items include word recall, following commands, constructional praxis, naming, ideational praxis, orientation, and word recognition. The rater-based items include remembering test items, comprehension, word finding ability, and spoken language ability. The original method of administration consisted of 3 trials of recognition with each trial beginning with the presentation of 12 target words alone followed by presentation of the targets in combination with 12 distracter words. In the studies presented here administration was restricted to one trial of recognition because previous analyses demonstrated high correlation between the score on the first trial and the mean of the 3 trials.

DR administration was carried out before word recognition to avoid interference from the second list. Although the exact amount of time between the initial presentation and the DR is not specified in the procedures, the median time to DR for MCI subjects was 5 minutes, and the interquartile range was from 1 to 8 minutes. Each subject was prompted with the following text: “I read you a list of words a few minutes ago; please tell me all the words you can recall.” Subjects were given up to 60 seconds but could report being finished sooner. Recalled words were recorded, and the assigned score was the nonrecalled words, or 10 minus the number of words recalled, with a score range from 0 to 10. The ADAS-cog12 is the total of the ADAS-cog11 plus the DR Score, and therefore has a range of 0 to 80.

Data Analysis

Baseline clinical measures [age of onset, Clinical Dementia Rating (CDR),¹⁰ and CDR sum of boxes (CDRsb), ApoE genotype status (defined as presence of at least one ApoE ε4 allele vs. no ApoE ε4 allele] and demographic features (age, sex, education) of the 2 groups were compared using t tests or Wilcoxon Rank Sum test for continuous variables and χ² test or Fisher exact test for categorical variables, as appropriate. Comparisons between the MCI and AD groups on DR, ADAS-cog11, and 12 scores at baseline and 12 months, and 12 months change scores were conducted using 2-sample t test or Wilcoxon Rank Sum test. In an effort to reflect common reporting practice in earlier AD clinical trials, the change for each measure was calculated in 2 ways: using nonimputed data and data imputed using the last observation carried forward (LOCF) method.

We calculated standardized change (change in 12-mo score/SD of change) for the 2 groups (AD vs. MCI) for the ADAS-cog11 and 12. The effect on standardized change resulting from the inclusion of DR in the ADAS-cog was computed by the following ratio: difference in standardized change between ADAS-cog11 and 12 divided by the standardized change in ADAS-cog11. Hence, a positive increase in percent standardized change indicates a gain in precision.

A second set of analyses similar to the one above used only those subjects with overlapping MMSE scores (ie, from 23 to 27) from each group to address the question of whether the differential utility of ADAS-cog11 versus ADAS-cog12 could be ascribed to diagnosis (AD vs. MCI). To further examine common reporting methods in clinical trials, we used longitudinal random coefficients regression models¹¹ as a sensitivity analysis of the difference in standardized change between ADAS-cog11 and 12.

Finally, we calculated estimated sample sizes to compare the mean change from baseline between 2 groups for a given effect size (33%) at 80% power for ADAS-cog11 and 12 under 2 conditions of the correlation between baseline and 12-month score (observed ρ from the 2 trials and a ρ = 0.8). All statistical analyses were conducted using R, version 2.6.2 (www.r-project.org).

RESULTS

The AD and MCI placebo groups of the 2 trials consisted of n = 111 and n = 259 subjects, respectively, of which n = 79 (71%) AD subjects and n = 192 (74%) MCI subjects completed the 12 months ADAS-cog assessment. For the overall samples, the AD group had a lower MMSE and higher CDR and CDRsb with longer disease duration. Age, sex, and education did not differ between the MCI and AD groups (Table 1).

TABLE 1.

Demographic and Clinical Features of Subjects From 2 Cohorts

Measure	AD (N = 111)	MCI (N = 259)	AD (MMSE: 26-24) (N = 30)	MCI (MMSE: 27-24) (N = 79)
Age (y; SD)	73.8 (8.0)	72.9 (7.6)	73.1 (7.2)	75.2 (6.9)
Sex (%female)	62 (56%)	121 (47%)	14 (47%)	39 (49%)
APOE4 presence (%)	73 (68%)*	136 (53%)*	19 (63%)	46 (58%)
Education (y; SD)	14.3 (3.3)	14.7 (3.1)	15.1 (2.9)*	13.5 (3.2)*
Education (y; %)
<12	14 (12.6)	27 (10.4)	3 (10)	14 (17.7)
12–15.9	47 (42.3)	99 (38.2)	11 (36.7)	36 (45.6)
≥16	50 (45.1)	133 (51.4)	16 (53.3)	29 (36.7)
Age of onset (y; SD)	69.8 (8.3)	69.9 (8.0)	69.6 (7.3)	72.6 (7.0)
Baseline MMSE (SD)	20.8 (3.6)*	27.4 (1.8)*	25.0 (0.9)	25.1 (0.8)
CDR
0.5	44 (40%)*	221 (100%)*	22 (73%)*	66 (100%)*
1	56 (51%)	0 (0%)	8 (27%)	0 (0%)
2	11 (10%)	0 (0%)	0 (0%)	0 (0%)
CDR-SOB	5.5 (2.5)*	1.9 (0.8)*	3.8 (1.4)*	2.3 (0.9)*

^*Significantly different between diagnoses at the P < 0.05 level.

AD indicates Alzheimer disease; CDR, Clinical Dementia Rating; MCI, mild cognitive impairment; MMSE, Mini-Mental Status Examination; SOB, sum of boxes.

Complete Sample

Table 2 provides the ADAS-cog11, 12, and DR scores at baseline and 12 months as well as the change scores for both the MCI and AD groups. As expected, ADAS-cog11 and 12 scores at baseline were lower (ie, better performance) in the MCI group (t = 14.1, P < 0.001 and t = 15.3, P < 0.001, respectively). At baseline, 52% of the AD group and 9% of the MCI group had 10 (floor performance) on the DR score. ADAS-cog11 and 12 unadjusted change scores for MCI group were also smaller than those for the AD group (t = 4.26, P < 0.001 and t = 3.89, P < 0.001, respectively). The DR change score was not significantly different for the MCI and the AD group (MCI change 0.20 vs. AD change 0.12; t = −0.4496; P = 0.654). The impact of adding DR to the ADAS-cog11 score is reflected in the comparison of the calculated standardized changes for the unadjusted and adjusted (by the LOCF method) at 12 months. For AD and MCI subjects, the unadjusted and LOCF adjusted scores were very similar (Table 2).

TABLE 2.

ADAS-cog and DR Baseline, 12mo and Change Scores

	All Patients		Patients with MMSE: 26-24
	AD (N = 111)	MCI (N = 259)	AD (N = 30)*	MCI (N = 79)†
Baseline ADAS-cog11	24.34 (9.5)	11.03 (4.2)	16.45 (6.1)	13.58 (4.3)
Baseline DR	8.93 (1.6)	6.20 (2.2)	7.83 (2.0)	7.43 (2.0)
Baseline ADAS-cog12	33.27 (10.3)	17.22 (5.9)	24.28 (7.0)	21.01 (5.7)
unadjusted
12mo ADAS-cog11	26.42 (11.4)	11.46 (6.1)	18.83 (8.1)	15.14 (6.1)
12mo DR	8.96 (1.9)	6.46 (2.44)	7.73 (2.7)	8.02 (1.9)
12mo ADAS-cog12	35.30 (12.5)	17.90 (7.90)	26.56 (9.9)	23.16 (7.1)
LOCF adjusted
12mo ADAS-cog11	27.43 (11.9)	11.40 (5.7)	18.62 (8.0)	14.68 (5.7)
12mo DR	9.07 (1.8)	6.48 (2.4)	7.78 (2.7)	7.81 (2.1)
12mo ADAS-cog12	36.43 (12.8)	17.86 (7.6)	26.40 (9.8)	22.48 (6.9)
unadjusted
Change ADAS-cog11	4.53 (7.7)	0.60 (4.2)	2.40 (8.4)	1.66 (4.7)
Change DR	0.12 (1.3)	0.20 (1.70)	−0.04 (1.9)	0.54 (1.6)
Change ADAS-cog12	4.61 (8.1)	0.78 (4.8)	2.36 (9.2)	2.19 (5.1)
LOCF adjusted
Change ADAS-cog11	4.09 (7.4)	0.40 (4.1)	2.41 (8.2)	1.06 (4.6)
Change DR	0.15 (1.3)	0.25 (1.8)	0.04 (1.9)	0.40 (1.81)
Change ADAS-cog12	4.21 (7.7)	0.64 (4.8)	2.45 (9.0)	1.43 (5.2)

^*AD cases with MMSE scores 24 and up.

^†MCI cases with MMSE score 24 to 26.

AD indicates Alzheimer disease; ADAS-cog; Alzheimer Disease Assessment Scale-cognitive subscale; CDR, Clinical Dementia Rating; DR, delayed recall; LOCF, last observation carried forward; MCI, mild cognitive impairment; MMSE, Mini-Mental Status Examination.

The standardized change at 12 months for AD subjects on the ADAS-cog11 was 0.589 (4.53/7.69), which was similar to the standardized change for the ADAS-cog12 of 0.569 (4.61/8.10). Thus, the addition of DR did not enhance the standardized change in the AD group. Conversely, for the MCI cohort, the standardized change at 12 months on the ADAS-cog11 was 0.142 (0.60/4.24) and on the ADAS-cog12 was 0.16 (0.78/4.88), indicating an increase in standardized change of 12%. Standardized changes for the LOCF-imputed 12-month scores indicated that addition of DR was associated with a 1.6% decrease in standardized change in the AD group, but resulted in a 37.5% increase in standardized change for the MCI group (Table 3). Sensitivity analysis using longitudinal random coefficients regression models gave similar results.

TABLE 3.

ADAS-cog: Standardized Change for 1-year Change by Diagnosis, Unadjusted, and LOCF Adjusted Scores

	One Year Change (Unadjusted)		One Year Change (LOCF Adjusted)
	AD	MCI	AD	MCI
MMSE (complete range)
ADAS-cog11	0.589	0.142	0.553	0.096
ADAS-cog12	0.569	0.16	0.544	0.132
Standardized change: ADAS-cog12 versus 11 (%)	−3.40	12.68	−1.63	37.50
MMSE (24–26)
ADAS-cog11	0.287	0.35	0.295	0.231
ADAS-cog12	0.258	0.429	0.273	0.274
Standardized change: ADAS-cog12 versus 11 (%)	−10.10	22.57	−7.46	18.61

AD indicates Alzheimer disease; ADAS-cog; Alzheimer Disease Assessment Scale-cognitive subscale; LOCF, last observation carried forward; MCI, mild cognitive impairment; MMSE, Mini-Mental State Examination.

Overlapping MMSE Range

The comparisons were repeated in the subgroups of AD and MCI subjects with overlapping MMSE scores (those with scores of 23 to 27). Baseline characteristics of these subgroups are also presented in Table 1. There was a significant difference in mean years of education between the 2 groups (t = 2.40; P = 0.05); the range for AD subjects was 11 to 20 years whereas the range for the MCI cohort was 3 to 20 years, but the distribution of subjects with < 12 years, 12 to 16 and ≥ 16 years of education did not differ by cohort. AD subjects scored higher (ie, had worse performance) on the CDRsb and higher on the ADAS-cog11 and 12 than their MCI counterparts.

At baseline 24% of the AD group and 20% of the MCI group had floor performance on the DR score. Of the n = 30 AD subjects at baseline, n = 27 (90%) had assessments at 12 months. Of the 79 MCI subjects at baseline, 56 (71%) subjects had assessments at 12 months. When the percentage gain in standardized change for the ADAScog12 compared with the ADAS-cog11 was computed for these subgroups, the standardized change again appeared to decrease for AD subjects (10.1%), whereas increasing for MCI subjects (22.6%).

Imputing the 12-month scores with LOCF methods for this subset resulted in 27 (90%) of the AD and 73 (92%) of the MCI subjects being included in the analyses. As with the overall sample, unadjusted and LOCF imputed scores remained relatively unchanged for ADAS-cog11 and 12. The gain in standardized change achieved by adding DR remained near 20% for the MCI cohort, and again there was a net loss in standardized change for the AD subjects (7.5%).

We conducted a sensitivity analysis using the longitudinal random coefficients regression model to further examine the impact of diagnosis on rate of change. Of the baseline measures that differed by cohort, only CDRSb was found to be related to each ADAS-cog outcome at the P < 0.15 level. However, because the CDR is directly related to subject diagnosis, it was not included in the model. Using the final models with diagnosis, time and the diagnosis by time interaction, rate of decline on any score (ADAS-cog11, DR, or ADAS-cog12) did not differ by diagnosis (all P values > 0.13). Sensitivity analysis using longitudinal random coefficients regression models gave standardized changes similar to those reported above (data not shown).

Impact of Apoliprotein E4 Allele

In general, there is no difference in effect, adjusting for ApoE4 status in ADAS-cog 11 or 12. There is a statistically significant interaction in DR change at 12 months between the group and ApoE4 status such at in the MCI group, 12-month change is greater in the E4+ than in the E4− group (0.49 vs. −0.14; P = 0.0125). There was no effect of ApoE4 status in the AD group.

Impact on Sample Size

Finally we evaluated the impact of these different standardized change scores on sample size estimates for typical effect sizes proposed in AD clinical trials. As can be seen from Table 4, using an estimate of correlation to be the observed correlation from the trial, the sample size was dramatically reduced for MCI group with the use of the ADAS-cog12 compared with the 11 with little to no effect in the AD group, even among those with high MMSE scores at baseline. If we assume a standard correlation for all groups (ρ = 0.8) among baseline and change scores, the ADAS-cog12 increases the sample size estimates in the AD groups whereas continuing to reduce it in the MCI groups. This information is provided graphically in Figure 1.

TABLE 4.

Sample Estimates for 12-month ADAS-cog Change Assuming 80% Power and Effect Size of 0.33

	Sample Size For Alzheimer		Sample Size For MCI
	(Observed ρ)	(Standard ρ)	(Observed ρ)	(Standard ρ)
MMSE (complete range)
ADAS-cog11	(0.71) 218	(0.8) 168	(0.72) 3877	(0.8) 2770
ADAS-cog12	(0.76) 214	(0.8) 179	(0.79) 2248	(0.8) 2141
Sample size difference: ADAS-cog12 versus 11	−4	11	−1629	−629
MMSE (24–26)
ADAS-cog11	(0.33) 2379	(0.8) 710	(0.64) 826	(0.8) 459
ADAS-cog12	(0.47) 2315	(0.8) 874	(0.70) 473	(0.8) 316
Sample size difference: ADAS-cog12 versus 11	−64	164	−353	−143

ADAS-cog indicates Alzheimer Disease Assessment Scale-cognitive subscale; MMSE, Mini-Mental State Examination.

FIGURE 1.

Power versus sample size for 12-month Alzheimer Disease Assessment Scale (ADAS-cog) change by instrument and observed versus standard correlation (effect size of 0.33). AD indicates Alzheimer disease; MMSE, Mini-Mental State Examination.

DISCUSSION

These results suggest that DR is relatively insensitive to change over a 1-year interval in subjects with AD. This is true even among those with mild AD as reflected in high MMSE scores. In subjects with AD, scores at the start of the study were significantly worse than those of the MCI group and the DR scores were near the floor, leaving little room for decline over the year. The addition of the DR item within the ADAS-cog12 served mainly to increase variance and resulted in a reduction in standardized change for the AD group in both unadjusted and adjusted analyses. By contrast, in the MCI group, the addition of DR within the ADAS-cog12 provided an increase in the standardized change over 1 year. This increase was apparent in both unadjusted and adjusted scores, and even in those MCI cases with MMSE scores overlapping with AD cases. These results suggest that the benefit provided by adding DR is associated with the MCI diagnosis rather than with the severity of mental status scores and it supports the notion of adding DR items in longitudinal studies of MCI groups to improve detection of overall cognitive change measures over time.

Even with the enhancement of the DR, very large sample sizes are required to measure change in the MCI group. This is particularly noticeable in those who had high MMSE scores. Perhaps, better measures of cognitive status are needed in this very minimally impaired group. It is also noteworthy that the MCI group was less educated than the AD group and the largest percent of individuals with less than high school completion occurred in MCI group with low MMSE scores (24 to 26). This may be associated with misdiagnosis in this subgroup that could contribute to the appearance of less decline than would be expected in a purer MCI group.

One of the obvious benefits of improved sensitivity and larger standardized changes is the ability to maintain statistical power to see a treatment difference with a smaller sample size. In the case of MCI we observe a standardized change increase from 13% to 38% depending on whether unadjusted or adjusted change scores are used. The DR item requires very little time and the improved sensitivity could translate to a savings in sample size of significant proportions, thereby providing improved study efficiency in this population. Conversely and of equal importance is the observation that DR may actually reduce sensitivity in subjects with AD.

These estimates were based on a single study for each population. However several reports support the general observation. For example, the original report on the DR item indicated that while 1-year change was observed in the mildest MMSE group the variance was quite high.^12,13 In one study, mean DR scores of a selective reminding test in patients with incident AD was below 2 out of 12, leaving little room to see deterioration over time.¹⁴

In summary DR may be useful in the longitudinal study of subjects with MCI. However, there is no evidence that it is useful in even mild AD. These results highlight the need to carefully select sensitive and relevant outcomes specific to the population under study, to maximize efficiency and economy in conducting clinical trials.