Using baseline cognitive severity for enriching Alzheimer's disease clinical trials: How does Mini-Mental State Examination predict rate of change?

Abstract

Background

Post hoc analyses from clinical trials in Alzheimer's disease (AD) suggest that more cognitively impaired participants respond differently from less impaired on cognitive outcomes. We examined pooled clinical trials data to assess the utility of enriching trials using baseline cognition.

Methods

We included 2882 participants with mild to moderate AD in seven studies from a meta-database. We used mixed effects models to estimate the rate of decline in Alzheimer's disease Assessment Scale-cognitive (ADAS-Cog) scores among Mini-Mental State Examination (MMSE) groups.

Findings

Baseline MMSE category was associated with baseline scores and rate of decline on the ADAS-Cog, adjusting for age and education (both P < .001). Greater baseline cognitive impairment was associated with more rapid progression.

Interpretations

Although we found significant differences in rate of decline, most differences between individuals were from baseline ADAS-Cog values. Since enrichment based on MMSE would reduce the recruitment pool while adding only slightly to detecting differences in rate of progression, it is not advised.

1. Background

Given the lack of success in trials of potential disease-modifying and symptomatic agents for Alzheimer's disease (AD), experts have recommended enriching trials with groups that are more likely to respond [1]. Post hoc analyses from some AD clinical trials have shown more rapid progression of disease with more severe baseline impairment [2], suggesting the possibility of a differential response to treatment. This has led to recommendations for selecting trials participants based on initial severity, which is usually based on scores on the Mini-Mental State Examination (MMSE) [3] or the Alzheimer's Disease Assessment Scale-cognitive (ADAS-Cog) [4]. However, the results of post hoc analyses have not been consistent across trials [5].

Recently, Ito and colleagues developed a mathematical model of disease progression in AD based on meta-analysis of summary data from the literature [6] and individual-level data from the Alzheimer's Disease Neuroimaging Initiative (ADNI) [7], with this model receiving approval from the Food and Drug Administration and the European Medicines Agency for simulating clinical trials. This model found that baseline MMSE was closely associated with baseline ADAS-Cog scores and the rate of decline on the ADAS-Cog over time. However, the implications of enrichment based on MMSE scores were not investigated using this model.

We empirically tested the potential efficiency of these recommendations for enrichment based on MMSE scores by comparing the rate of progression across a broad range of baseline cognitive severities assessed by the MMSE, using a recently developed meta-database of studies from the Alzheimer's Disease Cooperative Study (ADCS) [8] and ADNI [9].

2. Methods

2.1. Study overview and participants

Participants for the analysis were drawn from a meta-database consisting of 18 ADCS studies and ADNI, representing both clinical trials and observational studies in AD, MCI, and normal individuals (National Institutes of Health grant R01 AG037561) [10]. Inclusion criteria for the present analysis were (1) diagnosis of dementia due to AD; (2) completed the MMSE at baseline; (3) completed at least one assessment on the ADAS-Cog. Of the 19 studies with 6553 participants, we excluded 6 studies that enrolled only participants with MCI or normal cognitive function, 1 study that did not collect baseline MMSE data, and 3 studies that did not collect ADAS-Cog data, leaving 10 studies meeting these inclusion criteria. Of the 2888 participants with AD in these 10 studies, 6 were excluded due to missing data, yielding a total of 2882 participants for analysis. All diagnoses were based on National Institute on Neurological Disorders and Stroke - Alzheimer's Disease and Related Disorders Association (NINDS-ARDRA) criteria [11], with the additional requirement of a minimal severity based on clinical ratings (Table 1). Participants for most of the trials analyzed could continue using marketed antidementia drugs if they had been on stable doses before entry.

Table 1.

Placebo-controlled and observational studies included in the analyses

Study (code), dates	Design	Intervention	N	Duration (months)	Minimal severity
Selegiline, vitamin E, 1993–1996 [12]	RCT, moderate to severe AD	Vitamin E, selegiline	341	24	CDR 2 or greater
Prednisone 1995–1998 [13]	RCT, mild to moderate AD	Prednisone	138	16	MMSE 13–26
Conjugated estrogens 1995–1999 [14]	RCT, mild to moderate AD	Conjugated estrogens	120	15	MMSE 12–28
Nonsteroidal anti-inflammatory 1999–2001 [15]	RCT, mild to moderate AD	Rofecoxib, naproxen	351	12	MMSE 13–26
Simvastatin (LL) 2003–2008 [16]	RCT, mild to moderate AD	Simvastatin	406	18	MMSE 12–26
Divalproex 2003–2009 [17]	RCT, moderate AD	Divalproex	313	24	MMSE 12–20
Vitamins B 2003–2007 [18]	RCT, mild to moderate AD	B vitamins	409	18	MMSE 14–26
Huperzine 2004–2007 [19]	RCT, mild to moderate AD	Huperzine A	210	6	MMSE 10–24
Docosahexaenoic acid 2006–2009 [20]	RCT, mild to moderate AD	Docosahexaenoic acid	402	18	MMSE 14–26
Alzheimer's Disease Neuroimaging Initiative (ADNI) 2005–2010 [9]	Observational, AD, MCI, normal	None	800 (192 AD, 398 MCI, 229 normal)	36 (AD)	MMSE 20–26

Abbreviations: RCT, randomized controlled trial; AD, Alzheimer's disease; CDR, Clinical Dementia Rating Scale; MMSE, Mini-Mental State Examination; MCI, mild cognitive impairment.

NOTE. Studies were drawn from the Alzheimer's Disease Cooperative Study (http://www.adcs.org) and the ADNI (http://adni.loni.ucla.edu) and included participants with dementia due to AD and baseline MMSE assessments.

2.2. Measures

Dementia severity was rated using the MMSE, which is a brief measure of cognition assessing orientation, attention, concentration, memory, visual construction, and language. Scores range from 0 to 30 points, with lower scores indicating greater impairment. MMSE scores of approximately 21 to 25 are consistent with mild dementia, 11 to 20 with moderate, and 0 to 10 with severe, although cutoffs vary by study [21]. For this analysis, baseline MMSE scores were broken a priori into categories of 0 to 10, 11 to 14, 15 to 18, 19 to 22, 23 to 26, and 27 to 30 to allow comparisons among finer degrees of impairment.

The primary outcome measure was the ADAS-Cog, a standard scale for AD clinical trials that evaluates memory, reasoning, orientation, praxis, language, and word finding difficulty. Scores range from 0 to 70 errors, with higher scores indicating greater impairment. Clinical assessments were done at 6-month intervals over the duration of each study (Table 1).

2.3. Statistical analysis

The primary analyses were conducted using a mixed effects linear model (random coefficients model) [22], which adjusts for missing data in testing for differences in the intercepts (baseline scores) and slopes (rate of change) of the ADAS-Cog between groups defined by MMSE categories. The mixed effects model was used as it analyzes data from all participants (rather than just completers) and minimizes bias, and it has better controls for type I error in the presence of missing data [23]. Trials with duration less than 24 months would still be used in the estimation of the slope parameter by contributing data at the time points where observations were collected. A model was constructed with group effect, visit effect, and group by visit interactions, with age and education as covariates. Thus, for participant $i=\mathrm{1,2},\mathrm{...},n$ at visit $j=\mathrm{1,2},\mathrm{...},n_i$, the model was

$$ADA{S}_{i,j}=ag{e}_i+educatio{n}_i+grou{p}_i+tim{e}_{i,j}+grou{p}_i\times tim{e}_{i,j}+{\epsilon }_{i,j}$$

which includes both fixed effects of time at the group level and random effects of time at the individual level. An unstructured covariance matrix was used to model the independence of the slope and intercept parameters. Parameters were estimated using restricted maximum likelihood. The primary test of interest was the significance of the group by time interaction, which would indicate that the slopes differed by MMSE category. Analyses were performed using version 3.0.3 of the R programming environment [24]. Mixed model analyses were performed using version 3.1-118 of the NLME package for R [25].

3. Results

Participants with lower baseline MMSE categories tended to be older, less educated, and more likely to be female than those in higher categories (Table 2). Baseline MMSE category was strongly associated with baseline ADAS-Cog scores (P < .001). This association remained significant after adjusting for age and education (P < .001; Table 3 and Fig. 1). The rates of decline on the ADAS-Cog also showed significant differences overall (P < .001), although the slope change from one group to the next higher group was small relative to the differences in baseline scores (Fig. 1). Participants in lower (more impaired) MMSE categories at baseline showed greater rates of progression on the ADAS-Cog than participants in higher categories.

Table 2.

Clinical characteristics and ADAS-Cog ratings among participants with dementia due to AD based on MMSE category status

MMSE category		0–10	11–14	15–18	19–22	23–26	27–30	P value
N (total = 2808)		122	307	610	884	829	56
Age	2793	70.9 ± 8.7	74.1 ± 8.9	75.6 ± 8.2	76.1 ± 8.2	74.7 ± 7.6	73.2 ± 8.8	<.001
Education, % <high school	2799	28% (34)	18% (56)	17% (105)	14% (125)	9% (74)	7% (4)	<.001
Hispanic (%)	2799	4% (5)	7% (20)	6% (34)	4% (37)	3% (27)	4% (2)	.17
Married (%)	2808	75% (91)	74% (226)	69% (420)	67% (593)	77% (636)	77% (43)	<.001
Caucasian (%)	2799	90% (110)	88% (269)	88% (537)	92% (815)	91% (752)	93% (52)	.059
Female (%)	2799	65% (79)	65% (200)	60% (367)	61% (536)	51% (423)	59% (33)	<.001
Assigned to placebo, %	2808	20% (24)	39% (120)	41% (249)	46% (411)	48% (398)	54% (30)	<.001
ADAS-Cog (SD)
Baseline	2808	47.9 ± 8.6	38.1 ± 8.0	31.2 ± 7.6	23.4 ± 6.8	17.2 ± 5.7	13.5 ± 4.5	<.001
6 months	2165	50.5 ± 9.2	40.4 ± 8.8	34.1 ± 8.6	25.5 ± 7.8	18.6 ± 6.8	14.7 ± 5.3	<.001
12 months	1890	54.8 ± 8.2	44.4 ± 9.7	36.9 ± 10.0	27.8 ± 8.8	20.2 ± 7.6	17.0 ± 6.3	<.001
18 months	1103	55.3 ± 8.1	48.0 ± 10.0	39.0 ± 10.9	29.8 ± 10.0	21.9 ± 9.0	19.0 ± 6.2	<.001
24 months	343	58.0 ± 6.9	49.6 ± 8.8	41.7 ± 10.9	33.4 ± 11.0	24.6 ± 10.1	–	<.001

Abbreviations: ADAS-Cog, Alzheimer's disease Assessment Scale-Cognitive; AD, Alzheimer's disease; MMSE, Mini-Mental State Examination; SD, standard deviation.

NOTE. Summaries are presented as mean (SD) for continuous variables and percentage (number) for categorical variables. Items showing statistically significant differences among MMSE categories are highlighted in bold.

Table 3.

Mixed effects (random coefficients) model of ADAS-Cog change over time by MMSE category

	Unadjusted			Adjusted
	Estimate	Standard error	P value	Estimate	Standard error	P value
Intercept	47.87	0.618	<.001	47.40	1.325	<.001
Age, yrs	–	–	–	0.001	0.016	.944
Education
Less than HS	–	–	–	–	–	–
HS graduate	–	–	–	0.49	0.388	.203
College graduate	–	–	–	0.66	0.408	.106
Time, yrs	9.96	0.763	<.001	9.95	0.763	<.001
MMSE category
0–10	–	–	–	–	–	–
11–14	−9.90	0.730	<.001	−9.97	0.733	<.001
15–18	−16.73	0.677	<.001	−16.80	0.683	<.001
19–22	−24.64	0.659	<.001	−24.74	0.667	<.001
23–26	−30.83	0.661	<.001	−30.96	0.670	<.001
27–30	−34.45	1.095	<.001	−34.59	1.100	<.001
MMSE category × time
0–10	–	–	–	–	–	–
11–14	−1.82	0.871	.036	−1.82	0.871	.037
15–18	−2.76	0.810	<.001	−2.75	0.810	<.001
19–22	−4.54	0.793	<.001	−4.53	0.793	<.001
23–26	−6.31	0.793	<.001	−6.30	0.793	<.001
27–30	−6.73	1.130	<.001	−6.72	1.130	<.001

Abbreviations: ADAS-Cog, Alzheimer's Disease Assessment Scale-cognitive; HS, high school; MMSE, Mini-Mental State Examination.

NOTE. Estimates for slopes represent annual rates of change. The rate of change for each group would be the sum of the reference estimate (9.96 points/year for the most severe group) plus the estimate for the interaction, so that negative interaction terms indicate slower progression. Baseline ADAS-Cog scores for all categories were significantly different from the reference category of 0 to 10. Rates of progression were significantly different for all categories.

Fig. 1.

Predicted Alzheimer's Disease Assessment Scale-cognitive (ADAS-cog) score by Mini-Mental State Examination (MMSE) category. Slopes (rate of change) were significantly different across MMSE category (all P < .05). However, most of the differences in ADAS-Cog scores at the end of the trial were due to differences in baseline scores, as shown by the wide separation of groups at month 0.

Differences between groups at the conclusion of the trial reflected both differences in the baseline scores on the ADAS-Cog and differences in the rate of decline over the duration of the trial, with the former having a greater contribution than the latter. For example, the mean difference in ADAS-Cog scores between the lower (MMSE 15–18) and higher (MMSE 23–26) end of most clinical trials after 24 months was 21.2 points (Table 4). Of this, 14.1 points were due to differences present at baseline, leaving 7.1 points due to differences in slope over the trial period.

Table 4.

Estimated group means on the ADAS-Cog by MMSE category and follow-up assessment time

MMSE category	0 months	6 months	12 months	18 months	24 months
0–10	47.87	52.85	57.82	62.80	67.78
11–14	37.97	42.04	46.11	50.17	54.24
15–18	31.14	34.74	38.34	41.94	45.54
19–22	23.23	25.94	28.65	31.36	34.07
23–26	17.04	18.87	20.69	22.52	24.34
27–30	13.42	15.04	16.65	18.27	19.88

Abbreviations: ADAS-Cog, Alzheimer's Disease Assessment Scale-cognitive; MMSE, Mini-Mental State Examination.

4. Discussion

These analyses support prior observations that participants in AD clinical trials show different ADAS-Cog outcomes based on initial severity on the MMSE. However, these differences at the end of trials are primarily due to differences in baseline ADAS-Cog scores. Significant differences were observed in the slopes, or rate of progression, based on initial MMSE severity, but the magnitude of these changes was considerably smaller than the difference in baseline ADAS-cog scores. Notably, individuals with greater baseline levels of cognitive impairment (as measured by the MMSE) had greater rates of progression on the ADAS-Cog than individuals with lesser baseline cognitive impairment. These results are consistent with the results of the disease progression models of Ito and colleagues [6]^, [7], although they did not report individual-level analyses apart from ADNI.

These findings have significant implications for recommendations to enrich AD clinical trials based on initial severity. Enrichment using this criterion would select individuals with greater severity as being more likely to progress, which would be consistent with the larger slopes (and greater potential for slope reduction) seen in our analysis. However, such an approach runs counter to current approaches targeting individuals with lesser severity as having less neuropathology and being more likely to respond to treatment [26], which would be consistent with the larger contribution of baseline ADAS-Cog scores to the end-of-trial score than the change due to slope differences. These opposing recommendations highlight the potential limitations with analysis of observational studies and post hoc failed therapeutic trials, which cannot give a definitive depiction of the effects of a successful treatment.

Enrichment based on baseline severity would have adverse consequences by shrinking the recruitment pool for a clinical trial, reducing efficiency by requiring a longer enrollment period, more clinical sites, or increased recruitment effort, without a clear gain in efficacy by targeting likely responders or greatly increasing the rates of decline. As examples of this, we used the meta-database to examine the placebo arm of a clinical trial restricted to more severe samples and to less severe samples (Box 1). Enrichment based on MMSE status had only a small effect on the annual rate of change of the ADAS-Cog, and at the expense of excluding a large number of subjects. Enriching for baseline MMSE scores of 12–22 instead of 12–26 resulted in a difference of less than one point per year on the ADAS-Cog, but reduced the available sample pool by more than 800 subjects. Enrichment based on less severe MMSE scores resulted in a slower rate of progression on the ADAS-Cog compared with unenriched samples. Such results indicate that investigators should consider prospectively stratifying trials based on cognitive severity rather than enriching based on initial cognitive severity when attempting to improve AD trial design.

Box 1. Effects of restricting recruitment using baseline cognition as measured by the Mini-Mental State Examination (MMSE). When enriching for more severe disease using baseline MMSE, a slightly larger slope (rate of decline) was observed but offset by a large reduction in the size of the available sample in the meta-database meeting the MMSE cutoff. Enrichment for less severe illness resulted in a slightly smaller slope, as well as a large reduction in size of the available sample. Therefore restricting the upper range of MMSE scores is more efficient than restricting the lower range. Annual rate of change represents the slope of the mixed effects model fitted to the sample with the specified baseline MMSE range, while sample size is the number of subjects in the meta-database with the specified baseline MMSE range.

	Enrichment for more severe		Enrichment for less severe
	Baseline MMSE		Baseline MMSE
	12–26	12–22	16–26	20–26
ADAS-Cog annual rate of change	5.33	6.27	5.01	4.33
Size of available sample	2587	1767	2160	1505
Duration	12 months		18 months

Abbreviations: MMSE, Mini-Mental State Examination; ADAS-Cog, Alzheimer's Disease Assessment Scale.

Our analysis has several notable strengths, including the large sample size and the inclusion of a diverse sample of clinical trials across multiple sites. However, some limitations must be acknowledged. Our meta-database of clinical trials spans more than two decades, and secular changes in the conduct of clinical trials over this time frame could have affected results. Also, we only examined the MMSE as a measure of cognitive severity for predicting progression. More detailed assessment using neuropsychological testing may be able to identify individuals who will experience more rapid decline that the MMSE could not. However, such in-depth testing is usually not conducted until after the identification of potential participants is completed, and brief screening measures such as the MMSE are used to make initial classifications of severity. Any potential benefits of neuropsychological testing to enrich clinical trials samples must be balanced against the increased time and participant burden required in a screening process that includes such measures.

Although the goal of directing trials of therapies in AD toward individuals who are more likely to respond is a reasonable one, researchers must also consider whether the exclusion of potential participants not meeting the enrichment criteria would adversely affect the efficiency of the trial [27]. The use of baseline cognitive severity, as measured by the MMSE, serves as a predictor of disease progression in mild to moderate AD but may not predict treatment response. As such, further research to demonstrate its use is needed, and it cannot be recommended for enriching clinical trials.

Research in context.

• 1.Systematic review: We reviewed existing literature on the relationship between baseline cognitive severity in Alzheimer's disease (AD) clinical trials and subsequent rate of decline, which would suggest the former could be used to enrich clinical trials for individuals more likely to show therapeutic response. Several previous studies have shown an association between more severe cognitive impairment and rate of decline, but there are inconsistencies among these reports. To provide a more comprehensive picture, we analyzed data from seven studies in a meta-database of AD clinical trials and observational studies.
• 2.Interpretation: Our results confirm more severe baseline cognitive impairment is associated with more rapid progression of AD.
• 3.Future directions: Our results provide evidence that more, rather than less severe AD may be likely to respond to treatment. Clinical trials in AD should not exclude participants based on the severity of cognitive impairment, but include the severity as a moderator of treatment effects in analysis.