Mayo-PACC: a parsimonious Preclinical Alzheimer’s Disease Cognitive Composite comprised of public-domain measures to facilitate clinical translation

Abstract

Introduction:

We aimed to define a Mayo Preclinical Alzheimer’s disease Cognitive Composite (Mayo-PACC) that prioritizes parsimony and use of public domain measures to facilitate clinical translation.

Participants and Methods:

Cognitively unimpaired participants aged 65-85 at baseline with amyloid PET imaging were included, yielding 428 amyloid negative (A−) and 186 amyloid positive (A+) individuals with 7 years mean follow-up. Sensitivity to amyloid-related cognitive decline was examined using slope estimates derived from linear mixed models (difference in annualized change across A+ and A− groups). We compared differences in rates of change between Mayo-PACC and other composites (A+>A− indicating more significant decline in A+).

Results:

All composites showed sensitivity to amyloid-related longitudinal cognitive decline (A+>A− annualized change p<.05). Comparisons revealed that Mayo-PACC (AVLT sum of trials 1-5+6+delay, Trails B, animal fluency) showed comparable longitudinal sensitivity to other composites.

Discussion:

Mayo-PACC performs similarly to other composites and can be directly translated to the clinic.

Introduction

Preclinical Alzheimer’s cognitive composites (PACC) are modeled to be sensitive to subtle cognitive changes in preclinical Alzheimer’s disease (AD; i.e., amyloid positive [A+] cognitively unimpaired [CU]) [1, 2] and are commonly used outcome measures in longitudinal studies and AD clinical trials [3]. PACCs may similarly help with disease monitoring and potentially increase cross-sectional sensitivity to objective subtle cognitive decline in clinical settings [4, 5]. However, application of many PACCs is challenging in typical clinical settings because of frequent inclusion of component measures that are no longer readily available for purchase from the test publisher (e.g., the WAIS-R / WMS-R versions of Digit Symbol Substitution Test [DSST] and Logical Memory). Because of this limited access, some consider these measures clinically obsolete despite their continued relevance in research settings (but see [6]). Further, many national studies including the Alzheimer’s Disease Neuroimaging Initiative (ADNI) and the National Alzheimer’s Disease Coordinator Centers (NACC) have moved away from inclusion of proprietary measures [7], making future cross-validation of PACCs that depend upon such measures challenging. With the recent FDA approval of an anti-amyloid therapy and other therapies for Alzheimer’s disease potentially on the horizon, however, there has been increased discussion of the future role of embedded pragmatic clinical trials to evaluate clinical efficacy in clinical settings that offer greater generalizability of results [8, 9]. In addition, if new treatments are approved for the preclinical AD stage, clinicians need tools to aid monitoring that perform similarly to those used in clinical trials [10]. Therefore, a cognitive composite that can bridge the gap between research and clinical use is needed [11].

The MCSA is a population-based study of aging among Olmsted County, Minnesota residents that is an important resource for understanding the prevalence and incidence of biomarkers of Alzheimer’s disease and mild cognitive impairment (MCI), and interrelationships among biomarkers and cognitive performance [12]. This community-based cohort with limited exclusionary criteria offers increased generalizability of study results to general clinical settings than cohort studies with more restrictive inclusion criteria. To date, the MCSA frequently uses a global composite score as the primary cognitive outcome measure, derived from an entire battery of neuropsychological measures. Unlike this global cognition approach, PACCs apply a data-driven weighted or theoretical multidomain summation approach to maximize sensitivity to amyloid-related cognitive decline by comparing rates of change in individuals with (A+) and without (A−) amyloid, with greater change expected in those with elevated brain amyloid (e.g., A+>A− decline over time) [13–15]. Slight variations in PACCs exist due to cohort-specific measures and population characteristics, but they typically include some combination of memory, processing speed, executive functioning, global cognition, and more recently semantic fluency measures [3, 13, 15–17]. Donohue et al. [14] explored application of optimized weighted scores for the Alzheimer Disease Cooperative Study PACC (ADCS-PACC), and components and did not show any benefit relative to no optimization upon cross-validation [14]. Adaptations have been developed for different stages of the AD continuum and asymptomatic autosomal dominant carriers [18–20]. However, developing a composite that is sensitive to both early/preclinical and more advanced disease stages is challenging, and this is likely compounded by use of measures with ceiling or floor effects [20, 21].

The MCSA was recently part of a consortium of multiple longitudinal cohort studies that, along with Biogen scientists, aimed to empirically formulate a data-driven revised PACC (PACC-R) optimized for global clinical trials [19]. Linear mixed effects models and least absolute shrinkage and selection operator (LASSO) regression [22] was used to test the sensitivity of numerous candidate measures to amyloid-related cognitive decline within four different study cohorts including the MCSA, ADNI, the Australian Imaging Biomarkers and Lifestyle Flagship Study of Aging (AIBL), and the Charles F. and Joanne Knight Alzheimer’s Disease Research Center at Washington University in St. Louis. Results were considered collectively by the group to derive several candidate PACC-Rs that were then compared within each cohort [19, 20], resulting in a PACC-R that included word list learning, delayed recall and recognition, category fluency, Trails B (or similar), and DSST. The consortium applied the same methods to derive an alternative early AD/MCI Alzheimer’s Cognitive Composite (EMACC) to be used in cohorts with more advanced disease and includes AVLT 1-5, category fluency, Trails A, Trails B, and DSST [20]. Selection of measures for both the PACC-R and EMACC emphasized measures suitable for global clinical trials, avoidance of ceiling effects (e.g., exclusion of Mini Mental Status Examination; MMSE), and known sensitivity to cognitive change in Alzheimer’s disease. Both are helpful alternatives for use in research. A noted benefit of both composites is their exclusion of Logical Memory and MMSE when compared to ADCS-PACC that includes these measures. Logical Memory is problematic due to strong practice effects and the significant challenges of successfully adapting story memory measures for cross-cultural use [23]. MMSE is helpful for disease monitoring in MCI and dementia stages but has limited utility for preclinical disease stages [21, 24].

The aims of this study were to 1) define a Mayo-PACC comprised of public-domain measures with consideration of parsimony and psychometric properties, and 2) compare the sensitivity of the Mayo-PACC for amyloid-related cognitive decline (A+>A− decline over time) with other composite outcome measures using data from the Mayo Clinic Study of Aging (MCSA) [12]. Our primary hypothesis was that the Mayo-PACC would show similar sensitivity to amyloid-related longitudinal cognitive decline compared to the PACC-R [19]. We also explored comparisons to an approximation of the ADCS-PACC and a global composite.

Method

Participants

MCSA participants are randomly sampled by age- and sex-stratified groups using the resources of the Rochester Epidemiology Project medical records-linkage system, which links the medical records from all county providers [25]. Exclusion criteria are a terminal illness or hospice. Over 60% of residents contacted enroll and follow-up retention is 80%. Participants complete study visits every 15 months and a subset agree to undergo amyloid PET imaging. Study visits include an interview and completion of the Clinical Dementia Rating^® scale by a study coordinator [26], a neurological examination with medical history review and administration of the Short Test of Mental Status (STMS) [27], and neuropsychological testing [12]. The interviewing study coordinator, examining physician, and neuropsychologist each make an independent diagnostic determination of cognitively unimpaired (CU), mild cognitive impairment (MCI) or dementia. Only the neuropsychologist has access to the neuropsychological data for this diagnostic determination, with consideration given to normative data [28]. A final diagnosis is then achieved through consensus agreement [12, 29].

Consent Statement

The study protocols were approved by the Mayo Clinic and Olmsted Medical Center Institutional Review Boards. All participants provided written informed consent.

Inclusion and neuroimaging criteria

Inclusion criteria for the current study included cognitively unimpaired status at baseline visit, availability of amyloid PET within 3 years of the first MCSA visit (scan closest to the first MCSA visit was used if there was more than 1), and age 65-85. This age group was selected to maximize the likelihood of cognitive decline over time and to match inclusion criteria of prior PACC derivation studies [13, 19]. At least two follow-up visits were required, and all PACC component measures and z-scores had to be available. All available follow-up data were used (average 6 total visits and 7 years of follow-up). Amyloid positivity was defined as Pittsburgh Compound B PET (PiB-PET) meta-ROI SUVR ≥ 1.48 (centiloid 22)[30]. The meta-ROI includes prefrontal, orbitofrontal, parietal, temporal, anterior and posterior cingulate, and precuneus regions [31].

Composite scores

The four composites included in this manuscript and their respective component cognitive measures are listed in Table 1. Additional exploratory comparisons with additional composites are also provided in supplemental materials. Study-specific z-scores were derived for all component PACC measures using the A− group at baseline as the normative reference. PACCs were defined as the average of the z-scored test measurements. As in prior MCSA studies, z-scores for the Global-z composite represents the z-scores previously derived using CU participants 50 and older enrolled from 2004-2012, weighted by the age and sex distribution of the 2013 Olmsted County population [32, 33]. A priori selection of Mayo-PACC component variables prioritized parsimony and avoidance of measures not readily available for clinical practice (e.g., WAIS-R/WMS-R measures). Methods used to develop PACCs have been criticized for failing to consider psychometric properties, thus we also examined distributional properties, reliability, and practice effects as part of our process [3] (see statistical methods below). Variable selection was further informed by our prior experience applying linear mixed effects models and least absolute shrinkage and selection operator (LASSO) regression [22] to test the sensitivity of numerous candidate measures to amyloid-related cognitive decline as part of a collaborative effort, as described in the introduction [19, 20]. We avoided singular inclusion of delayed recall given that floor effects render measures of delayed recall less useful with progression of disease severity [20]. We instead used sum of trials for the AVLT (Trials 1-5 total + short delay recall + long delay recall) [34]. In line with our emphasis on public domain measures used in clinical settings, each measure in the Mayo-PACC is among the top 10 most frequently used instruments in the respective domain represented based on a survey of clinical neuropsychologists [35].

Table 1.

All composite scores (columns) and the component test score included in each (rows).

Measure	Mayo-PACC	PACC-R	ADCS-PACC1	Global-z2
AVLT Trials 1-5 total		X
AVLT Long delay recall		X	X	X
AVLT Sum of trials	X
AVLT Recognition hits		X
WMS-R Logical Memory II			X	X
WAIS-R Digit Symbol Substitution Test		X	X	X
Trails B seconds to completion	X	X		X
Category Fluency total				X
Category Fluency (Animals/Vegetables)		X
Animal Fluency	X
Boston Naming Test				X
MMSE from Short Test of Mental Status			X
WMS-R Visual Reproduction II				X
WAIS-R Picture Completion				X
WAIS-R Block Design				X

¹We derived an approximation of the ADCS-PACC [4], which required substituting measures not administered in the MCSA with similar measures of those constructs. The original ADCS-PACC has Free and Cued Selective Reminding Test total recall, WMS-R LM IIa (0-25), WAIS-R Digit Symbol Coding, MMSE. Our modified ADCS-PACC has AVLT long delay recall, WMS-R LM IIa+b (0-50), WAIS-R DSST, MMSE derived from Short Test of Mental Status.

²Global-z composite uses a single measure from each test administered in the MCSA. As in prior MCSA studies, z-scores for this composite represents the z-scores previously derived using CU participants 50 and older enrolled from 2004-2012, weighted by the age and sex distribution of the 2013 Olmsted county population [27, 28].

Note. ADCS: Alzheimer Disease Cooperative Study; AVLT: Rey Auditory Verbal Learning Test; MMSE from Short Test of Mental Status: Mini Mental Status Examination calculated from the Kokmen Short Test of Mental Status; PACC: Preclinical Alzheimer’s cognitive composite; Sum of Trials: sum of Trials 1-5 total, short delay, and long delay; WAIS-R: Wechsler Adult Intelligence Scale-Revised; WMS-R: Wechsler Memory Scale-Revised. Category fluency is not administered during the time between AVLT learning trials (1-5 total) and AVLT 30-minute delay in the Mayo Clinic Study of Aging.

Statistical methods

Psychometric properties:

Psychometric properties of component measures and composites were examined descriptively [3]. Test-retest reliability was determined by computing Pearson bivariate correlation coefficients. Practice effects and magnitude of change over time were calculated by determining the paired difference of two consecutive time points. Cohen’s d effect sizes were calculated using pooled SDs. Distributional properties were evaluated by examining the frequency of ceiling and floor effects at baseline and all visits combined, as well as by skewness and kurtosis values for all visits combined.

Sensitivity to longitudinal cognitive decline:

Slope estimates for amyloid status were obtained using linear mixed models (LMMs). A separate model was fit for each of the composite measures. All models were adjusted for age at current visit, age² at current visit, education level (in years), practice effects, and sex. Previous studies have shown there is typically a substantial practice effect from the first to second visit as well as smaller practice effects after the second visit [36–39]. Therefore, we account for practice effects by a term for difference between visit 1 and subsequent visits (not visit 1 vs. visit 1) as well as terms for time (years) since visit 1 and time (years) since visit 1 squared [39]. Between subject variability was accounted for by random intercept and slope terms per individual. Normality assumptions of the LMMs and overall goodness of fit were assessed graphically. Amyloid-related rate of decline was included as a linear term in each model (Amyloid status x Time from Baseline Interaction). Comparison of sensitivity to cognitive decline between two composites was calculated as the difference in the standardized amyloid-related slope estimates (i.e., difference in slopes between A+ vs A− individuals) from each respective model. A jackknife procedure was used to calculate the standard error and subsequent confidence intervals of any differences in amyloid-related rates of change. The primary comparison was Mayo-PACC vs. PACC-R. Secondary comparisons of Mayo-PACC with additional composites were also completed (also see supplemental materials). Due to the a priori focus on one primary comparison and exploratory nature of remaining analyses, comparisons were not adjusted for multiplicity. Alpha level was set at 0.05. Significance was determined based on whether the 95% Confidence Interval (CI) contained 0.

Results

Participant characteristics

Demographic characteristics are presented in Table 2. The A+ group was, on average, one year older and more likely to carry at least one copy of the e4 allele compared to the A− group. The two groups showed comparable education, sex, baseline CDR score, number of follow-up visits (mean 6) and number of follow-up years (mean 7). Across all available follow-up visits, 82% (506/614) remained CU, with a higher proportion remaining CU in the A− group (87%) compared to the A+ (72%) group. Sample sizes across specific intervals are available in Supplemental Tables 1–2.

Table 2.

Demographic characteristics and performance on component neuropsychological measures (raw) and composites (z) at baseline by biomarker group. All participants are cognitively unimpaired at baseline. Statistics reported are of the form mean (standard deviation, SD) or n (%).

	A− (n=428)	A+ (n=186)	p-value
Age, years	72.47 (4.81)	73.79 (5.03)	0.0021
Education, years	14.86 (2.57)	14.52 (2.66)	0.1361
Male sex	233 (54.4%)	102 (54.8%)	0.9272
APOE ε4 carrier	89 (21.0%)	88 (47.6%)	< 0.0012
CDR global score			0.0972
0	416 (97.7%)	175 (95.1%)
0.5	10 (2.3%)	9 (4.9%)
PiB SUVr	1.36 (0.07)	1.87 (0.39)	< 0.0011
Visit to PiB scan, years	0.92 (0.89)	1.07 (0.98)	0.0621
Number of MCSA visits	6 (2)	6 (2)	0.4401
Follow up, years	7.23 (2.70)	7.13 (3.02)	0.6921
Mayo-PACC, z-score	0.000 (0.745)	−0.106 (0.631)	0.0921
PACC-R, z-score	0.000 (0.676)	−0.080 (0.611)	0.1661
ADCS-PACC3, z-score	0.000 (0.694)	−0.118 (0.661)	0.0501
Global-z	0.062 (0.855)	−0.127 (0.813)	0.0111
MMSE from Kokmen STMS	28.42 (1.15)	28.32 (1.23)	0.3401
Kokmen STMS (original scale)	35.32 (2.06)	35.21 (2.16)	0.5391
AVLT Trials 1-5 Total	41.63 (9.16)	40.63 (8.57)	0.2041
AVLT Long delay recall	7.85 (3.42)	7.52 (3.23)	0.2581
AVLT Sum of Trials	57.75 (14.68)	56.13 (13.84)	0.2041
AVLT Recognition hits	12.80 (2.25)	12.77 (2.37)	0.8831
AVLT Recognition % correct	89.57 (9.05)	88.72 (11.52)	0.3301
WMS-R Logical Memory I total	23.75 (6.62)	23.44 (6.56)	0.5881
WMS-R Logical Memory II total	18.96 (7.42)	18.03 (7.02)	0.1461
WMS-R Visual Reproduction I	31.02 (5.03)	30.12 (4.74)	0.0381
WMS-R Visual Reproduction II	23.66 (7.96)	22.28 (7.94)	0.0481
TMT A total, seconds	34.87 (11.16)	36.89 (12.53)	0.0481
TMT B total, seconds	87.61 (40.96)	89.01 (31.91)	0.6781
WAIS-R Digit Symbol Coding	46.96 (9.89)	45.34 (9.44)	0.0591
Animal Fluency	20.27 (5.00)	19.40 (4.33)	0.0401
Fruit Fluency	13.25 (3.69)	12.95 (3.59)	0.3581
Vegetable Fluency	12.45 (3.72)	12.57 (3.45)	0.7101
Category Fluency total	45.97 (9.96)	44.92 (9.01)	0.2201
Boston Naming Test	55.95 (3.42)	55.33 (3.95)	0.0491
WAIS-R Picture Completion	14.00 (2.80)	13.62 (2.92)	0.1311
WAIS-R Block Design	26.87 (8.35)	24.92 (8.16)	0.0071

¹Linear Model ANOVA

²Pearson’s Chi-squared test

³Approximation of the ADCS-PACC.

Note. Composites are defined in Table 1. ADCS: Alzheimer Disease Cooperative Study; AVLT: Rey Auditory Verbal Learning Test; MMSE from STMS: Mini Mental Status Examination calculated from the Kokmen Short Test of Mental Status; PACC: Preclinical Alzheimer’s cognitive composite; Recognition % Correct: (true positives + true negatives)/total; Sum of Trials: sum of Trials 1-5 total, short delay, and long delay; TMT = Trail Making Test; WMS-R: Wechsler Memory Scale-Revised; WAIS-R: Wechsler Adult Intelligence Scale-Revised.

Psychometric properties of composites and component measures

Reliability.

All 4 composite measures had test-retest reliability coefficients ≥ .80 across all consecutive follow-up visits (see Figure 1 and Supplemental Table 1). Composites that included more component measures showed higher reliability coefficients (e.g., Global-z).

Figure 1:

Test-retest reliability (Panel A Pearson correlation coefficients) and magnitude of change (Panel B Cohen’s d effect sizes) for each composite (top panel) and test components for PACCs (bottom panel) for sequential visit pairs. Data from additional variables are available in supplemental materials.

Reliability coefficients for individual measures were generally highest for those with the best distributional properties (WAIS-R DSST, AVLT sum of trials). Measures with restricted ranges (ceiling effects) had the lowest reliability coefficients (MMSE from STMS, AVLT recognition). Time 1 to Time 2 reliability coefficients were generally slightly lower than subsequent visit pairs.

Magnitude of Change Across Visits.

Most composites, except Mayo-PACC, showed a significant practice effect (CI not containing 0) for the Time 1 to Time 2 comparison (Figure 1 and Supplemental Table 2). The magnitude of practice effects observed for the Time 1 to Time 2 comparison varied across composites and was larger for composites that included WMS-R Logical Memory (e.g., ADCS-PACC). Most composite CIs contained 0 for the Time 2 to Time 3 comparison suggesting stability, and remaining comparisons for later timepoints tended to show a subtle decline in these analyses that collapsed across A− and A+ groups.

Similarly, the magnitude of change observed for individual component measures varied, particularly for the Time 1 to Time 2 comparison (Figure 1 and Supplemental Table 2). The magnitude of practice effects observed from Time 1 to Time 2 were highest for memory measures, particularly WMS-R Logical Memory. Several measures did not show significant change for the Time 1 to Time 2 comparison (e.g., Trails B, Animal Fluency, Fruit Fluency, MMSE from STMS, BNT), whereas others show a small decline from Session 1 to Session 2 (e.g., Category Fluency total). Most practice effects were attenuated for the Session 2 to Session 3 comparison and beyond, showing stable or subtly declining performances.

Distributional Properties.

Several component measures showed evidence of non-normal distributions (e.g., ceiling effects, skewness and/or kurtosis) including the STMS (original and MMSE conversion), AVLT recognition, Trails B, and Boston Naming Test (BNT) (Supplemental Table 3). In contrast, composite scores showed good distributional properties despite inclusion of these component measures (Supplemental Table 4).

Model characteristics

Table 3 shows fixed effects estimates for predictor variables included in the candidate and 3 comparator models. Age at time of visit, age² at time of visit, sex, and education were statistically significant predictors of longitudinal performance in all models (all estimates p < 0.05). The practice effect term capturing test naïve status (not visit 1 vs. visit 1) was significant in the ADCS-PACC, PACC-R, and Global-z models, but not Mayo-PACC models, and the practice effect terms for time since visit 1 and time since visit 1 squared were significant for all models (Table 3). Graphical evaluation did not indicate that linearity assumptions were violated so no linear transformations were applied. A statistically significant difference in rate of decline between A− and A+ groups was found in all models [PiB status x Time from Baseline predictors (referenced to A−), p < 0.05]. Greater decline over time is observed for the A+ relative to the A− group (Figure 2). Model fit was acceptable for all composites; the correspondence between predicted values for a typical member of each group and smoothed longitudinal trends for all observed data provided additional support for model fit.

Table 3.

Standardized model estimates for the 4 mixed effects models.

	Mayo-PACC	PACC-R	ADCS-PACC1	Global-z
Fixed Effects	Estimate (95% CI)	Estimate (95% CI)	Estimate (95% CI)	Estimate (95% CI)
Age at visit, centered at 76.67	−0.436* (−0.505, −0.367)	−0.409* (−0.477, −0.341)	−0.412* (−0.481, −0.342)	−0.433* (−0.506, −0.359)
Age at visit^2	−0.091* (−0.128, −0.054)	−0.084* (−0.119, −0.049)	−0.087* (−0.124, −0.051)	−0.111* (−0.144, −0.077)
Sex (referenced to females)	−0.223* (−0.278, −0.168)	−0.39* (−0.445, −0.336)	−0.272* (−0.328, −0.216)	−0.165* (−0.223, −0.107)
Education, years	0.249* (0.194, 0.305)	0.25* (0.195, 0.305)	0.259* (0.202, 0.315)	0.318* (0.26, 0.377)
Not visit 1 vs. visit 1	0.002 (−0.019, 0.023)	−0.017* (−0.035, 0)	−0.026* (−0.045, −0.007)	−0.031* (−0.045, −0.016)
Years since visit 1	0.25* (0.171, 0.33)	0.261* (0.191, 0.331)	0.361* (0.285, 0.436)	0.26* (0.195, 0.325)
Years since visit 1^2	−0.188* (−0.258, −0.118)	−0.208* (−0.267, −0.149)	−0.24* (−0.306, −0.175)	−0.177* (−0.23, −0.125)
PiB status (referenced to A−)	0.008 (−0.046, 0.063)	0.01 (−0.044, 0.064)	−0.011 (−0.067, 0.044)	−0.018 (−0.076, 0.039)
PiB status x Time from Baseline	−0.132* (−0.18, −0.084)	−0.134* (−0.179, −0.089)	−0.154* (−0.201, −0.106)	−0.133* (−0.176, −0.09)
Random Effects	Mayo-PACC	PACC-R	ADCS-PACC	Global-z
SD of intercept	0.53	0.48	0.53	0.68
SD of residual	0.34	0.25	0.30	0.27
SD of slope	0.07	0.06	0.07	0.08
Correlation intercept & slope	0.26	0.20	0.21	0.25

^*p < .05

¹Approximation of the ADCS-PACC as defined in Table 1.

Note. PiB Status is A− when PiB SUVR is less than 1.48 and is A+ when PiB SUVr greater than or equal to 1.48. ADCS: Alzheimer Disease Cooperative Study; PACC: Preclinical Alzheimer’s Cognitive Composite; practice effects are modelled by a term for not visit 1 (e.g., visits 2+) vs. visit 1, as well as a term for time (years) since visit 1 and a term for time (years) since visit 1 squared; visit 1 is the baseline visit; the two age at visit terms (age at visit, centered and age at visit^2) capture longitudinal change, controlling for all other model terms.

Figure 2:

Model-derived trajectories of composite measures (solid lines, 95% CI as patterned gray shading) and observed trajectories (dashed lines, 95% CI as solid gray shading) from all participants separately depicted for A− (blue) and A+ (red) groups.

Sensitivity to longitudinal cognitive decline (A+>A− decline over time) – model comparisons

Comparisons of the difference in the standardized amyloid-related slope estimates between the candidate and comparator models are shown in Figure 3 and Supplemental Table 6. No statistically significant difference in the magnitude of amyloid-related rate of decline was observed between the Mayo-PACC and PACC-R models (primary comparison), or for exploratory comparisons (Mayo-PACC vs. ADCS-PACC and Mayo-PACC vs. Global-z). Additional exploratory comparisons similarly showed no difference in A+ vs. A− decline over time (PACC-R vs. ADCS-PACC, PACC-R vs. Global-z, ADCS-PACC vs. Global-z; see supplemental materials). Supplemental analyses also showed that the Mayo-PACC showed slightly better sensitivity to decline relative to composites with 2 measures or a single memory measure (see Supplemental Table 6).

Figure 3:

Pairwise comparisons of the standardized estimates of the annualized amyloid-related rate of decline for select composites.

Note. If the CI contains 0, the models are performing similarly in terms of their ability to separate A− and A+ groups over time (applies to all comparisons shown). If the difference is below 0, the first measure listed is separating A− and A+ groups more over time. If the difference is above 0, the second measure listed is separating A− and A+ groups more over time.

Results were similar when only 30-months of data were modeled (results not included).

Discussion

The Mayo-PACC is a psychometrically sound and parsimonious composite comprised of public-domain measures. As predicted, the Mayo-PACC was similar to the PACC-R in detecting amyloid-related cognitive decline. Mayo-PACC was also comparable to an approximation of the ADCS-PACC as well as a global z-score averaged across 9 measures. This is consistent with prior work comparing amyloid sensitivity across cognitive composites [40] and supports the relatively robust nature of composite scores comprised of varying component measures.

Neuropsychological tests show differential sensitivity to cognitive decline at different disease stages (i.e., preclinical, MCI, dementia), highlighting the potential need for composites sensitive to stage-specific cognitive phenotypes [16, 21]. In selecting measures for the Mayo-PACC, we prioritized preclinical change given that most participants in the MCSA are CU. Our use of AVLT sum of trials that combines learning trials 1-5 with both short and long-delay recall was intended to help avoid floor effects observed when using delayed recall alone in composite measures at later disease stages [21]. Semantic fluency and Trails B have also shown sensitivity across preclinical to mild AD dementia stages [21].

Prior derivations of PACCs have been criticized for a perceived failure to evaluate psychometric properties of composites and individual component measures [3]. Our results suggest robust distributional properties and reliability when 3 or more measures are combined. For example, although Trails B showed evidence of a mildly non-normal distribution, that was often offset by inclusion of other normally distributed variables in composites. Aside from the logistical challenge that limits its availability for purchase, the WAIS-R DSST is an ideal measure, with well-established psychometric properties and validity [41]. Thus, it was reassuring that our prioritization of Trails B (public-domain) over WAIS-R DSST did not negatively impact the overall psychometric properties of our chosen composite and did not significantly impact sensitivity to amyloid-related longitudinal decline (Mayo-PACC vs. PACC-R). While inclusion of more component measures was associated with higher reliability coefficients (e.g., Global-z), this comes at the cost of parsimony, time for administration and redundancy of some measures without improving sensitivity to amyloid-related cognitive decline. We predict that use of the Mayo-PACC will improve clinical applicability without sacrificing psychometric properties.

We examined practice effects of composites and component measures. Unlike other composites, Mayo-PACC did not show a significant initial practice effect (visit 1 vs. later visits) in mixed effects models in this sample. When considering individual component variables, memory measures were most prone to practice effects and Logical Memory delayed recall showed the largest initial practice effect (d=.21). Although practice effects were not examined by biomarker group in the present work, it is important to note that practice effects vary by biomarker group [37]. Thus, sample composition can contribute to variability in observed practice effects across different cohorts. Further, as seen by the significance of the linear and quadratic terms for years since visit 1, practice effects need not always be entirely accounted for by test naïve status alone. Future studies are needed to further unpack the complicated nature of practice effects and their impact on composite trajectories across biomarker groups.

In contrast to other PACCs, we chose not to include a global measure of cognition (e.g., MMSE) as these measures show prominent skew (i.e., ceiling effects) and are relatively insensitive to preclinical cognitive change [24, 42]. A relatively minor change in measures with limited variability such as the MMSE can lead to a large impact on z-scores, yielding “waterfall effects” that may be artificially driving larger effect sizes in composite scores when these measures are included. Indeed, Lim et al. [24] showed that removing the MMSE from the ADCS-PACC increased sensitivity to amyloid-related decline, and that inclusion of the MMSE increased the standard deviation of the rate of change approximately two-fold.

Our study has many strengths including 7-year average follow-up in a relatively large population-based sample. Nevertheless, there are limitations. First, while the Mayo-PACC is theoretically optimized for tracking cognitive decline across clinical stages (i.e., CU, MCI, dementia), we did not specifically test sensitivity to different stages in this manuscript and our sample only included individuals who were CU at baseline. Future work is needed to determine applicability of the Mayo-PACC in more advanced disease stages. Further, we observed an A+/A− effect on the Mayo-PACC over time based on our selected amyloid PET cut-points, but results can vary by amyloid load [43]. We also did not examine sensitivity within subgroups defined by tau or markers of neurodegeneration. While we advocate for a parsimonious composite measure that selectively includes component measures sensitive to cognitive decline, this does not negate the need for a battery of neuropsychological tests for thorough characterization of cognitive phenotypes. Finally, this cohort reflects the demographics of Olmsted County, Minnesota and results may not generalize to other populations, including populations targeted for clinical trials. It should be noted that important efforts have been made by other groups to develop PACCs in other countries [44], including harmonization of AD data collection [45].

In summary, the Mayo-PACC provides a parsimonious composite score that exclusively includes public domain measures that can be readily applied in other research and clinical practice settings and could be useful for future embedded pragmatic clinical trials. The Mayo-PACC demonstrated sensitivity to amyloid-related decline that is comparable to other PACCs and a global composite measure. This study examined psychometric properties and practice effects as part of the PACC creation process. Future work is needed to establish clinical meaningfulness of the Mayo-PACC, develop normative data to facilitate clinical application, and to further validate the Mayo-PACC along the AD continuum.

Research in Context.

1. Systematic review:

The authors reviewed the literature through primary source materials. Preclinical Alzheimer’s cognitive composites (PACC) are used as outcome measures in longitudinal studies and Alzheimer’s disease (AD) clinical trials. Methods used to develop PACCs have been criticized for failing to consider psychometric properties of component measures. A psychometrically sound cognitive composite that can bridge the gap between research and clinical use is needed. Use of public domain measures that can be readily applied in clinical settings will facilitate generalizability of the composite. These considerations were used to propose a new Mayo-PACC.

2. Interpretation:

The Mayo-PACC demonstrated sensitivity to amyloid-related decline that is comparable to other PACCs and a global composite measure. The Mayo-PACC could be useful for embedded pragmatic clinical trials in clinical settings.

3. Future directions:

Future work is needed to establish clinical meaningfulness, develop normative data, and further validate the Mayo-PACC across the AD continuum.