Cognitive modeling of the Mnemonic Similarity Task as a digital biomarker for Alzheimer's disease

Abstract

BACKGROUND

The Mnemonic Similarity Task (MST) is a popular memory task designed to assess hippocampal integrity. We assessed whether analyzing MST performance using a multinomial processing tree (MPT) cognitive model could detect individuals with elevated Alzheimer's disease (AD) biomarker status prior to cognitive decline.

METHOD

We analyzed MST data from >200 individuals (young, cognitively healthy older adults and individuals with mild cognitive impairment [MCI]), a subset of which also had existing cerebrospinal fluid (CSF) amyloid beta (Aβ) and phosphorylated tau (pTau) data using both traditional and model‐derived approaches. We assessed how well each could predict age group, memory ability, MCI status, Aβ, and pTau status using receiver operating characteristic analyses.

RESULTS

Both approaches predicted age group membership equally, but MPT‐derived metrics exceeded traditional metrics in all other comparisons.

DISCUSSION

A MPT model of the MST can detect individuals with AD prior to cognitive decline, making it a potentially useful tool for screening and monitoring older adults during the asymptomatic phase of AD.

Highlights

• The MST, along with cognitive modeling, identifies individuals with memory deficits and cognitive impairment.

• Cognitive modeling of the MST identifies individuals with increased AD biomarkers prior to changes in cognitive function.

• The MST is a digital biomarker that identifies individuals at high risk of AD.

1. BACKGROUND

Alzheimer's disease (AD) is marked by a gradual decline in memory and cognitive abilities that are often observed only after amyloid beta (Aβ) and phosphorylated tau (pTau) are already present. ¹ , ² , ³ , ⁴ Elevated levels of Aβ and pTau increase the risk of cognitive decline, ⁵ , ⁶ making this preclinical stage of AD a critical window for early detection and intervention. ⁷ During this phase, therapies for AD, including, but not limited to, those targeting Aβ and pTau, could be most effective, prior to irreversible neuronal loss. ⁸

Measuring Aβ and pTau is possible using positron emission tomography (PET) and cerebrospinal fluid (CSF), but both invasive and costly, limiting their general application in clinical settings. ⁹ , ¹⁰ Recent developments in blood testing for Aβ and pTau levels show promise in overcoming these barriers, ¹¹ , ¹² enabling them to become useful clinical tools. The early detection of cognitive impairment via digital biomarkers is also showing promise. ¹³ , ¹⁴ , ¹⁵ , ¹⁶ These non‐invasive assessments, which can often be remotely self‐administered, could complement blood tests in identifying individuals at future risk of decline, as they may detect different aspects of AD progression. Supporting this, work has found that combining blood biomarkers with cognitive tests offers a more accurate prediction of AD than using either method alone. ¹⁷ However, traditional cognitive tests have been less effective in identifying individuals at high risk of AD before cognitive symptoms appear. ¹⁸ This underscores the need for refined cognitive tasks that can detect subtle cognitive changes linked to AD pathology and aid in early diagnosis when combined with biomarker analysis.

The Mnemonic Similarity Task (MST) is a promising tool as it is designed to tax hippocampal function through its emphasis on pattern separation, a process central to rapidly learning new, arbitrary information. ¹⁹ , ²⁰ , ²¹ Performance on the pattern separation component of the MST (the Lure Discrimination Index [LDI]) has been associated with functional and structural changes within the hippocampus and related structures while the recognition memory aspect (REC) of the task has not. ²² , ²³ Given that the hippocampus (and entorhinal cortex, which serves as a gateway to the hippocampus) is one of the first affected by aging and AD, ²⁴ , ²⁵ , ²⁶ , ²⁷ it is unsurprising that performance declines with age and AD. ²⁸ , ²⁹ Further, work has demonstrated that the MST can predict early cognitive changes in AD, and this task has been used in multiple clinical trials, including A4 and HOPE4MCI. ³⁰ , ³¹ , ³² , ³³

The MST's traditional metrics are designed to be simple and robust but obscure potentially useful aspects of memory performance. Cognitive modeling of an individual's memory can give a richer understanding of mechanisms ³⁴ and how these are altered by aging or cognitive impairments. ³⁵ , ³⁶ , ³⁷ Recently we developed a multinomial processing tree (MPT) cognitive model to analyze performance on the MST using Bayesian methods that both fit individual participant performance and identified individual differences in memory and response strategies. ³⁸

Here, we applied this approach to determine whether it aided the MST's ability to discriminate various groups of individuals based on age, cognitive status, and Aβ/pTau status. We found that the MPT model was clearly superior to traditional metrics, particularly with regard to Aβ/pTau status, highlighting the MST's potential as an effective digital biomarker for early AD detection and monitoring.

RESEARCH IN CONTEXT

1. Systematic review: Using traditional sources, such as PubMed, the authors identified studies linking the Mnemonic Similarity Task (MST) to Alzheimer's disease (AD) risks. Most studies have shown that the MST can identify individuals with cognitive impairment, yet the MST's ability to identify older adults with increased AD biomarker status remains underexplored.

2. Interpretation: If digital biomarkers can be developed and validated to reflect AD pathology, they might offer a non‐invasive, cost‐effective way to predict Alzheimer's risk or monitor disease or treatment progression. We show how a cognitive‐modeling‐based analysis method not only classifies older adults with changes in memory and cognitive function but also identifies older adults with increased cerebrospinal fluid Aβ and pTau181 prior to clinical changes in cognitive ability.

3. Future directions: We encourage future studies to employ cognitive modeling of the MST as a digital biomarker to identify individuals at high risk for future cognitive decline.

2. METHODS

Data from this study came from two previously published works. Analyses 1 to 3 used participants from Stark et al. (2013), while analysis 4 used data from Trelle et al. (2021). Both used the same format as the MST, and both works attempted to identify cognitively “healthy” adults as part of their screening and assessment procedures.

2.1. Analysis 1: Predicting age group from cognitive modeling of MST

For predicting age group, people who were less than 40 years old (n = 27, age = 27.41 ± 5.7, 16 females) were classified as young, and those over 60 (n = 46, age = 71.33 ± 6.4, 28 females) were considered aged. All individuals were initially screened to be cognitively healthy without impairment using a battery of cognitive tasks. These include the Mini‐Mental State Exam (MMSE), ³⁹ Wechsler Memory Scale Logical Memory, ⁴⁰ Rey Auditory Verbal Learning Test, ⁴¹ Verbal Fluency, ⁴² Digit Span, ⁴³ Trails A and B, ⁴⁴ and Letter Number Sequencing, ⁴³ and the Wechsler Adult Intelligence Scale III. ⁴³ All individuals scored within 1.5 standard deviations of the mean of their age group for all neuropsychological measures.

2.2. Analysis 2: Predicting memory deficits in older adults using cognitive modeling of the MST

Significant work has used the Rey Auditory Verbal Learning Test (RAVLT) to differentiate older adults into separate groups based on cognitive function. ⁴⁵ , ⁴⁶ The RAVLT consists of learning a list of 15 words and recalling them after a delay of 15 min, and the delay score ranges from 0 to 15 and reflects the number of words correctly recalled after the delay. In the original report, older adults were split into thirds based on their RAVLT performance to parallel work in rodents that examined aged unimpaired (AU) and aged impaired (AI) groups. ²⁹ It is important to note that AI individuals (RAVLT of 5 to 8) are still within their age‐based norms and are not clinically impaired. AU individuals (RAVLT of 12 to 15) have a performance similar to young adults (this threshold is often used as part of the “SuperAger” criteria). However, here we used a threshold of 9 on the RAVLT to split older adults into either individuals with age‐related memory deficits (AMD) or no age‐related memory deficits (NMD) since performance above 9 is above age‐matched norms. ⁴⁷ Similar to prior work, individuals who scored higher than 9 were considered NMD (n = 31, age = 71.29 ± 6.79, 18 females), and those who scored 9 or below, but within normal limits of their age group, were considered AMD (n = 15, age = 71.40 ± 5.8, 10 females). ⁴⁶ , ⁴⁷ , ⁴⁸ , ⁴⁹

2.3. Analysis 3: Predicting cognitive status in cognitively older adults

To predict whether older adults were cognitively normal (CN) or had mild cognitive impairment (MCI) using the MST, the same 46 adults over the age of 60 from the previous two analyses were used for older adults who are cognitively intact (n = 46, age = 71.33 ± 6.4, 28 females). A further 10 individuals (age = 76.30 ± 6.78, 5 females) who were diagnosed with amnestic MCI were also included. Individuals with MCI were diagnosed by the University of California, Irvine (UCI) Alzheimer's Disease Research Center (ADRC). All individuals with MCI had a global Clinical Dementia Rating (CDR) of 0.5, a memory complaint, and impaired memory function on neuropsychological testing. A final diagnosis of amnestic MCI was reached by neurologists and neuropsychologists at clinical consensus conferences within the UCI ADRC. No participants had a history of neurological or psychiatric disorders, head trauma with loss of consciousness, drug abuse, or dependency.

2.4. Analysis 4: Predicting biomarkers of AD in CN older adults

To predict AD biomarker status, we used previously published data, ⁵⁰ collected as part of the Stanford Aging and Memory Study (SAMS). One hundred thirty‐three older adults (age = 68.8 ± 5.8, 83 females) were administered the MST and underwent a lumbar puncture to quantify AD biomarkers. All individuals had normal or corrected‐to‐normal vision/hearing, were right‐handed, were native English speakers, and had no history of neurologic or psychiatric disease. Further, each participant had a global CDR score of zero and performance within the normal range on a standardized neuropsychological test battery. Lastly, all participants were deemed CN during a clinical consensus meeting consisting of neurologists and neuropsychologists. The previously derived Aβ and pTau181 statuses were used in the present analyses (see Trelle et al. ⁵⁰ for details). Briefly, individuals with an Aβ42/Aβ40 ratio less than 0.0752 (n = 34, age = 70.33 ± 6.04, 21 females) were considered A+, and those with an Aβ42/Aβ40 ratio less than 0.0752 (n = 99, age = 68.13 ± 5.60, 63 females) were A−. Individuals with pTau181 greater than 42 pg/mL were considered T+ (n = 31, age = 71.86 ± 6.14, 21 females) with the remaining considered T− (n = 102, age = 67.73 ± 5.32, 63 females).

2.5.

The MST is a widely used cognitive task, that is, thought to critically tax hippocampal pattern separation (Figure 1A). ²³ , ²⁹ Both data sources used the traditional version of the MST, which consists of an incidental encoding phase and an explicit test phase. During the encoding phase, individuals made successive indoor/outdoor judgments for 128 images (2 s each, 0.5 ISI, color objects on a white background) via a button press. Immediately following the encoding phase, participants were given instructions for a recognition memory test, where they were told to identify objects as either “old” (the same picture as before), “similar” (indicating this is similar but not identical to the studied item, eg, a different exemplar, a rotation), or “new” via a button press. Here, participants saw 192 images (2 s each, 0.5 inter‐stimulus‐interval (ISI)) and responded to each of these images. Images consisted of 64 exact repeats from the encoding phase (targets), 64 completely novel images (foils), and 64 images that were similar but not identical to images seen during encoding (lures).

FIGURE 1.

Cognitive modeling of MST. (A) Sample stimuli used during incidental encoding phase and subsequent old/similar/new recognition task. (B) Probability tree diagrams of MPT model, demonstrating decision‐making process utilized within old/similar/new version of MST. Responses old (O), similar (S), and new (N) are modeled to be due to a decision tree. For example, on a repeat trial, the model assumes there is some probability, 𝜌, that at least a gist‐level successful retrieval is made. If that occurs, the modeled participant will respond “old.” If that fails, the modeled participant will respond “old,” “similar,” or “new” based on the guessing/bias parameters 𝛾. MST, Mnemonic Similarity Task.

Multiple behavioral metrics were extracted from the MST (Table 1), including the traditional REC and LDI scores. REC is a commonly used measure of recognition memory test and is the probability of “old” responses given to the target items minus the corresponding probability of “old” responses given to the foils (to correct for response bias). To quantify the ability to discriminate between similar lures, the LDI is the difference between the probability of giving a “similar” response to lure items and the probability of giving a “similar” response to the foils to account for any bias individuals may have in using the “similar” response overall. For a follow‐up analysis, we also quantified the rate of “old” responses for target trials (hits), the rate of “similar” responses for lure trials (correct rejections of lures), and the rate of “new” responses for foil trials (correct rejections of foils). Further, we attempted to obtain a readout of guessing by calculating the rate of “old” responses on foil trials, the rate of “similar” responses on foil trials, and the rate of “new” responses on target trials.

TABLE 1.

Descriptions of both traditional and model‐based metrics on the MST.

Metric	Type	Definition
REC	Traditional	Recognition memory score
LDI	Traditional	Reflects ability to discriminate between similar lures
p(Old|Repeat)	Traditional	Probability of responding “old” for repeats
p(Sim|Lure)	Traditional	Probability of responding “similar” to lures
p(New|Foil)	Traditional	Probability of responding “new” for foils
p(Old|Foil)	Traditional	Probability of responding “old” for foils
p(Sim|Foil)	Traditional	Probability of responding “similar” for foils
p(New|Repeat)	Traditional	Probability of responding “new” for repeats
𝜌	Modeled	Probability of remembering items at a gist level
λ	Modeled	Ability to discriminate remembered items from lures
𝜓	Modeled	Probability of remembering that an item was not studied
𝛾	Modeled	Probability of guessing “old”/bias to respond “old”
𝛾S	Modeled	Probability of guessing “similar”/bias to respond “similar”
𝛾N	Modeled	Probability of guessing “new”/bias to respond “new”

Abbreviations: LDI, lure discrimination index; REC, recognition memory.

2.6. Cognitive modeling

Cognitive modeling provides a useful tool for inferring latent psychological variables beyond traditional measurements. Previously, we used cognitive modeling to model subject‐level performance on the MST in young adults ³⁸ using the MPT framework, a common approach for cognitive modeling of REC tasks. The MPT framework assumes that cognitive processes can be divided into discrete categories or decision points (Figure 1B). Briefly, when a repeated item appears, we assume there is a probability (ρ) that the item is successfully matched with memory in at least a basic gist or “familiarity” form, leading to an “old” response. Failing that, we assume that a guess is made with unique probabilities (response biases) for each of the three responses. Similarly, when an unrelated foil is present, there is a probability (ψ) that the lack of a match to memory is sufficiently clear that a “no” response is made; failing that, a three‐choice guess is made.

When a similar lure is presented, there is an initial decision point involving recognizing some degree of match between the object presented and the memory of one previously encountered, based on the same ρ as above. This level of match is modeled to reflect a simpler item‐, gist‐, or familiarity‐based match (for both lures and repeated items). If this is unsuccessful, a three‐choice guess happens, as before. If successful, there is a second decision point based on a set of similarity‐based probabilities (δ) capturing whether the memory retrieval contains the richer details required to reject the item as only being similar to the studied item. If successful, a “similar” response is made; if it is unsuccessful, an “old” response is made.

Posterior distributions for metrics within MPT models were estimated at the subject level from trial‐by‐trial (condition (repeat, lure, foil), response (old, similar, new), and lure bin (one to five)) experimental data using Just Another Gibbs Sampler (JAGS), which is a program designed for Bayesian hierarchical models. ⁵¹ Models were run with eight separate Markov chain Monte Carlo chains with 1000 samples per chain after removing 1000 burn‐in samples. We used posterior means as point estimates for multiple metrics of interest (Table 1). These metrics include ρ, which reflects the probability of remembering items, λ, based on δ and designed to capture the ability to discriminate remembered items from lures, ψ, the probability of remembering that an item was not studied, γ (probability of guessing “old”), γ^N (probability of guessing “new”), and γ^S (probability of guessing “similar”). Supplemental information and code to run the model are provided at https://github.com/mdlee/mpt4mst.

2.7. Statistical analyses

All analyses were done in Python. Logistic regressions were run using statsmodels ⁵² to predict age group, clinical status, and biomarker status, for example, from various sets of metrics. Areas under the curve (AUC) measures were derived from receiver operating characteristic (ROC) curves of the logistic regressions. To compare model fits, we calculated the Bayesian information criterion (BIC) of each model. ⁵³ Absolute differences in BICs of greater than 2 were considered reliable. Considering metrics in isolation and in combination with other metrics from the same approach does allow for direct comparisons across the techniques. However, the metrics are not independent of each other, and the two approaches differ in the number of variables considered. Therefore, to identify how each variable acts in conjunction with the others, we performed an 8‐choose‐4 combinatorial analysis and quantified the number of times each metric appeared in the top third of AUCs from 8‐choose‐4 analyses. Independent sample t tests were used to examine group differences in traditional and model‐based metrics. ⁵⁴ To investigate group changes in guessing strategies, Kolmogorov–Smirnov tests were used because data were proportioned and therefore not normally distributed. For all analyses, p < 0.05 was considered reliable.

3. RESULTS

To investigate if the Bayesian MPT cognitive model could adequately fit the observed data, we conducted posterior predictive checks, which are the standard Bayesian approach to testing the ability of the model to redescribe the data on which parameter inferences are based, using the approach developed in Lee and Stark (2023). These checks demonstrated that the model was able to accurately redescribe the empirical data. We plotted these posterior predictive checks for each participant from Stark et al., 2013 (Figure S1A) and Trelle et al., 2021 (Figure S1B). Also, the measures from the MPT model were precise with the width of the confidence intervals for all the measures below 0.5. Previously, we demonstrated that the traditional REC correlated with ρ, while LDI correlated with λ (previously denoted by τ). ³⁷ This is because REC and ρ are both measures thought to reflect REC on a task, while LDI and λ are thought to be related to mnemonic discrimination. Our first goal was to assess the relationship between the traditional and modeled metrics in the two datasets. ²⁹ , ⁵⁰ As in the prior work, we found strong correlations between these variables in both datasets (Stark et al., 2013; REC vs ρ: r = 0.73, LDI vs λ: r = −0.90, Trelle et al., 2021; REC vs ρ: r = 0.77, LDI vs λ: r = −0.90). A full correlation matrix for both Stark et al. (Figure S2A) and Trelle et al. (Figure S2B) revealed similar correlations between datasets. These results demonstrate that the model‐based metrics derived here are similar to prior findings. With this, we performed four key analyses, using two datasets, to assess if traditional or model‐based metrics were superior at identifying individuals at risk for AD.

3.1. Analysis 1: Traditional metrics and model‐based metrics of MST equally predict age group status

Given that extensive work has demonstrated that older adults are impaired on the MST, we assessed whether cognitive modeling could enhance the ability to differentiate between younger and older adults. ²³ , ²⁹ Considered individually, there was no reliable difference in REC between age groups, while LDI was significantly lower in older adults (Figure S3A, REC: t(71) = 1.19, p = 0.28, Figure S3B, LDI: t(71) = 5.71, p < 0.0001). When examining modeled metrics individually, ρ showed no reliable age differences, while ψ and λ were lower in older compared to younger adults (Figure S3C, ρ: t(71) = 0.22, p = 0.83, Figure S3D, ψ: t(71) = 2.62, p < 0.05, Figure S3E, λ: t(71) = −5.69, p < 0.0001). A multiple logistic regression using the traditional LDI and REC as predictors achieved an AUC of 0.86 (Figure 2A, p < 0.0001). Model‐based metrics, with ρ, ψ, τ, and guessing strategies (γ, γ^N, and γ^S) as predictors, yielded a similar AUC of 0.84 (Figure 2A, top, p < 0.0001), suggesting that model‐based metrics did not outperform traditional metrics in predicting age group.

FIGURE 2.

Comparison of traditional (blue) to MPT model (red) performance in questions 1 to 3. ROC curves (top) demonstrating predictive value of traditional and model‐based measures and frequency of presence in top 30 AUCs in 8‐choose‐4 combinatorial analyses (bottom) are shown. (A) Comparison of age group predictions showing no significant difference between traditional measures and MPT model‐based measures. LDI emerges as the most frequent metric in the top third of AUCs. (B) Metrics derived from MPT model better predict the performance of high‐ versus low‐performing older adults. The gist‐based recognition memory signal metric (ρ) is the predominant metric in the 8‐choose‐4 analysis for predictive accuracy. (C) MPT model‐based metrics were superior at identifying healthy versus MCI. Within the 8‐choose‐4 analysis, γO is the leading metric for MCI prediction. AUC, area under a curve; LDI, lure discrimination index; MCI, mild cognitive impairment; MPT, multinomial processing tree; MST, Mnemonic Similarity Task.

To appreciate better the impact each variable might have in conjunction with the others, we performed an 8‐choose‐4 combinatorial analysis and identified how often each factor occurred in the top third of the resulting AUCs. This revealed that the LDI was the most common metric in distinguishing younger and older adults, appearing in virtually all the top‐performing models and almost twice as often as the most frequent MPT model‐based metric (Figure 2A, bottom). Thus, when considering the simpler task of predicting age group membership, we found no evidence that the MPT model was superior to the traditional approach.

3.2. Analysis 2: Model‐based metrics better identify memory ability of older adults

Aging is often associated with substantial heterogeneity, with a subset of older adults performing at higher levels on neuropsychological memory assessments compared to their peers. Therefore, we next asked if performance on the MST along with cognitive modeling could aid in dissociating across levels of cognitive function in healthy adults by discriminating NMD versus AMD. Considering each variable individually, REC and LDI levels were similar in NMD and AMD (Figure S3F, REC: t(44) = 0.22, p = 0.22, Figure S3G, LDI: t(44) = 1.72, p = 0.09). When measuring model‐based metrics, ρ was significantly higher in NMD compared to AMD individuals with no difference in ψ or λ (Figure S3H, ρ: t(44) = 3.10, p < 0.01; Figure S3I, ψ: t(44) = 1.248, p = 0.22; Figure S3J, λ: t(44) = −0.31, p = 0.76). When LDI and REC were combined, a multiple logistic regression did not successfully distinguish NMD versus AMD (AUC = 0.65, p = 0.13, Figure 2B). However, multiple logistic regression with model‐based metrics was able to stratify NMD from AMD with an AUC of 0.80 (p < 0.05). When assessing combinations of traditional and model‐based metrics in an 8‐choose‐4 combinatorial analysis, ρ emerged as the most consistent metric in the top‐performing models, with LDI appearing as a distant fourth most consistent (Figure 2B, bottom). This suggests that the MPT model provides a more accurate identification of memory ability in older adults than traditional metrics but that this is driven heavily by the model's estimate of how well individuals remember at least the gist of an item.

3.3. Analysis 3: Model‐based metrics better predict MCI status

We next investigated whether cognitive modeling of the MST could better identify individuals with MCI compared to traditional metrics. We found that individuals with MCI had significantly lower REC performance compared to CN older adults, but there were no differences between groups in LDI scores (Figure S3K, REC: t(52) = 4.73, p < 0.0001; Figure S3L, LDI: t(52) = 0.77, p = 0.44). We also found that ρ decreased in individuals with MCI, but no difference in groups for ψ or λ (Figure S3 M, ρ: t(52) = 5.51, p < 0.0001; Figure S3N, ψ: t(52) = 0.99, p = 0.33; Figure S3O, λ: t(52) = −0.16, p = 0.87). In the multiple logistic regression, we found that the combination of REC and LDI could classify MCI status with good accuracy (AUC = 0.81, p < 0.001, Figure 2C). However, cognitive‐model‐based metrics offered superior predictive power, achieving an AUC of 0.94 (p < 0.0001). Permutation analysis found that γ was the most influential metric, appearing in all the top third of models (Figure 2C, bottom). This suggests that the MPT model is superior at detecting MCI over traditional metrics largely due to the ability to derive differences in guessing strategy on the task.

3.4. Analysis 4: Model‐based metrics can better predict Aβ and Tau status in CN older adults

We next evaluated whether the MPT model of the MST could detect Aβ status in cognitively healthy older adults, classified as Aβ+ or Aβ− via CSF Aβ42/Aβ40 ratios. Aβ+ individuals had decreased REC scores but equivalent LDI performance compared to Aβ− counterparts (Figure S4A, REC: t(131) = 2.68, p < 0.01, Figure S4B, LDI: t(131) = 0.33, p = 0.74). Further, ρ was lower in Aβ+ compared to Aβ− older adults with no group differences in ψ and λ (Figure S4C, ρ: t(131) = 2.54, p < 0.05, Figure S4D, ψ: t(131) = 1.11, p = 0.27, Figure S4E, λ: t(131) = −0.53, p = 0.60). Multiple logistic regression with traditional metrics could modestly predict amyloid status (AUC = 0.64, p < 0.05, Figure 3A). On the other hand, multiple logistic regression with model‐based metrics better predicted amyloid status (AUC = 0.73, p < 0.05). When conducting an 8‐choose‐4 combinatorial analysis to investigate the impact each variable might have in relation to the others, γ was the most predictive metric among the top third of AUCs. Interestingly, γ was represented in nearly all the top models and twice as often as both traditional metrics (Figure 3A, bottom). The model thus better identifies asymptomatic individuals with elevated amyloid burden due to its ability to derive differences in guessing “old.”

FIGURE 3.

Utilization of the MPT model of the MST for predicting AD biomarker status. (A) Both traditional and MPT model‐derived metrics can predict Aβ status, with the MPT model‐based metrics showing superior predictive accuracy. The metric γO is identified as the most frequently occurring metric in the top results from n‐choose‐four analyses for predicting Aβ status. (B) The MPT model‐based measures, but not traditional measures, can successfully predict pTau status. The metric γO is again highlighted as the most common metric in the top 30 AUCs from an n‐choose‐four analysis for predicting pTau status. For panels A and B, Traditional measures are in blue, while the MPT model‐based metrics are in red. Aβ, amyloid beta; MPT, multinomial processing tree; MST; Mnemonic Similarity Task; pTau, phosphorylated tau.

While both Aβ and pTau are biomarkers for AD, pTau has a stronger link to cognitive decline and may better predict disease progression. Somewhat surprisingly, CN older adults with elevated pTau levels did not differ on either traditional or model‐based metrics (REC, LDI, ρ, ψ and λ) compared to those with normal pTau levels (Figure S4F–J, all ps > .10). Likewise, a multiple logistic regression with REC and LDI failed to predict pTau status (AUC = 0.50, p = 0.91, Figure 3B). Importantly, the logistic regression with the model‐based metrics did predict pTau status (AUC = 0.71, p < 0.05). When conducting an 8‐choose‐4 combinatorial analysis, the single clearly most reliable metric was γ, appearing more than twice as much as the next most important metric (ψ) (Figure 3B, bottom). Further, every metric from the MPT model was more represented than REC and LDI in the top third of models. Overall, the cognitive model outperformed traditional metrics in predictive accuracy, suggesting its effectiveness in early AD screening.

3.5. Changes in model‐derived guessing strategies with age, cognitive impairment, and biomarker status

Given that modeling guessing probabilities was informative for predicting AD biomarker status and cognitive status, we took a deeper dive into guessing strategies. We first explored how model‐based guessing strategies changed across age groups. Cognitive modeling suggested that younger adults tended to guess “similar” more frequently and “new” less frequently than older adults (Figure 4A, Kolmogorov–Smirnov test; γ^S: D = 0.37, p < 0.05; γ^N: D = 0.38, p < 0.05), without any significant age‐related differences for guessing “old” (γ: D = 0.25, p = 0.20). This pattern suggests an age‐related shift from guessing “similar” to “new.” Interestingly, however, no significant differences in guessing strategies were found between healthy older adults with and without memory deficits (Figure 4B, Kolmogorov–Smirnov test: all p > 0.10). In contrast, older adults with MCI were more inclined to guess “old” and less likely to guess “similar” or “new” compared to cognitively healthy older adults (Figure 4C, Kolmogorov–Smirnov test; γ: D = 0.6318, p < 0.01; γ^N: D = 0.4910, p < 0.05; γ^S: D = 0.5318, p < 0.05). These results underscore that aging and MCI distinctly affect guessing strategies.

FIGURE 4.

Influence of age, memory ability, cognitive status, and AD biomarker status on modeled guessing strategies in MST. (A) Compared to younger adults, older adults exhibit a higher likelihood of guessing “new” and a lower tendency to guess “similar.” (B) Among older adults, those with no age‐related memory deficits show no significant differences in guessing strategies when compared to individuals with age‐related memory deficits. (C) Individuals with MCI demonstrate a greater bias toward guessing “old” and are less inclined to guess “similar” or “new” relative to cognitively healthy older adults. (D) There are no differences in guessing strategies between Aβ− and Aβ+ older adults. (E) Older adults with elevated pTau levels display a marked shift from guessing similar toward guessing old compared to pTau‐negative older adults. For panels A–E, the metric for guessing “old” (γ) is represented in yellow, for guessing “similar” (γ^S) in pink, and for guessing “new” (γ^N) in blue. Aβ, amyloid beta; AD, Alzheimer's disease; MCI, mild cognitive impairment; MST, Mnemonic Similarity Task; pTau, phosphorylated tau.

We next investigated how guessing strategies on the MST varied with AD biomarker status in older adults. Cognitive modeling suggested that those with elevated amyloid displayed a tendency to guess “old” more frequently than their counterparts without elevated amyloid, but this failed to reach significance. Further, there were no significant differences in biases toward guessing “similar” or “new” (Figure 4D, Kolmogorov–Smirnov test; γ: D = 0.24, p = 0.08, γ^S: D = 0.21, p = 0.18, γ^N: D = 0.13, p = 0.75). We next investigated whether guessing strategies changed as a function of pTau status. We observed that individuals with elevated pTau levels were more likely to guess “old” and less likely to guess “similar,” with no change in the likelihood of guessing “new” (Figure 4E, Kolmogorov–Smirnov test; γ: D = 0.32, p < 0.05; γ^S: D = 0.34, p < 0.05, γ^N: D = 0.19, p = 0.34). These differences in predicted guessing strategies further highlight the benefits of the MPT cognitive model of the MST.

3.6. Addition of raw metrics of guessing does not match model‐based metrics

Given the differences in guessing across performance ability, impairment level, amyloid, and pTau status in older adults, we next asked whether we could derive guessing strategies based on response patterns. Specifically, we used the proportion of trials an individual responded “old” on foils as a measure of guessing old, the proportion of trials an individual responded “similar” on foils as a measure of guessing similar, and the proportion of trials an individual responded “new” on repeats as a measure of guessing new. REC and LDI incorporate these metrics in their calculations as part of their difference scores, intended to factor out differences in guessing rates. Any baseline shift in the probability of guessing “old” or “similar” would presumably affect both components of the difference metrics, removing what the model‐based analyses suggest could be highly informative. Therefore, we also added the raw proportion of trials people responded “new” for foil trials, the proportion of trials an individual responded “old” on repeats, and the proportion of trials an individual responded “similar” on lure trials. Using these six new metrics, we asked whether these metrics could increase the predictive value of the MST to the same level as that provided by the MPT model.

We first asked whether these new behavioral metrics could predict whether CN older adults exhibited memory deficits. In multiple logistic regression, we found that while the raw AUC appeared elevated, this model could still not reliably predict performance level (AUC = 0.78, p = 0.21, Figure 5A). We also investigated whether these differing models better fit the data by examining their respective BIC, which is a metric that reflects the goodness of fit of a model. ⁵⁵ Using this measure, we found that the MPT model‐based metrics better fit the data compared to the probability‐based traditional metrics (MPT model BIC: 76.49, raw traditional BIC: 81.86), suggesting that, despite the addition of the raw traditional metrics, cognitive modeling better predicts memory ability. Next, when assessing whether these new probability‐based metrics helped with the prediction of MCI status, we found that a multiple logistic regression with these raw traditional metrics significantly bolstered the predictive accuracy of the MST, yielding an AUC of 0.95 (p < 0.0001, Figure 5B), which aligns with the predictive strength of cognitive modeling and actually provided a better fit of the data than model‐based metrics (MPT model BIC: 49.77, raw traditional BIC: 45.12).

FIGURE 5.

Additional raw‐derived measures may improve upon traditional metrics but do not match the MPT model‐based metrics. (A) Adding raw metrics of guessing fails to predict memory ability, in contrast to cognitive modeling measures. (B) In cases of overt cognitive impairment, such as MCI, raw‐derived guessing measures show predictive value comparable to that of the MPT model. (C) Additional raw metrics fail to match the MPT model in predicting Aβ status. (D) While the additional raw measures improve the predictive ability of traditional approaches for pTau status, their performance does not exceed chance levels. For panels A–D, traditional measures are depicted in purple, and the MPT model‐based metrics are illustrated in red. Aβ, amyloid beta; MCI, mild cognitive impairment; MPT, multinomial processing tree; pTau, phosphorylated tau.

We next evaluated whether raw traditional metrics could match cognitive modeling in predicting amyloid status. A multiple logistic regression with these metrics did not match the predictive capacity of model‐based metrics (AUC = 0.68, p = 0.22, Figure 5C). Moreover, when comparing model fits using the BICs, model‐based metrics better fit the data (MPT model BIC: 167.18, raw traditional BIC: 177.12). We next asked if the new probability‐based metrics could predict pTau status. These new metrics did show a qualitative improvement in predicting pTau status, but this was not statistically reliable (AUC = 0.70, p = 0.08, Figure 5D) Further, model‐based metrics better fit the data compared to probability‐based metrics (MPT model BIC: 161.95, raw traditional BIC: 167.52), reinforcing the superiority of the MPT model in predicting amyloid and pTau status.

4. DISCUSSION

The MST is a widely used memory test that assesses changes in hippocampal integrity in various conditions including age‐related cognitive decline and AD. ²³ Given that this task is resistant to practice effects and can be easily performed remotely, it has emerged as an ideal candidate for clinical use as a digital biomarker for AD pathology, but direct tests of its validity are needed. We previously demonstrated that a MPT model could be applied to the MST, but we did not know whether this model would aid the predictive value of the MST. In this study, we compared the predictive value of traditional metrics versus model‐based metrics for predicting age group, MCI, and AD biomarkers. We found that the MPT model enhanced the ability of the MST to identify older adults at risk of developing AD prior to cognitive decline, demonstrating that the MST was well suited to enhance early diagnosis of AD.

4.1. Cognitive modeling of MST identifies differing cognitive capacities in older adults

A large body of work has demonstrated that advancing age is associated with significant impairment on the LDI metric of the MST, while REC remains stable with age. ²⁹ , ⁵⁶ , ⁵⁷ Similarly, we found that traditional behavioral metrics of the MST predicted age group with high proficiency (AUC = 0.85), and the MPT model did not increase the high predictive value of the MST. In permutation analyses, the LDI was most prominent, further supporting the hypothesis that age‐related impairments are due in part to deficits in hippocampal pattern separation.

While age‐related impairments are seen on many cognitive tests, there is typically significant heterogeneity within the aging population, with a subset of healthy older adults exhibiting AMD and others showing young‐like performance (eg, “SuperAgers” or “aged unimpaired”). Our analyses showed that traditional measures of MST performance did not readily distinguish these two but that the cognitive modeling approach could. This suggests that the MPT model may be more sensitive to differing memory abilities in older adults.

MCI, often considered a precursor to or risk factor for AD, is frequently missed, with perhaps only ∼8% of those affected accurately identified. ⁵⁸ We demonstrated that, like previous work, performance on the MST predicted MCI with an AUC of 0.81. ³¹ , ³² However, cognitive modeling significantly improved the predictive accuracy, reaching an AUC of 0.94. We note that multiple mechanisms can cause MCI. While the diagnoses of MCI followed the criterion for MCI due to AD, ⁵⁹ other mechanisms could have caused the cognitive impairment in a subset of these individuals. ⁶⁰ Here, MCI was diagnosed using a neuropsychological battery and a clinical consensus. Being another memory test, it is perhaps unsurprising that the MST predicted MCI. However, the MST was not part of the clinical diagnosis. Therefore, its accuracy could aid primary care clinicians who only have access to tests like the MMSE. We hypothesize that including the MST in primary care could improve cognitive impairment identification and reduce MCI underdiagnosis. This is, as yet, untested, and it is not known whether the MST can differentiate subtypes of MCI.

4.2. Cognitive modeling of MST predicts AD biomarker status in cognitively healthy older adults

Recent studies utilized comprehensive cognitive batteries to identify individuals at higher risk of AD, ¹⁴ , ³⁰ , ⁶¹ with longitudinal cognitive testing used to identify healthy adults with elevated Aβ and pTau levels. ⁶¹ , ⁶² , ⁶³ Several use a shortened version of the MST (aka BPSO) ⁶¹ , ⁶² and longitudinal changes on this could better predict memory impairment over 3 months compared to baseline neuropsychological scores. Importantly, none of the other tasks within the cognitive battery could exceed baseline neuropsychological scores.

Early detection of AD‐related pathology is paramount, but current clinical assessments are lagging pathologies, such as cerebral Aβ deposition by up to 20 years, highlighting the need for more sensitive tasks. ¹ , ⁴ A recent study demonstrated that combining multiple versions of the MST could modestly predict amyloid status in CN older adults. ³² Similarly, we found that MST performance modestly predicted Aβ status (AUC = 0.64), and the MPT model enhanced this prediction to an AUC of 0.73. Our results affirm that MST performance relates to Aβ status and show that inferred cognitive mechanisms improve its predictive value.

Amyloid accumulation, in the absence of pTau, does not correlate with cognitive impairment, ⁶⁴ but pTau is associated with hippocampal hyperactivity, neurodegeneration, and the transition to dementia. ⁶ , ⁶⁵ However, it is not known whether increased pTau levels can be predicted from performance on the MST. We found that, while the traditional MST metrics could not predict elevated pTau levels, integrating the MPT model improves prediction to an AUC of 0.71. This suggests that the MPT model can distinguish individuals with elevated pTau, highlighting its potential in identifying early markers of cognitive impairment associated with AD pathology.

4.3. Changes in guessing strategies as a function of cognitive impairment and biomarker status

When applying the MPT model to predict AD biomarkers and cognitive status, one emerging theme was the role of guessing strategies. Guessing strategies, or response biases, change in amnestic patients and individuals with dementia, so it is worthwhile to investigate how they change in people at risk of AD. Notably, individuals with increased biomarker levels tended to shift their guessing bias from “similar” to “old,” and this was more pronounced among those with MCI, suggesting a continuum of change. This shift toward guessing “old” aligns with other work demonstrating that MCI is associated with more liberal response biases on REC tasks. ⁶⁶ , ⁶⁷ One plausible explanation is that individuals build up gist representations during a task and, unlike younger adults, cannot rely on item‐level detail memory. Thus, without intact memory for details, individuals with MCI may overly rely on gist and show a more liberal response bias. ⁶⁷ , ⁶⁸ On the MST, this would cause the observed bias toward responding “old.” One potential explanation for this shift is reduced dentate gyrus‐based pattern separation, which causes a shift toward pattern completion, thereby increasing false memories. ⁶⁹ , ⁷⁰ Notably, these changes were more reliably seen using the MPT model than with raw traditional metrics. These outcomes underscore the superiority of cognitive modeling in recognizing individuals at risk of AD, supporting its utility as a digital biomarker for early detection of the disease.

The MST's three‐choice nature precludes the use of cognitive models like signal detection theory or dual process models, which assume unidimensionality in latent memory strength. In these models, responses are treated as arising from memory strengths between “new” and “old” (too strong to respond “new” but not enough for “old”). People, however, can actively generate evidence for “similar” responses (e.g., recognizing a different image of the same object). ²³ Despite this, work using these approaches on two‐choice tasks has also found a shift toward a bias toward responding old in MCI and AD, ⁷¹ , ⁷² , ⁷³ further suggesting that individuals at risk for AD exhibit changes in response patterns. There is, to our knowledge, one other cognitive model that has been developed for the three‐choice MST ⁷⁴ which uses linear ballistic accumulators to avoid the unidimensionality assumption of the MST.

Interestingly, different measures were important for predicting AD biomarkers compared to age or memory ability. This supports the idea that the mechanisms that contribute to pathological cognitive decline (ie, dementia) are not the same as the mechanisms that explain variability in memory ability in healthy older adults. The findings that LDI was the best predictor of age group and gist‐based information was the best at separating NMD and AMD suggest that these measures are most related to memory ability in older adults, but not necessarily related to AD. Conversely, the shift in structured guessing (ie, response bias) may be more directly linked to the pathological processes in AD.

4.4. Limitations and future directions

This study is not without limitations, and several future directions could be taken to better assess the utility of cognitive modeling of the MST. One critical aspect to consider is how well cognitive modeling performs relative to other, standard neuropsychological scores. In prior work using these same data, ⁵⁰ multiple neuropsychological episodic memory assessments showed no ability to discriminate amyloid of pTau181 status. However, this was not a clinical setting, and this direct comparison was not made.

Future work is also needed to determine how well MST performance measures could be used to create risk scores in the clinical setting. Ideally, this, combined with other measures, such as plasma, could help screen high‐risk individuals and identify who may need more invasive and costly measures. Future research should address this gap and explore how the MST can be utilized in clinical settings.

5. CONCLUSION

Here, we asked whether a MPT cognitive model of the MST could aid in the identification of individuals at risk of AD. We demonstrated that, in addition to predicting memory deficits and MCI, the MPT model of the MST could predict both amyloid and pTau status with AUCs of greater than 0.7 in older adults without signs of cognitive decline. This suggests that cognitive modeling of the MST holds significant potential as a non‐invasive, efficient screening tool within the clinical setting.

CONFLICT OF INTEREST STATEMENT

The authors declare no conflicts of interest. Author disclosures are available in the Supporting Information.

CONSENT STATEMENT

All human subjects provided informed consent.

Supporting information

Supporting Information

Supporting Information

Vanderlip CR, Lee MD, Stark CEL. Cognitive modeling of the Mnemonic Similarity Task as a digital biomarker for Alzheimer's disease. Alzheimer's Dement. 2024;20:6935–6947. 10.1002/alz.14163