Disease progression from mild cognitive impairment to dementia for patients with Alzheimer's disease or Lewy body pathology

Abstract

Background:

The majority of mild cognitive impairment (MCI) patients who converted to dementia in later years have Alzheimer's disease (AD) pathology. The second most common type of dementia is Lewy body (LB) dementia.

Objective:

In this project, we are interested in identifying the risk factors that predict who will develop AD dementia or LB dementia in later years.

Methods:

Cox proportional hazards model and machine learning survival methods for interval-censored data were used to identify the risk factors that predict the onset of dementia for MCI patients with AD or LB pathology.

Results:

We found that orientation, memory, and irritability scores were useful in predicting AD dementia onset, while daily living, depression, and executive function scores were identified as strong predictors in the LB cohort.

Conclusions:

Different neurocognitive domains were predictive for conversion to dementia from MCI in patients with AD or LB pathology. The depression scale and functional activities were found to be predictive of LB dementia while irritability or lability severity score from Neuropsychiatric Inventory Questionnaire was associated with the onset of AD dementia.

Introduction

Mild cognitive impairment (MCI) patients are often the target population in the disease-modifying therapy (DMT) trials.^1,2 The majority of MCI patients have the hallmark pathology of Alzheimer's disease (AD) with the accumulation of amyloid-β (Aβ) and tau proteins, leading to memory deficits and cognitive decline over time. The second most common dementia is Lewy body (LB) dementia, with the pathology of the accumulation of alpha-synuclein (α-syn). ³ For MCI patients with AD pathology, one tenth of them may have contributing LB pathology. ⁴ The annual conversion rate from MCI to AD dementia was higher than the rate for MCI to LB dementia among MCI patients (12% versus 6%).^5,6 These conversion rates were often estimated from the study population with or without AD or LB pathology. For that reason, that rate could be under-estimated for MCI patients with AD or LB pathology.

Some MCI patients will remain stable and not develop dementia (e.g., AD dementia or LB dementia or other types of dementia). ⁷ Therefore, it is important to identify risk factors that predict who will develop dementia in later years. In a case-control study with data from the Mayo Clinic Study of Aging, ⁸ as compared to the AD group, patients in the LB dementia group were found to be younger, have more males, have a history of depression, and have fewer Apolipoprotein E (APOE) ε4 copies.^7,9,10 In another study comparing the AD dementia group and the LB dementia group by using data from National Alzheimer's Coordinating Center (NACC), ⁶ all these patients had MCI status at some visits during the study and converted to either AD dementia or LB dementia before the last visit. They found that as compared to the AD dementia group, patients in the LB dementia group at baseline were more likely to be male, have more impairment on Trail Making Tests (Trail-A and Trail-B), have a higher percentage of depression and anxiety, and have better logical memory delayed recall. Based on these findings, the risk factors to predict AD or LB dementia onset may be different from each other.

In the existing literature, the visit with the dementia diagnosis (e.g., AD dementia) was used as the time for the conversion event. In fact, the actual conversion time from MCI to dementia is unknown. It is between the visit of the dementia diagnosis and the visit before that. Such data is known as interval-censored (IC) data in survival analysis.^11,12 Therefore, IC methods should be used to analyze such data properly. When only the visit with the dementia diagnosis was used in the data analysis, the logistic regression model and the right- censored survival models were commonly utilized. In the logistic regression models, the binary conversion status (stable or conversion) was the outcome in the analysis. Then, the detailed conversion time was not used in the analysis model, leading to poor model predictive performance. ¹³ The statistical models for right-censored data utilize the visit of the dementia diagnosis as the event time. In the right-censored models, the dementia onset is treated as the actual event time. But the dementia onset is interval-censored with the event occurring within a time interval. Therefore, the naive logistic regression model and the right-censored models do not fully utilize the information from the available data.

In addition to these traditional statistical models, machine learning methods for IC data may be used to further improve the predictive performance by relaxing the proportional hazards (PH) assumption in the traditional Cox model for IC data. Yao et al. ¹⁴ developed a random forest approach based on the weighted Kaplan-Meier estimate.^15,16 As compared to the traditional random survival forests that treat all terminal nodes with equal weights,^17–19 the conditional inference forest in their model assigns large weights to terminal nodes with a substantial number of subjects at risk. ¹⁴

Methods

Data sets

Data from the National Alzheimer's Coordinating Center (NACC) were used in this project, and data were downloaded on the date of May 31, 2024. There are three versions of the NACC database. We combined the first two versions (NACC-v1v2) as the training data, and the third version as the testing data set. Participants in the NACC-v1v2 were enrolled in the study between 2005 and 2015, while participants in the NACC-v3 were enrolled after March 2015. In general, participants in the first two versions had a longer follow-up than those in the NACC-v3 database. In this project, we were particularly interested in identifying the demographic and clinical cognitive measures that can be used to predict the conversion from MCI to AD dementia or LB dementia.

In the NACC dataset, a primary etiologic diagnosis is recorded by the clinician; there is the option to record another diagnosis if it is felt to be contributing to the cognitive disorder. From the NACC data, we identified two cohorts based on the primary etiologic diagnosis: the AD cohort and the LB cohort. In the LB cohort, as the sample size was small, we also included LB patients with LB as the contributing etiologic diagnosis (unless the primary diagnosis was AD). Patients with AD as the primary etiologic diagnosis and LB as the contributing etiologic diagnosis were included in the AD cohort. In the AD cohort, we had two groups:

• (1a) MCI-to-DementiaAD group: MCI patients with AD dementia conversion,

• (1b) MCI-with-AD stable group: MCI patients with AD as the primary etiologic diagnosis without dementia conversion.

In the LB cohort, we had the following two groups:

• (2a) MCI-to-DementiaLB group: MCI patients with LB dementia conversion,

• (2b) MCI-with-LB stable group: MCI patients with LB as the primary or secondary etiologic diagnosis without dementia conversion.

It is possible in the NACC database that a participant could have multiple conversions during the follow-up visits. Such cases introduce the complexity in data analysis and data interpretation. We only included participants with one clear conversion in the conversion group and no conversion in the stable group. For example, participants in the MCI-to-DementiaAD group had the cognitive status sequence:

$$\begin{array}{rl}MCI⟶& \cdots ⟶MCI\stackrel{convert}{⟶}ADdementia\\ ⟶& \cdots ⟶ADdementia,\end{array}$$

where the status between MCI was always MCI, and the status between AD dementia was always AD dementia. In other words, we included data of these visits in the analysis, and removed the visits before the very first MCI visit in such sequence.

The commonly used approaches were utilized to clean the variables. Variables with more than 50% missing were removed. The cognitive status change from MCI to either AD dementia or LB dementia is interval-censored. For MCI patients without conversion during the follow-up visits in the study, these patients were considered as censored at the last visit, including patients from (1b) and (2b).

Interval-censored data

Let $\mathbf{T}=\{T_1,T_2,\cdots ,T_K\}$ be a vector of visit times for a patient having K follow-up visits. It should be noted that the visit at $T_1$ is the very first visit for a patient having MCI status which may not be the baseline visit, and $T_K$ is always the last visit. At each visit, the diagnostic status is either MCI or dementia.

For MCI patients who have either AD dementia or LB dementia conversion, the actual conversion time (e.g., C) is unknown, but we know it is between two visits, say $T_{i-1}$ and $T_i$ , where $T_{i-1}$ is the visit time right before the conversion and $T_i$ is the visit that has the diagnostic status as AD dementia or LB dementia, $i=2,\cdots ,K$ . Then, we have the conversion time $C\in (T_{i-1},T_i]$ . For patients with conversion, the diagnostic status from visit $T_{i+1}$ to $T_K$ was the same as the diagnostic status at $T_i$ . For MCI patients without conversion, all the diagnostic status was MCI for all the visits from $T_1$ to $T_K$ .

Statistical models for interval-censored data

For interval-censored data, the traditional survival model is the Cox proportional hazards (PH) regression model for IC data. ²⁰ The Cox model is easy to fit but the PH assumption may not be satisfied in some cases (e.g., survival curve crossing). In such cases, machine learning methods may be used to improve the model fitting. Recently, Yao et al. ¹⁴ developed a survival random forest method for IC data based on the conditional inference framework. ¹⁷ We referred to that method as the RF-I model. That random forest model was shown to have similar performance as compared to the Cox model when the relationship is linear. For non-linear relationship data, the RF-I method had better performance than the Cox model with a low mean squared difference between the predicted survival function and the empirical survival function.

We also included the RF model for right-censored data (RF-R) in the comparison as this method was frequently used although this model may not be appropriate as the data are interval-censored, not right-censored. When the RF-R model was used, the information of visit time before the conversion event was not utilized in the analysis.

Model evaluation performance metrics

For interval-censored data, one simple performance metric is the proportion of the correct prediction on censored cases from the MCI stable groups and the correct prediction of conversion time between the observed time interval for patients from the conversion groups. We used P(MCI-to-DementiaAD) and P(MCI-to-DementiaLB) to denote the probability of correct prediction of the dementia conversion from MCI to AD dementia and LB dementia, respectively. We defined P(MCI-with-AD) and P(MCI-with-LB) similarly for MCI patients having MCI status without dementia conversion. It is often a challenge in interval-censored models to have high P(MCI-to- DementiaAD) and P(MCI-to-DementiaLB) values as the predicted conversion time should be within the observed time window, while the right-censored models only need to have the predicted time before the last visit time.

The integrated Brier score (IBS) is commonly used in interval-censored data analysis to evaluate the overall model fit performance. It is calculated from the squared distance between the empirical survival function $I(T_i>t)$ and the estimated survival function $\hat{S}(t|.)$ , where $T_i$ is the dementia onset time for the i-th patient, $i=1,2,\dots ,n.$ Suppose $X_i$ is the value of the covariate vector for patient i in a study. Then, the IBS is defined as

$$IBS(\hat{S})={\displaystyle \frac{1}{N}\sum _{i=1}^N{\displaystyle \frac{1}{t_{max}}\underset{0}{\overset{t_{max}}{\int }}[I((T_i>t)|X_i)-\hat{S}(t|X_i){]}^{2}dt,}}$$

When the distance between $I()$ and $\hat{S}()$ is small, the value of IBS is low. A low value of IBS is preferable in comparing different survival models.

Variable selection in interval-censored statistical processing

For each cohort, we first conducted the analysis by using the conversion time window as the outcome in the Cox interval-censored survival model with each covariate. These models were then ranked by the log-likelihood that measures the goodness of fit for a model. The features associated with the highest log-likelihood values were selected. In addition to the goodness of fit metric, we also checked the direction of the relationship between the outcome and each covariate. The ones that did not meet the expected direction were removed from the list. ²¹ In each cohort, we selected the top 30 measures to be used in the following model selection approach.

We used the forward model selection approach to select the features. The first feature is the one associated with the largest log-likelihood value. We then added each of the remaining features (29 features) to the model that already included the first feature. The second feature was the one whose model had the largest log-likelihood value and met the parameter direction. We repeated this forward model selection approach in conjunction with the parameter direction to select the top 15 features in the AD cohort. With a small sample size in the LB cohort, we were able to select the top 11 features. In many predictive models, the final number of features was often small. With 11 features in the LB cohort and 15 features in the AD cohort, they should be able to select the final features for cohort based on model performance metrics.

Results

For each cohort (LB and AD), we first used data from NACC-v1v2 to build predictive models by using the three survival methods (Cox model, RF-I model, and RF-R model). The predictive models were then used to be tested on the data sets from NACC-v3. The characteristics of the AD training cohort at the very first visit with the MCI status were presented in Table 1. The presented p-values were calculated by using the two-sample t-test for continuous outcomes (e.g., age), and the Fisher's exact test for categorical outcomes (e.g., proportion of female). In the AD training cohort, we had a total of 397 AD patients. Among them, one third of them (n = 266) had the conversion from MCI to AD dementia during the follow-up visits, while the remaining 131 patients remained MCI status. The MCI-to-DementiaAD group and the MCI-with-AD stable group had similar age, proportion of females, education years, and follow-up time in months. The AD dementia conversion group had 5.7% more patients who had at least one copy of APOE ε4 as compared to the MCI stable group.

Table 1.

Demographic table for baseline data from the NACC-v1v2 AD cohort.

Characteristics Sample size	Overall 397	Stable group 131	Conversion group 266	p
Follow-up months, mean (SD)	31.1 (21.49)	31.8 (21.70)	30.7 (21.40)	0.4192
Age in years, mean (SD)	81.9 (8.93)	82.3 (8.50)	81.7 (9.15)	0.7566
Sex, n (%)				0.2627
Male	132 (33.2%)	49 (37.4%)	83 (31.2%)
Female	265 (66.8%)	82 (62.6%)	183 (68.8%)
Education years, mean (SD)	15.2 (3.10)	15.1 (3.06)	15.3 (3.12)	0.4469
APOE ε4, n (%)				0.5601
No ε4 allele	205 (54.7%)	72 (58.5%)	133 (52.8%)
1 copy	145 (38.7%)	44 (35.8%)	101 (40.1%)
2 copies	25 (6.7%)	7 (5.7%)	18 (7.1%)

p is the difference between the stable group and the conversion group.

During an average follow-up of 30 months in the AD training cohort, the conversion rate from MCI to AD dementia was estimated to be 67% from this cohort. This conversion rate from MCI to AD dementia is higher than the reported conversion rate. In the AD conversion group, the average time from the conversion and the visit prior to conversion is 15 months with the SD of 8 months. In the literature, not all MCI patients had the AD pathology or the pathology of other dementia sub-types, which leads to a lower conversion rate than the rate here for the MCI patients with AD pathology.

As compared to the AD cohort, the LB cohort had a much smaller sample size (n = 37 in total), see Table 2. The follow-up time in this cohort was longer in MCI-to-DementiaLB group as compared to the MCI-to-DementiaAD group. The conversion rate from MCI to LB dementia was slightly higher: 75% as compared to 67% conversion rate in the AD cohort. Patients in the MCI-to-DementiaLB group were older than patients in the MCI-with-LB stable group. In the MCI-to-DementiaLB group, the majority of patients were male, while this trend was reversed in the MCI-with-LB stable group. Their education years were close to each other. For the APOE ε4 copies, the conversion group often had a higher proportion of patients with 1 copy or two copies than the stable group in each cohort.

Table 2.

Demographic table for baseline data from the NACC-v1v2 LB cohort.

Characteristics Sample size	Overall 37	Stable group 9	Conversion group 28	p
Follow-up months, mean (SD)	35.3 (25.52)	28.3 (24.90)	37.5 (25.70)	0.1541
Age in years, mean (SD)	76.8 (7.85)	71.3 (3.87)	78.6 (8.02)	0.0167
Sex, n (%)				0.3871
Male	23 (62.2%)	4 (44.4%)	19 (67.9%)
Female	14 (37.8%)	5 (55.6%)	9 (32.1%)
Education years, mean (SD)	15.6 (3.65)	15.3 (1.32)	15.6 (4.15)	0.9141
APOE ε4, n (%)				0.5482
No ε4 allele	19 (61.29%)	6 (75.00%)	13 (56.50%)
1 copy	10 (32.26%)	2 (25.00%)	8 (34.80%)
2 copies	2 (6.45%)	0 (0.00%)	2 (8.70%)

p is the difference between the stable group and the conversion group.

As compared to the AD cohort, the LB cohort had 6.6% fewer patients who have at least one copy of APOE ε4. MCI patients in the LB cohort were 5 years younger on average than MCI patients in the AD cohort. In addition, patients in the LB cohort had a high percentage of males: 62% versus 33% in the AD cohort.

Model training with NACC-v1v2 data

To build a predictive model by using the NACC-v1v2 data, we used the Monte Carlo Cross Validation (MCCV) instead of the traditional k-fold cross-validation method that is often used to save time but could introduce parameter estimation bias due to very limited simulations. ²² In the MCCV, we randomly sampled 90% of data as the training data to build a predictive model. Then, that model was used to test on the remaining 10% data. We used the MCCV with 5000 simulations to have reliable performance metrics for the AD cohort. For the LB cohort with a very small sample size (n = 9 in MCI-with-LB stable group and n = 28 in the MCI-to-DementiaLB group), we modified the MCCV to make sure we had some patients from each group in the training data and the testing data. In each simulation, we randomized selected 66% patients from the MCI-to-DementiaLB group and another 66% patients from the MCI-with-LB stable group as the training data. Then, the remaining 34% data were the testing data.

For the AD cohort, we presented the IBS values in Figure 1 with the IBS value as a function of the first D measures, where D = 1, 2, · · ·, 15. The IBS values of the RF-I model were relatively flat as the number of features went up. The Cox-I model started to have worse IBS values when the number of features was above 7. The RF-R model's IBS values were much larger than the two IC models. In Figure 2, we plotted the proportion of correct prediction for MCI-to-DementiaAD, and MCI-with-AD stable status. When the number of features was 3 or less, the correct prediction probabilities were found to be high for P(MCI-to-DementiaAD), but the variation of the prediction could be large due to a small number of features. We chose the number of features as 5 here to have a good balance between the correct prediction and the IBS value.

Figure 1.

The IBS values for the three models as a function of the first D measures (D = 1, 2, · · ·, 15) from 5000 MCCV simulations by using the AD cohort from the NACC-v1v2 data.

Figure 2.

The proportion of correct prediction for the MCI-to-DementiaAD group (top), and the MCI-with-AD stable group (bottom) as a function of the number of measures for the AD cohort using the NACC-v1v2 data.

These selected five measures were: (1) The orientation subscale from the Clinical Dementia Rating Sum of Boxes (CDR-SB) with the score range from 0 to 3; (2) total number of animals named in 60 s; (3) irritability or lability severity score from Neuropsychiatric Inventory Questionnaire (NPI-Q) ²³ ; (4) logical memory IIA delayed score that counts the total number of story units recalled; and (5) the status of angioplasty or endarterectomy or stent (see Table 3). The orientation subscale was suggested as an important measure to predict the conversion from MCI to AD.^21,24–26

Table 3.

Predictors of AD dementia or LB dementia with their baseline mean (SD) using the NACC-v1v2 data.

The AD cohort
Predictor	Overall (n = 397)	Stable (n = 131)	Conversion (n = 266)	p
TRAVEL	0.40 (0.78)	0.26 (0.64)	0.48 (0.83)	0.0092
STAYHOME	0.29 (0.45)	0.33 (0.47)	0.27 (0.45)	0.2082
Trail-B	154.4 (78.22)	147.0 (77.20)	158.0 (78.70)	0.1905
Trail-A	45.8 (23.10)	45.6 (22.50)	45.9 (23.40)	0.9555
ORIENT	0.14 (0.25)	0.06 (0.16)	0.18 (0.27)	< 0.0001
ANIMALS	14.8 (4.48)	15.9 (4.75)	14.3 (4.25)	0.0014
IRRSEV	0.28 (0.60)	0.24 (0.53)	0.30 (0.63)	0.5548
MEMUNITS	7.2 (4.58)	8.0 (4.76)	6.8 (4.45)	0.0419
CVANGIO	0.06 (0.23)	0.06 (0.24)	0.06 (0.23)	0.8382
The LB cohort
Predictor	Overall (n = 37)	Stable (n = 9)	Conversion (n = 28)	p
TRAVEL	0.56 (1.00)	0.44 (1.01)	0.59 (1.01)	0.5893
STAYHOME	0.37 (0.49)	0.22 (0.44)	0.42 (0.50)	0.2998
Trail-B	194.5 (87.04)	150.0 (66.30)	209.0 (89.10)	0.0991
Trail-A	53.4 (22.97)	48.0 (16.7)	55.2 (24.7)	0.3704
ORIENT	0.14 (0.23)	0 (0)	0.18 (0.24)	0.0406
ANIMALS	17.1 (4.87)	17.9 (4.94)	16.8 (4.91)	0.4976
IRRSEV	0.29 (0.72)	0.11 (0.33)	0.36 (0.81)	0.5179
MEMUNITS	10.7 (4.60)	12.3 (5.83)	10.2 (4.10)	0.4637
CVANGIO	0.14 (0.35)	0.11 (0.33)	0.14 (0.36)	0.8343

p is the difference between the stable group and the conversion group.

For the LB cohort, the IBS value of the RF-I model was lower when the number of features was 8 or less as compared to the cases with more features (see Figures 3 and 4). Both the Cox model and the RF-R model had the IBS values being higher than the RF-I model. For the probability of correct prediction in Figure 3, the predictive models with the number of features of 3 to 5 were seen to have good performance. To reduce the prediction variation, we would like to choose the final predictive model with 5 features to have robust predictions (see Table 3): (1) the level of difficulty related to travel which requires higher cognitive ability compared to basic daily tasks; (2) age; (3) preference on staying at home or going out; (4) Trail-B to measure executive function-related information processing speed; and (5) Trail-A to assess attention (see Table 3).

Figure 3.

The IBS values for the three models as a function of the first D measures (D = 1, 2, · · ·, 15) from 5000 MCCV simulations by using the LB cohort from the NACC-v1v2 data.

Figure 4.

The proportion of correct prediction for the MCI-to-DementiaAD group (top), and the MCI-with-AD stable group (bottom) as a function of the number of measures for the LB cohort using the NACC-v1v2 data.

Model testing

We conducted external validation on the NACC-v3 data set by using the final predictive model with the identified 5 features to predict AD dementia from MCI in the AD cohort, see Supplemental Table 1 for their baseline characteristics. The average follow-time was longer in the conversion group. They were similar with regards to age, education years, and female proportion. Although the conversion group had more patients with one or two APOE ε4 copies than the stable group, their difference was not significant. The conversion group had a much worse orientation score and memory score as compared to the MCI stable group.

The RF-I model had the smallest IBS value of 0.095, followed by the Cox model (0.104) and the RF-R model (0.127). These models had similar correct prediction probabilities being close to 50% although the RF-R model had 1% to 2% higher than the IC models.

For the LB cohort, we presented their baseline characteristics in Supplemental Table 2. With a very small sample size in NACC-v3 LB cohort, we did not find significant difference between the LB dementia conversion group and the LB stable group. The RF-R model had a smaller IBS value than the two IC models, but the RF-I model had a much higher proportion of correct prediction than the RF-R model (72% versus 56%). The Cox model had a similar proportion of correct prediction as compared to the RF-R model.

Discussion

Among the selected measures to predict conversion from MCI to dementia, neuropsychological total and sub-scale measures were identified from the two cohorts: orientation subscale and delayed memory score for the AD cohort and trail making tests for the LB cohort. The depression scale and functional activities were found to be predictive of LB dementia while neuropsychiatric measure was associated with the onset of AD dementia. This may fit with general cognitive profiles seen in LB and AD. It has been suggested that patients with LB dementia tend to have more deficits in processing speed, visuospatial and executive tasks. ⁷ Behavioral changes characterized by depressive symptoms are also common. Compared to LB dementia patients, those with AD dementia tend to make more semantic errors but fewer visuoperceptual errors on the Boston naming test.^27,28 However, the Boston test was not included in our model since it has many missing values in this database.

Trail-A and Trail-B predicted the onset of LB dementia as compared to the LB stable group. Although Trail-B was not on the top predictors in the AD cohort, it was ranked 8th in the predictive model in predicting the onset of AD dementia. Both Trail-A and Trail-B are frequently used to assess cognitive impairment, with Trail-B measuring more complex cognitive abilities than Trail-A. Trail-B was found to be associated with logical memory delayed score, gait speed, and grip strength by using data from the Long Life Family Study (LLFS). ²⁹ In another combat sports study, the change in Trail-B was statistically significant in the decliner group defined by the yearly caudate volumetric change. ³⁰

In this study, we included patients with both AD and LB pathology in the AD cohort when AD was the primary pathology. Among the 266 patients from the MCI-with-AD group, there was only one patient who had LB as the contributing cause of cognitive impairment and another patient with LB as the non-contributing cause of cognitive impairment. The low diagnosis rate of LB may be caused by the challenge in distinguishing its early symptoms from those of AD and/or mental illness. ³¹ This mixed group was found to be similar to the AD group regarding APOE ε4 rates, male proportion, and cardiovascular risk. ³² As the sample size for the mixed AD and LB group was so small, it is a challenge to run machine learning methods to identify risk factors.

Identifying characteristics that can be captured clinically and correlate with likelihood to decline can aid in decision making with patients—not only for planning reasons but also in choosing participants for clinical trials. Appreciating that there is always going to be overlap in symptoms and signs between AD and LB, the analysis we undertook indicates that models can be developed to predict change for specific etiologies of dementia; future steps would entail examining other cohorts and adding emerging biomarkers.