A Late Life Risk Index for Severe Cognitive Impairment in Mexico

Abstract

Background

Several dementia risk indices have been developed for older adults in high-income countries. However, no index has been developed for populations in low- or middle-income countries.

Objective

To create a risk index for predicting severe cognitive impairment among adults aged ≥60 in Mexico and to compare the accuracy of this index to the Dementia Screening Indicator (DSI).

Methods

This study included 3,002 participants from the Mexican Health and Aging Study (MHAS) interviewed in 2001 and 2012. The MHAS risk index included sociodemographic, health, and functional characteristics collected in 2001. A point value based on the beta coefficients from a multivariable logistic regression model was assigned to each risk factor and the total score was calculated.

Results

The MHAS risk index (AUC = 0.74 95% CI = 0.70–0.77) and DSI (AUC = 0.72 95% CI = 0.69–0.77) had similar accuracy for discriminating between participants who developed severe cognitive impairment from those who did not. A score of ≥16 on the MHAS risk index had a sensitivity of 0.69 (95% CI = 0.64–0.70) and specificity of 0.67 (95% CI = 0.66–0.69). A score of ≥23 on the DSI had a sensitivity of 0.56 (95% CI = 0.50–0.63) and specificity of 0.78 (95% CI = 0.76–0.79).

Discussion

The MHAS risk index and DSI have moderate accuracy for predicting severe cognitive impairment among older adults in Mexico. This provides evidence that existing dementia risk indices may be applicable in low- and middle-income countries such as Mexico. Future research should seek to identify additional risk factors that can improve the accuracy of the MHAS risk index.

INTRODUCTION

Dementia and severe cognitive impairment are significant public health concerns in the United States and abroad. The prevalence of dementia in high-income countries is similar to low- and middle-income countries with an estimated 7% of adults aged 60 and over in the United States living with dementia compared to 8.5% in Latin America [1]. However, it is expected that low- and middle-income countries, in particular Latin America, will experience a greater increase in dementia prevalence compared to North America, Europe, and other high-income countries. The prevalence of dementia in Latin American countries is projected to increase by 120% by the end of 2020 and 393% by 2040, compared to 49% and 172% in North America during the respective time periods [2].

The expected increase in dementia prevalence in Latin America will be largely due to population aging [3], but substantial differences in educational attainment among older adults from Latin American countries compared to the United States will also have an important role. Cross-national research using data from the Mexican Health and Aging Study (MHAS) and the United States Health and Retirement Study (HRS) indicates that approximately 30% of adults aged 50 and over in Mexico have completed seven or more years of education whereas 52% of adults aged 50 and over in the United States who have completed over twelve years of education [4]. Greater educational attainment is associated with a decreased risk for dementia [5] and preserved cognitive functioning during old age [6]. The observed benefits of education on cognition have been attributed to greater cognitive reserve among highly educated older adults compared to those who are poorly educated [7]. Cognitive reserve describes the use of alternative or more efficient neural pathways when performing cognitive tasks [8], which allows for an older adult to maintain normal levels of cognitive functioning despite the presence of dementia neuropathology [9].

There is growing consensus that administering treatments and interventions before clinical symptoms of dementia are observed may be effective strategies to prevent or delay the onset of dementia [10, 11]. This makes it necessary to create risk indices that can be used to identify individuals who do not have dementia, but who are at high-risk for the disease. The first dementia risk index included midlife measures for age, education, gender, physical activity, body mass index, total cholesterol level, and blood pressure and was able to moderately predict dementia status (area under the curve [AUC] 0.77) [12]. Additional risk indices that have used self-reported measures have also been shown to predict dementia status in North American populations with moderate accuracy [13–17]. Other risk indices that used brain imaging, genetic, and serum biomarkers can predict dementia status with high accuracy (AUC > 0.85) [18–20], but the high cost associated with collecting these measures limits the use of these indices in a clinical setting [21]. Additionally, some late life risk indices have been created using sample populations that include older adults with mild cognitive impairment (MCI) or who have concerns with their memory [13, 14, 22]. While dementia incidence is higher among older adults diagnosed with MCI compared to older adults in the general population [23], it has been reported that over 50% of older adults with MCI do not progress to dementia or revert back to a state of normal cognitive functioning [24]. Thus, including older adults with MCI or who have concerns about their memory in the analytic sample may decrease the accuracy of the risk index to predict dementia [25].

The expected increase in dementia prevalence in Latin American countries signifies the need to externally validate current indices in non-white populations [26] and, if necessary, to create risk indices to identify high-risk individuals for dementia in these populations. However, existing risk indices have not been validated in older adult populations in low- and middle-income countries such as Latin America, and it is unknown if these indices are able to accurately predict dementia in these populations or if risk indices need to be created specifically for these populations.

In the present study, we use data from the MHAS to create a late life risk index that can serve to identify older Mexican adults who are at high risk for developing severe cognitive impairment over an 11-year period. Additionally, we use data from the MHAS to compare with the Dementia Screening Indicator (DSI) [27], which is an existing late life dementia risk index that has been found to predict dementia with moderate accuracy in four cohort studies in the United States. Finally, we compared the ability of the DSI and MHAS late life risk index to predict severe cognitive impairment over an 11-year period.

MATERIALS AND METHODS

Mexican health and aging study and sample selection

The MHAS is the first nationally representative longitudinal study of aging in Mexico [28]. The survey instruments and overall design of the MHAS are highly harmonized with the U.S. Health and Retirement Study in order to conduct cross-national aging research between Mexico and the United States. The MHAS cohort was established in 2001 (Wave I) and included a total of 15,186 adults born in 1951 or earlier (birth year range 1896–1951) from urban and rural regions from all 32 states in Mexico. Participants were reinterviewed 2-years later in 2003 (Wave II) and 11-years later in 2012 (Wave III). The response and follow-up rates of the MHAS have been high with 91.8%, 93.3%, and 88.1% of eligible participants being interviewed in Waves I, II, and III, respectively [28]. Among those lost to follow-up by Wave II, 4.2% could not be located and 2.5% refused to complete the Wave II interview. Among those lost to follow-up between Waves II and III, 7.8% could not be located and 3.1% refused to complete the Wave III interview [28]. A total of 546 participants died between Waves I and II and 2,196 participants died between Wave II and Wave III. All three waves of data collection have included detailed measures for sociodemographic characteristics, family structure, current self-reported health conditions, lifestyle characteristics, history of migration to and from the United States, economic status, cognition, and participants’ childhood health and living conditions. The protocol for the MHAS has been approved by the University of Texas Medical Branch Institutional Review Board.

Data from Waves I and III were used in the present study to allow for a sufficient number of participants to transition from normal cognition to severe cognitive impairment. Figure 1 illustrates the selection of the final analytic sample for the present study (n = 3,002). The final sample included participants who during Wave I were aged 60 and older, were directly interviewed during Wave I, completed 2 or more cognitive assessments during Wave I, and did not have an evidence for severe or moderate cognitive impairment during Wave I. Participants who required a proxy because of illness, hospitalization, or temporary absence to complete the core questionnaire during Wave I were excluded from the sample because data for depressive symptoms, vision, hearing, self-reported overall health, and instrumental activities of daily living (IADLs; preparing a hot meal, shopping for groceries, taking medications, and managing money) were only collected from participants who were directly interviewed. Additionally, the final sample included participants that were directly interviewed who completed 2 or more cognitive assessments during Wave III or, if they were unable to be directly interviewed, had a proxy measure for cognition during Wave III. Finally, 73 participants who reported having never seen a physician or other medical professional were excluded because these participants were not asked about self-reported health conditions.

Fig. 1.

Selection of final sample.

A total of 1,783 participants died and 638 participants were lost to follow up by Wave III (Fig. 1). These participants were more likely to be male, were significantly older, completed fewer years of education, were less likely to be married, and were more likely to have self-reported hypertension, diabetes, heart attack, stroke, be a current or former smoker, be a former alcohol consumer, be disabled in ≥1 ADLs (walking across a room, bathing/showering, eating, and getting in/out of bed, using the toilet) or ≥2 IADLs, and to have high depressive symptoms compared to participants who were interviewed during Wave III.

Definition of severe cognitive impairment

Cognitive functioning of participants who received a direct interview was measured using the Cross-Cultural Cognitive Examination (CCCE) [29]. A detailed description of the administration and scoring of the CCCE in the MHAS has been provided by Mejia-Arango et al. [30]. The CCCE includes measures for verbal learning (immediate recall of verbal list of eight words, score 0–8 points), verbal memory (repeat list of eight words from verbal learning trial, score 0–8 points), visuospatial abilities (copying of two figures, score 0–6 points), visual memory (recall of two figures from visuospatial abilities trial, score 0–6 points), and attention (one minute to identify target stimulus (60) among a visual array of other stimuli, score 0–60 points). Measures for orientation (recite current day, month, and year, score 0–3 points), verbal fluency (name as many animals in 60 seconds, score 0–60), and numeracy (time to correctly count backwards from 20 to 11, score 0–60 points) were supplemented to the instrument in Wave III and were included in the assessment of cognitive status at Wave III.

The criteria of severe cognitive impairment for participants who received the CCCE was based on the diagnostic criteria for dementia proposed by the National Institute on Aging-Alzheimer’s Association workgroups [31]. Severe cognitive impairment was defined as scoring ≥1.5 standard deviations lower than what would be expected based on the participant’s age on 2 or more cognitive measures and difficulty performing 1 or more ADLs and/or 2 or more IADLs. This definition of severe cognitive impairment is also consistent with the criteria for dementia using the CCCE that has been validated in a clinical sample of 173 subjects in Mexico with a sensitivity of 84.2% and specificity of 100% [32]. Participants were required to have impairment in 2 or more IADLs because Mexican culture contributes to gender differences in IADLs (e.g., men do not typically shop for groceries or prepare a meal and women do not typically manage money).

Cognitive functioning for participants who required a proxy to complete the Wave III interview was measured using the Informant Questionnaire on Cognitive Decline in the Elderly (IQCODE) [33]. The IQCODE includes 16 questions and is administered by an interviewer to a family member or relative with knowledge of changes in the participant’s cognitive functioning over a 2-year period (1-much improved, 3-not much change, 5-much worse). The IQCODE can be summarized on a 1 to 5 scale by calculating the average of the 16 items. An average score of ≥3.4 has been reported to identify dementia patients with a sensitivity of 89% and specificity of 94% [33]. Severe cognitive impairment was defined as an average score ≥3.4 on the IQCODE and difficulty performing ≥1 ADLs and/or ≥2 IADLs.

Risk factors for severe cognitive impairment

All risk factors for severe cognitive impairment were measured at Wave I (2001). Risk factors were selected based on evidence from previous studies [34–39] and other dementia risk indices [16, 22, 40]. Additional risk factors that may be unique to Mexican older adults were also selected. Risk factors were defined according to six categories: (1) sociodemographic characteristics; (2) health conditions; (3) functional status; (4) lifestyle behaviors; (5) early childhood health and living conditions; and (6) social engagement. Sociodemographic characteristics included age (60–64, 65–69, 70–74, ≥75 years), current marital status (married/not married), gender (male/female), locality (rural/urban setting), paid and unpaid employment history (no/yes), and being able to speak English (no/yes). Educational attainment was dichotomized as having completed 0 to 3 years of education (no/yes) because 3 years was the median number of years of education completed in the final sample. Health conditions included self-reported measures for hypertension, diabetes, stroke, heart attack, arthritis, fair-poor vision, fair-poor hearing, ever having broken a bone, currently experiencing mild to severe pain, self-reported overall health status, and depressive symptoms. Depressive symptoms were assessed using nine questions from the Center for Epidemiological Studies Depression Scale that asked if a participant felt depressed, if everything they did was effort, had restless sleep, felt happy, felt lonely, felt that they enjoyed life, felt sad, felt tired, and felt they had a lot of energy. Participants who had six or more symptoms were classified as having high depressive symptoms [41]. Functional status included being unable to independently perform 1 or more ADLs, 2 or more IADLs, and having experienced a fall within the past two years. Lifestyle behaviors included smoking status (never, former, current), currently consuming alcohol (no/yes), and during the past two years having regularly engaged in exercise or hard physical work for three or more times a week (yes/no). Early childhood health and living conditions included having experienced a serious health event before 10 years of age or having experienced a traumatic head injury before 10 years of age. Living conditions during childhood were determined based on participants’ self-reported responses about having a toilet in the household, which is used as a proxy measure of childhood socioeconomic status [42]. Social engagement measures included talking with friends at least once a day in the past month and having spent any time doing volunteer work for the community, religious, educational, or charity organizations in the last two years.

Development of MHAS late life risk index

The MHAS late life risk index was created using a multistep processes based on earlier studies [12, 16]. First, bivariate analyses were conducted to identify risk factors associated with severe cognitive impairment at α ≤0.1 level of significance [16]. Second, risk factors that were identified during the bivariate analysis (step one) were included in separate logistic regression models that controlled for age, gender, and educational attainment. Third, risk factors that remained associated with severe cognitive impairment in step two at α ≤0.1 level of significance were included in a multivariable logistic regression model. A numerical value was assigned to each risk factor in the multivariable logistic regression model by multiplying each β coefficient by a common factor so that the risk factor with the lowest β coefficient would have a score of 1. Since the lowest β coefficient in the multivariable model was 0.10 and multiplying this value by 10 equals 1, all β coefficients were multiplied by 10 and rounded to the nearest whole number. The sum of these values was then calculated to obtain the MHAS late life risk index score, which ranged from 0 to 59 points.

The effectiveness of the MHAS late life risk index to discriminate between participants who developed severe cognitive impairment over the 11-year follow-up period from those who did not was determined using receiver operating characteristic (ROC) analysis. This analysis generates an ROC curve, which is a plot of the sensitivity and specificity for all possible cut offs of the risk index. ROC analysis was also used to calculate the sensitivity, specificity, positive predictive value (ppv), negative predictive value (npv), and accuracy (proportion of cases and controls who are correctly classified) for the optimal cut point of the risk index score. The optimal cut point was determined according to the Youden index, which identifies the score that maximizes the combined sensitivity and specificity [43].

Logistic regression was used to estimate the relative odds of severe cognitive impairment associated with a one-unit increase in the MHAS late life risk factor score. In addition, participants were categorized as low, moderate, and high risk according to the MHAS late life risk index score. The odds for severe cognitive impairment for each risk category were estimated using logistic regression models. Data analysis was conducted using R version 3.1 [44].

Internal validation of MHAS late life risk index

Bootstrapping was used to internally validate the MHAS late life risk index. Bootstrapping involves drawing participants at random with replacement from the full sample to generate a series of bootstrap samples. The sample statistic (e.g., AUC) is calculated in each of the bootstrap samples and the mean sample statistic is then calculated. We generated 3000 bootstrap samples and the mean AUC was calculated using the bootstrapped samples. A 95% confidence interval of the AUC was obtained using the distribution of estimated AUC values from the bootstrapped samples [45]. Bootstrapping was also used to estimate the sensitivity, specificity, ppv, npv, and overall accuracy and corresponding confidence intervals.

Dementia screening index

The ability of MHAS late life risk index to predict severe cognitive impairment over an 11-year period was compared to the DSI. The DSI is a late life dementia risk index that was designed to be used in a primary care setting to identify older adults who are high-risk to develop dementia within a six-year period [27]. The index was created using data from the Cardiovascular Health Study (CHS), Framingham Heart Study (FHS), Health and Retirement Study (HRS), and Sacramento Area Latino Study on Aging (SALSA) to identify a uniform set of dementia risk factors across all four cohorts. The final index included age (1 point per year older than 65), having completed fewer than 12 years of education (9 points), stroke (6 points), type II diabetes (3 points), being underweight (8 points; body mass index <18.5 kg/m²), needing assistance with managing finances or taking medications (10 points), and depressive symptoms (6 points; everything an effort, sleep was restless, could not get going, or current antidepressant use). The AUC values indicate that the DSI was able to accurately discriminate between high and low risk older adults in all four cohorts (CHS 0.68 [95% CI = 0.65–0.72], FHS 0.77 [95% CI = 0.73–0.82], HRS 0.76 [95% CI = 0.74–0.77], SALSA 0.78 [95% CI = 0.72–0.83]).

The DSI was chosen for the present study because all of the variables included in the DSI are available in the MHAS. However, direct measures for height and weight have only been collected in a subsample of 2,944 participants from the entire Wave I sample. A total of 583 participants in the final sample had direct measures for height and weight, and only 8 of these participants had a body mass index of less than 18.5 kg/m². Thus, being underweight was not included in the external validation of the DSI. The point values for the remaining risk factors of the DSI were applied to the variables in the MHAS and the total score was calculated. The AUC of the DSI and the sensitivity, specificity, npv, ppv, and overall accuracy of the optimal cut point as determined by the Youden index were compared to the values of the MHAS late life index.

RESULTS

Table 1 presents the baseline (Wave I) characteristics of the 3002 participants included in the final sample according to severe cognitive impairment status in Wave III. Participants classified as having severe cognitive impairment (n = 251, 8.4%) were significantly older, and more likely to be female, to have completed 0 to 3 year of education, to not be married, to have hypertension, to be diabetic, to have experienced a stroke, to report being in fair-poor health, to have arthritis, high depressive symptoms, not currently consuming alcohol, to have fair-poor vision, fair-poor hearing, to have fallen in the past two years, and to have limitations in 1 or more ADLs and 2 or more IADLs compared to participants who did not have severe cognitive impairment (all p < 0.05).

Table 1.

Sociodemographic, health, functional, lifestyle, social engagement, and early childhood health and living characteristics of final sample (n = 3,002)

	Cognitive Status in Wave III (2012)
Characteristic in Wave I (2001), n (%)	Non-Severely Impaired n = 2,751 (91.6%)	Severely Impaired n = 251 (8.4%)	Total cohort n = 3,002	p
Socio-demographics
Age (years)				<0.01
60–64	1258 (45.7)	54 (21.5)	1312 (43.7)
65–69	835 (30.4)	59 (23.5)	894 (29.8)
70–74	415 (15.1)	66 (26.3)	481 (16.0)
≥75	243 (8.8)	72 (28.7)	315 (10.5)
Gender				<0.01
Male	1170 (42.5)	79 (31.5)	1249 (41.6)
Female	1581 (57.5)	172 (68.5)	1753 (58.4)
Educational attainment				<0.01
0–3 years	1572 (57.2)	177 (70.5)	1749 (58.3)
≥4 years	1177 (42.8)	74 (29.5)	1251 (41.7)
Marital status				<0.01
Not married/consensual union	935 (34.0)	110 (43.8)	1045 (34.8)
Married/consensual union	1816 (66.0)	141 (56.2)	1957 (65.2)
Employment history (paid)				0.09
No	620 (22.6)	68 (27.2)	2311 (77.1)
Yes	2129 (77.4)	182 (72.8)	688 (22.9)
Employment history (non-paid)				0.73
No	1927 (70.7)	172 (69.6)
Yes	800 (29.3)	75 (30.4)
Locality				0.60
Urban	1788 (65.0)	159 (63.3)	1947 (64.9)
Rural	963 (35.0)	92 (36.7)	1055 (35.1)
Speak English				0.70
No	2558 (93.0)	235 (93.6)	2793 (93.0)
Yes	193 (7.0)	16 (6.4)	209 (7.0)
Health Conditions
Hypertension				0.04
No	1650 (60.2)	134 (53.6)	1784 (59.6)
Yes	1093 (39.8)	116 (46.4)	1209 (40.4)
Diabetes				<0.01
No	2419 (88.1)	197 (78.5)	2616 (87.3)
Yes	326 (11.9)	54 (21.5)	380 (12.7)
Heart attack				0.55
No	2665 (97.1)	242 (96.4)	2907 (97.0)
Yes	80 (2.9)	9 (3.6)	89 (3.0)
Stroke				<0.01
No	2689 (98.0)	238 (94.8)	2927 (97.8)
Yes	54 (2.0)	13 (5.2)	67 (2.2)
Self reported health				<0.01
Fair – poor	1771 (64.4)	192 (76.5)	1963 (65.4)
Good	833 (30.3)	55 (21.9)	888 (29.6)
Very good – excellent	146 (5.3)	4 (1.6)	150 (5.0)
Arthritis				0.02
No	2086 (76.1)	173 (69.2)	2259 (75.5)
Yes	656 (23.9)	77 (30.8)	733 (24.5)
High depressive symptoms				<0.01
No	1714 (62.7)	126 (51.4)	1840 (61.8)
Yes	1018 (37.3)	119 (48.6)	1137 (38.2)
Vision				<0.01
Excellent-good	1541 (56.3)	116 (46.4)	1657 (55.5)
Fair-poor	1195 (43.7)	134 (53.6)	1329 (44.5)
Hearing				0.03
Excellent-good	1993 (73.7)	165 (67.1)	2158 (73.1)
Fair-poor	713 (26.3)	81 (32.9)	794 (26.9)
Ever broken a bone				0.97
No	2316 (84.3)	211 (84.4)	2527 (84.3)
Yes	431 (15.7)	39 (15.6)	470 (15.7)
Chronic pain				0.15
No	1630 (59.3)	137 (54.6)	1767 (58.9)
Yes	1121 (40.7)	114 (45.4)	1235 (41.1)
Functional Status
≥1 ADL limitation				<0.01
No	2587 (94.0)	216 (86.1)	2803 (93.4)
Yes	164 (6.0)	35 (13.9)	199 (6.6)
≥2 IADL limitation				<0.01
No	2709 (98.5)	234 (93.2)	2943 (98.0)
Yes	42 (1.5)	17 (6.8)	59 (2.0)
Fallen past 2 years				<0.01
No	1691 (61.5)	125 (49.8)	1816 (60.5)
Yes	1059 (38.5)	126 (50.2)	1185 (39.5)
Lifestyle Behaviors
Smoking status				0.57
Never	1589 (57.8)	151 (60.2)	1740 (58.0)
Former	791 (28.8)	72 (28.7)	863 (28.8)
Current	370 (13.5)	28 (11.2)	398 (13.3)
Current alcohol consumer				<0.01
No	1943 (70.8)	205 (82.3)	2148 (71.8)
Yes	800 (29.2)	44 (17.7)	844 (28.2)
Physically active				0.09
No	1807 (66.2)	179 (71.6)	1986 (66.7)
Yes	921 (33.8)	71 (28.4)	992 (33.3)
Early Childhood Health and Living Conditions
Serious health event				0.78
No	2394 (88.3)	214 (87.7)	2608 (88.3)
Yes	317 (11.7)	30 (12.3)	347 (11.7)
Head injury				0.39
No	2623 (96.2)	232 (95.1)	2855 (96.1)
Yes	104 (3.8)	12 (4.9)	116 (3.9)
Toilet in household				0.20
No	1935 (70.8)	185 (74.6)	2120 (71.1)
Yes	799 (29.2)	63 (25.4)	862 (28.9)
Social Engagement
Talk ≥ 1 times per day with friends				0.26
No	1658 (61.8)	161 (65.4)	1819 (62.1)
Yes	1024 (38.2)	85 (34.6)	1109 (37.9)
Volunteer in past 2 years				0.11
No	2325 (84.6)	220 (88.4)	2545 (84.9)
Yes	423 (15.4)	29 (11.6)	452 (15.1)

Percentages based on column totals. Bivariate analyses conducted using chi-square tests. Totals may not sum to 3002 because of missing values. Italic indicates p < 0.05.

The bivariate analyses (step one) identified several risk factors for severe cognitive impairment. These characteristics included higher age (65–69, 70–74, ≥75 years), female gender, low education, not being married, hypertension, diabetes, stroke, high depressive symptoms, fair-poor overall health, fair-poor vision, fair-poor hearing, ADL impairment, IADL impairment, and having fallen within the past two years (all p < 0.1). After adjusting for age, gender, and educational attainment (step two), hypertension, diabetes, stroke, high depressive symptoms, IADL and ADL impairment, having fallen in the past two years, and fair-poor vision remained associated with severe cognitive impairment at a significant level of α ≤ 0.1. The risk factors identified in step 2 were then included in a multivariable logistic regression model (step 3). The β coefficients from this model ranged from 0.10 to 1.94 and the scores for each risk factor ranged from 1–19 points (Table 2). The total score for the MHAS late life risk index ranged from 0 to 59 points.

Table 2.

Final model and scores for each risk factor according to β coefficients

Characteristic	β		Odds Ratio (95% CI)	p	Score
Age (ref= 60–64)	~		~	~
65–69	0.57		1.77 (1.20–2.64)	<0.01	6
70–74	1.27		3.57 (2.39–5.36)	<0.01	13
75+	1.94		6.97 (4.65–10.52)	<0.01	19
Female (ref = male)	0.24		1.27 (0.94–1.74)	0.13	2
0–3 years education (ref = 4 + years)	0.28		1.33 (0.98–1.81)	0.07	3
Hypertension (ref =no)	0.10		1.11 (0.83–1.48)	0.5	1
Diabetes (ref=no)	0.84		2.32 (1.62–3.28)	<0.01	8
Stroke (ref= no)	0.69		2.00 (0.91–4.03)	0.07	7
High depressive symptoms (ref= no)	0.31		1.37 (1.00–1.86)	0.05	3
IADL impaired (ref = no)	0.97		2.63 (1.26–5.32)	0.01	10
ADL impaired (ref = no)	0.24		1.27 (0.76–2.05)	0.35	2
Fallen in past two years (ref = no)	0.17		1.19 (0.89–1.58)	0.23	2
Fair-poor vision (ref = good to excellent)	0.17		1.18 (0.88–1.57)	0.26	2
Total					59

The riskindex score was obtained by multiplying each β coefficient by 1 ÷ 0.10 and then calculating the column total.

Accuracy of MHAS late life risk index

The results of the ROC analysis indicated that the MHAS late life risk index was able to discriminate with moderate accuracy between participants who developed severe cognitive impairment from those who did not (Table 3). The AUC was 0.74 (95% CI = 0.70–0.77) and the optimal cut point as estimated by the Youden index was 16 points. A total of 1,029 (35.6%) participants had a risk index score of ≥16 points. This cut point had a sensitivity of 0.69 (95% CI = 0.64–0.70), specificity of 0.67 (95% CI = 0.66–0.69), ppv of 0.16 (95% CI = 0.14–0.18), npv of 0.96 (95% CI = 0.95–0.97), and a diagnostic accuracy of 68%.

Table 3.

Accuracy of the Dementia Screening Index (DSI) and MHAS Late Life Risk Index

	MHAS (95% CI)	DSI (95% CI)
Area under the curve	0.74 (0.70–0.77)	0.72 (0.69–0.76)
Sensitivity	0.69 (0.64–0.70)	0.57 (0.50–0.63)
Specificity	0.67 (0.66–0.69)	0.78 (0.76–0.79)
Positive predictive value	0.16 (0.14–0.18)	0.19 (0.16–0.22)
Negative predictive value	0.96 (0.95–0.97)	0.95 (0.94–0.96)
Accuracy	68%	76%

The sensitivity, specificity, positive predictive value, negative predictive value, and accuracy were calculated according to an optimal cut-off score of 16 points for the MHAS late life risk index and 23 points for the DSI.

A one-point increase in the MHAS late life risk index score increased the odds for severe cognitive impairment by 11% (OR 1.11, 95% CI = 1.09–1.12). Participants who scored ≥19 points, 13–18 points, and 7–12 points on the risk index were categorized as high, moderate, and low risk, respectively. Compared to participants who scored between 0–6 points on the MHAS late life index, those who scored between 13–18 points, and ≥19 points had 2.36 (95% CI = 1.43–4.01) and 7.34 (95% CI = 4.75–11.85) times higher odds, respectively, for developing severe cognitive impairment (Table 4, Fig. 2). Participants who scored between 7–12 points did not have significantly higher odds for severe cognitive impairment (OR = 1.36, 95% CI = 0.79–2.37).

Table 4.

Odds for severe cognitive impairment according to MHAS late life risk index score category

Risk score	Number	*Cases, N (%)	OR (95% CI)	p value
0–6	749	23 (3.1)	1 (ref)	~
7–12	774	32 (4.1)	1.36 (0.79–2.37)	0.27
13–18	646	45 (7.0)	2.36 (1.43–4.01)	<0.01
19+	721	136 (18.9)	7.34 (4.75–11.84)	<0.01

Total does not sum to 3,002 because of missing values. Odds ratios estimated using a logistic regression model in which risk index score category is the independent variable and severe cognitive impairment (yes or no) was the dependent variable.

^*Number of cases refers to the number of participants classified with severe cognitive impairment.

Fig. 2.

Association between MHAS late life risk index score category and odds for severe cognitive impairment. Odds ratio estimated using a logistic regression model. Participants who scored between 0 and 6 points were the reference category.

Several of the variables included in the MHAS late life index did not have a statistically significant relationship with severe cognitive impairment in the final model. Therefore, an abbreviated version of the MHAS late life index was calculated using a regression model that included age, diabetes, high depressive symptoms, and IADL impairment. The abbreviated MHAS late life index had a lower AUC compared to the full index (AUC = 0.70, 95% CI = 0.64–0.73) and the optimal cutoff of 3 points had a sensitivity of 0.64 (95% CI = 0.58–0.70), specificity of 0.69 (0.67–0.71), npv of 0.96 (0.95–0.97), and ppv of 0.16 (95% CI = 0.14–0.18).

External validation of the DSI

The DSI was able to discriminate between participants who developed severe cognitive impairment from those who did not with a similar level of accuracy as the MHAS late life risk index (Table 3). The AUC of the DSI was 0.72 (95% CI = 0.69–0.76) and the optimal cut point of the DSI according to the Youden index was 23 points. This cut point had a significantly lower sensitivity (0.57, 95% CI = 0.50–0.63), but significantly higher specificity (0.78, 95% CI = 0.76–0.79) compared to the optimal cut point for the MHAS late life index. The ppv (0.19, 95% CI = 0.16–0.22) and npv (0.95, 95% CI = 0.94–0.96) of the DSI were not significantly different compared to the MHAS late life risk index. The diagnostic accuracy of the DSI was 76%.

According to the DSI, a score of 9 points should be given to participants who have completed fewer than 12 years of education. Only 166 participants in the final sample completed 12 or more years of education and may not be a suitable cut-off for education given the low educational attainment in Mexico. Thus, a subsequent analysis was conducted in which a score of 9 points was given to participants who had completed between 0 and 3 years of education. The AUC for this modified version of the DSI did not change from the initial analysis (0.72, 95% CI = 0.69–0.76), but the optimal cut point of 18 points had a sensitivity of 0.69 (95% CI = 0.63–0.75), specificity of 0.66 (95% CI = 0.64–0.68), ppv of 0.15 (95% CI = 0.13–0.18), npv of 0.96 (95% CI = 0.95–0.97) and diagnostic accuracy of 66%, which are highly consistent with the values for the optimal cut point of the MHAS late life risk index.

DISCUSSION

This is the first study to create a late life risk index that can be used to identify older Mexican adults who are high-risk for developing severe cognitive impairment over an 11-year period. Importantly, this is also the first study to externally validate an existing dementia risk index using data collected from older adults living in a developing country. The MHAS late life index and DSI were able to predict severe cognitive impairment with moderate accuracy and the AUC for the MHAS late life risk index (0.74 95% CI = 0.70–0.77) and DSI (0.72, 95% CI = 0.69–0.76) are consistent with the AUC values for existing dementia risk indices [12, 14, 22, 46]. Additionally, the AUC of the MHAS late life risk index and DSI were both within the acceptable range of 0.70 to 0.80 proposed by Solomon and Soininen [26] to identify individuals who are candidates for lifestyle interventions or educational programs on preventing dementia.

Our finding that the DSI was able to discriminate between older Mexican adults who developed severe cognitive impairment from those who did not with a similar level of accuracy to the MHAS late life risk index is an important finding because it suggests that existing dementia risk indices are applicable to Mexico. Additionally, the level of educational attainment and prevalence of poor health conditions such as diabetes in Mexico is similar to other Latin American countries [47, 48]. Therefore, it is possible that existing dementia risk indices and the MHAS late life risk index may also be applicable to other Latin American countries. Moreover, demonstration of DSI as a predictive index for dementia later in life is an important contribution to the existing literature, and aging cohorts in other countries should test the accuracy of DSI in predicting cognitive impairment in their respective populations.

The findings presented in Table 4 and Fig. 2 provide evidence that the MHAS late life risk index can be used to classify older Mexican adults into high, moderate, and low risk groups for severe cognitive impairment. Compared to participants who scored between 0 and 6 points, those who had a score of 19 points or higher had 7.34 times higher odds of developing severe cognitive impairment and those who scored between 13 and 18 points had 2.36 times higher odds to develop severe cognitive impairment, while those who scored between 7 and 12 points did not have significantly higher odds. Therefore, participants who scored 7 to 12 points on the MHAS late life risk index may represent older Mexican adults who are at low risk to develop severe cognitive impairment, whereas those who scored 13 to 18 points and 19 points or higher represent older Mexican adults who are moderate and high risk during the 11-year follow-up.

Compared to the optimal cut point of 23 points on the DSI, the optimal cut point score of 16 points on the MHAS late life index had significantly higher sensitivity but significantly lower specificity (Table 3). These differences in the sensitivity and specificity of the optimal cut points between the two indices are likely due to differences between the two indices in the definitions of low education. The findings from a subsequent analysis using the DSI in which low education was defined as between 0 and 3 years did not change the AUC, but did produce an optimal cut point that yielded sensitivity, specificity, ppv, npv, and diagnostic accuracy values that were highly consistent with the optimal cut point of the MHAS late life risk index. Collectively, these findings provide evidence that the definition of low education does not substantially change the AUC of the risk index, but does influence the optimal cut point score that is used to calculate the sensitivity, specificity, ppv, npv, and diagnostic accuracy.

The similar findings between the MHAS late life risk index and DSI observed in the present study were somewhat expected given that all of the risk factors included in the DSI, with the exception of being underweight, were also included in the MHAS late life risk index. Advancing age [49], low education [50], diabetes [51], stroke [52], depression [53], and impairment in IADLs [54] have been associated with dementia and have been included in existing risk indices [12, 16, 40, 46]. Risk factors in the MHAS late life risk index that have not been included in other risk indices were impairment in ADLs, having fallen in the past two years, fair-poor vision, and female gender. ADL measures can be used to determine an older adult’s level of physical functioning [55, 56] and a decline in physical functioning has been shown to precede a diagnosis of dementia [57]. Previous studies have demonstrated that older adults with dementia are at an increased risk for falls [58]; however, the findings from the present study identified a history of falls to be associated with increased odds for severe cognitive impairment. A possible explanation is an increased frequency of falls may reflect early declines in balance and other motor skills due to pathological changes in the brain related to dementia [59]. Poor vision has been associated with an increased risk for Alzheimer’s disease [60]. This relationship may be due in part to the role that vision has in the likelihood of participating in mental and social activities that are protective against dementia [61]. Alternatively, many cognitive assessments rely upon vision to be completed. The cognitive battery administered to MHAS participants includes one assessment that involves vision (figure copying), but 187 participants in the Wave I sample could not complete the assessment because of vision problems.

While the risk factors included in the MHAS late life index are consistent with those included in existing indices, potentially important health conditions associated with dementia were not identified. For example, previous studies have reported that having experienced a heart attack is a risk factor for dementia [62], but this risk factor was not found to be significant in the MHAS late life risk index. This may be due to the low prevalence of heart attacks among participants of the MHAS, which decreases the statistical power to detect a significant relationship. Additionally, hypertension was included in the MHAS late life risk index, but this risk factor was weakly associated with severe cognitive impairment. This may be due to the age dependent relationship between hypertension and dementia risk. Hypertension during midlife has been identified as a risk factor for dementia [63], but greater declines in blood pressure in late life have been observed to occur among older adults who develop dementia compared to those who do not develop dementia [64].

There are important limitations of this study that need to be acknowledged. First, the MHAS does not include a clinical diagnosis of dementia and some participants may be incorrectly classified as having severe cognitive impairment or not having severe cognitive impairment. While the method used to determine severe cognitive impairment has been validated in a clinical setting in Mexico [32], the MHAS does not include biomarkers that can be used to determine the specific type of neurodegenerative disease that may be causing the severe cognitive impairment [31]. Furthermore, risk factors may differ for specific types of dementia, such as vascular dementia and Alzheimer’s disease, and classifying participants into a broad category of severe cognitive impairment may limit the accuracy of the MHAS late life index. A second limitation is that health conditions were ascertained based on self-reported data. Self-reported measures are low-cost and can be easily collected from participants, which makes these measures conducive to conduct-ing research in low- and middle-income countries, but self-reported measures are subject to measurement bias. A third limitation is that older adults who either died or were lost to follow up between Waves I and III are likely to be at the greatest risk for severe cognitive impairment. This is supported by the observed differences in sociodemographic, health, and functional characteristics between participants included in the final analytic sample and those who either died or were lost to follow up by Wave III. Fourth, genetic data has not been collected from the MHAS cohort, which prevented apolipoprotein (APOE) e4 allele status from being included in the risk index. The APOEe4 allele is the strongest genetic risk factor for late onset Alzheimer’s disease [65], but previous studies have reported that APOE e4 allele status does not improve the predictive accuracy of risk indices [12, 40]. In addition the MHAS does not have information on environmental and occupational exposures that have been associated with dementia and severe cognitive impairment [66]. Finally, the low prevalence of severe cognitive impairment in the final analytic sample needs to be considered when interpreting the npv and ppv of the MHAS late life risk index and DSI because the ppv increases and the npv decreases as the disease prevalence increases [67].

Despite these limitations there are important strengths of the present study. First, this is the only study to create a risk index for severe cognitive impairment using data from a developing country. Furthermore, this is the only study to externally validate an existing dementia risk index using data from a developing country. Second, the findings of the present study may be generalizable to other Latin American countries given the similarities in the educational and health characteristics of older adults in Latin America. Finally, the findings of this study have potentially significant clinical implications. The MHAS late life risk index includes sociodemographic, health, and functional measures that can be ascertained by a physician or other health professional during a routine physical examination and do not require a significant amount of time or cost to collect. These qualities imply that this risk index and similar risk indices can be included as part of a physical examination and be used to identify older Mexican adults that are high risk to develop severe cognitive impairment. This strategy can aid in prevention, treatment, and management protocols for severe cognitive impairment in Mexico.

In summary, this study created a late life risk index that was able to predict severe cognitive impairment among older Mexican adults over an 11-year period with moderate accuracy. Continued research is necessary to identify additional risk and protective factors for dementia and severe cognitive impairment that are specific to certain populations that may improve the diagnostic accuracy of the late life risk index for Mexico.