The CAIDE dementia risk score and the Honolulu-Asia Aging Study

Abstract

Introduction:

The CAIDE (Cardiovascular Risk Factors, Aging, and Incidence of Dementia) risk score is based on demographic, genetic, and modifiable risk factors in midlife and has been shown to be predictive of later-life dementia.

Objective:

To test the predictive capacity of the CAIDE dementia risk score among a cohort of Japanese-American men.

Methods:

Midlife measures were obtained from a sample of 3,582 Japanese-American men in the Honolulu Heart Program (1965–1968, average age=53.1 years). A follow-up exam in 1991 (average age=77.8 years) assessed cognitive impairment using the Cognitive Abilities Screening Instrument (CASI). Severe cognitive impairment was defined as a CASI score <60.

Results:

In this cohort, the CAIDE score demonstrates significant association with later-life severe cognitive impairment (OR=1.477, 95%CI: 1.39–1.58). However, the area under the receiver operating characteristic curve c-statistic suggests poor predictive ability (c=0.645, 95%CI: 0.62–0.67). Using a score cut-point of 10, the accuracy is acceptable (0.82), but the sensitivity is low (0.50).

Conclusion:

While the CAIDE dementia risk score at midlife is associated with later development of cognitive impairment in Japanese-American men, its predictive capacity in this population is weak.

INTRODUCTION

Recent reviews identified over 70 different prediction models for the detection of individuals at high risk for dementia or Alzheimer’s disease [1–3]. They vary widely based on the type of predictors they include, the age at screening, the length of follow-up time, and the sample from which the model was derived (e.g. population, primary care, cohort). Although some of these risk models perform well, none has shown high predictive capacity across a range of populations. Stephan et al. [2] suggest rather than add to an already saturated pool, researchers should focus on validating and improving existing risk models. In this study, we applied a widely known risk score for dementia, the CAIDE (Cardiovascular Risk Factors, Aging, and Incidence of Dementia) dementia risk score, in a population demographically distinct from the cohort in which the score was developed.

The CAIDE dementia risk score was developed using a large population-based cohort from Finland [4]. Participants were first examined in midlife, when extensive health data were collected, then followed over 20 years for the development of dementia. Based on data from the study, a risk score for predicting future dementia was constructed from multiple demographic and midlife measures. Later, the score was modified to place greater weight on age and education and to include carrier status for the apolipoprotein ε4 allele (APOE ε4). Though it demonstrated moderate predictive capacity in the Finnish cohort (c statistic=0.78, CI: 0.72–0.84), validation among additional cohorts is important to understand generalizability of the risk score.

The Honolulu Heart Program (HHP) began in 1965 with a cohort of over 8,000 Japanese American men born between 1900 and 1919 and living on the island of Oahu, Hawaii. They were followed over time and the cohort evolved into the Honolulu-Asia Aging Study (HAAS), which was designed to examine cognitive function and diseases of aging [5]. As such, data exist on both the midlife health of the participants and on later life cognitive function. We used these data to examine the predictive ability of the CAIDE dementia risk score in this population.

METHODS

Study population

Study participants came from the HAAS, which evolved from the HHP, both of which were reviewed and approved by the Institutional Review Board of Kuakini Medical Center, Honolulu, Hawaii. Beginning in 1965, a large cohort of Japanese men born between 1900 and 1919 and living on the island of Oahu, Hawaii, (n=8,006) were followed and reexamined two additional times separated by approximately 3-year intervals. A fourth exam was conducted from 1991 to 1993 for the remaining HHP members who agreed to participate. This was the baseline for the HAAS, which was designed to study cognitive function and diseases of aging [5]. The group of 3,734 men represented almost 80% of the surviving HHP cohort. Excluding respondents with missing data, the final sample for this study was 3,582 participants. The average length of time between the first and fourth exams was 25.2 years (+/− 1.1 years).

Definition of cognitive impairment

The Cognitive Abilities Screening Instrument (CASI) is a comprehensive screening instrument specifically designed for the cross-cultural assessment of cognitive impairment and dementia [6]. It combines measures of attention, concentration, orientation, memory, language, visual construction, abstraction, and judgement to create a composite score ranging from 1 to 100. For this analysis, a CASI score of less than 60 was used to indicate severe cognitive impairment (SCI) [7]. The recorded CASI score at the fourth exam was used to determine the presence (n=189) or absence (n=3,393) of SCI.

CAIDE dementia risk score

The CAIDE dementia risk score is comprised of dichotomized values for sex, systolic blood pressure (SBP), body mass index (BMI), total cholesterol, physical activity, and APOE ε4 status, as well as tertiles describing age and education in the original cohort. With the exception of APOE ε4 status, measures for each factor were taken during the first HHP exam (1965–1968) when the average age of the cohort was 53.1 years. Age, education, and physical activity were self-reported, while systolic blood pressure, body mass index, and blood cholesterol measures were obtained by health care professionals. The physical activity index was created using average time spent at different levels of physical activity (none to heavy) weighted by level of oxygen consumption associated with each activity (see [8] for more details). Genetic testing to acquire APOE ε4 carrier status was conducted at the fourth exam (1991–1993). The possible range for the CAIDE score is 0 to 18. In this all-male cohort, the observed range was 1 to 15.

Statistical analyses

Satterthwaite t-tests and Pearson’s chi-square tests were used to compare respondents who developed SCI to those who did not. Basic logistic regression was used to assess the effect of each component of the CAIDE score on the prediction of subsequent development of SCI. C-statistics were derived from the area under the receiver operating characteristic (AUC) curve. Risk score quartiles were identified using the distribution of the CAIDE score in the sample and were utilized to compare SCI prevalence and relative risk. We report predictive capacity (i.e. sensitivity, specificity, positive and negative predictive values, and accuracy) for two different cut-off points of the score: high risk = 9–18 and high risk = 10–18.

RESULTS

Table 1 compares respondents who developed SCI (CASI score <60) by exam four with those who did not. Those who developed SCI were significantly older (58.4 vs 52.3 years, p=<.0001), had less education (8.0 vs 10.6 years, p=<.0001), had higher systolic blood pressure (134 vs 130 mmHg, p=0.0094), and were more physically active (p=0.0007). The CAIDE dementia risk score was also significantly higher (9.49 vs 7.21, p=<.0001) among those who developed SCI. There was a 48% increase in the odds of SCI for each 1 point increase in the risk score (OR=1.477, 95%CI: 1.39–1.58). Body mass index, total cholesterol, and APOE ε4 carrier status demonstrated no difference between the two groups.

Table 1:

Comparison of characteristics among respondents who did or did not developed severe cognitive impairment (CASI score <60) by exam 4.

	SCI+ (n=189)	SCI− (n=3,393)
Characteristic	Mean (SD), % (#)	Mean (SD), % (#)	P value*	OR (95% CI)†
Age at exam 1, years	58.4 (5.4)	52.3 (4.4)	<.0001	1.256 (1.22–1.29)
Education, years	8.0 (2.9)	10.6 (3.2)	<.0001	0.731 (0.69–0.77)
Systolic blood pressure, mmHg	134.0 (21.4)	129.8 (18.3)	.0094	1.011 (1.00–1.02)
Body mass index, kg/m2	23.8 (3.0)	23.9 (2.8)	.7226	0.991 (0.94–1.04)
Cholesterol, mmol/L	5.6 (0.9)	5.7 (0.9)	.5239	0.949 (0.81–1.12)
Physically active	92.1% (174)	82.6% (2,803)	.0007	2.441 (1.43–4.17)
APOE ε4 carrier	19.6% (37)	18.4% (625)	.6902	1.078 (0.75–1.56)
CAIDE score	9.49 (2.0)	7.21 (2.4)	<.0001	1.477 (1.39–1.58)

Abbreviations: APOE ε4, apolipoprotein allele ε4; CASI, Cognitive Abilities Screening Instrument; CI, confidence interval; kg/m2, kilograms per meter squared; mmHg, millimeters of mercury; mmol/L, millimoles per liter; OR, odds ratio; SCI, severe cognitive impairment; SD, standard deviation.

NOTE: Bolding indentifies significant results.

^*P value for Satterthwaite t value or Pearson chi-square.

^†Odds ratio (OR) and 95% confidence interval (95% CI) for parameter estimate.

The values used to compute the CAIDE risk score are shown in Table 2. For each component of the score, beta coefficients and odds ratios for development of SCI from logistic regression models are given. In the HAAS cohort, age, education, BMI, physical activity, and APOE ε4 status were associated with development of SCI in univariate analyses. Table 2 also displays the c-statistic, or area under the receiver operating characteristic curve (AUC), for each component of the score as a measure of its predictive ability. The c-statistic gives an estimate of the discriminative ability of the test, with a value of 1.0 representing perfect prediction and 0.5 being the same as chance. The individual components of the CAIDE score demonstrate discriminative ability ranging from 0.506 (BMI, cholesterol) to 0.611 (education). Taking all components of the score together, the c-statistic is 0.645 (95% CI: 0.62–0.67).

Table 2:

Components of the CAIDE dementia risk score. There are eight parts, each with a designated value. An individual’s CAIDE score is the sum of the values for each of these eight items. Beta coefficients and odds ratios for development of SCI (CASI score <60) in the cohort are shown for each component, as is the area under the receiver operating characteristic curve (C-stat).

CAIDE component	CAIDE Risk Score	β coef	p *	OR (95% CI)†	C-stat (95% CI)‡
AGE AT EXAM 1
<47 years	0	0 (ref)
47–53 years	3	−0.2012	.0858	1.126 (0.59–2.16)
>53 years	5	0.5214	<.0001	2.320 (1.21–4.45)	0.587 (0.57–0.61)
EDUCATION
>=10 years	0	0 (ref)
7–9 years	3	−0.0568	.3589	1.829 (1.53–2.19)
0–6 years	4	0.7173	<.0001	3.966 (3.06–5.15)	0.611 (0.59–0.63)
SEX §
Women	0	NA	NA	NA
Men	1	NA	NA	NA	NA
SYSTOLIC BLOOD PRESSURE
<=140 mmHg	0	0 (ref)
>140 mmHg	2	0.0667	.1530	1.143 (0.95–1.37)	0.513 (0.49–0.53)
BODY MASS INDEX
<= 30 kg/m2	0	0 (ref)
>30 kg/m2	2	0.276	.0418	1.737 (1.02–3.96)	0.506 (0.50–0.51)
TOTAL CHOLESTEROL
<=6.5 mmol/L	0	0 (ref)
>6.5 mmol/L	1	−0.042	.4576	0.919 (0.74–1.15)	0.506 (0.49–0.52)
PHYSICAL ACTIVITY
Active	0	0 (ref)
Inactive	1	−0.2111	.0006	0.656 (0.52–0.83)	0.527 (0.51–0.54)
APOE ε4 STATUS
Non-carrier	0	0 (ref)
Carrier	2	0.2149	<.0001	1.537 (1.26–1.87)	0.535 (0.52–0.55)
TOTAL	18				0.645 (0.62–0.67)

Abbreviations: APOE ε4, apolipoprotein allele ε4; CAIDE, Cardiovascular Risk Factors, Aging and Incidence of Dementia; CASI, Cognitive Abilities Screening Instrument; CI, confidence interval; Coef, coefficient; kg/m2, kilograms per meter squared; mmHg, millimeters of mercury; mmol/L, millimoles per liter; NA, not applicable; OR, odds ratio; Ref, reference category; SCI, severe cognitive impairment.

NOTE: Bolding identifies significant results.

^*P value for Wald chi-square.

^†Odds ratio (OR) and 95% confidence interval (95% CI) for parameter estimate.

^‡C-statistic values, or area under the receiver operating curve (AUC), and 95% confidence interval (95% CI) for each component and for the combined score.

^§Estimates not available for all male cohort.

Table 3 shows the prevalence of SCI across quartiles of the CAIDE dementia risk score. In the lowest quartile, the prevalence of SCI is 0.7% compared to 14.4% in the highest quartile. The relative risk of developing SCI based on risk score is shown using the lowest quartile as the reference category. Those respondents with CAIDE risk scores in the highest quartile have a 900% higher risk of later development of SCI than those with scores in the lowest quartile (RR=10.131, CI: 4.66–22.04).

Table 3:

Prevalence of SCI at exam 4 across quartiles of the CAIDE dementia risk score and relative risk of SCI development.

CAIDE Score	SCI/All (n)	Prevalence of SCI	RR (95% CI)*
1–5	6/860	0.7%	1.000
6–7	22/1147	1.9%	2.014 (0.99–4.10)
8–9	66/916	7.2%	6.018 (2.79–12.96)
10–18	95/660	14.4%	10.131 (4.66–22.04)

Abbreviations: CAIDE, Cardiovascular Risk Factors, Aging and Incidence of Dementia; CI, confidence interval; RR, relative risk; SCI, severe cognitive impairment.

NOTE: Bolding identifies significant results.

^*Relative risk (RR) and 95% confidence interval (95% CI) for development of severe cognitive impairment relative to the first quartile.

Table 4 displays measures of predictive capacity of the CAIDE dementia risk score as a screening tool for future development of SCI. Two sets of values are given based on different cut-off values for defining ‘high risk’, one using a score of 9 or more as high risk and the other setting the cut-off at 10 or more. As expected, using a higher cut-off value captures fewer true positives, thus reducing the sensitivity of the test (0.76 vs 0.50). Conversely, the lower cut-off value captures fewer true negatives, resulting in a decrease in specificity (0.68 vs 0.83). The positive predictive value is low with either cut-off (0.12 and 0.14, respectively), suggesting more individuals at high-risk would be categorized as low-risk. The negative predictive value is high for either cut-off (0.98 vs 0.97), meaning most low-risk individuals would be properly categorized as low-risk. The accuracy of the lower cut-off is less than that of the high cut-off (0.69 and 0.82, respectively).

Table 4:

Predictive capacity of the CAIDE dementia risk score in the HAAS cohort.

	High risk of future development of SCI:*
Measure	CAIDE score 9+	CAIDE score 10+
Sensitivity	0.76	0.50
Specificity	0.68	0.83
Positive PV	0.12	0.14
Negative PV	0.98	0.97
Accuracy	0.69	0.82

Abbreviations: CAIDE, Cardiovascular Risk Factors, Aging and Incidence of Dementia; HAAS, Honolulu-Asia Aging Study; PV, predictive value; SCI, severe cognitive impairment.

^*High risk defined as a CAIDE score of 9 or more versus a score of 10 or more.

DISCUSSION

While we found a significant association of midlife CAIDE dementia risk score with later life severe cognitive impairment, the predictive ability of the score was low in our population of Japanese-American men (c-statistic=0.645; CI: 0.62–0.67) Further, poor sensitivity (0.50 or 0.76) means many of those truly at risk would not be identified as such. Using the higher cut-off (10+), the test would be correct more than 80% of the time. However, the majority of those correctly categorized would be true negatives.

We observed expected relationships between development of SCI and components of the CAIDE score. Specifically, those who developed SCI were older, had less education, had higher BMIs, and were more likely to have at least one APOE ε4 allele. Although elevated blood pressure has been associated with cognitive impairment in this cohort [9,10], we did not see this in our study using the CAIDE score definition. Additionally, cholesterol levels have been shown to be associated with risk of Alzheimer’s disease [11], but we saw no difference. It is possible that the use of anti-hypertensive or cholesterol-lowering medications may be masking any existing relationship [12]. Of interest to note is the protective effect of physical inactivity observed in this cohort. Leisure activity is believed to be one component that decreases the risk of dementia [13], which makes this finding counterintuitive. A previous study in the HAAS cohort found that physical activity benefitted primarily men with limited physical function rather than those with moderate or high physical function [14]. It is possible our cohort has mostly moderate to high physical function, negating the effect of physical activity. However, this does not explain the observed protective effect.

Although some researchers have been able to replicate a moderate predictive capacity of the CAIDE dementia risk score [15–17], others have found results similar to those in this study. Kerut et al. [18] obtained the same c-statistic of 0.64 in a sample of African-Americans. Licher et al. [19] and Anstey et al. [20] found predictive capacities not much better than chance when validating the CAIDE risk score across several cohorts (c-statistic range=0.49–0.57). However, it’s important to note that the cohorts used in these two studies were comprised of older adults and the follow-up time was short. The CAIDE risk score was developed using middle-aged respondents and a 20-year follow-up. It has been noted that some risk factors for dementia during midlife, such as high cholesterol or high BMI, may no longer confer increased risk in late-life [21–23]. Thus, the CAIDE dementia risk score may not perform as well among older adults.

Although the CAIDE risk score has been validated in other populations [24–29], the poor predictive capacity observed in this study may be a result of specific characteristics of our cohort (e.g. Japanese ancestry or male gender), suggesting limited generalizability. In a review of studies coding dementia, Wilkinson et al. [30] found a large heterogeneity in accuracy estimates and suggest that setting-specific validation may be the best option. Alternatively, the CAIDE risk score may be lacking important components for the prediction of future dementia. The original authors concede they did not have data for some factors they believe would have improved the score, including the presence of diabetes, waisthip ratio, and a family history of dementia [4]. The CAIDE risk score is important in that it highlights modifiable risk factors prior to disease development. Identification and manipulation of these factors in midlife could reduce the development of dementia among the elderly.

6. CONCLUSION

Although the predictive ability of the CAIDE dementia risk score was low in this cohort, there was a significant association between the midlife risk score and development of SCI 25 years later.