A simple method to rule out dementia with temporal orientation

Abstract

Objective

To explore the performance of a test of temporal orientation (TTO) comprising four items derived from the Mini-Mental State Examination over 4 years.

Methods

Responses were obtained from two large cohorts participating in longitudinal studies of aging in the United States (352 normal elderly, 98 persons with very mild probable or possible Alzheimer’s disease). Sensitivity, specificity, and predictive value (positive, PV+, negative, PV−) of the TTO were estimated for each of four annual visits.

Results

When four correct answers were treated as “oriented to time” and 0 to 3 correct answers were treated as “not oriented to time,” sensitivity (to the presence of AD) ranged from 46.0% to 69.2% and PV+ ranged from 32.1% to 49.5%. Specificity (for normal cognition) decreased from 93.2% at the first visit to 81.3% at the fourth visit; TTO performed most reliably in terms of PV−, the probability of normal cognitive function given orientation to time (TTO = 4), which ranged from 92.8% to 95.4%.

Conclusion

Given the stability and strength of the predictive negative value of a dichotomized TTO over time, a TTO could contribute to monitoring normal cognitive functioning in longitudinal studies in which cognitive status is not the primary focus. Prospective validation of the TTO is warranted.

1. Introduction

Data on the relationship of cognition to other health outcomes are frequently not gathered in epidemiologic studies because of the assumption that establishing intact cognitive function requires exhaustive, time-consuming, and expensive cognitive testing. However, when large sample studies involve subjective reports or recall, intact respondent cognition is critical. We hypothesized that a simple addition [1] could be made to health questionnaires that could reliably distinguish cognitively intact persons from persons with dementia or demonstrable impairment of cognition.

Temporal disorientation is rare in persons with normal cognitive function [2,3]; therefore, our motivation in developing a test of temporal orientation (TTO) was that it could have high specificity, ie, reliably identify persons with normal cognitive function. Although a four-point TTO score had a strong correlation with total Mini-Mental Status Examination (MMSE) score, it was not associated with education in persons with at least 7 years of education, unlike the MMSE [1]. In the present study we sought to replicate that result in independent respondents, as well as determine the sensitivity, specificity, and predictive values (positive, PV+; negative, PV−) for a TTO score that was dichotomized as 4 (100% correct) or 0 to 3 points (0 to 75% correct) [1]. We analyzed existing data from two large cohorts of elderly participants in longitudinal studies of aging and Alzheimer’s disease in the United States. We estimated these descriptors over time in these cohorts, simulating the use of the TTO “in the field” in longitudinal studies. This analysis was intended to simulate an epidemiologic study in which investigators are implicitly assuming normal cognitive function in participants from whom survey data are being collected, although the two groups described here do not represent longitudinal cohorts in the sense that they are not followed from a single starting point; rather, the first 4 years of each participant’s responses were analyzed.

2. Methods

Data were obtained from the first 4 years of observations collected under institutional review board–approved research projects in Alzheimer’s disease (AD) and aging being conducted at Oregon Health & Sciences University. Individuals in the AD cohort were identified as described below; those individuals with a consensus diagnosis as well as a dementia severity rating of mild at their initial evaluation were included in the analyses as AD. Individuals in the normal aging cohort were only included in the analyses as nondemented elderly (NDE) if they had no diagnosis of AD in their file for as long as the diagnostic variable was collected for them. In no case did an individual from the NDE cohort move into the AD cohort. However, not all individuals provided responses at all of the first four visits; and each of the two cohorts experienced some attrition from participating in the studies for which they were evaluated annually.

2.1. Subjects

All subjects described here are or were volunteers in institutional review board–approved research projects in Alzheimer’s disease and aging being conducted at Oregon Health & Sciences University. Participants with AD presented at the memory clinic with memory complaints, on referral by self, family, or health care provider. Each subject’s clinical history and exam findings were presented at a weekly case conference in which a consensus diagnosis was reached by a team of neurologists, geriatric psychiatrists, and neuropsychologists. Cognitively intact participants (NDE) were research subjects in studies of normal aging. These subjects were determined to be cognitively intact on the basis of the extensive neurological and neuropsychologic assessment they received to determine eligibility for studies of normal aging [4,5].

The sensitivity, specificity, and predictive value calculations presented here focus on those NDE (N = 352 at the first visit) who retained normal cognitive status for the 4 years of our study and on the clinic patients who met the National Institute of Neurological and Communicative Diseases and Stroke/Alzheimer’s Disease and Related Disorders Association criteria for probable or possible AD [6] and whose Clinical Dementia Ratings (CDR) [7,8] reflected early (mild) dementia (ie, CDR = 0.5). On the basis of the characteristics of the individuals with data in these databases, we identified individuals with consensus diagnosis of probable or possible AD and CDR = 0.5.

Consent to include personal and clinical data in the research database used in this study was signed by all participants at the time of their initial evaluation and enrollment; institutional review board approval was obtained for each protocol and for data sharing that resulted in these analyses. The data analyzed for this report included all AD patients and NDE subjects with normal cognitive status during 4 years whose data were archived as of September 2002.

2.2. Instruments

Several well-known and widely used instruments were administered to each participant.

2.2.1. CDR

Dementia severity was staged with the CDR scale [7,8]. This is based on semistructured interviews with the patient and a knowledgeable informant [7,8]. Patients are rated on six domains: judgment and problem solving, memory, community affairs, home and hobbies, personal care, and orientation. The domain ratings are 0 (no impairment), 0.5 (questionable impairment), and 1, 2, or 3 (mild, moderate, or severe impairment). The personal care domain has no 0.5 rating possible. The CDR global rating is based on a weighted combination of these domain scores [7,8]. In our analyses, all NDE had a global CDR = 0 at each visit (ie, no converters were included even if they had CDR = 0 at an earlier visit), whereas all persons with a diagnosis of AD had global CDR ratings of 0.5, suggesting very mild (early) dementia.

2.2.2. MMSE

Full-scale MMSE [9] scores are calculated as the sum of 1/0 correct/incorrect answers to 30 questions; scores range from 0 to 30 (worst to best). The MMSE was administered at each visit. The TTO score was based on four MMSE items: date, month, year, and day of week, ranging from 0 to 4 (worst to best) [1]. Because temporal disorientation is uncommon in persons with cognitive function [2,3], we dichotomized the TTO score to reflect “normal” performance, orientation to time (TTO = 4) and “not normal” performance, not oriented to time (TTO = 0 to 3) [1]. We also examined performance (% correct) in each group on the four items to determine whether any had a particularly low likelihood of being correct.

2. 3. Statistical methods

For simplicity of interpreting the data, we used only annual visits; if a participant missed a visit scheduled 12 months after the last visit but was assessed 13 to 23 months later, we did not use those data. This provided estimates of sensitivity and specificity that were based on responses given at roughly 12-month intervals.

Descriptive statistics were generated for each cohort to describe the respondents as well as the group’s performance of each of the four items and to estimate association between education and TTO; P values <.05 were considered significant.

After TTO scores (0 to 4) were computed, they were dichotomized (as 4 or 0 to 3). At each visit, individual’s two-level TTO [1] was entered into calculations of the sensitivity (true positives/all AD patients), specificity (true negative/all NDE), and predictive values (positive: (Prevalence × Sensitivity)/[(Prevalence × Sensitivity) + (1 − Prevalence) × (1 − Specificity)]; negative: (1 − Prevalence) × Specificity/{[(1 − Prevalence) × Specificity] + [Prevalence × (1 − Sensitivity)]}) [10] for distinguishing normal cognition from diagnosis of AD with CDR = 0.5 (ie, mild severity). These calculations were repeated at each of the first four annual visits in the longitudinal studies with all available data, without regard for whether the same persons contributed at each visit.

We used an estimate of the prevalence of AD in persons 65 years old or older, 11.3%, that was observed in two independent community-based cohorts [11,12]. Predictive power is critically dependent on the prevalence of a disease; we sought an estimate that had a high level of reliability. These two studies were reported to agree closely, despite being based on different communities and being at least a decade apart. Because our cohorts were generally older than 65 years, we believed this to be an adequately conservative estimate of prevalence to limit the bias in our calculations and conclusions.

All analyses were carried out by using SPSS 13.0 (2005; SPSS Inc, Chicago, IL) for the PC.

3. Results

The descriptive statistics for the two cohorts at baseline are shown in Table 1.

Table 1.

Descriptive statistics by cohort, mean ± standard deviation or percent

	NDE (n = 352*)	AD (n = 98*)
Education (y)	13.8 ± 3.0	14.4 ± 3.1
Age (y)	80.4 ± 9.7	73.1 ± 10.8
Gender (% female)	62	57
MMSE*	28.2 ± 1.6	24.4 ± 3.2
TTO*	3.9 ± 0.3	3.0 ± 1.1

^*Values at baseline, Y0. These cohorts were not compared on these descriptive variables.

Within each cohort we calculated the correlations between the TTO score (0 to 4) at each of the four visits and education (6 to 21 years; see Table 1 for means). None of the Spearman correlations were significantly different from zero (NDE range of coefficient values, −.04 to.11; all P > .10; AD range of coefficient values, −.07 to .20; all P > .15; data not shown).

Table 2 shows the proportion of each cohort responding correctly to the four TTO items.

Table 2.

Proportion of cohort with correct response per TTO item per year

Year	Group	“Year”	“Month”	“Day of Week”	“Today’s Date”
Y0 (baseline)	AD	86%	89%	71%	54%
	NDE	99%	100%	98%	96%
Y1	AD	98%	94%	80%	62%
	NDE	99%	99%	99%	91%
Y2	AD	96%	83%	54%	58%
	NDE	100%	99%	99%	87%
Y3	AD	85%	77%	62%	38%
	NDE	100%	100%	99%	82%

AD, persons with probable or possible AD and CDR = 0.5; NDE, persons with normal cognitive function and CDR = 0.

In both groups, the item least likely to be answered correctly was date (day of the month), and year was most likely to be correct; this was true at each visit except for AD patients in year 3, when day of week was slightly less likely to be answered correctly than date.

We then dichotomized the TTO score as 100% (4) versus “not” (0 to 3). Table 3 presents the diagnostic status of the respondents (AD, NDE) and their classification by the dichotomized TTO (not oriented, oriented). The samples of persons with AD who had a CDR of 0.5 were roughly halved each year, whereas roughly 80% of the NDE group size was maintained in succeeding years.

Table 3.

Sensitivity, specificity, and predictive value (positive, negative) for dichotomized TTO over time

	TTO	Cognitive Status		Sensitivity	Specificity	PV+	PV−
		AD	NDE
Year 0				58/(58+40) = 51.2%	328/(328+24) = 93.2%	.0579/.1182 = 49.5%	8.267/.8818 = 93.8%
	<4	58	24
	4	40	328
Year 1				23/(23+27) = 46.0%	246/(246+32) = 88.5%	.0520/.1540 = 33.8%	.7849/.8459 = 92.8%
	<4	23	32
	4	27	246
Year 2				15/(15+9) = 62.5%	199/(32+199) = 86.2%	.0706/.1935 = 36.5%	.7647/.8056 = 94.7%
	<4	15	32
	4	9	199
Year 3				9/(9+4) = 69.2%	150/(150+33) = 82.0%	.0784/.2383 = 32.9%	.7270/.7618 = 95.4%
	<4	9	33
	4	4	150

Any person with an AD diagnosis and a TTO score between 0 to 3 was designated true positive; a true negative was any NDE person with a TTO score of 4. At the first visit (Table 3), dichotomized TTO identified 51.2% of the 98 persons with AD as true positives (ie, sensitivity = 51.2%), whereas 93.2% of NDE were identified as true negative. The probability of being AD, given a TTO score between 0 and 3 (PV+), was 49.5%, whereas the probability that a respondent was normal, given a perfect TTO score (PV−), was 93.8%.

Because the sample of AD patients with CDR = 0.5 shrunk over time, the sensitivity of dichotomized TTO increased after an initial dip to 46.0% at the second-year visit to 62.5% at the third and 69.2% at the fourth annual visit. Conservatively assuming a constant prevalence rate (11.3%), PV+ dropped from 49.5% at the first visit to values in the 32% to 26% range during later visits.

The sample of NDE declined during the same period, but as noted, these individuals were included in these analyses only if diagnostic status did not change during at least the 4 years we analyzed, that is, NDE data were not missing over time because they were transitioning to questionable or incipient dementia (and it is likely that attrition in the AD group was due to severity transitioning), but rather as a result of attrition. The specificity of dichotomized TTO decreased linearly during 4 years from 93.2% initially to 82.0% 4 years later. Assuming the same constant value for the prevalence of AD, the PV− for the dichotomized TTO remained above 92% during all 4 years of this study; thus, a perfect TTO score was associated with at least 92% probability of normal cognitive status at each of the four visits.

4. Discussion

In this field test of TTO, we found that the four-point version was not associated with education in either group over time. The PV− of a dichotomized four-item TTO (100% vs less than 100%) was stable during 4 years in these cohorts, whereas sensitivity, specificity, and PV+ were generally low and far more variable over time. Sensitivity (to the presence of AD) ranged over time from 46.0% to 69.2%, specificity decreased from 93.2% to 82.0%, and PV+ ranged from 32.9% to 49.5%. Conversely, PV− remained constant during the 4 years, ranging from 92.8% to 95.4%.

Although these samples were derived from their respective communities, they were enrolled in the studies from which their data were derived on the basis of their cognitive status. Therefore, our estimates of sensitivity and specificity are artificial to some extent, and prospective validation of the TTO is necessary for more “naturalistic” estimates of these properties. It is also critical to note that individuals with mild cognitive impairment [13] were excluded (by definition and by consensus diagnosis as “cognitively intact”) from the NDE group as well as being excluded (by definition) from the AD cohort. Outside of the simulation setting in which this study was carried out, persons with mild cognitive impairment could considerably affect the discriminating properties of the TTO.

However, our results with respect to the probability of normal cognitive status, given orientation to time (PV−), both reflect our initial motivation in identifying the TTO and contrast with our observations of PV+, sensitivity, and specificity. If our sample was creating bias in these results, we would have expected a broader impact than only on PV−. Furthermore, our estimate of prevalence (11.3%) was based on similar results derived in two different community samples [11,12]. Higher estimates would have improved the apparent performance of the TTO with respect to PV+ and worsened it with respect to PV−. We used a prevalence estimate (11.3%) that is lower than a recent estimate (13.0%) from a community with many respondents older than 90 years [14], but it is higher than the average of 11 independent estimates derived between 1991 and 1996 (10.08%) [15].

The PV− reflects an important characteristic of the dichotomized TTO. The properties of the TTO might differ, depending on the context in which it is used. In longitudinal studies in which cognitive status is not the primary focus, particularly in older populations, a TTO is a simple addition that could make an important contribution to the identification of normal cognition over time. Because this report simulates the epidemiologic context, prospective studies are needed for broader validation, and the TTO should not be the only mechanism for detecting cognitive changes in aging cohorts.