Cognitive Assessment of Older Primary Care Patients with and Without Memory Complaints

Abstract

BACKGROUND

Dementia screening is currently recommended only for symptomatic patients.

OBJECTIVE

To evaluate memory complaints, a mental status test, and several cognitive tests as dementia screens in primary care.

DESIGN

Cross-sectional clinical epidemiologic study.

PARTICIPANTS

Three hundred thirty-nine comprehensively assessed, primary care patients aged ≥65 years.

MEASUREMENTS

Memory complaints were abstracted from chart review. Scores on Mini-Mental State Examination (MMSE) and domain-specific cognitive testing were compared to a dementia diagnosis based on Clinical Dementia Rating score ≥ 1, and areas under the receiver operating characteristic curves (AUC) were calculated. Classification and regression tree analyses were performed on memory complaints and tests with the highest AUCs.

RESULTS

Of 33 patients with dementia, only 5 had documented memory complaints. In 25 patients with documented memory complaints, no cognitive tests further improved identification of the 5 with dementia. In 28 patients with dementia but without memory complaints, an MMSE score < 20 identified 8 cases; among those with MMSE scores 20–21, a visual memory test identified a further 11 cases. Further cognitive testing could not detect 9 dementia cases without memory complaints and with MMSE scores ≥ 22.

CONCLUSIONS

In older primary care patients with memory complaints, cognitive screening does not help identify those who require further examination for dementia. Most patients with dementia do not report memory complaints. In these asymptomatic individuals, general mental status testing, supplemented by a memory test when the mental status score is equivocal, will identify lower-scoring patients who need dementia assessment. However, high-scoring asymptomatic dementia cases will remain undetected.

INTRODUCTION

Cognitive impairment and dementia pose growing health care burdens among older adults.¹ Patients with dementia often lack awareness of their cognitive impairments² or may not report symptoms to their physicians. Family members, when available, may report memory and other cognitive losses to physicians. Astute clinicians themselves may observe cognitive impairment in their patients. Early identification is potentially useful for etiologic diagnosis, education of patients and families,³ and appropriate treatment planning.^4,5 However, detection by primary care physicians remains low.^6,7

Evidence-based standards for clinical practice require that asymptomatic individuals be screened for conditions such as breast cancer and hyperlipidemia,^8,9 but screening for cognitive impairment or dementia remains controversial. Citing lack of evidence, current U.S. Preventive Services Task Force (USPSTF) guidelines argue neither for nor against screening asymptomatic adults for dementia.¹⁰ Evaluation of cognitive function is recommended by the USPSTF and others only if an impairment or deterioration in cognitive function is suspected,^4,10–12 although this recommendation has been challenged.^3,5,13–15 Besides lack of an empirical evidence base, barriers to screening include time and cost, lack of readily acceptable screening measures, lack of effective treatments, risk of distress and other adverse consequences in patients, and opposition to any routine mental health screening.^6,16–21 Our purpose in this article is not to reiterate these arguments but to contribute data on the relative accuracy of different cognitive measures for the detection of dementia, in older primary care patients, in the presence and absence of self-reported symptoms.

We have previously reported that only a minority of medical records of elderly primary care patients reflected reports of cognitive problems by patients, families, or physicians.⁷ We now report a comparison of several cognitive tests on their ability to correctly classify older primary care patients both with self-reported memory complaints, referred to hereafter as “symptomatic,” and without such complaints. Earlier studies on screening have focused on both general mental status tests^14,22,23 and domain-specific cognitive tests.^24,25 To our knowledge, previous studies have not examined the relative utility of these tests for the detection of dementia in relation to self-awareness, reflected in subjective memory complaints.

METHODS

Subjects

The Steel Valley Seniors Survey was a clinical epidemiologic study of dementia in primary care patients aged 65 years and older in a small-town community in southwestern Pennsylvania. Study procedures were approved by the University of Pittsburgh Institutional Review Board. From 1999 to 2001, participants were recruited from the offices of 15 physicians who provided care to older adults and agreed to provide access to patients and to medical records of consenting patients.⁷

In the physician’s office, written informed consent was obtained, demographic information was collected, and a general mental status test, the Mini-Mental State Examination (MMSE), was administered. The MMSE briefly assesses global cognitive function with scores ranging from 0 to 30; lower scores indicate worse performance.²⁶ A trained research nurse used a standardized chart abstraction protocol to review the medical record over the preceding 5 years for several variables including documentation of memory complaints.⁷

All participants who scored ≤24 on the MMSE, and a randomly selected comparison group of participants who scored ≥25, were offered a comprehensive assessment at home. The assessment included a cognitive test battery, brief medical history, physical and neurologic examination, depression symptom screen, questionnaire-based assessment of function, and detailed medication review, among other items.²⁷

Memory Complaint

Participants were designated as “symptomatic” if self-reported memory complaints were documented in their charts. Reports by family members, if any, were excluded because many participants were unaccompanied by relatives when they visited their physicians. Details and examples have been reported previously.²⁷

Cognitive Test Battery

The test battery included several simple tasks tapping a wide range of cognitive domains known to be affected in dementing disorders.

Verbal Memory The Hopkins Verbal Learning Test (HVLT) is a 12-item word list learning test with three learning trials, a 20-minute delayed recall, and forced choice recognition.²⁸

Visual Memory The modified Rey–Osterrieth Complex Figure (Rey Figure) tests immediate and delayed visual memory, visuospatial skills, and constructional praxis,²⁹ (initiating and completing new motor tasks that involve spatial organization, usually tested by ability to copy line diagrams).

Speed and Executive Functioning Trailmaking Tests A and B measure attention, processing speed, and visual search. Trailmaking B also measures executive functions including set-shifting and mental flexibility.³⁰ We report Trailmaking scores as number of correct connections divided by time (seconds) to complete, normalize the distribution, and make higher scores represent better performance.

Verbal Fluency This tests word generation in a 60-second period, including initial letter fluency (P and S) and category fluency (animals).³¹

Constructional Praxis The Clock Drawing Test involves constructional praxis with elements of executive functioning (planning, motor sequencing, selective attention).³²

Depressive Dymptom Screening A modified version of the Center for Epidemiological Studies Depression Scale³³ was administered. Our modification, the mCES-D³⁴ includes all original 20 items scored as present/absent.

Assessment of Dementia

The Clinical Dementia Rating (CDR) scale stages dementia based on cognition-related daily functioning derived from the entire comprehensive assessment except for the cognitive tests.³⁵ Six domains of functional status are assessed, including memory, orientation, judgment and problem solving, community affairs, home and hobbies, and personal care. A standard algorithm is used to generate a summary score^35,36 ranging from 0 (no dementia) through 0.5, 1, 2, and 3, reflecting questionable, mild, moderate, and severe dementia. A CDR rating ≥ 1 was treated as a diagnosis of dementia.

Statistical Analysis

Distributions are reported as percentages for categoric variables and as means, standard deviations (SD), and medians for continuous variables.

Two approaches were used to examine the classificatory properties of the cognitive tests.

Receiver Operational Characteristics (ROC) Method

For each cognitive test, ROC curves were generated by plotting sensitivity against 1-specificity for dementia at all possible cut points. Tests were rank ordered according to areas under the curves (AUC) and assessed for significant differences in AUC between pairs of tests. ROC curves and AUC were obtained using STATA 9.0.³⁷

Classification and Regression Tree (CART) Method

CART analyses were performed on the cognitive tests with the highest AUCs that were not statistically significantly different from one another. Based on recursive partitioning,^38,39 CART involves the segregation of different values of predictors through a decision tree composed of progressive binary splits. The optimal split is selected based on the impurity criterion (the reduction in the residual sum of squares because of a binary split of the data at that tree node). The validity of the final model was assessed using a modified version of the Gini index.^39,40

The CART analyses were used to identify the cognitive tests, which would provide the greatest accuracy in classifying subjects as to the presence of dementia (CDR ≥ 1 vs CDR < 1). We specified the model based on clinically plausible steps rather than purely statistical model selection. Thus, the first variable in the CART was the presence/absence of a memory complaint, followed by the MMSE as the next step. The domain-specific cognitive tests were then introduced together to identify the tests and cut points, which would provide the best classification beyond the MMSE. The CART procedure automatically selects the best cut points (splits) for achieving the best possible classification. In post hoc analyses, a tree was generated introducing depressive symptoms (mCES-D score) into the models before the MMSE or other cognitive tests.

A tenfold cross-validation was used to assess the predictive ability of the tree model. The numbers of demented and non-demented individuals at each of the terminal nodes in the CART tree were used to calculate the sensitivity, specificity, positive predictictive value (PPV), and negative predictive value (NPV) at each split. OPen-source statistical software, R version 2.3.1, was used.

RESULTS

Sample Selection and Size

Of 1,107 primary care patients aged 65+ years screened with the MMSE in the physicians’ offices, the comprehensive assessment was offered to 642 participants: 343 with MMSE scores ≤ 24 and a comparison group of 299 randomly selected from those with scores ≥ 25. Three of the 642 (0.5%) moved away, 3 (0.5%) died before the home visit, and 358 (55.8%) underwent the comprehensive assessments.

Demographic Characteristics

Among the 642 participants selected for comprehensive assessment, those who did and did not undergo the assessment were similar (P > .05) with regard to age (77.5 vs 77.7 years), sex, (68.7% vs 64.0% women), education (66.8% vs 63.3% with ≥12 years), and race (92.5% vs 93.5% white). However, those assessed had a marginally higher mean (SD) MMSE score than those who were not [24.5 (SD 3.4) vs 24.0 (SD 3.2), P = .047].

Prevalence of Memory Complaint and Dementia

Of the 358 assessed individuals, 26 (7.3%) had memory complaints documented in their charts, and 6 (23.1%) of those with memory complaints had dementia (CDR ≥ 1). Dementia was also present in 41 (12.4%) of 332 participants without memory complaints.

Complete cognitive data were available on 339 participants, 25 (7.4%) of whose charts revealed memory complaints, and 33 (9.7%) of whom had dementia (CDR ≥ 1).

Among 19 individuals who did not complete all the cognitive tests, 14 had dementia; they were included in the AUC calculations for the tests that they completed but not in the CART analyses.

Screening Properties of Cognitive Tests

Areas Under the ROC Curve The MMSE had the highest AUC at 0.9121, followed by the HVLT immediate and delayed recall (Table 1). However, the AUCs were not significantly different among the MMSE, HVLT immediate and delayed recall, Rey Figure immediate and delayed recall, and Trailmaking B. These tests, as a group, had significantly higher AUCs than the Rey Figure copy, Trailmaking A, Clock Drawing, and verbal fluency tests.

Table 1.

Comparison of Areas Under the Receiver Operating Characteristics Curve (AUC)* for Cognitive Tests

Tests	Number	AUC*	Significance level (P values) for pairwise comparisons of areas under the curve (AUC) between cognitive measures
			MMSE	HVLT immediate recall	Rey Figure delayed recall	Rey Figure immediate recall	HVLT delayed recall	Trailmaking B	Verbal fluency category	Rey Figure copy	Clock Drawing	Verbal fluency letter	Trail-making A
			0.912	0.906	0.896	0.887	0.881	0.868	0.808	0.798	0.793	0.787	0.764
MMSE†	358	0.912		0.72	0.63	0.29	0.08	0.47	0.001	<0.001	0.001	<0.001	<0.001
HVLT†immediate recall	357	0.906			0.70	0.44	0.22	0.57	0.001	0.002	0.004	<0.001	0.001
Rey Figure delayed recall†	352	0.896				0.47	0.27	0.93	0.02	0.01	0.001	0.01	0.01
Rey Figure Immediate recall†	355	0.887					0.65	0.82	0.03	0.003	0.004	0.01	0.03
bHVLT delayed recall	357	0.881						0.60	0.02	0.046	0.02	0.03	0.02
Trailmaking B†	341	0.868							0.01	0.001	0.02	0.003	<0.001
Verbal Fluency category	353	0.808								0.77	0.52	0.57	0.36
Rey Figure copy	355	0.798									0.94	0.68	0.63
Clock drawing	355	0.793										0.94	0.64
Verbal Fluency initial letters	357	0.787											0.97
Trailmaking A	351	0.764

*Area under the curve, referring to receiver operating characteristic curves for each test in relation to dementia (CDR ≥ 1)

^†These tests are not significantly different in AUC from one another.

CART

The CART model (Fig. 1) and the sensitivity, specificity, PPV, and NPV of the final tree are based on test cut points selected by the CART procedure to maximize correct classification.

Figure 1.

Outcomes of CART analyses for the presence/absence of memory complaints in participants with complete cognitive data. Sens = sensitivity; spec = specificity; PPV = positive predictive value; NPV = negative predictive value; dr = delayed recall.

Among 25 participants with memory complaints and complete cognitive data, of whom 5 (20%) had dementia, neither the MMSE nor any of the domain-specific cognitive tests further improved their classification into subgroups with and without dementia. Because this was a small subsample, lack of power is a potential explanation.

Among 314 participants without memory complaints, 28 (8.9%) had dementia. CART revealed that 8 of these dementia cases had MMSE scores < 20, 11 had MMSE scores 20–21, and 9 had MMSE scores ≥ 22. The 11 dementia cases with mid-range MMSE scores 20–21 were all detectable by a score < 7 on the visual memory test (Rey Figure delayed recall). The algorithm failed to detect 9 patients with dementia but without memory complaints and with MMSE scores ≥ 22.

As noted, 19 individuals who did not complete all the cognitive tests were excluded from the CART analyses. Of these, 14 had dementia, and only one had a documented memory complaint. Their MMSE scores ranged from 10 to 21, with a mean (SD) of 16.6 (3. 4), suggesting they would also have been detected by the MMSE cut points suggested by CART analysis.

In post hoc analyses, depressive symptoms (mCES-D) score was introduced into the tree but did not alter the previous classification. Eleven participants with incomplete mCES-D data were excluded from these post hoc models.

DISCUSSION

In a sample of older primary care patients, we used two complementary statistical approaches to examine the relative abilities of self-reported memory complaints and several cognitive measures to classify individuals according to their probability of having dementia. This classification is not equivalent to a diagnosis of dementia but rather identifies the subgroup of individuals for whom a diagnostic assessment for dementia might be warranted. Based on receiver operating characteristics, the general mental status test (MMSE) had the best classificatory ability although not significantly higher than tests of verbal memory, visual memory, speed/executive functioning. In CART analyses, among participants with memory complaints, cognitive testing did not further improve classification. In those without memory complaints, the MMSE was an effective first-pass screen, supplemented by the visual memory test among those with equivocal MMSE scores. Several participants with dementia did not complain of memory loss and could not complete the entire cognitive assessment, but did, in fact, obtain low MMSE scores; they would also have been detected using the MMSE. Approximately 3% of participants without memory complaints, and with higher scores on the MMSE, also had dementia, which would remain undetected by these steps. Depression can be associated with negative self-assessment including memory complaints,⁴¹ but did not alter the outcome of the tree analyses.

Generally, the term case finding describes the selective assessment of individuals with symptoms or risk factors, whereas the term screening refers to assessment of unselected individuals.⁴² Case finding requires a lower threshold of evidence for assessment because the individuals have already identified themselves as symptomatic or otherwise at risk. Our data suggest that case finding in patients with symptoms of memory loss should proceed directly to clinical assessment without further testing. However, focused cognitive screening can improve classification in those without memory complaints, identifying a smaller subgroup for clinical assessment. In a sense, the memory complaint itself functions as a positive “screen,” with high specificity albeit low sensitivity because most participants with dementia had no memory complaints.

Approaches previously suggested for dementia detection include waiting for spontaneous complaints from patients and/or families, routinely asking informants about changes in patients’ cognitive functioning,^43,44 screening with general mental status tests,^14,22,23 and screening with specific memory tests^24,25 Whereas families may validly report the patient’s memory problems,^43–46 they may also underreport patients’ difficulties,^20,47 and not all individuals have reliable informants.

In our sample, the MMSE was of considerable value in the detection of dementia. The MMSE is both widely used and widely criticized in the research literature^22,24; its superior performance in the current context is understandable by its relatively global scope. Much of the literature on cognitive screening is focused on general mental status tests. In studies of the MMSE using a cut point of 23/24, sensitivity ranged from 21 to 100%, whereas specificity ranged from 46 to 100%²²; the wide range reflects the range of dementia severity in these studies. Other global screening tests, such as the Mini-Cog, with a sensitivity of 76% and specificity of 89%,¹⁴ and the 7-Minute Screen, with a sensitivity of 92% and specificity of 96%,²³ have been recommended for their brevity. Tests of specific cognitive domains have been examined less frequently. In different studies, the MMSE had significantly higher AUC than the Trailmaking Tests and Clock Drawing²⁴ and performed better than verbal fluency, Trailmaking Test, and Word List Learning at discriminating between individuals with and without dementia.²⁵ In our study, the equally good performance of the memory and executive functioning tests is most likely explained by the weighting toward memory in current definitions of dementia, reflected in the CDR.³⁵ Our data suggest a complementary role for a specific memory test if used only when scores on the general mental status test are equivocal; we believe this to be a novel finding and a practical suggestion.

The predictive value of a screening test depends in part on the prevalence of the disorder in a given population. The prevalence of dementia is relatively high in specialty settings such as geriatrics or memory disorders clinics,^47,48 relatively low in epidemiologic surveys,⁴⁹ and possibly intermediate in primary care. Because the majority of older adults receive their health services solely in primary care, these are the settings in which the utility of cognitive screening most needs to be determined.^21,50

Our cognitive testing was performed by nurses with geriatric experience and stringent, systematic training and supervision in these specific assessments. In the typical primary care setting, this degree of training is neither available nor necessarily appropriate. Although these tests are simple, it is critical that clinicians who use them be adequately trained in their administration, scoring, interpretation, and limitations. They should not be portrayed as suitable for use by untrained personnel or as “do-it-yourself Alzheimer’s tests.”

A potential limitation is the relatively large proportion of participants who refused detailed assessment and might, thus, have introduced response bias⁵¹; however, they were similar to those who were assessed with regard to demographics and only marginally different in MMSE score. We were limited to chart review in determining whether patients had reported memory problems to their physicians, and the relatively small number with documented memory complaints may have reduced power for CART analyses in their subgroup. Our specific results may not generalize exactly beyond our sample; that is, in another sample, CART might reveal a different MMSE cut point to be ideal, and a different memory (delayed recall) test may provide equal or better classification. With a larger sample, we may have found that additional tests improved classification further; however, the practical likelihood seems low that a primary care assessment can include more than two cognitive tests.

This study contributes data on how different screening measures and memory complaints may be useful in detecting dementia, but the detection of dementia is not an end in itself. When and whether dementia should be detected in asymptomatic patients will depend on the potential costs, harms, and benefits of detecting dementia,⁵⁰ along with relevant ethical considerations in different populations. A comprehensive discussion of harms and benefits is beyond the scope of this article, but key issues include the burden on physicians of providing detailed assessments to screen positive patients, the relative clinical and public health impact of false positive diagnoses versus leaving cognitive impairment and dementia undetected, and of early detection of an untreatable disorder in an asymptomatic patient. However, all cognitive impairment is not dementia, and not all dementia represents irreversible neurodegeneration; early detection and diagnosis may provide providers and patients with opportunities to intervene appropriately and limit excess disability through secondary and tertiary prevention.