Two-Stage Screening for Early Dementia in Primary Care

Abstract

OBJECTIVE:

To compare two screening strategies for dementia in an urban primary care clinic, serving a low education, minority community comprised largely of Latino and African American patients.

METHODS:

Two hundred fifty seven patients underwent two-stage Patient-Based Screening (PBS) and Informant-Based Screening (IBS) followed by a diagnostic evaluation. In the first stage, PBS included brief tests of episodic memory (Memory Impairment Screen), semantic memory (Animal Fluency), and executive function (Reciting Months Backwards). For IBS, the first stage consisted of the short Informant Questionnaire on Cognitive Decline in the Elderly, administered to a family member or friend. Patients who screened positive in the first stage of either strategy underwent testing with the picture version of the Free and Cued Selective Reminding Test with Immediate Recall to identify memory impairment. Sensitivity, specificity, positive and negative predictive values were computed for various cutoffs of each test and combination of tests. Dementia was diagnosed using DSM-IV criteria without access to the screening test results.

RESULTS:

We identified 66 patients (25.7%) with previously undiagnosed dementia. Sensitivity was the same (77%) for both strategies but specificity was higher for IBS than PBS (92% versus 83%). IBS’s higher specificity makes it the preferred strategy if a knowledgeable informant is available.

CONCLUSION:

Unrecognized dementia is common in primary care. Case-finding can be improved using either PBS or IBS two-stage screening strategies.

INTRODUCTION

As new treatments and preventive approaches for Alzheimer’s disease (AD) emerge, they will be implemented in primary care settings where the majority of older adults receive their care. According to a systematic review of the prevalence of missed and delayed dementia diagnosis, providers identify from 9 to 41 percent of their patients with mild dementia (Bradford et al, 2009) despite the availability of cognitive screening tests for dementia (Tsoi et al, 2015; Yokomizo et al, 2014). Many dementia screening tests are influenced by education or cultural factors (Yokomizo et al, 2014), a real disadvantage in urban primary care clinics that serve diverse communities. Misclassification rates for the most common cognitive screening test, the Mini Mental State Exam, are high for minority patients and for individuals with low education (Kraemer, Moritz, & Yesavage, 1998). Similarly, the Montreal Cognitive Assessment is thought to have low specificity for detecting cognitive impairment in older adults with low education (Bernstein et al, 2011; Wong et al, 2015). Education adjusted cutoff scores may improve discriminative validity but they are inconvenient, and cannot be applied in a uniform fashion across groups that differ in race and culture (Teresi et al, 2001; Kraemer et al, 1998; Mayeux et al, 2011). As education is a predictor of dementia risk, adjustment may reduce predictive validity for the future onset of dementia (Sliwinski et al, 1997; Grober et al, 2015). Though dementia screening programs in primary care have improved recognition (Boustani et al., 2005; Eichler et al., 2015), specificity has been modest (Boustani et al, 2005; Borson et al, 2006; Doody et al, 2011), resulting in unnecessary anxiety for patients and families and inefficient use of resources (Borson et al., 2013).

Because low education is a risk factor for dementia onset (Xu et al, 2015), screening methods that work in low education groups are particularly important. The need to identify primary care patients with early dementia in a single clinic visit prompted our development of two-stage screening strategies that are efficient and have high specificity (Grober, Hall, Lipton, & Teresi, 2008a; Grober et al., 2008b). Two stage models have been widely used to optimize accuracy and efficiency of screening (Denny, et al, 2000; McNamee, 2003). In the first stage, a brief high sensitive screen is administered to all patients aged 65 years and older; patients who fail the rapid screen undergo second stage memory testing to identify memory impairment. Herein, we compare two first stage strategies, patient based screening (PBS) and informant-based screening (IBS). PBS consists of brief memory and executive function tests given to the patient. Informant-based screening (IBS) is conducted with a family member or friend. Patients who fail the first rapid stage of either strategy undergo a thorough assessment of episodic memory with the picture version of the Free and Cued Selective Reminding Test with Immediate Recall (pFCSRT+IR) to identify memory impairment. With a specificity of >80%, both strategies had a sensitivity of > 75% for early dementia in a geriatric primary care clinic treating high school educated Caucasian and African Americans (the development cohort: Grober, Hall, Lipton, et al., 2008a). Sensitivity and specificity did not differ materially by race or education, an advantage in racially and educationally diverse cohorts.

The goal of this study was to assess these PBS and IBS strategies in an urban primary care clinic that served a lower education, minority community comprised largely of Latino and non-Latino Black patients. We sought to determine how well each strategy identified early dementia cases in this new cohort. There were three objectives: 1) to determine the optimal cut scores for PBS and IBS for dementia screening in the new sample starting with the cut scores from the development sample; 2) to determine whether classification accuracy was the same for patients with low versus high levels of education and for Latino and non-Latino Black patients; and 3) to determine whether one strategy should be preferred over the other.

METHODS

Setting.

The Memory Screening Project was carried out in a low education, clinic population of patients aged 65 years or older that is largely Hispanic (53%) or Black (36%), at the Adult Primary Care Clinic of the Jacobi Medical Center in Bronx, NY. Because we wanted to reflect the demographics of the clinic population, patients of any race/ethnicity were eligible to participate as long as they were 65 or older, spoke English or Spanish, had an informant, and did not have a dementia diagnosis in their electronic medical record at the screening visit. Six hundred seventy-one participants completed the screening evaluation between April, 2008 and May, 2012. Experienced bilingual examiners approached eligible patients at their scheduled appointment, recruited interested patients, obtained written consent, and administered the 20-minute screening battery at the patient’s convenience, before or after their physician visit. Two hundred fifty-seven of the screened patients subsequently underwent the diagnostic battery that included neuropsychological tests and interviews with a family member or friend used to establish the “gold standard” diagnosis. They comprise the cohort in the current analyses. The project was approved by the Institutional Review Boards at the Albert Einstein College of Medicine and Jacobi Medical Center.

Screening strategies.

We pre-specified a set of cognitive tests for stage one of PBS based on their brevity and operating characteristics and for stage one of the IBS, we used the short Informant Questionnaire on Cognitive Decline in the Elderly (IQCODE: Jorm, 1994). For both PBS and IBS, the pFCSRT+IR was used for stage two. For purposes of the study, all participants received the entire screening battery. The Figure shows a flowchart that describes the two-stage screening process that emerged from our analyses.

Figure.

Flowchart of Recommended Two-Stage Screening Strategies.

Stage one: Rapid Screening Tests

Performance-Based Screening (PBS).

Stage one of PBS is comprised of three tests. The Memory Impairment Screen (MIS) is a four-minute, four-item, delayed free- and cued-recall controlled learning test of episodic memory. (Buschke et al., 1999) Participants read four words aloud and then identify each word (e.g., pink) when its cue is presented (color). After three to four minutes of distraction by the other two tests in PBS, the individual is asked for free recall followed by cued recall of words that are not retrieved by free recall. The MIS score is calculated as follows: [2 × (free recall)] + [cued recall]. Animal fluency (AF), the second test in the screen, is a measure of semantic memory in which participants are asked to generate the names of as many animals as possible in one minute (Rosen, 1980). The third test involves executive functioning. Participants are asked to recite the months of the year backwards (MB) which involves manipulating an overlearned sequence in a novel way and keeping track of one’s place. The first stage of PBS does not take any longer to administer than the MIS alone because AF and MB are administered in the three to four-minute interval between presenting the four MIS words and testing memory for them. Recommended cut scores that trigger second stage testing include a score of ≤ 4 on the MIS, fewer than 10 animals generated in one minute, or uncorrected error(s) reciting the months backward or the inability to do the task shown by a score of 2 or higher (Grober, Hall, McGinn, et al., 2008a). MB replaced Trails B used in the development sample to assess executive functioning.

Informant-Based Screening (IBS).

Stage one of IBS is accomplished with the short IQCODE that assesses change in memory and intelligence as rated by a family member or friend (Jorm, 1994). The short form includes 16 of the original 26 items and operates as well as the long form to distinguish between older adults with and without dementia. A five-point scale indicates the degree of change in daily activities (e.g., remembering recent conversations and events, making decisions); a score of three indicates no change. A five-year time frame was used which is long enough to observe functional decline but avoids the difficulty of finding informants who have ten years of contact with the participant. (Pisani, Inouye, McNicoll, & Redlich, 2003). Higher scores mean greater impairment. The recommended cut score to trigger second stage testing is an average item score of ≥3.2 (Grober, Hall, McGinn, et al., 2008a).

Stage Two: Assessing memory Impairment to identify those who screen positive for dementia.

Patients who fail the first stage of either strategy undergo memory testing with the pFCSRT+IR (Grober & Buschke, 1987; Grober, Buschke, Crystal, Bang, & Dresner, 1988) Unlike other tests of episodic memory, it begins with a study phase in which participants search a card containing four pictures (e.g., grapes) for an item that goes with a unique category cue (e.g., fruit). After all four items are identified, the card is removed and immediate cued recall of the four items is tested. The study phase is continued for the next group of four items until all 16 items have been identified and retrieved in immediate recall. There are three test trials, each consisting of free recall followed by cued recall for items not retrieved by free recall for a maximum of 48 items. Items not retrieved by cued recall are re-presented as reminders. Each separate trial is followed by 20 seconds of interference. Total recall is the sum of free and cued recall. A free recall score of <=25 indicated dementia in the development cohort (Grober, Hall, McGinn, et al., 2008a). Although total recall was not used as a predictor in that cohort, it was here added to enhance the sensitivity of second stage testing. A Spanish version of the pFCSRT+IR was constructed using standard back-translation methods (Grober, Ehrlich, Troche, Hahn, & Lipton, 2014).

“Gold standard” diagnosis.

Patients were diagnosed as having no memory impairment, memory impairment but no dementia, or dementia. As in the previous screening studies (Grober et al, 2008a; 2008b), patients with memory impairment but no dementia were combined with the patients whose memory was not impaired to comprise the no dementia group. A consensus diagnosis for each participant was established among a neuropsychologist (EG) and a geriatrician (AE) using DSM IV criteria for dementia (American Psychiatric Association, 1994) purposely without input from the patient’s primary care provider or knowledge of the screening test results in order to avoid diagnostic circularity. A report was generated for each patient containing the test scores in Table 1 along with the 5^th, 10^th, and 50^th percentile scores for each test based on the performance of the first 100 patients without dementia at baseline. Also included in the report were informant’s responses to the Clinical Dementia Rating (CDR) interview (Morris, 1993) augmented by their responses to the Alzheimer’s Disease Cooperative Study Activities of Daily Living Scale (Galasko, Bennett, Sano, & al., 1997; Sano et al., 2006).

Table 1.

Diagnostic Battery.

Patient Testing	Instrument
Memory	CERAD Word List Learning Test (Welsh et al., 1994)
	Name and address recall (Blessed, Tomlinson, & Roth, 1968)
	Spatial Location Memory (Grober, 1984)
Executive functions	WORLD backwards (Folstein, 1975)
	Category fluency using fruits and vegetables (Rosen, 1980)
	Judgment and problem solving questions (Morris, 1993)
	Intrusions in Word List Learning Test (Welsh, et al., 1994)
	Months backwards (Blessed, et al., 1968)
Other Cognitive functions	Orientation (Folstein, 1975)
	Judgment and problem solving questions (Morris, 1993)
	pFCSRT+IR Naming (Grober et al., 2000)
	Counting up, counting down (Blessed, et al., 1968)
	Self reported ADLs (Dartigues et al., 1997)
Mood	Geriatric Depression Scale (Yesavage et al., 1982–1983)
Informant Interview	Instrument
Memory and cognitive impairment	CDR interview (Morris, 1993)
ADL impairment	ADCS ADL Scale (Galasko, et al., 1997; Sano, et al., 2006) CDR interview (Morris, 1993)

CERAD: Consortium to Establish a Registry in Alzheimer’s Disease

CDR: Clinical Dementia Rating

ADCS: Alzheimer’s Disease Cooperative Study

Before meeting at the consensus conference, EG and AE reviewed the report, made an independent determination of the patient’s diagnostic status, and then rated the patient’s cognitive performance and activities of daily living using the CDR Scale (Hughes et al, 1982; Morris, 1993). At the conference, patients were discussed when there was any disagreement on diagnostic criteria or CDR box score. The final CDR rating was based on the pattern of box scores (Morris, 1993). Except for MB, diagnoses were made without knowledge of PBS or IBS results.

Statistical Methods.

The demographic features and test scores of the patients with and without dementia were summarized. Group comparisons were performed using Wilcoxon-Mann-Whitney tests, Pearson chi-square tests, or Fisher’s exact tests when the expected frequency in any cell was less than or equal to 5. Sensitivity, specificity and efficiency were computed for various cutoffs of each candidate test. The word “efficiency” refers to the proportion of patients who screened positive at a cutoff of the candidate test. Because of limited time and resources, the intent was to minimize the number of patients who would require second stage testing while maintaining high levels of sensitivity (>80%). Sensitivity and specificity of PBS and IBS were summarized and the differences between them were compared using McNemar’s test (McNemar, 1947). Logistic regression was used to assess the risks for dementia of a positive screen versus a negative screen on each strategy. To determine whether educational level (≤8 years of education versus > 8 years) or ethnicity (Latino, non-Latino Black), affected either strategy, sensitivity, specificity, PPV, and NPV values from the education or ethnicity strata were compared using either the Pearson’s Chi-square test or Fisher’s exact test. Non-Latino Whites and non-Latino others were not included in the ethnicity comparisons because of their small sample size.

RESULTS

Table 2 shows the demographic characteristics of the 257 patients by dementia status. Sixty-six participants (26%) met DSM IV criteria for dementia based on the independent diagnostic evaluation, the majority of whom (55%) had very mild dementia (CDR=0.5). Of the 191 participants who did not meet criteria for dementia, 23 had memory impairment but no dementia. Patients with dementia were older (p=0.003) but did not differ from patients without dementia by gender (p=0.99), self-reported ethnicity and race (p=0.20), years of education (p=0.59), years residing in the United States (p=0.38), or months until the diagnostic assessment (p=0.67). Patients with dementia endorsed more depressive symptoms than patients without dementia (p=0.02). The sample’s ethnicity/racial distribution mirrored the distribution of the clinic population of patients 65 years or older. Latino patients who identified their race as either White or Black were included with the other Latino patients.

Table 2.

Demographic information as a function of dementia status.

	No dementia	Dementia	Group
	(n=191)	(n=66)	Comparison
Age at screening (year)	72.6 (5.5)	75.8 (7.5)	P = .003
Female	69.6%	69.7%	P = 0.99
Self-reported ethnicity/race			P = 0.20
Latino	56.0%	59.1%
Non-Latino Black	33.0%	25.8%
Non-Latino White	6.8%	4.6%
Non-Latino Other	4.2%	10.6%
Education, years	9.2 (3.7)	8.9 (4.0)	P = 0.59
<=8 years	43.3%	43.1%	P= 0.97
> 8 years	56.7%	56.9%
Years in US	42.5 (16.7)	44.7 (17.8)	P = 0.38
Language of administration			P = 0.17
English	53.9%	63.6%
Spanish	46.1%	36.4%
Geriatric Depression Scale	3.0 (3.0)	4.0 (3.2)	P=0.02
Months from screening to diagnosis
Mean (SD)	14.3 (12.4)	14.1 (12.8)	P=.67
Median (range)	10.8 (0–54)	7.6 (0–54)

Table 3 shows the means and standard deviations of the screening tests by ethnicity/race and dementia status. Month Backwards (MB) was summarized using percentages of correct responses. As expected dementia-free patients performed better than patients with dementia on every measure.

Table 3.

Summary statistics for the candidate screening tests as a function of dementia status and ethnicity/race.

Means (upper row) and standard deviations (lower row) for all tests except Months Backwards indicated by percent correct responses

Race/Ethnicity	Dementia Status	MIS	AF	MB	Clock Drawing	IQCODE	pFCSRT+IR Free Recall	pFCSRT+IR Total Recall
Latino	No Dementia	7.0	13.2	63.6%	10.2	3.1	31.9	47.5
	(n=107)	1.4	3.6		2.7	0.2	6.6	1.2
	Dementia	4.9	11.0	25.6%	7.8	3.8	19.7	42.7
	(n=39)	2.5	3.5		4.0	0.4	8.6	6.8
Non-Latino Black	No Dementia	7.0	13.6	74.6%	10.1	3.1	28.9	46.8
	(n=63)	1.2	3.6		2.5	0.2	6.3	2.5
	Dementia	5.2	9.9	52.9%	7.7	3.9	20.5	43.3
	(n=17)	2.3	3.2		3.8	0.6	7.3	8.5
Non-Latino White	No Dementia	7.2	16.6	62.5%	10.5	3.2	27.2	47.1
	(n=13)	0.6	3.2		1.8	0.3	7.0	1.8
	Dementia	4.7	10.3	28.6%	8.0	4.0	20.7	39.3
	(n=3)	4.2	4.5		6.1	0.9	2.1	10.3
Other Non-Latinos	No Dementia	7.1	9.4	84.6%	8.8	3.2	29.5	47.4
	(n=8)	1.5	2.5		2.8	0.3	3.2	1.4
	Dementia	6.3	10.3	66.7%	8.6	3.9	22.6	45.3
	(n=7)	1.9	1.5		3.6	0.5	6.9	1.8

Abbreviations:

MIS: Memory Impairment Screen

AF: Animal Fluency

MB: Months Backwards

IQCODE: Informant Questionnaire on Cognitive Decline in the Elderly

pFCSRT+IR: picture version of the Free and Cued Selective Reminding Test

Stage One.

PBS.

When the recommended cut scores for both stages of the PBS were applied (MIS<=4, AF<=9, uncorrected errors on MB 2, FR<=25) sensitivity was lower than in the development cohort (70% versus 82%) (Grober, Hall, McGinn, et al., 2008a). Therefore, we explored other cut scores to improve sensitivity. Table 4 (top) shows the sensitivity, specificity, and efficiency of each stage one test at various cutoffs. As cut scores are raised more patients screen positive and have to undergo second stage testing indicated by the efficiency values. Sensitivity of the MIS was 20–40% lower at every cutoff compared to its sensitivity in the initial validation study (Buschke et al, 1999). By raising the MIS cut score from 4 to 5, sensitivity increased from 32% to 47%, specificity remained high (90%), with only a small increase in screen-positive patients. We evaluated three different approaches for combining PBS based tests to operationally define failing Stage 1. Definitions included failing either the MIS or MB), failing the MIS or AF or failing any one of the MIS, MB and AF (Table 4, middle). By using impaired performance on any test to trigger second stage testing (MIS<=5, AF<=9; MB>=2 (uncorrected errors) instead of the MIS <=5 alone, sensitivity increased from 47% to 88%. The loss of specificity was offset by improved specificity from second stage testing. Fifty four percent of the sample screened positive at stage one.

Table 4.

Sensitivity, specificity, efficiency, and 95% confidence intervals for Stage 1 of PBS and IBS at various cutoffs.

Candidate tests	Cutoff	Sensitivity	Specificity	Proportion who screen positive (efficiency)
Patient Based Screening (PBS): Stage 1
Memory Impairment Screen (MIS)	≤4	32 (21,44)	94 (89,97)	13 (9,18)
	≤5	47 (35,60)	90 (85,94)	20 (15,25)
	≤6	70 (57,80)	76 (69,82)	36 (30,42)
Animal Fluency (AF)	≤9	42 (30,55)	84 (78,89)	23 (18,28)
	≤10	48 (36,61)	75 (69,81)	31 (25,37)
Months Backwards (MB)	≥1	73 (60,83)	57 (50,64)	51 (44,57)
	≥2	65 (52,76)	69 (61,75)	40 (34,46)
MIS<=5 or MB>=2	-----	79 (67,88)	65 (58,72)	46 (40,53)
MIS<=5 or AF<=9	-----	68 (56,79)	69 (62,76)	40 (34,47)
MIS<=5 or MB>=2 or AF<=9	-----	88 (78,95)	58 (50,65)	54 (48,60)
Informant Based Screening (IBS): Stage 1
IQCODE	≥3.1	97(89,100)	64 (56,71)	52 (46,58)
	≥3.2	94 (85,98)	79 (72,85)	40 (34,46)
	≥3.3	88 (78,95)	88 (83,93)	31 (26,37)
	≥3.4	82 (70,90)	92 (87,96)	27 (22,33)

IBS.

Table 4 (bottom) shows the sensitivity, specificity and efficiency of various IQCODE cut scores. While a score of >=3.4 has good sensitivity (82%) and high specificity (92%), higher sensitivity was desirable. Again, the modest loss in specificity with the more sensitive cut score of >=3.2 was offset by second stage testing. Ninety four percent sensitivity was obtained by applying the same cut score of >=3.2 used in the development cohort (Grober et al, 2008a). Forty percent of the sample screened positive.

Stage Two.

Table 5 (top) shows the sensitivity and specificity of free recall and total recall separately at various cutoffs for identifying early dementia. Free recall of <=25 had the best overall classification accuracy though cutoffs of 26 and 27 achieved similar results. Total recall scores of <=45 missed half of the early dementia cases. Adding total recall (≤46) to free recall as an indicator of impaired memory (Table 5, bottom) improved sensitivity by 7 to 9 points over that obtained when only free recall was used as in previous studies (Grober, Hall, McGinn, et al., 2008a; Grober, Sanders, Hall, & Lipton, 2010). Classification accuracy was highest when free recall <=25 or total recall <=46 were used to identify early dementia cases.

Table 5.

Sensitivity, specificity, and 95% confidence intervals for free recall and total recall alone and combined.

pFCSRT+IR	Cutoff	Sensitivity	Specificity
Free Recall (FR)	≤24	68 (56,79)	81 (75,86)
	≤25	76 (64,85)	77 (71,83)
	≤26	76 (64,85)	75 (68,81)
Total Recall (TR)	≤44	41 (29,54)	92 (87,96)
	≤45	47 (35,60)	90 (85,94)
	≤46	58 (45,70)	85 (80,90)
FR<=24 or TR<=46		77 (65,87)	73 (66,79)
FR<=25 or TR<=46		83 (72,91)	70 (63,77)
FR<=26 or TR<=46		83 (72,91)	68 (61,75)

Table 6 shows the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) and their 95% confidence intervals (CI) for PBS in identifying early dementia overall, by education level and by ethnicity (Latino versus non-Latino Blacks). The overall sensitivity was 77% and specificity was 83%. Fifty-four percent of patients screened positive at the first stage by scoring ≤5 on the MIS, naming 9 or fewer animals or making uncorrected errors reciting the months backwards. Sensitivity, specificity, PPV and NPV did not differ significantly between patients with more than eight years of education compared with less educated patients (p-values range: 0.53–0.83) or between Latino patients and non-Latino Black patients according to every measure but none were significantly higher (p-values range: 0.14–1.00).

Table 6.

Sensitivity, specificity, positive and negative predictive values of Performance-Based Screening (PBS).

(Stage 1: MIS≤5 or AF≤9 or MB>=2) (Stage 2: pFCSRT+IR free recall ≤25 or total recall ≤46)

PBS	Sensitivity	Specificity	PPV	NPV
Overall	77 (65,87)	83 (77,88)	61 (50,72)	91 (86,95)
≤8 education years	82 (63,94)	81 (71,89)	61 (43,76)	93 (84,98)
>8 education years	73 (56,86)	85 (77,91)	63 (47,77)	90 (82,95)
Latino	79 (64,91)	85 (77,91)	66 (51,79)	92 (85,96)
Non-Latino Blacks	71 (44,90)	79 (67,89)	48 (28,69)	91 (80,97)

MIS: Memory Impairment Screen

AF: Animal Fluency

MB: Months Backwards

pFCSRT+iR: picture version of the Free and Cued Selective Reminding Test with Immediate Recall

PPV: positive predictive value

NPV: negative predictive value

Table 7 shows the sensitivity, specificity, PPV, and NPV for IBS in identifying early dementia overall, by education level and by ethnicity (Latino versus non-Latino Blacks). The overall sensitivity was 77% and specificity was 92%.

Table 7.

Sensitivity, specificity, positive and negative predictive values of Informant Based Screening (IBS).

(Stage 1: IQCODE≥3.2) (Stage 2: pFCSRT+IR free recall ≤25 or total recall ≤46)

IBS	Sensitivity	Specificity	PPV	NPV
Overall	77 (65,87)	92 (87,95)	76 (64,86)	92 (87,95)
≤8 education years	79 (59,92)	93 (84,97)	79 (59,92)	93 (84,97)
>8 education years	76 (59,88)	92 (84,96)	76 (59,88)	92 (84,96)
Latino	85 (69,94)	93 (86,97)	80 (65,91)	94 (88,98)
Non-Latino Blacks	65 (38,86)	92 (82,97)	69 (41,89)	90 (80,96)

IQCODE: Informant Questionnaire on Cognitive Decline in the Elderly

pFCSRT+iR: picture version of the Free and Cued Selective Reminding Test with Immediate Recall

PPV: positive predictive value

NPV: negative predictive value

Forty percent of patients screened positive at the first stage using the cut score of ≥3.2 on the IQCODE. Sensitivity, specificity, PPV and NPV values were very similar between education levels (p-values range: 0.78 – 0.81) and did not differ significantly between Latino patients compared to non-Latino Black patients (p=0.35–1.00) though there was a trend for higher sensitivity among Latinos (85% versus 65%, p=0.10).

The specificity of IBS was significantly higher than that of PBS (92% versus 83%) at equivalent sensitivities (77%) (McNemar’s statistic = 9.14, p=.004). Of 191 non-cases, both strategies correctly classified 154 patients and misclassified 12 patients. IBS correctly classified 20 non-cases that PBS misclassified as cases, whereas the PBS correctly classified only 4 non-cases that IBS misclassified as cases. In other words, when the two strategies diverged, IBS correctly classified five times as many non-cases as PBS.

Patients who screen positive on IBS (OR=37.0, (95% CI: 17.1,79.9) were at twice as likely as the patients who screen positive on PBS versus (OR=16.9,95% CI: 8.5, 33.7) to have a ‘gold standard” diagnosis of dementia. Of 614 eligibly screened patients only 257 received a “gold standard” clinical diagnostic assessment do to a funding gap. Comparisons between the screened group with and without diagnoses did not reveal significant differences in age, gender distribution, education, or performance on any of the screening tests. A similar proportion with and without diagnoses screened positive at stage one of the PBS (.54 versus .53). Twenty-nine percent without diagnoses screened positive on both PBS stages, consistent with the 26% of diagnosed dementia in the study.

For many of the patients we did follow-up, the interval between screening and diagnosis was extraordinarily long. Table 8 shows the non-significant differences in sensitivity and specificity of PBS and IBS at six-month intervals (p-values = 0.18–0.72). Sensitivity was highest when the diagnostic evaluation was conducted six to twelve months after screening and lowest, two or more years later.

Table 8.

Sensitivity and specificity of PBS and IBS at six-month intervals from date of screening to date of diagnosis.

Time interval	Number of Patients	PBS sensitivity	PBS specificity	IBS sensitivity	IBS specificity
≤6 months	96	81	80	81	88
6–12 months	42	100	91	89	100
12–18 months	29	83	87	83	96
18–24 months	28	75	79	75	92
>24 months	58	61	83	67	88

DISCUSSION

We compared PBS and IBS as approaches for identifying primary care patients with early dementia in a low education Bronx clinic largely comprised of Latino and African American patients. The first stage of PBS consisted of brief tests of episodic and semantic memory and executive functioning; IBS consisted of a brief informant interview. The second stage of both strategies consisted of the pFCSRT+IR to identify memory impairment. Twenty-six percent of the 257 patients met research criteria for dementia, indicating a significant burden of unrecognized dementia despite excluding patients with a medical diagnosis of dementia at the time of screening.

IBS outperformed PBS on several measures. While both strategies were equally sensitive to early dementia (77%), specificity was significantly higher for IBS than PBS (92% versus 83%). The risk of dementia was twice as high for screen-positive patients on IBS than PBS (OR: 37% versus 17%). Forty percent of the patients screen positive in IBS and have to undergo pFCSRT+IR testing compared to 54% for PBS. There was no evidence of educational bias in IBS. This makes sense because the IQCODE, stage one of the IBS, is unaffected by the educational level of the informant (Jorm, 2004) and because pFCSRT+IR performance, the second stage of the IBS, is not materially affected by years of education (Grober et al, 1998; Grober et al, 2010). IBS identifies patients with both functional decline and memory impairment rather than memory impairment alone, which may explain its higher specificity. IBS’s higher specificity, greater efficiency, and absence of educational bias make it the preferred strategy if a knowledgeable informant is available. Since low education is a risk factor for dementia onset (Xu et al, 2015), IBS has promise as an early dementia screener among low education primary care patients.

IBS was equally effective as a screener in this low education cohort (sensitivity 77%, specificity 92%) as it was in a primary care sample with 12 years of education (sensitivity 77%, specificity 91%) (Grober et al, 2008a). Our use of the IQCODE differed from the developers in two ways (Jorm, 2004). First we used a 5 year time horizon for assessing change rather than the original 10 year recall interval. Second, we used a cut score of 3.2, rather than the developers cut score of 3.44.

The sensitivity of the MIS was lower across a range of cut-scores than in previously studied better educated samples (Buschke et al, 1999). This may reflect the very mild degree of dementia in the current sample. The sensitivity of PBS was increased by combing the MIS with AF and MB with logical “ors”. None-the-less, the sensitivity (77%) and specificity (83%) of PBS was lower in the current sample than in the development cohort (82% and 88% respectively) (Grober et al, 2008a). These differences could result from differences in sample characteristics, determined by recruitment methods, education levels, and eligibility criteria.

In a study designed to validate screening tools for dementia, the disposition of persons with mild cognitive impairment (MCI) is an issue. MCI is conceptualized as a condition intermediate between cognitive normality and dementia. In some contexts, it would be reasonable to screen for either MCI or dementia, two groups that could be targets for intervention. In this study, our explicit goal was to screen for dementia. Among the 191 patients classified as “free of dementia”, 23 had amnestic MCI. Persons with aMCI are more likely to screen positive for dementia while being classified as normal (Bondi et al, 2014). Fifteen of the 23 aMCI patients displayed impaired pFCSRT+IR performance (FR<=25 or TR<=46). Eliminating MCI participants from the sample would make it easier to discriminate early dementia cases from those with no memory impairment though this approach would make results difficult to apply in realistic clinical settings.

This study has a number of weaknesses. Do to a funding gap, only 38% of screened patients had a timely gold standard assessment. Fortunately, we were able to compare the demographic and cognitive characteristics of the screened patients who did and did not have a gold standard diagnostic evaluation. There was no evidence of participation bias. The screened group with and without diagnostic evaluations did not differ in age or gender distribution, education, or performance on any of the screening tests. Another weakness was the interval between screening and diagnosis, which averaged 10 months. Screening status is fixed at the time of the initial assessment when patients screen positive or negative. If the gold standard status does not change over time, then the interval between screening and diagnosis does not matter. If the gold standard status changes, true positives can become false positives if disease “remits”, or false positives can become true positives as disease develops or worsens. One possibility is that those who screen negative and do not have dementia (true negatives), may develop dementia prior to the gold standard assessment; this would attenuate sensitivity and specificity. This effect would make our estimates conservative. Another possibility is that some individuals may screen positive while free of dementia (false positives) and subsequently develop the disease. This could lead to overestimates of sensitivity though we expect this effect would be modest. Another weakness was the limited power to detect differences in screening efficacy for Latino patients compared to non-Latino Blacks patients.

Despite the lack of an effective treatment to prevent the progression of AD pathology, there are still reasons to screen primary care patients aged 65 and older for early dementia. First, screening would reassure the “worried well” that their concern about declining cognition is age-related rather than dementia-related. Patients who screen negative on PBS or IBS can be highly confident (>90%) that they do not have dementia. Second, both strategies identified 77% of patients with very mild dementia, an improvement over the 9 to 41% of the mild dementia cases primary care physicians recognize (Mitchell et al, 2011; Connolly et al, 201). However, for non-Latino Blacks, the probability that a patient who screened positive on PBS has dementia (PPV) was quite low (48%). The value of identifying early dementia cases will ultimately be measured by comparing the potential benefits of appropriate medical treatment and care management that follow identification relative to the costs of failing to identify early dementia and the costs of the screening.

An advantage of two-stage screening is that it eliminates the need for additional neuropsychological testing because memory impairment is ascertained by the screener. However, screening positive is not a clinical diagnosis of dementia for many reasons including the false positive and false negative rates. Clinicians need to do additional assessments following diagnostic guidelines, including an interview with a family member or friend. We conducted our screening studies in two urban primary clinics. We expect results to be similar in similar settings but generalizability has not been assessed.

Our approach differs from other screeners in that the second stage, performed seamlessly after the first stage, is accomplished with the pFCSRT+IR. The use of the FCSRT to detect aMCI and dementia, predict future dementia and AD, and distinguish AD dementias from nonAD dementias has been well established in academic and clinical registry studies (e.g., Auriacombe et al, 2010; Derby et al, 2014; Donohue et al, 2014; Sarazin et al, 2007). Performance defines the core clinical phenotype for prodromal AD in the International Working Group criteria (Dubois et al, 2007; 2014). The test has excellent psychometric properties as a measure of memory (Grober et a, 2009). Finally, accumulating data demonstrate its association with CSF biomarkers, neuroimaging findings and autopsy-markers of AD (e.g., Grober et al, 1999; Rami et al, 2011; 2012; Sarazin et al, 2010; Wagner et al, 2012).

The strengths of the study include the administration of an early dementia screener in an urban primary care clinic in the course of routine care. The existence of educationally neutral screeners eliminates one of the barriers to screening asymptomatic primary care patients for dementia (Boustani, Peterson, Hanson, Harris, & Lohr, 2003; Brayne, Fox, & Boustani, 2007; Lin, O’Connor, Rossom, Perdue, & Eckstrom, 2013). In conclusion, unrecognized dementia in primary care settings can be reduced through effective and accurate two-stage screening.