Comparison of Two Informant Questionnaire Screening Tools for Dementia and Mild Cognitive Impairment: AD8 and IQCODE

Abstract

Background

Dementia and mild cognitive impairment (MCI) are under-recognized in community settings. This may be due in part to the lack of brief dementia screening tools available to clinicians. We compared two brief, informant-based screening tests: the AD8 and the Informant Questionnaire on Cognitive Decline in the Elderly (IQCODE) in a community-based neurology practice in the Midwestern United States

Methods

We examined 186 consecutive patients (44 controls, 13 with MCI, and 129 with dementia). Receiver operator characteristic curves were used to examine the ability of AD8 and IQCODE to discriminate between controls and MCI or dementia. Sensitivity, specificity, predictive values, and likelihood ratios were reported.

Results

AD8 differentiated healthy controls from MCI (p<.001) or dementia (p<.001), as well as MCI from dementia (p=.008). The IQCODE differentiated controls and MCI from dementia (both p<.001), and between controls and MCI (p=.002). Both AD8 (AUC = 0.953, 95% CI: 0.92–0.99) and IQCODE (AUC = 0.930, 95% CI: 0.88–0.97) provided discrimination between controls and patients with dementia; however the AD8 had superior sensitivity detecting dementia (99.2%) and MCI (100%) compared to the IQCODE (79.1% for dementia, 46.1% for MCI) with non-overlapping confidence intervals. Using published cut-offs (AD8 ≥ 2, IQCODE ≥ 3.4), only one case of dementia was missed with the AD8 while the IQCODE failed to detect dementia in 27 individuals. The AD8 detected MCI in all 13 individuals while the IQCODE misclassified 7 individuals.

Conclusion

Both the AD8 and IQCODE were able to detect dementia in a community setting. The AD8, however, was more successful than IQCODE in detecting MCI. If simple and efficient screening for early cognitive impairment is the goal, particularly in the early stages (e.g., for prevention trials or public screening), the combination of an informant interview (the AD8) and a brief performance measure could be considered as they meet the basic requirements of the Personalized Prevention Plan for Medicare beneficiaries.

INTRODUCTION

Despite the benefits of early detection, including effective treatment of the disease and its complications and enabling the patient (and family) to prepare for the future, dementia is still under-recognized in the community.^1,2 This is in part due to dismissal by patients and families of early signs of the disease as normal aging, denial, lack of time in a busy clinical practice and the lack of time-sensitive, effective screening tools.³ Given the brief time available to primary care physicians in a standard office visit, sensitive and specific cognitive impairment screening tools that are valid, easy to administer, and minimally time consuming are needed.^4,5

The screening for and assessment of cognitive impairment and dementia has traditionally been carried out by comparing individual performance on cognitive test measures with normative values.^6–8 However, since the diagnosis of Alzheimer disease (AD) and related dementias is founded on intra-individual decline in cognition with interference in accustomed daily activities, a complement to the conventional approach to dementia screening, (i.e., comparing cognitive function to normative values), is to assess cognitive change in an individual.^5,9–11 In assessing cognitive change it is necessary to have estimates of both current and pre-morbid levels of ability. Assessment of current cognitive ability poses little problem, but records of pre-morbid ability are rarely available. A possible solution to the problem of estimating cognitive decline is to use informants who have knowledge of both the subject’s current and pre-morbid behavior.^12,13 The informant is usually asked to rate change over a period of time from earlier in life (i.e., reported intra-individual change). Informant reports permit the use of patients as their own controls while eliminating the need for baseline assessments.^5,9 Moreover, assessing cognitive decline, rather than current functioning, does not require accounting for the level of education, pre-morbid intelligence and cultural differences. Other advantages of informant questionnaires are: relevance to everyday life, greater cross-cultural portability, applicability to people with limited education, less threat to the self-esteem of the person assessed, and the possibility of assessment by mail or telephone.^12,13 The main disadvantage of informant interviews has been that they have been time consuming. The Gold Standard for informant assessments, the Clinical Dementia Rating (CDR)¹⁴ can take 45–60 minutes to complete and thus is not practical in a typical clinical office visit. Therefore, more recently, there has been an effort to develop brief informant interviews to gauge intra-individual change.

Two commonly used informant questionnaires are the AD8^5,9–11 and the Informant Questionnaire on Cognitive Decline in the Elderly (IQCODE).^15–17 Both were developed and validated as part of longitudinal studies of memory and aging to assess the presence of dementia. The AD8 has 8 yes/no questions and takes 2–3 minutes to complete.⁹ The IQCODE has 16 items scored on a 5-point Likert scale and takes 5–7 minutes to complete.¹⁵ Both the AD8 and IQCODE can be self-administered by the informant without the assistance of medical or office staff. Because of the ease of use for dementia screening, they could prove useful for future public health initiatives. In this study, we compared the IQCODE and the AD8 as screening tools for dementia and mild cognitive impairment (MCI)¹⁸ in a community-based practice.

METHODS

Clinical Data

A total of 219 consecutive patients visiting a community-based Neurology/Memory Clinic in the Midwestern United States were asked to participate. As the purpose of this study was to investigate two informant-based dementia screening tests to detect dementia and MCI in a community setting, 33 patients with co-morbid medical conditions (e.g., sleep apnea, alcoholism, schizophrenia) that could potentially affect cognition but were not dementias were excluded from these analyses. Neither the AD8 nor IQCODE have been studied in these populations. The majority were patients seen in a Memory Clinic (N = 152), while additional controls (N = 34) were individuals without memory complaints seen in the setting of a General Neurology clinic for a final sample size of 186. Clinical data were collected between August 2006 and October 2008 and assessments were performed by one Board-Certified behavioral neurologist (MR) throughout the study. The majority of informants were adult children and/or spouses.

Informant Ratings of Cognitive Status

The AD8⁵ and IQCODE were independently completed by the informant, and the study clinicians were blinded to the scores. The AD8 gauges the informant’s perception of the target’s problems in the areas of memory, problem-solving abilities, orientation, and daily activities due to cognitive changes over the last “several years.” The AD8 is comprised of 8 items and respondents indicate their endorsement as Yes (1) or No (0). Items are summed resulting in a possible range of 0 – 8. The suggested cut-off score for dementia is 2 or greater.^5,9–11 The AD8 is highly correlated with Gold Standard evaluations including the CDR¹⁴ and neuropsychological testing⁹ and with imaging and cerebrospinal fluid (CSF) biomarkers of AD.¹⁹

The second informant measure was the IQCODE.^15–17 Informants rate the target’s cognitive change compared to 10 years prior. The IQCODE is comprised of 16 items representing everyday “situations where this person has to use his/her memory or intelligence.” Respondents use a 5-point Likert scale to indicate the degree of change (1 = Much Improved to 5 = Much Worse) compared to ten years prior. A total score is derived by calculating the average of completed items. The cut-off for dementia suggested in the literature ranges from 3.4 – 3.9.¹⁷

Cognitive Evaluation

Each patient and informant underwent a detailed history of cognitive and neurological symptoms and each patient received a comprehensive neurological and cognitive examination, performed by a single Board-certified Behavioral Neurologist (MR). As part of this comprehensive assessment, each patient was assessed for cognitive performance with the Mini-Mental State Examination (MMSE),²⁰ the Clock Draw task,²¹ a three-word recall to calculate a Mini-Cog score,^22,23 and a category fluency task (Animal Naming).²⁴

Clinical Diagnoses

Individuals with MCI were diagnosed according to published criteria²⁵ and categorized either as an amnestic or non-amnestic form based on the presence of memory impairment demonstrated by patient performance on testing and interviews with the patient and informant. Diagnoses of AD²⁶, Lewy body dementia (LBD)²⁷, vascular dementia (VaD)²⁸ and frontotemporal dementia (FTLD)²⁹ were made according to published criteria at the time of project initiation. Individuals were considered to have mixed dementia if they met criteria for two or more neurodegenerative disorders. Cognitively normal controls were recruited from a general neurology clinic and did not meet criteria for MCI or dementia. The AD8 and IQCODE were not used in establishing the clinical diagnosis.

Statistical Analyses

All analyses were conducted using SAS v9.3 (SAS Institute Inc., Cary, NC). Descriptive statistics were used to characterize and compare groups. Given the ordinal scale on which our different concepts were measured, we used non-parametric methods to compare the three diagnostic groups including Kruskal-Wallis tests for overall differences and post-hoc Wilcoxon rank-sum tests to test for specific differences (e.g., control vs. MCI). Regression analysis with estimate statements was used to obtain overall and specific group differences in means, while differences in proportions were tested with the chi-square test for overall differences and with the Marascuilo test³⁰ for specific comparisons. Spearman’s rho correlations were used to examine strength of associations between informant and performance measures. Sensitivity, specificity, positive and negative predictive values are reported. Receiver operator characteristic (ROC) curves and the area under the ROC curve (AUC) were generated to reflect graphically and quantitatively the ability of the AD8 and IQCODE to discriminate between cognitively normal patients and patients with dementia. Analyses were repeated to determine discriminative properties of the AD8 and IQCODE between cognitively normal patients and patients with MCI. Although nonparametric methods to estimate AUC are available, an underestimation compared to parametric AUC estimates has been reported particularly in the case of discrete (e.g., ordinal) data.³¹ We examined this by using Wilcoxon rank-sum statistics to calculate AUC to exclude the possibility that nonparametric methods would lead to significantly different AUC estimates. The following formulas were used to obtain 95% CI around the nonparametric AUC estimates:

$$\begin{array}{l}\sigma =[\text{AUC}(1-\text{AUC)}+(\mathrm{N}\text{diseased})(\mathrm{q}1-{\text{AUC}}^2)+({\mathrm{N}}_{\text{non}-\text{diseased}})(\mathrm{q}2-{\text{AUC}}^2)]/{{\mathrm{N}}_{\text{diseased}}}^{\ast }{\mathrm{N}}_{\text{non}-\text{diseased}}\\ \text{Where}\mathrm{q}1=\text{AUC}/(2-\text{AUC})\text{and}\mathrm{q}2=2\ast {\text{AUC}}^2/(1+\text{AUC})\\ 95\%\text{CI}=\text{AUC}\pm \text{sqrt}(\sigma )\end{array}$$

Another way to evaluate the utility of screening tests is with the likelihood ratio.^19,32 The likelihood ratio of any screening test is the probability that a positive test is found in persons with disease divided by the probability of the same finding in persons without disease.³² Likelihood ratios range from 0 to infinity with larger numbers providing more convincing evidence of disease; smaller numbers argue the disease is less likely; ratios close to 1 lack diagnostic value. Positive likelihood ratios greater than 5 or negative likelihood ratios less than 0.2 increase the probability of disease. Likelihood ratios were computed for both AD8 and IQCODE.

RESULTS

Participant Characteristics

Participant characteristics by group are summarized in Table 1. The sample consisted of 186 participants. The mean age of the patients was 77.8±8.2y with 13.2±2.9y of education. The cohort was 67% female and 38% were married with a mean MMSE score of 24.8±4.4. Clinical diagnoses for the group were 44 Controls, 13 MCI (10 amnestic, 3 non-amnestic), and 129 Dementia (64 AD, 40 mixed AD/VaD, 13 VaD, 10 LBD, 2 FTLD). Individuals with dementia diagnoses were rated as mild (69.8%), moderate (29.5%) or severe (0.8%). As expected, MMSE and Mini-Cog scores significantly differed across groups and were highest in the controls and lowest in the dementia group. However, Animal Naming and Clock Drawing were not able to capture differences between MCI and controls (p=0.85 and p=0.06, respectively).

Table 1.

Sample Characteristics

	Clinical Diagnosis			p-value
	Control (n=44)	MCI (n=13)	Dementia (n=129)	Overall differences	Control vs. MCI	Control vs. Dementia	MCI vs. Dementia
Variables Used in Establishing Clinical Diagnoses
Age (years)	72.9 (11.5)	77.0 (7.1)	78.2 (7.9)	.132	.234	.050	.602
Education (years)	15.1 (2.9)	13.7 (3.3)	12.9 (2.7)	.012	.212	.004	.293
Gender (% female)	84.1	46.2	62.8	.009	.039	.009	.516
MMSE	29.4 (0.8)	27.0 (1.4)	23.1 (4.1)	<.001	<.001	<.001	<.001
Mini-Cog	3.8 (0.6)	1.9 (1.1)	1.1 (0.9)	<.001	<.001	<.001	.006
Animal Naming (#/min)	15.5 (3.6)	14.7 (3.0)	9.8 (3.7)	<.001	0.848	<.001	<.001
3-word Delayed Recall	2.8 (0.6)	1.2 (0.9)	0.5 (0.7)	<.001	<.001	<.001	.003
Clock Draw (% abnormal)	0	30.8	45.2	<.001	.056	<.001	.561
Variables Under Study
AD8	1.1 (1.4)	3.7 (1.7)	5.2 (1.7)	<.001	<.001	<.001	.008
IQCODE	3.1 (0.4)	3.4 (0.5)	3.9 (0.6)	<.001	.002	<.001	<.001

Note: MCI = mild cognitive impairment; MMSE = Mini-mental State Exam, IQCODE = Informant Questionnaire on Cognitive Decline in the Elderly

Informant Ratings

AD8 and IQCODE scores differed significantly between the three comparison groups (overall and specific comparisons (Table 1). Both the AD8 and the IQCODE were correlated with all performance measures (all p-values <.001, Table 2).

Table 2.

Strength of Association between Informant Ratings and Performance Measures

	AD8	IQCODE
IQCODE	.831	--
MMSE	−.664	−.642
Mini-Cog	−.564	−.530
Animal Fluency	−.464	−.445
3-word Recall	−.548	−.496
Clock Draw	−.326	−.350

All p-values <.001 for Spearman’s Rho Coefficients

Key: IQCODE=Informant Questionnaire on Cognitive Decline in the Elderly; MMSE=Mini Mental State Examination

Discriminating Cognitively Normal Individuals from Dementia

We next examined the ability of the two informant scales to correctly classify individuals’ clinical diagnoses (Table 3). ROC curves were generated to measure the properties of the AD8 and IQCODE in classifying cognitively normal control individuals from those with dementia. Both the AD8 (AUC = 0.953, 95% CI: 0.92–0.99) and the IQCODE (AUC = 0.930, 95% CI: 0.88–0.97) discriminated dementia from no dementia. Although smaller, the non-parametric estimates showed similar trends as found using the parametric method (dementia vs. controls: AUC_AD8=0.883 (95% CI: 0.833–0.933) and AUC_IQCODE=0.839 (95% CI: 0.779–0.899). The AD8 had superior sensitivity (99.2%, 95% CI: 95.7–99.8) for the presence of dementia compared with IQCODE (79.1%, 95% CI: 71.3–85.2) with non-overlapping confidence intervals. The specificity and positive predictive value (PPV) did not differ between the AD8 and IQCODE. However the negative predictive value (NPV) of the AD8 was superior to the IQCODE (97.1% vs. 59.1% with non-overlapping confidence intervals). There was no difference in the likelihood ratio of a positive test between the AD8 and IQCODE, however the AD8 had a superior likelihood ratio of a negative test (0.01 vs. 0.24) compared with the IQCODE. Applying the published cut-offs (AD8 ≥ 2, IQCODE ≥ 3.4), the AD8 failed to detect dementia in only one individual while the IQCODE failed to detect dementia in 27 individuals.

Table 3.

Discriminative Ability and Psychometric Properties of the AD8 and IQCODE

Classification by Clinical Diagnoses
	AD8		IQCODE
	Not impaired (AD8 <2)	Impaired (AD8 ≥2)	Not impaired (IQCODE<3.4)	Impaired (IQCODE ≥ 3.4)
No impairment	34	10	39	5
MCI	0	13	7	6
Dementia	1	128	27	102

Psychometric Properties
	Dementia vs. No Dementia		MCI vs. No Dementia
	AD8	IQCODE	AD8	IQCODE
AUC	.953 (.915–.992)	.930 (.885–.975)	.899 (.821–.978)	.772 (.606–.938)
Sensitivity, %	99.2 (95.7–99.8)	79.1 (71.3–85.2)	100 (77.2–100)	46.1 (23.2–70.9)
Specificity, %	77.3 (63.0–87.2)	88.6 (76.0–95.1)	77.3 (63.0–87.2)	88.6 (76.0–95.1)
PPV, %	92.8 (87.2–96.0)	95.3 (89.5–97.9)	56.5 (36.8–74.4)	54.6 (28.0–78.7)
NPV, %	97.1 (85.5–99.5)	59.1 (47.1–70.1)	100 (89.9–100)	84.8–(71.8–92.4)
+ LR	4.4 (3.6–5.3)	6.9 (4.7–10.4)	4.4 (3.6–5.4)	4.1 (1.9–8.8)
− LR	0.01 (0.001–0.07)	0.24 (0.21–0.26)	0 (undefined)	0.6 (0.4–0.8)

Area under the curve (95% confidence interval)

Note: MCI = mild cognitive impairment; IQCODE= Informant Questionnaire on Cognitive Decline in the Elderly AUC = area under the curve (parametric analyses); PPV = positive predictive value; NPV=negative predictive value; LR=likelihood ratio.

Discriminating Cognitively Normal Individuals from MCI

We repeated these analyses by only considering individuals with MCI compared with cognitively normal older adults (Table 3). Although the sample size is small (n=13) both the AD8 (AUC = 0.899, 95% CI: 0.82–0.98) and IQCODE (AUC = 0.772, 95% CI: 0.61–0.94) discriminated cognitively normal individuals from those with MCI. Similarly to the normal-dementia comparison, the non-parametric AUC estimates were slightly lower but showing the same trend (AUC_AD8=0.886 (95% CI: 0.760–1.012 and AUC_IQCODE=0.674 (95% CI: 0.494–0.854). There were no differences in sensitivity, specificity, PPV, NPV or likelihood ratio of a positive test between AD8 and IQCODE for detecting MCI, however the likelihood ratio of a negative test favored the AD8 (0 vs. 0.6). Applying the published cut-offs (2 and 3.4), AD8 detected MCI in all 13 individuals whereas the IQCODE failed to detect MCI in 7 individuals.

DISCUSSION

Using brief informant interviews enabled us to detect dementia and MCI at the earliest stages by placing emphasis on reported intra-individual decline, rather than inter-individual comparisons of performance according to published norms.⁹ We found that two brief informant rating systems, the AD8 and IQCODE were able to detect the presence of cognitive impairment in community settings and were highly correlated with brief assessments of cognitive ability (MMSE, Mini-Cog, Clock Drawing, and Animal Naming) that are commonly used in community settings. Informants (spouses, adult children) using the AD8 were able to identify all but one individual with cognitive impairment. Using the IQCODE, informants were less successful in identifying individuals with cognitive impairment, characterizing 34 individuals with MCI or dementia as having no impairment. The false positive rates for the AD8 and IQCODE were not different.

Cognitive impairment due to dementia or MCI is under-recognized in the community.^1,2 There are a number of reasons this may be true, but at least part of the cause may be due to the lack of brief measures that can discriminate normal aging from very mild impairments in a time-effective manner. A number of brief performance-based dementia screening measures (e.g., the MMSE²⁰) are already in use, but may be (1) unable to detect or quantify change from previous levels of function; (2) insensitive to subtle changes in high functioning individuals (i.e., ceiling effects) who may score well within the normal range throughout the early stages of dementia; (3) unable to discern decline in individuals with poorer lifelong abilities; and (4) culturally insensitive thereby underestimating abilities of African Americans and other minority groups.^33,34 Informant assessments such as the AD8 are without ceiling effects and valid in assessing individuals regardless of age, gender, language, race or educational level.^2,5,9–11 The National Guideline Clearinghouse³⁵ recommends the combined use of an informant interview with a performance measurement to detect dementia and we have previously demonstrated that combining the AD8 with brief tests improve detection of cognitive impairment.¹¹ It is also important to consider whether patients are willing to be screened for dementia. Studies have reported a willingness of patients to be screened if there are perceived benefits to being screened and if the clinician can offer treatments for problems when detected.^36,37

The goal of any screening test is to separate people with a high probability of having the disease from those with a low probability. An effective screening test for dementia should be socially acceptable, safe, inexpensive, reliable, sensitive, specific, and brief. Gold standard informant interviews such as the Clinical Dementia Rating are not practical in a community setting – hence the need to develop and validate brief informant interviews – both the AD8 and IQCODE fulfill these criteria. The AD8 and IQCODE are highly correlated with performance on brief objective cognitive measures in common use. Consistent with previous studies, both the AD8 and IQCODE differentiated cognitively normal from individuals with dementia.^{5,9–11,15–17} The AD8, however, was better than IQCODE in detecting MCI which is consistent with a recent report that also found the IQCODE was not useful in detecting MCI.³⁸

There are a number of possible reasons for the differences. Although both the AD8 and IQCODE use informants, the domains assessed differ slightly. While the IQCODE covers two aspects of memory (acquisition of new information and retrieval of existing knowledge) and two aspects of intelligence (verbal and performance), the AD8 contains items that relate to memory, problem-solving abilities, orientation, and daily activities. Another explanation for why the AD8 may be better than IQCODE in detecting dementia at an earlier stage (i.e., MCI) is probably rooted in the history of their development. AD8 was originally developed from a battery of 56 questions.⁵ Questions that were very good at discriminating cognitively normal adults from moderate dementia but did not provide early discrimination were excluded. Other questions were so often endorsed by cognitively normal adults (forgetting day and date for example) that they provided little discrimination. The statistical models were modified to provide discriminations between CDR 0 and CDR 0.5 (that encompass both very mild dementia and MCI) because this is the dilemma posing most clinicians. This appears not to have been done (at least not consciously) for IQCODE. Thus, AD8 may be better at early disease detection precisely because it was fundamentally designed to detect early disease. Indeed a recent study demonstrated that the degree of functional impairment as measured by the CDR (which is the basis for development of AD8, but not IQCODE), predicted which MCI patients converted to clinical dementia.³⁹ Finally the AD8 is highly correlated with biomarkers of AD including amyloid imaging and cerebrospinal fluid tau and amyloid β-42.¹⁹ These biomarkers are now considered hallmarks for early diagnosis of Alzheimer disease and MCI and have been incorporated into revised criteria.^18,40

Our study has limitations. The sample size of the MCI cohort was small. With a larger sample size, the confidence intervals may narrow demonstrating statistical differences between the AD8 and IQCODE. Most, but not all patients underwent commonly used “Gold Standards” in research centers – the CDR¹⁴ for informant interviews or detailed neuropsychological tests. However the goal of this study was to demonstrate the utility of these brief informant interviews in the setting of a community practice where Gold Standard evaluations are generally not feasible. Our study was initiated before the publication of the revised criteria for MCI¹⁸ and Alzheimer’s disease.⁴⁰ However, the clinical features are not sufficiently different between the revised criteria and the previous iterations to affect clinical diagnosis. We did not collect biomarkers for this study but have previously demonstrated validation of the AD8 against clinical, neuropsychological and biomarker assessments.^9,19 Exclusion criteria for this study included medical conditions such as sleep apnea, schizophrenia, alcohol or drug use that could cause cognitive impairment but are not considered dementias. Neither the AD8 nor the IQCODE were designed to explore cognitive impairment in non-degenerative medical conditions nor are there currently any data to suggest that they would be able to do so. However the stated goals of this study were to determine if brief informant screening tools would improve detection of dementia and MCI in older adults coming to a community-based practice.

Alzheimer’s disease, MCI and related disorders will become a public health crisis and a severe burden on Medicare in the next two decades unless actions are taken to (1) develop disease modifying medications, (2) provide clinicians with valid and reliable measures to detect disease at the earliest possible stage, and (3) reimburse clinicians for their time to evaluate patients.² Dementia screening requires a consideration of the population-at-risk and the sensitivity and specificity of the instruments used.⁴¹ A large number of false positive individuals might expend limited health care dollars; a large number of individuals receiving false negatives would be denied treatment and miss opportunities to participate in clinical research. Thus, a staged dementia screening approach would make the most sense clinically and economically. The value of the AD8 is that it is not only brief but also corresponds to more detailed evaluations, neuropsychological testing and AD biomarkers,¹⁹ an argument that based on the currently available data, may not apply to the IQCODE. In this report, we demonstrated the AD8 may be superior to the IQCODE in detecting cognitive impairment at an earlier stage (i.e. MCI). In the environment of healthcare reform, it will be important for clinicians to use brief, sensitive and reliable methods to detect cognitive impairment in their patients. If simple screening for early cognitive impairment in the busy office setting is the goal the AD8 plus a brief performance measure such as the Mini-Cog^22,23 could be recommended, particularly because they meet the basic requirements of the Personalized Prevention Plan for Medicare beneficiaries.⁴²