Validation and diagnostic accuracy of the Alzheimer's questionnaire

Abstract

Background: accurately identifying individuals with cognitive impairment is difficult. Given the time constraints that many clinicians face, assessment of cognitive status is often not undertaken. The intent of this study is to determine the diagnostic accuracy of the Alzheimer's questionnaire (AQ) in identifying individuals with mild cognitive impairment (MCI) and AD.

Methods: utilising a case–control design, 300 [100 AD, 100 MCI, 100 cognitively normal (CN)] older adults between the ages of 53 and 93 from a neurology practice and a brain donation programme had the AQ administered to an informant. Diagnostic accuracy was assessed through receiver-operating characteristic analysis, which yielded sensitivity, specificity and area under the curve (AUC).

Results: the AQ demonstrated high sensitivity and specificity for detecting MCI [89.00 (81.20–94.40)]; [91.00 (83.60–65.80)] and AD [99.00 (94.60–100.00)]; [96.00 (90.10–98.90)]. AUC values also indicated high diagnostic accuracy for both MCI [0.95 (0.91–0.97)] and AD [0.99 (0.96–1.00)]. Internal consistency of the AQ was also high (Cronbach's alpha = 0.89).

Conclusion: the AQ is a valid informant-based instrument for identifying cognitive impairment, which could be easily implemented in a clinician's practice. It has high sensitivity and specificity in detecting both MCI and AD and allows clinicians to quickly and accurately assess individuals with reported cognitive problems.

Introduction

Given the expected increase in Alzheimer's disease (AD) prevalence in the USA [1] many clinicians will be faced with the prospect of evaluating many individuals for possible cognitive impairment. This problem may be further compounded by the possibility that screening for cognitive impairment may become mandatory under proposed healthcare reform [2]. Often, the first clinician a patient may see is a primary care physician who often has a limited amount of time to assess the individual. In addition to time constraints, many physicians do not screen for cognitive problems unless they receive complaints from patients or patients' families [3–5]. As a result dementia is not recognised by physicians until it is moderately advanced [6, 7]. Providers also cite a lack of confidence in diagnosing AD as a reason that nearly half of AD patients remain undiagnosed [3, 7, 8].

This necessitates the use of a brief and accurate screening instrument in order to determine, which patients require further assessment. The most common tool used to screen for dementia is the Mini-Mental Status Examination (MMSE) [9]; however, its scores can be biased by education level which can lead to false positive indications of impairment for individuals with low educational attainment and false negative indications of no impairment for highly educated individuals [10]. Informant-based questionnaires, such as the AD8 [11], IQCODE [12] and the DQ [13], have been developed in order to quickly and accurately identify clinical AD and have demonstrated good sensitivity and specificity (please see Supplementary data available in Age and Ageing online, Table S3). Although the Clinical Dementia Rating (CDR) [14] is widely used in clinical research settings, its utility in clinical practice is questionable given the length of time necessary for administration.

The Alzheimer's questionnaire (AQ) was designed to be a brief and easily administered assessment for use with collateral sources. A recent pilot study of the AQ demonstrated high sensitivity and specificity for detecting mild cognitive impairment (MCI) and clinical AD [15]. The intent of the current study is to validate the AQ as an accurate informant-based instrument in detecting both MCI and clinical AD.

Methods

Study sample

Three-hundred individuals were included in this study (100 CN, 100 MCI, 100 AD). The AD and MCI cases were drawn from the practices of three physicians and were between the ages of 56 and 93. The cognitively normal (CN) cases were between the ages of 53 and 93 and were recruited from a brain and body donation programme [16] in which the AQ was administered as part of their annual assessment. An exemption was granted for this study by the institutional review board as it fell under the categorisation of research involving the use of educational tests (cognitive, diagnostic, aptitude, achievement), survey procedures, interview procedures or observation of public behaviour.

The AD cases met NINCDS-ADRDA [17] criteria for a clinical diagnosis of probable or possible Alzheimer's disease. MCI cases were diagnosed as such based on Petersen criteria [18]. These criteria require the presence of subjective memory complaints and objective memory test performance that falls 1.5 standard deviations below age- and education-corrected mean values. Both single and multiple domain amnestic MCI cases were included. The CN cases were defined as having a global CDR score of 0 and were not impaired in any cognitive domain measured by neuropsychological testing. Individuals with MMSE scores below 20 were excluded so that the data better reflected a population seen in a primary care setting for cognitive complaints.

Consensus diagnosis with a neurologist, geriatric psychiatrist and neuropsychologist was used to determine the clinical status of CN individuals. Consensus diagnoses were made based on neuropsychological testing, neurological and physical exam and interviews with an informant, which assessed global cognitive status, functional status and mood and behavourial status. Clinician's diagnosis consisting of medical history, social history, neuroimaging, clinical laboratory results and neuropsychological testing was used for MCI and AD individuals. Individuals with any type of brain-related neurological or psychiatric illness were excluded.

The Alzheimer's questionnaire

The AQ [15] is a 21-item, informant-based dementia assessment. AQ items are divided into five domains including Memory, Orientation, Functional Ability, Visuospatial and Language. Items are posed in a yes/no format with the sum of points for ‘yes’ items equaling the total score that ranges from 0 to 27 with higher scores corresponding to greater impairment. Six items known to be predictive of a clinical AD diagnosis are weighted more heavily in the total score by being worth two points rather than one (please see Supplementary data available in Age and Ageing online, Appendix 1). The AQ was administered by a neurologist, geriatric psychiatrist and also by psychometrists trained by the neurologist and geriatric psychiatrist.

Items for the AQ are based on those from other informant-based assessments [11–14]. and were selected by a group of clinicians with extensive experience in dementia assessment. The items were selected based on their face validity to assess each of the AQ domains. Six items were selected to be weighted in the AQ total score as it was agreed that these items would clearly differentiate an impaired individual from a CN individual.

Statistical analysis

One-way analysis of variance (ANOVA) was used to discern group differences on age, education, MMSE score and AQ total score. Receiver-operating characteristic (ROC) analysis was used to determine sensitivity, specificity, area under the curve (AUC), likelihood ratios and cut-off scores for MCI and AD. Correlations between the mean domain scores were derived in order to assess internal consistency along with Cronbach's alpha. Analysis of covariance (ANCOVA) was used to compare group differences on the AQ total score while using age, education and gender as covariates in order to account for their effects. Bonferroni adjustment was used to correct for multiple comparisons.

Results

Demographic characteristics are displayed in Table 1. One-way ANOVA yielded statistically significant effects for age, education, MMSE score and AQ total score between the clinical groups. An additional one-way ANOVA found a statistically significant difference for gender on the AQ total score. ANCOVA which adjusted for age, education and gender was then used to analyse clinical group differences on the AQ total score [F = 327.68, (df = 2, 294), P < 0.001].

Table 1.

Demographic characteristics of study sample with mean MMSE and AQ scores

	CN	MCI	AD	Total	P-value
n	100	100	100	300	—
Age	77.98 (7.11)	74.82 (7.58)	78.17 (7.23)	76.99 (7.45)	0.002
Education	15.46 (2.89)	14.53 (2.50)	14.50 (2.48)	14.83 (2.63)	0.01
Gender (M/F)	38/62	60/40	59/41	157/143	0.002
MMSE	28.62 (1.44)	26.85 (2.50)	24.15 (2.51)	26.54 (2.76)	<0.001
AQ score	2.44 (2.54)	11.23 (4.80)	17.74 (4.78)	10.47 (7.53)	<0.001

CN, cognitively normal; MCI, mild cognitive impairment; AD, Alzheimer's disease, Mean (SD); MMSE normal range (26–30), AQ normal range (0–4).

Sensitivity, specificity, AUC, likelihood ratios (positive and negative) and cut-off scores for the AQ total score are displayed in Table 2. Two ROC analyses were carried out in order to derive AUC values. The first analysis used MCI as the outcome and CN as the reference while the other used AD as the outcome and CN as the reference. An additional ROC analysis was run with AD as the outcome and MCI as the reference in order to determine cut-off scores across a continuum. These analyses yielded high sensitivity and specificity for both MCI and AD.

Table 2.

Diagnostic accuracy of the AQ in MCI and AD

	Sensitivity (95% CI)	Specificity (95% CI)	AUC (95% CI)	LR+ (95% CI)	LR− (95% CI)	Cut-off score
MCI	89.00 (81.20–94.40)	91.00 (83.60–95.80)	0.95 (0.91–0.97)	9.89 (9.00–10.80)	0.12 (0.05–0.30)	5 ≤ 14
AD	99.00 (94.60–100.00)	96.00 (90.10–98.90)	0.99 (0.96–1.00)	24.75 (23.70–25.90)	0.01 (0.001–0.09)	15≤a

MCI, mild cognitive impairment; AD, Alzheimer's disease.

^aCut-off score derived through ROC analysis of MCI and AD cases with AD as outcome and MCI as the reference group; all other values for AD derived using CN as the reference group.

Internal consistency was high (Cronbach's α = 0.89). Correlations among the domain scores were moderate ranging from r = 0.45 to r = 0.69 (please see Supplementary data available in Age and Ageing online, Table S4).

Discussion

The results of this study show that the AQ is a valid measure of cognitive status and accurately identifies individuals with AD and MCI. In addition, the AQ requires approximately 3 min to administer and is easily interpreted. The rationale for weighting certain items on the AQ is that they reflect the presence of cognitive symptoms which are highly predictive of the clinical AD diagnosis [19]. Given that subjective memory complaints are common among older adults [20] using weighted items may assist in more accurately identifying individuals who are impaired. The AQ is not intended to replace a full diagnostic work-up that is done when assessing cognitive problems. It is intended to be a screening instrument used to determine which individuals require further evaluation.

The data for this study came from patients who were seen by dementia specialists so these results may not represent the general geriatric population. Another problem is that the AQ requires the use of an informant. In many cases, individuals may see a clinician by themselves or may not have a reliable informant. Additionally, the study sample was ethnically homogenous as the majority of participants were Caucasian. One other problem is that the clinical groups were very specific and did not include other diagnostic groups, such as vascular or frontotemporal dementia. In addition, MCI is a heterogeneous condition that can occur from multiple aetiologies and does not necessarily progress to clinical AD. Therefore, screening for this clinical entity can be problematic if other medical and social information is utilised. However, given recent interest in MCI as treatable not entity [21], instruments that identify individuals early in the disease process may help lead to better outcomes.

Overall, the AQ may be a useful tool to clinicians who require the use of a brief and accurate cognitive assessment. As mandates for cognitive screening among older adults are implemented [2], the AQ would fill the need for a brief and simple cognitive screening instrument.

Conflicts of interest

None declared.

Funding

This study was funded by the Arizona Alzheimer's Research Consortium, Banner Sun Health Research Institute (NIH P30 AG 019610, ADHS AGR 2007-37) and Banner Alzheimer's Institute. The Consortium only provided financial support for the project.

Supplementary data

Supplementary data mentioned in the text is available to subscribers in Age and Ageing online.