Telephone Assessment of Cognitive Function in the Late Onset Alzheimer’s Disease Family Study

Abstract

Context

Administration of cognitive test batteries by telephone has been shown to be a valid and cost-effective means of assessing cognition, but it remains relatively uncommon in epidemiological research.

Objective

To develop composite cognitive measures and assess how much of the variability in their scores is associated with mode of test administration (i.e., in person or by telephone).

Design

Cross-sectional cohort study

Setting

Late Onset of Alzheimer’s Disease Family Study conducted at 18 centers across the United States.

Participants

A total of 1,584 persons, 368 with dementia, from 646 families.

Main Outcome Measures

Scores on composite measures of memory and cognitive function derived from a battery of 7 performance tests administered in person (69%) or by telephone (31%) by examiners who underwent a structured performance-based training program with annual recertification.

Results

Based in part on the results of a factor analysis of the 7 tests, we developed summary measures of working memory, declarative memory, episodic memory, semantic memory, and global cognition. In linear regression analyses, mode of test administration accounted for less than 2% of the variance in the measures. In mixed-effects models, variability in cognitive scores due to center was small relative to variability due to differences between individuals and families.

Conclusions

In epidemiologic research on aging and AD, assessment of cognition by telephone has little effect on performance and provides operational flexibility and a means of reducing costs and missing data.

Alzheimer’s disease (AD) is a progressive illness that devastates the lives of millions of older people. Although some genetic and experiential risk factors have been identified, the pathophysiology of AD is not securely understood. The National Institute on Aging Genetics Initiative for Late Onset Alzheimer’s disease was designed to provide resources to help identify additional genes contributing to late onset AD. One part of that initiative is the Late Onset Alzheimer’s Disease Family Study which has been recruiting and clinically characterizing persons across the United States from families with multiple affected members and unrelated controls without dementia. Collecting uniform cognitive data is a substantial challenge in a study of this nature given multiple examiners from multiple centers. Further, the dispersion of family members across the United States presents logistic challenges that are most economically addressed by testing many affected and unaffected persons by telephone.

In this article, we evaluate the extent to which differences between modes of test administration and between centers affect cognitive performance. After undergoing a structured performance-based program of training and certification, research assistants administered a battery of 7 cognitive tests to >1500 older persons, nearly one third of whom were tested by telephone. We first developed summary measures of different forms of memory and global cognition. We then performed a series of linear and mixed-effects regression models to determine how much of the variability in performance between persons was attributable to test administration mode or to center.

METHODS

Participants

Subjects were recruited through 18 participating AD centers (see acknowledgments). As previously described [1], many index cases were recruited through one of the federally funded Alzheimer’s disease research centers. Media and other recruitment efforts directed other interested families to a toll free number at the National Cell Repository for Alzheimer’s Disease (http://ncrad.iu.edu) which assigned them to the nearest participating center. Each center also recruited unrelated control subjects up to half of whom could be spouses of participating family members. An eligible family was required to have at least 3 biologically related members willing to provide clinical data and a biological sample for DNA extraction. Each family included a proband diagnosed with AD after age 60 and a full sibling of the proband diagnosed with AD after age 60. A third family member could be a full or half sibling, parent, offspring, aunt, uncle, niece, nephew or first cousin of the proband and had to have AD (diagnosed after age 50) or mild cognitive impairment or be unaffected after age 60 as determined by cognitive testing and clinical evaluation. The determination of the eligibility of each family group was made by the coordinating center at Columbia University. Once criteria were met, other family members were eligible to participant. Informed consent was obtained from the participant or from a proxy if the participant lacked the capacity to consent. The study was approved by the Institutional Review Board of each participating center.

At the time of these analyses, 1,584 people had agreed to participate and completed the initial evaluation, including the cognitive testing. They had a mean age of 71.1 (SD = 11.2; range: 28–99) and had completed a mean of 14.2 years of schooling (SD = 3.0; range: 0–29); 61.4% were women. They represent 646 families, with 360 contributing a single family member, 197 with 2–4 participating members, 51 with 5–7, and 38 with 8 or more.

Clinical Evaluation

Data on demographic variables, diagnosis of dementia and AD, and medical history were obtained from each participant or an informant. Clinical classification of dementia and AD followed the guidelines of the joint working group of the National Institute of Neurological and Communicative Disorders and Stroke/Alzheimer’s Disease and Related Disorders Association. These require a history of cognitive decline and evidence of impairment in at least 2 cognitive domains, one of which must be memory to meet AD criteria [2]. In a subset of persons who could not be directly examined, clinical classification was based on a detailed review of medical records.

Genotyping of APOE polymorphisms (based on SNPs rs7412 and rs429358) was performed at PreventionGenetics (www.preventiongenetics.com). Genotyping was carried out in array tape using allele-specific PCR with universal molecular beacons. DNA sequencing of positive control DNA samples was completed to assure correct assignment of alleles.

Assessment of Cognitive Function

Cognition was measured with a battery of 7 brief tests [3]. Working memory was assessed with Digit Span Forward [4], Digit Span Backward [4], and Digit Ordering [5]. Two measures of episodic memory were included: immediate and delayed recall of story A from the Wechsler Memory Scale-Revised [4]. Semantic memory was assessed by asking persons to name members of two semantic categories (Animals, Vegetables) in separate 1-min trials [3, 5, 6]. In previous research, these tests have been shown to have adequate reliability [4, 7, 8], change in performance on them has been associated with a genetic risk factor for AD [3], and level of performance proximate to death has been associated with level of AD pathology on postmortem examination [3]. Administration of the test battery requires 10–15 min and can be done in person or by telephone.

After data collection began, we changed the category for the second fluency task from “fruits and vegetables” to “vegetables” to match the procedure used in Uniform Data Set by the National Institute on Aging Alzheimer’s Disease Centers [9]. In preliminary analyses, raw scores for “vegetables” were slightly lower than scores for “fruits and vegetables”, but each had comparable associations with animal fluency score, suggesting that they were measuring the same underlying ability. In computing cognitive scores, therefore, we treated scores for “vegetables” and “fruits and vegetables” as equivalent after converting them to a common scale.

Training and Certification

The test battery was administered by multiple research assistants at the 18 participating centers. To maximize uniformity of test administration and scoring, each research assistant underwent a structured 4-step program of training and certification coordinated by Rush University Medical Center personnel. The first step was to carefully read the cognitive assessment manual. Next, research assistants had to complete a minimum of two practice administrations of the battery at their site under the supervision of an individual with testing experience if not previous certification. Third, research assistants were required to score a set of 8 samples of story A with at least 95% accuracy. The Rush coordinators provided story samples and checked scoring accuracy. The procedure was repeated with new story samples until 95% accuracy was achieved. The final step involved giving the battery twice in succession by telephone to a coordinator at Rush without major errors of administration, data entry, or scoring, repeating the process (and providing feedback when needed) until the criterion was reached. Prescripted test responses were used to ensure exposure to a range of testing situations. We recertified test administrators at 12 month intervals, again requiring 95% accuracy in story scoring and two successive error free administrations of the test battery by telephone to the Rush coordinator.

Data Analysis

To minimize random variability, we developed composite measures of cognition in a 3-step process [5,10,11]. We began by hypothesizing two ways in which the tests could be grouped into functional domains. Next, we empirically grouped the tests by performing a factor analysis with varimax rotation and clustering tests with rotated loadings of .50 or higher on the same factor. Finally, we used Rand’s statistic [12] to test the concordance of the hypothesized groupings with the empirically-based groupings obtained in this cohort and in an independent group of subjects from the Rush Memory and Aging Project [13]. We formed composite measures of the hypothesized domains by converting raw scores on each component test to z scores, using the mean and SD of all participants, and then averaging the z scores to yield the composite measure. We also formed a composite measure of global cognition based on all 7 tests. To assess the effects of apolipoprotein E genotype on performance, we formed a no ε 4 reference group (i.e., ε 2/2, ε 2/3, ε 3/3) and contrasted it with one (i.e., ε 2/4, ε 3/4) and two (i.e., ε 4/4) ε4 allele groups, with separate analyses for those with and without dementia.

To assess the impact of mode of test administration on performance, we conducted a series of linear regression analyses of each composite measure, with separate models in each diagnostic subgroup. A first analysis included terms for age, sex, and education. The analysis was then repeated with an indicator for whether testing was done by telephone. We controlled for age, sex, and education in these and subsequent analyses because of their associations with cognitive performance.

To examine other sources of variability in performance, we constructed a series of mixed-effects regression models [14]. Each model had fixed effects for age, sex, education, and mode of test administration and random effects for center, family membership, and subjects within center.

Models were graphically and analytically validated. Programming was done in SAS [15].

RESULTS

Development of Composite Cognitive Measures

The test battery was administered to 1,584 individuals, 368 with dementia and 1,216 without it. The dementia subgroup was older (79.2 vs 68.6, t[902]=21.3, p<.001) and less educated (13.2 vs 14.5, t[1,504]=7.0, p<.001) than the no dementia subgroup and the distribution of sex was similar (60.6% vs 61.7%, χ² [1]=0.1, p=.709). Table 1 provides psychometric information on the test scores within each diagnostic subgroup. In those without dementia, the distribution of scores on each test was approximately normal. The level of test performance in the dementia group was lower than in the group without dementia, as expected. The distributions of scores in the dementia group were also approximately normal except for positively skewed memory performances which reflect the ubiquity of memory impairment in dementia.

Table 1.

Psychometric data on the cognitive tests and composite measures

Cognitive measure	No Dementia (n=1,216)			Dementia (n=368)
	Mean	SD	Skewness	Mean	SD	Skewness
Digit Span Forward	8.68	2.09	−0.24	6.32	2.87	−0.51
Digit Span Backward	6.48	2.25	0.46	3.61	2.36	0.28
Digit Ordering	7.73	1.80	0.43	3.53	2.85	0.19
Logical Memory Ia	11.86	4.17	−0.21	2.49	3.18	1.54
Logical Memory IIa	10.74	4.30	−0.18	1.30	2.54	2.48
Fluency (animals)	19.00	5.57	0.20	7.89	5.21	0.41
Fluency (vegetables)	16.87	5.33	0.26	6.36	4.67	0.45
Global cognition	0.33	0.51	−0.09	−1.10	0.62	0.00
Working memory	0.27	0.64	0.23	−0.92	0.90	−0.21
Declarative memory	0.37	0.60	−0.09	−1.24	0.52	0.81
Episodic memory	0.38	0.73	−0.22	−1.29	0.48	2.06
Semantic memory	0.35	0.66	0.16	−1.19	0.66	0.28

We hypothesized two ways in which the individual tests could be grouped into functional domains (Groupings 1 and 2 in Table 2). In one grouping, we specified two domains: working memory and declarative memory. In the second grouping we specified three domains with declarative memory subdivided into episodic memory and semantic memory.

Table 2.

Factor analyses of the cognitive tests

Cognitive test	Hypothesized cognitive domain a		Factor loading b		Factor loading c
	Grouping 1	Grouping 2	I	II	I	II
Logical Memory Ia	Declarative memory	Episodic memory	.88	.13	.87	.10
Logical Memory IIa	Declarative memory	Episodic memory	.89	.11	.89	.10
Fluency (animals)	Declarative memory	Semantic memory	.61	.24	.64	.29
Fluency (vegetables)	Declarative memory	Semantic memory	.65	.13	.66	.25
Digit Span Forward	Working memory	Working memory	.06	.82	.06	.82
Digit Span Backward	Working memory	Working memory	.15	.83	.19	.80
Digit Ordering	Working memory	Working memory	.30	.68	.34	.67

^aWe hypothesized that the individual cognitive tests could be grouped into two (Grouping 1) or three (Grouping 2) functional domains. Loadings are from factor analyses with varimax rotation of individuals without dementia in

^bthe Late Onset Alzheimer’s Disease study or

^cthe Rush Memory and Aging Project. Loadings >.50 are in boldface.

We next empirically grouped the tests in a factor analysis with varimax rotation. Because dementia severity can influence correlations among cognitive tests, we restricted the factor analysis to those without dementia. As shown by the first set of factor loadings in Table 2, this analysis identified two factors. We used Rand’s statistic, rescaled to range from -1 (complete disagreement) to 1 (complete agreement), to assess the concordance of the empirical grouping with the hypothesized groupings. The factor analytic results showed good agreement with the hypothesized two-domain (Rand statistic = 1.00, p = 0.029) and three-domain (Rand statistic = 0.62, p = 0.030) groupings. To test the generalizability of these results, we conducted an identical factor analysis of these same seven tests in a different cohort: 1,099 older persons without dementia from the Rush Memory and Aging Project [13]. As shown in the two right-hand columns of Table 2, these factor loadings were quite similar to those obtained in the Late Onset Alzheimer’s Disease Family Study cohort (Rand statistic = 1.00, p = 0.029).

Given the empirical support for the hypothesized groupings, we formed composite measures of each hypothetical cognitive domain. We also created a composite measure of global cognition based on all 7 tests in view of the positive correlations among all measures. To construct each composite measure, we converted raw scores on each test to z scores and then averaged the z scores of component tests to yield the composite score, as previously done for other composite cognitive measures [5,10,11]. The composite measure was treated as missing if more than half of the component test scores were missing. As shown in Table 1, these composite cognitive measures had relatively normal distributions in each diagnostic subgroup except for the skewed episodic memory distribution in those with dementia.

To assess the validity of the composite measures, we examined their association with apolipoprotein E (APOE) genotype in a series of linear regression models that controlled for age, sex, and education. In both the no dementia and dementia subgroups, inheritance of one or two ε4 alleles was associated with lower scores on all cognitive measures (data not shown).

Mode of Administration Effects

To enhance participation in the Late Onset Alzheimer’s Disease Family Study and to reduce its operational costs, we selected cognitive tests that could be administered either in person or by telephone. To date, 495 participants (31%) have been tested by telephone. They were younger than participants tested face-to-face (69.2 versus 71.9, t [1,050] = 4.7, p <.001), more educated (14.4 versus 14.1, t [1,504] = 2.0, p = .048), and less apt to have dementia (13% versus 28%, χ² [1] =41.2, p<.001). To assess the association of mode of test administration with cognitive performance, we constructed a series of linear regression models. Each model had an indicator for telephone versus in person testing and terms to control for the potentially confounding effects of age, sex, and education, with separate analyses for those with dementia and those without it. As shown in Table 3, administering the tests by telephone was associated with a slightly higher global cognitive score in those without dementia, but the effect accounted for less than 1% of the variability in global cognition. Among the memory systems measures, only working memory showed this effect with no association between administration mode and performance in the remaining measures. Among those with dementia, telephone administration was not associated with cognitive test performance. Overall, these data suggest that mode of test administration is not strongly related to performance on the composite cognitive measures.

Table 3.

The relation of mode of test administration to cognitive performance ^*

Cognitive measure	No Dementia				Dementia
	Estimate	SE	p	R2 Inc.	Estimate	SE	p	R2 Inc
Global cognition	0.096	0.027	<.001	.008	−0.087	0.083	.297	.002
Working memory	0.160	0.037	<.001	.014	−0.156	0.122	.204	.002
Declarative memory	0.048	0.031	.123	.001	−0.040	0.068	.554	<.001
Episodic memory	0.070	0.041	.086	.001	−0.013	0.064	.843	<.001
Semantic memory	0.026	0.036	.459	<.001	−0.063	0.089	.481	<.001

^*Estimated from linear regression models that controlled for age, sex, and education. Results show change in the cognitive measure associated with testing being done by telephone rather than in person. SE indicates standard error. R² inc. is the increment in adjusted R² due to mode of test administration.

Other Sources of Variability

The Late Onset Alzheimer’s Disease Family Study includes 18 centers contributing cognitive data on 3 to 377 individuals (median=65) representing 646 families, with 1 to 25 participating members. To examine center and familial effects, we constructed a series of mixed-effects regression models, with separate analyses for each composite cognitive outcome in each diagnostic subgroup. Each model had terms to account for the fixed effects of age, sex, education, and mode of test administration. We also included random effects for center, family, and subjects within center. As shown in Table 4, the amount of variability in the composite cognitive measures attributable to center was low in both an absolute sense (i.e., 0–2% in all instances) and in comparison to the variability attributable to familial aggregation and individual differences between persons within centers. The amount of variability attributable to family was somewhat larger ranging from 3–11%, but still substantially less than person-specific variability which ranged from 15–69%.

Table 4.

Proportion of variability in cognitive measures due to differences between centers, families, and persons^*

Cognitive measure	No Dementia			Dementia			All Participants
	Center Variance	Family Variance	Person Variance	Center Variance	Family Variance	Person Variance	Center Variance	Family Variance	Person Variance
Global cognition	.00	.03	.15	.00	.05	.30	.02	.05	.38
Working memory	.01	.04	.31	.01	.07	.69	.02	.05	.53
Declarative memory	.01	.04	.21	.00	.06	.16	.02	.06	.45
Episodic memory	.01	.07	.36	.01	.04	.15	.02	.08	.60
Semantic memory	.02	.04	.27	.01	.11	.27	.02	.06	.50

^*Estimated from mixed-effects regression models that controlled for age, sex, education, and mode of test administration.

COMMENT

As part of the Late Onset Alzheimer’s Disease Family Study, examiners from multiple centers administered a battery of 7 cognitive tests either in person or by telephone to >1500 older persons with and without dementia, >75% of whom represented persons evaluated as parts of families. Composite measures of global cognition and specific memory systems were derived from the individual tests and showed the expected associations with an external validity criterion. Relatively little of the variability in the composite measures was related to mode or site of test administration. Slightly more variability was related to family membership and most reflected person-specific factors. The results suggest that the battery provides a psychometrically sound, operationally flexible, and cost-effective means of assessing multiple memory systems in older persons.

A substantial body of research has examined cognitive testing of old people by telephone. Much of the research has focused on the level of agreement between telephone and in person testing. Studies with a repeated measurement design, including one using a cognitive test battery similar to the present one [16], have shown that testing the same individuals by telephone and face to face yields equivalent results [17–21]. This work has also shown that multiple domains of memory and cognition can be assessed [3, 16, 21, 22]; that persons with neurologic conditions, including mild cognitive impairment [23- 27], dementia [28, 29], and stroke [30], can be tested by telephone; and that conditions like diabetes that have been linked to cognitive impairment and decline based on in person testing also show these associations when testing is done by telephone [16, 31]. The present study represents an attempt to apply these findings in a multicenter epidemiologic study. The cross-sectional finding that mode of test administration contributes little to between person differences in cognitive performance regardless of domain tested or dementia status is consistent with prior work and extends it to a multicenter context. That neither mode of administration nor the effect of multiple testing sites contributed materially to the variance of cognition is likely due, in part, to the efforts expended in developing uniform test procedures and certification processes. However, it is also likely due, in part, to the large person-specific differences in cognitive performance among older persons with and without dementia.

Key operational aims of epidemiologic research on cognitive function are to minimize missing data and to maintain uniformity in test administration and scoring. The option of administering tests by telephone is likely to increase participation, particularly when subjects are geographically dispersed as in the present study. Use of tests amenable to telephone administration also has the advantage of allowing training and certification of examiners to be done by telephone. This increases efficiency, especially in a study with multiple examiners from multiple sites, by facilitating centralization of training and making it easier to recertify examiners at regular intervals. It also reduces travel costs and requires fewer trainers because the process of training, certification, and recertification can be spread out over time.

Because this study included family members, we were able to estimate the amount of variability due to family effects. Overall, only about 5% of the variability in cognitive performance was due to family effects in comparison to approximately 40% due to between person effects. The relative size of between person and familial effects appeared to vary across cognitive domains. For example, among persons with dementia, family effects accounted for more than 10% of the variability in semantic memory with between person effects accounting for less than 30%. By contrast, between person effects accounted for nearly ten times more variability in working memory than did family membership These data support the notion that genetic influences on cognition remain strong even during old age [32]. The use of families along with measures of different types of cognitive abilities provide an opportunity to identify these genetic variants.

In summary, the Late Onset Alzheimer’s Disease Family Study cognitive test battery provides psychometrically sound measures of global cognition and different forms of memory affected by advancing age and AD. The results provide further evidence of the utility of cognitive tests that are amenable to telephone administration [3].