Correlation Analysis of the In-Clinic and Telephone Batteries from the AREDS Cognitive Function Ancillary Study. AREDS Report No. 15

Abstract

Purpose

To determine whether the AREDS Telephone Battery can be substituted for the In-Clinic Cognitive Function Battery to assess cognitive function, so that participants could still provide follow-up information without having to come to the clinic.

Methods

Correlation analysis was performed on scores of the following in-clinic and telephone administrations: 1) Modified Mini-Mental State Examination (3MS), conducted in person vs. Telephone Interview Cognitive Status (TICS-M); 2) Wechsler Memory Scale-III (WMS-III), Logical Memory I and II; 3) Digits Backwards (a sub-test of the WMS-R); 4) Verbal Fluency; and 5) Letter Fluency (F,A,S).

Results

A total of 1,738 AREDS participants completed an In-Clinic Battery and a Telephone Battery within twelve months. Significant positive correlations were found for all tests, ranging from ρ = 0.89 between the 3MS and TICS-M scores (95% CI; 0.88–0.90), to ρ= 0.71 for Letter Fluency (95% CI; 0.68–0.74).

Conclusion

The linear relationships between the In-Clinic Battery and Telephone Battery scores support the hypothesis that the Telephone Battery is an appropriate substitute for participants who are unable to complete an in-clinic assessment of cognitive function.

INTRODUCTION

The Age-Related Eye Disease Study (AREDS) includes a clinical trial of the effect of high-dose zinc and anti-oxidants (beta-carotene and vitamins C and E) on age-related macular degeneration (AMD) progression and cataract. Although some observational studies of non-demented older adults have suggested that high antioxidant dietary intake or the use of antioxidant supplements is associated with better cognitive performance,^1–4 other studies have reported no protective effect.^5–8 To date, there are no published data from randomized trials assessing the use of high-dose antioxidant or zinc supplements on cognitive function in non-demented older adults.

AREDS is a multicenter prospective cohort study of the clinical course, prognosis and risk factors for AMD and cataract. The study also included a randomized, placebo-controlled clinical trial of treatment with high-dose antioxidant vitamins and/or zinc on the incidence of advanced AMD and vision loss. A 25% reduction in the risk of developing advanced AMD was observed with the combination treatment.⁹

The AREDS Cognitive Function Study was implemented in June 2000. The primary purpose of this ancillary study is to measure the potential beneficial or deleterious effects of the AREDS study medications on cognitive function and to investigate potential associations between cognitive function and the development or progression of either AMD or cataract.

As AREDS participants age, their mobility may be diminished, thus limiting or even eliminating their ability to return for follow-up clinic visits. The AREDS Telephone Battery was included in this ancillary study to test whether a telephone interview could replace the in-clinic assessment of cognitive status for participants who are unable to return to the clinic. The design of this study included a baseline in-clinic administration and a subsequent baseline telephone administration. At least 6 months elapsed between the two administrations.

Having a baseline telephone evaluation serves two purposes: it allows for a comparison of results between two methods of administration, and it provides longitudinal data on participants who move away from a clinical center or are otherwise unable to return for in-clinic follow-up. The purpose of this analysis is to validate the use of the AREDS Telephone Battery in place of the In-Clinic Battery for AREDS participants unable to attend in-clinic visits for the duration of the AREDS Cognitive Function Ancillary Study.

METHODS

Study Population

Details of the AREDS study design have been published elsewhere¹⁰ and are briefly described here. A total of 4,757 persons, 55–80 years of age at time of enrollment, were entered into the study at 11 clinical centers between 1992 and 1998. Persons were categorized into AMD categories, which were determined by the size and extent of drusen, the extent of retinal pigment abnormalities, and the presence of advanced AMD (determined from photograph grades at a reading center). In addition, visual acuity at baseline and at the time of administration of the cognitive function tests were recorded.

Cognitive Function Batteries

The AREDS In-Clinic Battery included the following neuropsychological tests: the Modified Mini-Mental State Examination (3MS),¹¹ Verbal Fluency: “Animal Category” and “Letter Fluency,”¹² Wechsler Memory Scale–Revised (Logical Memory I and Logical Memory II),¹³ Buschke Selective Reminding Test (8 Trial),¹⁴ and Digits Backwards.¹⁵ A measure of depression from the Center for Epidemiologic Studies’ Depression Scale (CES-D)¹⁶ was also included. The AREDS Telephone Battery comprised all instruments included in the In-Clinic Battery with the exception of the Modified Mini-Mental State Examination and the Buschke Selective Reminding Test. The Telephone Interview Cognitive Status–Modified (TICS-M)¹⁷ was used in place of the 3MS.

This report describes In-Clinic Battery data collected between July 2000 and February 2002, and Telephone Battery data captured between June 2001 and August 2002. Trained and certified interviewers administered the instruments during an AREDS visit in a quiet room. The In-Clinic Battery was administered in the following standardized order: the CES-D, WMS-RLogical Memory I, 3MS, Letter Fluency FAS, Buschke SRT, Animal Category Fluency, Digits Backwards, and WMS-R Logical Memory II. The Telephone Battery was administered in the following standardized order: CES-D, WMS-R Logical Memory I, TICS-M, Letter Fluency FAS, Animal Category Fluency, Digits Backwards, TICS-M Recall, and WMS-R Logical Memory II. The length of the In-Clinic Battery was approximately 25–30 minutes, while the length of the Telephone Battery was approximately 20–25 minutes. Informed consent was obtained from all AREDS participants who agreed to complete the battery of cognitive instruments.

Statistical Analysis

Correlation analysis using Pearson’s ρ was done for both batteries administered no more than 12 months apart. A correlation analysis was conducted on the “raw” scores and predicted scores computed from a regression analysis adjusted for age and depression score. All analyses were carried out using SAS version 8.0.

RESULTS

Of the 4,757 AREDS participants, 1,738 (36.5%) completed all related instruments of the AREDS in-clinic and telephone batteries within 12 months of each other. Of the 3,019 participants not included, 1,748 participants did not have cognitive testing (58%), 605 (20%) did not complete either the In-clinic (n = 26) or Telephone battery (n = 579); 647 (21%) completed both batteries more than 12 months apart; and 19 (<1%) completed both batteries less than 12 months apart but had one or more missing instruments. Of the 1,748 participants who did not have cognitive testing, 397 (23%) died prior to the implementation of the Cognitive Function Ancillary Study and 1,351 (77%) refused or otherwise did not participate in the ancillary study due to the following reasons: too much of a time commitment (n = 206); refused all contact and follow-up (n = 238); illness (n = 145); diagnosis of or suspected Alzheimer’s disease/dementia (n = 120); high anxiety/fear (n=113); transportation problems (n = 75); relocation (n = 61); stopped in-clinic visits (n = 47); unable to contact/locate (n = 38); language barrier (n = 10); hearing loss (n = 6); and general refusal/other/unknown (n = 292).

Participant characteristics for the 1,738 participants are presented in Table 1. The mean age at the time of the In-clinic Battery administration was 75 years (ranging from 61 to 87 years). Fifty-seven percent of the participants were female, 73% had more than a high school education, and 97% were white. The participants had a median length of time between the in-clinic and telephone administrations of 9.8 months, ranging from 4.7 to 12.0 months. Participants included in this report were younger on average (68 versus 69 years at randomization), more educated (73% versus 59% with more than a high school education), and had a higher percentage of “white race” participants (97% versus 95%) compared with the 3,019 AREDS participants not included. Participants included in this report did not differ from the remaining AREDS participants with respect to gender.

TABLE 1.

Participant Characteristics.

Characteristics	n = 1,738
Age, yrs [Mean (SD)]
In-clinic administration	74.9 (5.0)
Telephone administration	75.7 (5.0)
Time between administrations, months [Mean (SD)]	9.7(1.7)
Gender [N(%)]
Male	743 (43)
Female	995 (57)
Race [N(%)]
Caucasian	1,685 (97)
Other	53 (3)
Education [N(%)]
High school or less	476 (27)
> High school	1,262 (73)
CES-D Score [Mean (SD)]
In-clinic administration*	7.5 (6.8)
Telephone administration	9.8 (8.8)

^*One participant missing an in-clinic CES-D score.

The mean, median and standard deviation of in-clinic and telephone instrument scores are presented in Table 2. The approximate symmetry of the distributions of scores suggests that the Pearson’s correlation coefficient was appropriate to use for the estimation of correlation.

TABLE 2.

Distribution of In-Clinic and Telephone Battery Scores

Instrument	Mean	Median	Standard Deviation	Instrument Min-Max	AREDS Min-Max
3MS	93.6	95.0	5.7	0–100	51–100
TICS-M*	36.5	36.0	5.4	0–51	5–51
Logical Memory I	38.0	38.0	10.6	0–75	0–67
Logical Memory I*	42.6	43.0	11.2	0–75	0–72
Logical Memory II	22.2	22.0	8.3	0–50	0–45
Logical Memory II*	25.4	26.0	9.1	0–50	0–48
Letter Fluency	38.9	38.0	13.3	—	4–91
Letter Fluency*	37.8	37.0	14.0	—	2–99
Animal Category	17.6	18.0	4.9	—	1–44
Animal Category*	16.6	17.0	5.0	—	0–35
Digits Backward	6.4	6.0	1.9	0–12	0–12
Digits Backward*	7.1	7.0	2.4	0–12	0–12
Composite	0.2	0.4	4.2	—	−16.9–12.8
Composite*	0.2	0.4	4.2	—	−20.3–13.2

^*Indicates telephone interview using this instrument.

The mean scores for each cognitive function instrument decreased with age and with increased depressive symptoms as assessed by the CES-D, were higher for female and Caucasian participants, and were lower for participants with less education (data not shown). Because age and CES-D scores varied with the time of administration and are significantly associated with the instrument scores, adjusted scores were computed from a linear regression model. The mean, median and standard deviations of the adjusted in-clinic and telephone cognitive function instrument scores are presented in Table 3.

TABLE 3.

Distributions of Age- and CES-D Adjusted In-Clinic and Telephone Battery Scores

Instrument	Mean	Median	Standard Deviation
3MS	93.6	93.6	1.7
TICS-M*	36.5	36.5	1.6
Logical Memory I	38.0	38.0	2.1
Logical Memory I*	42.6	42.7	2.9
Logical Memory II	22.2	22.2	1.7
Logical Memory II*	25.4	25.5	2.5
Letter Fluency	38.9	39.0	2.2
Letter Fluency*	37.8	37.9	2.1
Animal Category	17.6	17.6	1.1
Animal Category*	16.6	16.7	1.3
Digits Backward	6.4	6.4	0.3
Digits Backward*	7.1	7.1	0.5
Composite	0.0	0.0	6.0
Composite*	0.0	0.0	5.9

^*Indicates telephone interview using this instrument.

Unadjusted Instrument Scores

The scatter plots for the unadjusted in-clinic and telephone instrument scores are shown in Figure 1. The estimated correlation coefficient and 95% confidence intervals are presented in Table 4. There was evidence of a positive but weak correlation between the 3MS and TICS-M scores (ρ = 0.44, 95% CI; 0.40–0.49). A lack of low to medium scores on both instruments may have resulted in a diminished correlation (Fig. 1a).

FIGURE 1.

Scatter plots for the in-clinic and telephone Instrument scores (a) Logical Memory I, (b) 3MS and TICS-M, (c) Logical Memory II, (d) Letter Fluency, (e) Animal Category Fluency, (f) Digits Backwards, and (g) Composite scores.

TABLE 4.

Results of the Analysis of Unadjusted Scores for In-Clinic vs. Telephone Instruments

In-clinic vs. telephone	Estimated correlation coefficient	95% Confidence Interval
3MS vs. TICS-M	ρ = 0.44	(0.40, 0.49)
Logical Memory I	ρ = 0.67	{0.64, 0.69)
Logical Memory II	ρ = 0.71	(0.68, 0.73)
Letter Fluency	ρ = 0.79	(0.77, 0.81)
Animal Category	ρ= 0.62	(0.58, 0.65)
Digits Backward	ρ = 0.35	(0.31, 0.40)
Composite	ρ = 0.77	(0.74, 0.79)

The scatter plots in Figures 1b and 1c portray a significant positive correlation between the in-clinic and telephone scores for the Logical Memory I and Logical Memory II instruments (Logical Memory I: ρ = 0.67, 95% CI; 0.64–0.69 and Logical Memory II: ρ = 0.71, 95% CI; 0.68–0.73).

There was also a significant linear association between the in-clinic and telephone scores from the Letter Fluency (ρ = 0.79, 95% CI; 0.77–0.81) and Animal Category Fluency (ρ = 0.62, 95% CI; 0.58–0.65) instruments (Figs. 1d and 1e).

The Digits Backwards instrument had a weak but significant linear relationship between in-clinic and telephone scores (ρ = 0.35, 95% CI; 0.31–0.40), as shown in the scatter plot in Figure 1f. The result indicates that there is a tendency for telephone scores to increase as in-clinic scores increase for the Digits Backwards instrument, suggesting that there is enough of a linear relationship to rely on telephone scores as an approximate score of the in-clinic scores.

The composite scores, calculated as the sum of the z-scores for each instrument within each battery, had a strong positive relationship as shown in Figure 1g and confirmed by a significant linear association between the in-clinic and telephone composite scores (ρ = 0.77, 95% CI; 0.74–0.79).

Adjusted Instrument Scores

Correlation coefficients between instrument scores adjusted for age and depression at the time of administration are presented in Table 5. A significant positive correlation was found between the adjusted 3MS and TICS-M scores (ρ = 0.89, 95% CI; 0.88–0.90). When age at administration and CES-D depression score are held constant, the TICS-M scores may be comparable to the 3MS scores.

TABLE 5.

Results of the Analysis of Adjusted Scores for In-Clinic vs. Telephone Instruments

In-clinic vs. telephone	Estimated correlation coefficient	95% Confidence Interval
3MS vs. TICS-M	ρ = 0.89	(0.88, 0.90)
Logical Memory I	ρ = 0.87	(0.86, 0.88)
Logical Memory II	ρ = 0.86	(0.84, 0.87)
Letter Fluency	ρ = 0.71	(0.68, 0.74)
Animal Category	ρ = 0.82	(0.81, 0.84)
Digits Backward	ρ = 0.79	{0.76, 0.81)
Composite	ρ = 0.83	(0.81, 0.84)

The significant positive correlation between scores for the Logical Memory I and Logical Memory II instruments increased after adjustment (Logical Memory I: ρ = 0.87, 95% CI; 0.86–0.88 and Logical Memory II: ρ = 0.86, 95% CI; 0.84–0.87).

Analysis of the adjusted scores confirmed the significant linear association between the in-clinic and telephone adjusted scores from the Letter Fluency (ρ = 0.71, 95% CI; 0.68–0.74) and Animal Category Fluency (ρ = 0.82, 95% CI; 0.81–0.84) instruments. This implies that the Letter Fluency and Animal Category Fluency instruments give consistent scores either through telephone administration or in-person administration after adjustment for age and depression at the time of administration.

The Digits Backwards instrument had a strong significant linear relationship between in-clinic and telephone adjusted scores (ρ = 0.79, 95% CI; 0.76–0.81), notably improved over the unadjusted analysis (ρ = 0.35).

The relationship between the composite scores remained strong (ρ = 0.83, 95% CI; 0.81–0.84).

DISCUSSION

Because many elderly patients become unable to return regularly to clinic sites in large, long-term studies of elderly participants such as AREDS, the need for a telephonic mental status assessment for participants unable to complete clinic visits is apparent. Our findings suggest that a telephonic assessment can serve as a useful surrogate for an in-clinic assessment in evaluating overall cognitive function status using a composite score. An instrument-to-instrument substitution of the telephone for the in-clinic scores would be acceptable for four of the six common instruments between the batteries (Logical Memory I and II, Letter Fluency, and Animal Category Fluency). The correlations are improved after adjustment for age and depression at the time of administration. More specifically, the correlations of the TICS-M versus 3MS and the in-clinic versus telephone scores of the Digits Backwards instrument were improved more dramatically than the other correlations of in-clinic versus telephone instruments, suggesting that age and depression may influence TICS-M and 3MS scores along with Digits Backwards scores more than the scores on other instruments. A reason for the influence of age and depression on the TICS-M and 3MS may be that these instruments are global measures of cognitive function and they are more sensitive to age and depression, which are known risk factors for cognitive impairment. Overall, the linear relationship between the scores of the In-Clinic Battery and those of the Telephone Battery support the hypothesis that the Telephone Battery is an appropriate substitute for participants who are unable to complete an in-clinic assessment of cognitive function.

Much evidence supports the reliability and validity of cognitive function data captured via the telephone. In a comparison of a telephone version to the in-person version of the Mini-Mental State Examination (MMSE), Roccaforte et al.¹⁸ obtained a correlation of 0.85. A Swedish study examining multiple aspects of cognitive function among 230 subjects found correlations from 0.51 for short-term memory to 0.80 for higher-order intelligence when comparing a telephone battery to an in-person interview.¹⁹ A UK cross-sectional study of 120 older adults found the TICS-M to be correlated with the MMSE (r = 0.57).²⁰

Potential study limitations must be noted. The optimal design would be to administer the batteries within a shorter time frame to get a true measure of the test-retest reliability of the two instruments. A one-year lag between administrations increases the chance of cognitive decline. In general, the overall mean scores tended to be higher on the telephone battery (latter administration) as compared to the in-clinic (first administration). The increase in score on the Telephone Battery suggests that participants may have used various tools for assistance or that learning occurred. For example, anecdotal evidence from clinical center staff indicates that during the telephone interviews many participants recognized the Wechsler stories as ones they had previously heard. The Digits Backwards instrument may be more susceptible to the inclination of participants to use tools for assistance, considering the difficulty many persons have in mathematics, or at least in mental computations, and the excess of high telephone scores associated with low in-clinic scores as demonstrated in the scatter plot. The possible use of assistance may decrease the ability of the various instruments to detect change over time. The large number of participants excluded for the reasons mentioned earlier is another potential limitation of this study. Reasons mentioned as to why these participants were excluded could be more likely to occur in weaker, older participants that may or may not be cognitively impaired. Therefore, the results of this paper could potentially be based only on younger and healthier, both physically and mentally, individuals.