Measuring attention in very old adults using the Test of Everyday Attention

Introduction

The concept of attention refers to a person’s information processing capacity (Mirsky, Anthony, Duncan, Ahearn, & Kellam, 1991; Shumway-Cook & Woollacott, 2000). Performing tasks that require more attention than available will result in reduced performance on those tasks. The Test of Everyday Attention (1994, 1996) (TEA) was developed to measure attention performance in persons with attentional deficits (I. H. Robertson, Ward, Ridgeway, & Nimmo-Smith, 1996). The TEA has been validated in persons aged 18 to 80 years old, but not in the very old (> 80 years old), who are most vulnerable to diminished cognitive functioning.

The TEA is strongly based on Posner and Petersen’s neuro-anatomical model of attention (Petersen & Posner, 2012). They propose that attention is fractioned in at least three different systems with all distinct neuro-anatomical bases: 1) a selection system to select relevant processes or stimuli and to inhibit irrelevant ones; 2) a vigilance system responsible for maintaining preparedness in the absence of external signs; and 3) the orientation system to engage and disengage attention in space, e.g. to focus and to take attention away. The first two systems are assessed in the TEA. The test measures different aspects of the selection system, namely divided attention and attention switching (I. Robertson, Ward, Ridgeway, & Nimmo-Smith, 1994). Sustained attention is part of the second system and reflects vigilance (Oken, Salinsky, & Elsas, 2006; Petersen & Posner, 2012).

Despite the important role of attentional capacity in the elderly, there are few tests to measure attention, specifically multiple domains of attention, in very old adults. Except for assessing multiple attentional domains, this test is based largely on everyday materials, which is useful to identify an individual’s attentional problems in real life. The validation of the Test of Everyday Attention was limited to persons aged 80 and younger and there are no published studies concerning the suitability of the TEA in very old adults. Validated measures of attention are needed in order to measure changes in the attentional domain and to understand their impact in very old adults. The number of people living to very old ages is steadily increasing; from 2000 to 2010, the US population aged 80 and older increased from 7,000,000 to over 9,000,000, a trend that is expected to accelerate in coming decades with the aging of the Baby Boom generation (U.S. Census Bureau, 2014). Therefore, the purpose of this study was evaluating use of the TEA in persons aged 80 and older living in the community.

Methods

The MOBILIZE (Maintenance of Balance, Independent Living, Intellect, and Zest in the Elderly) Boston Study (MBS), is a longitudinal population-based study of older adults living in the community in the Boston area (Leveille et al., 2008). Eligibility criteria of these persons aged 70 and older from the City of Boston and 5 neighboring communities included being able to communicate in English, walk at least 20 feet without personal assistance, and plans to remain in the area for 2 or more years. Moderate or severe cognitive impairment, measured as a score less than 18 on the Mini Mental State Examination (MMSE) was an exclusion criterion (Escobar et al., 1986; Folstein, Folstein, & McHugh, 1975). A third assessment wave, referred to as MOBILIZE Boston II (MBS II), took place approximately six years from baseline (2011–2015) and included the TEA and a number of other neuropsychological tests. Eligible participants included those who were alive and still residing in the Boston area, and could walk without personal assistance. Participants who experienced severe health deterioration or relocated to a nursing facility were excluded. The current analysis includes 249 persons aged 80 and older who completed the MBS II clinic or home visit assessments. A small number (n=39) could not come to the clinic for their assessment and instead completed the neuropsychological assessment in a home visit. The tests were administered by an experienced research assistant trained in the testing protocol by a neuropsychologist with expertise in administering these specific tests including the TEA, in older and chronically ill adults. Participants signed informed consent at the start of the assessment visit. The institutional review boards of the Hebrew SeniorLife and University of Massachusetts Boston approved all procedures and methods. A detailed description of the original MBS design and methods is described elsewhere (Leveille et al., 2008).

Measurements

This cross-sectional study used data from the MBS baseline interview, including sociodemographic characteristics and new assessment data from the MBS II that included health characteristics, the TEA subscales and the neuropsychological battery.

Sociodemographic and Health Characteristics

Sociodemographics including age, gender, race and educational level were assessed at the MBS baseline home interview (Leveille et al., 2008). Education level was represented by two categories: high school or less (< 12 years of education) and college attendance or graduate school (≥ 12 years of education). Health characteristics include vision, hearing difficulties and self-rated health. In the MBS II assessment, distant vision was assessed using the Good-Lite Chart light box with letter charts, which participants had to read at a 10-foot text distance (Wood et al., 2005). Poor vision was defined as the lowest performing quartile. Participants were asked if they had a hearing problem (yes/no) during the interview. Self-rated health was measured on a 5-point scale and dichotomized into fair/poor and good/very good/excellent self-rated health.

Test of Everyday Attention

We measured attention using the TEA, developed by Robertson, Ward Ridgeway, and Nimmo-Smith (I. H. Robertson et al., 1996). The TEA provides a comprehensive assessment of attention measuring multiple domains in adults and was standardized for persons aged 18 to 80 years in a group of British volunteers (I. H. Robertson et al., 1996). The test has high test-retest reliability across different ages and performs well across cultural groups (Crawford, Sommerville, & Robertson, 1997; I. H. Robertson et al., 1996). The TEA is based on an imaginary vacation trip to Philadelphia, Pennsylvania (USA). For the present study, we used 4 subscales of the TEA: Visual Elevator (attentional switching), Map Search (visual selective attention), and Telephone Search (selective attention) and the Telephone Search While Counting with dual task (sustained and divided attention). After piloting the TEA with a small number of very old volunteers, the instructions for some of the TEA subscales were modified slightly to make them as simple and clear as possible for American study participants. For instance, in the TEA manual, the instructions for the Map Search were as follows: “The symbol here shows where gas stations can be found in the Philadelphia area. There are many symbols like this on the map”. Instead of these sentences, we instructed the participants “This is the symbol that you will be asked to find on a map of Philadelphia. The symbol will be much smaller than this on the map.” Practice sessions were provided according to the TEA testing guidelines. To accommodate vision and hearing problems, participants were offered magnifying glasses and use of an audio amplifier with headphones before each of the tests.

Attention switching

The Visual Elevator test is a self-paced task where participants are asked to imagine that they are in an elevator and need to count up and down according to a series of large bold arrows that pointed upward or downwards, shown on display cards. The accuracy score was based on how many correct final floor numbers the participant achieves out of the 10 hypothetical elevator rides shown. The timing score was calculated by counting the time taken for the correctly performed switches (where the elevator switches a number of times going up or down). The Visual Elevator Test measures attentional switching and mental flexibility (I. H. Robertson et al., 1996).

Selective attention

As mentioned above, the Map Search is a timed visual search task where participants are asked to search for and circle gas pump symbols on a busy colored map of the Philadelphia area. There are also other symbols on the map such as restaurant symbols, with 80 items of each. The total score is calculated according to the number of gas pump symbols circled within 2 minutes.

In the Telephone Search Test, participants are asked to look for matching symbols beside telephone numbers while they are searching a list of plumbers in a simulated telephone directory. The average time-per-target score is calculated by dividing the total time by the number of correctly detected symbols.

Sustained attention

The Telephone Search While Counting Test resembles the previous test. Participants are additionally asked to count a number of series of audio tones presented by a tape recorder. The average time per correctly identified symbols, again, was calculated.

Divided attention

To obtain a measure of divided attention (Dual Task Decrement score), the time-per-target score from the prior Telephone Search task is subtracted from the time per target score weighted for accuracy of tone counting.

Neuropsychological measures

The neuropsychological battery measured a broad range of cognitive functioning including attentional capacity, executive functioning and memory.

In addition to the TEA, the other test of the attentional domain was the Trailmaking Test (TMT) Part A. The TMT part A requires the participant to connect number targets on a paper in sequential order using a pencil (e.g. 1–2-3–4) (Lezak, 2004; Tombaugh, 2004).

The domain of executive functioning includes the TMT Part B, TMT Delta, Clock-in-the-Box Test (CIB) and the Letter fluency Test (F, A, S words). The Trail Making Test part B requires the individual to connect both number and letter targets in an alternating sequence (e.g. 1-A-2-B-3-C) (Lezak, 2004; Tombaugh, 2004). The difference score between Part A and Part B resulted in TMT Delta, calculated to control for information processing speed and motor function and is used in other studies to measure executive functioning (Ble et al., 2005; Drane, Yuspeh, Huthwaite, & Klingler, 2002; Eggermont, Milberg, Lipsitz, Scherder, & Leveille, 2009). The Clock-in-the-Box Test (CIB) requires the participant to read written instructions and to execute those instructions by drawing a clock and setting the clock to the correct time (Chester et al., 2011; Grande, Milberg, Rudolph, Gaziano, & McGlinchey, 2005).

Verbal fluency was tested with 3 phonemic fluency tasks, where participants are asked to name as many words as possible, beginning with the given letters F, A and S for 60 seconds for each letter (Lezak, 2004).

The Hopkins Verbal Learning Test (HVLT) was used to assess memory. It contains a 12-item word list learning test including immediate recall, delayed recall and a 24-item word recognition test (Glisky & Kong, 2008). The MMSE is a short multidimensional assessment instrument, providing information about general cognitive function, also largely in the memory domain (Folstein et al., 1975).

Statistical analysis

Completion of the TEA battery was examined according to demographic and health characteristics. Mantel-Haenzel Chi-square tests (1 d.f.) were used to test group differences for ordinal variables. For categorical variables, overall Chi-square tests were used.

Secondly, the TEA subscale scores were assessed along with the percentage of participants unable to complete the test. In addition, the reasons for not completing TEA subscales were reported (where only one reason per subscale could be identified). Pearson correlation coefficients were calculated to assess the association between the TEA subscales and the neuropsychological test battery components. We interpreted correlations of R (rho) ≥ .50 as strong correlations and R = .30 – .49 as moderate correlations (Cohen, 1988). All analyses were performed with SPSS v21.0 (IBM Corp., Armonk, NY).

Because we did not use all original components of the TEA, we were not able to replicate the original factor structure of the TEA (I. H. Robertson et al., 1996), and accordingly factor analysis was not included in our analyses.

Results

Study Sample Characteristics

Of the initial 328 participants from the original MBS cohort who would be aged 80 and older in the new assessment wave, 17 (5%) had died and 8 (2%) refused to participate. Another 53 persons (16%) were excluded because of severe health deterioration or relocation to a nursing facility or out of the Boston area. Only 1 person had a missing record. This resulted in a final study sample of 249 participants aged 80 to 101 years.

Participants aged 80 years and older in the MBSII assessment wave had a mean age of 87 years (SD=4, range 80 to 101) with 166 women (67%) and 83 men (33%), similar to the original gender distribution of the MBS cohort. A total of 67 (27%) of the 249 participants had incomplete TEAs, meaning they were missing at least one subscale. However, 90% of participants were able to complete at least three of the four tests, 94% completed 2 or more tests and 96% completed at least one test.

Older adults with an incomplete TEA were more likely to have fewer years of education, to be African-American, to have fair or poor self-rated health and to have hearing problems, compared to people who completed the four TEA subscales (Table 1).

Table 1.

Completion of the TEA according to demographic and health characteristics in adults aged 80 years and older, MOBILIZE Boston Study II.

Characteristics	Completed TEA (n=182 ) percent	Incomplete TEA (n= 67 ) percent	p-value*
Age groups
80–89y	78	70
90–102y	23	30	.24
Women	65	70	.48
Education:
High school or less	22	42
Attended college	78	58	.002
Race: White	86	69
Black	10	25
Other	4	6	.006
Fair/poor self-rated health†	7	19	.005
Hearing problem‡	57	74	.017
Vision problem £	22	27	.42
Cognition (MMSE<23)	17	63	≤.001

^*Mantel -Haenzel chi-square test (1 d.f.), except for race comparisons, which used chi- square test for overall differences (2 d.f.)

^†Self-reported health (fair/poor vs. good/very good/excellent)

^‡Self-reported hearing problem (yes/no)

^£**** Poor vision was defined as the lowest performing quartile

TEA – Test of Everyday Attention

TEA subscales

The distributions of the TEA subscales, including the percentages of participants unable to complete each subscale, are presented in Table 2. Proportions of participants with missing TEA subscales ranged from 8% on the Map Search up to 19% on the Visual Elevator Test. In general, reasons for not completing TEA subscales had to do with failure to comprehend test instructions despite repetition and practice. Most of these people had evidence of possible cognitive difficulties (MMSE score < 23), for example, 69% of those who did not complete the Visual Elevator test had low MMSE scores. Also vision problems played a role in not completing tests, mainly on the Map Search test (15 out of 19 missing tests). Other participants did not attempt the test or refused, resulting in missing data, particularly on the Telephone Search While Counting test (10 out of 31 participants with missing tests). The reasons for not completing the tests are displayed in Table 3.

Table 2.

Test of Everyday Attention Subscale Scores and Percent Unable to Complete Test

Subscale	Mean (SD)	Median	Range	Incomplete
Visual Elevator*	4.67 (1.80)	4.20	2.20 ; 18.20	48 (19%)
Map Search **	32.25 (15.58)	32.00	0.00 ; 67.00	19 (8%)
Telephone Search***	5.54 (3.20)	4.60	2.50 ; 26.10	22 (9%)
Telephone w/ Counting ****	7.03 (4.97)	5.40	2.80 ; 40.40	31 (12%)
Dual task decrement score	4.90 (9.54)	1.75	−3.10 ; 57.20	45 (18%)

^*Timing score, counting the time for correct switches (/time per switch)

^**Number detected in 2 minutes

^***Time per target

^****Time per correctly identified symbol

Table 3.

Reasons for not completing TEA subscales

Reason not completed	Visual Elevator N (% miss)	Map Search N (% miss)	Telephone Search N (% miss)	Telephone w/ Counting N (% miss)
Vision problem	7 (15%)	15 (79%)	12 (55%)	6 (19%) **
Cognitive problem*	33 (69%)	4 (21%)	6 (27%)	15 (49%)
Unable to do	4 (8%)	-	-
No attempt/ refused	4 (8%)	-	4 (18%)	10 (32%)
TOTAL (N)	48	19	22	31

^*Reported by the interviewer or by a MMSE score <23.

^**Includes 3 participants with a hearing problem

TEA – Test of Everyday Attention

There were differences in demographic and health factors associated with selected incomplete tests. For example, less education and African-American race were associated with missing data only in the attention switching task. Poor self-rated health, and hearing and vision problems were associated with inability to complete the Map and Telephone Search Tests (selective attention). (Data not shown).

Correlations

Among participants who completed the TEA, scores on the Trail Making A Test correlated most strongly with several TEA domains, specifically within the domains of attention switching, selective and sustained attention (R= .60-.70, Table 4). The correlation was moderate between Trail Making A and divided attention (Dual task decrement) (R = .41) and selective attention (Map Search) (R = −.49). Trail Making B was moderately correlated with sustained attention and attention switching. Weakest correlations were observed between the TEA subscales and HVLT tests of recall and recognition.

Table 4.

Correlation between TEA subscale scores and Neuropsychological Tests

Neuropsych. Test	Visual Elevator	Map Search	Telephone Search	Telephone w/ Counting	Dual task decrement
Trails A	.60**	−.49**	.64**	.70**	.41**
Trails B	.49**	−.51**	.55**	.46**	.20**
Trails B-A Delta	.33**	−.41**	.45**	.32**	.06
Clock-in-a-box	−.41**	.35**	−.36**	−.39**	−.34**
FAS words	−.33**	.34**	−.40**	−.43**	−.31**
HVLT: Immed. Recall	−.17*	.37**	−.33**	−.37**	−.24**
HVLT: Delayed recall	−.18**	.36**	−.31**	−.31**	−.26**
HVLT: Recognition	−.13	.25**	−0.34**	−.30**	−.22**
MMSE	−.37**	.39**	−.45**	−.48**	−.37**

^*Pearson correlation coefficient, p-value significant at the 0.05 level

^**significant at the 0.01 level

TEA – Test of Everyday Attention

Discussion

The results of this study show that in community-living adults aged 80 years and older, the TEA is a valuable measure of attentional resources. The TEA subscales were significantly correlated with another neuropsychological test that measures attention (TMT A). In general, this population-based sample of very old adults was able to complete most of the TEA subscales. The Visual Elevator Test, which measures the domain of attentional switching, was the most challenging test for these older participants, as evidenced by the proportion of incomplete tests related to difficulties in understanding the instructions or in completing the practice test.

This is partly in line with the original TEA paper, where Robertson and colleagues also found a correlation between the Telephone Search Test (time per target) and Trails B (r= .63). In de factor analysis they have also done, Trails B loaded on the same factor as Map Search and the Telephone Search Test, namely ‘visual selective attention/speed’. (I. H. Robertson et al., 1996)

The growing recognition of the importance of dual task performance in older adults requires validated tools to measure the attentional domain. Previous studies have shown that attention-demanding ‘dual tasks’ affect the gait pattern. Older adults who performed a second task while they walked showed reduced gait speed (Pajala et al., 2005; Shkuratova, Morris, & Huxham, 2004; Springer et al., 2006). In addition, the consistency of the gait pattern (gait variability) was altered in idiopathic elderly fallers and patients with Parkinson’s disease (Springer et al., 2006; Yogev et al., 2005), which has been associated with increased fall risk (Barak, Wagenaar, & Holt, 2006; Hausdorff, Rios, & Edelberg, 2001; Springer et al., 2006). We showed that most very old adults in our study population were able to perform the cognitive ‘dual task’ of the TEA’s subscale, i.e., Telephone Searching While Counting.

Notably, the TEA was strongly correlated with the Trail Making Test part A, which also measures attention. However, the moderate correlations between several subscales of the TEA and the TMT B and TMT Delta, MMSE, Clock-in-a-Box and Letter Fluency may indicate either that some TEA subscales also involve executive functioning or, these other neuropsychological tests also are measuring attentional abilities (e.g., several items on the MMSE require attentional skills). In those TEA subscales, reaction time also may play an important role, which was described previously (Chan, Hoosain, & Lee, 2002).

Robertson and colleagues described several possible constraints on the validity of the TEA, such as vision, verbal intelligence and various clinical syndromes. In their study, the TEA subscales were not mainly affected by hearing. Also, they excluded participants with peripheral vision problems who reported difficulty in detecting the symbols on the Map Search test. Another possible constraint was intelligence. In their study, the Visual Elevator test was the only test with a partial correlation coefficient exceeding .3, reporting the correlation between the TEA subtest and the The National Adult Reading Test (NART) IQ score (adjusted for age). Robertson and colleagues therefore suggest that participants with intelligence below the average and a Visual Elevator score just below the average score should not be seen as an impaired performance on the attentional domain (I. H. Robertson et al., 1996). In the present study, reasons for not completing the TEA subscales mainly had to do with failure to comprehend test instructions despite repetition and practice. Approximately one-fifth of older adults were unable to complete the Visual Elevator Test, which is known to be weakly associated with intelligence. It should be noted that the MBS cohort comprises a relatively highly educated cohort of elders, reflecting the demographics of Boston in general. Additionally, it is reflecting those older adults who were most likely to be eligible and interested in participating in this longitudinal study (Leveille et al., 2008). However, two-thirds of the participants who did not complete the Visual Elevator Test scored below the standard MMSE cutpoint for dementia screening (score below 23), which may explain the high proportion of incomplete tests. On the other hand, cognitive problems played only a minor role in failure to complete the other TEA subscales, suggesting that the Visual Elevator Test was the most cognitively challenging for this population. It may be that attentional switching is particularly sensitive to age-associated changes, consistent with reported age-related decrements in performance of complex tasks that draw on executive functioning (Tun & Lachman, 2008). Moreover, higher education does not protect against age-related decrements in attentional switching (Van Gerven, Meijer, & Jolles, 2007). As previously mentioned, although our sample was population-based, MBS participants had more years of education than the general population of older adults. It is likely that we would have had more incomplete tests in a sample with less education.

Similar to Robertson and colleagues’ original TEA study, vision problems were sometimes recorded in the MBS II as a reason for not completing the test despite accommodations using magnifying glasses (I. H. Robertson et al., 1996). However, hearing difficulties were not recorded by the interviewers in our study as a main reason for not completing any of the TEA subscales. Participants were able to hear the given instructions and hear the tones in the Telephone While Counting Test, with the exception of only 3 participants. These participants were already unable to do the testing because of vision problems. Interviewers adjusted the volume of the tones or used an amplifier with headphones when needed. However, people who reported having hearing problems during the health interview were more likely to have an incomplete TEA.

Compared to Robertson’s study, we did not examine the TEA in various clinical syndromes, such as stroke, Alzheimer’s disease or progressive supranuclear palsy. The MBS initial eligibility criteria excluded persons with moderate or severe dementing illness and those with serious mobility problems, though it is possible that some participants may have mild cognitive impairment. Our findings suggest that some subscales of the TEA, such as the Visual Elevator test, would not be feasible for subgroups of older adults with serious cognitive impairments.

Overall, our results suggest that some minor modifications to the tests might reduce the missing data in this very old population. First of all, the arrows on the Visual Elevator test could be enlarged for visually impaired persons. Also, the Visual Elevator tasks could be portrayed in one long wide sheet to avoid the line wrapping that created confusion for several participants. Because participants often did not grasp the idea of an elevator going up and down, it may be that this concept is too abstract and other tests of attentional switching may be needed to capture the full range of functioning in this domain. In general, the study staff reported that participants were fatigued by the lengthy instructions for the TEA and might understand the test better if the introduction was abbreviated. Instead, older adults may perform better on the test if the instructions focused only on the most practical information needed to guide them in performing the tests.

Our study has several strengths, including the substantial sample size of a very old population-based cohort. Key to the current study, we collected specific information about reasons for incomplete tests. Furthermore, we used an attention test covering multiple attentional domains accompanied by an extensive neuropsychological battery.

Our analysis was limited in that we could not fully test convergent validity because we included only some subscales of TEA. We needed to limit subject burden because the TEA was part of a large battery of physical and cognitive tests. However, we found strong correlations between the Trail Making A score and 3 out of 5 TEA subscale scores, and moderate correlations with the other 2 TEA scores. Trail Making Part B was the only other test showing strong correlations with TEA domains (2 out of 5 subscale scores). A number of the other neuropsychological tests were moderately correlated with the TEA subscales, reflecting the attentional demands of many of these standard cognitive tests in this old population. Using the truncated set of TEA subscales, we could not perform a factor analysis comparable to the original TEA validation, as described previously. However, the TEA has shown good validity and test-retest reliability in persons aged 18–80 years old. Robertson and colleagues generally showed good reliability for 1-week test-retest on TEA Versions A and B in a sample of 154 volunteers and with Versions B and C after another week for a subsample (n=39) of the larger group (I. H. Robertson et al., 1996). Further research is needed to assess test-retest reliability in the population over age 80 years.

In conclusion, these results demonstrate the feasibility and potential value of the TEA for measuring multiple domains of attention in most very old adults. Nevertheless, future research is needed to determine whether adaptations of the test will reduce missing data and make the test more suitable for older persons across a broader range of education and cognitive functioning.