Eye Tracking Analysis of Visual Cues During Wayfinding in Early Stage Alzheimer’s Disease

Abstract

Introduction

Persons with Alzheimer’s disease (AD) have profound impairment in wayfinding, potentially related to a deficit in visual attention and selection of relevant environmental information. This study sought to determine differences in visual attention to salient visual cues and non-salient cues (building features) in older adults with and without AD during active wayfinding in a large scale, virtual reality spatial task.

Methods

Fifteen subjects (7 with AD and 8 controls without AD) were asked to find their way repeatedly during 10 trials in a virtual simulation of a senior retirement community. Subjects wore eye tracking glasses to capture visual fixations while wayfinding. The least square means (LSMs) and their standard errors (SEs) for percentage of fixations and duration of fixations on salient and non-salient cues were estimated from the linear mixed effects models and compared by group (AD or control) and cue type.

Results

The group by cue type interaction was significant for both percentage of fixations (F(1,13)=6.79, p=. 02) and duration of fixations (F(1,13)=4.87, p=. 04). The AD group had significantly lower percentages of fixations on salient cues, LSM = 57.91, (SE=2.44) compared to controls, LSM = 66.40 (SE=2.19); p=.03. Persons with AD had a higher percentage of fixations on building features LSM=31.65 (SE=2.18) than controls, LSM=24.54 (SE=1.95); p=.02. Shorter durations of fixations on salient cues were experienced by the AD group, LSM = 38.89 (SE=1.69) than the control group, LSM=44.69 (1.55); p=.02.

Discussion/Conclusion

Individuals with AD may have difficulty selecting relevant information for wayfinding as compared to normally aging individuals; and attend more frequently than controls to irrelevant information. This may help explain the wayfinding difficulties seen in AD.

Introduction

Alzheimer’s disease (AD), the most prevalent cause of dementia [1], produces a deficit in spatial cognition, leading to impairments in wayfinding in both novel and familiar environments [2–5]. Wayfinding deficits occur early in the disease, with over half of all persons with early stage AD having difficulty finding their way even in familiar environments [3]. Deficits in spatial cognition in AD and MCI are well documented, and include deficits in learning routes [6], recalling landmarks necessary for wayfinding, and in using allocentric based strategies [7].

During wayfinding, persons must search, select, and process sensory information such as paths and visual cues (or landmarks) [8]. Individuals must select relevant cues that help to distinguish one area from another from the environment. Cues have qualities, such as size, color, shape, and meaning that make them useful for wayfinding. Caduff and Timpf theorize that cues must be salient such that they elicit the visual attention of the wayfarer in order to be useful for wayfinding. After selection, wayfarers must then focus on the cue and store its attributes in memory [9]. People cannot attend to all information in the environment; instead, they must allocate mental resources to specific environmental features – using selective attention. Selective attention can be influenced by exogenous features such as the color, form, or location of objects [9]. Wayfinding, as a task involving goal-directed behavior, involves cognitive processes that can be affected by Alzheimer’s disease [3,10–12].

In normal aging, there are changes in visual search patterns and attention that may affect how visual cues are selected and remembered in wayfinding tasks Visual attention and visual search abilities, which are compromised due to aging [13,14] and further due to AD, may be the cause of deficits in the recognition and use of environmental cues for wayfinding [8,15]. Eye tracking studies using static scenes have found that persons with AD show impaired search patterns, disorganized visual search patterns, and longer visual fixation times do older adults without disease [16]. Longer fixation times may be related to problems disengaging from cues [16]. In several studies, subjects with AD were shown to fixate less frequently than controls on incongruent elements presented in a scene [17,18] or on novel stimuli than did normal controls [19]. Thus, persons with AD may affect a person’s ability to recognize important changes or elements within a scene. This beginning evidence suggests that persons with AD may have less efficient visual search patterns, they may fixate longer and more often on irrelevant information and less often on relevant information, and they may not notice environmental changes. However, empirical studies of fixation are limited, and their results have not been consistent [10].

Absent from the literature are studies that examine visual fixations in persons with AD while actively finding their way in an environment. Eye movements are thought to be somewhat task dependent. One recent study examined eye movements of healthy older (not with AD) versus younger adults while passively learning short, pre-determined routes with cues present at intersections; and again during static visual images of the intersections in the route. They found that older individuals fixated less often than the younger group on navigationally important features such as wayfinding cues [8]. However, this study did not record eye tracking while participants found their way actively in a larger scale space and did not include persons with AD; in fact, we found no studies that examined visual fixations in persons with AD while actively wayfinding in large scale spatial environments.

Thus, the purpose of this study was to determine differences in visual fixations to salient visual cues and non-salient building features between persons with AD and those without AD in a large scale, virtual reality, spatial task. It was hypothesized, based on the review of literature, that persons with AD would have fewer overall fixations on salient cues than those without AD; and that persons with AD would fixate more often and longer on non-salient cues, such as repetitive building features, than those without AD. We also hypothesized that persons with and without AD would fixate more on salient than non-salient cues.

Materials and Methods

Participants

The current study is a part of a larger study on wayfinding in aging and Alzheimer’s disease (AD). In the parent study, older community-dwelling adults with and without early stage AD were asked to find their way in a virtual reality simulation of a large senior residence [20]. The inclusion criteria for the study were: 1) Age 62 or older; 2) for the control group, no diagnosis of cognitive disease, and Mini Mental Status Scores (MMSE) of ≥ 27, indicating a low probability of dementia; [21] 3) visual acuity of 20/40 with correction and not color blind; 4) able to move a joystick. Alzheimer’s disease or Mild Cognitive Impairment (MCI) due to AD had to be diagnosed by a health care provider using established criteria [22,23] and test in the early stage of the disease using the Clinical Dementia Rating Scale (.5 – 1) [24]. Persons with early stage AD versus later stage were chosen for this study because changes in wayfinding are shown to occur very early in the disease [3,25] and the study aimed to determine if there are early changes in the selection of salient visual cues for wayfinding in persons with very early stage AD.

For this portion of the study, a subsample of 15 subjects (3 with early stage AD, 4 with MCI due to AD, and 8 control) out of the total 83 were selected to analyze their eye tracking data. The MCI and AD groups were combined and called the AD group in the parent study and this analysis, giving a sample size of 7 AD and 8 control. The AD and MCI subjects were combined due to the small sample size, and also because they were very similar in terms of cognitive abilities. In fact, the criteria for the diagnosis of MCI and early stage AD are very similar (differing only in that AD involves functional deficits) that there is overlap in the diagnoses [26].

The reason not all subjects were included was due to the large amount of resources required for coding the eye tracking videos (over 40 hours per subject and over 3200 eye tracking data points captured per subject). Subjects whose eye tracking videos were the most complete and who represented the population in the study with equal amounts of males and females and a range of ages were selected. The mean age of subjects was 76 years (range 65-86), and 8 out of 15 (53%) were female.

Procedure.

Subjects were recruited from the community and memory clinics. Written informed consent was obtained from the subjects and/or decision makers for those without consent capacity. Those enrolled completed a demographic survey, the Mini-Mental Status Examination (MMSE), Montreal Cognitive Assessment (MoCA), and digit span tests.

Wayfinding testing took place over a two-day period and is reported in depth elsewhere [20]. Briefly, subjects were asked to find their way to a specific location in a virtual reality simulation of a large senior retirement community for five consecutive trials for each of two days (10 total trials). The VR simulation was projected on a 12- foot screen. Subjects tried to find their way by moving throughout the environment using a joystick. Trials ended when the subject found the location or when three minutes elapsed (to avoid subjects becoming frustrated or overly tired). VR environments have been shown to be a valid tool for assessment of spatial navigation, with results from VR transferring to real world environments even in persons with cognitive impairment [27,28].

For the eye tracking portion of the study, while wayfinding, subjects wore eye tracking glasses (Applied Science Industries Mobile Eye-XG) [29]. The glasses were lightweight goggles that contained a small video camera and an optical device that track eye movements using pupil-corneal reflection, with a visual range of 50 degrees horizontal and 40 degree vertical. The output from the eye tracker is a video recording of the visual scene, superimposed with the eye gaze (cross hairs), so that the movement of the eyes during the virtual navigation was recorded.

The ASL Result Plus Gaze Map Module [30] was utilized for analysis of the data. Videos were analyzed frame by frame by drawing lines encompassing the objects/areas at points of fixation, called areas of interest (AOI). Objects belonged to two categories – salient cues, and non-salient cues. The non-salient cues included building features such as doors, the floor, lights, corners, and handrails; these were not helpful for wayfinding, as they were repetitive throughout the environment and did not distinguish one area from another. The salient wayfinding cues were eleven colorful, large, high contrast, simple objects placed at key decision points. These included a picture of a sun, a rainbow mobile, a large American Flag, a bunch of red balloons, a picture of children, a butterfly mobile, a rainbow mobile, a picture of a large fish, a picture of a cardinal, a red car mobile, a sun picture, and a wall hanging of a tiger rug (Figure 1). The qualities of the helpful cues (large, colorful, placed at key decision points) were determined based on prior research on older adults and wayfinding cues [31–33]. Visual fixations off screen and on furniture were not included in the analysis because they did not meet the criteria for being salient or non-salient cues.

Fig 1.

View of one hallway of the wayfinding environment showing a salient cue (an American flag) at akey decision points. Doors, lights, rails, and floors are examples of nonsalient cues that were repetitive and not helpful for the wayfinding task.

Measures

Fixations.

Visual fixations were identified by the ASL gaze map software using established algorithms [30]. To reflect fixations, the software provided two summary outcome variables were defined for salient and non-salient cues: 1) Percentage of fixations, defined as the number of fixations on salient and non-salient cues, out of the total number of fixations; and 2) Duration of fixations, defined as the duration of all fixations on salient and non-salient cues, out of the total duration of all fixations.

Demographics and Cognitive Measures.

Subjects completed a demographic survey to determine age, gender, and years of education. In addition, the Digit Span Forwards (DSF) and backwards (DSB) tests were administered to test working memory since the ability to recall landmarks may be dependent, in part, on working memory; and spatial learning has been shown to be impacted by working memory [34]. In the DSF test, subjects were asked to repeat an increasingly longer series of numbers; the highest amount of numbers they can recall is their score. The DSB is administered the same, except subjects must state the numbers in reverse order [35]. The MoCA is a 10 minute, 30 item screening tool that assesses short term memory, visuospatial memory, executive functioning, attention/working memory, language and orientation [36]. Higher scores indicate less probability of cognitive disease. The MoCA has a sensitivity of 83% in detecting mild cognitive impairment and 94% for dementia [37].

Statistical Analysis

The demographic and cognitive measures of the AD and control groups were summarized, and differences between two groups were evaluated using t-, Wilcoxon, or Fisher’s exact tests as appropriate. Repeated measures of two fixation outcomes, percentage and duration of fixations on cues were analyzed using linear mixed effects (LME) models that generalize classical analysis of repeated measures and allow for data missing at random (MAR). Twenty repeated measures (10 time points, with outcome for salient and non-salient cue at each time point) were nested within subjects, and heterogeneous autoregressive correlation structure of the first order was specified. The covariates included gender, time (trial number) entered as a class variable to model potentially non-linear patterns, cue type (salient or non-salient), study group (AD v. control), and study group by cue type interaction. The least square means (LSMs) and their standard errors (SEs) for the levels of the interaction term were output from each model and reflected average fixations on salient and non-salient cues over time. T-tests comparing the LSMs by study group and by cue type produced formal tests of study hypotheses. The 95% confidence intervals (Cis) were estimated for the differences by group for each cue type and by cue type within each group. All analyses were performed using SAS 9.4, with LME modeling implemented in the MIXED procedure.

Assuming moderate correlation coefficient of 0.6 between pairs of repeated measures, the between-group differences of one adjusted standard deviation were detectable as statistically significant with power of 0.80 or greater in two-sided tests at 0.05 level of significance.

Results

The summary of demographic and cognitive measures of study sample is presented in Table 1. The AD and control groups were similar with respect to age, education level, or gender; but the AD group had significantly lower MMSE and Montreal Cognitive Assessment (MoCA) scores as expected (Table 1). Digit span forwards and backwards testing showed no differences between the groups. All subjects except for one person in the AD group had completed fixation data across 10 trials. The available data from one subject that had missing data on trials 7-10 were included in the LME models under the MAR assumption.

Table 1.

Comparison of demographic and cognitive variables between study groups

Demographic Variables	Control Group (n=8)	AD Group (n=7)	t (df)	Chi square (df)	p value
Age (M, SD)	75.00 (1.20)	76.57 (5.03)	−0.478 (13)		0.640
Years education (M, SD)	15.75 (2.96)	15.29 (2.56)	0.322 (13)		0.753
Female (n, %)	4 (50%)	4 (47%)		0.077 (1)	0.782
DSF (M, SD)	6.25 (1.17)	5.86 (0.90)	0.722 (13)		0.475
DSB (M, SD)	4.00 (1.07)	4.14 (1.07)	−0.258 (13)		0.800
MMSE (M, SD)	29.00 (1.20)	26.43 (2.30)	2.661 (13)		0.027*
MoCA Total Score (M, SD)	25.13 (2.41)	19.00 (3.51)	3.880 (13)		0.003*

Note. AD=Alzheimer’s disease; DSF=Digit Span Forward; DSB=Digit Span Backward; MMSE=Mini-Mental State Examination; MoCA=Montreal Cognitive Assessment.

There was no appreciable change in differences between groups as time progressed (Figure 2); therefore, average differences over time between groups for salient and non-salient cues were evaluated. Compared to males, females had a lower percentage of fixations by 1.46 (SE=0.63), p=.04 and lower fixation duration by 5.56 (SE=0.74), p<.01. The group (AD versus control) by cue type interaction was significant for both percentage of fixations (F(1,13)=6.79, p=.02) and duration of fixations (F(1,13)=4.87, p=.04), and the corresponding LSMs are presented in Table 2. As seen from the table, after adjusting for gender, the AD group had significantly lower percentages of fixations and durations of fixations on salient cues compared to controls. For the non-salient cues, the percentage of fixations but not duration of fixations was significantly greater in the AD group compared to controls. Within each group, people fixated significantly more and spent more time fixating on salient versus non-salient cues. The difference in total fixations between non-salient and salient cues in the control group was −41.85, 95% CI (−50.59, −33.13), p<.01; in the AD group the difference was −26.27, 95% CI (−36.00, −16.53), p<.0001. For the fixation duration, the difference by cue was −22.00, 95% CI (−27.15, −16.84), p<.0001 in the control group, and −14.71, 95% CI (−20.36, −9.06), p<.0001.

Fig 2.

Least Squares means for percentage of fixations and duration of fixations on salient and non-salient cues at each time point.

Table 2.

Comparison fixations on salient and non-salient cues by group.

Outcome	AD group, LSM (SE)	Control group,LSM (SE)	P-value (95% CI for group difference)
Percentage of fixations on salient cues	57.91 (2.44)	66.40 (2.19)	.03 (−13.38, −0.83)
Percentage of fixations on non-salient cues	31.65 (2.18)	24.54 (1.95)	.02 (1.55, 15.43)
Percentage of duration of fixations on salient cues	38.89 (1.69)	44.69 (1.55)	.02 (−10.39, −1.22))
Percentage of duration fixations on non-salient cues	24.18(1.17)	22.69 (1.07)	.34 (−1.77, 4.75)

Discussion

The most important finding from this study is that those with AD fixated less often and spent less time fixating on the salient visual cues than did similarly aged, cognitively intact individuals; and the AD group fixated more often on the non-salient cues. Despite the differences between groups, both control and AD groups fixated more frequently and had longer duration of fixations on salient visual cues than non-salient visual cues. Interestingly, the duration of fixations on non-salient visual cues was not significantly different between the groups; indicating that those with AD did not fixate longer (or have difficulty disengaging from) non-salient visual cues. The results of the study suggest that subjects with AD had more difficulty selecting and/or visually attending to the salient visual cues over time than did those without AD. This finding is important, because in order to find one’s way in the large-scale VR environment, it was necessary to identify the salient cues, which were present at each decision point, and all of the other building features were repetitive and unhelpful.

The unique contribution of this study is that visual fixations were tracked while subjects actively attempted to find their way during a lifelike wayfinding task over multiple trials. Prior studies have not examined eye tracking in this population while subjects actively found their way over time. However, the results of this study are congruent with other studies examining visual fixations in persons with AD compared to controls in static scenes. These studies have shown that persons with AD may not select relevant information from a scene [16]. Our study findings showed that persons with AD employ selective visual attention while wayfinding, but do not select salient visual cues for wayfinding as often as the controls. These differences in selective attention may partially explain the wayfinding deficit seen in persons with AD.

Persons with AD may not select relevant cues for wayfinding due to several reasons. They may not recall the cue from prior exposures or recognize it as relevant for wayfinding due to memory impairment. They may also not encode the cues into a cognitive map due to hippocampal atrophy seen in AD; prior studies have shown that persons with AD are less likely to use hippocampal based strategies [7,38]. Additionally, those with AD may spend more attentional resources on the physical action of moving and staying within a path (thus spending more time looking at the building features); leaving them with have fewer resources to encode wayfinding cues.

An interesting finding from the study was that females had less percentages and durations of fixations than did males. Research on wayfinding has shown a strong male performance advantage, and that females are more reliant on salient visual cues than males for wayfidning [20,32,39]. However, the differences in types of cues attended to by males versus females is understudied. Future studies should examine these differences further.

A surprising finding in this study is that the AD group did not have a longer duration of fixations on non-salient cues when compared to the control group. Prior studies have shown that persons with AD have longer fixation durations during visual search paradigms in which subjects are asked to find a target among distractors on a computer screen [16,40]. These findings have led to support for a hypothesis that persons with AD have problems disengaging from visual information [41], although there is significant variability in findings among other studies [42]. Ameiva et al. in a review of inhibitory processes in AD concluded that evidence for inhibitory process changes in AD is not conclusive and may be task dependent [42].

This study had a limitation of a small sample size, which was offset by the availability of 10 repeated measures that allowed to reduce the error variance and detect between-group differences of approximately one standard deviation. Smaller differences such as those of 1/3 to ½ of the standard deviation may also be meaningful [43–45], but were not observed in this study, and the non-significant finding for duration of fixations on non-salient cues corresponded to mean difference of 1.5% and less than 1/5 of the standard deviation, which corresponds to a small effect size in Cohen’s classification [46]. Future studies might focus on eye tracking/visual fixations in older adults and track them over extended time intervals to enable determination of changes over the course of AD progression. In addition, research on methods to enhance visual attention to salient cues may provide a method to improve wayfinding ability in persons with AD.

In conclusion, the results of this study showed that persons with AD had less numerous fixations and shorter fixations on salient visual cues than did normal older adults without disease while wayfinding in a simulated large-scale spatial environment. These results support a hypothesis that deficits seen in wayfinding tasks in large scale space may be due to the selection and recall of salient cues whilst wayfinding, rather than problems disengaging from cues. These results provide information that may explain the decline in wayfinding ability seen in older adults with AD.