Differential impact of partial cortical blindness on gaze strategies when sitting and walking - an immersive virtual reality study

Abstract

The present experiments aimed to characterize the visual performance of subjects with long-standing, unilateral cortical blindness when walking in a naturalistic, virtual environment. Under static, seated testing conditions, cortically blind subjects are known to exhibit compensatory eye movement strategies. However, they still complain of significant impairment in visual detection during navigation. To assess whether this is due to a change in compensatory eye movement strategy between sitting and walking, we measured eye and head movements in subjects asked to detect peripherally-presented, moving basketballs. When seated, cortically blind subjects detected ~80% of balls, while controls detected almost all balls. Seated blind subjects did not make larger head movements than controls, but they consistently biased their fixation distribution towards their blind hemifield. When walking, head movements were similar in the two groups, but the fixation bias decreased to the point that fixation distribution in cortically blind subjects became similar to that in controls - with one major exception: at the time of basketball appearance, walking controls looked primarily at the far ground, in upper quadrants of the virtual field of view; cortically blind subjects looked significantly more at the near ground, in lower quadrants of the virtual field. Cortically blind subjects detected only 58% of the balls when walking while controls detected ~90%. Thus, the adaptive gaze strategies adopted by cortically blind individuals as a compensation for their visual loss are strongest and most effective when seated and stationary. Walking significantly alters these gaze strategies in a way that seems to favor walking performance, but impairs peripheral target detection. It is possible that this impairment underlies the experienced difficulty of those with cortical blindness when navigating in real life.

1. INTRODUCTION

When navigating in naturalistic environments, the sequential acquisition and processing of visual information is influenced by many factors including task demands, heading direction (Warren, Kay, Zosh, Duchon & Sahuc, 2001), optic flow (Harris & Carre, 2001, Warren et al., 2001), and a division of attentional resources to salient/interesting targets (Broman, West, Munoz, Bandeen-Roche, Rubin & Turano, 2004). In turn, this information allows for correct orienting, speed control and obstacle avoidance (Patla, Niechwiej, Racco & Goodale, 2002), while maintaining the ability to monitor the environment for less predictable items of interest (Jovancevic, Sullivan & Hayhoe, 2006, Jovancevic-Misic & Hayhoe, 2009). What happens, then, if the size of the visual field is reduced in both eyes? Can afflicted individuals compensate effectively? Studies have shown that partially blind subjects can maintain their ability to interpret optic flow information by implementing compensatory eye movement strategies (Li, Peli & Warren, 2002). However, measures of sequential visual information acquisition (e.g. Bowers, Mandel, Goldstein & Peli, 2009, Vargas-Martin & Peli, 2006), are traditionally performed while seated, using two-dimensional (2D) experimental displays, which cannot accurately capture stimulus conditions or visually-guided behaviors in natural, three-dimensional (3D) environments. The development of 3D, immersive, virtual environments has allowed for more naturalistic experimental conditions, thus, eliciting behavior that closely mimics that found in the real world (Jovancevic-Misic & Hayhoe, 2009).

Partial cortical blindness, due to unilateral primary visual cortex damage, affects ~1% of humans older than 49 years of age (Gilhotra, Mitchell, Healey, Cumming & Currie, 2002, Zhang, Kedar, Lynn, Newman & Biousse, 2006a, Zhang, Kedar, Lynn, Newman & Biousse, 2006b). Because conscious vision is lost in the same portion of the visual field in each eye, this deficit is ideal to study visually guided function with a large but spatially consistent loss of visual information. When tested under standard (2D, static) laboratory conditions, subjects with unilateral cortical blindness display altered search strategies, ineffective scanning and abnormal saccade patterns (Hildebrandt, Giesselmann & Sachsenheimer, 1999, Ishiai, Furukawa & Tsukagoshi, 1987, Kerkhoff, 1999, Pambakian, Wooding, Patel, Morland, Kennard & Mannan, 2000, Zangemeister, Oechsner & Freksa, 1995, Zihl, 1995). They also fixate primarily into their blind hemifield (Ishiai, Furukawa & Tsukagoshi, 1987), a spontaneously developed compensatory strategy that allows more of the relevant visual information to be captured across the seeing hemifield (Gassel & Williams, 1963, Pambakian, Wooding, Patel, Morland, Kennard & Mannan, 2000). A recent study that looked at simulated driving behavior in the cortically blind found a significant decrease in detection rates of pedestrians appearing in the blind hemifield (Bowers et al., 2009). This result is troubling, but because the study did not record eye movements, we do not know if the failure to detect pedestrians was due to an absence of compensatory gaze strategies in the subjects tested or to other factors.

Given that the main problems reported by the cortically blind involve walking, driving and navigating (Warren et al., 2001), we asked: (1) whether compensatory gaze strategies seen in stationary, blind subjects are maintained when they physically walk in a naturalistic, 3D environment, and (2) whether these strategies are effective both for accurate navigation and for the detection of unexpected objects/events in the environment. By using an immersive, virtual reality system, participants could be tested either while seated and stationary, as well as when physically walking and interacting with the 3D naturalistic environment. This allowed us to assess changes in visual behavior induced by simply going from a seated to a walking condition, while maintaining an identical, controlled visual environment. Understanding how the cortically blind interact with their dynamic visual world and why they exhibit their worse functional deficits when navigating is essential for ultimately designing more effective rehabilitation strategies for this patient population.

2. METHODS

2.1 Subjects

Twelve adult subjects - seven males and five females, ranging in age from 50 to 83 years – were recruited into this study, which was approved by the University of Rochester’s Research Subjects Review Board. Experiments were conducted with the subjects’ informed, written consent and in compliance with the tenets of the Declaration of Helsinki.

Six of the subjects (CB2, 5, 6, 8–10) were tested 6 to 40 months after suffering unilateral visual cortical damage that caused homonymous visual field defects, except for one case in which the damage had occurred 19 years previously (see Table 1). All cortically blind subjects possessed large visual field losses in at least one quadrant of one visual hemifield in each eye, as determined using monocular Humphrey (24-2) visual fields (Figure 1) and Goldmann visual fields (Supplementary Figure 1). Magnetic resonance images of their brain showed all cortically blind subjects to possess damage to the primary visual cortex and/or optic radiations of one brain hemisphere (Supplementary Figure 2). Prior to enrollment in the study, each participant underwent thorough neurological and neuro-opthalmological examination by a licensed neurologist and neuro-ophthalmologist, respectively. This was done to document their stroke-induced deficits and to verify the absence of visual neglect and other significant medical, ocular or neurological irregularities that could interfere with the physical act of walking or with performance of the virtual reality task.

Table 1.

Subject Demographics

Subject	CB2	CB5	CB6	CB8	CB9	CB10	C1	C2	C3	C4	C5	C6
Gender	F	F	M	M	F	M	F	M	F	M	M	M
Age (years)	65	50	83	63	57	72	73	53	66	50	73	63
Handedness	R	R	R	R	R	R	R	R	R	R	L	R
Months since lesion	8	40	24	230	7	10	-	-	-	-	-	-
Affected hemifield	R	R	R	R	L	R	-	-	-	-	-	-
Mobility aids	N	N	N	N	N	N	N	N	N	N	N	N
Still driving	Y	Y	N	Y	N	N	Y	Y	Y	Y	Y	Y

M – Male; F- Female; Y – Yes; N – No; L – Left; R - Right

Figure 1.

24-2 Humphrey visual fields illustrating the visual field defects in each of the six cortically blind subjects tested in the present study. The dark regions indicate areas of deficit over the central 25° of vision in each eye. For an indication of the full extent of the defect over the entire visual field of each eye, refer to the Goldmann Visual fields in Supplementary Figure 1.

Six visually intact individuals (C1-6) were also recruited to serve as controls. All exhibited normal motor and visual performance and lacked any significant neurological history (Table 1).

2.2 Virtual reality (VR) testing

VR testing was performed using one of two head-mounted displays (HMD1 and HMD2), both from the same manufacturer (Virtual Research Systems Inc., Aptos, CA). The two HMDs were functionally equivalent, with no statistically significant differences in eye or head movements found between them within a given subject group (controls or CB - Supplementary Figure 3).

2.3 HMD1

Seven of the subjects (CB2, CB5, CB6, C3-6) were fitted with a Virtual Research V8 virtual reality HMD (Virtual Research Systems Inc., Aptos, CA) that immersed them in a virtual representation of Penn Railway Station (Figures 2 and 3). Virtual dimensions were geometrically identical to real world dimensions to diminish the occurrence of any virtual, motion-induced, visuo-motor conflicts. Head movements were tracked with a HiBall-3000™ Wide-Area High Precision Tracker (3^rd Tech Inc., Durham, NC), which measured both linear and angular motion and providing 6 degrees of freedom. The image was updated following each change in head position with a latency of 30–50 ms. Movements of the left eye were specifically monitored and recorded using a helmet-mounted ASL501 eye-tracker (infra-red bright pupil), sampling at 60 Hz (Applied Science Laboratories, Bedford, MA).

Figure 2. Virtual reality setup.

A. Picture of Virtual Research Systems VR 1280 head-mounted display with helmet-mounted infrared eye tracker and InertiaCube (HMD2) used to perform the present experiments. B. Video frame from the virtual Penn Railway Station environment in which a basketball is present and fixated by the subject. The subject’s fixation position is indicated by the black crosshairs superimposed on the video record of the scene. C. Origins of the basketballs presented in the present experiment, and plotted on a two-dimensional representation of a subject’s virtual field of view. Note the relatively even distribution of basketball origins in the eight octants of the virtual field of view.

Figure 3. Schematic diagram of L-shaped walking path.

Red dashed line represents a typical path walked between the four sawhorses. The remaining objects in the environment represent staircases and lampposts. The video images represent views of the virtual space as subjects approached each sawhorse.

The subjects’ center of gaze was represented by crosshairs output by the eye tracker and superimposed on video recordings of the virtual scene via a video mixing board. Visual displays were generated at a rate of 60 Hz by a Silicon Graphics computer and displayed in stereo on two helmet-mounted LCD screens with 640 × 480 pixel resolution and a 60° diagonal field of view (48 deg horizontal ×36 deg vertical dimensions). Visual and audio information were recorded at 30 Hz with a digital video recorder and then burned onto DVDs as mpg-2 files. Subjects were required to wear a lightweight backpack containing a battery, video multiplexer and camcorder designed to record the multiplexed video stream.

2.4 HMD2

Five of the subjects (CB8-10, C1-2) were fitted with a Virtual Research VR1280 virtual reality HMD (Virtual Research Systems Inc., Aptos, CA - Figure 2A) that immersed them in the same virtual environment as HMD1. Subject head orientation was tracked with an InertiaCube 2+ (InterSense Inc., Bedford, MA), which measured both linear and angular movements, and providing 3 degrees of freedom – roll, pitch, yaw. The position of the head in space was tracked by a Precision Position Tracker (PPT) system (World Viz LLC, Santa Barbara, CA), with a latency of 18 ms, an accuracy better than 0.5 cm, and which provided the additional 3 translational degrees of freedom.

Movements of the left eye were monitored and recorded using a helmet-mounted H-MIN6 eye-tracker (pupil-cornea reflection), sampling at 120 Hz (Applied Science Laboratories, Bedford, MA) (Figure 2A). Recorded data included both time and horizontal and vertical eye position in relation to the head (recorded at 60 Hz). Data from both the head and eye tracker were compiled into an output data file that included timestamp, eye position, head position, ball position, and ball diameter in the image plane recorded every 16 ms. In addition, each subjects’ center of gaze was represented by crosshairs output by the eye tracker and superimposed on video recordings of the virtual scene (Figure 2B) via a video mixing board.

Visual displays were generated at a rate of 60 Hz and displayed in stereo on two LCD screens with 1280 × 1024 pixel resolution and a 60° diagonal field of view. Visual information was recorded at 30 Hz with a digital video recorder.

2.5 Eye position calibration procedure

While wearing either HMD, the subjects’ eye position was calibrated to the virtual environment using a commercial calibration software (EyePos, ASL Laboratories) before testing for each condition, as well as every time the subjects removed or repositioned the HMD. Our criterion was that eye position should be accurate to 0.5 degrees of visual angle or better. During calibration, subjects were asked to precisely and sequentially fixate nine, black numbers displayed in a regular 3×3 array across a rectangular area 24×18° in the center of the virtual field of view. The numbers were displayed on a gray, uniform background. Each number was fixated for 3 seconds before the calibration software (EyePos, ASL Laboratories) was manually triggered to store the eye position with respect to the fixation location in the virtual field of view. Precision of the eye position with respect to the calibration grid of 9 numbers was also re-measured at the end of the sitting and walking conditions in order to verify that calibration accuracy was not lost during performance of the task. If the eye position deviated by >0.5deg from each letter, the data from that condition were thrown out, the subject was recalibrated and testing was repeated.

2.6 Visual Detection Task

Subjects were asked to visually detect and track flying basketballs as they appeared in the virtual world. During the task, basketballs were added to the virtual scene every 4 seconds. They appeared randomly from one of eight octants in the peripheral virtual field of view (Figure 2C). Balls originated at the edge of the field (~ 12 virtual meters from the subject) defined by the current head position. After appearing, the balls moved toward the center of the field over a 2 second period before disappearing at ~1 virtual meter away from the subject’s head. Balls were distributed so that ten balls appeared from each octant during the duration of a testing session, for a total of 80 balls/sitting or walking condition.

Subjects were asked to saccade to each basketball as soon as they detected it and then to visually track it until it disappeared. The task was performed while seated in a chair and allowed to freely gaze within the virtual environment (sitting condition), and while walking in an L-shaped path (Figure 3) while, again, being allowed to freely gaze around in the virtual environment (walking condition). No restraints were placed on gaze or head movements in either of the sitting or walking conditions. In the walking condition, four sawhorses were used to demarcate the walking path. The subjects were allowed several practice trials in each of the conditions in order to familiarize themselves with the virtual environment and the basketball detection task before data were recorded.

2.7 Data Analysis

Both video recordings and raw data output files were generated from each test session and exported for analysis. Video recordings were imported into iMovie and analyzed frame by frame to detect errors in eye tracking. An “error” was identified when the crosshairs were not present on the screen or when there were multiple crosshairs within a single video frame. Such frames and the associated raw data were excluded from further analysis. The data output from the eye and head tracker were then processed to extract information about basketball detection rates, the magnitude of head movements, and the horizontal distribution of fixations for both the sitting and walking conditions and path accuracy (in the walking condition only).

Basketball detection was defined as having occurred when the position of the eyes (crosshair) landed within 1° of visual angle from the nearest edge of the ball for more than 10% of the time that the ball was present on the screen (greater than 200 ms). This method decreased the chance that a ball might be classified as “detected” following only a brief, coincidental eye movement that happened to fall within 1° of the ball’s location.

In order to assess the impact of having a left or right blind hemifield on visually-guided behavior in the virtual environment, we first measured several fixation parameters when no basketballs were present in the virtual field of view, i.e. when subjects were simply gazing within the environment and waiting for the next ball appearance. Fixations were defined as the center of gaze remaining within 1°of a given location for a minimum duration of 100 ms. An average of 171 fixations were analyzed per condition (sitting versus walking) per subject. To test for the existence of a lateralization bias, as previously reported in the literature on cortically blind subjects, the distribution of fixations along the horizontal dimension of the virtual field of view was compared across experimental conditions (sitting versus walking) and experimental groups (cortically blind versus controls). We also measured fixation durations and the number of fixations performed over a task-dependent interval for each condition (specifically, over the first two minutes in the sitting condition and over the first five L-shaped paths completed in the walking condition).

Next, we categorized fixation location and the identity of items in the environment upon which fixations landed at each ball onset. Categories included sawhorse, lamppost, ground, wall, and “other” objects (murals, columns, stairs, windows, doors, etc.). Each category was also designated as “ahead” (within the central 20° of the frame), “to the side” (laterally outside the central 20° of the virtual field of view), upper and lower quadrants.

The magnitude of lateral head movements was estimated by calculating the standard deviation of the horizontal position shift of a feature/object in the virtual environment (e.g. the top left corner of a sawhorse) during a one-minute period. In the walking condition, movements of the environment due to a direction change (when turning right or left at a sawhorse) were excluded from the analysis; only movements occurring while walking straight towards a sawhorse were taken into account. It should also be noted that our measure of head movements includes both translational and rotational components of the head motion, which could not be disambiguated in our experimental setup.

Each person's "path efficiency" was also assessed during the walking condition. To do so, the position of the head in space was tracked and used to measure the path length walked by each subject (black lines in Figure 7D). We then expressed this path length as a percentage of the maximum possible path length, measured as the sum of the center-to-center distances between consecutive sawhorses (gray lines in Figure 7D). The higher this percentage, the more "efficient" the path walked (i.e. most similar to the total possible path length). We also measured the total time each subject stood still and the total number of completed paths over the entire duration of the task.

Figure 7. Effect of vision loss on fixation category parameters.

A. Percent of fixations made in each category over the duration of the sitting and walking tasks, separated into fixations made by controls and cortically blind (CB) subjects. B. Percent of fixations placed into the upper or lower quadrants of the visual field, as a function of subject group, across the sitting and walking conditions. Error bars = SEM, * = p<0.05.

Finally, as an additional verification of behavioral consistency within subject groups, we tested three of the controls (C3-5) on two different occasions, separated by 5.5 ± 0.9 months. All aspects of testing and analysis were identical in the two visits.

2.8 Statistical Analysis

The distribution of fixation and feature locations was analyzed on both an individual and group basis. At the group level, the mean and standard deviation of the horizontal component of fixation/feature position in the virtual field of view were analyzed with between-group Student’s t-tests and a repeated measures one-way analyses of variance (ANOVA), with condition (sitting versus walking) being the within-subject factor and group (control versus cortically blind) being the between-subject factor. Follow-up one-way ANOVAs were performed as necessary. T-tests were used to determine significance at the individual level. All inferential tests were two-tailed and p<0.05 was considered statistically significant.

3. RESULTS

3.1 Basketball detection

We contrasted the ability of CB and visually intact subjects to detect peripherally appearing basketballs while either sitting and stationary or physically walking an L-shaped path. Both cortical blindness and task condition (sitting/walking) influenced the ability to detect basketballs, with a significant main effects of condition (sitting versus walking - F(1,10)=27.87, p<0.001) and subject group (F(1,10)=42.92, p<0.001) on the percentage of basketballs detected relative to the number of balls that appeared (Figure 4A; see Table 2 for all descriptive statistics). Overall, CB individuals detected fewer basketballs than controls (sitting (p=0.004); walking (p=0.004)), and walking elicited a lower detection rate than sitting in both CB subjects (p=0.01) and controls (p=0.02). However, the lack of a significant interaction suggests that walking impairs basketball detection similarly in both subject groups.

Figure 4. Effect of vision loss on basketball detection.

A. Plot of the percentage of basketballs detected across the entire visual field by control and cortically blind (CB) subjects, as a function of condition (sitting or walking). B. Plot contrasting basketball detection rates for balls originating in the sighted and blind hemifields of cortically blind (CB) subjects. C. Basketball detection rates plotted separately for the favored and unfavored hemifields of controls. Error bars = SEM, * = p<0.05.

Table 2.

Descriptive statistics for gaze and performance parameters

	CONTROLS		CORTICALLY BLIND
	Sitting	Walking	Sitting	Walking
Parameter	Avg. ± SD	Avg. ± SD	Avg. ± SD	Avg. ± SD
Avg. fixation depth (deg)	1.1 ± 0.8	1.3 ± 1.2	4.7 ± 2.1	1.9 ± 0.4
Horiz. SD fixations (deg)	4.5 ± 1.7	6.7 ± 1.8	6.8 ± 1.1	5.3 ± 1.8
Horiz. SD head mvts (deg)	0.11 ± 0.08	0.73 ± 0.17	0.11 ± 0.03	0.84 ± 0.29
Avg. fixation durations (ms)
-Entire visual field	316 ± 12	158 ± 10	227 ± 19	161 ± 15
-Favored/blind hemifield	362 ± 35	157 ± 5	226 ± 26	155 ± 12
-Unfavored/sighted hemifield	248 ± 47	154 ± 25	220 ± 11	168 ± 12
Number of fixations (sitting/2 min; walking/5 paths)
-Favored/blind hemifield	89 ± 33	118 ± 35	211 ± 97	184 ± 48
-Unfavored/sighted hemifield	46 ± 19	95 ± 98	96 ± 111	65 ± 9
Number of completed paths (per walking condition)	-	10 ± 4.0	-	11 ± 6.4
Time standing still (sec)	-	20.4 ± 6.2	-	23.4 ± 14.1
Path efficiency (%)	-	69.5 ± 8.8	-	77.3 ± 4.5
Basketball detection (%)
-Entire visual field	99.1 ± 1.4	89 ± 6.5	80.3 ± 8.3	57.9 ± 13.2
-Favored/blind hemifield	100 ± 0	93 ± 10	48.8 ± 18.6	40.4 ± 19.9
-Unfavored/sighted hemifield	98.5 ± 2.3	85.2 ± 10.8	98.5 ± 1.8	76.5 ± 13.9

To assess whether detection performance of CB subjects was uniformly poor or different between the sighted and blind hemifields, we analyzed basketball detection according to the balls’ hemifield of origin. Seated CB individuals were able to detect 98.5 ± 1.8% of basketballs originating in their sighted hemifield, and 48.8 ± 18.6% of balls originating in their blind hemifield (Figure 4B). The difference was significant (p<0.001). Walking CB subject detected only 76.5 ± 13.9% of balls originating in their sighted hemifield, a significant decrease from their detection rate when seated (p<0.005). By the same token, they detected 40.4 ± 19.9% of balls originating in their blind hemifield when walking, which though lower, was not statistically different from their detection rate in the sitting condition.

To control for the effect of hemifield in age-matched control subjects, we contrasted basketball detection separately for the preferred and non-preferred hemifields (visual hemifields in which subjects consistently placed most or least fixations, respectively – see section 3.2 below for a quantitative definition). The detection rates are presented in Table 2 and hover between almost 100% in the sitting condition and 85–93% in the walking condition. An ANOVA revealed a main effect of condition (F(1,10)=12.8, p=0.005), but not group and no significant interaction, suggesting that detection performance in the favored and unfavored hemifields was decreased similarly when going from sitting to walking conditions (Figure 4C).

3.2 Fixation Distribution and Horizontal Head Movements

Given the drop in basketball detection rates between the sitting and walking condition across subjects, an interesting question was whether this was associated with a specific change in gaze strategy between the two conditions and groups. Prior studies had shown that stationary CB subjects bias their fixation distribution towards the blind hemifield (Ishiai et al., 1987; Pambakian et al., 2000). We also observed such a bias under our seated/stationary experimental condition. We then assessed whether the strength of this bias was altered by the act of performing a task while physically walking. By using the same virtual environment and basketball detection task in the seated and walking conditions, it was possible to isolate the effects of physically walking.

A raw plot of fixation locations across the virtual field of view showed that in controls, the majority of fixations landed around the vertical meridian in both the sitting and walking conditions (Figure 5A, C). However, each control displayed a slight preference for one hemifield over the other (Supplementary Figure 4A, C). The virtual hemifield in which the majority of fixations landed (the "favored" hemifield) differed between subjects and was unrelated to handedness (Table 1). However, it was consistent for a given subject across separate study visits (data not shown), across conditions (sitting versus walking – Supplementary Figure 4A, C) and following removal, replacement and recalibration of the HMD (data not shown). Although controls placed significantly more fixations into their favored hemifield in both the sitting and walking conditions (sitting p<0.001; walking p=0.009 – Supplementary Figure 4C), their fixations were generally centered close to the vertical midline. A fixation bias magnitude was computed for each subject as the average absolute horizontal distance of fixations from the midline of the virtual field of view (midline represented as 0°). Control bias magnitude hovered around 1° both when sitting and when walking.

Figure 5. Effect of vision loss on fixation and head movement parameters.

A. Plot of fixation distribution for one control participant under each of the two experimental task conditions tested – sitting and walking the L-shaped path. The vertical gray line in each graph denotes the vertical midline or meridian of the virtual field of view. Note the relatively well-centered distribution of fixations under both conditions in this subject, and the greater horizontal spread of fixations in the walking relative to the sitting condition. B. Plots of fixation distribution in one cortically blind participant, for each of the two experimental task conditions tested. The subject’s 24-2 Humphrey visual field composite (averaged across both eyes) appears in the upper right corner, with the black regions representing the visual deficit (<5dB of sensitivity) and light gray and white representing intact vision (~20–30dB sensitivity). Note the propensity of this cortically blind subject to place the majority of his fixations into the virtual hemifield corresponding to the side of his visual defect. In addition, there is a relatively broad horizontal distribution of fixations in the sitting condition, which is decreased in the walking condition. C. Magnitude of horizontal fixation bias represented as the average horizontal fixation position (depth) relative to the vertical meridian of the virtual field of view, plotted as a function of condition (sitting or walking) for the two subject groups (controls and cortically-blind). D. Plot of the average standard deviation (SD - an indication of spread) around the mean fixation location as a function of condition. E. Average horizontal standard deviation (SD) of head movements over a one-minute period, plotted as a function of condition. All error bars = SEM, * p<0.05.

CB subjects also had a “favored” hemifield – they placed most of their fixations in the virtual hemifield corresponding to the side of their blind hemifield of vision (Figure 5B, Supplementary Figures 4B, D). However, these fixations landed further away from the vertical midline than in controls, so that the strength or depth of the fixation bias into the blind hemifield when sitting was greatly exaggerated (4.7 ± 2.1°) relative to that in sitting controls (p=0.002, Figure 5C). Walking CB subjects also placed most of their fixations in the virtual hemifield corresponding to the side of their blind hemifield (p=0.002, Figure 5B, Supplementary Figures 4B, D). However, the depth of their bias was significantly decreased relative to the sitting condition (p=0.03), such that it was no longer significantly different from the average fixation depth of walking controls (Figure 5C). A repeated measures one way ANOVA confirmed this change in strategy on the part of CB subjects, revealing a significant main effect of group (control versus CB; F(1,10)=21.9, p=0.001) and a significant interaction between group and condition (F(1,10)=6.7, p=0.03) on the strength of fixation bias.

Aside from the fixation bias, other differences in fixation distribution emerged between controls and CB subjects. We measured and compared the average horizontal standard deviation of fixation locations in the two groups as a means of assessing fixation distribution spread (Figure 5D). A significant group×condition interaction (F(1,10)=9.06, p=0.01) was revealed. A follow-up one-way ANOVA suggested that the CB had a significantly broader fixation distribution than the controls when sitting (p=0.02), but not when walking. This suggests that seated CB subjects fixated over a greater width within the virtual field of view compared to seated controls. However, they maintained this fixation width when walking. In contrast, controls significantly widened their fixation distribution when going from sitting to walking (p=0.04), narrowing the difference between the two subject groups in the walking condition.

Overall, these fixation patterns suggest that when seated and performing a visual detection task in a naturalistic, 3D environment, CB subjects exhibit a large fixation bias towards their blind hemifield, coupled with a larger than normal horizontal spread of fixations. Controls have a smaller offset in average fixation distance from the vertical meridian towards their preferred hemifield and the width of their fixation distribution across the virtual field of view is significantly smaller than in CB subjects. However, when asked to interact with the 3D virtual environment by physically walking an L-shaped path, both subject groups changed their oculomotor strategies, albeit differently. While both groups continued to place most of their fixations into the same, preferred hemifield as when sitting, CB subjects reduced the depth of their fixation bias and decreased the width of their fixation distribution. Controls maintained a similar bias depth but increased the spread of their fixation distribution.

To assess whether changes in fixation strategies were associated with changes in head movements, we estimated the spread of horizontal head movements across groups. Head movements were generally small, with a standard deviation averaging just above 0.2° in the sitting condition and close to 1° in the walking condition, with no significant differences between controls and CB subjects (Figure 5E). Both groups showed a similar, significant increase in the standard deviation of horizontal head movements when going from the sitting to the walking condition (controls (p<0.001); CB (p<0.001); Figure 5E).

3.2 Fixation Durations and Numbers

When comparing average fixation durations across the entire visual field of CB and control subjects (Figure 6A), there were significant main effects of condition (F(1,10)=9.73, p=0.01) and group (F(1,10)=6.15, p=0.03). Overall, fixation durations were shorter when walking versus sitting, and CB subjects made shorter fixations than controls when sitting, but not walking. Generally, fixation duration was inversely related to number of fixations. We counted the number of fixations made during the first two minutes of the sitting condition and during the first 5 paths in the walking condition (Figure 6B, C). Note that the different time frames used to analyze fixation numbers in the two conditions only allowed for comparison across hemifields and groups within each condition rather than across conditions. When sitting, CB subjects made more fixations than controls over a 2 minutes period (Figure 6B). When walking, they made a similar number of fixations over the first 5 paths as controls (Figure 6C).

Figure 6. Effect of vision loss on fixation duration and number of fixations.

A. Plot of average fixation durations for control and cortically blind subjects as a function of condition (sitting or walking) across the entire visual field. B. Number of fixations in the sitting condition over a two-minute period for the control and cortically blind subjects. C. Number of fixations in the walking condition, counted over the first 5 paths completed by each subject group. Error bars = SEM, * = p<0.05.

3.5 Fixation Category

In addition to assessing fixation location while balls were absent from the field of view, we also assessed what items or features in the environment subjects were fixating at the time of each ball’s appearance. Overall, there were no significant differences between specific object categories - sawhorses, lampposts, ground, walls, and “other” objects (murals, columns, stairs, windows, doors, etc.) fixated by controls and CB subjects in the sitting and walking conditions (Figure 7A). Neither did fixation on any one object category at the time of ball onset predict whether a particular ball was detected or missed in both the sitting and walking conditions. What did predict ball detection in both conditions for CB subjects was fixation into the preferred hemifield at the time of ball appearance: 82±9% of the time when seated CB subjects detected a ball, they were fixating in the virtual hemifield on the side of their blind field at its onset. When walking, this value was 70±11%. For comparison, when sitting and walking controls detected a ball, they were fixating into their favored hemifield 64±16% and 59±19% of the time, respectively, which was not significantly different from chance.

When sitting, both subject groups fixated the wall in front of them most frequently (controls 58.2 ± 23.2%; CB 60.4 ± 24.3% of the time). When walking, the percentage of fixations on walls decreased significantly, replaced instead, by increased fixations on the sawhorses and the ground (Figure 7A). Of interest was the observation that CB subjects increased their fixations to the ground by ~20% more than controls, while lowering the incidence of their fixations on the “other” object category by ~30% (while controls did not – see Figure 7A). This suggests that in the walking condition, the direction of attentional resources to the ground in CB subjects was slightly stronger than in controls.

Additional analysis showed that both seated CB subjects and controls placed ~70% of their fixations into the upper quadrants of the virtual field of view and only ~30% of fixations into the lower quadrants, a significant difference (p<0.001, Figure 7B). There was only a significant main effect of quadrant (F(1,10)=8.532, p=0.02), suggesting that there were no differences between the two subject groups in terms of fixations in the upper versus lower quadrants. When walking, controls placed ~85% of their fixations in the upper quadrants and ~15% in the lower quadrants. In contrast, walking CB subjects placed significantly fewer fixations in the upper quadrants than controls (p=0.005), and significantly more fixations into their lower quadrants (p=0.005) (Figure 7B), supporting the prior observation that they were placing additional fixations on the ground relative to controls (Figure 7A). A significant interaction was also found (F(1,10)=12.97, p=0.005), suggesting that walking had a differential effect on each subject group.

3.6 Walking Path Performance

A one-way ANOVA revealed no significant differences between the CB and controls on the amount of time spent standing still (Figure 8A), number of completed paths walked (Figure 8B), or path efficacy (Figure 8C). Even visual inspection of the paths taken by participants in the two groups (Figure 8D) showed a lack of significant differences between controls and the CB. Oscillating “instabilities” were observed in all walking paths (Figure 8D) and were largely due to the natural swaying patterns generated by a walking individual shifting weight from one foot to the next. Importantly, there were no significant differences between walking styles or paths between our two experimental groups, suggesting that subjects were uniformly capable of fast, accurate navigation, and that they swayed to a similar extent.

Figure 8. Effect of vision loss on walking parameters.

A. Time spent standing still during the entire duration of the walking task. B. Number of completed circuits during the walking task. C. Path efficiency as a function of maximal distance between sawhorses. D. Schematic representation of the first 5 paths walked by a control and a cortically blind (CB) subject. The light gray lines denote ideal path measured from sawhorse center to center. The thin black lines denote the actual path walked by each subject. Note the slight waviness, likely due to body sway, and the high degree of similarity between the control and the cortically blind subject's performance.

3.7 Consistent visual behavior of controls across visits

Both ANOVA and within-condition Student’s t-tests revealed a lack of statistically significant differences between visits 1 and 2 for all parameters measured in three visually intact controls (Supplementary Figure 5). Specifically, there was no significant difference between visits for the side of the fixation bias (not shown), the absolute distance of fixation locations from the horizontal midline (a.k.a. magnitude of fixation bias: F(1,6)=2.898, p=0.149), the horizontal SD of fixation locations (F(1,6)=0.972, p=0.370, fixation durations (F(1,6)=1.261, p=0.312) and the horizontal SD of head movements (F(1,6)=1.124, p=0.338). The fact that all three controls exhibited such consistency between visits 1 and 2 for the parameters examined suggests that measures obtained from single study visits, which involved analyzing >100 fixations and head movements in each subject/condition, were sufficiently robust to yield significant, consistent results.

4. DISCUSSION

4.1 Detection of peripherally presented, moving objects when sitting and walking

Cortically blind subjects immersed in a naturalistic, dynamic, 3D virtual environment detected ~80% of peripherally-presented basketballs while seated and stationary, compared to almost 100% detection performance by age-matched, visually-intact controls. Balls appeared from randomly selected locations in the peripheral virtual field of view, in head-centered coordinates. The vast majority of the balls that were missed by the cortically blind originated from their blind hemifield, with a near perfect detection rate of balls appearing from their sighted hemifield. Thus, although CB subjects were free to move their eyes and head at will, and in spite of the fact that they displayed a distinct bias in fixation distribution towards their blind hemifield, this was not enough to prevent them from missing ~50% balls originating from that hemifield. Ambulation significantly reduced overall detection performance relative to the sitting condition in both subject groups. Interestingly, ambulation decreased detection rate for balls originating in the intact hemifield of CB subjects more than it decreased detection for balls originating in the blind hemifield, or in controls. All in all, our findings show that the act of walking impairs the ability to detect peripherally-presented objects relative to the sitting condition by about the same amount in both subject groups. However, it is equally important to note that CB subjects start off at ~20% lower level of performance than controls in the sitting condition. We have no real indication yet as to what level of detection performance on our task corresponds to what level of comfort navigating a busy street or shopping mall in reality. We posit here that 80% detection performance or better on our task (such as attained by sitting CB subjects and walking controls) corresponds to a reasonably good, easy and effective interaction with a busy, dynamic real-world environment. However, we also posit that a drop to 58% detection performance (in walking CB subjects) may mean missing enough peripheral information that dynamic interaction with a complex visual environment may become truly difficult.

In an attempt to better understand why walking CB subjects and controls were less able to detect peripherally presented objects that originated in an easy, head-centered coordinate system, we performed an analysis of fixation distribution in space, head movements and fixation category (i.e. what specific objects and features in the environment were being fixated at ball onset).

4.2 Fixation characteristics while seated

Seated, cortically blind subjects differed from normally sighted controls by placing the majority of fixations in the virtual hemifield corresponding to the side of their visual field defect. This behavior, previously reported for visual search and other simple tasks involving examination of 2D static images (Chedru, Leblanc & Lhermitte, 1973, Ishiai et al., 1987, Meienberg, Zangemeister, Rosenberg, Hoyt & Stark, 1981, Pambakian et al., 2000, Zangemeister et al., 1995), is thought to represent a compensation for the visual field defect, which places more visually-relevant information into the intact hemifield (Gassel & Williams, 1966, Ishiai et al., 1987, Pambakian et al., 2000). The six cortically blind subjects recruited for this study were tested at least 6 months after their brain injury, which is sufficient time for them to develop such a compensatory fixation strategy (Pambakian et al., 2000, Zihl, 1995). To test the possibility that the uneven, left-right distribution of fixations across the virtual field-of-view might represent an artifact of eye tracker calibration or uneven positioning of the virtual reality helmet on the head, we removed the HMD, replaced it, and recalibrated it multiple times during each testing session. Yet despite these adjustments, subjects maintained the side of their bias across testing conditions. In the case of three controls, the side of the bias was also maintained across different testing sessions, performed several months apart. Such consistency would not be present if poor eye tracker calibration or HMD position was biasing fixation distribution across the virtual field-of-view.

In addition to placing most of their fixations in one or the other hemifield, when sitting, the cortically blind had wider horizontal fixation distributions and their fixations were centered deeper into their preferred hemifield than in controls. Wider fixation distributions probably resulted from increased eye movements in these individuals, as previously reported (Ishiai et al., 1987, Pambakian et al., 2000, Zihl, 1995) and may have allowed them to maintain relative efficacy in performing the detection task.

Since saccades towards each ball were used as a measure of “detection”, an important consideration was whether saccades are impaired in this patient population. However, previous studies, including one from our own laboratory (using some of the same subjects tested here), showed cortical blindness not to alter saccade dynamics during visual search (Martin, Riley, Kelly, Hayhoe & Huxlin, 2007, Zangemeister, Oechsner & Freksa, 1995). Our subjects displayed similar eye movement patterns in between basketball appearances, hovering around a general, relatively central (in controls) or slightly eccentric (in CB subjects) area around the middle of the virtual field of view (Supplementary Figure 6). Neither group seemed to make systematic, scanning eye movements across the virtual field of view in search of the next basketball. Instead, they appeared to “wait” for the balls to appear, from a relatively central position.

The fact that cortically blind subjects missed about 50% of balls originating in their blind hemifield in spite of increased fixations towards that hemifield is interesting. One strategy that could have allowed for better detection performance would have been for them to place and maintain fixation at the extreme edge of the virtual field of view corresponding to the side of their blind field. None of the subjects employed such a strategy. In addition, cortically blind subjects have slower reaction times and miss more targets during visual search than normally sighted individuals (Zihl, 1995), even when given an unlimited amount of time (Hildebrandt, Giesselmann & Sachsenheimer, 1999). Thus it is likely that the shorter “effective” stimulus presentation time for balls appearing in the blind field contributed to decreased detection rates.

4.3 Fixation characteristics and navigation accuracy when walking

Ambulation significantly reduced the differences in fixation parameters (horizontal distribution, duration, number) observed between the two subject groups in the sitting condition. Specifically, the strength of the fixation bias exhibited by cortically blind subjects while seated was not maintained when walking. Ambulation did not alter the horizontal spread of fixations in the cortically blind, but elicited a significant broadening in the controls’ distribution. Given that their eye movement strategies while stationary allowed for relatively good peripheral detection performance, one might ask why cortically blind subjects did not maintain the same strategy, or adopt a more exaggerated gaze strategies (in the case of the horizontal spread), while walking. A possible explanation is that a more extreme bias would mean a greater deviation of average eye position in the orbit. Normal subjects typically adjust their head to keep the eyes centered in the orbit (Pelz, Hayhoe & Loeber, 2001) or at least within a desired range of orbital eccentricities - see "customary ocular motor range" (COMR) described by Stahl (Stahl, 1999, Stahl, 2001). Thus, the visual system might impose limits on fixation behavior, probably because more extreme eye positions would mean using more peripheral, lower-acuity regions of the visual field to perform the task. A second possibility is that subjects may have reached their visual system's limit for processing information effectively (and rapidly) from a broad area of space. It has been suggested that humans with peripheral field loss also have difficulty updating and/or maintaining spatial relationships with objects in the environment when walking (Fortenbaugh, Hicks & Turano, 2008, Turano, Yu, Hao & Hicks, 2005). Subjects either choose not to make the head/eye movements that could assist them in navigation, or when they do, it interferes with updating of their spatial representation of the environment (Turano et al., 2005).

At the same time, walking does indeed place additional demands on the visual system that could limit the ability to compensate for visual field loss by more biased oculomotor strategies. For instance, when walking, the allocation of attentional resources is divided among many factors including task goals, heading direction, interpretation of optic flow, and obstacle avoidance (Warren et al., 2001). The attentional field of view has been shown to be decreased by visual loss and impairment (Bowers et al., 2009, Hassan, Turano, Munoz, Munro, Roche & West, 2008, Leat & Lovie-Kitchin, 2006). It is likely that cortically blind subjects have smaller attentional fields than controls. Perhaps, instead of dividing attention equally between basketball detection and proper navigation of the L-shaped path, they might preferentially allocate attention to task components with the greatest risk, the most perceived benefit and/or the most reliable information (Araujo, Kowler & Pavel, 2001, Grasso, Prevost, Ivanenko & Berthoz, 1998, Jovancevic et al., 2006, Jovancevic-Misic & Hayhoe, 2009, Miyasike-Dasilva, Allard & Mcilroy, 2010). In our paradigm, our two main task components (walking and detecting basketballs) differed in terms of their visual "reliability" in world coordinates. Basketballs originated from random (i.e. unreliable) locations in the virtual field's periphery, which moved along with the person (specifically his/her head). The walking path, on the other hand, was clearly delineated by four sawhorses, which remained visually and spatially constant (i.e. reliable) within a particular area of the virtual world during the entire session. Our results show that cortically blind subjects were as fast and accurate at navigating the L-shaped path as controls, but while doing so, they became more impaired at detecting basketballs, particularly those originating in their intact hemifield. Fixation category analysis confirmed that they looked significantly more at the ground and in lower quadrants of the virtual field of view than controls. This suggests that they were indeed directing most of their limited attentional resources towards path-associated visual cues important for walking accurately, rather than adopting strategies that would have allowed them to maximize basketball detection.

4.4 Fixation category analysis

To better understand the impact of our subjects’ fixation distribution on ball detection versus the ability to navigate accurately, we assessed what specific objects or features in the environment were fixated by each subject at the time of ball onset, and what general region of space these objects/features were located in. In the vertical dimension, there was no significant difference in fixation targeting between the two subject groups in the sitting condition. Both primarily looked in the upper two quadrants of the virtual field of view, at the wall and “other” objects (murals, columns, stairs, windows, doors) in front of them. However, in the horizontal dimension, cortically blind subjects were more likely to detect a ball if they were fixating in the virtual hemifield corresponding to the side of their blind field. This lends support to the notion that the fixation bias towards the blind hemifield of vision is a beneficial, compensatory strategy for the cortically blind.

When walking, control subjects maintained their fixation preference for the upper quadrants of the virtual field of view, but they switched from primarily targeting the wall to primarily targeting the ground. A focus on upper field quadrants while targeting the ground can most likely be explained if the subject was looking “ahead”, in the direction of heading. It has indeed been reported that when walking, gaze is generally aligned with the direction of heading, but falls several steps ahead of one’s current position (Grasso, Prevost, Ivanenko & Berthoz, 1998, Hollands, Patla & Vickers, 2002).

Walking cortically blind subjects also targeted the ground with their fixations, but they did so more frequently than controls, and at the expense of looking at “other” objects. Consistent with this behavior, they directed more of their fixations to the lower quadrants (and less to the upper quadrants) than controls. Thus, cortically blind subjects appeared to look at the ground more directly in front of them. The use of peripheral vision has been often implicated in the visual control of walking including climbing stairs (Miyasike-Dasilva, Allard & McIlroy, 2010), monitoring foot trajectory (Marigold & Patla, 2008, Timmis, Bennett & Buckley, 2009), and gait speed (Marigold & Patla, 2008). Studies have found that the lower visual fields are critical for maintaining stable foot placement and monitoring the terrain (Marigold & Patla, 2008). Perhaps the reduction in peripheral vision in our cortically blind subjects required them to monitor characteristics of the path and the environment more carefully and in a more restricted manner, closer to each individual foot placement. This would imply an additional draw on attentional resources that may have contributed to the blind subjects’ decreased ability to detect peripherally presented basketballs when walking.

4.5 Head movements do not compensate for visual field defects

In addition to limits the visual system might impose on ocular eccentricity, we also noted that cortically blind subjects, who function without a large [peripheral] portion of their visual field in both eyes, compensated for this loss by primarily changing their horizontal fixation distribution (within limits discussed above), but not by making larger, lateral head movements. This is somewhat surprising in view of prior experimental results with humans and monkeys fitted with goggles that restricted their useful field of view (Crawford & Guitton, 1997, Gauthier, Obrecht, Pedrono, Vercher & Stark, 1987, Sandor & Leger, 1991). These subjects altered head movements to compensate for the reduced field of view. So why do we not see this in cortically blind subjects performing our task? There are several possible reasons: (1) all of the basketballs appeared from within the virtual field of view in a head-centered coordinate frame, within a small field of view (48×36°), which should have reduced the need to make large head movements in order to detect a basketball; (2) the useful fields-of-view in prior, simulation studies (Crawford & Guitton, 1997, Gauthier et al., 1987, Sandor & Leger, 1991) were significantly smaller than the residual fields-of-view of our subjects. Perhaps a certain decrease in field size needs to occur before compensatory head movements are invoked; (3) prior simulation studies did not use gaze-contingent methods to decrease field-of-view. As such, head movements may have been the only means by which subjects could reach the required targets; (4) the precision of visually-guided behavior, and even ocular kinematics, are greatest when the eyes are centered in the orbit (Biguer, Jeannerod & Prablanc, 1985, Stahl, 2001). In order for cortically blind subjects to navigate the walking path correctly with a laterally positioned head, they would be forced to make extreme eye deviations. As discussed earlier, this would impair performance due to decreased peripheral acuity; (5) the head is "heavy", and is made even heavier by wearing a virtual reality helmet. Even without that, the head is slower and requires more energy to move than the eyes in the orbit. Ballard and colleagues showed in a block-copying task (Ballard, Hayhoe & Pelz, 1995), that head movements are so costly that subjects will use working memory rather than make large head movements; and finally (6) we inferred head movements. Instead of measuring rotational head movement directly, we extracted the information based on the lateral movement of objects in the environment via the video recordings. Admittedly, this motion could be a consequence of either translational or rotational motion of the head, and when walking, a large component of it could be explained by body sway. However, it is unlikely that this metric would underestimate head motion, considering that the virtual world (and, hence, the video frame) is head-centered, and moved with the head.

4.7 Conclusions

The present experiments examined gaze characteristics and visual performance in cortically blind subjects immersed in a dynamic, 3D virtual environment. While sitting, these subjects displayed some of the same compensatory visual behaviors previously described under traditional, 2D testing conditions. Specifically, they placed most of their fixations over a broad area, relatively deep in the virtual hemifield on the side of their blind field. This behavior allowed them to detect close to 80% of peripherally-presented basketballs. Although walking, cortically blind subjects still targeted most of their fixations towards their blind field, other aspects of their bias were eliminated and they switched to directing their gaze primarily at the ground within lower quadrants of the field of view. This suggests a switch in focus from detecting basketballs to concentrating on the near walking path. Indeed, this was associated with a significant impairment in peripheral detection performance for balls appearing in both the intact and blind hemifields, coupled with normal and accurate navigation of the L-shaped path. Since controls also showed a decrease in basketball detection rate while walking, it seemed likely that walking placed an additional attentional load on all subjects. However, the differential gaze targeting to the lower hemifield in cortically blind subjects was likely responsible for a critical drop in their ball detection rate below a threshold that may correspond to significant impairement and difficulty when navigating in real life. A next step in this research will be to quantify the relationship between performance in our small, virtual field of view and the larger field of view present in real world situations. However, the results obtained presently already suggest a possible explanation for the visual dysfunctions reported by cortically blind subjects: perhaps altering (by training) some of the gaze strategies developed spontaneously by this patient population could improve their peripheral visual detection performance during navigation-intensive tasks such as walking and driving.