The Preferred Retinal Locus Used to Watch Videos

Abstract

Purpose

Eccentric viewing is a common strategy used by people with central vision loss (CVL) to direct the eye such that the image falls onto functioning peripheral retina, known as the preferred retinal locus (PRL). It has been long acknowledged that we do not know whether the PRL used in a fixation test is also used when performing tasks. We present an innovative method to determine whether the same PRL observed during a fixation task was used to watch videos and whether poor resolution affects gaze location.

Methods

The gaze of a group of 60 normal vision (NV) observers was used to define a democratic center of interest (COI) of video clips from movies and television. For each CVL participant (N = 20), we computed the gaze offsets from the COI across the video clips. The distribution of gaze offsets of the NV participants was used to define the limits of NV behavior. If the gaze offset was within this 95% degree confidence interval, we presumed that the same PRL was used for fixation and video watching. Another 15 NV participants watched the video clips with various levels of defocus blur.

Results

CVL participants had wider gaze-offset distributions than NV participants (P < 0.001). Gaze offsets of 18/20 CVL participants were outside the NV confidence interval. Further, none of the 15 NV participants watching the same videos with spherical defocus blur had a gaze offset that was decentered (outside the NV confidence interval), suggesting that resolution was not the problem.

Conclusions

This indicates that many CVL participants were using a PRL to view videos that differed from that found with a fixation task and that it was not caused by poor resolution alone. The relationship between these locations needs further investigation.

The visual system may adjust to a loss of foveal vision by using a functioning peripheral area of the retina to perform visual tasks that the nonfunctioning fovea would normally accomplish.¹ This pseudofovea is known as the preferred retinal locus, or PRL.^2–4 Previous studies have shown that most people with bilateral macular disease use a single retinal area during fixation that is normally located near the edge of the central scotoma.^1,2,5,6 Typically, fixation tests are used to determine the location of the PRL. The problem that has been long acknowledged is that we cannot determine whether the PRL measured in a fixation test is also used when performing other tasks. There is some evidence of the PRL varying with luminance⁷ and multiple PRLs.^8–11 It is not clear whether these multiple PRL locations are truly separate or whether they are unrepeatable local increases in frequency caused by the stochastic nature of short measurement periods. Only a handful of people with truly separate PRLs, and PRLs that can be used at will, have been reported.^9,10,12 Sullivan and Walker¹³ found that the area used to fixate while pointing may be larger than the fixational PRL, be offset from the measured fixational PRL, and vary between eyes. Fixation is typically measured monocularly, but most visual tasks are performed binocularly; yet often, monocular PRLs are not in corresponding retinal locations.^14,15 Thus, comparing monocular fixation measurements to activities performed binocularly may be misleading. A method of measuring the binocular PRL has been reported,¹⁶ though the instrument is not widely available and its reported measurement error is smaller than might be expected (and is much less than that with a different method that we have not published).

Apart from the study of pointing,¹³ to our knowledge, there are no systematic approaches to identify the location of the PRL used while performing activities of daily living. Here we present an innovative method to determine whether the same PRL measured during a fixation task was used to watch videos. This approach rests on two assumptions: (1) that calibration of a gaze-tracking system, in which the participant is asked to look at a fixation target in multiple locations, determines the fixational PRL (i.e., participants are expected to use the same fixational PRL when fixating at each target during the process, so, once calibrated, the system tracks the fixational PRL), and since we calibrated with both eyes viewing, we track the binocular fixational PRL; and (2) that participants using a PRL to watch a video will look in similar locations to people with normal vision (NV). This democratic center-of-interest (COI) approach uses the tendency of people with NV to look at the same things most of the time in directed videos (i.e., those in which the presentation of the content was planned).^17,18 For our analysis, we assume that people with central vision loss (CVL) will do the same but with less ability or due to impaired vision, including reduced spatial resolution, contrast sensitivity, and impacts of crowding (identifying objects of interest), poor eye movement control (being able to direct the gaze to the target location), and unstable fixation (holding the gaze at the visual target).¹⁹ Also, people with CVL may look at slightly different aspects of objects of interest, for example, looking at external features of a face more than internal features.^20,21 Thus, the gaze of a person with CVL at an object might be offset from that of a person with NV. The direction of that offset is not likely to be consistent (systematic bias), as the external features are distributed around the face and are likely to have offsets that vary between objects. So, for our analysis, it may introduce a wider distribution of gaze locations, but not a bias direction.

In our first study, we measured the difference between the (binocular) gaze location and the (binocular) fixational PRL (as found by the gaze-tracking calibration process) in a group of people with CVL and a case-matched, NV, control group. We hypothesize that some participants with CVL will use the same PRL as that used in a fixation test while others would use a different PRL location. We anticipated high variability between participants due to individual differences that probably relate to the differences in the shape and location of the central scotomas.

Further, we set out to determine whether the reduced resolution experienced by people with CVL exclusively explains the fact that CVL participants often did not look in about the same place as the NV participants. To answer this question, in our second study, additional NV participants wore hyperopic defocus lenses of different powers to induce different optical blurs while they viewed the video clips. Blur induced by defocus and diffusive (translucent) lenses has been previously used to simulate impaired vision.^22–27 We examined whether they still located the COI despite the blur. We anticipated that the difference between the presumed video-watching PRL and the fixational PRL would not significantly change, if poor resolution was the cause of the gaze offset.

Methods

Gaze Data Collection

Participants were seated at a table 1 m from a 27-inch 16 × 9 display on which the clips were presented at 30 frames per second. At the participant viewing distance of 1 m, the display had a width of 33.4° of visual angle. A chin and brow rest was used during the experiments to reduce head movements. Data were collected at a 1000-Hz sampling rate using an EyeLink 1000 eye tracker (SR Research, Mississauga, ON, Canada). The clips were displayed and data collected with a MATLAB (Mathworks, Inc., Natick, MA, USA) program using the Psychophysics,²⁸ Video,²⁹ and EyeLink Toolboxes. At the beginning of the experiment, the eye tracker was calibrated using a 9-point calibration procedure with care taken over gaze transitions between calibration locations and the instruction to look at the target so that it could be seen. If the average error exceeded 1.5° during a 9-point test of the calibration, the calibration was performed again. At the beginning of each trial, participants were instructed to watch the stimulus “normally, as you would watch television or a movie program at home.” At the end of each clip, the participant was asked to describe the contents of the clip.³⁰ Participants watched one clip per trial. Although the gaze of one eye was tracked, we calibrated binocularly; therefore we were tracking the binocular fixational PRL (as described in more detail below).

Video Clips

As described previously,^30,31 there were 200 directed video clips, chosen to represent a range of genres and types of depicted activities. The genres included nature documentaries (e.g., BBC's Deep Blue, The March of the Penguins), cartoons (e.g., Shrek, Mulan), and dramas (e.g., Shakespeare in Love, Pay It Forward). The clips were 30 seconds long and were selected from parts of the films that had relatively few scene cuts, which was reflected in the average number of cuts per minute in our clips being 9, as compared to approximately 12 per minute in contemporary films.³² The clips included conversation, indoor and outdoor scenes, action sequences, and wordless scenes where the relevant content was primarily the facial expressions and body language of one or more actors. Clips were shown at the full width of the display (33.4°). As the clips had a variety of aspect ratios, the height varied, but all were seen centered vertically in the display, in a letterbox format.

Determination of Whether the Fovea Was Used to Fixate

The participant was considered to be using the fovea if the retinal location used to fixate a target when viewing monocularly was within the 95% confidence interval of a sample of 179 healthy eyes from three studies (Hu SY, et al. IOVS 1994;35:ARVO Abstract 1527).^33,34 That confidence interval represents the between-individual variations in the location of the fovea relative to the center of the optic nerve head (ONH). Fixation was recorded monocularly using a Nidek MP-1 (Nidek Technologies, Padova, Italy). The participants were asked to look at the red cross so that they could see it.³⁵ Their fixation location was recorded at 25 Hz for approximately 30 seconds. After removing fixations outside the 99% confidence interval (considered outliers), the monocular fixational PRL was defined as the mean location of the fixation distribution. Custom software developed in MATLAB plotted a best-fit ellipse to the margins of the ONH using points marked by the experimenter. The ONH center was defined as the center of this fitted ellipse. If the monocular fixational PRL was within the 95% confidence interval of the healthy eyes, then it was considered to be at the fovea. For Table 1, if either monocular fixational PRL was found to be at the fovea, the CVL participant was considered to be using foveal vision (though the quality of vision, such as visual acuity, was reduced compared to healthy eyes). However, even when the fovea was available in one or both eyes, this does not mean that the fovea was used for the task. For example, in geographic atrophy, it is not uncommon for a reader to use a PRL outside the atrophic ring rather than the intact fovea in a residual island of visual field, presumably because it provides a wider field of view, as reading letter by letter is tedious.

Table 1.

Some Demographic and Vision Characteristics of Participants in the CVL Group. The “Foveal Vision” Column Indicates Whether at Least One Eye Had a Monocular PRL That Was at the Fovea

Participants

In total, there were 20 participants with CVL and 75 participants with NV.

NV Control Group for Center-of-Interest Determination

The NV-control group consisted of 60 participants with NV who have been described previously.^30,31 Recruitment was stratified with three equally sized age groups: under 60 years, 60 to 70 years, and greater than 70 years, each with equal numbers of men and women. Each NV-control participant watched a different subset of 40 video clips from a set of 200 clips.

Study 1: Gaze Offset When Using a PRL

The CVL group consisted of 20 participants with CVL (median 63.5; range, 29–87 years) from the community in and near Boston, Massachusetts. Vision characteristics of the CVL group are reported in Table 1. One CVL participant, patient P17, had normal single-letter visual acuity (20/19), but substantial problems with functional vision from multiple, small scotomas from parafoveal retinal lesions. Participants in the CVL group watched a subset of 20 video clips. The NV-match group consisted of 20 of the 60 NV-control participants, who were selected to approximately match the ages of the 20 CVL participants (median 62.5; range, 27–85 years).

Study 2: Gaze Offset When Vision Was Blurred

The NV-defocus group consisted of 15 additional NV participants (median 29; range, 21–67 years) who were not members of the NV-control group. This group has been reported previously.³¹ They watched the same subset of 20 video clips seen by the CVL group wearing varying levels of spherical defocus lenses to produce optical blur while watching the same clips as described previously. There were five levels with lenses selected for each participant, ranging from 0 to +9 diopters, to produce visual acuities 20/16, 20/50, 20/125, 20/320, and 20/800 at the 1-m viewing distance. Some demographics for the three groups are shown in Table 2.

Table 2.

Demographic Characteristics of Participants in Each Group

The Institutional Review Board of the Schepens Eye Research Institute approved all studies. The research followed the tenets of the Declaration of Helsinki. Informed consent was obtained from each participant prior to data collection, and they received a vision assessment and a cognitive assessment. All participants had a Montreal Cognitive Assessment^36,37 score of 17 or better. Apart from the NV-defocus group, participants were shown the clips wearing habitual, not necessarily optimal, optical correction. The NV-defocus group had an optimal correction for the viewing distance and positive lenses adjusted to obtain the required visual acuities at the 1-m viewing distance.

COI Determination

We randomized and assigned different video clips to each NV-control participant. Overall, each of the 200 video clips was viewed by 12 NV-control participants. For each video clip, we removed blinks, saccades, and other lost data, which should leave fixations and pursuits. For each frame (33 ms), for each participant, the gaze position data points (0–33) were averaged. While the gaze of all participants is often in one location (Fig. 1A), it can be distributed across more than one location, such as in scenes when there are two people speaking. Thus, as described in more detail previously,³⁸ we used a novel method to determine the democratic COI. First, for each frame in each clip, for all the available data from all NV-control participants (up to 12), a kernel density estimate of the average gaze positions was computed (Fig. 1B). Then, we integrated the area under the region of the density estimate for all potential positions of a rectangular box across the frame interpolating with a symmetrical Gaussian function. The democratic COI was defined as the location of the center of the box with the highest integral value (Fig. 1B). The benefit of this approach over averaging or taking the median of the gaze points is that it better accounts for multimodal gaze distributions (Fig. 1C).

Figure 1.

Example of democratic COI determination. (A) Gaze locations of the 24 NV-control participants during one frame of one video clip. (B) Kernel density estimate of those gaze locations in that video frame shown as a heat map, with red representing a higher density. The dashed rectangle represents the box used to determine the democratic COI. (C) Kernel density estimate with a bimodal gaze distribution and the placement of the democratic COI.

Gaze-Offset Determination

For each participant, for each frame in each video clip, we calculated the gaze offset from the democratic COI position as the difference, in degrees, between the gaze coordinates and the democratic COI coordinates. CVL and NV-defocus participants were compared to the democratic COI of the NV-control group. For each NV-match participant, since they were members of the NV-control group, the democratic COI was recalculated without including gaze data from that particular participant. The gaze-offset distance was calculated as the length of the vector from the democratic COI, whose origin was defined as (0, 0), for both horizontal and vertical gaze offsets, such that:

The spread of the gaze-offset distribution was measured for each clip for each participant using the bivariate contour ellipse area (BCEA),³⁹ which has been used to measure the spread of fixation data.⁴⁰

Thus, if a participant always looked at the democratic COI, then the offset distribution would have no spread and would be located at the origin. Since NV viewers tend to look in the same place most of the time (Fig. 1A),^17,18 the offset distributions of NV-match participants were expected to be located near the origin and to have small spreads. People with CVL have poor fixation quality and eye movement control.^23,41 Thus, we expected the offset distributions to have a larger spread than that of NV-match participants. If the CVL participant used the fixation PRL to look at the democratic COI, we expected the distribution to be centered at the origin, whereas if another retinal location was used, a video-watching PRL, we expected the distribution to have its center away from the origin.

If a CVL participant was unable to locate the democratic COI or chose to look at other objects, then the distribution should be wide and centered: centered because, in directed video clips, on average, gaze is near the center of the video image (see Fig. 6 in Ref. 18). To simulate this, for each NV-match participant we used the scanpath from one video clip to determine the gaze offset when compared to the democratic COI of a different video clip. In this case, the viewer's scanpath and the democratic COI were not related. As an illustration, the gaze-offset distribution for one of the 20 is shown in Figure 2. In that simulation, the distribution center was at (0.01°, 0.22°) and the BCEA was 14.2 deg². For all 20 simulations, the average location of the center of the gaze-offset distributions was (0.8°, 0.4°) with gaze offsets varying between 1.4° and 2.9°. Thus, if a CVL participant followed a scanpath that differed from the NV-control group and used the fixational PRL to watch the video, the expectation is for a gaze-offset distribution that has its center near the origin (the democratic COI), with a 95% confidence limit of 3.0° based on our simulation. Thus, a gaze-offset distribution that has its center within 3.0° of the origin could result from having followed an alternative scanpath (i.e., not the democratic scanpath), but also could result from following the democratic COI.

Figure 2.

2-Dimensional histogram of gaze-offset distances for one NV-match participant when a COI for a different clip was used. This simulates the pattern expected when viewing a different COI from the NV-control group. The democratic COI from the NV-control group is located at (0, 0) and is indicated by the white cross.

To investigate whether a CVL participant with a gaze offset less than 3° was following an alternative video scanpath, we used the normalized scanpath salience (NSS).¹⁷ NSS measures the coherence between the viewer's video scanpath and that of the comparison group (here the democratic scanpath of the NV-match group). The above method used to determine the gaze-offset distributions for a simulated alternative scanpath using an unrelated scanpath was used to measure the NSS scores expected from using an alternative scanpath. For the 20 video clips using the unrelated scanpaths, the average NSS score was 0.77 and the upper 95% confidence limit was 2.03. Thus, if the NSS score was above 2.03 and the gaze-offset was small (<3°), then the viewer was probably following the democratic scanpath and not an alternative scanpath.

NV-defocus participants were expected to have a wider gaze-offset distributions than the NV-match participants, reflecting increased difficulty identifying objects of interest, and would remain centered if the NV-defocus participants were able to identify the objects of interest despite the blur. That could change with the amount of blur.

Results

Study 1: Gaze Offset When Using a PRL

We compared the distributions of gaze-offset distances between the three groups. NV-match participant distributions varied in BCEA (spread), but were all close to the COI (Supplementary Fig. S1). Their average BCEA was 4.61 (range, 2.7–8.6) deg² and the average gaze offset was 1.16° (range, 0.8°–1.7°). These gaze offsets may represent gaze calibration errors. The maximum error was consistent with our calibration criterion of less than 1.5° error (before data were collected).

As shown in Figure 3, CVL participants had wide distributions, and many CVL participants had distributions that are clearly not centered, being shifted away from the COI, which corresponds to the fixational PRL. The distributions that were shifted away from the COI indicated that the participant was using a different PRL to view the video clips than when looking at a fixation target. The BCEA of the CVL group (average 15.2, range, 2.3–33 deg²) was larger than for the NV-match group (mixed-effects regression, z = 3.81, P < 0.001). The average gaze offset varied between individuals and was usually larger than for the age-matched NV participant (Fig. 4). The average gaze-offset distances of the CVL group were larger than the NV-match group (z = 5.14, P < 0.001; Fig. 5B).

Figure 3.

2-Dimensional histogram of gaze-offset distances in the 20 participants with CVL. Each part of the figure represents the distribution of gaze-offset distances for one CVL participant. The white crosses represent the location of the democratic COI.

Figure 4.

Comparison between average gaze-offset distance in CVL participants and average gaze-offset distance in age-paired NV participants. Error bars represent standard deviations across the clips. A gaze-offset distribution that has its center within 3.0° (represented with a gray dotted line) of the origin could result from having followed an alternative scanpath (i.e., not the democratic scanpath) or from following the democratic COI. Only P2 had a low NSS score, so may have followed an alternative scanpath.

Figure 5.

Gaze offset and gaze-offset distance comparatives. (A) Scatterplot of individual gaze offsets of participants in the three groups. Inset shows the magnified central region. The ellipse represents the 95% confidence interval of a bivariate normal distribution fit to the gaze offsets of the 20 NV-match participants. (B) Comparison of average gaze-offset distance among the three groups. The CVL group showed a much larger gaze-offset distance than the other two groups (P < 0.001). Error bars represent 95% confidence interval across participants (N = 20 for CVL and NV-match groups; N = 15 for NV-defocus group).

Four CVL participants were considered to be using their fovea in at least one eye (Table 1: P2, P4, P5, and P17). Their BCEA was relatively smaller than that of the other CVL participants (z = −1.71; P = 0.09). This was particularly clear for P4 and P5, who had “central,” “small” gaze-offset distributions (Figs. 3, 5). P2 had a gaze offset that was very close to the 95% confidence interval for the NV-match group (Fig. 5). P17 had a gaze-offset distribution that was not centered (Figs. 3, 5). P17 had macular scarring in both eyes that produced multiple, small, central scotomas, so a gaze offset might have been made to obtain a more open view of objects of interest. In this small sample, there was a trend for higher binocular letter CS scores to be related to smaller BCEA values (z = 1.93; P = 0.054) and smaller gaze offset (z = 1.79; P = 0.074). We found no relationship between the spread of the gaze offset and visual acuity, age, or sex.

Nine of the 20 subjects with CVL had an average gaze offset that was less than 3° (subjects P2, P3, P4, P5, P7, P8, P10, P11, and P14), and thus might have followed an alternative video scanpath rather than the democratic video scanpath (see analyses of unrelated video clips described above). Of those nine subjects, eight subjects had a NSS score that was above the 95% confidence limit of NSS scores (2.03) when using an unrelated scanpath, and thus were probably following the democratic scanpath. One subject, P2, had a low NSS score (0.9), and thus might have been using an alternative scanpath. P2 had geographic atrophy with central islands in each eye that included the fovea (visual acuity of 20/84 in the right eye and 20/34 in the left eye). While the foveas were used for fixation of small stimuli (e.g., letters during visual acuity measurement and fixation stimuli), it is likely that he used an alternative location for viewing video. Thus, we expected an offset gaze distribution. The lack of a gaze offset and a low NSS score could have resulted from the use of more than one video-watching PRL.

Study 2: Gaze Offset When Vision Was Blurred

When NV participants watched the videos with spherical-defocus blur, the gaze-offset distributions were very similar to the NV-match group (Supplementary Fig. S2). BCEA varied between blur levels (range, 7.5°–11.5°), being interestingly larger for the lower blur but not significantly different (z = −1.71, P = 0.087) (Supplementary Fig. S3), and smaller than for the CVL group (z = −2.61, P = 0.009). The gaze-offset distances did not vary significantly between blur levels (z = −1.63, P = 0.103). Further, the gaze-offset distances for the blur conditions were not different from the NV-match group (z = 0.33, P = 0.74; Fig. 5B). This indicates that the NV-blur participants were able to locate the objects of interest, and thus their gaze offsets, on average, did not differ from the COI.

The distribution of gaze offsets of the NV-match participants was used to define the limits of NV behavior. We calculated a bivariate normal distribution that defines an elliptical 95% confidence interval from the distribution of gaze offsets of the NV-match participants (see inset in Fig. 5A). If a gaze offset was within this 95% degree confidence interval, then the PRL used to watch video was not significantly different from that used for fixation. The largest average gaze-offset distance of a CVL participant was ∼13° (Fig. 4). Eighteen of the 20 CVL participants had gaze offsets that were outside the NV confidence interval (further from the COI than expected; see inset in Fig. 5A). None of the NV-defocus participants had a gaze offset that was outside the confidence interval.

Discussion

We present an innovative method to determine the location of the video-watching PRL relative to the binocular fixational PRL in a group of people with CVL and in two control groups. We hypothesized that some participants with CVL would use the same PRL as that used to fixate a simple target, while others would use a different PRL location to look at objects of interest in video clips. Consistent with previous studies,^17,18 NV-match participants had centered distributions (Supplementary Fig. S1), confirming that they looked in about the same place at about the same time as one another. In agreement with our hypothesis, many CVL participants were using a PRL to view videos that differed from that found with a fixation task (Fig. 3), since their gaze-offset distributions were shifted from the COI location (fixational PRL). One possible explanation for the large variability in the gaze-offset distributions of the CVL group may be the differences found within the visual acuity or contrast sensitivity of each participant (Table 1) and the variability of the locations of the PRL developed by each participant and the location of the binocular scotoma relative to the PRL. Our results are in agreement with previous studies that also found that vision impairments causing CVL do not have homogenous effects and there are large individual differences in functional adaptations to CVL.^42,43 Further questions include: Is there any relation between the locations of the video-watching PRL location and the fixational PRL? Are the participants with offset PRLs moving the PRL away from the scotoma border so that objects of interest are not obscured? We presume that the scotoma border determines the location of the movie-watching PRL. Future studies should calculate the binocular fixational PRL orientation and compare it to the gaze-offset orientation found using the gaze-tracking system. We hypothesize that the movie-watching PRL location is placed in the same orientation plane of the fixational PRL, away from the scotoma center.

Further, we examined whether reduced resolution could exclusively explain the lack of scanpath coherence found in people with CVL. We examined whether NV participants wearing defocus blur lenses located the COI despite the defocus blur vision. We anticipated that the difference between the presumed movie-watching PRL and the fixational PRL would not significantly change. As predicted, NV participants watching the same videos with spherical defocus blur lenses also produced uniform gaze-offset distributions (Supplementary Fig. S2), no different from that found with no blur (Supplementary Fig. S1). Our results suggest that reduced resolution does not cause the offset. Also, the spread of gaze offsets (BCEA) of the NV-defocus group were much smaller than those of the CVL group, showing that having a fovea and no central scotoma, even in the presence of reduced resolution, is not sufficient to produce the broad spread of gaze offsets, which presumably come from poor fixation control. Thus, the use of defocus lenses to simulate CVL viewing conditions failed to recreate the visual experience of people with CVL. Therefore, further studies should focus on examining and quantifying additional factors, such as oculomotor control patterns, for individuals with CVL and NV participants under similar conditions. Importantly, for simulations of CVL that include a central scotoma, we consider it essential to evaluate individuals with NV performing the visual tasks with simulated scotomas in a gaze-contingent paradigm, as blur is not enough. Recent studies have shown the benefit of training NV participants to develop a fixational PRL^44,45 and, promisingly, to develop oculomotor re-referencing. Interestingly, previous and current works in our laboratory have already contributed to improving the latency and accuracy for gaze-contingent systems by predicting the trajectory of saccadic eye movements (Wang SW, in press, 2017).^46,47

In summary, our results indicate that most CVL viewers do look in about the same place, as do NV participants when watching videos. Many of them typically use a PRL to view video that differs from that found with a fixation task. The relationship between these locations needs further investigation. Traditional methods to measure PRL should be replaced with alternative methods that adequately adapt to the different visual tasks performed by the participants, and should be performed with binocular viewing. Training and evaluating NV participants to develop PRL using accurate gaze-contingent systems is key to simulate impairing visual conditions realistically.