Loss of Central Vision and Audiovisual Speech Perception

Abstract

Communication impairments pose a major threat to an individual's quality of life. However, the impact of visual impairments on communication is not well understood, despite the important role that vision plays in the perception of speech. Here we present 2 experiments examining the impact of discrete central scotomas on speech perception. In the first experiment, 4 patients with central vision loss due to unilateral macular holes identified utterances with conflicting auditory-visual information, while simultaneously having their eye movements recorded. Each eye was tested individually. Three participants showed similar speech perception with both the impaired eye and the unaffected eye. For 1 participant, speech perception was disrupted by the scotoma because the participant did not shift gaze to avoid obscuring the talker's mouth with the scotoma. In the second experiment, 12 undergraduate students with gaze-contingent artificial scotomas (10 visual degrees in diameter) identified sentences in background noise. These larger scotomas disrupted speech perception, but some participants overcame this by adopting a gaze strategy whereby they shifted gaze to prevent obscuring important regions of the face such as the mouth. Participants who did not spontaneously adopt an adaptive gaze strategy did not learn to do so over the course of 5 days; however, participants who began with adaptive gaze strategies became more consistent in their gaze location. These findings confirm that peripheral vision is sufficient for perception of most visual information in speech, and suggest that training in gaze strategy may be worthwhile for individuals with communication deficits due to visual impairments.

INTRODUCTION

Speech processing is often thought of as an auditory phenomenon; however, vision also plays an integral role in perceiving speech information. For example, visual input can increase intelligibility when auditory information is degraded, such as in tasks where speech is mixed with background noise.¹ Visual information also influences perception under optimal listening conditions. When people are presented with conflicting auditory and visual information, for example, an auditory “aba” and a visual “aga,” they will often perceive a sound that is entirely different from the auditory stimulus (for example, “ada”). This is known as the McGurk effect.²

Understanding the connection between vision and speech perception has important implications for the aging population. Hearing impairments are present in 80% of individuals who have reached 80 years of age. In addition, 24% of those individuals will suffer from a significant visual impairment.³ The combination of hearing and visual loss in aging can make communication very difficult, with the result being that considerable time and effort is required to carry on basic daily activities. Hearing and vision loss are predictors of loneliness in the elderly^4-6 and decreased quality of life.^7,8

Research has suggested that peripheral vision may be sufficient to influence speech perception, and that foveal vision may not be necessary for much of the visual gain in speech tasks.^9,10 The successful use of peripheral vision for speech perception may depend in part on the gaze strategy adopted and the ability to place important visual information on an unimpaired area of the retina. Previous literature, using static fixation points or letters, has suggested that people with central vision loss will often adopt a preferred retinal locus (PRL), whereby they gaze eccentrically in order to fixate the desired visual object on a consistent peripheral retinal location.^11-15 This enables them to optimize their functional vision by placing objects on an unaffected retinal location.

The research presented here directly examines the link between visual impairment and speech perception, by assessing the performance of individuals with real or simulated central vision loss on speech perception tasks. The first experiment involves a group of 4 individuals with unilateral full thickness macular holes. These participants had small unilateral discrete scotomas concentric with the fovea. Here we explore whether peripheral vision can be sufficient for visual speech perception if the scotoma size and direction of gaze permit visibility of the talker. The second experiment involved a group of participants who experienced simulated vision loss by means of a gaze-contingent artificial central scotoma of 10 degrees. Here we determine how a group of normally sighted individuals respond to an artificial scotoma over a series of days. Our goals in these studies are to highlight the potential deficits in communication with central vision loss and to explore how the patient's fixation pattern can contribute to the loss and to rehabilitation.

EXPERIMENT 1

Introduction

Experiment 1 serves 2 purposes. First, it provides a strong test of the efficacy of peripheral vision for communication, since central vision is not available. Second, the experiment allows us to examine strategic responses to central vision loss. Patients actively control how they sample the visual world in response to their deficits, and some adaptations may be more effective than others.

Methods

Participants

All participants spoke English as their first language. Two groups of participants were tested in this experiment: a patient group consisting of 4 patients who had unilateral macular holes (see Table 1) and a control group of 12 undergraduate students (4 males, 8 females, mean age = 21.08, range = 17–31). All macular holes were confirmed as Stage IV macular holes with ocular coherence tomography.

TABLE 1.

Clinical characteristics of the patients included in the study

Participant	Age	Gender	Acuity (RE)	Acuity (LE)	Scotoma diameter (approx)	Affected eye
P1	77	M	6/6	6/120	< 2°a	LEFT
P3	65	F	6/90	6/6	< 2°a	RIGHT
P2	77	F	6/60	6/7.5	4°	RIGHT
P4	74	F	6/6	6/90	7°	LEFT

^aParticipants 1 and 3 did not perceive a central region of loss, which suggests that they were filling in for their loss of photoreceptors due to their macular holes.

The approximate diameter of the scotoma was assessed using Amsler grid tests; however, scotoma size could not be assessed for 2 of the participants using this method because their scotomas were quite small. We classified these participants as having scotomas less than 2 degrees in diameter.

Hearing status varied across subjects. However, each subject served as their own control, with performance compared for the 2 eyes. Of the 4 participants, 3 self-reported normal hearing, and 1 wore bilateral hearing aids during the experiment to correct for his hearing loss.

Stimuli

The stimuli were created from 8 vowel-consonant-vowel (VCV) utterances (/aba/, /ada/, /aga/, /ava/, /atha/, /ala/, /azha/, and /ara/), spoken by a single fluent male speaker of English and recorded on digital videotape. The video clips were converted to gray scale and recorded to DVD.

The utterances were presented in congruent and incongruent forms. In the congruent form, the 8 utterances were played normally, and the auditory matched the visual consonants. In the incongruent form, an auditory /aba/ was dubbed onto 4 of the visual utterances (/ada/, /ava/, /atha /, and /ala), maintaining the timing with the visual stimulus of the original sound track. The 4 visual utterances were chosen based on their ability to elicit the strongest McGurk effect in a previous study (that is, the visual stimulus modified the way the auditory stimulus was heard).¹⁶

Equipment

Participants were tested in a sound isolation booth. They were seated with their head stabilized in a chin rest with their eyes 57 cm from a 20-inch video monitor (JVC Model TM-H1950G). The stimuli were played with a Pioneer DVD player, Model V7400. The DVD trials were controlled by custom software, which also recorded the participants' responses. Responses were made on a keyboard with 8 possible key responses corresponding to each utterance. The auditory signal was amplified using a R300 reference amplifier (interM Corp, South Korea) and played through Studio 20 v.3 speakers (Paradigm Electronics Inc, Toronto, ON, Canada) located on each side of the screen. Participants entered responses on a keyboard.

Monocular occlusion was controlled by having the participants wear PLATO goggles (Translucent Technologies, Toronto, ON, Canada) with computer-controlled liquid-crystal shutter lenses. These lenses are designed to be either opaque or clear depending on the voltage passed across the liquid-crystal lens. In the opaque condition, light is scattered in order to obscure the eyes while maintaining constant illumination.

Gaze position was recorded using the EyeLink II system (SR Research Ltd, Mississauga, Canada), which simultaneously records a participant's eye position and head position with a headband-mounted camera system, using calibration, validation, and drift correction parameters outlined in Buchan, Paré, and Munhall.¹⁷

Design and Procedure

Participants first completed a general practice, giving them a chance to become familiar with the task and the monocular viewing. Following this, participants completed 5 blocks: a practice block of the 8 congruent stimuli and 4 experimental blocks of 16 trials (8 congruent stimuli and 2 repetitions of 4 incongruent stimuli). The trials were presented randomly within each block. Participants were told to watch the screen and to press the key corresponding to the consonant they heard.

While the video played, participants had 1 lens obscured and 1 lens clear. During the practice block, 1 lens was obscured for the first 4 trials and the other was obscured for the last 4 trials. For the experimental blocks, 1 lens was obscured for the first and fourth block, and the other was obscured for the second and third block. The order in which the unaffected eye was obscured was counterbalanced across participants. For the undergraduates, the order in which their right and left eye was obscured was counterbalanced in a similar manner. Between trials, participants had both lenses clear to see the keyboard and to perform a drift correction to maintain the eye tracking calibration.

Eye-Tracking Analysis

To compare the fixation locations for the affected and unaffected eyes, the data were analyzed in 2 ways. The first analysis was a qualitative one, using figures with the fixations for the unaffected eye and affected eye over-laid on a still image of the talker. The second analysis was used to infer whether participants might be adopting a PRL. The position of fixation during viewing by the affected eye relative to the position of fixation during viewing by the unaffected eye was calculated. It was assumed that the region where participants gazed with their intact eye was also the region participants would place in a preferred retinal locus with their affected eye. The center of the fixations during viewing with the unaffected eye was calculated by taking the average location of all the samples that occurred within fixations. Using this center point, vectors were calculated for every gaze sample within fixations when participants used their affected eye. The orientation of each vector relative to the unaffected eye center point was calculated. All the eye-tracking data was collected only during the period of time when the talker was speaking, and fixations which were not on the screen were not included in the analysis.

Results

Behavioral Data

Performance on the congruent trials was very high for all of the participants. More than 99% of the trials were correctly identified, and this performance did not differ between eyes. For the incongruent trials, the McGurk effect was observed if the response was anything other than /aba/, the auditory stimulus. With their unaffected eyes, the patients perceived the McGurk effect on every single trial. With their affected eyes, perception of the McGurk effect remained quite high. The 2 participants with the smallest scotomas perceived the McGurk effect on every trial, while the 2 patients with the larger scotomas showed some change in the perception of the illusion. One participant perceived the McGurk effect in 15 of 16 trials, and 1 participant perceived the McGurk effect in 10 of 16 trials. Performance of undergraduate participants was also high. Figure 1 displays the proportion of McGurk responses for the 4 participants, as well as the proportion of McGurk for the undergraduate participants. Undergraduate participants also perceived the McGurk effect in almost every incongruent trial (mean percent of non-/aba/ trials = 96%, SE = 0.4).

FIGURE 1.

The proportion of McGurk responses (proportion of non-/aba/ responses) for 4 patients with macular holes using their affected eye and their unaffected eye, and the mean proportion of McGurk responses for the undergraduate participants.

Gaze Data

Figure 2 shows the fixations of participants during the McGurk trials. It can be seen from these figures that the location of fixations with the unaffected eyes during the McGurk trial participants varied considerably. Note that participants 1 and 3 showed strong audio-visual integration without foveating the mouth region and, thus, even with their unaffected eyes, relied on peripheral vision.

FIGURE 2.

Participants' fixations with unaffected eyes (black) and affected eyes (white) during the McGurk task. The spatial center of fixations with the unaffected (white triangle) and affected (black triangle) eyes are shown, as well as the dimensions of the scotoma (or maximum dimensions for participant 1 and 3).

The participants appeared to shift their gaze when they used their affected eye, which may suggest that participants were attempting to use eccentric retinal regions with which to view aspects of the face. However, participants did not appear to shift their gaze sufficiently in order to remove their scotoma from the regions that they fixated with their unaffected eye. The 2 participants with the largest scotomas both naturally fixated on the mouth with their unaffected eye. Participant 4 makes a larger adjustment of fixation location than participant 2, which may limit the occlusion of the mouth by the scotoma. Participant 2 made little adjustment, and the McGurk effect was considerably reduced.

Orientation of Fixations with the Affected Eye

For each participant, an angular histogram was calculated to display the amount of time with the scotoma placed at different orientations with respect to the center point of fixations with the unaffected eye. On these histograms, displayed in Figure 3, the angle of each bar depicts the orientation of the scotoma placement from the center point of unaffected eye fixations, and the length of each bar depicts the total time the scotoma was placed at each orientation. To provide consistency between subjects, only the first 2 experimental blocks were considered in creating these.

FIGURE 3.

Angular histograms display the time (in samples) spent with the scotoma at a particular orientation with respect to the center of fixations made with the unaffected eye. The length of the bar represents the amount of time (in samples), and the angle of the bar represents the orientation.

The histograms show that for most participants, the placement of the scotoma tends to be clustered at 1 orientation with respect to placement of the unaffected eye. This seems to be especially true of the participants with larger scotomas. For instance, participant 1, with a scotoma that is likely less than 2 degrees in diameter, showed the least consistent orientation of his scotoma. Participants 4 and 2, with larger scotomas, tended to place their scotomas in a more specific orientation. Although the orientation was fairly consistent for each individual, there was no consistent preferred direction for the placement of the scotoma across participants.

Discussion

Experiment 1 shows that the speech perception of participants with small central vision loss is only moderately affected. Two participants with macular holes of less than 2 degrees in diameter perceived the McGurk effect on every trial with their affected eyes. These participants fixated around the eye region with both their affected and unaffected eyes. This combination of gaze location and performance demonstrates that direct gaze to the mouth is not necessary for this task. This is consistent with previous findings indicating that when young participants with normal vision fixated on the eyes, there was still a strong McGurk effect.⁹ The 2 participants with slightly larger scotomas also perceived the McGurk effect on the majority of trials. However, of the 2 participants with larger scotomas, participant 2 showed a more significant decline in visual influence on speech perception than participant 4. This result is likely because participant 2 placed the scotoma over the mouth of the talker and thereby obscured valuable speech information. In contrast, participant 4 shifted the scotoma away from the mouth. The importance of visibility of the mouth has been shown in a study by Preminger et al.,¹⁸ which suggests that when the mouth is masked, performance is reduced on a consonant recognition task.

The gaze data indicate that participants tended to place their scotoma in a consistent orientation relative to the location of fixations in the unaffected eye, but that they did not choose a location far enough away to completely compensate for the potential occlusion of speech information. If it is assumed that the area of interest is the same as the region of the face that was fixated with the unaffected eye, then it appears as though participants were not placing their scotoma sufficiently far away to have an effective preferred retinal locus. This may be due to the fact that the participants in this study were not accustomed to using their eye with visual loss, as they could normally rely on vision in their unaffected eye. If participants were given the opportunity to become familiar with using their affected eye they might learn to adapt to their scotoma and use a more eccentric viewing angle. Previous studies have shown that it takes some time for participants with simulated scotomas to adapt a consistent preferred retinal locus.¹⁹

Another possibility is that participants were not shifting their gaze significantly because they were not required to do so, given the limited size of their scotoma. If information from the face is distributed across the face, as suggested by several experiments,^9,20 then it would not be necessary to move a small scotoma to avoid blocking essential areas. Perhaps participants with larger scotomas would be forced to shift their gaze more significantly.

In summary, the findings from Experiment 1 suggest that individuals with limited central visual loss are able to make use of much of the visual information that is available during speech processing. This suggests that patients with small bilateral damage to their central retina do not require additional support for communication. Patients with larger amounts of central vision loss may experience a slight decline in performance if they obscure the mouth with their scotoma. The gaze patterns suggest that participants tend to shift their gaze in a consistent direction when using their affected eye; however, the shift was very small and did not provide evidence for the adoption of an immediate effective preferred retinal locus. In Experiment 2, we test whether a larger scotoma size and a training period to adjust to a scotoma leads to more significant shifts in gaze patterns.

EXPERIMENT 2

Introduction

Undergraduate participants with an artificial gaze-contingent scotoma were tested on a speech perception task over 5 days to allow time for participants to adapt to their scotomas. If the small shift that was seen in Experiment 1 is indicative of a lack of experience with a unilateral scotoma, then the longer time course of Experiment 2 should allow participants to adapt and learn to shift their gaze more considerably throughout the experiment.

The gaze-contingent scotomas were 10 visual degrees, which is considerably larger than those in Experiment 1. If the small shift in gaze in Experiment 1 was due to the small scotoma size, having a larger scotoma will ensure that participants are required to shift their gaze more considerably to avoid blocking a large area of interest. The larger size of the scotoma also allows for an examination of whether speech perception remains intact with more extensive visual loss. In Experiment 1, speech perception was only minimally affected by small central scotomas. From Experiment 2, it is possible to assess whether a larger scotoma of 10 visual degrees affects speech perception.

The speech perception task in Experiment 2 was a speech-in-noise task, in which participants were presented with single words in the presence of background acoustic noise. The presence of visible speech normally enhances the intelligibility of speech under these conditions.¹ Difficulties with speech perception in noise is a common communication problem, and thus this task presents a natural context for this test.

Methods

Participants

Six male and 6 female participants served as participants (mean age = 23, range = 21–23). Participants L and H were left eye dominant, all other participants were right eye dominant. Subjects had normal or corrected to normal vision, as determined by self report.

Stimuli

The stimuli were created from audiovisual recordings of 474 mono-, di-, and trisyllabic words selected from the MRC psycholinguistic database,²¹ spoken by a fluent male speaker of Canadian English. The audio was degraded by the addition of a commercial multi-talker noise signal (Auditec, St. Louis, Missouri, USA). The signal-to-noise level was produced using RpvdsEX software (Version 5.4), and the level was determined by pilot testing to achieve audio-only accuracy levels below 40%.

Two sets of stimuli were created with the degraded audio: an audiovisual set and an audio-only set with no visual information. The stimuli were played with a resolution of 750 horizontal lines using Experiment Builder (Version 1.4.1., SR Research Ltd.). The equipment used to play the stimuli and record eye movements was identical to that used in Experiment 1.

Design

This experiment was a 5-day repeated measures design. On the first day, subjects completed 1 scotoma-free session, followed by 1 experimental session for which participants had a gaze-contingent scotoma of 10 visual degrees. The position of fixation was tracked, and a black circular scotoma was rapidly displayed on the screen. This technique effectively simulates central vision loss. The artificial scotoma was created using Experiment Builder (SR Research), which allows image objects to be rapidly superimposed on a stimulus movie at locations determined by the subject's gaze. Following this, participants completed 1 experimental session (ie, with a scotoma) each day for 5 days in total. During each session in the study, 79 stimuli were presented randomly: 59 audiovisual speech stimuli and 20 audio-only stimuli. The audio-only stimuli controlled for changes in performance due to auditory learning alone. Stimulus words were not repeated for an individual participant and were randomly assigned to sessions across subjects. Participants acted as their own controls.

Procedure

Each participant was seated in the sound booth and the eye tracker was fitted and focused. All participants were tested monocularly with a patch on their non-dominant eye. Their dominant eye was determined by the Porta test.²² Participants were asked to repeat the word spoken by the talker, or state that they did not understand the word. They were told that their response would initiate the next trial. Participants' responses were scored as correct or incorrect by the person leading the experiment. Subjects' head positions were controlled using a chin rest. Between each trial, participants performed a drift correction to maintain the eye tracking calibration. For this activity, the scotoma would disappear and subjects would be required to fixate on a central dot.

Data Analysis

To analyze gaze data, the number of samples, down-sampled to 100 Hz, was used to measure the time an individual gazed in a particular area. As with Experiment 1, the region fixated without a scotoma (ie, control condition) was contrasted to the region fixated with the scotoma. To examine whether participants adequately shifted gaze to avoid blocking this region, a circle was drawn in the dimensions of the scotoma with its center at the mean of gaze positions for the control condition (the “control circle”). The percentage of gaze samples outside the control circle was calculated for the audiovisual trials (see Figure 4). The 7 individuals who had over 50% of samples outside of the control circle on the majority of experiment days were classified as the adaptive gaze strategy group, and the 5 individuals who had less than 50% of their samples outside of the control circle were classified as the nonadaptive gaze strategy group. This classification was used for subsequent analyses.

FIGURE 4.

The proportion of gaze samples outside the control circle (y-axis) for each participant (x-axis) for each experimental day. The red line indicates the 0.5 proportion cutoff criteria for gaze strategy groups. Participants with over half of the data outside the control circle for the majority of experimental days were grouped in the adaptive gaze strategy group, and participants with less than half of the data outside the control circle for the majority of experimental days were grouped in the nonadaptive gaze strategy group.

Statistical analyses were carried out using Statistical Package for the Social Sciences (SPSS). For all of the statistical tests an alpha of .05 was used unless otherwise specified. Any violations of the assumption of sphericity were corrected using the Greenhouse-Geisser correction of adjusted degrees of freedom.

Results

Behavioral Data

To determine the degree to which a visual image enhanced speech perception for the various tasks, a score for the “percent of visual gain” was calculated with the following equation: (% words correct for the audiovisual stimuli) − (% words correct for the audio-only stimuli).²³ All phonemes in a word had to be correctly identified to be scored as a correct identification.

Figure 5 shows the proportion visual gain for the control and experimental sessions for the 2 groups of participants. It can be seen that participants were impaired in their ability to use visual information to aid speech-in-noise intelligibility when an artificial scotoma blocked central vision. However, when participants used an adaptive strategy to maximize the use of peripheral vision, they were able to use much of the visual speech information available. As seen in Figure 5, the adaptive gaze strategy group performed at the level of the control condition by the third experimental day and showed a linear improvement in percent visual gain across the 5 experimental days. The nonadaptive gaze strategy group performed significantly below their control condition and did not show a consistent improvement over the 5 experimental days. Note that the nonadaptive gaze strategy group showed considerably more audiovisual gain in the control session than the adaptive gaze strategy group. The small N and the well-known individual differences in audiovisual speech perception in noise prohibit any interpretation of this difference. However, it is the relative change in gain rather than the absolute level that is of concern, and both groups show decreases in gain when the scotoma is introduced.

FIGURE 5.

Mean proportion visual gain for participants in the adaptive gaze strategy and nonadaptive gaze strategy groups. Error bars represent standard error of the means. Dashed lines indicate the mean performance level of the control session.

A 6 × 2 mixed model ANOVA was conducted to evaluate the effect of day (including control) and gaze strategy (adaptive and nonadaptive) on percent visual gain. A significant interaction was observed between day and gaze strategy, F(5,50) = 7.34, P < .01. Main effects for day and gaze strategy were also observed, F(5,50) = 11.17, P < .01, and F(1,10) = 12.51, P < .01, respectively. The interaction reflects the marked difference in effect of the scotoma on the adaptive and nonadaptive strategies over days.

As displayed in Figure 5, the adaptive gaze strategy group shows a linear increase in visual gain across the 5 experimental days, F(1,6) = 24.29, P < .05. In contrast, there was no significant linear trend across the 5 experimental days in the nonadaptive gaze strategy group, F(1,4) = .28, P > .05.

Gaze Data Results

Gaze data was analyzed to determine whether participants were placing their scotomas in a consistent location and direction that might be indicative of a PRL. The spatial variability, the variance of the Euclidian distances to the center of the spread of fixations for all trials on a given day measured in visual degrees, is displayed in Figure 6. As shown in Figure 6, the adaptive gaze strategy group decreased spatial variability from experimental days 1 to 5, as confirmed by a trend analysis indicating a significant linear change, F(1,4) = 24.65, P < .05. Conversely, there was no trend observed for the nonadaptive gaze strategy group, F(1,4) = .162, P > .05. This result suggests that fixations tended to be in an increasingly consistent location for participants in the adaptive gaze strategy group, but not for the participants in nonadaptive gaze strategy group.

FIGURE 6.

Mean spatial variability for participants in the adaptive gaze strategy and nonadaptive gaze strategy groups. Error bars represent standard error of the means.

The angular orientation was calculated for participants in the adaptive gaze strategy group to measure the direction in which participants shifted their gaze. The angular orientation was calculated and used to create angular histograms using the technique described in Experiment 1. Only the samples that occurred outside the control circle were used in the calculation. A qualitative assessment of the angular histograms indicated that individuals tended to place their scotoma at a consistent orientation throughout the 5 experimental days, as illustrated in Figure 7, which displays a typical participant's performance. Out of the 7 participants who developed an adaptive gaze strategy, 4 participants placed the scotoma to the right of the area of interest (defined as the average gaze location on the control day), 2 participants placed the scotoma superior to the area of interest, and 1 participant placed the scotoma to the left of the area of interest.

FIGURE 7.

Angular histograms for a typical participant for the 5 experimental days. Histograms represent the number of gaze samples that occurred outside the control circle and their orientations compared to the average of the control condition. This participant developed an adaptive gaze strategy that consistently placed the scotoma to the right of the area of interest. Note that the scale for number of samples changes across days.

The mean standard deviations for these angular histograms, outlined in Table 2, illustrate the concentration of gaze locations in a particular region of the visual field. It can be seen that the standard deviations decreased across the 5 experiment days, which suggests that participants were shifting their gaze in a more consistent direction. A trend analysis indicates that there was a significant negative trend in standard deviations across the 5 experimental days for the adaptive gaze strategy group, F(1,6) = .6.68, P < .05. This confirms that individuals in the adaptive group were placing their scotoma at a more consistent orientation relative to the control condition across the experimental days.

TABLE 2.

Mean standard deviation of orientation angles for experimental days for the Adaptive Gaze Strategy Group

	Mean SD
Day 1	48.75
Day 2	30.05
Day 3	38.37
Day 4	33.62
Day 5	17.44

To examine whether the amount of time an individual spent using an eccentric gaze location (% outside the control circle) predicted an increase in visual speech intelligibility (visual gain), a linear regression was performed, first using the entire data set and then using only the data from the adaptive gaze strategy group. Both analyses showed that the percent of samples outside the control scotoma significantly predicted percent visual gain, R² = .48, F(1,58) = 52.91, P < .01; β = .69 and R² = .24, F(1,33) = 10.17, P < .01; β = .49, for the entire data set and for the adaptive group alone, respectively. This indicates that a large amount of the variance in visual gain (48% for all participants, 24% for the adaptive group) is accounted for by its linear relationship with the percent of time spent outside the control circle. These findings suggest that for these large scotomas, the extent to which a participant shifted his or her gaze has an important role in determining how much speech information they are able to perceive visually.

Discussion

When individuals viewed audiovisual speech with an artificial central scotoma, there was a bimodal distribution of gaze patterns. Participants in the adaptive gaze strategy group learned to place their scotoma at an eccentric location, which allowed them to view the area of the face that they viewed during the control condition. The other group of participants, the nonadaptive gaze strategy group, did not learn to move their scotoma off of the area of the face that they viewed during the control day. These 2 groups differed in their ability to use visual information as a way to aid audition in understanding single spoken words when the auditory signal was degraded. The adaptive gaze strategy group was able to perform at the level of the control day during the experimental condition. Additionally, the adaptive gaze strategy group showed a linear increase over the 5 experimental days in percent visual gain. This linear increase in percent visual gain was significantly predicted by the amount of time they spent at an eccentric gaze location. Conversely, the nonadaptive gaze strategy group performed worse than the control condition during each of the 5 experimental days and showed no linear increase in percent visual gain.

The larger scotomas in Experiment 2 demonstrate the effects of larger central vision loss. Although the participants did not have true visual impairments, and thus were not living with the visual loss on a daily basis, the fact that the scotoma was artificial meant that it was discrete and that its dimensions were controlled. The findings indicate that visual speech perception is diminished by central scotomas, but that this deficit can be overcome by appropriate gaze strategies. These strategies are learned quickly over a few hours of exposure to the talker, but do not always spontaneously occur to the participants.

By definition, a PRL is an undamaged, discrete, and consistent eccentric area of the retina that individuals place a target image on.¹² In this study, the fixation patterns of participants were examined with the more dynamic natural stimuli of a talker articulating words. Although we did not ask participants to fixate on a certain target, and therefore cannot draw firm conclusions about whether participants were adopting PRLs, the gaze behavior of participants in the adaptive gaze strategy group is suggestive of a PRL. Participants in the adaptive gaze strategy group consistently shifted gaze to an eccentric location, at a distance far enough to avoid blocking the inferred area of interest (estimated based on the control trials) with the scotoma. The shift tended to occur in a consistent direction, and became increasingly consistent across the experiment.

The majority of studies investigating PRLs have shown that people have a tendency to choose a PRL that places the scotoma to the right or above the point of interest.^14,15,24,25 In Experiment 2, 6 out of 7 participants in the adaptive gaze strategy group placed the scotoma to the right of or superior to the area of interest. The consistency between these results and those from previous studies suggests that these trends are independent of the task and stimuli as both the task and stimuli differ considerably from those used in previous experiments.

CONCLUSION

The experiments presented here explore the influence of central visual impairment on speech perception. The results suggest that patients who have a small central scotoma have great potential for normal communication, particularly if they adopt an appropriate gaze strategy. With a larger central vision loss, visual speech intelligibility is somewhat more disrupted by visual impairment; however, this can also be overcome with an appropriate gaze strategy. From the perspective of understanding how speech is visually processed, this result is in agreement with research suggesting that peripheral vision is sufficient for most speech perception.^9,10

From a clinical standpoint, this result suggests that many people with central vision loss may naturally overcome any deficit to communication that results from their visual impairment. For those individuals who do experience difficulties in speech perception as a result of their visual loss, it may be possible to train a more effective gaze strategy to improve their function. In previous studies, patients with central scotomas have been successfully trained to adopt a PRL for reading.^26,27 Similarly, patients with homonymous hemianopia were successfully trained to be more efficient in their eye movements in order to improve performance on a number of tasks, including activities of daily living.²⁸ In patients with bilateral vision loss, it might be possible to devise a similar rehabilitation strategy for communication that involves instructing patients to gaze eccentrically and use their peripheral vision.