The Invisibility of Scotomas I: The Carving Hypothesis

Abstract

SIGNIFICANCE

Veridical depictions of scene appearance with scotomas allow better understanding of the impact of field loss and may improve the development and implementation of rehabilitation. Explanation and depiction of the invisibility of scotoma may lead to patients' understanding and thus better compliance with related treatments.

PURPOSE

Simulations of perception with scotomas guide training, patient education, and rehabilitation research. Most simulations incorrectly depict scotomas as black patches, although the scotomas and the missing contents are usually invisible to patients. We present a novel approach to capture the reported appearance of scenes with scotomas.

METHODS

We applied a content-aware image resizing algorithm to carve out the content elided under the scotomas. With video sequences, we show how and why eye movements fail to increase the visibility of the carved scotomas.

RESULTS

Numerous effects, reported by patients, emerge naturally from the scotoma carving. Carving-eliminated scotomas over natural images are barely visible, despite causing substantial distortions. Low resolution and contrast sensitivity at farther eccentricities and saccadic blur reduce the visibility of the distortions. In a walking scenario, static objects moving smoothly to the periphery disappear into and then reemerge out of peripheral scotomas, invisibly.

CONCLUSIONS

Scotoma carving provides a viable hypothetical simulation of vision with scotomas due to loss of neurons at the retinal ganglion cell level and higher. As a hypothesis, it generates predictions that lend themselves to future clinical testing. The different effects of scotomas due to loss of photoreceptors are left for follow-up work.

Understanding visual field loss is essential for the development of vision rehabilitation or restoration devices and techniques.¹ Portions of the visual field may become insensitive to visual stimuli in various diseases at any point along the visual pathway from the retina to the cortex. Such field losses are measured clinically using perimeters, and the part of the visual field that is “blinded” is called a scotoma. The perimetric results are essential for detection and the diagnosis of diseases. If the damage is post-chiasmatic, the scotomas are typically bilateral, and they result in binocular homonymous scotomas delineating parts of the field of view that are rendered invisible with both eyes open. In pre-chiasmic diseases or injuries (optic neuropathy or retinopathy), the field losses are monocular. Monocular scotomas in both eyes may overlap in part, and the binocularly overlapping scotomatous areas result in binocular scotomas.

In this paper, we address only the perception with localized discrete binocular scotomas surrounded by a seeing field. We do not address the perception at the edge of the normal field (i.e., the loss of the temporal crescent with loss of one eye²) or scotomas that extend into the limit of the normal field such as hemianopias, quadranopias, and tunnel vision. We briefly address the case of a bitemporal hemianopia that, in combination with esotropia, results in a binocular vertically elongated central scotoma mostly surrounded by seeing field.³

Simulations/illustrations of the perception of patients with such scotomas guide clinicians' training and patients' and caretakers' education, as well as research efforts. In almost all simulations, the scotomas are incorrectly depicted as black (or gray) patches visibly occupying part of the field and thus occluding part of the scene. In these illustrations, contents elided by scotomas are not visible, but the location, shape, size, and boundaries of the scotoma are visible. In fact, the scotomas are usually not apparent and thus are “invisible” (no field occupied with black or gray space), with the contents in the scotomatous field missing from the patient's consciousness.¹ The focus of this article is this “invisibility” of the existence of scotoma. Several studies reported spatial distortions as a form of visibility of scotomas or as a consequence of the scotomas (Schuchard RA. IOVS 1991;32:ARVO Abstract 816).^4–8 Such distortions were illustrated in one simulation of central scotoma⁹ and are addressed in this paper.

A few studies have directly tested the validity of the black patch simulations in patients with field losses and found them to be incorrect.^3,10,11 In one study, black patch simulations of vision with early (arcuate) and late (tunnel vision) glaucomatous field loss were rejected by all 50 patients, and 26% reported no visible loss.¹⁰ Seventy-six percent of 29 patients with age-related macular degeneration (AMD) rejected the black patch simulation taken from the National Eye Institute website.¹¹

Fletcher et al.¹² found that most patients with AMD and binocular scotomas were completely unaware of a visual field loss, even with very large scotomas. However, 44% reported objects disappearing on them. Another article questioned patients with severe pericentral glaucoma (within the 10-2 Humphrey Field Analyzer field) about their perception of an Amsler grid.¹³ Only 5% of patients reported seeing black patches. Many of the patients could detect missing lines in the grid or report seeing such lines as blurred or gray. Note, however, that the Amsler grid is a perimetric test and does not represent vision with a natural view of the environment.

Despite all the counterevidence, most simulations of vision with scotomas, central or peripheral, continue to depict black (or gray) patch(es) occluding parts of the scene in the patient's view. Such simulations are presented on most websites^14–16 and textbooks.^17–19 The high prevalence of such wrong simulations confuses patients who may easily see that they are wrong (at least for them), and when they report that discrepancy to their relatives, the confusion spreads to the caretakers.

One way to understand how scotomas can exist, be measurable by perimetry, and yet not be perceived (be invisible) is to consider optical scotomas caused by optical devices. An optical scotoma is caused by spectacles-mounted magnifying devices, such as high-power positive spectacles lenses used to correct hyperopia, in particular due to aphakia.^20,21

These lenses were typically produced in a lenticular shape with a round central high-power lens surrounded by a plano or lower power lens used to mount the lens in the frame (Fig. 1). The lens magnifies the view through the lens, which results in a ring-shaped loss (ring scotoma) of a section of the field of view surrounding the lenticular lens that can be measured with a perimeter (Fig. 1A). Observing or photographing a scene²² through such a magnifying lens does not result in a black or any other colored ring, although careful examination reveals the loss of content that falls within the ring-shape scotoma surrounding the content seen through the central lenticular lens because of the magnification (Fig. 1B). The loss of content (elided content) is not difficult to note upon careful foveal examination of the photograph. In actual use, however, the elided content is most of the time located at far peripheral eccentricities (>40°). Therefore, the contents are affected by the reduced resolution and contrast sensitivity in the periphery and by being outside the patient's attention, making the loss of content very difficult to note. However, under eye²⁰ or head²¹ movement, objects falling into the ring scotoma (the elided details) may occasionally pop into the visible central part of the field of view and become apparent, startling the patient in what is called the “jack-in-the-box” phenomenon. This (optical) effect illustrates the relevance of image dynamics to the perception of the effects of scotomas, which we address further hereinafter.

FIGURE 1.

“Invisibility” of perimetrically recorded optical scotomas. (A) Monocular (right eye) perimetry of a normally sighted person wearing a +10 D lenticular spectacle lens (39-mm-diameter lenticular). (B) A photograph taken by a camera held behind the same lens depicting the user's view.²² The magnification ring scotoma recorded in (A) is only notable in (B) by noticing the missing content (missing parts of the entrance on the right and missing parts of posters) between the (magnified) view through the lens and the view outside the lens. (C) Monocular (left eye) perimetry from a patient with left homonymous hemianopia wearing upper and lower horizontal peripheral prisms (40Δ ≈ 20° at 20° upper and lower periphery) on the left lens. The apical scotomas are easily measurable on the right seeing side, in addition to the expanded field of view measured to the left. (D) A photograph of the view through the peripheral prisms lens.²³ For a patient with left hemianopia, the left half of this image will not be visible, except as seen through the right side of the prisms. The apical scotomas in (C) are not explicitly noticeable but can be appreciated by noting that the portions of the scene (i.e., the person's face) are not visible, in or out of the prism, as it is replaced by the view through the prism of the portions of the scene farther to the left.

A second common optical scotoma is the scotoma manifested at the apex of any prism included within the field of view, called the apical scotoma.²⁴ Here too, despite the clear record of a scotoma obtained in perimetry (Fig. 1C), the view through the lens (even monocularly, with or without hemianopia) does not include an explicit black patch that occludes contents and/or occupies the field of view (Fig. 1D).²² Instead, the shifted image in the prisms replaces part of the unshifted image at the same location of the prisms, and the realization of content loss is cognitively implied and requires attention. The peripheral prism, which shifts a portion of the field of view for patients with homonymous hemianopia, is usually placed at more than 20° eccentricity above and below the line of sight²⁵ (Fig. 1C) so that all the elided content details in the apical scotoma and visible details in the surrounding areas are subject to the reduced visual acuity and contrast sensitivity peripherally, making the loss of content less apparent. This eccentricity-dependent loss of resolution is not illustrated in these photographs captured by a camera (Figs. 1B, D) but is simulated later in this article.

In both the aforementioned optical scotoma examples, the scene details under the scotoma in the field of view, the elided contents are replaced by visual content from another part of the scene through the optical devices, the magnified central area in the case of the ring scotoma, and the shifted scene in the prism case. Importantly, the scotoma, which can be appreciated cognitively, is not explicitly visible as a patch in the field of view, but the part of optical device that occupies the scotomatous field now shows another part of field of view due to the optical effect (i.e., magnification and shift).

A different type of scotoma that may be measured perimetrically yet be invisible can occur in patients with complete bitemporal hemianopia. Damage to the optic chiasm due to disease, surgery, or trauma results in heteronymous (bitemporal) scotomas. The residual fields in both eyes may be nonoverlapping and thus may leave the binocular field of view largely intact, except for the loss of the temporal crescents bilaterally. A binocular vertical central scotoma can be measurable in patients with bitemporal hemianopia and manifest esotropia (Fig. 2A).^3,26 The patient's perception is inconsistent with the common wrong illustration in Fig. 2B; rather, these patients report the effects shown in Figs. 2C and D. Note that the elided content here is not replaced by anything; it is simply eliminated (carved out). These patients' perception can be simulated optically by changing the magnitude of the strabismus; for example, by bifold mirrors in Fig. 2D or variable prism in the phoropter, we can vary the horizontal extent of the scotoma (i.e., the width of the elided content), resulting in perception of shrinking or widening of the contents (e.g., the face of the examiner, as in Fig. 2D). Because the vertical meridians on both sides of the scotoma represent the same visual direction (straight ahead), they are perceived to be abutting, rendering the scotoma invisible. This is a special situation, but we are hypothesizing that similar elided contents and thus carved-out perceptions occur with other central and peripheral discrete scotomas surrounded by the seeing retina.

FIGURE 2.

Invisibility of the scotoma from bitemporal hemianopia combined with left esotropia. (A) A schematic drawing of a chiasmal lesion causing a vertical scotoma (marked as black), as wide as the esotropia angle. The inset shows a dichoptic field diagram with the loss of both peripheral crescents and a vertical central scotoma. (B) A typical (incorrect) simulation of the field loss is illustrated as a black stripe in the middle of the scene that matches the field diagram in (A). (C) The actual view of the scene as perceived by such a patient with bitemporal hemianopia.³ Because of the elided contents at the center (under the black strip in B), small mismatches may be detected across the vertical meridian under careful examination. (D) Photographic depiction of a similar effect of optical scotoma created by slightly misaligned wall mirrors found in a bathroom. Although no scotoma or black vertical strip is explicitly visible (except for the thin line between the mirror boundaries), the missing elided image content (nose) due to the scotoma can be inferred cognitively.

The physiological blind spot is a monocular discrete scotoma that marks the optic nerve head. It is an ellipse measuring approximately 5° × 7° centered approximately 15° temporally in both eyes' visual fields. Because everyone can experience the blind spot themselves, its properties and invisibility have been studied more than any other type of scotoma. When covering one eye, the blind spot in the other eye becomes a binocular scotoma, yet it is not visible, although it can be demonstrated and easily measured perimetrically. It is important to consider the lessons from observations with the physiological scotoma, yet it is also important to note that it is a special scotoma, as the brain has not had a representation of the space under that scotoma except through the other eye.

Although it was clear to some for centuries that scotomas are not seen by patients as black patches in the field of view, it was not obvious how they do look, let alone how to illustrate, or simulate the vision with them. Understanding and illustrating the invisibility of scotomas or the possible impact of them on the view by the remaining functional field right next to the scotoma are not simple.

Many types of scotomas are not explicitly observed and are out of awareness even when they are located centrally within the visual field. Therefore, we advance here the hypothesis that the visual pattern under the scotoma, the elided content, is carved out of the perceived scene and is not replaced by anything, as shown previously for the special case of bitemporal hemianopia with esotropia (Fig. 2). This carving-out requires the surrounding scene content from just outside the scotoma to somehow shift into the void created in the scene by the scotoma. In the general case, such a shift must result in local spatial distortion of the content outside the scotoma. Local distortions will interact with the image motions because of eye movements or self-motion. These interactions, which may be perceived as local motion, are addressed hereinafter.

Scotoma Carving

We illustrate/simulate the invisible scotomas based on the carving out of elided contents by modifying the Seam Carving image processing technique,²⁷ designed as a content-aware image rescaling/resizing process. It is now widely used in image processing packages such as Photoshop (Adobe Inc., San Jose, CA). We therefore call this simulated visual perception process Scotoma Carving. The seam carving algorithm may be applied to remove a pre-designated part of the image (i.e., the scotoma) by forcing these portions to be removed. This removal of contents from the designated part of the image without replacing any is the main effect resulting in the invisibility of the scotomas. It is used to illustrate the effects that most content-aware carving may cause and enable us to examine the consistency of such effects with scotomas as reported by patients and observed in studies. The use of this specific content-aware removal algorithm is not meant to directly represent the visual mechanism underlying the perception with scotoma but rather to illustrate the appearance, to a patient, of a scene with elided contents carved out. Of course, some visual mechanism that will provide similar effects is necessary for this hypothesis even to be considered. Preliminary validity of this simulation is supported by the consistency of the results with effects reported by patients with scotomas, as shown in the rest of this paper.

In the seam carving algorithm,²⁷ “seams” (one-pixel-wide meandering paths of least energies through the image) that progress either top to bottom (vertical; Fig. 3B) or left to right (horizontal) are calculated and later removed. A seam contains only one pixel in each row or column for the vertical or horizontal seams, respectively. The algorithm selects seams to minimize the removal of salient contents (hence content-aware). A seam is constructed by using an energy function (local contrast measure) that measures the visual saliency of the pixels. At each step, the next pixel (from among neighboring pixels in the next row for a vertical seam), the one with the lowest energy is selected. In the images produced here, the (3 × 3) Prewitt operator²⁸ was used as the energy function. A seam's total energy is calculated by summing the energy calculated for each pixel. Once all the seams are constructed, seams with the lowest cumulative energy rank are removed progressively from the image one by one, resulting in a reduction in the size of the image by one pixel in the orthogonal direction (i.e., horizontal size reduction of one pixel for a vertical seam removal).

FIGURE 3.

“Thin man”⁶ perceived by patients with cortical paracentral field loss. (A) Field diagram of a small (5 × 8°) bilateral homonymous paracentral scotoma. (B) The binocular scotoma mask is shown over the father's left shoulder. The six first seams to be removed in scotoma carving are annotated in colors. These seams are converging into the scotoma from above and then diverging below the scotoma. (C) The scotoma carving shrank the left shoulder. The scotoma itself is invisible. Additional local distortions (i.e., pots on the stove, lines on the shirt, and the narrower wall of the furnace) are only noticeable by carefully comparing (B) and (C).

The seam carving approach may be applied to preferentially remove the part of the scene underlying a scotoma. That portion of the image is pre-set to have minimal energy (or even negative energy), so seams will always pass through the scotoma and thus are removed first. The algorithm is forced to select the seams near the scotoma to pass within the scotoma, whereas the seam portions outside of the scotoma (before and after passing through the scotoma) are free to search for the best path to remove additional (low energy/less salient) pixels to complete the seam. The lowest cumulated energy seams are removed in order until all the pixels under the scotoma are removed. More details of the scotoma carving computations applied here are in Appendix 1, available at http://links.lww.com/OPX/A621.

The scotoma carving process produces seams widely distributed across the image to prevent visible distortions but results in more distortion near the scotoma because all seams converge into it. If scotoma carving is applied using vertical seams, the right and left sides of the scotoma will be forced to close into apposition. Similarly, for horizontal seams, the regions above and below the scotoma will close. When both vertical and horizontal seams are applied, the distortions are in all directions around the scotoma. These distortions get smaller (more distributed) farther away from the scotoma. The image processed by scotoma carving ends up smaller than the original image. That reduction in the image size is by the size of the scotoma. However, if such an effect happens in natural vision, it will not be noticeable, as the visual field of view (200° horizontally) is much larger than typical scotomas (about 10°), and there is no clear visible boundary of the full visual field.

Distortions similar to those occurring with the scotoma carving algorithm have been reported by patients. Patients with small right paracentral homonymous hemianopias (Fig. 3A) reported seeing the left shoulder of a person appear thinner than the right shoulder.⁶ A simulation of this paracentral scotoma using the scotoma carving illustrates a similar effect (Fig. 3). In this case, the carving was applied using vertical seams resulting in the shrinking of the horizontal shoulder line. The same patients were presented also with two vertical lines laterally shifted from a fixation target to the right and left.⁶ The right line passing through the scotoma was reported by both patients to be shorter than the left line, representing vertical distortion (shortening). We have replicated this effect by applying scotoma carving (not shown). Andrews and Campbell²⁹ reported that a horizontal line extending over time through the physiological blind spot appears shorter than a similar line extending at the same time outside the physiological blind spot. In this dynamic case, the difference in size was more pronounced than with the static images used by Safran et al.⁶ and others, although the difference may have more to do with the difference in eccentricity than with dynamism. All these reported effects are consistent with the results of scotoma carving process. Other, related, spatial distortion effects were reported in patients with quadranopias due to cortical lesions (Ramachandran VS. IOVS 1992;33:ARVO Abstract 1348).⁸

The selection of the direction of the seam carving to perform may be an important decision in the carving algorithm. Preferably, the direction of applied carving should follow what may be taking place in the visual system, but evidence for that is limited. The few examples listed previously suggest that carving takes place in the direction orthogonal to the test stimulus (e.g., vertical seam carving for the width of the shoulder in Fig. 3), but in almost all clinical studies, only one direction was evaluated. Because of the curved and diverging paths taken by seams outside the scotoma, the distortions, although mainly orthogonal to the direction of carving, may be affecting other directions.

Another consideration may be that the carving should take place in the direction orthogonal to the line connecting the fovea with the discrete peripheral scotoma (i.e., vertical seam carving for left and right peripheral scotomas and horizontal seam carving for upper and lower peripheral scotomas, respectively). The benefit of such a choice is that the distortions imparted by the seam path outside of the scotoma would be directed mostly away from the fovea, keeping the distortion farther in the periphery. Note that fixation is naturally aimed at higher saliency (high contrast portions of the scene), and as a result, the seams would be likely diverted away from the object of interest being fixated. The algorithm would result in lower distortions if the direction of the seam carving is aligned with the longer dimension of the scotoma, which would require fewer seams to be carved out. This last rule will coincide with the second rule when carving arcuate or ring scotomas, which may be the case in glaucoma and retinitis pigmentosa.

In this paper, we implemented horizontal and vertical carving and selected the seam carving directions to follow the aforementioned directional selection rules. For paracentral scotomas, we implemented such orthogonal seam carving (i.e., applying vertical seam carving for lateral scotomas and horizontal seam carving for vertical scotomas, respectively). In the case of multiple scotomas, the orthogonal carving algorithm was applied separately to each scotoma to follow that rule. For central scotoma, this last rule may not be applicable, and thus, we have alternated horizontal and vertical seam carvings. This approach is also more efficient in cases of roughly round scotomas, as is frequently the case with central scotomas. Because of the curved path of the seams and because of the combined effect of nominally horizontal and vertical seams, the distortions outside the carved scotoma may be affected in all directions not just in the horizontal and vertical directions (as the seams may deviate by up to 45° from the primary direction).

The scotoma carving images shown in Figs. 3 and 4 result in substantial local spatial distortions of the scene outside of the carved scotomas. These distortions are noticeable when the scotoma carved image is compared side by side with the original image, as seen in these figures. However, without comparison with the uncarved image (not accessible by patients), these substantial distortions are barely noticeable. In addition, extended foveation of the carved image as one investigates details at different locations from the figures is not available for real patients because the scotomas are moving with changes in fixations, remaining eccentric. Comparison of details is sometimes possible within one image, such as the difference between the two shoulders (Fig. 3).

FIGURE 4.

Scotoma carving simulation of vision with paracentral arcuate scotomas from glaucoma (eccentricity of ~15°). (A) Typical vision with scotoma simulation (three black paracentral patches with the black cross with white border marking the associated fixation position) adapted from Crabb et al.¹⁰ Image reproduced with permission. Superimposed yellow and red outlines indicate the location of those scotomas at two different fixation locations (yellow and red crosses), representing saccadic eye movements. Scotoma carving simulation results in the removal of objects under the scotomas marked in (A) by the dashed yellow outlines (B) and the solid red lines (C). The local spatial distortions are not easily noted when maintaining fixation at the cross. The distortions pop out as motion artifacts when images in (B) and (C) are abruptly flipped (see Video 1, available at http://links.lww.com/OPX/A622). With guided fixations and blurred frames inserted between fixations as simulation of saccadic blur, the motion, and the distortions, scotomas are invisible (see Video 2, available at http://links.lww.com/OPX/A623).

Although Fig. 3 simulates the view with a paracentral scotoma near the fovea, the images in Fig. 4 simulate the effect of farther peripheral scotomas (arcuate scotomas) located at approximately 15° of eccentricity and therefore unlikely to be noticed if foveation is maintained at the indicated fixation locations. If the images are viewed at the fixation marked in Fig. 4A under the correct display size from the appropriate viewing distance, the distortions fall at the appropriate far eccentricity, where they are very difficult to note.^30,31 A simulation of eccentric scotoma effects can be generated by incorporating the decline in resolution and contrast sensitivity with eccentricity.³² Although the eccentric scotoma effects can be examined foveally over the entire image, the patients with the scotoma cannot examine them because the scotomas move with the fovea. This aspect is addressed further hereinafter (Fig. 5)

FIGURE 5.

“Flipbook” simulations of saccadic effect presentation with scotoma carving. (A) A panorama image of the full farmhouse scene spanning 109°, which is too large to be presented on most displays. The 10 small crosses (yellow, green, and blue) mark the selected locations of fixation implemented in the video frames. For 2 of the 10 fixations, we show the positions of the corresponding arcuate scotomas carved out (blue and green). (B) A sample frame from the green fixation with corresponding green arcuate scotomas shown in (A) carved out. When fixating on the center fixation mark, it is hard to see even the large distortions on the right and almost impossible to note the distortion caused by the carving on the left. (C) Another frame of the flipbook corresponding to the blue arcuate scotoma in (A) was first processed to simulate the loss of resolution as a function of eccentricity³² and then carved out. In both (B) and (C) and all fixations in Video 3 (available at http://links.lww.com/OPX/A624), it is almost impossible to see the effects of the smaller lower scotoma. The observer may stop the video at any frame and examine the whole frame foveally. This should enable the detection of the effects of scotoma carving including some dramatic distortions that are not visible when fixating centrally.

The Interaction of the Scotomas with Vision Dynamics

Objects in the scene that move slowly into a scotoma merely disappear, yet patients do not experience the scotoma itself, which remains invisible. One can experience this by closing one eye and moving a finger through the physiological blind spot in the other eye. Here too, part of the finger disappears—but it just disappears into nothingness. Neither the scotoma nor the outline of the scotoma emerges or becomes visible. This is an example of the role of motion in perception with scotomas. There are various types of motion to be considered. The most common is the whole retinal image abrupt motion due to saccades that occurs approximately three times per second. Another commonly encountered movement is the radial optic flow of the world image across the retina in locomotion. This smooth optic flow is much faster at farther retinal eccentricities than near the focus of expansion. Another motion is the smooth sweeping of the whole static world image across the retina with the pursuit of a smoothly moving object of interest. In all three cases, the perception of a stable world is maintained despite the movements of the world image on the retina. Another motion of interest is the movement of one or a few objects of interest across the field of view while fixation is maintained on another static object of interest. This results in a mostly static retinal image except for the few moving objects. The interaction of the invisible scotoma with retinal image motions is of interest. After showing scotoma carving effects on static images, we then deal with the effects of these most common saccadic movements and optic flow on the invisibility of scotomas.

Effects of Saccades

Most simulations of vision with scotomas are limited to static images. The videos that we found online typically present smooth panning movements of the camera with the black (or gray) patch scotomas fixed in the frame.^33,34 The latter reference does include a few simulations of saccadic-like movements.

Smooth pursuit eye movements require tracking a moving object, which is not the situation shown in most of these videos, except for our simulation of vision with central scotoma³⁵ that included some smooth movement of the scotoma within the scene and the video frame.³⁶ In a Public Broadcast Service television program, featuring a low-vision device based on an early head-mounted display,⁹ the simulations also showed smooth movement of the scotoma either at the center of the frame or within the frame; however, the eye movements measured and shown in the same program were all saccades, yet these were not simulated. Tracking a smoothly moving object using peripheral vision is not easy.^37,38 Simulations of vision with scotomas should account also for the change in the location of scotomas across the scene with eye movement. As patients' saccade across a scene, scotomas and their associated distortions move abruptly around the scene with every saccade. The temporal changes in local distortions across a saccade result in apparent local motions that may attract attention to the distortion. However, patients do not report noticing such effects or perception of motions because of changes in local distortions during saccadic eye movements, indicating another type of invisibility of the effects of scotomas.

The simplest way to represent the effect of saccades is to switch quickly between the images in Figs. 4B and C, as done in Video 1 (available at http://links.lww.com/OPX/A622), representing an abrupt change in fixation. When watching this video without a fixation marker, the viewer is free to look around the scene examining the difference between two frames and may notice the apparent motions during switching between two images and detect the local spatial distortions.

However, if a blank or blurred frame representing saccadic suppression, or merely saccadic blur,^39,40 is inserted between the pre- and post-saccade frames, the distortions and the apparent motions are largely masked (Video 2, available at http://links.lww.com/OPX/A623). If the observer performs a saccadic eye movement exactly when the images are switched (guided gaze-contingency), these obviously apparent image motions become invisible, because they are masked by the observer's saccade—even without the simulated blur (not shown). We approximate gaze-contingent display using a “guided-gaze” stimulation, where the image is switched 200 milliseconds after (i.e., an average saccadic delay) the fixation target shifted (Video 2, available at http://links.lww.com/OPX/A623).

Another way of incorporating the effects of saccades in the scotoma carving simulations is the “flipbook” presentation (see Video 3, available at http://links.lww.com/OPX/A624). Here an image larger than the angular width of the display, a panorama image (Fig. 5A), is used (representing a 109° wide scene). A series of fixation locations are selected within the larger image at various locations representing fixations of a person (with early glaucoma field loss) scanning the view with saccades. Scotoma carving is applied using the simulated patient's scotoma positioned relative to each selected foveal fixation location. In these simulations, the same arcuate scotomas used in Fig. 4A are used but flipped upside down to represent lower arcuate scotomas (Fig. 5A). The processed image is then cropped to fit the display size (45° wide) with the fixation always at the center (Fig. 5B). The image in Fig. 5B is obtained by taking the fourth-fixation cross (green) from the left side of Fig. 5A. If the images are viewed on a 24-inch high-definition monitor (approximately 20 inches wide) from a distance of approximately 24 inches, they span approximately 45° of visual angle. The series of these images representing a sequence of 2-second fixations across the larger image is presented in Video 3 (available at http://links.lww.com/OPX/A624), which simulates the effect of saccadic eye movement with the central guided fixation. Because most naturally occurring saccades are smaller than 15°,⁴¹ there is a substantial overlap between consecutively presented frames, making the image shift obvious. However, if the observer maintains fixation at the center of the display, the motion artifact caused by the changing distortions between the images (which are mainly taking place at approximately 15° retinal eccentricities) is not visible. During this simulated fixation movement, the whole retinal image is updated, and therefore, any changes in the images from before to after fixation are not easy to track. As a result, in this flipbook simulation video, no saccadic blurred frame was required to mask the transition. The reader may pause the video on any frame to examine the frame carefully. If the reader fixates on the fixation target, the peripheral scotoma carving is hardly noticeable because of lower resolution and contrast sensitivity of the peripheral vision. In a few frames, only the effect of one of the lateral scotomas may be noted this way, and in many frames, no scotoma is noted. The effect of the lower smaller arcuate scotoma is observable, but only if one is searching for it deliberately. The motion associated with the local distortions due to this lower scotoma is never observable. We provide examples of similar processing with additional 10 images of various characteristics demonstrating that the invisibility of the scotomas, the distortions, and the motion artifacts are a general effect and not specific to the image in Fig. 5 (Video Set 1, available at http://links.lww.com/OPX/A651).

Effect of Eccentricity

It is well established that the contrast sensitivity declines with increased retinal eccentricity.³² That effect, which is easy to measure, is nevertheless invisible to people with normal sight without steady fixation and when attempting to distinguish details in the periphery. Under casual observation, people perceive their full field of view to be sharp in focus and at high contrast. Peli et al.³² applied the pyramidal multiscale vision model⁴² to simulate the effect of reduced contrast sensitivity with retinal eccentricity. The validity of these simulations was then verified using wide-angle natural images.⁴³ This effect combined with scotoma carving was applied here in Fig. 5C and next Figs. 6C and D and Fig. 7. In inspecting those scotoma carved images either with fixation maintained at the center of the image or with the simulation of the effect of eccentricity (Fig. 5C), it is difficult to note the image details carved out without prior examination and comparison with the outlined scotoma image. Removal of the image regions within the scotomas using the carving algorithm results in spatial distortions around the elided areas (e.g., straight lines becoming curved). Distorted spatial details are visible when affecting familiar objects such as faces, but even in these cases, the distortions are frequently difficult to discern without comparing the processed with the original image.

FIGURE 6.

Central scotoma carving. (A) A scene spanning 32° with the location of a 6° central disciform scotoma marked (yellow circle). (B) Alternating scotoma carving. Applying vertical seam alternating with horizontal seam to the original scene in (A). Distortions are apparent in comparing the images in (A) and (B), but the image in (B) is completely acceptable if examined without comparison. (C) The original image in (A) was processed with the Peli model to simulate the decline in resolution and contrast sensitivity with retinal eccentricity.^32,43 (D) The image in (C) after alternating scotoma carving representing the view of a patient with a disciform scotoma. The most notable effect is the “blurriness” of the view, as reported by patients. The central elided content from (C) is invisible here.

FIGURE 7.

(A) The image in (Fig. 6A) with the fixation and scotoma position shifted 6.5° down and 8° rightward from the center. Note the increased blur near the left edge of the image and decreased blur on the lower right in comparison with (Fig. 6C). (B) The image in (A) after alternating scotoma carving. The distortions become more apparent even without comparison with the image in (A) because they are not at or near the perspective vanishing point. (C) The image in Fig. 6A with the fixation and scotoma position shifted 6.5° up and 10° rightward from the center to fall onto the high-rise building. (D) The image in (C) after alternating scotoma carving. Note the increased blur on the left and below in this image in comparison with Fig. 6C. The distortions here are apparent even without comparison with the image in (C) because they affect the windows grid pattern in addition to not being near the perspective vanishing point. The angular size of the printed images is much smaller than the simulated size. The reader may enlarge each image to fill the page by zooming in. Even at that scale, the images are approximately only 65% of the simulated angular size.

The Effect of Optic Flow

When a patient with peripheral (arcuate) scotoma(s) is walking or driving, the image of the scene sweeps smoothly across the retinas spreading peripherally from the focus of expansion, which is at the direction of the movement. Static objects near the focus of expansion move/expand very slowly. Static objects at farther eccentricity from the focus of expansion move faster than the objects near the focus of expansion as they move farther peripherally. A patient walking straight may be looking in any direction, but assuming, first, fixation at the focus of expansion (note the first and third fixation markers in Video 4, available at http://links.lww.com/OPX/A625), static objects at the periphery will move away to the farther periphery following the optic flow and thus smoothly move into and then out of the peripheral scotomas (Video 4, available at http://links.lww.com/OPX/A625). When a peripheral object (or parts of an object) enters and later reemerges from the scotomas with no eye movement, the change in the object's appearance (similar to jack-in-the-box phenomenon) may be expected to be visible, as it is not masked by saccadic blur (see Appendix 2, available at http://links.lww.com/OPX/A650). However, patients with such peripheral scotomas do not report such changes, just as they do not report changes in static images with or without saccadic eye movements. Normally sighted readers may experience the effect by walking while looking straight ahead with one eye closed. The optic nerve head scotoma in the other eye is at approximately the same eccentricity (15°) and of similar size (5 × 7°) as the early arcuate scotomas in glaucoma, yet the physiological scotoma remains invisible during this movement. This invisibility of the scotomas may be accounted for by the lack of attention to the periphery and by the lower resolution at that peripheral eccentricity. It is also the case that the visual system is accustomed to such disappearance and reemergence of objects during walking, as a closer object may temporarily occlude a farther object, which will then reappear farther peripherally. Thus, the effect of the lateral scotomas is similar to the familiar occlusion imparted by the closer objects within the optic flow during walking. Note that the objects occluded, or covered by the scotomas may be static or may be moving on their own within the scene.

The walking patient is likely looking at the focus of expansion, but nothing prevents the walking patient from looking elsewhere from time to time. In most cases, the shifts of gaze will be saccadic. In these cases, the effect of saccadic blur will be similar to the effect discussed previously and thus further contributes to the invisibility of the scotoma or the local motion due to changes in periscotomas distortions.

As the gaze is shifted laterally away from the focus of expansion and lands on a peripheral static object in the environment, that object will appear to move laterally triggering smooth pursuit eye movements to track the object with the fovea. This will also result in sweeping of the peripheral scotomas across the scene at an angular speed related to that of the fixated object, which will be slower than that of more peripheral or farther static objects. The fixated object may be static as addressed previously, but it may be moving within the environment such as another pedestrian walking in the same direction and speed as the patient, or in any other direction. If the fixated object is another pedestrian walking in the same direction and speed as the patient (see the second fixation marker in Video 4 [available at http://links.lww.com/OPX/A625] placed on the head of the person walking in the same direction as the patient), the pedestrian remains at the same retinal eccentricity. Fixating on the pedestrian means that the pedestrian image remains fixed on the patient's retina while the whole environment is moving as the optic flow because of the patient's walking. This situation is similar to the fixation at the focus of expansion except that movement of the environment across the scotoma may be faster on the farther side and slower on the other side if the pedestrian walking in the same direction is laterally off from the patient's walking path. All these conditions are shown in Video 4 (available at http://links.lww.com/OPX/A625) illustrating the scotoma carving on a video of a patient walking in a virtual shopping mall with other pedestrians walking around in various directions and speeds. All the conditions described previously are illustrated, and if the observer is fixating on the marked fixation targets on the correct angular size display (100° horizontal span), the invisibility of the scotomas and the distortions caused by the carving in all these situations may be appreciated (Video 4, available at http://links.lww.com/OPX/A625). More complex interactions between the scotoma and other pedestrians occur when the patient is fixating a static target off the focus of expansion (gaze away from walking direction), which results in sweeping of the scotomas across the field of view while other pedestrians walking in various directions may interact with the “moving” scotomas.

Carving a Central Scotoma

Distortions due to the carving of peripheral scotomas were found to go largely unnoticed as the peripheral image content is out of the viewer's attention, and the local spatial distortions are subjected to lower peripheral resolution and contrast sensitivities.³⁰ We also evaluated the impact of scotoma carving applied centrally, representing the central field loss due to disciform macular degeneration where the full thickness of the retina is affected. Patients with central scotomas due to AMD also reported that they did not notice the scotomas as a black patch,^1,11 although they frequently report distortions.^44–46

In Fig. 6, we illustrate the effects of a central 6° diameter scotoma using the scotoma carving. The first observation is that there is no apparent black patch (Fig. 6B), which matches the patients' reports. Without comparing Fig. 6B with the original image, Fig. 6A, it is difficult to notice the loss of the elided content at the center of the image. Because of the fixating (and thus the central scotoma) near the vanishing point of the scene, the converging roads after the scotoma carving appear natural without a careful examination or manual tracing of the lanes. The distortions around the carved-out scotoma are substantial, but the distortions of both the buildings and the road surfaces in Fig. 6B appear completely acceptable as architectural features seen in a geometric one-point (a central vanishing point) perspective. Fig. 6B shows the result of alternatingly carving with horizontal and vertical seams (see Appendix 1, available at http://links.lww.com/OPX/A621, for more details). Upon side-by-side comparison, the carved-out missing contents (several buildings and trees) are easily noticeable.

All the processing and observations of the simulated central scotoma described previously have been with high-resolution (4096 × 3349) images examined with normal foveal vision. Some patients with macular scotoma describe distortions of architectural structures such as door frames or ceiling lines where the expected underlying structure is easy to compare with the appearance with the scotoma. They do, however, describe their vision, first, as “blurred” because they are forced to use their pericentral peripheral retina with its lower-contrast sensitivity and lower resolution. When the eccentricity effect^32,42,43 is applied to the original image (Fig. 6C) and then the high-resolution central vision is carved out, the carved image displays only the residual blurred pericentral vision of the reduced resolution outside the scotoma (Fig. 6D). The lower resolution further masks, to some extent, spatial distortion, although not completely, as the degradation at this near periphery is not as severe as near the glaucomatous arcuate scotomas.

A patient may be looking at the vanishing point of perspective as seen in Figs. 6 (i.e., the center of the image) or looking at different locations in the scene. Fig. 7 illustrates the effect of such fixations away from the vanishing point on the appearance of the same scene with the same scotoma. When the fixation and the scotoma are shifted down and to the right (Fig. 7A), away from the vanishing point, the first effect of the scotoma carving (Fig. 7B) is the increased blur to the left and upper parts of the image because of the farther eccentricity from the fovea. The distortions surrounding the carved scotoma are somewhat more noticeable due to the effect on the local image details away from the vanishing point, but still are not very striking at that location. In Fig. 7C, the fixation is shifted up and to the right so that the scotoma is carving a large portion of the high-rise building. The result of the scotoma carving is shown in Fig. 7D. In addition to the change in the blur, which is increased here toward the lower left side of the image, the distortions here are very apparent as a deviation from the rectilinear shape of the building, both of the whole building and of the grid-like structure of the windows. These effects illustrate the interaction of the local distortions with the specific image content. When the distorted areas are consisting of a so-called carpenter's world, the distortions are easier to notice and are more visible. This is consistent with reports of patients with central scotoma who describe distortions of architectural features such as ceiling lines and door frames. We provide examples of similar processing with additional 10 images of various characteristics demonstrating that the invisibility of the scotomas, the distortions, and the motion artifacts are a general effect and not specific to the image in Figs. 6 and 7 (Video Set 2, available at http://links.lww.com/OPX/A652).

DISCUSSION

Our interest here is focused on the invisibility of scotomas under natural daily viewing conditions. Numerous publications addressed the improved visibility of the scotoma under special (laboratory) conditions, as described hereinafter. The fact that special conditions were needed to enhance the visibility of the scotoma, or its boundary, using blinking⁴⁷ or dynamic spatial noise,⁴⁸ indirectly demonstrates that the scotomas are in fact invisible under natural viewing conditions.

Sir David Brewster⁴⁹ pointed out, as early as 1832, the surprising effect that a target falling in the optic nerve scotoma will disappear and be replaced with the color of the uniform background surrounding the target and the scotoma. This “filling-in” effect has been frequently described since then, and similar effects were documented for patients with discrete retinal lesions (i.e., disciform macular scar,⁵⁰ chorioretinal scars from presumed toxoplasmosis⁵¹) or with cortical scotomas.⁵²

Our scotoma carving hypothesis is distinct from, and conflicting with, the filling-in hypothesis that has been advanced to explain the phenomenon of the invisibility of the scotomas. The so-called filling-in phenomenon was reported to occur only in special scenarios where the scene under and around the scotoma is either a uniform, colored surface^49,53 or a noise pattern.⁵⁴ In the filling-in, the content under the scotoma is elided, and then the field of view occupied by the scotoma (such a “hole”) is said to be replaced by the repeatable surrounding pattern. We note that filling-in with these scenarios is consistent with just carving out the elided content, without any replacement or filling-in. Thus, in the scotoma carving, nothing occupies the field of view of the scotoma (no hole), which is just simply removed from consciousness. The scene contents surrounding the scotoma are indistinguishable from the elided content in these special scenarios; a color surface remains a color surface, a noise pattern remains a noise pattern, and even periodic wallpaper-like patterns remain periodic patterns,^55,56 A carving-out process without any replacement (filling-in) will similarly result in invisible scotomas for these types of spatial patterns.

The filling-in effect has been demonstrated with periodic patterns^50,57 and static noise⁵⁰ patterns. However, filling-in has not been reported to occur with complex natural scenes of everyday life, as we used here. We have found only one report that studied the filling-in effect with complex natural images or scenes on 10 patients with cortical scotomas,⁵⁸ and this report concluded “that filling-in is impossible for complex visual patterns.” In fact, no one has even suggested what filling-in effects in natural scene might look like.

The shrinking of lines passing through the invisible scotomas, as demonstrated by Andrews and Campbell²⁹ and Safran et al.⁶ is direct evidence for carving out and against filling-in. Under any type of filling-in, the length of the lines should not be changed. Whatever mechanisms that might cause filling-in when applied to a natural scene should create some false filling-in effects, as it cannot fill in with the elided information based on the surrounding information (e.g., smaller detail than scotoma size). It has been demonstrated that the filling-in effect draws information from a very narrow ring stimulus surrounding the scotoma. Color filling-in was shown to be achieved by a mere 0.05° wide ring of color surrounding the physiological scotoma and by a texture ring of only 0.2° width at the edge of that scotoma.⁵⁹ We are preparing a second manuscript on the invisibility of scotomas, which includes an actual filling-in effect, where complex image content from outside the scotoma is used to create spatially correlated content inside the scotoma (Scotoma Replacement, in preparation). This second process applies in cases of field loss due to photoreceptors loss. Our scotoma replacement process does account for the thin ring filling-in.⁵⁹ However, that process does not apply to scotomas due to loss at the retinal ganglion cell or higher neurons, which are addressed here.

A major difference between the scotoma carving hypothesis and the filling-in hypothesis is the different appearance of the image content that is not directly affected by the scotoma. The filling-in hypothesis does not require, nor does it allow any changes in the appearance of the content outside of the scotoma. In the filling-in, the hole in the image caused by the scotoma (i.e., the field of view covered by the scotoma) is filled in, and thus, the space outside the scotoma should not be affected by that loss. It is similar to the content-aware filling algorithms such as the Magic Eraser applied in the Google Pixel 6 (Google, Mountain View, CA) or the Object Eraser applied in the Samsung Galaxy (Samsung, Suwon, South Korea), which result in distortion only in the applied area (i.e., scotoma). In the scotoma carving case, the removal of the elided content leaves a void in the image/scene, which, when it is closed, as the carving process requires, pulls in the image content outside of the scotoma from both sides of the scotomas toward each other (the right side toward the left and vice versa, and the upper toward the lower, etc.), resulting in local spatial distortions. In applying the scotoma carving algorithm, the seams that are forced to pass through the scotoma diverge outside the scotoma, resulting in the distortions reduced farther away from the scotoma (Fig. 4B). That effect is apparent with the scotoma carving processing that removes whole seams. Because a typical discrete scotoma occupies a very small fraction of the visual field, the scotoma carving ends up with virtually no visible distortions farther away from the scotoma. All these phenomena are consistent with the description of distortions provided by patients.

Local distortions outside scotomas have been reported for central^1,60,61 and peripheral scotomas.^62,63 The scotoma carving process resulted in local distortions outside the scotoma area, as we have shown here in a preliminary way. These may or may not be visible in peripheral locations or even centrally. The invisibility of local distortions in the normal periphery has been previously shown.^30,31 Results reported about the nature of peripheral scotoma distortions are not uniform across all studies, and therefore, not all are consistent with the effect of scotoma carving effects. Gerrits and Timmerman⁵¹ reported that, for patients with AMD, a slit of light going through the scotoma was never reported to be complete (a break in the line was noted). However, Ramachandran^52,64 did find a completion of bright line across scotomas—although the process took a few seconds to occur. Zur and Ullman⁵⁰ reported that two of their subjects observed a small break in the line going through their AMD scotomas, whereas a third patient reported a complete line but with a central segment of lower contrast. When six such lines were combined into a grating pattern, they were seen as complete by all three subjects. Such line(s) through a scotoma will be completed by scotoma carving but may be interrupted, depending on the angle of intersection with the scotoma border. A related effect was reported by Safran and Landis,⁶⁵ where patients reported a missing section in the Amsler grid, which was, nevertheless, much smaller than the scotoma measured perimetrically. However, the same patients did not report such gaps when looking at natural images, again pointing to the nature of the Amsler grid as a perimetry device.

The missing (elided) content under a scotoma and at the borders of transitions between two residual areas around a scotoma is frequently barely noticeable with complex natural images after the application of the scotoma carving, even when the carved images are examined carefully. The loss of content is easier to note when the carving eliminates an object of interest (i.e., a face), but even in these situations, the visual system seems to be forgiving, especially when the effect is in peripheral vision or after saccadic eye movement as we demonstrated. Sergent⁶⁶ noted in the context of visual field loss that “perception is aimed at objects, not at patterns of light” leading to “intrinsic difficulty in attending to the pattern of light rather than to the object as such.”

The apparent invisibility or low visibility of scotoma borders that are brought into apposition by the carving process is similar to the low visibility of cuts in wallpaper.⁵⁶ Wallpaper patterns are specifically designed to work well when carved out to fit the dimensions of the walls where the abutting cuts may not be matching sections of the pattern, yet they are not too disturbing to the eye, especially in peripheral vision. Similar designs are applied to cloth materials printed for shirt or dress manufacturing. The acceptability of such mismatched patterns to the visual system may be related to the amodal completion of occluded parts of objects. Amodal perception is the perception of the whole of an object when only parts of it affect the retina.⁶⁷ The light pattern mismatches seen with the scotoma carving are similar to the light patterns at the edges of occluding surfaces. Occluding surfaces or parts of objects are very common in the environment, so they are familiar to the visual system and are not bothersome. For example, the carved-out parts of the cars in Fig. 4 appear to be occluded behind walls. The dynamic changes of occlusion and uncovering of such objects when one moves about in the environment are also very common and appear to be similar to the changes that occur at carved scotomas, as the simulated person is moving in the environment (see Video 4, available at http://links.lww.com/OPX/A625). Note that such optic flow movements are larger in magnitude in the peripheral vision where they are less noticeable because of the reduced resolution and contrast sensitivity at higher eccentricity.

Schuchard⁵⁷ evaluated the perception of Amsler grid test patterns in patients with long-standing bilateral macular scotomas. He found significant and substantial underreporting of the scotoma, as also reported by Safran and Landis⁶⁵ and Achard et al.⁶⁸ The size of the area with no visible grid lines within an Amsler grid reported by patients was found to be much smaller than the scotoma mapped perimetrically. However, when not performing the Amsler grid task, these same patients did not perceive such gaps when looking at natural images. Our interest here is limited to the effects noted when observing natural scenes. The application of the scotoma carving algorithm is subject to numerous artifacts when applied to degenerate test stimuli such as the Amsler grid or a few lines.

The distortion reported by patients with disciform AMD and illustrated here has been interpreted by Massof⁹ to suggest a perceptual closure of visual space around the scotoma analogous to the closure of a drawstring bag. This drawstring closure hypothesis is similar but not identical to our scotoma carving. In particular, the cause and nature of the distortions outside the scotoma are different in nature in these two cases. The reports of these spatial distortions surrounding AMD central scotomas led to the design of field remapping vision aids using counter distortions.^9,69–72 These authors proposed that the distortion introduced around the scotomas may be fully or largely compensated by a remapping image processing. This aligns with the surround distortion after the scotoma carving and highlights the potential of (scotoma carving) simulations to aid in designing and evaluating vision rehabilitation devices and techniques. However, our findings of the low impact of the elided content and the practical invisibility of the distortions associated with the scotoma carving show that such treatments may not be very effective or valued, except perhaps for the central scotoma case. Even there, the effect may be noticeable only in some cases, depending on local image content.

If an artificial scotoma (a square of uniform gray level) is placed over a repeated periodic grating background, the artificial scotoma is perceived to be replaced by the repeated pattern,⁵⁴ but no such filling-in effect was found when the field of view surrounding the artificial scotoma was filled with square grid of random letters. An important distinction between the filling-in of natural scotoma (optic nerve head scotoma or pathological scotoma) and the artificial scotomas is that the filling-in of natural scotomas is apparent immediately upon testing, whereas filling-in phenomena with artificial scotomas take time to develop.^56,73–75 The invisibility of monocular scotomas was reported immediately after the covering of the fellow healthy eye.^51,76 The scotoma invisibility phenomena discussed here also develop instantaneously. This aspect is particularly important when the interactions between the scotomas and eye movements are considered, as discussed hereinafter. Spatial distortion reported around central disciform scotomas caused, as a result of chorioretinal scaring after wet AMD, has been thought to be related to physical deformation of the retina near the scar at the scarring process.^77,78 The scotoma carving process results in such distortions without postulating any retinal mechanical distortions (Fig. 6). The stipulation of retinal distortion as a cause for metamorphopsia assumes retinotopy mapping in the cortex. Indeed, such a mapping occurs at the lower levels of the cortex, but it is known to change toward object-centered mapping at higher cortical centers.^79,80 More importantly, even at the lower cortical level, the retinotopy is neither simple nor linear. There is retinotopy in the gross sense that stimuli to the right of the fovea are mapped that way, but stimuli twice as far to the right are mapped farther to the right but may not necessarily be twice as far. Such relationships occur only with connected content. There are numerous examples of distorted spatial relationships under various conditions such as perisaccadic mislocalization⁸¹ and compression⁸² and with numerous spatial illusions.

The simulation of the effect of retinal eccentricity on the resolution used here (Figs. 5C, 6, 7) was applied using the pyramidal multiscale model applying measured contrast detection thresholds.⁴² That parameter-free model was used to generate simulations that were quantitatively verified for normal central vision of natural images.⁸³ The peripheral vision eccentricity effect was added based on the measurement of contrast sensitivity across the retina and data from numerous other studies cited in Peli et al.³² The eccentricity model was fitted to derive a single eccentricity factor using experiments with complex images presented on a 64° wide display.⁴³ Other computational models of vision across the visual field were proposed and applied to such simulations.^84–87 However, verifications with complex images were not reported. When these simulations of peripheral vision were combined with representation of scotomas, the central scotomas were either left out of the presented images^32,84 or depicted as highly visible gray patches.⁸⁵ Instead of eliminating the elided content under the scotoma, blurred versions of that content have been presented by Geisler and Perry⁸⁸ and Jones et al.⁸⁹ for glaucomatous arcuate scotomas and by Goldstein et al.³⁵ for central scotoma in AMD (using the Geisler's algorithm). However, these processes use the image data from within the scotomas, which is not accessible to a visual system with the scotomas and only blended/blurred that image content and did not fully delete the data from within the scotomas. In our scotoma carving process, the scotoma areas remain within the displayed image, and the image content covered by the scotoma is completely eliminated.

In considering visual simulations, presenting images at the correct angular span is considered crucial if the observers are asked to observe the images for themselves. The angular image size was explicitly addressed for the simulations presented by Capilla et al.⁸⁴ and by Peli et al.^83,90,91 The angular image size was explicitly manipulated by varying viewing distances in verifying the simulations.⁸³ Wide peripheral images are difficult to display at the correct size. We used images cropped to the angular size of 45 or 60° so that readers can match the size on their display and thus experience the eccentricity effects in the (in)visibility of the scotoma. However, if the simulation is of peripheral scotomas that do not include the fovea, the relationship derived in the study by Peli et al⁹⁰ ensures that change in display angular size with distance maintains the effect of eccentricity on the content, enabling changes in viewing distance without disrupting the effect if readers maintain fixation. Only when the scotoma includes the fovea resulting in eccentric fixation being simulated does the display size have to be maintained to veridically illustrate the effect of eccentricity.

Clinical Significance

Patients with glaucoma are not aware of their field loss, leading to calling the glaucoma the “silent thief” of sight.⁹² We have demonstrated and explained here that the patients do not gain awareness of the field loss even when saccadic eye movements result in uncovering objects from under the scotomas and covering other objects, or with the gradual disappearing and reemerging of objects from the scotoma during smooth pursuit eye movements or due to optic flow when walking or driving. When the patients are told by their doctors that they have field loss, are shown the perimetric diagrams, as a proof, and see online the wrong common simulations with the black patches representing such loss, they immediately and easily determine that these illustrations do not represent their own visual perception.¹⁰ Understandingly, they conclude that all these issues do not apply to them. It is little wonder that these patients’ compliance with drug treatments prescribed to them is very low.^93,94 It is likely that proper explanation of the invisibility of the field loss, as presented here, despite its complexity, will educate the patients about the nature of the disease and its visual effect and will encourage better compliance despite the invisible loss. Furthermore, with such better explanations, patients may be taught to spot and monitor their invisible field loss and thus be more aware of the risk to their vision and the need for treatment.

Varieties of training of eccentric fixation or modifying the preferred retinal locus was developed over the years. Many of these required awareness of the scotoma boundary and the location of the preferred retinal locus relative to the scotoma.⁹⁵ Such performance would be impeded by the invisibility of the scotoma. In one study, the outline of the scotoma was graphically presented within the functioning retina around the scotoma, and the patient was trained in reading with this artificial visibility of the scotoma. Only one of eight patients with AMD who practiced reading did not improve his reading speed after that brief practice.⁹⁶ This is an example of the consideration of the invisibility of the scotoma in directly designing a rehabilitation technique/device.

Limitations

In this paper, we addressed only discrete relatively small scotomas surrounded by functional vision on all sides. The processing and analyses do not apply directly to other types of scotomas such as homonymous hemianopia, loss of the temporal crescent on one side,² and tunnel vision due to loss of peripheral field.⁹⁷ In all these cases, the scotomas extend into the edge of the normal peripheral field. Understanding and simulating perception with these scotomas may require a different approach. However, these do share one clear characteristic with the discrete scotomas addressed here: the lack of conscious appreciation of the lost field. The fact that this occurs in these conditions is surprising and dramatic, especially when one considers that the central borders of the scotoma in many cases are very close to foveal vision and, in some cases (homonymous hemianopia), practically splitting the foveal field, yet they remain invisible. Another very dramatic demonstration of this effect is with retinitis pigmentosa patients with very narrow residual tunnel vision. We have seen in the laboratory three patients with only 5, 10, and 20° diameters of central residual fields, respectively, who reported driving regularly (this may be legal in some states). One of them who worked as a traveling salesman reported four minor collisions over the last year. Amazingly, all these patients denied any perceived restriction of their visual field. Currently, we do not know how to simulate these field losses in a way that is comparable to the scotoma carving.

The field of view outside the normal visual field (i.e., behind one's head) is invisible. It clearly does not appear black or have any appearance at all, nor is there any transition noted between the seen field of view and the elided image content behind the head. That transition occurs at the far periphery where resolution and contrast sensitivity are both very poor, which may account for the invisibility of the transition. The transition from seeing to nonseeing visual field in hemianopia is reported to have similar perceptual characteristics.^66,98 Despite taking place at field eccentricity with much higher resolution and contrast sensitivity, the missing field is not perceived as a black area (despite frequently being simulated this way), and the border or transition from seeing to nonseeing is not apparent. Similar effects are reported by patients with tunnel vision due to advanced retinitis pigmentosa or glaucoma. These conditions are outside of the scope of this paper, and application of scotoma carving algorithm to these field losses does not yet contribute much to the understanding of the phenomenology.

CONCLUSIONS

The concept of scotoma carving presented here is indeed a hypothesis aimed at providing a plausible mechanistic illustration of the well-established facts that scotomas are not visible, not as black patches or in any other previously suggested ways. Our proposed scotoma carving was applied using the seam carving algorithm, but this is not intended to suggest this specific algorithm or model as the postulated physiological mechanism. The carving of elided content in the scotoma may not need any active process or mechanisms. The content may be simply ignored, as is the scene behind our head.⁵³ Rather, we suggest that the invisibility of spatiotemporal content within the scotoma results in a collapse of perceptual spatial relations across the scotoma. That process is, by definition, different from “filling-in” and results in local distortions outside of the scotoma, unlike filling-in. Content-aware processing in the brain, similar in general nature to seam carving, would be an effective way of distributing and diffusing the distortions as the distance from the scotoma increases. Our preliminary presentation is strongly supported by the fact that numerous results in the literature regarding observations and studies with various discrete scotomas are consistent with the scotoma carving effects. The scotoma carving process itself has been developed only preliminarily and needs further work. Once that is completed or sufficiently advanced, we will be able to use the technique to create specific (and quantitative) predictions that may be tested with patients both in terms of appearance and perception and for quantitative performance on psychophysical tasks. We are currently piloting studies in which the existence of distortions of the type described qualitatively by patients with homonymous pericentral cortical scotomas⁶ will be documented and measured quantitatively.

Supplementary Material

ovs-100-515-s001.docx ^{(111.3KB, docx)}

Download video file ^{(6.4MB, mp4)}

Download video file ^{(5.6MB, mp4)}

Download video file ^{(13.5MB, mp4)}

Download video file ^{(10.5MB, mp4)}