Resolving visual motion through perceptual gaps

Abstract

Perceptual gaps can be caused by objects in the foreground temporarily occluding objects in the background or by eyeblinks, which briefly but frequently interrupt visual information. Resolving visual motion across perceptual gaps is particularly challenging, as object position changes during the gap. We examine how visual motion is maintained and updated through externally-driven (occlusion) and internally-driven (eyeblinks) perceptual gaps. Focusing on both phenomenology and potential mechanisms such as suppression, extrapolation, and integration, we present a framework for how perceptual gaps are resolved over space and time. We finish by highlighting critical questions and directions for future work.

The challenge of processing motion through perceptual gaps

Incoming visual information is often interrupted by externally- and internally-driven events such as occlusion (see Glossary) and eyeblinks, producing perceptual gaps (Figure 1). For example, cars moving on a highway are often fully or partially occluded by other traffic. Similarly, eyeblinks, which can occur 15–20 times per minute [1] and may last for up to 500 ms [2,3], prevent visual information from reaching the brain. One way of bridging these perceptual gaps is to bias perception towards the representation of the last visible information prior to the gap [4]. Retaining information about object identity, for example, can be achieved by relying on the object features last seen [5], as object identity rarely changes during the gap. In contrast, retaining visual motion through the gap is challenging, as a moving object changes position while out of sight. Thus, the visual system needs to update position information during perceptual gaps. To be able to update object position, the direction of motion and the rate of movement needs to be maintained to successfully estimate when and where an object will re-appear. Together, updating position and maintaining a representation of velocity allows for resolving visual motion through perceptual gaps which is critical to prepare for actions, navigate the environment successfully, and avoid collisions.

Figure 1. Illustration of perceptual gaps.

Perceptual gaps can be externally-driven (occlusion) or internally-driven (eyeblinks). In both cases, even though incoming visual information is unavailable during the gap, we still perceive visual events as continuous.

Recent developments in neuroimaging and behavioural paradigms have increased our ability to investigate the mechanisms for updating motion information during perceptual gaps. For example, advances in analysis methods for time-resolved neuroimaging methods such as electroencephalography (EEG) and magnetoencephalography (MEG) enable a deeper understanding of inherently dynamic processes [6] and have recently been employed to examine processes such as motion extrapolation [7,8]. In addition, modelling techniques have been used to examine how these processes could be instantiated in the brain [9].

Using eyeblinks and occlusion as models, the current review draws on evidence from behavioural and neuroscience studies to discuss how visual motion information is maintained and updated through externally- and internally-driven perceptual gaps. While saccades also interrupt visual information and share some of the characteristics and mechanisms of bridging perceptual gaps caused by eyeblinks and occlusion, we will not focus on this special case in detail (Box 1). We begin by reviewing physical motion and illusory motion processing, which we discuss in the context of perceptual gaps. Then we consider the experience of visual motion during occlusion and eyeblinks and identify possible mechanisms that support motion processing through these gaps. We conclude that motion perception before and after the perceptual gap is critical to make accurate predictions and integrate and resolve perceptual gaps across space and time.

BOX 1: Perceptual gaps caused by saccadic eye-movements.

Disruptions in visual sensory input can also arise from saccadic eye-movements. A saccade is a ballistic eye-movement where fixation is transferred from one spatial location to another, enabling rapid sampling of our visual environment. While both saccades and eyeblinks produce perceptual gaps through physical movement of the eye, we focus on eyeblinks in our review for two reasons. First, eyeblinks cause a physical obstruction between the retina and sensory input with the closing of the eyelid, similar to externally-driven perceptual gaps with object occlusion. In contrast, during a saccade, light still impinges on the retina and visual processing is thought to be inhibited through saccadic suppression. However, sensory input inhibition is incomplete. With training, suppression can actually be dampened if stimulus presentation during a saccade is task relevant [133,134]. In fact, moving task relevant stimuli during saccade flight can be detected and used to support post-saccadic gaze correction [135]. Second, during a saccade, the spatial position of objects needs to be updated for both the saccadic eye-movement and the external motion adding significant complexity to comprehending motion processing during such perceptual gaps. Integration of information across saccades could be supported by receptive field remapping, where neurons with receptive fields which that are about to receive salient information due to a saccade activate start responding in expectation [136,137]. Remapping may also occur for the relocation of attention across saccades, adding complexity for determining which receptive fields are prospectively activated. Information about an attended object may also be maintained across saccade, adding a role for trans-saccadic memory [138]. However, despite these differences, it is notable that eyeblinks and saccades may have similar underlying mechanisms which bridge the perceptual gap they cause. These include suppression around the eye-movement and prediction of post eye-movement sensory input via prior knowledge.

The neural substrates of motion processing

Both physical and illusory motion processing can potentially inform our understanding of how visual motion is resolved through perceptual gaps. Before and after the perceptual gap, the object moves visibly along a motion trajectory. Processing physical motion before the gap informs predictions about the object’s position during the gap. Furthermore, physical motion after the gap can reinforce the perception of an object trajectory retrospectively. Maintenance of a motion trajectory before and after occlusion has such a strong impact that if the object that passes behind the occluder differs in visual features from the one that emerges at the expected position and time, it is perceived as a single object that changed slightly, and not as two separate objects [10–13]. Further, visual illusions of motion such as apparent motion demonstrate that representations of spatiotemporal content can occur in the absence of explicit visual stimulation. Thus, the mechanisms supporting illusory motion processing may also support motion trajectory maintenance during perceptual gaps.

Physical motion processing

To resolve visual motion across perceptual gaps, physical motion information needs to be integrated over space and time (from before to after the perceptual gap). Visual motion processing is supported by a hierarchy of regions and motion selectivity is observed as early as the retina [14,15] (Figure 2A). Direction-selective neurons in primary visual cortex (V1) process motion as a local, one-dimensional feature and the integration of these local signals into coherent motion patterns is thought to occur in V5/MT (in humans often referred to as hMT) [16]. In addition to V5/MT, areas MST [17], V3A [18], V6 [19] and regions in the intraparietal sulcus (IPS) also represent global motion [20,21]. Global motion signals have even been reported in V2 and V3 [22]. These motion-selective areas also integrate motion over time [23,24]. In particular, the lateral intraparietal area (LIP) within non-human primate IPS has been characterized as temporally integrating motion information over long delays in perceptual decision making tasks [e.g., 25].

Figure 2. Physical and illusory motion processing.

(A) Physical motion processing pathway. The earliest motion-selective cells are within the retina. From there information is relayed upstream to the LGN, which projects mostly to V1 but with some direct projections to V5/MT. V1 also projects directly onto V5/MT, which has dense connections with regions around the intraparietal sulcus (IPS). While V1 neurons have small receptive fields and are thought to represent local motion, V5/MT represents global motion. (B) Apparent motion occurs when the position of an object is varied quickly over time between discrete locations. The subjective experience is that the object moves smoothly between those locations.

Motion processing prior to perceptual gaps may play a role in predicting motion trajectories during perceptual gaps. Motion illusions such as the flash-lag effect [26] demonstrate that we predict the current position of a moving object based on its past trajectory. The flash-lag effect occurs when a stationary object is flashed right next to a moving object and participants (falsely) perceive the position of the stationary object to lag behind the moving object. Transcranial magnetic stimulation (TMS) to V5/MT reduces the flash-lag effect [27], indicating that extrapolation relies on activity in V5/MT. Extrapolating motion trajectories inferred from physical motion into perceptual gaps is likely one of the key processes involved in resolving motion information through gaps.

Illusory motion processing

When objects move out of sight during perceptual gaps such as occlusion, we still have a sense that the object persists and keeps moving [10,28]. Due to the lack of physical motion input during occlusion, perceived motion must be internally-generated or maintained. One of the most striking examples of internally-generated motion is apparent motion where we perceive illusory motion in response to discrete visual stimulation in different positions of the visual field [29] (Figure 2B). For instance, when we watch a movie, we readily perceive fluid motion even though the movie consists of a sequence of stationary images. While there are some differences in the conscious experience of motion induced by apparent motion and perceptual gaps, the maintenance of internally-generated motion signals may rely on similar processes in both cases.

Motion perception in apparent motion closely resembles the perception of physical motion, suggesting we actually perceive a stimulus moving along an illusory motion trajectory [30]. The experience of such apparent motion is so strong and spatiotemporally specific that it can interfere with the detection of another object along the illusory motion trajectory [31–33]. Perception of apparent motion involves similar cortical regions as perception of physical motion. Several studies have shown the involvement of V5/MT during apparent motion [34–37]. For example, single-pulse TMS applied to V5/MT reduced the impact of apparent motion on detection of targets along the apparent motion path [37]. In addition to V5/MT, parietal areas have also been implicated in apparent motion perception. LIP in macaques responds strongly to apparent motion when perceived in the preferred direction of the recorded neurons [38]. There is also evidence that V1 is critical for illusory motion awareness [39–41] with increased activity observed along the apparent motion path [42,43]. As there is no true visual stimulation in the visual field locations corresponding to the apparent motion path, the activation in V1 must be driven by feedback connections from higher level visual areas. Although long-range lateral connections within V1 may also support apparent motion processing [44], they are unlikely to explain all activity. In particular, predictive activity related to the apparent motion illusion has been shown to transfer from V1 in one hemisphere to the other across an eye-movement [45]. This indicates a role for feedback as there are limited callosal connections within V1 [46]. Collectively, these studies demonstrate a position-specific representation for illusory apparent motion stimuli that is similar to physical motion, suggesting the same may exist for internally-maintained moving objects in perceptual gaps.

Although similarities exist, there are also important differences between internally-maintained motion perceived in apparent motion and during perceptual gaps such as occlusion. First, the experience of object persistence through periods of occlusion is spontaneous and automatic, occurring as soon as an object moves out of sight. In contrast, for apparent motion to occur, the object needs to re-appear. Second, when an eyeblink or occlusion occurs, the object disappears gradually, and this may be critical for eliciting the internally-maintained motion. In contrast, apparent motion can be elicited with sudden offsets. These differences in phenomenology suggest that a combination of physical and illusory motion processing is central to resolving perceptual gaps.

The phenomenology of perceptual gaps

In natural vision, perceptual gaps are frequent, but these interruptions do not break the perceived continuity of motion trajectories. For eyeblinks in particular, it has been suggested that the perception of continuity is supported by both the suppression of the visual consequences of an eyeblink [3] and an underestimation of the eyeblink duration [47–50]. In eyeblink suppression, efference copy may trigger preparation for the visual consequences of closing the eyelid by lowering visual sensitivity prior to the gap [3]. Blink suppression is thought to reduce the prominence of the gap so visual information before and after the gap is perceived as one continuous visual event. Suppression alone may not be sufficient because the measured neural suppression of light sensitivity is much less than the perceived change in luminance caused by the closure of the eyelid [3,47,49–51]. Blink duration underestimation may also contribute by minimizing the perceived gap. For stationary stimuli, this underestimation may promote continuous and stable perception. However, for moving stimuli, underestimated blink durations could lead to inaccurate position tracking during perceptual gaps. Furthermore, the extent of blink duration underestimation is not modulated by the actual eyeblink duration [47], making position estimation of moving stimuli more challenging. Collectively, while suppression and underestimated blink duration perception likely contribute to minimizing non-informative sensory input, they are not sufficient for resolving visual motion across perceptual gaps because position information needs to be updated.

One way of updating position information during perceptual gaps is to use the physical motion information before the gap to predict the trajectory during the gap. Extrapolated trajectories have been shown to extend into perceptual gaps. For example, position information is represented when an object dynamically moves into the retinal blind spot [52,53]. Similarly, when an eyeblink occurs [54], recent evidence suggests that extrapolation occurs involuntarily into perceptual gaps (Figure 3). Participants observed an object moving smoothly on a trajectory and when they blinked the object would disappear. When reporting the object’s last visible position prior to the eyeblink, participants reported the position to be displaced in the direction of motion, suggesting that motion trajectories were extrapolated. However, when the object just disappeared at a random location in the absence of an eyeblink, there was no overshoot, indicating that the extrapolation is not linked to disappearance of the object per se. Thus, the gradual disappearance of the object may be an important feature when extrapolating trajectories into perceptual gaps. When an object suddenly vanishes, the perceived last visual position is not consistent with the extrapolated motion trajectory [55,56], suggesting that sudden offsets overwrite extrapolated positions [7,57–61]. In contrast, when an object disappears gradually, extrapolation seems to extend into the perceptual gap, as the gradual nature of disappearance makes it more likely that the object continues to exist [28]. Importantly, extrapolation may not bridge the entire perceptual gap. In an eyeblink paradigm [54], participants perceived continuity in motion when the moving object jumped backward during the eyeblink. This might reflect the fact that extrapolation only covers the perceived duration of the gap, which is shorter than the physical duration. Thus, while extrapolation likely enables motion information to be extended into the perceptual gap, it may not be enough to bridge larger gaps.

Figure 3. Involuntary motion extrapolation through a perceptual gap.

Participants observed a stimulus moving along a circular trajectory. When an involuntary blink occurred, the stimulus disappeared, and the participants were asked to indicate where the last visible position was. The results showed that independent of blink duration the response was biased to reflect a perception of the moving stimulus ahead of time. When the stimulus disappeared without a blink occurring, this overshoot was not observed suggesting that the sudden offset overwrote the extrapolated trajectory. (Maus et al. 2020, Figure 2b, redrawn with permission [54])

To be able to update positions during perceptual gaps, velocity information must be maintained. Infants have been shown to be able to consistently incorporate velocity information into the prediction of object-reappearance after occlusion [62,63]. Similarly, the perceived position of moving objects during eyeblinks has been shown to be influenced by velocity, although the estimation of velocity may be inaccurate [54]. It seems that this ability to maintain velocity information is mainly a perceptual feature of the visual system and not a motor feature, as ocular pursuit velocity decreases substantially while tracking an object during occlusion [64]. However, when contrasting groups who are free to engage in ocular pursuit versus groups who fixate at the re-appearance location, there is an advantage in estimating when the object will re-appear for the ocular pursuit group [65]. This suggests that ocular pursuit prior to occlusion does help to maintain velocity information through the gap. However, second order derivatives of velocity (i.e., acceleration) do not seem to be updated during occlusion, even if acceleration before occlusion is psychophysically detectable [66].

Motion trajectories across perceptual gaps are perceived to be valid when the object’s position after the gap is consistent with the physical motion trajectory observed before the gap. However, when an object after occlusion re-appears in an unexpected position or at an unexpected time, object continuity is interrupted [67–72]. This highlights that the predicted motion trajectory is integrated with the physical motion after the gap. As small prediction errors can occur when extrapolating motion, for example, over long distances or when distractors are present [73], small differences between the predicted and actual position after re-appearance should not interrupt the perception of a valid trajectory. In non-human primates, repeated small, systematic differences in time or position result in an adaptation to the latency or endpoint of saccades predicting the re-appearance of the object behind an occluder [74]. Similarly, if an object is systematically displaced during an eyeblink, humans’ post-eyeblink fixation adjusts towards the displaced position within ~35 trials [75,76]. This indicates that the process of motion extrapolation incorporates feedback from the re-appearance after perceptual gaps, highlighting that prediction occurs within the context of integrating physical motion trajectories across the gap.

Internally maintained motion trajectories are not only anchored by physical motion perception before and after the perceptual gap but are also influenced by the physical presence of the occluder. In apparent motion paradigms, it has been shown that large temporal or spatial gaps erode the perception of smooth motion trajectories [77,78]. However, the presence of an occluder can maintain the perception of smooth motion trajectories despite large gaps [79]. In addition, the shape of the occluder has been shown to influence perceived illusory trajectories [80,81]. When a curved occluder is presented adjacent to apparent motion, the illusory moving object is often perceived to follow a curved trajectory [81]. The likelihood of perceiving the curved illusory trajectory increases as time between disappearance and re-appearance of the object increases, suggesting that perception of the illusory trajectory changes, based on the time that has passed since the object disappeared. These findings demonstrate that the physical presence of the occluder plays a key role in maintaining motion trajectories.

In sum, resolving visual motion through perceptual gaps likely involves a combination of suppression, extrapolation, and integration. However, there are differences when comparing mechanisms which contribute to externally- or internally-driven perceptual gaps. Some of these differences relate to the nature of the interruption caused by blinks versus occlusion. For example, the duration of eyeblinks is relatively constant [47], while the duration of occlusion varies with the size of the occluder and the speed of motion. Similarly, the speed of gradual disappearance and re-appearance is skewed for eyeblinks with the gradual disappearance taking a short time and reappearance taking longer (~50 ms to get from open to closed and ~300 ms to get from closed to 97% open) [2]. In contrast, when the speed of object motion remains the same, the gradual disappearance and re-appearance of an object during occlusion happen at the same rate. Finally, we can adjust blink frequency and blink timing in response to task demands [82], allowing for minimal loss of relevant information. In contrast, occluding events can generally not be avoided, as external occluders are rarely under the control of the observer.

Mechanisms supporting motion processing through perceptual gaps

Where is object information maintained during perceptual gaps?

Multiple motion-sensitive cortical regions have been implicated in maintaining representations during perceptual gaps. Similar to representations during apparent motion [83,84], position-specific information has been shown to be represented in early visual cortex during perceptual gaps [85–87]. In one fMRI study [87], participants viewed an object moving on a circular trajectory that was dynamically occluded in one quadrant. Within retinotopically defined areas of V1, V2, and V3 that corresponded to the position of the occluder, the results showed higher activation during occlusion in comparison to a control condition in which the object disappeared (Figure 4). Interestingly, the signal evoked by the occluded object in early visual regions was not shape-specific suggesting only position information was maintained behind the occluder. This is consistent with behavioural findings from the occlusion and eyeblink literature, showing that participants are often unaware of color or shape changes during perceptual gaps [10,12,88,89]. The lack of identity information during occlusion might point to a difference between representations during apparent motion and perceptual gaps, as object position and identity have been shown to be represented in V1 during apparent motion [90]. However, identity information must be retained to some degree, as temporal frequency, for example, has been demonstrated to influence perceived duration during occlusion [91]. One possibility is that identity information of occluded objects is represented in areas other than early visual cortex. For example, a non-human primate study reported that neurons in the banks of the superior temporal sulcus respond when people, but not other objects, become occluded [92].

Figure 4. Position-specific information during occlusion.

Participants viewed an object moving on a circular trajectory. In the occlusion condition, the object moved behind an occluder without visible edges (quadrant and trajectory lines depicted were added for demonstration only). Then the object re-appeared on the other side of the occluder. In the disappearance condition, the object vanished as soon as it came in contact with the occluder and re-appeared after a delay on the other side of the occluder. In the occlusion condition, the object was perceived to persist behind the occluder, while no object was perceived during the gap in the disappearance condition. In line with this perceptual experience, the neuroimaging results showed that there was position-specific information in visual cortex for the occlusion but not the disappearance condition. (Erlikhman & Caplovitz (2017), Figure 3, redrawn with permission [87])

Human neuroimaging and non-human primate physiology studies also show a contribution of areas along the dorsal visual pathway [93], presumably capturing extrapolated motion paths across the perceptual gap [94–99]. For example, a study on non-human primates examined posterior parietal cortex responses during visual motion tracking with a joystick [96]. The animals had to track a moving object which was briefly occluded in a subset of trials. The results showed that neurons in LIP displayed direction-selective activity for both occluded and visible objects while neurons within MST only responded to the object when it was visible. Relatedly, results of an fMRI study [98] on humans revealed activity in IPS and V5/MT when an object is temporarily occluded versus when it suddenly disappeared.

How is motion information maintained through perceptual gaps?

Extrapolation is an inherent part of motion processing. When motion signals are processed, previous trajectory information is still present throughout the visual system while current position information stimulates the retina [100]. Recent time-series neuroimaging work has demonstrated that there is a neural representation of predicted positions of a moving object ahead of time [7,8,101]. For example, the results of an EEG study [7] have shown that the position of an object can be decoded ahead of time when the object follows a predictable motion sequence, showing that position information is available before the object arrives there. Similar automatic anticipatory motion effects can be observed in motion illusions such as the flash-lag effect. Extrapolation is likely the driving mechanism behind the flash-lag illusion [57] and explicit models have related extrapolation to the effect [9,102,103]. In particular, neural populations in an artificial neural network learn to extrapolate without supervision through spike-timing dependent plasticity [9]. Comparing the extrapolation shift observed in the neural network with the magnitude of the flash-lag effect observed in humans [104], shows that the degree of extrapolation is highly similar (Figure 5A). This indicates that extrapolation occurs spontaneously and automatically when a system is introduced to motion, highlighting its central role in motion perception.

Figure 5. Evidence for extrapolation and suppression.

(A) The flash-lag effect (FLE) describes the illusion that a stationary flash is perceived to lag behind a stimulus in motion when they are actually aligned. This effect is due to extrapolation of the moving object resulting in the moving object being perceived ahead in time. The FLE scales with speed and this relationship can be fit along a horizontal asymptote (blue). When training a model, extrapolation occurs spontaneously with the receptive field (RF) of each neuron biasing perception ahead of time. This shift in RF also scales with motion speed (red). When correlating the FLE lag with the RF shift, there is a near perfect correlation, suggesting that the mechanism of extrapolation occurs artificially in a similar way as behaviorally. (Burkitt and Hogendoorn (2021), Figures 7 and 8, redrawn with permission [9]) (B) In another study (Golan et al., 2016), neural responses were recorded intracranially from visual cortex while patients viewed images of different things such as faces, houses, and objects. In some trials, a gap was inserted (i.e., black screen) to interrupt visual information and in other trials the authors looked at the interruption of visual information by spontaneous eyeblinks. Looking at the results, there was a strong evoked response after the stimulus re-appeared after the gap but not after the spontaneous eyeblink. This indicates that re-appearance signals are suppressed after an eyeblink but not after an artificial gap. (Golan et al. (2016), Figure 5, redrawn with permission [112])

There is also evidence from brain stimulation and patient studies that activity in parietal regions is modulated by position prediction effects [105,106]. Inhibitory TMS to the IPS reduces the impact of motion extrapolation on target detection [106]. These parietal effects may be related to the interruption of temporal processing of visual events [107,108], which may support motion extrapolation. Together, these findings suggest extrapolation extends into perceptual gaps and that parietal areas play a central role in supporting the representation of extrapolated motion trajectories.

As outlined in the phenomenology section, sensory suppression may also facilitate motion processing through perceptual gaps. Physiology research on non-human primates showed that neurons in early visual areas (V1-V4) respond differently when a perceptual gap occurs due to an eyeblink versus a sudden artificial gap (i.e., blank screen) [109,110]. When the gap was caused by an eyeblink, the visual response was suppressed in comparison to when the gap was caused by a blank screen. Similarly, human work has shown that even when retinal illumination is held constant during the eyeblink, activity in mid-level visual, parietal and prefrontal areas are suppressed [111]. In a recent patient study [112], intracranial recordings were obtained from visual areas while patients viewed images of different objects (e.g., faces, houses, tools). When comparing the neural data in trials where a spontaneous blink occurred with trials where an artificial gap (blank screen) was inserted, the re-appearance response in a higher-level visual area was suppressed for eyeblinks (Figure 5B).

Although efference copy could trigger blink suppression, the visual characteristics of an object gradually disappearing may also initiate suppressive effects [113]. This is more consistent with the observation of suppression in occlusion which is unrelated to motor movements. When there is no reliable depth information in a visual scene, intersections of the object contours are the only cues for occlusion [114,115]. The contours created by occlusion are not due to the object’s shape but are accidental contours that only appear because of occlusion. In a monkey physiology study, V4 responded strongly to true object contours while suppressing accidental contours created by occlusion of stationary objects [116]. The suppression of these accidental contours in V4 might be involved in the perception of a whole object behind the occluder instead of separate object fragments. It is likely that such suppression occurs in dynamic occlusion too. Together, the gradual nature of occlusion and eyeblinks seem to support the perception of persistence through perceptual gaps by maintaining extrapolated trajectories and suppressing post-blink sensory onsets.

There are several candidate regions that may support the integration of physical motion information with internally-maintained motion trajectories. Regions within the medial temporal lobe (MTL), such as the hippocampus, have been shown to represent temporospatial predictions. For example, visual space is represented within human entorhinal cortex in grid-type cells [117–119] with a subgroup of these cells firing in a direction-selective fashion prior to an upcoming saccade [120]. In line with these predictive signals in MTL, several fMRI studies have demonstrated that the hippocampus carries predictive signals for learnt sequences [121–125]. For example, participants in one study [121] were trained to respond to a visual cue with an action (right or left button press) which was then systematically associated with a specific visual outcome stimulus. During an fMRI scan, the trained participants completed the full trial sequences (i.e., cue + action = outcome), partial trial sequences (i.e., cue + action), and trials where they only saw the visual outcome stimulus. The results showed that the full sequences were represented in hippocampus while the outcome only stimulus could only be read-out from early visual areas. On trials where the full sequence was correctly decodable in hippocampus, the outcome could be predicted more reliably from V1. This suggests that the sequential, learned order of visual events is represented in hippocampus which feeds information about expectations back to visual cortex. These and other findings have led to the suggestion that completion of gaps between places or other events is one of the central functions of the hippocampus [126,127]. As resolving motion information through perceptual gaps relies on predicting sequences of object positions based on physical motion trajectories, the MTL seems like a potential candidate for integrating physical and internally-maintained motion signals through perceptual gaps.

In addition to regions in the MTL, another sub-cortical structure, the cerebellum, may play a role in resolving visual motion through perceptual gaps. The cerebellum is involved in representing temporal information that is relevant for motor and cognitive function [128,129]. During occlusion the cerebellum has been shown to be involved in time prediction [130–132]. For example, the results of an fMRI occlusion study [130] show that a subregion of the cerebellum was recruited when the velocity of an object changed over time. In this experiment, a moving object became dynamically occluded and re-appeared either too early or too late (velocity judgment task) or shifted to the right or the left (spatial judgement task). The posterior cerebellum became active only during the velocity judgment task with functional connectivity analysis showing a stronger interaction with IPS, the frontal eye fields, and MT. This finding suggests that the cerebellum is involved in predicting visual motion when a change over time is likely. Future work will have to examine whether regions in the cerebellum are recruited to resolve visual motion across perceptual gaps independently of task demands.

Concluding remarks

Resolving visual information through perceptual gaps is a fundamental part of everyday vision, as eyeblinks and occlusion occur frequently. Here, we highlighted why moving stimuli pose a particular challenge to bridging perceptual gaps and showed that mechanisms such as suppression, extrapolation, and integration are important to update visual motion through periods of occlusion and eyeblinks.

Generally, maintaining and updating motion information happens automatically and without conscious effort. However, many studies require participants to explicitly perform a task related to the perceptual gap, which may change the nature of the experience. In future work it will be important to test more naturalistic conditions, which will require a focus on neural activity in the absence of explicit behaviour. The focus on early visual areas in previous neuroimaging studies looking at spatial information during perceptual gaps, is partly driven by the inherent limitations of fMRI. Due to the temporal lag in fMRI, the only way of answering whether information is maintained during occlusion is to focus on retinotopically organized areas. In the future, alternative approaches to examining the nature of the object representations beyond early visual regions could involve techniques with high-temporal resolution such as M/EEG or methods with a finer spatial scale such as ultra-high field and laminar fMRI (see Outstanding Questions). M/EEG would allow the examination of object representations during the perceptual gap, as activations at each timepoint are independent. Laminar fMRI could potentially provide layer-specific activations which might make it possible to distinguish between feedforward and feedback information. Ultimately, integrating neural data with behavioural findings will be critical to elucidate how different mechanisms contribute to the perceptual experience.

Outstanding questions.

How do suppression, extrapolation and integration interact during perceptual gaps? How does the representation change if the object never re-appears? How is information before and after occlusion integrated in the absence of clear extrapolation?

What features beyond position are represented during perceptual gaps? How do task demands change the way we resolve perceptual gaps?

Are there representations during perceptual gaps in areas beyond visual cortex? How do different cortical regions interact to process visual motion during occlusion or eyeblinks?

What role do the MTL and cerebellum play in resolving visual motion through perceptual gaps caused by eyeblinks and occlusion?

Highlights.

• Perceptual gaps occur frequently due to eyeblinks and occlusion

• Processing information through perceptual gaps is particularly challenging for moving stimuli as dynamic information changes while the object is briefly out of sight

• Our perceptual experience suggests that stimulus information such as position is maintained throughout the gap

• Suppression, extrapolation, and integration all play a role in representing visual motion through perceptual gaps

Glossary

Accidental contours

True object contours mark the shape of an object. In contrast accidental contours are created by objects being partially occluded, generating a new contour where the objects intersect.

Apparent motion

The impression of movement produced by the rapid succession of still objects in different locations.

Blink suppression

Suppression of the experience of an eyeblink and the sensory input prior and during the eyeblink to support continuous visual perception throughout eyeblinks.

Efference copy

Neural copy of a motor command to prepare for the perceptual consequences of movement.

Extrapolation

Representing motion trajectories ahead of time to compensate for processing delays.

Eyeblinks

Closure of eyelids that occurs largely involuntarily (i.e., not requiring conscious attention) but can also be controlled at will. Spontaneous eyeblinks occur ~15–20 times per minute and last ~100–300ms.

Flash-lag effect

Visual motion illusion that is thought to be driven by extrapolation. While an object is moving, another object is flashed alongside the moving object briefly. Because of extrapolation, the moving object is (wrongly) perceived to be ahead of the flashed object.

Illusory motion

Perceiving motion when no physical motion occurs. Examples include visual illusions such as apparent motion.

IPS

Intraparietal sulcus. Region of the parietal lobe that has been shown to be involved in motion processing.

MST

Medial superior temporal area. MST is part of the motion complex and is involved in higher-level motion processing.

Occlusion

Period of object invisibility typically due to movement behind another object.

Perceptual gap

A period of time where visual processing is interrupted by internally-controlled events (e.g. eyeblinks) or external events (e.g. occlusion).

Physical motion

Motion signals that are evoked by an object changing position over time. We refer to physical motion when the moving object is visible.

Receptive field remapping

Activation of receptive field prior to saccade, in preparation for stimulation of the receptive field once saccade has landed.

Salient

properties of an object which attracts attention above other objects.

V5/MT

Middle temporal area. MT is the motion complex in the primate brain. When referring to humans, MT is often referred to as hMT.