Contextual modulation refers to the change in a neuron’s responsivity caused by image structure placed outside of its classical receptive field and to the effect of surround image structure on the perceptual properties of target regions contained within. This definition identifies a visual process whose study has enjoyed contributions from both physiology and psychophysics. Add some computational modelling to the mix and it becomes clear that the study of contextual modulation is a truly multi-disciplinary venture, a fact reflected in the range of approaches to the subject described in the articles in this volume.
Why a special issue on the subject? Studies of contextual modulation run into the hundreds, perhaps thousands, covering a range of phenomena, some conventional, some idiosyncratic. What seemed to us to be missing from the literature, however, was a focus on the possible functions of contextual modulation. What benefit, if any, does it confer? The articles in this volume go some way towards addressing this question. Each is a mini-review/opinion or original research article focusing on one or more proposed functions of contextual modulation. Readers of this volume will learn that contextual modulation may serve, among other things, orientation pop-out, texture-boundary segmentation, detexturization, gain control, recalibration, second-order processing, feature selectivity, color constancy and increased coding efficiency for border extraction. Although no doubt many of these functions are intimately related and in some cases mediated by similar neural processes, the above list testifies to the idea that contextual modulation likely serves multiple functions.
In the physiological domain contextual modulation typically refers to neuronal responses to stimuli placed within the classical receptive field, or CRF, that are modulated by stimuli placed outside of it in what is often termed the ‘extra-classical receptive field’, or ERF. Importantly, the CRF must be activated before the ERF has any influence. Hallum and Movshon, drawing on single-unit recordings of monkey neurons in cortical areas V1 and V2, suggest that the ERF might constitute the basis for detecting second-order stimuli, that is stimuli that vary not in luminance or chromaticity but in luminance contrast, color contrast, orientation, spatial frequency, etc. According to Hallum and Movshon, neurons with ERFs perform a ‘double-duty’: their CRFs are sensitive to luminance variations at particular orientations and spatial frequencies, while their ERFs render them sensitive to coarser-scale variations in various second-order properties, again with particular orientations and spatial frequencies.
Schmid and Victor on the other hand propose that ERF neurons fulfil distinct functions in V1 and V2, with V1 neurons responsible for enhancement of salient features leading to orientation pop-out (in which an element of one orientation appears to ‘pop-out’ when surrounded by elements of a different orientation), and V2 neurons responsible for texture segmentation, the same role for contextual modulation proposed by Hallum and Movshon. Nurminen and Angelucci go further. They argue for at least three functions of the neuronal ERF, each mediated by physiological mechanisms operating at different spatial scales involving connections both within and between visual areas. Nurminen and Angelucci identify contrast normalization, increased coding efficiency for border extraction, and the enhancement of salient features in the visual scene as three functions of contextual modulation. Continuing with the theme that contextual modulation operates across multiple visual areas, Krause and Pack review single-unit recording studies in a range of striate and extra-striate regions (e.g. MT, MST, V4, IT) and argue that contextual modulation, with contrast normalization as its core mechanism, improves feature selectivity throughout the visual system. They consider for example the well-known “aperture problem” in motion perception. Motion signals from direction-selective V1 neurons, because they only sample a portion of the stimulus, are often ambiguous with respect to the direction of motion of the larger object to which they are responding. Krause and Pack suggest that contextual modulation serves to selectively enhance those V1 responses that are most informative with respect to the object’s true direction, making easier the job of higher visual areas such as MT, which integrate V1 motion signals in order to signal object motion.
The phenomenon of orientation pop-out, considered by Schmid and Victor to be a V1 function of contextual modulation, is one of a number of phenomena identified by Gheorghiu, Kingdom and Petkov in support of their argument that a key role of contextual modulation is to ‘de-texturize’ the image. They coin the term ‘de-texturization’ to capture the idea that one of the roles of the neuronal ERF is to signal the presence of contours and object boundaries at the expense of textures. They base their conclusion on model simulations of CRF-ERF interactions in response to a variety of stimuli, including images of natural scenes, as well as on psychophysical studies of contour-shape and texture-shape after-effects. Gheorghiu et al.’s findings with shape after-effects reinforce the idea that contextual modulation is not just a property of neurons in V1 and V2, but of neurons in higher visual areas, such as those involved in shape coding.
The contrast normalization function of contextual modulation identified physiologically by Nurminen and Angelucci as well as by Krause and Pack underpins the themes of three of the other psychophysics articles in this volume. Clifford approaches the question of the function of contextual modulation by focussing on one of its best-known effects: the Tilt Illusion. In the Tilt Illusion the perceived orientation of a grating patch is altered (sometimes repulsed, sometimes attracted) when surrounded by a differently oriented grating. On the basis of a review of the literature of the Tilt Illusion, Clifford suggests that it is caused by a contrast gain-control mechanism that enables the system to flexibly change how orientation is encoded. The changes may be in response to changes in the environment, resulting in orientation-code recalibration, or in response to perturbations that occur within the visual system, resulting in orientation-code error correction. Recalibration and error-correction are fundamental properties of our sensory systems, Clifford argues, and thus the gain control model of the Tilt Illusion likely applies to many other forms of contextual modulation.
Webster reminds us that contextual modulation operates temporally as well as spatially. Based on findings from studies of chromatic adaptation he argues that the normalization process from adaptation leading to color constancy exemplifies how contextual modulation confers important benefits to the organism such as perceptual constancy and improved discrimination. Color constancy is the central theme of Werner’s contribution. She argues that the contextual influences that help to achieve color constancy operate at two different spatiotemporal scales: one slow and spatially extensive, the other one fast and localised to regions of the scene bathed in a common illumination.
This brief introduction to the contents of this volume will hopefully pique the interest of readers and encourage them to delve further into a field of study that not only boasts a long tradition but also continues to be one of the most important areas of vision research.