A review of visual aftereffects in schizophrenia

Abstract

Psychosis—a cardinal symptom of schizophrenia—has been associated with a failure to appropriately create or use stored regularities about past states of the world to guide the interpretation of incoming information, which leads to abnormal perceptions and beliefs. The visual system provides a test bed for investigating the role of prior experience and prediction, as accumulated knowledge of the world informs our current perception. More specifically, the strength of visual aftereffects, illusory percepts that arise after prolonged viewing of a visual stimulus, can serve as a valuable measure of the influence of prior experience on current visual processing. In this paper, we review findings from a largely older body of work on visual aftereffects in schizophrenia, attempt to reconcile discrepant findings, highlight the role of antipsychotic medication, consider mechanistic interpretations for behavioral effects, and propose directions for future research.

1. Background

Understanding the computational and neural mechanisms that function abnormally in psychosis—a complex constellation of symptoms that reflect a separation from reality—has long been a challenge. This understanding is essential, however, to developing effective and targeted pharmacological and psychological treatments for schizophrenia—the illness in which psychotic symptoms are most enduring, severe, and disabling. Implicit or explicit in many mechanistic accounts of schizophrenia is that psychotic symptoms arise due to inappropriate creation or use of stored regularities to guide the interpretation of incoming information (Gray et al., 1991; Hemsley, 2005; Sterzer et al., 2018). The idea here is that perception is sculpted by a combination of current sensory input, context, and past experience, and that schizophrenia is associated with an abnormality in this integrative, modulatory, and inferential process, which leads to abnormal perceptions and beliefs.

The visual system provides a test bed for investigating the role of prior experience and prediction: accumulated knowledge of the world informs our current perception (Barlow, 1990; Clifford et al., 2000; Knil and Richards, 1996; Rao and Ballard, 1999). Importantly, robust behavioral paradigms have been developed to precisely quantify predictions in the visual system, and parallel work in non-human primates provides a basis for interpretation at the level of single neurons. Accordingly, the visual system holds a unique advantage as a model system for understanding the role of experience in how individuals perceive and interpret the world and holds a clear translational value in understanding abnormalities in psychosis.

One form of altered neural coding and perception due to prior sensory input, in which continuous exposure to a stimulus leads to saturation in the responses of neurons coding the available stimulus features, is called adaptation. In the visual domain, there are well-established methods of quantifying the magnitude of sensory adaptation at the perceptual level and clear indications as to how these perceptual phenomena map onto neural sensitivity changes.

Namely, the magnitude of adaptation can be quantified by measuring the strength of what are known as visual aftereffects: illusory perceptions that follow continuous presentation of a real visual stimulus. Aftereffects are thought to be due to extended firing, during the initial stimulus presentation, of neurons selective to the features of that initial stimulus, leading to inhibition/saturation of the response of those neurons and associated disinhibition of neurons that are selective to complementary features, as part of the brain’s effort to maintain homeostasis with regards to neuronal firing rates (Bednar, 2012). The perceptual result, the aftereffect, is illusory perception of ‘the opposite’ of the initial stimulus (e.g., the complementary color, opposite direction of motion, etc.). For example, stare at the cross at the center of Figure 1 for about one minute, then close your eyes. The image of a face you likely perceive while your eyes are closed is an example of a negative afterimage and arises due to adaptation to luminance. Analogous aftereffects are found for other features, from basic ones such as color and orientation to more derived ones such as facial expression and gender. Importantly, primate neurophysiology work has provided insights into the specific neuronal populations that are adapting for various types of aftereffects to occur—indeed, the notion that different aftereffects relate to adaptation in different and specific neuronal populations has led to the adaptation paradigms’ moniker of ‘the psychophysicists’ microelectrode’ (Frisby, 1979).

Figure 1.

Negative afterimage. Stare at the crosshair for 30–60 seconds, then close your eyes. The inverse of this image should be visible and is an example of a visual aftereffect.

Characterizing abnormalities in the predictions that derive from different types of prior sensory experiences and in their influence on current visual perception, then, has the potential to provide a nuanced and biologically-informed perspective on prediction abnormalities in psychosis generally, but also stands to inform etiological conceptualizations of the subjective alterations in visual perception reported by individuals with schizophrenia more specifically (e.g. Freedman and Chapman, 1973; Silverstein et al., 2017). Experimental evidence for such perceptual alterations comes from a range of tasks (reviewed in Butler et al., 2008; Chen, 2011; Silverstein, 2016; Skottun and Skoyles, 2007; Yoon et al., 2013), and a role of abnormal prediction in these alterations has previously been proposed (Silverstein, 2016). Thus, studying visual aftereffects, along with providing insights into global disease mechanisms, may also shed light on the origins of specific aspects of clinical phenomenology.

Despite this potential for visual aftereffects to inform current mechanistic theories of schizophrenia, the bulk of the studies of visual aftereffects in schizophrenia were conducted prior to the 1970’s. Our aim here is to revive and critically review this literature, reconcile discrepant findings, highlight the role of antipsychotic medication, consider mechanistic interpretations for behavioral effects, and propose directions for future research.

2. Experimental paradigms

Before reviewing findings, we begin by describing the types of visual aftereffects that have been investigated in schizophrenia, which are depicted in Figure 2, and the outcome measures that have been reported. After that, we summarize how aftereffects are quantified experimentally, as well as which factors are known to influence these aftereffects in healthy individuals. We will also briefly highlight what is known about the neuronal bases of these aftereffects. Common to all aftereffect paradigms is an adapter stimulus that the subject views for some period of time, typically followed by a test stimulus on which the subject is instructed to report. The influence of the adapter stimulus on the perception of the test stimulus is the general measure of interest.

Figure 2.

Schematic depiction of how single neurons implement visual aftereffects, in this case, the tilt aftereffect. The response of a neuron that responds preferentially to leftward orientations is depicted in the solid line, and the response to a neuron that responds preferentially to rightward orientations is depicted in the dotted line. Stimulus presentation is depicted on the x-axis. Upon presentation of a leftward orientation grating, the neuron with a preferred leftward direction begins to respond vigorously, but soon begins to adapt with repeated presentation of this stimulus (i.e. neural firing is attenuated). Upon the offset of this leftward grating and onset of a blank screen, the neuron returns to a below-baseline level of firing. The neuron with a rightward orientation preference remains at baseline, thus resulting in an increase in firing rate of the neuron with a rightward orientation preference relative to the neuron with a leftward orientation preference, which gives rise to the perceptual phenomenon of a rightward bias in the perception of stimulus orientation (i.e. aftereffect). Adaptation is illustrated using orientation as an example here, but it transpires in an analogous fashion with other stimulus properties.

2.1. Negative Afterimages

(Craik, 1940; Kelly and Martinezuriegas, 1993; Virsu and Laurinen, 1977): Upon adapting to a visual stimulus with a given color, subjects will report an illusory image that has the same spatial configuration, i.e. outlines, as the original stimulus but in which each location has a color that is complementary to that shown at the corresponding location in the original stimulus (e.g. green becomes red). Grayscale stimuli can also induce negative afterimages (cf. Figure 1), and in that case each location in the illusory image has a brightness that is the complement of the corresponding location in the original stimulus (i.e. bright becomes dim and vice versa).

2.2. Tilt Aftereffect

(Gibson and Radner, 1937; Greenlee and Magnussen, 1987; Wolfe and O’Connell, 1986): Upon adapting to a grating (or other pattern) that is oriented at a particular angle, subjects are presented with a similar grating that has a slightly different orientation. The original adaptation period results in a perceptual bias so that the angle of the subsequent grating is perceived as farther removed from that of the original grating than it actually is.

2.3. Movement Aftereffect

(Anstis et al., 1998; Mather et al., 2008; Wohlgemuth, 1911): Upon adapting to movement in a particular direction, subjects are asked to report on the movement direction of a test stimulus. The aftereffect consists of the illusory perception that this test stimulus moves in a direction opposite to that of the adapting stimulus. Studies in schizophrenia have, by and large, measured what is known as the Archimedes spiral motion aftereffect. In this paradigm, the adapting stimulus is a spiral rotating in a particular direction. Once the spiral stops rotating, the now-stationary stimulus appears to rotate in the direction opposite to the adapting stimulus.

2.4. Figural Aftereffect:

This term is used in a number of ways in the psychological literature. In a broader sense, the term covers several different aftereffects, including the tilt aftereffect (Day et al., 1959; Wohlgemuth, 1911). In the context of research related to schizophrenia, a series of studies have used this term for an aftereffect that is evident when a set of two vertical lines, one placed straight above the other, is viewed following exposure to an adapting stimulus (Kohler and Wallach, 1944; Prysiazniuk and Kelm, 1965). The adapting stimulus consists of a similar set of two vertical lines but, in addition to being placed above each other, the two are shifted horizontally: one leftward and the other rightward. As a result, the lines in the test stimulus are perceived as shifted horizontally in the opposite directions (i.e. rightward and leftward, respectively).

2.5. McCollough Effect

(McCollough, 1965): The McCollough effect is an aftereffect that is substantially different from the ones discussed so far. It centers not on adaptation to a particular visual feature, but on adaptation to a conjunction of two features. Typically, the subject adapts to a red-colored grating that has one orientation (e.g. horizontal) as well as a green-colored grating that has an orientation orthogonal to the first grating (e.g. vertical). Then, the subject is presented with a test image that is one of the gratings (or, in the classic demonstration, a patchwork of both) but that has no color. The aftereffect consists of the illusory perception that (any part of) a test stimulus that matches one of the adapters in orientation, has a color that is complementary to the color of that adapter.

3. Experimental measures of aftereffects, and factors that influence them

Work on healthy individuals has identified several methods for quantifying the strength of aftereffects. Some studies simply ask subjects to report whether an aftereffect is present or how strong it is, typically by comparing it to some physical reference stimulus that is not, itself, affected by the aftereffect (Georgeson and Turner, 1985; Greenlee and Magnussen, 1987). Another straightforward method involves measuring the duration of the aftereffect rather than its subjective magnitude, by asking subjects to indicate when the aftereffect ends (Leguire and Blake, 1982; Suzuki and Grabowecky, 2003; van de Grind et al., 2001; Verstraten et al., 1998) and sometimes also when it starts (i.e. latency; Suzuki and Grabowecky, 2003). A final, less direct, method involves physically adjusting the test stimulus itself to counteract the aftereffect that affects perception of the test stimulus, in order to quantify the amount of physical change required to exactly cancel the aftereffect. For instance, to measure the strength of an upward movement aftereffect such a nulling, or cancelling, procedure would involve presenting a test stimulus that contains some downward motion rather than being fully stationary, and determining how much downward motion is required for the test stimulus to appear stationary (Blake and Hiris, 1993; Castet et al., 2002). For other aftereffects analogous procedures have been used (Georgeson and Turner, 1985; Kelly and Martinezuriegas, 1993; Thakkar et al., 2018; Wagner, 1971). Each of these methods of measuring aftereffects comes with advantages and disadvantages that matter in specific cases (Brascamp et al., 2010; Castet et al., 2002; Pantle, 1998), but the methods qualitatively agree in the majority of cases.

Across essentially all aftereffects, one factor that influences aftereffect strength is timing: aftereffects get stronger with more prolonged adaptation, and their strength decays over time after the adapter is removed (Greenlee and Magnussen, 1987; Harris and Calvert, 1989; van de Grind et al., 2004; Wolfe and O’Connell, 1986). In addition, different kinds of aftereffects are affected by visual stimulation parameters such as contrast or retinal location. We will briefly address such influences below because they play a role in our later discussions related to schizophrenia. Specifically, for any influence of disease status on aftereffect strength we need to consider whether disease-related alterations in the way a stimulus is represented in early visual processing (e.g., due to impairments in gain control, contrast sensitivity, ability to fixate ones’ gaze, perceptual organization, coherent motion detection) can mediate the strength of aftereffects. One prominent factor that influences effectively all aftereffects is the spatial correspondence between the adapter stimulus and the test stimulus. Some aftereffects have a great deal of such spatial specificity: for negative afterimages, for example, the afterimage appears at the exact retinal location where the adapter stimulus was processed before. For other aftereffects the degree of spatial specificity is reduced, in that aftereffects are still experienced for test stimuli with a location that is somewhat displaced from the adapter location (Afraz and Cavanagh, 2009; Knapen et al., 2009; Wenderoth and Wiese, 2008). Nevertheless, even in those cases aftereffect magnitude is largest at the exact location of the test stimulus.

In addition to these general effects of time and spatial location, which affect aftereffects of all kinds, there are stimulus factors that play a role for specific kinds of aftereffects. As with the factor of spatial correspondence, these factors typically relate to the degree of neural overlap between the adapter stimulus and the test stimulus. For instance, when the movement aftereffect is viewed on a stationary test stimulus, it is strongest after adapting to a slowly moving stimulus; yet for a test stimulus that jitters rapidly (like TV static), faster adapters yield the strongest motion aftereffects (van Boxtel et al., 2006; Verstraten et al., 1998). To give another example: the tilt aftereffect is stronger when the physical orientation of the test stimulus is only slightly different from that of the adapting stimulus (by about 10–20 degrees) than when the orientation difference is larger (Clifford, 2002; Schwartz et al., 2007). In both these examples, and also with regard to location correspondence between adapter and test stimulus, the most plausible explanation is that aftereffects have their origin in neurons that are tuned, i.e. that respond to only a limited range of stimulus features. As a result, the degree to which perception of a given test stimulus draws on neurons that have actually been affected during the adaptation period depends on the degree of correspondence between adapter stimulus and test stimulus in the relevant dimensions. The role of neural tuning in aftereffects will be discussed in more detail in the next section.

A factor of influence that is not captured by the above descriptions in terms of neural correspondence, is stimulus contrast—a measure of the difference in brightness between a stimulus’ darkest and brightest regions. Several studies in healthy populations have investigated the degree to which adapter contrast impacts aftereffect strength. Unsurprisingly, negative afterimages (which reflect adaptation to darker and brighter regions in an image) become stronger when adapter contrast is increased (Gilroy and Blake, 2005; Kelly and Martinezuriegas, 1993). For both movement aftereffects and tilt aftereffects, strength depends on the contrast of both the adapter and the test stimulus. In particular, aftereffect strength increases with increasing adapter contrast but decreases with increasing contrast of the test stimulus (Keck et al., 1976; Nishida et al., 1997; Parker, 1972). These effects, however, may interact with other stimulus parameters, as one study reported the relation between test stimulus contrast and the strength of the tilt aftereffect to reverse as a function of test stimulus duration (Harris and Calvert, 1989).

4. The neural basis of aftereffects

In general terms, aftereffects are thought to arise from a net reduction in responsivity, gradually arising during the adaptation period, of neurons that respond to the adapting stimulus (Figure 2). This reduced responsivity carries over to the neurons’ response to a subsequent test stimulus, resulting in the perceptual effect. This type of change in neural responsivity over time is commonly observed at the single-neuron level (Albrecht et al., 1984; Barlow and Hill, 1963; Kohn and Movshon, 2003), and there is no doubt that it happens during aftereffect paradigms. The perceptual phenomena that define aftereffects, however, are likely the result of a more complex interplay between many neurons that differ in the degree to which their responsivity is affected during the adaptation period (Grunewald and Lankheet, 1996; Mather, 1980; Sutherland, 1961; Tolhurst and Thompson, 1975). For instance, the perception of a contour’s orientation is plausibly rooted in the collective pattern of neural responses among neurons that are tuned to orientation, i.e. that each responds optimally to a given, preferred, orientation and less to different orientations. Individual neurons differ in their preferred orientation, and neurons with different tuning preferences interact through both excitatory and inhibitory connections (Li, 1998). The tilt aftereffect, then, would correspond to an overall imbalance, due to differential sensitivity to the earlier adapter, in response to the test stimulus across this population of orientation-tuned neurons.

One relevant implication that emerges from this broad summary of the neural basis of aftereffects is that different aftereffects arise from different neural populations, depending on the populations’ selectivity to visual input. For instance, negative afterimages emerge, to a substantial degree, due to adaptation at the level of retinal and subcortical neurons (Craik, 1940; Li et al., 2017; Zaidi, 2012), whereas tilt aftereffects are likely related to changes among the orientation-tuned neurons of the primary visual cortex (Clifford, 2002) and motion aftereffects plausibly depend on the responses of movement-tuned neurons in higher visual cortical regions (Huk et al., 2001; Mather, 1980).

5. Critical review of findings in schizophrenia

In the following section, we review findings in schizophrenia. Because existing findings do not paint a coherent picture when comparing across different studies, we also explore clinical and experimental factors that may account for variability across studies. Table 1 summarizes the mixed existing evidence for altered visual aftereffects in schizophrenia, with some studies finding evidence for a greater influence in patients of the adapter stimulus on perception of the test stimulus, some finding less influence, and some finding no difference. Note that null results should be interpreted with caution as sample sizes are small and studies are likely underpowered.

Table 1.

Study	Aftereffect	Measure	Groups and sample sizes	Medication status	Group effect	Relationship with other measures
Abraham and McCallum (1973)	Spiral aftereffect	Duration	1) Psychotic inpatients with first- rank (n unknown) 2) Psychotic inpatients without first-rank symptoms (n unknown) 3) Healthy controls (n unknown)	Not reported	SZP > HC	SAE duration decreased with improvement in clinical status following treatment
Barrett and Logue (1974)	Spiral aftereffect	Occurrence	1) Inpatients with schizophrenia (n=35) 2) Inpatients with organic brain disease (n=35)	Not reported	SZP > brain injury*
Calver et al. (1991)	Tilt aftereffect	Strength	1) SZP (n=8; tested before and after medication injection) 2) HC (n=8)	Medicated	SZP > HC*	Greater tilt aftereffect in SZP before injection than after injection
Claridge (1960)	Spiral aftereffect	Duration	1) Hysteria inpatients (n=16) 2) Dysthymia inpatients (n=16) 3) Schizophrenia inpatients (n=16) 4) Healthy controls (n=16)	Not reported	SZP > HC Hysteria=HC < Dysthymia
Day et al (1967)	Tilt aftereffect Figural aftereffect	Strength	1) SZP inpatients (n=20) 2) non-SZP inpatients (n=20) 3) Healthy controls (n=20)	Mostly medicated	No group difference
Hersen et al (1972)	Spiral aftereffect	Occurrence Latency Duration	1) SZP inpatients (n=20) 2) Organic brain disease (n=20) 3) Healthy controls (n=20)	Not reported	Occurrence: HC > SZP > Brain disease Latency: HC < SZP and brain disease Duration: no difference
Herrington and Claridge (1965)	Spiral aftereffect	Duration	1) First-episode psychosis inpatients (n=30) 2) Healthy controls (n=18)	Not reported	No difference	No effect of antipsychotic treatment
Kelm (1962)	Figural aftereffect	Strength	Experiment 1 1) SZP inpatients (n=5) 2) HC (n=5) Experiment 2 1) SZP inpatients (n=10) 2) Alcoholic inpatients (n=10)	Not reported	SZP < HC*
Kelm (1968)	Figural aftereffect	Strength	1) SZP inpatients (n=12) 2) non-SZP inpatients (n=8)	Unmedicated for 48 hours	SZP < non-SZP
Krishnamoorti and Shagass (1963)	Spiral aftereffect	Duration	1) Psychotic inpatients (n=26) 2) Healthy controls (n=20)	Unmedicated for 7 days	No difference
Rokem et al (2011)	Tilt aftereffect	Strength	1) Schizophrenia outpatients (n=16) 2) HC (n=20)	Mostly medicated	No difference
Schein (1960)	Spiral aftereffect	Duration	1) Psychotic inpatients (n=12) 2) HC (n=23)	Not reported	No difference	No difference between subjects taking different classes of medications
Surguladze et al (2012)	McCollough effect	Occurrence Latency	1) SZP (n=50, inpatients and outpatients) 2) First-degree relatives (n=61) 2) 50 HC	All medicated	Frequency: SZP < hC and relatives Latency: SZP > REL > HC	No relationship to medication dose Longer latency correlated with greater positive symptoms in SZP and schizotypy scores in relatives
Thakkar et al (2018)	Negative afterimage Tilt aftereffect	Strength	131 HC	Unmedicated		Negative afterimage: no relationship with schizotypy Tilt aftereffect: negatively related to schizotypy scores
Tress and Kugler (1979)	Movement aftereffect	Duration	1) SZP inpatients (n=11) 2) HC (n=15)	Not reported	No difference
Wertheimer (1954)	Figural aftereffect	Strength	1) SZP inpatients (n=15) 2) HC (n=15)	Not reported	SZP < HC*
Wertheimer (1957)	Figural aftereffect	Strength	Experiment 1: 1) SZP inpatients (n=15) 2) HC (n=15) Experiment 2: 1) SZP inpatients (n=17) 2) HC (n=17)	Not reported	SZP < HC

One clear source of variability across studies is the type of aftereffect measured. This factor is relevant as different aftereffects involve adaptation occurring in different neuronal populations. All of the studies finding greater aftereffect strengths and durations in schizophrenia measured the movement and tilt aftereffect. On the other hand, studies of the figural aftereffect found, by and large, reduced aftereffect strength in schizophrenia patients. In addition, the one study of the McCollough effect also reported reduced afterimage strength in patients. It is worth noting here that the validity of what has been referred to as the figural aftereffect in clinical studies is debatable. More specifically, the direction of effects in many individuals is in the opposite direction to what you would expect given neural adaptation, thus bringing into question its comparability with other measures of visual aftereffects (Day et al., 1967).

Another source of variability across studies involves the experimental parameters—namely, duration and visual properties of the test and adapter stimuli. Few studies have manipulated these parameters within a single experiment; however, differences in parameter values may explain discrepancies across studies. Of those studies comparing the duration of movement aftereffects between schizophrenia and healthy controls, two studies reported longer aftereffect durations in schizophrenia (Abraham and McCallum, 1973; Claridge, 1960), five studies reported null findings (Herrington and Claridge, 1965; Hersen et al., 1972; Krishnamoorti and Shagass, 1963; Schein, 1960; Tress and Kugler, 1979), and no studies observed longer durations in controls. With the aforementioned caveat regarding null findings in mind, it is nevertheless interesting that those studies reporting greater duration of the movement aftereffect in patients used relatively long adapter durations (1 minute), whereas those that reported null results used shorter adapter durations (15–30 seconds). In this context, it is worth noting that aftereffect strength depends not only on stimulus timing (see above), but also that the physiological events that underlie aftereffects may differ depending on the timescale of adaptation (Kanai and Verstraten, 2005; Wolfe and O’Connell, 1986). Indeed, there is some evidence that results of aftereffect studies involving schizophrenia patients may differ, even qualitatively, depending on stimulus timing as well as other parameters. Specifically, in their investigation of the tilt aftereffect in individuals with schizophrenia (before and after antipsychotic injection), Calvert et al (1991) manipulated the spatial frequency of the adapter stimulus as well as the duration of the test stimulus. For higher contrast adapter stimuli, patients before injection had larger tilt aftereffects than controls at short test durations; this difference was not observed at longer test durations and was, in fact, reversed in sign. That is, for lower contrast stimuli, patients prior to injection had a larger tilt aftereffect only at long test durations. However, these findings should be interpreted with caution given the very small sample sizes (see Table 1). Using the figural aftereffect, Kelm (1962, 1968) also manipulated adapter durations and the duration between adapter and test stimuli. Predictably, aftereffects diminished in strength with increasing duration between adapter and test stimuli; for patients, the direction of the aftereffect actually reversed at long adapter-test intervals. Group differences in aftereffect strength in these studies are difficult to interpret, however, as they only reported differences in the absolute magnitude of aftereffects. In summary, there is some tantalizing, albeit incomplete, evidence that hints at the importance of characterizing the temporal dynamics of visual aftereffects in schizophrenia, and how they vary under different perceptual conditions, for understanding the location of functional abnormalities within the visual processing hierarchy.

A further source of experimental variability across studies is the operational measure of aftereffects used, with different studies reporting occurrence, strength, duration, and/or latency. It is difficult to disentangle the extent to which variability in conclusions across studies is due to differences in outcome measure used as the outcome measure is often confounded with aftereffect type. For motion aftereffects, only the occurrence, latency, and duration of effects have been reported; for the figural aftereffect, only strength has been reported. This lack of overlap between the measures used to quantify the two types of aftereffects should be kept in mind when considering the generally opposite directions of effect that have been reported when comparing patients and controls in terms of these aftereffects. For instance, it is certainly possible that the time course of aftereffects differs in schizophrenia patients, such that duration and strength are differentially affected, but no studies have systematically addressed this. The use of latency and duration measures is also problematic, as it is confounded with general reaction time, which is known to be slowed in schizophrenia across most response domains (reviewed in Nuechterlein, 1977).

Another factor that certainly merits consideration in explaining altered aftereffects in schizophrenia is antipsychotic medication. These drugs achieve therapeutic effects by blocking dopamine (Seeman and Lee, 1975; Seeman et al., 1976), but also act on other neurotransmitter systems. Adaptation at the cellular level is plausibly modulated by these systems (McCormick and Williamson, 1989). Indeed, healthy observers administered antipsychotic medications experience a reduction in the aftereffect strength related to tilt, motion, and color (Harris et al., 1986; Harris et al., 1983; but see Janke and Debus, 1972; Lehmann and Csank, 1957). These findings cannot be accounted for by drug effects on retinal blurring, perceived luminance of the adapter, drowsiness, blinking, or loss of fixation (Harris et al., 1986; Harris et al., 1983). Studies comparing the effect of antipsychotics with other psychotropic medications suggest that drug effects on aftereffects are primarily due to dopaminergic action. The influence of medication in schizophrenia patients is less clear from the literature. As evidenced in Table 1, many studies do not report medication status of patient samples, and there is no clear pattern of findings across those studies that do report medication status. A handful of studies have explicitly reported the effects of medication. Consistent with studies in healthy individuals administered antipsychotic medications, the strength of the tilt aftereffect was greater in patients immediately prior to a neuroleptic injection (when levels of the medication were presumably low) compared to immediately after (Calvert et al., 1991). Additionally, duration of the movement aftereffect decreased following a treatment-related decrease in first-rank symptoms in an inpatient population (Abraham and McCallum, 1973). Although the nature of the treatment is not described, there was most likely a pharmacological component. On the other hand, Lloyd and Newbrough (1964) observed no change in duration of the movement aftereffect before and after antipsychotic administration in schizophrenia patients. Notably, these patients only underwent a 10-day wash-out period, and previous medication use may have had a lingering effect on aftereffects. However, even in a never-medicated group of first-episode patients that improved following antipsychotic treatment, cessation of drugs 8–15 weeks later did not influence the duration of the movement aftereffect (Herrington and Claridge, 1965). Additionally, studies that have reported correlations between aftereffect measures and normalized medication dose have found no evidence for a relationship (Rokem et al., 2011; Surguladze et al., 2012). In summary, there is clear and strong evidence from healthy observers that antipsychotic drugs reduce aftereffect strength and duration. Although some patient studies also support such a relationship between antipsychotic use and reduced aftereffects, other studies report no such relationship. Findings from patient studies are trickier to interpret however given the possibility of receptor changes due to long-term medication use and the likely confounding effect of improvement in clinical status with medication administration. Importantly, because of the direction of the effect of antipsychotic drugs in healthy individuals, findings of increased (rather than decreased) aftereffect strength and duration in schizophrenia patients cannot plausibly be accounted for by neuroleptics.

Another potential factor leading to variability across studies is illness stage and clinical status. Unfortunately, many studies do not report illness stage or clinical status of participants. In his review of movement aftereffect duration in schizophrenia, Harris (1994) made inferences about the clinical status of participants based on age and argued that duration of the movement aftereffect was greatest in acutely ill, untreated schizophrenia patients. These findings are akin to data on contrast sensitivity, with observations of increased contrast sensitivity in unmedicated, first-episode patients (Kelemen et al., 2013; Kiss et al., 2010; Shoshina et al., 2015) and decreased contrast sensitivity in chronic patients, regardless of medication status (Butler et al., 2005; O’Donnell et al., 2006; Skottun and Skoyles, 2007; Slaghuis, 1998). Indeed, Abraham and McCallum (1973) found that duration of the movement aftereffect decreased with improvement in clinical status following treatment. On the other hand, Surguladze (2012) found that more severe positive symptoms were associated with increased latency to report the McCollough effect. It is certainly possible that the influence of clinical status on visual aftereffects is domain- (i.e. motion versus color) and/or measure-specific (i.e. duration versus latency). Alternatively, longer illusion latency and longer illusion duration may both be caused by response slowing, possibly secondary to distraction, which has been found to be more related to positive than negative symptomatology (Green and Walker, 1986).

6. Mechanisms for altered visual aftereffects in schizophrenia

In the following section, we consider possible CNS and non-CNS explanations for altered visual aftereffects in schizophrenia and, following these explanations, we interpret these data in the context of recent comprehensive theoretical accounts of the disorder. Arguably, the most compelling explanation for altered visual aftereffects is that patients with schizophrenia differ from healthy controls in the adaptability of neuronal populations. The plausible difference, discussed above, in exact cortical origin of various aftereffects therefore offers the potential to distinguish visual processing levels in which adaptation is, and is not, affected.

Differences in visual aftereffects may also be secondary to differences in perceived contrast. As noted above, contrast sensitivity is decreased in chronic schizophrenia patients (Butler et al., 2005; O’Donnell et al., 2006; Skottun and Skoyles, 2007; Slaghuis, 1998) and increased in those early in the course of illness (Kelemen et al., 2013; Kiss et al., 2010; Shoshina et al., 2015). Given that aftereffect strength typically increases with increased contrast (see earlier section), it is certainly possible that the observed increases in aftereffect strength and duration in younger, more acutely ill samples may be explained, at least in part, by increases in perceived contrast of the adapter stimulus.

Increased visual processing duration or activity may also contribute to findings of increased aftereffect strength and duration in schizophrenia. For example, visible persistence, which refers to the duration for which a stimulus is visible following termination, has been found to be longer in patients with schizophrenia—as much as twice as long—compared to healthy controls (Schwartz et al., 1994). Longer visible persistence effectively lengthens the adapter stimulus duration, which would be expected to increase aftereffect strength. Whether the relatively short absolute duration by which visible persistence is lengthened in schizophrenia can account for findings of longer and stronger motion and tilt aftereffects is unknown.

Non-CNS explanations for altered visual aftereffects must also be considered. One such factor is eye movements. As noted above, aftereffects all display some degree of spatial selectivity, so breaks in fixation would be expected to yield weaker aftereffects. Indeed, there is a robust literature citing impairments in fixation and gaze control in schizophrenia (e.g. Barton et al., 2007; Hutton and Ettinger, 2006; Thakkar et al., 2011; Thakkar et al., 2015) and fixation losses may explain those studies reporting weaker aftereffects in patients. Only one study to date (Rokem et al., 2011) used an eye tracker to monitor eye movements and exclude trials with large losses of fixation; in this study, there was no difference between patients and controls in the strength of the tilt aftereffect. Two older studies visually inspected eye position during induction of the figural aftereffect. Wertheimer (1954) did not observe any appreciable deviations from fixation, and Kelm (1968) excluded those subjects with large deviations from fixation. In both of these studies, a smaller figural aftereffect was still observed. Although that suggests that smaller figural aftereffects cannot be accounted for by losses of fixation, they do not rule out the possibility of smaller fixational eye movements, that are not easily detected by visual inspection, contributing to reduced figural aftereffects in patients. Notably, increased fixational eye movements certainly cannot account for findings of increased strength and duration of aftereffects. Furthermore, there is no evidence for differences in microsaccade rate in individuals with schizophrenia (unpublished analyses in Demmin et al., 2018; Egana et al., 2013). Nevertheless, considerations related to eye movements could also form a motivation for future work to focus on aftereffects that have a limited degree of spatial selectivity, such as face aftereffects, and that would therefore not be strongly affected by differences in fixation control, and certainly not by small fixational eye movements (Afraz and Cavanagh, 2009).

Group differences in response bias and criterion must also be considered as possible confounding factors. For instance, there may be group differences in the criterion level of visibility of an aftereffect needed to report its offset or onset, or there may be group differences in the cognitive tendency with which subjects report the test stimulus as opposite to the adapter stimulus, particularly as the decision becomes more difficult. To date, these factors have not been appropriately controlled for. One way to address these possible response-level confounds is to use specific variants of the nuller/cancellation method of measuring aftereffect strength, that render an influence of response bias or criterion less likely (Brascamp et al., 2018; Thakkar et al., 2018).

On a broader scale, we can interpret findings of altered visual aftereffects in the context of both computational accounts of visual aftereffects and unified and biologically plausible theoretical frameworks for understanding schizophrenia. One such framework holds that symptoms arise from an imbalance in cortical excitation and inhibition (E/I imbalance) due to NMDA receptor hypofunction (Coyle, 2012; Kantrowitz and Javitt, 2010; Olney et al., 1999) and GABA-ergic dysfunction (Egerton et al., 2017; Phillips et al., 2015; Schmidt and Mirnics, 2015; Thakkar et al., 2017). In this context it is noteworthy that a prominent theory of visual aftereffects postulates that they arise due to short-term changes in lateral interaction strength between feature detectors in a process analogous to the Hebbian learning process by which feature selectivity develops in sensory cortices across development (Bednar and Miikkulainen, 2000). Thus, alterations in visual aftereffects may reflect E/I imbalance and, more specifically, abnormal lateral interactions within visual cortex in schizophrenia. In the early phase of schizophrenia, one possible mechanism for stronger than normal aftereffects is an increase in the rate at which the weights of excitatory connections between inhibitory neurons (Gibson et al., 1999) are updated (or between pyramidal cells and interneurons; Thomson et al., 2002). The increased inhibition for features present in an adapting stimulus would then lead to increases in activity at normally inhibited neurons (i.e., especially those that signal features complementary to those whose processing has been inhibited), as part of the brain’s efforts to maintain homeostasis in firing rates. This hypothesis of increased connection strength during the acute phase of schizophrenia is based on evidence of (a) hyperactivation of the locus coeruleus (LC) in schizophrenia (Alsene & Bakshi, 2011; Yamamoto & Hornykiewicz, 2004) and modeling data indicating that elevated LC firing rate increases both gain and the rate of Hebbian learning (Verguts & Notebaert, 2009); (b) modeling data demonstrating that when cortical activity levels increase, membrane time constants are lowered, causing increases in synchronized firing between cells and aberrant local network formation (Chawla, Lumer, & Friston, 1999); and c) increased baseline synchrony in early schizophrenia (Rivolta et al., 2014; Silverstein et al., 2012; Sun et al., 2013), and increased synchrony being associated with the presence of hallucinations in people with schizophrenia in general (reviewed in Uhlhaas and Singer, 2010). With increasing illness chronicity, many patients would transition to a state of weakened lateral interactions, which we would expect to be associated with weaker aftereffects, consistent with some of the data reviewed above. This change from the acute stage to the chronic stage has been observed in the domain of contrast sensitivity as described above, as well as in the domain of perceptual organization (Silverstein, 2016; Silverstein and Keane, 2011), and it may account for attenuated contextual modulation of vision more broadly (Butler et al., 2008; Phillips et al., 2015), as well as other aspects of disorganization (Olypher et al., 2006; Phillips et al., 2015; Phillips and Silverstein, 2003, 2013) and perception (Butler et al., 2008; Phillips et al., 2015; Rokem et al., 2011; Silverstein, 2016; Silverstein and Keane, 2011) in chronic schizophrenia. While much data fit within this overall framework, longitudinal testing of the same people across different stages of illness are needed to provide strong confirmatory evidence. It would also be helpful, in the short-term, to obtain, in parallel within the same individuals, putative measures of both baseline inhibition strength, for instance as indexed by measures of surround suppression (Anderson et al., 2017; Tibber et al., 2013), and of changes in inhibition strength, as indexed by aftereffects.

A second prominent candidate explanation of schizophrenia is altered predictive coding. Predictive coding accounts of brain function posit that perception is an inferential process by which higher cortical areas generate models of the world based on context and past experience, which are communicated to sensory areas. Mismatches between predictions and sensory information (i.e. prediction errors) lead to either an update in the model or action by the organism to change the sensory input to be more consistent with predictions (Clark, 2013; Friston, 2005). Formalized within a Bayesian framework, sensory information (likelihood) is combined with predictions (priors) to compute the most likely cause of that sensory data (posterior)—which is then experienced as the percept. Psychosis is argued to arise due to perceptual experiences being more influenced by the likelihood distribution (i.e. sensory data) and less influenced by prior beliefs, due either to reduced reliability, or precision, of priors, increased precision of sensory data, or a combination thereof (reviewed in Sterzer et al., 2018; but see Powers et al., 2017). Visual adaptation, and by extension visual aftereffects, have also been explained in the context of prediction (Clifford et al., 2007). In formal Bayesian observer models (Stocker and Simoncelli, 2005), exposure to the adapter stimulus results in an increase in the precision of the likelihood function in the range of values around the adapter’s parameters. Although aftereffect paradigms do not typically measure visual discrimination per se, visual adaptation does regularly result in increased discrimination, depending on the precise relation between the adapter and test stimulus (Clifford, 2002; Clifford et al., 2001; Regan and Beverley, 1985). In this framework, larger aftereffects in individuals with schizophrenia can be interpreted as larger increases in the precision of the likelihood function, which would have the net effect of shifting percepts towards the sensory data, rather than prior beliefs, as has been posited in schizophrenia (Sterzer et al., 2018).

Consistent with the notion that perceptual inference is biased towards sensory data in psychosis, individuals with schizophrenia have been found to be less susceptible to those visual illusions that arise due to the influence of higher-level information on current sensory input (e.g. Dakin et al., 2005; Keane et al., 2013; Schneider et al., 2002; Tibber et al., 2013). A notable example here is the hollow mask illusion, in which a concave face is perceived to be convex. This illusion is argued to arise due to the powerful influence of top-down knowledge that faces are convex (Gregory, 1970). This illusion is weaker in individuals with schizophrenia, particularly those that are acutely psychotic (Keane et al., 2013; Schneider et al., 2002). However, there is also data suggestive of a greater influence of history and knowledge on perception (Corlett et al., 2019). In the visual system, previous exposure to an un-ambiguous image confers better discrimination of that same image presented later in a degraded (i.e. ambiguous) form, and this advantage is greater in individuals prone to psychosis. These apparently contradictory findings have been reconciled by highlighting the potentially critical role of where in a sensory hierarchy predictions are formed, noting that weaker priors from within the visual system, in this case, might lead to greater reliance on top-down information from outside the visual system to help structure the input. Generally, these effects of more or less influence of prior history and context on current processing are cast, in Bayesian terms, as a change in the prior distribution. However, modeling work would suggest that for aftereffects, past history (in this case, exposure to the adapter) exerts an influence on current processing by increasing the precision of the likelihood function. Such an explanation may also warrant consideration for other paradigms and illusions. Finally, it should be noted that a predictive coding failure for aftereffects would be consistent with the hypothesis of excessive updating of weights at excitatory connections onto inhibitory cells in schizophrenia (an aspect of the excitation/inhibition perspective, described above), and so an integration between the two accounts may be possible.

7. Conclusions and future directions

In conclusion, there is tantalizing evidence from a largely older body of work suggesting an abnormal use of prior experience in sculpting perception in schizophrenia in the form of altered visual aftereffects. However, there are clear inadequacies in the current literature. We thus intend this paper to function as much a review of the current literature as a motivation for further investigating visual aftereffects and a call for more rigorous studies. There are clearly significant sources of variability in these findings that prevent conclusive interpretation and that are rooted, at least in part, in differences in relevant stimulus dimensions (i.e. tilt, motion, etc.) and stimulus properties (i.e. duration, spatial frequency, etc.). Indeed, there is evidence for stronger motion and tilt aftereffects in schizophrenia but weaker figural aftereffects, and these findings appear to be moderated by adapter and test durations as well as spatial frequency. One important direction for future work will be to explore what kinds of aftereffects are altered in schizophrenia and to map out the temporal and featural stimulus conditions under which the group differences emerge. In addition, future experimental work should aim to rule out confounds related to response bias and criterion as well as eye movements. To the extent that results can be explained by CNS factors, these types of studies can clarify the extent to which altered visual aftereffects originate in the retina, the subcortical visual pathway, V1, and/or higher visual areas involved in integration and visual cognition. This will further shed light on which neuronal populations are involved in altered adaptation and its dynamics in schizophrenia.

Clinical factors, including duration of psychotic illness, symptom profile, and diagnostic criteria used to define the disorder, are a further source of variability. Because most of the research on aftereffects was done prior to the publication of the DSM-III, it is not clear to what degree older findings apply to DSM-5-defined schizophrenia. Furthermore, aftereffects seem to be stronger in unmedicated individuals in the acute phase of the illness, and antipsychotic medications reduce adaptation duration and strength in healthy observers. Thus, a second important direction of future research will be to gather data using current diagnostic criteria and to account for the effects of medication and illness stage. Future work should also explore the extent to which altered visual adaptation, as measured by visual aftereffects, may explain abnormalities in subjective visual experience as well as psychotic symptoms more broadly. Indeed, prior work has noted a relationship between an altered use of context on visual perception that is strongest in acutely ill patients and is related to disorganization (Keane et al., 2014; Silverstein et al., 2013; Silverstein et al., 2000; Uhlhaas et al., 2006; Uhlhaas et al., 2005; Uhlhaas and Silverstein, 2005).

Another potentially fruitful direction for future research is to investigate the extent to which altered abnormalities in neuronal adaptation, measured using aftereffects, may reflect illness vulnerability. There are two studies that suggest this to be the case. In healthy undergraduates, weaker tilt aftereffects, but not negative afterimages, were related to positive schizotypal traits (Thakkar et al., 2018). In the only study of visual aftereffects in unaffected relatives of schizophrenia patients, relatives tended towards increased latency of the McCollough effect (Surguladze et al., 2012), and longer latencies were associated with greater positive schizotypal traits in these relatives.

Finally, experimental data from visual aftereffect paradigms are amenable to computational modeling approaches, which can aid in dissecting component processes that may be altered in schizophrenia spectrum disorders. Future model-fitting efforts may provide insights into the extent to which putatively altered visual aftereffects in schizophrenia are explained by abnormal lateral interactions within visual cortex (Bednar and Miikkulainen, 2000) and/or abnormal Bayesian inference (Stocker and Simoncelli, 2005).

In conclusion, visual aftereffects may serve as a useful paradigm for understanding both altered visual experience in schizophrenia and more global dysfunction related to predictive processes and short-term synaptic plasticity. Although there is compelling evidence for altered aftereffects in schizophrenia, future work using more rigorous psychophysics and comprehensive and current clinical characterization is necessary to paint a clearer and more complete picture of aftereffects in schizophrenia and to fully realize their potential for understanding the disease.

Figure 3.

Illustration of aftereffect paradigms. In the top left is a legend to all panels. Each panel shows one paradigm, all involving an adapter stimulus (left column in each panel) followed by a test stimulus (right columns). Aftereffects constitute altered perception of the test stimulus, so each panel also illustrate subjective perception (bottom rows) in addition to on-screen stimuli (top rows). Aftereffects shown: negative afterimages (A), tilt aftereffect (B), movement aftereffect (C), figural aftereffect (D), and McCollough effect (E). See text for further details.

Highlights.

• Abnormal use of prior experience to interpret sensory information in schizophrenia

• Visual aftereffects measure influence of prior experience on perception

• Evidence for alterations of visual aftereffects in schizophrenia

• Significant variability in findings due to type of aftereffect and clinical factors

• Findings allow for mechanistic interpretations for altered visual aftereffects