When Predictive Mechanisms Go Wrong: Disordered Visual Synchrony Thresholds in Schizophrenia

Abstract

Patients with schizophrenia display an impaired sense of temporal continuity, and we showed that they judge events as being simultaneous even in case of large onset asynchronies. We check here whether this means a fusion of events in time, or on the contrary, a segregation of events and a deficit in coding time-event structure. Subjects decided whether 2 squares were displayed simultaneously or asynchronously on the screen and gave their response by hitting a left or right response key. The implicit processing of asynchrony was explored by means of the Simon effect, which refers to the finding that manual responses are biased to the side of the stimulus. We checked whether responses were biased to the side of the first or second square, when squares were asynchronous and displayed on opposite sides. Results revealed an enlarged time window in patients irrespective of the squares’ position (intra- vs interhemispheric presentation). But for asynchronies eliciting “synchronous” judgments, patients’ responses were biased to the side of the first square. In contrast, controls were biased in all cases to the side of the second square. The inverse effects observed below thresholds in patients and controls cannot be attributed to a generalized deficit. In controls, elementary predictive mechanisms would allow anticipation of upcoming events, whereas patients appear to process squares as if isolated rather than following each other. Predictive mechanisms would be impaired in patients, who would rather rely on reactive mechanisms in order to perceive asynchrony.

Introduction

A disturbed sense of continuity has been reported in patients with schizophrenia.^1,2 Patient’s own reports illustrate this disturbance “Time splits up and doesn’t run forward anymore. These arise uncountable disparate now, now, now, all crazy and without rule or order.”³ This alteration might be related with patients’ impairments in a range of tasks regarding time processing at different timescales, ie, duration evaluation^4–9 or discrimination of successive events in time.^10,11 Impairment in the sense of continuity might be especially related to difficulties in coding time-event structure, as suggested in recent experimental studies.¹² However, it is still unclear how impairments observed in experimental studies relate with clinical evidence. In fact, experimental studies usually suggest that patients fuse events even though they are separated by a large delay, whereas clinical evidence suggests a fragmentation of events in time. In order to resolve this conundrum, we dissociate here effects at implicit and explicit levels in both patients with schizophrenia and controls.

The fact that the sense of continuity is related to the coding of time-event structure is based on ideas stemming from phenomenology. Even the simplest conscious perceptual experiences are embedded in a continuous flow of conscious experience. For example, when at a concert where several instruments are being played all at once, the musicians sometimes play a note at the same time and sometimes the notes are played successively. When played in succession, the note that has just been played is not only remembered but also phenomenologically perceived, even as it is “retained” in its being-past. “Its being-past is something now, something present itself, something perceived.”^13(p219) Similarly, the “yet-to-come” tone is perceived, as it is “protained” in its being-future. For Husserl,¹³ the experience of continuity requires thus the integration of past, present, and future moments. Especially important for the present study, “now” includes an expectation regarding future moments.¹³ As suggested by Varela,¹⁴ this might result from neural constraints. In fact, the processing of information, even the simple display of a square on a computer screen, requires time, thus implying that, rather than being coded as single points in time, events have a duration. This means different events overlap in time even if their onset is shifted in time. The result would be a sense of continuity rather than the perception of discrete moments.

The idea that events overlap and are fused in time is also supported experimentally. Brecher,¹⁵ eg, showed that subjects need a delay of 30–100 ms between 2 event onsets to distinguish the first from the second. This delay might represent “the subjective present” described by Husserl.¹³ We recently showed that patients with schizophrenia needed larger onset asynchronies than controls to correctly judge that 2 bars were presented one after another rather than simultaneously,¹⁶ independent of a response bias effect.¹² At a phenomenological level, this would imply less smoothness in the flow of conscious experience, with a reduced ability to perceive a recent event as passed and a to-come event as future. However, a difficulty to report an asynchrony explicitly does not necessarily mean that events are fused together at an implicit level. Even if patients do not consciously discriminate asynchronous stimuli for short stimulus onset asynchronies (SOAs), they may nonetheless perceive a difference implicitly. Our question is whether, in the case of patients, all events included within the same “simultaneity window” are fused together into one single percept. In order to evaluate the implicit processing of asynchronies, we used the Simon effect.

The Simon effect refers to the finding that performance is faster and more accurate when the stimulus appears on the same side as the responding hand, even if stimulus location is irrelevant to the task.^17,18 The Simon effect thus reflects a tendency to press the button on the side of the stimulus but requires no explicit judgment. The visuomotor Simon effect is indeed believed to rely on direct activation of the manual response through visual stimulation.¹⁹ In our task, 2 stimuli were presented simultaneously or asynchronously. The stimuli were both on the left or right side of the screen or one was on the left and one on the right. Subjects had to hit a left-hand response key with the left hand when the stimuli were judged to be simultaneous and a right-hand response key with the right hand when they were judged to be asynchronous. When the 2 stimuli are on the same side, subjects’ response should be biased to the side of the stimuli, thus reflecting the classical Simon effect. When the 2 stimuli are on opposite sides, however, a classical Simon effect cannot be expected, and in particular, no Simon effect can occur in case of 2 simultaneous stimuli. It is only when the 2 stimuli are asynchronous that a Simon effect may be observed again. The first square indeed appears by itself on the screen for a short duration, until the second square is displayed. If this short-duration stimulus is subconsciously detected as such, ie, an isolated stimulus on one side of the screen, a Simon effect should be observed on the side of this first square. In contrast, if the second square is expected, the Simon effect could be observed on the side of this second square even in the absence of a conscious perception of asynchrony. Exploring the Simon effect will thus yield information about the implicit coding of time-event structure.

Our predictions are the following. First, patients are expected to require larger asynchronies than controls to report them. The question is whether this means a real fusion of events in time. It is explored by measuring the Simon effect when the 2 stimuli are on opposite sides. In case of fusion, events are perceived as identical on both sides. There cannot be any bias to either side, and no Simon effect should be observed. In contrast, if events are segregated below threshold, a Simon effect might be observed. In that case, the type of bias (on the side of the first or the second stimulus) will give indications regarding how information has been processed, eg, with or without an expectation regarding the second stimulus. Most importantly, it will indicate whether patients and controls process asynchronies in the same way or not at an implicit level, ie, when they explicitly judge stimuli as simultaneous.

Methods

Subjects

Eighteen stabilized outpatients with schizophrenia took part in the study. They were diagnosed according to the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition criteria.²⁰ Symptoms were assessed with the help of the Positive and Negative Syndrome Scale.²¹

The control group matched the patients’ group in terms of gender, age, and level of education (Fs < 1) (table 1).

Table 1.

Demographic and Clinical Data of the Participants

	Patients	Controls
Gender (M/F)	9/9	9/9
Age (mean ± SD)	35.7 ± 6.3	34.3 ± 6.4
Years of education (mean ± SD)	11.8 ± 2	12.5 ± 1.8
Medication (typical/atypical/no medication)	3/14/1
Dose of chlorpromazine equivalents	275 mg/day
PANSS positive symptoms (mean ± SD)	15.4 ± 4
PANSS negative symptoms (mean ± SD)	19 ± 7.3
PANSS general symptoms (mean ± SD)	35.8 ± 12.8
PANSS total (mean ± SD)	70.3 ± 21.2

Note: PANSS, Positive and Negative Syndrome Scale.

The project was approved by the local ethics committee. All subjects gave their informed written consent prior to testing, in accordance with the recommendations laid down in the Helsinki Declaration.

Details concerning exclusion criteria and the equipment (computer, 120 Hz monitor and 50 Hz eye tracking) can be found in online supplementary material.

Stimuli

Stimuli were 2 squares (0.8° × 0.8°) displayed at 1 of the 4 corners of a virtual square (5.5° × 5.5°) located in the middle of the screen. They were thus presented in 2 of 4 possible locations. When presented at the top or bottom of the virtual square, they were interhemifields, whereas when they were located on the right or left of the virtual square, they were in the same hemifield (figure 1).

Fig. 1.

Illustration of the 4 Possible Target Locations. Two squares are presented at the same time or asynchronously in 1 of 4 possible locations: upper, lower, right, or left.

Contrast levels were chosen to be as similar as possible to those used in our previous experiment.¹² To reduce the influence of a transient response, squares increased gradually in luminance from 0.02 to 12 cd/m², over a presentation interval of 75 ms.

Procedure

Subjects had to decide whether the 2 squares displayed on the screen appeared at the same time or not. Each trial began with the presentation of the central fixation point, which remained on the screen throughout the trial. Subjects had to fixate this point for 500 ms, this being checked by continuous eye tracking. Stimuli then appeared either simultaneously (SOA = 0 ms) or asynchronously. Twelve levels of SOA were used (from 0 to 92 ms in steps of 8.3 ms). Squares stayed on the screen until subjects had responded. The stimuli and fixation point were then removed from the screen, and the next trial started after a delay of 1000 ms. Subjects were instructed to hit a left response key with the left hand in the case of simultaneous squares and a right response key with the right hand in the case of asynchronous squares. No emphasis was put on response speed.

Each target location (upper, lower, right, or left) was equally represented. Each combination of position (same or different hemifields) and SOA (12 levels) was tested 20 times in random order, yielding a total of 480 trials.

Threshold Evaluation

To control for a tendency to give asynchronous responses, the individual data were subjected to the following probability-based correction:

where P(0) is the percentage of “simultaneity response” for simultaneous squares (SOA = 0). This correction ensures that “asynchronous responses” taken into account in the following analysis cannot be attributed to false alarms. The “thresholds” were then derived from a linear adjustment between the SOAs and the corrected rate of “simultaneous” responses (rate of “simultaneous” responses = a × SOA + b) for each subject. They were calculated as the SOA corresponding to equivalent rates of simultaneous and asynchronous responses (50% for both). In other words, the threshold corresponds to the “point of subjective equality.”

Results

There was a similar false alarm rate before data correction in patients (22.4%) and controls (24%) when squares were synchronous (F < 1). Although high and possibly related to the fact that there was uncertainty regarding the location of the target squares, these false alarm rates were clearly lower than in our preceding study (around 40%).

An ANOVA was conducted on thresholds with the group (patients vs controls) as between-group variable and with the type of presentation (intra- vs interhemifield) as within-group variable. The threshold was significantly higher in patients (50.1 ms) than controls (41.7 ms): F[1, 34] = 4.8; P < .05. Although the difference between patients and controls is slightly more apparent when the squares are displayed in the same hemifield (49.7 ms in patients vs 39.8 ms in controls, F[1, 34] = 4.5, P < .05) rather than interhemifield (50.6 ms in patients vs 43.7 ms in controls, F[1, 34] = 3.8, P = .059), there was no interaction between presentation (intra- or interhemifield) and group (F < 1) (figure 2). These data were confirmed in the analyses on response times (RTs) (see online supplementary material).

Fig. 2.

Rates of Responses “Simultaneous” as a Function of the SOAs and Targets’ Location, Intrahemifield (Left Panel) vs Interhemifield (Right Panel) in Patients (Continuous Line) vs Controls (Dotted Line). Thresholds are indicated by the straight lines: 49.7 ms in patients vs 39.8 ms in controls when targets are presented in the same hemifield (left panel) and 50.6 ms in patients vs 43.7 ms in controls when they are presented interhemifields (right panel).

An effect of presentation (intra- vs interhemifield presentation) became apparent, however, when the rates of “synchronous” responses observed for all SOAs were taken into consideration. We performed an ANOVA with the group as between-group variable (patients vs controls) and with type of presentation (intra- vs interhemifield) and the SOAs (from 0 to 92 ms) as within-group variables.

There was an effect of intra- vs interhemifield presentation, (F[1, 34] = 7.8, P < .01), with subjects overall making 3.6% more errors, ie, more “simultaneous” responses, in the case of presentation interhemifield than in case of squares presented within the same hemifield. This effect interacts significantly with the SOA (F[11, 374] = 2.7, P < .005) but not with group (Fs < 1; there was neither any significant 3-way interaction between group, type of presentation, and SOA). Whereas there was no effect of intra- vs interhemifield presentation for SOA = 0 ms (F < 1), a significant effect of presentation was found for SOAs ranging from as little as 8.3 ms (difference of 5.8%, F[1, 34] = 5.7, P < .05), right up to 41.7 ms (differences between 5% and 10%, Fs > 4.4, Ps < .05). These results show a disadvantage for interhemifield presentation, which is expected since it requires an interhemispheric transfer of information. More surprisingly, this effect is observed at very short asynchronies, which yield simultaneous judgments. If this had reflected a true absence of asynchrony perception, then performance should have been equivalent whatever the target conditions.

A sensitivity to short asynchronies was also supported by analyses on RTs, at least in patients (see online supplementary material). The Simon effect was used to examine this further.

As emphasized in the Introduction, the Simon effect reflects the tendency to respond with the hand that is on the same side as the stimulus. A sensitivity to short asynchronies may be due to 2 different mechanisms, and these possibilities can be disentangled by exploring the Simon effect occurring when the squares are displayed on opposite sides. First, subjects’ responses might be influenced by a sense of direction and by expectation regarding the second square. In that case, their response should be biased on the side of the second square onset. Alternatively, they might be sensitive to the first square’s onset, which remains on the screen by itself for a duration equivalent to the SOA. In that case, the response should be biased on the side of the first square.

Due to the sensitivity to short asynchronies revealed by the advantage for intrahemifield presentation, we were especially interested in the Simon effect observed below threshold. Because the threshold varies between subjects, we averaged the percentage of “simultaneous” responses for SOAs below and above individual thresholds. The number of SOAs taken into account was thus adapted to each subject, with the highest SOA considered below threshold being at least 12 ms below the subject’s threshold.

It should be noted that we checked that the basic Simon effect was not altered in patients relative to controls, by comparing rates of “simultaneous” responses when the 2 squares were displayed on the left or on the right (see the online supplementary material).

The critical analysis, though, concerned the Simon effect in the case of squares displayed in 2 different hemifields (first square on the left, second on the right, or vice versa). This analysis showed a significant interaction between group, SOAs (sub- vs suprathreshold), and presentation sides (F[1, 34] = 5.7, P < .05). Regarding SOAs below threshold, patients gave 4.7% more responses using the left response key (“simultaneous” responses), when the first square was on the left and the second on the right, than when the locations were reversed (F[1, 17] = 5.5, P < .05). This bias to the side of the first square was consistent across patients, being reversed in 3 patients among 18 only, and the Simon effect was still significant when using the nonparametric Wilcoxon test (T = 20, z = 2.04, P < .05). In contrast with patients, the responses of control subjects were biased to the side of the second square (figure 3). Below threshold, controls gave 5.7% more responses with the left-hand key when the second square was on the left than when it was on the right (F[1, 17] = 4.9, P < .05). The effect was reversed in 5 among 18 controls and tended to be significant with the Wilcoxon test (T = 35, z = 1.7, P = 0.08).

Fig. 3.

(A) Rate of responses “simultaneous” as a function of SOAs when squares are presented in 2 different hemifields, either the first square on the right side and the second on the left (continuous line) or vice versa (dotted line) in controls (left panel) and patients (right panel). Thresholds for interhemifield presentation are indicated by the straight lines. The mean rate of responses “simultaneous” below threshold, calculated as a function of individual responses, is illustrated in (B) as a function of the side of the second square in patients (left panel) and controls (right panel).

The profile was similar in both groups for SOAs above threshold, with a bias to the side of the second square. There was a higher percentage of responses with the left-hand key (“simultaneous” responses) when the second square was on the left rather than on the right (by 6% in patients, F[1, 17] = 6.8, P < .05 and by 7.4 % in controls, F[1, 17] = 12.9, P < .005). In patients, the inverted profile below and above threshold produced a significant interaction between SOA (below vs suprathreshold) and the side of the second square, (F[1, 17] = 15.2, P < .005).

Correlations

There was no correlation between the medication dose in chlorpromazine equivalents and their intrahemifield threshold (r = −.03) or interhemifield threshold (r = .04). In patients but not in controls, there was a correlation between the mean threshold of asynchrony detection and the amplitude of the Simon effects observed both below the threshold (r = −.56, N = 18, P < .05) and above threshold (r = .57, N = 18, P < .05). The larger the bias on the side of the first square below threshold, the lower the threshold in patients. In contrast, the larger the bias on the side of the second square above threshold, the higher the threshold.

Discussion

Our results replicate previous results showing that the threshold of asynchrony detection is higher in patients than in controls.^12,16 Despite the simplicity of the test, there was still a threshold difference between patients and controls. The extent of the threshold difference may seem small (around 10 ms) but represents an increase of about 20% compared with the threshold of controls. Given the temporal precision of most cognitive functions (language and motor control), this difference may constitute an important drawback for patients, especially as it appears to increase in the case of more complex tasks¹² and because the results show impairments even when stimuli are processed in the same hemisphere, thus generalizing previous results. Coupled with results showing similar disturbances in the auditory modality¹⁶ and reduced sensitivity to onset asynchrony,²² the results add to the literature showing impairments related to time in patients. They suggest deficits at an additional and more elementary level than duration perception, which also involves memory^5–8 or critical flicker fusion and masking, which involves spatial fusion.^10,11

Our main question was whether the extended window of synchrony perception observed in patients implies, or not, a fusion of the events occurring within the same temporal window. The results not only suggest that this is not the case but also reveal a qualitative difference in the way patients and controls detect asynchronies. The Simon effect analysis indeed suggests that in the present paradigm implicit processing differs qualitatively in patients and controls. The Simon effect produces a tendency to hit response keys on the same side as the stimulus. In case of 2 stimuli displayed in different hemifields, the Simon effect shows whether responses are biased to the side of the first or second square. The results show patients and controls are biased by the second square when the SOA is large enough. For short SOAs, however, and contrary to controls, patients were biased to the side of the first square. This suggests that the first square is detected at least implicitly, even though its duration is very short. This is consistent with past studies that showed patients’ sensitivity to short-duration stimuli.²³ That this response profile does not persist above threshold shows patients are performing the task correctly and do not mistakenly give a response after the first square. If that had happened, the influence of the first square should have increased when it stayed by itself on the screen for longer, ie, for longer SOAs. Such was not the case. In addition, the RTs increase around the threshold (see online supplementary material) shows that patients do not answer in an impulsive way after the first stimulus. It seems thus that both patients and controls follow instructions and are influenced by the last event for large SOAs.

Despite this, the mechanisms at play clearly differ between groups below threshold. The results in patients are consistent with a simple feed-forward processing of information. The first displayed square is necessarily processed in a feed-forward way. The display of this square represents the only event occurring on the screen, at least for a short period. The fact that it is displayed alone on the screen should elicit an automatic “visuomotor” Simon effect.¹⁹ This should occur even if the first stimulus is not consciously perceived as isolated. This effect fades slowly with time,^19,24 and the Simon effect elicited by the first stimulus thus disappears as the SOA between the 2 consecutive squares increases. When the SOA is large enough and when the second square is clearly dissociated from the first one, its appearance can then elicit a Simon effect on its side. All effects observed in patients might thus occur in a feed-forward way and independently from a perception of succession. This hypothesis is further supported by the correlation between the threshold of simultaneity/asynchrony discrimination and the amplitude of the Simon effect on the side of the first square at short SOAs. This correlation suggests that patients’ judgment of asynchrony at threshold is facilitated by the perception of an isolated square. What seems to lack in patients is thus the comparison between the onsets of the stimuli. It is as if they would rather process 2 isolated stimuli, thus making it difficult to compare their onsets. Controls, on the contrary, appear to be sensitive to the succession of the 2 squares even below threshold. This is consistent with the fact that thresholds derived from temporal order judgments are usually lower than those derived from simultaneity/asynchrony judgments.²⁵ It is likely that this relies on top-down processes, at least in our paradigm. The results in patients certainly suggest that the perception of succession is not automatic. As a matter of fact, the perception of succession depends on the active comparison of the squares’ onsets because it cannot rely on a passive perception of motion (the perception of illusory motion was indeed prevented by the fact that stimuli stayed on the screen once displayed). This means that due to the task controls direct their attention toward the expected second stimulus. This might explain the lack of Simon effect on the side of the first stimulus, and the fact that this effect occurs rather on the side of the second stimulus.

The theory of predictive coding is especially suited to explain the effects observed here. According to predictive coding, the representation activated in visual perception by sensory information would be permanently compared with following upcoming information in order to correct interpretation.^26,27 If this is true, it means that processing is dynamic and inherently codes time succession. It might thus provide the basis for an implicit coding of time processing. According to the present results, however, an additional step would be required to compare the onsets of 2 separate stimuli.²⁸ This might involve the configuration of early predictive mechanisms to attend not only to 1 but also to 2 successive events that are separated in space. As many models, predictive coding can be adapted to include the possibility of early predictive coding to be guided by higher level expectations.²⁹ This means that following the instructions explicitly would lead to the configuration of early predictive mechanisms at an implicit level. In the framework of this model, patients might be impaired due to 2 possible impairments. They may have a difficulty to configure early predictive mechanisms according to the instructions. Else the early predictive mechanisms might themselves be impaired. Either possibility is consistent with the literature, inasmuch as both types of mechanisms are supposed to rely on feedback connections from high-level structures to earlier levels, which have been suggested to be impaired in schizophrenia both at a neurobiological level^30–32 and at a functional level.^33–35 An impairment of predictive mechanisms would be consistent with difficulties observed in the motor domain^36–38 and with a proposal from Gallagher and Varela.³⁹ A difficulty to configure task sets might also be related to decision-making processes that are known to be impaired in patients.^40–42

One of the limitations of our study is that most patients were treated with antipsychotic medication. An effect of treatment cannot be ruled out, although no positive correlation was found between the increase in patients’ threshold and treatment in chlorpromazine equivalent. A recent study has also shown that even first-episode patients having been treated for less than 6 weeks are impaired at detecting asynchronies between 2 rectangles onsets.⁴³ In any case, the present study confirms the existence of enlarged temporal windows at least in treated patients. This impairment at an explicit level is not associated with a fusion of events but rather a paradoxical fragmentation of information displayed within the temporal window, as suggested by implicit motor responses. We propose that patients are unable to anticipate immediate upcoming events, consistent with earlier clinical descriptions^1,44 and recent proposals.³⁹ Within the framework of Husserl’s work, this would mean a deficit in the integration of future moments in the subjective present, ie, a deficit in protention. Inasmuch this integration underlies the sense of continuity, such an impairment might be involved in the patients’ disturbance of the sense of continuity. Even though an elementary disturbance, it might rely on impaired anteroposterior connectivity and might impact on a wide range of everyday life activities, such as language, reasoning, or motor control, which require anticipating events on a very short timescale.

Funding

French National Institute for Health and Medical Research (INSERM); Centre Hospitalier Régional Universitaire of Strasbourg (Inner project grant from the University Hospital of Strasbourg, API- HUS n°3494); the Medicine Faculty of Strasbourg.

Supplementary Material

Supplementary material is available at http://schizophreniabulletin.oxfordjournals.org.