Past trials influence perception of ambiguous motion quartets through pattern completion

Abstract

There are many celebrated examples of ambiguous perceptual configurations such as the Necker cube that abruptly and repeatedly “switch” among possible perceptual states. When such ambiguous configurations are presented intermittently, observers tend to see the same perceptual state on successive trials. The outcome of each trial apparently serves to “prime” the outcome of the following. We sought to determine how long the influence of a past trial persists by using ambiguous motion quartets as stimuli. We found large, significant effects of all four most recent trials, but the results were not consistent with any priming model. The results could be explained instead as perceptual completion of two kinds of temporal patterns, repeating and alternating. We conclude that the visual system does not passively remember perceptual state: it analyzes recent perceptual history and attempts to predict what will come next. These predictions can alter what is seen.

Visual perception under ordinary circumstances is an ongoing process. Current visual information is integrated with past information as part of a perceptual cycle (1), and it is not surprising that, for example, the outcome of a given trial in a psychophysical experiment is affected by what has occurred in recent trials. The observer's response time on a particular trial, for example, is significantly affected by recent task history (2–8). This trial-to-trial effect of the past on the present is particularly pronounced in the perception of motion quartets, a commonly used apparent motion stimulus. A motion quartet consists of a brief display of two tokens presented at opposite ends of a diameter of an invisible circle followed a short time later by presentation of two other tokens on a possibly different diameter. With proper choice of timing, the observer sees apparent motion carrying one token of each pair to a token of the other (Fig. 1). The direction of perceived motion implies a correspondence between each token in the first pair and one of the tokens in the second. This pairing of tokens represents the visual system's solution to the motion correspondence problem (9–11).

Fig. 1.

A motion quartet. The pair of disks marked A appears for 250 ms and then disappears. After a short delay (250 ms), the pair marked B appears for 250 ms. The observer sees apparent rotational motion that carries the first pair of dots into the second. The angle θ between the two diameters affects the probability that the direction of apparent motion is clockwise or counterclockwise. For many observers, the movement is roughly equally likely to be clockwise as counterclockwise when θ = 90°.

The perceived motion is compelling, but it can also be ambiguous. When the tokens are all identical and the angle θ between the diameters is ≈90°, many observers are as likely to see movement in the clockwise direction as they are in the counterclockwise direction. By varying θ, the experimenter can vary the probability that the observer will perceive movement in one direction or the other. When θ is near 180°, motion is almost always seen as counterclockwise, and when it is near 0°, motion is almost always seen as clockwise.

The perceived direction of motion is affected by proximity, the similarity between potentially corresponding tokens (9–14), and the direction of motion perceived during recent trials. Ramachandran and Anstis (15) found that the tendency for the observer to perceive the same direction of motion persisted even with delays of 10–30 s between the end of one trial and the beginning of the next and that “some observers tended to see the same [direction of motion] for indefinitely long periods that were impractical to measure” (p. 138). Other researchers have reported (16) or studied (17–19) this effect.

In Fig. 2, we plot an estimate of the probability of seeing counterclockwise motion, P[C], as a function of θ for one observer who reported the direction of motion of a motion quartet configuration on 2,800 successive trials. The sequence of events during each trial is shown in Fig. 3. The black curve is the estimate based on all of the trials, and the red curve is the estimate based on those trials for which the previous response was counterclockwise C. The red curve has shifted to the left: at any angle the observer is more likely to respond counterclockwise C just after responding C. In contrast, the blue curve is the estimate based on those trials for which the previous response was clockwise C, and the curve is shifted to the right. At any angle the observer is less likely to respond counterclockwise C just after responding C.

Fig. 2.

The effect of the most recent trial. The curves are estimates of the probability P[C] of seeing counterclockwise motion C as a function of motion quartet angle θ: the black curve is an estimate based on all trials; the red curve is an estimate based on trials where the response on the previous trial was counterclockwise; and the blue curve is an estimate based on trials where the response on the previous trial was clockwise. The red and blue curves are shifted away from the black in opposite directions. The intersection of each curve with the horizontal black dotted line is the point of subjective indifference for the corresponding condition. The PSI is reduced after C (facilitation) and increased after C (inhibition).

Fig. 3.

Experiment 1: time course of a trial. The fixation point appears alone for 400 ms at the center of the display area. Then a motion quartet is displayed, and, after the display, the observer is free to respond.

Leopold and coworkers (20, 21) found dependencies on the past for other ambiguous figures when they were presented intermittently. They conclude that “the neural expression of a state of perceptual organization may have an inherent storage capacity that promotes the reestablishment of the same state during the subsequent dynamic processing of visual information” (p. 1076).

But what exactly is remembered in this inherent storage? At one extreme, we could hypothesize that the influence of the past was effectively confined to the most recent previous trial. That is, given knowledge of the outcome of the most recent trial, the direction of perceived motion in the current trial is conditionally independent of all other past trials. We refer to this hypothesis as the Markovian hypothesis. Leopold et al. (20) and Maier et al. (21) implicitly assume this hypothesis.

Alternatively, the responses to trials before the most recent one may exert their own cumulative influence on the current trial. It is plausible that a longer sequence of counterclockwise trials would prime or facilitate a counterclockwise response more so than a shorter sequence, but we need to develop a precise statement of such a priming model before we attempt to test it. We can formulate these intuitions by a priming hypothesis composed of two claims. (i) Changing a C to a C at any point in the past cannot decrease the probability of responding counterclockwise in the current trial. That is, the result can be an increase in the probability of responding C or no change in probability (perhaps because the change is so far in the past that it has no influence). This first assumption implies, for example, that CCC must lead to a greater bias toward C than CCC, if there is any difference at all between their effects. (ii) The second assumption of the priming hypothesis is that the effect of changing a C to a C or vice versa must decrease (or remain the same) as we go farther back into the past.

We measured the point of subjective indifference (PSI, the angle θ for which the observer is as likely to see clockwise as counterclockwise motion) conditional on the responses to the three most recent trials. For convenience, we refer to any sequence of three or more preceding trials (e.g., CCC) as a conditioning sequence. We also use the conditioning sequence to denote the corresponding estimate of the PSI from data as well as the true, underlying PSI that is being estimated. It will be clear from context which is intended. With that notation, the ordering of true PSIs dictated by the priming hypothesis is as follows:

[1]

and

[2]

In interpreting the sequences above, it may help to remember that a higher PSI implies a lower probability of C at any angle. According to the priming hypothesis, this probability is highest after CCC (and the PSI is lowest) among all sequences of length 3, and after CCC it is lowest (and the PSI is highest). As we go along the terms in Eq. 1, each successive term is derived from the previous either by changing a C to a C or by sliding a C forward in time. With each change, threshold for C can only decrease, if it changes at all. Note, in particular, that, if the Markovian hypothesis is true, all of the inequalities in Eq. 1 are actually equalities: only the outcome of the most recent trial matters and it is always C in Eq. 1. Eq. 2 can be analyzed similarly.

The priming hypothesis does not lead to a prediction of the relative order of CCC, the last term in Eq. 1, and CCC, the first term in Eq. 2. We do not know whether the combined effect of changing the second and third most recent trial outcomes outweighs the effect of the most recent, or vice versa.

Materials and Methods

Apparatus. Stimuli were presented on a Dell P780 17-inch computer monitor controlled by a Dell Workstation running windows xp. The monitor was close to flat, varying less than 1 mm in depth across the horizontal extent of the screen. The controlling program was written in the psychophysical toolbox (22, 23). Each observer was positioned in a chin rest at a distance of 50 cm from the screen. The observer's eyes were at the height of the center of the screen (where a fixation cross appeared), and the screen was frontoparallel to the observer's horizontal line of sight.

Stimuli. The observer saw motion quartets, illustrated in Fig. 1. The key independent variable is the angle between the lines through the fixation point defined by pairs A and B, denoted θ. The angle can range from 0° to 180°, and in experiment 1 we used the seven values 40, 60, 80, 100, 120, 140, and 160. In experiment 2, we used the values 60, 80, 90, 100, and 120 and the inducing angles 40° and 140°. We did not know in advance how much the PSI (and the psychometric function) would shift in response to repeating sequences, and consequently we used a wide range of angles to be certain that we could capture all of the psychometric functions conditional on the recent history of responses. The angles chosen in experiment 1 are not symmetric around 90° for reasons discussed in Results.

The angle was varied randomly from trial to trial. We define the angle so that when it is near 0°, the observer's response is overwhelmingly likely to be clockwise (C), and when the angle is near 180°, the observer's response is overwhelmingly likely to be counterclockwise (C). We fit Gaussian psychometric functions to data and estimated the PSI, the angle at which the observer is as likely as not to see motion in the counterclockwise direction. The background was neutral with luminance 69 cd/m². The intensities of the circular disks in each pair were 112 cd/m². Each disk was 2.23 cm (2.56° of visual angle) in diameter. The centers of the disks in each pair were separated by 3.72 cm (4.26° of visual angle).

Procedure. Experiment 1. The time course of a single trial is illustrated in Fig. 3. The observer first saw a fixation cross that remained visible for the duration of the trial. After 400 ms, a pair of circles (pair A), placed symmetrically about the fixation point, appeared for 250 ms. The first pair then disappeared, and the screen was blank except for the fixation cross for a further 250 ms. A second pair of circles (pair B), also placed symmetrically about the fixation point, appeared for 250 ms and then disappeared. The observer then responded. As noted above, we refer to the basic sequence pair A–blank–pair B–blank as a motion quartet. The observer was asked to observe the motion and to report whether the apparent motion induced went in the counterclockwise or clockwise direction (Fig. 1) by pressing a key on a computer keyboard.

Experiment 2. This experiment was the same as experiment 1 except that trials consisted of a group of four motion quartets. Stimulus duration and interstimulus timing were unchanged. The observer responded to only the final motion quartet in each group. The time course of a single trial is illustrated in Fig. 4.

Fig. 4.

Experiment 2: time course of a trial. During each trial, the observer sees four motion quartets but reports the direction of motion of only the last one. Each motion quartet has the same composition and timing as those used in experiment 1. The delay between successive motion quartets in a trial is 800 ms, the average time between trials measured in experiment 1. The angles of the first three motion quartets were either 40° or 140°, intended to induce perception of clockwise or counterclockwise motion, respectively. The sequence 40, 140, 40 is shown in the figure, intended to induce perception of the sequence CCC. The angle of the final motion quartet on each trial was drawn from a wider range. See text.

Design. Experiment 1. The observer completed 20 blocks with brief rests between. Each block consisted of 140 trials, 20 repetitions of a motion quartet at each of seven angles. The order of trials was randomized within each block. The randomization was changed for each block. In summary, each subject completed 2,800 trials in total.

Experiment 2. The observer completed 20 blocks. Each block specified 120 trials in random order. Each trials consisted of three repetitions of each of the eight possible inducing sequences (e.g., CCC) combined with the five possible response angles for the last motion quartet in each group, in random order. The observer overall saw 9,600 motion quartets and responded to 2,400 = 20 × 3 × 8 × 5 of them.

Participants. Experiment 1. Twenty subjects who were unaware of the purpose of the experiment participated. All were recruited by posted advertisement and were paid $12 per hour for their participation. Subjects required 2 h or less to complete all trials. Experiment 2. Sixteen subjects who were unaware of the purpose of the experiment participated. As in experiment 1, all were recruited by posted advertisement and were paid $12 per hour for their participation. Subjects required ≈3–5 h to complete all trials.

Results

We divided the data into eight subsets, depending on what the preceding three trials were (e.g., CCC). We fitted separate Gaussian psychometric functions to each subset by the method of maximum likelihood, obtaining an estimate of the PSI , the angle where the observer is as likely to see clockwise as counterclockwise movement. The PSI estimates averaged across subjects for the two sequences in Eqs. 1 and 2 are plotted in Fig. 5. The dashed horizontal line marks the average PSI across all subjects and sequences (88.3°), slightly less than 90°.

Fig. 5.

Experiment 1: results conditional on three previous trials. The responses for each of 20 subjects were divided into eight groups, conditioned on the responses to a conditioning sequence that was the three previous trials (CCC, CCC, etc.). A psychometric function was fitted to each of the eight groups for each subject, resulting in eight estimates of the PSI. The mean of these PSIs across subjects is plotted on the vertical axis, labeled by the corresponding conditioning sequence. The conditioning sequences are arranged in the same order, left to right, as shown Eqs. 1 and 2. The predictions of the priming hypothesis are that the PSIs for the first four conditioning sequences should be nonincreasing, and those for the second four conditioning sequences should be nonincreasing. The two violations of the predicted ordering are enclosed in dotted ellipses. Both are highly significant.

Note, first of all, the evident skew-symmetry in the PSI sequences when C and C are exchanged. We intentionally introduced an asymmetry in the distribution of angle values used in estimating psychometric functions in experiment 1 to test whether our results depended on the exact distribution of angles. Observers saw more angles biased toward counterclockwise than clockwise. The near-perfect skew-symmetry of the data in Fig. 5 (and also in Fig. 6) suggests that our results are not sensitive to the exact distribution of angle values used.

Fig. 6.

Experiment 1: results conditional on four previous trials. The responses for each of 20 subjects were divided into 16 groups, conditioned on the responses to a conditioning sequence that was the four previous trials (CCCC, CCCC, etc.). A psychometric function was fit to each of the 16 groups for each subject, resulting in 16 estimates of the PSI. The mean of these PSIs across subjects is plotted on the vertical axis, labeled by the corresponding conditioning set. The conditioning sequences are arranged in the same order, left to right, shown in Eq. 3. The predictions of the priming hypothesis are that the PSIs for each group of four conditioning sets should be nonincreasing. The two violations of the predicted ordering are enclosed in dotted ellipses. Both are highly significant.

It is clear that CCC < CCC (t₁₉ = 4.20; P < 0.001) and CCC < CCC (t₁₉ = 4.96; P < 0.001). That is, there are large effects (11.3° and 10.6°, respectively) associated with trials before the most recent. We reject the Markovian hypothesis.

The prediction of the priming hypothesis is that both sequences be monotonically nonincreasing. However, CCC > CCC (t₁₉ = -2.69; P < 0.007) and CCC < CCC (t₁₉ = -4.34; P < 0.001), contrary to the priming hypothesis. No simple priming model can explain these results. Replacing a C in the past by a C should not decrease PSI, but we find that CCC > CCC. Replacing a C in the past by a C should not increase PSI, but we find that CCC < CCC.

The two order violations involve the alternating sequences CCC and CCC. Note that, for example, the sequence CCC has C as the most recent trial and more occurrences of C than C in the three most recent trials if compared with CCC. Yet C is more likely after the latter than after the former. This outcome and the similar outcome for CCC are inconsistent with the priming model, but it is consistent with priming based on an increased probability of seeing the continuation of the alternating sequences CCC → C and CCC → C.

To further investigate the effects of repeating and alternating sequences, we regrouped the data according to the responses on the four most recent trials. Of greatest interest are the repeating sequences, CCCC and CCCC, and the alternating sequences, CCCC and CCCC. In Fig. 6, the PSIs for the 16 sequences are grouped into four groups, and the prediction of the priming hypothesis is that each group of four should be nonincreasing going from left to right. The predictions derived from the priming hypothesis for the four groups are as follows:

[3]

The groups are based on the priming hypothesis (the first two lines are just Eqs. 1 and 2 with a C added on the right, and the third and fourth lines are just Eqs. 1 and 2 with a C added instead). Note, again, the evident skew-symmetry in the PSI sequences when C and C are exchanged.

Examining Fig. 6, we see that repeating sequences of length four induce a powerful bias to continue the “run” and that even repeating sequences of length three such as CCCC also induce a bias (all four are significantly different from the mean response marked by a dashed line). The alternating sequences of length four also induce a strong tendency to continue the alternation (and thereby violate the priming hypothesis), but the same tendency is not statistically significant when the alternating sequence is only of length three and the first item in the sequence is not consistent with the alternation (e.g., CCCC). This outcome implies that events as far back as four places in the past influence the current response: the influence of the past does not die off quickly.

To summarize, the results of the first experiment were inconsistent with both the priming hypothesis and its special case, the Markovian hypothesis. In particular, responses as far back as four trials before the current trials (the limit of analysis) had large effects on perception of the current trial. Compare the effects of CCCC and CCCC or CCCC and CCCC in Fig. 6. In both cases, the change in the fourth most recent trial produces a large change in perception of the current trial.

The pattern of failure was consistent with the hypothesis that the observer's visual system detected two kinds of sequential patterns, repeating patterns and alternating patterns, and that the observer's perception of the stimuli was biased toward completion of these patterns (and no others of length three or four).

In a second experiment, we examined whether the effect of past trials is present even when the observer simply observes the motion without responding. That is, must the observer make an explicit perceptual judgment and/or response on an earlier trial in order for it to affect a later trial? At first glance, in the absence of responses, it would seem that we cannot determine what the observer has experienced on each trial. However, for very extreme angles, the observer is overwhelmingly likely to make one or the other response exclusively. We determined from psychometric functions in experiment 1 that, for angles of 40° or less, the observer is likely to respond C and for angles of 140° or more, C (P > 0.98). Consequently, we can use motion quartets with these extreme angles to insert “trials without responses,” where we very likely know what the observer's response would have been without asking the observer. The sequence of events during a single trial is illustrated in Fig. 4.

The analyses are identical to the first analyses (on sequences of length 3) reported in experiment 1. We grouped the responses for each possible induced sequence and estimated the parameters of a Gaussian psychometric function. The PSI parameters are plotted in Fig. 7 in the same format as in Fig. 5.

Fig. 7.

Experiment 2: results conditional on three previous trials. The responses for each of 16 subjects were divided into eight groups, conditioned on the responses to a conditioning sequence that was the three previous trials (CCC, CCC, etc.). The averages of the separate threshold estimates for each subject are plotted versus conditioning set as in Fig. 5.

Note, first of all, that the data have the same skew-symmetry as observed in Figs. 5 and 6. The observers are treating the two possible directions of perceived motion interchangeably. The main difference that is evident in comparing Figs. 5 and 7, is that the sequences CCC and CCC no longer result in a bias to see C and C, respectively. If we compare the mean for CCC to the overall mean, we find no significant difference (t₁₅ = 0.581, n.s.), and the results for CCC are similar (t₁₅ = 0.031, n.s.).

However, the biases induced by the alternating sequences CCC and CCC are significantly different from the overall mean in the same direction as observed in experiment 1 for CCC (t₁₅ = 2.45; P = 0.014, one-tailed) and for CCC (t₁₅ = -2.41; P = 0.015, one-tailed). The magnitudes of the biases induced by both alternating sequences are actually larger in experiment 2 than in experiment 1, but not significantly so.

To summarize, we asked observers to respond to every fourth trial in a long series of motion quartets where we presented unambiguous motion quartets in a specified pattern during the first three trials. The tendency to extend repeating sequences vanished when the observer did not need to explicitly judge or to explicitly respond to the three immediately preceding trials. The tendency to extend alternating sequences remained, comparable in magnitude to that found in the first experiment.

Discussion

Facilitation or priming theories can readily explain the tendency to extend repeating sequences that leads to a bias to “complete” the sequence, but they cannot be used to explain a tendency to complete alternating sequences. Adaptation or fatigue theories (24) might seem, at first glance, to be appropriate models to explain why C should be more likely followed by C, all else being equal. An adaptation theory is similar to a priming theory except that past occurrences of C reduce the likelihood of perceiving C on the current trial, past occurrences of C reduce the likelihood of perceiving C. For any plausible adaptation theory, CCC should lead to a strong bias toward C, as the cumulative effects of fatigue leads to an ever-increasing tendency to see the other direction of motion. Of course, CCC resulted in a strong bias toward C, not C, in our data. Adaptation theories even in conjunction with priming theories cannot account for our results.

We propose an alternative explanation, the pattern completion hypothesis: (i) the visual system detects repeating and alternating patterns and (ii) detection of a pattern results in a bias in perception toward completion of the pattern. Fecteau and Munoz (25) have proposed that stimulus salience plays an important role in creating sequential effects. They are primarily concerned with the effect of the most recent trial, but their conjecture is consistent with our results and might serve to explain one puzzling aspect of our data. Repeating patterns induced little or no effect in experiment 2, but they produced a very strong effect in experiment 1. The observers are not simply ignoring the inserted trials to which they do not respond. If that were the case, then we would expect little or no effect of alternating patterns in experiment 2. We find instead an effect as strong as that in experiment 1. It may be that, in the absence of response, observers simply pay more attention to stimuli that represent a change in direction than to stimuli that repeat. If so, then adding a secondary monitoring task (e.g., detecting a slight reduction in brightness of the tokens during a trial) might restore the effect of repeating sequences under the conditions of experiment 2.

We emphasize that the sequential effect manifests itself in perception and is not merely the result of a change in the observer's strategy. A signal detection observer, told that the prior odds of signal are 3:1, will likely press the “Signal” response key more often, but we have no reason to believe that his or her perceptual experience on a signal trial is any different from what it would have been with different odds (26). The observer simply resolves ambiguous perceptual events more often in favor of a declaration that a “Signal” is present. There are similar sequential effects in absolute judgments tasks, where observers are asked to provide a numerical estimate of the sensory magnitude of stimuli. These effects are typically attributed to memory distortions and the strategic use of rating scales, not to changes in what is perceived (27–31).

In contrast, in the perception of motion quartets, the percept is rarely ambiguous: the motion quartet evidently moves one way or the other. The percept itself has been altered by the pattern-induced bias. We noted above reports by earlier researchers that describe how the ambiguous percept becomes “locked” into one of its two possible perceptual states and remains there for many trials. Indeed, in pilot experiments, we each experienced the peculiar frustration of seeing the motion quartet move in the same direction for many trials in a row while knowing that the perceptual evidence was on average perfectly ambiguous.

Last, we review evidence for the existence of sequential pattern detectors for repeating and alternating patterns in human perception.

Behavioral Evidence. There is considerable evidence that observers spontaneously detect sequential patterns. Infants of 2 months make anticipatory eye movements when stimuli are alternately placed on left and right halves of a display (32), and older children respond to violations of complex sequential patterns (33). People spontaneously detect patterns in sequences and use them to improve performance even if they are not told that such patterns may be present (34, 35). Observers also tend to perceive patterns where there are none (36, 37).

Possible Neural Correlates. There is some neurophysiological evidence for neural pattern detectors that respond to repeating sequences and to alternating patterns. Huettel et al. (38) asked observers to view rapidly presented, random sequences of two token types, circle and square, and to classify each token as it was presented. Huettel et al. found marked increases in hemodynamic responses in prefrontal cortex when either local alternating or repeating patterns were disrupted. In addition, violations of repeating sequences also led to increased hemodynamic responses in insula, caudate, and putamen, whereas violations of alternating sequences did not. Sommer et al. (39) report similar increases for event-related potentials measures in response to violations of repeating and alternating sequences of tones and a similar pattern in response time. Squires et al. (40) asked observers to listen to random sequences of low- and high-pitched tones. The authors examine the effect of the four most recent trials on a composite discriminant score based on cortical event-related potentials. They found strong effects of repeating sequences and a small effect of alternating sequences. They do not show that the effect of alternating sequences rises above the level of chance.

The stimuli used in the fMRI and event-related potential experiments just described are unambiguous sequences of tokens (e.g., tones or lights) and what is recorded is typically interpreted as violations of expectancy. Of course, there are no predictive patterns present in our stimuli and no violations occur. Here we have found that observers are biased to see or report motion consistent with local repeating and alternating patterns, and only these two kinds of patterns. Whenever the effect of past trials is to bias perception so as to be consistent with local repeating or alternating patterns, presumably the observer's expectations are fulfilled, not violated. Nevertheless, the two experiments just described are intriguing given that they isolate precisely the two kinds of patterns that we have identified experimentally.

We emphasize that, whereas there is considerable evidence suggesting that neural processing detects repeating and alternating patterns and develops expectations based on them, our study demonstrates that these pattern-based expectations alter what we actually perceive.