The Push-pull of Serial Dependence Effects: Attraction to the Prior Response and Repulsion from the Prior Stimulus

Abstract

In the “serial dependence” effect, responses to visual stimuli appear biased toward the last trial’s stimulus. However, several kinds of serial dependence exist, with some reflecting prior stimuli and others reflecting prior responses. One-factor analyses consider the prior stimulus alone or the prior response alone, and can consider both variables only via separate analyses. We demonstrate that one-factor analyses are potentially misleading and can reach conclusions that are opposite from the truth if both dependencies exist. To address this limitation, we developed two-factor analyses (model comparison with hierarchical Bayesian modeling and an empirical “quadrant analysis”), which consider trial-by-trial combinations of prior response and prior stimulus. Two-factor analyses can tease apart the two dependencies if applied to a sufficiently large dataset. We applied these analyses to a new study and to four previously published studies. When applying a model that included the possibility of both dependencies, there was no evidence of attraction to the prior stimulus in any dataset, but there was evidence of attraction to the prior response in all datasets. Two of the datasets contained sufficient constraint to determine that both dependencies were needed to explain the results. For these datasets, the dependency on the prior stimulus was repulsive rather than attractive. Our results are consistent with the claim that both dependencies exist in most serial dependence studies (the two-dependence model was not ruled out for any dataset) and, furthermore, that the two dependencies work against each other. If true, traditional one-factor analysis techniques serve only to identify the stronger dependency.

Recently encountered visual information alters perceptual decisions. Often, the alteration manifests as a repulsion from the prior stimulus, as in perceptual phenomena like the tilt aftereffect (Gibson & Radner, 1937; Webster, 2015). More recently, the opposite pattern has been reported (Cicchini et al., 2014; Fischer & Whitney, 2014; Kiyonaga et al., 2017), in which the influence of the prior stimulus on current perceptual judgments appears to be attractive. That is, when participants are presented with a series of stimuli of the same type (e.g., gratings that differ in orientation across trials, Figure 1A), responses to the current stimulus appear biased toward the orientation of the stimulus on the immediately preceding trial (Figure 1B). This attractive effect has been labeled “serial dependence” (Fischer & Whitney, 2014). Like repulsive aftereffects, attractive serial dependencies have been observed for a wide range of stimulus classes, including orientation, spatial location, motion direction, numerosity, timing, identity, gaze direction, ensemble statistics, attractiveness, and gender (Alais et al., 2018; Fischer et al., 2020; Fornaciai & Park, 2018b; Jepma et al., 2014; Liberman et al., 2014; Suárez-Pinilla et al., 2018; Taubert, Alais, et al., 2016; Taubert, Van der Burg, et al., 2016; Xia et al., 2016), suggesting that the effect reflects fundamental mechanisms of perception.

Figure 1.

(A) Experiment 1 Schematic. On each trial, participants were presented with a grating stimulus and asked to report its orientation. Responses could begin when the stimulus appeared, but the stimulus was replaced by a mask after 200ms. Stimuli are not to scale. ITI: Inter-trial Interval. (B) Graphical Depiction of Serial Dependence. Errors on the current trial are plotted as a function of the orientation difference between the current trial and the prior trial. If the error is in the same rotational direction as the orientation difference (e.g., both Clockwise/CW or both Counterclockwise/CCW), this suggests attraction, whereas if they are of opposite rotations, this suggests repulsion, as indicated by the colored quadrants. The bias effect typically operates within a restricted range of differences between the current trial and the prior trial.

The attractive serial dependence effect has been explained as reflecting a “continuity field” (Fischer & Whitney, 2014), which promotes visual stability by biasing current perceptual representations toward the recent past through a (potentially weighted) averaging mechanism (Alais et al., 2018; Cicchini et al., 2018; Fischer & Whitney, 2014). However, it is unclear whether this attraction is towards the low-level visual properties of the prior stimulus or towards the higher-level percept of the prior stimulus (which manifests as attraction towards the prior response, in the case of a perceptual error on the prior trial). Adjudicating between these two possibilities is difficult because the response on the current trial can be influenced by many factors (see also Jepma et al., 2014; Jones et al., 2013) that sum together in a complicated mixture to influence behavior. In addition, outside of the serial dependence literature, there is substantial evidence that a response on one trial can make participants more likely to make a similar response on the next trial (Annis et al., 2018; Jesteadt et al., 1977), an effect that we refer to as “response hysteresis” (Schwiedrzik et al., 2014). We use this term for any observed attraction to the prior response, while noting that it could arise from a variety of psychological mechanisms, including changes in beliefs about the base rate of stimuli (Zhang et al., 2014; c.f. Urai et al., 2019), systematic changes in how much evidence participants require for initiating responses (Wagenmakers et al., 2004), a tendency to repeat decisions without regard to the stimulus (Akaishi et al., 2014; Braun et al., 2018), or one interpretation of continuity field theory (in which the response simply provides an index of the participant’s higher-level percept). But whatever the source of response hysteresis, its existence poses a problem when drawing conclusions about serial dependence effects: if the response to the prior trial is accurate, then attraction to the prior stimulus and response hysteresis are confounded.

The conflation of attraction to the prior stimulus and response hysteresis is exacerbated by standard analysis practices, one-factor analyses that considers the prior stimulus alone or the prior response alone, or considers both factors, but in separate analyses. In a standard one-factor analysis, the magnitude of dependence is determined by non-parametrically smoothing the errors or by fitting a derivative of Gaussian function (Figure 1B). The derivative is a convenient model since it has an amplitude parameter that corresponds to the maximum average error elicited by prior stimuli, which is taken as the magnitude of serial dependence. We use this derivative function as a tool to highlight the kinds of biases on the current trial that would arise from the summation of two dependencies with opposite signs, one whose amplitude is positive (Figure 2A, first column), and one whose amplitude is negative (Figure 2A, second column). A typical one-factor analysis – plotting a dependence as a function of either the prior orientation or prior response separately – shows only a summation of these two latent influences, with the summation influenced by participants’ response variability (Figure 2A, third column). For example, if the two forces mirrored each other perfectly and participants were relatively accurate, the latent influences would combine to produce a flat line – an observed lack of dependence despite robust latent dependencies (Figure 2A, bottom row).

Figure 2.

Analysis of serial dependence effects, showing a traditional one-factor analysis (A) and a two-factor analysis (B) that examines combinations of prior response and prior stimulus. A) Hypothetical results of a serial dependence experiment, in which the prior stimulus induces two effects simultaneously. The first column shows an attractive dependence, the second a repulsive dependence, and these are summed to give the third column, the observed dependence. The repulsive effect differs across the rows, while the attractive effect is the same in all 3 rows. In each column, the x-axis of the analyzed dependence might reflect relative stimulus of prior trial, or relative response of prior trial. However, the observed results would only be a simple sum of the two effects (third column) if both dependencies are of the same type (both prior stimulus, or both prior response) or in the special case that they are different (one prior stimulus, one prior response) but accuracy is perfect, making them equivalent. B) Predicted error for all combinations of prior stimulus and prior response. This two-factor analysis allows identification of two dependencies when one is caused by prior stimulus and the other by prior response. The plots in (B) show all combinations of the prior response, drawn from the first column of (A), and prior stimulus, drawn from the second column of (A), with these two effects summing to produce predicted error. When the prior trial is accurate, the combination corresponds to the predictions along the diagonal of the plots in (B), which is equivalent to the summation plots in the third column of (A). The colors indicate the sign and magnitude of the predicted error for the current trial. C) Predicted error for combinations of prior response and stimulus, averaged to aid visualization. A typical experiment will not contain every possible combination of prior stimulus and response, but key information is retained by averaging errors within quadrants.

The example in Figure 2A, bottom row, demonstrates just how misleading one-factor analyses can be. In this example, a one factor-analysis based on the prior stimulus produces a null result. In addition, a one factor-analysis based on the prior response produces a null result. However, both conclusions are incorrect. In reality, there is both an effect of the prior stimulus and the prior response. Thus, it is potentially misleading to use a one-factor analysis and yet all prior studies in the serial dependence literature are based one-factor analyses. Furthermore, the situation in Figure 2A is not uncommon. To preview our results, of 5 datasets that we analyzed, two of them produced seemingly null results for one-factor analyses based on only the prior stimulus or prior response and yet a two factor-analysis revealed this to be caused by offsetting dependencies. Furthermore, the results of all 5 datasets were consistent with offsetting dependencies.

This problem is widely acknowledged, and several studies have sought to isolate separate dependencies experimentally (Bae & Luck, 2020; Cicchini et al., 2017; Fischer & Whitney, 2014; Fornaciai & Park, 2019; Fritsche et al., 2017; Pascucci et al., 2019b; Pascucci & Plomp, 2021; Suárez-Pinilla et al., 2018), but these studies often produce contradictory results. This might reflect a common challenge: the psychological operations that contribute to different sequential effects, like attraction to the prior stimulus or response hysteresis, are not easily manipulated independently. For example, one approach instructs participants to withhold their responses on a subset of trials, asking whether the attraction persists on trials after which there was no response (Fischer & Whitney, 2014; Fornaciai & Park, 2019; Pascucci et al., 2019b; Pascucci & Plomp, 2021). However, a lack of a motor response does not imply a lack of a perceptual decision (Pascucci et al., 2019b), and the prior decision may be sufficient to cause dependencies (Akaishi et al., 2014). To reduce the role of unrealized decisions, some studies instructed participants to ignore the stimulus (Pascucci et al., 2019b), or informed them that prior stimuli were irrelevant to the task (Fornaciai & Park, 2018a), but this introduces a confound with attention, and the continuity field is believed to be sensitive to how well participants attend to the prior stimulus (Fischer et al., 2020; Fischer & Whitney, 2014). To assess attention, some researchers have asked participants to report on a subset of the otherwise ignorable stimuli (Pascucci et al., 2019b; Pascucci & Plomp, 2021), but then it is difficult to guarantee that attentional checks do not reintroduce unrealized decisions. In summary, although it is straightforward to manipulate participants’ responses, it is more challenging to manipulate their decisions while also equating every aspect of perceptual processing.

Given the difficulties inherent to isolating dependencies experimentally, we explored whether dependencies could be isolated statistically. The standard serial dependence paradigm, in which participants simply report the stimulus on each trial, may in some cases provide sufficient constraint to untangle the effect of the prior stimulus from the effect of the prior response, provided that: 1) the analysis considers combinations of prior response and prior stimulus (i.e., a two-factor analysis); and 2) there is sufficient trial-by-trial variability in these combinations (e.g., participants’ accuracy varies across trials). In the hypothetical scenario of Figure 2A, if one dependency (e.g., first or second column) is driven by the prior response and the other is driven by the prior stimulus (e.g., the other column), a one-factor analysis of either the prior stimulus or prior response is one that collapses over the other factor. Rather than collapsing over one factor, errors can be assessed across all possible combinations of prior stimulus and prior response (Figure 2B) with an explicit model, or and the model’s results can be compared to a model-free approach (e.g., averaging across combinations of prior stimulus and response, as in Figure 2C). These two-factor analyses can provide insight in cases where the one-factor approach fails. For instance, in the scenario where the two effects perfectly mirror each other (third row of Figure 2), the trials for which the prior response is not perfectly accurate exhibit systematic errors on the current trial that can be used to identify the two sequential dependencies (Figure 2B, C).

We developed a two-factor, statistical modeling approach that can identify two dependencies, where the combinations of prior stimulus and response arise from natural variations in trial-by-trial accuracy. The approach first uses model comparison to determine whether a dataset is constraining enough to identify multiple dependencies. If there is sufficient constraint, the parameters of the model that is best supported by the data can be inspected to determine the direction of those dependencies. We applied this framework to a new experiment that collected considerably more data than a typical serial dependency study. We then applied the technique to previously published datasets (Fischer & Whitney, 2014; Pascucci et al., 2019b; Samaha et al., 2019).

Methods

Experiment 1

As discussed above, disentangling stimulus and response effects requires variability between prior responses and prior stimuli. To gain the data needed to tease apart a dependency on the prior stimulus from one arising from the prior response, we used a challenging orientation judgment task (i.e., performance was not perfectly accurate) with a high trial count (i.e., good statistical power in terms of combinations of prior stimulus and prior response). The stimulus parameters and experimental procedure followed the design of Samaha and colleagues (2019). This task involved briefly presented stimuli corrupted by noise. As such, the results of this study may not generalize to all serial dependence tasks, in which the stimuli are uncorrupted by noise and participants are allowed to view the stimuli for extended durations. Hence, we additionally analyzed four published datasets that used a range of protocols (described below).

Participants

Power analyses indicated that approximately thirteen participants would be sufficient both to detect a single dependence and to reliably distinguish between one versus two dependencies (Appendix B). To allow for the possibility that data from some participants would be unusable, sixteen participants were run, including author PS, with all except PS being awarded course credit. All participants had normal or corrected-to-normal vision, and all provided usable data. The procedure was approved by the University of Massachusetts Amherst Institutional Review Board.

Stimulus Parameters

Stimuli were presented on an LCD monitor (ASUS VG248QE, 1920 × 1080 cm, 100 Hz refresh rate, 1920 × 1080 resolution), viewed from approximately 60.96 cm. Stimuli were displayed using the Psychophysics Toolbox (Version 3.0.14; Brainard, 1997; Pelli, 1997) and custom MATLAB code (2015a, MathWorks).

Throughout the experiment, participants fixated on a light gray dot (0.08°). Grating stimuli (sine wave with 1.5 cycles per degree and phase 0 subtending a circular region of 2°) were presented on a medium gray background. Mask stimuli consisted of white noise rendered at 100% contrast. Participants were cued to make orientation reproduction responses with a circle (6° radius) and made the response by clicking a mouse near the circle (within 50 pixels of the circle).

The signal-to-noise ratio of the grating was reduced by averaging the grating with white noise (Samaha et al., 2016, 2019). The contrast of the grating was determined by a pilot study (3 participants, data not shown), in which white noise (100% contrast) was averaged with a grating stimulus presented at a range of contrast levels. The main experiment used a contrast (10%) that elicited responses that were within ±25° of the true orientation approximately 80% of the time.

Procedure

The trial structure is illustrated in Figure 1A. Each trial began with the presentation of a grating, surrounded by a circle. To encourage fixation, a centrally presented dot was visible continuously. The grating was replaced by a mask after 200ms, but participants could initiate responses immediately after target onset¹. The mask remained on the screen until participants finished their response. A circle surrounded the grating and mask, and participants responded by using a computer mouse to click on the circle. They were instructed to report the orientation of the grating stimulus by clicking a point on the circle such that an imagined line connecting the center of the grating (i.e., the fixation point) to the clicked point would be parallel to the stripes of the grating. They first practiced the task for 10 trials with an experimenter present and available to answer questions about the task. Participants could respond on either side of the circle. Trials were separated with a variable fixation period (randomly determined on each trial with a draw from a discrete uniform distribution ranging from 300–500 ms in steps of 20 ms).

The orientation on each trial was drawn at random from a discrete, uniform distribution, sampling integers between 0–179°. Participants completed 15 blocks, each with 101 trials, yielding 1,515 trials per participant, and 24,240 trials in total. The median duration of the experiment was 48 minutes (range: 34–86).

Published Datasets

Four published datasets were reanalyzed (Fischer & Whitney, 2014 experiment 1b; Pascucci et al., 2019a experiments 1 and 2; Samaha et al., 2019). See Table 1 for an overview of the methods. Note that, in the original publications, the data were preprocessed to a different extent than in our analyses. Preprocessing in the original studies included: A) excluding outlier responses (Pascucci et al., 2019b; Samaha et al., 2019); B) centering each participant’s errors to remove any overall clockwise or counterclockwise bias in that individual (Pascucci et al., 2019b; Samaha et al., 2019); and/or C) regressing out biases caused by the orientation on the current trial (Pascucci et al., 2019b), whereby participants respond less accurately for certain orientations, regardless of the stimuli or responses preceding those orientations (e.g., Jastrow, 1892; Wei & Stocker, 2015). Preprocessing steps B) and C) were built into our main analyses (model comparison, described below) by including regressors for these effects. Of note, any bias for particular orientations can cause a spurious dependence on the prior response, and so they must be considered when modeling response dependencies (Appendix A; Fritsche, 2016).

Table 1:

Protocols for studies of serial dependence in orientation judgments.

Study	Experiment	Inducer Duration	Inducer Eccentricity	Inducer-Target Position	Inducer Contrast	Inducer SF	Mask Duration	Mask-to-Response Delay	ITI Averagea	Trials
Current Study	1	200	0	Same	10b	1.5	0	0	400	24240
Fischer Whitney (2014)	1bc	500	6.5	Same	25	0.33	1000	250	2000	3296
Pascucci et al. (2019)	1	400	8.5	Varied	50	0.5	400	500	500	5600
Pascucci et al. (2019)	2	400	0	Same	50	1.2	400	500	500	4400
Samaha Switzky Postle (2019)	1	33	0	Same	8.5b d	1.5	0	6300e	700	6000

^aIn some experiments ‘Average ITI’ includes both the ITI and a period of cueing at the start of the target trial. In all experiments, the reported ITI does not include participants’ response time.

^bContrast refers to Michelson contrast of gratings, which were convolved with white noise.

^cIn the original experiment, this was listed as a control experiment for experiment 1. In experiment 1, the orientations on each trial were explicitly counterbalanced, whereas in 1b they were randomized.

^dThe reported amplitude averages across two conditions. The conditions kept the signal-to-noise ratio of the stimuli equal, but halved the grating and noise contrasts.

^eThis is an average across five conditions (600, 3450, 6300, 9150 and 12000). Authors observed that delay did not have a substantial effect.

Note: Eccentricity: distance in degrees visual angle from fixation to the center of the stimuli; Contrast: Percent Michelson; SF: Spatial Frequency, cycles per degree; ITI: Inter-trial Interval. All timing is reported in milliseconds.

Analyses

All datasets were analyzed in three ways: 1) different hierarchical Bayesian models were fit to the data and compared to each other, assessing whether the data were sufficiently constraining to distinguish between effects of the prior orientation and prior response (a two-factor analysis); 2) standard one-factor serial dependence analyses (e.g., plotting smoothed errors) based on the raw data (these plots will not necessarily match the previously published plots, which were based on pre-processed data); and 3) a non-parametric quadrant analysis (a two-factor analysis) that sorts trials into one of four quadrants depending on the relative direction of the prior stimulus and prior response. When reported, confidence intervals are across participants (Morey, 2008).

Bayesian Analyses

Three hierarchical Bayesian models were fit to each dataset. The models allowed for one of the three combinations of effects of the prior orientation and prior response (i.e., two models with only one dependency and one model with both dependencies). All three models also included terms to capture rotational bias (i.e., a participant’s tendency to err in a clockwise or counterclockwise direction by a small amount on all trials) and biases towards particular salient angles. The latter was parameterized as either a bias towards the nearest cardinal angle (e.g., 0° or 90°), and thus a bias away from oblique angles, or a bias towards oblique angles (e.g., 45°, 135°), and thus away from cardinal angles, with the latter possibility termed “the oblique effect” in the literature (Jastrow, 1892; Wei & Stocker, 2015). Here, we provide a brief description of the Bayesian models that were used to analyze the data. A more detailed description of the models, including how they handle preferences for oblique/cardinal angles, is given in Appendix A.

The models were additive, assuming that error on the current trial reflected the summation of any dependencies from the prior trial along with biases that did not depend on the prior trial (i.e., rotational and periodic biases). One model included a dependence on the prior stimulus, a second included a dependence on the prior response, and third included both dependencies. Added together, the serial dependence biases and the cardinal/oblique/rotational biases determined the average error as a function of the current orientation and the stimulus/response from the prior trial. Trial-by-trial variability in this average was modeled with a normal distribution.

Serial dependencies in orientation were modeled using the derivative of the density function for a circular normal distribution². All models were estimated hierarchically, allowing for parameters to vary by participant while simultaneously allowing the participant-specific parameters to constrain each other via a “population-level” distribution (Kruschke, 2015, Chapter 8). The population-level distribution can be informally understood as an average across participants – an estimate produced by analyzing participants separately and then summarizing those individual analyses. Analyzing them hierarchically rather than separately accounts for individual differences while simultaneously pooling information across participants to gain greater reliability.

Models were constructed and fit using the Stan language (Carpenter et al., 2016), using its interface (RStan, 2.18.2) for the R computing language (R Core Team). Stan draws samples from an approximation of the posterior distribution using a modified version of the Hamiltonian Monte Carlo algorithm (Duane et al., 1987; Hoffman & Gelman, 2014). The validity of this approximation was assessed in two ways. First, chains were monitored for divergences, an indication that the numerical simulation methods in the algorithm are compromised (Betancourt, 2018). Second, the split-$\widehat{R}$ (“split r-hat”) for each parameter was calculated (Gelman & Rubin, 1992). In all results, there were no divergences and for each parameter the split-$\widehat{R}$ was below 1.1 (Gelman et al., 2013, Chapter 14).

Model comparison was based on the models’ predictive abilities with an approximation to a leave-one-out (LOO) cross-validation score (Vehtari et al., 2017, 2019). Specifically, models were compared based on their Pareto-smoothed, importance sampling leave-one-out plus (PSIS-LOO+) score, as calculated with the loo software package (Vehtari et al., 2020).

Median Window

In addition to the two-factor Bayesian modeling, each dataset was also analyzed with a one-factor analysis (e.g., Figure 2A), a median window based on either the prior stimulus or response. The median error on the current trial for each participant was calculated using a sliding, median window, centered on 200 equally spaced orientation differences between −90° and 90° (width of ±12°, following Samaha et al., 2019). The smoothed error for orientation differences was calculated between both (1) the prior and current orientations and (2) the response to the prior stimulus and the current orientation. To eliminate boundary artifacts, windows near ±90° included circularly wrapped copies of the data (e.g., data from stimulus orientations of 78–89° were considered to be adjacent to an orientation of −90°). The across-participant distribution of median errors was then used to calculate confidence intervals.

Quadrant Plots

The hierarchical Bayesian models make parametric assumptions about the functional form of the underlying serial dependencies (i.e., circular normal distributions) to increase statistical power. To check that the data show features consistent with the conclusions from model comparison, we also developed a non-parametric two-factor quadrant plot analysis that can be used to visualize the data in terms of separate dependencies. This was achieved by sorting each trial into one of the four categories for the combinations of whether the prior stimulus was clockwise or counterclockwise as compared to the current stimulus, and whether the prior response was clockwise or counterclockwise as compared to the current stimulus, as shown in Figure 2C. For each of these four possible prior stimulus/response categories, current trial errors were averaged within participants, and confidence intervals were calculated across the averages.

For the quadrant plots, if the prior stimulus is the only dependence, there should be a main effect of prior stimulus in the 2 × 2 grid of quadrants (as seen in Figure 3A, repulsion from the prior stimulus results in a clockwise bias for the left column and a counterclockwise bias for the right column). If the prior response is the only dependence, there should be a main effect of prior response (as seen in Figure 3B, attraction to the prior response results in a clockwise bias for the top row and a counterclockwise bias for the bottom row). If both dependencies exist, there should be an interaction. For instance, if the two dependencies are in opposite directions, the two “accurate” quadrants (upper-right and lower-left) will not display much of a bias because the dependencies counteract each other, whereas the two “inaccurate” quadrants (upper-left and lower-right) will show biases, with one clockwise and the other counterclockwise (see Figure 3C). In contrast, if the two dependences are in the same direction, the two accurate quadrants will show a bias, with one clockwise and the other counterclockwise, while the two inaccurate quadrants will not display much of a bias because the two dependencies counteract each other (see Figure 3D). Thus, the quadrant analysis can display whether the data are consistent with the presence of two serial dependencies that are either in opposition or in agreement (this is also true of the hierarchical Bayesian model, as seen in Appendix Figure B2, which breaks down the power analysis according to whether the direction of the two dependencies is the same or opposite, revealing equivalent statistical power in both cases).

Figure 3.

A guide to interpreting quadrant plots, with examples of a dependence only on the prior stimulus (A), a dependence only on the prior response (B), or two dependencies that are in opposite directions (C) or the same direction (D).

We use the quadrant plot as a visual aid, rather than a statistical test, owing to a sampling bias that makes it unsuitable for inferential tests. This sampling bias results in the average magnitude of difference from the prior stimulus/response being larger in the accurate quadrants compared to the inaccurate quadrants. For instance, the average extent to which the prior stimulus is clockwise from the current stimulus (i.e., the right column) is likely to be greater for the upper-right accurate quadrant than for the lower-right inaccurate quadrant. This is because whenever the prior stimulus is very clockwise from the current stimulus (a datapoint falling to the far right of the x-axis), the response to the prior stimulus is very likely to also be clockwise from the current stimulus (thus falling into the upper-right quadrant), whether the participant made a small or a large error: only extremely large errors on the prior trial would cause the prior response to cross into the inaccurate (lower-right) quadrant. In contrast, whenever the prior stimulus is only slightly clockwise from the current stimulus (falling just to the right of zero on the x-axis), the response to the prior stimulus is much more likely to fall counterclockwise from the current stimulus (thus falling into the lower-right quadrant): even a small error on the prior trial might cause the response to cross into the bottom-right quadrant. Thus, on average, datapoints falling in the upper-right quadrant will be biased towards the right side of the quadrant and datapoints falling in the lower-right quadrant will be biased towards the left (more central) side of the quadrant.). The critical point is that, if there is only one dependency, the main effect outcomes seen in Figure 3A and 3B are unlikely, because of this sampling bias. To reach statistical conclusions about whether two dependencies exist versus one, or whether the dataset contains sufficient constraint to answer this question, we recommend use of model comparison, followed by use of quadrant plots as a check to see if the data generally support the conclusions reached with model comparison.

Results

Bayesian Analysis

Bayesian models were applied to the trial level data. Three models were fit that allowed for either 1) an effect of only the prior orientation, 2) an effect of only the prior response, or 3) an effect of both the prior orientation and the prior response. Models were compared by their PSIS-LOO+ score (Vehtari et al., 2017, 2019), which estimates the model’s ability to accurately predict held out data. This should adjust for the extra model flexibility inherent in the model with both dependencies. In Experiment 1 (Figure 4), model comparison favored the model with both effects as compared to either of the single effect models (difference ± standard error: full model vs. stimulus only: −103.72 ± 16.04; full model vs. response only: −149.99 ± 19.61). Thus, Experiment 1 provided enough constraint to estimate two separate dependencies.

Figure 4.

Model comparison (predictions for held out data) between a model with only the prior stimulus, only the prior response, or both dependencies. In all experiments, the model with both was either the most predictive model or tied for most predictive. The y-axis shows the relative expected log predictive density, estimated with PSIS-LOO+ for each model, as compared to the most predictive model (i.e., the most predictive model is always 0, with error shown by the shaded rectangle). Error bars extend two standard errors of the mean of the difference.

With five times more data than is typically collected in serial dependence studies (see Table 1), Experiment 1 had higher statistical power for dissociating the effect of the prior response from the effect of the prior stimulus (see also Supplementary Figure C3, which shows that the magnitude of errors in Experiment 1 was comparable to the other datasets, suggesting that model comparison was reliable for Experiment 1 primarily because of trial count). We next applied model comparison to four previously published datasets (Figure 4). Across the datasets, only one other – Pascucci et al., experiment 1 – provided enough constraint to reliably separate stimulus and response effects. That is, although one-factor analyses of the remaining three experiments suggested dependencies (Appendix A, Appendix C), model comparison did not allow a definitive conclusion as to whether there were dependencies on the previous stimulus, previous response, or both (i.e., at least two models tied for being the most predictive, Figure 4).

To assess why the models could not be reliably distinguished in some cases, we performed a best-case scenario simulation study that assumed independence between prior responses and prior stimuli and eliminated individual differences and bias effects other than prior responses and prior stimuli. The models were fit to artificial datasets generated in this best-case scenario manner, with different datasets generated assuming either one or two dependencies (Appendix B). These simulations had two goals: first, to confirm that a comparison of Bayesian models could in principle determine whether a dataset contained one versus two dependencies, and, second, to estimate how many trials would be required for that determination to be reliable in this best-case situation. The simulations revealed that approximately 10,000 trials are required to reliably distinguish between two effects (this trial count is not broken down by subject because this situation assumed no individual differences). At 5,000 trials, two components may be identified in only 75% of experiments. Considering that none of the 4 previously published datasets contained more than 10,000 total trials (Table 1), it is unsurprising that model comparison failed to reach clear conclusions for 3 of them. In light of these results, we report here additional analyses of the two experiments that were able to separate stimulus and response dependencies, namely the present experiment, which contained more than 10,000 trials, and Experiment 1 of Pascucci et al. (for completeness, analyses for all experiments are presented in Appendices A and C).

Before interpreting the results, we assessed whether the winning model adequately captured the data by comparing observed data to predicted data in terms of errors smoothed with a median window and plotted separately as a function of either the prior orientation or prior response (Figure 5, left column for observed data and middle column for data generated from the model with both dependencies). Note that the model predictions exhibited “peripheral bumps” (Fritsche et al., 2017), whereby the dependence on the prior response appears to swap from attractive to repulsive at approximately ±45°. This occurred even though each individual derivative of a circular normal distribution function is either exclusively repulsive or exclusively attractive. In the simulated datasets, the peripheral bumps reflect the sum of two overlapping dependencies; the same effect is illustrated in the middle row of Figure 2A.

Figure 5.

Model results from the model with both dependencies. Model predictions (middle) can be compared to observed data (left), and each estimated dependency is shown in isolation (right), revealing that errors are attracted to the prior response and repelled from the prior stimulus. The two experiments for which model comparison could distinguish stimulus and response effects are in the different rows. Left: Preprocessed data were smoothed with a moving median window. Ribbons span the 95% confidence intervals, across participants. Middle: To check whether the model captured the data, the model with both dependencies was used to generate artificial datasets using parameters estimated from the real data. The generated datasets were preprocessed as in the left column. Ribbons span the 95% highest density interval across the preprocessed data. Right: The 95% highest density interval for the posteriors of the estimated dependencies. Unlike the median smoothing (left/middle column) that summates the dependencies, these plots show each dependency in isolation after removing the contribution of the other dependency and after removing any preferences for oblique or cardinal angles and any bias to rotate responses. PPD: Posterior Predictive Distribution.

Critically, the directions of the two dependencies in the models were unconstrained: both effects could be attractive, both could be repulsive, or one could be attractive and the other repulsive. Nevertheless, for both experiments that provided sufficient constraint for disentangling stimulus and response effects, the results indicated an attraction to the prior response and repulsion from the prior stimulus (Figure 5, right column). Furthermore, as seen in third column of Appendix Figure C4, the results from the other three datasets were consistent with the claim that repulsion from the prior stimulus and attraction to the prior response is a general property of serial dependence experiments. More specifically, when applying the model that contained both dependencies, none of the datasets produced results that were reliably attractive to the prior stimulus (i.e., the 95% highest density interval for the prior stimulus effect was either indeterminate between attraction versus repulsion or decidedly favoring repulsion) and all datasets produced results that were reliably attractive to the prior response (i.e., the 95% highest density interval for the prior response effect was decidedly favoring attraction).

Quadrant Plots

The Bayesian analyses suggest that the prior stimulus and prior response exerted opposite influences on errors, but this conclusion appears to contradict the moving window analysis, which does not show a clear repulsion from the prior stimulus nor an unequivocal attraction to the prior response (Figure 5, left column). As discussed in relation to Figure 2, this may reflect the limitations of a one-factor analysis such as the median window (Figure 2A), whereby two opposing dependencies can counteract each other. We therefore sought to visualize the effects suggested by the Bayesian model with an alternative, non-Bayesian analysis that we refer to as a “quadrant plot”. Quadrant plots group together trials according to whether the prior trial was clockwise or counterclockwise relative to the current trial, both in terms of the prior stimulus and prior response (e.g., Figure 2C and Figure 3). The winning Bayesian model predicted, for both datasets, that the response and stimulus dependencies should counteract each other, specifically in the two diagonal quadrants (bottom left and top right), resulting in nearly unbiased responses. The model also predicted that the two dependencies should work in the same direction for the off-diagonal quadrants, resulting in large errors in the upper-left and lower-right quadrants, but with these errors of opposite direction in the two quadrants. This pattern of results is clearly seen for both experiments that offered sufficient constraint to separate the effects (Figure 6; quadrant plots for the remaining experiments are provided in Appendix Figure C2).

Figure 6.

Errors differ across combinations of clockwise and counterclockwise prior stimulus and prior response in (A) Experiment 1 and (B) Pascucci et al. Experiment 1. Datapoints and colors in each quadrant correspond to the average error for the associated combination of clockwise/counterclockwise prior stimulus and prior response (compare to Figure 2C). Inset numbers in each quadrant indicate how many trials fell into that quadrant. Error bars cover the 95% confidence intervals.

Discussion

Recent stimuli and recent responses influence perceptual decisions. For instance, in visual aftereffects, such as the tilt aftereffect, perception is repelled away from recently viewed stimuli. In contrast to visual aftereffects, the serial dependence effect appears to show that perception is attracted towards recently viewed stimuli. However, this appearance may be misleading, considering that stimuli elicit a cascade of processes that range from detecting visual primitives to making perceptual decisions, with effects at one level potentially counteracting effects at another level. We claim that analyses of serial dependence should assume dependencies on both the prior response and prior stimulus. To address this mixture of processes, we developed new analysis techniques, ran a new experiment that collected enough data to reliably apply these techniques (five times more than is typically collected), and re-analyzed four previously published serial dependence experiments. The results lead to two separate conclusions.

First, we draw the methodological conclusion that typical experimental procedures in the study of serial dependence can in theory tease apart these two sequential dependencies, although in practice the amount of data required is substantially larger than is typically collected. When using a one-factor analysis (i.e., an analysis that considers the prior stimulus and collapses over prior response, or vice versa), perfectly opposing dependencies and relatively low response accuracy can lead to mistaken conclusions (e.g., the last row of Figure 2A suggests the absence of any effect). However, a two-factor analysis (i.e., an analysis that considers combinations of prior stimulus and prior response) can identify both dependencies when there are a sufficient variety of combinations of the two variables (i.e., ample data on the “off-diagonal” in Figure 2B; see also Figures 3C and 6). For our analyses, we estimated that, at minimum, experiments should contain 10,000 total trials (Appendix B), substantially more than were collected in many serial dependence experiments (e.g., Table 1). This estimate represents a “best-case” scenario using simulated data, but the necessary total trial count, and the breakdown of that total into a certain number of subjects and trial count per subject, will depend on experiment-specific factors like the accuracy of responses, variability across participants, the magnitude of the individual effects, and the specific analysis method.

Our methodological conclusion regarding use of a two-factor analysis is related to, but different than use of a two-factor model (Pascucci et al., 2019b). Pascucci et al. compared performance of a one-factor model (attraction to the prior stimulus) versus a two-factor model that contained both repulsion from the prior stimulus and attraction to the prior response, with these factors summing up to produce a single response function. They applied both models to data, finding that the two-factor model provided a better explanation. For instance, the two-factor model explained an experiment in which repulsion from the prior stimulus was found when the prior stimulus was unreported, but attraction to the prior stimulus was found when the prior stimulus was reported. However, this was a one-factor analysis that considered only the orientation of the prior stimulus rather than considering trial-by-trial combinations of prior stimulus and prior response. As shown in Figure 2, a one-factor analysis can be misleading, and this is still true even if a two-factor model is fit to the one-factor analysis. For instance, the current experiment produced a completely null result when using a one-factor analysis of prior stimulus. Application of the two-factor model to data collapsed over prior response would lead to a parameter identifiability problem because the results would be equally explained by setting both factors to zero, or by having two factors that perfectly counteracted each other. In contrast, as shown in the model results in Figure 5 and the quadrant analysis in Figure 6, when using a two-factor analysis, the apparently null result is revealed as two highly reliable factors that counteracted each other. In support of Pascucci et al.’s two-factor model, when we used a two-factor analysis, the two-factor model did significantly better than either one-factor model at explaining the data for two of the five datasets, and for the other three datasets, the two-factor model was either the best model or not reliably different than the best model (the latter cases occurred when the one-factor model with only prior response was the best model). In no case was there evidence against the two-factor model. Thus, we claim that typical serial dependence experiments contain both a dependence on the prior stimulus and a dependence on the prior response (see also Moon & Kwon, 2022). More importantly, we find that conclusions about the direction of any dependencies should be based on trial-by-trial two-factor analyses.

Second, we conclude that when applied to data from the present experiment and from four previously published studies, our analysis failed to support the existence of an attractive effect towards the distal properties of the prior stimulus (as opposed to the percept of the stimulus, which may better align with the prior response). That is, by using analysis techniques that tease apart response and stimulus effects, we failed to find an attraction to the prior stimulus in any dataset. For two of the five experiments, stimulus and response effects were teased apart successfully, and for these two experiments we found that the response effect was an attraction to the prior response whereas the stimulus effect was repulsion from the prior stimulus. In the remaining three experiments, no conclusion could be drawn about an effect of the previous stimulus. For all datasets, when applying a model with both dependencies, the response effect appeared to be attractive while the stimulus effect was either repulsive or indeterminate.

The existence of two dependencies may help explain conflicting results in the literature. As noted in the introduction, several studies have observed positive dependencies on the prior stimulus when participants withhold responses (Fischer & Whitney, 2014; Fornaciai & Park, 2018a; Pascucci et al., 2019b). Other studies find repulsive effects even when responses are given (Bae & Luck, 2020). If there is only one kind of dependency, it is not clear from these studies why the dependency should be so changeable, including reversals in the direction of the dependency. However, all of these results used one-factor analysis techniques, which analyzed the data in terms of the prior stimulus or prior response while collapsing over the other factor. In contrast, our results suggest that most experiments encourage two (or more) dependencies, and that for a given experiment the balance may be tilted in favor of either attraction or repulsion within a one-factor analysis, or even balanced to give the appearance of no dependence. If this is conclusion is correct, care must be taken when interpreting changes in the magnitude of a dependence analyzed with a one-factor analysis. For example, a manipulation that appears to reduce or even reverse an attraction effect as inferred with a one-factor analysis may in fact reflect a relatively subtle change in the balance of power between a robust attraction to the prior response that counteracts partially or wholly a robust repulsion from the prior stimulus. Unless a two-factor analysis is used, it is not clear whether the reduction in attraction effect reflects a reduction in the attractive dependency or an increase in the repulsive dependency.

There are two important caveats to our conclusions. First, the statistical modeling technique assumed that the magnitude of stimulus and response dependencies are constant across trials (the parameters of the circular normal distribution used to capture each kind of dependency were allowed to vary between participants but were assumed to be fixed for each participant across trials). This is likely a simplification. For instance, perception of the prior stimulus is likely to vary across trials due to factors such as fluctuations in vigilance; in the extreme case, a participant may fail to look at the prior stimulus entirely, precluding any opportunity for dependence on that stimulus. By failing to acknowledge these fluctuations, the model likely underestimates the magnitude of the stimulus dependence. Such underestimation could have contributed to the ambiguous effect of the previous stimulus in the three datasets for which model comparison resulted in a tie. However, when modeling data that allowed identification of two dependencies, the stimulus effect was repulsive rather than attractive, meaning that the only positive evidence we have for any effect of the previous stimulus is for a repulsive effect, in line with the classic tilt aftereffect. Although this limitation may have led to underestimation of effect magnitude, it is unlikely that it resulted in mistakenly identifying the direction of the stimulus dependence as negative when it was truly positive.

The second caveat is that our results, although supportive of an interpretation of the serial dependence effect as a mixture of two dependencies (Bae & Luck, 2020; Fornaciai & Park, 2019, 2020; Fritsche et al., 2017; Pascucci et al., 2019b), do not reveal the psychological processes that caused the two dependencies. Nor do they reveal whether the psychological mechanisms are distinct or shared, nor how they relate to the wider literature on sequential effects (Kiyonaga et al., 2017). When we were able to statistically isolate both dependencies, they resembled effects well-documented before the proposal of a continuity field: A repulsive visual aftereffect and an attractive response effect. We suspect that the repulsion from the prior orientation is a manifestation of the tilt-aftereffect, given that the effect is ubiquitous and automatic, it can occur after brief exposure to the inducer (e.g., under 10 ms; Sekuler & Littlejohn, 1974), does not require awareness of the inducer (Kanai et al., 2006), is sensitive to a range of low-level features (Greenlee & Magnussen, 1987; Harris & Calvert, 1985, 1989; Morant & Mikaelian, 1960; Parker, 1972), can occur across different screen locations and spatial frequencies (Jacob et al., 2021; Morant & Mikaelian, 1960; Parker, 1972), and could arise from mechanisms that have been observed in single-cell recordings of early visual neurons (Clifford et al., 2000; Dragoi et al., 2000; Gutnisky & Dragoi, 2008; Patterson et al., 2013; Wissig & Kohn, 2012). These prior results suggest that a repulsive tilt aftereffect is caused by merely viewing the inducing stimulus, and so there is every reason to expect that such perceptual aftereffects play some role in a typical serial dependence effect study.

We further suspect that the attraction to the prior response is mediated by a process that lies “further along” the pathway than the early-stage processes implicated in the repulsive tilt aftereffect (e.g., the processes involved in making a perceptual decision). It remains possible that the process is perceptual, as implied by the continuity field; e.g., if participants “saw” an orientation that was not presented and accurately reported that illusory orientation, then subsequent attraction toward the (inaccurate) perception of the inducer would manifest as attraction to the prior response (see also Cicchini et al., 2017; Fischer & Whitney, 2014; St John-Saaltink et al., 2016). This account of the attraction effect agrees with the characterization of serial dependence as a manifestation of perception enacting Bayesian inference (Cicchini et al., 2018; Kalm & Norris, 2018; van Bergen & Jehee, 2019). However, it is also possible that participants perceive the prior orientation accurately but respond inaccurately, and in that case, the attraction could reflect their prior decision about the inducer (Akaishi et al., 2014; Braun et al., 2018). Indeed, multiple high-level processes have been proposed as accounts of response hysteresis, including shifting beliefs about the base rate of stimuli (Zhang et al., 2014), slow changes in how much evidence participants require before initiating responses (Wagenmakers et al., 2004), or the integration of low-level sensory information into a decision (Pascucci et al., 2019b). Finally, it possible that there are three (or more) effects, with an attraction to the prior response coexisting with both a repulsion and attraction to the prior stimulus – our analyses cannot rule out this possibility.

In sum, we suggest that studies of the serial dependence effect ought to be analyzed with two-factor analyses that consider combinations of prior stimulus and prior response. We developed two new techniques for doing so, with the first assessing the reliability of separate dependencies using model comparison, and the second producing a way of visualizing the separate dependencies in a quadrant plot. As applied to several datasets, these techniques failed to find evidence of an attraction toward the prior stimulus. Furthermore, when they were able to identify separate dependencies, they revealed a repulsive stimulus effect and an attractive response effect.

Appendix A: Bayesian Models of Serial Dependence

To account for the circular nature of orientation, the commonly used derivative of a Gaussian model was replaced with a derivative of the density function of a von Mises distribution.

The density function of a von Mises distribution with a width parameter, $w$, for an orientation, $x$ (ignoring, here, whether the orientation is reported or presented), centered on orientation 0 is given by

$$f\left(x|w\right)=\frac{\exp \left(w\cos \left(x\right)\right)}{I_0\left(w\right)}$$ 1

In that density function, $I_0\left(\cdot \right)$, is the modified Bessel function of the first kind, of order 0. It normalizes the density function so that its integral is constant between $\pm \pi$ ($x$ is in radians with period of $2\pi$). The derivative of this density function with respect to $x$ is equal to the following.

$$f^{\prime }\left(x|w\right)=\frac{w\sin \left(x\right)\exp \left(w\cos \left(x\right)\right)}{I_0\left(w\right)}$$ 2

This derivative has only a single parameter, $w$, which governs both the width and height. To serve as a model for serial dependence, this derivative must be rescaled, introducing an amplitude parameter, $a$.

$$g\left(x|w,a\right)=\frac{aw\sin \left(x\right)\exp \left(w\cos \left(x\right)\right)}{I_0\left(w\right)f^{\prime }\left(2\mathit{arctan}\left(\sqrt{\sqrt{4w^2+1}-2w}\right)|w\right)}$$ 3

The value ($\left(\sqrt{\sqrt{4w^2+1}-2w}\right)$ is the orientation at which the derivative (Equation 2) peaks, itself a function of $w$. So, the rescaling divides the derivative by its maximum (forcing the maximum to 1) and then multiplies by $a$. With this rescaling, the parameter $a$ can be interpreted as the maximum error, which is equivalent to the amplitude of the derivative of a Gaussian that has previously been used to assess serial dependencies (e.g., Fischer & Whitney, 2014).

In modeling the error of each trial, either one or two distinct rescaled derivative functions (Equation 3), with separate parameters, were summed to capture the two dependencies. Thus, the independent variable, $x$, that induces a particular dependence may be the difference between the current orientation and the prior orientation, or the difference between the current orientation and the prior response, depending on which dependencies are included in a model.

Two additional sources of bias were incorporated into all models. First, each participant may exhibit a general clockwise or counterclockwise rotation bias – an offset that affects all orientations equally by rotating all responses somewhat (all prior studies analyzing serial dependence that account for this bias instead do so in a preprocessing step, removing the average orientation error from each participant prior to model fitting). Second, when reporting orientations, participants tend to be more erroneous on those orientations that are intermediate between the cardinal and oblique axes (Appelle, 1972; Jastrow, 1892; Wei & Stocker, 2015), reflecting a bias either toward or away from the oblique/cardinal orientations (in the current datasets, the bias was toward the oblique orientations and away from the cardinal orientations). This anisotropy was also present in our data (Figure A1). Some serial dependence studies have accounted for this anisotropy through preprocessing (Pascucci et al., 2019), but not all. We accounted for it in the Bayesian model by including a sinusoidal term that cycled twice over the range of orientations (i.e., contained two peaks over the possible range of orientations, as in Figure A1). Because this term could be positive or negative, the model was able to capture biases in either direction regarding preferences for oblique angles versus preferences for cardinal angles.

The anisotropy (Figure A1) can cause a spurious attraction to or repulsion from the prior response (Fritsche, 2016). If a spurious attraction is present, it can be revealed by shuffling all trials, then plotting errors on the current trial as a function of the relative response of the last trial. Critically, random shuffling of the trial sequence removes actual sequential dependencies, but it does not remove spurious response dependences that arise from a bias to give certain orientations more often than other orientations. After shuffling (Figure A2, second column), some datasets exhibited a spurious dependence on the prior response (e.g., Pascucci et al. Experiment 2, Samaha et al.).

To understand the spurious effect, consider a sequence of responses that are perfectly accurate except for a shift of responses towards oblique angles for stimuli that are neither cardinal nor oblique, as in Figure A1. The key question is how that error on the current trial compares to the prior response. The effect on the data – even though the experimental design includes equal sampling of all orientations – is an overrepresentation of oblique prior responses. This is because both current trials whose orientations are close to cardinal, and prior trials whose orientations are close to cardinal, elicit cardinal responses, and these together give the appearance of attraction to the prior response. More generally, any bias towards specific orientations will give the appearance of an attractive response bias because such a bias results in more prior trials for which that orientation was reported and more current trials for which that orientation is reported.

To investigate the role of spurious dependencies in the Bayesian analyses, the raw data were preprocessed to remove the periodic patterns of the type shown in Figure A1, in a manner that is analogous to the non-sequential biases of the Bayesian models. Preprocessing was done by regressing out both the rotational clockwise/counterclockwise bias and a sinusoidal bias from each participant’s errors. Preprocessing largely mitigated the spurious dependence on the prior response (Figure A2, third column) and visually reduced the dependence on the prior response in the unshuffled data (Figure A2, compare first and fourth columns). In one dataset, the preprocessing may have been insufficient to remove all spurious dependencies (Figure A2, Samaha et al., second row, Fischer & Whitney, third row), indicating that participants in that experiment may have exhibited a bias for particular orientations that was not captured by the sinusoid regressor used in preprocessing and the Bayesian model. Of note, this was a dataset for which the Bayesian analyses were unable to distinguish between dependencies on the prior orientation and prior response; perhaps the inability to remove these biases contributed to the models’ indiscriminability. More generally, these analyses demonstrate the importance of either pre-processing the data to remove biases for particular orientations, prior to additional analyses, or modeling the data with a model that explicitly includes such biases as part of the model.

A schematic of the hierarchical Bayesian model is given in Figure A3. This diagram is designed to give a high-level overview of the relationships among parameters, and how the model relates these parameters to the data. The distributions assigned to each parameter that is not a prior are listed to the right of the diagram, and the priors are listed with the filled, square nodes.

Figure A1.

Errors were largest on orientations intermediate between the cardinal and oblique axes (e.g., between 0° and 45° and between 45° and 90°). Each point corresponds to the error on a single trial, with data from all participants pooled. The x-axis follows the convention that 0° and 180° are horizontal and increasingly positive orientations are more counterclockwise. Blue lines indicate best fitting sinusoids (minimum squared error).

Figure A2.

When errors depend on the orientation of the current trial there is a spurious dependence on the prior response but adjusting for that dependence mitigates the confound. Each row contains the smoothed errors from a different dataset, using all trials (as in Figure 3A). First Column: Errors were smoothed with a moving median window, without preprocessing. Second Column: Errors were smoothed after the trials were shuffled. The shuffling means that any dependence on the prior orientation or response must be spurious. Third Column: Errors were preprocessed, regressing out both a rotational clockwise/clockwise bias and a sinusoidal bias. Trials were shuffled and the errors were again smoothed. Fourth Column: Errors were preprocessed as in the third column, but the original trial order was preserved. In all columns, ribbons span 95% confidence intervals across participants.

Figure A3.

Hierarchical Bayesian Model of Serial Dependence. Observed data, $\mathrm{y}$, are indicated with a shaded node. They were modeled with a normal distribution with standard deviation $\sigma$ and location $\mu$. The parameter $\mu$ is the output of a deterministic function, the summation of biases due to the oblique/cardinal preferences ($\beta$), clockwise/counterclockwise rotational biases ($\gamma$), and serial dependencies caused by either the prior response, the prior orientation, or both. Nodes are grouped with the square “plates”, indicating over which subsets of the data the node is replicated. The dashed plate around $a$ and $w$ indicates where the three models differed: in two models, there was a single $x$ (i.e., a single dependence modeled by a derivative of von Mises), but in the full model there were two kinds of $x$ (i.e., two dependencies). The parameter $a$ is the amplitude of the rescaled derivative of von Mises with width $w$ (Equation 3). The four parameters – $\beta$, $\gamma$, $a$, and $w$ – were estimated for each participant, hierarchically. These hierarchies were modeled with a normal distribution for each of $\beta$, $\gamma$, and $a$, and a half-normal distribution for $w$. The location and scale of these normal (or truncated-normal, truncated at 0) distributions are given by $\mu$ and $\sigma$ in the diagram, respectively. The priors on the population-level effects are given by the filled square nodes. $N\left(\mu ,\sigma \right)$: Normal with location $\mu$ and scale $\sigma$; $\mathit{TN}\left(\mu ,\sigma \right)$: truncated-normal with location $\mu$ and scale $\sigma$; $\Gamma \left(\zeta ,\tau \right)$: Gamma with shape $\zeta$ and rate $\tau$.

Appendix B: Power Analysis of Serial Dependence and Bayesian Model Recovery

A frequentist power analysis was conducted to guide how many participants to recruit, and a related model recovery simulation was performed to validate the Bayesian model. The primary inferential questions of this study hinge on estimated amplitudes of the derivative of von Mises, a function that might indicate either an attractive or repulsive effect of either the prior orientation or the prior response, depending on the sign of the amplitude. These checks were designed to 1) roughly calculate how many participants would be required to reliably reject a null model, and 2) explore the ability of the Bayesian model comparison to adjudicate between one versus two dependencies. Both the power analysis and the model recovery simulations provide information about the former, but only the model recovery provides information about the latter.

Statistical significance in studies of serial dependence is usually reported based on a permutation test for the estimated amplitude of a derivative of a Gaussian function, collapsing across participants. However, the statistical tests reported in this study were not based on permutation tests, but instead Bayesian methods. As a rough approximation, we conducted an analytic power calculation based on a mixed-effects linear model. Estimates of the amplitude of the derivative of Gaussian vary, but peak around a few degrees. We used the conservative estimate of 1°. The one study that reported variability in the amplitude across participants estimated a standard deviation of 0.91° (based on four participants; Fischer & Whitney, 2014). Assuming a standard deviation of 1° both across participants and across conditions, the sampling distribution for this average has a standard error of $\sqrt{3/2n}$, where $n$ is the number of participants (Wood, 2017, sec. 2.4). With these approximations, power to detect a single amplitude with an alpha of 0.05 would exceed 0.8 only if there are at least 13 participants.

To assess whether the Bayesian analyses can distinguish between one versus two dependencies, a model recovery simulation was conducted. Datasets for a single participant were simulated using versions of the derivative of circular normal model that had either one or two dependencies, and then model comparison was run on those simulated datasets to determine whether they were better fit by a model that had either one or two dependencies. This simulation was designed specifically to look at whether the true number of dependencies could be isolated from errors, without knowing the magnitude or direction of those dependencies. Hence, we made two simplifications in the models that were used to simulate and fit the simulated data, relative to the models described in Appendix A (i.e., simplified relative to the versions that we used to fit the observed data). First, the simplified derivative of von Mises model excluded biases due to both rotational clockwise/counterclockwise biases, and the periodic errors of Figure A1. Second, the simplified model was non-hierarchical, involving only a single participant’s data. There were two versions of this simplified model. One version, a full model, included two dependencies, whereas the other included only a single dependency.

These two versions of the circular normal models were used to simulate datasets. To simulate a dataset, parameters were first drawn from the prior distribution of the model. Since the priors do not constrain the sign of the amplitude parameters, this meant that a dataset simulated from the full model might include two attractive dependencies, two negative dependencies, or one attractive and one negative dependence. Using the sampled parameters, the responses of a single participant were simulated (per dataset, either 5,000; 10,000; or 15,000 trials). When generating data, the inputs to the circular normal were calculated before calculating the errors. For example, in the reduced model (only one dependence), every trial was first associated with a random orientation value between $\pm \pi$, which was used as the input to a derivative of a circular normal (whose parameters were given by the draw from the prior). The output of the circular normal was then summed with a Gaussian noise term, producing the error for a trial. In the full model, the procedure was the same, except every trial was associated with two independent inputs to two independent circular normals (analogous to the prior orientation and participants’ response to that orientation). Both the (simplified) reduced and full model were then fit to each dataset. The models were compared with PSIS-LOO (Vehtari et al., 2017). This procedure was repeated 100 times for each trial count.

When the data were generated with only a single dependency (i.e., generated by the reduced model), the full model was very rarely selected (Figure B1, right panel), indicating high selectivity. However, the procedure was somewhat asymmetric, with the full model winning in only about 75–95% of datasets generated with two dependencies (Figure B1, left panel). Nevertheless, with at least 10,000 trials, the full model was chosen correctly at least 80% of the time, and the reduced model was correctly chosen less than 20% of the time. When considering datasets generated from dependencies that had the same or opposite signs separately, these patterns held (Figure B2).

These analyses were designed to show two things. First, that the new experiment would provide enough data to make reliable conclusions about serial dependence, and second, that one versus two dependencies were identifiable with the Bayesian analyses. The 20/80% cutoff highlighted in this appendix is arbitrary and adopted here out of convention. In the experiment, sixteen participants contributed usable data. If a frequentist test were performed on estimates of the amplitude of dependencies in those participants, the analytic power calculation suggests that the test would be likely to have enough observations to detect a true effect (given that 80% power was estimated to be achieved with 13 participants). Each participant contributed 1,515 trials, for a total of 24,240 trials. The model recovery shows that, with this amount of data, the Bayesian analysis is likely able to identify one versus two dependencies.

Figure B1.

Results of Model Recovery. One versus two dependencies are often recoverable, given sufficient trials. The plot shows the proportion of times the Bayesian analysis as applied to simulated data chose the full model. The two panels indicate whether the simulated datasets were generated with the full (i.e., two effects) or reduced (i.e., one effect) model. Dots mark the average proportion and error bars encompass the 95% highest density interval of the posterior distribution for the proportion parameter, given a beta prior and binomial likelihood (Jeffreys’ prior). Dashed line marks 0.2 and 0.8.

Figure B2.

Results of Model Recovery, Split by Amplitude of Dependencies in Simulated Datasets. Data plotted as in Figure B1, but results for the full model are shows as groups in which the dependencies had opposite signs (Mixed) or the two positive or two negative amplitudes (Same). The number of datasets within each group is shown at the top of the figure.

Appendix C: Supplementary Analyses Across Experiments

In the main paper, we focused on the two datasets for which model comparison indicated that there was enough constraint to estimate stimulus and response effects, presenting just the posterior distribution from the winning model. In this appendix, we provide further details for the remaining datasets and models that did not win model comparison.

First, consider that each of the three Bayesian models fit the observed data closely, when judged with a single-factor analysis (compare Figures A2, first column and C1, all columns). Figure C1 contains the posterior predictive distributions for the models (i.e., predictions of raw data, including rotational and oblique biases), for all five datasets. As discussed in the main text, this kind of single-factor analysis is unable to show how the estimated effect of, for example, the prior stimulus, does or does not change based on the prior response. Moreover, the effects are small relative to the variability in the data. When plotted this way, the three models exhibit only minor differences in all five datasets, and it is clear that they each capture the major trends in the data.

Next, consider the raw data. In the main text, we suggested that dependencies on the prior stimulus and response could be distinguished when there were enough trials with different combinations of prior response and stimulus. Model comparison suggested that there was enough constraint in only two out of five datasets (show in rows 1 and 4, in Figure C1). As compared to these two datasets, the remaining three exhibited several differences. First, the trial count for Experiment 1 was an order of magnitude higher than all other datasets (Table 1): based on the analyses reported in Appendix B, it was the only one of the five that contained enough trials to consistently distinguish one from two dependence. Second, the dependencies in the Samaha et al. dataset were minimal, as discussed in their manuscript, and as seen in Figure C2 (all four quadrants’ colors are pale and the errors deviate little from zero). Third, the datasets differed in the number of trials and the size of the errors occurring under each possible combination of offsets of the prior stimulus and prior response from the current trial (the four quadrants): the datasets from both Fischer & Whitney (Figure C2 Panel B) and Experiment 1 from Pascucci et al. experiment 1 (see Figure 5 Panel B) contained approximately the same pattern of attraction and repulsion, but an analysis of trial numbers across conditions (not shown) revealed that the dataset from Pascucci et al. Experiment 1 (which provided enough constraint for model comparison) contained approximately twice as many trials in the crossed combination (the off-diagonal quadrants, in which the prior stimulus and response were of opposing signs) than the dataset from Fischer & Whitney (which yielded ambiguous model comparison results). Finally, in Experiment 2 from Pascucci et al. (Figure C2 Panel C) although there were substantial numbers of trials in the crossed combination conditions, the errors in these crossed combinations were minimal (the off-diagonal quadrants are pale in color and associated with small errors).

Third, consider differences in participants’ accuracy across experiments (Figure C3). Differences in accuracy correspond to differences in the proportions of the combinations of prior orientation and response. Errors were largest in Experiment 1 (the current study) and the dataset from Samaha et al., as expected considering that the protocols in each of those tasks were designed to elicit a substantial portion of errors.

Finally, consider the posterior distributions for the dependencies in each of the three models (Figure C4). When considering only an effect of the prior orientation or the prior response (Figure C4, first two columns), most datasets provide enough constraint to unambiguously estimate the sign of the modeled dependence. In agreement with Figures A2 and C2, the sign of that dependence suggests an attraction, for both stimulus and response. However, as discussed in the main text, considering only a single dependence without adjusting for the other is not justified. Indeed, when a model allowed for two dependencies (rightmost column), the sign of those dependencies was ambiguous exactly when model comparison failed to distinguish the models (i.e., for the Samaha, Fischer and Whitney, and Pascucci Experiment 2 datasets).

Figure C1.

Data simulated from each of the three models (first three columns) closely matches the observed data (fourth column, reproduced from Figure A2). All model rows show the 95% highest density interval for the posterior predictive distribution, calculated as in Figure 5. Experiments are separated by rows, and models are separated by columns.

Figure C2.

Datasets that resulted in ambiguous model comparison also show ambiguous dependencies. Data plotted as in Figure 6. Inset numbers in each quadrant indicate how many trials fell into that quadrant. Error bars span 95% confidence intervals.

Figure C3.

Accuracy varied across experiments. Horizontal lines mark the median absolute error. Boxes span the 25^th and 75^th quantiles, while whiskers extend to the largest or smallest value that is 1.5 times the interquartile range. Whiskers not shown for the dataset from Fischer & Whitney, considering that it contained only four participants. Individual dots show individual participants.

Figure C4.

Estimated dependencies across the different models. Experiments are separated by rows, models by column. Each panel show the 95% highest density interval for the modeled dependencies. Two models contained only one dependence (first two columns), and so there is only a single curve. In some datasets, the sign of the dependencies differed across the different models.