Limited interaction between translation and visual motion aftereffects in humans

Abstract

After exposure to a moving sensory stimulus, subsequent perception is often biased in the opposite direction. This phenomenon, known as an aftereffect, has been extensively studied for optic flow stimuli where it is known as the visual motion aftereffect (MAE). Such visual motion can also generate the sensation of self-motion or vection. It has recently been demonstrated that fore-aft translation in darkness also produces an aftereffect. The current study examines the interaction between visual MAE and vestibular translation aftereffects. Human subjects participated in a two-interval experiment in which the first interval (adapter) was visual, translation, or both combined congruently or in conflict. Subjects identified the direction of the second (test) interval of either visual or translation using a forced-choice technique. The translation adapter had no influence on visual test stimulus perception, and the visual adapter did not influence vestibular test stimulus perception in any subjects. However, congruent visual and translation induced a significantly larger perceptual bias on the translation test stimulus than was observed for a translation only adapter. The congruent adapter caused the MAE to be diminished relative to a visual only adapter. Conflicting visual and vestibular adapters produced an aftereffect similar to that seen when the single adapting stimulus was the same modality as the test stimulus. These results suggest that unlike visual and translation stimuli whose combined influence on perception can be predicted based on the effects of each stimulus individually, the effects of combined visual and translation stimuli on aftereffects cannot be predicted from the influences of each stimulus individually.

Introduction

A theme in sensory perception is that exposure to a stimulus makes it seem more neutral such that perception of subsequent stimuli are shifted in the opposite direction. Perhaps, the longest known example of this is the visual motion aftereffect (MAE) in which after viewing a moving image, a static pattern is perceived as moving in the opposite direction. This is also known as the “waterfall illusion” and was first described by Robert Addams more than 175 years ago (Addams 1834) and has now been described for several types of visual stimuli (Thompson and Burr 2009). One type of visual motion which is commonly experienced is the expanding visual pattern caused by moving through a visual environment known as optic flow (Gibson 1950). Vection, an illusion of self-motion, can result from viewing an expanding visual pattern (Johansson 1977; Andersen and Braunstein 1985; Ohmi and Howard 1988). Few studies on the MAE have tried to disambiguate the perception of environmental versus self-motion or vection, although a recent report demonstrated that vection may be separate from the visual MAE (Seno et al. 2010).

When combined with self-motion, the visual MAE can be inhibited. This has been demonstrated when forward self-motion is combined with a congruent visual stimulus (Wallach and Flaherty 1975; Harris et al. 1981). However, the visual MAE was similar to that of a stationary observer when the visual and self-motion stimuli were in conflict (Harris et al. 1981). These findings suggest that the visual MAE may actually be a response to an incongruent visual and vestibular stimulation, which is consistent with the hypothesis that the MAE may be a form of sensory error correction (Anstis et al. 1998).

In addition to visual motion, aftereffects have also been demonstrated in response to whole body translation in darkness. Fore-aft translation in darkness produces a translation aftereffect (TAE) such that after translation, perception of a subsequent movement is modulated in the opposite direction (Crane 2012). A priming effect has also been described for visual motion but is limited to shorter inter-stimulus intervals (Kanai and Verstraten 2005). Much less is known about TAE than MAE, and they should not be assumed to be mechanistically similar.

Given that both visual and translation stimuli produce aftereffects, it raises the question of whether aftereffects from different modalities are combined. Previous data have demonstrated that the MAE can be inhibited by congruent translation (Wallach and Flaherty 1975; Harris et al. 1981). However, these studies only examined the perception of visual stimuli, and they do not address potential effects of multisensory adaptation on vestibular perception. It has also been shown that perception of a stimulus with visual and vestibular components yields a more reliable perception than a unimodal stimulus (Angelaki et al. 2009), and this has been demonstrated for visual-vestibular heading discrimination (Fetsch et al. 2009). It has been shown that adaptation to visual stimuli consisting of two combined directions (i.e., forward expansion with lateral translation) results in a visual MAE that is predicted by a linear combination of the two directions of motion (Mather and Moulden 1980; Verstraten et al. 1994). Linear combinations have also predicted aftereffects from combined visual and auditory stimuli (Sato et al. 2007; Wozny and Shams 2011) but have not yet been tested with regard to combined visual and vestibular stimuli.

The current study examines potential interaction between visual MAE and TAE by studying both aftereffects under similar conditions with analogous stimulus parameters. Although both visual MAE and TAE are demonstrated, no interaction between the two modalities was found with a cross-sensory adapter. However, a combined visual-translation adapter yielded a significantly enhanced aftereffect for translation in darkness. The visual MAE was decreased when tested with a visual-translation adapter in comparison with a visual only adapter.

Methods

Equipment

Motion stimuli were delivered using a 6-degree-of-freedom motion platform (Moog, East Aurora, NY, model 6DOF2000E) as previously described (Crane 2012; Roditi and Crane 2012). Responses were collected using a three-button control box: the center button was pressed by the subject to initiate each stimulus. The two buttons at either end were used to identify the perceived direction of the test stimulus as forward or backward. Subjects were seated upright in a padded racing seat (Corbeau, Sandy UT, model FX-1) mounted on the platform which included high lumbar and seat bolsters. The head was fixed using an appropriately sized American football style helmet which was rigidly fixed to the motion platform with an inflatable liner to prevent decoupling of the head as previously described (Crane 2012). The facemask of the helmet was removed so the visual stimulus was not obscured.

The visual stimulus consisted of a computer generated star field which simulated movement of the observer through a random-dot cloud. The stimuli were presented on a horizontal color LCD screen measuring 115.6 by 64.8 cm with a resolution of 1920 × 1080 pixels (Samsung model LN52B75OU1FXZA). The subject was seated 50 cm from the screen that filled a 98° field of view in azimuth. A fixation point consisted of a 2 × 2 cm midline cross at eye level. Disparity was provided using red–green anaglyph glasses made with Kodak (Rochester, NY) Wratten filters #29 (dark red) and #61 (deep green). The colors were adjusted such that the two intensities were similar when viewed through the respective filters, and the brightness of the visual objects when viewed through one filter relative to the other was better than tenfold.

The platform motion was synchronized with the visual stimulus. The custom software controlling the platform and visual stimulus (moogdots) had a variable to control the delay between the two stimuli. Prior to performing any experiments, the synchronization was verified by having the visual display simulate an object fixed in space. An earth-fixed laser was then projected onto the virtual earth-fixed object. The delay variable was adjusted using a relatively high-frequency sway stimulus (2 Hz) such that the virtual object and laser remained aligned. The calibration was subsequently tested using a variety of movements and frequencies (0.1–2 Hz) to verify the calibration was accurate.

During both visual and vestibular test stimuli, an audible white noise was produced from two platform-mounted speakers on either side of the subject to mask the sound of the motion platform’s motors as previously described (Roditi and Crane 2012). Although no masking noise was needed for the visual condition, it was still used for consistency.

Stimulus

The visual and vestibular motion was fore-aft with a profile consisting of a sine wave in acceleration which contained no discontinuities in acceleration, velocity, or position, and they have previously been described and used for threshold determination (Benson et al. 1986; Soyka et al. 2011; Roditi and Crane 2012; Valko et al. 2012). The vestibular stimulus included a small amount of sway with a maximum amplitude of 0.6 mm to minimize use of noise and vibration to determine the size of the vestibular stimulus (Crane 2012).

Visual stimuli were presented on a color LCD screen measuring 115.6 by 64.8 cm with a resolution of 1920 × 1080 pixels (Samsung model LN52B75OU1FXZA) and a refresh rate of 60 Hz. The subject was seated 50 cm from the screen that filled a 98° horizontal field of view. The visual stimulus consisted of a star field which simulated movement of the observer through a random-dot cloud. Each star consisted of a triangle 0.5 cm in height and width at the plane of the screen, adjusted appropriately for distance. The star density was 0.01 per cubic cm. The depth of the field was 130 cm.

In one-interval experiments, the stimulus was 0.5 s (2 Hz). In the two-interval experiments, the first-interval (adapter) stimulus was 1.5 s (0.66 Hz) during which the platform moved or visually simulated 15 cm of motion. The first interval was followed by a 0.5 inter-stimulus interval (ISI) during which the fixation point was visible but there was no visual or platform motion. After the ISI, the second (test) stimulus was delivered, and this stimulus lasted 0.5 s and had a maximum of 5 cm of motion which was adjusted based on the subject’s previous responses. These stimulus parameters were carefully considered. Although a longer, higher velocity adapting stimuli would likely yield a stronger aftereffect, the platform movement had a limited range of motion. Subject comfort as well as the duration of the trial blocks had to be considered. The minimum ISI of 0.5 s was used because we wanted to insure there was no residual vibration from the adapting stimulus and that the subject was able to distinguish the adapter and test stimuli. The current stimulus parameters were based on the results of some preliminary studies (data not shown) and are the same as those used in the previous study describing TAE (Crane 2012).

Experimental procedure

Prior to stimulus delivery, the subject heard a 500 Hz, 0.125 s single tone to signal that the next stimulus was ready. The stimulus was delivered immediately after the subject pressed the start button. After the stimulus was delivered, two 0.125 s tones were played in rapid succession to indicate that the stimulus had been delivered and prompt the subject to press one of two response buttons. The subject then pushed one of two buttons to indicate forward or backward motion was perceived. Thus, this was a two-interval forced-choice task. Tones were played from speakers mounted to the motion platform to eliminate any potential auditory localization cues. When a response button was pressed, a key click sound was played which did not depend on the accuracy of the response, but indicated that the subject’s selection had been recognized by the program. If no response was entered within 2 s, a “timeout sound” was played (a low frequency buzz). This amount of time was almost always sufficient to report a response. A time-out was used was to allow subjects the opportunity not to enter a response if there was a lapse of attention. Subjects were instructed to enter a response based on their perception but were encouraged to choose a best guess in uncertain situations.

The staircase used in all experiments shifted the next stimulus toward a more backward stimulus for each forward response and vise versa (i.e., one up, one down). The step size started at 16 cm/s and decreased by half with each reversal to a minimum of 4 cm/s. This staircase tended to converge at the point of subjective equality (PSE).

A single-interval vestibular stimulus with 50 trials was conducted to measure potential bias. There was also a two-interval vestibular stimulus where the first interval was an adapter which consisted of randomly interleaved forward or backwards motion (100 trials). These experiments were essentially the same as those previously reported as a measure of fore-aft vestibular aftereffects (Crane 2012) with the exception that instead of being in complete darkness, a fixation point was visible for the present study.

Care was taken to make the visual motion stimulus as similar to the platform motion as possible. Visual MAE is frequently measured by modifying the coherence (essentially the amount of noise) in the test stimulus (Hiris and Blake 1992). Although visual MAE does occur with 100 % coherence or static test stimuli, these are low-level MAE that likely occurs early in the visual processing pathway (Moulden 1980). Including dynamic elements and binocular disparity in the visual test stimulus by decreasing the coherence was done to make the resulting MAE more likely to occur in higher levels of the visual pathway (Nishida and Sato 1995; Mather et al. 2008) such as MST (Culham et al. 2000; Ashida et al.2007) where both visual and vestibular stimuli are represented (Britten and van Wezel 1998; Page and Duffy 2003; Gu et al.2010). Coherence does not have an analogous parameter for a vestibular stimulus. To make the visual stimulus as analogous to the vestibular stimulus as possible, the displacement of the visual motion rather than coherence was varied with the staircase. It was necessary to have a visual test stimulus with coherence degraded so that it was ambiguous for small movements, but with largest motion (5 cm), the direction could be unambiguously identified (i.e., <5 % errors). We found during preliminary experiments that the coherence required to reliability identify the direction of the visual stimulus varied significantly between subjects. The coherence used for the test stimulus in each subject was determined based on performance of a single-interval task in which coherence was modified but displacement remained constant at 5 cm. The coherence for future visual test stimuli in that subject was determined based on the lowest coherence where the direction of the stimulus could be reliably determined in both directions. This coherence used for each subject varied from 30 to 60 % (mean ± SD, 46 ± 10). A single-interval visual experiment was then conducted during which the previously determined coherence was fixed and the amount of apparent motion was varied. This verified that all subjects could identify the maximal motion (5 cm) in each direction but had some uncertainty with smaller movements.

The remaining experiments included two stimulus intervals: an adapter (first interval) and a test stimulus (second interval). Within a block of trials, the adapting stimulus was either visual, vestibular, or combined (visual and vestibular). The test stimulus was always either vestibular or visual with the modality of the test stimulus also fixed within each block. In all blocks, forward and backward adapting stimuli were randomly interleaved such that within a block, 50 % of stimuli had a forward adapter and 50 % had a backward adapter. After an inter-stimulus interval (ISI) in which no motion occurred, a test stimulus was delivered. For each adapter, there were two staircases: one that started with 5-cm forward motion and the other that started with 5-cm backward motion. This was done so that the direction of the adapting stimulus could not be used to predict the direction of the test stimulus. Thus, each of the blocks with a unisensory adapter (visual or vestibular) included 4 randomly interleaved staircases each of which included 25 stimulus presentations as previous used for TAE (Crane 2012). Thus, these aftereffect blocks included 100 stimulus presentations. Blocks of trials were also conducted with multisensory, both visual and vestibular, adapters. In these trials, potential effects of the visual and vestibular stimuli being either congruent (in the same direction) or in conflict (the difference in direction was 180°) were also interleaved within these blocks increasing the number of stimulus presentations in the block to 200. The order blocks that were completed was varied between subjects.

Prior to starting the experiment, subjects were shown some test stimuli in which visual and vestibular stimuli were congruent with the visual stimulus 100 % coherent. This allowed them to experience the platform motion and understand the relationship between platform motion and visual motion so they would report the visual motion direction as if it represented self-motion through a fixed environment.

Subjects

Informed written consent was obtained from all participants. The protocol including the written consent form used was approved by the University of Rochester Research Science Review Board and conducted according to the principles expressed in the Declaration of Helsinki.

A total of 11 subjects (6 female) participated. Three subjects did not complete one of the test conditions due to lack of availability (in one subject, it was the multisensory adapter with the visual test stimulus and in the other 2, it was the multisensory adapter with a vestibular test stimulus). All the subjects completed the unisensory conditions. Multiple testing sessions on different days were required to complete the protocol since each session was limited to no more than 90 min to decrease subject fatigue. Ages ranged from 21 to 69 (mean ± SD, 39 ± 19). Two subjects (#4 and #5) were familiar with the design and purpose of the experiment. The responses in these subjects were similar to those of the naïve subjects, and it was thought that knowledge of the experimental design did not influence perception. The other subjects had participated in previous experiments in the laboratory using the motion platform but were otherwise naïve to the design and purpose of the experiment. Subjects understood that the adapter stimulus could be followed by a test stimulus that was either forward or backward motion.

Subjects were screened prior to participation. The screening included caloric testing, an audiogram, visual acuity testing, and screening questions to rule out any known history of vestibular disease or cognitive deficit. Based on these results, the subjects had normal peripheral vestibular function and hearing.

Analysis

The percentage of forward responses for each stimulus level were plotted as a function of the stimulus used. A cumulative Gaussian function with confidence intervals was determined from those data points using a Monte Carlo (MC) maximum likelihood criteria allowing for a small lapse rate (0.00–0.05) which was fit to the data set as previously described (Wichmann and Hill 2001a, b), as used by others (Fetsch et al. 2009; MacNeilage et al. 2010) and in the current laboratory (Crane 2012). Data from each subject were resampled from the original data. The resampled data had the same number of data points, but due to the random resampling, some data points were included multiple times and others not at all. A psychometric function was then fit to the resampled responses. This resampling and curve fitting were repeated 2,000 times on each trial block so that multiple estimates of the mean (point of subjective equality or PSE) as well as the sigma could be generated and 95 % confidence intervals determined.

The potential influence of an adapting stimulus was determined by comparing the distribution of means determined using the MC method. In aftereffect blocks, the mean was calculated 2,000 times using random resampling of the data (Wichmann and Hill 2001) for each adapting stimulus (Fig. 1). The mean from each of calculation in one test condition was compared with the mean in the other condition, and the number of times one distribution had a larger mean than the other was tabulated. If a test condition was compared with itself using this method, one would expect the mean to be larger in one set of resampling and curve fitting 50 % of the time. A p value was calculated using this method by taking twice the fraction of the fits in the distribution with the smaller average mean that is greater than the fraction of the fits in the distribution with the larger mean. Thus, if a block of trials is compared with itself, p will be about 1. If the mean of the smaller distribution is larger in 1 of 2,000 instances, then p = 0.001. If it is larger in 50 of 2,000 instances, then p = 0.05. Another way to think about this method is that it essentially quantifies the amount of overlap between two distributions. In cases in which there was no overlap between these two distributions, the p value is given as <0.001. For this test, a p value of p = 0.01 was considered significant.

Fig. 1.

Measurement of an aftereffect in an example subject. This example is for a visual adapter (100 % coherence) and visual test stimulus (45 % coherence). A circle is shown for each stimulus level presented with the number of stimuli proportional to the size of the circle. The smallest circles represent a single stimulus presentation, and the largest circles represent 8 stimulus presentations. In this example, both intervals were visual motion and the platform remained stationary. A solid line represents the forward adapter, and the dashed line represents a backward adapter. The cumulative distribution function that best approximates the data is shown as a black line. The data were randomly resampled and fit ×2,000. The bottom panel represents histograms of the biases determined from the resulting fits. In these plots, the scale of the x axis was compressed for clarity. In this example, there was no overlap in the histograms (p < 0.0005) indicating a highly significant difference

The possibility that aftereffects of multimodal (visual and vestibular) adapting stimuli could be predicted from the aftereffects of unimodal adapters was considered. In addition to simply adding the magnitude of the two unimodal aftereffects, a maximum likelihood estimate (MLE) was calculated as previously described (Ernst and Banks 2002) using the thresholds determined from the unimodal aftereffects as the reliability estimates. The relative reliability of the vestibular (w_vest) and visual (w_vis) unimodal conditions was calculated using the sigma (δ) found for each condition (Eq. 1):

$${\mathcal{W}}_{\mathrm{vest}}=\frac{1}{1+{\sigma }_{\mathrm{vest}}^2∕{\sigma }_{\mathrm{vis}}^2}\text{and}{\mathcal{W}}_{\mathrm{vis}}=\frac{1}{1+{\sigma }_{\mathrm{vis}}^2∕{\sigma }_{\mathrm{vest}}^2}$$ (1)

These relative reliability estimates were then used with the mean ($\stackrel{‒}{x}$) of the visual and vestibular conditions to make the MLE prediction (Eq. 2).

$$\mathrm{MLE}={\mathcal{W}}_{\mathrm{vest}}{\stackrel{‒}{x}}_{\mathrm{vest}}+{\mathcal{W}}_{\mathrm{vis}}{\stackrel{‒}{x}}_{\mathrm{vis}}$$ (2)

Results

The experiments were well tolerated without significant motion sickness issues. All subjects were able to reliably identify the stimulus directions at the extremes of the test stimulus used.

Blocks of trials were analyzed for each subject using the curve fitting described in Methods. This allowed determination of the bias from the mean psychometric function with 95 % confidence intervals (CI). The sigma or width of the psychometric function was also determined from these fits as a measure of the threshold (Fig. 1). As a control, the bias was determined for a single-interval visual and vestibular task. In this one-interval experiment, the bias was near zero in all subjects for the vestibular (translation) task (Fig. 2a). For the visual direction task, the mean of the psychometric function was significantly (MC method, p < 0.01) shifted in the negative direction in 6 of 11 subjects, indicating a tendency to perceive small movements as forward motion (Fig. 2b). The mean threshold (sigma) for the vestibular task was 0.78 ± 0.16 cm/s (mean ± SE), which is in the range of previous estimates of direction perception thresholds for translation (Fig. 2c). There was no significant correlation between subject age and the vestibular threshold (Pearson correlation coefficient, R = −0.21, p = 0.53).

Fig. 2.

Single-interval direction perception. Error bars represent 95 % CI. Positive values indicate the point of subjective equality or mean is shifted forward, and negative values indicate a backward shift. The final column in each panel (marked C) represents the mean data. a Single-interval vestibular direction determination. b Singleinterval visual direction determination from an optic flow stimulus with decreased coherence. c The width (sigma) of the cumulative distribution function. The coherence used for visual stimuli through-out the experiment in each subject is given as a number at the base of each bar

Care was taken to make the visual task as similar as possible to the vestibular task by varying the amount of apparent motion and keeping the coherence constant across stimulus presentations. The coherence needed to be degraded for an aftereffect to be perceived using the current protocol, but had to be good enough that the direction could be reliably identified for the largest translation (5 cm, peak velocity 20 cm/s). The range of coherences which met these criteria varied between subjects. The coherence for visual test stimuli was set individually as described in the Methods and ranged between 30 and 60 % (values shown in Fig. 2c). The coherence used within an individual subject remained constant throughout blocks of trials. Although the coherence for the test stimulus was degraded, for the visual adapting stimulus, the coherence was always 100 %. The mean visual threshold was 3.7 ± 1.1 cm/s. The correlation between sigma and the coherence used was poor (Pearson correlation coefficient, R = 0.06) and not significant (p = 0.86). The subject age was also not significantly correlated with the coherence used (^R = 0.50, p = 0.11), or with the visual threshold (R = 0.20, p = 0.55).

A two-interval experiment was conducted to see whether an initial vestibular and/or visual stimulus influenced perception of a second vestibular or visual stimulus. The influence of the adapting stimulus was determined by comparing the means of the psychometric function with that of opposite adapting stimulus direction (Fig. 1).

A vestibular adapter followed by a vestibular test stimulus produced a significant (MC method, p < 0.01) TAE in 3 subjects such that the test stimulus was biased where subsequent perception was likely to be opposite that of the adapting stimulus (Fig. 3a). In one subject, there was a significant (MC method, p < 0.01) priming effect (Fig. 3a), such that the perception of the test stimulus was more likely to be perceived in the same direction as the adapter. This result is similar to that previously published (Crane 2012). Although the sigma was higher after the adapting stimulus (1.9 ± 0.5 cm/s) when compared with that of the single-interval trial, the difference was at the border of significance (two-tail paired t test, p = 0.05, degree of freedom (DOF) = 10).

Fig. 3.

Effect of a unisensory adapter on the test stimulus. Upward pointing open triangles represent a forward adapter, and downward triangles represent a backward adapter. The mean is the stimulus level and the subject is equally likely to identify as forward or backward. Thus, if the mean is shifted in the positive (forward) direction, it indicates a neutral stimulus that is more likely to be perceived as backwards. Error bars are the 95 % CI. Circled subject numbers indicate a significant difference between a forward and backward adapter (p < 0.01). Significant aftereffects (the forward adapter mean is greater than the backward adapter mean) are marked with the subject number circled with a solid line. The significant priming effect (a, subject 8) is shown with a dashed circle to indicate the mean with the backward adapter and is greater than that for the forward adapter. The final column in each panel (marked C) represents the mean data. a Vestibular adapter and vestibular test. b Visual adapter and visual test, also known as a visual MAE. c Visual adapter and a vestibular test; the visual adapter had no significant influence on perception of the vestibular test stimulus. d Vestibular adapter combined with a visual test

A visual motion aftereffect (MAE) was in the same direction and was significant (MC method, p < 0.01) in 7 of the 11 subjects, with the remaining subjects demonstrating a shift in the PSE in the aftereffect direction (Fig. 3b). The visual stimulus was designed to be analogous to the vestibular stimulus and not to produce the largest possible MAE. No subject exhibited a priming effect with the visual adapter and test stimulus. The sigma was significantly greater than in the single-interval experiment at 10 ± 2 cm/s vs 4 ± 1 cm/s (paired t test, p = 0.004, DOF = 10).

A major goal of this work was to determine whether a visual adapter could influence vestibular perception and vice versa. In the current protocol, a visual adapter did not significantly bias vestibular perception in any subject (Fig. 3c). Similarly, a vestibular adapter did not produce a bias in visual perception in any subject (Fig. 3d). The sigma of the vestibular test stimulus after a visual adapter was similar to that of the single-interval translation test at 0.72 ± 0.14 cm/s (paired t test, p = 0.78, DOF = 10). Likewise, the sigma of the visual test stimulus after a vestibular adapter was similar to that of the single-interval visual test stimulus at 3.7 ± 0.8 cm/s (paired t test, p = 0.99, DOF = 10).

The possibility that a multisensory (visual + translation) adapting stimulus could alter perception of a test stimulus in a way different from a single sensory adapting stimulus was considered in 2 blocks of trials. In both blocks, the adapting stimuli consisted of a forward or backward translation stimulus combined with forward or backward visual stimulus. Thus, the adapting stimuli could be either congruent or in conflict depending on the relative directions of the platform and visual motion. The test stimulus in one block was vestibular, and in the other, it was visual.

Use of a congruent adapter stimulus and a vestibular test stimulus produced a significant (MC method, p < 0.01) priming effect in 2 subjects, an aftereffect in 6 subjects, and no significant effect in one subject (Fig. 4a). The congruent adapter had a larger effect on biasing the test stimulus than a vestibular adapter alone (paired t test, p = 0.001, DOF= 8); thus, a congruent adapter enhanced the vestibular aftereffect and vestibular priming. The conflict adapter produced fewer significant differences (MC method, Fig. 4c) and was not significantly different from the vestibular only adapter (paired t test, p = 0.6, DOF = 8).

Fig. 4.

Effect of a multisensory adapter on the test stimulus. Upward pointing arrows represent the portion of the adapting stimulus matching the test stimulus was forward. Thus, for the conflict-vestibular condition, upward pointing arrows represent forward translation and backward optic flow, but for the conflict-visual condition, they represent forward optic flow and backward translation. As with Fig. 3, error bars represent the 95 % CI, circled subject numbers indicate a significant difference for that subject between forward and backward adapter conditions, and the final column (C) represents mean data

When the visual test stimulus was combined with the congruent adapter 4/10 subjects had a significant aftereffect (Fig. 4b), this was fewer than had a significant effect with a visual only adapter. The average aftereffect size (mean of forward minus backward adapter) was 11.9 ± 2.3 cm/s (mean ± SE) with a single modality adapter, and 7.1 ±1.6 with a congruent adapter, a difference which was not significant (paired t test, p = 0.2, DOF = 9). Use of a conflict adapter produced an aftereffect in 5/10 subjects which was on average smaller than the visual only adapter condition at 8.8 ± 2.9 cm/s and also not significantly different (paired t test, p = 0.5, DOF = 9).

The data were combined across subjects (Fig. 5) which demonstrated a relatively large visual MAE (Fig. 5a) with a visual adapter which was not present when a vestibular adapting stimulus was used. The TAE was small when averaged across subjects (Fig. 5b), but larger when combined with a visual and vestibular adapting stimulus. In examination of the mean data, it becomes clear that most of the aftereffect with both visual and vestibular conditions occurred with a backward adapter stimulus and relatively little occurred with a forward stimulus (Fig. 5a, b).

Fig. 5.

Mean across subjects by test condition. Error bars represent ±1 SEM and for clarity are only shown in one direction for conditions where an adapting stimulus was used. a The mean of the psychometric function using a visual test stimulus. b The mean of the psychometric function using a vestibular test stimulus. c The average threshold (sigma or width of the psychometric function) for visual and vestibular test stimuli. d The average threshold for vestibular test stimuli

The threshold (sigma) of the visual stimulus was larger (Fig. 5c) than the threshold for vestibular motion (Fig. 5d) which may have been related to the degradation in the visual stimulus (decreased coherence). The coherence was intentionally decreased on the visual test stimuli so that the MAE could be demonstrated using conditions analogous to those used to demonstrate a TAE. For both the visual MAE and TAE, the thresholds were lowest for conditions that did not produce an aftereffect (single-interval tests, and unimodal cross-sensory adapters).

The potential for the visual MAE and TAE using a bimodal (visual + vestibular) adapting stimulus to be predicted by the responses to a unimodal stimulus was considered (Fig. 6). For analysis of the bimodal adapter, only the unimodal data from the subjects that completed the corresponding bimodal condition were included. Simple addition of the two unimodal conditions predicted an aftereffect similar to the condition when a single adapting stimulus was combined with a similar test stimulus due to the aftereffect of the cross-modal adaptation being minimal. However, the addition model did predict the TAE (Fig. 6b) with a bimodal stimulus although it overestimated the visual MAE (Fig. 6a). A maximum likelihood estimate (MLE) was calculated by weighing each unimodal adapter by the sigma (Fig. 5c, d) as previously described (Ernst and Banks 2002). Because the sigma was lower in conditions which did not produce an aftereffect, the MLE greatly underestimated the TAE and visual MAE to bimodal adapters (Fig. 6).

Fig. 6.

Mean aftereffect by condition. For conditions in which a combined (visual + vestibular) adapter was used (i.e., conflict and congruent), the predicted response from addition of the visual and vestibular adapter alone is also shown (diamonds), as well as the response predicted using a maximum likelihood estimate (MLE) which weighs the visual and vestibular thresholds based on the observed sigma. Error bars represent ±1 SEM. a Visual MAE (i.e., visual test stimulus). b Translation aftereffect (vestibular test stimulus)

Discussion

An isolated optic flow stimulus is ambiguous because it could represent environmental motion, self-motion through a fixed environment, or an expanding object fixed in space. Use of other sensory cues help disambiguate these possibilities and mold perception. Thus, when an optic flow stimulus is combined with congruent self–motion, the optic flow stimulus is more likely to be perceived as motion through a fixed environment. It has been shown in the visual psychophysics literature that visual MAE is inhibited when combined with congruent self-motion which explains why they are not usually perceived during daily activities (Thompson and Burr 2009). If the MAE is a response to conflicting sensory cues, as is the case when visual motion presented when stationary, then removing the conflict with congruent translation and visual motion would predict a decrease in the MAE (Harris et al. 1981).

In addition to anecdotal reports of the MAE being inhibited in self-motion situations such as driving a car, the inhibition of MAE when combined with congruent self-motion is grounded in two classic studies, the first involved having the subject rock back and forth while exposed to an optic flow stimulus only during one direction of self-motion (Wallach and Flaherty 1975). That paper found that 12 of 32 subjects reported a visual MAE when the forward phase of motion was combined with an expanding visual stimulus. However, 31 of 32 subjects reported the MAE when the expanding stimulus was presented, while the subject was stationary. The other study also involved alternating fore-aft translation with the visual stimulus presented only in one direction (Harris et al. 1981). In that study, the MAE was quantified using a cancelation method and decreased by about 75 % when an expanding visual stimulus was in phase with forward motion. Both of these studies used visual adapters in only one direction in each block of trials and found the inhibition of the MAE was limited to an expanding stimulus presented during the forward phase of motion and was not observed with a contracting stimulus nor a contracting or expanding stimulus during backward movement. Both studies were also potentially confounded because the subjects experienced both directions of translation but only one direction of optic flow.

Another important difference between the previous work on the effect of translation on the visual MAE (Wallach and Flaherty 1975; Harris et al. 1981) and the current experiments is the type of test stimulus used. Wallach and Flaherty used a static pattern with the subject asked to report if an aftereffect was perceived. Harris et al. used a stimulus with the potential for movement using a nulling task but at full coherence. Thus, neither of these previously studies decreased the coherence of the visual test stimulus as was done in the current experiments. The MAE measured with low-level MAE is considered low level because they likely occur early in the visual pathway (i.e., the retina or V1) based on previous evidence such as position specificity (Anstis and Gregory 1965) and poor interocular transfer (Wade et al. 1993). Including dynamic elements in the visual test stimulus as can be done by decreasing the coherence which leads to the MAE occurring at a higher level (Nishida and Sato 1995) in the visual pathway where vestibular stimuli might also be considered. Unlike low-level MAE, these high-level or second-order MAE is known to occur at least in part in area MST (Culham et al. 2000; Ashida et al. 2007) where both visual and vestibular stimuli are represented (Britten and van Wezel 1998; Page and Duffy 2003; Gu et al. 2010). Thus, it was felt decreasing the coherence of the test stimulus and including binocular disparity to create second-order MAE that might be more likely to demonstrate an interaction with visual stimuli.

In the current study, there was a large amount of variation between individual subjects. The visual MAE was relatively reliably observed with 7 of the 11 subjects demonstrating a significant response (Fig. 3b) and the remaining subjects demonstrating a shift in the PSE in the direction predicted by a MAE. The TAE had much more variation with a large effect seen in only 3 of 11 subjects (Fig. 3a) and a large effect in the opposite direction seen in one subject. The TAE would likely have been more universally observed if a longer ISI, such as 1 s, was used as the 0.5 s ISI used here is known to yield a TAE in some subjects and a priming effect in others (Crane 2012), although even this previous study demonstrated large intra-individual variation. The reason for the short ISI in the current series was to maximize possible influences of visual stimuli, as well as to examine the possible influence of visual stimuli on the vestibular priming effect as well as the TAE. Previous studies which examined the potential interaction between the TAE and MAE also found significant intra-individual variation with 12 of 32 subjects demonstrating a MAE when congruent motion was combined with a visual stimulus (Wallach and Flaherty 1975), a similar ratio to the 4 of 11 that demonstrated an effect in the current series (Fig. 4b). Thus, there seems to be a large amount of intra-individual variation inherent in both the TAE and the MAE–TAE interactions. The source of this variation is not clear and was not correlated with age or gender. This may be related to the individual experience which has previously been cited as a reason for suppression of the MAE when combined with forward translation more than with backward translation (Harris et al. 1981). Some of the between-subject variability may be related to cue validity or the degree of attention the subject is giving to different aspects of the stimulus (Shimozaki et al. 2012).

The perception of visual motion and self-motion is not independent as it has long been recognized that exposure to optic flow can produce a sensation of self-motion or vection (Fischer and Kornmuller 1930). Vection appears only after the optic flow stimulus has been present for a several seconds (Brandt et al. 1973; Bubka et al. 2008), but the latency is much less when combined with a vestibular stimulus (Melcher and Henn 1981; Wong and Frost 1981), and it can be attenuated when combined with conflicting self-motion (Teixeira and Lackner 1979). Vection also produces an aftereffect (Brandt et al. 1974) which is differentiated from the visual MAE by its longer duration (Seno et al. 2010). Quantification of the vection aftereffect in these studies has been limited because it was not compared with actual self-motion.

It has recently been reported that fore-aft translation in darkness also produces an aftereffect (Crane 2012). Given this finding, the interpretation of previous studies on self-motion and MAE (Wallach and Flaherty 1975) becomes more difficult. Because in these experiments, forward and backward motion was alternated, one might expect that the aftereffect of backward motion would occur during the forward phase of motion and vice versa; thus, the perception of the self-motion could be exaggerated relative to the visual expansion. Meanwhile, the visual expansion always occurred in the same direction; thus, the subject likely became desensitized to it over time (Goldstein 1957; Carlson 1962). Furthermore, for short durations (0.5 s or less) after translation, some subjects experience a priming effect (Crane 2012) during which the perception is shifted toward continued motion in the same direction which if combined to a visual MAE may cause some cancelation. Thus, in the previous studies, it is unclear how the translation aftereffect may have interacted with the visual MAE.

The prior literature establishes that a vection aftereffect can be differentiated from the visual MAE (Seno et al.2010), the existence of fore-aft translation aftereffects in the absence of visual stimulation (Crane 2012), and that visual MAE are influenced by fore-aft translation (Wallach and Flaherty 1975; Harris et al. 1981). However, the nature of the interaction between these has not been well defined. The current experiments demonstrated that a visual optic flow stimulus does not influence perception of subsequent fore-aft translation and that fore-aft translation does not influence the perception of visual stimuli. This was an unexpected finding given that in previous reports forward translation while viewing an expanding visual stimulus decreased the visual MAE.

The use of an adapting stimulus consisting of congruent visual and vestibular motion with a visual test stimulus (Fig. 4b) most closely approximated the previous experiments demonstrating MAE inhibited with congruent vestibular motion was present (Wallach and Flaherty 1975; Harris et al. 1981). In the current experiment, fewer subjects had a significant MAE under these conditions when compared with a purely visual adapter, and the overall size of the MAE was smaller. Thus, the current results are consistent with the previously published findings although it was a small difference that did not reach significance in the current paper. This difference could be attributed to methodological differences including the much longer period of adaptation (minutes) used in the prior studies examining the relationship between visual and vestibular motion.

The more striking effect of a multisensory adapter was that a congruent adapting stimulus influenced perception of subsequent translation such that both aftereffects and priming were significantly enhanced relative to a translation adapter alone (Figs. 4a, 5b).

It is difficult to predict either the visual MAE or TAE in response to a bimodal adapter based on the unimodal aftereffect. A simple additive interaction between TAE and visual MAE predicts a response similar to the unimodal aftereffect because a visual adapter had minimal influence on a vestibular test stimulus and vice versa (Fig. 6). Attempts at using a Bayesian MLE to predict the aftereffect of a bimodal adapter also failed because the sigma was much lower for conditions that did not yield an aftereffect. Thus, the MLE predicted a minimal aftereffect to a bimodal adapting stimulus, and a sigma smaller than either aftereffect with a unimodal adapter but neither of these predictions was observed. The reason for this may be partly because the relative weights used (sigma) did not reflect the relative reliabilities, but because a bimodal adapter produced an enhancement of the TAE and a decrease in the visual MAE, no single weight could be applied to explain both these results.

Perception of translation itself is a multisensory phenomenon. The adapting translation stimulus had a peak velocity of 20 cm/s and peak acceleration of 42 cm/s/s which could likely be sensed by proprioception in addition to vestibular cues (Walsh 1961; Gianna et al. 1995). Since the sensory experience of translation in darkness was multimodal, the addition of optic flow likely strengthened the perception of translation, thus also strengthening the aftereffect and priming effect. No such strengthening occurred when the visual adapter was in conflict which supports this theory. One possible explanation is that the TAE is not a purely vestibular phenomenon but may occur at a higher level after visual, vestibular, and proprioceptive cues are integrated. In this situation, more congruent cues could lead to a stronger perception of self-motion and a stronger TAE. With translation, aftereffect perception may not be a result of recalibration to conflicting stimuli but rather a response to translation itself as perceived through multisensory cues.

The visual MAE was weakened when combined with congruent translation which could be explained by the MAE being caused by recalibration to conflicting sensory stimuli. When there is visual-vestibular sensory mismatch (i.e., optic flow without translation), this perceived mismatch may be minimized over time causing the aftereffect after resolution of the optic flow stimulus (Wallach and Flaherty 1975; Thompson and Burr 2009). If there is no mismatch such as optic flow being combined with a congruent translation, the compensation process is not necessary and the aftereffect is smaller.

The current results do not directly address the potential dichotomy between visual MAE and the recently described vection aftereffect (Seno et al. 2010). The visual adapting stimulus was only 1.5 s which was likely too short to induce a sensation of vection (Brandt et al. 1973; Bubka et al. 2008), and the interleaved forward and backward visual adapters likely further inhibited vection perception.

The current results indicate that there is no direct interaction between visual MAE and TAE. This finding is perhaps not surprising given that prior studies on visual MAE have demonstrated that the MAE is most robust when the test stimulus closely resembles the adapting stimulus (Cameron et al. 1992; Bex et al.1996). Despite the attempts in this study to make the visual and translation stimuli as analogous as possible by using the same duration, peak velocity, and acceleration, they do not have a common aftereffect. More interesting is that an adapter with both visual and vestibular components inhibits the visual MAE but enhances the TAE. Although perception of combined visual and vestibular headings can be predicted from perception of single sensory components of these stimuli (Gu et al. 2008; Fetsch et al. 2009, 2012), such predictions do not hold true for aftereffects. This may represent a fundamentally different mechanism for the visual MAE and TAE: The visual MAE may be a response sensory mismatch in visual and vestibular stimuli suggesting a role in differentiation of self versus external motion, while the TAE may be the result of perceived translation which may have multisensory cues.