On interference effects in concurrent perception and action

Abstract

Recent studies have reported repulsion effects between the perception of visual motion and the concurrent production of hand movements. Two models, based on the notions of common coding and internal forward modeling, have been proposed to account for these phenomena. They predict that the size of the effects in perception and action should be monotonically related and vary with the amount of similarity between what is produced and perceived. These predictions were tested in four experiments in which participants were asked to make hand movements in certain directions while simultaneously encoding the direction of an independent stimulus motion. As expected, perceived directions were repelled by produced directions, and produced directions were repelled by perceived directions. However, contrary to the models, the size of the effects in perception and action did not covary, nor did they depend (as predicted) on the amount of perception–action similarity. We propose that such interactions are mediated by the activation of categorical representations.

Introduction

Earlier studies on concurrent perception and action have mainly focused on the structural or central capacity limitations that lead to dual-task decrements in performance (e.g., De Jong, 1993; Johnston & McCann, 2006; Levy, Pashler, & Boer, 2006; Pashler, 1994; Schweickert, 1983; see also Hazeltine, Ruthruff, & Remington, 2006). In more recent years, however, it has become increasingly clear that not only the timing of two concurrent tasks but also their “content” determines the pattern of interference that arises (e.g., Hamilton, Wolpert, & Frith, 2004; Hommel, 1998; Hommel, Müsseler, Aschersleben, & Prinz, 2001; Lindemann, Stenneken, Schie, & Bekkering, 2006; Müsseler & Hommel, 1997; Schubö, Aschersleben, & Prinz, 2001; Stevanovski, Oriet, & Jolicoeur, 2006; Zwickel, Grosjean, & Prinz, 2007, 2008). Müsseler and Hommel (1997), for example, showed that the ability to identify a masked arrow during the preparation of a left or right button press depends on the spatial relationship between the arrow and the button location. Arrows pointing in the direction of the prepared movement led to a decrement in perceptual performance compared to when button and arrow directions differed. These kinds of interference effects have been termed specific (Müsseler, 1999), to distinguish them from the unspecific effects that are typically observed under dual-task conditions (e.g., the psychological refractory period effect; Pashler, 1994).

Although specific interference effects have become the focus of an ever increasing number of investigations (e.g., Hamilton et al., 2004; Lindemann et al., 2006; Müsseler & Hommel, 1997; Schubö et al., 2001; Stevanovski et al., 2006; Zwickel et al., 2007), they are always reported as arising either in perception or action. To the best of our knowledge, no study has obtained effects of perception on action and action on perception in the same experiment. It has therefore been difficult to determine whether such effects might tradeoff between these two “sides” of performance and whether they actually share a common causal origin. These questions are particularly interesting in light of two recent models that have been proposed to account for such interference effects (Hamilton et al., 2004; Schubö et al., 2001). Although the models differ in scope, they both predict that the size of the interference effects in perception should vary with the amount of similarity between what is simultaneously produced and perceived. One of the models also predicts that interference effects in perception and action should co-occur and their effect sizes should covary.

The aim of this study was to further our understanding of how perception and action specifically interact by testing these model predictions. To do so, we developed a paradigm in which we could measure specific interference effects in perception and action in the same experiment. We then systematically varied the similarity between the events that needed to be simultaneously perceived and produced. To anticipate, both models fail to account for the data of the experiments and we offer a modification to the models in the “General discussion”. Before discussing the predictions and the paradigm, we employed in more detail, we introduce the findings and models of Schubö et al., (2001) and Hamilton et al., (2004).

The Schubö model

Schubö et al. (2001) asked participants to produce (without visual feedback) sinusoidal hand movements while observing independent sinusoidal stimulus motions, both of which could vary in amplitude. The results revealed a form of contrast effect (CE): Perceived amplitudes were repelled by produced amplitudes. For example, medium-amplitude motions were perceived as smaller during the production of large as compared to small movement amplitudes. Schubö et al. (2001) also found that produced amplitudes were repelled by the perceived amplitudes, thereby establishing the mutual (bidirectional) nature of their effects. However, in order to avoid sequential effects in their paradigm, different balancing schemes (experiments) were required to measure the two types of effects. As a consequence, no influence of action on perception and of perception on action could be detected within the same experiment.

Schubö et al. (2001) advanced a model that was rooted in the framework of common coding (Prinz, 1990, 1997) and inspired from models of reaching in the presence of stationary visual distractors (Tipper, Howard, & Jackson, 1997). According to the model, a common set of representational elements code for perceived and produced amplitudes. Furthermore, motion and movement amplitudes were assumed to be coded in a distributed and graded fashion. That is, elements corresponding to the perceived/produced amplitudes are activated most and the amount of activation levels off for surrounding elements as one moves away from the relevant amplitudes. Because of this graded activation, some elements of the motion and movement activation curves may overlap. The critical assumption of the model is that these overlapping elements become inhibited in order to minimize the amount of interference between perception and action. This, however, causes the means of the activation curves to shift away from each other, thereby leading to the repulsion effects that were observed (Schubö et al., 2001).

The Hamilton model

In the study of Hamilton et al. (2004), participants watched video clips in which actors lifted identically looking boxes that varied in weight. The weight of the boxes was to be judged by the participants. Critically, while watching the videos, participants were asked to lift and hold a light or heavy weight themselves. The results also showed a form of CE, in that holding a heavy weight led to lower weight judgments than holding a light weight. Thus, the perceived weights were repelled by the held weights.

This CE was accounted for by a model grounded in the MOSAIC framework (Haruno, Wolpert, & Kawato, 2001). The model contains multiple modules, each associated with a different weight (left side of Fig. 1). During action observation, each of these modules predicts the movement kinematics that would result if its associated weight were lifted (arrows coming from the left in the figure). These kinematic predictions interact with observed kinematics via a comparison process (depicted in the middle of the figure). As a result, different responsibilities are assigned to the respective modules (reflected by the thickness of the lines from the middle to the right in the figure). The value of a module’s responsibility corresponds to the similarity between its prediction and the observed movement pattern. Thus, high similarity leads to a high responsibility and low similarity to a low responsibility. These graded responsibilities are thought to reflect noise in the system. To arrive at a weight judgment the responsibilities are normalized, multiplied with the weights associated with the modules, and added. The crucial assumption to explain the CE is that hefting a certain weight occupies the corresponding module. As a consequence, the judgment will not rely on this module and this loss of information results in a judgment bias away from the held weight. We now turn to the specific predictions made by this and the Schubö model.

Fig. 1.

Depiction of the Hamilton model (see text for details)

Predictions of the models

Table 1 summarizes the predictions of the models. The Schubö model, due to its symmetric composition, predicts a mutual specific interference effect in action and perception, that is, influences on action and on perception should co-occur (mutual interference prediction). According to this model, the potential conflict that could arise from overlapping features along a given perception-action dimension, such as amplitude (Schubö et al., 2001), weight (Hamilton et al., 2004), or direction (current experiments), is resolved by inhibiting those features. Shunting the overlapping feature elements leads to a loss of these elements in both action and perception coding. In other words, if a CE in action occurs because of the inhibition of overlapping representational elements, this inhibition should cause a CE in perception as well.

Table 1.

Predictions of the Schubö and Hamilton models

	Mutual interference	Monotonic relationship	Amount of overlap
Schubö model	√	√	√
Hamilton model	?	?	√

Predictions of the models are indicated by a “√”, “?” indicates that the model is neutral with respect to this prediction

The Schubö model also predicts that the size of the interference effects (CEs) in perception and action should be monotonically related, that is, they should both increase and decrease together. This follows from the same assumption as above: overlapping features are inhibited and no longer participate in perception and action coding. If the amount of feature overlap is large, many representational elements will be shunt from both action and perception coding, which should result in two large CEs. Conversely, if the amount of overlap is small, only a few elements will be inhibited, thereby leading to smaller CEs. The relationship between the CEs in action and perception, however, does not have to be linear because the size of the CEs depends on the the shape of the two activation curves. Therefore, without making further assumptions, it is only possible to predict a monotonic relationship. Thus, the second prediction was that the size of the interference effect in action should be monotonically related to the size of the interference effect in perception (monotonic relationship prediction). This prediction can be evaluated by calculating, across participants, the monotonic correlation between interference effects in perception and action because people who show a large CE in action should also show a large CE in perception (for a similar logic, see Franz, Gegenfurtner, Bülthoff, & Fahle, 2000). Additionally, the relationship between interference effects in perception and action can be tested by within-participant correlations of the CEs in action and perception across trials.

The Hamilton model, on the other hand, makes no predictions about an influence of perception on action. Therefore, finding a mutual interference effect or a monotonic relationship would not contradict the Hamilton model, but simply lend credit to the Schubö model. As mentioned above, to the best of our knowledge, no study has obtained specific interference effects in action and perception in the same experiment. Even though Schubö et al. (2001) found CEs in both perception and action, these effects were obtained in different experiments.

Similarly, Zwickel et al. (2007) asked participants to produce movements to the right or left while watching a moving dot on a screen. This dot deviated at pseudo-random time points either to the right or left. Participants’ task was to react as fast as possible with a button press to this deviation. Longer reaction times were found when the dot deviated into the direction of the hand movement. This was what was expected if hand movements repelled the stimulus away and therefore back to the horizontal. In Zwickel et al. (2008) participants had to judge a motion direction while performing hand movements. In both studies despite a CE in perception was found it was not possible to assess the influence on action because movements were varied around motions but not the reverse. A CE in action was found by Grosjean, Zwickel, and Prinz (2008) where motions were either above or below the produced movements which allowed assessing the influence on action but prevented measuring an effect in perception. Importantly, none of the studies looked at the influence on action and perception at the same time. Therefore, a new paradigm was necessary to close this experimental gap and test the first two model predictions.

The third prediction is that the size of the CE should increase the more similar action and perception become on the dimension along which the interference occurs (amount of overlap prediction). Both models make this prediction. In the Schubö model, when the motion/movement amplitudes are more similar to each other, the amount of overlap between the two activation curves will increase. This, in turn, will lead to a larger amount of inhibited elements and therefore to larger CEs. The involvement of a similar mechanism in the Hamilton model also leads to the amount of overlap prediction (Fig. 1): When the held weight is more similar to the observed weight, the occupied weight module is closer to the observed weight module. This means that the occupied module will have gained more activation by the observed weight and therefore its loss will lead to a stronger bias of the perceived weight. An absence of an amount of overlap effect would therefore call both models into question.

Although the experiments of Schubö and colleagues (Schubö et al., 2001; Schubö, Prinz, & Aschersleben, 2004) included an amount of overlap manipulation by using different sizes of produced and perceived amplitudes, only the results for the medium-sized amplitudes were reported. Similarly, in the experiment of Hamilton et al. (2004), the held and observed weights differed in similarity, but no statistics were reported as to whether amount of overlap had an influence on the size of the effects. Also our own studies up to now did not include an amount of overlap manipulation. Once again, new data were needed to test the predictions.

Current paradigm

Figure 2 depicts the experimental setup. Participants sat in front of a display and moved a hand-held stylus on a graphics tablet. The tablet and the participants’ hands were covered from view by a wooden board. On half of the trials, a moving stimulus (dot) was presented on the display while they performed their movements. The task of the participants was to move as accurately as possible while trying to encode the direction of stimulus motion for later report.

Fig. 2.

The experimental setup used in the current study (see text for details)

Example trial sequences for a motion trial (Fig. 3a) and a no-motion trial (Fig. 3b) are presented in Fig. 3. Participants were asked to move their hand in a given direction (−25° in this example) and to terminate their movements between 500 and 1,000 ms. Movement direction was instructed during a training phase where visual feedback of required and produced directions were available (“Methods” for details). In motion trials, the onset of the hand movement triggered a reference stimulus (RS) motion (of 0° in this example), which lasted for 500 ms. The stimulus moved without leaving a trace on the display. After 500 ms (i.e., at RS motion offset) the screen was cleared and 500 ms later (i.e., 1,000 ms after movement onset), a rotating test stimulus (TS) appeared. The TS consisted of a dotted line that rotated back and forth around the direction of the RS motion. Participants were asked to stop the rotating TS with a key press when they thought it matched the direction of the previously seen RS motion. In no-motion trials, the display was cleared at movement onset and remained blank until the participants pressed a key after terminating their movement.

Fig. 3.

Example trial sequences for a motion trial (a) and a no-motion trial (b). In no motion trials no reference or test stimuli were shown indicated by a blank gray frame in the figure (see text for details)

The overall direction of the hand movements and stimulus motions was varied between experiments (see “Stimuli and movements” for details). In the horizontal configuration, the RS moved from left to right and participants were asked to produce upward or downward hand movements (Fig. 4). This configuration was rotated by 90° in the vertical configuration. That is, participants saw RS motions that moved upwards and had to produce movements to left or right. The required movement lengths were 20 cm in the horizontal configuration and 12 cm in the vertical configuration. The specific directions (upward, downward, left, right) of the hand movements were blocked in order to avoid using a cue to signal the required movement direction. This prevented any cue-stimulus (perceptual-perceptual) interactions, which may have confounded the results of earlier studies (e.g., Schubö et al., 2001, 2004).

Fig. 4.

Depiction of the different reference stimulus and hand movement directions, and how the angles were coded for each type of configuration (horizontal, vertical). For the purpose of illustration, two required movement directions are presented for each configuration. In the experiments, only one movement was produced on a given trial. When no reference or test stimulus was shown (no motion trials) a blank gray frame is depicted in the figure

The CE in perception was defined as the difference between perceived RS directions for upward and downward hand movements (horizontal configuration) or for rightward and leftward movements (vertical configuration). Specifically, the angle at which the TS was stopped was taken as the perceived angle and half the difference in perceived angles between the two movement directions provided a measure of the CE in perception.1

To quantify the CE in production, produced movement directions for motion and no-motion trials were compared. The angle of the (virtual) straight line connecting start and end positions of the hand movement was taken as the produced angle. The difference between produced angles for motion and no-motion trials was then taken as the CE in production.

Based on the findings of Schubö et al. (2001) and Hamilton et al. (2004), a CE in perception should manifest itself by a repulsion of the perceived RS direction by the produced movement direction. For example, upward movements should lead to lower perceived angles than downward movements (Fig. 5a) for the same RS motion. In production, the CE should be reflected in produced angles being further away from the motion direction on motion trials than on no-motion trials (Fig. 5b) for the same required movement directions.

Fig. 5.

Expected perceived angles for upward and downward movements when a CE in perception occurs (a). Expected produced directions for motion and no-motion trials if a CE in production occurs (b). The angular differences are exaggerated for the purpose of illustration

In this context, the mutual interference prediction would mean that overlap in directional features of hand movements and stimulus motions should lead to CEs in both action and perception. A monotonic relationship would be revealed when large/small CEs in action covary with large/small CEs in perception because shunting large/small amounts of directional features would affect both representations to a large/small degree. Finally, if the CE is a consequence of amount of overlap between features of action and perception, large differences in angles between produced and perceived movements should lead to smaller CEs (amount of overlap prediction) because of a smaller amount of overlap.

To preview, Experiments 1 and 2 were designed to address the mutual interference and monotonic relationship predictions. In Experiments 2–4, the angles between the hand movements and RS motions were systematically varied (covering a range from 10° to 100°) to test the amount of overlap prediction (i.e., the influence of perception-action similarity).

Experiment 1

The aim of this experiment was to explore (1) whether CEs in perception and action can be obtained in the same experiment (mutual interference prediction), and (2) whether the size of the individual interference effects covary across or within participants (monotonic relationship prediction). This was achieved by employing the paradigm described above with roughly horizontal RS motions and upward and downward movements that deviated by 25° from the horizontal axis.

Methods

Participants

Twenty-four right-handed individuals (mean age = 25 years; age range = 20–35 years; 3 males, 21 females) participated in the experiment. In this, as well as in the subsequent experiments, all participants reported normal or corrected-to-normal vision and no motor impairments, were not aware of the purpose of the study, and were paid for their time.

Apparatus

Data collection took place in a dimly illuminated and sound-attenuated chamber. An IBM-compatible microcomputer controlled the experiment and was connected to a 21” color monitor that was used for stimulus presentation. Participants performed their movements by displacing a stylus on a graphics tablet (Wacom Ultrapad A3E graphics tablet, 457 mm horizontal, 305 mm vertical). The x and y positions of the stylus were sampled in synchrony with the monitor’s refresh rate (70 Hz). The graphics tablet was covered with a board to prevent participants from seeing their own movement. The tablet was placed below and aligned with the monitor so that the horizontal midpoints of the tablet and monitor corresponded.

Stimuli and movements

In motion trials a red circle with a diameter of 6 mm was used for the RS motion. It started its trajectory vertically centered and 10 cm to the left of the vertical midline of the display area (Horizontal configuration in upper-left part of Fig. 4). The RS motion moved within 500 ms a 20 cm long straight path without leaving a trace on the black background of the display. At the approximate viewing distance of 60 cm, the circle diameter corresponded to 0.57° and the trajectory length to 18.92° of visual angle. The circle moved at a constant speed of 37.84°/s either 4° above, 4° below, or along the horizontal midline.

The TS appeared 15° above or below the horizontal midline immediately after movement recording offset. It consisted of 5 equally spaced circles (identical to the RS circle) arranged along a virtual line. The first and last circles of this line would correspond to start and end positions of RS motions at the respective angles. Immediately after its appearance, the TS began rotating downwards or upwards around the (fixed) circle located at the start position. The direction in which the TS started to move was varied pseudo-randomly and balanced within each block. The rotation speed was 0.2° every refresh of the monitor (i.e., approximately every 14 ms). When the end position of 15° on the other side of the horizontal midline was reached, the direction of TS motion reversed (i.e., moving at 0.24 Hz). In no-motion trials, no TS was shown.

Participants always moved the stylus with their right hand. They started their movements on the graphics tablet horizontally aligned with the RS start position on the screen. The relationship between stylus and screen movements was the same as between a computer mouse movement and its cursor on a screen, that is, movements away from the participants on the graphics tablet corresponded to upward movements on the screen. This mapping was trained during practice trials with off-line feedback of the produced movements (see the “Procedure” for more details).

The required movement trajectories were straight lines that deviated, depending on the movement condition, by either 25° upward or downward from the horizontal axis (Horizontal configuration in lower-left part of Fig. 4). The required length of the movements was 20 cm (i.e., the same length as the RS motion trajectories).

Design

Movement direction (upward, downward) and trial type (motion, no motion) were manipulated within participants. The RS motion angle (+4°, 0, −4°) was varied between participants. This was done to reduce variance in perceived motions that would be caused by different visual stimuli. This seemed especially important given that accuracy of perceived motion directions differs for different directions (e.g., Loffler & Orbach, 2001). Participants either performed downward movements for the first six blocks and then switched to upward movements for the last six blocks or followed the reverse sequence. The order of the movement directions was counterbalanced across participants. Each block consisted of 20 pseudo-randomly arranged trials consisting of 2 trial types × 2 TS starting directions × 5 repetitions. This led to a total of 240 trials.

Procedure

Before a trial started, movements of the stylus were reflected with a gain of 1:1 by motions of a small cursor on the display. The cursor consisted of a white disk with a diameter of 2 mm (0.19° of visual angle). Participants started a trial by moving the cursor into the RS start position. A tone (1,760 Hz, 15 ms) signaled that participants could start moving whenever they felt ready. This tone was played 1 s after the cursor entered the RS start position. As soon as the participants moved out of the circle, the cursor disappeared (movements were always performed without online visual feedback), and the RS started to move in motion trials or disappeared in no-motion trials. In motion trials, a second tone (880 Hz, 15 ms) was played when the RS motion finished (after 500 ms) and the RS disappeared from the display. During the following 500 ms, stylus movements were still recorded (i.e., for a total of 1000 ms). The end of the recording was marked by a third tone (440 Hz, 15 ms).

As illustrated in Fig. 4, in motion trials, the TS started its rotation 1,000 ms after the start position had been left. Participants stopped the rotating line by pressing the space bar with their left hand when they felt that the TS matched the previously seen RS motion direction. This judgment was performed without any speed restrictions. Participants were informed that the RS motions looked very similar and therefore should pay attention also to small differences. Immediately after the key press the screen went blank and the trial ended. In no-motion trials participants started the next trial by pressing the space bar. The next trial started after an inter-trial interval of 1,000 ms.

Prior to each block, except for the first block of a given movement-direction sequence, five practice trials were run. Additionally, prior to the first block of a sequence, a training phase consisting of practice trials was administered. A practice trial started by displaying the to-be-produced trajectory as a straight line in red for 2,000 ms. Participants then moved with the stylus into the start position and waited for 1,000 ms for the first tone which indicated, as described above, the start of the movement phase. The rest of the tone sequence resembled that of an experimental trial. Participants were told that the third tone indicated the end of the movement phase and that they should not move the stylus thereafter.

Feedback of the movement trajectory was given in the form of an overlay of two lines. The white line reflected the actually produced trajectory and the red line the required movement trajectory (i.e., a straight line). This off-line feedback was meant to help participants to acquire the movements and therefore was not available during the experimental blocks. The end of the training phase was determined by a performance based system or when more than 30 training trials had been completed. After each practice and experimental trial, if applicable, an error message was displayed. These error messages informed the participant whenever they lifted the pen, reversed movement direction or moved after the third tone. On average, participants needed about 19 trials to complete the training. The entire experiment lasted between 55 and 90 min.

Data analysis

Movement trajectories were first aligned by setting the x and y stylus values to a common (0,0) coordinate position. Tangential velocity profiles were then obtained by numerical derivation and low-pass filtering at 8 Hz using a fourth-order and zero-lag Butterworth filter. Movement onset/offset was defined as the first/second moment at which 5% of peak tangential velocity was reached. Using these markers, four types of measures were used to quantify the movements. Movement time was set as the difference between offset and onset times. Movement length was computed as the sum of the euclidian distances between adjacent samples along the trajectory from movement onset to offset. Lastly, movement end angle was defined as the angle, relative to the horizontal axis, of the (virtual) line connecting the stylus positions at movement onset and offset. Downward and upward movements were coded with positive and negative angles, respectively (Fig. 4).

To ensure that participants followed the task instructions, trials were excluded when (a) the stylus was moved after the tone that signaled the end of the movement and before the perceptual judgment was provided (late movement), (b) the pen was lifted during movement (pen lift), (c) participants reversed the direction of drawing (movement reversal), (d) the difference between the movement end angle and the required angle was larger than 20° for deviations away from the horizontal axis, larger than 9° for deviations toward the horizontal axis, or the produced trajectory length was shorter than half of the required trajectory length (trajectory failure), or (e) the difference between perceived and RS angles was larger than 5° or the response occurred earlier than 200 ms after the TS started its rotation (concentration failure).

To assess the influence of motion perception on movement production (CE in production), mean absolute movement end angles were computed for each movement direction, trial type, and participant. The means were then averaged within participants across the two movement directions. A CE should show up as a repulsion of the produced movements from the RS (roughly horizontal) motion directions, and thus higher absolute end angles for motion than no-motions trials. The CE corresponds to this difference in end angles and was tested for significance with a two-tailed paired t test with the factor trial type (motion, no motion).

If produced movement directions also lead to a repulsion of perceived motion directions, downward movements should lead to larger perceived angles than upward movements. This CE in perception was assessed by submitting the mean perceived angles for upward and downward movements to a two-tailed paired t test with the factor movement direction (upward, downward).

Finally, Spearman’s rank correlation was used to test for a monotonic relationship across participants between the size of the CEs in perception and production. This means that a value close to 1 should be obtained if participants who show large CEs in perception also tend to show large CEs in production. In contrast, large CEs in perception that co-occur with small CEs in production would produce values close to −1. If the CE size that a participant showed for perception had no relationship to the size the participant showed in production a value close to 0 would be expected.

To detect a potential relationship within participants, Spearman’s rank correlations were calculated between the movement end angles and the perceived angles across trials for upward and downward movements separately. If the CEs in production and perception are related within participants, then low downward movements (high negative angles) should be associated with high perceived angles (high positive angles), and high upward movements (high positive angles) should be associated with low perceived angles (high negative angles). Therefore, negative correlations should be observed for both, downward and upward movements. This allowed us to test them with one single t test. To do so, the 48 (24 participants times 2 movement directions) ρ correlation coefficients were Fisher z’ transformed and tested against 0 with a t test. This trialwise correlation included only motion trials because measures for the effect in action and perception were only available in these trials.

Results

The mean percentages of excluded trials were 3.44, 0.28, 0.19, 1.60, and 3.21% for late movements, pen lifts, movement reversals, trajectory failures, and concentration failures, respectively, resulting in a total percentage of discarded trials of 8.72%.

Produced movements

Figure 6 shows the mean produced and required upward and downward movements for one exemplary participant. These trajectories were obtained by resampling the horizontal and vertical coordinates for each trajectory at 101 (0–100%) equally spaced time points. The coordinates at each time point were then averaged across trials for each type of movement (for details see, e.g., Spivey, Grosjean, & Knoblich, 2005). The trajectories were relatively straight, but slightly overshot the required movement angles. Average movement length (197 mm) and time (773 ms) were within the required range. These descriptive statistics indicate that the participants were able to perform the movement task required of them. Importantly, on average, participants showed a CE in production because the end positions (angles) of the movements were higher for motion than no-motion trials.

Fig. 6.

Mean upward and downward produced and required movement trajectories as a function of trial type (motion, no motion) for one exemplary participant in Experiment 1

This shift is quantified in Table 2, which contains the mean absolute end angles for motion and no-motion trials. As neither Experiment 1 nor 2 showed an effect of RS motion angle on CE size in production, motion trials are reported collapsed across the different RS motion angles. The difference in end angles between the two trial types represents the CE in production and is graphically presented in Fig. 7 (empty circle for the 25° required movement angle). As indicated by the positive CE value, on average, participants’ movements veered away from the RS motion (on motion trials). This CE was significant (t[23] = 3.98, p < 0.01).

Table 2.

Mean absolute movement end angle and mean perceived angle (in degrees)

Required movement angle	Trial type		Movement direction
	Motion	No motion	Up/left	Down/right
Experiment 1
25°	28.58 (0.57)	27.79 (0.58)	−0.65 (0.65)	0.01 (0.73)
Experiment 2
10° (large AO)	15.20 (0.46)	14.29 (0.39)	−0.77 (0.92)	−0.14 (0.84)
35° (small AO)	38.91 (0.55)	38.29 (0.56)	−0.90 (0.91)	−0.04 (0.96)
Experiment 3
30° (large AO)	36.31 (0.78)	–	−0.72 (0.28)	0.94 (0.24)
80° (small AO)	78.93 (0.48)	–	−0.30 (0.33)	0.96 (0.23)
Experiment 4
10° (large AO)	17.74 (1.07)	–	−0.88 (0.87)	0.25 (0.84)
100° (small AO)	103.52 (0.67)	–	0.05 (0.94)	−0.50 (0.92)

Mean absolute movement end angles are reported as a function of trial type (motion, no motion) and the absolute value of the required movement angle (small and large amount of overlap conditions) for Experiments 1–4. Similarly, mean perceived angles are displayed as a function of movement direction (up/left, down/right). End angles are averaged across the two movement directions (up/left, down/right). All angles are coded relative to the horizontal midline for Experiments 1–2 and relative to the vertical midline for Experiments 3–4. Standard errors of the mean values are listed in brackets

AO amount of overlap

Fig. 7.

Size of the CE in production and perception as a function of the absolute value of the required movement angles for Experiments 1 (25°) and 2 (10° and 35°). Required movement angles are coded relative to the horizontal axis. Whiskers indicate 95% confidence intervals

Perceived motions

Mean perceived angles are presented in Table 2. The perceived angles were higher for downward than upward movements and half the difference between these values represents the CE in perception. This CE (filled circle for the 25° condition of Fig. 7) was positive and significantly different from 0 (t[23] = 3.46, p < 0.01).

Relationship between CEs in production and perception

Figure 8 presents the CE in perception as a function of the CE in production across participants. Each data point corresponds to the CEs for one participant. Despite notable inter-individual differences in the size of the CEs in production and perception, no systematic relationship between the interference effects was present. This was statistically confirmed by a non-significant correlation value of 0.16 (p = 0.45) across participants. The within-participant correlation across all participants and movement directions was not significant either (mean ρ = −0.01, t[47] = −0.36, p = 0.72). See “Data analysis” for details how the correlations have been calculated.

Fig. 8.

Size of the CE in perception as a function of the size of the CE in production for Experiment 1. Each data point corresponds to one participant

Discussion

In Experiment 1, it was possible to obtain a CE in both action and perception. Specifically, on average, hand movements were found to veer away from perceived motion directions and perceived directions were repelled by the direction of the concurrently produced movements. This mutual specific interference effect was predicted by the Schubö model. In contrast to the monotonic relationship prediction of their model, however, no systematic relationship between the size of the CEs in action and perception was observed (within or across participants). If the variability in CE sizes across participants was caused by differing amounts of overlapping representational elements, then a relationship between the CEs would have been expected. We return to this issue in the “General discussion”.

Experiment 2

The aim of Experiment 2 was to replicate the basic results of Experiment 1, as well as test the amount of overlap prediction made by the models. They both predict that the size of the CE should increase with increasing similarity between action and perception. Accordingly, the angular difference between the produced movements and stimulus motions was systematically varied in this experiment. This was achieved by employing pairs of movement directions that varied by either ±10° (large amount of overlap condition) or ±35° (small amount of overlap condition) from the horizontal axis. Thus, the two amount of overlap conditions differed by 25°.

Method

Except where noted, the method used in this experiment was identical to that used in Experiment 1.

Participants

Twenty-seven right-handed individuals (mean age = 25 years; age range = 18–34 years; 7 males, 20 females) took part in the experiment.

Apparatus, stimuli and movements, design, and procedure

In this experiment, only two RS motion directions (+4°, −4°) were used. More critically, there were now four different movement directions. In the large amount of overlap condition, participants performed movements that deviated by 10° above or below the horizontal midline. In the small amount of overlap condition, the movements deviated by 35° above or below from horizontal. The two amount of overlap conditions were blocked and counterbalanced across participants. Each of the 4 movement directions was performed for 3 consecutive blocks, leading again to a total of 240 trials. On average, 20 trials were needed to complete training and the experiment lasted between 60 and 95 min.

Data analysis

The data were analyzed in the same way as before, except that trajectory failures were defined as movements that deviated by more than 5° from the required angles toward the horizontal midline or more than 20° away from it. The 9° criterion used in Experiment 1 was replaced by a 5° criterion to ensure that the produced movements did not overlap with the ±4° stimulus motion directions in the 10° movement direction condition.

The influence of motion perception on movement production was again assessed by computing and averaging the mean absolute end angles across the downward and upward directions, for each trial type, amount of overlap condition, and participant. Then, a two-way repeated-measures ANOVA on these values was performed with the within participant factors amount of overlap (small, large) and trial type (motion, no-motion). Similarly, to detect a change in motion perception as a function of movement production, a two-way repeated-measures ANOVA on mean perceived angles was computed with the within participant factors amount of overlap (small, large) and movement direction (upwards, downwards). Finally, to test whether a systematic relationship between CEs in production and perception was present, the same monotonic correlation analyses as in Experiment 1 were performed but separately for the small and large amount of overlap conditions.

Results

The data of three participants had to be excluded. One participant was excluded because of a technical problem. The other two were excluded because, after applying the exclusion criteria, some conditions had zero observations. These zero observation conditions resulted in both cases from about 48% excluded trials that were mainly due to concentration failures (88%). For the remaining participants (N = 24), the mean percentages of excluded trials were 3.39, 1.94, 1.60, 4.93, and 4.79% for late movements, pen lifts, movement reversals, trajectory failures, and concentration failures, respectively, resulting in a total percentage of discarded trials of 16.65%.

Produced movements

The average movement length (194 mm) and time (716 ms) were within the expected ranges. As shown in Table 2, participants again had a tendency to overshoot the required movement angles in both amount of overlap conditions. Importantly, the produced end angles clearly differed for small and large conditions (as they should after filtering of trajectory failures).

A CE was obtained for the 10° and 35° required movement angles (Fig. 7). However, the size of the CEs was similar for both amount of overlap conditions. These observations were confirmed in the ANOVA, which yielded a significant main effect of trial type (F[1, 23] = 18.70, MSE = 0.76, p < 0.001), but no interaction between trial type and amount of overlap (F[1, 23] = 1.28, MSE = 0.40, p = 0.27). The remaining significant main effect of amount of overlap (F[1, 23] = 1,270.53, MSE = 10.75, p < 0.001) indicates that mean end angles differed for the two amount of overlap conditions.

Perceived motions

Downward movements led to larger perceived angles than upward movements (Table 2). Figure 7 presents the size of the corresponding CEs in perception for the 10° and 35° required movement angles. Although a CE was present for both amount of overlap conditions, they were roughly equal in size. These patterns were corroborated in the ANOVA, which revealed a significant main effect of movement direction (F[1, 23] = 12.12, MSE = 1.11, p < 0.01), but no significant effects including the factor amount of overlap (both Fs < 1).

Relationship between CEs in production and perception

No systematic relationship (across participants) held between the size of the CEs in production and perception for either amount of overlap condition. This was confirmed by a non-significant correlation value of 0.19 (p = 0.37) for the small amount of overlap condition and of 0.19 (p = 0.37) for the large amount of overlap condition. Similarly, the mean within-participant ρ values of −0.01 and −0.40 for the small and large amount of overlap conditions were not significantly different from 0 (t[47] = −1.11, p = 0.27; t[47] = −0.37, p = 0.71).

Discussion

In line with Experiment 1, no monotonic relationship between the size of the CEs in action and in perception was found. Importantly, even though CEs were obtained for both amount of overlap conditions in action and perception, no difference in the size of the CEs as a function of amount of overlap was observed for either action or perception. This overall pattern of results is in opposition to the amount of overlap prediction of the Schubö and Hamilton models. To avoid a premature conclusion about the amount of overlap prediction, we further decreased the amount of overlap between the produced and perceived directions in the next two experiments.

Experiment 3

In this experiment, the amount of overlap between the direction of the produced movements and the concurrently perceived motions was further decreased to verify that we did not simply miss a potential influence of amount of overlap in Experiment 2. Consequently, the angular separation between the two amount of overlap conditions was increased to 50° by using pairs of movement directions that varied by either ±30° (large amount of overlap) or ±80° (small amount of overlap) from the vertical axis. The setup was also rotated counterclockwise by 90°, such that the motions moved roughly vertically and the movements went to the left/right of vertical. There were three reasons for this rotation. First, it tested whether findings for upward and downward movements would generalize to leftward and rightward movements. Second, given the dimensions of the graphics tablet (457 mm horizontal, 305 mm vertical) participants could produce longer movements without risk of slipping off the tablet. Third, matching of ‘up’ and ‘down’ movements on the graphics tablet to motions on the screen is not as natural as ‘right’ and ‘left’ which have an obvious interpretation in both cases. Finally, given that the focus of this experiment was on the influence of amount of overlap, no-motion trials were omitted and CEs in production were no longer considered.

Method

Except where indicated, the method was identical to Experiment 2.

Participants

Twenty-five right-handed individuals (mean age = 26 years; age range = 18–35 years; 3 males, 22 females) participated in the experiment.

Apparatus, stimuli and movements, design, and procedure

The setup of this experiment resembled that of Experiment 2, but rotated counterclockwise by 90°. Therefore, angles were coded relative to the vertical midline (vertical configuration in Fig. 4). The movement directions deviated to the left or right from the vertical midline by 30° in the large amount of overlap condition and by 80° in the small amount of overlap condition. Accordingly, the RS start position was moved to a horizontally centered position, 7.5 cm below the horizontal midline (vertical configuration in upper-left part of Fig. 4). The RS trajectory length and required movement length were shortened to 12 cm (11.42° of visual angle) to make sure that participants stayed on the graphics tablet in the large amount of overlap condition and deviated by (+4°, −4°) from the vertical. No-motion trials were eliminated from the design. Consequently, only two blocks of 20 trials were run for each movement direction (in a counterbalanced order), resulting in a total of 160 trials. The mean number of practice trials needed in the training phase was 23 trials and the experiment lasted between 45 and 65 min.

Data analysis

Movements were considered as trajectory failures when produced end angles were closer than 5° to the horizontal midline or deviated by more than 20° from the required angles. The 5° criterion was chosen to ensure that only movements with an upward component were analyzed. Given the absence of no-motion trials, CEs in production and correlations between the CEs in production and perception were no longer computed.

Results

The data of one participant had to be excluded because of conditions with zero observations resulting from about 65% excluded trials, 39% thereof being trajectory failures and 56% beeing concentration failures. For the remaining participants (N = 24), the mean percentages of excluded trials were: 1.07, 1.22, 2.40, 10.73, and 7.81% for late movements, pen lifts, movement reversals, trajectory failures, and concentration failures, respectively, resulting in a total percentage of discarded trials of 23.23%.

Produced movements

Table 2 shows that participants had a tendency to overshoot the required movement angle in the 30° condition and slightly undershoot it in the 80° condition. Otherwise, average movement length (157 mm) and time (705 ms) were within the expected ranges.

Perceived motions

Perceived angles were larger (i.e., more to the left) for rightward than leftward movements (Table 2). However, Fig. 9 shows that the corresponding CEs for the 30° and 80° conditions were similar in size. The results of the ANOVA confirmed these observations. There was a significant main effect of movement direction (F[1, 23] = 34.30, MSE = 1.5, p < 0.001), but no significant effects including the factor amount of overlap (main factor: F[1, 23] = 1.62, MSE = 0.72, p = 0.22, interaction F[1, 23] = 1.69, MSE = 0.54, p = 0.21).

Fig. 9.

Size of the CE in perception as a function of the absolute value of the required movement angles for Experiments 3 (30° and 80°) and 4 (10° and 100°). Required movement angles are coded relative to the vertical axis. Whiskers indicate 95% confidence intervals

Discussion

Despite increasing the angular separation between the amount of overlap conditions to 50°, no difference in the size of the CEs was found. In the next experiment we performed a final test of the amount of overlap prediction.

Experiment 4

Until now, the angular separation between the movements and motions was never increased beyond 90°. We therefore tested the amount of overlap prediction for the last time by choosing a small amount of overlap condition that was ±100° away from the vertical axis. With a large amount of overlap condition requiring ±10° movements, the separation between the amount of overlap conditions was 90° .

Method

The method was identical to Experiment 3, except where noted otherwise.

Participants

Twenty-five right-handed individuals (mean age = 26 years; age range = 18–36 years; 6 males, 19 females) participated in the experiment.

Apparatus, stimuli and movements, design, and procedure

The large amount of overlap condition involved 10° movements to the right or left of the vertical midline. In the small amount of overlap condition, 100° movements (10° below the horizontal midline) either to the right or left had to be produced. On average, participants needed about 21 trials to learn to perform these movements and the experiment lasted between 55 and 75 min.

Data analysis

The movement rejection criteria were changed to ensure that produced movements did not overlap with the stimulus motions and had a clear downward component to them. Movements that came closer than 5° to the vertical and horizontal midlines or deviated by more than 20° from the required angles were considered trajectory failures.

Results

The data of one participant had to be excluded because of conditions with zero observations resulting from about 94% excluded trials with 5% thereof being trajectory failures and 91% concentration failures. For the remaining participants (N = 24), the mean percentages of excluded trials were 1.72, 1.72, 0.94, 11.80, 6.09% for late movements, pen lifts, movement reversals, trajectory failures, and concentration failures, respectively, resulting in a total percentage of discarded trials of 22.27%.

Produced movements

Average movement length (122 mm) and time (715 ms) were within the expected ranges. Moreover, participants overshot the required angles (Table 2), but to a larger degree in the large than in the small amount of overlap condition.

Perceived motions

Perceived angles were larger for rightward than leftward movements for the large amount of overlap condition, but this difference was reversed for the small condition (Table 2). That is, when the movements had a downward component to them (small amount of overlap condition), the CE actually turned into an assimilation (attraction) effect (AE). This can be clearly seen in Fig. 9, where the “CE” was positive for the 10° required movement angle and negative (indicating assimilation) for the 100° movement angle. These findings were reflected in the results of the ANOVA, which revealed a significant main effect of movement direction (F[1, 23] = 9.54, MSE = 0.22, p < 0.01) and a significant interaction between movement direction and amount of overlap (F[1, 23] = 19.87, MSE = 0.84, p < 0.001). There was no main effect of amount of overlap (F < 1). Separate ANOVAs for the large and small amount of overlap conditions with the factor movement direction confirmed that the CE and AE were indeed significant (F[1, 23] = 40.57, MSE = 0.38, p < 0.001; F[1, 23] = 5.15, MSE = 0.68, p < 0.05, respectively).

Discussion

For the first time, the amount of overlap manipulation had an influence on the CE. However, in contrast to the predictions of the models, the size of the CE did not decrease with smaller amount of overlap; it actually changed its sign and turned into an AE. That is, on average perceived directions were attracted by produced directions. Based on the pattern of effects observed in the previous experiments, this change appears to occur rather abruptly at around 90°. This finding is considered in more detail below.

General discussion

The current experiments investigated how visual motion perception and concurrent movement production mutually interact. Participants were asked to produce hand movements (without visual feedback) while observing the motion of an independent stimulus. By varying the direction of the movements relative to the direction of the motions, it was possible to show that, on average, perceived directions were repelled by produced directions and produced directions were repelled by perceived directions. As far as we know, this is the first time that CEs in action and perception have been found in the same experiment. However, contrary to what was expected based on two recent models (Hamilton et al., 2004; Schubö et al., 2001), the size of the CEs in production and perception did not covary across or within participants, nor did the size of the effects systematically decrease with an increase in angular separation between the movement and motion directions. Interestingly, the CE in perception actually turned into an AE when the angular separation increased beyond 90°, that is when the movements were produced with a downward component while the motions had an upward component. The only prediction of the Schubö and Hamilton models (Table 1) that found empirical support was the mutual interference prediction, which states that influences on action and on perception should co-occur.

For economical reasons the influence of concurrent motions on the CEs in production was only tested in Experiments 1 and 2. However, given the null results in Experiments 1 and 2 it is very unlikely that a monotonic relationship had been observed if CEs in production had also been measured in Experiments 3 and 4.

While movements were employed in action and in perception in the current study we do not think that overlap on another dimension than direction is necessary for specific interference effects to show up. For example, the same effect would be expected if a line instead of a moving dot was used. Indeed in a similar but unpublished experiment we also found a CE in action with a static line. In addition, also a static object without orientation information induces CEs. This is because its location specifies a direction. For example, in a study by Tipper et al. (1997) hand movements were influenced by stationary distractors. Tipper and colleagues accounted for their results by proposing that representations of the performed hand movements to the target and of the potential hand movements to the distractors interfered. We differ from this account, based on our findings of interference also in perception, in that we suggest that interference takes place between the representations of perception and action events. Importantly, the reported effect is not unspecific, that is, effecting a general shift in one direction rather it shifts rightward hand movements to the right while leftward hand movements are shifted to the left.

In the experiments movement direction was blocked. This was an attempt to minimize interference by non-motor representations which could occur if visual cues for movement directions were used. Therefore, the current experiments cannot exclude the possibility that the categorical influence (i.e., CE size did not depend on the amount of overlap, see below) was caused by the small amount of reprogramming or the short movement time that was needed for task performance, which might have prevented development of detailed spatial representations.2 Although, in the end, this is an empirical question and it would be interesting to look at it in future studies, there are two reasons why this does not challenge our explanation. First, while it is true that movements were always performed only in one direction during a block, participants were still required to produce a specific direction and as can be inferred from Table 2 succeeded in doing so. Additionally, they were retrained on the specific direction at the beginning of each block. Therefore, there is reason to believe that participants encoded the specific direction; simply coding, for example, ‘right upward’ would not suffice to achieve the criterion. Second, even if coding of categorical movements was caused by the small need to reprogram, the current findings are still interesting, because they show that interaction between action and perception can take place at a categorical level and that categorical coding seems to be the preferred coding if possible. If the need to invoke more detailed representations added to this interference effect an amount of overlap component would then be a question of further studies in which different directions are required for each movement. Interestingly, we are not aware of any study that reported such an effect. The problem with such a paradigm would be, how to measure an effect on action, given the large expected variability when movements change from trial to trial (simply having two movement directions would probably not suffice to exclude categorical coding). It is also unlikely that longer movement times would add an amount of overlap component to the CE, given that comparing Experiments 1 and 2 of Grosjean et al. (2008) suggests that if anything, the size of CE in action decreases with longer movement times.

The size of the effects (<1°) obtained in the present study are somewhat smaller compared to that of related effects, such as the induced shift of a moving dot by surrounding tilted lines (e.g., Westheimer, 1990). Arguably, this limits the practical relevance of such phenomena to situations in which visual motion perception and movement production require high levels of accuracy (e.g., endoscopic surgery). Nonetheless, the generality and robustness of these effects have now been demonstrated in a number of tasks and for a variety of perception-action dimensions (e.g., amplitude, weight, and direction; Hamilton et al., 2004; Müsseler & Hommel, 1997; Schubö et al., 2001, 2004; Zwickel et al., 2007, 2008), which has made them very useful for testing models of the perception-action interface. Importantly, despite the small size of the effect, the CE was obtained at a high level of significance which makes it unlikely that we failed to support the hypotheses because of too low statistical power. Additionally, amount of overlap had an influence for an angular separation of more than 90° but this influence was in opposition to the prediction of the models. We now consider whether other models of perception-action, action-action and perception-perception interactions could explain the present findings.

Related findings and models

Two particularly relevant models have been proposed by Tipper et al. (1997) and Welsh and Elliott (2004) to account for perception-action interference effects in pointing tasks. Tipper et al. (1997) observed that hand movements to target objects deviated away from near distractor objects but toward far distractors. Repulsion from the near distractor was attributed to the inhibition of a potential movement to the distractor location, the distributed representation of which was assumed to overlap with the representation of the movement to the target. As in the Schubö model, which was inspired from this model, the inhibition of overlapping representational elements was posited to lead to a shift of the resultant reach away from the close distractor. To account for the attraction to the far distractor, it was further assumed that the amount of inhibition was proportional to the saliency of the distractor object, with saliency being related to the proximity of the distractor to the moving hand. Therefore, far distractors led to less inhibition than near distractors. As a consequence, the resultant reaching movement will actually combine representational elements of reaches to the target and distractor, thereby leading to an attraction effect for far distractors. This model could, in principle, account for the change from repulsion to attraction that was observed in the present experiments. However, it also predicts that the amount of repulsion should gradually diminish with smaller amount of overlap, which is inconsistent with the current results.

The response activation model of Welsh and Elliott (2004) is similar to the model of Tipper et al. (1997) in that it postulates that interference effects arise from the parallel activation of reaches to the target and distractor. It differs from it, however, in that whether attraction or repulsion occurs depends not on the spatial but on the temporal relationship between distractor and target stimuli. If distractor onset precedes target onset by enough time, the distractor response can be inhibited and repulsion arises. If, however, the target follows the distractor very close in time, the amount of inhibition is not large enough to prevent a combined response and attraction occurs. The time course assumptions of the response activation model cannot, however, explain the present pattern of results either. Namely, the absence of an amount of overlap effect for angular separations smaller than 90° and the reversal of the effect beyond this value.

It is interesting to note that the horizontal axis (midline) has been found to play a special role in line copying tasks (Meulenbroek & Thomassen, 1991, 1992; Van Sommers, 1984). In these tasks, participants had to produce rapid small back-and-forth movements in directions that were self-selected from a set of possible directions (Meulenbroek & Thomassen, 1992) or instructed by the experimenter (Meulenbroek & Thomassen, 1991; Van Sommers, 1984). In both cases, the possible directions were visually shown. Participants movements had a tendency to veer away from the horizontal, that is, to produce lower orientations than required when producing movements below but close to the horizontal, and higher orientations than required when producing movements above but again close to the horizontal. These results underline the qualitative difference between movements produced below and above the horizontal. It is unclear, however, what predictions about the CE could be derived from these results, given that in the current experiment the visual motions did not coincide with the required directions. What is more, any general form of repulsion from the horizontal would be subtracted away by comparing motion and no-motion trials and would not show up in the present effects.

For action-action interactions, interference effects (AE) that are influenced by amount of overlap have also been reported. For example, Swinnen, Dounskaia, Levin, and Duysens (2001) asked participants to produce vertical movements with their left hand while concurrently performing rapid movements in certain directions with their right hand. The results showed that the movement of the left hand was biased in the direction of the concurrent right hand movement. The AE was largest for orthogonal directions and smallest for vertical movements with the right hand. Amount of overlaps between these two extremes led to intermediate effect sizes. This pattern of interference was attributed to the spread of neural activity via interhemispheric connections. When the two directions were similar and therefore led to the activation of similar movement population vectors (representations) in each hemisphere, spreading activation did not lead to a strong change in direction coding. Less similar vectors, however, resulted in more interference. Thus, this model predicts an influence of amount of overlap as well.

Amount of overlap effects were also observed for perception-perception interactions (e.g., Marshak & Sekuler, 1979; Westheimer, 1990). For example, in Marshak and Sekuler (1979)’s experiment, participants watched random dots that moved in two different directions. One of these directions was always horizontal. Participants’ task was to judge the direction of the other motion. The results showed that this motion was perceived as being repelled from the horizontal motion. Additionally, the size of the repulsion decreased with larger angles between the two motions. Mahani, Carlsson, and Wessel (2005) proposed a model to account for such perceptual interference effects in which motion repulsion is considered a side-effect of clustering algorithms (see also Navalpakkam & Itti, 2007). The underlying idea is that when concurrent features are assigned to two different tasks, features of the two tasks might become mixed. According to Mahani et al. (2005) the probability of misclassifying features of a given motion is higher for features that are less typical for one motion and more typical for the other motion. This loss of “untypical” and the gain of more “typical” features leads to a CE. However, this mechanism would also predict an amount of overlap effect.

Given that amount of overlap effects have been found in action-action and perception-perception paradigms but not in the current paradigm, it seems that interference effects between perception and action differ from interference effects within perception and action. In the following we will argue that this difference depends on the kind of representations that are inhibited. Specifically, we will argue that while detailed spatial relationships are preserved for action-action and perception-perception interactions, this information is “lost” when functionally independent actions and perceptions interfere. Instead, interactions between perception and action are mediated by the activation of categorical representations. Indeed, to the best of our knowledge, there is only one study that reported an interference of detailed spatial representations between action and perception (Ehrenstein, Cavonius, & Lewke, 1996). Importantly, in this study, action and perception were not functionally independent (Zwickel et al., 2007, 2008), which led to an AE.

A categorical account

One way to account for the the absence of an amount of overlap effect within the range of about 90° is to assume that, although specific angles are represented at some level of the system, interference arose between categorical representations of the movement and motion directions. For example, all hand movements within the upper-right quadrant of the graphics tablet may have led to the activation of the same “right upward” representation, irrespective of the specific angle that was required. Similarly, stimulus motions that moved roughly vertically may have always activated the same “upward” representation. Therefore, at this level of the system, the amount of overlap manipulation in Experiments 2 and 3 did not lead to activation of different representations. This means that, for example, the 35° movement to the right led to the same activation of the “right upward” representation as the 10° movement to the right did. Given the same activation in these two cases no difference in interference size would be expected. In addition, no correlation between the CE sizes in perception and action would be expected as the variance of the CE sizes in action and perception would only be caused by additional sources of variance that are unrelated to the processes and representations that lie at the origin of the interference effects themselves. In other words, a large CE in perception would not be caused by the a difference in representation at the level where action and perception interfere but by earlier or later processing. While this would explain why no influence of amount of overlap and no monotonic relationship would be expected, the next section will deal with the issue why mutual interference should still arise in this model.

While movement production and motion perception would activate one categorical representation at the level of interference, these categories themselves could be represented in a distributed fashion, such that, for example, activating the category “upward” would involve the activation of elements that code for motions slightly to the right and left of vertical as well. In this way, upward movements to the left or right would lead to the activation of a common set of elements with upward motions, but the amount of representational overlap would not depend on the specific directions involved. By assuming, as did Schubö et al. (2001), that these common elements are inhibited, a CE would arise and it would not vary in size as long as the movement has an upward component.

For downward movements, however, a different categorical representation, such as “right downward”, could have been activated. Given that it would not overlap with the representation of an upward motion, no inhibition would be necessary to keep the motor and perceptual activities separate. However, as discussed above (Mahani et al., 2005), some elements between the perceptual and motor representations may be exchanged by chance, leading to assimilation. Interestingly, this would predict that the size of the AE should increase with an decrease in amount of overlap because a random exchange of elements from less similar movements and motions would involve an exchange of more dissimilar elements.

The involvement of categorical representations in perceptual processes is not new (e.g., Braine, 1978; Postma & Laeng, 2006) and spatial categories have already been shown to influence, among other things, the perception of locations (Huttenlocher, Hedges, & Duncan, 1991) and visual search times (Wolfe, Friedman-Hill, Stewart, & O’Connell, 1992). In Wolfe et al. (1992), visual search times were found to be faster when distractors and targets came from different categories (e.g., tilted to the right vs. tilted to the left) than when distractors and targets belonged to the same category (e.g., both tilted to the right). The present results add to this by showing that categorical representations also play a critical role in determining how concurrent perception and action interact.

When similarity is linked to comparability, the present account is related to the explanation of contrast and assimilation advanced by Aarts and Dijksterhuis (2002). Aarts and Dijksterhuis (2002) found that priming individuals with a certain speed could have a contrastive or assimilative effect on later speed judgments. However, the direction of the effect depended on whether the prime and test stimuli were perceived as comparable or not. For example, whether priming individuals with a turtle led to higher speed estimates of a human depended on whether participants believed that animals and humans are comparable or not. When participants read about the similarity between animals and humans, a CE was observed. However, reading about differences between humans and animals led to AEs. In the current context, one might argue that CEs were observed as long as participants perceived hand movements and stimulus motions as similar or comparable. AEs resulted, however, when decreasing the amount of overlap led to no perceived similarity or comparability between what was produced and perceived.

Taken together with the findings reviewed in the previous section, the present results suggest that the nature of interference effects depends on the type of events that are involved. When both events come from the same domain, that is within action or perception, interference seems to arise between detailed, as opposed to categorical, representations. Two studies are especially informative in this regard because they showed that the interference between spatial features of stimuli and responses depends not only on their physical properties but is more flexible. Hommel (1993) instructed participants to press one of two buttons in response to the frequency of a tone (low/high). The tones were presented at task-irrelevant left or right locations. In another condition, participants were instructed to turn on a light in response to the frequency of the tones by pressing a button on the opposite side of the light. Shorter reaction times were obtained for spatial correspondence between the locations of the tone and the button in the button condition, and between the locations of the tone and the light in the light condition. Thus, the interference effects were modulated by how the participants represented the goal of their actions (button vs. light location). Similarly, Stevanovski, Oriet, and Jolicoeur (2002) showed that interference effects change as a function of whether a stimulus (e.g., <) is interpreted as an arrow head (i.e., pointing to the left) or a headlight (i.e., projecting to the right). Analogously, in the current experiments, the observed interference might have occurred at a level where physical properties of the stimuli and actions played a minor role. It is thus not surprising that other researchers have already suggested that categorical codes underly various types of interference effects (e.g., de C. Hamilton, Joyce, Flanagan, Frith, & Wolpert, 2005; Hommel, 1998; Kunde & Wühr, 2004; Lindemann et al., 2006).

Assimilation versus contrast

One issue that has not been considered until now is that some studies reported facilitatory effects between produced and perceived events (e.g., Brass, Bekkering, & Prinz, 2001; Craighero, Fadiga, Rizzolatti, & Umiltà, 1999; Stürmer, Aschersleben, & Prinz, 2000; Vogt, Taylor, & Hopkins, 2003). However, in these studies, action and perception where not functionally independent from each other. For example, Vogt et al. (2003) showed that grasping movements where performed faster when the go signal depicted a picture of a congruent versus incongruent grasp end position. Crucially, in all these studies, hand movements were performed in response to the interfering pictures and therefore action and perception where not functionally independent. There are also two studies that have reported AEs between two functionally independent produced and perceived events (Repp & Knoblich, 2007; Wohlschläger, 2000). Interestingly, their experiments differ from the current ones and those of Hamilton et al. (2004) and Schubö et al. (2001) in that they used ambiguous visual or auditory stimuli. For example, Wohlschläger (2000) investigated the influence of directional hand movements on the perception of an ambiguous motion that could be interpreted as rotating clock- or counterclockwise. What he found was that the direction of perceived motion was biased in the direction of the produced movement. The critical difference between this study and those considered so far could be that, under ambiguous stimulus conditions, one often relies on non-perceptual information to achieve a stable percept. This may have lead to more integration (assimilation) between perceptual and action-related information than in the type of experiments used here, where participants may have been able to keep their perceptual and motor tasks more separate from each other.

Similarly, Repp and Knoblich (2007) found that the direction of movements on a piano keyboard influenced perceived changes in pitch of an ambiguous tone sequence. The tone sequence was more often judged as rising when participants moved from left to right than when they moved from right to left. However, this influence was obtained for expert piano players, but not for non-expert piano players. The authors suggested that this effect was related to the existence of overlearned movement-auditory effect associations for the experts. Given the relative novelty of the movements and motions used in the current study, the difference between the results of Repp and Knoblich (2007) and the current ones could be related to their use of ambiguous stimuli and/or the involvement of overlearned movement-effect associations. That said, future research is necessary to resolve the apparent conflict between these effects and the type of effects considered here.

Summary

In the present study, mutual interference (contrast and assimilation) effects between the direction of concurrently produced movements and perceived motions were obtained. However, the sizes of the effects in production and perception were not correlated nor did they gradually increase with increases in the angular amount of overlap between the produced and perceived directions. These findings were inconsistent with two models that have been proposed to account for such effects. To deal with this problem, we posited that such effects are mediated by the activation of categorical representations. An aim for future studies should be to determine which directions these categorical representations contain and how much people vary in their category structure. This would further our understanding of how perception and action interact and possibly even help prevent interference effects in certain situations.