Abstract
Target selection is biased by recent experience. For example, a selected target feature may be stored in memory and bias selection on future trials, such that objects matching that feature are “primed” for selection. In the present study, we examined the role of action history in selection biases. Participants searched for a uniquely colored object. Pre-trial cues indicated whether participants should respond with a keypress or a reach movement. If the representation of the feature that biases selection is critically bound with its associated action, we would expect priming effects to be restricted to cases where both the response mode and target color are repeated. However, we found that responses to the target were faster when the target color was repeated, even when the response switched from a reach to a keypress, or vice versa. Priming effects were even observed following “no-go” trials where a response was withheld, and priming effects transferred across response modes when eye movement recordings ensured that participants did not saccade to the target. These results demonstrate that target features are represented in memory separately from their associated actions and can bias selection on subsequent trials even when a different mode of action output is required.
Keywords: Visually-guided reaching, selection for action, priming of popout, trial history, memory
Attentional mechanisms allow observers to select subsets of a visual scene for more detailed processing. A great deal of research has focused on what factors guide this selection process (e.g., Chun & Jiang, 1998; Egeth, Virzi, & Garbart, 1984; Maljkovic & Nakayama, 1994; Theeuwes, 1992, Treisman & Gelade, 1980; see Lamy & Kristjánsson, 2013 for a review). However, behavioral goals typical require action outputs. Furthermore, recent evidence suggests that the motor system is integrated with higher-level selection processes (e.g., Cisek, 2007; Gallivan, McLean, Smith, & Culham, 2011; Song & Nakayama, 2006; Song, Rafal, & Mcpeek, 2011; Song & Nakayama, 2009; Spivey, Grosjean, & Knoblich, 2005). Therefore, the role of action in the guidance of selection deserves further scrutiny.
One area where the role of action remains relatively unexplored is in understanding how memory representations of previous actions might bias selection. It has been shown that properties of recently selected targets, or “selection history” (see Awh, Belopolsky, & Theeuwes, 2012), have a marked effect on selection. For example, Maljkovic and Nakayama (1994) asked observers to report the shape of a uniquely colored target with a keypress response; response times were shorter when the target color was repeated on consecutive trials compared to when the target color was switched, demonstrating a “priming of popout” (POP) effect. The authors proposed that the POP effect is driven by an implicit memory trace associated with the previously selected target that biases selection automatically on subsequent trials (see also: Becker, 2008a; Leonard & Egeth, 2008; see Kristjánsson & Campana, 2010, for a review). Repetition of simple feature targets (Goolsby & Suzuki, 2001; Wolfe, Butcher, Lee, & Hyle, 2003), conjunction targets (Hillstrom, 2000), target locations (Maljkovic & Nakayama, 1996), and the spatial configuration of display elements (Chun & Jiang, 1998) can also bias selection.
Recent experience also affects performance when an action output is required. Eye movements to a target are faster when a target feature is repeated on consecutive trials (Becker, 2008b; 2008a; McPeek, Maljkovic, & Nakayama, 1999). Reach movements, too, show evidence of color POP; hand movements to a target are faster and more direct (i.e., less curved) when the target color is repeated on consecutive trials (Moher & Song, 2013; Song & Nakayama, 2006). In a recent study, we examined whether the action output itself has any effect on subsequent responses in a visually-guided reaching task. We examined reach movement trajectories on a trial-by-trial basis, and found some trials where the trajectory was initially aimed at a non-target object before being re-directed midflight to the correct target object. We found increased distractor interference following these “partial error” responses when the target color was repeated, suggesting that the action output itself does have an effect on subsequent responses (Moher & Song, 2013). There is additional evidence to suggest that specific action responses within keypress or reaching behavior may be bound together in memory with stimulus properties and can bias subsequent responses (Hommel, 2004); for example, an observer might aim an arm movement in a curved direction to avoid an obstacle while reaching to a target, and re-use a similar movement trajectory on a subsequent trial with the same target even when the obstacle has been removed (e.g., Jax & Rosenbaum, 2007; Jax & Rosenbaum, 2009).
While these studies highlight a role for the motor system in memory-based selection bias, it is unclear how much the mode of action itself (e.g., keypress, eye movement, reach movement) plays a role. For example, imagine a woman wants to know where her keys (with a red keychain) are on the desk. She might search the desk until she find the keys, then look back to the computer to check her email. After a moment, she decides to take a sip of coffee out of her (red) mug. Because the red keychain was recently selected for vision, will there now be a bias in favor of selecting red objects (such as the red coffee mug) for action?
A number of studies have suggested that selection for vision and action share similar underlying mechanisms (Song & Mcpeek, 2009; Song & Nakayama, 2007a), with movement trajectories reflecting the degree of competition for target selection (e.g., Song & Nakayama, 2006; Spivey, Grosjean, & Knoblich, 2005), suggesting that a shared “priority map” guides selection regardless of the required response mode (Song, Rafal, & Mcpeek, 2011; Zehetleitner, Hegenloh, & Muller, 2011). Furthermore, recent studies have found evidence that areas previously associated with specific motor effectors, such as frontal eye fields and superior colliculus, are involved in motor-unspecific target selection, presenting plausible neurophysiological locations of this type of priority map (Cisek & Kalaska, 2010; Gallivan et al., 2011; Song et al., 2011). If selection history is encoded in this type of motor-unspecific priority map, it might be expected that biases towards recently selected target features, locations, or objects are encoded independently from action output. While Makovski & Jiang (2011) found evidence that long-term attentional biases are abstracted from action, it is unknown whether short-term biases in selection priority transfer across different response modalities.
In the present study, we address this issue directly in a POP task. POP provides a well-studied model to investigate the role of action history in guiding selection, as target color repetition effects have been demonstrated in keypress, eye movement, and reaching tasks (e.g., Becker, 2008a; Becker, 2008b; Kristjánsson & Campana, 2010; Leonard & Egeth, 2008; Maljkovic & Nakayama, 1994; McPeek, Maljkovic, & Nakayama, 1999; Moher & Song, 2013; Song & Nakayama, 2006). Furthermore, the mechanisms driving POP are debated, and a study on the role of response mode in POP may shed light on competing theories. Maljkovic and Nakayama (1994) initially proposed that POP effects were driven by an enhanced representation of the target-defining feature only. However, others have argued that POP can operate via representations of whole objects (Kristjánsson, Ingvarsdottir, & Teitsdottir, 2008) or events (Hillstrom, 2000; Huang et al., 2004). Action history might plausibly be represented in a higher-level episodic memory trace of the previous event, but if POP effects are driven only by enhancement of the target-defining feature, POP effects should be independent from response mode.
We asked observers to either reach to a color singleton target, or press a key to indicate which side of a color singleton target was chipped. We used a discrimination rather than detection task for the keypress condition in an effort to approximately equate the attentional requirements of the two tasks. Previous studies have shown that subtle target feature discrimination in a popout task requires a shift of focal attention (e.g., Bravo & Nakayama, 1992; Maljkovic & Nakayama, 1994), and selection of a popout target for saccades or reaches also similarly requires a shift of focal attention (e.g., Song & Nakayama, 2006; Song & Nakayama, 2007a). Song & Nakayama (2007a) also directly compared these two response types in a popout task, and found that requiring a subtle perceptual discrimination or requiring goal-directed saccades or reaches leads to equivalent results.
Target color and response mode varied randomly from trial-to-trial. If the mode of action is bound together with the representation of the target feature, possibly in some type of event file (e.g., Hommel, 2004) in which the previous trial event is represented (e.g., Hillstrom, 2000), we might expect an effect of color repetition to occur only when the response mode is also repeated. This represents the action-dependent priming hypothesis. This hypothesis predicts that dependent variables for both action responses (i.e., response initiation latency, reach curvature) and keypress responses (i.e., response time) will be affected by repetition of the target color only when the response mode is repeated; when the response mode is not repeated, there should be no effect of target color.
However, if the target-defining feature is represented independently from any action, similar to the original proposal of Maljkovic & Nakayama (1994), then we expect priming effects to transfer completely across response mode. This represents the action-independent priming hypothesis. According to this hypothesis, we expect that repeating the target color will speed responses and/or reduce reach curvature regardless of the response mode used on the previous trial.
Experiment 1
In Experiment 1, we varied both the target color and the response mode randomly on a trial-by-trial basis. We investigated whether repeating the target color and/or response mode from one trial to the next affected both keypress and reach movement responses. These data allowed us to assess whether actions and target features are encoded in memory, and how those representations in memory might interact and influence subsequent behavior.
Method
Participants
20 Brown University undergraduate students (10 Male, mean age = 21.6 years) participated for course credit. All participants were right handed with normal or corrected-to-normal color vision. The protocol was approved by the Brown University Institutional Review Board.
Apparatus
Similar methods including apparatus, stimuli, and data analysis procedures have been implemented in our previous work (Moher & Song, 2013). Stimuli were projected from behind a plexiglass display that was arranged upright on a table perpendicular to the observer’s line of vision, facing the seated observer at a distance of approximately 48 cm. Three-dimensional hand position was recorded at a rate of approximately 160 Hz using an electromagnetic position and orientation recording system (Liberty, Polhemus) with a measuring error of .03 cm root mean square. A motion tracking marker was secured with a Velcro strap near the tip of each observer’s right index finger. The observer’s index finger was rested on a Styrofoam block placed in front of them on the table, located 27 cm from the screen along the z-dimension (i.e., the axis that is bounded by the observer and the display). The finger was aligned with the bottom of the display along the y-dimension (i.e. the axis that is bounded by the top and bottom of the display), and the horizontal midline of the display along the x-dimension (i.e. the axis that is bounded by the left and right sides of the display). A standard USB keyboard was used to record button press responses. The observer’s left hand was placed on the keyboard, located between the observer and the display (Figure 1A). Stimulus presentation was conducted using custom software designed with MATLAB (Mathworks) and Psychtoolbox (Brainard, 1997).
Stimuli
All stimuli appeared on a black background. A fixation cross appeared at the center of the screen at the beginning of each trial, with a width and length of 0.7 cm. Three diamonds appeared on the screen during each trial. The diameter of each diamond was 3 cm. Each diamond had a “chip” on either the top or the bottom (Figure 1) meaning that the top or bottom edge was a flat edge (1 cm) instead of pointed. The three diamonds were equally spaced, and placed at 4, 8, and 12 o’clock on an imaginary circle surrounding fixation with a radius of 11 cm, with an inter-item distance of 18.9 cm (measured from center to center). The diamonds were rendered in either red or green. The two colors were equiluminant using photometer calibration. On each trial, one diamond appeared in the randomly selected target color and the remaining diamonds were rendered in the other color. The target diamond was randomly selected to one of the three locations described above on each trial, and the distractor diamonds were placed at the remaining two locations.
Cues preceding each trial indicated the mode of response required on the upcoming trial (Figure 1B). Cues were either an image of a keyboard (from http://www.awicons.com/, 1 cm vertically and 5.8 cm horizontally), or an image of a hand (from http://www.clker.com/, 2.4 cm vertically and 2 cm horizontally). Those same images were used with a black “X” superimposed as a warning when observed initiated the wrong response.
Procedure
Nine-point calibration was conducted with the hand tracking system at the beginning of the session. Participants were instructed to keep the index finger on their right hand in the starting position and their left hand on the “z” and “x” keys until the target stimuli appeared. Each trial began with the presentation of the response cue (an image of either a keyboard or a hand), informing participants whether they should use their right hand to reach (reach trial) or their left hand with the keyboard (keyboard trial). After two seconds, the three diamonds appeared (Figure 1B). On touch trials, participants were instructed to point to the uniquely colored diamond with their right index finger as quickly as possible. On keyboard trials, participants were instructed to press the “z” key with their left hand if the top of the uniquely colored diamond was chipped, and the “x” key with their right hand if the bottom of the uniquely colored diamond was chipped.
If the participant moved the right hand on a keyboard trial or pressed a key on a touch trial, they were presented with a warning image, and the trial was counted as incorrect. If the participant did not respond within 1.5 seconds, the trial was counted as incorrect. The display remained on the screen for an additional 200 ms after the participant’s response to encourage participants to rest briefly on the target. This allowed us to obtain complete speed profiles for each trial. Participants were given auditory feedback following every trial – a high beep for accurate responses, low beep for inaccurate responses, and two low beeps if the time limit elapsed before a response was executed. There was a 1 second intertrial interval.
The experiment began with 50 practice trials, followed by 8 blocks of 50 trials each. Participants were given an opportunity to rest between each block. Each session lasted approximately one hour.
Data analysis
When the participant’s finger came within approximately 1.3 cm of the display on the z-dimension and simultaneously within approximately 2 cm of the target diamond on the x and y dimensions within the one-second time limit, a response was considered correct. If this threshold was passed for a non-target diamond, or the participant did not cross any reaching threshold during the time limit, the trial was counted as incorrect.
Hand movement data were analyzed offline using custom MATLAB (Mathworks) software. Three-dimensional resultant speed scalars were created for each trial using a differentiation procedure in MATLAB. These scalars were then submitted to a 2nd order, low-pass Butterworth filter with a cutoff of 10 Hz. Movement onset was calculated as the first time point on each trial after stimulus onset at which hand movement speed exceeded 25.4 cm/s. Movement offset was defined as the first subsequent measurement on each trial when speed decreased to below 25.4 cm/s. Each individual trial was visually inspected (e.g., Song & Nakayama, 2006, 2007a, 2007b 2008); for trials where the default threshold clearly missed part of the movement or included substantial movement back to the starting point, thresholds were adjusted manually to more appropriate levels for that trial (0.58% of all trials). Initiation latency was defined as the time elapsed between stimulus onset and movement onset. Movement time was defined as the time elapsed between movement onset and movement offset.
Trajectories for calculating curvature were measured in two-dimensional XY space by calculating a line from the start to the end point of the movement, and measuring the orthogonal deviation of the actual movement from that line at each sample throughout the movement. Curvature was defined as the maximum point of deviation (unsigned) in centimeters divided by the length of the line from the start to the end points of the movement in centimeters (see e.g., Desmurget, Jordan, Prablanc, & Jeannerod, 21997; Moher & Song, 2013; Song & Nakayama, 2006). For keyboard trials, response time and accuracy of keyboard responses were recorded and analyzed using custom MATLAB software.
We conducted 2×2 ANOVAs with factors of target color (repeat vs. switch) and response mode (repeat vs. switch) on measures of error rate and response time for button press responses, and error rate, curvature, initiation latency, and movement time for hand movement responses (see Table 1). All direct comparisons involve measures from the same response mode of the present trials; in other words, responses on keypress trials were compared across responses on keypress trials, whereas responses on reach movement trials were compared across responses on reach movement trials. Two participants were removed from analysis for poor performance on the keypress task (accurate response executed on fewer than 50% of all keypress trials).
Table 1.
Dependent Variable | Response Mode | Color | |
---|---|---|---|
Repeat | Switch | ||
Initiation latency | |||
Repeat | 481 ± 21 ms | 507 ± 23 ms | |
Switch | 495 ± 22 ms | 506 ± 23 ms | |
Movement time | |||
Repeat | 450 ± 9 ms | 453 ± 10 ms | |
Switch | 456 ± 8 ms | 454 ± 10 ms | |
Curvature | |||
Repeat | 0.088 ± .006 | 0.097 ± .007 | |
Switch | 0.096 ± .006 | 0.102 ± .008 | |
Reach error rate | |||
Repeat | 1.7 ± 0.9 % | 1.8 ± 1.0% | |
Switch | 2.5 ± 1.0% | 3.3 ± 1.0% | |
Keypress response time | |||
Repeat | 847 ± 27 ms | 929 ± 31 ms | |
Switch | 885 ± 30 ms | 913 ± 33 ms | |
Keypress error rate | |||
Repeat | 7.2 ± 1.5 % | 7.8 ± 1.9% | |
Switch | 10.1 ± 2.8% | 10.2 ± 2.7% |
Note: Error terms reflect standard error of the mean (s.e.m.)
Because color was the target-defining feature in this task, we expected target color repetition effects to be more robust and consistent than location repetition effects. In addition, we did not necessarily expect location repetition effects for reaching movements; for example, a previous eye movement study found inhibition of return, rather than priming of popout, when the target location was repeated (e.g., Bichot & Schall, 2002). Furthermore, we found no consistent three-way interactions between target color, target location, and response mode for any dependent measure in separate 2×2×2 ANOVAs conducted for each experiment (for each dependent measure, p > .1 for the three-way interaction in at least two of the three experiments). Thus, we primarily focused our analysis on color repetition effects. Nevertheless, at the end of the results section of Experiment 3, we provide a brief summary of all location repetition effects that were found consistently across all three experiments.
For all analyses, we removed all trials in which participants used the wrong response mode on the current or previous trial, reach trials where a large number of movement samples were dropped due to computer error or no movement was initiated, trials where the previous response was inaccurate, and the first trial of each block (17.66% of all trials). In addition, using a recursive trimming procedure (e.g., Van Selst & Jolicoeur, 1994), we removed any trial that was considered an outlier in each experimental condition along measures of initiation latency, movement time, curvature, or keypress RT. This eliminated 1.6% of all trials.
Results
There were two independently manipulated factors of interest in the current study: target color (repeat vs. switch), and response modality (repeat vs. switch; see Figure 1C). If the target feature and response mode from a single trial are encoded in memory and bias subsequent responses, we would expect both factors to affect performance on subsequent trials. Furthermore, if these two representations in memory are linked together, we might expect an interaction between these two factors.
Keypress responses
We found that overall performance was high and there were no main effects or interactions for keypress error rate (ps > .1). The rest of the section focuses on RT.
First, we address the role of response mode repetition. If there was an effect of response mode repetition, it would suggest that the response mode of the previous trial was encoded in memory and directly affected subsequent behavior. A cost in switching from one response mode to another (e.g., the previous trial required a reach movement response and the current trial requires a keypress response) might be expected, given the frequent and varied costs associated with switching reported in the literature (e.g., Garavan, 1998; Rogers & Monsell, 1995). Although RT was longer when the response mode switched overall (899 ms vs. 888 ms), this effect did not reach significance, F(1,17) < 1. Thus, there was no switch cost associated with switching from a reach movement to a keypress response. This lack of a significant switch cost here (and inconsistent effects in Experiments 2 and 3) may be attributable to the long delay between the onset of the cue and the onset of the display of 2 seconds; previous studies have shown a “preparation effect” whereby observers can overcome initial task-switching costs with sufficient time between the task cue and stimulus onset (e.g., Monsell, 2003; Koch, 2001).
Next, we examine the effect of target color repetition. Previous studies have suggested that repeating the target color in a popout search task results in faster RT (e.g., Maljkovic & Nakayama, 1994). Consistent with these results, we found shorter response times when the target color was repeated (866 ms vs. 921 ms), F(1,17) = 51.36, p < .001. This suggests that the target color was encoded in memory and did bias subsequent responses.
Finally, we assessed whether the target color repetition effects were influenced by whether the response mode from the previous trial was repeated. This analysis allowed us to address the two competing hypothesis noted in the introduction: the action-independent and the action-dependent priming hypotheses. The action-dependent hypothesis predicts an interaction between these two factors, with a benefit to repeating target color present only when the response mode is also repeated, since the representation of the target color would be tightly bound to its associated action. The action-independent hypothesis, on the other hand, predicts a POP effect will occur regardless of whether the response mode is repeated, since the target would be represented independently from its associated action.
We did indeed find a significant interaction, F(1,17) = 12.52, p < .01 (Figure 2), indicating that the magnitude of POP was affected by whether the response mode was repeated from the previous trial. To further examine this interaction effect, we next conducted simple main effects analyses to determine whether the POP effect was significant only when the response mode was repeated. Instead, we found that there was a significant reduction in response time when the target color was repeated when the response mode was repeated (847 vs. 929 ms), F(1,17) = 56.93, p < .001, and when the target color was repeated and the response mode was switched (885 ms vs. 913 ms), F(1,17) = 6.64, p < .05. Thus, the interaction does not reflect an absence of the POP effect when the response mode is switched, but instead reflects a reduction in the magnitude of the POP effect when the response mode was switched.
Together, the data are most consistent with the action-independent hypothesis. However, it is notable that the magnitude of POP was reduced when the response mode switched, suggesting some degree of dependence; this is discussed in greater detail in the General Discussion. Nevertheless, the key finding of these data is that POP can transfer across different response modalities.
Reach movement responses
To assess the effect of both response mode repetition and target color repetition on the spatial and temporal characteristics of reach movements, we next conducted ANOVAs on reach movement variables, including reach accuracy, initiation latency, movement time, and reach curvature.
Reach target selection errors were more frequent when the previous trial required a keypress response (2.9%) rather than a reach response (1.8%), F(1,17) = 6.74, p < .05, and curvature towards a distractor was also greater following a keypress response (.099 vs. .093 ms), an effect that approached significance, F(1,17) = 4.24, p < .06. These data suggest there was some moderate switch cost associated with shifting from a keypress response to a reach movement. There was no effect of response mode on initiation latency or movement time, ps > .1.
Next, we addressed the main effect of target color repetition. Previous studies have found that repeating the target color can reduce the time to respond to the target as well as reach curvature in a visually-guided reaching POP task (e.g., Song & Nakayama, 2006). We found that initiation latency was faster when the target color was repeated (488 ms) than when it was not (507 ms), F(1,17) = 19.29, p < .001, suggesting that the target color was encoded and affected subsequent reach movements. Curvature was also modestly reduced when the target color was repeated (.092 vs. .100), an effect that approached significance, F(1,17) = 3.95, < .07. There was no effect of target color on reach error rate or movement time, ps > .1.
Unlike the keypress responses, there was no interaction between color repetition and response mode for any measures, ps > .1. Again, this demonstrates that responses were faster when the target color was repeated regardless of whether the previous response mode was also repeated, consistent with the action-independent priming hypothesis. Although the interaction was not significant, it is worth pointing out that for the one measure where there was a significant main effect of color repetition (initial latency), the magnitude of the benefit for repeating target color was much greater when the response mode was repeated (26 ms) than when it was switched (11 ms; see Figure 2).
Summary
Together, the data from both reach movements and keypress responses refute the action-dependent hypothesis and instead favor the action-independent hypothesis. POP did transfer from keypresses to reach movements and vice versa. This outcome indicates that target selection does transfer across response modalities. Thus, the representation of the target feature is not critically bound with its associated action response in memory.
Although the interactions did not reach significance for reach movement responses, the magnitude of repetition benefits was consistently greater when the response mode was repeated than when it was switched (Figure 2). These patterns suggest that while priming does transfer across different response modes, there is still some relationship between representations of target properties and their associated actions. We discuss this in greater detail in the General Discussion.
Experiment 2
In Experiment 1, both types of responses, keypresses and reach movements, required motor output. That is, even a keypress response required a finger movement on every trial. Thus, it is possible that transfer of POP across response mode occurred because an overt response was made to the target on all trial types. In Experiment 2, we changed the keypress task to incorporate “no-go” trials in which a response to the target was withheld. Recent studies have found divergent results regarding whether passive viewing of a target stimulus elicits priming effects following a “no-go” trial (e.g., Kristjánsson, Saevarsson & Driver, 2013; Yashar, Makovski, & Lamy, 2013). However, these studies have used a central cue prior to stimulus onset to indicate no-go trials; the present study differs in that target selection is necessary to determine whether the current trial is a no-go trial because the response type (go vs. no-go) was indicated by a target property. Previous studies have suggested that target selection may be a necessary component to trigger subsequent POP effects (e.g. Goolsby & Suzuki, 2001). The present study is designed to determine whether target selection alone, in the absence of any overt motor response, is sufficient to form representations of target feature that bias subsequent responses regardless of the response mode required.
A second possible concern from Experiment 1 is that eye movements were unmonitored. Therefore, it is possible that observers moved their eyes to the target on every trial, and that POP effects transferred across different response modes only because an eye movement was executed to the target in each case. In Experiment 2, we used an eyetracker to monitor eye fixation and observers were explicitly instructed to maintain fixation. These manipulations were guided in part by Makovski and Jiang’s (2011) study in which they used eye monitoring and no-go trials to address similar concerns in a study on transfer of contextual cueing across different response modalities.
Method
Except where otherwise noted, the methods were identical to Experiment 1.
Participants
34 Brown University undergraduate students (10 Male, mean age = 19.5 years) participated for course credit. All participants were right handed with normal or corrected-to-normal color vision. The protocol was approved by the Brown University Institutional Review Board.
Apparatus
The keyboard and reach tracking apparatus were identical to the previous experiment. In addition, eye position was monitored online with an Eyelink II eyetracker (SR Research). Eye position was recorded at a rate of approximately 160 HZ. Nine-point calibration and validation was conducted at the beginning of each session, and drift correction was performed at the beginning of each block. Drift correction was also conducted during blocks as needed.
Stimuli
Stimuli were identical to Experiment 2 with the following exception. When the participant’s eye went outside the fixation boundary, a picture of a pair of eyes appeared at fixation (from http://www.proprofs.com/, 9 cm vertically and 13.7 cm horizontally), to inform the participant that they broke fixation on that trial.
Procedure
The procedure was identical to Experiment 1, except in the following ways. The correct response on keypress trials in which the target was chipped on the bottom was to withhold a response (i.e. a “no-go” trial). When the target was chipped on top, participants were instructed to press the “z” key with their left hand. After the practice block, participants were instructed to keep their eyes focused on the cue appearing at the center during the trial. If the participant moved their eyes outside of a fixation box measuring 6.8 cm on each side surrounding the center of the cue, a pair of eyes appeared and the trial was terminated. The boundary box was large to minimize the need for drift correction during the block, because a drift correction procedure would increase the intertrial interval, potentially affecting memory-based selection biases. However, to ensure that participants were not executing smaller eye movements towards the target within the boundary box, we examined eye movement data on all non-rejected trials. We defined target-aimed eye movements as instances where eye velocity exceeded 34.7°/s (30 cm/s) and maximum deviation from initial position was at least 2 cm up, left, or right (downward saccades were not of concern, as there was no target at the bottom of the screen). Target-aimed eye movements by this definition occurred on only 4.5% of all non-rejected accurate trials. These data suggest that observers were not frequently making saccades towards the target within the boundary box.
Participants completed up to 8 blocks of trials. Some participants completed fewer than 8 blocks due to fatigue and/or discomfort issues with the eyetracker. The average number of blocks completed was 6.97.
Data analysis
Data analysis was identical to Experiment 2, except for the exclusion criteria described in the results section below. For trials where the default threshold clearly missed part of the movement or included substantial movement back to the starting point, thresholds were adjusted manually to more appropriate levels for that trial (0.26% of all trials).
For all analyses, we removed all trials in which participants used the wrong response mode or moved their eyes on the current or previous trial, reach trials where a large number of movement samples were dropped due to computer error or no movement was initiated, trials where the previous response was inaccurate, and the first trial of each block (34.6% of all trials). As in Experiment 1, we used a recursive trimming procedure (e.g., Van Selst & Jolicoeur, 1994), to remove any trial that was considered an outlier in each experimental condition along measures of initiation latency, movement time, curvature, or keypress RT. This eliminated 1.4% of all trials. Two participants were removed from analysis for poor overall performance on the keypress task (accurate response executed on fewer than 50% of all keypress trials).
Results
Go vs. no-go trials
One concern from Experiment 1 is that POP effects transferred across different response modes only because an overt action response was required for both reach movement and keypress trials. Thus in Experiment 2, the keypress task was changed such that no overt response was required when the bottom of the target was chipped. This means that keypress trials in which the bottom of the target was chipped were classified as “no-go” trials, while trials where the top of the diamond was chipped were classified as “go” trials.
We predicted that color repetition effects would be unaffected by whether the previous trial was a “go” or a “no-go” trial, abating any concerns about POP transfer relying on an overt action response. To test this prediction, we conducted 2×2 ANOVAs with factors of target color (repeat vs. switch) and previous trial type (go vs. no-go) on measures of error rate, curvature, initiation latency, and movement time for reach movement responses and measures of error rate and response time for keypress responses. Because the go vs. no-go manipulation only applied to keypress trials, and not to reach movement trials, this analysis was limited to cases where the previous trial was a keypress trial and not a reach movement trial. Since we were interested only in whether or not the previous trial type (go vs. no-go) interacted with target color repetition effects, we focus only on the results of the interaction between the factors of target color and previous trial type. There were no interactions between previous trial type and target color, Fs < 1, suggesting that the whether the previous trial was a go or no-go response did not impact color repetition effects. This confirms our prediction that color repetition effects were largely unaffected by whether the observer did or did not make an overt response to the target on the preceding trial.
Therefore, we collapsed across go and no-go responses for all subsequent analyses, treating both “go” and “no-go” keypress responses as the same category (keypress responses), though keypress RT data reflect only “go” trials since “no-go” trials do not have a response time measurement. For the main analysis, we conducted 2×2 ANOVAs with factors of target color (repeat vs. switch) and response modality (repeat vs. switch) on measures of error rate, curvature, initiation latency, and movement time for reach movement responses and measures of error rate and response time for keypress responses (see Table 3).
Table 3.
Dependent Variable | Response Mode | Color | |
---|---|---|---|
Repeat | Switch | ||
Initiation latency | |||
Repeat | 437 ± 12 ms | 452 ± 12 ms | |
Switch | 467 ± 14 ms | 478 ± 16 ms | |
Movement time | |||
Repeat | 438 ± 9 ms | 437 ± 9 ms | |
Switch | 431 ± 8 ms | 427 ± 9 ms | |
Curvature | |||
Repeat | 0.091 ± .004 | 0.099 ± .005 | |
Switch | 0.095 ± .004 | 0.096 ± .005 | |
Reach error rate | |||
Repeat | 1.0 ± 0.3 % | 1.1 ± 0.3% | |
Switch | 1.0 ± 0.3% | 1.8 ± 0.5% | |
Keypress response time | |||
Repeat | 764 ± 20 ms | 807 ± 19 ms | |
Switch | 747 ± 21 ms | 767 ± 18 ms | |
Keypress error rate | |||
Repeat | 3.9 ± 1.0 % | 3.1 ± 0.8% | |
Switch | 4.7 ± 0.9% | 8.8 ± 1.5% |
Note: Error terms reflect standard error of the mean (s.e.m.)
Keypress responses
As in Experiment 1, we start by examining what happens when the response mode is switched from one trial to the next. The error rate was reduced when the response mode was repeated (3.5%) rather than when it was switched (6.7%), F(1,31) = 32.52, p < .01. This result reflects a moderate switch cost associated with switching response mode from one trial to the next.
Surprisingly, RT was longer when the response mode was repeated (786 ms) rather than when it was switched (757 ms), F(1,31) = 5.99, p < .05. However, this is likely attributable to the design of the experiment. When looking at keypress data, any trial categorized as a “response mode repeat” trial refers to a keypress trial preceded by a keypress trial; thus, “response mode repeat” trials were preceded by either “go” or “no-go” trials with the equal probability. “Response mode switch” trials, on the other hand, refer to a keypress trial preceded by a reach movement trial; these trial types are never preceded by a “no-go” trial, since there are no “no-go” trials in the reach movement response condition. Although keypress response type on the previous trial (go vs. no-go) did not interact with target color repetition effects, which was our primary interest, overall keypress RT was longer following no-go trials (797 ms) than following go trials (768 ms). This difference likely explains why RT was longer on response mode repeat trials than response mode switch trials for the keypress condition.
Next, we examined the effect of target color repetition to determine whether repeating the target color on consecutive trials resulting in facilitated responses (e.g., Maljkovic & Nakayama, 1994). Participants made fewer response errors (i.e., pressing a key on a “no-go” trial or not pressing a key on a “go” trial) when the target color was repeated (4.3%) than when it was switched (5.9 %), F(1,31) = 4.99, p < .05. Furthermore, RT was shorter when the target color was repeated (756 ms) than when it switched (787 ms), F(1,31) = 14.44, p < .01. These data are consistent with previous POP findings, demonstrating faster and more accurate responses when the target color is repeated on consecutive trials.
Finally, we assessed the interaction between response mode and target color to determine whether POP effects were dependent upon a repetition of the response mode. When the response mode switched, participants made fewer errors when the target color was repeated (4.7%) than when it was switched (8.8%), while when the response mode was repeated, participants actually made slightly more errors when the target color was repeated (3.9%) than when it was switched (3.1%), F(1,31) = 7.22, p < .05. Subsequent simple main effects analyses revealed that the benefit for repeating target color when the response mode was switched was significant, F(1,31) = 8.89, p < .01, but that there was no effect of repeating target color when the response mode was repeated, F(1,31) < 1. The reduction in errors in the response mode switch case demonstrates once again that POP transfers across response mode. The overall error rate was much lower on response mode repeat trials than response mode switch trials, as reported above. Therefore, the lack of POP for response mode repeat trials in the error rate measure may reflect a ceiling effect, whereby task performance was too high to detect a slight difference in errors committed.
As in Experiment 1, there was a greater RT benefit when the response mode was repeated (color repeated: 764 ms, color switched: 807 ms, 43 ms benefit) than when the response mode was switched (color repeated: 747 ms, color switched: 767 ms, 20 ms benefit; see Figure 3). However, the interaction only approached significance, F(1,31) = 2.92, p < .1.
Together, these data again demonstrate that POP transfers across response mode, providing further evidence against the action-dependent hypothesis and further support for the action-independent hypothesis.
Reach movement responses
Next we examine the effect of target color repetition and response mode on reach movement measures including reach error rate, initiation latency, movement time, and reach curvature.
First, we examined the role of response mode repetition to determine whether the response mode required on the previous trial had any impact on reaching behavior on the current trial. Initiation latency was shorter when the response mode was repeated (445 ms) than when it was switched (472 ms), F(1,31) = 13.42, p < .01. Movement time was also shorter when the response mode was repeated (438 ms vs. 429 ms), F(1,31) = 12.85, p < .01. There was no effect of response mode on reach error rate or curvature, Fs > .1. These results are similar to those reported in Experiment 1, though we found here that switching response mode lengthened movement times rather than increasing error rates.
Next, we examined the effect of target color to determine whether POP effects were present on reaching trials in Experiment 2, as they were in Experiment 1 and Song & Nakayama (2006). As in Experiment 1, we found that initiation latency was shorter when the target color was repeated (452 ms) than when it was switched (465 ms), F(1,31) = 13.74, p < .01. Curvature was also marginally reduced when the target color was repeated (.093 vs. .097), an effect that approached significance, F(1,31) = 3.36, p < .08. There were no effects of target color repetition on reach error rate or movement time, ps > .1. These results replicate Experiment 1, demonstrating a POP effect for reaching responses when the target color was repeated.
As in Experiment 1, there were no significant interactions between response mode and target color for all reach movement measures, ps > .1. Also as in Experiment 1, the magnitude of the priming benefit for initiation latency was slightly greater when the response mode was repeated (15 ms) than when it was switched (11 ms; See Figure 3).
In Experiment 2, we controlled for eye movement responses to prevent observers from executing saccades to the target regardless of response type. We also included no-go trials to determine whether priming transfer might occur even when no overt response of any kind is executed. Even with these changes, we still found that target selection biases transferred across different response modes. These data allow us to reject the action-dependent hypothesis, and instead conclude that the representation of a target feature can bias subsequent responses even when the mode of response differs from the previous one. This is consistent with the action-independent hypothesis, suggesting that the target feature is represented independently from its associated action.
Experiment 3
In Experiments 1 and 2, different response types were associated with different hands. Thus, it may be the case that transfer only occurred because different brain hemifields were recruited for different response modes. If the same hand is used for both responses, it is possible that transfer will not occur because of a strong binding between response mode and target stimulus within the response-relevant brain hemifield. Alternatively, transfer may be more complete if the same hand is used to execute both types of response.
We chose the discrimination task in Experiments 1 and 2 specifically because it requires a shift of focal attention, since reach movements to a popout target also require a shift of focal attention (Song & Nakayama, 2006, 2007a). Yet, on the surface level, the task demands differed, in that the keypress task required discrimination of target properties while the reaching task did not. Thus, in Experiment 3, we had participants use their right index finger to respond on both keypress and reach trials, and the task for both trials was similar – execute a response to the target (either a keypress or a reach) only when the top of the target diamond is chipped.
Method
Except where otherwise noted, the methods were identical to Experiment 2.
Participants
15 Brown University undergraduate students (7 Male, mean age = 20.2 years) participated for course credit. All participants were right handed with normal or corrected-to-normal color vision. The protocol was approved by the Brown University Institutional Review Board.
Apparatus
The keyboard and reach tracking apparatus were identical to the previous experiment. However, in Experiment 2, the keyboard was placed directly in front of the participant, and the starting position for reach movement trials was a finger resting on top of the “m” key.
Stimuli
Stimuli were identical to Experiment 2 with the following exception.
Procedure
The procedure was identical to Experiment 2, except in the following ways. On reach movement trials, observers were instructed to only reach to the target when the top of the target diamond was chipped.
Participants completed up to 8 blocks of trials. Some participants completed fewer than 8 blocks due to fatigue and/or discomfort issues with the eyetracker. The average number of blocks completed was 7.27.
Target-aimed eye movements again occurred infrequently, on only 3.4% of all non-rejected accurate trials. This again suggests that observers were not making saccades towards the target within the boundary box.
Data analysis
Data analysis was similar to Experiment 2. For trials where the default threshold clearly missed part of the movement or included substantial movement back to the starting point, thresholds were adjusted manually to more appropriate levels for that trial (0.2% of all trials).
For all analyses, we removed all trials in which participants used the wrong response mode or moved their eyes on the current or previous trial, reach trials where a large number of movement samples were dropped due to computer error or no movement was initiated, trials where the previous response was inaccurate, and the first trial of each block (29.4% of all trials). As in Experiment 1, we used a recursive trimming procedure (e.g., Van Selst & Jolicoeur, 1994), to remove any trial that was considered an outlier in each experimental condition along measures of initiation latency, movement time, curvature, or keypress RT. This eliminated 0.7% of all trials. One participant was removed from analysis for poor overall performance on the keypress task (accurate response executed on fewer than 50% of all keypress trials), and another two were removed because they had no observations in at least one of the cells in one of the ANOVAS.
We conducted 2×2×2 ANOVAs with factors of target color (repeat vs. switch), previous response type (go vs. no-go) and response mode (repeat vs. switch) on measures of error rate and response time for button press responses, and error rate, curvature, initiation latency, and movement time for hand movement responses (see Table 4). All direct comparisons involve measures from the same response mode of the present trials; in other words, responses on keypress trials were compared across responses on keypress trials, whereas responses on reach movement trials were compared across responses on reach movement trials. All data reported are from “go” trials.
Table 4.
Dependent Variable | Previous Response | Response Mode | Color | |
---|---|---|---|---|
Repeat | Switch | |||
Initiation latency | ||||
Go | Repeat | 639 ± 23 ms | 693 ± 25 ms | |
Switch | 664 ± 25 ms | 731 ± 39 ms | ||
No-Go | Repeat | 677 ± 24 ms | 730 ± 33 ms | |
Switch | 694 ± 25 ms | 733 ± 27 ms | ||
Movement Time | ||||
Go | Repeat | 378 ± 14 ms | 371 ± 15 ms | |
Switch | 390 ± 15 ms | 389 ± 16 ms | ||
No-Go | Repeat | 395 ± 18 ms | 382 ± 11 ms | |
Switch | 397 ± 18 ms | 377 ± 17 ms | ||
Curvature | ||||
Go | Repeat | 0.114 ± .007 | 0.120 ± .013 | |
Switch | 0.121 ± .010 | 0.120 ± .012 | ||
No-Go | Repeat | 0.122 ± .007 | 0.110 ± .011 | |
Switch | 0.120 ± .015 | 0.121 ± .011 | ||
Reach error rate | ||||
Go | Repeat | 5.3 ± 1.8% | 4.6 ± 1.9% | |
Switch | 1.8 ± 1.0% | 4.2 ± 2.2% | ||
No-Go | Repeat | 7.0 ± 2.2% | 7.0 ± 2.2% | |
Switch | 4.6 ± 1.5% | 4.1 ± 1.3% | ||
Keypress response time | ||||
Go | Repeat | 769 ± 35 ms | 790 ± 30 ms | |
Switch | 756 ± 28 ms | 822 ± 32 ms | ||
No-Go | Repeat | 818 ± 31 ms | 829 ± 41 ms | |
Switch | 803 ± 39 ms | 809 ± 44 ms | ||
Keypress error rate | ||||
Go | Repeat | 1.1 ± 0.8% | 2.0 ± 1.3% | |
Switch | 3.2 ± 1.5% | 1.7 ± 0.8% | ||
No-Go | Repeat | 2.4 ± 1.1 % | 3.5 ± 1.7% | |
Switch | 2.3 ± 1.4% | 2.9 ± 1.1% |
Note: Error terms reflect standard error of the mean (s.e.m.)
Results
The full data set can be seen in table 4.
Keypress responses
RT was longer following no-go responses (815 ms) compared to go responses (784 ms), F(1,11) = 5.77, p < .05. There were no other previous trial type main effects or interactions for keypresses. As in Experiment 1, we found no effect of response mode on RT or accuracy, Fs < 1.
As in the first two experiments, target selection biases transferred across response modalities, even when no overt response was executed on the previous trial. RT was shorter when the target color was repeated (787 ms) than when it was switched (812 ms), F(1,11) = 5.91, p < .05 (see Figure 4). There was no interaction with response mode, previous response type, and no three-way interaction, Fs < 1. This result suggests that the selection bias observed in the first two experiments is not limited to cases where a different hand and/or different task is used for different response modes. Instead, in the present experiment, the RT benefit was actually slightly larger in magnitude when the response mode switched (color repeated: 780 ms, color switched: 815 ms, 35 ms benefit) than when the response mode was repeated (color repeated: 794 ms, color switched: 809 ms, 15 ms benefit). This suggests that the reduction in POP transfer in the first experiment may have been due in part to differences in the task and/or hand used for different response modalities; we explore this idea further in the General Discussion. There were no main effects of target color or interactions between target color and other factors for other reach movement measures.
Reach movement responses
Initial latency was shorter following go trials (682 ms) relative to no-go trials (709 ms), F(1,11) = 13.59, p < .01. Movement time was 15 ms shorter following go trials compared to no-go trials when the response mode was repeated, but 2 ms longer following go trials when the response mode switched, an effect that approached significance, F(1,11) = 3.78, p < .08. There were no other main effects or interactions for previous trial type for reach movements, ps > .1.
Although the error rate difference was small, error rates were slightly higher when the response mode was repeated (5.9%) than when it was switched (3.7%), F(1,11) = 14.02, p < .01. This pattern was not found in the first two experiments; however, a previous eye movement did find a modest task-switch benefit with a two-second preparation interval (Barton et al., 2006).
Again, initiation latency was shorter when the target color was repeated (669 ms) than when it was switched (722 ms), F(1,11) = 9.18, p < .05 (see Figure 4). There was no interaction with response mode, previous response type, and no three-way interaction for initiation latency, Fs < 1. The magnitude of the benefit was almost identical when the response mode was repeated (54 ms) as when it was switched (53 ms). Thus, target selection biases transferred across response modes, even when no overt action was required on the previous trial.
We found a small effect of target color repetition on movement time, with longer movement times when the target color was repeated (390 ms) than when it was switched (380 ms), F(1,11) = 5.72, p < .05. The movement time effect may be driven by the much faster latencies in the target color repeated condition, as rapidly initiated movements often exhibit longer movement times (e.g., Song & Nakayama, 2007; 2008). Still, the magnitude of the initiation latency (53 ms) was more than 5 times larger, and thus target color repetition sped overall responses. There were no main effects of target color or interactions between target color and other factors for other reach movement measures.
Again, these data suggest that target selection biases transfer across response mode even when the tasks are similar and the same hand is used. In fact, by looking at the magnitude of response time benefits, transfer appears more complete here than in the first two experiments. Possible explanations for the reduction in magnitude in the first two experiments are discussed further in the General Discussion.
Target location repetition effects
Across all three experiments, we examined target location repetition effects by conducting separate ANOVAs with factors of target color repetition and response mode (and previous trial type in Experiment 3 only). There was a consistent RT benefit to repeating the target location across all three experiments for keypresses, with shorter RT when the location was repeated (Exp. 1, 867, Exp. 756 ms, Exp. 3, 773 ms) than when it was switched (Exp. 1, 905, Exp. 2, 777 ms, Exp. 3, 814 ms), ps < .05. This is consistent with previous keypress studies showing that repetition of target location speeds responses (e.g., Maljkovic & Nakayama, 1996). In all three experiments, there was no interaction between location repetition and response mode, ps > .1. Thus, repetition of target location speeds keypress responses, regardless of the response mode on the previous trial, again consistent with the action-independent hypothesis. However, in all three experiments, the magnitude of the location repetition benefit was larger when the response mode was repeated (Exp. 1, 52 ms, Exp. 2, 33 ms, Exp. 3, 59 ms) than when the response mode switched (Exp. 1, 23 ms, Exp. 2, 9 ms, Exp. 3, 22 ms). This pattern is similar to the patterns of color repetition effects on both reach movements and keypress responses in Experiments 1 and 2, suggesting the possibility that there is some relationship in memory between the location and the associated action (see General Discussion). There were no other consistent location repetition effects found across the three experiments.
General Discussion
Numerous studies have demonstrated that the visual characteristics of a scene bias target selection, resulting in robust priming effects when those characteristics are repeated. Here, we examined the role of action history in target selection biases by requiring participants to engage in different types of action responses to a popout target. In Experiment 1, we found evidence to suggest that repeating the target color sped responses regardless of whether the response mode was repeated from the previous trial. These data directly refute the action-dependent priming hypothesis, which stated that the representation of a recently selected target feature is critically bound to its associated action in memory and thus produces priming effects only when both the response mode and target color are repeated. Instead, the data demonstrate that POP transfers across different response modes. This is consistent with the action-independent priming hypothesis, and with Maljkovic and Nakayama’s (1994) claim that POP is driven by a representation of the target feature itself, which is also supported by a computational model (Strauss & Heinke, 2012)
In Experiment 2, we found a similar pattern of results even as observers maintained eye fixation. This refutes a possible alternative interpretation of Experiment 1 in which POP only transfers across response modes because observers move their eyes to the target regardless of response. Furthermore, POP effects following keypress trials were not affected by whether an overt action was required (a “go” trial) or not (a “no-go” trial) on the previous trial. Thus, target features are encoded and bias target selection on future trials even when there is no overt action taken towards that target. In Experiment 3, we found even more robust transfer of priming effects when responses were executed with the same hand for both response modes and the task was similar for both response modes. This refutes a possible concern that there is a tight binding between action and response, but priming transfer was possible in Experiments 1 and 2 because the two response modes were easily distinguishable due to the use of separate hands and separate tasks. Across all three experiments, we found location repetition benefits for keypress responses regardless of the previous response mode, again consistent with action-independent target selection biases.
Previous neurophysiological research has shown that no-go trials may activate components of the motor system, which are then inhibited by frontocortical mechanisms before the action is executed (e.g., Band & Boxtel, 1999; Miller, Riehle, & Rehquin, 1992). Thus, it is possible that the “no-go” response may be associated with a particular type of action, even if that action is not overtly executed. However, the present results demonstrate that target selection biases formed on no-go trials can transfer across different modes of responding; thus, after an observer withholds a keypress response to a target stimulus, performance will be improved for a reach movement to a target stimulus matching that color or location on a subsequent trial, and vice versa. This suggests that not only are target selection biases formed even when no overt response is required, but also that these biases are not critically bound to the withheld action with which they are associated.
Action representations in memory
The magnitude of priming benefits for color repetition was consistently reduced when the response mode switched, relative to when the response mode was repeated, in the first two experiments (see Figures 2 and 3). This pattern was observed for location repetition effects on keypress responses across all three experiments. This suggests that in addition to the representation of the target feature, there may also exist either a separate representation of the entire event that can bias selection (e.g., Hillstrom, 2000), or a separate representation of the previous action itself that is co-activated with the representation of the target feature when both are repeated. In other words, while POP effects do transfer across response modes, it might be the case that the feature representation is more robustly activated when the response mode is also repeated, suggesting some relationship between target feature and action response.
This reduction in the magnitude of POP effects may be attributable to the ideomotor principle (e.g., Greenwald, 1970; see Shin, Proctor, & Capaldi, 2010, for a review). In particular, the idea of “event files” (e.g., Hommel, 1998; 2004; Hommel, Müsseler, Aschersleben, & Prinz, 2001) may provide a framework for understanding transfer of target selection biases. An event file refers to a bound representation in memory of a perceptual event, task context, and associated action response. When a familiar perceptual event is encountered in its associated task context, the associated action from the event file is activated. The notion of event files is consistent with the reduction in POP effects observed in the current study when response mode was switched, though these conclusions should be interpreted with caution, as those reductions were not always statistically significant. However, the fact that POP still occurred even when response mode switched, albeit at a smaller magnitude, suggests that target features may be represented in memory separate from event files, or that bindings within event files may not be as strong as previously thought.
In a previous experiment, Makovski and Jiang (2011) reported transfer of attentional bias across response modalities in a contextual cueing task (Makovski & Jiang, 2011). The present study builds upon that finding in a POP task, demonstrating that short-term biases can also transfer across response modalities. Furthermore, contextual cueing reflects an attentional bias that is based upon task-relevant information; that is, observers learn that particular element configurations predict the location of the target. In a POP task, the color of the target on one trial is entirely uncorrelated with the color on the next trial. Thus, POP effects reflect a bias that is not based on any task-relevant information, and therefore represents a distinct category of selection bias that is automatic rather than task-driven.
One possible explanation for reduction in the magnitude of priming effects in the present study, but no reduction in the transfer of contextual cueing from one response mode to the other (Makovski & Jiang, 2011), is that short-term attentional biases include a representation of motor actions, possibly within an event file, but that these action representations are not encoded in long-term memory-based attentional biases. Previous studies have investigated the shift of attentional templates from working memory to long-term memory (Carlisle, Arita, Pardo, & Woodman, 2011) and suggested that attentional control settings and strategies may shift when long-term mechanisms are involved (Moher et al., in press). These studies provide a plausible framework for this claim, but further studies are needed to fully understand the role of action representation as attentional biases shift from short-term to long-term memory.
There could be alternative explanations for the reduction in the magnitude of POP effects when the response mode switched. For example, it might be the case that the magnitude of POP was reduced because the response hand was switched from one trial to the next, and not because the response mode was switched. In Experiment 3, where the response hand stayed consistent, there was more complete transfer from one response mode to the other (although still there were reductions in the magnitude of the POP effect for location repetitions in keypress responses). It is possible that the use of separate hands allowed for representations binding response mode and target features to be formed, while using the same hand for both response modes blocked formation of these representations. Further experiments are needed to tease apart these alternatives and better understand the role of action in trial history effects on target selection. Nevertheless, the current results clearly demonstrate that target selection biases do transfer across different response modes. Thus, there is some representation of previously selected target features in memory that is not critically bound to its associated action.
Integrated attention-action systems
In recent years, there has been an increased focus on the role of the motor system in target selection. For example, a number of studies have examined how trial history and distractor salience affect target selection for action (Song & Nakayama, 2006; 2007b), and how motor output can be used to measure competition for selection (Song & Nakayama, 2009; Spivey et al., 2005; Spivey & Dale, 2006; Welsh & Elliott, 2005). Action outputs themselves can even bias future target selection (e.g., Moher & Song, 2013). Furthermore, brain areas previously associated only with motor output, such as frontal eye fields and superior colliculus, have been shown to play a role in effector-independent target selection (Gallivan et al., 2011; Song et al., 2011).
Along with the present results, the studies described above demonstrate the need to account for the motor system in models of attentional selection. To understand how target selection is influenced by recent experience, it would be useful to consider the role of action history and its interaction with more traditional components involved in selection history effects that consider the physical characteristics of the visual environment (Chun & Jiang, 1998; Maljkovic & Nakayama, 1994; 1996; Wolfe et al., 2003). This is particularly important because behavior in the real world often requires an observer to engage in varying modes of interaction (e.g., reaching, looking, grasping, moving) with objects in their surrounding environment. In the present study, we found that POP effects did not depend on repeated response modes; however, other trial history effects might be more dependent on action history. The paradigm described in the present studies may offer a useful basis for future experiments to further investigate the role of the motor system in selection history effects.
Conclusions
The data reported here demonstrate that a short-term target selection bias, priming of popout, transfers across different response modalities. Specifically, repeating the target color biased subsequent performance, even when the response mode (keypress or reach movement) switched from one trial to the next. This suggests that a high-level representation of the target feature is encoded and biases target selection separately from its associated action.
Table 2.
Dependent Variable | Previous Response Type | Color | |
---|---|---|---|
Repeat | Switch | ||
Initiation latency | |||
Go | 459 ± 16 ms | 466 ± 17 ms | |
No-Go | 483 ± 14 ms | 497 ± 15 ms | |
Movement time | |||
Go | 441 ± 8 ms | 437 ± 9 ms | |
No-Go | 438 ± 9 ms | 439 ± 9 ms | |
Curvature | |||
Go | 0.098 ± .004 | 0.099 ± .008 | |
No-Go | 0.096 ± .007 | 0.095 ± .005 | |
Reach error rate | |||
Go | 1.1 ± 0.5% | 1.8 ± 0.7% | |
No-Go | 1.3 ± 0.5% | 1.4 ± 0.6% | |
Keypress response time | |||
Go | 750 ± 23 ms | 787 ± 21 ms | |
No-Go | 779 ± 19 ms | 815 ± 22 ms | |
Keypress error rate | |||
Go | 4.2 ± 1.3% | 3.6 ± 1.1% | |
No-Go | 4.1 ± 1.7% | 2.6 ± 0.8% |
Note: Error terms reflect standard error of the mean (s.e.m.)
Acknowledgments
This project is supported by Brown University Salomon faculty research award and NIGMS-NIH (P20GM103645) to J.H.S, and the Center for Vision Research fellowship and the Brown Training Program in Systems and Behavioral Neuroscience NIH T32MH019118 to J.M. We would also like to thank Susanne Ferber, Dietmar Heinke, and an anonymous reviewer for helpful comments.
References
- Awh E, Belopolsky AV, Theeuwes J. Top-down versus bottom-up attentional control: a failed theoretical dichotomy. Trends in Cognitive Sciences. 2012;16(8):437–443. doi: 10.1016/j.tics.2012.06.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Band G, Van Boxtel G. Inhibitory motor control in stop paradigms: review and reinterpretation of neural mechanisms. Acta Psychologica. 1999;101(2):179–211. doi: 10.1016/s0001-6918(99)00005-0. [DOI] [PubMed] [Google Scholar]
- Barton JJS, Greenzang C, Hefter R, Edelman J, Manoach DS. Switching, plasticity, and prediction in a saccadic task-switch paradigm. Experimental Brain Research. 2005;168(1–2):76–87. doi: 10.1007/s00221-005-0091-1. [DOI] [PubMed] [Google Scholar]
- Becker SI. The mechanism of priming: Episodic retrieval or priming of pop-out? Acta Psychologica. 2008a;127(2):324–339. doi: 10.1016/j.actpsy.2007.07.005. [DOI] [PubMed] [Google Scholar]
- Becker SI. The stage of priming: Are intertrial repetition effects attentional or decisional? Vision Research. 2008b;48(5):664–684. doi: 10.1016/j.visres.2007.10.025. [DOI] [PubMed] [Google Scholar]
- Bichot N, Schall J. Priming in macaque frontal cortex during popout visual search: Feature-based facilitation and location-based inhibition of return. Journal of Neuroscience. 2002;22(11):4675–4685. doi: 10.1523/JNEUROSCI.22-11-04675.2002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bravo M, Nakayama K. The role of attention in different visual search tasks. Perception & Psychophysics. 1992;51(5):465–472. doi: 10.3758/bf03211642. [DOI] [PubMed] [Google Scholar]
- Carlisle NB, Arita JT, Pardo D, Woodman GF. Attentional templates in visual working memory. Journal of Neuroscience. 2011;31(25):9315–9322. doi: 10.1523/JNEUROSCI.1097-11.2011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Chun MM, Jiang Y. Contextual cueing: implicit learning and memory of visual context guides spatial attention. Cognitive Psychology. 1998;36(1):28–71. doi: 10.1006/cogp.1998.0681. [DOI] [PubMed] [Google Scholar]
- Chun M, Nakayama K. On the functional role of implicit visual memory for the adaptive deployment of attention across scenes. Visual Cognition. 2000;7:65–81. [Google Scholar]
- Cisek P. Cortical mechanisms of action selection: the affordance competition hypothesis. Philosophical Transactions of the Royal Society B: Biological Sciences. 2007;362(1485):1585–1599. doi: 10.1098/rstb.2007.2054. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cisek P, Kalaska JF. Neural Mechanisms for Interacting with a World Full of Action Choices. Annual Review of Neuroscience. 2010;33(1):269–298. doi: 10.1146/annurev.neuro.051508.135409. [DOI] [PubMed] [Google Scholar]
- Egeth HE, Virzi RA, Garbart H. Searching for conjunctively defined targets. Journal of Experimental Psychology: Human Perception & Performance. 1984;10(1):32–39. doi: 10.1037//0096-1523.10.1.32. [DOI] [PubMed] [Google Scholar]
- Gallivan JP, McLean DA, Smith FW, Culham JC. Decoding effector-dependent and effector-independent movement intentions from human parieto-frontal brain activity. Journal of Neuroscience. 2011;31(47):17149–17168. doi: 10.1523/JNEUROSCI.1058-11.2011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Garavan H. Serial attention within working memory. Memory & Cognition. 1998;26(2):263–276. doi: 10.3758/bf03201138. [DOI] [PubMed] [Google Scholar]
- Goolsby B, Suzuki S. Understanding priming of color-singleton search: Roles of attention at encoding and “retrieval. Perception & Psychophysics. 2001;63(6):929–944. doi: 10.3758/bf03194513. [DOI] [PubMed] [Google Scholar]
- Greenwald AG. Sensory feedback mechanisms in performance control: with special reference to the ideo-motor mechanism. Psychological Review. 1970;77(2):73–99. doi: 10.1037/h0028689. [DOI] [PubMed] [Google Scholar]
- Hillstrom A. Repetition effects in visual search. Perception & Psychophysics. 2000;62(4):800–817. doi: 10.3758/bf03206924. [DOI] [PubMed] [Google Scholar]
- Hommel B. Event files: Evidence for automatic integration of stimulus-response episodes. Visual Cognition. 1998;5(1–2):183–216. [Google Scholar]
- Hommel B. Event files: feature binding in and across perception and action. Trends in Cognitive Sciences. 2004;8(11):494–500. doi: 10.1016/j.tics.2004.08.007. [DOI] [PubMed] [Google Scholar]
- Hommel B, Müsseler J, Aschersleben G, Prinz W. The theory of event coding (TEC): A framework for perception and action planning. Behavioral and brain sciences. 2001;24(5):849–878. doi: 10.1017/s0140525x01000103. [DOI] [PubMed] [Google Scholar]
- Huang L, Holcombe AO, Pashler H. Repetition priming in visual search: Episodic retrieval, not feature priming. Memory & Cognition. 2004;32(1):12–20. doi: 10.3758/bf03195816. [DOI] [PubMed] [Google Scholar]
- Jax SA, Rosenbaum DA. Hand path priming in manual obstacle avoidance: Evidence that the dorsal stream does not only control visually guided actions in real time. Journal of Experimental Psychology: Human Perception & Performance. 2007;33(2):425–441. doi: 10.1037/0096-1523.33.2.425. [DOI] [PubMed] [Google Scholar]
- Jax SA, Rosenbaum DA. Hand path priming in manual obstacle avoidance: Rapid decay of dorsal stream information. Neuropsychologia. 2009;47(6):1573–1577. doi: 10.1016/j.neuropsychologia.2008.05.019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Koch I. Automatic and intentional activation of task sets. Journal of Experimental Psychology: Learning, Memory, and Cognition. 2001;27(6):1474–1486. doi: 10.1037//0278-7393.27.6.1474. [DOI] [PubMed] [Google Scholar]
- Kristjánsson Á, Campana G. Where perception meets memory: A review of repetition priming in visual search tasks. Attention, Perception & Psychophysics. 2010;72(1):5–18. doi: 10.3758/APP.72.1.5. [DOI] [PubMed] [Google Scholar]
- Kristjánsson A, Ingvarsdottir A, Teitsdottir UD. Object- and feature-based priming in visual search. Psychonomic Bulletin & Review. 2008;15(2):378–384. doi: 10.3758/pbr.15.2.378. [DOI] [PubMed] [Google Scholar]
- Kristjánsson Á, Saevarsson S, Driver J. The boundary conditions of priming of visual search: From passive viewing through task-relevant working memory load. Psychonomic Bulletin & Review. 2013;20(3):514–521. doi: 10.3758/s13423-013-0375-6. [DOI] [PubMed] [Google Scholar]
- Lamy DF, Kristjánsson A. Is goal-directed attentional guidance just intertrial priming? A review. Journal of Vision. 2013;13(3):1–19. doi: 10.1167/13.3.14. [DOI] [PubMed] [Google Scholar]
- Leonard C, Egeth H. Attentional guidance in singleton search: An examination of top-down, bottom-up, and intertrial factors. Visual Cognition. 2008;16(8):1078–1091. [Google Scholar]
- Makovski T, Jiang YV. Investigating the role of response in spatial context learning. The Quarterly Journal of Experimental Psychology. 2011;64(8):1563–1579. doi: 10.1080/17470218.2011.564291. [DOI] [PubMed] [Google Scholar]
- Maljkovic V, Nakayama K. Priming of pop-out: I. Role of features. Memory & Cognition. 1994;22(6):657–672. doi: 10.3758/bf03209251. [DOI] [PubMed] [Google Scholar]
- Maljkovic V, Nakayama K. Priming of pop-out: II. The role of position. Perception & Psychophysics. 1996;58(7):977–991. doi: 10.3758/bf03206826. [DOI] [PubMed] [Google Scholar]
- McPeek R, Maljkovic V, Nakayama K. Saccades require focal attention and are facilitated by a short-term memory system. Vision Research. 1999;39(8):1555–1566. doi: 10.1016/s0042-6989(98)00228-4. [DOI] [PubMed] [Google Scholar]
- Miller J, Riehle A, Requin J. Effects of preliminary perceptual output on neuronal activity of the primary motor cortex. Journal of Experimental Psychology: Human Perception & Performance. 1992;18(4):1121. doi: 10.1037//0096-1523.18.4.1121. [DOI] [PubMed] [Google Scholar]
- Moher J, Lakshmanan BM, Egeth HE, Ewen J. Inhibition drives early feature-based attention. Psychological Science. doi: 10.1177/0956797613511257. (in press) [DOI] [PMC free article] [PubMed] [Google Scholar]
- Moher J, Song JH. Context-dependent sequential effects of target selection for action. Journal of Vision. 2013;13(8):1–13. doi: 10.1167/13.8.10. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Monsell S. Task switching. Trends in Cognitive Sciences. 2003;7(3):134–140. doi: 10.1016/s1364-6613(03)00028-7. [DOI] [PubMed] [Google Scholar]
- Rogers R, Monsell S. Costs of a predictable switch between simple cognitive tasks. Journal of Experimental Psychology: General. 1995;124(2):207–231. [Google Scholar]
- Shin YK, Proctor RW, Capaldi EJ. A review of contemporary ideomotor theory. Psychological Bulletin. 2010;136(6):943–974. doi: 10.1037/a0020541. [DOI] [PubMed] [Google Scholar]
- Song J-H, Nakayama K. Role of focal attention on latencies and trajectories of visually guided manual pointing. Journal of Vision. 2006;6(9):982–995. doi: 10.1167/6.9.11. [DOI] [PubMed] [Google Scholar]
- Song J-H, Nakayama K. Fixation offset facilitates saccades and manual reaching for single but not multiple target displays. Experimental Brain Research. 2007a;177(2):223–232. doi: 10.1007/s00221-006-0667-4. [DOI] [PubMed] [Google Scholar]
- Song J-H, Nakayama K. Automatic adjustment of visuomotor readiness. Journal of Vision. 2007b;7(5):1–9. doi: 10.1167/7.5.2. [DOI] [PubMed] [Google Scholar]
- Song J-H, Nakayama K. Target selection in visual search as revealed by movement trajectories. Vision Research. 2008;48(7):853–861. doi: 10.1016/j.visres.2007.12.015. [DOI] [PubMed] [Google Scholar]
- Song J-H, Nakayama K. Hidden cognitive states revealed in choice reaching tasks. Trends in Cognitive Sciences. 2009;13(8):360–366. doi: 10.1016/j.tics.2009.04.009. [DOI] [PubMed] [Google Scholar]
- Song J-H, Mcpeek RM. Eye-Hand Coordination During Target Selection in a Pop-Out Visual Search. Journal of Neurophysiology. 2009;102(5):2681–2692. doi: 10.1152/jn.91352.2008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Song J-H, Rafal RD, Mcpeek RM. Deficits in reach target selection during inactivation of the midbrain superior colliculus. Proceedings of the National Academy of Sciences. 2011;108(51):E1433–E1440. doi: 10.1073/pnas.1109656108. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Spivey MJ, Dale R. Continuous dynamics in real-time cognition. Current Directions in Psychological Science. 2006;15(5):207–211. [Google Scholar]
- Spivey MJ, Grosjean M, Knoblich G. Continuous attraction toward phonological competitors. Proceedings of the National Academy of Sciences. 2005;102(29):10393. doi: 10.1073/pnas.0503903102. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Strauss S, Heinke D. A robotics-based approach to modeling of choice reaching experiments on visual attention. Frontiers in Psychology. 2012;3(105):1–11. doi: 10.3389/fpsyg.2012.00105. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Theeuwes J. Perceptual selectivity for color and form. Perception & Psychophysics. 1992;51(6):599–606. doi: 10.3758/bf03211656. [DOI] [PubMed] [Google Scholar]
- Van Selst M, Jolicoeur P. A solution to the effect of sample size on outlier elimination. The Quarterly Journal of Experimental Psychology. 1994;47A(3):631–650. [Google Scholar]
- Welsh T, Elliott D. The effects of response priming on the planning and execution of goal-directed movements in the presence of a distracting stimulus. Acta Psychologica. 2005;119(2):123–142. doi: 10.1016/j.actpsy.2005.01.001. [DOI] [PubMed] [Google Scholar]
- Wolfe JM, Butcher SJ, Lee C, Hyle M. Changing your mind: On the contributions of top-down and bottom-up guidance in visual search for feature singletons. Journal of Experimental Psychology: Human Perception & Performance. 2003;29(2):483–502. doi: 10.1037/0096-1523.29.2.483. [DOI] [PubMed] [Google Scholar]
- Yashar A, Makovski T, Lamy D. The role of motor response in implicit encoding: Evidence from intertrial priming in pop-out search. Vision Research. 2013:1–36. [PubMed] [Google Scholar]
- Zehetleitner M, Hegenloh M, Muller HJ. Visually guided pointing movements are driven by the salience map. Journal of Vision. 2011;11(1):1–18. doi: 10.1167/11.1.24. [DOI] [PubMed] [Google Scholar]