Abstract
It is known that looming motion can capture attention regardless of an observer’s intentions. Real-world behavior, however, frequently involves not just attentional selection, but selection for action. Thus, it is important to understand the impact of looming motion on goal-directed action to gain a broader perspective on how stimulus properties bias human behavior. We presented participants with a visually-guided reaching task in which they pointed to a target letter presented among non-target distractors. On some trials, one of the pre-masks at the location of the upcoming search objects grew rapidly in size, creating the appearance of a “looming” target or distractor. Even though looming motion did not predict the target location, the time required to reach to the target was shorter when the target loomed compared to when a distractor loomed. Furthermore, reach movement trajectories were pulled towards the location of a looming distractor when one was present, a pull that was greater still when the looming motion was on a collision path with the participant. We also contrast reaching data with data from a similarly designed visual search task requiring keypress responses. This comparison underscores the sensitivity of visually-guided reaching data, as some experimental manipulations, such as looming motion path, affected reach trajectories but not keypress measures. Together, the results demonstrate that looming motion biases visually-guided action regardless of an observer’s current behavioral goals, affecting not only the time required to reach to targets but also the path of the observer’s hand movement itself.
Keywords: visually-guided action, looming motion, selection bias, movement trajectories
1. Introduction
Everyday behaviors often require reaching towards objects in order to manipulate them in a goal-directed fashion. For example, cooking dinner might involve turning down the stove, grabbing a spice jar, and then reaching to a spatula. These actions are often executed in the context of a cluttered environment, such as a messy kitchen, where multiple objects compete for attention and action.
In such an environment, selection of a single object for more detailed processing is typically required for a guided action response (e.g., Song & Nakayama, 2006). In many cases attentional selection can be guided towards task-relevant properties in a top-down fashion based on current behavioral goals (e.g., Egeth, Virzi, & Garbart, 1984; Gottlieb, 2007; Green & Anderson, 1956; Posner, 1980). However, sometimes the physical properties of an object can also automatically bias attentional selection regardless of an observer’s intentions (e.g., Theeuwes, 1992; Yantis & Jonides, 1984). Guidance of selection for action can be broken down along similar lines; for example, objects matching task-relevant properties tend to compete more strongly for action (e.g., Castiello, 1999). However, action can also be automatically directed towards a perceptually salient object, such as a color singleton, even when that object is not task-relevant (e.g., Kerzel & Schönhammer, 2013; Wood et al., 2011).
One object property that is known to be behaviorally relevant, but whose impact on goal-directed action remains poorly understood, is looming motion. Looming motion, or a sudden increase in the perceived size of an object, is consistent with a rapidly approaching object and thus likely to signal threat to an observer. For example, looming motion would be perceived when a predator attacks, or when a ball is thrown in an observer’s direction.
Previous studies have shown that when an attended object appears to loom, that object triggers an automatic behavioral response in infants and monkeys (e.g., Schiff, Caviness, & Gibson, 1962; Schmuckler, Collimore, & Dannemiller, 2007). Franconeri and Simons (2003) further showed that attention is automatically directed towards objects exhibiting looming motion even when attention is initially directed elsewhere. They proposed a behavioral urgency hypothesis, whereby stimuli that typically signal the need for an urgent behavioral response, such as looming objects, capture attention automatically (see also, e.g., Von Mühlenen & Lleras, 2007). Lin, Franconeri, & Enns (2008) further showed that looming stimuli produce stronger attentional capture when they loom from the periphery and when they appear to be on a collision path with the observer.
These studies have focused on attentional selection. Attention and action are closely linked (e.g., Cisek, 2012; Song & Nakayama, 2009; Spivey & Dale, 2006), and a reach to a target among distractors requires a shift of focal attention (e.g., Song & Nakayama, 2006). Therefore, attentional selection provides a key roadmap for understanding selection for action. Indeed, a number of studies have shown that selection for action is guided in much the same way as attentional selection (e.g., Song & Nakayama, 2006; Moher & Song, 2014). However, action is not merely a readout of concluded higher-level cognitive processes like attention; instead, action plans may be initiated before the selection process is finalized (e.g., Spivey, Grosjean, & Knoblich, 2005; Song & Nakayama, 2008). Thus, the factors that guide attentional selection do not necessarily coincide with the factors that guide selection for action. Indeed, in a recent study we found a dissociation between attentional selection in a psychophysical task and selection for action in a visually-guided reaching task. Specifically, we found that increasing the perceptual salience of a color singleton distractor increased capture in the psychophysical task, but facilitated selection for action in the reaching task (Moher, Anderson, & Song, in revision). In another recent study, Buetti & Kerzel (2009) examined the Simon Effect (e.g., Simon & Rudell, 1967) in both a keypress task and a reaching task. They also found differences between the two measures; for example, the magnitude of the Simon Effect on response times was greater for a keypress response than a reaching response. Therefore, it is important to study guidance of selection for action separately from attentional selection, because in some cases these two processes may involve non-overlapping mechanisms (see also, e.g., Adam & Pratt, 2004).
In the present study, we examine whether looming motion automatically biases selection for action. One possible outcome is that goal-directed action is automatically biased towards looming objects, such that looming motion speeds responses when the looming object is a target, but looming motion disrupts performance when the looming object is a distractor. Another possibility is that selection of a looming object for action is more difficult than selection of a non-looming object because observers seek to avoid possible collisions (see e.g., Merchant, Zarco, Prado, & Pérez, 2009, for a review of interception and collision avoidance). A third possibility is that looming has no effect on goal-directed action.
In Experiment 1, participants searched for and subsequently reached to a letter target in a three-object display. Prior to display onset, pre-masks appeared at the locations of the objects. On some trials, one of the pre-masks was initially small but grew in size over a brief period, and thus appeared to be “looming” in the direction of the participant from behind the display. We examine whether goal-directed action is biased towards looming objects by examining the temporal and spatial aspects of reach movements towards looming targets as well as non-looming targets in the presence of looming distractors. In Experiment 2, we vary the path of looming motion to explore whether objects on a collision path with the participant have more or less impact on goal-directed action. For both reaching experiments, we also present data from a similarly designed keypress version of the task to highlight the similarities and differences between traditional psychophysical approaches and visually-guided reaching studies in exploring the guidance of selection.
2. Experiment 1A: Looming motion and goal-directed action
2.1 Materials and Methods
Methods were largely adapted from Moher & Song (2013).
2.1.1. Participants
Twelve Brown University undergraduates participated in the study in exchange for class credit (3 male, mean age: 20.2 years). All participants for all experiments reported here were right-handed with normal or corrected-to-normal vision and normal color vision. The protocol was approved by the Brown University Institutional Review Board in accordance with the Code of Ethics of the World Medical Association (Declaration of Helsinki) for experiments involving humans. One participant was removed from analysis due to technical problems during the experiment.
2.1.2. Apparatus
Stimuli were presented on an upright Plexiglas display facing the seated participant at a distance of approximately 48cm. A projector behind the display projected the images onto the Plexiglas. An electromagnetic position and orientation recording system (Liberty, Polhemus) was used to record the three-dimensional hand position at a rate of approximately 160 Hz with a measuring error of .03 cm root mean square. A motion-tracking marker was fastened to the tip of each participant’s right index finger using a Velcro strap. A Styrofoam block was placed 27 cm in front of the participant, between him or her and the display. This was the starting block on which participants rested their index finger at the beginning of each trial. Stimulus presentation was conducted using custom software designed with MATLAB (Mathworks) and Psychtoolbox (Brainard, 1997).
2.1.3. Stimuli
All stimuli were presented on a black background. A fixation cross measuring 0.5 cm×0.5 cm (0.6° of visual angle) appeared at the center of the screen before each trial. Three letters measuring 1.6 cm wide (1.9° of visual angle) and 2.7 cm tall (3.2° of visual angle) appeared on the screen during each trial. They were equally spaced and placed at 4, 8, and 12 o’clock positions on an imaginary circle surrounding fixation at a distance of 10 cm (11.9° of visual angle) from fixation. The letters closely resemble the block letters used in digital clocks so that any letter could be obtained from subtracting any of the seven line segments from a block 8. The letters used include: U, H, F, E, P, C, L, and S. Each display contained one target letter: either a U or an H, randomly selected for each trial. Participants were instructed to reach out and touch the target letter. On each trial, three “8”s appeared prior to the appearance of the stimuli. These figure 8s served as placeholder masks in order to prevent participants from beginning their search until the targets appeared. On looming trials, which consisted of half of all trials, one of the three “8”s initially appeared as 0.5 cm wide (0.6° of visual angle) and 0.9 cm tall (1.1° of visual angle) before rapidly expanding to its full size (300% increase in horizontal and vertical size at a linear growth rate) during the brief (150 ms) looming period (Figure 1).
2.1.4. Procedure
Nine-point hand calibration was conducted at the beginning of the experiment. Each trial began with a fixation cross at the center of the display. Participants were instructed to keep their finger in the starting position until the letter stimuli appeared. The trial would not proceed if the participant prematurely moved their finger off of the starting position. One second after the fixation cross appeared, three figure 8s appeared as placeholder pre-masks. In the looming condition, one randomly chosen placeholder mask appeared initially smaller than the others. After 1 s, the smaller mask grew to the same size as the other letters in the display over the course of 150 ms, resulting in the appearance of looming motion. The masks were then removed to reveal the letters immediately following the end of the looming animation. After every trial, an auditory feedback tone was played to indicate whether the participant’s response was accurate (high-pitch beep) or inaccurate (low-pitch beep). If the participant did not touch one of the three letters within 1500 ms, the trial was marked as incorrect and a tone was played to notify the participant. There was a 1 s intertrial interval.
The experiment began with 10 practice trials, followed by 8 blocks of 60 trials each (one participant completed only seven blocks due to technical issues, but was still included in the analysis). Participants were given an opportunity to rest between each block. Each testing session lasted approximately one hour.
2.1.5. Data Analysis
Data analysis for the eleven included participants was largely adapted from methods reported in Moher & Song (2013). When the participant’s finger came within approximately 1.3 cm of the display on the z-dimension (i.e., the axis bounded by the participant and the display) and simultaneously within approximately 2.4 cm of the center of the target letter on the x (i.e., the horizontal axis on the display) and y (i.e., the vertical axis on the display) dimensions within the time limit, a response was considered correct. If this threshold was passed for a non-target letter, or the participant did not cross any reaching threshold during the time limit, the trial was counted as incorrect.
Hand movement data were analyzed offline using custom MATLAB (Mathworks) software. Three-dimensional resultant speed scalars were created for each trial using a differentiation procedure in MATLAB. These scalars were then submitted to a 2nd order, low-pass Butterworth filter with a cutoff of 10 Hz. Movement onset was calculated as the first time point on each trial after stimulus onset at which hand movement speed exceeded 25.4 cm/s. Movement offset was defined as the first subsequent measurement on each trial when speed decreased to below 25.4 cm/s. Initiation latency was defined as the time elapsed between stimulus onset and movement onset. Movement time was defined as the time elapsed between movement onset and movement offset.
Trajectories for calculating curvature were measured in two- dimensional xy space by calculating a line from the start to the end point of the movement, and measuring the orthogonal deviation of the actual movement from that line at each sample throughout the movement. Curvature was defined as the maximum point of deviation (unsigned) in centimeters divided by the length of the line from the start to the end points of the movement in centimeters (see, e.g., Desmurget, Jordan, Prablanc, & Jeannerod, 1997; Moher & Song, 2013; Song & Nakayama, 2006).
We primarily compared performance on trials with a looming target to trials with a looming distractor. However, trials where no object loomed were included in the experimental design for movement trajectory analyses purposes. These trials provided a baseline trajectory that accounted for kinematics associated with reaching to targets in this task context independent from any looming motion (e.g., Song & Nakayama, 2006; Song & Nakayama, 2008). These non-looming trials, however, were not included in other analyses, as the presence of looming motion prior to target onset may have served as an alerting cue regardless of whether the looming object was a target or distractor. This alerting cue could subsequently speed the time required to initiate a response on looming trials relative to non-looming trials, making comparisons of timing measures in particular difficult to interpret.1
To compare movement trajectories across conditions, we resampled each individual trial to create 101 samples equally spaced in time along the horizontal and vertical dimensions of the display (e.g. Moher & Song, 2013; Spivey et al., 2005; Song & Nakayama, 2008). We calculated distractor attraction scores from these resampled movements in order to assess whether movement trajectories were pulled towards the location of looming distractors (Fig. 2A). To compute distractor attraction scores for each participant, for each target location, we calculated the difference in hand position at each resampled point between the mean trajectory of accurate trials where a looming distractor was present and the mean trajectory of accurate baseline trials where no looming object was present. Positive scores indicated that the hand was being pulled more towards a given location when a looming distractor was present at that location relative to baseline. Negative scores indicated that the hand was being pushed away from the location of the looming distractor. These scores were calculated for each participant separately to each of the six possible target and distractor combinations, and subsequently averaged together, creating a distractor attraction score at each of 101 resampled movement points which reflected the degree to which the hand was pulled towards the distractor location relative to baseline. We used a cluster-based analysis to determine when during the movement the distractor disrupted trajectories according to distractor attraction scores (e.g., Maris & Oostenveld, 2007).
In an additional analysis, we directly compared the trajectories of hand movements on looming distractor trials to determine whether hand movements were pulled generally towards the center of all items when a distractor was present, or were pulled specifically towards the location of that looming distractor. For side targets, we contrasted trials with a distractor present at either the top or the opposite side from the target, and positive scores indicated that the hand was pulled more towards the opposite side location when a distractor was at that location as opposed to at the top location. For targets at the top of the display, positive scores indicated the distractor was pulled more leftwards for a left-side distractor than for a right-side distractor.
Each individual trial was visually inspected (e.g., Song & Nakayama, 2006, 2007a, 2007b, 2008); for trials where the default threshold clearly missed part of the movement or included substantial movement back to the starting point, thresholds were adjusted manually to more appropriate levels for that trial (5.1% of all trials in Experiment 1A, 2% of all trials in Experiment 2A). In addition, we removed all trials in which a large number of movement samples were dropped due to computer error or no movement was executed (0.4% of all trials in Experiment 1A, 0.6% of all trials in Experiment 2A).
2.2. Results and Discussion
Consistently, we found that reach movements were executed more rapidly when a looming target was present compared to looming distractor present trials. Initiation latency was shorter when the looming object was a target (415 ms) compared to when the looming object was a distractor (434 ms), t(10) = 3.6, p < .01 (Fig. 2B). Movement times were shorter for looming target trials (385 ms) than looming distractor trials (403 ms), t(10) = 4.14, p < .01, and also less curved for looming target trials (0.119) relative to looming distractor trials (0.157), t (10) = 4.64, p < .001. Overall accuracy was very high (>98%), and there was no effect of looming condition on accuracy t(10) = 1.76, p > .1.
On trials where a looming object was present, the looming motion provided no useful information to the participant in finding the target location. Nevertheless, looming motion directly impacted selection for action, as looming targets were reached to more quickly and with less movement deviation than non-looming targets that were presented concurrently with a looming distractor. Thus, selection for action was automatically biased by looming motion.
The most plausible interpretation of this result is that looming motion automatically attracted attention and action. If this were the case, we would additionally expect that movement trajectories were pulled in the direction of looming distractors (e.g., Buetti & Kerzel, 2009; Song & Nakayama, 2006). Although we did find an increase in movement curvature on looming distractor trials relative to looming target trials, this measure does not indicate whether hand movements were biased towards the location of the looming distractor, away from the looming distractor, or simply more deviated in both directions on looming distractor trials.
In Fig. 2C, we plot the distractor attraction score (see methods for details) at each point of the movement for looming distractor trials to determine whether hand movements were pulled in the direction of the looming distractor relative to baseline non-looming trials. Distractor attraction scores were greater than zero continuously from 2% through 90% of the movement (shaded area between dashed lines in Fig. 2C), p < .001, indicating that the hand was consistently pulled in the direction of looming distractors. Furthermore, this result is not because observers were simply delaying target selection by moving their hand towards the center of all items when a distractor was present2 (e.g., Kerzel & Schönhammer, 2013). If that were the case, reach movements would be unaffected by the specific looming distractor location. Instead, however, we found that from 37% through 69% of the movement, hand position was affected by looming distractor location. Specifically, the hand was pulled closer to a given non-target location when the looming distractor appeared there rather than at the other non-target location.
To summarize, hand movements were initiated more rapidly and arrived at their target in a shorter period of time when the looming object was a target compared to when the looming object was a distractor. Furthermore, there was greater deviation towards distractors when they exhibited looming motion. Together, these results suggest goal-directed action is automatically biased towards looming motion.
3. Experiment 1B: Looming motion and visual search
In the next experiment, we presented participants with displays identical to those in Experiment 1A. However, we changed the response requirements. Rather than a visually-guided reach movement, participants pressed a key to indicate whether a “U” or “H” target was present. This allowed us to compare the results from the visually-guided reaching data in Experiment 1A to a more traditional visual search dataset.
3.1. Materials and Methods
12 Brown University undergraduates participated in the study in exchange for class credit (4 male, mean age: 19.3 years).
Experiment 1B was identical to Experiment 1A except that rather than pointing to the target, participants were required to press a key to indicate the identity of the target letter (“U” or “H”) on a keyboard that was placed on the table in front of the participant. Dependent variables for this experiment were response time and accuracy of keypress responses.
3.2. Results and Discussion
As in Experiment 1A, looming distractors disrupted performance relative to looming targets. Participants were faster to respond when the looming object was a target (688 ms) than when it was a distractor (763 ms), t(11) = 8.34, p < .001 (Fig. 2D). Additionally, errors were more frequent in the looming distractor condition (5.5%) compared to the looming target condition (3.1%), t(11) = 4.29, p < .01.
These data show a similar pattern of results to previous studies on looming-based attentional capture (e.g., Franconeri & Simons; Von Mühlenen & Lleras, 2007); namely, that visual search was biased by looming motion. Previous studies have varied the set size of search items to determine whether looming motion affected attentional selection. If looming motion biases attentional selection, participants are likely to select the looming item first regardless of the number of items in the display. Thus, RT to looming targets should remain relatively stable regardless of the number of display items, while RT to trials with a looming distractor present will increase as the number of display items increases. Alternatively, if looming motion affects some type of post-selective processing, then RT will still be faster to looming targets but will increase with increasing set size even for looming targets, as the looming motion does not guide selection towards the target but instead speeds processing following selection (e.g., Treisman & Gelade, 1980).
In Experiment 1A and 1B, however, we only used a fixed set size of three items. Thus, it is difficult to determine whether the keypress results reported in the visual search task of Experiment 1B reflect changes in the attentional selection process, or changes in other processes that occur before or after attentional selection. For example, the results of Experiment 1B could be interpreted to reflect faster processing of letter identity following selection of looming objects. In other words, the selection process itself was not affected by looming motion, but processing of the looming letter after selection was faster than processing of other letters after selection (see e.g., Moher, Ashinoff, & Egeth, 2013; Wolfe, 2003). Another possible explanation of the results from Experiment 1B is that slower RT on looming distractor trials reflects inhibition rather than enhancement of the location of the looming distractor. This would be consistent with previous studies showing that applying inhibition can require time and limited central resources, resulting in slower responses (e.g., Chao, 2010; Moher & Egeth, 2012; Watson & Humphreys, 1997). One advantage of using the visually-guided reaching approach in Experiment 1A is that movement trajectories indicate the level of competition from non-target objects (e.g., Spivey et al., 2005; Song & Nakayama, 2008). Thus, in Experiment 1A, we can rule out a post-selective account and an inhibitory account of the data, because movement trajectories are pulled towards the spatial location of a looming distractor.
Another notable difference in comparing results between Experiments 1A and 1B is that the looming benefit (i.e., the difference in RT between looming target and looming distractor trials) was much greater in magnitude in Experiment 1B (75 ms) than the looming benefit for total response time (initiation latency plus movement time) in Experiment 1A (37 ms) 3. The possible implications are discussed in the General Discussion.
In Experiment 1, as indicated in Fig. 1, looming objects always grew equally in size in all directions. Thus, looming objects appeared to be looming towards the participant along the axis connecting the participant and the display. However, in real-world situations looming objects often exhibit motion along multiple axes. Particularly for objects not directly in front of an observer, this additional motion would indicate whether the looming object was on a collision path with the observer or not. An object on a collision path might compete more strongly for selection for action, as that object would be almost guaranteed to require some type of behavioral response (e.g., Lin et al., 2008). In Experiment 2, we manipulated the direction of looming motion such that objects loomed either towards (Fig. 3A) or away from the participant (Fig. 3B) to determine whether objects on a collision path with the participant bias selection for action more strongly than objects on a non-collision path.
4. Experiment 2A: Does the direction of looming motion impact goal-directed action?
4.1. Materials and Methods
12 Brown University undergraduates participated in the study in exchange for class credit (5 male, mean age: 19.2 years).
Experiment 2A was identical to Experiment 1A except in two respects. Firstly, due to a minor change in protocol, the duration of the looming pre-mask period was changed to 167 ms. Secondly, the nature of the motion of looming objects was changed. In Experiment 1A, looming objects grew outward in all directions simultaneously, such that they appeared to be moving forward towards the participant with no movement along the horizontal or vertical axes. In Experiment 2A, the looming object could appear to be looming towards the participant on either the horizontal and/or vertical axis, and thus appear to be on a collision path with the participant, or away from the participant on the horizontal and/or vertical axis, thus appearing to be on a miss path. For objects at the top of the display, a collision path meant that the object expanded leftwards, rightwards, and downwards, and a miss path meant that the object expanded leftwards, rightwards, and upwards. For side objects, a collision path meant that the object expanded inwards and upwards (Fig. 3A), whereas a miss path meant that the object expanded outwards and downwards (Fig. 3B). Looming motion was equally likely to be a collision or miss path for each trial where a looming object was present. It is important to note that the looming object always moved in the direction of the participant along the z-axis as in Experiment 1 (i.e., the axis bounded by the participant and the display). However, for the present experiment we categorize looming motion path as having collision or miss paths referring to motion on the x and y axes (i.e., the horizontal and vertical axes of the display).
Data analysis was largely similar to Experiment 1A. We conducted 2×2 repeated measures ANOVAs with factors of looming object (target vs. distractor) and looming motion path (collision vs. miss from the observer) on all dependent variables. We again calculated distractor attraction scores for looming distractors, but for this experiment we conducted scores separately for distractor that loomed on a collision path or miss path. In addition, to assess whether the direction of looming motion of looming targets (rather than distractors) impacted hand movements, we conducted a separate analysis in which we examined trajectories on trials where a looming target appeared on the left or right side of fixation only, eliminating from analysis trials where the target appeared at the top of the display. For this analysis, we compared trajectories as a function of whether the looming motion was on a collision or miss path. We calculated the difference between the position of the hand along the x-axis on those trials to the position of the hand along the x-axis on non-looming trials at each point of the resampled movement. That number was calculated as positive if the hand was pulled in the direction of looming motion (i.e., if a rightward looming target caused the hand to deviate more to the right relative to a non-looming target), and negative if the position of the hand was opposite the direction of looming motion. Thus, a positive number at any point in the movement indicates that the hand was pulled along the x-axis in the direction in which the target object loomed.
4.2. Results and Discussion
As in Experiment 1, hand movements were more efficient when the target object loomed relative to when one of the distractors loomed. Initiation latency was shorter for looming target trials (423 ms) than for looming distractor trials (440 ms), F(1,11) = 5.2, p < .05. Movements showed a similar pattern, with shorter movement times for looming targets (431 ms) compared to looming distractor trials (450 ms), F(1,11) = 11.7, p < .01 (Fig. 4), and less curvature on looming target trials (.099) relative to looming distractor trials (.128), F(1,11) = 11.4, p < .01. Error rates were again very low (<2%) with no difference in error rate between looming targets and looming distractors, F(1,11) < 1. There were no effects of looming motion path on any of these measures, nor any interactions, ps > .05.
We again measured the impact of looming distractors on reach movement trajectories with the distractor attraction score. As in Experiment 1A, reach movement trajectories were pulled towards the location of looming distractors; the distractor attraction score was greater than zero from 11% through 92% of the movement, p < .001. Unlike Experiment 1A, hand position was unaffected by the specific looming distractor location. However, while looming distractors in Experiment 1A maintained a constant center, looming distractors in Experiment 2A did not. As a result, it is possible that participants had a less precise representation of the looming distractor location initially, making it difficult to differentiate movement trajectories as a function of the looming distractor’s location.
Next, we compared trajectories for distractors on either a collision path or a miss path (Fig 5A) by examining distractor attraction scores for each trial type (Fig. 5B). It is important to note that distractors on a collision path with the participant’s location also happen to loom towards the target’s location, and distractors on a miss path from the participant also loom away from the target. For example, a looming distractor from the left side on a collision path with the participant also happens to loom towards a target at the top or right side. Thus, one possible outcome would be that hand movements were pulled in the same direction as the direction of looming motion of the distractor. If that were the case, we would expect more deviation towards looming distractors when they loomed on a miss path, as if the participant’s hand were being pulled outwards as the distractor loomed away from the target. Instead, we found that distractor attraction scores were greater from 27% through 31% of the movement when the looming motion was on a collision path with the participant rather than on a miss path, p < .001. Thus, objects on a collision path with an observer have a stronger influence on selection for action (cf., Lin et al., 2008), resulting in a stronger pull on hand movement trajectories.
This result suggests that not only is goal-directed action biased towards the location of looming objects, but also that the implied motion path of those looming objects can further impact goal-directed action. We next examined whether hand movements towards looming targets were impacted by the path of looming motion. For example, it might be the case that hand movements were pulled in same horizontal direction of looming targets appearing on the left or right side. Thus, if a left-side looming target loomed on a collision path with the participant, we might expect movement trajectories to be pulled more rightwards than if a left-side target loomed on a miss path (Fig. 6A). For a right-side target, we might expect trajectories to be pulled more leftwards when the target loomed on a collision path with the participant. We measured the pull along the x-axis as a function of whether the looming motion was on a collision or miss path for targets appearing on the left and right side of fixation (see methods for details). Hand movements were indeed pulled in the direction of looming motion when the target loomed, with movement biased in the same direction as the looming direction of the target from 13% through 29% of the movement, p <.001 (Fig. 6B). That is, when the object loomed rightwards, the hand was pulled to the right, and when the object loomed leftwards, the hand was pulled to the left. Unlike the analysis with looming distractors, it is unclear whether this effect occurs because participants are pulled in the same direction as the object they select, or whether objects compete more strongly for selection for action when they are on a collision path with the participant. Either interpretation is consistent with hand movements being pulled inwards when the left and right side targets loom on a collision path with the participant. In either case, though, this result provides further evidence that visually-guided reach movement trajectories are affected by looming motion path.
Together, these results suggest that not only does the presence of a looming object impact goal-directed action, but also that the path of the looming motion itself affects goal-directed action. Specifically, when a target or non-target item exhibits looming motion, hand movement trajectories are affected by the path of looming motion, and distractors compete more strongly for selection for action when they are on a collision path with an observer.
5. Experiment 2B: Looming motion direction and visual search
5.1. Materials and Methods
12 Brown University undergraduates participated in the study in exchange for class credit (3 male, mean age: 19.3 years).
Experiment 2B was identical to Experiment 2A, except that keypress responses were required instead of visually-guided reaching responses. For data analysis, we again conducted 2×2 repeated measures ANOVAs with factors of looming object (target vs. distractor) and looming path (collision vs. miss) on all dependent variables.
5.2. Results and Discussion
As in Experiment 1, response times were faster when the looming object was a target (669 ms) compared to when the looming object was a distractor (755 ms), F(1,11) = 79.3, p < .001 (Fig. 7), and error rates showed a similar pattern with more frequent errors on looming distractor trials (5.8%) compared to looming target trials (3.5%), F(1,11) = 13.4, p < .01. There was no effect of looming path for response time, F(1,11) < 1. Error rates were higher when the looming object was on a collision path with the participant (6.9%) than when it loomed on a miss path (3.9%), F(1,11) = 10.4, p < .01. However, there were no interactions between the looming object (target vs. distractor) and looming path for response time or error rate, Fs < 1.
Lin et al. (2008) found response times and search slopes in a visual search task were reduced when targets exhibited looming motion on a collision path with an observer. Although we did find that error rates were higher for objects that loomed towards the participant regardless of whether the looming motion occurred on a target or distractor object, we did not find any advantage in performance for targets looming towards the participant in the present experiment. One possible explanation for this discrepancy is that the strength of the looming motion in the present study was weaker than the motion in Lin et al. (2008), and thus did not produce the same behavioral effects. The looming objects on a collision path with the participants in Lin et al.’s (2008) study grew more than 4 items as large in horizontal and vertical size during the looming period, while looming objects only grew about 3 times as large in our study. Furthermore, the objects in Lin et al.’s study were on a collision path with the participant’s head, whereas our objects were only designed to be on a collision path with the participant’s body in general. Finally, we did not vary set size and thus could not determine whether targets on a collision path with the participant produced shallower search slopes than targets not on a collision path. These differences may explain why we did not observe faster response times for targets looming towards the participant. However, this highlights another advantage of the visually-guided reaching approach used in Experiment 2A. Although the manipulation of looming motion direction was not sufficiently strong to generate response time effects in a visual search task, we were able to detect differences in reach movement trajectory as a function of looming motion for both looming targets and looming distractors in a visually-guided reaching task. Thus, visually-guided reaching may provide a particularly sensitive measure to stimulus-driven effects on target selection processes.
As in Experiment 1, the looming benefit was greater in magnitude for keypress responses (86 ms) relative to reach movement responses (36 ms)4. The potential implications of this difference are discussed in the General Discussion.
6. General Discussion
In two experiments, we found that selection for action was automatically biased towards looming motion. Reach movement responses were initiated more quickly and movement time was shorter when the looming object was the target relative to when the looming object was a non-target distractor, even though the looming motion did not predict the location of the target. Furthermore, reach movement trajectories were pulled in the direction of looming distractors. In Experiment 2, we found that distractors that loomed on a collision path with the participant had a greater impact on selection for action than distractors that loomed on a miss path, indicating even stronger biases for looming objects that appeared to be headed towards the participant’s body.
These results expand the behavioral urgency hypothesis proposed by Franconeri & Simons (2003). Those authors proposed that objects exhibiting properties that typically require an immediate behavioral response, such as looming motion, capture attention automatically. In the present study, we have shown that goal-directed action responses are also automatically biased towards looming motion. This suggests that object properties signaling the need for an urgent behavioral response impact not only attentional selection, but bias hand movements themselves. Further research examining other object properties that may signal behavioral urgency is needed to fully understand the relationship between properties associated with the need for an immediate reaction and the automatic action responses that those properties trigger.
In conjunction with both visually-guided reaching studies, we presented data from similarly designed visual search versions of the same tasks in which keypress responses rather than reach movement responses were required. The contrast between these two approaches highlights several important issues. First and foremost, the data from reach movement studies provides information about how looming motion affects selection for action, rather than just attentional selection. This is an important distinction, as selection for action is required for many real-world behaviors (e.g., Castiello, 1999), and there are some differences in the way the physical properties of objects guide attentional selection and selection for action (e.g., Buetti & Kerzel, 2009). In Experiment 1, we further demonstrated an inherent advantage of examining reach movement trajectories in that this technique allows us to infer the strength of competition for selection from target and non-target objects without varying the set size of the display. For example, trajectories were pulled towards rather than away from looming distractors in Experiment 1, suggesting that looming distractors did not trigger inhibition, but rather attracted selection for action. Furthermore, this deviation occurred almost immediately in the movement, providing insight into the time-course of the selection bias induced by looming motion. In Experiment 2, we found that a manipulation of looming motion path that did not affect the timing of reach movement or keypress responses did affect reach movement trajectories. Thus, reach movement trajectories proved to be a particularly sensitive dependent measure of our experimental manipulations, relative to keypress responses or reach movement timing measures.
Finally, we found that the effect of looming motion on the total response time to a target, as a function of whether the looming object was a target or distractor, was much greater for keypress responses than for reach movement responses. This involves comparing across two different types of responses; thus, any interpretations of this difference should be taken with caution. Nevertheless, the difference in magnitude may be informative in understanding how looming motion affects target selection processes for different types of action.
A recent study by Skarratt, Gellatly, Cole, Pilling, and Hulleman (2013) examined how looming motion affects both attention and motor responses. They compared response times to looming and receding motion in a visual search task and a temporal order judgment (TOJ) task. While the former requires a rapid motor response, the latter is judged solely on accuracy and thus can be taken as a reflection of purely perceptual processes. Skarratt et al. (2013) found that while looming motion triggered faster overall responses than receding motion in the visual search task (see also; Skarratt, Cole, & Gellatly, 2009), there was no looming motion advantage relative to receding motion in the TOJ task. Thus, they suggested that while object motion in general attracts attention, looming motion specifically has a further behavioral effect in that it primes motor responses. They argued that such a mechanism would prove adaptive in that looming motion may often signal the presence of a threat and thus necessitate a rapid motor response.
In the present study, it may be the case that the priming of visuomotor respones by looming motion was even more powerful than in Skarrat et al. (2013), because reach movement responses were involved rather than keypresses. This priming of reach movement responses provided by the mere presence of looming motion could negate some, but not all, of the looming advantage in the selection process for looming targets relative to looming distractors in goal-directed action. This would be especially likely if the stage of processing that looming motion speeds up in the visually-guided reaching task operates concurrently with the selection for action process (cf., Pashler, 1994). However, another possible explanation for the difference in the size of the looming advantage could be procedural differences between the reaching and keypress versions of the task. Both tasks were similar in that they required discrimination of letter identities in order to find the target. Furthermore, previous studies have shown that reaching to a target among distractors requires a shift of focal attention (e.g., Song & Nakayama, 2006). Therefore, for the purpose of matching attentional demands, a discrimination task is a more appropriate comparison to a reaching response rather than a detection task (e.g., Moher & Song, 2014; Song & Nakayama, 2007b). Still, the response required in the keypress task required an additional step of pressing the key associated with the particular target that was present. For the reach task, the participant simply had to reach to the target’s location regardless of its identity. Thus, the response selection process for the keypress task may have been longer, providing more opportunity for looming motion to reduce the time needed to complete the task. In any case, the differences in looming advantage between the two tasks further highlight the importance of considering how the type of action output required may impact the selection process.
While we manipulated whether the looming object was on a collision course with the participant’s body, we did not design our experiment to examine whether the path of implied motion of the looming object interfered with the path of the participant’s hand movement to the target. It is possible, for example, that an observer would seek to avoid a collision between their hand and the implied path of the looming object. Indeed, previous studies have shown that observers adjust movement paths to avoid collisions with stationary objects on or near the path to the target (e.g., Tipper, Howard, & Jackson, 1997). Future studies that focus on this particular type of collision with looming objects may provide useful information in understanding how selection for action may be affected by anticipated collisions during movement execution.
A number of previous studies have found that goal-directed action is automatically biased towards perceptually salient feature singleton objects (e.g., Kerzel & Schönhammer, 2013; Wood et al., 2011). The present results build on these findings to show that looming motion can also bias reach movement trajectories automatically. These findings highlight the degree to which the bottom-up properties of a stimulus can direct selection for action regardless of an observer’s intentions. A great deal of literature has been devoted to studying the impact of bottom-up factors on attentional selection. However, this recent research shows that selection for action is also subject to biases driven not by an observer’s goals, but instead by the physical properties of objects. It is important to consider the consequences of these misdirected movements in everyday behavior. For example, a roadside billboard exhibiting looming or other types of motion may be dangerous not only because it captures attention and subsequently delays a driver’s response to a slowing car in front of them, but also because a driver’s actions may also be impacted by this salient distractor. As a result, the driver might inadvertently steer towards the location of the billboard and thus away from the center of the road (see e.g., the “moth effect”, Kitamura & Matsunaga, 1994).
Highlights.
Reaches are faster and more direct towards a target exhibiting looming motion
Reaching movements are automatically attracted towards looming distractors
Greater disruption from distractors looming on a collision path with the observer
Reaching is a more sensitive tool to evaluate looming effects than psychophysics
Acknowledgments
This project is supported by Brown University Salomon faculty research award and NIGMS-NIH IDeA P20GM103645 to J.H.S. J.M. is supported by the Center for Vision Research fellowship and the Brown Training Program in Systems and Behavioral Neuroscience NIH T32MH019118, and J.S. is supported by the Janey Scholars Program.
Footnotes
For completeness, we report the descriptive statistics for non-looming trials here. Experiment 1A: Initiation latency (IL; 472 ± 17 ms), movement time (MT; 383 ± 23 ms), Accuracy (99.1 ± 0.2 %), and curvature (0.126 ± 0.009). Experiment 1B: RT (744 ± 25 ms), accuracy (96.1 ± 1.1 %). Experiment 2: IL (482 ± 18 ms), MT (431 ± 16 ms), Accuracy (98.2 ± 0.7 %), and curvature (0.101 ± 0.01). Experiment 2B: RT (739 ± 23 ms), accuracy (94.2 ± 1.4 %).
We thank Dr. Dirk Kerzel for raising this possibility.
Although this difference should be interpreted with caution since we are comparing across different types of responses, we nevertheless conducted a mixed-design ANOVA with looming object (target vs. distractor) as a within-subject factor and response type (reach vs. keypress) as a between-subjects factor and found a significant interaction, F(1,21) = 12.64, p < .01, supporting the notion that the looming benefit differed between experiments.
A mixed-design ANOVA revealed a significant interaction between looming type and response type, F(1,22) = 18.95, p < .001, confirming the difference in looming benefit between the two experiments. However, as in Experiment 1 this difference should be interpreted with caution because it involves a comparison across different types of responses.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
References
- Adam JJ, Pratt J. Dissociating Visual Attention and Effector Selection in Spatial Precuing Tasks. Journal of Experimental Psychology Human Perception and Performance. 2004;30(6):1092–1106. doi: 10.1037/0096-1523.30.6.1092. [DOI] [PubMed] [Google Scholar]
- Buetti S, Kerzel D. Conflicts during response selection affect response programming: Reactions toward the source of stimulation. Journal of Experimental Psychology Human Perception and Performance. 2009;35(3):816–834. doi: 10.1037/a0011092. [DOI] [PubMed] [Google Scholar]
- Castiello U. Mechanisms of selection for the control of hand action. Trends in Cognitive Sciences. 1999;3:264–271. doi: 10.1016/s1364-6613(99)01346-7. [DOI] [PubMed] [Google Scholar]
- Cisek P. Making decisions through a distributed consensus. Current Opinion in Neurobiology. 2012;22(6):927–936. doi: 10.1016/j.conb.2012.05.007. [DOI] [PubMed] [Google Scholar]
- Chao H. Top-down attentional control for distractor locations: The benefit of precuing distractor locations on target localization and discrimination. Journal of Experimental Psychology Human Perception and Performance. 2010;36(2):303–316. doi: 10.1037/a0015790. [DOI] [PubMed] [Google Scholar]
- Desmurget M, Jordan M, Prablanc C, Jeannerod M. Constrained and unconstrained movements involve different control strategies. Journal of Neurophysiology. 1997;77(3):1644–1650. doi: 10.1152/jn.1997.77.3.1644. [DOI] [PubMed] [Google Scholar]
- Franconeri SL, Simons DJ. Moving and looming stimuli capture attention. Perception and Psychophysics. 2003;65(7):999–1010. doi: 10.3758/bf03194829. [DOI] [PubMed] [Google Scholar]
- Gottlieb J. From Thought to Action: The Parietal Cortex as a Bridge between Perception, Action, and Cognition. Neuron. 2007;53(1):9–16. doi: 10.1016/j.neuron.2006.12.009. [DOI] [PubMed] [Google Scholar]
- Green BF, Anderson LK. Color coding in a visual search task. Journal of Experimental Psychology. 1956;51(1):19–24. doi: 10.1037/h0047484. [DOI] [PubMed] [Google Scholar]
- Kerzel D, Schönhammer J. Salient stimuli capture attention and action. Attention, Perception & Psychophysics. 2013 doi: 10.3758/s13414-013-0512-3. [DOI] [PubMed] [Google Scholar]
- Kitamura F, Matsunaga K, Nagao I. Influence of looking at hazard lights on car-driving performance. Perceptual and Motor Skills. 1994;78(3c):1059–1065. doi: 10.2466/pms.1994.78.3c.1059. [DOI] [PubMed] [Google Scholar]
- Müller HJ, Geyer T, Zehetleitner M, Krummenacher J. Attentional capture by salient color singleton distractors is modulated by top-down dimensional set. Journal of Experimental Psychology Human Perception and Performance. 2009;35(1):1–16. doi: 10.1037/0096-1523.35.1.1. [DOI] [PubMed] [Google Scholar]
- Lin JY, Franconeri S, Enns JT. Objects on a collision path with the observer demand attention. Psychological Science : a Journal of the American Psychological Society/APS. 2008;19(7):686–692. doi: 10.1111/j.1467-9280.2008.02143. [DOI] [PubMed] [Google Scholar]
- Maris E, Oostenveld R. Nonparametric statistical testing of EEG- and MEG-data. Journal of neuroscience methods. 2007;164(1):177–190. doi: 10.1016/j.jneumeth.2007.03.024. [DOI] [PubMed] [Google Scholar]
- Merchant H, Zarco W, Prado L, Pérez O. Advances in Experimental Medicine and Biology. Vol. 629. Boston, MA: Springer US; 2009. Behavioral and Neurophysiological Aspects of Target Interception; pp. 201–220. [DOI] [PubMed] [Google Scholar]
- Moher J, Egeth HE. The ignoring paradox: Cueing distractor features leads first to selection, then to inhibition of to-be-ignored items. Attention, Perception & Psychophysics. 2012;74(8):1590–1605. doi: 10.3758/s13414-012-0358-0. [DOI] [PubMed] [Google Scholar]
- Moher J, Anderson BA, Song J-H. Stimulus-driven control of selection: Dissociable effects of salience on attention and goal-directed action. (in revision) [Google Scholar]
- Moher J, Ashinoff BK, Egeth HE. Detection is unaffected by the deployment of focal attention. Frontiers in Cognitive Psychology. 2013;4(284):1–10. doi: 10.3389/fpsyg.2013.00284. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Moher J, Song JH. Context-dependent sequential effects of target selection for action. Journal of Vision. 2013;13(8):1–13. doi: 10.1167/13.8.10. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Moher J, Song J-H. Target Selection Bias Transfers Across Different Response Actions. Journal of Experimental Psychology Human Perception and Performance. 2014:1–14. doi: 10.1037/a0035739. [DOI] [PMC free article] [PubMed] [Google Scholar]
- von Mühlenen A, Lleras A. No-onset looming motion guides spatial attention. Journal of Experimental Psychology Human Perception and Performance. 2007;33(6):1297–1310. doi: 10.1037/0096-1523.33.6.1297. [DOI] [PubMed] [Google Scholar]
- Pashler H. Dual-task interference in simple tasks: Data and theory. Psychological Bulletin. 1994;116(2):220–244. doi: 10.1037/0033-2909.116.2.220. [DOI] [PubMed] [Google Scholar]
- Posner M. Orienting of attention. The Quarterly Journal of Experimental Psychology. 1980;32(1):3–25. doi: 10.1080/00335558008248231. [DOI] [PubMed] [Google Scholar]
- Schiff W, Caviness JA, Gibson JJ. Persistent fear responses in rhesus monkeys to the optical stimulus of “looming. Science. 1962;136:982–983. doi: 10.1126/science.136.3520.982. [DOI] [PubMed] [Google Scholar]
- Schmuckler MA, Collimore LM, Dannemiller JL. Infants’ reactions to object collision on hit and miss trajectories. Infancy. 2007;12(1):105–118. doi: 10.1111/j.1532-7078.2007.tb00236.x. [DOI] [PubMed] [Google Scholar]
- Simon JR, Rudell AP. Auditory S-R compatibility: The effect of an irrelevant cue on information processing. Journal of Applied Psychology. 1967;51:300–304. doi: 10.1037/h0020586. [DOI] [PubMed] [Google Scholar]
- Skarratt PA, Cole GG, Gellatly ARH. Prioritization of looming and receding objects: Equal slopes, different intercepts. Attention, Perception & Psychophysics. 2009;71(4):964–970. doi: 10.3758/APP.71.4.964. [DOI] [PubMed] [Google Scholar]
- Skarratt PA, Gellatly ARH, Cole GG, Pilling M, Hulleman J. Looming Motion Primes the Visuomotor System. Journal of Experimental Psychology Human Perception and Performance. 2013:1–14. doi: 10.1037/a0034456. [DOI] [PubMed] [Google Scholar]
- Song JH, Nakayama K. Role of focal attention on latencies and trajectories of visually guided manual pointing. Journal of Vision. 2006;6(9):982–995. doi: 10.1167/6.9.11. [DOI] [PubMed] [Google Scholar]
- Song JH, Nakayama K. Target selection in visual search as revealed by movement trajectories. Vision Research. 2008;48(7):853–861. doi: 10.1016/j.visres.2007.12.015. [DOI] [PubMed] [Google Scholar]
- Song JH, Nakayama K. Hidden cognitive states revealed in choice reaching tasks. Trends in Cognitive Sciences. 2009;13(8):360–366. doi: 10.1016/j.tics.2009.04.009. [DOI] [PubMed] [Google Scholar]
- Song J-H, Nakayama K. Automatic adjustment of visuomotor readiness. Journal of Vision. 2007a;7(5):1–9. doi: 10.1167/7.5.2. [DOI] [PubMed] [Google Scholar]
- Song JH, Nakayama K. Fixation offset facilitates saccades and manual reaching for single but not multiple target displays. Experimental Brain Research. 2007b;177(2):223–232. doi: 10.1007/s00221-006-0667-4. [DOI] [PubMed] [Google Scholar]
- Spivey MJ, Dale R. Continuous dynamics in real-time cognition. Current Directions in Psychological Science. 2006;15(5):207–211. [Google Scholar]
- Spivey MJ, Grosjean M, Knoblich G. Continuous attraction toward phonological competitors. Proceedings of the National Academy of Sciences. 2005;102(29):10393–10398. doi: 10.1073/pnas.0503903102. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Theeuwes J. Perceptual selectivity for color and form. Perception and Psychophysics. 1992;51(6):599–606. doi: 10.3758/bf03211656. [DOI] [PubMed] [Google Scholar]
- Tipper SP, Howard LA, Jackson SR. Selective reaching to grasp: evidence for distractor interference effects. Visual Cognition. 1997;4:1–38. [Google Scholar]
- Treisman A, Gelade G. A feature-integration theory of attention. Cognitive Psychology. 1980;12(1):97–136. doi: 10.1016/0010-0285(80)90005-5. [DOI] [PubMed] [Google Scholar]
- Watson DG, Humphreys GW. Visual marking: Prioritizing selection for new objects by top-down attentional inhibition of old objects. Psychological Review. 1997;104(1):90–122. doi: 10.1037/0033-295x.104.1.90. [DOI] [PubMed] [Google Scholar]
- Wolfe JM. Moving towards solutions to some enduring controversies in visual search. Trends in Cognitive Sciences. 2003;7:70–76. doi: 10.1016/s1364-6613(02)00024-4. [DOI] [PubMed] [Google Scholar]
- Wood DK, Gallivan JP, Chapman CS, Milne JL, Culham JC, Goodale MA. Visual salience dominates early visuomotor competition in reaching behavior. Journal of Vision. 2011;11(10):1–11. doi: 10.1167/11.10.16. [DOI] [PubMed] [Google Scholar]
- Yantis S, Jonides J. Abrupt visual onsets and selective attention: evidence from visual search. Journal of Experimental Psychology Human Perception and Performance. 1984;10(5):601–621. doi: 10.1037//0096-1523.10.5.601. [DOI] [PubMed] [Google Scholar]