Toward an integrated account of object and action selection: A computational analysis and empirical findings from reaching-to-grasp and tool-use

Abstract

The act of reaching for and acting upon an object involves two forms of selection: selection of the object as a target, and selection of the action to be performed. While these two forms of selection are logically dissociable, and are evidently subserved by separable neural pathways, they must also be closely coordinated. We examine the nature of this coordination by developing and analyzing a computational model of object and action selection first proposed by Ward [Ward, R. (1999). Interactions between perception and action systems: a model for selective action. In G. W. Humphreys, J. Duncan, & A. Treisman (Eds.), Attention, Space and Action: Studies in Cognitive Neuroscience. Oxford: Oxford University Press]. An interesting tenet of this account, which we explore in detail, is that the interplay between object and action selection depends critically on top-down inputs representing the current task set or plan of action. A concrete manifestation of this, established through a series of simulations, is that the impact of distractor objects on reaching times can vary depending on the nature of the current action plan. In order to test the model's predictions in this regard, we conducted two experiments, one involving direct object manipulation, the other involving tool-use. In both experiments we observed the specific interaction between task set and distractor type predicted by the model. Our findings provide support for the computational model, and more broadly for an interactive account of object and action selection.

1. Introduction

A fundamental aspect of object-directed action, pointed out by Allport (1987), is that it depends on “two essential forms of selection: Which action? and Which object to act upon?” (p. 395). These two forms of selection are logically and experimentally dissociable, and of course there exists an abundance of research focusing on either one or the other in isolation. The two forms of selection have also been proposed to depend, to some extent, on separable neural pathways (Borowsky et al., 2005; Creem & Proffitt, 2001; Milner & Goodale, 1995; Ungerleider & Mishkin, 1982). However, in addition to recognizing this separability, it is also necessary to understand how object and action selection are coordinated during goal-directed behavior. As pointed out again by Allport (1987), such “integration or coordination is manifestly essential for the coherence of action” (p. 396).

A growing body of research indicates that action- and object-selection processes are indeed far from independent (Prinz & Hommel, 2002). First, the process of object selection can influence action selection. For example, Tucker and Ellis (1998) found that attending to photographs of familiar objects having handles (e.g., frying pans) facilitated button-presses on the same side of space as the handle (see also Tipper, Paul, & Hayes, 2006). Similar facilitatory effects of object viewing on grip selection have also been reported (Ellis & Tucker, 2000). Castiello (1996) found that, under certain circumstances, the shape of distractor objects influenced hand shape in reaching-to-grasp. Finally, a previous study of our own found that distractor objects strongly affording potential grasping actions (e.g., distractors with handles facing toward the acting hand) slowed responses to targets more than distractors with weaker affordances for grasping (Pavese & Buxbaum, 2002).

At the same time, other work has argued for an influence of action selection on object selection. Specifically, it has been proposed that during visual search, the intention to perform a specific action can affect the saliency of objects affording that action. Work by Craighero and colleagues (Craighero, Bello, Fadiga, & Rizzolatti, 2002; Craighero, Fadiga, Rizzolatti, & Umilta, 1998; Craighero, Fadiga, Rizzolatti, & Umilta, 1999; Craighero, Fadiga, Umilta, & Rizzolatti, 1996) and others (Musseler, Steininger, & Wuhr, 2001) found that preparation of a specific grasp facilitated response times to detect visual stimuli whose orientations were congruent with the planned grasp, but not incongruently oriented stimuli. A related study by Bekkering and Neggers (2002) found that fewer saccades were made to distractor objects differing in orientation from a predefined target, if subjects were prepared to grasp the target as opposed to merely pointing at it (see also Hannus, Cornelissen, Lindemann, & Bekkering, 2005). Further studies with neglect patients have shown that object affordances can both improve (Humphreys & Riddoch, 2001; Riddoch, Humpheys, Edwards, Baker, & Willson, 2003) and prevent object detection (Rafal, Ward, & Danziger, 2006). Finally, an important series of studies by Tipper and colleagues demonstrated that distractors close to the acting hand yielded more interference than distractors far from that hand (Tipper & Howard, 1997; Tipper, Lortie, & Baylis, 1992). Furthermore, the amount of interference was dependent on the ease of programming a potential movement to the distractor as compared to the target. Effects such as these suggesting preparation of responses to both targets and distractors were only observed when a reaching response was required, and not when the response was verbal (Meegan & Tipper, 1999).

Taken together, such previous findings suggest the presence of a bi-directional, mutually constraining relationship between the processes underlying object selection and action selection.¹ An important next step toward characterizing this integration would be to construct models that address not only both forms of selection, but also their dynamic interaction.

To be viable, any such model must align itself at least broadly with a set of basic observations from neuroscience. As we have already noted in passing, a range of findings have led to the conclusion that there exist a set of distinct, dissociable processing pathways running between posterior perceptual cortices and frontal motor areas. At a minimum, the data have been taken to indicate a distinction between a dorsal “where” pathway, primarily responsible for representing object location, and a more ventral “what” pathway, responsible for representing object form or identity (Ungerleider & Mishkin, 1982). Beyond this, further work has suggested that at least part of the dorsal processing stream may also be characterized as a “how” system, mapping from object morphology to specific motor responses (Barde, Buxbaum, & Moll, 2007; Borowsky et al., 2005; Creem & Proffitt, 2001; Milner & Goodale, 1995; Rizzolatti & Matelli, 2003). The characteristics of these separable processing pathways suggest that object and action selection may depend upon the pathways differentially. In particular, the task of action selection would seem quite closely aligned with the function of the putative “how” pathway, while object selection would seem to align more with the functions of the “what” and “where” pathways. Given this alignment, it seems reasonable to require that any model seeking to capture the relations between object and action selection be designed such that these two forms of selection occur within structurally dissociable processing streams.

In summary, putting together the messages proceeding from behavioral and neuroscientific work, the computational challenge is to show how object and action selection can occur within architecturally segregated networks, but nonetheless be closely coordinated.

2. A computational model of object and action selection

A pioneering effort to model the interactions between object and action selection was reported by Ward (1999). The model took the form of a rather simple neural network, containing three functional components: (a) a set of units involved in representing perceptual inputs, mapping these into separate pathways representing object location (i.e., a “where pathway”; Ungerleider&Mishkin, 1982), and object color and shape (a “what” pathway), (b) a set of units representing dimensions of motor response, including reach direction and hand shape and (c) a set of top-down inputs, biasing processing toward stimuli with specific characteristics or toward specific responses.

For all its simplicity, the Ward (1999) model is appealing for at least two reasons. First, its architecture implements the idea that object and action selection take place within separable processing pathways, as the neuroscientific data broadly suggest. Second, it provides a rudimentary account of how these parallel processing pathways might interact during object-directed reaching.

In order to further explore the dynamics of the Ward (1999) model, we implemented a modified version of the account, resulting in the neural network model shown in Fig. 1. Full details of our implementation, and its minor differences from Ward's original, are discussed in the following section and in Appendix A. Briefly, in line with Ward (1999), the model contains components relating to perceptual inputs and to action outputs, as well as top-down biasing inputs. On the perceptual side of the network, external inputs are applied to units coding conjunctively for spatial location (limited to ‘left’ and ‘right’) and object features including color and shape. These are linked with units representing color, shape and location. Units coding for location link to output units indicating reach direction (left vs. right), and units coding for shape link to units indicating a specific mode of manual action, e.g., grasp shape. (As discussed further in Appendix A, an additional output group coding for verbal color-naming responses was included for symmetry.)

Fig. 1.

Our adaptation of the Ward (1999) model. Circles: processing units; line segments: bi-directional excitatory connections; large arrows: points of input and output. Not shown: Top-down inputs, lateral inhibitory connections among units within each boxed group, and excitatory self-connections. Full details of the implementation are provided under Section 2.1 and in Appendix A.

Although this simple network is not intended as a neuroscientific model, it is worth noting that its components, as well as their connectivity, do map in a gross way onto specific neural systems. The object-representation portion of the model, although descended in our model as in Ward's from earlier psychological accounts of attention (in particular Phaf & Van der Heijden, 1990), assumes the same general form as more neurobiologically oriented models of object selection (e.g., Deco & Rolls, 2004). Perhaps more important, there is substantial evidence indicating that reach direction and hand configuration, represented separately in the model, are represented in different sectors of neocortex. For example, disabling the anterior portion of the intraparietal sulcus (AIP: Gallese, Murata, Kaseda, Niki, & Sakata, 1994) or the ventral premotor cortex (Fogassi et al., 2001) in monkeys leaves reaching intact but disrupts hand shaping during grasping. Similar dissociations between reaching and grasping have been observed in humans when AIP functioning is disrupted by lesions (Binofski et al., 1998) or transcranial magnetic stimulation (Tunik, Frey, & Grafton, 2005). Finally, there is a clear relationship between the top-down biasing signals, present in both the original Ward (1999) model and our update of that model, and the top-down biasing role attributed to the prefrontal cortex in recent theories of cognitive control and plan representation (e.g., Miller & Cohen, 2001).

In line with both neuroscientific (Desimone & Duncan, 1995; Duncan, Humphreys, & Ward, 1997) and behavioral (Tipper, Howard, & Jackson, 1997) theories of selection, the present model accords a central role to competition. As in the Ward (1999) model, connections between groups are reciprocal and excitatory, and connections between units within each group are inhibitory. This pattern of connectivity means that if inputs representing two objects are applied, the features of one object (color, location, shape) will tend to compete for representation with the features of the other, with one object's features eventually winning out over those of the other. The outcome of this competition is influenced by the model's excitatory top-down inputs, which can bias the model toward one object by supporting activation of one or more of its features.

A final critical assumption of the model, which once again aligns broadly with neuroscientific knowledge, is that activation can flow via both feedforward and feedback connections: As in Ward's original (1999) model, activation propagates not only from perception-related units to action-related units, but also in the opposite direction.

Despite the model's simplicity, its key assumptions – pathway segregation, biased competition, and bi-directional information flow – lead to rather complex and interesting dynamics, which we believe may shed light on the interaction and coordination of object and action selection in human behavior. In the next section, after providing some further implementational details, we provide an analysis of these dynamics, culminating in the identification of a strong and testable prediction of the model.

2.1. Model implementation

2.1.1. Network architecture

The model was implemented using the LENS neural network simulator (Rohde, 1999). The structure of the model is as diagramed in Fig. 1. Not shown in the figure are lateral inhibitory connections running reciprocally between each pair of units within all groups (perceptual input, location, color, shape, reach direction, manual action, and color name) and excitatory connections from each unit to itself. The connection weights employed for the simulations reported below were 3.0 for feedforward and feedback connections between groups, 1.0 for excitatory self-connections, and −2.0 for lateral inhibitory connections.

2.1.2. Unit activation function

Units assumed activation values between 0 and 1, based on their inputs. The net input of each unit at time step t was computed as

$${\mathit{\text{net}}}_j(t)=(1-\tau ){\mathit{\text{net}}}_j(t-1)+\tau \left(\sum _i{a}_i(t)w_{\mathit{\text{ij}}}+\beta \right)$$ (1)

where a_i is the activation of unit i, and w_ij is the weight of the connection from unit i to unit j, β is an inhibitory bias term set at −2.0 for all units, and τ is an integration constant set at 0.1. Net input for input units included a term for external input, which was either +2.2 or −2.2 (values chosen to generate an activation of 0.8 in the absence of other inputs. In the LENS simulator, this was accomplished by “soft-clamping” inputs at a strength of 0.8). Net input for units receiving top-down modulation (color and hand shape) included a term for this input, which assumed a value of either 0 or 0.25. Unit activations were based on the logistic function:

$$a_j(t)=\frac{1}{1+e^{-{\mathit{\text{net}}}_j(t)}}$$ (2)

2.1.3. Simulation procedure

At the outset of each simulation trial, net inputs were initialized at 0, and unit activations at 0.5. Each trial began with a period of 50 time steps run without external input, allowing the network to reach a steady baseline state. Next, top-down inputs were applied (as specified in the following section) for a period of 30 time steps. Finally, perceptual inputs were applied, and the trial continued for a maximum of 100 time steps. A response was considered to be generated when one reach direction unit and one manual action unit crossed an activation threshold of 0.7.

2.2. Model behavior

2.2.1. Object and action selection

In order to make clear how object and action selection occur in the model, it is useful to consider the model's behavior when inputs are presented to represent two objects: on the left, a blue object with a slender shape calling for a pincer grip, and on the right a red object with a broader shape calling for a full-hand grasp.² Assume further that the task being modeled requires locating and grasping a blue target, and that the appropriate task set is implemented by applying top-down input to the color unit representing blue. Fig. 2 shows how activations for a subset of units in the model evolve over time, given this set of inputs. The plot begins on the time step where the top-down input begins. This weak biasing input raises the activation of the blue color unit slightly. This change in activation feeds back to the visual input layer, with the result that when the external inputs are applied (vertical lines in the figure), the input unit representing blue/left activates more strongly than the one representing red/right – even though these units receive the same external input. Positive feedback between the blue/left input unit and the blue color unit, combined with lateral inhibition, leads the blue color unit to activate strongly, and suppresses the red color unit. Activation of the blue/left input unit grows as well, leading to increasing activation of the left location unit. This, in turn provides feedback support to all input units on the right, resulting in increasing activation of the slender/left unit, which in turn supports activation of the corresponding shape unit. As activation in the location and shape units builds, so does activation in corresponding reach direction and manual action units, resulting in strong activation of the units appropriate to the blue object (reach: left, manual action: pinch). This outcome is determined entirely by the initial top-down input. When top-down input is instead applied to the red color unit, the opposite reach and manual action units are ultimately activated.

Fig. 2.

Activation profile for selected units in the model given input representing a slender blue object on the left and a broad red object on the right, with top-down input to the blue color unit. The labels above each plot correspond to the group labels in Fig. 1. The y-axis indicates unit activation in the range (0, 1). The x-axis indicates cycles of processing, beginning with the onset of the top-down color input. The vertical bar in each plot indicates the onset of the stimulus inputs, 30 cycles following onset of the top-down input.

Note that, by the end of the interval displayed, the model has completed both of the steps Allport (1987) identified as necessary to object-directed action, deciding both the “category and/or mode of action” (e.g., pinch vs. palm) via the manual action units and “where the action is to be directed” via the reach direction units. In the present example, the outcome of these decisions was determined indirectly by the top-down input to the color unit, which biased processing toward blue objects. In this case, the top-down input most immediately guides object selection. Note, however, that top-down inputs can also directly bias the process of action selection by activating the model's action-related units. For illustration, consider a case where the system receives the same inputs as in the previous example (left: blue, slender; right: red, broad), but this time around the task is to grasp whichever target affords a pincer grip. The appropriate task set, in this case, is implemented by applying top-down input to the manual action unit coding for a pincer grip. Fig. 3 shows the time-course of activation for the same units as Fig. 2. As in the previous example, biases induced by top-down input feedback to the input layer, leading indirectly to the selection of all of the features belonging to the relevant object, and to the appropriate action outputs.

Fig. 3.

Activation profile for selected units as in Fig. 2, but with top-down input to the palm output unit.

Together, these samples of the model's behavior illustrate two of its most important characteristics. The first is the reciprocal and interactive relationship between object selection and action selection. The second is the critical role of top-down inputs in determining the outcome of both forms of selection, and the details of their interaction. As in Ward (1999), these top-down inputs are intended within the present model to represent pre-established plans of action, e.g., ‘grasp the blue object’ or ‘grasp whichever object affords a pincer grip.’ The overall function of the model thus resonates with the proposal by Duncan (1993) that human behavior, in general, may be best understood as resulting from a three-way interaction among stimulus conditions, candidate actions and plan representations. On a more concrete level, the model gives rise to a testable prediction concerning the impact of task set on distractor interference in object-directed reaching, as we shall now explain.

2.2.2. A plan-dependent distractor effect

Consider a scenario in which a subject is asked to reach to and grasp a target object, which can be identified and differentiated from a distractor object by its color. Imagine, further, that there are two kinds of trials the subject may encounter. In one trial-type (match trials) the distractor object is identical to the target object, except for its color. In the other (non-match trials), the distractor also differs from the target in both its shape and in the action it most strongly affords. Assume, finally, that the dependent measure is time from stimulus presentation to reach completion. The model we have been considering gives rise to the prediction that the relative impact of the two distractor conditions should depend critically on the subject's preparatory set or plan of action. Specifically, it should depend on whether selection of the action to be performed must await target identification, or instead is known and prepared ahead of time.

The former of these task or plan conditions can be simulated in a fashion very similar to our first illustrative example above (see Fig. 2), where top-down input was applied to the blue color unit. In that example, the affordance of target and distractor objects differed. This parallels the non-match trial condition, as we have labeled it. The model can also be used to simulate the experiment's other trial type (match), where target and distractor share the same affordance, by providing the same top-down input to the blue color unit and then applying inputs representing left: blue/slender, right: red/slender. Fig. 4 (left panels) plots the activation of the left reach direction unit and the pinch action unit, in these two scenarios. In order to simulate reaching-time data, we assume that reach completion requires completion of both object and action selection. Thus, in the model, reaching completion is considered to occur on the first processing cycle on which units in both the reach direction and manual action portion of the model have crossed a predefined activation threshold (see Section 2.1 above). Reaching time is then quantified as the number of processing cycles between stimulus onset and reach completion. Here, as in all of our simulations, cycles of processing are assumed to relate linearly to units of time.³

Fig. 4.

Activation profiles for manual action:pinch and reach direction:left units, formatted as in Figs. 2 and 3. The horizontal line in each panel marks the response threshold of 0.7. The dotted line and its label indicate the reaction time in cycles. Upper-left: contingent action condition, non-match distractor. Lower-left: contingent action condition, match distractor. Upper-right: planned action condition, non-match distractor. Lower-right: planned action condition, match distractor.

In comparing the upper left (non-match) and lower left (match) panels in Fig. 4, note that the overall reaction time is smaller in the match case. This is largely because action selection, i.e., activation of the pinch unit, is faster on match trials. In the non-match condition, action selection must wait until the target object is identified, since the two objects presented have different affordances. In the match case, where both objects invite the same action, action selection can (and does) occur earlier. In contrast, object or location selection is slowed in the match condition relative to the non-match condition, since the early activation of the pinch unit results in transient support for the distractor object, through feedback connections. In the end, however, this slowing of object selection in the match condition is outweighed by the speedup on action selection, resulting in an overall faster reaction time for this condition.

These simulation results address the situation where the subject must wait until the target object is identified in order to select the action to perform upon that object. What happens in the case where the action is known ahead of time, and becomes part of the preparatory set? Let us assume that the subject we are simulating knows ahead of time that the target object will be slender, and thus that the appropriate action will be to use a pincer grip. This can be simulated by applying top-down input to the blue color unit (since color is still needed uniquely to identify the target object), but now also to the pinch output unit. Fig. 4 (right panels) shows the model's behavior under these conditions. In contrast to what was observed with top-down input only to the color group, reaching time is faster in the non-match condition than the match condition. There is once again a speedup in action selection in the match condition. However, because the task-relevant action is ‘known’ from the outset, this gain is smaller than before. Ultimately the speedup in action selection is outweighed by the slowdown in object selection in the match condition. This slowdown is larger than in the contingent action case (left panels in Fig. 4); the earlier onset of action selection leads to more prolonged top-down activation of the distractor object in the match condition.

One way of summarizing the behavior of the model is in terms of the interference induced by distractor objects in each condition. This can be defined as the reach completion time when inputs are applied representing target and distractor objects minus the completion time when only the target object is represented in the input. Fig. 5 shows interference measures for each task condition (planned and contingent action) and distractor types (match and non-match). Of course, the precise pattern obtained depends on the parameters of the model. However, as documented in Appendix B, what remains invariant across parameter settings is the presence of an interaction between task and distractor condition, according to which the difference in interference between non-match and match trials is larger (more positive) in the contingent action condition than in the planned action condition. This interaction is clear in Fig. 5, where the difference is positive in the contingent action condition and negative in the planned action condition.

Fig. 5.

Interference scores from the model in the four conditions shown in Fig. 4.

2.3. Testing for plan-dependent interference

If the foregoing simulations are on target, they suggest that relations between object and action selection in object-directed behavior may be most clearly understood by explicitly taking into account the role of plan representations or preparatory set. Indeed, the interaction illustrated in Fig. 5 may be described as a plan-dependent distractor effect, since the pattern of behavior depends not only on the relationship between the distractor and target objects, but also on the initial plan, implemented as a pattern of top-down input biasing both object and action selection.

Of course, the findings from the model are only compelling if the observed interaction can also be seen in human behavior. Some support for this possibility comes from a previous study in which we investigated the effects of pre-planning an action (or not) in different experiments with different subjects, stimuli, and response requirements (Pavese & Buxbaum, 2002). In an experiment in which actions were blocked and could thus be planned in advance, we found that matching distractors slowed response times more than non-matching distractors. In contrast, in another experiment in which actions were contingent on target affordances, non-matching distractors slowed response times more than matching distractors (see Fig. 7, top).

Fig. 7.

Top: Summary of results from Pavese and Buxbaum (2002), Experiments 2 and 3. Left: Bars labeled Variable show mean distractor interference effects from Experiment 2, averaging across target conditions. Here, subjects grasped target objects (coffee mugs) with handles, and pointed to targets without handles, in the presence of distractors with handles (match) or without (non-match). Bars labeled Fixed show mean interference scores from Experiment 3, averaged across response conditions. In one response condition, subjects pressed a button with the left or right hand to indicate the left/right location of a separate target button object, in the presence of distractors that were either buttons (match) or handles (non-match). In another, subjects reached for handle targets, in the presence of either handle (match) or button (non-match) distractors. Right: Differences between mean interference scores (non-match minus match) for fixed and variable conditions. In both panels, error bars show standard error, averaged across conditions contributing to the displayed means. Middle: Results of Experiment 1. Left: Interference scores by block type and distractor type. Right: Differences between interference scores for distractor conditions (non-match minus match) for each block types. Error bars in both plots show the 95% confidence interval, calculated so as to partial out variability due to the main effect of subject (as recommended by Loftus & Masson, 1994). Bottom: Results of Experiment 2, presented as for Experiment 1.

While these results fit with the performance of the model, there is the possibility that at least some portion of the observed plan-dependent interference effects was the result of cross-experiment variation. The present study sought to overcome this limitation by manipulating action planning as a within-subject factor. We conducted two experiments testing for the predicted interaction between distractor condition and task setting. In each, subjects reached to and acted upon a target object, identified by color. On some trials this target was presented on its own (baseline trails), on others with a distractor object, differing in color. When present, the form of the distractor either matched that of the target, thus inviting the same action (match trials), or differed, calling for a different action (non-match trials). Each experiment contrasted a condition where the identity of the target object was consistent across trials, allowing the mode of action to be pre-planned (fixed-target condition) with a condition where the identity of the target changed unpredictably across trials, meaning that action planning had to wait until the target was identified (variable target condition).

3. Experiment 1

3.1. Methods

3.1.1. Participants

Twenty right-handed college-aged students (ages 18–22, 10 women) participated in Experiment 1. Handedness was assessed using the Edinborough Handedness Inventory (Oldfield, 1971). Participants gave informed consent and were paid for their participation. The experimental protocol was approved by the University of Pennsylvania Institutional Review Board.

3.1.2. Apparatus

The display apparatus consisted of a wooden structure (21 in. high× 23 in. wide×8 in. deep) with four platforms (each 4 in. wide×4 in. deep; arranged in two rows of two platforms) fitted with touch-sensitive microswitches. Only the two lower platforms, mounted 2 in. from the base of the apparatus and 6 in. apart from each other, were used in the present experiment. The apparatus also included a start button mounted on the tabletop immediately in front of the subject, aligned with center of the base of the display.

The stimulus objects employed are shown in Fig. 6 (top). Each involved either a pencil or a drawer handle mounted on a wooden block (15.0 cm wide, 6.5 cm high, and 5 cm deep for pencil stimuli, 16.5 cm wide, 9 cm high and 9 cm deep for handle stimuli).⁴ The pencil or handle could appear either in blue or purple, making for four stimulus object-types in all. As explained further below, only blue stimulus objects were used as intended targets for reaching; purple objects always played the role of distractor.

Fig. 6.

Top: Experimental setup for Experiment 1, showing the handle and pencil target objects employed. Bottom: The tools and target objects used in Experiment 2.

Portable visual occlusion spectacles (PLATO, Translucent Technologies, Inc.) were used to control the timing of stimulus presentation. The lenses of these goggles can rapidly (∼5 ms) switch between their light scattering, occluding state and their transparent state, during which 90% of incident light is transmitted. The display platform, goggles, and a start button were all connected to a PC. Custom software was used to time the trials, play auditory stimuli, and record responses.

3.1.3. Procedure

Participants were seated directly in front of the display apparatus with the center of the apparatus at midline. This placed the two lower platforms approximately 11 in. away from the front of the body and approximately 10 in. below eye level when looking straight ahead. Before each trial the experimenter placed the target on the platforms while the participant's goggles were occluded. To control the start position on each trial and to obtain a measure of movement initiation time, participants started each trial by depressing the start button with the right index finger. The experimenter began the trial after determining that the participant was ready (seated with head and body midline and pushing the start button). At the start of the trial, a tone was played, and after 500 ms the goggles cleared. The participant then initiated his or her movement by lifting off the start button and using the right hand to move the target forward on its platform. The target was to be identified by color, with target objects always appearing in blue, and distractors always in purple. Participants were instructed to contact only the protruding (pencil or handle) portion of the target. The instructions indicated that, if the target was a pencil, the subject was to use a pincer grip. If the target was a handle, a full-hand grasp was to be used. The touch-sensitive microswitches were used to record the time at which the target was perturbed. If the subject failed to respond within 10 s, reaction time was not recorded and a long error tone was played. Response accuracy was judged and recorded by the experimenter.

Each participant completed two fixed-target blocks and two variable-target blocks. The four blocks followed an ABBA design, and the first block's type was counterbalanced across subjects. Each block consisted of 60 trials, which included 20 baseline (no distractor) trials, 20 match trials (distractor identical to target except for color), and 20 non-match trials (distractor different from target in both color and shape), interleaved in a random order. Subjects were told before each block began whether the target type would be fixed (consistent target object) or variable (unpredictable target object), and if fixed, whether the target object would be the handle or pencil. The left/right position of the target object, and on variable-target blocks the identity of the target, were counterbalanced across trials.

3.1.4. Analysis

We judged that measuring only reaching times, rather than detailed reach or grasp kinematics, would be sufficient to test the central prediction of the model, which ismost simply expressed in terms of the time needed to complete both object and action selection, rather than either one independently. The apparatus allowed for the collection of three measurements of the time it took to complete the task: initiation time (time between the goggles opening and the participant lifting off the start button), movement time (time between lifting off the start button and contact with the stimulus), and total reaction time (RT; the sum of initiation time and movement time). Although our interest in this study was the time it took participants to plan movements, we chose to analyze RT rather than initiation time, which may seem to offer a purer measure of planning time. This decision was made because the initiation time could have been influenced by strategy (see Tipper et al., 1997). For example, as a strategy participants may have initiated movements as quickly as possible, potentially before selecting which object (the target or distractor) to move toward. If this occurred, these decisions would have to be made after liftoff, thereby lengthening the portion of the movement captured with the movement time measurement. By using total RT as our dependent variable, we could account for planning done after liftoff.

In designing our analyses, we considered that raw RTs might be affected by the specific locations of target and disctractor (Chieffi, Gentilucci, Allport, Sasso, & Rizzolatti, 1993; Jackson, Jackson, & Rosicky, 1995) or by the nature of the manual response (pinch or grasp), neither of which were relevant to the prediction being tested. To minimize the impact of these factors, and to isolate the effect of distractors, we focused most of our analyses on interference scores. The interference scores were computed for each condition by subtracting the mean for the no-distractor condition from the corresponding match and non-match distractor conditions. By using an interference score, we could compare the unique effects of distractors for trials with the same movement requirements.

In order to compute interference scores, the mean RT was computed for each combination of target object, target location, distractor type, and block, omitting from analysis any trial in which the RT was three or more standard deviations above the participant's mean RT or in which the participant moved the distractor instead of the target. For each combination of target object, target location, and block, the mean RT for the no-distractor trials was subtracted from match and from non-match trials. The resulting interference scores were then averaged so as to yield four overall interference scores for each subject, specifically, scores for match and non-match trials in variable-target blocks, and scores for match and non-match trials in fixed-target blocks. These interference scores were then analyzed in a repeated measures ANOVA with factors for task (fixed-target vs. variable-target) and distractor type (match vs. non-match).

3.2. Results

Mean interference scores for each block type and distractor condition are shown in Fig. 7. The ANOVA yielded no significant main effects (effect of block type, F(1,19) = 1.63, n.s.; effect of distractor condition F(1,19) = 0.02, n.s.). However, as predicted, a significant interaction between block type and distractor condition was obtained (F(1, 19) = 5.03, p < 0.05). Specifically, reaching on non-match trials was slower, relative to match trials, in variable-target blocks than in fixed-target blocks. Numerically, reaching on non-match trials was slower than on match trials in variable-target blocks, and faster than on match trials in fixed-target blocks; however, in neither case did these pairwise relationships approach statistical significance (paired t-tests: variable trials t(19) = 1.17, n.s.; fixed trials t(19) = 1.183, n.s.).

Although our predictions, and thus our analyses, were focused on total RT, exploratory analyses were conducted to evaluate the respective contributions of initiation and movement time. In order to ascertain whether the interaction effect obtained at the level of total RT was driven differentially by these components, we conducted a three-way ANOVA with factors for block type, distractor condition, and RT component (initiation time vs. movement time). This ANOVA naturally yielded the same interaction between block type and distractor condition as the earlier ANOVA on total RT (F(1,19) = 5.03, p<0.05). However, there was no significant three-way interaction (F(1,19) = 0.52, n.s.). In other words, the interaction between block and distractor conditions did not differ significantly between initiation time and movement time. Two-way ANOVAs conducted independently on initiation time and movement time did not reveal a significant interaction between block and distractor type for either measure (initiation time: F(1,19) = 0.50, n.s.; movement time: F(1,19) = 2.56, n.s.). In line with our initial argument for using total RT as a primary measure, we assumed that the absence of the interaction at the level of initiation and movement times might reflect the presence of variation in strategy across (and perhaps within) subjects, in terms of the degree of reach planning conducted prior to liftoff. In order to explore this, we adopted an approach introduced by Meegan and Tipper (1998). For each subject we calculated the ratio of mean initiation time to mean total RT, reasoning that subjects for whom this ratio was relatively large were likely to be “pre-planners,” i.e., they were likely to be completing a relatively large portion of the planning process prior to liftoff. This ratio measure was then incorporated as a covariate into the previous ANOVAs on initiation time and movement time. For initiation time, though not movement time, this revealed a significant interaction between block type and distractor condition (initiation time: F(1,19) = 6.24, p< 0.05; movement time: F(1,19) = 0.08, n.s.). Among initiation times, there was also a three-way interaction among block, distractor and the initiation-time ratio (F(1,19) = 7.03, p<0.05). A correlation analysis confirmed that, across subjects, the ratio of mean initiation time to mean total RT correlated with the size of the interaction between block type and distractor condition (r = 0.53, p< 0.05). That is, the magnitude of the interaction effect correlated with the degree to which subjects behaved like “pre-planners.”

3.3. Discussion

Experiment 1 tested for a predicted interaction between task conditions and distractor type in object-directed reaching. Based on a modified version of the Ward (1999) model, we predicted that the relationship between target and distractor affordances (matching vs. non-matching) would differ, depending on whether the identity of the target object was predictable. Consistent with the model, reach completion with non-match distractors was found to be slower, relative to match distractors, when the target object varied than when it was held constant. Though small in absolute size, the observed interaction is consistent with the idea, implemented in the model, that object selection and action selection are parallel and interactive processes, whose relative timing and mutual effects depend on the details of the task domain.

Although the strong predictions of the computational model apply to total RT, rather than to initiation or movement time, exploratory analyses indicated that the same interaction effect could be detected at the level of initiation times when analyses controlled for individual differences in strategy.

Despite the confirmation of the predicted interaction, pairwise comparisons between distractor conditions within each block type did not reach statistical significance. Although this result may appear to contradict the interaction effect, this is in fact not the case. First, the pairwise contrasts necessarily involved smaller effect sizes than the interaction analysis, making type II error more of a risk. Second, it is possible that the interaction effect may vary less across subjects than the pairwise relationships. In fact, the computational model prompts us to suspect the second of these two possibilities. As documented in Appendix B, when the parameters of the model are varied, the interaction effect remains relatively stable while the relationship between distractor conditions in the fixed-target task varies widely. Thus, the strong predictions of the model bear on the interaction effect more than on the pairwise contrasts between distractor conditions, at least in the fixed-target case.

4. Experiment 2

Experiment 2 sought to replicate and extend the results of Experiment 1, by testing for the same interaction with a different set of objects and actions. In contrast to Experiment 1, which involved direct manual contact with the target, Experiment 2 examined the case of tool-use. In the studies of object and action selection reviewed in Section 1, object affordances were defined by the ease with which an action could be performed on the object by the hand. Given the prominence of tool-use in the behavioral repertoire of humans, an important question is whether the previously observed relationships between objects, actions, and plans will be maintained when affordances are defined by the ease with which an action can be performed on a given object by a tool. To this point few, if any, studies have directly assessed this question. However, related studies in both humans and monkeys indicate that an attentional ‘envelope’ around the acting hand may expand to include a hand-held tool (Iriki, Tanaka, & Iwamura, 1996; Longo & Lourenco, 2006; Maravita & Iriki, 2004), and that the pattern of attentional allocation in hemispatial neglect is similarly malleable as a function of whether or not a tool is held and used (Farne & Ladavas, 2000). If a tool may act as a proxy for the hand in terms of attentional allocation, it seems plausible that tools might also assume the role of the hand in other aspects as well. In particular, holding a tool might affect basic representations of available actions, as well as representations of object affordances, and this has indeed been suggested (Goldenberg & Hagmann, 1998; Witt, Proffitt, & Epstein, 2005). If this is the case, then the mechanisms identified in our computational model should extend to the case of tool-use, and the basic pattern of results obtained in Experiment 1 should be replicable in the present experiment.

The participant's task was to use a hand-held tool to move a target stimulus. The tools and target stimuli are shown in Fig. 6 (bottom). Participants moved the targets using either a spear or hook tool, which was held throughout each block. Two different types of targets were used: targets with holes in them and targets with eyes attached to them. Either tool could be used to manipulate either target type. However, each tool was designed to be used more readily for moving one target type than the other. That is, the spear tool could more readily move the hole targets, and the hook tool could more readily move the eye targets, regardless of target position. In this sense, each of the target objects carried a stronger affordance for one tool than the other.

As in Experiment 1, target objects were presented either alone or with a matching or non-matching distractor, and distractor effects were compared between conditions where the target object varied unpredictably across trials and where the target remained constant.

Our laboratory frequently investigates older individuals who have suffered strokes. To leave open the option of studies comparing healthy and brain-damaged participants, we often recruit healthy, active subjects from a population that is age-matched to our stroke population.⁵ We chose to study older adults in the present experiment, with this objective in mind. Although this represents a change from Experiment 1, it should be noted that the same effect observed in Experiment 1 was also observed in an earlier set of experiments with older adults (Pavese & Buxbaum, 2002), as detailed in Section 1. Thus, we reasoned that if the same effect obtained in the present experiment, it could be interpreted to hold generally across reaching-to-grasp and tool-use situations, at least among older adults.

4.1. Methods

4.1.1. Participants

Nineteen right-handed older adults (13 females) participated in Experiment 1. Their average age was 60 (range 41–78), comparable to the participants in Pavese and Buxbaum (2002). Average length of education was 14.6 years. All participants had normal or corrected-to-normal vision and normal color vision. Participants gave informed consent and were paid for their participation. The experimental protocol was approved by the Albert Einstein Healthcare Network Institutional Review Board.

4.1.2. Materials and procedure

The display apparatus from Experiment 1 was used. The two tools, the four targets (two exemplar hole targets and two exemplar eye targets), and the four distractors are shown in Fig. 6 (bottom). As described below, only one target object and at most one distractor object were presented on each trial. The tools were 10 in. long and weighed approximately 200 g each. Both holes and eye were mounted on wooden blocks that were 8.9 cm×8.9 cm×8.9 cm.

The procedure for individual trials was identical to the one used in Experiment 1, except that subjects began each trial with the base of the tool handle depressing the button. Participants completed two testing sessions on consecutive days. Session one included four variable-target blocks, each containing 106 trials. Half of these, randomly intermixed, were the trials of interest: compatible trials where the target was the one most easily used engaged using the tool currently in hand (spear tool/hole target and hook tool/eye target combinations). The other half of trials were incompatible trials, where using the tool was still feasible, but more awkward (hook tool/hole target and spear tool/eye target combinations). Only data from compatible trials in the variable-target blocks were examined because the incompatible trials served as filler trials so that participants could not prepare a movement to the target before seeing it.

Session two included four fixed-target blocks, each containing 53 compatible trials. In both sessions, half of the blocks were completed with the spear tool and half were completed with the hook tool. The ordering of blocks was randomly selected, as was the side the target was presented on across trials. Before both sessions, half of the participants completed 40 practice trials with the spear and then 40 practice trials with the hook. The other half of the participants completed the opposite order of practice trials. To further reduce the effects of practice, the first 10 trials of each block were removed from analysis. When these first 10 trials of each block, as well as incompatible trials on variable-target blocks, were removed from analysis, each participant completed 192 compatible trials in both the fixed- and variable-target conditions.

In all blocks of both sessions, one distractor object was sometimes presented along with the target object. As in Experiment 1, targets and distractors were distinguished using color, with targets colored blue and distractors colored purple. Distractors were presented on two-thirds of all trials. One half of the distractor trials, the target and distractor were the same object-type (e.g., both had holes; match trials). On the other half of distractor trials, the target and distractor were different object-types (e.g., the target had an eye and the distractor had a hole; non-match trials). If the participant incorrectly moved the distractor, a buzzer sounded to indicate an error.

In summary, Experiment 2 had a 2 (task condition: fixed or variable)×2 (distractor type: match or non-match) design in which both variables were manipulated within participants.

4.1.3. Analysis

Trials in which RTs were three or more standard deviations above the participant's mean RT were discarded, as were trials in which the participant moved the distractor instead of the target. Interference scores were calculated as in Experiment 1, and entered into a two-way repeated measures ANOVA.

4.2. Results

The ANOVA yielded no significant main effect of block type (F(1,18) = 0.008, n.s.), or distractor type (F(1,18) = 1.48, n.s.). However, it did indicate a significant interaction between these two factors, F(1,18) = 4.98, p<0.05. In fixed blocks, there was no significant difference between match and non-match distractors (53 and 47 ms, respectively). In variable blocks, there was more interference from non-match than match distractors (62 ms vs. 41 ms, paired-t = 2.3, p < 0.05; Fig. 7, bottom).

As for Experiment 1, exploratory analyses were conducted to evaluate the contributions of initiation time and movement time to the interaction effect observed at the level of total RT. The results obtained closely paralleled those obtained in Experiment 1. A three-way ANOVA with factors for block type, distractor condition and RT component indicated no difference in the size of the block×distractor interaction between initiation and movement times (three-way interaction: F(1,18) = 1.18, n.s.). As in Experiment 1, independent two-way ANOVAs on initiation and movement time indicated no significant interaction between block and distractor for either one on its own (initiation time: F(1,18) = 0.83, n.s.; movement time: F(1,18) = 2.72, n.s.). However, again as in Experiment 1, when these analyses were repeated with a covariate indicating initiation time as a proportion of total RT, this yielded a significant interaction between block and distractor among initiation times (F(1,18) = 8.74, p < 0.01), though not for movement times (F(1,18) = 0.34, n.s.). As before, the two-way interaction for initiation time was accompanied by a three-way interaction among block, distractor, and the RT-proportion covariate (F(1,18) = 10.26, p < 0.01). Correlation analysis indicated, once again, that subjects for whom initiation time made up a relatively large proportion of total RT were also prone to show a relatively large interaction between block and distractor type in their initiation times (r = 0.61, p < 0.01).

4.3. Discussion

Experiment 2 sought to replicate the findings of Experiment 1 in a slightly different task context. Despite the use of tools to act upon target objects, instead of direct manual contact as in Experiment 1, Experiment 2 again yielded a significant interaction between target predictability and distractor type. As in Experiment 1, the differential impact of a match vs. non-match distractor depended on the degree to which actions could be pre-planned; reach completion with non-match distractors was again slower, relative to match distractors, when the target object varied than when it was held constant. These data suggest that the relationships between object affordances, actions, and plans remains similar whether the effector is a hand or a tool. This, in turn, suggests that the top-down effects of planned actions on object selection extends to tool actions;when holding a given tool, attention is directed preferentially to objects that are readily manipulated by that tool.

As in Experiment 1, exploratory analyses indicated that the interaction predicted by the model could be detected not only at the level of total reaching times, but also at the level of initiation times, after controlling for individual differences in strategy.

As in Experiment 1, the pairwise contrast between distractor conditions in the fixed-target case did not reveal a statistically significant difference. As noted earlier, and documented in Appendix B, this is entirely consistent with the computational model. The model does appear to predict a consistent difference between distractor conditions in the variable-target case, however, and such a difference was observed in the present experiment.

For reasons explained earlier, the present experiment involved participants in a different age range than those studied in Experiment 1. If further information were not available, it would be impossible to rule out the possibility that the observed effect stems from an interaction between age and reaching task, and so would not have been observed had younger subjects performed the tool-use task, or had older subjects performed the task from Experiment 1. The data from Pavese and Buxbaum (2002, see Fig. 7) ease this ambiguity, by showing that older subjects do show the predicted interaction even when tool-use is not involved.

5. General discussion

Selection for action in the context of object manipulation comprises two forms of selection: selection of a target object and selection of the action to be performed on that object. While a great deal is known about these forms of selection considered independently, comparatively little work has focusedontheir interrelations. Where data is available, it supports the view that action and object selection can overlap in time and influence one another reciprocally. A rudimentary computational framework for understanding this interactive relationship between object and action selection was proposed by Ward (1999).

As covered in Section 1, we explored the dynamics of a slightly modified version of Ward's model, applying the model to a variety of scenarios involving object-directed reaching in the presence of a distractor. Our simulations highlighted the critical role of representations of task set, or action plans, manifesting at the behavioral level as an interaction between (a) task or plan type (planned action or contingent action) and (b) distractor type (match to target or non-match). In two experiments, we observed the specified interaction in the behavior of subjects performing object-directed reaching. In Experiment 1, this was observed in the setting of direct manual action upon familiar objects (pencils, handles). In Experiment 2 the interaction was observed in a task involving the use of novel tools. In both experiments, the interaction reflected differences in the pattern of interference by non-match distractors when actions were planned in advance or deferred until after object selection. Specifically, non-match distractors caused less interference, relative to match distractors, when actions could be planned in advance. Indeed, in both experiments the trend was toward greater interference from match distractors when actions could be pre-planned, and greater interference from non-match distractors when they could not.

The present results replicate the pattern observed in a previous experiment involving a related task (Pavese & Buxbaum, 2002). However, they also extend those earlier findings in two ways. First, the present results confirm that the pattern we observed across separate studies with different subjects and stimuli in our earlier work can be replicated based on a within-subject manipulation in a single experiment. Second, they show that the results are robust across different sets of objects and actions, and whether or not hand-held tools are involved. The result with tool-use is of particular interest because it suggests that the Gibsonian notion of environmental “affordances” for action (the degree to which the environment and the actor are complementary, Gibson, 1979) can be extended to tool – object interactions, at least in humans.

According to the account embodied in the computational model, the observed interaction reflects the principle that a distractor object can affect reach execution by impacting either object selection or action selection, with the weighting between these determined by the nature of the task or plan. When the action to be performed is known and prepared from the outset, object selection becomes the rate-limiting process, and reach execution is slowed by a distractor that affords the pre-planned action. When the action to be performed is not known ahead of time, but is instead contingent on the target object, the process of action selection contributes more significantly to reaction time, and reach completion is slowed by a distractor that invites a different action than the target object.

At the most general level, the computational and empirical results presented here support the view that a comprehensive understanding of the relationship between object and action selection must additionally take into account the contribution of task set or plan representations, implemented as top-down biases on both kinds of selection process (see Duncan, 1993).

While the data we have presented are informative, it is important to note some aspects of our experimental approach that place limits on the data's interpretation. First, the only aspect of the reaching process measured in our experiments was response time (initiation time, transport time, and their sum). As discussed in Section 1, in conjunction with our model, we considered reach completion to depend on completion of both object and action selection processes. Clearly, a more detailed empirical test of the model would call for some method of tracking the progress of object and action selection simultaneously but independently. For example, additional information about object selection might be gathered through eye-tracking, and action selection could be more directly followed by tracking the hand aperture from movement initiation to completion.

Another limitation of our experimental design pertains specifically to the fixed-target condition. This condition was intended to present the subject with a situation in which the mode of action (e.g., pinch vs. palm) could be anticipated and, in principle, prepared ahead of target presentation. However, the fixed-target condition not only allowed the subject to anticipate the mode of action; it also allowed anticipation of the identity of the target object (e.g., pencil vs. handle). This leaves open an alternative explanation for our findings, which pertains only to object selection. Specifically, in the fixed-target condition, the identity of the target could have formed part of the search template, leading distractors matching the target to interfere with target selection more than in the variable-target condition. Indeed, although we simulated the fixed-target condition by applying top-down input to the units specifying mode of action (as well as to the color units), very similar results are obtained if top-down input is applied to the object-form units instead. This concern is mitigated by the deliberately close relationship between our fixed-target condition and the experimental task used by Bekkering and Neggers (2002). Although, as reviewed in Section 1, that study found improvements in discrimination among oriented targets when the task was to grasp a target of a specific orientation, this effect was not observed when the task was to point to the target. Since in both situations the form of the target was known to the subject ahead of time, the results from the Bekkering & Neggers (2002) study indicate that action anticipation per se can have an effect on attentional selection.

As we have suggested, the results of the present experiments provide empirical support for the computational framework introduced by Ward (1999), further developed as part of the present research. Given the complexity of the issues involved – fully evident in the behavioral phenomena we have examined here – explicit computational models of this kind provide a highly useful tool for expressing and developing theories of object and action selection. The specific model we have analyzed in the present work seems to offer a promising basis for further development. As we have discussed, one of its appealing aspects is that it captures rich aspects of behavior while adhering to basic constraints proceeding from neuroscientific research. Having stressed this, however, it is also obvious that such neuroscientific constraints are presently implemented in an extremely rudimentary and abstract fashion. Even despite this, Ward's (1999) original work shows how the model can be used to address neuropsychological findings relevant to object and action selection. One worthwhile project might be to incorporate further neuroscientific detail into the model, expanding the range of neuroscientific data with which it can make contact.

Appendix A. Appendix A: Departures from Ward (1999)

The model described in Section 2 was based closely on the model proposed by Ward (1999). However, there were minor departures from Ward's (1999) model, and for completeness we enumerate these here:

1. In the Ward model, units representing manual action (Ward's “how” group) did not receive input directly from units representing object form. Instead, they received input from two groups, one coding conjunctively for object location and manual action (“where × how”) lying between the how group and the object location group, and the other coding conjunctively for object shape and object action (“what × how”), interposed between the how group and the object shape group. The motivation for this arrangement derived from the fact that the details of a manual action may depend on the location of the target object and the observation that some actions (e.g., grasping) depend on object shape more strongly than other actions (e.g., pointing). Because the data at issue in the present work raised neither of these issues, and for simplicity, we chose the more straightforward approach of linking the manual action group directly to the object shape group.

2. In the Ward model, reach direction was considered to be determined directly by the units coding for object location. We chose to include a separate unit group to code for reach direction in order to acknowledge the distinction between perceptual codes for object location and motor codes related to reach trajectory.

3. Our model, unlike Ward's, includes units coding for color names. These units were included simply in order to ensure that the color units in the object-representation portion of the model had dynamics identical to the units representing object shape, with the justification that color representations are presumably linked to motor outputs (naming responses) in just the same way as object shape and location (compare Cohen & Huston, 1994). In further simulations, comparable results were obtained with a model that excluded the color-naming group, with compensatory changes to the weights connecting input to color and the feedback weights within the color group.

4. Our model used a slightly different activation function from the one used by Ward. This change, which is inconsequential with respect to the qualitative behavior of the network, was made in order to allow the use of gradient-descent learning algorithms in future simulations using the model.

Appendix B. Appendix B: Parameter space partitioning analysis

The simulations described in the main text yielded an interaction between task (fixed-target vs. variable-target) and distractor type (match vs. non-match). In order to establish whether this was an invariant property of the model, as opposed to one that was heavily parameter dependent, we employed the parameter space partitioning (PSP) algorithm described by Pitt, Kim, Navarro, and Myung (2006). This carves the parameter space of a model into regions yielding qualitatively different patterns of performance, and yields an estimate of the volume of each region. For the purposes of applying PSP, the model's parameters were permitted to vary in the ranges indicated in Table 1. In addition to the parameters specified in the main text, the PSP analysis also introduced three new parameters, which were included with the objective of exploring a wide space of possible model configurations. Two of these parameters (color overlap and shape overlap) governed the degree to which the input patterns for the two colors (shapes) resembled one another. With color overlap set to its minimum value of 0, the color of a left-positioned blue object was represented with inputs of 1.0 to the left/blue input unit and 0.0 to the left/red input unit. With color overlap set to its maximum value of 1, the inputs became 0.5 and 0.5. (Note that, to maximize transparency, soft-clamping was not used in these simulations.) Shape overlap had the analogous effect on inputs to the input units representing shape. The third parameter, labeled affordance overlap, was intended to capture the fact that object shapes do not uniquely afford a single manual action, but afford different actions to different degrees. Links were added to the model running from the shape:broad unit to the action:pinch unit and from the shape:slender to the action:grasp unit. The weights of these connections were set to a × 0.5o, where a is the parameter controlling feedforward/back excitation and o is the affordance overlap parameter. The remaining (standard) connections between the shape and manual action units were given weights equaling a × (1 − 0.5o).

Table 1.

Model parameters and their ranges in the PSP analysis.

Parameter	Minimum	Maximum
Feedforward/back excitation	0	10
Self-excitation	0	10
Lateral inhibition	−10	0
Bias	−10	0
Top-down input: color	0	1
Top-down input: manual action	0	1
Preparatory cycles	0	50
Color overlap	0	1
Shape overlap	0	1
Affordance overlap	0	1
Response threshold	0	1

In the course of the PSP analysis, at each parameter setting explored, the model was tested on all combinations of task conditions (fixed-target, variable-target) and distractor types (none, match and non-match). Based on the resulting reaction times, interference scores were computed for fixed/match, fixed/non-match, variable/match and variable/non-match conditions. The model's behavior was then characterized in terms of: (1) which (if either) distractor type yielded greater interference in the fixed-target condition, (2) which distractor type yielded greater interference in the variable-target condition, and (3) whether the difference between the two distractor effects (non-match minus match) was greater in the fixed condition, the variable condition, or neither. Parameter settings were excluded from the results if they yielded an incorrect reach direction or manual action output, if they led the model to cross threshold prior to the presentation of the stimulus objects, or if they prevented the model from crossing threshold within 100 cycles of stimulus presentation.

The results of the PSP analysis are shown in Fig. 8. The pattern obtaining in the largest region of parameter space was the one shown in Fig. 5: in the variable condition, interference was greater for non-match distractors, but in the fixed condition there was greater interference from match distractors. Although this was the most frequent pattern generated by the model, obtaining in 48.7% of the parameter space, it was not the only pattern the model could produce. In 37.7% of the parameter space, non-match distractors caused more interference than match distractors in the variable condition, but in the fixed condition no difference was observed between the two. The third most common pattern, occupying 13.7% of parameter space, involved greater interference for non-match distractors in both fixed and variable conditions. Thus, the pair-wise differences between the two distractor types (especially in the variable condition) was quite sensitive to changes in parameterization. What remained essentially invariant across parameter space was a particular relation between these two pairwise differences, that is, a particular interaction between condition (fixed vs. variable) and distractor type. Across the vast majority of parameter space (>99.99999%) a positive interaction between these two factors was obtained, such that (VN – VM) > (FN – FM), where VN indicates interference in the variable/non-match condition, VM the variable/match condition, FN the fixed/non-match condition, and FM the fixed/match condition.

Fig. 8.

Results of the PSP analysis. The pie chart shows the percentage of parameter space yielding qualitatively different patterns of behavior, defined in terms of the relationship between interference effects in different task and distractor conditions. F = Fixed-target condition, V = Variable-target condition, N = Non-match distractor, M = Match distractor. Not shown are patterns that occupied less than 0.01% of parameter space, which collectively accounted for 0.004% of the parameter space.

Analogous results were obtained with several different versions of the model, including a version that excluded color overlap, shape overlap, and affordance overlap, a version that excluded the color-naming output group, and a version that allowed for different thresholds, top-down input strengths and preparation times for fixed and variable conditions. In all cases, the specified interaction held throughout the vast majority of parameter space.