Catching fly balls in virtual reality: a critical test of the outfielder problem

Abstract

How does a baseball outfielder know where to run to catch a fly ball? The “outfielder problem” remains unresolved, and its solution would provide a window into the visual control of action. It may seem obvious that human action is based on an internal model of the physical world, such that the fielder predicts the landing point based on a mental model of the ball’s trajectory (TP). But two alternative theories, Optical Acceleration Cancellation (OAC) and Linear Optical Trajectory (LOT), propose that fielders are led to the right place at the right time by coupling their movements to visual information in a continuous “online” manner. All three theories predict successful catches and similar running paths. We provide a critical test by using virtual reality to perturb the vertical motion of the ball in mid-flight. The results confirm the predictions of OAC, but are at odds with LOT and TP.

Introduction

How does an outfielder in the game of baseball know where and when to run to catch a fly ball? The “outfielder problem” is a particularly demanding case of the general problem of interception (Fajen & Warren, 2007). To catch a fly ball, the fielder must not only intercept a moving target, but must do so at a specific place and time. Catching involves two phases guided by different processes: moving the body close to the landing point of the ball (e.g. Chapman, 1968), and moving the hand relative to the body to make the final catch (e.g. Peper, Bootsma, Mestre, & Bakker, 1994). The outfielder problem concerns the first phase. Moreover, the fielder must accomplish this with sparse visual information; at an initial distance of over 30 m, most absolute distance cues (binocular disparity, vergence, motion parallax, optical expansion) are not detectable (Cutting & Vishton, 1995), nor are variables such as the ball’s physical velocity and spin. Approaches to the outfielder problem differ in important ways, such as whether an internal world model or an informational coupling is used to guide behavior. Thus, determining how fielders solve this problem carries implications for general principles of model-based and online control in perceptual-motor tasks.

The first solution to come to mind is trajectory prediction (TP). Saxberg (1987) proposed that fielders perceive the initial conditions of the ball’s motion and compute its trajectory to predict where and when the ball will land. TP thus requires accurate perception of the ball’s distance, speed, and direction of motion, as well as variables such as spin, air density, and gravity that can significantly affect the trajectory (Adair, 1990; McBeath, Nathan, Bahill, & Baldwin, 2008). To predict where the ball will land, these variables must be combined in a sophisticated internal model of projectile motion that takes them all into account. Evidence for such a model is lacking; even skilled baseball players can’t identify correct ball trajectories or predict landing points (Shaffer & McBeath, 2005).

Alternatively, fielders could use visual information to continuously guide their movements, leading them to the landing position at the right time without explicitly predicting either variable. By directly coupling their movements to optical variables in an online manner, this approach avoids dependence on an internal model and computation of the trajectory. Two theories of this type have been proposed.

Optical acceleration cancellation (OAC) (Chapman, 1968; McLeod & Dienes, 1996, McLeod, Reed, & Dienes, 2006; Michaels & Oudejans, 1992) posits that radial movement toward and away from the ball is controlled by cancelling the vertical acceleration of the ball’s optical projection (i.e. the tangent of the ball’s elevation angle, tanα, or its projection on a vertical plane, see Figure 1). If the ball’s optical velocity (dtanα/dt) is increasing, the fielder should accelerate backward, and if it is decreasing, accelerate forward, thereby keeping the optical velocity approximately constant. Tangential movement perpendicular to the direction of the ball is controlled independently by matching the ball’s lateral position (Chapman, 1968), either by keeping the ball in a constant bearing direction (azimuth ψ) or by holding its horizontal angle to home plate (β) constant (Fajen & Warren, 2007; McLeod, et al., 2006; Tresilian, 1995).

Figure 1.

Convention for angles and directions used by OAC and LOT. The fielder moves with velocity v, with radial (v_r) and tangential (v_t) components, while moving to catch the ball which originates at home plate (upper left). Angles to the ball are calculated in several planes: elevation angle (α), azimuth angle (ψ), horizontal angle between ball and home plate (β), ball-heading angle (δ), linear optical trajectory angle (γ).

OAC has been criticized on the grounds that the visual system is relatively insensitive to acceleration (Calderone & Kaiser, 1989; Schmerler, 1976). OAC, however, only requires detecting an increase or decrease in velocity with a threshold sufficient to bring the fielder close enough to the landing point to reach the ball (Tresilian, 1995). Visual judgments by seated observers are equivocal on this point (Babler & Dannemiller, 1993; Brouwer, Brenner, & Smeets, 2002; Oudejans, Michaels, Bakker, & Davids, 1999; Zaal & Michaels, 2003), and may depend on task context or fielder skill level. We thus chose to approach the problem by testing whether the theory can account for actual catching behavior.

In contrast, linear optical trajectory (LOT) theory (McBeath, et al., 1995) posits that the fielder runs so as to keep the apparent trajectory of the ball linear (from the fielder’s point of view), such that it appears to move in a constant direction γ relative to the horizontal. LOT has the virtue of not requiring that ball acceleration be detected; instead, LOT only requires that departures from a straight trajectory (i.e. constant γ), be detected. However, LOT does not produce a unique solution or a well-defined mapping from optics to action, but rather constrains the fielder’s movement to a family of paths; an additional constraint, such as that suggested by OAC to control radial movement, must be applied in order to generate a unique path. If such a constraint were applied, though, OAC and LOT would still differ in that LOT theory predicts that the fielder’s radial and tangential movements are coupled to maintain a constant γ, implying that any factor affecting one would have an impact on the other. In OAC, radial and tangential movements are independent, which provides a means by which the two theories can be tested.

Distinguishing the three theories (TP, OAC, and LOT) in the real world is complicated by the fact that they all predict successful catches with the same starting and ending positions, and only small differences between predicted paths (e.g. McLeod, Reed, & Dienes, 2002; Shaffer & McBeath, 2002). One way to dissociate them is to use virtual reality to create physically impossible trajectories by perturbing the ball in flight and testing which relations are held constant (Marken, 2005). Specifically, when we change the trajectory of the ball from parabolic to linear in mid-flight, thereby perturbing its vertical motion, TP, LOT, and OAC make different predictions about the fielder’s response. TP holds that the ball’s landing point is predicted from initial conditions, and thus a mid-flight perturbation should not affect the fielder’s movement. OAC predicts that the fielder’s radial velocity will be adjusted so that dtanα/dt shifts to a new constant value, whereas the tangential velocity will be unaffected so that the horizontal angle β is unchanged. On the other hand, LOT predicts that the fielder’s radial and tangential velocity will be coupled to maintain γ, preserving a constant ratio between tanα and tanβ over the perturbation.

Independently, McLeod, Reed, Gilson, & Glennerster (2008) performed a similar manipulation of the ball’s trajectory and speed, testing four soccer players heading the ball in a 3.5 m × 3.5 m virtual environment. They reported results consistent with a generalized version of the OAC theory, according to which the player keeps α increasing at a steadily decreasing rate (McLeod, et al., 2006), and inconsistent with TP and LOT. However, the ball’s anterior-posterior and lateral landing positions were varied in separate experiments. In the present study, we vary the landing position in both directions simultaneously in a 12 m × 12 m virtual environment, eliciting large radial and tangential movements of the outfielder, to provide a stronger test of the three theories.

Methods

Participants

Twelve (8 male, 4 female) experienced college baseball and softball players with normal vision took part in the study.

Apparatus

Data collection took place in the VENLab, a 12 m × 12 m area in which participants were able to move about freely. Participants wore a Cybermind Visette-Pro HMD (60 deg Horizontal × 46.8 deg Vertical Field of View, 640 × 480 pixel resolution for each eye) covered with a black shield, so the surrounding field was dark, and also wore a baseball glove on the left hand. Synoptic images of a virtual environment were generated on an Onyx2 graphics workstation (SGI, Mountain View California) using World-ToolKit software. The virtual world consisted of a Julesz-textured ground plane, blue sky, and a white baseball with red stitching (diameter=7 cm). Head and glove position (4 mm RMS) and orientation (0.1° RMS) were measured using IS-900 hybrid inertial/ultrasonic trackers (InterSense, Burlington, Massachusetts), and head position and orientation were used to update the view of the environment at 60 Hz.

Task

Participants were asked to catch virtual fly balls presented in the HMD, while moving freely. Balls started at a distance of 36.22 m and landed at one of four locations relative to the participants’ initial position (Figure 2): ±4.00 m in the forward-backward direction and ±4.00 m in the left-right direction, similar to real-world conditions tested by McLeod, Reed, & Dienes (2001). The task as seen by a participant wearing the HMD is shown in Movie 1, while an external view of the participant is shown in Movie 2. Following 16 practice trials with normal trajectories, 8 normal trials and 8 perturbed trials were tested at each final position, in random order. Normal fly balls followed a parabolic path, with normal gravitational acceleration (9.8 m/s²) and no air resistance. Perturbed fly balls followed a parabolic trajectory until the apogee (25.00 m), at which point the ball descended on a linear path with a constant vertical velocity (11.11 m/s); this altered its vertical optical acceleration, but not its lateral optical motion.

Figure 2.

Schematic of the experimental conditions. (a) Top view. Crosses show the landing point of the ball for the four final ball positions, located +/− 4.00 m in the forward-backward and left-right directions. Axis conventions are shown, with +Z being up. The distances are not to scale. (b) Side view of the trajectory of the ball in normal and perturbed conditions.

Movie 1.

Download video file ^{(453.3KB, mov)}

The task as seen by a participant wearing the HMD.

Movie 2.

Download video file ^{(893.8KB, mov)}

The task as seen by an external viewer.

Analysis

Since we were interested in the initial phase of catching (moving to the landing position), we defined “catches” as trials in which participants brought either the head or the glove within one meter of the ball. About half of the fly balls in each condition were registered as catches (forward normal 47%, backward normal 44%, forward perturbed 55%, backward perturbed 52%), and only those trials were included in subsequent analyses. The percentage of catches was lower than might be expected in a comparable real-world situation, possibly due to the encumbrance of the HMD and its limited field of view. Nevertheless, we replicated the basic pattern of paths observed by McLeod et al. (2001), in which the fielder’s path tends to curve slightly toward home plate when running forward, and curve in a symmetrical direction when running backward (see Figure 3 for paths from one participant).

Figure 3.

Paths for a representative participant, for normal (blue) and perturbed (red) trajectories. Each trial started at X=0 and Y=0, with the final ball positions shown with green circles.

Head and glove positions were filtered with a Butter-worth filter (5 Hz cutoff) and time series of the angles α, β, γ and ψ (Fig. 1) and their derivatives were calculated. Linear regressions were performed on tanα, tanβ, ψ, and γ vs. time and tanα vs. tanβ for each trial. A mean R² value across all trials was calculated for each experimental condition for each participant. Because no significant differences between fly balls to the right and left of the participant were observed, these trials were grouped together. To avoid transients at the beginning and end of each trial, only data between 1.00 s and 3.47 s were analysed for the whole trial, while data between 1.00 s and 2.17 s were analyzed for the first half of the trial and between 2.30 s and 3.47 s for the second half. Comparison of R² values between the first and second half of trials, and between normal and perturbed fly balls, was made using a z test following a Fisher z-transformation.

Results

Optical Acceleration Cancellation (OAC)

OAC holds that the rate of change of tanα (elevation angle) is constant on normal trials, implying a linear relationship between tanα and time. We thus computed the linear regression of tanα on time for each trial (see Figure 4 for a representative participant; data for all participants may be seen in the auxiliary files). In the normal conditions, mean R² values exceeded .99 for both backward and forward fly balls (Table 1). In the perturbed condition, however, the slope changed after the perturbation, yielding significantly lower R² values in that condition (z(72)=8.61, p<.0001, for forward and z(72)=6.63, p<.0001, for backward fly balls). Because a shift in the slope of tanα to a new constant value would be expected if the fielder cancelled the new optical acceleration caused by the perturbation, we performed separate regressions for the first and second half of each trial. The mean R² values were high on both halves of the trial (>.99 in most conditions), confirming that slopes were also linear following the perturbation. The exception was the second half of the perturbed backward trials, in which the fit (R²=.84) was significantly lower (z(32)=13.52, p<.0001) due to the flat slope. The sum of squared error in this condition, indicative of departure from linearity, however, was not significantly different from the backward normal trials (t(11)=1.65, p=.1272), which suggests that the linear increase in tanα with time held, even in the perturbed condition. This confirms that the ball’s optical acceleration was approximately zero both before and after the perturbation, consistent with OAC.

Figure 4.

Time series of tan(α) for a representative participant, collapsed across left and right fly balls. The time of perturbation is indicated by a tick mark on the x-axis. Thick cyan and magenta lines show the fits of the model for the normal and perturbed fly balls, respectively (see Model).

Table 1.

Mean R² values across all participants, for the first half of the trial and second half of the trial separately, and for the entire trial. Trials where participants moved to the left and right were grouped together.

	Forward Normal	Backward Normal	Forward Perturbed	Backward Perturbed
tan(α) vs. time
First	.9994	.9999	.9993	.9998
Second	.9916	.9933	.9924	.8408*,**
Entire	.9931	.9941	.5914**	.8494**
tan(β) vs. tan(α)
First	.9938	.9961	.9939	.9954
Second	.8594*	.9553*	.8847*	.8347*,**
Entire	.8511	.9408	.5217**	.6874**
tan(β) vs. time
First	.9938	.9961	.9939	.9954
Second	.9048*	.9558*	.9338*,**	.9593*
Entire	.8618	.9476	.8766	.9485
ψ vs time
First	.8873	.9780	.9011	.9634
Second	.9633*	.9519*	.9613*	.9297*
Entire	.8657	.8869	.8746	.9016
γ vs time
First	.8892	.9786	.9001	.9643
Second	.9639*	.9527*	.9626*	.9316*
Entire	.8658	.8871	.8750	.9020

^*Significant difference (p<.05) between R² for the first and second half of a trial.

^**Significant difference (p<.05) between R² for normal and perturbed fly balls.

Linear Optic Trajectory (LOT)

LOT (McBeath et al., 1995) posits that fielders run in such a way as to keep the ball’s optical trajectory straight (i.e. a constant angle γ) throughout a trial, which is equivalent to maintaining a linear relationship between tanα and tanβ (between the ball’s elevation angle and horizontal angle to home plate). To test this, we performed a linear regression of tanβ on tanα for each trial (Table 1, Figure 5). In the normal condition, the R² values were high, with mean values of .8511 and .9408 for forward and backward fly balls respectively. However, the fits were significantly lower in the perturbed condition, with mean R² values of .5217 and .6874 for forward and backward fly balls (z=4.0887, p<.0001 and z=5.417, p<.0001, respectively). As before, this might be explained by a shift to a new linear slope following the perturbation, but this account appears unlikely for two reasons. First, separate regressions for the second half of the trial yielded significantly lower R² values than for the first half in all conditions (z≥4.92, p<.0001). Second, an analysis of the horizontal angle tanβ over time (Table 1, Figure 6) revealed no differences in slope for the whole trial between normal and perturbed trials (z=−.36, p=.7188, for forward and z=.05, p=.9601, for backward fly balls). Thus, contrary to LOT, the perturbation affected tanα but not tanβ, indicating that the participant’s radial and tangential movements are not actually coupled.

Figure 5.

Plot of tanα vs. tanβ for the same representative participant as in Figure 4. Thick cyan and magenta lines show the fits of the model for the normal and perturbed fly balls, respectively (see Discussion).

Figure 6.

Time series of tan(β) for a representative participant, collapsed across left and right fly balls. The time of perturbation is indicated by a tick mark on the x-axis. Thick cyan and magenta lines show the fits of the model for the normal and perturbed fly balls, respectively (see Model).

Tangential Movement

Chapman (1968) originally proposed that, to control tangential movement, the fielder simply matched the lateral position of the ball. This might be implemented by nulling the rate of change (or acceleration) of the horizontal angle between the ball and home plate (tanβ) or of the bearing direction of the ball (azimuth ψ). We tested these strategies by regressing tanβ and ψ on time during the first and second halves of each trial (Figures 6 and 7, Table 1). Mean R² values were high for both variables (>.866), with significant differences between the first and second half of the trials in both normal and perturbed conditions (|z|≥2.0297, all p<=.05); R² increased in the backward conditions for ψ and decreased in the other conditions. The strong fits both before and after the perturbation suggest that either variable could be used to control the fielder’s tangential movement.

Figure 7.

Time series of ψ for a representative participant, collapsed across left and right fly balls. The time of perturbation is indicated by a tick mark on the x-axis. Thick cyan and magenta lines show the fits of the model for the normal and perturbed fly balls, respectively (see Model).

Discussion

Our results confirm that participants are able to respond to a physically impossible trajectory and catch perturbed fly balls. Surprisingly, they were equally successful at catching both perturbed and normal fly balls, in a manner consistent with the Optical Acceleration Cancellation theory (Chapman, 1968). Participants reacted to the perturbation by recovering a linear increase in tanα (elevation) over time, with a different slope. This linear relationship was quite strong, with R² values exceeding .99 in most conditions, both before and after the perturbation. Moreover, participants maintained this relationship while moving both radially and tangentially. The recovery of linearity after the perturbation is surprising because it requires a large correction in running speed to restore a constant optical ball velocity, and a varying running speed to maintain it. This finding provides strong evidence that participants cancel the ball’s optical acceleration to control their radial movements.

The fielder’s mid-flight adjustments are inconsistent with the Trajectory Prediction theory (Saxberg, 1987) that the ball’s trajectory and landing point are predicted from the first part of the ball’s trajectory; if TP were used to guide fielders to the correct place to catch the ball, there would be no reason to expect the linear increase in tanα.

Contrary to the Linear Optic Trajectory theory (Mac-Beath et al., 1995), the perturbation in the ball’s vertical trajectory did not affect the fielder’s tangential movement. Although tanα (elevation angle) was changed by the perturbation, tanβ (horizontal angle to home plate) was not affected, and a linear relationship in time for tanβ was observed throughout the entire trial, regardless of the perturbation. The change in tanα but not tanβ resulted in a change of γ (trajectory angle) that cannot be accounted for by LOT. Instead, it appears that radial and tangential movements of the fielders are controlled independently, consistent with OAC.

If tanα and tanβ are independent, how are the fielder’s tangential movements controlled? We tested two versions of Chapman’s (1968) hypothesis that the fielder matches the lateral position of the ball, either by nulling the rate of change in the horizontal angle between the ball and home plate (tanβ) or in the bearing direction of the ball (ψ). The high correlations indicate that either variable could be used to control the fielder’s tangential movement. This is not surprising because they are closely related, but the bearing direction ψ is more general because ”home plate” need not be in view to determine the angle.

These results converge with those of McLeod et al. (2008), who found that soccer players head the ball in a manner consistent with a generalized version of OAC (McLeod et al., 2006). In their version of the theory, the player keeps α increasing at a steadily decreasing rate, whereas in Chapman’s (1968) original version, this is accomplished by maintaining a constant rate of increase in tanα. The results in perturbed fly ball conditions seem somewhat at odds with the generalized version of OAC: after the perturbation, α and tanα actually decrease in our experiment (top of Figure 4), or increase at an increasing rate (bottom two panels of Figure 5 in McLeod et al., 2008). In addition, McLeod et al. (2008) needed to introduce a memory component to explain the tendency for dα/dt to return toward its pre-perturbation trajectory, whereas if tanα is used to guide movement, this tendency is simply a consequence of keeping dtanα/dt constant at its post-perturbation value, without having to invoke a memory component. We therefore believe that keeping dtanα/dt constant provides a better control principle for catching fly balls.

Model

Our results are consistent with OAC for the fielder’s radial movements, and a constant bearing strategy for tangential movements. To determine whether this theory could actually generate the observed data, we created a dynamical model in which the fielder’s radial velocity is controlled by nulling the acceleration of tan(α), while simultaneously and independently controlling tangential velocity by nulling the velocity of tan(ψ), similar to the constant bearing model of target interception (Fajen & Warren, 2007). The model treats the fielder as analogous to a damped spring with a fixed point at d²tan α/dt²=0 and dtanψ/dt=0. The equations of motion are

$$\begin{array}{l}{\ddot{v}}_r=-b_r{\stackrel{.}{v}}_r-k_r\ast min(d^{2}tan(\alpha )/{dt}^2,c_r)\\ {\ddot{v}}_t=-b_t{\stackrel{.}{v}}_t-k_t\ast min(dtan(\psi )/dt,c_t)\end{array}$$ (1)

where b_r and b_l are damping terms, k_r and k_t are stiffness terms, and c_r and c_t are thresholds that approximate a sigmoidal response to the optical variables.

Model predictions were simulated using Euler integration with a step size of 1/60 of a second, using the actual ball positions from the experimental data for the ball trajectory. The fielder was assumed to not move for the first 0.5 s of the trial to simulate the delay between the ball’s launch and the fielder’s first step. Parameters were fit separately for each participant using NOMAD, a generalized search program using MATLAB script (Abramson, Audet, & Dennis, 2004). This routine minimized the squared error between the x- and z- positions of the model and the data, summed over all time points in all catching trials. The mean R² value was .5994 for the x-position and .4174 for the z-position (the best possible values were .8068 and .6836, respectively, determined by calculating R² values between the mean trajectory of each participant and all trials). Fits of the model are shown in Figures 4, 5, 6, and 7 with thick magenta and cyan lines for the normal and perturbed conditions respectively, using the parameters derived for the representative participant (b_r=19.38 1/s, b_t=.19 1/s, k_r=116.48 m/s, k_t=223.52 m/s², c_r=4 rad/s², c_t=4 rad/s). The model thus approximates the fielder’s movements when catching a fly ball by directly coupling optical variables to action.

Conclusion

In sum, the findings demonstrate that OAC can account for the fielder’s radial movements, even following large perturbations to the vertical motion of the ball. In contrast, the perturbations had no effect on the fielder’s tangential movements, refuting the key prediction of LOT that they should be linked. Finally, the rapid responses to mid-flight perturbations show that the fielder’s movements are continuously controlled, contrary to the standard TP theory. The results thus suggest that perception is used to guide action by means of a continuous coupling of visual information to movement, without requiring an internal model of the ball’s trajectory.