Neural processing of imminent collision in humans

Abstract

Detecting a looming object and its imminent collision is imperative to survival. For most humans, it is a fundamental aspect of daily activities such as driving, road crossing and participating in sport, yet little is known about how the brain both detects and responds to such stimuli. Here we use functional magnetic resonance imaging to assess neural response to looming stimuli in comparison with receding stimuli and motion-controlled static stimuli. We demonstrate for the first time that, in the human, the superior colliculus and the pulvinar nucleus of the thalamus respond to looming in addition to cortical regions associated with motor preparation. We also implicate the anterior insula in making timing computations for collision events.

1. Introduction

The detection of looming and estimation of time to collision (TTC) are fundamental for survival in the environment. These faculties are readily observable in most locomotor animals, and influence drivers', cyclists' or pedestrians' critical decisions on a daily basis. For an object moving at a constant speed, TTC can be computed using instantaneous distance and velocity. In most environments, however, such parameters are not directly available to the observer, and require estimation from three-dimensional scene information. As an object approaches the observer, the optical size of the object on the retina (τ) increases exponentially with time, as does the rate of expansion of the object, or looming [1]. It has been proposed that the optical variable τ, based on the relative rate of image dilation, can be used as an estimation of TTC [2]. Consistent with this interpretation, fear or defence responses to symmetrical expansion of a closed contour object in the visual field have been elicited in human infants and a variety of animal species [3,4].

Despite a wealth of research on the behavioural aspects of looming detection and TTC computation in humans, little has been done to establish how processes are represented in the human central nervous system. In the search to understand the neural basis of looming detection, the majority of research has been carried out using invasive methods on insect and avian visual systems. In the locust, the Lobula Giant Movement Detector (LGMD) neuron and the Descending Contralateral Movement Detector (DCMD) have been found to respond to looming stimuli [5]. In the pigeon, the tectofugal pathway in particular has been implicated in the detection of looming. This pathway consists of the optic tectum, nucleus rotundus and telencephalic entopallium. Three classes of neurons (tau, rho and eta cells) have been located in both layer 13 (the stratum griseum centrale) of the optic tectum [6] and the nucleus rotundus [7], which are sensitive to TTC, angular velocity and the moment when an object reaches a specific optical size. The tectofugal pathway is the avian homologue of the superior colliculus (SC) and pulvinar nucleus (Pu) in mammalian species [6] with layer 13 of the optic tectum thought to be the equivalent of the intermediate stratum of SC [8]. The SC has also been found to be involved in defensive, escape or cringe behaviour in avian species [9] and rodents, and it is a subpopulation of cells in deep and intermediate grey regions of the SC that are more closely associated with these behaviours [10,11]. Such behaviours would be consistent with a potential threat, such as a looming predator in the central or peripheral vision. Furthermore, the SC has connections to multiple regions of the Pu in primates [12], which in turn has reciprocal connections with a distributed network of cortical regions involved in vision, attention and multi-sensory processing. Thus, both these subcortical structures operate within a corticothalamic network and could allow for the modulation of motor and visual processing that is necessary for rapid response to threatening stimuli.

More recently, two studies have used human functional magnetic resonance imaging (fMRI) in order to determine the neural correlates of looming detection and TTC decision-making. Within our own laboratory, comparative judgements on pairs of looming stimuli were found to cause increases in BOLD response in superior parietal and motor cortex [13]. Complementary to this, the supplementary motor area was found to vary with the likelihood of collision in an egocentric TTC judgement task [14]. This sensorimotor response suggests that there is motor preparation in response to an approaching object even though execution is not intended, underlining the direct and impelling nature of looming events. The methodology employed in these studies, however, did not look at subcortical activations associated with looming, particularly those in the SC and thus could not confirm any equivalence to avian models. Furthermore, neither of these two studies employed a task which necessitated accurate TTC judgements, but rather used binary choice judgements.

In the present study we aim to look at the neural correlates, in humans, of estimating TTC for the approach of looming objects. Looming is associated with an increase in the relative rate of image dilation on the retina. Measuring the neural response to looming stimuli provides a challenge, as it is also inherently linked with changes in both local and global luminance and motion properties, which themselves can be represented neurally. In order to account for low-level visual effects, we used point dot stimuli that minimized the overall change in luminance across the time-course of an event and allowed us to present stimuli with similar local, but not global motion properties. A second challenge is to establish whether neural systems are responding to potential collision or just a change in a low-level motion property that might be dissociated from a collision stimulus. Gibson noted that ‘magnification reaches an explosive rate in the last few moments before contact … This accelerated expansion specifies imminent collision.’ ([1], p. 188). In this respect, a system that is sensitive to accelerated expansion (looming) is sufficient to act as a collision warning system, although this will not provide a precise estimate of when the collision will occur. In this first foray into establishing human neural processing of collisions, we focus specifically on the response to accelerated expansion and contrast this with both accelerated contraction and stimuli that have the same average local motion speed, and contain elements that accelerate while changing direction, but do not display accelerated expansion.

2. Methods

(a). Participants

Ten (seven males and three females) neurotypical, paid volunteers aged between 20 and 40 took part in this study. The study was approved by a local ethical committee; all participants were screened according to standard fMRI scanning guidelines and gave their written consent to take part in the study.

(b). Stimulus presentation

Stimuli were projected on to a screen at the end of the scanner bore via a projector in the scanner control room. Participants viewed the screen while lying in the bore of the scanner via a mirror positioned approximately 15 cm from their eyes. The screen refresh rate was 60 Hz and the resolution was 1024 × 768 pixels. The horizontal and vertical extent of the screen was 34° and 30°, respectively. Stimuli were presented monocularly to the left eye, with the right eye covered by a patch for the duration of the experiment. Thirty repetitions of each condition were presented in total using an n-back sequence so that each stimulus was preceded by each other stimulus an equal number of times. Furthermore, the inter-stimulus interval was varied between 7.5 and 9 s. Both these procedures seek to reduce bias in estimating event-related per cent signal change because of activity related to previous trials.

Stimuli were generated using Vizard (Worldviz) software that uses OpenSceneGraph libraries to present perspective correct three-dimensional stimuli. This allows all stimuli to undergo the transformations that would occur with a solid object moving in depth. For the moving stimulus, we used a ball composed of point lights, placed at 500 vertices around a sphere (the dots to the rear of the sphere were not occluded). These points expanded during flight as would texture elements on a real sphere, but the size of each element did not increase. In addition, we added a rotation of 45° s⁻¹, to each sphere during flight (1 s), randomized across three axes (pitch, roll and yaw) for each trial, which gave a compelling impression of a structured object moving in depth.

For all stimuli, two vertical lines were set behind the main stimuli at 3.87 m and their colour (green, yellow or blue) indicated what trial type was about to occur. These lines were also used as part of the TTC task for the receding condition. The experimental sequence always commenced with the ball fading into the scene over 1.5 s. This then underwent motion for 1 s and then disappeared some time before a TTC button response was required. The looming ball (figure 1a) was 24 cm in diameter and travelled at 3.1 m s−1, it disappeared 0.25 s before reaching the observer (0.77 m from the observer and 17.7° optical size). The receding ball (figure 1b) created exactly the same series of images as the looming ball, but starting with the largest size (17.7°), and contracting down to the smallest. The observer pressed a button when they judged that the ball would reach a size that would pass through two yellow vertical lines 3.87 m away (although this size was never actually reached on any trial). This condition provided matched stimuli for the looming ball in terms of representing a structured three-dimensional object in motion, which required a TTC judgement and also had an equivalent distribution of local motion, but it presented accelerated contraction, rather than accelerated expansion. The random condition faded-in at the final size of the looming ball (17.7°), but the dot elements then underwent random motion at a speed equivalent to the mean rate of dot motion in the looming ball approach. The observer pressed a button when a blue square appeared approximately 0.25 s after the dot motion ceased. This presented a control for the manual response in the other conditions, but in this case the response was not based on anticipated arrival. This condition also acted as a low-level visual control. The dot elements had equivalent local motion properties averaged across the 1 s event and the elements changed the direction of motion (directional acceleration), but importantly there was no isotropic acceleration, which is a key feature of motion in depth. The comparison of random, looming and receding stimuli allows us to identify areas that respond to motion patterns signalling motion in depth and then whether this activation is owing to approach (collision) or just any motion in depth. All velocities and temporal parameters were randomized by ±10 per cent across trials so as to avoid habituation when making a TTC motor response.

Figure 1.

Sequence of events in a looming trial (a) and receding trial (b). (1) Lines change colour to cue stimulus type. (2) Ball appears by slowly fading in for 1.5 s. (3) Ball looms towards/recedes from viewer for 1 s. (4) Ball disappears and participant has to make a TTC judgement after a random interval.

(c). fMRI data acquisition and pre-processing

fMRI data were collected using a Siemens Trio 3 T scanner with an eight-channel head array coil. Functional images were collected using 38 slices covering the whole brain (slice thickness 3 mm, inter-slice distance 0 mm, in-plane resolution 3 × 3 mm) with an echo planar imaging sequence (TR = 3 s, TE = 29 ms, flip angle = 90°). All experiments in this study employed an event-related design and data were collected over three runs of 141 volumes each, with the first four volumes of all runs being discarded to allow for T1 equilibration effects. fMRI data pre-processing and data analysis were carried out using Brain Voyager QX. Prior to analysis, all images were corrected for slice timing using the middle slice as a reference slice. Images were realigned to the first image in the first session and resultant realignment parameters were used as regressors in the first-level general linear model (GLM) analysis. All images were transformed to Talairach space. Region of interest (ROI) analysis at the individual level was carried out on unsmoothed data; however, whole brain contrasts were carried out on smoothed (5 mm) images.

(d). First- and second-level GLM analysis

Beta values were estimated using the GLM in order to convolve the haemodynamic response function (HRF) with the time series of events. We removed low-frequency noise with a high-pass (GLM-Fourier) filter. An event in this case was classed as the onset of the stimulus fade-in to the offset of the stimulus some time before TTC (2500 ms). The period between offset and TTC was not included in the event time in order to avoid directly modelling early motor responses. A correction for serial correlations was employed using a first-order autoregressive model (AR-1). Six regressors were added to each model in order to model potentially confounding rotational and translational minor head movements in x, y and z coordinates; this was considered particularly important in the event that looming stimuli was associated with minor ‘avoidance’ head movements. At the group level, a p < 0.001 (uncorrected threshold) was used alongside a 10 voxel (vx.) cluster level extent threshold for general comparisons.

(e). ROI event-related analysis

The SC is a particularly difficult structure to image using fMRI because of its small size and its proximity to major blood vessels in the brain stem, subjecting it to cardiac noise [15]. Wall et al. [16] recently presented data which questioned the validity of using a standard (6 s) canonical HRF citing that a 4 or 5 s HRF was actually optimum for modelling activity in the SC. They noted a significant improvement in the detected activation in the SC without the need for complex scanning procedures or analysis. Thus, in order to maximize statistical inference, we did an additional ROI analysis on anatomically defined regions of the left and right SC. Each ROI comprised a 27 mm³ cube drawn over (and within) the SC using landmarks derived from a standard brain atlas ([17], see figure 2). In order to allow us to assess BOLD responses which may not be precisely modelled by a 6 s HRF, raw event-related time courses were extracted from each individual SC ROI. These per cent signal change time courses were averaged across all participants for left and right SC separately. Average BOLD response onset (mean for all conditions) was adjusted to 0 per cent and the data were then divided by the maximum data point in the dataset.

Figure 2.

27 mm³ ROI of left and right SC (inset) and the location of SC within the brain (main image, coronal perspective).

Bell et al. [18] found that a high-intensity stimulus that prompts a saccade resulted in earlier neural activation (approx. 50 ms) in the primate SC than a lower intensity stimulus, suggesting that neural responses in the SC can be mediated by light intensity. However, Bell et al. [18] used a stimulus that was 160 times brighter than its comparator, whereas the brightness of our stimuli in the fovea varied only by a factor of 5 for a distant ball (point lights compressed) versus a near ball (point lights dispersed). As our looming and receding stimuli have perfectly matched but reciprocal intensity ramps, the raw time courses also allowed us to determine whether there were temporal differences in the BOLD peak response owing to specific clusters of frames. The looming event presented a dense patch of dots in the centre of the screen at onset and a larger sparser patch of dots at completion; however, the converse were true for the receding stimuli. Examining time course values within the SC ROIs allowed us to assess this influence. If the BOLD response is consistently time locked to the start of the event (regardless of sequence order), we would expect a temporally consistent response peak for each condition regardless of the strength of that response; however, if the BOLD response is time locked to frames with a high level of luminance in the central visual field, we would expect a consistent 1 s offset across conditions.

3. Results

(a). Behavioural results

Compared with a veridical time estimate (TTC = 0), participants tended to respond only fractionally before TTC for the receding balls (mean = −0.043 s, s.d. = 0.229, d.f. = 9; t = −0.59, p = 0.57) and significantly after TTC for the looming balls (mean = 0.233 s, s.d. = 0.205, d.f. = 9; t = 3.60, p < 0.01). Late responses to looming balls may have been related to where participants considered their face to be in relation to the screen when asked to press the button ‘when they thought the ball would hit them in the face’. Although participants reported finding the receding condition harder, the more accurate responses may relate to being able to visibly see the point of contact. Given this pattern of results, the receding condition was considered a good cognitive as well as low-level visual control for the looming stimulus. Motor responses to the square cue in the random condition were compatible with a reaction time to the onset of a stimulus (mean = 0.537 s after, s.d. = 0.107, d.f. = 9; t = 15.74, p < 0.001). This suggests that participants were simply reacting to the appearance of the square rather than trying to anticipate its onset.

(b). MRI results

Making a TTC judgement to receding stimuli (receding > random, p < 0.001 unc., vx > 10) activated middle frontal gyrus (MFG), superior frontal gyrus (SFG), inferior parietal lobe (IPL) and post-central gyrus (PCG; table 1). In contrast, making a TTC computation specific to looming stimuli (looming > random, p < 0.001 unc., vx > 10) activated MFG, cingulate gyrus and anterior insula. Viewing looming balls in comparison with a high-level TTC control (looming > receding, p < 0.001 unc., vx > 10) resulted in the activation of inferior frontal gyrus, anterior and mid-insula, as well as basal ganglia and thalamic medial pulvinar (mPu).

Table 1.

Activated regions for timing computation and looming. The whole brain contrasts receding ball > random motion and looming ball > random motion were run in order to identify brain regions that were involved in making a time to collision judgement to a moving stimulus in comparison with a low-level visual and motor control stimulus. The contrast looming > receding was run to identify activation that was specific to perceiving an approaching object in comparison with a control matched for making a TTC judgement. All activations presented were at the p < 0.001 uncorrected level with a voxel extent threshold = 10.

condition	peak x	peak y	peak z	t	voxel n
receding > random
superior frontal gyrus	−6	−1	64	6.29	67
middle frontal gyrus	39	56	10	6.67	120
inferior parietal lobule	51	−46	40	5.20	26
post-central gyrus	−21	−49	64	5.49	33
looming > random
middle frontal gyrus	45	29	22	5.21	24
cingulate gyrus	3	11	44	6.34	68
anterior insula	−36	19	4	5.43	14
looming > receding
head of caudate nucleus	−9	2	4	5.58	25
external globus pallidus	−9	−1	−5	5.45	23
medial pulvinar nucleus of the thalamus	9	−31	−2	6.03	16
inferior frontal gyrus	48	26	13	5.78	11
anterior insula	39	22	7	6.41	20
mid-insula	36	−7	−11	7.01	29

Following the onset of stimulus fade-in each condition resulted in a 6 s peak BOLD response in left and right SC for the looming ball (figure 3). ROI t-tests showed that there were significantly increased responses in the left and right SC when viewing looming stimuli compared with both receding stimuli (L: t = 1.634, p < 0.05; R: t = 1.395, p < 0.01) and random dot motion (L: t = 1.664, p < 0.05; R: t = 1.157, p < 0.05); response peaks in the reverse and random conditions showed no significant difference however.

Figure 3.

Event-related time course per cent signal change values for bilateral SC. The event start and endpoints are marked by the dashed grey line; looming onset is marked by the solid grey line. Black thick line, looming; red dotted line, receding; green dashed lines, random.

There was no suggestion of a consistent 1 s offset in peak BOLD response when comparing the receding and looming stimuli in SC; thus, there appeared to be little consequence of having reciprocal luminance intensity ramps within trials. Although our three whole-brain contrasts presented no activation in visual cortices, in acknowledgement that activation in the SC could still be owing to low-level visual features of the looming stimulus, we carried out three repeat contrasts at a much more lenient threshold (p < 0.05 unc.) in order to ascertain whether low-level effect, such as average luminance intensity across trials were apparent in the BOLD response. The only contrast resulting in activation in visual cortices was receding > random; this activation was located in the anterior end of the calcarine fissure (x = 7, y = −72, z = 9) and occipital gyrus (x = −24, −85, −12), anatomically corresponding to V1v and V2d, respectively. The looming condition resulted in no additional visual activation in comparison with receding or random stimuli; thus strengthening our claim that SC response to a looming object is not simply a result of low-level visual features.

4. Discussion

This study has shown that an extensive network of sub-cortical and cortical regions respond preferentially to visually looming stimuli compared with salient stimuli with no accelerated expansion properties. At the sub-cortical level, a response to looming in human SC and mPu is in line with research findings in both the pigeon and the locust [5–7]. This finding also fits well with the role of SC in detecting visually salient stimuli and orienting or defensive movements of the head and eye to salient stimuli [11]. Activation associated with looming stimuli in the basal ganglia is also consistent with previous research. Inhibitory mechanisms in the basal ganglia network have been inferred to be involved in ceasing the ongoing motor behaviour when a looming, potentially dangerous, object is detected [19]. The basal ganglia output nucleus has efferent projections to the thalamus and pre-motor areas of the brain stem, including SC [20] and may play a role in mediating motor responses to looming stimuli entering personal space.

The mPu receives inputs from intermediate layer of the SC and as well as having projections to and from striate and extrastriate regions. It is also connected to a range of higher cortical regions, such as parietal, temporal, orbital frontal and cingulate cortex as well as the amygdala [12], and could thus play an important role in motor preparation with regard to looming stimulus. A recent review by Kaas & Lyon [21] which predominantly focused on primate literature implicates a subset of medial nuclei in the pulvinar as the subcortical component of the dorsal visual stream, with the majority of pulvinar projections to middle temporal (MT) visual area emanating from medial regions. A neural network that circumvents V1 in this way may provide a more direct pathway for the purpose of responding to the approaching stimulus. Given that the Pu of the thalamus plays a role in attentional modulation and orienting [22], looming stimuli may simply be more attentionally engaging, thus resulting in tectopulvinar activation. However, we argue that a looming-detection mechanism, which works on such a principle would still be effective for a rapid response to an approaching object.

Activations across frontal, parietal and cingulate cortex were present when making a TTC judgement to both looming and receding stimuli in comparison with random motion control. These activations are consistent with previous research and reflect the cognitive demand of the task [13]. Additional activation present in frontal and parietal cortical regions when making a TTC judgement to receding stimuli are consistent with the cognitive difficulty associated with making a computational TTC judgement in comparison with a low-level control [23] and encoding spatially salient aspects of the external environment [24]. Previously, comparative judgements made between two looming stimuli have been found to selectively increase the BOLD response in superior parietal and motor cortex [13]; in our study, these regions were only activated for receding stimuli. The receding condition was the only condition which required the use of multiple objects in the simulated environment as the participants had to judge TTC to a distant set of lines rather than to the face. This suggests that there is either a specific or additional role for parietal or motor cortex in making relative TTC judgements involving multiple objects in visual space as opposed to absolute judgements with a single object.

The anterior insula was more active when making a TTC judgement to looming stimulus in comparison with both the control and receding stimuli. This may bear some connection to this region's involvement in making duration judgments [23], error detection [25] and interoceptive awareness on a variety of tasks [26]. Together, this research suggests that this cortical region may be particularly attuned to making timing judgements with respect to objects in the environment which are approaching personal space.

Future research needs to focus on elucidating the role of attentional orienting in looming detection and associated tectopulvinar response. The two may be closely linked as looming stimuli tends to be alerting. Furthermore, disambiguating the contribution of different visual cues that are inherently linked to a looming object is a challenging next step. In this experiment, we controlled for global changes in luminance using point light stimuli, but manipulating object properties such as visible texture can bias the percept of looming [27]. The progression from point light stimuli, towards more natural textures that will produce different luminance ramps, is a balance that needs to be struck between ecological validity and experimental control in fMRI.

We conclude that an extensive network of regions is involved in both low-level detection of looming, sensorimotor response and higher level TTC estimation. The network contains the tectopulvinar early warning system, a motor preparatory system and possibly a more sophisticated computation system involving the anterior insula. These preliminary findings do have implications for humans' performance during everyday activities, such as driving or sports. If errors occur in collision detection, these may be owing to failure at a sub-cortical level rather than in higher level cognitive processing. The dedicated nature of these early detection systems may also shed light on the ‘footballer's dilemma’: being able to react very rapidly to objects on a collision course, but after the event being unable to elucidate what was perceived or how it was intercepted (or avoided).