Perceptual deficits in patients with impaired recognition of biological motion after temporal lobe lesions

Abstract

“Biological motion” may be defined by the pattern of movement of a small number of lights attached to the major joints of a human performing simple actions. Normal observers watching such displays immediately recognize a person and his or her actions. In the present study, we investigated the effects of lesions of anterior cortical regions on the perception of biological motion. We measured the performance on psychophysical static and motion tasks and on object and action recognition tests in four stroke patients who presented with a disorder of recognition of biological motion. We relate our results to the finding that neurons in the rostral part of the superior temporal gyrus (the superior temporal polysensory area) respond selectively to biological motion, and to the idea that the superior temporal polysensory area integrates the late stages of the dorsal and ventral cortical visual streams, as well as to recent functional MRI studies on biological motion.

Biological motion consists of the dynamic patterns generated by animal movements (1). It can be isolated in the laboratory by viewing, in the dark, a set of small lights attached near the joints of moving humans and other animals. From the pattern of lights generated by the movement, observers readily recognize actions such as running, walking, stair climbing, and bicycle riding. In the macaque, the superior temporal polysensory area (STP) (located in the anterior portion of the superior temporal gyrus) appears to process biological motion; some STP neurones respond selectively to patterns produced by biological but not other types of movement (2, 3). The presumed human homologue of macaque STP is selectively activated when subjects discriminate biological motion and not when they are viewing the same stimuli but are making a discrimination of the direction of motion (4).

Here we present data from four stroke patients who were unable to perceive biological motion after lesions that included the anterior temporal lobe. These patients performed normally on tasks of color and static form discrimination but were impaired on certain motion discriminations.

Methods and Results

Case Histories. The criterion for these four patients' inclusion in the present study was their inability to recognize biological motion, although their performance on basic form, color, and some visual motion tasks was by and large normal. They were stroke patients with right-hemisphere lesions involving the anterior portions of the temporal lobe. The normal control subjects were all right-handed and age-matched to the patients (ages 40–65). All of the subjects involved in this study gave informed consent according to the procedures of the Boston University Human Subjects Committee.

The neuropsychological and psychophysical results used in this study were obtained between 4 and 6 weeks after the occurrence of the stroke, when the patients were in a stable neurological condition but still were inpatients in the stroke rehabilitation unit.

Case 1. JR was a right-handed 61-year old bartender, admitted for rehabilitation after a massive right-hemisphere cerebral infarct. MRI of the brain revealed a large lesion in the right hemisphere involving the anterior and lateral portion of the temporal lobe anterior to the tip of the temporal horn of the lateral ventricle, a small portion of the basal ganglia and the globus pallidus, and slightly extending into the parietal lobe (Fig. 1A). On examination, the only neurological signs were left homonymous hemianopsia with spared 10° of the central visual field, left hemiparesis, and inability to copy simple shapes (e.g., cube, table, and daisy).

Fig. 1.

Axial slices of CT or MRI images of the patients' brains illustrating the locus of the lesions. (A) MRI images of patient JR's right-hemisphere lesion in the middle cerebral artery territory involving most of the temporal and parietal lobes and slightly the basal ganglion. (B) Patient RJ's brain; images obtained by CT scan show the right-hemisphere lesion involving the convexity of the temporal lobe with some extension to the basal ganglia and slight involvement of the anterior portions of frontal and parietal lobe. The major portion of the lesion involves the anterior right temporal lobe. (C) Patient IF; MRI images showing the area of increased signal intensity in the very anterior portion of the right temporal lobe an area of hemorrhage posterior to it, involving the posterior temporal lobe. (D) Patient LH; images obtained by CT scan, showing the lesion involving the anterior temporal lobe and the posterior limb of the internal capsule.

Case 2. RJ was a right-handed 57-year-old janitor, admitted after a right middle cerebral artery ischemic infarction. Computed tomography (CT) scan of the brain showed a low-density area involving the convexity of the right temporal lobe with extension into the basal ganglia and superiorly involving slightly the posterior-lateral frontal and parietal lobes (Fig. 1B). There was a marked weakness of the left arm and leg. Visual fields were full on confrontation, and copying simple shapes was normal.

Case 3. IF was a right-handed 59-year-old astronomy professor, admitted after an acute intracerebral hemorrhage in the right hemisphere. The MRI scan of the brain revealed a large lesion, ≈2.7 cm × 6.5 cm × 4.5 cm, involving the most anterior portion of the superior temporal lobe, and an additional area of different signal intensity measuring ≈2 cm × 1.5 cm was seen anterior to the recent hemorrhage. Biopsy confirmed that this was a malignant tumor. Clinically, the patient presented with left-hemiparesis and left-homonymous hemianopsia with sparing of the central 10°.

Case 4. LH was a right-handed 39-year-old instructor of physical education, admitted after craniotomy and clipping of a rupture of a right internal carotid aneurysm at the level of the posterior communicating artery. A postoperative CT scan of the brain showed a lesion involving the anterior portion of the temporal lobe and the posterior portion of the internal capsule. On examination, there was a dense left-hemiparesis and hemianopsia with macular sparing of the central 10°.

Neuropsychological Assessment. Verbal and performance IQ tests were prorated from the scores of six verbal and four performance tests of the Wechsler Adult Intelligence Scale Revised Test. All four patients were normal on verbal IQ; however, all were similarly impaired on the performance IQ (range 63–73 percentile). None of the patients was found to suffer from visual spatial neglect or from any form of visual agnosia. Discrimination of contrast sensitivity (5) and color (6) was normal, as was 2D shape detection and discrimination from noisy background or by total flux. However, JR and RJ were impaired on spatial localization. Each of these tests has been described in detail (7, 8).

Recognition of Biological Motion. The motion information in the “biological motion” display encodes nonrigid articulations of a human actor's limbs moving relative to one another. Johansson (1) demonstrated the sufficiency of biological motion for form perception by placing lights on the points of articulation of a human actor. The form of such an actor remains obscure while the person is static. But as he or she articulates, the shape becomes immediately obvious. The only information available in the display is provided from the dynamic source. The stimulus was presented on a videotape by using scenes from Johansson's original movie (9). In the display, one sees only the pattern of lights attached to the joints of the human actor during the performance of such actions as walking, stair climbing, riding a bicycle, push-ups, shaking hands, and hugging. The patients were shown the biological motion movie and were asked to verbally identify what they saw; they were not given feedback.

All four patients failed to recognize biological motion. They could not perceive a man walking or performing common actions. Instead, they reported seeing “white balls”, “golf balls”, or “pool balls.” When the display showed the man walking, all four patients reported that the balls were rolling from left to right; when the man riding the bicycle was shown, they reported seeing balls bouncing on the screen; and two men shaking hands were perceived as “balls going back and forth.” However, all four patients reported correctly the global direction of the “point lights” in the display, indicating that the failure of recognition was not due to “motion blindness.”

Perceptual Tasks. For normal observers and for a large class of patients with occipital-temporal or occipital-temporal-parietal lesions (4, 7, 9–13), the recognition of the “biological motion illusion” is obvious. Not only is the general shape of the body clearly and immediately visible, but it is as easy to tell in which direction the stimulus is moving as it is to discern the action portrayed by the movement. The surprising failure of the four patients to recognize biological motion, in contrast to their quick and accurate report of the direction of the”point lights“in the display, motivated us to complement their neuropsychological assessment with evaluation of visual motion perception and visual cognitive abilities.

Object and Action Recognition. Gollin figures test (14). The test stimuli consisted of a graded series of incomplete outline drawings. There were, in total, three drawings of each item, the third portraying a full outline of the object. Drawings of the 20 different objects were presented on individual cards, and subjects were asked to name or describe the object portrayed by the stimulus. The error scores for JR, RJ, IF, and LH were, respectively, 90%, 60%, 60%, and 50%, as compared with 15% for eight matched controls.

Action recognition (15). A videotape of 10 pantomimed actions was presented on a black-and-white TV monitor as silhouettes in motion (e.g., shoveling, sawing, plucking, pulling combing, or rowing). The ability to recognize the pantomimed action was assessed by asking the subject to identify in a four-alternatives forced-choice task the picture of the imaginary object whose use was depicted in the pantomime. The error scores for JR, RJ, IF, and LH were, respectively, 60%, 60%, 20%, and 40%, as compared with 5% for six matched controls.

Psychophysical Tests of Motion Perception. We used four psychophysical motion tasks to measure the ability to perceive different aspects of motion. In all experiments except Experiment 4, random dots were used to minimize familiar position cues and to isolate motion mechanisms. The stimuli were presented 2° to the left or right of fixation, so that we could compare performance for stimuli presented in the right and left visual fields (except in Experiment 4, which was presented with central fixation). In all cases, the stimuli were fully within the normal portion of the right or left visual field.

Experiment 1: Boundary from differences in motion direction (10, 16, 17). The display (subtending 8 × 8 deg^2e) contained two adjacent regions translating coherently, but in different directions, thus producing a discontinuity in the velocity field (Fig. 2A). The discontinuity appears as an imaginary vertical line with a notch. The notch (subtending 1.4 deg²) was always presented within 1° above or below the center of the imaginary boundary. Two angular differences between the two areas are used, 10° and 39.5°. Using constant stimuli and the two-alternative forced-choice procedure, we asked observers to determine whether the notch was above or below the center of the display.

Fig. 2.

Boundary from differences in motion direction. (A) A schematic view of the stimulus. The differential movement of two halves of the random dot pattern reveals a boundary with a rectangular notch. The arrows indicate (schematically) that the boundary and notch result from a direction difference in the display. (B) The bar graphs represent percent correct responses from normal controls and the four patients for identifying the location of the notch (above or below the fixation mark) for angular differences between the two areas of 10° and 36.9°.

For the large angle, all four patients had normal performance, whereas for the smaller angular difference, the scores of JR, RJ, and LH were significantly below those of IF and the normal control subjects (Fig. 2B).

Experiment 2: Discrimination of form from relative motion (18). This test addresses the subject's ability to extract 2D form from relative motion, by either direction or speed differences. The display consisted of a dense field of random dots with one of four 2D forms defined by motion contrast between the background (ellipse, triangle, square, or oblong) and a cluster of adjacent dots. The figure always appeared at the center of the display and moved horizontally along an imaginary runway. The shape subtended 2° × 2° and was undetectable when all of the dots were either stationary or moved in the same direction as the background. The speed of the background was 3°/sec. There were two conditions: (i) 2D form from direction difference (Fig. 2A). The figure and the background moved with the same speed but in different directions. The direction difference varied from easy (90° difference) to hard (1°). (ii) 2D form from speed differences (Fig. 2C). In this condition, the figure moved in phase with the background, but faster. The speed difference varied from 100% to 5% faster.

The stimuli were displayed by an adaptive staircase method (7), and direction difference or speed difference thresholds were calculated from the last six reversals. In a four-alternatives forced-choice task procedure, subjects were asked to identify the camouflaged 2D moving shape that appeared in the center of the display.

The performance of JR, RJ, and LH was not significantly different from that of the normal control subjects. Subject IF, however, was impaired on this task (Fig. 3B). He reported always seeing a shape in the middle of the display, but when the direction differences between the figure and the background were <15°, he was unable to correctly identify the shape. The good performance of LH, RJ, and JR is somewhat surprising, because in the direction discrimination task, they scored in the impaired range for directions of 10°. A possible explanation for this discrepancy may be that in the direction discrimination task (Experiment 1), subjects are asked to use direction to extract form and perform a spatial localization, whereas in this task, they use direction only to extract form.

Fig. 3.

Discrimination of form from relative motion. (A) A schematic view of the 2D form from direction differences stimulus. The black arrows indicate schematically the motion vectors of the figure (shown inside the elipse) and of the background. (Upper) Arrows portraying the motion vectors are equal in length but have different angle, referring to the direction discrimination task. (Lower) The arrows have the same direction (are shown parallel) but different lengths, indicating that the difference between the form and the background is solely in speed. (B) The bar graphs show the threshold of angle difference (y axis) necessary for reliable shape discrimination for 17 normal control subjects and the four patients. (C) The display is almost identical to A, except that the figure moves in phase with the background but differs in speed from the background. (D) The bar graphs show the threshold of speed difference (expressed in percentage on the y axis) required by normal controls (n = 17) and the four patients to discriminate reliably the camouflaged shape.

In the form from the speed differences test, RJ and LH had normal performance. JR and IF, on the other hand, were significantly impaired (Fig. 3D). RJ's poor performance on this task is surprising in view of his excellent speed discrimination ability, suggesting that his deficit may involve an inability to use speed information to extract form.

Experiment 3: Perception of 3D structure from motion. This task, described in ref. 7, investigates the subjects' ability to perceive 3D structure from motion. The display portrays two random dot kinematograms, one above the other, and is shown schematically in Fig. 4A.

Fig. 4.

Perception of 3D structure from motion. (A) A schematic representation of the two dynamic random dot fields used. (Upper) The structured stimulus generated by the orthographic projection of a square-shaped random dot velocity field onto a transparent cylinder that is rotated. (Lower) A velocity field representing the totally unstructured stimulus. (B) Comparison of data from JR, RJ, IF, LH, and 11 normal controls for the discrimination between a 100% structured cylinder point lifetime of 400 msec and an unstructured display.

The dots in the display had a finite lifetime (400 msec) and appeared and disappeared asynchronously. The dots in one of the kinematograms, the “structured” display, were the orthographic projection of points on the surface of a hollow cylinder rotating with 30°/sec around its vertical axis. The other kinematogram, the “unstructured” stimulus, was generated by randomly shuffling all of the velocity vectors (i.e., the path the dots travel in their lifetime) in the structured display. In a two-alternative forced-choice procedure, subjects were asked to determine which of the two kinematograms portrayed the rotating cylinder (which was always presented at 100% structure). The spatial positions (up vs. down) of the structured and unstructured fields were randomly assigned. There were 50 trials.

For stimuli shown in their contralesional visual field, JR, IF, and LH scored just above chance on this task, whereas RJ's performance was not statistically different from that of the normal control subjects (Fig. 3D).

Experiment 4: Long-range motion. This test addressed the subjects' ability to solve the “correspondence problem” (19). Correspondence is the process by which an object maintains its identity when seen in different places at different times, such as when objects presented in sequential-static pictures are seen in coherent motion. To solve correspondence implies first a process of selection and then matching of critical attributes of objects.

The task described here is adapted from refs. 4 and 20. Correspondence was measured by the ability of the subjects to determine the direction of rotation in a display consisting of two pairs of 2D Gaussian modulated sinusoidal gratings (Gabor patches) arranged at the four corners of an imaginary square. The stimuli consisted of four separate frames, displayed twice in succession for a total of eight frames in one display. Fig. 4A shows schematically the configuration of each of the four frames. Only the position of the Gabor patches is changed, not their orientations. The separation between centers of like Gabor patches is 3.6°. For a 45° rotation (from one frame to the next), each Gabor patch traverses a distance of 1.4° of visual angle. The frame duration was kept constant at 80 msec with an interstimulus interval of 48 msec, putting this display well within the realm of long-range motion. Gabor patches of five different central spatial frequencies were used [1.0, 1.7, 3.0, 5.0, and 10.0 cycles per degree (cyc/deg)], the Gabor patch of 5 cyc/deg being the reference spatial frequency in all comparisons. In each trial, eight frames interleaved with seven interstimulus blank intervals are displayed in one of two position sequences corresponding to clockwise or counterclockwise rotation. In a two-alternative forced-choice procedure with the method of constant stimuli, subjects were asked to discriminate clockwise from counterclockwise motion. Each combination of spatial frequencies was presented 12 times.

To be certain that the patients perceived the whole stimulus, we used a static control test. In this task, four Gabor patches of the same spatial frequencies as in the experimental condition were presented in the same configuration. The only difference was that the Gabor patches did not change position (a static display). One patch, placed randomly in one of the four positions in the display, differed in spatial frequency from the other three by the same ratios as in the experimental task. The subjects were asked to fixate on the central-hair crossmark and to pick out the odd Gabor patch. There were 20 trials in this task.

Compared with the performance of the normal controls, all four subjects were impaired on the long-range motion task (Fig. 4 C–F). However, they scored 100% on the static control task.

Discussion

In two patients, RJ and IF, the lesion was primarily in the anterior temporal lobe, whereas in the others, the lesions included portions of both the parietal and anterior temporal areas.

All four patients had normal performance on color and form detection and discrimination. Object recognition was normal in all patients except under the more difficult conditions where it was impaired relative to normal controls. Performance of the patients on psychophysical motion tasks varied. On the direction discrimination test, all four patients scored below the normal controls, and RJ's performance was particularly poor. IF was slightly impaired on the 2D form from direction; 2D form from speed was impaired in JR; and IF was impaired on the 2D form from speed differences. Only one of the four patients (RJ) had normal scores on 3D structure from motion. Finally, all patients were severely impaired on the long-range motion test, and all were completely unable to perceive biological motion (which was the initial criterion of including them in this comparative study).

Patients with an involvement of the posterior temporal and parietal lobes in their lesion were impaired on detecting 3D structure from motion cues. This is consistent with results from experiments on monkeys, where the discrimination of structure from motion can be impaired either by lesions of inferior temporal cortex (TE) (21) or of area MT (22), located in the superior temporal sulcus. Furthermore, the only patient who was normal on structure from motion had a lesion so anterior that it probably spared both the homologues of area TE and MT in humans.

The finding that all of the patients failed on both long-range and biological motion raises the possibility that both deficits reflect a common underlying dysfunction. However, for at least two reasons, it is unlikely that long-range and biological motion reflect a single mechanism. First, in a previous study, we reported a patient severely impaired on long-range motion but normal on biological motion (23). Second, Mather et al. (24) reported psychophysical evidence that biological motion is mediated not by long-range motion but by local motion mechanisms.

All of the patients were impaired on recognition of objects from degraded incomplete information (Gollin figures test). What, if any, might be the relationship between these recognition deficits and the deficit on recognition of biological motion? In monkeys, neurons in the anterior portions of the temporal lobe (STP) are sensitive to biological motion. STP receives direct information from the inferior temporal cortex, a “ventral” system area crucial for object recognition (2) and from areas MST and FST, “dorsal” system areas involved in the analysis of complex stimulus motion (25). STP can be viewed as integrating the outputs of the ventral system for object recognition and the dorsal system for the analysis of motion, and thus its damage would be expected to yield a deficit on biological motion perception. We suggest that the impairment in biological motion in our patients is, indeed, due to damage to the human equivalent of STP or its inputs. This is consistent with recent functional MRI studies in normal subjects showing that recognition of biological motion produces activations in the anterior part of the superior temporal sulcus, probably corresponding to the macaque STP (4, 26).

The pattern of RJ's behavioral impairments and cortical damage is particularly supportive of this view. His performance was normal on all stages of motion perception except for biological and long-range motion. Furthermore, his lesion was largely confined to the most anterior portion of the temporal lobe, presumably involving the equivalent of STP. In the other cases, the lesions included large portions of the anterior reaches of the dorsal or ventral system or both.

Finally, it should be noted that normal perception of biological motion is possible in the face of severe impairments of other kinds of motion perception, as we reported in the case of patient AF (7, 9, 11, 13, 27, 28). Because AF's lesion was particularly destructive of the dorsal system, we assume that his normal performance on these tasks depended on the ventral system and the unaffected STP.

Fig. 5.

Long-range motion. (A) Schematic representation of the four frames used to produce apparent motion. Circles labeled with the same letter represent Gabor functions of the same spatial frequency. Clockwise rotation is produced by presenting the frames in order {1, 2, 3, and 4}. Counterclockwise rotation is produced by presenting the frames in order (1, 4, 3, and 2). Each frame was composed of two pairs of Gaussian modulated sinusoidal gratings (Gabor functions). The two pairs of Gabors in each frame are oriented at 90° to each other and centered on a crosshair fixation point. The Gabors of each pair have the same spatial frequency. The Gabors in successive frames are rotated about the central fixation point by 45° from the previous frame. To produce a 360° rotation of the display, each trial was composed of eight frames. (B–E) Results from each of the four patients compared with 16 normal controls. The data for the normal control group are presented as a shaded area representing mean ± 1 SD. The patient's data are represented as mean and standard errors. The graphs represent percent correct as a function of spatial frequency of the varying Gabors (1.7, 3, 5, and 10 cyc/deg) against the Gabor pairs of 5 cyc/deg.