Impaired perceptual memory of locations across gaze-shifts in patients with unilateral spatial neglect

Summary

Right-hemisphere lesions often lead to severe disorders in spatial awareness and behavior, such as left hemispatial neglect. Neglect involves not only pathological biases in attention and exploration, but also deficits in internal representations of space and spatial working memory. Here we designed a new paradigm to test whether one potential component may involve a failure to maintain an updated representation of visual locations across delays when a gaze-shift intervenes. Right-hemisphere patients with varying severity of left spatial neglect had to encode a single target location and retain it across an interval of 2 or 3 seconds, during which the target was transiently removed, before a subsequent probe appeared for a same/different location judgment. During the delay, gaze could have to shift to either side of the remembered location, or no gaze-shift was required. Patients showed a dramatic loss of memory for target location after shifting gaze to its right (towards their ‘intact’ ipsilesional side), but not after leftward gaze-shifts. Such impairment arose even when the target initially appeared in the right visual field, before being updated leftward due to right gaze; and even when gaze returned to screen center before the memory probe was presented. These findings indicate that location information may be permanently degraded when the target has to be remapped leftward in gaze-centric representations. Across patients, the location-memory deficit induced by rightward gaze-shifts correlated with left neglect severity on several clinical tests. This paradoxical memory deficit, with worse performance following gaze-shifts to the ‘intact’ side of space, may reflect losses in gaze-centric representations of space that normally remap a remembered location dynamically relative to current gaze. Right gaze-shifts may remap remembered locations leftward, into damaged representations; whereas left gaze-shifts will require remapping rightward, into intact representations. Our findings accord with physiological data on normal remapping mechanisms in the primate brain, but demonstrate for the first time their impact on perceptual spatial memory when damaged, while providing new insights into possible components that may contribute to the neglect syndrome.

Introduction

Brain lesions can cause severe disturbances in spatial awareness and spatial behavior. In particular, left spatial neglect is a frequent and disabling multi-component syndrome, usually observed after extensive unilateral right-hemisphere damage (Driver & Vuilleumier, 2001; Driver, Vuilleumier, & Husain, 2004; Halligan, Fink, Marshall, & Vallar, 2003; Mesulam, 1999). The patient’s awareness and behavior is typically biased towards the ipsilesional side of space, leading to neglect for contralesional information (Bisiach, 1993; Karnath, 2001; Vallar, 1998). These patients may show striking failures in perceiving, orienting, reporting, imagining and/or remembering contralesional stimuli, even though their primary sensory-motor functions often remain intact (Driver et al., 2004; Heilman, Watson, & Valenstein, 2003). The neuropsychological mechanisms underlying this syndrome remain incompletely understood, yet they may provide a unique window on the neural processes involved in normal spatial cognition.

Brain lesions responsible for spatial neglect may involve various regions in parietal, frontal, and superior temporal cortex, plus related subcortical regions (Doricchi & Tomaiuolo, 2003; Hillis et al., 2005; Husain & Kennard, 1996; Karnath, Fruhmann Berger, Kuker, & Rorden, 2004; Mort et al., 2003), typically with some right-hemisphere dominance. All these regions are reciprocally interconnected within a distributed large-scale network that plays a crucial role in space representation, as well as in attention and other processes (Mesulam, 1999). Chronic persistence of left spatial neglect after a right-hemisphere stroke is usually associated with extensive vascular lesions that will disrupt several different brain regions (Buxbaum et al., 2004; Maguire & Ogden, 2002; Samuelsson, Jensen, Ekholm, Naver, & Blomstrand, 1997). Accordingly, spatial neglect is increasingly considered as a multi-component syndrome, often resulting from a complex combination of impairments, each with potentially distinct neural substrates (Driver et al., 2004; Halligan et al., 2003; Husain & Rorden, 2003).

Yet, the exact nature of the different neglect components is still unresolved. Major contributing deficits may include spatial biases in attention (Bartolomeo & Chokron, 2002; Driver & Vuilleumier, 2001; Rafal, 1994) and motor intentions (Bisiach, Geminiani, Berti, & Rusconi, 1990; Heilman et al., 2003; Mattingley, Husain, Rorden, Kennard, & Driver, 1998), as well as local perceptual biases towards parts over wholes (Halligan et al., 2003; Lamb, Robertson, & Knight, 1990), potential deficits in spatial working-memory (Husain et al., 2001; Pisella, Berberovic, & Mattingley, 2004), and non-spatial deficits in vigilance (Robertson, Mattingley, Rorden, & Driver, 1998). Several key aspects of neglect are attributed to some loss or distortion in internal representations of contralesional space (Bisiach, Ricci, & Modona, 1998; Halligan et al., 2003; Heilman et al., 2003; Karnath, 2001; Mesulam, 1999). Visual deficits in neglect patients may manifest in relatively complex or combined spatial coordinates, unlike strictly retinotopic visual field-cuts (Karnath, Schenkel, & Fischer, 1991; Ladavas, 1987; Vuilleumier, Valenza, Perrig, Mayer, & Landis, 1999), suggesting a possible role for dynamic representations of space in which locations are coded by combining visual information with extraretinal signals related to current gaze, trunk, or even limb position (Behrmann, Ghiselli-Crippa, Sweeney, Di Matteo, & Kass, 2002; Driver & Vuilleumier, 2001; Pouget & Driver, 2000).

Here we designed a new paradigm to test the implications of such dynamic recoding of spatial locations for neglect. We were particularly interested in the potential role of gaze-centric representations, which are now known to exist in some parietal and frontal areas in both monkeys (Colby, Duhamel, & Goldberg, 1995; Duhamel, Bremmer, BenHamed, & Graf, 1997; Umeno & Goldberg, 2001) and humans (Medendorp, Goltz, Vilis, & Crawford, 2003; Merriam, Genovese, & Colby, 2003). We investigated the possible impact of damage to such neuronal populations on perceptual spatial memory (rather than on oculomotor behavior per se). Recent single-cell recordings have revealed that visual responses of parietal and frontal neurons can be dynamically updated when gaze-shifts are made after stimulus onset (Colby et al., 1995; Umeno & Goldberg, 2001). Based on these findings, we predicted that right-hemisphere patients with left spatial neglect might exhibit a specific pattern of deficits in a spatial task requiring them to maintain location information for several seconds across changes in gaze position - as is typically the case in many normal everyday situations, and also in several clinical tests. By examining the effects of different directions of gaze-shifts on spatial memory, we could test a new and counterintuitive prediction, as explained below.

In monkeys, neurophysiological results during ‘delay’ paradigms have shown that neurons in parietal and frontal cortex can maintain spatially-selective activity when a location in their receptive (and/or motor) field remains task-relevant during the delay, even without any sensory input currently presented at that location (Andersen, Bracewell, Barash, Gnadt, & Fogassi, 1990; Chafee & Goldman-Rakic, 1998; Umeno & Goldberg, 2001). Critically, such activity may exhibit dynamic ‘remapping’ during gaze or head shifts (Colby et al., 1995; Duhamel et al., 1997; Graziano, Yap, & Gross, 1994; Umeno & Goldberg, 2001). Thus, when a shift of gaze will relocate a previously stimulated position into the receptive field of a particular neuron for the first time, this neuron may respond to the ‘remembered trace’ of the stimulus, despite never being directly stimulated (Duhamel, Colby, & Goldberg, 1992; Umeno & Goldberg, 2001). Recently, human fMRI studies have also reported activations in parietal and prefrontal cortex that might correspond to similar remapping when gaze-shifts change a remembered target location (Medendorp et al., 2003; Merriam et al., 2003; Tobler et al., 2001). These findings indicate that dynamic representations of space exist within parietal and frontal areas, that can remap remembered locations gaze-centrically during delays with changes in eye-posture. In other words, the maintenance of a location in visual space across changes in gaze might not involve activity in a fixed neuronal sub-population, but rather more dynamic and changeable patterns of activity across neurons that integrate retinal and extraretinal signals. The goal of our new paradigm was to test for any functional consequences of such spatial remapping in neglect patients (and whether these apply to perceptual memory, rather than just oculomotor behavior) by comparing their memory for visual locations as a function of different types of gaze shifts during the delay.

Consider how a to-be-remembered location should normally be maintained by neurons with gaze-centric representations, when gaze is shifted (Colby et al., 1995; Pouget & Driver, 2000; Umeno & Goldberg, 2001). For a location initially encoded at fixation, a rightward gaze-shift should remap it leftwards in gaze-centric terms, and so presumably into neuronal subpopulations of the contralateral right hemisphere (Colby et al., 1995; Medendorp et al., 2003; Merriam et al., 2003; Umeno & Goldberg, 2001). However, such remapping might be severely disturbed in a neglect patient, for whom right-hemisphere damage could destroy some of the neuronal populations representing leftward locations within gaze-centric maps (Behrmann et al., 2002; Driver & Vuilleumier, 2001; Pouget & Driver, 2000). As a result, the trace of the remembered location should become degraded or lost in neglect patients; and hence not be re-mappable out of the ‘black-hole’ of the gaze-centric map, even if gaze subsequently returned to the initial fixation. By contrast, a leftward gaze-shift should remap the initially fixated location rightward in gaze-centric maps, hence presumably into the intact left hemisphere (Medendorp et al., 2003; Merriam et al., 2003), so that spatial memory should now be preserved. Our experiments directly tested this otherwise paradoxical prediction: neglect patients should fail to retain location information following gaze-shifts towards the ipsilesional (“good”) right-side of space; but not (or less so) after gaze-shifts to the contralesional (“bad”) left-side. Moreover, if deficits in such dynamic gaze-centric representations of space play a functional role in neglect, we might expect that impairments during gaze-shifts on our new test may be related to the severity of left neglect on standard clinical measures.

Note that our predictions contrast with other theoretical proposals about the neural mechanisms of neglect. In particular, Pisella and Mattingley (2004) highlighted the possible role of spatial-remapping deficits in these patients, as we have also emphasized in other work (Driver & Vuilleumier, 2001; Husain et al., 2001) (see also Sapir, Hayes, Henik, Danziger, & Rafal, 2004). However, Pisella and Mattingley (2004) made a different prediction, namely that in right-hemisphere patients with left neglect, internal representations of spatial locations across the whole visual field would be lost or ‘overwritten’ by new information during any leftward gaze-shifts (while rightward shifts would only affect locations in the previous left visual field). This prediction is in fact opposite to our own prediction (namely that rightward gaze-shifts should be more detrimental, as these remap locations into leftward gaze-centric representations that may be damaged). Pisella and Mattingley (2004) prediction was based on theoretical considerations, but to our knowledge never directly tested. Here; we provide such an empirical test, which could also assess our own remapping hypothesis.

Our prediction (i.e. deficits after rightward gaze-shifts due to leftward remapping) is also distinct from the impairments found in some patients during ‘double-step’ saccadic tasks (Duhamel, Goldberg, Fitzgibbon, Sirigu, & Grafman, 1992; Heide & Kompf, 1998), where a very rapid sequence of eye-movements is required, with the target ‘stepping’ from the first to the second location before the first saccade is initiated. Deficits for the second saccade may arise in this task if the first saccade is towards the contralesional side in some parietal patients with (Heide & Kompf, 1998) or without (Duhamel, Goldberg et al., 1992) neglect. This has been attributed to the loss of some anticipatory efference-copy of oculomotor commands (Duhamel, Goldberg et al., 1992). This should not be a limiting factor in our paradigm, where explicit perceptual memory for location was tested over much longer delays (several seconds rather than ∼100ms). Moreover, any loss of efference-copy for contralesional gaze-shifts that might arise should lead to results opposite to our remapping hypothesis (i.e. to deficits after leftward, rather than rightward gaze-shifts).

Our new paradigm provides for the first time a direct empirical test for disentangling these different theoretical hypotheses about “spatial remapping” deficits in neglect (cf. Pisella & Mattingley, 2004), by probing explicit memory for a single spatial location after delays with gaze-shifts to either side, relative to delays with no gaze-shifts. Our results clearly accorded with our hypothesis. Patients with left neglect were impaired at maintaining a spatial location in memory across delays of several seconds only when they made a transient gaze-shift to the far-right, shifting the to-be-remembered location leftward gaze-centrically. In contrast, patients showed no loss in spatial memory when gaze shifted to the far-left during the delay. Furthermore, the deficit after rightward gaze-shifts was reliably correlated with neglect severity in individual patients, consistent with a contributory role to the clinical syndrome.

Methods

Participants

We recruited 7 consecutive patients who had a single focal right-hemispheric stroke, intact visual fields, and evidence of left spatial neglect in one or more clinical tests (Karnath et al., 2004; Mort et al., 2003). Five patients (2 females, 3 males; mean age = 69.5) were tested on one variant of our paradigm (Experiment 1); and two others (2 males, mean age = 66.0) were tested on a second control version (Experiment 2). Patients were not selected on the basis of a specific lesion site or a specific neglect test, in order to assess possible remapping deficits in our paradigm independently of any particular clinical profile that could be related to distinct neglect components, and to avoid any selection bias when testing for such deficits for the first time. Table 1 gives basic information for each patient. The initial clinical assessment of neglect was based on a battery of standardized paper-and-pencil tests (Rousseaux et al., 2001; Wilson, Cockburn, & Halligan, 1987). A shorter series of clinical tests was also given on the same day as our experimental paradigm, including Mesulam shape cancellation (Mesulam, 1985); Albert line cancellation (Albert, 1973); star cancellation and letter-string reading from the Behavioral Inattention Test (Wilson et al., 1987); plus a bisection task on lines with various lengths (Schenkenberg, Bradford, & Ajax, 1980), and spontaneous drawing of a clock from memory (Rousseaux et al., 2001). All patients had normal or corrected acuity, with intact visual fields on both sides, but stable left visual extinction on double simultaneous stimulation. All were right-handed and had no other neurological or psychiatric diseases.

Table 1.

Clinical Characterstics of the Patients

Patient	Sex, Age	Lesion	Time Post Stroke	Motor Deficit	Visual Field	Albert Cancellation Test (Total Misses)	Mesulam Cancellation Test (Total Misses)	Star Cancellation Test (Total Misses)	Letter-string Reading Test (Total Misses)	Line Bisection Test (% Left Deviation)
Experiment 1
VK	m, 56	FP infarct	20 mo	L hand weakness	intact	0	17	8	7	8.6
JH	m, 63	MCA infarct	4 mo	L hemiparesis	intact	2	8	1	6	6.2
JJ	f, 69	F infarct	1 mo	L arm weakness	intact	0	9	12	13	3
FD	f, 80	F infarct	2 mo	L arm weakness	intact	0	29	5	10	2.4
ED	m, 69	FTP infarct	8 mo	L hemiparesis	intact	0	5	2	0	0
Experiment 2
GA	m, 74	MCA infarct	2 mo	L hemiparesis	intact	2	15	9	25	6.6
MM	m, 58	MCA infarct	18 mo	L hemiparesis	intact	0	18	8	2	6.1

F = frontal; L = left; P = parietal; R = right; T = temporal; W = white matter; f = female; m = male; mo = months.

Seven healthy elderly subjects matched for age, gender and education also participated as controls for completeness (Expt 1: 2 females, 3 males, mean age = 67.1; Expt 2: 2 males, mean age= 64.5; all right-handed). All participants were tested during a single session, with brief resting breaks between different tests or different blocks. All participants gave informed consent in accord with local ethics.

For each patient, brain lesions were confirmed by clinical MRI scans and subsequently reconstructed on axial MRI slices by two neurologists (MH and PV) using MRIcro (Rorden & Brett, 2000), according to previously described methods (Mannan et al., 2005). Lesioned areas were transformed to a 3D region-of-interest (ROI) corresponding to the lesion volume, and then normalized to a standard brain template using MRIcro and SPM (Ashburner & Friston, 1997). Finally, normalized lesion ROIs were superimposed on a T1 MRI template and submitted to exploratory mapping analyses using MRIcro (Rorden & Brett, 2000), to compare lesion site and extent as a function of the severity of the experimental deficits observed (see below).

Experimental procedure

All stimuli were presented on a large laptop screen (1280 × 854 pixels, ∼35 × 47° visual angle) with a white background, using a Matlab toolbox (MathWorks Inc., CA) running Cogent 2000 software (www.fil.ion.ucl.ac.uk/cogent2000.html). Patients sat at ∼50 cm from the screen. Each trial began with a fixation cross at the screen-center for 1 sec (Fig. 1). An initial ‘sample’ target-dot (size ∼1°) was then presented at an unpredictable location on the right or left side relative to the fixation cross. The color of this target-dot could be either green or red (50% each), and had to be reported verbally as soon as it appeared. Correct color judgments required participants to foveate the target due its small size and eccentric location relative to initial fixation. The colored dot could be presented either alone or together with black background-dots (set size = 7 or 15 in Expt 1; always fixed at 15 in Expt 2). The latter factor was included because background landmarks can sometimes influence spatial memory in normals (Humphreys, 1998); but in fact the background factor had no influence on the critical results here, and so background effects will be reported only briefly for completeness.

Fig. 1.

Illustrative sequences of events within trials of different types in Experiment 1. (A) In the ‘no-remapping’ condition, the initial sample display with its target-dot, and the final probe display with another target-dot for spatial same/different judgment, were separated by an empty delay of 2 sec. (B, C) In the ‘remapping’ conditions, the interval between the sample target-dot (in initial display) and the probe target-dot (in final display) included a small peripheral letter presented at the far-left of the screen (B, gaze-left/remap right condition); or at the far-right (C, gaze-right/remap-left condition). Subjects had to shift gaze to the peripheral letter in order to identify it. On each trial, position of the probe-target in the final display could either match the sample-target (in a random half of trials), or be slightly displaced (in the other half). The sample and probe targets could appear alone (A-C), or together with 7 or 15 background black dots, randomly distributed on the screen (D-F). This background factor was orthogonal to the no-remapping, gaze-left, or gaze-right manipulation. When present, background-dots had identical positions in the sample and probe display for a given trial (but different jittered positions in different trials).

The position of both the target and any background-dots was randomized over successive trials, appearing at any location within a virtual 4×4 grid centered on the screen (∼35 × 40°), with an additional random jitter of up to ±4° in any direction within each cell of the grid). The initial sample-target appeared on the left or right side of this grid with equal probability (we refer to this factor as grid-side to avoid any confusion with changes in hemifield due to gaze-shifts). All background-dots (7 or 15 items, if present) appeared at jittered locations in the same 4 × 4 virtual grid (Fig. 1). The target-dot plus any black background-dots remained visible until the participant named the target color (red or green).

Immediately after the color response, the experimenter pressed a key to initiate the next phase of the trial, namely the delay period, during which the initial sample-target (along with any background-dots) disappeared for a few seconds (Fig. 1). The sequence of subsequent events during the delay differed slightly between the two variants of our paradigm (Expt 1 and 2), but both variants included two critical conditions (given in separate blocks) that either did or did not involve remapping of the target location during the delay.

For Experiment 1, in the “no-remapping” condition, the display remained entirely blank for the whole delay period of 2 sec (Fig. 1AD). In the “remapping” condition, the delay period contained a single small letter (any from the alphabet, Geneva font, upper case, size ∼1.5°) that was presented on either the far-left or far-right of the screen (Fig. 1BE and CF, respectively), in unpredictable and randomized order. We ensured during previous pilot tests that correct identification of this letter could not be made in peripheral vision due to its small size and eccentric position on the screen, but always required subjects to saccade in order to foveate the letter, thus imposing a gaze-shift in the corresponding direction. However, the letter had sufficient duration (2 sec) to be detected by the patients on most trials despite their neglect for left space. Moreover, observation by the experimenter confirmed that patients saccaded in the direction of the peripheral letter initially (rather than, say, always exploring the right side even for a unilateral peripheral letter presented on the left). Note that because the no-remapping and remapping conditions were given in separate blocks, in the remapping condition the patients knew in advance that a letter would appear during the delay on every trial, either on the right or left side (half of the trials each, randomly intermingled to allow a close comparison of gaze-shift directions).

At the end of the 2-sec delay, the critical ‘probe’ display was presented, in which the colored target-dot reappeared, together with black background-dots if these had been present in the sample display. If present, black dots reappeared in the same number and locations in the probe display as in the preceding sample display; whereas the colored target-dot could either reappear at its previous location (50% of trials) or be slightly shifted (2° to right or left, 25% each). Participants had now to make an explicit spatial perceptual-memory judgment, verbally reporting whether this target-dot had changed its location or not (same-different response). The probe display remained visible until this response was made. The three different set-sizes of dot displays (i.e., 0, 7 or 15 background dots) were equiprobable and randomly intermixed. Each of 6 possible sub-conditions (same or different target position; 3 background set-sizes) was repeated 12 times to produce blocks of 72 trials. Target color randomly varied from trial to trial.

For Experiment 2, the ‘remapping’ conditions were similar to Experiment 1 except for one single modification during the delay, which now included an additional step between sample and probe targets: a small digit (∼1°, identity selected randomly from digits 1-9) was now also presented at the screen-center (for 1 sec), just after the offset of the peripheral letter (again shown for 2 sec), but prior to the probe display onset (see Fig. 4BC). Participants had now also to name aloud this central digit. This additional manipulation was intended to force them to fully return gaze to screen-center after the transient gaze-shifts to a peripheral letter on either side (see below). The total interval between sample and probe targets was thus extended to 3 sec (vs 2 sec in Expt 1). The ‘no-remapping’ condition was exactly as in Experiment 1 (Fig. 4A), with a blank screen during the delay (now extended to 3 sec to match the new remapping conditions).

Fig. 4.

Illustrative event sequences and results for Experiment 2. (A) In the ‘no-remapping’ condition, the target-dot in the initial sample display and the final probe display were separated by an empty delay interval, now extended to 3 sec to match the new ‘remapping’ conditions. (B,C) In the new ‘remapping’ conditions, the interval between the sample-target in the initial display and the probe-target in the final display again included a peripheral letter presented (for 2 sec) at either (B) the far-left of the screen (gaze-left/remap-right), or (C) the far-right of the screen (gaze-right/remap-left). To be identified, this small peripheral letter required a gaze shift towards it. But now this peripheral letter was always followed by a small central digit (for 1 sec) that forced gaze to return to screen-center prior to the probe display. In all three remapping conditions, the color target-dot could appear alone (as shown in A-C) or with 15 black background-dots. The probe target had either the same or a slightly different position than the sample target (half of the trials each), and patients made a same/different verbal response. (D) Mean accuracy of target location memory for patients in this experiment, showing a similar pattern to those from Experiment 1 (cf. Fig. 2b). There was again a disproportionate impairment in the gaze-right/remap-left condition (Gaze-R, rightmost two bars) relative to the two other conditions, irrespective of whether targets were initially presented on the left or right of the display grid (white versus black bars). The gaze-left/remap-right (Gaze-L) condition and the no-remapping condition again did not differ.

In both variants of the task (Expt 1 and 2), the no-remapping condition was always run first, to familiarize participants with the task sequence and same/different location judgments (in one single block of 72 trials). This was followed by the remapping condition (two successive blocks of 72 trials each), in which gaze-left and gaze-right trials were given in intermingled randomized order. Hence the side of the peripheral letter during the delay was unpredictable, but participants could expect that such a letter appeared on one side or the other during the delay on these trials. In each condition and experiment, the task began with 16 practice trials, until subjects felt comfortable with the task sequence. All participants were able to learn the task quickly and easily reported the sequence of responses (i.e., color, letter, and location judgment in Expt 1; color, letter, digit, and location judgment in Expt 2) after this short practice. To simplify the patients’ task, all responses were made verbally but typed into the laptop by the experimenter at the trial end, allowing a later analysis of accuracy for different steps in the task (as well as ‘stopwatch’ estimate of response latencies, see below).

Our critical measure in both experiments concerned how location memory for the target was affected by the different events during the fixed delay between sample and probe displays. Specifically, our hypothesis predicted (see Introduction) that patients with left neglect would show a disproportionate impairment in maintaining an accurate representation of target location across the 2-second delay when they must shift their gaze towards the small letter at the far-right during that delay (e.g. Fig. 1C), and hence must remap the target location contralesionally towards the left-side in gaze-centric space. By contrast, we predicted that they should show no such impairment when gaze must shift to letters at the far-left, since the target location would then remap to the intact/ipsilesional right-side in gaze-centric maps (e.g. Fig. 1B). This contrasts with the prediction of Pisella and Mattingley (2004), according to which location information in memory should be particularly degraded or overwritten whenever gaze shifts leftward. On the latter account, performance should be impaired in the remapping condition that required gaze to be shifted out to the far-left letter, rather than when gaze was shifted to the far-right during the delay, as we predicted instead.

Results

Experiment 1

Target-color judgments (green/red response for the initial sample-dot) were recorded for each side where targets appeared on the screen (i.e. left or right of the display grid), and for each of the different background set-sizes (0, 7, or 15 black dots). These responses were 100% correct across all conditions, in both patients and controls.

The more important data concerned the accuracy of target-location judgments (same/different response to the probe-dot), calculated separately as a function of the two same factors (grid-side, background set-size), plus for each of the critical remapping conditions (no-remapping; gaze-right/remap-left; or gaze-left/remap-right), using only those trials where the peripheral letter presented during the delay was correctly identified when present (i.e., in the gaze-shift conditions).

As expected, healthy controls correctly identified the peripheral letter on most trials (mean 98%). Importantly, their accuracy in judging the remembered target location was equivalent for gaze-right/remap-left and for gaze-left/remap-right conditions (see Fig. 2A), with only slightly worse performance after such gaze-shifts (91% correct) relative to the no-remapping condition (97%); but this difference was not significant (t(5)= 1.69, p= .16), and the slight bilateral trend might reflect nonspecific dual-task factors. Thus, controls showed no evidence for any asymmetric cost of remapping after right vs left gaze-shifts (Fig. 2A). Accuracy was also slightly better in the presence of background-dots (7 dots: 95.4%; 15 dots: 94.6%) than when the target appeared alone (92.1%), consistent with previous findings showing some facilitation of location memory when relative coding of positions is possible for normals (Humphreys, 1998). A repeated-measure ANOVA (background set-size x gaze-condition x grid-side of target) confirmed this small beneficial influence of background-dots in controls (F(2,8)= 4.64, p= .046), but the slight difference between gaze-shift conditions was again not significant (F(2,8) = 2.24, p = .168); no other main effects or interactions were significant.

Fig. 2.

Accuracy of target location memory for Experiment 1 in the same/different location task. (A) Results from five healthy age-matched controls. There is only a mild decrease in performance for both remapping conditions (Gaze-L= gaze-left/remap-right, Gaze-R= gaze-right/remap-left) relative to the no-remapping condition; and importantly no difference or asymmetry between the two remapping conditions. (B) Results for the five patients with right-hemisphere damage and left spatial neglect for Experiment 1. There is a dramatic impairment in performance specifically for the gaze-right/remap-left condition (Gaze-R, rightmost two bars) relative to all other conditions. There is also a small additive trend for worse performance when targets were initially presented on the left vs right side of the display grid (white vs black bars).

Like normal controls, the patients were able to correctly report the peripheral letters on most trials in the two gaze-shift conditions, although unsurprisingly they were slightly better for letters presented on the far-right of the screen (95.0%) than on the far-left (83.3%), consistent with their neglect. This asymmetrical trend did not reach conventional significance overall (F(1,4)= 3.22, p= .142), probably because left neglect was relatively mild in a few patients (a point we return to later). Note that this trend for a greater difficulty with far-left letters could not bias our measure of location memory in favor of our prediction, since any impact on the subsequent judgments of target location would actually work against our hypothesis that spatial memory will be worse following a gaze-shift to the far-right letter. Moreover, only trials with correct letter-reports were further analyzed for subsequent location judgments.

Our critical prediction of asymmetric remapping deficits was strongly supported (Fig. 2B). Patients were strikingly impaired at remembering target position in the gaze-right/remap-left condition (57.5% correct overall), compared with gaze-left/remap-right (81.3%) or no-remapping trials (88.5%). Performance was also worse for targets initially presented on the left side of the display grid (71.6%) than on the right (81.3%), but this was independent of (i.e., additive to) the severe impairment on gaze-right/remap-left trials (see Fig. 2B). Repeated-measure ANOVA on percent-correct location judgments confirmed a highly significant effect of remapping condition (gaze-right/remap-left; gaze-left/remap-right; or no remapping; F(2,4)= 15.2, p= .002); a marginal effect of grid-side (right or left, F(1,4)= 7.62, p= .051); but no interaction (F(2,8)= .14). No other terms were significant, including background set-size and background x remapping interaction (all F < 1).

Direct pairwise comparisons further confirmed that, for targets initially presented on the left grid-side, location memory was selectively impaired in the gaze-right/remap-left condition, both relative to no-remapping (t(4)= 4.39, p= .012) and, more critically, relative to gaze-left/remap-right (t(4)= 3.38, p= .027). The same pattern was observed for targets on the right grid-side also (t(4)= 3.47, p= .025; and t(4)= 3.99, p= .016, respectively). This impairment induced by rightward gaze-shifts arose for targets on either side of the display grid in all five patients (see Fig. 3, Experiment 1 data). By contrast, leftward gaze-shifts did not differ from no-remapping for targets on either grid-side (all t(4)< 1).

Fig. 3.

Accuracy of target location memory in individual patients. Location memory judgments (percent correct) are shown for each neglect patient from Experiment 1 (n= 5) and from Experiment 2 (n= 2), for the three critical conditions with respect to spatial remapping (Gaze-L= gaze-left/remap-right; Gaze-R= gaze-right/remap-left) and separately for each side of the screen (left vs right grid positions). Performance is lower in the gaze-right condition than the other two conditions for every patient. Note that this effect is larger for some patients than others, but still present across them all. By contrast, performance is similar in the no-remapping and gaze-left conditions overall.

These findings reveal that shifting gaze to the far-right (which would induce remapping of the remembered target location leftwards in gaze-centric representations) induced a dramatic loss in the patients’ perceptual memory for target position across a 2-second delay, even for targets initially shown on the right-side of the screen. By contrast, shifting gaze to the far-left did not disrupt spatial memory in our patients, relative to the no remapping condition. These results therefore accord with our hypothesis about dynamic gaze-centric remapping of locations across delays with intervening changes in gaze direction (with a deficit arising when the target should be remapped in the contralesional/lefttward direction in gaze-centric terms). Since the delay interval between sample-target offset and probe-target onset was fixed (at 2 sec), no difference in delay duration can account for this striking difference in location memory between gaze-right/remap-left versus gaze-left/remap conditions. (See also Appendix, for details on stopwatch estimates of the time elapsed between sample-target offset and location judgments, which indicate no systematic time difference between the critical remapping conditions, thus arguing against any speed/accuracy tradeoff in the patients).

Finally, we also note for completeness that location judgments in the patients, unlike in the controls, did not improve with an increasing number of background dots (mean correct judgments on probe display: 78.9%, 80.0% and 81.7% for 0, 7, and 15 black dots, respectively; F(2,8) = .12), irrespective of remapping conditions or grid-side. This suggests that the patients may have some additional deficit in coding relative positions between neighbouring items in the display (see Humphreys, 1998), perhaps due to the local bias that is common in neglect patients (Halligan et al., 2003; Lamb et al., 1990) and/or impairments in between-objects spatial coding (Humphreys, 1998). However, any such problem was clearly unrelated to the striking deficit in memory for target location in the gaze-right/remap-left condition, as the latter deficit arose regardless of the number of background items (see above).

Experiment 2

One potential limitation of our remapping task could be that the asymmetric effect of gaze-shifts on location memory might, in principle, depend on the retinal position of the probe-target when it reappears, rather than the updated location of the target during the delay. After gaze-shifts to the far-right, neglect patients might perhaps have some difficulty in returning to judge the probe-target, which might initially appear in their left visual hemifield. This possibility seems unlikely because all patients rapidly became familiar with the need to return centrally after reporting the peripheral letter, and in fact never failed to detect the probe for the required location judgment at the end of delay. Moreover, our patients actually showed a slightly greater difficulty in reporting far-left than far-right letters (see above), such that any tendency to linger at the periphery might arguably apply more for gaze-left than gaze-right conditions. Finally, our latency estimates did not show slower responses to the probe after gaze-right than gaze-left shifts (see Appendix and Annex Table 2). Nevertheless, we ran a variant of our paradigm in two other patients (Expt 2), which was specifically designed to rule out the possibility that an asymmetric effect of gaze-shifts on location memory might result from the position of the probe when it reappears. This variant was similar to the first experiment in all respects except that it always enforced a return of fixation back to the screen-center (to report a central digit briefly presented there), prior to onset of the probe (see Methods and Figure 4a-c).

Table 2.

Estimates of Latency for Memory Judgments in Each Patient and Each Experiment

	Correct Location Judgments		Incorrect Location Judgments
	L	R	L	R
Patients	Gaze-shift	Gaze-shift	Gaze-shift	Gaze-shift
Experiment 1
ED	7.000	4.365	7.135	5.110
FD	3.220	2.84	3.500	2.980
JH	6.143	4.937	6.359	7.624
JJ	2.943	3.931	4.229	3.198
VK	2.106	2.177	2.218	2.695
Experiment 2
GA	6.600	8.590	8.709	6.210
MA	5.413	5.842	6.594	6.774

In this experiment, controls were flawless in reporting successively the sample-target color, the peripheral letter on either side, and the subsequent central digit. On the critical location judgments for the final probe-target, their performance did not differ between gaze-right/remap-left/return-to-center (87% correct) and gaze-left/remap-right/return-to-center (86% correct); while it was, as might be expected, slightly better in the no-remapping condition (97% correct), albeit not significantly so (p> .25, chi-square tests in each case). Again, location judgments in control subjects were marginally better with than without background black-dots (93% vs 88%, respectively).

Both neglect patients also correctly reported the sample-target color on all trials, and across both remapping conditions. In the ‘remapping’ blocks, the patients were reasonably accurate for the peripheral letters (GA: 83% and 92% for far-left and far-right, respectively; MM: 94% and 100%), and for the subsequent central digit (GA: 75%, MM: 100%). Only trials with correct performance for both the peripheral letter and central digit were considered further (but the pattern of location-memory performance is similar if all trials are included).

Critically, location judgments for probe-targets were significantly worse in the gaze-right/remap-left/return-to-center condition, as compared with the gaze-left/remap-right/return-to-center condition, or the no remapping condition (see Fig. 4D for means, and Experiment 2 data in Fig. 3 for individual results). This pattern arose regardless of whether the initial sample-target was shown on the left_grid-side (MM: 1/18, 12/17, and 32/36 correct, for gaze-right, gaze-left, and no-remapping trials, respectively; GA: 6/14, 10/13, and 24/36); or shown on the right grid-side (GA: 2/10, 12/14, and 32/36 correct; MM: 8/18, 13/17, and 34/36). This pattern was found reliably in each patient, as confirmed by chi-square tests on the number of correct and incorrect trials. The two critical remapping conditions differed significantly within each patient (GA: χ²(1)= 10.3, p< .001; MM: χ₂(1)= 14.6, p< .001). Moreover, the gaze-right condition was also worse than no-remapping (GA: χ₂(1)= 14.1, p< .001; MM: χ²(1)= 47.1, p< .001); whereas the gaze-left condition did not differ from no-remapping (GA: χ²(1)= .02, MM: χ²(1)= .028, both ns).

Again, as in Experiment 1, the presence of background dots did not influence position memory in the neglect patients, irrespective of grid side as well as remapping conditions (GA: 64 vs 59%, MM: 71 vs 69%, for mean percent-correct with or without distractors, respectively; chi-square < 1).

The results of Experiment 2 therefore replicate and extend Experiment 1, now demonstrating a selective deficit for the gaze-right/remap left condition even when gaze was forced to return to screen-center prior to onset of the probe that tested location memory. These data thus confirm that the asymmetric performance resulted from the transient gaze-centric location of the remembered target during the delay, not from the probe location at onset. Again, the delay between sample-target offset and probe-target onset was constant for all conditions here (now 3 sec), ruling out any difference due to stimulus timing. Moreover, there was no difference in the estimate of total time elapsed from target-sample offset to final location judgments in the two remapping conditions (see Appendix and Annex Table 2).

Selective deficit for gaze-right/remap-left regardless of grid-side

In addition to our major finding of a deficit in the gaze-right/remap-left conditions, which applied for targets that initially appeared on either the left or right side of the grid, both Experiments 1 and 2 also showed a slight tendency for worse location memory when targets were presented on the left of the grid (and thus of the screen), relative to the right side, even in the no-remapping condition. This deficit was relatively small and strictly additive to the cost of right-gaze/remap-left conditions, without any significant interaction between these two effects. It is possible that the grid-side trend may result from some other general factor contributing to degraded location information for left space in neglect patients. But (as suggested by a reviewer) it might also be questioned whether individual variability in the effect of grid-side might somehow relate to the magnitude of memory cost for the gaze-right/remap-left conditions. For example, it could perhaps be contended that the grid-side effect might reflect an inability to encode any location on the left of the screen, such that the far-left location of peripheral letters during gaze-shifts would produce no competition or “load” for target memory; whereas locations of any stimulus on the right side (including far-right letters) would be unintentionally encoded into memory and somehow compete with the target representation.

However, although location memory was better overall on the right than left grid-side across both experiments (t(6)= 3.53, p= .012), our patients were still able to encode and maintain left-grid target locations (e.g. mean 82% correct in no-remapping trials). Moreover, we found no significant correlation between the size of the grid-side effect in individual patients (accuracy for right-grid minus left-grid targets) and their ‘remapping cost’ (accuracy for gaze-left minus gaze-right conditions); r= .22, p= .84. We therefore conclude (in accord with the additive pattern of results found above) that the slight impairment for left versus right grid-side targets is a separate phenomenon to the more dramatic effect of gaze-right/remap-left (leading to severe impairment) versus gaze-left/remap-right (revealing no impairment). This of course is consistent with the fact that gazing to the far-right would relocate both grid-sides into contralesional space in gaze-centric terms.

Relation of remapping deficit to clinical and anatomical characteristics

Although we would not suggest that the remapping deficit identified here is the sole (or even the major) determinant of neglect, we hypothesized that it may contribute to several important aspects of the disorder. Neglect patients will frequently make rightward saccades in daily life, and also during many clinical tests (e.g. search, cancellation, bisection, drawing). As found in our new paradigm here, this might degrade or erase previously encoded location information, if the latter needed to be remapped leftward in internal gaze-centric representations. Thus, a remapping deficit induced by transient gaze-shifts could potentially exacerbate pathological losses for spatial information on the contralesional side to current gaze direction, even when these locations have recently been inspected (as in our paradigm).

Accordingly, we assessed whether the severity of the experimental remapping deficits found in each individual patient (Fig. 3) might relate to the severity of neglect on standard clinical measures (Table 1), especially for tests that are likely to induce spontaneous gaze-shifts, such as cancellation tasks, line bisection, reading, or drawing. Since our patients were consecutive cases selected for showing signs of neglect in at least one test among a standard battery (but with intact visual fields, after a single stroke), they showed somewhat different degrees of impairments across different clinical tasks (see Table 1). Moreover, while all of them showed some remapping deficit specific to the gaze-right/remap-left condition, they did so to different degrees (see Fig. 3). For each patient from Exp. 1 and 2, we could therefore calculate the ‘remapping cost’ in target-location memory (gaze-left minus gaze-right accuracy), and then test for any correlation of this remapping cost with individual patient scores for standard neglect tests given on the same day (Table 1).

Remarkably, positive correlations were found with several of these clinical measures, particularly for omission rate on Mesulam cancellation (Fig. 5a; r(6)= .66, p< .05) and total error rate (omissions + misplacements) on clock drawing (r(6)= .63, p= .05). The correlation was marginally significant for deviation magnitude on line bisection (Fig. 5b; r(6)= .61, p= .07), while it was also positive but non-significant for other neglect tests, such as the star cancellation (r(6)= .51) and letter-string reading tasks (r(6)= .41) from BIT (Wilson et al., 1987). Overall, the strongest correlation (Fig. 5c, r(6)= .75, p< .05) was observed with a global ‘neglect severity index’ that averaged the percentage of left-sided omissions across all tests, including the three cancellation tasks, letter-string reading, clock drawing, plus percentage deviation on line bisection.

Fig. 5.

Correlation of the experimental remapping deficit with severity of clinical neglect. Remapping cost was computed as the difference in accuracy for gaze-left minus gaze-right conditions (x axis) and then related to several clinical measures (y axis). There were positive correlations with: (A) the total number of items missed in Mesulam cancellation; (B) the magnitude of ipsilesional deviation on line bisection; and (C) a composite score of neglect severity, summing deficits across several different clinical tests (see main text). (D) There was no reliable correlation with other clinical variables, such as the time since stroke onset or lesion volume.

By contrast, we found no reliable correlation between the different neglect tests (e.g., bisection vs Mesulam cancellation or drawing, all r > .46, p > .31), consistent with other studies including larger patient groups (e.g. Agrell, Dehlin, & Dahlgren, 1997; Hier, Mondlock, & Caplan, 1983); (although see Buxbaum et al., 2004; Halligan, Marshall, & Wade, 1989). Furthermore, the experimental remapping costs did not correlate with other, less specific clinical factors (Fig. 5d), such as the patients’ age; time since lesion; nor lesion volume (r(6)= .35, p= .46).

The same correlation pattern was still found (see Fig. 5) when we excluded the two patients tested on the modified version of the remapping task (MA and GA, Expt 2).

Finally, we conducted an exploratory anatomical analysis to examine the possible relation of this new remapping impairment with brain-lesion sites in individual patients. Since we did not select our patients by a single anatomical criterion (but for presence of a single right-hemisphere stroke with intact visual fields and left neglect signs), their lesions involved several different brain regions typically associated with spatial neglect (Husain & Kennard, 1996; Karnath et al., 2004; Mesulam, 1999; Mort et al., 2003) and might therefore provide some hints as to the major neural substrates of the newly identified impairment. Because the critical remapping manipulation was essentially similar in both variants of our paradigm (Expt 1 and 2), and the pattern of results very consistent across all cases (see Fig. 3), our exploratory anatomical analysis considered all 7 patients together. Figure 6a shows individual lesions for each of them, as reconstructed on axial brain slices and normalized to a standard T1 MRI template (see Methods). Normalized lesion ROIs obtained from reconstruction were then used to determine the overlap of lesions and to compare patients with the largest (ED, GA, and MM) versus the smallest (VK, FD, and JH) right-gaze costs, by performing a median-split subtraction of the ROIs from individual patients, with MRIcro software (Rorden & Brett, 2000). This analysis indicated that the more severe remapping impairments were associated with posterior brain damage involving the inferior parietal lobe and underlying white-matter, extending into the temporal lobe and basal ganglia (Fig. 6b). Note however that these lesion data are preliminary, since the main purpose of the present study was not anatomical, but rather to test for direction-specific remapping deficits, as clearly identified here for the first time.

Fig. 6.

Lesion reconstruction for all patients. (A) Individual lesions are shown for each patient (in rows), superimposed on a normalized MRI brain template. (B) Median-split subtraction analysis, comparing the lesions in the three patients with the most severe deficit in spatial remapping (difference in accuracy for gaze-left minus gaze-right conditions) versus the three patients who had the least severe deficit in spatial remapping (see main text). Each color in the scale bar shown at right codes for a 16.67% frequency of lesion in one or the other group, except for the central purple color that represents −16.67 to +16.67%. Brain areas implicated more in patients with the more severe behavioral remapping deficit in our experiments are shown in yellow (frequency of lesion 100%), involving the inferior parietal and superior temporal lobe, with extension into subcortical regions and paraventricular white-matter.

Discussion

Our two experiments reveal a striking failure of right-hemisphere patients with left spatial neglect to maintain an accurate perceptual memory for the position of a single visual target across delays of 2-3 seconds, but only under certain conditions. This deficit exclusively arose when their gaze had to shift towards the ‘good’ (far-right) side during the delay. It still arose after rightward gaze-shifts when gaze was then returned to screen-center, prior to testing location memory. The latter finding indicates that the internal representation of the remembered location was permanently lost or degraded when it had to be briefly maintained in the ‘bad’ side of space, during a transient gaze-shift towards a more ipsilesional location, and could not be recovered by returning gaze centrally. By contrast, shifting gaze to the contralesional side (far-left) did not disrupt their location memory whatsoever, as compared with a control condition where no remapping was required during the delay.

These results accord with our paradoxical remapping prediction, derived from single-neuron recordings (and more recent fMRI studies) showing dynamic gaze-centric representations of visual locations. Using a new paradigm to test this remapping hypothesis directly, we demonstrated that the patients’ deficit was specific to ipsilesional gaze-shifts, consistent with the idea that these require leftward updating of the to-be-remembered location in gaze-centric coordinates, unlike gaze-shifts towards the contralesional side that require rightward updating instead. These findings agree with neurophysiological data on gaze-centric updating in both monkey single-cell studies (Colby et al., 1995; Duhamel et al., 1997; Umeno & Goldberg, 2001) and recent fMRI studies in healthy humans (Medendorp et al., 2003; Merriam et al., 2001), which revealed neural activity in frontal and parietal cortex associated with remapping of remembered locations during unstimulated delays. But our results go beyond these studies, in showing that a disruption of gaze-centric remapping mechanisms can have a direct impact on an explicit measure of perceptual spatial memory, and not just on oculomotor or visuomotor behavior. Our results also imply a causal role for the damaged brain regions in such perceptual memory for location across gaze-shifts (whereas causality cannot be inferred from single-cell recordings or fMRI alone).

Pisella and Mattingley (2004) recently drew attention to the possible importance of gaze-centric remapping for neglect patients, but their theoretical speculations differed from our own hypothesis and findings. These authors suggested that contralesional gaze-shifts (i.e. towards the left side of space) should ‘overwrite’ all remembered locations within damaged spatial maps in neglect patients, while ipsilesional/right shifts should affect only a portion of left visual space. Although we agree with the importance of remapping effects in neglect, our results revealed a pattern very different from their proposal. We found that gaze-shifts to the far-right (not far-left) disrupted performance in neglect patients, for both grid-sides, just as we predicted on the basis that rightward gaze-shifts require leftward mapping in gaze-centric coordinates.

Our patients performed well in the no-remapping condition (Fig. 1AD and 4A), showing only slightly more errors for targets on the left vs right grid-side (Fig. 2B and 4D). Furthermore, they were not impaired at all after gaze-shifts to the far-left (Fig. 1BE and 4B). This outcome is noteworthy since the latter condition required patients to gaze in the contralesional/neglected direction, and since it runs counter to the prediction of Pisella & Mattingley (2004). But this also shows that saccades per se during the delay did not disrupt location-memory in the patients. Instead, location memory was disproportionately impaired only after rightward gaze-shifts (Fig. 1CF and 4C), when the remembered target location should require updating into a leftward position within internal gaze-centric map(s)(Duhamel et al., 1997; Medendorp et al., 2003; Merriam et al., 2001; Pouget & Driver, 2000). Healthy controls showed no asymmetry between the two remapping conditions, as expected given their intact spatial maps. Further, in patients, the deficit on gaze-right trials arose equally for targets initially presented on the right or left side of the display grid (Fig. 2B and 4D), consistent with gaze to the far-right shifting all these locations leftward within gaze-centric representations. Thus, even when targets were initially seen on the ipsilesional/right side of the screen, location memory was severely impaired after gaze was transiently shifted more towards the far-right side in these patients.

One referee made an ingenious suggestion that during the gaze-right condition, the location of the far-right letter might automatically enter into spatial working-memory (despite its irrelevance to the prescribed memory task), to disrupt memory for the target in a way that the location of the far-left letter might not, possibly due to a more general difficulty in encoding or storing left locations in spatial memory. However, we consider it somewhat unlikely that our findings of a major deficit specific to the gaze-right condition could be explained away by such a general problem. Although there was a slight disadvantage for locations on the left side of the screen, relative to the right side, left-screen locations were successfully encoded and maintained by neglect patients over delays without gaze-shifts. Moreover, the effect of grid-side neither interacted nor correlated with the remapping cost caused by rightward gaze, indicating distinct and additive sources for these two effects. But note that even if the referee’s suggestion was correct, our results would still highlight a key new finding in neglect; by revealing selective losses in location memory after gaze-shifts towards rightward but not leftward stimuli, and contradicting the previous remapping prediction of Pisella and Mattingley (2004).

In addition, our findings cannot simply be explained away by some form of “leftward inattention” or “extinction” for contralesional/left stimuli during rightward gaze-shifts (as also queried by a referee). Indeed, it is important to emphasize that no target was physically present on the screen during the delay period. Thus, the to-be-remembered location would only shift ‘leftward’ during the gaze-right conditions to the extent that a memory trace of it was subject to gaze-centric remapping. Whether one then attributes the memory deficit in this situation to gaze-centric leftward ‘inattention’ or “extinction” for a remapped memory trace, or rather to losses in gaze-centric representations of leftward locations, may be more a terminological issue than a difference in substance (as for other aspects of the longstanding attentional/representational debate concerning neglect, (e.g. see Pouget & Driver, 2000)). At the neural level, there may be considerable overlap between representations that allow spatial memory across delays while combining retinal and extra-retinal information, with those that direct attention or determine salience for selected locations (e.g. see Ashburner & Friston, 1997; Corbetta, Kincade, & Shulman, 2002). Nevertheless, it is an interesting issue for future research to consider whether similar deficits to those found here might arise if patients merely shifted covert attention to the peripheral letter and back, rather than executing overt gaze-shifts. On a strictly gaze-centric interpretation, the same pattern might not be expected; whereas on a more attention-centric account it might be. However, such experiment would have to overcome some practical obstacles, in requiring brain-damaged patients to shift covert attention very substantially without any saccades.

Another issue for future research is whether the deficit found in our paradigm might also apply in a task requiring short-term memory for other non-spatial properties (e.g. shape or color) across the same gaze-shift manipulations. Some previous research (Pisella et al., 2004) already showed that neglect patients may selectively fail to retain location, but not shape and color, albeit in the different context of a four-item visual working memory task.

Taken together, our new results converge with previous suggestions (Behrmann et al., 2002; Driver & Vuilleumier, 2001; Pouget & Driver, 2000; Vuilleumier & Schwartz, 2001) that brain lesions in neglect patients may damage neural populations that combine extra-retinal (e.g., oculomotor) postural signals with representations of visual space (Andersen et al., 1990; Colby et al., 1995); and that such losses may contribute to deficits in spatial memory and exploration in neglect patients (Husain et al., 2001; Mannan et al., 2005; Pisella et al., 2004); (see also Sapir et al., 2004). The proposal that impairments to some aspects of spatial working memory might underlie some components of neglect also appears consistent with the anatomical overlap of brain regions implicated in spatial working memory and spatial attention (Corbetta et al., 2002; Husain & Rorden, 2003). However, our study goes beyond previous hypotheses about spatial working memory deficits, by showing that memory for a single spatial location is critically dependent on the direction of an intervening gaze-shift, with dramatic losses in memory when gaze shifts towards one direction (rightward), but no such losses for the same information when gaze shifts to another (leftward) direction, or when no gaze shift is made, despite similar retention delays. These data suggest that spatial working memory deficits associated with neglect may result in part from damage to spatial representations that are dynamically maintained in gaze-centric coordinates, as found in parietal and frontal cortical areas (Colby et al., 1995; Medendorp et al., 2003; Merriam et al., 2003; Umeno & Goldberg, 2001)

Further studies should examine any impact of other brain lesions in our paradigm, including patients without any signs of spatial neglect or patients with left hemisphere strokes. Nevertheless, here we recruited a series of consecutive patients with a variety of right-hemisphere lesions, and with different degrees of neglect on different tests, allowing us to demonstrate a clear correlation with clinical neglect severity, without having to select a priori only a subset of patients with specific symptoms. Left-hemisphere lesions typically do not cause severe neglect (Beis et al., 2004), possibly because both sides of space can be represented in the right hemisphere, whereas only the contralateral/right side may be represented in the left hemisphere (Heilman et al., 2003; Mesulam, 1999). Thus, unilateral left hemisphere lesions might not be expected to produce severe remapping deficits, since the intact right hemisphere may still be able to represent bilateral regions of space, and hence to remap a remembered location rightward following a leftward gaze-shift.

We performed a preliminary anatomical analyses on our right-hemisphere patients, tentatively suggesting that the remapping deficits might be more pronounced when lesions extend into inferior parietal and superior temporal regions, plus adjacent subcortical structures, that are all typically associated with enduring clinical neglect (Doricchi & Tomaiuolo, 2003; Karnath et al., 2004; Mort et al., 2003). Importantly, however, our main conclusions stand regardless of residual issues about detailed anatomy or laterality. Our study clearly reveals for the first time a selective deficit in right-hemisphere patients with neglect, which was specifically induced by ipsilesional/rightward gaze-shifts during delays, indicating a failure when leftward gaze-centric remapping was required. Furthermore, we could demonstrate a reliable correlation between this remapping deficit and neglect severity on standard clinical tests, particularly for cancellation tasks known to provide sensitive clinical measures (Ferber & Karnath, 2001).

The remapping deficit identified here might contribute to several other manifestations of neglect, including an abnormal tendency of patients to revisit previously inspected locations during search tasks (Husain et al., 2001; Mannan et al., 2005; Wojciulik, Husain, Clarke, & Driver, 2001). We found that the severity of remapping deficit was more strongly correlated with some clinical tests than others, and most reliable for complex cancellation tasks and clock drawing (but less so for reading and line bisection). We surmise that many spontaneous gaze-shifts are likely to occur during the former tests, especially cancellation (Behrmann, Watt, Black, & Barton, 1997) and clock drawing (Di Pellegrino, 1995). Such gaze-shifts may then exacerbate the pathological losses in internal representations for contralesional space. Similarly, we suggest that the specific deficit in dynamic remapping across gaze-shifts might also contribute to some otherwise ‘paradoxical’ behaviors in neglect. For instance, during drawing tasks, neglect patients may initially orient to contralesional locations to outline a figure, but then seem to ‘forget’ these when drawn to details on the ipsilesional side, failing to return to the previously drawn elements to add details (Di Pellegrino, 1995). Likewise, neglect patients may be able to mark the leftmost corners of a cancellation sheet, yet then fail to cancel left items near those recently acknowledged and inspected locations (Halligan et al., 2003). The specific remapping deficit identified here could lead to the location of left page-extremities being lost from memory after their initial fixation, due to subsequent rightward gaze-shifts and the leftward remapping that is then required. In daily life, remapping deficits could analogously contribute to rapid forgetting of information about spatial scenes, even in familiar settings, whenever patients make rightward gaze-shifts, as they frequently do (Bartolomeo & Chokron, 2002; Halligan et al., 2003).

Future studies might extend our new paradigm to situations where eye-movements can be recorded online, and might compare performance during covert shifts of attention relative to overt saccadic shifts (Barton, Behrmann, Black, & Watt, 1997; Behrmann et al., 1997). Eye-movements could not be recorded in our study due to clinical constraints, but our procedure with small target-dots and small peripheral letters did force all subjects to make eye-movements in the remapping conditions (as witnessed by the experimenter during pilot tests and experimental sessions). As noted above, neural systems responsible for saccadic eye-movements closely overlap with those implicated in spatial attention and gaze-centric representations of space (Bremmer, Graf, Ben Hamed, & Duhamel, 1999; Colby & Goldberg, 1999; Corbetta et al., 1998), and follow-up studies would be needed to determine whether covert attentional shifts can produce similar deficits as remapping effects found here. Note that our results may accord with previous data from parietal patients, showing impaired remapping of “inhibition of return” with exogenous visual cues in a spatial orienting task (Sapir, 2004 #55). However, the latter observations differed from our findings, because they concerned putatively reflexive effects of spatial orienting, and the deficits were observed after both ipsilesional and contralesional gaze-shifts. .

More generally, our results provide to our knowledge the first direct evidence of gaze-centric remapping effects in a purely perceptual memory task. Other recent work (Khan, Pisella, Rossetti, Vighetto, & Crawford, 2005; Khan, Pisella, Vighetto et al., 2005) has examined gaze-centric remapping in a very different visuo-motor disorder (optic ataxia), which involves misreaching subsequent to superior-parietal damage, and usually dissociates from neglect caused by more inferior parietal lesions (Karnath et al., 2004; Mesulam, 1999; Mort et al., 2003). Changes in gaze-position can modulate reaching to remembered targets in optic ataxia (Khan, Pisella, Rossetti et al., 2005; Khan, Pisella, Vighetto et al., 2005), suggesting a role for dynamic gaze-centric remapping that remains intact in the latter patients (unlike here). The findings in optic ataxia thus differ from the present data in many respects, including the nature of the disorder and the brain regions implicated (Perenin, 1997); the task and type of response (motor reaching vs explicit spatial memory (Bridgeman, Lewis, Heit, & Nagle, 1979; Goodale & Humphrey, 1998)); and whether or not gaze returned to its initial position prior to the final response. This was an important aspect of the second variant of our paradigm (Expt 2 here); whereas gaze remained deviated during final reaching in optic-ataxia studies (Khan, Pisella, Rossetti et al., 2005; Khan, Pisella, Vighetto et al., 2005). Hence the latter effects might primarily reflect a deficit in visuo-motor transformation, depending on gaze-angle during reaching, rather than during the delay itself; whereas our perceptual spatial-memory task clearly revealed an effect due to gaze-posture during the delay (see especially our Experiment 2). Thus, results from optic-ataxia studies (Khan, Pisella, Rossetti et al., 2005; Khan, Pisella, Vighetto et al., 2005) agree with our own data in showing the importance of gaze-centric spatial representations, but the specific issues and conclusions are very different.

Although our results converge with single-cell (Colby et al., 1995; Duhamel et al., 1997; Umeno & Goldberg, 2001) and neuroimaging studies (Medendorp et al., 2003; Merriam et al., 2001) on the existence of gaze-centric remapping, they differ from previous findings on double-step saccadic deficits in brain-damaged patients (Duhamel, Goldberg et al., 1992; Heide & Kompf, 1998), as mentioned in the Introduction. In double-step saccade tasks, a visual target is flashed briefly at two different locations in very rapid succession (within ∼100-200ms), so that the second target disappears prior to the first saccade. Patients with right parietal damage, either with neglect (Heide & Kompf, 1998) or without neglect (Duhamel, Goldberg et al., 1992; Heide & Kompf, 1998), may fail to saccade correctly to a second rightward target after a first leftward saccade; but they can correctly follow the target with their eyes when it first steps right then left (Duhamel, Goldberg et al., 1992; Heide & Kompf, 1998), or when saccadic steps are slower (∼500ms) in either direction (Duhamel, Goldberg et al., 1992). The deficit for rapid double-saccades might result from loss of an anticipatory motor efference-copy (Duhamel, Goldberg et al., 1992; Heide & Kompf, 1998), required to predict the retinal displacement of the second target (now removed) after the first eye-movement. Such anticipatory activity was presumably not required in our task, which involved much longer delays and concerned explicit spatial memory, potentially recruiting different neural circuits than rapid and automatic oculomotor responses (Bridgeman et al., 1979; Goodale & Humphrey, 1998). Moreover, any double-step deficit affecting efference-copy for contralesional saccades in our task would actually have led to an opposite pattern of results (i.e., deficits for gaze-left conditions), indicating that our findings cannot be explained by prior saccadic observations. In future studies using our paradigm, the role of different delay durations between the first target onset, saccadic onset towards a peripheral location, and the final memory probe, could be further investigated. This might allow distinctions between defects in the memory trace arising due to remapping errors, from any loss in anticipatory efference signals, and/or spontaneous decay since the target offset or since the saccade onset.

To sum up, our findings extend recent data from single-cell studies (Colby et al., 1995; Duhamel et al., 1997; Umeno & Goldberg, 2001) and fMRI (Medendorp et al., 2003; Merriam et al., 2001) on gaze-centric remapping during delay periods in the normal brain, to reveal the drastic functional consequences of damage to neural systems underlying such representations of space in patients. Our study indicates not only that dynamic remapping arises during gaze-shifts, but also shows for the first time that this may contribute to perceptual location memory, not just to oculomotor behavior or reaching. Our study also implies that the brain regions damaged in our patients play a causal role in these processes (which cannot be concluded from fMRI activation nor single-cell data alone). Thus, in neglect patients, a target location previously encoded and fixated can be forgotten within seconds if a gaze-shift towards the ipsilesional side intervenes, being permanently lost from memory even if gaze is then returned to center prior to the memory test. We propose that while such remapping deficits are by no means the only components of neglect, they might contribute to several of its clinical manifestations (consistent with the correlations found here with clinical measures); might explain some otherwise paradoxical aspects of the syndrome (Halligan et al., 2003); and can be attributed to damage involving gaze-centric representations of contralesional visual space (Colby et al., 1995; Medendorp et al., 2003; Merriam et al., 2001; Umeno & Goldberg, 2001). These findings provide new insights into the functional implications of dynamic representations of space in the human brain, and into the dramatic but highly specific impact of brain damage on spatial awareness.

Appendix

Stopwatch estimates of latencies for location memory judgments

The critical findings in both Experiments come from the accuracy of explicit same/different judgments for the location of the probe-target, relative to the initial sample-target, after a fixed delay interval (always constant at 2 sec in Experiment 1, and at 3 sec in Experiment 2). The latencies for participants to make these same/different judgments were also estimated, by computing the time-elapsed between offset of the sample-target and response to the probe-target (as entered by the experimenter who hit a ‘stopwatch’ key on the computer when the subject responded). These stopwatch latency estimates are reported for completeness here (see Annex Table 2). These data indicate no major difference in the time taken by the patients to make location judgments in the different remapping conditions (t(4)= 1.03, p= .36); nor any correlation between the critical memory cost for gaze-right/remap-left versus gaze-left/remap-right conditions (difference in percent-correct), against the total time elapsed from sample-target offset to same/different judgment for the probe-target (Annex Table 2). No such correlation was found, regardless of whether we used the raw memory cost, or ‘normalized’ this cost to individual scores in the no-remapping condition (all p> .22). Our estimate of response latencies did however reveal some other effects that would be expected in our paradigm, indicating that it has some sensitivity. In Experiment 1, background set-size (i.e. number of black dots) influenced the speed of color judgments for the initial sample target-dot, both in the controls (F(1,4)= 7.97, p= .047; mean 1480, 1510, and 1590 ms for 0, 7, or 15 black dots, respectively) and in the patients (F(2,4)= 7.57, p= .014; mean 2020, 2180, and 2210 ms for 0, 7, and 15 black dots with a target on right grid-side; 2620, 2790, and 2890 ms for 0, 7, and 15 black dots with a target on left grid-side). This is just as expected for a standard visual search task. But note that all of our most critical results concerned the significant difference in accuracy for target location memory (same/different responses on the probe-target) across the different remapping conditions (see main text). Moreover, the difference in accuracy of spatial memory between the critical remapping conditions did not depend on the number of background dots (see main text). Similarly, latency estimates for the letter reports did not indicate any significant asymmetry between left vs right stimuli, suggesting no greater difficulty for detecting a single left than a single right item at fixed and predictable peripheral locations in the remapping condition.