Electrophysiological evidence for the left-lateralized effect of language on preattentive categorical perception of color

Abstract

Previous studies have shown that the effect of language on categorical perception of color is stronger when stimuli are presented in the right visual field than in the left. To examine whether this lateralized effect occurs preattentively at an early stage of processing, we monitored the visual mismatch negativity, which is a component of the event-related potential of the brain to an unfamiliar stimulus among a temporally presented series of stimuli. In the oddball paradigm we used, the deviant stimuli were unrelated to the explicit task. A significant interaction between color-pair type (within-category vs. between-category) and visual field (left vs. right) was found. The amplitude of the visual mismatch negativity component evoked by the within-category deviant was significantly smaller than that evoked by the between-category deviant when displayed in the right visual field, but no such difference was observed for the left visual field. This result constitutes electroencephalographic evidence that the lateralized Whorf effect per se occurs out of awareness and at an early stage of processing.

Behavioral, electrophysiological, and neuroimaging studies have shown that early processing of color stimuli can be affected by the categories encoded in the language of the observer (1–13). More recently, this “Whorfian” effect in color processing has been demonstrated to be stronger for stimuli exposed in the right visual field (RVF) than in the left visual field (LVF) and to disappear in the presence of a concurrent demand on verbal memory. This finding suggests that the use of lexical information in the left hemisphere is the origin of differential visual hemifield responses to color stimuli (14–23). There has been some suggestion that the left cerebral hemispheric areas implicated in important language functions may serve as a top-down control source that modulates the activity level of the visual cortex (24–28), but little is known about whether this lateralized effect occurs at early, preattentive perceptual processing stages or at postperceptual decision/response phases. Visual mismatch negativity (vMMN) is widely held to reflect the brain's early and automatic change in event-related potentials (ERPs) in response to a novel stimulus (29–31). In the present study, the vMMN component is observed in order to assess the lateralized Whorfian response to task-irrelevant changes in color stimuli. There have been several related studies in this area, however. Thierry and associates (25, 26) showed in nonlateralized studies with task-irrelevant responses that Greek speakers’ vMMN components, starting around 100 ms poststimulus, peaking around 200 ms, and maximal over parieto-occipital scalp areas, show sensitivity to the basic lexical distinction in the Greek language between light and dark blue, whereas English speakers show no such effect. Further ERP studies have confirmed early Whorfian effects in nonlateralized paradigms (27, 28). [Fonteneau and Davidoff (12) were the first to show a neural correlate of color categorical perception. They found the peak latency in ERP for a small color difference bridging a color-term boundary to be shorter than that for an equally sized color difference not bridging a color-term boundary.] In this study we examine, and establish, effects on the vMMN component of lexical color categories in a lateralized paradigm requiring color-irrelevant overt responses using a within-subject design.

Taking vMMN as an electrophysiological index of preattentive change detection, we show here that the lateralized Whorf effect takes place early, automatically, and out of awareness. We used colors from a set of four colors of constant brightness and saturation. The colors were a neutral green (G1), a bluish green (G2), a greenish blue (B1), and a neutral blue (B2). They were approximately equivalent in lightness and not quite equally spaced in hue (Fig. 1A). (See Materials and Methods for details.) The ERPs of 30 Mandarin Chinese speakers were recorded while performing a visual oddball task (Fig. 1B). Participants were instructed to focus on the fixation mark and were asked to press a button when the fixation mark changed from the plus sign (“+”) to the circle (“○”). There was no mention of color in the instructions. Each trial presented two colored squares (hereafter “colors”) flanking the fixation mark (Fig. 1B). Within each block of trials one of the two colors adjacent to the blue–green boundary (G2 or B1) served as standard throughout, and each of the four colors served as deviant once (as required). Trials were presented in four blocks of 540 each. Within each block, 70% of the trials consisted of two identical standard colors (either G2 or B1) with a “+” fixation mark (the standard condition); 10% of the trials consisted of the same two standard colors with a “○” fixation mark (the “target” condition); 10% of the trials consisted of the standard color in the RVF and a deviant color in the LVF (the LVF–deviant condition); in the remaining 10% of trials the standard color appeared in the LVF, and the same deviant in the RVF (the RVF–deviant condition). The two blocks containing each standard color were distinguished from each other by one having the within-category deviant and the other the between-category deviant.

Fig. 1.

(A) Print-rendered version of the four colors used. (B) Experimental design and a sample of stimuli presented in the four experimental blocks.

Earlier, nonlateralized research (25, 26, 28) found that cross-category color deviants elicit a greater vMMN response than within-category deviants, revealing language-influenced unconscious change detection at early stages of processing. From the fact that linguistic categories are activated rapidly and influence early visual processing in two visual hemifields differentially, we predicted that the vMMN effect evoked by the between-category deviant would be larger than the vMMN effect evoked by the within-category deviant only or preferentially when the deviant was presented in the RVF. We also expected a larger vMMN effect from the between-category deviant in the RVF than in the LVF.

Results and Discussion

Target Stimuli.

Both the behavioral and electrophysiological responses to the target stimuli indicated a high rate of successful target detection. Behaviorally, the mean proportion of correct hits was 98.2% (SD = 2), and the mean number of incorrect hits was less than two. The mean reaction times were 447 ms (SD = 51). Together these results suggest a high degree of narrowing of attention to the fixation mark. Grand-averaged ERPs evoked by the frequent and target stimuli are shown in Fig. 2A. The target stimuli evoked larger N2 and P3 components. The N2 component occurred with a central scalp distribution, and the P3 component occurred with a parietal-occipital scalp distribution, as shown in Fig. 2B. The mean amplitude of N2 was measured from 200 to 300 ms, and that of P3 was measured from 300 to 500 ms. Paired t tests were conducted across all the electrodes. The mean amplitudes of both the N2 and P3 components evoked by the target were significantly larger than those evoked by the standard (P < 0.05).

Fig. 2.

(A) Grand-averaged ERPs at Cz for standard (solid red line) and target (dashed blue line) stimuli. (B) Scalp topographic maps of N2 (Left) and P3 (Right) for target stimuli.

Deviant Stimuli.

There was an obvious early negative deflection in all the deviant ERPs in relation to the standard ERPs (Fig. 3 A and C, solid green line) except for the within-category RVF deviant (Fig. 3 A, red dashed line), beginning at ∼130 ms poststimulus, lasting until about 190 ms, and peaking at ∼160 ms. This early deviance-related negativity was defined as the vMMN, parallel to mismatch negativity (MMN) in auditory stimuli and to vMMN in other visual stimuli (27, 32, 33). The vMMN was maximal over the lateral parietal-occipital scalp; so electrodes O1, O2, PO7, PO8, PO3, PO4, P5, P6, P7, and P8 were chosen for further analysis. It is important to distinguish a deviance-related mismatch in the relevant time range (100–250 ms), which is a preattentive vMMN, from an attention-related N2b modulation (28, 34). The two patterns show differences in that the vMMN starts earlier, is registered in more posterior regions, and is not followed by an attention-driven, anterior P3a modulation [see ref.(28 for discussion]. The deviance-related negativity recorded here was similar to that observed by Clifford et al. (28), who took the further step of noting that it was restricted to the lower visual field, further confirming that was a preattentive vMMN, and not an N2b.

Fig. 3.

Grand-averaged ERPs (linear derivation of electrodes O1, O2, PO7, PO8, PO3, PO4, P5, P6, P7, and P8; epochs ranged from −100 to 500 ms for graphic display) (A and C) and scalp topographic maps displaying the topographic distributions of the mean amplitude, 130–190 ms (B and D). A and B show ERPs and maps of standard and RVF deviants, respectively. C and D show ERPs and maps of standard and LVF deviants, respectively.

We found no significant difference in peak latency across all the deviants and standard stimuli (F_[4,116] = 1.18, P < 0.323). The mean amplitudes of ERPs in the time window of 130–190 ms evoked by deviants were compared by paired t tests to those elicited by standards at these electrodes. The between-category RVF, between-category LVF, and within-category LVF deviants evoked significantly larger vMMN (P < 0.05 in each case). The vMMN evoked by the within-category RVF deviants did not differ significantly from that of the standard stimuli (P = 0.299).

We then compared the vMMN effects for both deviants and both visual fields in a 2 × 2 repeated-measures ANOVA [pair type (between-category vs. within-category) × visual field (RVF vs. LVF)]. The vMMN component was measured for each visual field in the time window of 130–190 ms by subtracting the ERPs of the standard stimuli from the ERPs corresponding to each deviant. There was a significant interaction between pair type and visual field (F_[1,29] = 6.612, P < 0.016). For the RVF deviants, the vMMN effect of within-category deviants was significantly smaller than the vMMN effect of between-category deviants (t_[29] = −2.906, P < 0.007) (Fig. 4). For the LVF deviants, there was no significant difference between the two pair types (t_[29] = 0.565, P = 0.577).

Fig. 4.

Mean amplitude of the vMMN effects by deviants. Error bars represent SE. *, significant difference at P < 0.05; ns, nonsignificant.

We also compared the vMMN effects in the RVF and LVF for each pair type (Fig. 4). For the between-category pairs, the vMMN effect from the RVF deviants was significantly larger than the vMMN effect from the LVF deviants (t_[29] = −2.198, P < 0.036). For the within-category pairs, deviants in the LVF evoked an MMN component that was numerically larger than evoked by deviants in the RVF, but not significantly so (t_[29] = −1.211, P = 0.236).

We have observed a significant interaction in the vMMN component between color pair type (within-category vs. between-category) and visual field (left vs. right). Specifically, a significantly smaller vMMN effect was elicited by the within-category deviant relative to the between-category deviant when the deviant was exposed in the RVF. In contrast, there was no such difference in the LVF, although both within-category and between-category deviants provoked reliable vMMN effects compared with the standard stimuli. This result converges with previous behavioral and functional MRI data of the lateralized Whorfian effect (14, 15, 17, 24). Color patches of an unexpected hue exposed briefly in the RVF only, and hence received initially in the left, language-dominant cerebral hemisphere, elicit less of a surprise response in the brain when they belong to the same lexical category as an expected color than do colors perceptually equidistant from the expected color but belonging to a different lexical category. No comparable effect occurs with colors exposed in the LVF. Previous research on early ERP evidence of categorical color perception has shown that it varies predictably according to language-specific terminology (25, 26) and that it very probably is automatic and preattentive (28). The present study has shown that the categorical modulation of the vMMN by color difference is lateralized to the RVF and so presumably is controlled by the left cerebral hemisphere, where most language functions are housed and where this finding confirms much behavioral (2, 14–17, 19) and neurophysiological (23–26, 28) evidence. Lateralized Whorfian effects on early color processing strongly appear to be automatic and preattentive.

Materials and Methods

Participants.

Thirty native Mandarin Chinese speakers (14 male, overall mean age 20.9 y), recruited from South China Normal University, participated in the experiment. All participants gave informed consent and were paid for participation. All were strongly right-handed and had normal or corrected-to-normal vision. None had any color vision deficiency, as measured by the Ishihara Color Test, or any history of neurological impairment or psychoactive medication use.

Stimuli and Procedure.

The stimuli shown in Fig. 1 were presented on a 17-inch cathode-ray tube monitor with a resolution of 1024 × 768 pixels and a refresh rate of 75 Hz at a viewing distance of 70 cm. The International Commission on Illumination (CIE) xyY values measured by a spectrometer were as follows: G1 = 0.256, 0.374, 73.5; G2 = 0.242, 0.342, 72.5; B1 = 0.228, 0.308, 76.8; B2 = 0.215, 0.275, 62.9. The xyY values of the gray background were 0.321, 0.347, 34.70. The delta E separations of the stimuli were as follows: G1–G2, 13; G2–B1 14.83; B1–B2 17.18. The fact that the B1–B2 distance is greater than the other two cannot have affected the main result of a categorical effect only in the RVF, because exactly the same stimulus configurations were presented to both visual fields.

All participants were given a blue–green lexical boundary test 1 wk before the ERP experiment. We assumed that a week was sufficient time between the lexical boundary test and the ERP observations to obviate possible priming effects for color difference. It would have been preferable to perform the lexical boundary test after the ERP task to eliminate the need for such assumptions, and replications should follow the latter procedure to simplify the logic of interpretation. Crucially, the major result could not have been affected by any such priming, because the lexical boundary test was administered bilaterally, and the major result turned on the observation of a visual field difference.

On each trial, a 7 × 7 cm square stimulus (one of the four colors, G1, G2, B1, or B2) was presented centrally on a gray background for 500 ms at a viewing distance of 70 cm, followed by a gray blank screen. Participants were instructed to name the stimulus with the corresponding Mandarin Chinese word (“green”) or (”blue”). Each stimulus was presented 15 times in a total of 60 randomized trials. All participants’ naming accuracies were above 95%.

In the ERP experiment, participants were tested in a dimly lit, sound-attenuated booth and were seated in a comfortable chair. During each trial, two 2 × 2 cm colored squares were presented for 200 ms against a gray background, one on each side of the fixation mark, followed by a blank screen for 900 ms. The separation distance of the two squares was 8.5 cm, and the inner edge of each square was 3.5° to one side of the fixation cross. Within a block, a single color, G2 or B1, served as standard throughout. In “standard” trials both stimuli were the same standard color. Within-category deviant trials were of either type (G1, G2) or (B1, B2) and were distinguished further by the side on which the deviant occurred (LVF or RVF). Between-category deviant trials were of two types, each further distinguished by visual field. One type presented G2 as standard and B1 as deviant; the other type presented B1 as standard and G2 as deviant. Thus, there were four subtypes of deviant trials. Participants viewed four blocks of trials, each containing one of the subtypes of deviant trials. Each block consisted of 540 trials and tested one of the standard stimuli against one of its two possible deviants. Within each block, 70% of the trials were “standard,” consisting of the standard color in both positions along with the standard “+” fixation mark, 10% of the trials were “target” trials, presenting the standard color in both positions with the nonstandard “○” fixation mark. The remaining 20% of trials were divided evenly between trials presenting the deviant color in the LVF or in the RVF, both sets using the standard “+” fixation mark. For each standard color there was one block with a within-category deviant and one block with a between-category deviant. The presentation order was pseudorandomized within each block, such that the first five trials were standard, and two deviants never appeared in succession. Block order was counterbalanced across participants. Participants were asked to press the spacebar with the left or right index finger when the fixation cross changed to a circle. The finger responses were counterbalanced across participants.

EEG Recording and Analysis.

Electrophysiological data were recorded continuously in reference to the tip of the nose, with a sampling frequency of 1,000 Hz from 64 Ag/AgCl electrodes mounted on an Easycap according to the international 10–20 system. The recordings were amplified with two 32-channel BrainAmp MR Plus amplifiers. Data were filtered online with a 0.1–100 Hz band pass and 50-Hz notch filter, and refiltered offline with a 0.8–20 Hz band pass zero-phase shift digital filter (slope 24 db/Oct). Bipolar horizontal and vertical electrooculograms were recorded simultaneously to monitor eye movements. The Brain Vision Analyzer software package was used to analyze data. Epochs ranged from −100 to 800 ms after the onset of stimulus. Baseline correction was performed according to prestimulus activity. Ocular artifacts were corrected mathematically by subtracting them from the EEG recordings. Epochs with potentials exceeding ±100 μV at any cap electrode were rejected automatically. Data were segmented and averaged according to stimulus type: standard, target, between-category deviant in RVF (bRVF), between-category deviant in LVF (bLVF), within-category in RVF (wRVF), within-category in LVF (wLVF). All hardware and software were obtained from Brain Products (http://www.brainproducts.com/index.php).