Visual word processing and experiential origins of functional selectivity in human extrastriate cortex

Abstract

How do category-selective regions arise in human extrastriate cortex? Visually presented words provide an ideal test of the role of experience: Although individuals have extensive experience with visual words, our species has only been reading for a few thousand years, a period not thought to be long enough for natural selection to produce a genetically specified mechanism dedicated to visual word recognition per se. Using relatively high-resolution functional magnetic resonance imaging (1.4 × 1.4 × 2-mm voxels), we identified a small region of extrastriate cortex in most participants that responds selectively to both visually presented words and consonant strings, compared with line drawings, digit strings, and Chinese characters. Critically, we show that this pattern of selectivity is dependent on experience with specific orthographies: The same region responds more strongly to Hebrew words in Hebrew readers than in nonreaders of Hebrew. These results indicate that extensive experience with a given visual category can produce strong selectivity for that category in discrete cortical regions.

Human extrastriate cortex contains a number of regions that respond selectively to specific categories of visual stimuli (1): the fusiform face area (FFA), which responds selectively to faces (2, 3); the parahippocampal place area, which responds selectively to scenes (4, 5); and the extrastriate body area and fusiform body area, which respond selectively to human bodies and body parts (6–8). Each of these regions can be found in roughly the same anatomical location in most subjects. How do they arise in cortex? Here we test the hypothesis that category-selective regions in the ventral visual pathway can be created through visual experience without a strong genetic predisposition for that specific selectivity.

Visually presented words are an ideal stimulus class for testing this hypothesis because they are (i) extensively learned, (ii) visually distinctive, and (iii) unlikely to have been the focus of evolutionary selection pressures (9). Each of us has had extensive visual experience with written words, yet our species has only been reading for a few thousand years (and literacy has only been prevalent for a few hundred years), which is not thought to be long enough for natural selection to produce a genetically specified brain region dedicated to visual word recognition (10–12). Therefore, if a cortical region is found that responds selectively to visually presented words, compared with other visual stimuli, this finding will serve as an existence of proof that extensive experience can be sufficient to produce selectively responsive regions of visual cortex.

Previous neuroimaging (10, 12–17), intracranial recording (18), magnetoencephalography (19, 20), electroencephalography (EEG) (21, 22), and neuropsychology studies (23–25) in both normal and dyslexic readers (26–29) have identified a region in the left inferior occipitotemporal cortex, often termed the “visual word form area” (VWFA) (16, 17, 30), that may be involved in visual word recognition (for review, see ref. 11). However, two important questions remain, which we addressed here.

First, how specific is the response of this region to visual words? That is, does the VWFA respond more to words than to any other category of visual stimuli? Cohen and Dehaene (16, 17, 30), who coined the term VWFA, define it as the region in the nonretinotopic cortex that responds more to words than checkerboards, leaving the question of specificity open. Many prior studies either fail to address the selectivity of word-responsive regions in detail or have potential confounds, such as differences in attention between word and nonword stimuli. Although selectivity for words compared with stimuli other than checkerboards has been reported [e.g., line drawings (14, 18), geometric symbols (20), and black-and-white pictures of faces and houses (24, 31)], this selectivity could still arise from simple visual feature differences. For example, Hasson et al. (14) identified a region responding more to alphanumeric character strings and written words than line drawings of faces, buildings, or tools, but did not test other more visually similar classes of stimuli. Given the large number of low-level features that differ between alphanumeric characters and line drawings, these findings leave the specificity of this region unclear. Polk and Farah (10, 12) identified voxels in some subjects that responded more to visually presented words than to digit strings, but these voxels often showed no difference in response between words and geometric shapes, suggesting that they were not selective for words. Further, a number of studies have reported no evidence for a region of ventral visual cortex selectively responsive to visual words or letters (32–36). Finally, it has been argued that the VWFA is activated in a number of different tasks that do not require visual word form processing (37) and may instead subserve general perceptual processing for any meaningful visual stimulus (38, 39). Thus, no consensus exists in the literature on the selectivity of the putative VWFA.

Second, if strong selectivity for visually presented words in fact exists, does it depend on experience with words? An alternative hypothesis is that an apparently word-selective region might instead reflect a preexisting selectivity for visual features present in word stimuli regardless of experience. To test whether the specific selectivity (if it exists) comes from experience, it is necessary to compare neural responses in people with and without relevant experience.

To answer these two questions of the selectivity and origin of the putative VWFA, we used relatively high-resolution (1.4 × 1.4 × 2 mm) functional MRI (fMRI) to scan both Hebrew and non-Hebrew readers while they viewed English words, consonant strings, Hebrew words, digit strings, Chinese characters, and line drawings. We found a small region in the left hemisphere of most participants selectively responsive to letter strings (responding more to words and consonant strings than to other stimuli). Furthermore, this region responded more strongly to Hebrew words in Hebrew readers than in non-Hebrew readers. Taken together, our data support the hypothesis that extensive experience can produce strong selectivity in regions of the extrastriate cortex.

Results

Word Selectivity in the Ventral Visual Cortex.

First we looked for selectivity for English words in English readers (who could not read Hebrew). Our initial screen entailed testing for a higher response to English words than line drawings of objects in a blocked-design experiment (Fig. 1). This criterion represents a necessary but not sufficient condition for a region to be regarded as word-selective. In 20 of 23 participants, we identified clusters of voxels outside of the posterior occipital cortex in the region of the occipitotemporal sulcus and fusiform gyrus that responded more to English words than to line drawings (Fig. 2). We refer to the candidate word-selective cluster in each participant as the words versus drawings region of interest (WvD ROI). The location of the WvD ROI in our participants is consistent with the location of the putative VWFA reported previously (14, 16, 17, 30). Voxels with a greater response to words than line drawings were also sometimes observed in the occipital cortex, probably reflecting differences in the retinotopic spatial envelope of line drawings and words (14), and on the lateral surface of the brain close to the superior temporal sulcus, likely corresponding to regions previously identified as involved in phonological processing (33, 40). However, coverage of the occipital and superior temporal regions was limited by the restricted slice prescription available during high-resolution scanning, and data from these regions were not obtained in all participants. Here we focused on clusters of apparently word-selective voxels in the ventral visual cortex.

Fig. 1.

Experimental design and stimuli. In blocked-design runs (A), participants saw 15-sec blocks of English words, consonant strings, Hebrew words, and line drawings interleaved with blocks of fixation only and either fixated passively (half the runs) or responded whenever there was an immediate repeat of the same stimulus. In event-related runs (B), participants saw English words, consonant strings, digit strings, Hebrew words, Chinese characters, and line drawings on interleaved trials and reported whether the stimulus was moving downward or to the right.

Fig. 2.

Regions selectively responsive to English words. Representative functional slices from 8 non-Hebrew readers showing voxels selectively responsive to English words compared with line drawings (P < 10⁻⁴). Selectively responsive voxels were predominantly found in the left (L) hemisphere close to the occipitotemporal sulcus.

In 18 of 20 participants, a WvD ROI was found only in the left hemisphere. In two participants, clusters were found in both hemispheres, and we collapsed the data across hemispheres. The WvD ROI averaged 45 voxels in size. For comparison, in 12 participants, we also localized face-selective voxels on the fusiform gyrus (defined by the contrast faces versus objects). The FFA was identified in every participant, and, consistent with previous reports (14), the left FFA was largely adjacent and medial to the WvD ROI. The FFA averaged 485 voxels bilaterally (right, 315 voxels; left, 170 voxels) and was significantly larger than the WvD ROI in the same participants [t(11) = 5.0, P < 0.0001].

As noted previously, a higher response to English words than line drawings represents a necessary but not sufficient criterion for a word-selective region. To provide a stronger test of the selectivity of the WvD ROI in more detail, we measured the response in this region in separate event-related runs in 11 participants viewing English words, consonant strings, Hebrew words, digit strings, Chinese characters, and line drawings [Fig. 3A and supporting information (SI) Fig. 5]. Note that the resulting response magnitudes are not subject to a selection bias (41) because the data used to define the region were independent from the data used to quantify the response. To characterize the selectivity in this region, we performed a series of planned comparisons contrasting the blood oxygen level-dependent response in each of the new stimulus classes to that elicited by the two benchmark conditions used in the localizer experiment, English words and line drawings. We found a significantly higher response to English words than to each of the other stimulus classes [Hebrew words, t(10) = 3.933, P < 0.003; digit strings, t(10) = 2.559, P < 0.03; Chinese characters, t(10) = 7.655, P < 0.0001] except for consonant strings [t(10) = 0.517, P > 0.6]. Words differ from all other stimulus classes in terms of pronounceability, but the similarity in response to words and the harder-to-pronounce consonant strings, as well as the difference in response between consonant and digit strings (which are at least as pronounceable as consonant strings), argue against pronounceability as an explanation of the selectivity observed. The similarity in response to English words and consonant strings suggests that this region is better characterized as being selective for letters or letter strings than for words per se (see SI Table 1).

Fig. 3.

Estimated hemodynamic response functions for the English words versus line drawings ROI in the event-related runs. In non-Hebrew readers (A), the response to English words was significantly greater than to all other stimuli except consonant strings. A similar pattern of response was found in the Hebrew readers (B), except that the response to Hebrew words was much greater than that seen in non-Hebrew readers.

Significantly increased responses relative to line drawings were also found for digit strings [t(10) = 2.711, P < 0.022] and Hebrew words [t(10) = 5.537, P < 0.0001]. However, there was no significant difference in the response to Chinese characters and line drawings [t(10) = 1.634, P > 0.132]. Thus, the WvD ROI also exhibits some selectivity for digits strings and Hebrew words even in non-Hebrew readers possibly due to their visual similarity to English words, which is much greater than for Chinese characters (see SI Text).

To determine whether any regions show selectivity for English words versus consonant strings, we directly compared the response to English words and consonant strings in the blocked-design experiment. Although we were able to identify scattered voxels in many participants exhibiting a preference for words over consonant strings (54 voxels per participant on average), these were not found in any consistent anatomical location across participants. The most significant voxels were found close to the superior temporal gyrus and likely reflect phonological activation for words over consonant strings (33, 40). In the ventral visual cortex, voxels showing a stronger response to English words than consonant strings could only be identified in 13 of 20 participants in whom we localized a WvD ROI and averaged 17 voxels. However, this selectivity for English words over consonant strings was only replicated in event-related data in one of six participants in whom we had collected both blocked-design and event-related data. Thus, we found no clear evidence for regions of the ventral occipitotemporal cortex showing selectivity for English words compared with consonant strings.

In summary, in the vast majority of participants, we were able to identify a region of the extrastriate cortex, predominantly in the left hemisphere and close to the occipitotemporal sulcus and fusiform gyrus, responding selectively to visual words and consonant strings, even when compared with similar visual stimuli (digit strings and Hebrew words). For ease of reference, we refer to this region as the candidate letter string-selective region (cLSSR).

Role of Experience.

The strong response to letter strings in the cLSSR suggests that the selectivity of this region is based on our extensive experience with visual words. However, it remains possible that this selectivity might exist even if we had never learned to read, and that it instead results from some preexisting selectivity for the visual features present in letter string stimuli. The only way to firmly establish the role of experience in generating the observed selectivity is to test people with and without the relevant experience. Therefore, we next compared the response in the cLSSR to Hebrew words in individuals who either did or did not read Hebrew.

Using the same contrast of English words versus line drawings described previously, we identified a cLSSR in 11 of 13 Hebrew readers (who were also proficient English readers) (Fig. 4). In seven subjects these selective clusters were found only in the left hemisphere, in two subjects only in the right hemisphere, and in two subjects in both hemispheres. The average size of the cLSSR was 41 voxels, which was not significantly different from the size found in non-Hebrew readers [t(29) = 0.305, P > 0.75].

Fig. 4.

Regions selectively responsive to English and Hebrew words in Hebrew readers. Representative functional slices from three Hebrew readers showing voxels selectively responsive to English words (Left) and Hebrew words (Right) compared with line drawings (P < 10⁻⁴). Color scale as for Fig. 2.

Having isolated a cLSSR in Hebrew readers, we compared the selectivity observed in separate event-related runs to that in the non-Hebrew readers tested previously. For all stimulus classes except Hebrew words, the pattern of response across the stimulus categories (Fig. 3B) was similar to that observed in the non-Hebrew readers (Fig. 3A): The response to English words was significantly greater than to digit strings [t(10) = 2.531, P < 0.03] and Chinese characters [t(10) = 7.993, P < 0.0001], and there was no significant difference in the response to English words and consonant strings [t(10) = 0.401, P > 0.69]. Critically, however, the response to Hebrew words was much greater in the Hebrew readers than in the non-Hebrew readers. A two-way ANOVA with group and stimulus type as factors showed a main effect of stimulus [F(3.3, 66.2) = 31.071, P < 0.0001], but no main effect of group [F(1, 20) = 4.137, P > 0.055] and a stimulus-by-group interaction [F(3.3, 66.2) = 7.787, P < 0.0001]. This stimulus-by-group interaction was no longer present when Hebrew words were excluded from the analysis [F(2.7, 53.8) = 0.106, P > 0.94]. Paired t tests comparing the responses for each stimulus class found that only for Hebrew words was there a significant difference in response between Hebrew and non-Hebrew readers [Hebrew words, t(20) = 4.425, P < 0.0001; all others, t(20) < 1.4, P > 0.16]. This stronger response to Hebrew words in the Hebrew readers compared with the non-Hebrew readers suggests that experience shapes the selectivity of this region.

An alternative account of the data, however, is that the increased response to Hebrew in Hebrew readers reflects increased attention (i.e., top–down modulation) and not the effect of long-term experience on cortical selectivity (i.e., bottom–up processing). Consistent with this attentional account, the response to Hebrew words in Hebrew readers was significantly greater than the response to English words [t(10) = 2.685, P < 0.023], although the Hebrew readers were also proficient readers of English. We therefore examined the data from the one-back task, where attentional biases should, if anything, work in the opposite direction: greater attention to Hebrew words for nonreaders of Hebrew than readers of Hebrew, arising from greater difficulty with Hebrew words for non-Hebrew readers (as confirmed by analysis of the behavioral data; see SI Table 2). Despite this opposite attentional bias, the blocked one-back data showed the same effect seen in the event-related data: a significantly stronger response for Hebrew words in the Hebrew readers than in non-Hebrew readers (see SI Fig. 6).

Do Hebrew and English words selectively activate the same regions in Hebrew readers? To compare the cLSSR identified with English words in Hebrew readers with regions showing Hebrew word selectivity, we directly contrasted Hebrew words and line drawings in the blocked-design data. In every Hebrew reader with a region selective for English words versus line drawings, we found a similar region selective for Hebrew words versus line drawings (Fig. 4). In every participant, these two regions overlapped considerably, with 79% of the voxels in the English word-selective region also found in the Hebrew word-selective region. The average size of the Hebrew word region was 100 voxels, almost twice the size of the cLSSR in the same participants, although this difference only just reached significance [t(10) = 2.2, P < 0.05]. Thus, similar occipitotemporal regions are selective for Hebrew and English words in readers of both languages (see also SI Fig. 7).

In summary, the response in the cLSSR to Hebrew words is much greater in Hebrew readers than in non-Hebrew readers. Thus, visual experience shapes the selectivity of the visual cortex and can lead to functionally selective cortical regions visible with fMRI.

Discussion

We report here a small region in the ventral occipitotemporal cortex identified in >85% of participants tested that responds selectively to visually presented words and letter strings compared with a variety of control stimuli. For ease of reference, we refer to this region as the cLSSR. Further, and most importantly, we show here that the selectivity of the cLSSR is shaped by experience. The response of this region to visually presented Hebrew words was substantially stronger in Hebrew readers than in non-Hebrew readers. Taken together, these findings indicate that the striking selectivity of at least one region in the ventral visual pathway for a particular stimulus class originates from extensive experience with that stimulus class. Although these findings do not demonstrate the experiential origins of other selective regions in the ventral visual pathway (FFA, parahippocampal place area, extrastriate body area, and fusiform body area), they serve as proof that extensive experience can produce selective regions of cortex.

Of course, the role for experience demonstrated here does not rule out some role for genes, which are necessary for many aspects of brain development and adult function; here we argue only that experience, rather than genetics, specifies the precise selectivity of the cLSSR. Indeed, the fairly consistent anatomical location of the cLSSR across subjects suggests that genetic contributions might predispose this region to develop the letter string-selectivity we observed (42). For example, the location of the cLSSR might be specified genetically by its connectivity (i.e., receiving inputs from visual regions and sending outputs to language-related regions), with its precise selectivity arising from learning mechanisms based on this connectivity. Although we and others (9) consider it unlikely that the precise selectivity of the cLSSR could be specified genetically in the very short period since people began reading, the hypothesis cannot be ruled out entirely until we gain a better understanding of how genes control cortical organization.

Our study demonstrates strong and robust selectivity for visual words in the extrastriate cortex, even when compared with visually similar categories of stimuli such as digit strings and Hebrew words. As detailed in the introduction, prior studies have provided only weak and inconsistent evidence for such selectivity in part because they tested few stimulus conditions (43). We speculate that many of the inconsistencies in the prior literature result from the fact that the cLSSR is very small compared with the size of typical imaging voxels, so when imaging at standard resolutions the response profile of this region will inevitably be averaged with neighboring cortex, and in some cases the region may be missed altogether (see also ref. 7). The present data resolve these ambiguities by showing that when scanning is conducted at relatively high resolution, a region that is strongly selective for letter strings (but not for words vs. consonant strings) can be found in most subjects tested.

Our finding that the cLSSR did not respond differentially to words and consonant strings does not necessarily imply that words and letter strings are processed in the same way in this region. We have shown only that there is no large-scale clustering of responsiveness to words versus consonant strings (but see refs. 17 and 44 and SI Fig. 8). Thus, it is still possible that the cLSSR contains groups of neurons separately encoding only words or only consonant strings, but that these neurons are not segregated on a scale visible at the current resolution. Indeed, fMRI adaptation studies have shown differential sensitivity to words and their anagrams within the putative VWFA (45). However, recent behavioral work (46) suggests that letter strings and words are processed by common mechanisms in the left hemisphere: Consonant string distractors (but not Chinese or line drawing distractors) interfere with word processing to the same extent as word distractors.

An experiential origin of letter string-selectivity suggests that we should see changes in selectivity within the left hemisphere of children learning to read. Although a cross-sectional fMRI study of people ages 6 to 22 years found no relationship between reading ability and activity in the left inferotemporal cortex (35), a recent electrophysiological study (47) reported the absence of selectivity for letter strings versus symbols in the left hemisphere of children who could not yet read words, but robust selectivity in adults. Further, developmental changes in word processing in the ventral stream may extend into adolescence (48).

The experiential origins of selectivity we describe here are also consistent with studies of novel object learning. In a recent study (49), we found that 10 h of training on a novel object category produces a nearly 2-fold response to trained compared with untrained categories in some voxels in the ventral visual pathway, which was not present before training. Similarly, neurophysiological studies in nonhuman primates have found that training on discrimination and recognition of visual objects can lead to changes in the selectivity of neural responses in the inferior temporal cortex, a region critical for object recognition (50, 51). Finally, experience with visual objects can lead to clustering of neurons with similar selectivity within the inferior temporal cortex (52).

Although the present study resolves longstanding questions about the existence and experiential origins of letter string-selectivity, important questions remain. What exact computations are conducted in this region, and what kinds of representations does it extract from visually presented words? Important clues come from studies using fMRI adaptation, which suggest that representations in this region are case-independent (45, 53). Such case-independent representations also point to the role of experience in shaping the selectivity of letter string-selective regions of cortex. Second, although our findings demonstrate the role of experience in shaping the selectivity of the cLSSR, they leave unanswered the question of why this region lands in a very similar location across subjects and why it apparently cannot “move over” to adjacent cortex when damaged in adulthood (54, 55). Finally, what is the time course of the development of the cLSSR (56), what does this region do before children learn to read, and (how) does it differ in dyslexic versus normal children (26–29)?

Experimental Procedures

All experiments and procedures were approved by the institutional review boards of Massachusetts Institute of Technology and Massachusetts General Hospital. All participants gave informed consent before starting the experiments.

Participants.

There were 41 right-handed participants (ages 18–45): 25 (11 male, 14 female) native English speakers with no experience of reading Hebrew (non-Hebrew readers) and 16 (11 male, 5 female) Hebrew readers who started to read Hebrew in childhood either before or shortly after learning to read English. Seven Hebrew readers were native Hebrew speakers, and the remaining nine were native English speakers with extensive experience of Hebrew. No significant differences were observed between native and nonnative Hebrew speakers, either in the size of the cLSSR or in the pattern of selectivity. Therefore, the data for Hebrew readers were collapsed across native language.

All participants were scanned in an initial blocked-design experiment. Twelve native English speakers and all Hebrew readers were also scanned in event-related runs.

One non-Hebrew reader and two Hebrew readers were excluded for excessive head motion, and one non-Hebrew reader and one Hebrew reader terminated the scan session early.

Stimuli.

Participants saw concrete English words (uppercase), English consonant strings (uppercase), Hebrew words, line drawings, digit strings, and Chinese characters. Separate sets of stimuli were used for the blocked-design and event-related runs. English words, consonant strings, digit strings, and Hebrew words all subtended ≈1.5° of visual angle in height and ≈1° in width for each character. Chinese characters subtended ≈4° square, and line drawings subtended up to 6° along their longest axis. For further details on the characteristics of the orthographic stimuli, see SI Text.

Scanning.

Participants were scanned at the Martinos Center for Biomedical Imaging at Massachusetts General Hospital in a 3T Siemens (Erlangen, Germany) Trio magnet with a custom-built eight-channel phased-array surface coil. Functional images were acquired with an EPI sequence with GRAPPA (echo time = 46 msec; 128 × 128 matrix; field of view 180 × 180 mm; 12–15 slices approximately parallel to the base of the temporal lobe with a slice thickness of 2 mm and an interslice gap of 0.4 mm). For blocked experiments, repetition time = 3 sec; for event-related designs, repetition time = 1.5 sec.

Participants were scanned in both blocked-design and event-related runs. Blocked-design runs were used to identify word-selective regions. Event-related runs were used to characterize the responsiveness of these regions across different types of stimuli.

Blocked-Design Runs.

Participants viewed four runs during which 15-sec blocks (30 stimuli per block) of English words, consonant strings, Hebrew words, or line drawings were presented interleaved with blocks of no visual presentation. Stimuli were presented for 200 msec with an interstimulus interval of 300 msec. Each run contained 21 blocks and lasted for 5 min and 15 sec. For half of these runs, participants simply had to maintain fixation. In the other half, participants performed a one-back task, responding every time the same image was presented twice in a row (see SI Table 2 for behavioral data).

Event-Related Runs.

Participants viewed five runs during which English words, consonant strings, digit strings, Chinese characters, line drawings of objects, and Hebrew words were presented on interleaved trials. Each trial lasted for 1.5 sec, with a 300-msec stimulus presentation followed by 1,200 msec of fixation. During each trial, the stimulus moved either to the right or down, and the participant's task was to report the direction of motion using a button box (see SI Table 3 for behavioral data). Each run contained 150 trials (25 per stimulus class) interspersed with periods of fixation and lasted for 5 min and 39 sec.

FFA Localizer.

If time permitted, participants also viewed two blocked-design runs during which 15-sec blocks of photographs of faces or objects were presented interleaved with blocks of no visual presentation. Each run comprised 21 blocks with 20 images in each stimulus block and lasted 5 min and 15 sec; participants performed a one-back task.

Analysis of Imaging Data.

Data were analyzed with Freesurfer (www.surfer.nmr.mgh.harvard.edu/) (57, 58), froi software (www.froi.sourceforge.net), and custom Matlab code. Data were motion-corrected and, for the blocked-design runs only, smoothed with a Gaussian kernel of 3-mm FWHM. Significance maps of the brain were computed by performing t tests for pairwise comparisons of conditions and thresholded at P = 0.0001 (uncorrected).

Preprocessing did not involve any spatial normalization of subjects in a common reference space (e.g., Talairach). Given the anatomical variability between subjects, such normalization would obscure finer spatial patterns in activations, certainly at the high resolution that we used.

To avoid selection bias, all analyses were performed by using one data set to define ROIs and an independent data set to examine the pattern of response across conditions.

Within the WvD ROI, statistical analysis was performed on the peak responses during event-related and blocked-design runs. For event-related data, we averaged the responses at 4.5 and 6 sec poststimulus onset. For blocked-design data, we averaged responses at 6, 9, 12, and 15 sec after onset of the block.

Abbreviations

cLSSR

candidate letter string-selective region

FFA

fusiform face area

fMRI

functional MRI

VWFA

visual word form area

WvD ROI

words versus drawings region of interest.