Cognitive Dimensions of Orthographic Stimuli Affect Occipitotemporal Dynamics

Abstract

Previous research documented letter-string specific cortices in the ventral visual stream near the left occipitotemporal junction (i.e., anterior fusiform gyrus). These neural areas potentially code the perceptual elements comprising orthographic stimuli, and thus function as feature detectors in high-level vision. While abundant evidence supports this region’s role in detecting isomorphic perceptual features, any influence cognitive dimensions (e.g., the lexicality of letter-strings) may play in modulating this area’s processing remains an open question. To investigate this, we examined the spatiotemporal dynamics of high-density magnetoencephalographic signals, recorded as subjects completed a rhyme-judgment task on stimuli varying in the cognitive property of lexicality. Our data demonstrate that the time course of occipitotemporal cortices discriminates cognitive attributes of orthographic stimuli. The dynamics in this brain region may indicate interactive processes unfolding later in the time course, when more anterior fronto-temporal circuits are activated by semantic correlates of real words.

Introduction

This paper is concerned with clarifying the role of left occipitotemporal (LOT) cortices in language processing. Numerous studies have implicated the most posterior aspect of the left inferior temporal gyrus, usually extending ventrally to include the anterior fusiform gyrus, as a necessary component of the normal reading system. However, the precise function(s) performed by LOT cortices remain incompletely understood, which limits the field’s capacity to characterize the unique contribution(s) of this neural region in relation to the other classically-recognized members of the language processing circuitry. Early on, the lesion-deficit approach showed that damage to LOT/anterior fusiform regions could cause pure alexia, or letter-by-letter reading (Damasio and Damasio 1983). More recently, functional neuroimaging studies in normal subjects have examined whether LOT areas are fully specialized for orthographic stimuli, or perform more generic mid- to high-level visual analyses. Activation of LOT cortices is typically stronger for pseudowords, relative to words, like other left hemispheric regions subserving language processing (Price et al. 2003). In fact, a recent review concluded that no brain regions are consistently more active during real word processing (Mechelli et al. 2003). Presumably, pseudo- and real word reading utilizes the same neural areas, and the greater activation simply indicates that pseudoword processing taxes the entire system to a greater extent. This scenario limits the capacity of hemodynamic data to allocate the components of language processing amongst the neural structures involved in normal reading. For example, relative to words, reading pseudowords places increased demand on the neural region(s) translating orthography into phonology, but since pseudowords induce greater activation throughout the system, ascribing this function to a structure becomes more complicated.

However, there are other ways to dissociate components of the language processing system. Thus far, most studies have used the lexical decision task or word reading, and made comparisons across different categories of stimuli. Another approach is to use multiple tasks with a single class of well-controlled stimuli, and make comparisons across the different tasks (McDermott et al. 2003). For example, one can probe phonological processing with the rhyme-judgment task, or selectively burden the semantic system through the semantic-judgment task. Across multiple studies, such manipulations underlie the reasonable consensus that LOT/anterior fusiform cortices serve prelexical processing, and may act as a direct interface with the semantic system (Jobard et al. 2003). Under this view, all orthographic stimuli activate this brain region, but frequently encountered words benefit from modulation through the semantic system; thus LOT cortices might provide a direct route to meaning through top-down semantic mediation. In contrast, pseudowords would not receive this top-down modulation, and after a brief ‘search’ process codes would be transferred to anterior brain areas serving phonological decoding.

Functional neuroimaging studies in acquired-dyslexia patients have provided additional evidence for semantically mediated processing in LOT cortices.Price et al. (2003) acquired functional magnetic resonance imaging (fMRI) data from a phonological dyslexic and a surface dyslexic completing reading tasks. The patient with phonological dyslexia could not read pseudowords or words with low imageability. He also made semantic mistakes such as reading ERROR as “wrong,” suggesting that semantic mediation compensated orthography-to-phonology translation. This patient activated LOT cortices, left inferior frontal areas, and right temporal regions during the fMRI protocol. Remarkably, he successfully read highly imageable words without the left superior temporal areas (lesion site) that others implicate as necessary for orthography-phonology conversion (Simos et al. 2000). Conversely, the patient with surface dyslexia was able to read words with regular spelling-to-sound relationships and pseudowords, but not irregular words (e.g., yacht). Price et al.’s patient suffered from bilateral atrophy of the anterior temporal lobes, with damage extending posteriorly along the inferior temporal gyrus into LOT cortices. Despite such damage, her fMRI results revealed a completely normal reading system with the notable exception of left superior temporal and LOT neural regions. Critically, in comparison to a control group, she showed enhanced activity in all neural areas involved in phonological processing and reduced activity in semantic areas, which supports behavioral observations of a strict reliance on phonological processing during reading (Price et al.). Overall, these data indicate LOT cortices are not necessary for successful orthography-to-phonology translation, but are necessary for successful reading in the absence of an intact phonological processing system (i.e., semantically mediated reading).

Cognitive models propose temporal structure to the sub-processes involved in reading, and in doing so provide another venue for exploring structure-function relationships. Utilizing this information requires techniques with high temporal resolution, and several neuromagnetic investigations have applied these cognitive models toward interpreting the functional significance of activated neural regions based on time course data. Such studies describe a letter-string-specific response occurring at ~150 ms in LOT/anterior fusiform areas. The amplitude of this response is larger for syllables relative to individual letters, larger for words relative to syllables, and larger for words relative to length-matched geometric symbols (Tarkiainen et al. 1999). This neural response also exhibits a latency effect consistent with letter-string specificity (Tarkiainen et al.), and was shown to be diminished or even absent in a related study of developmental dyslexics (Helenius et al. 1999). However, recent evidence indicates that this response does not discriminate the cognitive dimension of lexicality (Cornelissen et al. 2003). Converging evidence is provided by intracranial recordings, which describe similar letter-string specificity in roughly the same neural area (Allison et al. 1994). In sum, the available electromagnetic evidence focuses on the early aspects of the time course, and suggests a high-level visual analysis role for this brain region.

Thus, conclusions derived from meta-analyses of hemodynamic studies and fMRI in language-impaired patients are in disagreement with those obtained by electromagnetic approaches. The hemodynamic data indicates that LOT regions play some role in the more cognitive aspects of language processing through interaction with fronto-temporal neural circuits, whereas the electromagnetic evidence suggests that the function of LOT cortices is more limited to bottom-up driven high-level visual processing. However, this discrepancy may be artificial in that electromagnetic studies have been limited in their scope to the initial 200 ms of the time course. Given that semantic processing and/or semantic mediation of lexical retrieval may not occur until later in the epoch, when left perisylvian regions become active, a potential role for LOT areas in such processing cannot be ruled out. In fact, several studies using linguistic stimuli, but focusing on different cognitive phenomena, show activation in LOT cortices up to ~600 ms into the time course (Dale et al. 2000; Dhond et al. 2001, 2003), which may indicate a functional role in higher-level integrative processes involving more anterior language areas.

In the current study, we extracted the entire time course of the magnetic signal in LOT cortices as subjects performed rhyme-judgments on words and pseudowords. This task limits the effect of distinct reading strategies, and also ensures that all stimuli undergo complete orthography-phonology conversion. As for semantic mediation, we assume that the effect is manifested obligatorily on lexical access. Presumably, the effect is ubiquitous and underlies many of the reaction time (RT) differences commonly observed in psycholinguistic experiments, such as pseudohomophones being processed faster than pseudowords (McCann and Besner 1987). Thus, if semantic areas do modulate processing in LOT regions, we should observe such a modulation, even though our task makes no explicit demands on semantic processing or lexical retrieval.

To extract the time course of LOT cortices, we used a 248-channel neuromagnetometer. Magnetoencephalography (MEG) non-invasively measures magnetic fields that emerge from postsynaptic currents generated through the activity of parallel-oriented pyramidal cells of the neocortex (Hämäläinen et al. 1993). The technique combines excellent temporal resolution with good spatial accuracy. By investigating the spatiotemporal dynamics, we can observe how processing changes in LOT cortices as later activity commences in more anterior fronto-temporal regions. Our results indicated that both words and pseudowords induced equivalent activation in LOT cortices during the first ~250 ms of the time course; however, after ~250 ms, this region remained active during the processing of words, but became mostly dormant during pseudoword processing.

Materials and Methods

Subjects

Eleven native English speakers age 18–41 years (mean age = 28 years) were paid to participate in the experiment (8 males and 3 females). One male subject’s data was discarded due to poor signal-noise ratios. All subjects were strongly right-handed (range: 75–100; Oldfield 1971), had normal or corrected to normal vision, and denied any history of neurological or psychiatric disease. Each subject provided informed consent to a protocol approved by the relevant Institutional Review Boards.

Experimental paradigm

Subjects performed a rhyme-judgment task while supine in a dimly lit, magnetically shielded room (MSR). The experiment consisted of 5 blocks, each lasting approximately 70 seconds with a 15-second inter-block interval. Thus, overall recording time was ~7 minutes. In each block, subjects viewed (duration = 600 ms; stimulus-onset-asynchrony = 1200 ms) 47–49 non-targets and 7–9 targets in pseudo-randomized order. Each block contained a total of 56 stimuli and, on average, an equal number of targets and non-targets from each stimulus condition. The target stimulus set consisted of 20 pseudowords and 20 words, each rhyming with the word “trail.” Most target stimuli were orthographically dissimilar to “trail” (e.g., whale), which deterred task performance based on orthography alone. The non-target stimulus set consisted of 80 high-frequency concrete nouns (range: 1.01 – 1.78 log; mean: 1.45 log; Kucera and Francis, 1967), 80 pronounceable pseudowords, and 80 consonant strings. No stimulus in the non-target set rhymed with the word “trail,” and no stimulus in either set was repeated. All stimuli were 4–6 letters long and presented in white 36-point Courier font on a black background. Stimulus presentation alternated with a white fixation cross. We included consonant strings in our stimulus set only for comparison to an earlier experiment, thus these data will be reported separately. Furthermore, orthography-to-phonology translation (purpose of task) cannot be performed on consonant strings, which makes any comparison with the other stimuli misleading. To create pseudowords, we shuffled the phonemes of the concrete nouns; thus, phonemic units present in the corpus of words were preserved in the pseudowords. Particular care was also taken to ensure that pseudowords resembled real English words in all respects, with the exception of lexical and semantic status (i.e., we screened the stimulus set for pseudohomophones and other ‘special’ pseudowords). Subjects responded with a button press when a word or pseudoword rhyming with “trail” was observed, and did not respond to other stimuli (i.e., go/no-go task). Before MEG acquisition, subjects were asked to limit blinking during stimulus presentation to reduce associated artifacts. However, during the inter-block intervals, subjects were told via visual display to blink freely. An LCD projector outside the MSR projected stimuli onto the middle of a screen positioned ~60 cm above the subject.

Data Acquisition

With an acquisition bandwidth of 0.1–200 Hz, neuromagnetic responses were sampled continuously at 508 Hz using a Magnes 3600 WH equipped with 248 axial-gradiometer sensors (4-D Neuroimaging, San Diego, CA). Each sensor is coupled to a SQUID (superconductive quantum interference device), which acts as a low-noise magnetic flux-to-voltage converter. All MEG data were subjected to a global noise filter subtracting the external, non-biological noise obtained through the MEG reference channels, and stowed for offline analyses. Along with an electrooculogram (EOG), we recorded a photodiode signal to ensure precise timing in stimulus delivery.

Prior to MEG measurement, five coils were attached to the subject’s head and the locations of these coils, together with three fiducial points and the scalp surface, were determined with a 3-D digitizer (Fastrak 3SF0002, Polhemus Navigator Sciences, Colchester, VT). Once the subject was positioned inside the MSR, an electric current was fed to the coils. This induced a measurable magnetic field and allowed the coils to be localized in reference to the sensors. Since the coil locations were also known in head coordinates, all MEG measurements could be transformed into a common coordinate system. With this coordinate system (including the scalp surface points), we coregistered MEG data and structural MRI data using the BrainVoyager 2000 software (Version 4.9; Brain Innovations, The Netherlands).

T1-weighted axial images were acquired on a GE Signa Horizons LX 1.5T MR scanner using a neuro-vascular head coil, and a 3-dimensional SPGR sequence with the following parameters: TE = minfull, TR = 20 ms, Flip angle = 30 deg., FOV = 240 × 240 mm, matrix = 256×256, slice thickness/gap = 1.5/0, NEX = 1. The resulting voxel resolution was 0.94 × 0.94 × 1.5 mm. The volume covered extended from the top of the head to the bottom of the cerebellum, including the external auditory meati bilaterally.

MEG Data Analyses

MEG data was split into 1-second epochs, which included a 200 ms pre-stimulus baseline. Artifact rejection was based on a fixed threshold method (EOG > 100 uV or MEG level > 1.5 pT), supplemented with visual inspection. Epochs in which the subject responded (targets) were also rejected. For each subject, two average bins were created (word and pseudoword), and each bin contained a minimum of 60 trials (out of 80 possible). After averaging, we filtered the MEG signals (1–44 Hz) and performed source localization using a spherically-symmetric conductor model. We used contour plots to identify time periods with clear dipolar field patterns and minimal interference from nearby simultaneously active brain areas. Each dipolar distribution was modeled as a single equivalent-current-dipole (ECD) using the subset of sensors covering both magnetic flux extrema. ECD’s had to maintain > 0.90 goodness-of-fit (GOF) over a 10 ms interval (i.e., 5 data points) to be accepted as a reliable source. Furthermore, dipolar fields had to exhibit dissipation and subsequent reorganization to be acknowledged as a distinct source and entered into a separate ECD model. A more detailed description of our source localization procedures is available (Wilson et al. in press). We used the Brain Electrical Source Analysis software (BESA 5.0.4; MEGIS Software GmbH, Germany) for all MEG data pre-processing and source modeling.

Results

Behavioral Data

Error rates for the rhyme-judgment task were too low (0.94%) for further analyses. Subjects distinguished words rhyming with “trail” faster than pseudowords (mean RT: words = 604 ms, pseudowords = 621 ms), although this difference was not significant (paired t-test, p > 0.25). The relative mean RT is consistent with past studies, and it is likely that the lack of significance is due to the limited degrees of freedom (df = 9).

MEG Data

Both words and pseudowords not rhyming with “trail” evoked a strong MEG signal. In each subject, contour plots indicated initial responses to be bilateral and near the occipital pole, consistent with early visual processing. Activation then progressed anterior and became predominantly left-lateralized. By 200 ms post-stimulus, both conditions had activated LOT cortices in all subjects. In contrast, only a subset of subjects (words 5/10, pseudowords 4/10) activated the right hemisphere homologue, and the magnitude thereof was qualitatively weaker. During the next 50–100 ms, activation spread further anterior into left perisylvian regions, with an earlier latency for words relative to pseudowords. For the remainder of the epoch, both words and pseudowords induced robust activation in a distributed network of classic left hemisphere language areas (i.e., superior temporal gyrus/sulcus, frontal operculum, and temporal-parietal regions; see Figure 1a –b). In contrast to the numerous sources per region per subject detected in the left hemisphere, activity in right hemisphere homologues was sparse. Collapsed across subjects and conditions, only right superior temporal and temporal-parietal regions (including supramarginal and angular gyri) showed notable activation, and relative to left hemisphere homologues the magnitude was diminished (i.e., far fewer sources per subject; see Figure 1a–b).

Figure 1.

Sources active 125–600 ms post-stimulus onset, in all subjects, have been projected to the surface of a standardized 3-D rendering of a participant’s MRI for easier visualization. The different colors represent different participants (i.e., all sources detected in the same participant are the same color). Sources in the (a) word condition and (b) pseudoword condition were strongly left-lateralized, and displayed remarkable spatial consistency within-subject. Each condition evoked substantial activation in the entire network of classically-recognized left hemisphere language processing areas. As shown, activation tended to cluster in posterior fusiform gyri, LOT/anterior fusiform region, left superior temporal areas, and to a lesser extent parietal regions. In some subjects, substantial activation was also present in the left inferior frontal gyrus.

The spatial aspect of our data did not indicate salient differences between conditions (see Figure 2), but the time course clearly differentiated words from pseudowords. Specifically, activation in LOT cortices was robust for both conditions early on (i.e., before 250 ms), but significantly dissipated later in the time course of pseudoword processing. For statistical evaluation, all ECDs localizing to LOT cortex were grouped into one of three latency bins (i.e., dipoles peaking before 275 ms, 275–425 ms, or after 425 ms). Each latency bin spanned ~150 ms of stimulus processing, as activity commenced in this region ~125 ms after stimulus onset and the behavioral response occurred at ~600 ms. After binning the data, we performed a repeated-measures ANOVA with condition (2 factors) and latency bin (3 factors) as within-subject variables, and number of reliable sources (i.e., 0.90 GOF) as the dependent measure. The validity of our dependent variable as a metric of regional activation has been repeatedly demonstrated (Simos et al. 1998, 1999, 2000; Breier et al. 1999; Papanicolaou et al. 1999). Since number of sources is a count variable, it is appropriate to re-express the data using a square-root transformation (Tukey 1977). This transformation stabilizes variance and decreases skewness associated with count variables; thus, we applied the square-root transformation before performing ANOVA.

Figure 2.

Representative subject. Word elicited sources are depicted in red and pseudoword responses in blue. Both conditions evoked robust activity in a network of left hemisphere regions, including the superior temporal gyrus/sulcus and LOT cortices. Right occipitotemporal areas were also activated in this subject. As shown, source areas across the two conditions overlapped almost entirely. Right hemisphere sources were also detected (temporal lobe, not shown), but were far less numerous. The 2-D MRI plots are shown in radiological convention, and the cylindrical bar on each ECD (3-D rendition only) represents the orientation of the cortical current.

The assumption of sphericity held in our data set, and all reported values assume sphericity. The main effect of condition was significant F(1,9) = 9.22 (p < 0.02), with more LOT activation in the word condition. The effect of latency bin was also significant F(2,18) = 12.54 (p < 0.001), and pairwise comparisons revealed more sources before 275 ms, relative to the 275–425 ms (p < 0.01) and after 425 ms bins (p < 0.01). The condition-by-latency bin interaction effect was significant F(2,18) = 3.68 (p < 0.05), and within-subject contrasts showed only the linear component to be informative F(1,9) = 5.16 (p < 0.05). To explore the interaction effect, we contrasted the two conditions in each latency bin. As shown in Figure 3, this set of analyses indicated significantly more word-elicited sources after 425 ms (paired t-tests; t(9) = 2.53, p < 0.05). The 275–425 ms latency bin showed a similar trend, but it was not significant (p < 0.08). LOT activity before 275 ms was stronger during pseudoword processing, but this effect did not approach significance.

Figure 3.

Time course of LOT cortices. Sources localizing to LOT/anterior fusiform cortex during each of the three latency bins for both conditions. The black line represents the word condition and the gray line refers to the pseudoword condition. The ordinate displays estimated marginal means of the dependent measure (i.e., number of sources per latency bin, after data transformation). LOT activation was significantly greater for words after 425 ms (p < 0.05), but this trend is clearly visible in the middle latency bin (275–425 ms) indicating LOT cortices distinguish cognitive dimensions of orthographic stimuli shortly after ~275 ms. Error bars represent one standard error of the mean.

Discussion

We extracted the time course of LOT activity as subjects completed a rhyme-judgment task on words and pronounceable pseudowords. Overall, our spatiotemporal maps showed considerable overlap in the distributed set of left hemispheric regions serving word and pseudoword processing, and thus are in agreement with past neuroimaging studies of language processing (Jobard et al. 2003; McDermott et al. 2003; Mechelli et al. 2003; Price et al. 2003; Wilson et al. in press). Furthermore, the early activation (< 200 ms) in LOT regions is consistent with past MEG studies and intracranial recordings focusing on similar phenomena (Allison et al. 1994; Helenius et al. 1999; Tarkiainen et al. 1999; Cornelissen et al. 2003). However, the present results extend these past findings by indicating the time course of LOT cortices discriminates the lexicality dimension of linguistic stimuli.

Previous research on the function(s) of LOT cortex in language processing has tended to support two marginally distinct positions. On the one hand, LOT areas perform high-level visual analyses specialized for perceptual features of orthographic stimuli (Allison et al. 1994; Helenius et al. 1999; Tarkiainen et al. 1999; Cornelissen et al. 2003); and on the other, LOT regions perform this function but additionally participate in more abstract integrative processes malleable to top-down modulation from more anterior language areas (Dhond et al. 2003; Jobard et al. 2003; Marinkovic et al. 2003; Price et al. 2003). The current data lend support to the latter position as these cortices remained active well beyond visual analyses stages, and further this later activity distinguished stimuli differing only in the cognitive attribute of lexicality.

Investigating how lexico-semantic properties affect processing in LOT regions has not been a focus of previous studies, although substantial supporting data can be gleaned from related contexts. For example, studies of dual-process models suggest that LOT cortices remain active after 300 ms during the processing of pseudohomophones and words, but not standard pseudowords (Simos et al. 2002). Pseudohomophones and pseudowords differ only in the semantic dimension, thus this later activity may result from phonological codes contacting semantic areas which in turn feedback information toward LOT regions in attempt to resolve the word form. LOT cortices also show word-repetition effects that are not limited to early aspects of the time course. Marinkovic et al. (2003) reported repetition effects early in the epoch, which could indicate preferential processing of recently activated word forms, and late in the epoch (~400 ms) when repetition effects were also present in more anterior language areas. Perhaps the later, more distributed priming effect reflects integrative processes involving the core network of language processing regions computing the semantic attributes necessary for successful task performance. Lastly, there is data from a verb inflection task indicating differential processing for regular and irregular verbs in LOT regions ~340 ms into the epoch (Dhond et al. 2003). According to this group, the greater activation for irregular verbs does not appear until widespread fronto-temporal language circuits are engaged, which may indicate LOT cortices receive substantial top-down modulation from widely distributed brain regions during the computation of irregular verbs by the language processing system (Dhond et al. 2003).

Conclusion

The current data indicate a progression of neural activity starting bilaterally in posterior occipital areas, spreading anterior toward LOT/anterior fusiform regions, and becoming more strongly left lateralized as activity reaches anterior fronto-temporal language regions. Within this progression, phonological codes are assembled and retrieval of lexical and semantic attributes is obligatorily attempted. In the current study, we demonstrated that stimuli possessing a semantic dimension elicit greater activation in LOT cortices later in the time course. This later activity may be indicative of integrative processes involving a distributed network of left hemisphere language areas, and dynamic interaction amongst fronto-temporal and LOT circuitry may be a precursor for successful computation of semantic codes from input orthography.