Imaginative Language: What Event-Related Potentials have Revealed about the Nature and Source of Concreteness Effects

Abstract

Behavioral and neuropsychological evidence suggest that abstract and concrete concepts may be represented, retrieved, and processed differently in the human brain. As reviewed in this paper, data using event-related potential measures, some in combination with visual half-field presentation methods, have offered a detailed picture of the nature and source of concreteness effects. In particular, the results provide strong evidence for multiple mechanisms underlying the behavioral processing differences that have long been noted for concrete and abstract words and, further, suggest an intriguing, unique role for the right hemisphere in associating words with sensory imagery.

1. Introduction

Language, the human capacity to communicate productively via structured arrangements of symbolic forms, is undoubtedly one of the most complex products of biological intelligence. To successfully comprehend language, perceptual events experienced through different modalities (for example, spoken, written, or signed words) must be rapidly analyzed, combined, and integrated with knowledge stored in long-term memory. Thus, language processing is rapid, and important language events unfold continuously at multiple time scales, from the millisecond-level timing differences that differentiate some phonemes to the several minutes over which a discourse unfolds. Remarkably, the human brain has the ability to accomplish such complex processes rapidly and effortlessly over time.

Given the speed with which language (and the neural and cognitive processes needed to process it) unfolds, measures with high temporal resolution play a particularly important role in building an understanding of how and when the brain extracts meaning from linguistic input. Therefore, in addition to linguistic analyses and psycholinguistic experiments utilizing behavioral measures, the study of language processing has been critically enhanced by the use of event-related brain potential (ERP) measures.

2. Event-related potentials and the study of language

Language—and indeed all of perception, cognition, and action—arises from electrochemical activity in the sensory receptors and muscles, relayed to and from the brain. This electrical activity can be picked up noninvasively using sensors placed on the body and scalp. By measuring and examining changes in this activity associated with different types of stimuli and different task conditions, one can begin to map out the stream of biological processes that underlie various aspects of cognitive processing. In practice, sensors are placed on volunteers in the laboratory, and the signals picked up by those sensors are amplified, converted from analog to digital form, and stored for later analysis. Those signals can then be examined in parallel with information about the stimuli that the volunteers were exposed to and the responses, if any, that they made (Fabiani et al. 2007). Electrophysiological measures thereby provide data that unfold over time and track, with millisecond-level precision, the timescales critical for language processing. Moreover, they can be obtained without the need for any extraneous task, allowing measurements to be made under naturalistic conditions, as participants simply listen or read for comprehension.

In particular, ERPs are measures of brain electrical activity that are timelocked to an event of interest, such as the onset of a word, a participant’s response, or any other type of temporally well-defined point of interest. The primary source of scalp-recorded ERPs comes from cortical pyramidal cells, which tend to have an open-field arrangement (in which neurons are aligned) and become active in relative synchrony, so that their summed potentials generate a large enough signal to be detected at the scalp. With appropriate recording conditions, attention to potential artifacts, and well-designed experimental protocols (see Luck 2005), ERPs provide a direct, multidimensional measure of brain activity, revealing with high sensitivity not only whether two groups, conditions, or stimuli are processed differently, but when and how—for example, whether the change is in the size, timing, and/or scalp topography of the response.

ERPs have been used to study perception, cognition, emotion, and motor planning for more than half a century now, and, as a result, many aspects of the ERP have been linked to specific cognitive and neural processes (Luck & Kappenman 2012). Thus, the recording of ERPs provides a set of well-characterized dependent variables for the study of language and the variety of cognitive processes that go into successful comprehension (for a review of how ERP measures have been used to address a wide range of language questions, see Kutas & Federmeier 2007). As a very brief overview, early aspects of the ERP waveform (in the first ~ 200 ms) reflect sensory processing and effects of attention. For example, visual evoked responses include the P1, N1, and P2. As discussed later, modulations of P2 amplitude have been observed in the context of language processing and linked to differential attentional states associated with, for example predictive processing. The N400 is a negative-going ERP response that peaks around 400 ms after the onset of a word (or other meaningful stimulus) and has been linked to semantic processing. N400s are smaller than usual for stimuli that are repeated or that appear in a supportive context, such as following a semantically related item or in a congruent sentence (Kutas & Federmeier 2011). The P600 is a positive-going response that peaks after about 500 ms and is sensitive to syntactic aspects of language inputs (Swaab et al. 2012). With measures like these, researchers have been able to make headway on a number of core questions about the nature of language representations and processes—one example of which we review next.

3. Concreteness effects and the nature of semantic representations

At its heart, language comprehension entails linking the symbolic forms that are words (and morphemes) to representations of the referents of those forms, stored in what is generally known as ‘semantic memory’. A major challenge in the study of the human mind and brain, therefore, is to understand the nature and organization of semantic representations. Traditional linguistic approaches have focused on what are sometimes thought of as basic semantic features, such an animacy and countability, and have tended to treat these features as abstract in nature and situated within the language system. Functional and cognitive approaches to linguistics, however, have highlighted the relationship between language and other aspects of cognition and, as such, have emphasized the importance of spatial and sensory features for various aspects of language structure (for example, Wilcox 2004). The role of sensory features has also been a matter of debate within the psycholinguistic and cognitive neuroscience literatures on semantic representation and processing. Views of semantic memory range widely, from accounts that posit that all types of words (and other meaningful representational forms, such as pictures) converge on a single, amodal common store to accounts that argue that different types of inputs engage multiple, different types of representational systems, mediated by different brain areas and/or systems (for various views, see for example, Caramazza et al. 1990; Paivio 1991; Pylyshyn 1980; Riddoch et al. 1988; Shallice 1988). Adjudicating between these views has proven difficult, in part because the same empirical data are accounted for in different ways by the varying theories.

It has been well established that there are processing differences between words as a function of concreteness. Concrete words—that is those, such as ‘table’, that refer to entities that can be readily experienced by the senses—are learned earlier and processed more quickly and accurately than abstract words, such as ‘justice’ (Bergelson & Swingley 2013; Gerhand & Barry 2000; Paivio 1991; Schwanenflugel et al. 1988). Similarly, concrete sentences are easier to comprehend (Schwanen-flugel & Shoben 1983) and to judge for meaningfulness and truthfulness (Belmore et al. 1982). The long-standing controversy is whether such processing differences are best explained as arising because of a qualitative difference in the representations and/or processing systems engaged by these word types (that is, supporting dual/multiple systems views of semantic memory) or, instead, because of quantitative differences in the information accessed by these words from a single system.

Accounts consistent with single system models, such as context-availability theory (Schwanen-flugel 1991), claim that processing differences for abstract and concrete words are attributable to quantitative differences in the amount and quality of available information. This account argues that abstract words and their contexts contain less information and/or less strong connections to associated knowledge than do concrete words. In support of context-availability theory are findings that processing differences between concrete and abstract words are reduced in strong contexts—that is, when even abstract terms have sufficient contextual support (Schwanenflugel 1991; Schwanenflugel et al. 1988).

In contrast, multiple system models, such as the dual-coding theory (Paivio 1991, 2007), claim that concrete words are not only encoded verbally, but also represented non-verbally in an imagery-based system. Accessing the semantics of a concrete concept, then, might involve some kind of perceptual and/or motor stimulation, which, in turn, affords concrete words and sentences their processing and memory benefits. In support of the idea that modality-specific systems are activated during conceptual processing, studies have documented a ‘perceptual-switch effect’, in which switching between modalities incurs processing costs (for example, Spence et al. 2001). For example, Pecher et al. (2003) asked participants to determine whether an object had a specific visual feature (for example, ‘banana–yellow’) and showed that they were faster to do this when the preceding trial had also queried visual features as compared to when the previous trial asked about gustatory features (for example, ‘apple–tart’). Similarly, studies examining whether motor representations are evoked by words have shown priming between words denoting similar motor actions, such piano and typewriter, which are used with similar manual movements (for example, Myung et al. 2006). At issue is whether such simulations constitute a necessary—or at least routine—aspect of semantic access, and whether they are in fact responsible for the concreteness effects that have been observed across a range of tasks and for words denoting concrete concepts with a wide variety and mix of feature types.

In sum, whereas context-availability argues for a quantitative difference between word types within a single system, models like dual coding theory argue for a qualitative difference based on activity in different systems. Numerous behavioral studies have sought to empirically invalidate one or the other of these explanations. Unfortunately, in spite of the relative consistency of the experimental findings, there has been considerable disagreement concerning the underlining mechanism of concreteness effects. Part of the difficulty in differentiating between these accounts based on behavioral data alone is that such end-state measures necessarily summate across all of the multiple cognitive processes that unfold before the response—meaning that there are multiple, viable accounts of any differences that are observed.

3.1 ERP effects of concreteness

Time-course data provided by the ERP technique offer a form of evidence that is different from and complementary to the standard reaction time results on which most of the literature on the semantic processing of concrete and abstract materials is based. Because they can be used to unpack the multiple processes that are summated in behavioral measures, ERPs hold great promise for questions concerning the nature of processing differences. Such data can potentially clarify the issue by providing a window to both semantic memory structures and processes. In a series of studies, Holcomb and colleagues (Holcomb et al. 1999; Kounios & Holcomb 1994; West & Holcomb 2000; see also Barber et al. 2013) examined concreteness effects using ERPs, ultimately providing evidence for an account that combines aspects of the dual-coding and context-availability views. They found that, relative to abstract words, concrete words elicited more negative-going potentials in the time window of the N400 (300–500 ms) over posterior electrode sites, as well as a sustained frontal negativity from 300 ms up to 900 ms (West & Holcomb 2000, who refer to this effect as the ‘N700’). The differing timecourses and scalp distributions of these subcomponents of the ERP concreteness effect suggest that there are multiple underlying sources of concreteness-based processing differences, and subsequent work has shown that these subcomponents are also functionally separable.

The timing and distributional characteristics of the posterior concreteness effect link it to the well-studied N400 component, which has been associated with semantic processing in general and with semantic access in particular (for a comprehensive review, see Kutas & Federmeier 2011). Since the N400 is larger for words with more orthographic neighbors and for words with more lexical associates in long-term memory (Laszlo & Federmeier 2011), the larger amplitude N400 responses observed to concrete words suggest that these items generally evoke more activity in the semantic system. This pattern supports the claims of context-availability theory that concrete words have richer semantic associations. In addition, this N400 concreteness effect is consistent across nouns, adjectives, and verbs, as well as words whose word class is ambiguous, such as duck (Huang et al. 2010; Lee & Federmeier 2008), regardless of task demands. However, this effect is sensitive to context. N400 differences between concrete and abstract words are attenuated and/or eliminated when the words are repeated (Kounios & Holcomb 1994) or are embedded in a predictive context (Holcomb et al. 1999). Again, patterns on the N400 are thus consistent with the predictions of the context-availability theory (Schwanenflugel 1991; Schwanenflugel et al. 1988).

A different pattern of effects is seen for the sustained frontal negativity, which has been shown to be sensitive to word type and to task demands. This effect has been documented for nouns, adjectives, and unambiguous verbs, but not for ambiguous words used as verbs (Lee & Federmeier 2008). Importantly, the presence/size of the effect is modulated by task demands, such that it is larger in tasks making reference to word semantics and/or to imagery than for lexical decisions or tasks that emphasize surface properties of the words (Gullick et al. 2013; West & Holcomb 2000). The fact that a similar effect is seen during explicit visual imagery (Farah et al. 1989; note that the reference electrode used in this study is different, changing the apparent topography of what would be a frontal effect with the reference used in the previously cited work) suggests that sensory imagery processes play a role in concreteness effects. These findings are thus consistent with views, such as the dual-coding account (Paivio 1991, 2007), which attribute concreteness effects to the availability of perceptual and/or motor imagery (simulation) primarily or exclusively for concrete words.

Overall, then, these studies show that concreteness has multifaceted effects on processing. In the absence of context, concrete words elicit more semantic activity than do abstract words in a system that seems to be shared across many types of words, as well as by other types of meaningful inputs like pictures. Moreover, particularly when task demands encourage deeper processing, concrete words seem to elicit more mental imagery. Based on their ERP work, Holcomb and colleagues therefore developed what they call the context-extended dual-coding theory, which combines aspects of both context-availability and dual-coding accounts of concreteness effects and emphasizes the role of both structural and contextual factors in language comprehension processes (Holcomb et al. 1999; see also Levy-Drori & Henik 2006).

3.2 Neural underpinnings of concreteness effects

The observation that the part of the concreteness effect linked to imagery is biggest over the front of the head (when an average mastoid reference is used) cannot be used to infer that the neural generators for this effect are frontal, as mappings between surface topography and the location of the underlying neural generators (the ‘inverse problem’) are not possible without additional constraints (for attempts to model the generators of ERP concreteness effects, see Adorni & Proverbio 2012). However, information about the brain areas involved in concreteness effects can be gleaned from other measures with high spatial resolution (for example, Moseley & Pulvermüller 2014), or from ERP experiments that build in additional manipulations. Both sources of evidence suggest that the two hemispheres of the brain make different contributions to concreteness effects.

Evidence from functional magnetic resonance imaging (fMRI) and positron emission tomography (PET) studies generally supports the idea that the left and right cerebral hemispheres differ in their processing of concrete and abstract words (for example, Binder et al. 2005; Fiebach & Friederici 2003; Kiehl et al. 1999; Noppeney & Price 2004; Pexman et al. 2007; Sabsevitz et al. 2005). However, specific findings across studies are variable. One source for this variability is differences in stimulus control (Pexman et al. 2007). A challenge for studies of concreteness effects is that the concrete and abstract conditions are generally comprised of different lexical items, which may therefore differ along dimensions other than their concreteness. Compounding this difficulty is the fact that many words in English (and in other languages) are word class and/or semantically ambiguous, and the concreteness of the referent concept may differ across these different usages.

To avoid these difficulties, Huang et al. (2010) adopted a new approach to the study of concreteness effects, by taking advantage of polysemy. Unlike homonyms, which have distinct, unrelated meanings, polysemous words have distinct but related meanings, which may nonetheless differ in their concreteness. For example, the word book can be used to refer to both a concrete physical object (for example, green book) or to its more abstract intellectual content (for example, interesting book). As in these examples, then, it is possible to use adjectives to manipulate the concreteness of the same lexical items.

Huang et al. (2010) showed that concrete, compared to abstract, adjectives elicited typical ERP concreteness effects (Holcomb et al. 1999; Kounios & Holcomb 1994; Lee & Federmeier 2008; West & Holcomb 2000), with larger N400 responses and sustained frontal negativity to the concrete words. Of critical interest, then, was whether concreteness effects would also be observed at the noun—that is, for the same lexical item, as a function of the concreteness of the integrated concept set up by the prior adjective. In order to further assess the neural source of concreteness effects, Huang et al. (2010) combined the concreteness manipulation with the use of visual half-field presentation, in order to bias processing to the contralateral hemisphere. Visual half-field presentation takes advantage of the anatomy of the visual system: items apprehended in the visual periphery make initial contact with the primary visual cortex in the hemisphere contralateral to the visual field of presentation (Banich 2003). The critical nouns of the pairs, therefore, were lateralized to either the left or right visual field (LVF, RVF) to allow an assessment of each hemisphere’s sensitivity to concreteness.

Indeed, there were concreteness effects on the critical nouns, driven by the concreteness of the noun sense established by the adjective, and these concreteness effects were strikingly different in the two hemispheres. When processing is initially biased to the left hemisphere (LH), concretely modified nouns elicit an enhanced frontal P2 response and a smaller N400. The P2 is part of the normal visual evoked response and has typically been linked to the detection and analysis of higher-level visual features, guided by attention (Luck & Hillyard 1994). In sentence processing paradigms, the amplitude of the frontal part of the P2 has been found to be modulated by contextual constraint with LH—but not right hemisphere (RH)—processing: targets in strongly constrained sentence contexts (that is, contexts that elicit strong, consistent predictions for particular endings) elicit more positive P2 responses than those in weakly constrained contexts (Federmeier et al. 2005; Wlotko & Federmeier 2007). Thus, being able to form strong context-based expectations for upcoming words seems to change how the perceptual processing system allocates attention and analyzes subsequent stimuli. In this case, consistent with prior work showing that the LH uses language context information to make predictions about likely upcoming words (for example, Federmeier 2007), concreteness-related effects in the LH patterned with the constraint of the context provided by the adjective. That is, the LH used the more constraining context information provided by the concrete adjectives to anticipate the subsequent nouns and their semantic features, resulting in an enhancement of the frontal P2 response and a reduction in the N400. Note that in this paradigm, then, the typical N400 concreteness pattern was actually reversed, since the adjective context made the semantic features of the upcoming noun more accessible when that context induced a concrete reading of the word. No frontal negativity was found when the nouns were initially presented to the LH. In contrast, when processing was initially biased to the RH, no P2 or N400 effects of concreteness were observed. Instead, nouns preceded by concrete adjectives elicited a sustained frontal negativity between 500 and 900 ms. Thus, the RH seems to play a critical role in eliciting sensory imagery to words.

This study thus provides further evidence for multiple, separable effects of concreteness, mediated by different neural systems distributed across the two cerebral hemispheres. These effects are not confounds arising from the use of different sets of lexical items, but can be seen even on the same word when the concept evoked by that word is more or less concrete. Indeed, the fact that these effects depend on the comprehender’s ability to create an integrated representation of the adjective noun pair was demonstrated in a companion study looking at comprehension in healthy older adults (Huang et al. 2012), who showed young-like concreteness effects at the single word level (on the adjective) but failed to show concreteness effects at the noun, due to impairments with rapidly integrating information across words (see review by Wlotko et al. 2010).

In another series of studies, Huang & Federmeier (2013) have replicated and extended these findings, using phrases consisting of two adjectives followed by a noun. In critical trials, half of the phrases had two abstract adjectives (for example, honorific military pin) and the other half had two concrete adjectives (for example, shiny metallic pin). Either the second adjective or the noun was lateralized, in order to probe for hemispheric processing differences. The results showed that, irrespective of position or part of speech, when stimuli were lateralized to the LH, words preceded by the concrete modifiers elicited enhanced P2 responses, consistent with the use of prediction-based processing mechanisms, whereas when the same stimuli were presented to the RH, concrete words elicited a sustained frontal negativity, highlighting the critical role of the RH for language-induced mental imagery.

3.3 Concreteness effects in adjective ordering

Although the studies just reviewed have examined concreteness differences that arise from the successful integration of multiple words, it is also the case that the concreteness of individual lexical items may affect the ease of that integration process itself. When multiple adjectives are used to modify a noun, they tend to appear in a consistent sequence, and native speakers of a language generally have strong, consistent intuitions about how to order a series of adjectives. For example, an English speaker who wishes to talk about an encyclopedia that is not paperback and not light would call it a heavy hardback encyclopedia and not a hardback heavy encyclopedia. Adjective sequencing has been linked to a number of grammatical and semantic factors (for example, informativeness, definiteness, nouniness, etc.), though none seems to provide a complete account of speakers’ preferences. One additional factor that has not been widely considered is the concreteness of the adjectives being combined. However, concreteness is correlated with adjective ordering, with more abstract adjectives often occurring further from the noun than more concrete ones (for example, lovely shallow pond). Thus, concreteness may be influencing how people preferentially sequence words in phrases involving modified nouns. Moreover, comparisons across normal and dispreferred orders (shallow lovely pond), which have provided some of the key empirical data about adjective ordering, are also likely to involve concreteness differences in the words, unless this is specifically controlled for (which it typically has not been).

To look at the role of concreteness in adjective sequencing, Huang & Federmeier (2012) systematically manipulated adjective order preference and concreteness in phrases consisting of two adjectives and a noun. Some phrases were structured like those in prior studies, in which the preferred order has an abstract adjective followed by a concrete one (exhaustive hardback encyclopedia). However, the study also used phrases in which the preferred order has a more concrete first adjective (heavy informative encyclopedia). Huang & Federmeier (2012) found that for all conditions, except those in which both adjectives were concrete, the processing of second adjectives in dispreferred orders impacted lexico-semantic predictability. That is, preferred orders were associated with facilitated N400 responses compared to dispreferred orders. Thus, by controlling concreteness, they were able to avoid confounds that affected ERP patterns in prior work (for example, Kemmerer et al. 2007) and thereby establish that adjective ordering affects semantic, rather than syntactic, aspects of processing.

In the special case in which both the first and the second adjectives were concrete, the preferred order showed a sustained, frontally-distributed negativity compared to the dispreferred order. The timing and the distribution of this effect is similar to the typical frontal ERP concreteness effect, which has been linked to imagery-based processes (for example, Huang et al. 2010). Because in this condition both adjectives afford imagery, it seems likely that participants would elaborate on the mental image initiated with the first adjective when they apprehend a second concrete adjective. However, when concrete adjectives are encountered in a dispreferred order, participants seem less able to further engage imagery—which, in turn, may be contributing to comprehenders’ intuitions that these phrases are more difficult to understand.

4. Summary and future directions

ERP studies have thus provided key insights into the nature of the processing differences between concrete and abstract words, and, more generally, the characteristics of semantic representation in the brain. In particular, ERPs reveal that concreteness effects are neither just qualitative nor quantitative in nature; they are multifaceted. Concreteness effects arise both from quantitative differences in the amount of long-term memory activation entailed by particular types of words (which, in turn, is dependent on context), and from qualitative differences in the tendency for those words/concepts to elicit mental imagery (which, in turn, is malleable by task demands). The ERP results thus support theories, like the context-extended dual-coding theory, that allow for multiple types of semantic codes, and multiple types of processes involved in accessing those codes.

Moreover, studies that combine ERP measures with visual half-field presentation designs have further revealed that both hemispheres can comprehend language (see Federmeier et al. 2008), but do so differently over time—and that these differences make important contributions to the multi-faceted nature of concreteness effects. The LH is sensitive to the differential contextual constraint provided by concrete and abstract concepts, and the RH enriches the comprehension of concrete language through the engagement of sensory imagery. Such findings compel a more complete inclusion of hemispheric differences in theoretical accounts of concreteness effects.

Given that the two hemispheres differ in how they use the information provided by concrete words, future work might examine their ability to switch between concrete and abstract senses. In addition, individual differences have been reported in lexical/semantic processing (Tanner & Van Hell 2014), but it is not clear whether individual differences also exist in the imagery generation processes that mediate the integration of linguistic input and sensory imagery. Moreover, concreteness effects are common in studies of first (Bergelson & Swingley 2013) and second (De Groot & Keijzer 2000) language learning, both showing that concrete words are acquired earlier than abstract words. However, the underlying mechanisms still remain unclear. Research using ERPs to focus on concreteness effects in language acquisition thus could have an important impact on pedagogy.

Overall, the findings suggest that semantic processing may consist of both modal and amodal components, distributed across multiple neural systems and available in parallel as a function of stimulus properties and task demands. Moreover, the findings emphasize the critical role that time and timing play in semantic processing—and thus the critical data that multidimensional, temporally sensitive measures, such as ERPs, provide for language processing research. Moving forward, the question that arises centers on how the semantic system ensures that diverse, distributed information is available when needed as language comprehension unfolds over time.