The English Lexicon Mirrors Functional Brain Activation for a Sensory Hierarchy Dominated by Vision and Audition: Point-Counterpoint

Abstract

The meanings of most open class words are suffused with sensory and affective features. A word such as beach, for example, evokes polymodal associations ranging from gritty sand (tactile) and crashing waves (auditory) to the distinctive smell of sunscreen (olfactory). Aristotle argued for a hierarchy of the senses where vision and audition eclipse the lesser modalities of odor, taste, and touch. A direct test of Aristotle’s premise was recently made possible with the establishment of the Lancaster Sensorimotor Norms (2019), a crowdsourced database cataloging sensorimotor salience for nearly 40,000 English words. Neurosynth, a metanalytic database of functional magnetic resonance imaging studies, can potentially confirm if Aristotle’s sensory hierarchy is reflected in functional activation within the human brain. We correlated sensory salience of English words as assessed by subjective ratings of vision, audition, olfaction, touch, and gustation (Lancaster Ratings) with volumes of cortical activation for each of these respective sensory modalities (Neurosynth). English word ratings reflected the following sensory hierarchy: vision > audition > haptic > olfaction ≈ gustation. This linguistic hierarchy nearly perfectly correlated with voxel counts of functional activation maps by each sensory modality (Pearson r=.99). These findings are grossly consistent with Aristotle’s hierarchy of the senses. We discuss implications and counterevidence from other natural languages.

INTRODUCTION

Words differ in their sensorimotor and affective salience. For example, apple strongly evokes color imagery, whereas loud sound is the hallmark of thunder. It has long been argued that differences in the perceptual and emotional grounding of object concepts mediate semantic representation in language (Augustine, 390 C.E.; Barsalou, 1999; Gallese & Lakoff, 2005; Locke, 1685). Aristotle (c 350 BCE) held that sensory features vary in their weighted contributions toward knowledge representation. Sensation, Aristotle argued, is subject to a hierarchy with vision and audition eclipsing the lesser senses of taste, touch, and smell. Many western philosophers (e.g., Hume, Locke) have echoed Aristotle’s claims of a sensory hierarchy dominated by vision and audition, and this assumption has since permeated modern cognitive science (Locke, 1685; Bedny & Saxe, 2012; Landau & Gleitman, 1985).

Aristotle’s sense hierarchy was the product of logic, observation, and intuition rather than empirical data. During Aristotle’s time, no natural language was exhaustively characterized with respect to its sensorimotor salience. As such, it was impossible to provide empirical confirmation that any particular language recapitulates a given sensory hierarchy. Such a resource was recently made available with the establishment of the Lancaster Sensorimotor Norms (Lynott et al., 2019).

The Lancaster Sensorimotor Norms represent a database of crowd-sourced scale ratings for the salience of nearly 40,000 English words on numerous sensorimotor variables (Lynott et al., 2019). Beach, for example, has the following feature salience on a 0-5-point scale: visual (4.44), auditory (2.44), gustatory (0.22), haptic (2.22), olfactory (2.88), interoceptive (0.16). Brysbaert and colleagues estimate that the average adult lexicon is composed of approximately 42,000 words (Brysbaert et al., 2016). Therefore, at nearly 40,000 words, the Lancaster Sensorimotor norms approximate the majority of the lexicon of everyday English.

One mode of testing for a sensory hierarchy is through implicit distributional data embedded within language. Such data provide indirect inference about cognition that may be used to reverse engineer hypotheses about neurolinguistic processes. For example, one might infer that a language with an impoverished odor lexicon devalues olfaction in its conceptual system. Such inference would rely on deduction rather data. A more rigorous empirical approach would link language to brain structure and/or function. For example, relative dominance of vision over olfaction would be supported by more extensive patterns of evoked functional brain activation across many different studies. Neurosynth (Yarkoni et al., 2011) is a publicly accessible database offering the potential to yield such inference.

Neurosynth generates metanalytic functional activation maps for user-specified search terms by automatically extracting activation coordinates from published functional neuroimaging papers while also linking each study with a set of keywords that occur with high frequency in the original published manuscripts. Meta-analyses can then be performed comparing the results for studies with and without a given keyword of interest. For example, a query of “visual” will generate a single activation map reflecting aggregated activation for “visual” across all indexed studies in the Neurosynth database (https://neurosynth.org/analyses/terms/visual/).

Study aims: Coupling language and the brain

The Lancaster Sensory Norms offer a comprehensive listing of sensorimotor salience within the English lexicon, whereas Neurosynth offers a way to evaluate functionally defined brain regions. Linking these resources can potentially inform whether a sensory hierarchy in the human brain is mirrored within English. Our aims here are threefold:

1. We will test whether the lexicon of English reflects Aristotle’s sensory hierarchy (e.g., vision > audition) via the Lancaster norms.

2. We will test whether the extent of functional brain activation for each sensory modality reflects Aristotle’s sensory hierarchy (e.g., vision > audition) via Neurosynth.

3. We will examine the bivariate correlation between sensory salience of English words (Lancaster) relative to functional activation (Neurosynth).

METHODS

For the lexical analysis we imported all entries from the Lancaster Sensorimotor Norms into R statistical software (version 3.5.2) and derived grand means for Vision, Audition, Olfaction, Gustation, and Somatosensation.

For the functional activation analysis, we first exported functional brain activation maps from Neurosynth, thresholding for multiple comparisons using a voxelwise False Discovery Rate (FDR) of 0.01. We used maps generated for association tests, which evaluate for the presence of a non-zero association between search term use (e.g., visual) and activation of a particular voxel (i.e., whether activation occurs more consistently for studies that are linked with a particular term than for studies that are not, effectively controlling for base-rate differences across regions). We obtained analytic maps for each sensory modality by matching category terms from the Lancaster Sensorimotor Norms. The search term “haptic” returned no results, so we used “tactile”. Similarly, “smell” returned no results, so we used “olfactory”. We queried the following search terms:Visual (N=3,110 studies), Auditory (N=1,252 studies), Tactile (N=225 studies), Taste (N=80 studies), and Olfactory (N=74 studies). We obtained voxel counts for each functional activation map by summing all positively-signed FDR-corrected voxels. Python scripts for obtaining Neurosynth images, and the images themselves, can be found at https://osf.io/92ry5/.

For the language-brain cross-correlation, we conducted Pearson correlations of the grand means of each sensory modality from the Lancaster norms against the respective voxel counts of their functional activation from Neurosynth.

Results

Figure 1a shows differences in the distributions of sensory salience across the 39,708 words of the Lancaster database. The relative ranking across sensory modalities on a 0-5 Likert scale was: vision (μ =2.90) > audition (μ=1.51) > somatosensory (μ=1.07) > olfactory (μ =0.39) = gustatory (μ =0.34). ¹

Figure 1.

Sensory-Brain and Sensory-Language Correlations

The rank volume(s) of functional activation for each sensory modality were as follows: Visual (25,891 voxels), Auditory (11,242 voxels), Tactile (8,271 voxels), Taste (3,159 voxels), and Olfactory (2,395 voxels).

Figure 1c represents a near perfect correlation (Pearson r=.99, p<.0001) between English word norms (Lancaster) and functional activation maps (Neurosynth).

An unexpected finding was that the bivariate correlation between the number of studies constituting each modality and their respective voxel counts was also near perfect (Pearson R=.98, p<.0001). We assessed whether this potential source of bias (i.e., more studies = more activated voxels) was a pervasive artifact in functional neuroimaging by randomly sampling 60 unique terms from the Neurosynth database (e.g., attention, risk, hyperactivity), and correlating their respective voxel counts with the number of referenced studies comprising each activation map. This correlation between activation and number of studies was negligible (Pearson r=. 14, p=.30). R scripts for this analysis can be found at https://osf.io/92ry5/.

General Discussion

A near perfect correlation exists between functional voxel counts and sensorimotor ratings of English words. Brain regions that process olfaction and gustatory detail elicit less extensive functional activation patterns relative to vision and audition. These relative differences in activation volumes across sensory modalities are strongly correlated with lexical diversity in English. That is, more visual activation across functional imaging studies is correlated with a more extensive visual lexicon in English. In contrast, olfaction tends to elicit much smaller swathes of functional activation, and English has a correspondingly sparse odor lexicon. An extreme interpretation of this correlation is that the Aristotelian functional sensory hierarchy reflected within the human brain is recapitulated within one of the world’s most commonly spoken natural languages. This language recapitulation perspective is, however, open to numerous alternative hypotheses.

Foremost, any causal claim about language recapitulates the human brain is premised on the temporal assumption that the brain preceded language. More specifically, patterns of human brain activation for the respective senses preceded the ontogeny of the English lexicon. With respect to brain morphology, this evolutionary assumption seems self-evident. However, the argument becomes much more nuanced when considering brain function. Patterns of brain activation tend to show significant variability across cultures and languages, even differing within the same individual (e.g., bilingualism) for stimuli probed in different languages (Gutchess et al., 2006). Deacon (1998) argued that such neuroplasticity imposes constraints on the evolution of language and brain. That is, natural language is not simply a recapitulation of brain structure. Rather, the human brain has experienced punctuated co-evolution in tandem with language. According to Deacon (1998), many recent evolutionary changes in human brain structure were driven by language-brain feedback mechanisms conferred by the adaptive advantages of linguistic symbol systems. Thus, language has conformed to the unique constraints of a changing brain.

Similarly, the human brain is actively adapting to unprecedented opportunities afforded to our species by language. A strong test of the language recapitulation hypothesis would involve confirming that Aristotle’s sensory hierarchy is a linguistic and neural human universal. Alternatively, the observed associations could be idiosyncratic to English. At the neural level, it is not currently possible to evaluate this claim. Within the domain of natural language, the Lancaster Sensorimotor norms also lack close parallels in scope between other languages (but see Chen et al., 2019; Speed & Majid, 2017). Therefore, it is not currently possible to conduct direct cross-linguistic comparisons using identical methodology.² Nevertheless, several recent studies using more indirect assays of word usage across languages suggest that the traditional Aristotelean sensory hierarchy of visual and auditory dominance is in fact a phenomenon that is unique to English (Majid et al., 2018; Speed & Majid, 2019).

Majid and colleagues (2018) conducted the most extensive cross-linguistic investigation to date examining the ways that a diverse range of natural languages weigh different senses within their lexicons. It has previously been argued that English is effectively ‘mute’ in its capacity to describe odors and tastes (Stevenson et al., 2007). Odors are notoriously difficult for English speakers to name and identify (Stevenson et al., 2007). Moreover, the English odor lexicon is sparse. Our repertoire for describing odors and tastes is limited to vague descriptions (e.g., that smells/tastes bad) and similes (e.g., that smells like a sewer). An impoverished odor and taste lexicon is consistent with Aristotle’s sensory hierarchy. This conclusion is, however, upended when examining the lexicons of many non-Western languages including Turkish, Cantonese, Farsi, and Malay, all of which have richly developed taste and odor lexicons—some rivalling that of vision (Majid et al., 2018). This distributional linguistic evidence suggests that the Aristotelian sensory hierarchy is not a human universal.

Future Directions & Concluding Remarks

Caution is warranted in interpreting the correlations we observed within and between the Neurosynth and Lancaster norms. Although both databases reflect many observations, the correlations we reported involve only five data points, one for each sensory modality. The overwhelming strength of such correlations could be driven by an outlier such as vision which tends to have much higher lexical salience and diffuse brain activation relative to any other modality. Despite these caveats, it seems clear that the Lancaster Sensorimotor norms offer a significant advance toward elucidating modality-specificity within one of the world’s most commonly spoken natural languages. Neurosynth affords a parallel evaluation of the relative strength of the primary senses as indexed by functional activation. The linguistic and neuroimaging data generated by these different resources nearly perfectly aligned. If one were to draw a conclusion based solely on the strength of this single correlation, it would be that Aristotle was correct. However, a wider lens incorporating higher salience for olfaction and gustation in non-Western languages casts doubt upon the universality of this claim.

Strong alignment between brain activation and sensory salience in English suggests two possibilities. First, language could recapitulate an evolutionary sensory hierarchy embedded within the functional architecture of the human brain. Alternatively, natural language and brain function could mirror one another through co-evolution. The strongest confirmatory test of the recapitulation hypothesis would involve determining whether a universal hierarchy of the senses is present in the human brain across cultures. Alternatively, the sensory salience of a particular language could flexibly mirror the neural constraints shared by a particular culture. For example, speakers of a language such as Malay with a strongly developed odor lexicon might show more extensive patterns of functional activation for olfaction than English speakers. A lack of cross-linguistic neuroimaging data currently precludes such a test. Yet, a wealth of natural language data are more consistent with the premise of flexible sensory hierarchies.

Aristotle proposed a hierarchy of the senses such that vision and audition eclipse smell, taste, and touch.

We examined whether this Aristotelian hierarchy of the senses is reflected in natural language and in patterns of brain activation.

We correlated sensorimotor salience of many English words with the volumes of functional activation elicited by studies of vision, audition, smell, taste, odor, and touch.

The Aristotelian hierarchy was recapitulated both in language and in functional brain activation.