Neural specialization for ‘visual’ concepts emerges in the absence of vision

Abstract

The ‘different-body/different-concepts hypothesis’ central to some embodiment theories proposes that the sensory capacities of our bodies shape the cognitive and neural basis of our concepts. We tested this hypothesis by comparing behavioral semantic similarity judgments and neural signatures (fMRI) of ‘visual’ categories (‘living things,’ or animals, e.g., tiger, and light events, e.g., sparkle) across congenitally blind (n=21) and sighted (n=22) adults. Words referring to ‘visual’ entities/nouns and events/verbs (animals and light events) were compared to less vision-dependent categories from the same grammatical class (i.e., animal vs. place nouns, light vs. sound, mouth, and hand verbs). Within-category semantic similarity judgments about animals (e.g., sparrow vs. finch) were partially different across groups, consistent with the idea that sighted people rely on visually learned information to make such judgments about animals. However, robust neural specialization for living things in temporoparietal semantic networks, including in the precuneus, was observed in blind and sighted people alike. For light events, which are directly accessible only through vision, behavioral judgments were indistinguishable across groups. Neural responses to light events were also similar across groups: in both blind and sighted people, the left middle temporal gyrus (LMTG+) responded more to event concepts, including light events, compared to entities. Multivariate patterns of neural activity in LMTG+ distinguished among different event types, including light events vs. other event types. In sum, we find that neural signatures of concepts previously attributed to visual experience do not require vision. Across a wide range of semantic types, conceptual representations develop independent of sensory experience.

Quotations

“Critics delight to tell us what we cannot do. They assume that blindness and deafness sever us completely from the things which the seeing and the hearing enjoy, and hence they assert we have no moral right to talk about beauty, the skies, mountains, the song of birds, and colours. They declare that the very sensations we have from the sense of touch are “vicarious,” as though our friends felt the sun for us! They deny a priori what they have not seen and I have felt. Some brave doubters have gone so far even as to deny my existence. In order, therefore, that I may know that I exist, I resort to Descartes’s method: “I think, therefore I am.” Thus I am metaphysically established, and I throw upon the doubters the burden of proving my non-existence.” – Helen Keller (1908)

“The properties of an organism’s body limit or constrain the concepts an organism can acquire. That is, the concepts by which an organism understands its environment depend on the nature of its body in such a way that differently embodied organisms would understand their environments differently.” – Stanford Encyclopedia of Philosophy (Shapiro & Spaulding, 2021).

Introduction

Within the first weeks of life, infants identify living things by looking for faces, bodies, and biological motion (Simion et al., 2008; Opfer & Gelman, 2011; Spelke, 2022). Watching animals such as elephants and blue jays offers information about their shape, color, texture, and behavior (e.g., De Vries, 1969; Massey & Gelman, 1988; Setoh et al., 2013). Some conceptual categories, like light events (e.g., ‘sparkle’, ‘glow’) and colors (e.g., ‘blue’, ‘yellow’), can only be directly experienced through vision. Vision is an important source of direct sensory evidence about many conceptual categories. Here we ask how visual experience contributes to the cognitive and neural basis of concepts.

Many embodied accounts of cognition propose that the sensory capacities of our bodies constrain the concepts we can entertain (e.g., Barsalou, 1999; Lakoff & Johnson, 1980, 1999; Tucker & Ellis, 1998; Glenberg & Kaschak, 2002; Thompson-Schill et al., 1999; Casasanto, 2009; Shapiro & Spaulding, 2021; see also Locke, 1690; Hume, 1739; Berkeley, 1732). Laurence & Margolis (2024) dub this the ‘different-body/different-concepts hypothesis.’ Neurally, such theories propose that concepts are represented in the modality-specific sensory systems though which they were acquired. Consistent with this idea, a large neuroimaging literature suggests that conceptual retrieval activates sensory regions of the brain (Thompson-Schill, 2003; Barsalou, 2008; Meyer & Damasio, 2009; Pulvermüller & Fadiga, 2010; Kiefer & Pulvermüller, 2012; Meteyard et al., 2012; Yee & Thompson-Schill, 2016; Martin, 2016; Reilly et al., 2024; Tyler & Moss, 2001; Barsalou et al., 2003; Barsalou, 2010; Pulvermüller, 2001; Gallese & Lakoff, 2005; Zwaan, 2014). Color words activate visual color areas, motor words motor control regions, and auditory words auditory cortices (Martin et al., 1995; Chao & Martin, 1999; Hauk et al., 2004; Wallentin et al., 2005; Simmons, Martin, & Barsalou, 2005; Beilock et al., 2008; Hoenig et al., 2011; Fernandino et al., 2016; Simmons et al., 2007; Halpern et al., 2004; Kiefer et al., 2008; Kemmerer et al., 2008; Kuhnke, Kiefer, & Hartwigsen, 2020). One interpretation of these data is that the sensory means through which concepts are acquired shapes their cognitive and neural basis.

Sensorimotor experiences can differ widely across people. One question is whether such differences lead to different conceptual representations. Unlike experts in music or a particular sport, whose sensory experience differs from that of non-experts in nuanced ways, congenitally blind individuals lack visual experience entirely. Even so, behavioral studies find shared use of ‘visual’ words across blind and sighted people (Marmor, 1978; Zimler & Keenan, 1983; Landau & Gleitman, 1985; Shepard & Cooper, 1992; Connolly et al., 2007; Lenci et al., 2013; Saysani et al., 2018; Kim et al., 2019; Bedny et al., 2019; Wang et al., 2020; Kim et al., 2021). For instance, Landau & Gleitman (1985) found that a congenitally blind child, Kelli, understood and produced verbs like ‘look’ and ‘see’ as well as color words (e.g., ‘green’) around the same age as sighted children do. Congenitally blind adults make subtle distinctions among verbs that refer to light events based on light intensity and periodicity (e.g., ‘sparkle’ vs. ‘flash’; Lenci et al., 2013; Bedny et al., 2019). Blind and sighted people share knowledge of large animal appearance (e.g., what is the shape and size of a tiger?), despite the fact that direct sensory access to large animals is primarily visual (Kim et al., 2019). Blind and sighted people have similar intuitions about how color varies across object tokens (e.g., two pieces of paper are more likely to have the same color than two cars) (Kim et al., 2021) and about the similarity space of colors (e.g., orange is more similar to red than to green) (Shepard & Cooper, 1992; Marmor, 1978; Saysani et al., 2018). Thus, seemingly visual information is acquired by humans who do not have direct sensory access to it.

People born blind could acquire ‘visual’ knowledge in a variety of ways, including by analogy to other senses (e.g., touch, audition), but humans are prodigious social learners, and learning through language likely makes a significant contribution to visual knowledge in blindness. Indeed, languages of the world convey rich information about the senses. English has a large ‘visual’ lexicon (Viberg, 1983; Sweetser, 1990; Levinson & Majid, 2014; San Roque et al., 2015; Winter et al., 2018). Recently, large language models (LLMs) have aptly demonstrated that semantic representations of ‘sensory’ information can be acquired via language alone (Abdou et al., 2021; Patel & Pavlick, 2022; Li et al, 2021; Wei et al., 2022; Sharma et al., 2024; Gurnee & Tegmark, 2023; Marjieh et al., 2022, 2024). For example, LLMs can reconstruct the similarity space of colors (red is more similar to orange than to blue), the spatial locations of US states on a map, and object shapes (Abdou et al., 2021; Gurnee & Tegmark, 2024; Sharma et al., 2024; Marjieh et al., 2024).

Exactly how much humans learn about vision from language and what kinds of representations are acquired from language remain open questions. Compared to LLMs, humans have more modest memory resources and access to far less linguistic data (Warstadt & Bowman, 2022; Frank, 2023). It has also been suggested that precisely because LLMs learn from language alone, their representations are shallow (Lake & Murphy, 2023; Bender & Koller, 2020; but see Chalmers, 2023). Some differences in visual knowledge have also been observed across blind and sighted people in prior behavioral studies. Although color similarity judgments (e.g., orange vs. blue) are similar across blind and sighted people on average, individual blind people’s judgments are more variable (Shepard & Cooper, 1992; Marmor, 1978; Saysani et al., 2018). Blind individuals rate large animals (e.g., tiger, rhinoceros) as less familiar, and animal appearance knowledge differs somewhat across blind and sighted groups (Kim et al., 2019). Blind and sighted people show low agreement about the colors of animals and common objects, including plants. For example, in one study, 100% of sighted people and about 50% of blind participants labeled carrots as orange (Kim et al., 2021). Likewise, semantic similarity judgments about fruits and vegetables are influenced by color in sighted but not blind participants (Connolly et al., 2007). In sum, the available behavioral evidence suggests that sighted people and people born blind share ‘visual’ knowledge, but this knowledge is not identical across groups. In particular, blind and sighted people disagree about some aspects of the appearance of living things.

Given that differences in behavior across groups are relatively subtle, one interpretation of these results is that direct sensory access is not as central to conceptual representation as predicted by the ‘different-body/different-concepts’ hypothesis (Mahon et al., 2009; Mahon & Caramazza, 2011; Vannuscorps & Caramazza, 2016; Bedny & Saxe, 2016; Bedny et al., 2019; Bedny, 2020). Alternatively, it is possible that subtle behavioral differences between blind and sighted people reveal more fundamental changes in the format of their ‘visual’ conceptual representations (e.g., Connolly et al., 2007; Yee, Chrysikou, & Thompson-Schill, 2013; Yee, Jones, & McRae, 2017). Neuroscience evidence can help distinguish between these interpretations.

1.1. Neural evidence regarding the relationship of concepts and sensory experience

Neuroscience studies have provided some of the strongest evidence for the idea that concepts are embodied in sensorimotor systems. Classic neuropsychological work proposes that semantic deficits for living things arise as a result of damaged visual knowledge (Allport, 1985; Warrington & Shallice, 1984; Warrington & McCarthy, 1987; Farah & McClelland, 1991; Gaffan & Heywood, 1993; Moss et al., 1997; Tranel et al., 1997; Humphreys & Forde, 2001; cf. Caramazza & Shelton, 1998; Caramazza & Mahon, 2003). fMRI studies find that thinking about living things (e.g., animals) activates distinctive neural structures, and such findings have been attributed to the retrieval of visual knowledge central to living things concepts (Perani et al., 1999; Martin et al., 1996; Okada et al., 2000; Thompson-Schill et al., 1999).

Analogously, parts of the posterior portion of the left middle temporal gyrus (LMTG+) respond preferentially to action verbs over concrete nouns (Kable et al., 2002, 2005; Davis et al., 2004; Bedny et al., 2008, 2014; Martin et al., 1995; Bedny & Thompson-Schill, 2006; Yu et al., 2012; Lapinskaya et al., 2016; Elli et al., 2019), which has been attributed to the importance of visual motion information for action verb representations (Kable et al., 2002, 2005; Tranel et al., 2003; Kemmerer et al., 2008; Noppeney, 2008; Pulvermüller & Fadiga, 2010; Kemmerer & Gonzalez-Castillo, 2010; Damasio & Tranel, 1993). In the current study, we tested whether these neural signatures of vision-dependent concepts differ across congenitally blind and sighted people. The emergence of group differences would provide support for the idea that the sensory capacities of our bodies shape our conceptual representations. Alternatively, it is possible that neural signatures previously attributed to the visual dependence of concepts develop similarly in people born blind. This finding would provide evidence for the body-independence of concepts.

1.2. The current study: Comparing the neural basis of living things and light events across congenitally blind and sighted people

We compare the neural basis of two visual categories, living things and light events, across congenitally blind and sighted people. Vision is thought to provide a key source of information about these categories. Living things and light events also span a wide range of semantic types, from concrete entities to events, and thus together offer a broad perspective on the contribution of vision to the neural instantiation of concepts. Finally, as noted above, living things and light events are associated with distinctive and consistent neural signatures in sighted people. In other words, both categories dissociate neurally from other categories of concepts (i.e., activate distinctive regions of the brain and/or produce distinctive neural patterns of activity as measured by multivariate methods). One hypothesis is that such dissociations arise because vision plays a privileged role in the acquisition of these concepts (e.g., Warrington & Shallice, 1984; Tranel et al., 2003; cf. Caramazza, 1998). If so, we would expect to find some or all of these neural dissociations to be absent, weakened, or different (e.g., in neural location) in people born blind. Alternatively, if the same neural signatures are observed in congenitally blind people, this finding would provide strong support for the idea that visual experience is not central to their acquisition.

1.3.1. Neural responses to living things concepts in sighted people and predicted responses in blind people

In sighted people, words referring to living things elicit distinctive neural responses in temporoparietal semantic brain networks, particularly in the precuneus (PC; Fairhall & Caramazza, 2013a, 2013b; Fairhall et al., 2014; Peer et al., 2015; Wang et al., 2016; Silson et al., 2019; Rabini, Ubaldi, & Fairhall, 2021; Deen & Freiwald, 2022; Aglinskas & Fairhall, 2023). Responses to living things in the precuneus are elicited by words and images alike (Fairhall & Caramazza, 2013a; Fairhall et al., 2014; Kanwisher et al., 1997; Grill-Spector et al., 2004; Konkle & Caramazza, 2013; Connolly et al., 2016; Devlin et al., 2002; Noppeney et al., 2006; Mahon et al., 2009). This distinguishes the precuneus from lateral ventral occipito-temporal cortex (VOTC), which responds to images of living things but not to words referring to them (see Bi et al., 2016 for a review). Moreover, in sighted people, classifiers trained on patterns of neural activity in the precuneus generalize across images of living things and words referring to living things (Fairhall & Caramazza, 2013a), making these responses a good test case for comparing living things representations across blind and sighted people.

1.3.2. Neural responses to light event concepts in sighted people and predicted responses in blind people

The second ‘visual’ category examined in the current study is light events (e.g., ‘sparkle’). Unlike animal nouns or motion verbs (e.g., ‘clap’, ‘hit’), light events are only directly accessible through vision. Light events are situated in time and encoded in most languages, including English, by verbs (Talmy, 1975; Langacker, 1987; Frawley, 1992). Previous studies have found that event words elicit neural responses in the LMTG+ relative to words describing objects and properties (e.g., Kable et al., 2002, 2005; Davis et al., 2004; Bedny & Thompson-Schill, 2006). Whether the LMTG+ encodes modality-specific or modality-invariant conceptual representations has been debated. Early studies attributed responses to motion verbs in the LMTG+ to the retrieval of visual motion information (e.g., Damasio & Tranel, 1993; Martin et al., 1995; Kable et al., 2002; 2005). Subsequent work showed that people born blind also activate this region during motion verb comprehension, suggesting that it supports modality invariant representations (Noppeney et al., 2003; Bedny et al., 2012; Bottini et al., 2020). However, an alternative interpretation is that visual motion information represented in this region in sighted people is replaced by auditory or sensorimotor motion information in people born blind (e.g., the visual image of bouncing is replaced by the sound of bouncing) (Yee, Chrysikou, & Thompson-Schill, 2013; Yee, Jones, & McRae, 2017; see also Striem-Amit et al., 2018; Bi, 2021; Kiefer, Kuhnke, & Hartwigsen, 2023; Campbell & Bergelson, 2022). Relative to motion verbs, light verbs provide a stronger test of the contribution of visual experience to conceptual representation because they are directly accessible only through vision. If LMTG+ responses to light events are observed in congenitally blind people, this would suggest that such representations develop equivalently regardless of whether they are learned via direct sensory access.

1.3.4. Current experimental design

We compared ‘visual’ categories to multiple non-visual categories within the same general semantic class (entities/nouns vs. events/verbs). Among entity concepts, we compared ‘more visual’ living things (birds and mammals) to ‘less visual’ non-living things (manmade and natural places) (e.g., Warrington & Shallice, 1984; Warrington & McCarthy, 1987). Among event concepts, we compared visual light emission events (e.g., ‘sparkle’, ‘glow’) to non-visual events, including sound emission events (e.g., ‘beep’, ‘squeak’), hand actions (e.g., ‘prod’, ‘stroke’), and mouth actions (e.g., ‘slurp’, ‘lick’). Participants heard pairs of words from the same semantic category (e.g., ‘to sparkle, to glow’ or ‘the robin, the owl’) and judged their semantic similarity on a scale of 1 to 4. Pairing words within semantic categories ensured that participants could make detailed semantic judgments. We used individual-subject fMRI analysis (Fedorenko et al., 2010; Nieto-Castañón & Fedorenko, 2012) to identify previously established neural networks responsive to entities/nouns and events/verbs in each participant. Within these networks, we compared neural responses to ‘visual’ and non-visual concepts using univariate and multivariate approaches.

Methods

2.1. Participants

Twenty-one congenitally blind adults (13 females, 8 males; age range 18-67 years, M = 39.14 ± 13.81 SD) and twenty-two sighted age- and education-matched controls (16 females, 6 males; age range: 19-62 years, M = 37.55 ± 13.25 SD) participated in the study (Supplementary Table 1). Blind participants lost their sight due to pathologies of the eyes or optic nerve anterior to the optic chiasm (i.e., not due to brain damage), and had at most minimal light perception since birth. Throughout the experiment, all participants (sighted and blind) wore a light exclusion blindfold to match their visual input. Sighted and blind participants were screened for cognitive and neurological disabilities (self-report). Participants gave written informed consent and were compensated $30 per hour. The study was reviewed and approved by the Johns Hopkins Medicine Institutional Review Boards. Four additional blind participants were scanned but excluded from the final sample because they were older than 70 years of age (n=2), they were not blind since birth (n=1), or they gave similarity judgments different from those of the group (n=1, correlation with the group lower than 2.5 SDs from the average for both verbs and nouns).

2.2. Stimuli and procedure

While undergoing functional magnetic resonance imaging (fMRI), participants heard pairs of words and judged how similar the two words were in meaning on a scale from 1 (not at all similar) to 4 (very similar), indicating their responses via button press. Word stimuli fell into 1 of 2 grammatical classes (entities/nouns, events/verbs), facilitating our investigation of ‘visual’ categories spanning both classes (i.e., animal nouns, light verbs). Within these classes, words were further divided into 4 categories (entities/nouns: birds, e.g., ‘the crow’; mammals, e.g., ‘the fox’; manmade places, e.g., ‘the barn’; natural places, e.g., ‘the swamp’; events/verbs: light emission, e.g., ‘to sparkle’; sound emission, e.g., ‘to squeak’; hand-related actions, e.g., ‘to pluck’, mouth-related actions, e.g., ‘to bite’) (Figure 1, Table 1, see Appendix 1 for full list of stimuli). These categories captured a wide range of semantic categories within each grammatical class, including categories for which visual information is thought to play a comparatively less important role (e.g., places, sound emission). Words were matched across several variables, including number of syllables and familiarity (see Elli et al., 2019 and Appendix 2 for details). Word pairs were presented in blocks of 4 and were grouped by semantic category within blocks. Each word appeared once within a block. Blocks were 16 s long and were separated by 10 s of rest. The experiment included a total of 144 blocks evenly divided into 8 runs.

Figure 1:

In-scanner semantic similarity judgments across sighted and blind participants. (A) Item-wise correlations (Spearman’s rho ρ) between blind and sighted average group ratings. Confidence intervals (95%) are indicated via shading. (B) Leave-one-out within-group correlations (Spearman’s ρ). Error bars: ± standard error of the mean.

Table 1:

Example stimuli from each category.

Entities / Nouns	Animals	Birds	the crow – the dove	the goose – the owl
		Mammals	the fox – the lion	the giraffe – the hippo
	Places	Manmade	the barn – the garage	the shrine – the temple
		Natural	the swamp – the bay	the canyon – the crater
Events / Verbs	Actions	Hand	to prod – to pluck	to stroke – to pummel
		Mouth	to gnaw – to bite	to slurp – to lick
	Emissions	Light	to glow – to sparkle	to shine – to flash
		Sound	to beep – to ring	to squeak – to bang

Our experimental design enabled us to perform multivariate analysis (MVPA) of neural responses to each category. Whereas univariate analysis measures the magnitude of neural activity corresponding to different experimental conditions, multivariate analysis measures the distinctiveness of patterns of activity across conditions, offering a more sensitive approach. Because multivariate analysis can capture more subtle differences in representational content, it is well suited to address cognitive questions. In the current study, we use multivariate analysis (linear classification) to ask whether entity/noun categories and event/verb categories are differentially represented in blind and sighted participants’ brains. To facilitate such analysis, we created two non-overlapping subsets of words that were exclusively presented in either even or odd runs. This enabled us to train linear classifiers on neural responses to one set of words and test the classifiers on neural responses to a different set of words, ensuring that any above-chance classification effects reflect differences in the neural patterns associated with semantic categories and not word forms. Words in each semantic category were divided into two non-overlapping sets of 9 words. Within each such set, we created all possible pairs within a category (e.g., ‘the seagull, the parrot’, 36 pairs per set per category). There were no cross-category pairs.

2.3. Behavioral data analysis

Due to a response box malfunction, 19/21 blind and 19/22 sighted participants contributed to behavioral data analysis. In-scanner similarity judgments were first standardized (z-scored to mean=0 ± 1SD) within each participant to account for individual differences in Likert scale use, and then standardized within grammatical class (i.e., events/verbs, entities/nouns) to a [0,1] range (i.e., $x=\frac{x-x_{min}}{x_{max}-x_{min}}$) within each participant. To assess agreement in semantic judgments within and across blind and sighted groups, we correlated item-wise ratings within each semantic category using Spearman’s rho (ρ) rank correlations. This analysis asks whether blind and sighted participants agree regarding which pairs within a semantic category are most similar in meaning (see Appendix 3 for details).

2.4. fMRI data acquisition and preprocessing

MRI structural and functional data of the whole brain were collected using a 3 Tesla Phillips scanner with a 32-channel head coil. We collected T1-weighted 3D-MPRAGE structural images using a pulse sequence in 170 sagittal slices with 1mm isotropic voxels (TE/TR=7.0/3.2ms, FoV=240x240 mm, 288x272 acquisition matrix, scan duration=5:59’). We collected T2*-functional BOLD images using parallel transverse ascending echo planar imaging (EPI) sequences in 36 axial slices with 2.5 x 2.5 x 2.5 mm voxels (TE/TR=30/2000ms, FoV=192x172mm, 76x66 acquisition matrix, 0.5mm gap, flip angle=70º, scan duration=8:04’).

Data were analyzed using FSL, Freesurfer, the Human Connectome Project workbench, and custom in-house software written in Python (Dale, Fischl, & Sereno, 1999; Smith et al., 2004; Glasser et al., 2013). Functional data were motion corrected using FSL’s MCFLIRT algorithm (Jenkinson et al., 2002), high pass filtered to remove signal fluctuations at frequencies longer than 128 seconds/cycle, mapped to the cortical surface using Freesurfer, spatially smoothed on the cortical surface (6mm FWHM Gaussian kernel), and prewhitened to remove temporal autocorrelation. Covariates of no interest were included to account for confounds related to white matter, cerebral spinal fluid, and motion spikes.

2.5. fMRI data analysis

2.5.1. Univariate analysis

Univariate analyses were used to test whether regions previously associated with animal nouns (i.e., the precuneus) and light verbs (i.e., the LMTG+) exhibit characteristic category-specific responses in the absence of visual experience. Each of the entity/noun and event/verb categories was entered as a separate predictor in a general linear model (GLM) after convolving with a canonical hemodynamic response function and its first temporal derivative. Each run was modeled separately, and runs were combined within-subject using a fixed-effects model (Dale et al., 1999; Smith et al., 2004). Group-level random-effects analyses were corrected for multiple comparisons across the whole cortex at p < .05 family-wise error rate (FWER) using a nonparametric permutation test (cluster-forming threshold p < .01 uncorrected) (Winkler et al., 2014; Eklund, Nichols, & Knutsson, 2016; Eklund, Knutsson, & Nichols, 2019).

2.5.2. ROI definition

We defined regions of interest in each participant to enable individual-subject analyses of responses to ‘visual’ vs. ‘non-visual’ categories. Regions of interest were defined in cortical areas (search spaces) previously shown to respond to entity and event concepts in sighted people (see Crepaldi et al., 2013, for a review). Within these noun and verb responsive areas, we compared responses to ‘visual’ concepts across blind and sighted people. These areas also responded more to nouns vs. verbs or vice versa in whole-cortex analysis (p < .05 uncorrected) in the current study. We defined 4 entity/noun-preferring search spaces: left precuneus (LPC), left inferior parietal lobule (LIP), left lateral inferior temporal cortex (LlatIT), and left medial ventral temporal cortex (LmedVT); and 1 event/verb-preferring search-space: left middle temporal gyrus/inferior parietal cortex (LMTG+) (Supplementary Figure 5). Within these search spaces, we defined individual subjects’ functional ROIs for each participant.

Although the left inferior frontal gyrus also responded more to events than entities in the current study, we previously found that it showed weak and category-invariant decoding in sighted adults (Elli et al., 2019); therefore, we did not use this ROI.

Search spaces were first defined in the blind and sighted groups separately and then combined across groups, such that each search space (e.g., blind LPC + sighted LPC) included all the voxels responding more to events or entities in either group. This procedure is inclusive to avoid omitting above-threshold activation in either of the groups. Next, we defined individual-subject ROIs within each search space by selecting every participant’s top 300 active vertices for the events/verbs>entities/nouns (verb ROI) or entities/nouns>events/verbs (noun ROIs) contrasts (see Appendix 4 for details).

Following past work demonstrating occipital activation during language processing in blind individuals (Röder et al., 2002; Amedi et al., 2003; Bedny et al., 2011, 2012, 2015; Lane et al., 2015), we additionally defined two ROIs in occipital cortex in each participant: left and right V1-V2 (BA17-18) from the PALS-B12 Brodmann area atlas included in FreeSurfer (Van Essen, 2005).

2.5.3. MVPA ROI analysis

We used MVPA (PyMVPA toolbox; Hanke et al., 2009) to assess the extent to which patterns of activity in entity- and event-responsive ROIs distinguish between entity categories and between event categories.

For each ROI in each participant, we trained a linear support vector machine (SVM) classifier to separately decode among the 4 event categories and the 4 entity categories (chance 25%). We submitted to this analysis the z-scored beta parameter of the GLM associated with each vertex for each semantic category in each run (2 grammatical classes * 4 categories per class = 8 total observations per vertex per run) (see Appendix 5 for details). Within each of the entity- and event-responsive ROIs, we used one-tailed Student’s t-tests to test the classifier’s accuracy against chance (25%), and two-tailed independent samples Student’s t-test to compare the accuracy for events and entities. We used repeated measures ANOVAs to test for interactions between groups, ROIs, and grammatical class (entities/events). We evaluated significance using a combined permutation and bootstrapping approach (Schreiber & Krekelberg, 2013; Stelzer, Chen, & Turner, 2013) (see Appendix 5 for details). The same approach was used to assess the statistical significance of decoding accuracies for entity categories and event categories within the two occipital ROIs.

Next, to evaluate how well the classifier performed on pairwise distinctions among entities (e.g., birds vs. mammals) and among events, we inspected the confusion matrices generated by the classifier. The confusion matrices contain the classification and misclassification frequencies for any pair of categories, which can be compared using a signal detection theory framework (Swets, Tanners, & Birdsall, 1961; Green & Swets, 1966; Haxby, Connolly, & Guntupalli, 2014). We assessed the discriminability between 1) animals vs. places within entity-responsive ROIs and 2) light events vs. all other event categories in the LMTG+ by computing the nonparametric estimate of discriminability (Pollack & Norman, 1964; Grier, 1971; Stanislaw & Todorov, 1999). An A′ of 0.5 corresponds to chance performance, whereas an A′ of 1 indicates perfect discriminability. Because A′ values did not follow a normal distribution, we used one-sample Wilcoxon signed rank tests to compare A′ values to chance performance, and a repeated measures permutation ANOVA (5,000 permutations) using the permuco package in R (Frossard & Renaud, 2021) to test for interactions between groups, ROIs, and classification error type in entity-responsive brain regions. Wilcoxon signed rank tests use the test statistic V, which represents the sum of the positive ranks, or the distance of all observed values greater than the chance-level from the chance-level.

Results

3.1. Behavioral results

Between-group agreement

Semantic similarity judgments made by blind and sighted people were significantly correlated across groups for every semantic category. Some categories were more similar across groups than others (Figure 1A): between-group similarity was highest for mouth events (ρ=0.93), and lowest for birds (ρ=0.6) and mammals (ρ=0.68). Between-group similarity was lower for animal nouns (bird, mammal) than for place nouns (manmade, natural) (animal nouns: ρ=0.7, 95% CI = [0.61, 0.78]; place nouns: ρ=0.86, 95% CI = [0.81, 0.9]). Light events, the only purely visual category, showed similar agreement across groups compared to other event/verb types (light events: ρ=0.85, 95% CI = [0.77, 0.9], mouth, hand, and sound events: ρ=0.88, 95% CI = [0.85, 0.91]).

Within-group agreement

Blind and sighted participants showed significant within-group agreement for all categories (blind: nouns ρ=0.23 ± 0.24 SD, verbs ρ=0.33 ± 0.28 SD; sighted: nouns ρ=0.42 ± 0.26 SD, verbs ρ=0.5 ± 0.24 SD). Overall, there was lower agreement among blind participants than among sighted participants for both entities/nouns and events/verbs (entities/nouns: main effect of group, F_(1,37)=8.99, p=.005; events/verbs: main effect of group, F_(1,37)=5.65, p=.02). An ANOVA comparing within-group agreement across entity/noun categories revealed a marginal group by semantic category interaction (group x entity semantic category interaction, F_(3,111)=2.24, p=0.09; Figure 1B). Post-hoc Tukey-adjusted pairwise comparisons revealed a significant difference between groups only for mammals (Figure 1B, blind ρ=0.22 ± 0.24; sighted ρ=0.49 ± 0.24). No group by semantic category interaction was observed in within-group agreement for events/verbs (F_(3,111)=1.46, p=0.23).

Average similarity ratings

People born blind rated entities/nouns as more similar to each other (Supplementary Figure 1A; repeated measures ANOVA, 2 groups (sighted, blind) x 4 noun semantic categories (birds, mammals, manmade pl., natural pl.): main effect of group, F_(1,37)=7.47, p=0.01). This effect was qualified by a marginal group by semantic category interaction, F_(3,111)=2.57, p=0.06), whereby the group difference was more pronounced for birds and mammals (Supplementary Figure 1A; repeated measures ANOVA, 2 groups (sighted, blind) x 4 noun semantic categories (birds, mammals, manmade pl., natural pl.).

For events/verbs, there were no significant group or group by condition interaction effects in average similarity ratings (all ps > 0.1; Supplementary Figure 1A; see Appendix 6 for details).

Reaction times

There were no group or group by condition interaction effects in reaction time data among entities/nouns or events/verbs (all ps > 0.5; Supplementary Figure 1B; see Appendix 6 for details). Across both groups, participants were faster to make judgments about animals (birds, mammals) compared to places (manmade, natural; repeated measures ANOVA, 2 groups (sighted, blind) x 4 entity/noun categories (birds, mammals, manmade places, natural places): main effect of semantic category, F_(3,111)=7.79, p<0.0001). Participants across groups were also faster to make judgments about mouth actions compared to all other event/verb categories (repeated measures ANOVA, 2 groups (sighted, blind) x 4 noun categories (hand, mouth, light, sound): main effect of semantic category, F_(3,111)=7.36, p=0.0002).

In sum, subtle differences in behavioral judgments were observed across groups for living things (i.e., animal nouns), a partially vision-dependent category, but not for light events, an entirely vision-dependent category. These findings suggest that visually acquired knowledge is used by sighted people to judge similarity between some ‘visual’ categories, i.e., animals. However, direct sensory access is not necessary for acquiring typical meanings of sensory categories, i.e., light events.

3.2. fMRI results

3.2.1. Do selective univariate responses to living things concepts emerge in the absence of visual experience?

We observed similar neural signatures of living things concepts across groups. In both sighted and blind participants, animal nouns (birds and mammals) activated a sub-region of the PC more than place nouns (Figure 2, animals > places). The animal response observed in the blind group was in an analogous location to previously reported responses to living things (i.e., people) in the PC of sighted participants (e.g., Fairhall et al., 2013b). This result suggests that the emergence of a preferential response to living things concepts in the PC does not require vision.

Figure 2:

Whole-cortex results for animals>places: (A) Sighted; (B) Blind. Group maps are shown p<0.01 with FWER cluster-correction for multiple comparisons. Voxels are color coded on a scale from p=0.01 to p=0.00001. The average PPA location from separate cohort of sighted subjects (Weiner et al., 2017) is overlaid on the place noun response observed in the current study. The two overlap in both groups, with the focus of the place noun response located more anteriorly. The average people-preferring precuneus location from a separate cohort of sighted subjects (Fairhall & Caramazza, 2013b) is overlaid on the animals response observed in the current study. These also overlap in both blind and sighted participants. Increased activation for animals over places is observed in the left precuneus in sighted participants at a lower statistical threshold (p < 0.05 uncorrected). See Supplementary Figure 3 for full whole-cortex results.

Consistent with prior findings, preferential responses to place nouns over animal nouns were also observed in sighted participants on the medial surface, in the retrosplenial complex, inferior to the responses to animal nouns. A similar response to place nouns was observed at a more lenient statistical threshold in the blind group (p<0.01, uncorrected). This retrosplenial complex region has previously been identified as part of the ‘place’ processing network in sighted participants (Ino et al., 2002; Rauchs et al., 2008; Epstein, 2008; Dilks et al., 2022). In both groups, preferential responses to places were also observed in medial VOTC, near but anterior to the canonical location of the parahippocampal place area (PPA) (Weiner et al., 2017), although this response was weaker and more distributed in the blind group, extending into early visual cortices (group-by-condition interaction, Figure 5B).

Figure 5:

Group-by-condition interactions of univariate responses in occipital cortices. Group maps are shown p<0.01 with FWER cluster-correction for multiple comparisons. Voxels are color coded on a scale from p=0.01 to p=0.00001. (A) Peak percent signal change averaged across all occipital regions in which group-by-grammatical class (events vs. entities) interactions were observed. (B) Peak percent signal change in occipital regions in which group-by-entity category (animals vs. places) interactions were observed (occipital pole, anterior medial VTC).

Both groups also exhibited preferential univariate responses to entities/nouns over events/verbs in parietal and temporal regions previously associated with concrete entities, including the posterior parietal, lateral inferior temporal, and medial occipitotemporal cortices, as well as the PC (Supplementary Figure 3).

In sum, selective responses to living things were observed in temporoparietal networks of congenitally blind and sighted participants alike, particularly in the precuneus. Neural specialization for ‘living things’ concepts, a putatively visual category, develops with and without vision.

3.2.2. Multivariate decoding of animals vs. places throughout entity-responsive network in blind and sighted groups

MVPA revealed that animals were robustly discriminable from places throughout entity-responsive regions in both sighted and blind participants (all ps < .05), including in PC (sighted: V = 210, p = 0.0004, blind: V = 173, p = 0.007; see Supplementary Figure 5 and Supplementary Table 3 for results in each ROI), although the sighted group exhibited higher discriminability overall (repeated measures ANOVA, 2 groups (sighted, blind) x 4 ROIs (LPC, LIP, LlatIT, LmedVT): main effect of group F_(1,41)=37.30, permuted p = 0.0002; main effect of ROI F_(3,164)=1.55, permuted p = 0.2).

Inspection of the confusion matrices showed that in both groups, neural patterns for birds were more likely to be confused with mammals than places (repeated measures ANOVA, 2 groups (sighted, blind) x 2 error types (bird-mammal, bird-place) x 4 ROIs (LPC, LIP, LlatIT, LmedVT): main effect of error type F_(1,82)=32.04, permuted p = 0.0002). This effect was qualitatively similar but smaller in the blind group (error type x group interaction F_(3,246)=10.60, permuted p = 0.003). Similarly, mammals were more likely to be confused with birds than with places (repeated measures ANOVA, 2 groups (sighted, blind) x 2 error types (bird-mammal, bird-place) x 4 ROIs (LPC, LIP, LlatIT, LmedVT): main effect of classifier error type F_(1,82)=28.25, permuted p = 0.0002), and this effect was qualitatively similar across groups but smaller in the blind group (error type x group interaction F_(3,246)=23.92, permuted p = 0.0002). These results suggest that all of the neural features of the dissociation between animals and places in temporoparietal semantic networks develop without visual access.

Together, the univariate and the multivariate evidence suggests that neural representations of living things concepts, a partially ‘vision-dependent’ category, develop qualitatively similarly regardless of visual experience.

3.2.3. Responses to visual light events in LMTG+ across blind and sighted people

There were no differences across blind and sighted groups in the LMTG+’s response to light events or any other event category in univariate analysis (Figure 3; individual-subject ROI analysis, repeated measures ANOVA, 2 groups (sighted, blind) x 4 event categories (hand, mouth, light, sound): group x event category interaction, F_(3,123)=1.14, p=0.34; main effect of group, F_(1,41)=0.06, p=0.81; main effect of semantic category, F_(3,123)=7.16, p=0.0002). In other words, the LMTG+ of both groups showed a robust response to light events that was higher than the response to entities/nouns. Multivariate analysis revealed that spatial patterns of neural activity in LMTG+ distinguish between different types of events (i.e., light, sound, hand, and mouth) and this is equally true for blind (t₍₂₀₎=3.91, permuted p=0.0004) and sighted (t₍₂₁₎=3.88, permuted p=0.0003) participants. There were no differences in decoding accuracy between the groups (repeated measures ANOVA, 2 groups (sighted, blind): main effect of group, F_(1,41)=0.94, p=0.34; Supplementary Figure 4). Neural populations in the LMTG+ are therefore sensitive to semantic distinctions between event categories in both sighted and blind people.

Figure 3:

Whole-cortex results for events/verbs > entities/nouns on the left lateral surface: (A) Sighted; (B) Blind. Group maps are shown at p<0.01 with FWER cluster-correction for multiple comparisons. Voxels are color coded on a scale from p=0.01 to p=0.00001. (C) Peak percent signal change (PSC) from the 5% most active vertices for events/verbs>entities/nouns in the LMTG+ (left: sighted; right: blind). Note that this figure can be used to evaluate differences among events and among entities in the LMTG+ ROI, as well as differences between groups in entity/event responses. This figure cannot be used to evaluate within-group differences between events and entities because the ROIs were defined as the most event-selective vertices; thus the difference between events and entities may be exaggerated due to statistical bias. See Supplementary Figure 3 for full whole-cortex results.

We next looked at light events in greater detail, because of their visual nature. Light events were distinguishable from hand events in both groups (blind: V = 158, p = 0.0009, sighted: V = 170, p = 0.0001). In the blind group, light events were also distinguishable from both sound emission events (V = 92, p = 0.007) and mouth actions (V = 127, p = 0.009). In the sighted group, light events were not distinguishable from sound emission events (V = 86, p = 0.65) and were marginally distinguishable from mouth actions (V = 113, p = 0.046). We constructed confusion matrices based on classifier error patterns to probe the ‘representational space’ of the LMTG+ across groups, these confusion matrices are a measure of which event categories have the most similar representations. Consistent with the idea that the LMTG+ of blind and sighted people shares a similar representational space, the confusion matrices for the blind and sighted groups were significantly correlated (Figure 4B; r(30)=0.55, p=0.03).

Figure 4:

Classifier responses and confusion matrices for entity categories in the LPC (A) and for event categories in the LMTG+ (B). Bar graphs display the correct responses and errors for classification of animals vs. places (LPC) and light vs. all other event categories (LMTG+) within each participant group. Note that in the two lightest bars reflect the number of errors made in both directions (e.g., “light-sound” = mean of light (real) – sound (predicted) and sound (real) – light (predicted)). Chance: 25%. Confusion matrices (columns = real, rows = predicted) display the percentage of correct responses (diagonals) and errors (off diagonals) for classification of the relevant categories in each ROI. See Supplementary Figure 5 for results from all ROIs. Key: Mml = mammal, ManP = manmade place, NatP = natural place.

In the blind group, the LMTG+ was the only region that showed higher decoding for verbs than nouns (t₍₂₀₎=−2.68, permuted p=0.01), providing evidence for LMTG+ selectivity for events in this population. In the sighted group, decoding for verbs and nouns was not different in the LMTG+ (t₍₂₁₎=−0.28, permuted p=0.78), whereas it was higher for nouns in LPC and LmedVT (Supplementary Figure 4; Supplementary Table 4). A 3-way repeated measures ANOVA (2 groups (sighted, blind) x 5 ROIs (LMTG+, LPC, LIP, LlatIT, LmedVT) x 2 grammatical classes (entities/nouns, events/verbs)) revealed an ROI x grammatical class interaction but no 3-way interaction with group (two-way ROI x grammatical class interaction, F_(4,164)=6.40, permuted p<0.0001; 3-way interaction F_(4,164)=1.31, permuted p=0.26). This result suggests that entity/noun and event/verb selectivity develop similarly across the cortex regardless of visual experience.

In sum, we find that neural signatures of light events are similar across congenitally blind and sighted people. In both sighted and blind participants, the LMTG+ responds more to events than entities and distinguishes among different semantic categories of events, including light events and other event types. Thus, the neural basis of a conceptual category that is only directly accessible through vision, i.e., light events, develops similarly in people with and without direct sensory access. These results suggest that vision is not necessary for the emergence of category-specific neural responses to ‘visual’ events.

3.2.4. Responses to words in occipital networks of blind and sighted people

Prior studies have identified responses to sentences and words in the occipital cortices of congenitally blind people as well as some sensitivity to properties of spoken words in sighted people (e.g., Sadato et al., 1996, 1998; Burton, 2002; Burton et al., 2003; Röder et al., 2002; Amedi et al., 2003; Bedny et al., 2011; Lane et al., 2015; Seydell-Greenwald et al., 2020). These results have been described as evidence for ‘cross-modal plasticity,’ i.e., the recruitment of visual networks for non-visual tasks. Consistent with this prior work, in the current study, group differences emerged exclusively within occipital cortices.

First, and consistent with prior work, sighted participants showed either deactivation or activity that was not different from rest for all word categories in early occipital cortices (e.g., Bottini et al., 2020). Numerous previous studies find deactivation in visual cortices of sighted people during attentive cross-modal auditory and tactile tasks, a response pattern thought to be related to the suppression of irrelevant information from the visual modality (e.g., Hairston et al., 2008; Kawashima et al., 1995; Murphy et al., 2016; Laurienti et al., 2002). A similar phenomenon is likely to account for the suppression of activity in visual cortex of sighted people while listening to words in the current study. By contrast, the congenitally blind group showed above-rest responses to entity and event words in several early occipital areas, consistent with prior evidence that visual cortices participate in spoken language tasks in this population (Figure 5).

Second, blind and sighted groups showed different preferences across semantic categories in occipital cortex, in line with the idea that the functions supported by visual areas is different across blind and sighted populations. Importantly, however, group differences did not pattern with the ‘visual’ status of the categories. As discussed above, an anterior PPA-like medial VOTC region showed a preference for places in sighted and blind people, but a larger effect was observed in the sighted group. The place-preferring activation of blind participants was more diffuse, expanding into posterior early occipital ‘visual’ networks. In blind participants, a left-lateralized network of early visual areas exhibited increased responses to places over animals, with above-baseline responses observed for both categories (Figure 5B). We suggest that this finding reflects the posterior expansion of semantic place responses into early occipital networks in blind people. By contrast, the same early visual areas of sighted participants exhibited deactivation for both animals and places.

We failed to find any evidence for enhanced responses to light events or living things (i.e., animals) in early visual cortex of sighted people compared to blind people (Supplementary Figure 6). Although concrete nouns are generally more imageable than verbs (see Appendix 2 for imageability ratings of stimuli used in the current study), sighted participants exhibited greater deactivation for nouns compared to verbs in a network of right-lateralized early visual areas (medial, ventral, and dorsal surfaces of the occipital pole; Figure 5, see also Figure 2). By contrast, blind participants exhibited equivalent above-baseline activity for both nouns and verbs in these regions (Figure 5A). Differential responses of early visual cortices to spoken words across blind and sighted people is consistent with prior evidence of plasticity in this population (e.g., Röder et al., 2002; Bedny et al., 2011; Collignon et al., 2013; see Pascual-Leone et al., 2005; Merabet & Pascual-Leone, 2010; Bedny, 2017 for reviews).

Multivariate decoding among semantic categories in early visual networks was weak in blind and sighted people alike (Supplementary Figure 6). In early visual regions defined using a Brodmann area atlas (V1-V2; BA17-18), we observed above-chance decoding exclusively in the right hemisphere of blind participants (decoding of entity categories: t₍₂₀₎=2.51, permuted p=0.009; event categories: t₍₂₀₎=2.33, permuted p=0.01). Thus, despite the fact that early visual cortices showed above-rest univariate responses to events and entities in blind people, these regions do not robustly encode finer-grained distinctions among semantic categories. We found marginal decoding among entities in the left hemisphere of sighted participants (V1-V2; BA17-18 entity categories: t₍₂₁₎=1.56, permuted p=0.07; event categories: t₍₂₀₎=0.45, permuted p=0.34).

In sum, responses in early visual networks of blind and sighted people were not related to the ‘visual’ status of the stimuli. We failed to find any evidence that ‘visual’ words (animal nouns or light events) activate visual cortices in sighted but not blind people. Reponses of early visual networks to spoken words were therefore not predicted by whether or not an individual had accessed the referent of the words through vision.

Discussion

4.1. Semantic similarity judgments for ‘visual’ words across groups

Consistent with prior evidence that people born blind have rich ‘visual’ semantic knowledge, similarity judgments were positively correlated across groups for all semantic categories, including ‘visual’ ones (e.g., Marmor, 1978; Landau & Gleitman, 1985; Shepard & Cooper, 1992; Lenci et al., 2013; Saysani et al., 2018). In line with the claim that vision plays an important role in learning about living things, semantic similarity judgments of blind and sighted people differed more for birds and mammals than for places (e.g., ‘barn’, ‘garage’) (i.e., slightly lower correlations between groups and higher similarity judgments on average for birds and mammals among blind people) (Allport, 1985; Warrington & Shallice, 1984; Warrington & McCarthy, 1987; Farah & McClelland, 1991; Gaffan & Heywood, 1993; Moss et al., 1997; Tranel et al., 1997; Humphreys & Forde, 2001; see Bi et al., 2016, for related arguments). One prior study also found that animal appearance knowledge differs partially across sighted and blind people (Kim et al., 2019). In particular, blind and sighted people’s judgments about animal shape, size, and texture is overlapping but not identical, and labels of animal colors differ across groups (Kim et al., 2019). Together with this prior evidence, the results of the current study suggest that for sighted people, visually derived information about the surface features of animals influences semantic similarity judgments (see Connolly et al., 2007; Kim et al, 2021 for related evidence with regard to fruits and vegetables).

In contrast to living things, which can in principle be accessed through non-visual modalities (e.g., touch, audition), light events (e.g., ‘sparkle’) are directly accessible only through vision. We might therefore expect judgments about light verbs to differ even more across blind and sighted people. Contrary to this prediction, we found that semantic similarity judgments for light events were just as correlated across blind and sighted groups as judgments about non-visual events/verbs. This result corroborates prior behavioral studies that report similar judgments for light event concepts across blind and sighted people (Lenci et al., 2013; Bedny et al., 2019). In sum, these results suggest that shared sensory experience of a concept’s referent does not predict shared semantic knowledge as measured by semantic similarity judgments. If this were the case, we would expect to observe larger differences between groups in semantic similarity judgments about light events compared to judgments about living things.

One factor that could influence the degree to which shared sensory experience influences semantic similarity judgments is the availability of other shared non-visual information that could be used to make the same judgments. Prior evidence suggests that the degree to which semantic judgments of animals are influenced by appearance knowledge varies among sighted people as a function of ecological expertise. While sighted adults living in industrialized societies rely on surface-level visual appearance when judging the semantic similarity of living things (animals and plants), people with more biological expertise (e.g., members of cultural groups that live in closer contact with nature) tend to rely more on abstract causal information such as behavioral and ecological patterns (Murphy & Medin, 1985; Boster & Johnson, 1989; López et al., 1997; Proffitt, Coley, Medin, 2000; Bailenson et al., 2002; Medin & Atran, 2004). The participants in the current study were mostly recruited from the urban environment of Baltimore, although we did not measure their ecological expertise. Subtle differences in semantic similarity judgments about birds and mammals across blind and sighted urbanites could partly reflect the fact that the average sighted U.S. city-dweller knows little else about what distinguishes a sparrow from a finch besides what they look like. Future work comparing blind and sighted people with different levels of animal expertise could resolve this question. Another factor that may influence the coherence of similarity judgments across blind and sighted groups is the degree to which the ‘sensory’ information in question can be readily learned through other sources, such as language. Semantic distinctions among light verbs are arguably low dimensional: light emission verbs fall along dimensions of intensity and periodicity (Faber & Usón, 1999), whereas differences between the shapes, colors, and sizes of birds are complex and seemingly arbitrary, potentially making these features harder to acquire efficiently through linguistic communication.

In sum, subtle group differences in behavioral judgments were observed for living things, and there was high agreement across groups for light events. These results suggest that people can develop shared representations of purely visual concepts, such as light events, with and without direct sensory access to their referents.

4.2. Similar neural responses to living things and light events across sighted and blind people

Behavioral evidence from the current study and prior work is open to multiple interpretations. The observed difference in behavioral judgments between blind and sighted people for birds and mammals suggests that sighted participants use visually acquired appearance knowledge to make semantic similarity judgments about these categories. Do such behavioral differences between blind and sighted people reflect fundamental differences in conceptual representation? Or do they reflect small quantitative differences in knowledge analogous to those typically observed across subsamples of the sighted population (e.g., urbanites vs. naturalists) (e.g., Carey, 2011; Yee & Thompson-Schill, 2016; Marti et al., 2023)? In a similar manner, the absence of group differences in light event judgments could reflect the use of qualitatively similar conceptual representations across groups or mask profound differences in representation (e.g., sighted people use visual representations and congenitally blind people use linguistic ones to arrive at similar judgments). Neural evidence offers complementary insights by testing whether previously identified neural signatures of ‘visual’ concepts emerge in the absence of visual experience. The current neural findings support the view that ‘visual’ concepts develop in qualitatively similar ways across sighted and blind adults.

4.2.1. Specialization for living things in temporoparietal semantic network of people born blind

We find that both sighted and blind people exhibit robust neural specialization for living things in temporoparietal networks. Multivariate analysis revealed distinct neural patterns for living things (i.e., animals) and non-living things (i.e., places) across temporoparietal regions previously associated with the retrieval of entity concepts (e.g., Fairhall et al., 2014; Deen & Freiwald, 2022; Elli et al., 2019). In addition, selective responses to living things emerged in the PC of both blind and sighted participants. These results are consistent with prior findings from sighted adults proposing that the PC supports living things representations (Devlin et al., 2002; Fairhall & Caramazza, 2013a; 2013b; Fairhall et al., 2014; Deen & Freiwald, 2022; Peer et al., 2015; Elli et al., 2019). Our findings suggest that neural specialization for living things concepts develops independent of vision. In other words, whether people have direct sensory access to a concept’s referent does not appear to influence the neural basis of its representation.

Consistent with prior literature, we also observed selective responses to place words in the retrosplenial complex and the medial VOTC of both blind and sighted people. Medial VOTC responses were located anterior to the typical location of the perceptual PPA in sighted people (e.g., He et al., 2013; Wang et al., 2016; Fairhall et al., 2014; Steel et al., 2021; Häusler et al., 2022; Epstein & Kanwisher, 1998; Weiner et al., 2017; Silson et al., 2016, 2019; see also Baldassano et al., 2013). Previous studies have suggested that unlike the posterior PPA, which is involved in place perception, anterior PPA represents mnemonic and/or conceptual information related to places (Silson et al., 2016, 2019; Steel et al., 2021; Häusler et al., 2022). Together, these results point to a vision-independent ‘double dissociation’ in the neural instantiation of living things and place concepts.

In sum, neural signatures of ‘living things’, a partially vision-dependent category for sighted people, develop similarly in people with and without visual experience, suggesting that these networks are robust to differences in sensory experience. This neural evidence points to the ‘body independence’ of living things concepts.

4.2.2. Similar neural signatures of light event concepts in people born blind and sighted

Unlike animals, light events (e.g., ‘sparkle’, ‘glow’) can be perceived only through vision. Despite this, we observed similar neural responses to light emission events among sighted and congenitally blind adults. In both populations, the LMTG+ exhibits distinctive neural responses to light events relative to entities (univariate analysis) as well as to other event categories (e.g., hand actions; multivariate analysis).

The current results expand on prior work showing that representations of motion verbs (e.g., ‘roll’, ‘bounce’) in the LMTG+ are similar across congenitally blind and sighted people (Noppeney et al., 2003; Bedny et al., 2012). One interpretation of these prior findings is that the LMTG+ of people who are sighted represents visual motion information, while the LMTG+ of people born blind undergoes ‘cross-modal plasticity’: visual information is replaced with sensory information from other modalities (e.g., audition, touch) (Yee, Chrysikou, & Thompson-Schill, 2013; Yee, Jones, & McRae, 2017; see also Pascual-Leone & Hamilton, 2001; Bavelier & Neville, 2002). This explanation cannot account for the current findings on light events, which can only be accessed through the visual modality. It is not clear what aspects of auditory/tactile experience could inform the learner that, for example, shining is more similar to glowing than to sparkling or flashing.

We speculate that people born blind use linguistic evidence to acquire the same light event concepts that sighted people acquire (see also Landau & Gleitman, 1985). Large language models (LLMs) trained on linguistic data alone can generate human-like semantic judgments about sensory phenomena (Abdou et al., 2021; Patel & Pavlick, 2022; Li et al, 2021; Wei et al., 2022; Sharma et al., 2024; Gurnee & Tegmark, 2024; Marjieh et al., 2022, 2024). Sensory semantic content can therefore in principle be learned from language alone, i.e., without access to sensory information from any modality. Precisely how people born blind learn ‘visual’ meaning from language remains to be understood.

It is possible that the LMTG+ of blind and sighted people represents different types of information, i.e., language-derived information in blind people and visual motion information in sighted people. This possibility cannot be ruled out by the available neural data but also lacks any positive empirical support. Across several studies and a variety of semantic categories, the LMTG+ of sighted and blind people exhibits similar neural responses to event concepts. By contrast, both the current and prior studies find functional differences across early visual networks of blind and sighted groups (e.g., responses to spoken language and braille in V1) (e.g., Röder et al., 2002; Sadato et al., 1996; 1998; Amedi et al., 2004; Collignon et al., 2011; 2013; Striem-Amit et al., 2015; Abboud & Cohen, 2019). In our view, the most parsimonious account of this evidence is that conceptual representations in the LMTG+ develop in qualitatively similar ways in people with and without direct visual access.

It is also worth mentioning that the LMTG+ responds not only to perceptible events (e.g., ‘to sparkle’, ‘to run’), but also abstract events (e.g., ‘to think’, ‘to love’), as well as to event nouns (e.g., ‘the hurricane’) (Davis et al., 2004; Bedny et al., 2008; Bedny et al., 2014; Noppeney et al., 2003; Bedny et al., 2012). This, together with the evidence that patterns of neural activity in the LMTG+ distinguish among different event types (Elli et al., 2019), suggests that the LMTG+ encodes modality-independent semantic representations of event concepts.

In sum, across a broad range of semantic types, from living things to light events, neural signatures of concepts develop similarly in individuals with and without direct sensory access. This evidence provides support for the hypothesis that concepts are ‘body-independent.’

4.3.1. The relationship of the current findings to prior neuroscience evidence for embodied concepts

Neural data have played a significant role in motivating the view that concepts are grounded in sensory experience and have contributed to the ‘different-body/different-concepts hypothesis’ (e.g., Pulvermüller, 2001; Barsalou, 2010; Gallese & Lakoff, 2005; Zwaan, 2014; Meteyard et al., 2012; Kiefer & Pulvermüller, 2012; Yee & Thompson-Schill, 2016; Reilly et al., 2024). How do we reconcile the current evidence with prior studies that report activation of sensory systems during semantic tasks? We speculate that some of the neural activity observed during conceptual tasks that was interpreted as sensory in prior work is not in fact sensory. This appears to be the case for responses to action verbs observed in the LMTG+, which were originally interpreted as reflecting the activation of visual motion representations. In sighted people, the LMTG+ is located near visual motion perception regions, including area MT+ and biological motion perception areas in the superior temporal sulcus (STS) (Grossman et al., 2000; Isik et al., 2017; Wurm & Caramazza, 2019). The original studies proposing that the LMTG+ represents visual motion information were conducted prior to the advent of modern functional localization techniques and used group analyses prone to the error of ‘blending’ regions that are proximal but functionally distinct in individual participants (Fedorenko et al., 2010; Nieto-Castañón & Fedorenko, 2012). It is therefore possible that neighboring visual and conceptual responses were not separated in this prior work. More recent evidence suggests that modality-specific sensory responses to visual motion are separable from responses to words referring to visual motion (e.g., to run), although there are also shared action representations across language and vision (Bedny et al., 2008; Wurm & Caramazza, 2019)

There is also evidence that under some task conditions, language referring to perceptible qualities (e.g., color) or objects can activate high-level sensory representations (e.g., Hsu et al., 2011; Wang et al., 2020; Seydell-Greenwald et al., 2023). For example, when asked to make highly detailed perceptual judgments about the colors of named objects (is a school bus more similar in color to an egg yolk or to butter?), sighted people activate high-level color perception regions (Hsu et al., 2011). Whether such neural responses should be considered ‘part of a concept’ has been hotly debated (Yee & Thompson-Schill, 2016; Mahon & Hickok, 2016; Leshinksaya & Caramazza, 2016; Machery, 2010).

A ‘dual theory’ of concepts accommodates both the observation that sensory regions can be activated during conceptual tasks and the observation that blind and sighted people exhibit similar neural responses to ‘visual’ categories (Osherson & Smith, 1981; Margolis & Laurence, 2003; see also Bi, 2021). According to this view, people with direct sensory access to perceptible categories have a two-part conceptual representation that includes ‘abstract conceptual cores’ as well as ‘sensory identification procedures’ used to identify referents of that category (Osherson & Smith, 1981; Margolis & Laurence, 2003). The conceptual cores are activated obligatorily and shared across people regardless of sensory experience. By contrast, the sensory identification procedures are retrieved optionally depending on the task and context, are only retrieved for some perceptible categories, and vary across people based on their sensory experiences (Yee & Thompson-Schill, 2016). Some recent evidence comparing the neural basis of color knowledge across blind and sighted people is potentially consistent with this view. Blind and sighted people activate similar semantic networks when judging object color, but sighted people additionally activate occipital visual-perceptual regions (Bottini et al., 2020; Wang et al., 2020). One interpretation of this result is that the regions activated by both sighted and blind people represent abstract conceptual cores, whereas the visual-perception regions activated only by sighted people support perceptual identification procedures.

The conceptual cores vs. identification procedures hypothesis still leaves open the question of why visual-perceptual regions (i.e., occipital cortices) are recruited by sighted people for some visual concepts (e.g., object colors) but not others (i.e., living things and light events). Likewise, it is unclear why neural responses to living things and light events are indistinguishable across sighted and blind people making detailed semantic judgments, while subtle group differences are observed in object color labeling tasks (Bottini et al., 2020; Wang et al., 2020).

One speculative possibility is that sighted people are more likely to store and retrieve long-term perceptual representations of object colors than light events. More generally, ‘visual’ identification procedures might be less relevant for semantic categories that can be identified without retrieving information from long-term memory. Unlike representations of object appearance, light events are low-dimensional (flashing = bright light changing periodically), dynamic (i.e., unfolding over time), and variable across instances (flashing streetlights vs. flashing lightning). For instance, when told to ‘drive until you see the flashing light’, a sighted person might identify the flashing event without the need to store or retrieve a long-term sensory memory of flashing.

Regardless of which of these explanations, if any, is correct, the available data suggest that some purely visual concepts (e.g., light events) are behaviorally and neural indistinguishable in individuals with and without direct sensory access. Moreover, differences in sensory experience and resulting differences in appearance knowledge (i.e., about living things) do not necessitate changes in the neural basis of semantic representations. Even in the face of dramatic sensory differences, people acquire shared sensory concepts with similar neural bases. Such evidence is difficult to reconcile with the ‘different-body/different-concept hypothesis.’ Social and inferential learning via linguistic evidence establishes shared conceptual representations across people.

Data statement

Anonymized fMRI data from the current study are available on OpenICPSR (https://www.openicpsr.org/openicpsr/project/198163/version/V3/view). Code used in the current study can be found on Open Science Framework (https://osf.io/f4dj2/?view_only=13b637d0bde049d684077b331c606bc7) and GitHub (https://github.com/NPDL/NPDL-scripts).