Mapping the neuronal building blocks of human language with language models

Abstract

Humans can convey new and highly diverse information through language. This ability to form and combine words into elaborate phrases and sentences enables us to express inexhaustible meanings and is fundamental to human cognition^1–5. However, understanding the microscopic cellular building blocks and cortical landscape that precisely underlie human language has remained a challenge. Here we used wide-scale single-neuronal recordings combined with natural language processing models to identify fine-grained linguistic representations across the human frontotemporal cortex during language production. We find that, whereas certain neurons represented the detailed grammatical relationships between words or their parts of speech, others tracked the sentences’ higher-order syntactic structure, their phrase transitions and sequence. Collectively, these neurons reliably captured the words’ syntactic and semantic properties but also dynamically incorporated their specific sentence contexts, therefore enabling them to encode information combinatorially and at highly granular levels of detail. We show how these cell populations were locally organized and how their microscale representations differed from that of their wider field potential patterns. We also show how these neurons were distributed broadly across the frontotemporal cortex, but how their ability to encode linguistic information was left-lateralized and varied between cortical regions. Together, these findings identify some of the most basic cellular building blocks by which linguistic information is encoded in humans and begin to define the cortical landscape of language at a combined micro (cellular), meso (local population) and macro (regional) scale.

Wide-scale recordings reveal neurons in the human brain that encode fundamental components of language such as the grammatical relationships between words, their parts of speech and the higher-order syntactic structure of phrases and sentences.

Main

Language is a fundamental cognitive process unique to humans that allows us to communicate complex and highly diverse meanings. It also enables us to construct new and potentially inexhaustible expressions through grammatical rules that govern how we arrange and combine words and that can generalize across sentences beyond simple repetition. Linguistic studies based on behavioural observations have accurately described how we use the properties of words, such as their grammatical function and syntactic relationships, to construct unique sentences^1–3. They have also described how we use the hierarchical structure of phrases and sentences to communicate specific ideas and thoughts⁶. Recent neuroimaging^7–11 and electrocorticographic recordings^5,12–15, in turn, have identified broadly distributed regions across the frontotemporal cortex that reliably engage in speech production and sentence construction^16–19 and that differentiate grammatically well-formed sentences from unstructured word lists or sounds, suggesting their sensitivity to syntactic structure^20,21. They have also found neural responses that reflect the merger of words into phrases⁶ and their semantic properties^22,23, revealing a broad macroscopic network of cortical areas that could support human language.

However, understanding the microscopic organization and cortical landscape by which linguistic information is encoded by neurons in humans or the cellular processes that underlie natural speech has remained a longstanding challenge. While recent investigations have revealed how the phonetic components of words are encoded by neurons^24,25, they do not reveal the cellular processes by which we produce meaningful speech or through which we arrange and combine words into phrases and sentences. To convey meaning through language, for example, humans use abstract grammatical categories (such as adjectives and nouns) and dependencies (such as the relationship between nominal subjects and objects) that can generalize across sentences and that can describe complex relationships such as actions and outcomes. Yet, how such grammatical features are encoded by neurons or whether they generalize across sentences largely remains undefined. Little is also known about whether neurons can represent the higher-order syntactic structure of sentences or how they encode their constituent phrases.

Another prominent question in neurolinguistics is whether syntactic information is dissociable to some degree from that of semantic information, or how these core aspects of language are represented at a cellular scale^26,27. Whereas previous imaging studies have suggested that linguistic information is probably represented broadly across the brain^28,29, little is also known about how such information is distributed across cortical regions or whether speech processes are lateralized at a basic cellular level. Finally, little is known about how the local activities of individual neurons and their tuning properties relate to those of the populations’ broader field potential patterns, or what their local organization within the cortex may be.

Here we performed single-neuronal recordings across the human frontotemporal cortex and tracked their action potential (AP) and local field potential (LFP) activities over long-term durations as participants produced natural speech. Using speech tracking, parsing, modelling and decoding techniques, we describe the detailed organization and encoding properties of the neurons. We also describe their distribution, regionality and lateralization, together providing a detailed examination of linguistic information encoding during language production at a combined micro (cellular), meso (local population) and macro (regional) scale.

Recording neurons during natural speech

Recordings were obtained from neurons across the human frontotemporal cortex using semichronically implanted microelectrode arrays (96 channel configuration)^30–32. These arrays were implanted as part of planned neurosurgical care for epilepsy monitoring^30,33,34 and were located in the frontal^35,36, anterior temporal^20,37 and posterior temporal^38–40 regions that have been shown to reliably engage in speech production and sentence construction^41–43 and to display robust language-selective responses^41,42,44 by validated language localizers^28,41,45 (Fig. 1a). They were also placed in participants who were awake, after implantation, and were therefore capable of producing natural speech, together providing a rare opportunity to study the activities of individual neurons during natural language production (see the ‘Microelectrode recordings’ section of the Methods). In total, we recorded from 579 putative neurons in 8 participants (3 female, 5 male, aged 27 to 52 years) and 14 sessions and only well-isolated single units with stable waveform morphologies consistent with those of cortical neurons were used (Fig. 1b and Extended Data Fig. 1a).

Fig. 1. Tracking linguistic representations by neurons across the human frontotemporal cortex during natural speech.

a, Sagittal views of microelectrode locations. Eight participants (579 neurons) were recorded from, with four sites located in the left hemisphere (n = 254 neurons) and four in the right hemisphere (n = 325 neurons). An overlay of the recorded sites (left), the recording sites by hemisphere⁶⁴ (middle) and the approximate microelectrode dimensions (right) are shown. The highlighted areas represent regions that were previously identified to display language-selective responses on validated language localizers^28,41,45 and reliably engage in speech production^{41–43,65,66}. b, AP voltage tracings and waveform morphologies of well-isolated single units (one example out of n = 579 neurons) at varying timescales (left); each line represents a single AP (right). c, AP activities and raw audio waveforms time-aligned to each word. Spike timings for each cell are shown in black, and firing rate profiles are shown in red. n = 3 out of 579 neurons. d, For illustration, a constituency structure of a sentence, with each word labelled based on its respective part of speech (personal pronoun (PRP), verb (VB), adverb (RB), preposition (IN), determiner (DT), noun (NN)), constituents (noun phrase (NP), verb phrase (VP), adverb phrase (ADVP), prepositional phrase (PP)), constituency depth and ordinal position. e, Constituency structures of three representative sentences. f, Peri-word histograms (top) and raster plots (bottom) of four representative neurons, aligned to the onset of word utterance. The mean ± s.e.m. neuronal activity in spikes (sp) per second (s) is colour coded based on the linguistic feature that the neuron preferentially responded to. The coloured lines and dots reflect the features to which the neurons were preferentially tuned, whereas the grey lines and dots reflect the neuronal activity for all other features (Extended Data Fig. 1b). MD, modal.

Extended Data Fig. 1. Single-unit waveform and selectivity to linguistic features.

a. Left, example of a single-unit waveform isolated from an electrode within the Utah micro-electrode array. Right, the hierarchical structure of the constituency parser for an example sentence, with the rightmost branch for each hierarchical layer is marked in red. The corresponding number indicates the layer number at which the particular branch closes. Closing level was determined by counting the number of hierarchical levels, starting from the word (leaf node), upward until it reached a node that was no longer at the rightmost branch of the parse tree b. Examples of PSTH plots of four representative neurons out of n = 579 (Fig. 1f). Neuronal activity (mean ± s.e.m.) is colour-coded based on the linguistic feature that the neuron preferentially responded to whereas the grey lines reflect neuronal activity for all other features. A cell might be selectively tuned to the words’ parts of speech (e.g., adverb) whereas others may be selectively tuned to their constituents (e.g., verb phrase), constituency depth or closing level. The sample size (n) is contingent upon the number of words in the in-category and out-of-category conditions, which varies with linguistic feature, participant and session. c. Control analysis to ensure the percentage of selective neurons was not due to specific subsets of words by randomly sampling 80% to 95% of words. The blue dots represent the percentage of selective neurons for each of sub-sampling, and the thick black line labels their median. The black box labels the first and third quartile of the distribution and the whiskers label 1.5× the inter-quartile range from the box. The sample size (n) is contingent upon the number of words in the in-category and out-of-category conditions, which varies with linguistic feature, participant and session.

During recordings, the participants produced various phrases and sentences during natural speech^{42,43,46–48} (see the ‘Language production and audio recordings’ section of the Methods). The sentences were also constructed de novo during natural language production (rather than simply read or repeated) and differed broadly in context and structure (Fig. 1c). For example, the participant may construct a declarative sentence such as “They came home for the weekend” or they may produce a sentence with a subject–verb inversion, such as “When is the payment due?” (Extended Data Table 1a; see the ‘Language production and audio recordings’ section of the Methods). Together, the participants produced 10,460 words across 1,895 sentences, for an average of 747 ± 168 (mean ± s.e.m.) words and 135 ± 28 sentences per neuron per session. All words produced by the participants were transcribed using a semiautomated platform and were aligned to each neuron’s AP activity (see the ‘Language production and audio recordings’ section of the Methods). The participants were English speakers and displayed normal language function on preoperative testing (see the ‘Participants’ section of the Methods).

Extended Data Table 1.

Speech production, linguistic features, and the neuronal selectivity

a. Representative prompts and sentences produced by the participants during natural speech. These sentences varied in structure and content and were constructed de novo during natural language production. b. We extract constituents, parts of speech, constituency depth, and closing levels using a constituency parser. We also included the ordinal position of each word that increases linearly with the length of a sentence. To allow for consistency across comparisons, constituency depth, closing level, and ordinal positions were all capped at eight (note that ordinal position was set at 1 – 9 to reflect the actual word position). c. To confirm the consistency of neuronal selectivity, the percentage of neurons that displayed significant selectivity for each syntactic feature was triply calculated using rank-sum tests, logistic regression tests and linear regression tests (linear regression tests were only performed for the continuous variables). We also performed multiple regression analysis with all features combined (Lasso regression) as well as sequential feature selection analysis to further account for potential collinearity between features. Lastly, we included non-syntactic variables (surprisal, word frequency, and word length) in the multiple regression models to verify the robustness of the percentage of responding neurons. All main results are reported in the paper based on the rank-sum test. d. We examined the dependencies and the dependency depth obtained from the dependency parser. To allow for consistency across comparisons, dependency depth and ordinal position were also all capped at eight.

Linguistic representations by neurons

Neurons within the population responded selectively to the linguistic properties of the words. Natural language processing models such as context-sensitive constituency parsers allow for some of the core linguistic properties of words to be reliably labelled and tracked within natural speech^1–3,49. Thus, for example, whereas the word’s part of speech reflects its grammatical properties (such as an adjective), the sentential constituents reflect how the word or group of words functions as a grammatical unit (for example, within a noun phrase). Furthermore, whereas the word’s constituency depth describes its hierarchical syntactic relationship to other words in a sentence, its ordinal position describes its rank order (Fig. 1d,e, Extended Data Fig. 1a and Extended Data Table 1b). By tracking the neurons’ AP activities in relation to each planned word (pre-articulation period, −400 ms to −100 ms from word utterance; see the ‘Single-neuronal analysis’ section of the Methods) and by using a constituency parsing approach (see the ‘Sentence parsing and linguistic feature labelling’ section of the Methods), we find that 9.2% (n = 53) of the neurons responded selectively to the words’ parts of speech (one-sided Mann–Whitney U-test, α = 0.05, P values were adjusted with false discovery rate (FDR) correction for multiple feature comparisons; Figs. 1f and 2a–c)—for example, transiently increasing their firing rates when the upcoming word was a modal verb (Fig. 1f and Extended Data Fig. 1b). By contrast, other neurons responded selectively to the sentence’s constituents (16.2%, n = 94), whereas 15.9% (n = 92) responded selectively to the word’s constituency depth (distance from the root of the constituency tree) and 11.1% (n = 64) to closing level (phrase mergers). Together, these proportions of neurons were significantly higher than expected from chance (χ² test, P < 0.05) and remained consistent when randomly subsampling words (bootstrapping test, n = 100, P > 0.05 for all features; Extended Data Fig. 1c). They were also consistent when tested across statistical techniques such as multiple regression analyses and sequential feature selection that accounted for collinearities between features (χ² test, P > 0.05 with Bonferroni correction across comparisons; Extended Data Table 1c), together suggesting that these neurons reflected the linguistic properties of the words.

Fig. 2. Selectivity and robustness of linguistic representations at the cellular scale.

a, The proportions of neurons out of n = 579 that displayed selectivity to specific linguistic features (left; Mann–Whitney U-test, α = 0.05, P values were FDR corrected for multiple feature comparisons). Right, coincidence matrix displaying the distribution of cells that displayed selectivity to specific linguistic features. Inset: the percentage of neurons exhibiting overlapping selectivity with lower-order acoustic features (for example, pitch). b, Dependency structure of a representative sentence, with grammatical relationships (dependency), dependency depths and ordinal positions. Here the labels are nsubj (nominal subject), advmod (adverbial modifier), prep (prepositional modifier), pobj (object of a preposition) and det (determiner). c, The proportions of neurons (out of n = 579) that displayed selectivity to dependencies and dependency depth (left; defined as Mann–Whitney U-test, α = 0.05, P values were FDR corrected for multiple feature comparisons). Top right, a coincidence matrix of the distribution of cells that displayed selectivity to specific linguistic features. Bottom right, the number of neurons and their overlap based on their encoding properties including, for example, constituency (const.) depth, dependency (dep.) depth and ordinal position. d, Population decoding performances of individual words for each linguistic feature. The mean performance (triangles) is shown relative to chance levels (dotted lines) and across iterations of train–test splits (histograms, one-sided permutation test; see the ‘Decoding linguistic features from neural activity’ section of the Methods). Additional observation-averaged decoding performances that demonstrate decoding consistency across bootstrapping numbers, training datasets and participants are further shown in Extended Data Fig. 2. e, Dimensionally reduced representation of linguistic features across population neurons (uniform manifold approximation and projection (UMAP); see the ‘Single-neuronal analysis’ section of the Methods). Each selected neuron is colour coded based on the feature to which it displayed significant selectivity. In the case of neurons responding to multiple features, all features are plotted and overlaid.

These linguistic representations by neurons were also robust. Using an L2 regularized balanced multinomial logistic regression classifier to predict the linguistic properties of words not used for model training (70:30 split into training and testing; see the ‘Decoding linguistic features from neural activity’ section of the Methods), we find that all of the linguistic features that were tested could be reliably decoded from the neurons’ activity patterns on a word-by-word level (one-sided permutation test, P < 0.05; Fig. 2d). Thus, for example, the activities of the neurons could accurately predict the constituents to which the words belonged when tested on randomly selected words or sentences not used for model training. The highest single-word decoding accuracies were observed for the words’ constituency depth (31.5% observed versus 14.3% chance; one-sided permutation test, P < 10⁻³, n = 1,000), followed by closing level (27.1% observed versus 11.1%, P < 10⁻³), constituents (25.1% observed versus 10.0% chance; P = 0.005) and parts of speech (18.4% observed versus 10.0% chance, P = 0.031), with slightly higher decoding performances being found when using observation-averaged activity patterns (Extended Data Fig. 2a,b). The linguistic properties of the words could therefore be reliably decoded from the activity patterns of the neurons.

Extended Data Fig. 2. Consistency and robustness of decoding performance.

a. To confirm that observation-averaged population decoding performances were not affected by the number of observations that were averaged, the observations were bootstrapped using n = 10 (top), 20 (middle), and 30 (bottom) observations. b. To further ensure that observation-averaged decoding performances were robust, both neuronal selection and decoding model training were performed only on the training datasets and then decoded from neuronal activity on the testing data. c. Linguistic features were decoded from increasing portions of selective neurons randomly obtained from each participant. To describe the relations between the number of neurons and the observation-averaged decoding accuracy, an exponential fit was provided for each feature (black dotted line). The darker coloured lines label the mean of decoding accuracy, whereas the shaded areas label 3 standard errors. d. Observation-averaged population decoding accuracies across constituency and dependency parsers are plotted by participant and session. e. Observation-averaged decoding accuracies for each linguistic feature are listed for each participant. f. To verify that the linguistic features can be robustly decoded from neuronal activity regardless of sentence content and context, we divided the sentences into different clusters based on their Vicuna model embeddings. Altogether, the observation-averaged decoding accuracies were not significantly different when using 3 or 4 clusters (multivariate linear model, Wilks’ Λ = 0.96, F(12, 284) = 0.54, p = 0.89, and Wilks’ Λ = 0.95, F(18, 541) = 0.59, p = 0.91, respectively).

Finally, given the above findings, we examined whether these linguistic representations by neurons were generalizable. By randomly sampling cells from the recorded population and by accounting for their sparse spiking using observation-averaged decoding, we find that decoding performances were similarly significant across all tested features (one-sided permutation test, P < 0.001) and that decoding performances consistently plateaued as the number of tested neurons increased (exponential fit; Extended Data Fig. 2c). We also find similar observation-averaged decoding performances when comparing neuronal activities across the different participants and recording sessions (Friedman test, P = 0.08) as well as similarly significant decoding when examined across all tested features (92% and 86% of all sessions and all participants, respectively; one-sided permutation test, α = 0.05; Extended Data Fig. 2d,e). Lastly, using a natural language processing model to further cluster the words based on variations in their sentence contents (see the ‘Generalizability of linguistic representations’ section of the Methods), we find largely consistent decoding performances for all linguistic features (multivariate linear model, P > 0.05; Extended Data Fig. 2f), suggesting that these linguistic representations by the neurons were not only robust but also generalizable.

Specificity of linguistic encoding

In contrast to constituency parsers, whereby a sentence is decomposed into constituent units, dependency parsers principally describe the words’ grammatical relationships, which grow hierarchically based on the relationship between the head and dependent words (Fig. 2b and Extended Data Table 1d). Thus, using a dependency parsing approach (see the ‘Sentence parsing and linguistic feature labelling’ section of the Methods) to further test the specificity of neuronal responses, we find that 10.2% (n = 59) of the neurons responded selectively to the grammatical relationships (dependencies) between the words—for example, changing their activities based on whether a planned word will be the direct object or nominal subject of a governing word. Furthermore, 12.1% (n = 70) of the neurons responded selectively to the word’s dependency depth (Fig. 2c and Extended Data Table 1c), reflecting the hierarchical relationship between the head and dependent word. Overall, both the dependency relationship and depth could be reliably decoded from the neurons’ response patterns (23.2% and 27.2% observed single-word decoding versus 10.0% and 11.1% for chance, respectively; one-sided permutation test, P < 0.05; Fig. 2d), suggesting that these features were represented reliably.

Given the above findings, it is possible that the neurons’ responses may be partly explained by overlapping word features—for example, a word can be both a noun and belong to a noun phrase. However, when we evaluated the specificity of the neurons across all possible feature pairs, we found that only 2.2% of them displayed an overlap. Among all the pairs, only parts of speech and dependencies displayed a significant overlap across tests of independence (2.6% of neurons; two-sided permutation test, P = 0.028; χ² test, χ²₁ = 18.8, P = 4.1 × 10⁻⁴; Fisher’s exact test, odds ratio = 4.32, P = 2.1 × 10⁻³; Bonferroni corrected for multiple comparisons). We also found little correlation in the degree to which the neurons were modulated by parts of speech and dependencies when compared on a neuron-by-neuron basis (linear regression test, P = 0.18), further confirming that their responses were specific.

These neurons also appeared to display a high degree of specificity for features that were closely related. For example, even though the words’ constituency and dependency depths are often highly related, both of which increase with sentence progression, we found little overlap between cells that encoded constituency and dependency depths (one-sided permutation test, n = 1 × 10⁵, P = 0.31 (constituency and dependency depth), P = 0.14 (constituent and dependence relationship); Fig. 2c,e). There was also little overlap between neurons that responded selectively to either the constituency depth or those that simply reflected the words’ ordinal positions (one-sided permutation test, n = 1 × 10⁵, P = 0.33), which also grows with sentence progression, suggesting that these linguistic features were distinguishable at the cellular level.

Finally, we examined whether other lower-order aspects of speech, such as the sound-spectral properties of words, could have partly explained these neuronal responses. However, when tracking the words’ acoustic features, we find that, of the neurons that responded to variations in word pitch (12%, n = 72 out of 579 neurons), only a few (≤2.2%) overlapped with any of the neurons that displayed linguistic selectivity (two-sided permutation test, P > 0.1 across all features; Fig. 2a). We also found little difference in the proportions of neurons that displayed linguistic selectivity when considering other related features such as word frequency, length and surprisal (multiple regressions analysis, with added features; see the ‘Single-neuronal analysis’ section of the Methods; Extended Data Table 1c), suggesting that the neurons’ responses were indeed robust to variations in these lower-order speech properties.

Combinatorial linguistic encoding

Although the above findings identified neurons that reliably reflected the words’ linguistic properties, it remained unclear whether and to what degree they encoded their specific combinations. For example, even though the word ‘dog’ in the sentences ‘The dog ate the bone’ and ‘The man walked the dog’ is the same, the specific combination of linguistic features that describe it differs (although the word ‘dog’ is a noun in both sentences, it is a nominal subject in the former and an object in the latter). Similarly, although ‘dog’ and ‘bone’ are both nouns, they hold broadly different semantic meanings^1–3,49.

Therefore, to further study the combinatorial encoding properties of the cell population, we constructed embedding models that could simultaneously capture these features and whereby each linguistic feature (for example, parts of speech, dependencies) was represented by a unitary value (Fig. 3a). Thus, for example, when producing the word ‘dog’ in ‘The dog ate the bone’, these embeddings would reflect not only the part of speech to which the word ‘dog’ belongs but also its constituency depth and serial order as a one-hot vectoral projection (see the ‘Syntactic and semantic embedding models’ section of the Methods). To further account for the semantic properties of the words, we also provided vectoral projections of the words’ meanings based on Word2Vec embeddings²⁷. Lastly, to allow for valid comparison and to limit the possibility of overfitting, we dimensionally reduced the models’ embeddings using a principal component analysis to calculate their predictivity—a measure of how well the models’ embeddings can reliably predict the observed neuronal activity patterns (see the ‘Language model predictivity’ section of the Methods).

Fig. 3. Population organization and temporal structure of information encoding by neurons.

a, Neuronal predictivity by syntactic (one-hot vectors), semantic (word2vec), a combination of syntactic and semantic and higher-order contextual embeddings (LLaMA Vicuna) was defined as the correlation between the recorded (observed) and model (predicted) activities (see the ‘Language model predictivity’ section of the Methods). b, The distribution, quartiles, 1.5 × interquartile range (whiskers), mean values (thin lines) and median values (thick lines) of population predictivities. Each dot corresponds to a single session (n = 14; grey circle: outlier at −0.3). Chance predictivities are shown in grey. Statistical analysis was performed using one-sided permutation tests (n = 10⁵); contextual features versus syntactic (P = 0.008, 95% confidence interval (CI) = 0.0076–0.058), semantic (P = 0.042, CI = −0.0015–0.069) and combined semantic–syntactic features (P = 0.008, CI = 0.0069–0.054); see the ‘Language model predictivity’ section of the Methods. c, The relationship between context length and population predictivity, with predictive performance plateauing (reaching 1/e) at 5 past words. Data are mean± s.e.m. d, Swap-sentence control (top). n = 14. Data are mean ± s.e.m. Predictivity = −0.01; one-sided permutation test, P < 10⁻⁵, n = 10⁵, CI = 0.051–0.063. Bottom, pseudosentence input control. n = 14. Data are mean ± s.e.m. Predictivity = −0.03; one-sided permutation test, P < 10⁻⁵, n = 10⁵, CI = 0.069–0.082). e, Normalized contextual model predictivity for the neural population, time aligned to word utterance (left). Right, normalized predictivity across time and network layers for individual neurons, calculated from −2,000 to 2,000 ms (100 ms increments) in relation to word utterance to evaluate contextual integration^{56,57,67–69}. f, The average peak predictivities of selective neurons are illustrated by the size of the solid circle; the transparent circle indicates 2 s.d. of the predictivity (note that the datapoints for constituents and dependency overlap). *P < 0.05, **P < 0.01.

Here we find that the activities of cells within these populations could be reliably predicted by the specific combinations of linguistic features that defined the words (one-sided permutation test; P < 0.05 for all tested models), with the highest mean predictivity observed for models that included both syntactic and semantic information (mean predictivity = 0.015 ± 0.01 (mean ± s.e.m.); Fig. 3b). However, we also find that most individual neurons preferentially encoded either syntactic information or semantic information alone, with only 1.8% of the cells encoding both syntactic and semantic information for the features tested (binomial test with Bonferroni correction for multiple comparisons, P > 0.05 for all features). Thus, while many of the neurons were preferentially tuned to respond to specific linguistic features, they also reliably encoded information about the combinations of linguistic features that uniquely defined the words.

Finally, beyond encoding syntactic and semantic information about the words, these cell populations were also informative of the words’ specific sentence contexts. Here, using large language models^24,25 (Vicuna; Extended Data Table 2; see the ‘Higher-order contextual embedding models’ section of the Methods) that could capture the sentences’ contexts (for example, ‘bark’ within ‘The dog has a loud bark’ versus ‘The tree has thick bark’), we find that the contextual model was more predictive of the neurons’ activities than models that simply included the words’ syntactic or semantic features (population predictivity, 0.05 ± 0.01 (mean ± s.e.m.); one-sided permutation test P < 0.05, n = 10⁵; Fig. 3b). Notably, models that incorporated progressively longer context windows (from 2 to 10 words preceding the spoken word) were increasingly predictive of neuronal activity (plateauing at approximately 5 words; exponential fit; Fig. 3c), By contrast, predictivity was at chance when randomly shuffling the sentences in relation to neuronal activity (population predictivity at −1,000 ms = −0.01; one-sided permutation test, P < 10⁻⁵, n = 10⁵; Fig. 3d) or when replacing the words spoken with randomly selected tokens (selecting an entirely new set of words; population predictivity = −0.03; one-sided permutation test, P < 10⁻⁵, n = 10⁵; Fig. 3d), suggesting that the activities of these neurons reflected the words’ contexts. We also find that the predictivities for the neurons remained consistently high well before word onset (population and single-neuronal predictivity, one-sided permutation test, P < 1 × 10⁻⁵, n = 10⁵; range −2,000 to 2,000 ms, tested in 100 ms increments; Fig. 3e), peaking approximately 1 s before utterance, and displayed a close temporal relationship between the neurons’ peak predictivities, their timings, model layers and linguistic selectivity (regression model, P = 0.02; Fig. 3f, Extended Data Figs. 3–6 and Supplementary Video 1). Together, these cell populations therefore appeared to provide richly detailed representations of the syntactic, semantic and contextual properties of the words before utterance.

Extended Data Table 2.

Embedding models, training, and size

Open-source large embedding models were used for testing the model source, size of the model, and training or fine-tuning performed^122–126. To ensure that the findings obtained from these models are both generalizable and robust, we performed additional validation experiments using a diverse suite of large language models that varied in their architectural depth, parameter scaling, and training corpora. Here, we evaluated models spanning several orders of magnitude in size across different model families, including Falcon (1B and 7B), LLaMA (3B and 7B), and Bloom (560 M, 1B, and 7B), as well as the Pythia family (3B and 7B).

Extended Data Fig. 3. Neuronal predictivity across participants, recording sessions, layers and models.

A leave-one-participant-out procedure was used to determine whether any participant disproportionately contributed to population predictivity across (a) participants and (b) sessions. Here, absent points reflect the subjects or sessions that were dropped. Overall, predictivities were consistent across sessions (Kruskal Wallis test, H(13) = 1.1, p = 1.00, effective size = 0; peak layer). c. Neural population predictivity across layers for the contextual model (blue) and the shuffle control (grey) ± s.e.m. Predictivity was significantly higher than random shuffling the neuronal activity (one-sided permutation test, p < 10⁻⁵, n = 10⁵, 95% CI = 0.048–0.060), and reached a peak at the 14^th layer of 0.07 (compared to 0.01 in the shuffle control). Predictivity was calculated for each layer respectively. d. Exponential fit for the predictivity as a function of layer. Example models from each family with the highest peak predictivity (Bloom (1b), Pythia (3b), Falcon(rw7b), and Vicuna (7b)) and their exponential fits (dashed lines).

Extended Data Fig. 6. Predictivity for selective neurons by time and layer.

Individual neuronal predictivities by the contextual model across different times are plotted. Single neurons are grouped by their selectivity for the linguistic features.

Diversity and distribution of neurons

Collectively, these neurons were distributed broadly across the recorded cortical areas and hemispheres. Neuronal recordings were made here from frontal^35,36, anterior temporal^20,37 and posterior temporal^38–40 regions shown to reliably engage in speech production^28,41–43 and to display robust language-selective responses^41,42,44 by validated localizers^28,41,45 (Fig. 1a). By tracking their encoding properties, we find that the proportions of neurons that responded selectively to at least one or more features (289 out of 579) were similar in the frontal, anterior temporal and posterior temporal regions (χ² test, χ²₂ = 0.87, P = 0.65). They were also similar when comparing the left to right hemisphere (χ² test, χ²₁ = 0.51, P = 0.47; Fig. 4a), and there was little difference in the distribution of neurons based on the particular linguistic features tested (χ² test, P > 0.05; aside from dependencies and constituents; Fig. 4a), suggesting that they were widely distributed.

Fig. 4. Diversity and distribution of neurons across frontal-temporal cortical regions and hemispheres.

a, The distribution of neurons across participants, hemispheres and brain areas. Top, the outer ring illustrates the proportion of selective neurons; individual segments are partitioned by participant. The brightness of these segments labels the hemisphere: lighter tones represent the left hemisphere and darker tones represent the right hemisphere. Within the inner circle, grey scales identify the anatomical areas: anterior temporal cortex (dark grey), posterior temporal cortex (mid-grey) and posterior frontal cortex (light grey). Bottom, the proportions of neurons that displayed selectivity for one or more features, grouped by cortical areas and hemispheres (Extended Data Fig. 7a). b, Neuronal modulation across hemispheres and cortical areas. The z score from neurons significantly responding to one or more syntactic features were grouped by hemispheres (top) or brain areas (bottom). One-sided permutation tests (n = 10⁵) yielded the following results: left versus right (P = 0.036, 95% CI = −0.0051–0.11), anterior versus posterior temporal (P = 0.012, CI = 0.014–0.13), anterior temporal versus frontal (P < 10⁻⁵, CI = 0.16–0.34) and posterior temporal versus frontal (P = 0.0013, CI = 0.085–0.28). The left subplots display the average z scores across all linguistic features for neurons that responded significantly to any feature; the yellow–green bar indicates the mean and the yellow–orange bar indicates the median of the z scores. The right subplots illustrate the average z scores for neurons selective to each specific feature across hemispheres (top) and across cortical areas (bottom). ***P < 0.001.

However, the strength of modulation and the degree to which the neurons were informative of the linguistic features preferentially lateralized to the left hemisphere and differed across cortical regions. Specifically, when we examined the degree of modulation, which reflects the extent to which each neuron changes its activity in response to specific features (their z-scored change in activity; see the ‘Single-neuronal analysis’ section of the Methods), we found that neurons in the left hemisphere displayed significantly stronger modulation than neurons in the right hemisphere (n = 579; rank-sum test, z = 8.06, P = 7.7 × 10⁻¹⁶), including to lower-order word features such as pitch (rank-sum test, z = 4.53, P = 5.8 × 10⁻⁶; Fig. 4b (top) and Extended Data Fig. 7a,b). This difference between hemispheres was consistent across cortical areas (rank-sum test, P < 3 × 10⁻⁶), with the posterior temporal cortex exhibiting the most significant difference (rank-sum test, z = 8.32, P = 9.0 × 10⁻¹⁷). Furthermore, when compared across cortical areas in both hemispheres (Kruskal–Wallis test, H₂ = 33.15, P = 6.3 × 10⁻⁸), the strongest modulation was found in the prefrontal cortex (rank-sum test, P < 0.05; for all features except for closing level; Fig. 4b (bottom) and Extended Data Fig. 7c). Neural predictivities from the contextual embeddings were also highest in the left compared to right hemisphere (rank-sum test, z = 3.3, P = 9.5 × 10⁻⁴) but highest in the anterior temporal cortex (Kruskal–Wallis test, H₂ = 616, P = 1.6 × 10⁻¹³⁴; Extended Data Fig. 7d,e). Thus, even though these neurons were distributed broadly across cortical sites, the strength of modulation and the degree to which they could effectively encode linguistic information appeared to be regionalized and preferentially lateralized to the left hemisphere.

Extended Data Fig. 7. Proportions of selective neurons and predictivities across hemispheres and cortical regions.

a. Proportion of neurons responding to linguistic features by hemisphere and area. Each subplot labels the proportion of selective neurons for each linguistic feature. Here, the neurons are grouped both by hemisphere and brain area. Further, the percentage of selective neurons responding to specific linguistic features was grouped by (b) hemispheres and (c) cortical regions. Population predictivities at the peak layer were determined in relation to (d) hemispheres and (e) cortical regions (mean ± s.e.m.) with dots labelling sessions. To capture the entire contextual window, predictivities were calculated over the full seconds prior to utterance.

Local cortical organization

Finally, we evaluated how the activities of these neurons related to their broader field potential patterns at the local cortical level⁵⁰. Whereas APs reflect the activities of individual neurons (the basic computational units by which information is encoded), LFPs predominantly reflect the broader synchronous activity of the local neural population, including their synaptic and subthreshold membrane potentials. By tracking their corresponding activities on a site-specific level (with each electrode measuring less than 100 µm in diameter and approximately 5 µm at its tip), we find that many of the sites displayed linguistic selectivity based on their recorded LFPs. Specifically, 23% of the sites (n = 303 of the 1,344 electrode contacts across all arrays) displayed linguistic selectivity, showing consistent changes in LFPs to specific linguistic features (for example, parts of speech; Fig. 5). However, we also find that the proportions of LFP sites that displayed selectivity were significantly lower than the proportion of neurons that displayed selectivity (50%; n = 289 out of 579 across all linguistic features; χ² contingency test, χ²₁ = 141.0, P = 1.6 × 10⁻³²) and that most sites in which the LFPs were selective contained no selective neurons (233 out of 303 channels; χ² contingency test, χ²₁ = 2.3, P = 0.13). Moreover, only 19% of the electrode sites contained both neurons and LFPs that encoded the same linguistic features (one-sided permutation test, P = 0.33, n = 1,000, Extended Data Fig. 8). Thus, while many of the recorded sites contained LFP signals that were selective, the linguistic features encoded were often distinct from those encoded by the individual neurons when examined at the μm level.

Fig. 5. Diversity of single-neuronal and LFP activity patterns.

Left, comparison of single-neuronal AP to LFP activities. By having access to both single-neuronal APs and population LFPs from the same microelectrode locations (with each electrode contact measuring approximately less than 100 µm in base diameter and 5 µm at the tip), these two core activity metrics can be compared on a μm scale at a site-specific level. The z scores for all neurons and their LFP activity were recorded at the same microelectrode sites. We then sorted and visualized the single neuronal z scores and applied this same sorting order to the LFP z scores. Middle, the z scores of single-neuronal activities were significantly higher than those from LFPs (two-sided permutation test, P < 10⁻⁴, n = 10⁵, 95% CI = 0.19–0.36). Right, LFP response to linguistic features. We identified overlapping responsive LFP channels by counting the instances in which a single channel demonstrated significant responses to two distinct linguistic features simultaneously. The coincidence matrix shows the percentage of channels that significantly respond to one or more linguistic features.

Extended Data Fig. 8. Linguistic feature decoding based on neuronal AP and LFP activities.

a. Observation-averaged population decoding performance for each linguistic feature by single neuronal (blue) and local field potential (orange) activities. The histograms represent the distribution of decoding performances across different train-test splits. The triangle represents mean decoding performance, and the dotted line represents chance-level performance. b. Coincidence matrix displaying the percentage of sites that displayed selectivity to specific linguistic features based on single-neuronal (y-axis) and LFP (x-axis) activities.

Compared with their local field activities, the neurons also displayed a significantly stronger modulation (defined as z-scored differences in activity; two-sided permutation test, P < 10⁻⁴) and decoding performances for many of the neurons were significantly higher despite their sparse spiking activities (one-sided permutation test, P < 10⁻⁴, n = 10,000 for all features except constituency depth and closing level; Extended Data Fig. 8). Notably, 8.5% (49 out of 579) of the neurons displayed an extreme degree of selectivity to specific linguistic features, as defined by a z score of 2.0 or more⁵¹, whereas almost none of the LFP sites displayed such selectivity (1.2%; χ² contingency test, χ²₁ = 31.5, P = 2.0 × 10⁻⁸; Fig. 5). Thus, in contrast to the LFP signals, many of the neurons appeared to specialize in encoding specific linguistic features based on their tuning properties.

Discussion

Using wide-scale single-neuronal and local field potential recordings from across the human frontotemporal cortex in combination with natural language processing models, parsing and decoding techniques, we identify a finely detailed representation and partitioning of linguistic information during the production of natural speech. Many of the neurons changed their activities selectively based on the linguistic properties of the upcoming words even when they were closely correlated. They could also reliably predict their linguistic properties even when taken from sentences that differed in their context and topic—suggesting a process that could allow linguistic information not only to be encoded at a cellular level but to also be reliably read-out from the activity patterns of the neurons during natural speech.

To construct meaningful speech, linguistic analyses suggest that humans recursively assemble words into larger structures in a way that enables us to convey specific meanings^4,5. Here we found cells that reflected not only basic lexical representations such as parts of speech, but that also tracked the higher-order syntactic structure of the sentences, from how the words grouped into phrases to their phrase mergers and order. These neurons responded selectively to these linguistic features while displaying little response to other features such as the sound-spectral properties of speech suggesting a process that could plausibly support the recursive (for example, hierarchical versus serial^5,16,52,53 or modular⁵⁴) nature of language. They also reveal some of the basic building blocks through which the syntactic structure of language can be robustly encoded and that could enable the generative construction of sentences during language production.

Collectively, we find that these cell populations provided a richly detailed picture of the words, phrases and sentences being produced. These cells uniquely described syntactic features such as the grammatical relationships between words and their parts of speech. They also distinguished the syntactic properties of the words from their semantic meanings, suggesting that these representations were dissociable at a cellular level. When modelled in aggregate, the activities of these populations could also reliably capture the specific combinations of features that described the words, even when decoded from distinct sentences. They could even capture contextual information about the specific sentences being construction, with contextual model predictivities increasing up to a second or more before utterance—revealing long contextual integration windows⁵⁵ that are largely consistent with those predicted by prior behavioural^56,57 and neurophysiological^22,23 studies. Together, such a ‘bookkeeping’ process could therefore allow these cell ensembles not only to provide a combinatorial representation of the words being produced but to also enable their linguistic properties to be encoded at a highly granular level.

Another important finding from these recordings is that information encoded by these neurons was distributed broadly across the frontotemporal cortex. Previous imaging studies and more recent electrocorticographical recordings have demonstrated widespread engagement of areas that are more prominent on the left language-dominant hemisphere. These homotopic areas of activation span large parts of the lateral temporal and prefrontal cortices^7–11 and have been shown to extend well beyond primary speech regions, such as the ventral premotor cortex^41,58. They have also been found to reliably engage in language production and perception by validated language-localizer tasks, together revealing a broad amodal language-selective network of areas that are involved in human speech^28,59. Here, consistent with these studies^28,60, our observations provide tentative evidence for the distributed but left-lateralized and regionalized representation of linguistic information at the cellular scale. They are also consistent with recent animal studies that have suggested the redundant and distributed representation of sensorimotor information⁶¹, suggesting a process that could allow linguistic information to be both widely accessible and concentrated in specialized areas.

Finally, when examined locally, we find an interesting distinction between the encoding properties of individual neurons and their populations’ wider field potential patterns. Specifically, linguistic information represented by neurons was found to be largely distinct from that of their LFPs, even when recorded from the same focal cortical sites. Moreover, in contrast to their surrounding LFP patterns, the activities of many of these neurons appeared to be highly tuned to or specialized in responding to specific features, suggesting a hierarchy of information at a microscale level. By providing site-specific AP-to-LFP comparisons, these findings therefore highlight the heterogeneity of linguistic representations at a local population level and offer a prospective link by which to begin bridging our understanding of language processes across study modalities.

Together, these findings reveal some of the most basic neural elements that underlie human language and offer a prospective approach for studying its cortical organization at a combined micro, meso and macro scale. Given these observations, it is interesting to conjecture whether similar mechanisms or linguistic representations may be shared between speech production and perception, or whether neural activities may be optimally described by certain language processes^5,16,52,53 and models⁵⁴. Further investigations could also determine whether these findings generalize to other communicative modes, such as writing or scripted speech and further clarify how expressive elements like prosody and tone are represented within these contexts^1–5,62,63. Additional studies will be needed to examine neuronal responses from specific eloquent areas, such as the left inferior frontal gyrus, and to record more widely across the brain in order to paint a more complete picture of language production. Here our findings provide an initial starting point from which to begin addressing these questions, and through which to study the basic cellular elements that underpin human language.

Methods

Single-neuronal recordings

Participants

All aspects of the study were approved and performed in strict accordance with guidelines set by the Massachusetts General Hospital Institutional Review Board (IRB) and Harvard Medical School for the ethical conduct of research. All of the participants included in the study were scheduled to undergo epilepsy monitoring^30,33,34,70. The decision for surgery was made by a multidisciplinary team that commonly included neurologists, neurosurgeons, radiologists and neuropsychologists. Operative planning was made independently by the surgical team and without consideration of study participation. Once the decision to undergo surgery was made, the participant’s research candidacy was reviewed. The participants were enrolled only if the surgical plan was for cortical surface grid recordings for seizure monitoring. The patients were aged at least 18 years, and they had intact language function and were able to provide informed consent. A total of eight participants was included in the study (3 female and 5 male, mean age of 40 years; range, 27 to 52 years). All of the participants that were consented were free to withdraw from study participation at any time without impact on clinical care.

Microelectrode recordings

For each participant, multi-electrode microarrays (Utah arrays; Blackrock Microsystems) were implanted into an area planned for surgical resection. These micro-arrays (used for research) were placed within the cortex in tandem with the surface cortical grids (used for clinical monitoring) and were confirmed to be positioned in areas planned for resection based on intraoperative cortical mapping, as described previously^30,71–73. The micro-arrays are configured in a 10 × 10 electrode channel mapping separated by 400 μm (96 recording electrodes covering an approximate area of 4 × 4 mm). For placement, the electrode arrays were implanted using a semi-automated impactor and were connected to a temporarily implanted female port. A total of eight arrays was implanted into eight participants. On the left hemisphere, two were in the anterior temporal cortex, one was in the posterior temporal cortex and one was in the prefrontal cortex. On the right hemisphere, one was in the anterior temporal cortex, two were in the posterior temporal cortex and one was in the prefrontal cortex. Together, these recordings lay in regions that have been shown to reliably engage in language production and sentence construction^{41–43,65,66} and to display robust language-selective responses^{41,42,44,65,74–76} and activation by validated language localizers on imaging studies^28,41,45 (Fig. 1a). Each participant had a single array. The voltages were recorded and amplified via a 96-multichannel acquisition processor, and a band-pass filter was applied to the neuronal signals (150Hz to 8 kHz; 1-pole low-cut and 3-pole high-cut with 1,000× gain). The signals were then digitized at 30 kHz and stored offline. Audio recordings were tracked and aligned through a synchronization trigger that was integrated into both the neuronal and audio recording systems (Natus Medical). After clinical monitoring, all arrays were removed.

Single-unit isolation and LFP processing

To select and cluster APs of putative neurons, an automatic waveform detection was commonly performed using Kilosort 3.0 (https://github.com/MouseLand/Kilosort). The selected APs were then manually curated using Phy (https://phy.readthedocs.io/en/latest/, version 2.0b6). Candidate units were clustered based on their voltage waveform morphologies, spiking rates and their distribution profiles of inter-spike intervals, auto-correlation and cross-correlation profiles. We excluded units that did not demonstrate waveform stability over the course of recordings or that did not have inter-spike-interval profiles or waveform morphologies consistent with those of cortical neurons. Units from different sessions were recorded at separate time periods and across different linguistic contexts and were therefore sorted separately^77–79. Finally, to confirm the quality of recordings, we used SpikeInterface to extract the measures of the single-unit sorting (v.0.102.3). Here the signal-to-noise ratio was defined as the ratio of the maximum amplitude of the mean spike waveform to the s.d. of the background noise on the same channel. Noise was computed through the median absolute deviation of the peak channel. Isolation distances were used to quantify the separation of putative units from their neighbours in principal component space. Here, after projecting spike waveforms into a lower-dimensional principal component space (across 5 principal components), the isolation distances were calculated using the Mahalanobis distance. Lastly, the inter-spike intervals for each unit were defined by the time intervals between consecutive spikes. Any putative units that had a high percentage of ISIs lower than 2 ms or firing rates lower than 0.1 Hz were excluded. Across all the single units, the mean signal-to-noise ratios were 2.5 ± 0.16 (mean ± s.d.), the mean isolation distances were 37.6 ± 2.7 and the mean peak inter-spike intervals (that is, the mean times that the inter-spike interval distributions peaked) were 8.7 ± 0.4 ms and were largely consistent with previous literature using analogous recording techniques^32,80,81.

LFPs were obtained from the same electrode contacts as our single-neuronal recordings. Here, for each electrode of the Utah array (96 active channels in total per array), the raw voltages were processed through a series of notch filters to eliminate the AC powerline at 60 Hz and its higher-order harmonics. Then, a fourth-order Butterworth low-pass filter was applied with a cut-off frequency of 300 Hz to capture LFPs. Finally, the filtered signals were downsampled to 1 kHz for downstream analysis.

Language production and audio recordings

We tracked the activities of neurons as part of real-time conversations to study their responses during natural language production^{42,43,46–48,76,82}. Here the participants responded freely to questions and prompts that were given ad hoc, therefore allowing them to produce phrases and sentences in a manner that reflected natural speech (Extended Data Table 1). The sentences were also constructed de novo (for example, rather than being simply read or repeated) and therefore enabled neuronal responses to be evaluated independently of explicit sensory cues. Thus, the participant may be prompted with a question such as “Do you want a phone right now?”, to which the participant may respond with the compound sentence “Yes, keep it there just in case, and then just bring that back over”. Therefore, in alignment with growing efforts in the field to study natural language^{21,42,43,46–48,76,82–87}, the sentences produced by the participants varied naturalistically in structure and content, were constructed de novo and were representative of natural language production.

Together, the questions heard by the participants could be divided into 20 main topics and covered wide-ranging fields that included: dates and times, feelings and emotions, spatial locations and orientations, personal descriptions, comfort and reassurances, personal thoughts, conditions and situations, thoughts about specific events, general commentary, health, disagreement and personal conflicts, general activities, opinions, validation and agreement, uncertainty about events, passive questions, active questions, binary questions, visual attributes and social greetings. Quantifying the diversity of question topics across the participants, we also calculated the Shannon entropy of the topics for each participant as: $H\left(X\right)=-\sum _{x\in X}p\left(x\right)\log p\left(x\right)$, where x labels the topics (n = 20), and p(x) is the frequency of prompts belonging to each category. Given that each participant received an average of 14.1 ± 4.9 topics and with a theoretical maximum entropy of 3.0, using an even distribution across 20 topics, we find that the entropy ranged from 1.4 to 2.3, with an average of 2.0 ± 0.3 (mean ± s.d.), meaning that the topics were distributed broadly across the participants.

To align each word to the AP activities and LFPs, each word was transcribed and timestamped from the audio data in an automated manner using either Vosk API (16 kHz 16 bit mono; vosk-model-en-us-aspire-0.2) or WhisperX (https://github.com/m-bain/whisperX). After automatic transcription, each word and its timestamp were then manually validated and corrected by an independent scriber (Audacity; v.2.3). Words containing participant-identifying information were replaced by a set of common names that were independent of the participants.

Single-neuronal and local field responses to linguistic features

Sentence parsing and linguistic feature labelling

A sentence parsing approach was used to identify linguistic features on a word-by-word level as produced by the participants during natural speech. This approach, based on natural language processing models, allows for the basic linguistic properties of words (for example, their parts of speech) to be reliably labelled and tracked^1–3,49,88. Here we used two complementary parsing techniques.

Constituency parsing for each sentence was obtained from the AllenNLP package^49,88. Here the word’s part of speech (for example, adverb) was labelled following the Penn Tree Bank, and similar labels were grouped (Extended Data Table 1b; although note that different parsers have also been shown to display differences in labelling)⁸⁹. The word’s constituent label was determined on the basis of which constituent the word belonged to, and was obtained by traversing the parse tree upward from the word to the next highest hierarchical level. The constituency depth of each word in the constituency tree was determined by the word’s hierarchical distance to the sentence root, and the word’s ordinal position was its order within the sentence. Finally, the closing level was determined by the number of finished phrases after the word was produced (Extended Data Fig. 1a). Words that were not located at the rightmost branch of the parse tree were assigned a closing level of 0, as they did not close any syntactic structures. Levels with a single child node were not counted as an extra level.

Dependency parsing for each sentence was similarly obtained from the AllenNLP package^49,88 that followed the model of Deep Biaffine Attention for Neural Dependency Parsing⁹⁰. Built on the verb of a sentence as the root, the dependency tree grows hierarchically with child nodes showing direct dependency on their parent node. Here the grammatical relationship between words (for example, object of a preposition) was determined based on their dependency. To allow comparison across features, the top 10 most frequent categories for each feature were selected for neuronal analysis. For depth, closing level and position, a maximum of 8 was capped on features with excessive layers/number of words with word position then shifted the range to 1–9 (Extended Data Table 1b).

Single-neuronal analysis

Selectivity of neuronal responses

To evaluate selectivity of the neurons to specific linguistic features (for example, parts of speech), we compared their firing rates for words of the target category (such as adverbs) against all words in other categories (for example, all non-adverbs; Extended Data Table 1b, d). This classification was performed for each linguistic category (such as nouns, verbs) and for each neuron. Statistical significance was determined using the one-sided Mann–Whitney U-test (α = 0.05), with P values adjusted using the FDR method for multiple comparisons across categories. To illustrate the specificity of neuronal responses within the population, we combined the selectivity of all neurons and performed a dimension reduction algorithm using UMAP transformation⁹¹. Here to characterize the tuning properties of the neurons during the canonical word planning period²⁷, we used a window from −400 to −100 ms aligned to word onset. However, the precise window was found to have little effect on the proportions of selective neurons identified (−400 ms or −1,100 ms before word onset; χ² test, P > 0.25 for all syntactic features).

Modulation of neuronal responses

While the selectivity of a neuron describes whether the neuron responds preferentially to a specific linguistic feature (for example, adverbs), its neuronal modulation quantifies the magnitude of this response. We calculated the baseline-normalized z scores for individual neuronal firing rates as:

$$Z_i=\frac{{\mu }_i-{\mu }_{\mathrm{other}}}{{\sigma }_{\mathrm{other}}}$$

where the index i labels the firing rates for the linguistic feature to which the neuron is selective, ‘other’ labels the firing rates for all other features; μ labels the mean and σ labels the s.d. To assign a z score to each linguistic feature (for example, parts of speech), we identified the category within that feature that yielded the most significant response (the lowest P value) and selected its corresponding z score.

Controls for generalizability of neuronal responses

To ensure that the selectivity of the neurons was not biased by the particular sets of words, we performed an additional bootstrapping procedure in which we randomly subsampled the words until reaching 80%, 85%, 90% and 95% of the number of words in the original dataset. We then repeated this procedure 100 times to ensure that we broadly sampled across the available words (Extended Data Fig. 1c).

Controls for collinearity with multiple regression analysis

To confirm the selectivity of the neurons and to further confirm that their responses were not influenced by potential collinearities between the features, we performed a multiple regression analysis. Here we used Lasso regression (regularization α = 0.01) to predict neuronal firing rates from one-hot embeddings of linguistic features, thus accounting for the potential effects of multicollinearity between variables^51,92. This yielded standardized coefficients (β) for each feature. Significance was then determined by randomly shuffling the neural firing rates 10,000 times to generate a null distribution of β values. Finally, P values were derived by comparing observed coefficients against this distribution, with a Bonferroni correction applied to account for multiple comparisons. Second, we evaluated for the potential effect of other related properties such as word frequency, length and surprisal (accounting for 7.9%, 3.6% and 11.9% of the neurons, respectively), but found little effect on the proportion of selective neurons (Extended Data Table 1c). Here, word-level surprisal was obtained from the Vicuna-7B model whereby input text for each sentence was tokenized, and token-level surprisal was computed as the negative log-probability given its preceding context (derived from the model’s output logits)⁹³. Word-level surprisal was defined as the sum of surprisals across all subword tokens comprising that word given their sentence context, whereas word frequency was calculated more simply based on the occurrence of each unique word across all the recorded sentences and sessions. Word length was calculated as the number of letters in each word and was top coded at eight characters to reduce the influence of outliers⁹⁴. Word surprisal, frequency and length were all discretized to allow for comparison with the other linguistic features. Finally, we separately identified neurons that maintained consistent selectivity using a sequential feature selection approach that removes correlated features or features that are uninformative^51,95 and again found little effect on the proportion of selective neurons (Extended Data Table 1c reports on the percentages of neurons that were consistently selected in both approaches).

Controls for specificity and overlap between neurons

To further evaluate whether and to what degree the tuning properties of the neurons overlapped, we calculated the percentage of cells that demonstrated significant selectivity for two or more distinct features (for example, parts of speech and dependencies). Here, for every feature pair, we first constructed a neuron × feature binary matrix indicating selectivity for each linguistic feature (Fig. 2a,c). Meanwhile, we independently shuffled the selectivity assignments for each feature across neurons. This preserved the marginal selectivity rates of neurons for each individual feature, while breaking only the joint structure—the association between a neuron being selective for one feature and also being selective for another feature. This permuted data, in turn, is the null distribution needed to test whether co-selectivity is different from expected under independence. Next, we calculated the co-selectivity for each pair of features using the shuffled data. These steps were repeated for 10⁵ iterations to generate a null distribution of co-selectivity values for each feature pair^51,96. The observed overlap for each feature pair was then compared against this null distribution to determine whether it differed significantly from chance-level independence (P < 0.05, two-sided permutation test, Bonferroni corrected for the number of pairs tested). Additional confirmatory analysis of independence included a χ² test and Fisher’s exact test that compare the observed to expected overlapping frequencies of neurons for all possible pairs (P < 0.05, Bonferroni corrected for the number of pairs tested). A χ² test and a Fisher’s exact test were also performed to confirm the robustness of the results. To further examine the relationship between neural activity (the neurons’ firing rates) and feature pairs (parts of speech and dependencies), we first identified the subset of neurons that exhibited significant selectivity to both parts of speech and dependencies. For these neurons, we then calculated their z-scored firing rates by comparing the features to which they were selective versus other categories. Finally, we performed a linear regression analysis in which we predicted the z scores for one feature (dependencies) using the z scores of the other (parts of speech), using the neurons as observations. Here, the P value reflected whether the slope of the linear regression was significantly different from zero.

Controls for lower-order word properties

To evaluate neuronal responses in relation to low-order word properties, we quantified the frequency (pitch) of each uttered word (Parselmouth interface for Praat, v.0.4.3)⁹⁷. The mean pitch was extracted across the duration of each word and aligned to the word production planning period.

Comparing neuronal activities across hemispheres and cortical areas

To allow for comparison across cortical areas (anterior temporal cortices, posterior temporal cortices and posterior frontal cortices) and hemispheres (left and right), we evaluated differences in the proportions of neurons that displayed selectivity using a χ² test. We also evaluate the degree of modulation using Wilcoxon rank-sum test for hemispheres and a Kruskal–Wallis H-test for cortical areas. We next used a one-sided permutation test to further confirm differences in modulation across areas and hemispheres. Finally, two-way ANOVA was used to confirm these findings and to further test potential interaction effects.

Comparing single-neuronal to local field activities

In addition to the APs, we evaluated the LFP activities recorded from each site^50,98–100. We calculated the selectivity and modulation of the LFPs that were recorded across all channels. To further allow for a comparison between single neuronal activities and LFPs^50,101, we spatially aligned the activity of each neuron with the specific LFP channels from which they were recorded. We then sorted and visualized the single-neuronal z scores and applied this same sorting order to the LFP z scores (Fig. 5). A χ² contingency test was used to evaluate the proportions of neurons and LFP sites that displayed selectivity and a one-sided permutation test was used to evaluate the overlap in linguistic selectivities (n = 1,000). A two-sided permutation test was used to compare their degrees of modulation (n = 10,000).

Neuronal decoding analysis

Decoding linguistic features from neural activity

Decoding model

A multinomial logistic regression classifier (Scikit-learn package v.0.21.3) was used to decode the linguistic features from neuronal activity with L2 regularization to minimize the loss function as

$$\underset{\mathbf{w},c}{\text{min}}\frac{1}{2}{\mathbf{w}}^T\mathbf{w}+C\sum _{i=1}^n\text{log}\left(\text{exp}\left(-y_i\left({\mathbf{x}}_i^T\mathbf{w}+c\right)\right)+1\right)$$

where C is the inverse of the regularization strength; vector w are coefficients and c is the bias; x_i represents the feature vector of neuronal firing rates, $y_i\in [-\mathrm{1,1}]$ labels the actual category for word index $i\in [1..n]$, which labels the number of observations (words). We used balanced class weights, with a regularization strength of 0.0001. The mean decoding accuracies were obtained from the independent observations on the test dataset.

Neural population decoding

As described previously^27,102–105, to perform the population-level decoding analysis, we aggregated neuronal activity across all participants and sessions. Here we performed a stratified sampling to split the training and testing datasets^51,106. Specifically, for a given feature (for example, parts of speech), we randomly selected 70% of trials belonging to a specific category (such as nouns) for the training set and an independent 30% of trials for the testing set. These were repeated 1,000 times with independently drawn random partitions on each iteration for the non-shuffled data. The neural-feature matrix $X_{\mathrm{combined}}=\left[X^{\left(1\right)}\right|X^{\left(2\right)}|\cdots X^{\left(S\right)}]\mathit{ϵ}{\mathrm{ℝ}}^{{W\times N}_{\mathrm{total}}}$, where $N_{\mathrm{total}}={\sum }_{s=1}^S{N}_s$ is the total number of significant neurons pooled across S sessions, and W labels the number of words. $X^{\left(s\right)}$ is composed by $x_{w,n}^{\left(s\right)}$, which is the firing rate of neuron n in session s for word w. The mean decoding accuracy was obtained as the average decoding performance across the sampling iterations.

Thus, whereas the individual words may differ across sessions, their categories (for example, words that are nouns) could be directly decoded and compared. The neural activity vectors for these subsampled words were then horizontally concatenated across all selective neurons from all participants and sessions to create a unified training and testing matrix, in which the rows represent the aggregated feature observations and the columns represent the combined pool of relevant neurons from all sessions. This process was then repeated for every category (such as nouns) per linguistic feature (parts of speech), with the resulting matrices vertically concatenated to form the final datasets. Finally, to assess statistical significance and estimate chance-level performance, we conducted a permutation test by randomly shuffling the feature labels and repeating the entire modelling procedure to generate a null distribution (n = 1,000). The P value for the permutation test was then derived by calculating the frequency with which the shuffled accuracies exceeded the actual decoding accuracy (P < 0.05; Fig. 2d).

Single and bootstrapped observations

All primary decoding results in the main text, including those shown in Fig. 2, are provided for individual words. However, to further account for the sparse spiking of neurons per word and to ensure high fidelity of decoding^27,107–110, we also generated new observations within the same category by randomly selecting and averaging the neural firing rates across ten observations, and then averaged the firing rates to create a new observation. This procedure therefore further accounts for the neurons’ sparse spiking per word while importantly preserving the linguistic features being decoded. To account for the difference of word numbers in each section, we used this bootstrapping with replacement to standardize the number of observations per category across session. Here the above process was repeated until we created the same number of observations (500 new observations per unique category for the training dataset and 200 for the testing set). This procedure, including the randomized data partitioning and bootstrapping, was repeated for 1,000 iterations to ensure robust estimation and to account for variability in the sampling process. Finally, we confirmed the consistency of observation-averaged decoding by repeating the bootstrapping procedure but varying the number of bootstrapped observations between 10, 20 and 30 (Extended Data Fig. 2a) as well as by selecting neurons that specifically displayed selectivity on model training (Extended Data Fig. 2b) and by selecting only neurons that displayed selectivity for individual participants (Extended Data Fig. 2e). Unless specified, all decoding analyses were performed using observation-averaged activity.

Generalizability of linguistic representations

To examine the generalizability of the linguistic representations across contexts, we used Vicuna model embeddings to cluster the sentences into topics based on their model embeddings, as described previously^111,112. This step clusters the sentences into distinct topics based not only on the lexical meanings of the words but also on their sentence contents and contexts. We used the top-most layer of the model’s hidden embeddings for all words per sentence, and categorize into three or four equal clusters (k-means; Extended Data Fig. 2f).

To further confirm the generalizability of linguistic representations across the participants, we performed a bootstrapping procedure with 5 observations, then trained a classifier on a random selection of 50 observations for each category, and tested on randomly selected 20 for each category. This process was then repeated with 50 repetitions for each session. We then applied a Friedman test to examine whether dropping one participant significantly impacts the observation-averaged decoding accuracy. We also confirmed the relationship between the decoding performance and the number of neurons recorded^27,113 by randomly sampling cells from across the participants and showing that the number of cells correlated with the decoding performance (Extended Data Fig. 2c). Finally, for further comparison, we repeated these analyses but now evaluated decoding performances across individual cortical areas and hemispheres.

Language models

Syntactic and semantic embedding models

Embedding models were used to capture the combination of linguistic features that described the specific words being produced. For example, even though the word ‘dog’ in the sentence ‘The dog ate the bone’ and ‘The man walked the dog’ is the same, the models distinguish its varying syntactic roles—such as nominal subject versus object—as well as the distinct semantic representations between different nouns like ‘dog’ and ‘bone’^1–3,49,88. To first examine the combination of syntactic features that defined each word, we constructed embedding models that tracked all features obtained from the constituency and dependency parsers by representing each category (for example, nouns within parts of speech) with a one-hot vector of zeros and ones, and concatenating them for all syntactic features. This combination of features therefore provided a comprehensive representation of all tested features per word (Fig. 3a). Here, the embedding vector $\mathbf{E}\left(w\right)$ was written as

$${\mathbf{E}}_{\mathrm{syn}}\left(w\right)={\mathbf{v}}_1\left(w\right)\oplus {\mathbf{v}}_2\left(w\right)\oplus \cdots \oplus {\mathbf{v}}_N\left(w\right)$$

where w is the word for which the embedding is being constructed and v_i(w) is one-hot vector representation of the specific value of the ith feature category for word w.

Second, we examined the lexical semantic properties of each word²⁷ using a pretrained semantic embedding model (word2vec)¹¹⁴ to obtain a vectoral representation for each word. This model maps the semantic meaning of words onto an embedding space in which words that have different meanings also have different vectors. Embeddings were obtained from the word2vec model trained on the Google dataset (about 100 billion words, gensim, v.4.3.2). Out-of-vocabulary words (approximately 7% of all words) were excluded. We also verified that zero-vector padding of these out-of-vocabulary words had little effect on predictivity results (two-sided permutation test, n = 10⁵; P = 0.37 and P = 0.60 for semantic only and combined embeddings; see Language model predictivity).

Finally, to examine the words’ semantic and syntactic properties, we combined the syntactic and semantic vectors to form a shared embedding model which was defined here as

$${\mathbf{E}}_{\mathrm{comb}}\left(w\right)={\mathbf{E}}_{\mathrm{syn}}\left(w\right)\oplus {\mathbf{E}}_{\mathrm{sem}}\left(w\right)$$

where E_sem is the semantic embedding vector for word w.

Higher-order contextual embedding models

While the above embedding models enabled us to examine how combinations of features are encoded by neurons on a word-by-word level, they did not determine whether the neurons also reflected their specific contexts (for example, the word ‘bark’ within ‘The dog has a loud bark’ versus ‘The tree has thick bark’)^6,115–118. Therefore, to further address this question, we also used contextual models that were capable of further capturing these contextual variations^22,23. Here we used Vicuna-7B (v.1.1), a large language model that was fine-tuned from the LLaMA architecture to serve as a contextual framework for generating high-dimensional vector embeddings. Vicuna-7B is an open-source autoregressive language model based on a transformer architecture and consists of 32 transformer layers and 4,096 embedding dimensions and was further tuned on large-scale, user-shared conversations.

Here the contextual model embeddings were obtained by inputting words from each sentence produced by the participants, such that each sentence was associated with corresponding hidden embeddings for each layer^23,42. Next, to extract word embeddings across varying contextual lengths for each sentence (Fig. 3c), the word strings were generated by sliding a window of size N across the preceding word sequence. These word strings were then processed through the Vicuna-7B model to extract hidden state representations for all layers:

$${\mathbf{E}}_{\mathrm{contextual}}\left(w_i,L\right)={\mathbf{H}}_L\left(w\right)$$

where H_L(w) is the hidden state vector extracted from layer L of the model for word sequence $\left\{w_1,w_2,\cdots ,w_N\right\}$. In combination, this approach therefore allowed us to systematically manipulate the amount of preceding context provided to the model, ranging from two words (that is, N = 2) to extended ten-word sequences. The hidden state activations across all layers were then extracted at the last token and concatenated, thus providing contextual information for each word in relation to its specific sentence context and its varying context lengths. Further validation experiments include using a diverse suite of large language models (Extended Data Fig. 3 and Extended Data Table 2). All models and tokenizers were loaded through Huggingface in Python¹¹⁹ and run with a NVIDIA GeForce RTX 4090 GPU.

Predictivities of neurons across language models

Language model predictivity

We examined the degree to which embeddings from each of the language models (syntactic, semantic and higher-order contextual models) were predictive of the neurons’ activity patterns. First, to allow for comparison across models and to limit overfitting, we first performed principal component analysis (optimal dimensionality was determined as 30 principal components; Extended Data Fig. 4a) from each model layer (constrained within a range of [−10,000, 10,000] to mitigate the influence of outliers). These matrices were then temporally assigned using non-overlapping 100 ms bins using embeddings for the bins that occupied each word. Similarly, we calculated the neural firing rates for individual neurons by binning their APs into 100 ms intervals. The overall population neural activity was calculated from the combined firing rate patterns of all neurons and were smoothed by a three-bin moving average.

Extended Data Fig. 4. Predictivity across models and their linguistic feature mappings.

a. Population predictivity as a function of PCs. Neuronal predictivities at the peak layer were calculated by increasing the number of principal components of contextual model embeddings. b. Population predictivity by model size. Contextual model predictivities at the peak layer are grouped by the sizes of the models. Across the models, there is no significant difference in predictivity in relation to model size (linear regression test, t(10) = 1.27, Pearson’s r = 0.37, p = 0.23). Here, the number of parameters (b) is counted in the billions. c. Individual neurons were categorized based on their linguistic selectivity, independent of temporal offsets. The resulting plot illustrates how average predictivities of individual neurons in their peak encoding layer vary based on the neurons’ selectivity over time for all selective neurons. d. Predictivity of individual neurons based on their selectivity to specific linguistic features. Here, the peak predictivity of neurons and the degree to which they were predictive are colour-coded based on their response selectivity to specific linguistic features. The bar plot on the top illustrates the counts of all selective neurons per layer in relation to peak predictivity.

Next, we calculated predictivity¹²⁰, which is the Pearson correlation coefficient between the language model’s predicted neuronal activities and the observed neuronal activity:

$$\text{Predictivity}=\frac{\mathrm{cov}(y,\hat{y})}{\sqrt{\mathrm{var}\left(y\right)\times \mathrm{var}\left(\hat{\mathit{y}}\right)}}$$

where y labels the actual neuronal firing rates, and $\hat{y}$ labels the predicted neuronal firing rates based on the model’s embeddings. cov($·$) labels the covariance matrix, and $\mathrm{var}(·)$ labels the variance.

Finally, to predict neuronal activity from the models’ embeddings, we performed a Ridge regression (Python scikit-learn package; v.0.21.3) using the dimensionally reduced embeddings from each contextual model and layer to minimize the cost function:

$$\underset{\mathbf{w},c}{\text{min}}\alpha {\mathbf{w}}^T\mathbf{w}+\sum _{i=1}^n(y_i-({\mathit{x}}_i^T\mathbf{w}+c){)}^2$$

where y_i is the neuronal activity for observation $i\in [1..n]$; vector w are coefficients and c is the bias; x_i represents the feature vector of language model embeddings; α is the regularization strength that was set as 0.01. The predictivities were then calculated separately for each session. This approach therefore enabled us not only to obtain an aggregate account of the predictivities of all recorded neurons but also to evaluate their consistency across participants and sessions. The regression weights were trained using data from all but one sentence within each session, and were then used to predict neuronal responses for the withheld sentence. By repeating this process across all sentences, we obtain a full set of predicted activities for each layer of the model.

For single-neuronal predictivities, we calculated the predictivity from each individual neuron’s firing rates and then averaged their predictivities (Fig. 3e (right) and 3f), while the mean population predictivities were obtained by calculating the predictivity from the mean firing rates from all neuronal firing rates in a session and then averaging across sessions (weighted by the number of neurons; Fig. 3e (left)). Here predictivities in Fig. 3b were calculated from the averages during the pre-utterance period and were evaluated with fixed temporal offsets that aligned neuronal activity (obtained before utterance).

To establish a baseline chance performance for predictivity, we generated a null distribution by randomly permuting the average neuronal activities and recalculating the predictivity on this shuffled dataset. This randomization procedure was conducted for the same number of iterations as the primary modelling analysis. We then performed a one-sided permutation test to determine whether the predictivity of the actual data was significantly higher than that of the shuffled control. For comparison of predictivities across hemispheres and brain areas, we used a rank-sum test and a Kruskal–Wallis test, respectively.

Neural predictivity controls

To confirm that the predictivities reflected information specifically captured by the language models, we performed an additional control analysis using randomly generated non-meaningful stimuli as inputs to the contextual model. Here the tokenizer was replaced with a random drawing of a number (index of a token) to ensure that no linguistically meaningful information was input into the model. Furthermore, to confirm that the predictivity reflects the contextual information specific to each sentence and cannot be simply explained by the order of words, we also performed the swap sentence control. We first grouped the sentences produced by the participants based on their word length to ensure matched comparisons. Within these word-length-matched groups, we then randomly shuffled the sentences. These swapped-sentence embeddings were then used to calculate the predictivity of neuronal population activity. To allow for consistency with our contextual predictivity analyses above, both of these controls were performed at the population level, and both results were reported at −1,000 ms before utterance.

Neural predictivity dynamics and mapping

To characterize the temporal evolution of neural predictivity, we systematically shifted the temporal alignment between neuronal firing rates and contextual model embeddings by introducing a series of lead and lag offsets in 100 ms increments. This approach enabled us to identify the specific latency at which neural activity most strongly reflects the linguistic information encoded in the contextual model embeddings across a wide temporal window, spanning from 2,000 ms before word onset (pre-articulatory planning) to 2,000 ms after onset (post-articulatory processing). This was applied to both the population’s firing rates (Fig. 3e (left)) and the individual neuronal firing rates (Fig. 3e (right)).

Finally, to illustrate the relationship between the neurons’ peak predictivity and their linguistic feature selectivity, we calculated the neuron’s average predictivity for each model layer and time point. This approach therefore enabled us to map the layer and temporal lag at which neuronal predictivity peaked for each neuron based on its encoding properties (Fig. 3f and Extended Data Fig. 4). For all contextual predictivity results in Fig. 3c,d and Extended Data Figs. 3 and 4, we used the −1,000 ms timepoint relative to the embeddings, which was determined as the approximate predictivity peak observed in our time-resolved population analysis. However, the exact timing used for analyses was not critical as the contextual model significantly outperformed the other syntactic, semantic and combined models (one-sided permutation test, P < 0.004 at −1,000 ms across all comparisons; P < 0.042 at −400 ms to −100 ms interval across all comparisons; n = 10⁵) and exceeded chance for both tested intervals (one-sided permutation test, P < 10⁻⁵, n = 10⁵).

Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.

Online content

Any methods, additional references, Nature Portfolio reporting summaries, source data, extended data, supplementary information, acknowledgements, peer review information; details of author contributions and competing interests; and statements of data and code availability are available at 10.1038/s41586-026-10691-5.

Extended data figures and tables

Extended Data Fig. 5. Predictivity of individual neurons.

Each subplot indicates the contextual model predictivity of a single neuron, with the x-axis labels time and the y-axis labels the layer. Neurons are arranged based on the time at which maximum predictivity occurs.

Author contributions

J.C. performed the analysis. J.C., S.S.C. and Z.M.W. obtained and curated the data. J.C., Y.K., M.J., H.H. and Y.J.K. processed the data. J.C. and M.J. devised the parsing approach. J.C. and Z.M.W. designed the study and wrote the manuscript. Z.M.W. developed and directed the project and supervised all aspects of the research.

Peer review

Peer review information

Nature thanks Andrea Martin, Matthew Nelson and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Funding

J.C. is supported by AAUM and Mussallem Transformative Scholar Award; M.J. by CIHR, the NARSAD Young Investigator Grant and the Foundations of Human Behavior Initiative; S.S.C. by R44MH125700 and the Tiny Blue Dot Foundation; and Z.M.W. by NIH R01DC019653 and U01NS121616.

Data availability

All primary data supporting the main findings of this study are available at Zenodo¹²¹ (10.5281/zenodo.20128462).

Code availability

All code necessary for reproducing the main findings of this study are available at Zenodo¹²¹ (10.5281/zenodo.20128462). All software used in this study are listed in the Reporting Summary.

Competing interests

The authors declare no competing interests.

Extended data

is available for this paper at 10.1038/s41586-026-10691-5.

Supplementary information

The online version contains supplementary material available at 10.1038/s41586-026-10691-5.