Family identity is represented within vocal categories by the gerbil auditory cortex

SUMMARY

Vocalizations are integral to social communication, with acoustic features that may convey both meaning and information about an animal’s identity or emotional state. The neural representation of vocalizations must therefore permit animals to generalize across one set of acoustic features to recognize meaning, and a different set of features to recognize vocalizer identity. To test this idea, we recorded the responses of gerbil core auditory cortex (AC) neurons to a large array (n>1500) of variants drawn from four vocalization categories and produced by 5 different families. Each vocalization category could be decoded from a small AC population with high accuracy, despite the acoustic variance across vocalization renditions of different families-of-origin. Further, a larger AC population was required to decode family identity, using a unique set of acoustic features for each category. Thus, AC activity can be used to simultaneously predict vocalization category and family identity, and is robust to the natural range of acoustic variance.

eTOC blurb:

in ‘t Zandt et al. find that auditory cortex populations can both generalize across hundreds of vocalizations to represent call categories, while simultaneously parsing family identity information. Overlapping populations of different sizes are used to extract these different types of information, revealing a novel computation underlying naturalistic vocalization processing.

INTRODUCTION

Vocalizations often convey many pieces of information simultaneously. For example, spoken words convey meaning, while also providing pivotal information about the talker’s identity^1,2. In fact, acoustic variance is a central feature of vocal communication that allows any single speech sound to convey both meaning and social information^3–7. A vowel’s formant frequencies can vary as a function of the specific talker⁸, voice fundamental frequency can vary as a function of talker sex⁹, and voice onset time can differ between regional accents¹⁰ – each conveying a talker’s traits (e.g., regional origin, status, age, size). While acoustic variance increases the informational capacity of speech, it also creates a challenge for auditory processing: the nervous system must represent meaning associated with a vocal category by generalizing across inter-talker acoustic variance, while also representing identifying features by discriminating between each individual’s variance. Thus, to understand how the nervous system represents both vocalization categories and identity information, it is essential to use a large array of vocalizations with a natural range of acoustic variance^5,11,12.

Vocalizations can convey meaningful information in many species, such as alarm calls that signal nearby danger¹³ and rat ultrasonic calls signal emotional state¹⁴. Animals also rely on vocalizations to navigate complex social landscapes. Many acoustic features (e.g. frequency, duration, envelope) provide identifying information about individuals or group membership^15–19, and these features may differ across vocalization categories^4,17,20. Thus, natural vocal processing involves the simultaneous extraction of both meaning and identity information from a variable set of acoustic signals.

Robust responses to vocalizations are measured throughout the auditory neuraxis^21–27, yet single neurons with highly selective responses are uncommon^28,29. Rather, the representation of vocalization categories is typically associated with population codes^21,25,30. Auditory cortex (AC) population codes can independently discriminate phonemes³¹ and social traits, such as age³², from vocalizations despite significant acoustic variability. While these studies point to AC as an important locus for vocalization processing, they do not characterize both vocalization category and social information for the same neural population. In contrast, naturalistic social communication requires the simultaneous processing of meaning and identity information. Therefore, whether and how the AC represents social information across many categories remains uncertain.

Mongolian gerbils (Meriones unguiculatus) are a highly social species, with a rich repertoire of acoustically-distinct vocalizations that are associated with unique behaviors^33,34. The usage of this repertoire differs between families³⁵, suggesting that gerbil vocalizations may convey information about an animal’s family identity. Here, we asked whether gerbil AC neuron populations can simultaneously support vocal category discrimination, while also retaining the ability to distinguish between the family-of-origin for each vocalization. To test this, we performed wireless recordings from populations of AC single neurons in adult Mongolian gerbils while presenting >1500 vocalizations drawn from four different categories and 3 different families.

RESULTS

Gerbil vocalizations can be acoustically classified by category and family identity

To test whether auditory cortex (AC) activity can be used to categorize gerbil vocalizations, we used a large stimulus set with a comprehensive range of natural acoustic variance. Since gerbils exhibit family-specific usage of vocalization categories³⁵, we obtained our stimulus set by recording vocalizations from 5 separate gerbil families (2 parents, 7–9 pups, postnatal (P) day 25–27). Vocalizations were first identified using a supervised deep neural network (DAS³⁶) trained to identify individual, non-overlapping vocalizations (model performance F1 score: 0.77; Figure S1A-B). An unsupervised approach was then used to determine whether their acoustic features differed across families. Specifically, spectrograms of each individual syllable (i.e. variants) were embedded in a 32-dimensional latent space using a variational autoencoder (Figure S1C-D; see STAR METHODS³⁷). A Gaussian mixture model was applied to these latents to generate clusters with similar acoustic attributes. A subset of clusters were then manually merged into broader groups that replicated vocal categories described in Peterson et al. (2024). Our experiments focused on 4 vocalization categories: combination (Combo), downward frequency modulated (DFM), upward frequency modulated (UFM), and Warbles (Figure 1A-B). We extracted 50–250 variants per family per category, yielding a stimulus set of >2500 unique variants (of which ~1500 were used in any single recording session).

Figure 1: The acoustic attributes of gerbil vocalizations can be classified by both category and family identity.

A. Schematic of enlarged home cage used for overnight vocalization recordings. B. Spectrograms of variants from each of 4 vocalization categories from each of 5 families. C. UMAP representation of vocal latents. Colors correspond to class labels in B. D. Random forest classifier performance on predicting vocalization categories trained on the vocalization latents. Error bars = 1 standard deviation (SD) across 100 SVM iterations. E. Random forest performance on predicting family identity trained on the vocalization latents. Classifier performance shown for 5, 3, and 2 families’ vocalizations. Family-ID classifiers were trained on a single category at a time. Error bars = 1 SD. See also Figure S1.

To visualize how vocalization acoustics were distributed across the four categories, we used a UMAP representation to project the latents that explained the most variance into a 2D space (Figure 1C). We quantified how accurately the vocalizations could be classified into these 4 categories by training a random forest classifier on the latent representations of the vocalizations (see STAR METHODS). Across 100 iterations of randomly sampled train and test sets, the classifier performed 96–98% on all four vocal categories (Figure 1D), indicating that family-invariant vocal categories can be described by their acoustics across hundreds of variants.

Although it is known that gerbils exhibit family-specific usage of vocalization clusters³⁵, it is not known whether the acoustic properties of any vocalization category differ between families. To test whether vocalizations within each of the four categories could be classified by family-of-origin, we performed the classifier analyses described above, but trained the models for one vocalization category at a time and used family identity as the classifier labels. When trained on randomly sampled vocalization latents from all 5 families, family identity could be decoded with 35–43% accuracy, as compared to chance performance of 20% (Figure 1E, left). Across all neural experiments, two families were kept constant for all subject animals. When trained with just these 2 families, family identity could be decoded with >65% accuracy across all four vocalization categories, as compared to 50% chance (Figure 1E, right). These results indicate that gerbil family identity information is embedded within the acoustics of single vocalization categories, although this information is not as robust as category itself.

Since the latents represent abstract, uncorrelated components of the stimulus set, we compared the above analyses to acoustic features that have been studied in the auditory neuroscience literature^21,25,29,38. In fact, vocalization category and family identity each displayed distinct acoustic features, as computed on the filtered waveforms (Figure 2A). Using the classifier procedure, vocalization category could be decoded with 90–96% accuracy (Figure 2B) and family identity could be decoded with 68–78% accuracy (Figure 2C). We examined the importance of each feature to the classification and found that start, stop, and median frequency best explained the acoustic differences across categories, while the acoustic differences across families were distributed across multiple features (Figure 2D).

Figure 2: Acoustic features of the stimulus set.

A. Distributions of acoustic features calculated for each syllable. Top row: Distributions across categories. Rows 2–5: Distributions within category, across Family 1 and Family 2. B. Mean random forest (RF) classification of vocal category using the 12 acoustic features. C. Mean random forest (RF) classification of family identity (per category) using the 12 acoustic features. D. Relative feature importance of acoustic features to category and family classification.

Heterogeneous vocalization responses underlie AC neuron category-selectivity

Since the acoustic features of each vocalization could be classified based on category and family-of origin (Figures 1–2), we next asked how well they could be classified by AC neuron activity. A 64-channel silicon probe was implanted in the left AC of adult gerbils (n=5; see STAR METHODS) and, following recovery, telemetric recordings were obtained from awake, freely moving animals while the stimulus set was presented from an overhead speaker (Figure 3A). As illustrated in Figure 3B, the stimulus set consisted of the four vocalization categories described above, vocalization bouts, noise stimuli, and pure tones. For each vocalization category, variants from 3 families were presented, with one of the families always being the subject animal’s own family. Thus, each neuron was stimulated with ~1500 unique variants. Vocalizations were presented in 5-minute blocks of the same category, with family identity of the variants randomly shuffled throughout these blocks. Each individual variant was presented 5 times.

Figure 3: Heterogeneous responses of AC units to gerbil vocal categories.

A. Schematic of gerbil implanted with wireless electrophysiology system. B. Stimulus presentation for the whole experimental session (F1, F2, & F3 represent Family ID 1, 2, & 3). C. Overlays of probe tracts on the AC sections from the Radtke-Schuller⁷⁰ stereotaxic axis. D. Two example single units’ (SU) responses to all trials within a vocal category. Top: Mean peri-stimulus time histogram (PSTH) to all trials in a given vocal category (shading = 95% confidence interval). Bottom: Raster plot of spike times on every trial. Inset: Mean waveform of the single unit. E. Example SU #2 UFM responses, with trials reordered based on individual variant mean zFR. Left: PSTH and raster plot, ordered by highest to lowest mean zFR. Right: Zoomed in to top 10 variants with highest mean zFR across the 5 trials of each variant. Grey and white shading delineates boundaries between unique variants. F. Quantification of percentage of AC population that responds to increasing number of vocal categories. G. Quantification of percentage of responsive AC population that responds significantly to each vocal category. H. Distribution of selectivity index of all responsive units. I. Quantification of within-category versus across-category coefficient of variation on category-responsive neurons’ firing rates. Each line = 1 category-responsive neuron. Error bars = SEM across all category-responsive neurons. (***p<0.01, paired t-test). J. Quantification of the percent of unique variants that category-responsive neurons respond to. Grey circles = individual gerbils. Error bars = SEM across gerbils.

We recorded auditory responses from 857 AC single units across the 5 subject animals (n=53–299/animal; mean = 171.4, SD = 85.4), confirmed by referencing the electrode tracts to the gerbil stereotaxic atlas⁷⁰ (Figure 3C). We observed a range of single unit responses to the vocalization categories with some neurons responding reliably to all four categories (Figure 3D, top), and others showing high vocalization selectivity (Figure 3D, bottom). Within the entire population, 37% (n=316) of AC neurons exhibited a significant change in firing rate to at least one vocalization category (“vocalization-responsive”; paired t-test between spontaneous and vocalization-evoked firing rates with Bonferroni correction; p<2.5e-3). “Category-selective” neurons that responded significantly to only one category comprised 15% (n=125) of the population (Figure 3F). The remaining 22% (n=191) of neurons responded to 2 or more categories. The majority of vocalization-responsive neurons responded to the Combo and DFM vocalizations, with smaller number of neurons responding to UFMs and Warbles (Figure 3G). For vocalization-responsive units, a selectivity index (SI) was computed to quantify the specificity of category responses. An SI of 1 indicates that the unit responded to only 1 category, while an SI of 0 indicates the unit responded equally to all 4 categories. Figure 3D illustrates neurons with SI values of 0.41 and 0.92. The median SI for the entire vocalization-responsive population was 0.72 (Figure 3H), suggesting that vocalization-responsive AC neurons displayed some category selectivity.

The above analyses used the mean category responses, but ignored individual variant responses. Figure 3E displays one AC neuron’s responses to each variant, sorted by the response magnitude (Figure 3E), and illustrates two characteristics: First, AC neurons responded reliably to the same call across a recording session, as assessed by the lower coefficients of variation for the trials of the same call versus different calls of the same category (Figure 3I). Second, strongly-responsive units were activated by only ~50% of unique variants within a category (Figure 3J). These results suggest sparse category-encoding at the single neuron level.

Single AC neurons can be modulated by family identity

Given the disparity of within-category responses described above, we asked whether AC neuron responses could plausibly contribute to the encoding of family identity. Many single AC neurons were modulated by family identity for one or all vocalization categories tested (Figure 4A-B). Of the category-responsive neurons, 22% (n=68) displayed a significantly modulated firing rate to family identity for at least one vocal category (one-way ANOVA with Bonferroni correction; p<2.5e-3). However, the proportion of AC neurons that was responsive to own-family, as compared to other-family, was not significantly different for any vocalization category (Figure 4C). Further, the median selectivity index for family identity was low, ranging from 0.24–0.29 across vocal categories (Figure 4D).

Figure 4: Single units in AC are modulated by family identity of vocalizations.

A. Example single unit response to all trials of Combos produced by 3 gerbil families. Top: Mean peri-stimulus time histogram (PSTH) to all trials in a given vocal category. Bottom: Raster plot of spike times on every trial. B. Eight example single unit average PSTHs to all variants in a vocalization category, separated by which family produced the variants (shading = 95% confidence interval). C. Percent of population that is responsive to own v other family for each vocalization category. Colored points and error bars = mean and 95% confidence interval across all animals. Grey lines = individual animals. D. Selectivity index of category-responsive cells to family identity.

Vocalization category and family identity can be simultaneously decoded from AC populations

To ask how well vocal category and family identity could be decoded from AC population activity, we built a linear support vector machine (SVM) decoder using single trial, z-scored 30ms PSTHs. Figure 5A provides a visualization of the z-scored PSTHs for the vocalization-responsive pseudopopulation, with all PSTHs for a single category averaged for each neuron. Note that this visualization does not capture the variant-to-variant differences in PSTHs that are present within our test data (Figures 3D-E, 4A-B, S2A-B). To compare vocalization category and family identity decoding, these analyses were performed in parallel such that the same trials were used for each decoder. Single-trial PSTHs (300 trials per family per category) for the entire population were randomly sampled and split into train and test sets. The vocalization category SVM was trained to form 4 linear hyperplanes that best separated the population activity for one category from the activity of the 3 other categories (Figure 5B, top). In order to perform family identity decoding for our pseudopopulation, we restricted the analysis to the 2 families that all animals had in their stimulus set (since ‘own’ family was not the same for all animals). Therefore, the family identity SVM formed one hyperplane that best separated Family 1 and Family 2 within the same vocal category (Figure 5B, bottom). This procedure was performed 100 times with new subsamples of the trials. Since the decoders performed either equally or better when restricting the population to the vocalization-responsive units (Figure S2C-D), only vocalization-responsive units were used in subsequent analyses. Further, the PSTH decoders always outperformed a simple firing rate decoder (Figure S2E-F).

Figure 5: Vocalization categories and family identity can be decoded from AC population activity.

A. Category-averaged 30ms-bin PSTHs (zFR = z-scored firing rate) for every vocalization-responsive neuron (n=316). Neurons are sorted by the vocalization category they showed the largest response to. B. Schematic of linear SVM classifiers (top: vocalization category, bottom: family ID for 1 category at a time). C. Vocalization category SVM performance (n=100 iterations resampling 300 trials from each category). Colors = pseudopopulation performance (n=316 neurons). Light grey circles = individual gerbil performance. Error bars = 1 SD across 100 iterations. Chance = 25%. Dark grey circle and error bars = SVM trained on shuffled labels. D. Vocalization category SVM performance versus number of neurons sampled from the population. Error bars = 1 SD across 20 iterations. Solid line = sigmoidal fit to the means. Dashed line = inflection point of the fit. E. Family ID SVM performance (n=100 iterations resampling 300 trials from each family, within a category). Colors = pseudopopulation performance (n=857 neurons). Light grey circles = individual gerbil performance. Error bars = 1 SD across 100 iterations. Chance = 50%. Dark grey circle and error bars = SVM trained on shuffled labels. F. Family ID SVM performance versus number of neurons sampled from the population, for each vocal category. Error bars = 1 SD across 100 iterations. Solid line = sigmoidal fit to the means. Dashed lines = inflection points of the fit. See also Figure S2, S3, S5.

We found that both vocalization category and family identity could be decoded from AC activity significantly above chance, despite being tested with hundreds of unique vocalization variants. Vocalization category performance was 89–94% across the 4 categories (Figure 5C; one-sample t-test p<4.05e-163), close to the upper limit defined by the acoustic classifier in Figures 1D and 2B. Even in individual gerbils, with responsive neural sample sizes of 25–89 single units, performance was significantly above chance, demonstrating robust vocal category representations by AC populations. Further, knocking out any single animal’s neurons from the population still yielded performance significantly higher than chance (one-sample t-test p<1.5e-65, Figure S5D). There were differences in performance across the four vocalization categories, with performance ranging from 89% for the Combos to 94% for the Warbles. Combos may be more difficult to classify because they include acoustic features from DFMs and Warbles. We further asked how many neurons are required to perform vocalization category classification by subsampling the pseudopopulation with increasing population sizes (Figure 5D). We fit the mean performance with a sigmoidal function to determine the inflection point, and found that 37 neurons were required to perform 62% correct.

Across all 4 vocalization categories, the AC population classified family identity with 68–78% accuracy, significantly above the chance performance of 50% (one-sample t-test p<1.5e-65; Figure 5E). This was comparable to decoding of family identity from the latents that represent individual vocalization acoustic features (Figure 1E, right). For most vocalization categories, family identity could be decoded from an individual gerbils’ AC activity, and even after knocking out a single animal’s neurons from the population (Figure S5E). Family identifying features were present in all of the vocalization categories, although the SVM performed best for Combo vocalizations. The population size needed to perform family ID decoding across vocalization categories ranged from 167–178 units (Figure 5F), approximately 5 times as many that were required to decode vocalization category, suggesting that these features are more difficult for AC neurons to represent.

Since both vocalization category and family identity could be decoded from AC activity, we assessed the dimensionality of the population representations which can have implications for cognitive flexibility and memory capacity^39,40. A low-dimensional representation, where population variance was restricted to the category coding-axis, would allow AC to discriminate between vocalization categories, but not families. At the other extreme, a maximally high dimensional representation would permit AC to discriminate between any pair of vocalization categories and family ID combination.

Dimensionality was assessed with the participation ratio, shattering dimensionality, and cross-condition generalization of our dataset. The participation ratio quantifies the dimensionality based on how distributed variance is across all dimensions of the data (e.g., a value of 1 would be minimally dimensional and 7 would be maximally dimensional for a mean-centered, 8-condition dataset). We found that our data had a participation ratio of 5.1 (Figure S3C-D), indicating that AC populations compress vocalization representations into a lower dimensional space than is expected from a random geometry. The shattering dimensionality measures the mean performance of linear decoders on all possible dichotomies of the data (Figure S3E-F). A value of 0.5 indicates the lowest dimensionality, as the decoders did not perform above chance across all dichotomies, while a value of 1 would indicate maximal dimensionality, as all dichotomies are linearly separable. We calculated a shattering dimensionality of 0.725, again demonstrating that the AC population has a low dimensional geometry that may underlie generalization across variants to represent vocal categories.

Finally, we computed how well an SVM decoder could generalize across conditions in our dataset. A high cross-condition generalization would suggest a low-dimensional, abstract representation, while a low value would suggest that the condition is represented high-dimensionally and uniquely across conditions. For category encoding, we found high cross-family generalization, suggesting that categories are largely invariant to subtle acoustic differences across family ID (Figure S3G). We found relatively poor cross-category generalization of family (Figure S3H), suggesting that the representation of family identity in the AC is high dimensional and unique for different categories.

Individual AC neurons can be specialized for family or category decoding

The inclusion of both category and family identity variance in the stimulus set permitted us to ask whether single AC neurons were specialized for the extraction of one type of information. Thus, we reran the SVM 1000 times and compared the contribution of each neuron to the vocalization category and the family identity SVMs to their weights in a null decoder trained with shuffled trial labels (Figure 6A). A null decoder will equally weigh all neurons in the population across 1000 iterations, since no particular neurons give the null SVM more information to perform classification. Thus, this analysis compares the contribution of neurons relative to a perfectly distributed population code (all neurons contributing equally). A paired t-test (p<2.5e-3) was used to compare the true and null contribution for all family decoders and for each category, the results of which we used to label each cell as strongly contributing only to category, only to family identity, to both, or to neither decoder (Figure 6B).

Figure 6: AC neurons contribute to both category and family identity decoding.

A. Schematic demonstrating what is meant by an SVM weight. B. Absolute values of SVM weights for all responsive units (n=316), shaded by whether they contribute to that vocalization’s category classification, family identification, both, or neither. C. Quantification of the number of units contributing to either or both, or neither of the classifiers described in B. D. Performance of SVMs only using 20 sampled neurons from the category only, family only, both, or neither populations.

A greater number of vocalization-responsive cells contributed significantly to either the vocalization category or the family identity decoder than to both decoders (Figure 6C), suggesting the existence of ‘specialists’. However, neurons that contributed significantly to both decoders performed classification just as well as the specialists (Figure 6D). This suggests that single neurons can read out information to more than one downstream decoder without impairing either decoder’s performance. We found that vocalization category can be decoded from ‘family only’ and ‘neither’ cells at ~50% when subsampling 20 neurons of each neuron type (Figure 6D), suggesting that even though these neurons don’t contribute strongly on their own, vocalization category can be extracted from a purely distributed code at the population level. Since family identity could not be decoded from ‘only category’ or ‘neither’ cells, this suggests that family identity cannot be extracted from a purely distributed code, but rather a sparse code with a few strongly contributing neurons. The exception was Combo vocalizations, where family ID could be decoded significantly above chance from the ‘category only’ cells (1-sample t-test, p=2.75e-7), suggesting again that the combinatorial nature of these vocalizations offers the population more information from which to extract both categorical and social information.

AC represents family identity within each vocalization category using multiple acoustic features

We next sought to determine which acoustic features were used to decode vocalization category and family identity by leveraging the large number of individual variants. Specifically, we asked which acoustic features were common among well-decoded variants versus poorly-decoded variants. First, the family identity decoder was run 500 times, sampling new trials each time. Across all iterations, we computed the percent of iterations that family identity was correctly decoded from the population response to that variant (Figure 7A). Then, we compared the acoustic features from the best- versus worst-decoded 10 variants for each family and each category. When comparing the z-scored difference between Family 1 features and Family 2 features for well and poorly decoded syllables, the features with the largest differences differed for each category (Figure 7B). For example, peak frequency and duration most strongly distinguished family identity in DFMs, since well-decoded Family 1 variants had shorter durations and higher median frequencies (Figure 7C). Amplitude modulations and duration cues led to better family identity decoding from UFM variants, while amplitude modulations and peak frequency distinguished Warbles from different families. Thus, while simple frequency cues likely underpin coarse vocal categorization, combinations of more complex acoustic cues subserve AC encoding of family identity.

Figure 7: AC uses heterogeneous acoustic features to represent family ID within vocal categories.

A. Mean percent correct of individual variants across 500 iterations of the neural SVM decoder. Black points = top 10, best-decoded variants. Grey points = bottom 10, worst-decoded variants. B. Top z-scored acoustic features that differed most between the best-decoded and worst-decoded variants. C. Raw acoustic features values of the top versus bottom variants for each category, separated by family ID (independent t-test; ***p<0.001, **p<0.01, *p<0.05, †p<0.1, ns(not significant)=p>0.1). See also Figure S4, S6, and Table S1.

Comparison between standard coding properties and vocalization responses

Finally, we examined whether standard coding properties could explain the observed vocalization-evoked responses by presenting tones (30 ms, 1.5–50kHz) and amplitude modulated noise stimuli (depth: 100% depth, rates: 2–64Hz; passbands: 1–10, 1–25, 5–25, 25–40, or 30–55 kHz). 43% of AC units (n=367) were responsive to either vocalizations, tones, or AM noise (Figure S4A). Of these sound-responsive neurons, 86% were responsive to vocalizations, 43% to tones, and 31% to AM noise, indicating that AC neurons may be more easily driven by vocalizations than artificial stimuli. We found neurons that responded to tonal frequencies that spanned the entire vocalization frequency range (Figure S4C-D), indicating that we adequately sampled the tonotopic axis. Further, the best frequencies of tone-responsive neurons to a given category generally matched the average frequency of that category (Figure S4B). To assess this quantitatively, we computed the phi correlation (a value of 1 is a perfect positive correlation) between the presence of a vocalization response for each category and the presence of a tone or AM response for matched frequencies (Table S1). We found moderate response correlations with tone responses (0.37–0.64) and weak correlations with AM responses (0.12–0.36), suggesting that frequency tuning is a strong, but not the sole, contributor to vocalization category responses.

We further examined these properties at the individual animal level to determine what might explain differences in single animal decoding performance for different vocalization categories. Most animal’s neurons were biased to best frequencies <20kHz, suggesting why there were more Combo- and DFM-responsive units (Figure S5B-C). Animal 3, with the most neurons with best-frequencies in the ultrasonic range, had the highest performance on UFM and Warble family identity decoding (Figure 5E) and the greatest impact on UFM and Warble family identity decoding when removed (Figure S5E), again suggesting that frequency tuning is a strong contributor to vocalization responses.

We also considered whether animal position could have influenced our data. For example, since animals were freely moving, interaural level could have changed during the course of testing. To determine whether positional factors were a source of AC neuron response variance, we asked how every tone-responsive neuron’s (n=157) z-scored firing rate was modulated as a function of the animal’s distance to the speaker, head orientation to the speaker, velocity, or acceleration (Figure S6). The 5th and 95th percentiles for the slope fit R² values were 0.004 to 0.25 across the four behavioral variables, with only 2 neurons having slope fits that reached significance (Pearson correlation with Bonferroni correction; p<2.5e-3), suggesting that movement and position had a relatively small influence on neuronal responses.

DISCUSSION

Vocalization processing involves two distinct requirements: generalizing across many voices to categorize a sound’s meaning and, inversely, distinguishing between many voices to recognize identifying information. In humans, both meaning and identity information can be extracted from vocalizations within 300 ms of sound onset^10,41–45, consistent with simultaneous processing of these two attributes⁴⁶. The neural representation of these two types of information have typically been explored in separate studies that do not consider the impact of natural acoustic variability. Here, we asked how adult gerbil AC populations concurrently represent vocalization category and family identity using a >1500 stimulus set of gerbil vocalizations belonging to four vocal categories and recorded from five gerbil families.

While a neural representation of vocalization categories has been characterized in several species^25,38,47, these studies used relatively few variants and did not determine whether the representation was robust to the natural range of variance. However, several studies have tested neural categorization by presenting synthetic variants that were morphed from a small set of natural vocalizations^{21,23,47–50}. These studies show nonlinear neural transformations of vocalizations into categories across an artificially manipulated acoustic axis, and imply a similar transformation for a natural range of variants. However, morphing could plausibly introduce unnatural feature combinations that would not be generalized in natural vocalizations. Finally, a few studies have shown that AC neurons can categorize a few dozen natural variants of phonemes or vocalizations^31,32, although these studies did not challenge the neural populations on both category and social identity classification. Taken together, these studies provide deep insight into the neural basis of vocalization categorization, while leaving open the question of how social identity cues are represented simultaneously within categories.

Our study confirms and extends these results by challenging each AC neuron with ~1500 natural variants that derived from 4 vocalization categories and 3 separate gerbil families (Figure 1). Using this approach, we demonstrated that decoding of both vocalization category and family identity was robust to natural acoustic variability. Consistent with previous reports, we found a relatively small population of vocalization-responsive neurons. However, the 316 vocalization-responsive AC neurons were sufficient to classify single trials into the 4 vocalization categories at a performance of 89–94% (Figure 5C), which is comparable to classification performance based on acoustic features (Figures 1D, 2B). Although this performance is higher than reported for classification of dozens of human phonemes by ferret AC³¹, this is likely explained by the ferret study having used a greater number of vocalization categories. Similar to a report on zebra finch auditory pallium²⁵, we find that a small number of cells is required to perform categorization, perhaps due to the gerbil AC population having relatively strong category-selectivity at the single neuron level (median SI = 0.72; Figure 3H).

Behavioral evidence from many species suggests that certain vocalizations signal individual or colony-specific identity^{15,17,19,51,52}, but the neural mechanisms supporting this behavior remain uncertain. AC neuron responses can be tuned to conspecific vocalization features in squirrel monkey, marmoset, cat, and macaque^53–55 and these responses have even been shown to be modulated by familiarity with a vocalizer^56,57. However, identity-specific information may differ across vocal categories¹⁷, and representing identity would require AC neurons to parse subtle, high-dimensional, acoustic features on a category-by-category basis.

Here, we report that family identity can be decoded from an AC population for each of the vocalization categories (Figure 5E), suggesting that social identity is represented independently and concurrently within each category. This finding is consistent with a recent behavioral study showing that zebra finches can perform social discrimination from vocalizations across their repertoire, and that the features used for identification differ for each vocalization category¹⁷. We show that family identification requires a larger AC population than does vocalization categorization (Figure 5F), suggesting that this information is not as strongly encoded at the level of single variants and thus requires more neurons to represent the salient acoustic features (Figure 7). In general, decoder performance was similar when using firing rate versus 30ms bins (Figure S2), suggesting that temporal coding is not always necessary for simultaneous extraction of category and family identity. Previous work reports that AC encoding for AM rates⁵⁸ and zebra finch vocalizations are improved with a temporal code²⁵, but these studies assess only modulation rates <40Hz, which are within the range to which AC neurons are able to synchronize^59,60. Our stimuli, such as the modulations in the Warbles, have AM rates that exceed 40Hz, suggesting that AC neurons are likely to use a rate code to represent these modulations.

One limitation of our design is that single syllables were used as stimuli, whereas gerbils (and other rodents) typically emit vocalizations in bouts³⁵. Therefore, it is possible that vocalization repetition provides stronger social identity information, and temporal integration by a downstream decoder would be able to extract family-specific sound cues and perform better classification. Indeed, female zebra finch neurons in NCM, a higher auditory area, respond preferentially to their mate’s full song compared to familiar and unfamiliar songs⁵⁷ and bout responses, but not syllable responses, are enhanced in female AC after 3 days of male social experience⁵⁶. A separate possibility is that primary AC serves to extract family-specific features and projects to a downstream decoder that compares those features to a family prototype, subserving ‘recognition’. This would be consistent with cognitive models positing that human voice recognition is a distinct mechanism from discrimination, given dissociations of these deficits in patients using lesion case studies^61–63. Further, a hierarchical model for efficient coding in the auditory neuraxis posits that encoding becomes increasingly efficient as position in the hierarchy increases, temporal integration gets longer, redundancy decreases, and neuronal encoding becomes better explained by multidimensional feature models^64,65. In fact, efficient representations of vocalization categories have been identified in earlier areas of the auditory hierarchy^23,48,66,67, although it is not clear how robust they are to naturally-varying identifying information. Our study is consistent with an efficient AC representation of vocalization categories using sparse single-unit responses, which is family-invariant. We further found that family information can also be extracted from vocal categories, but less efficiently (i.e., a greater number of neurons is required). This leaves open the possibility that a downstream region may integrate across multiple syllables within a bout to more efficiently encode family identity.

While our findings support an AC contribution to family identification, this interpretation is limited by the absence of behavioral measures in gerbil for family recognition or categorization. Therefore, it is necessary to develop a paradigm that measures gerbil discrimination of vocalization categories and family-of-origin such that results can be compared to neural responses. Such a paradigm could systematically vary the features identified as important for extracting either category or identity information (Figure 7) and ask how animals’ discrimination compares with a neural classifier. For example, subtly time stretching and pitch shifting DFMs could reveal how these features concurrently affect family encoding and at what level animals can still perceptually discriminate. We expect that finer variation of multiple acoustic features would be needed for animals to discriminate identity, while coarser variation of fewer features would allow for category discrimination.

The neural computations underlying categorization of naturally-variable vocalizations are not well understood. Previous work suggests that acoustically, identity information is higher dimensional than category information. That is because the acoustic features that signal identity are comprised of multiple, small differences⁶⁸, and depend on vocal category¹⁷. We demonstrate here that the AC can classify both vocal category and family identity from a single vocalization syllable, and the features used to perform this dual computation differ somewhat between categories. Our population has a lower dimensionality geometry than would be expected from chance (Figure S3), suggesting that there is a transformation from the acoustic information either at the level of the AC or inherited from upstream regions that aids with categorical representations. Further, we find stronger cross-family generalization of category than of cross-category generalization of family (Figure S3), suggesting that family identity coding is higher dimensional and not easily abstracted from all categories. Our analysis of the acoustic features underlying category versus family identity coding support this (Figures 2 & 7), since both at the acoustic and neural level more features are required to extract family identity information. A separate finding is that more single neurons are primarily specialists for either vocalization category or family ID decoding, with few neurons strongly contributing to both (Figure 6). Previous work has suggested that mixed-selective neurons are necessary for increasing dimensionality and that a population of purely-selective neurons limits the dimensionality of the population⁴⁰. It is possible that the contribution of both generalists and specialists to the decoders ensure that the AC strongly represents category, yet remains flexible enough to extract social information.

Overall, our findings provide insights into the population coding properties that support naturalistic vocalization processing. Our results are consistent with previous work showing that auditory cortex populations represent vocal categories³¹, and we show that this is true even in the face of acoustic variance that conveys social identity. These category and family identity representations are underpinned by a mix of both specialist and generalist neurons that work together to perform distinct computations of category generalization and family identification. Together, our findings reveal how the auditory cortex deals with naturalistic variance to provide an efficient representation of multiple types of information, permitting vocal animals to extract meaning and social identity from the same stimuli. Future investigations should build on these findings by testing how these neural computations underlie behavior decisions in naturalistic social paradigms.

RESOURCE AVAILABILITY

LEAD CONTACT

Estelle in ‘t Zandt, eei3718@nyu.edu

MATERIALS AVAILABILITY

This study did not generate new unique reagents.

DATA AVAILABILITY

All data and custom scripts are available here: https://nyu.box.com/s/xz9mw96mmpt8vzqoncn4lrxu3v5shacb

STAR METHODS

EXPERIMENTAL MODEL AND STUDY PARTICIPANT DETAILS

Five adult Mongolian gerbils (Meriones unguiculatus, 2 females, 3 males) were used in the study. Gerbils were bred in an onsite vivarium, weaned at postnatal (P) day 30, and housed on a 12 hour light/dark cycle with full access to food and water. All experimental protocols were approved by New York University’s Animal Use and Welfare Committee.

METHOD DETAILS

Stimulus set:

Vocalizations recorded from 5 families of gerbils (7–9 pups P25–27) from 7pm-7am. The two families shared across all experiments (Family 1 and Family 2) had 8 and 7 pups, respectively. Parents and all pups were placed in an enlarged cage with Alpha-Dri bedding, food, water, a roofless-house, and some bedding from the home cage. Video recordings were obtained under red LED light with a FLIR Blackfly-S camera (30 frames per second) and vocalizations were recorded at 125kHz sampling rate from 4 Avisoft microphones (placed at the four roof corners of the cage, angled towards the middle).

The signals from the 4 microphones were averaged to improve the signal-to-noise ratio³⁵. We trained a network to automatically extract vocalizations from the averaged microphone signal using Deep Audio Segmenter (DAS³⁶).To train the network, we extracted sample bouts from 8 gerbil families recorded overnight as described above. This was to ensure that the training set was not biased by the vocalizations from one particular family or by subtle changes in the background microphone signal across recording days. The network was trained by labeling bouts with one of 10 labels: alarm, noisy alarm, upward frequency modulated (ufm), noisy ufm, low-frequency, noisy-low frequency, warble, noisy-warble, chirp, and noise. This ensured that the network extracted vocalizations that were not overlapping with other vocalizations or with noise such as scratching, digging, etc. This network performed between 30–80% performance on correctly labeling variants. A second network was trained to detect noise using the noise-labeled variants from the first network, as well as the original training set. This network performed at 60% or higher for all categories and produced much fewer noise-contaminated variants (Figure S1A-B). The vocalization category labels used for DAS to extract variants were not used for any subsequent analyses, only for extracting vocalizations. Therefore, the model’s performance was evaluated after grouping all vocalizations, overlapping vocalizations, and noise. As demonstrated in Figure S1B, 80% of user-labeled vocalizations were extracted with the network. We evaluated the model’s performance for noise, vocalizations, and overlapping vocalizations using the python sklearn f1_score function (average=None).

Using the onsets and offsets output by DAS, we extracted variants from the 8 families we recorded overnight vocalizations from. The onsets and offsets from DAS showed some jitter in cropping the first or last few milliseconds of the vocalizations, so all variants were padded with 40ms to ensure that the whole vocalization was extracted. We generated spectrograms (nperseg=512, noverlap=256, spec_min_val=−8, spec_max_val = −5) from these onsets and offsets and trained a variational autoencoder to represent each spectrogram with 32 latent values³⁷. We fit a Gaussian mixture model (cov=‘diag’) to cluster the vocalization latents into 200 clusters. These clusters were then manually labeled by an experimenter to be flat, stacked, spectral, flat, sweeping, short, mixed, ufm, warble, loud, and noisy. Then, we used the manual labels to pull all the GMM clusters that belonged to the broad vocal categories: Combos, DFMs, UFMs, and Warbles^33–35. For all experiments, the same trained VAE and GMM models were used on all vocalizations, so the cluster labels remained constant across all families recorded from.

The vocalizations from 4 broad categories were extracted: upward frequency modulated (UFM), warbles, downward frequency modulated (DFM), and combination vocalizations (combos). These variants were manually sorted through to check if they were truly single, non-overlapping calls and that there was at least a 100ms quiet period preceding the vocalization, to allow for a reasonable baseline period before the vocalization was presented. Variants belonging to these 4 vocalization categories were filtered with a custom python script using librosa’s soft mask vocal background subtraction. All vocalizations were normalized to a root mean squared (RMS) value of 0.004 (48dB for a numpy float32 wavfile). A custom script was used to present the vocalizations with a mean inter-stimulus-interval (ISI) of .47s. This large file was divided into multiple 5 minute blocks for presentation during the experiment.

Non-vocal control stimuli were generated to test the spectrotemporal tuning properties of recorded cells. Pure tones with frequencies from 1.5–55kHz (30ms duration, 2ms rise/fall ramp) were presented randomly in 2 min blocks of 5 trials each with mean ISI of 640ms (+/− 300ms). Amplitude-modulated (AM) noise stimuli were also presented to test the rate tuning of neurons. White noise was bandpassed at 1–10kHz, 1–25kHz, 5–25kHz, 25–40kHz, or 30–55kHz and modulated at 100% depth with rates of 2, 4, 8, 16, 32, and 64Hz. AM stimuli were presented 10 times each. All stimuli were normalized to a root mean squared level of 0.004.

Surgeries

Animals were anesthetized with isoflurane/oxygen, secured in a stereotaxic device (Kopf), and a 4 shank 64-channel silicon probe (Cambridge Neurotech P1–236 array) was implanted in the left core auditory cortex (AC) (primary AC or the anterior auditory field⁶⁹). The anterior-most shank of the probe was implanted 3.5mm anterior to lambda, 4.6mm lateral, and 1–1.2mm ventral to the surface at a 25 degree angle in the mediolateral plane. The probe was attached to a custom microdrive to allow for post-surgery advancement of the probe through AC. A ground wire was inserted into the contralateral hemisphere approximately 2mm medial and 1mm anterior from lambda.

Stimulus presentation and data collection:

Stimuli were presented as wavfiles using a custom script using the pyaudio package on a Raspberry Pi through a HiFi Berry DAC2 Pro sound card. Stimuli were amplified using a TDT z-bus stereo amplifier and presented through a Fountek NeoCd1.0 1.5” Ribbon Tweeter. At an RMS value of 0.004, the stimuli were presented at 67–75dB, as measured from either the closest position in the arena (directly below) or the furthest corner. Four ultrasonic microphones (Avisoft CM16/CMPA48AAF-5V) were synchronously recorded using a National Instruments multifunction data acquisition device (PCI-6143) via BNC connection with a National Instruments terminal block (BNC-2110). The recording was controlled with custom python scripts using the NI-DAQmx library (https://github.com/ni/nidaqmx-python) which wrote samples to disk at a 125 kHz sampling rate. The electrophysiological data was simultaneously recorded at a 12.5kHz sampling rate using a wireless data logger (White Matter LLC) that received a synchronization signal from the NIDAQ. The neural data were upsampled to 25kHz during downloading.

Implanted animals were placed in the experimental rig (22×14×14in.). The speaker was located on the roof, so sounds were presented from overhead. After 5 minutes of silence to habituate the animal, the stimulus presentation began. The order of stimulus presentation was the same every session within a block, but the block order was semi randomized across sessions. The full experimental session lasted 2–3 hours.

Histological electrode site confirmation

The experimental endpoint was determined either by the microdrive reaching its deepest point in the tissue such that no new AC neurons could be recorded from, or the recording SNR dropping to a level where single units could no longer be sorted. Animals were deeply anesthetized with isoflurane and overdosed with 0.3mL euthasol. Once the animal stopped breathing but before the heart stopped beating, a transcardial perfusion was performed with 10mL of saline and 20mL of 10% formalin. The brain was extracted and stored in 10% formalin for at least 2 days. The brain was notched on the right hemisphere and placed in agar so that it could be sectioned into 60um sections using a vibratome (Leica). Sections were mounted onto gelatin-subbed slides and left to dry for 24 hours. Sections were then stained with 0.25% thionin and coverslipped with Permount. After drying, sections were imaged to locate the probe. Probe sites were aligned to the Ratdke-Schuller gerbil stereotaxic atlas⁷⁰ to determine the exact implantation sites.

Electrophysiological recording and preprocessing

Raw electrophysiological data were preprocessed using custom python scripts that bandpassed the signal from 0.2–4kHz and referenced the signal by common median subtraction. Individual action potentials from single neurons were extracted using Kilosort 2.5 and manually reviewed using Phy2. Inclusion criteria for a single unit included having at least a whole session firing rate greater than .1Hz, a contamination percentage below 20, and visually inspecting the principal components of the waveform and interspike-interval histograms for noise.

Quantifying acoustic features

We used the similarity_features function from the VocalPy package⁷¹, a python implementation of the Sound Analysis Pro Sound Analysis Tools library, to compute median pitch (fundamental frequency), peak frequency, entropy, frequency modulation, and goodness of pitch. In addition, duration and global FM was computed from the audio (global FM = (stop frequency - start frequency)/duration)). Start and stop frequencies were computed by taking the median fundamental frequency within the first and last thirds of each vocalization (Peterson et al. 2024). Bandwidth was computed by taking the difference between the start and stop frequencies. Amplitude modulation (AM) was computed on the smoothed envelope (smoothing sigma=0.5ms). The smoothed envelope was converted to dB and the find_peaks function from scipy was used to count the number of positive AMs in the signal >6dB⁷². Global AM was the (number of positive AMs)/duration and peak AM was the fastest AM in the signal.

Video tracking

Video tracking was performed using SLEAP⁷³ on all sessions to determine the location and orientation of the animal in relation to the overhead speaker. The SLEAP model was trained using a keypoint skeleton containing the nose, ears, head, and body points of a single animal. The fill_missing function was used to interpolate the tracks in video frames that SLEAP did not explicitly label the animal.

QUANTIFICATION AND STATISTICAL ANALYSIS

Acoustic classifiers

To determine if the acoustics of the vocalizations could be classified based on their vocalization categories and by the identity of their family-of-origin, we used a random forest classifier. The latent values from the variational autoencoder that explained the most variance were used as input and the labels were either the vocalization categories or the family IDs from which they were recorded. The vocalization category classifiers were fit 100 times, using either 2, 3, or 5 families’ vocalizations as input. Each iteration randomly sampled 50 syllables of each category from each family and split the variants into an 80/20 train/test set. We also ran this classifier for category and family (2 families) using the z-scored acoustic features instead of the latents, using the same procedure described above. For the acoustic feature random forest classifier, the relative feature importances were averaged across all iterations.

Electrophysiological analyses

Quantification of responsiveness to vocal categories, family identities, tones, and noise

Firing rates were computed by summing the number of spikes in a given epoch window and dividing by the length of this window. Epochs for which firing rate was computed included the spontaneous period (100ms before the vocalization’s background started), the pre-vocalization background period (i.e. pad), and the duration of the vocalization itself. A neuron was counted as responsive to a vocal category if the firing rate during the vocalization epoch across all trials of that vocalization was significantly different (excited or suppressed) from the firing rates of both the spontaneous epoch and the pad epoch (paired t-test with Bonferroni correction; p<2.5e-3). From these analyses, we counted the number of cells that were significantly responsive to 0, 1, 2, 3, or all 4 categories, as well as which categories these neurons were responsive to.

To determine whether more neurons were responsive to an animal’s own family than the other families, we quantified the responsiveness of cells specifically to each family ID. If a neuron was responsive to a given category, then we compared the spontaneous, pad, and evoked firing rates for each family of that category individually. If the evoked firing rates were significantly different from both the spontaneous and pad (paired t-test with Bonferroni correction; p<3.3e-3), the unit was counted as ‘family responsive’.

We also computed responsiveness to tones and noise. For tones, the firing rates 100ms before tone-onset were compared to the firing rates from tone onset to 50ms after tone offset (to account for some neurons displaying prolonged tone firing). A neuron was counted as tone-responsive if there was at least one tone frequency for which the evoked firing rate was significantly different than the spontaneous firing rate (paired t-test with Bonferroni correction; p<2.6e-4). The neuron’s best frequency was determined for tone-responsive units as the frequency that elicited the maximum change in firing compared to baseline. Noise-responsiveness was determined by comparing the 100ms period before noise onset and the firing rate during the full 1 second amplitude-modulated noise stimulus (paired t-test with Bonferroni correction; p<3.3e-4).

To quantify variation in the responses to individual variants, we compared the coefficient of variation (CV) of trials from the same variant versus different variants from the same category. CV was computed for each category-responsive neuron by dividing the standard deviation of the stimulus-evoked firing rate by the mean firing rate. CV was quantified for all syllables that were shared across all subject animals. For within-syllable CV, standard deviation and mean were computed from the 5 repeats of the same syllable. For across-category variation, we sampled 1 random trial from 5 different variants in the same category, and repeated this sampling for the number of iterations that matched the number of unique variants for that category. For a given category-responsive neuron, the mean CV was computed by averaging all the within-syllable CVs versus all the across-category CVs. The within-syllable and across-category CVs were compared at the population level with a paired t-test for each category.

To quantify the percent of category variants single neurons responded to, we used a threshold measure of z-scored firing rate, since there were only 5 trials per variant. A response to a single variant was counted if there was a >0.25 absolute change in z-scored firing rate from both the spontaneous and pre-vocalization pad period (excitation or suppression).

Selectivity index

For each of the vocalization-responsive units, a selectivity index (as described in Sharma et al. 2024⁷⁴) was computed to determine how specific a neuron’s response was for 1 versus all categories. An SI=1 indicates that a neuron only responds to 1 category and not to any of the others, while SI=0 indicates equal firing to all four categories. The formula is the following:

$$SI=\frac{n-\sum _{i=1}^n\left(\frac{{\bar{r}}_i}{{\bar{r}}_m}\right)}{n-1}$$

where n represents the number of categories (4) or families (3), r_i represents the absolute value of the difference between the mean category/family firing rate and the mean spontaneous category/family firing rate, and r_m is the maximum of r_i for all i. For vocalization category selectivity (Figure 3G), the mean category firing rates were used, pooling across family ID). For family ID selectivity, the mean family firing rates within a given category were used. Note that only category-responsive neurons were used to compute family ID selectivity for a given category. To determine if a unit was family-modulated, we used a one-way ANOVA to test if the firing rates were significantly different across families (one-way ANOVA, p<2.5e-3). The number of units that were responsive to family identity for at least 1 category was divided by the total number of vocalization-responsive units to determine the percentage of the responsive population that was family-modulated.

Population decoders

Classification of single trial, z-scored 30ms-binned PSTHs was performed using a linear support vector machine (SVM) using the Python sklearn package, with parameters C=1.0, tol=1e-3, penalty=‘l1’, dual=False, and max_iter=10000. The shape of the decoder input had the rows, or features, N_neuronsxB_timebins, and columns, or samples, T_trials. Since one family was always the subject animal’s own family, only 2 families were used for identity decoding performance in this analysis (so that we could include all variants that were presented to all neurons in the dataset). The decoding analysis was performed 100 times, sampling 300 trials from each category and family per iteration to ensure equal sampling of the dataset. On a given iteration, the data were separated 80/20 into train and test sets, and the percent correct of the decoder on the test set was saved. The subsampled trials on a given iteration were also used to train a null decoder for which the trial labels were shuffled. To ensure fair comparison of vocalization category and family ID decoding, the same sampled trials that were used for decoding family ID were further subsampled so that there were always 300 trials per class (for family: 300 from Fam1 and 300 from Fam2; for vocalization category: 150 from each family, so that there were 300 trials per class). The exact same decoding analysis was performed using either a pseudopopulation of all units, a pseudopopulation of all vocalization-responsive units, or using each individual gerbil’s vocalization-responsive units. We also performed the decoding analysis as described using the z-scored firing rates rather than the z-scored PSTHs (input: N_neuronsxT_trials). Finally, we performed both category and family identity decoding by leaving out individual animals’ responsive neurons from the population, per the same procedure described above.

We determined the effect of population size on decoding performance by performing the decoding as described, but also subsampling increasing numbers of neurons in the population. We sampled 2, 5, 10, 25, 50, 100, 200, and 316 neurons 20 times each. The mean performance was fit with a sigmoidal function by bootstrapping the individual iteration performances 500 times. The sigmoid function was: Where x is the number of neurons, y is the SVM performance, L is the range of the function, k is the steepness of the curve, x₀ is the inflection point, and b is the lower asymptote of the function. The bootstrap was initialized with x₀ as the number of neurons needed for the average performance across all neuron samplings and b was constrained with a lower bound of chance performance (25% for vocalization category, 50% for family ID).

We further analyzed the weights of neurons in the PSTH SVM to determine the relative contribution of individual neurons to either category or family ID decoding. We performed the SVM 1000 times, as well as a null decoder that was trained on shuffled trial labels. and saved the coef_ for each iteration. Since the PSTH SVM input was N_neuronsxB_bins, each iteration produced 8 weights per neuron. Further, we were interested in the relative contribution of each cell, so we computed the mean absolute value of these weights for each cell in a given iteration. We then divided the mean(abs(weight)) of each cell by the sum of all the weights to determine the percent contributed by each neuron. A neuron was determined to significantly contribute to a decoder if its ‘percent contribution’ across the 1000 iterations was significantly higher than in the null decoder (paired t-test with Bonferroni correction; p<2.5e-3).

Geometric analyses: participation ratio, shattering dimensionality, and cross-condition generalization

The following analyses used trial-averaged, z-scored firings rates for all neurons for all variants that were shared across all experiments. The variants were then sorted into 8 experimental conditions: Combo-Family1, Combo-Family2, DFM-F1, DFM-F2, UFM-F1, UFM-F2, Warble-F1, Warble-F2. All variants of a condition were averaged to form a “condition-averaged matrix”. To compare the geometrical analyses to a maximally-high dimensional version of the data that shared the same firing rate properties as the real data, the order of the neurons were randomly shuffled independently for each condition, effectively rotating the population activity into an orthogonal dimension for each condition. This maintains the firing rate properties of the population for a given condition, but disentangles the relationship of single neurons’ firing rates across conditions.

The participation ratio was computed from the eigenvalues of the covariance matrix of the condition-averaged matrix^39,75, and of 100 random rotations of this matrix. The participation ratio formula is as follows:

$$PR=\frac{{\left({\mathrm{\Sigma }}_j{\lambda }_j\right)}^2}{{\mathrm{\Sigma }}_j{\lambda }_j^2}$$

where l_j is the j^th eigenvalue of the covariance matrix. A larger participation ratio indicates even spread of the variance across more dimensions, while a smaller participation ratio indicates that the variance is concentrated in fewer dimensions, indicating a lower dimensionality.

The shattering dimensionality is a complementary metric that measures how many dichotomies of the data can be linearly decoded. The 8 conditions of this experiment can be divided into 35 unique dichotomies (for example, [C-F1, C-F2, D-F1, D-F2] vs. [U-F1, U-F2, W-F1, W-F2]). For each dichotomy, the population response to 200 variants from each half of the conditions were randomly sampled. A decoder was trained on labels “dichotomy 1” or “dichotomy 2”. This was repeated 10 times per dichotomy to get a mean performance for a given dichotomy. The shattering dimensionality was computed as the average decoding performance of all 35 unique dichotomies. This procedure was compared to a null decoder that was trained on shuffled dichotomy labels. To compare the shattering dimensionality to a maximally-high dimensional dataset, the same random rotation procedure as described above was performed, and shattering dimensionality was performed on the randomly-rotated data.

Cross-condition generalization was assessed by training the SVM on one condition of the data and testing on the other. First, to test cross-family category generalization, the SVM was trained as described above to decode category from Family 1 data and tested on Family 2 data. Then, to test cross-category identity generalization, the SVM was trained to decode family identity from 1 category, and then tested on a 2nd category. This was repeated for all combinations of the dataset.

Determining the contribution of acoustics to family identity decoding

The family identity SVM was run 500 times, sampling new trials each iteration. Across all iterations, we computed the percent correct for individual variants by counting the number of times the decoder correctly labeled a single trial population response as Family 1 or Family 2. The average across the 5 trials of a given variant was used as that variant’s percent correct. Next, we compared the acoustic features of the top-decoded variants from each family to the features of the worst-decoded variants of each family. We chose 10 syllables to use the same number across all categories, but to avoid the “bottom” Combo syllables having significantly greater than chance performance. If the population was using a given feature to perform family identification, then a given feature should be strongly different for well-decoded variants, but not different for poorly-decoded variants. For each vocal category, the z-scored acoustic features were sorted by the difference of those features between Family 1 and Family 2 to identify the order of most to least differing features. From this ordering, we compared the raw acoustic feature values (not z-scored) using an independent t-test between the top 10 Family 1 and Family 2 syllables, and the bottom 10 Family 1 and Family 2 syllables.

Movement analyses

For all sessions with tone-responsive neurons, the head and nose points were extracted for all frames to compute a position and direction vector for the animal’s head. The neurons’ best frequency was used for all movement analyses. For a given neuron’s best frequency, the position and orientation of the head were computed for the duration of the tone (30ms). The numpy.diff function was used to compute the velocity and acceleration from the head position data. For each tone-responsive unit, the linear correlation between velocity, acceleration, distance to the speaker, and the orientation to the speaker was correlated with the z-scored firing rate on each best-frequency tone trial (scipy linregress and pearsonr functions). The slopes and R² values of those correlations are summarized in Figure S6.

KEY RESOURCES TABLE.

REAGENT or RESOURCE	SOURCE	IDENTIFIER
Chemicals, peptides, and recombinant proteins
Permount	Fisher Scientific	SP15-100
Deposited data
Electrophysiology data	NYU Box	https://nyu.box.com/s/xz9mw96mmpt8vzqoncn4lrxu3v5shacb
Experimental models: Organisms/strains
Gerbil (Meriones unguiculatus)	Charles River Laboratories	www.criver.com
Software and algorithms
Experimental acquisition code	Custom code	https://github.com/Aramist/lab-acquisition.git
Preprocessing and analysis	Custom code	https://nyu.box.com/s/xz9mw96mmpt8vzqoncn4lrxu3v5shacb
MATLAB 2022	MathWorks	www.mathworks.com
Adobe Illustrator 2022	Adobe	www.adobe.com

Highlights:

• Small auditory cortex populations extract call type from thousands of vocalizations

• A larger population is needed to extract family identity from the same set of calls

• Individual neurons can be specialists for one type of information

• Populations extract call type or family identity using unique acoustic features