BattyCoda: A novel open-source software for bat call annotation and classification

Abstract

The field of acoustic communication needs tools that facilitate the annotation and labeling of animal calls. Bat acoustic libraries gathered over the past few decades have primarily focused on compiling echolocation calls, which have been leveraged to develop machine learning algorithms capable of classifying bat species. However, because these classification methods require large training datasets, they have not yet been generalized to classify types of bat communication calls. Communication call repertoires in bats are wide, and distinct syllables occur with varying frequency, with some call types being recorded only rarely. Furthermore, collecting Classification communication calls poses greater technical challenges, making these calls more difficult to capture reliably. Here, we present BattyCoda, an open-access, customizable tool to categorize and label bat communication call types within the repertoire of a species using small training datasets (tens to hundreds of labeled calls). In this work, we compiled an initial training dataset of 11 types of big brown bat (Eptesicus fuscus) calls, tested the performance of various candidate classifiers, and assessed the final classifier’s training sample size sensitivity. We found that the best performing classifier achieved a balanced accuracy of ~50 %, with common call types achieving classification accuracies over 70 %. Our tool can greatly facilitate annotating bat calls in recordings by providing accurate labels for common call types, while also assisting researchers in categorizing rarer communication calls. BattyCoda has the potential to build research capacity in the field of acoustic communication by expanding the availability of libraries including a wider range of bat calls and species, thereby enabling the exploration of new hypotheses.

1. Introduction

The study of acoustic communication benefits from the comparative approach of studying diverse species that produce a wide repertoire of social sounds. For decades, zebra finches and other songbirds have been the champion animal models, and several methods have been developed to label and categorize each syllable (Elie and Theunissen, 2016; Pearre et al., 2017; Sahu et al., 2022). As technology and research advance, we can expand our species of study and delve into the communication sounds produced by mammals such as non-human primates, cetaceans, rodents, and bats.

Bats are the second largest order of mammals, with over 1400 species, each displaying unique communication calls. Most bat species are acoustic specialists that use echolocation (sonar-like vocalizations) to build a scene of their surroundings. By emitting high-frequency sounds and listening to the echoes that reflect off objects in their environment, they can navigate in low light conditions, enabling them to thrive in their nocturnal niche. Echolocation in bats has been widely studied. Bat researchers currently have access to commercial software such as Kaleidoscope (Wildlife acoustics) and SonoBat; and open-source software such as WEST-EchoVision (Tabak et al., 2022) and Waveman (Chen et al., 2020) that can be used to accurately identify bat species based on the acoustic features of their echolocation calls. These algorithms can be successfully employed to classify high numbers of bat species (Yoh et al., 2022), yet they are based on the echolocation calls produced by the animals and do not attempt to label communication calls within the repertoire of each species. Each bat species also produces its own repertoire of communication calls that differ in form and function from the echolocation calls, and for most species these calls are less well described. Thus, distinguishing within species among different types of communication calls remains more elusive, and labeling and categorizing calls that have not yet been described poses a new challenge. Our program aims to simplify the task of categorizing specific communication syllables within each specie’s repertoire. Contrary to echolocation calls, these communication calls are often produced in hard-to-reach roosts and during social interactions, thus, researchers have been less privy to these sounds. With over 1400 species of bats, approximately 1 % (15 species referred to in the literature to this date) of all bat species have been systematically studied in terms of their acoustic communication (Chaverri et al., 2018; Salles et al., 2019; Salles and Bohn, 2022). With broad repertoires and large differences across species, building adequate libraries is strenuous work.

Though several deep learning algorithms exist to classify dolphin, birds and other animals acoustic signals, most require hundreds or thousands of labeled exemplars as training data (Armitage and Ober, 2010; Fundel et al., 2023; Maegawa et al., 2021). This need for a high number of annotated and labeled calls to train deep learning neural nets presents a hurdle in advancing the study of bat auditory communication and, thus, the comparative approach to the neural mechanisms of acoustic communication across taxa.

Communication calls emitted by bats range in how often they are produced and can vary across geographical locations for different populations of the same species. Furthermore, they usually present individual signatures that make categorical classification difficult even for a trained human eye. For example, from the analysis of data for a previous publication from our group (Salles et al., 2024), reaching 45 clean (i.e., high signal-to-noise ratio, no overlap with other calls) exemplars for one category (QCF syllables of big brown bats) took 3 months of data collection in the lab and another 6 months of annotation to find them across the over 20,000 recorded calls. Considering this problem, we set to design a customizable framework that would enable simple classification based on small training data sets. Our software, BattyCoda, is an annotating tool that employs the acoustic features extracted by the R software package WarbleR (Araya-Salas and Smith-Vidaurre, 2017) from isolated calls and a simple classifier to suggest a categorical label for each call. The classifier can be re-trained with every additional dataset and can be set to skip human supervision during annotation if the confidence level for the categorization is above a threshold set by the investigator. In this way, calls that are easy to classify and for which we have many exemplars (i.e., echolocation calls) can be immediately labeled by the program without being checked by the annotator, while rarer calls can get a label suggestion from the classifier but can be confirmed or re-annotated by the investigator. In this way, the software makes call labeling faster than a human researcher combing through every individual call in the dataset. It reduces ambiguity in the categorization by suggesting the most likely category for each call based on its acoustic features. In this work, we present our new tool for acoustic call labeling, which can be used across taxa when labeled datasets are limited. BattyCoda is unique in the field as it is to our knowledge the only tool specifically designed to label and classify communication syllables within bat species. We also discuss the customization possibilities and compare the performance of different classifiers for our training data.

2. Methods

2.1. Acoustic recordings

Wild Eptesicus fuscus were captured in mistnets at abandoned mines in Vermont and New Hampshire under Vermont Fish and Wildlife Scientific Collection Permit (Authorization # SR-2014–09) and a New Hampshire Fish and Game Department Scientific Collection Permit (no number assigned). Nine adult male bats were physically identified as belonging to the species Eptesicus fuscus and were kept in the Dartmouth College animal care facility for several weeks under IACUC Protocol number is hofs.ht.2. At night, pairs of bats were flown together in a large, tunnel-shaped outdoor flight enclosure (7.6 m W × 7.6 m L × 4.0 m H) consisting of a metal frame and mesh walls. The enclosure was in a farm field distant from any anthropogenic light sources. Echolocation and communication calls produced by the bats in the enclosure were recorded at a sampling rate of 300 kHz using ultrasound-sensitive microphones (model CM16; Avisoft Bioacoustics, Glienicke, Germany) and a data acquisition board (Avisoft UltraSoundGate 416H) connected to a laptop computer running Avisoft RECORDER software. This acoustic recordings were used as the data set that was first manually annotated and then tested on the classifiers for this work.

2.2. Front-end labeling with BattyCoda

The first challenge bat researchers encounter when investigating communication calls is labeling calls into different call-type categories. Thus, we developed a platform that can be accessed online to label calls in audio files where call time events have been previously identified (segmented). This platform, or ‘front-end of BattyCoda’, is fully customizable for the species of interest (Fig. 1). The researcher can add a template figure of labeled spectrograms to aid the visual comparison with the new calls. Each new call is presented as a spectrogram with the following parameters: (nperseg = 2 ** 8, noverlap = 254, nfft = 2 ** 8). Additional audio channels are presented below the primary channel. A time-compressed spectrogram is also shown to visualize the syllable in its acoustic environment (500 ms). A list of possible labels is displayed to the right of the main spectrogram, and the annotator can choose from one before moving on to the next call. The researcher can manually enter this list as they set up the environment for the target species call classification. In this work, we first used the annotation capabilities of BattyCoda to manually inspect and label each call from the previously recorded dataset. Among the numerous species of bats, only a few have had their communication vocalization repertoire described (Salles et al., 2019). Among them, Big brown bats (Eptesicus fuscus) have well-described communication signals with a defined syllable nomenclature (Gadziola et al., 2012; Montoya et al., 2022; Wright et al., 2013). Thus, we chose this species as the target for the initial setup of the back-end classifier. We selected the 11 most common E. fuscus communication call type labels as described in Montoya et al., 2022 and used this as a template for the manual annotation of a database of 1506 calls (Fig. 2). The communication syllables for this species of bat are named: Frequency Modulated bouts (FMBs), Chevron Shaped call (CS), Downward frequency modulation (DFM), Long shallow frequency modulation downward to long frequency modulation (LsDFM-LFM), U shape call to long downward frequency modulation (U-LDFM), Single frequency modulation (SFM), Single humped frequency modulation (sHFM) Long frequency modulation (LFM), Long downward frequency modulation (LDFM), Quasi-constant frequency (QCF), Quasi-constant frequency to downward frequency modulation (QCF-DFM), Long quasi-constant frequency to chevron shape (LQCF-CS), and Upward frequency modulation (UFM).

Fig. 1.

Frontend of BattyCoda. The top left spectrogram shows the current call on the preferred channel. Other channels are available (if the recording was made with an array) in the spectrograms below it. The top right shows the same call spectrogram but zoomed out in time to see the acoustic context. Between the top two spectrograms, the customizable list of labels is displayed. The bottom right displays the species-specific template added by the user; in this case, calls of Eptesicus fuscus from Montoya et al., 2022.

Fig. 2.

Spectograms showing example vocalizations of the Eptesicus fuscus call repertoire. A) Echolocation calls B) Frequency Modulated Bouts - FMBs C) Chevron Shaped call - CS D) Downward frequency modulation - DFM E) Long shallow frequency modulation downward to long frequency modulation - LsDFM-LFM F) U shape call to long downward frequency modulation - U-LDFM G)Single frequency modulation - SFM H) Single humped frequency modulation- sHFM I) Long frequency modulation - LFM J) Long downward frequency modulation - LDFM K)Quasi-constant frequency - QCF L) Quasi-constant frequency to downward frequency modulation QCF-DFM M) Long quasi-constant frequency to chevron shape - LQCF-CS N) Upward frequency modulation - UFM.

2.3. Compiling an initial training dataset for automated back-end labeling

In addition to user-generated labels assigned in the front-end of BattyCoda, our aim was to have the program automatically assign a call type label in the back-end. To select and train a classification algorithm, we compiled an initial training dataset from the manually previously labeled calls. This then became the initial training data set for BattyCoda. Now, new data can be run through the program and depending on the level of accuracy set by the user, the program will automatically label new calls or show the investigator the call with a suggested label for approval or correction. For example, the investigator can set a threshold of 55 % accuracy and with the current training of the classifier, the program would automatically label calls identified as DFM, SFM, sHFM, FMB and Echolocation. While other calls would be displayed to the investigator with a suggested label that the investigator can correct. New labeled data for each species is added to the next iteration in the training of the classifier.

2.4. Selecting a classification algorithm

We considered a broad range of classification algorithms for evaluation varying in complexity and in their approaches to handling non-linear relationships, feature interactions, and class imbalances. We selected six candidate models for testing: support vector machines (SVM), k-nearest neighbors (KNN), random forest, multinomial logistic regression, XGBoost, and linear discriminant analysis (LDA) (see Appendix A for brief descriptions of each classifier). We used implementations of these candidate models that were capable of performing multi-class classification natively without needing to reduce the classification problem to multiple binary classifications. As expected, the distribution of observations for each call type in our dataset was greatly imbalanced, with echolocation calls representing the vast majority of the calls (almost 60 % of the calls, Fig. 3). To simulate a real-world scenario with a typical distribution of call types, we deliberately kept an imbalanced design in our data. This empirical distribution allowed us to evaluate how our classifiers would perform under such conditions. We randomly selected 70 % of the data for training and used the remaining 30 % to evaluate classifier performance. We scaled the features in the training data to standardize the 26 acoustic features extracted using WarbleR (Appendix B). We used the same scaling parameters (i.e., mean and standard deviation) to standardize the test data. This approach prevents data leakage by ensuring that information from the test set is not used during model training.

Fig. 3.

Mean classification accuracy by call type of our linear discriminant analysis model across 100 tests of training and testing data. Error bars represent standard error of the mean; white numbers indicate the number of exemplars per call type.

For this study, we deployed five of our six classifiers in the mlr3 package in R (Bischl et al., 2023), except multinomial logistic regression, which is not currently available within mlr3. We performed multinomial logistic regression using the “nnet” package (Venables and Ripley, 2002). We parameterized each model in the manner most appropriate for our classification problem and dataset. Hyperparameter tuning for the learning process was conducted using an automated random search with a batch size of five and a maximum of 50 evaluations. Five-fold cross-validation was used to evaluate model performance during hyperparameter tuning. The balanced classification accuracy across classes served as the performance measure for selecting the optimal hyperparameter combination for each classifier. The predictive performance of the classifiers was also assessed using the balanced accuracy metric, which calculates the recall of each class separately and averages them, ensuring that call types with the largest sample sizes do not overshadow the performance of underrepresented call types. Recall is calculated as the number of correctly identified calls in a class divided by the total number of calls in that class in the dataset. To account for stochasticity and ensure robust, generalizable results, we updated the training/testing data splits and re-ran each model 100 times.

2.5. Testing sample size sensitivity of the classification algorithm

One of the key challenges when building accurate classifiers for bat communication calls is that the sample size for certain types of call is often small. Therefore, we also tested the performance of our LDA classifier when trained with increasingly smaller subsets of our original training dataset. To achieve this, we followed the same training and testing procedure as described in Section 2.3, with the addition of an initial stratified sampling step that compiled a random subset of the original training dataset while maintaining the same distribution of exemplars across classes. Replicating the proportional distribution of observations across classes of the original dataset ensured that the subsets displayed the same level of imbalance of the original dataset and allowed us to evaluate accuracy while emphasizing the sample size of the minority class (i.e., call type with the fewest number of exemplars in the dataset). We ran this model using 80 % (n_total = 847, n_{minority class} = 8), 60 % (n_total = 637, n_{minority class} = 6), 45 % (n_total = 479, n_{minority class} = 5) and 20 % (n_total = 216, n_{minority class} = 2) of our original training dataset.

3. Results and discussion

The field of acoustic communication in bats is growing, yet necessary tools are lacking to advance the field. Syllable call labeling for bat acoustic data remains a slow and tedious process, usually done by a human annotator one syllable at a time. Our new software, BattyCoda, will support an increase in the speed at which manual annotation can happen. In this software and with a small sample size of labeled training data, a simple classifier provides a prediction regarding the potential label of the call. The user can set the accepted level of recall (confidence in the assigned label) for the program to automatically label the call. If a call label falls below the set threshold of confidence, the user has the ability to manually inspect the call and provide a user-generated label.

3.1. LDA was found to be the best performing classification algorithm

Our results show that LDA was the most accurate classification algorithm for our data (Table 1). Linear discriminant analysis demonstrated the highest predictive performance, with an average balanced accuracy of 0.496. XGBoost followed, with an average balanced accuracy of 0.458. The superior performance of LDA was surprising, given that it ranked among the most simple and least flexible classification algorithms relative to the other candidates, and in similar studies has been found to be a poor performer when compared to other methods (Armitage and Ober, 2010; Bourel and Segura, 2018). Classification by LDA works by transforming features into linear combinations that maximize the separation among classes (i.e., variance within groups versus among groups) and then using these transformed features to classify observations (Tharwat et al., 2017). Linear discriminant analysis works best when the delineations in the predictive features across classes are linear and well-defined (Bourel and Segura, 2018; Qin et al., 2022; Zhang et al., 2023). Our results suggest that this may be the case for bat call types, as there are clear patterns in our acoustic features distinguishing the call types (Fig. 4). The other candidate models may be overfitting the minority classes or may be hampered by potential noise in the data from acoustic features that are not contributing to the separation of classes. The fact that machine-learning model, XGBoost, a state-of-the-art, ensemble tree-based model, was found to be a worse classifier than LDA, may be evidence that such models could be overfitting the training data. Given that small sample size and severe class imbalance are expected characteristics of these types of bat communication call datasets, simpler classification algorithms, such as LDA, may be more suitable.

Table 1.

Results of classifier performance tests. Average balanced accuracy across 100 runs is reported for each classification algorithm along with 95 % confidence intervals. LDA = linear discriminant analysis; KNN = k-nearest neighbors; SVM = support vector machines.

Classifier	Balanced accuracy	95 % CI Upper	95 % CI Lower
LDA	0.496	0.506	0.486
XGBoost	0.458	0.465	0.450
Random Forest	0.453	0.462	0.444
Logistic Regression	0.452	0.460	0.443
KNN	0.426	0.435	0.417
SVM	0.414	0.420	0.407

Fig. 4.

Clustered heatmap of average values of the acoustic features for each call type. Similar call types are grouped using k-means clustering. Acoustic features were scaled prior to analyses, call type-averages for these features range from 3 to − 2.

3.2. Classification accuracy of call types using LDA

In our LDA models, echolocation calls were the most accurately classified, with nearly 90 % (0.896) of calls correctly labeled on average across runs (Fig. 3). FMB and sHFM calls were also classified with relatively high accuracy, at 71 % (0.711) and 70 % (0.702), respectively. The call types that were most often misclassified were UFM and LsDFMLFM, with only 21 % (0.211) and 22 % (0.217) of calls classified correctly on average, respectively. The classification accuracy of the call types correlated with the number of exemplars in our dataset. As expected, echolocation calls, the most overrepresented call type in our dataset (60 % of all calls), were classified with very high accuracy. Given that echolocation calls are likely to be overrepresented in other empirical data, the high-accuracy classification of echolocation calls could potentially reduce the manual inspection to just 40 % of the calls for a BattyCoda user. Our data suggest that other common communication calls are also classified with high accuracy, reducing the load even further. For example, sHFM and FMB syllables show high similarity of acoustic features but can be easily separated by the classifier (Figs. 4 and 5). Thus, for call types occurring at high numbers, user-driven categorization becomes less necessary, so calls can be labeled automatically. For rarer call types with lower levels of classification accuracy, automated-categorization can still be valuable for the user, helping reduce bias and improve annotation time across users by accepting the suggestion of the classifier, as it is easier to accept a suggestion than make one anew (Proctor and Schneider, 2018). Nonetheless, as the number of observations increases for rarer call types as new user-provided calls get integrated into the BattyCoda training dataset, so will the accuracy of their classification.

Fig. 5.

Confusion matrix for one of the training and testing rounds for LDA (set.seed = 47). The specific seed value used in this round (47) indicates the starting point for the random number generator, ensuring that data splitting and model training are reproducible. Correct classifications are displayed in the green-colored cells. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

In addition to sample size, our results suggest that classification accuracy can also be affected by the level of acoustic similarity with other call types. For example, LDFM calls have a large number of exemplars, but display a lower classification accuracy, as these calls are often classified as echolocation calls (Fig. 5). This is supported by the fact that LDFM and echolocation calls have many acoustic features in common (Fig. 4) and can sometimes be hard to distinguish even for human observers, needing to use duration and acoustic context to improve accuracy. It is noteworthy that the communication calls in our study fall into broader functional categories such as aggression/territoriality, distress, or affiliative/appeasement calls; therefore, call types within these broader categories may be harder to distinguish as they share acoustic features. Generally, for E. fuscus bats, affiliative/appeasement calls are long in duration and lower in frequency and intensity, while distress calls are loud, broadband, and show frequent amplitude modulations. Aggression/territorial calls, on the other hand, are shorter than the other two types, and often occur in sequences (Montoya et al., 2022).Some communication calls may resemble echolocation calls. A theory on why these calls may be acoustically similar suggests that the bats can co-opt communication calls made while bats fly to gather navigation information from echoes of these calls (Moss, 2018). Yet, it remains to be tested to determine to what extent these calls are used for navigation. Other theories suggest that the mechanical and respiratory constraints of producing calls in flight are minimal and coupled to flight itself (Voigt and Lewanzik, 2012); this may limit the feature range bats can exploit while flying, thus producing communication calls that resemble echolocation calls may be advantageous from an energetic perspective.

3.3. LDA was found to be robust for smaller training data sizes

We tested the minimum training sample size needed by our model to achieve a comparable level of performance. We found in this second test that our LDA model was robust across training data sizes, achieving average balanced accuracies similar to the average balanced accuracy of our LDA model trained with the entire training dataset (Table 2). The classification accuracy for each class was also found to be comparable to the results obtained for the original training dataset (Appendix C). Our results are particularly encouraging given that researchers are usually faced with the challenge of small sample sizes when collecting bat communication data. One of the reasons why obtaining large sample sizes for each call type is challenging is that bats emit most of their communication calls when roosting in places that are hard to access for humans (Kunz and Lumsden, 2003). Furthermore, even if calls can be recorded at their roosting site, some bats live in large colonies and their calls are embedded in a cacophony, making it difficult to separate clean exemplars for training data sets. When paired during experimental designs, the bats will emit few communication calls, often of varying types, so acquiring many exemplars of each call type time consuming. Lastly, there are over 1400 species of bats, and for most, their call repertoire has not been described, making this tool especially useful for researchers looking to start building labeled call libraries for under-described species.

Table 2.

Results of minimum training sample size tests. Average balanced accuracy across 100 runs is reported for each training dataset size along with 95 % confidence intervals.

Training data size	Balanced accuracy	95 % CI Upper	95 % CI Lower
80 %	0.497	0.523	0.471
60 %	0.495	0.522	0.468
45 %	0.48	0.509	0.455
20 %	0.451	0.48	0.422

3.4. Limitations

As with any classification tool, BattyCoda’s performance is constrained by both data availability and algorithmic limitations. For instance, although LDA was the best performing classifier in our study, it is also among the least flexible classification algorithms that we tested. The linear assumptions of this method may restrict performance in cases where the boundaries between call types are non-linear or feature interactions are more relevant. Furthermore, low classification accuracy of rare call types will remain a challenge when few labeled exemplars are available for training. As repertoires for communication calls of bats grow and individual syllables are defined more precisely, we expect these limitations to become less important.

4. Conclusion

In this work, we developed BattyCoda, a new tool for manual annotation and automated classification of acoustic data from different taxa using only a few exemplars for each syllable category. We found that linear discriminant analysis was the best performing classification algorithm for this purpose, achieving a balanced accuracy of ~50 %, and classification accuracies over 70 % for the most common bat call types. Additionally, we found that the classification accuracy of our tool remained relatively stable when trained with successively smaller datasets. BattyCoda is fully customizable to enable the user to train the classifier on any pre-segmented data set and upload both a figure template for the calls and a list of potential labels for the user and classifier to choose from.

Our future research will focus on using this tool to describe the vocal communication repertoire of bat species for which it hasn’t been described before. Further, we will use those repertoires to describe the social-behavioral ecology of those bats, hoping that it can aid in the conservation of lesser-known species. As we collect data from different species and build species-specific training data sets, we will make them available to the public so that other labs attempting to study those bats can more readily label and classify their data. Though initially developed for bat acoustic data, we believe that BattyCoda furthers the field of bioacoustics by providing a straightforward tool that can help describe the vocal repertoire of species for which acoustic libraries are still not compiled.

Supplementary Material

Appendix A. Supplementary data

Supplementary data to this article can be found online at https://doi.org/10.1016/j.ecoinf.2025.103195.

Data availability

All data and accompanying scripts used in this manuscript, as well as the BattyCoda software, are available on GitHub (https://github.com/angiesalles/battycoda).