Optimising predictive models to prioritise viral discovery in zoonotic reservoirs

Abstract

Despite the global investment in One Health disease surveillance, it remains difficult and costly to identify and monitor the wildlife reservoirs of novel zoonotic viruses. Statistical models can guide sampling target prioritisation, but the predictions from any given model might be highly uncertain; moreover, systematic model validation is rare, and the drivers of model performance are consequently under-documented. Here, we use the bat hosts of betacoronaviruses as a case study for the data-driven process of comparing and validating predictive models of probable reservoir hosts. In early 2020, we generated an ensemble of eight statistical models that predicted host–virus associations and developed priority sampling recommendations for potential bat reservoirs of betacoronaviruses and bridge hosts for SARS-CoV-2. During a time frame of more than a year, we tracked the discovery of 47 new bat hosts of betacoronaviruses, validated the initial predictions, and dynamically updated our analytical pipeline. We found that ecological trait-based models performed well at predicting these novel hosts, whereas network methods consistently performed approximately as well or worse than expected at random. These findings illustrate the importance of ensemble modelling as a buffer against mixed-model quality and highlight the value of including host ecology in predictive models. Our revised models showed an improved performance compared with the initial ensemble, and predicted more than 400 bat species globally that could be undetected betacoronavirus hosts. We show, through systematic validation, that machine learning models can help to optimise wildlife sampling for undiscovered viruses and illustrates how such approaches are best implemented through a dynamic process of prediction, data collection, validation, and updating.

Introduction

Identifying the probable reservoirs of zoonotic pathogens is challenging.¹ Sampling wildlife for the presence of an active or previous infection (ie, by testing for seropositivity) represents the first stage of a pipeline for the proper inference of a host species,² but sampling is often limited on a phylogenetic, temporal, and spatial scale by logistical constraints.³ Given such restrictions, statistical models can play a crucial role by helping to identify which pathogen surveillance targets are a priority, by narrowing the set of plausible sampling targets by either ruling out clades of low-likelihood hosts4, 5 or predicting clades at a high risk of being hosts.⁶ For example, machine learning approaches have generated candidate lists of probable, but unsampled, primate reservoirs of Zika virus, bat reservoirs of filoviruses, and avian reservoirs of Borrelia burgdorferi.7, 8, 9

At the same time, host predictions are rarely validated empirically.¹⁰ Occasional case studies suggest both success and failures. For example, models predicted Eonycteris spelaea as an undetected bat host of filoviruses,⁷ which was later confirmed by field sampling in southeast Asia.11, 12 Similarly, models of mosquito–Zika virus interactions predicted Culex quinquefasciatus as a probable vector,¹³ which was rapidly validated by experimental competence trials.14, 15 A 2019 model of Nipah virus in India also predicted several bat species as undetected hosts.² However, experimental infection of the predicted Rousettus aegyptiacus showed that this species could not support virus replication.¹⁶ Further, Nipah virus was found circulating in Pipistrellus pipistrellus in 2021, a species with a low predicted probability of being a host.¹⁷ More generally, predictions from most models are either untested or opportunistically validated, allowing for little insight into which approaches have greatest predictive accuracy. Systematically validating predictions would provide crucial insight into the broader usefulness (or inefficacy) of different models in zoonosis research. Moreover, these modelling approaches are generally developed in isolation; the implementation of multiple modelling approaches collaboratively and simultaneously, as part of a model-to-validation workflow, could reduce redundancy and apparent disagreement at the earliest stages of pathogen tracing at the same time as advancing predictive analytics by addressing inter-model reliability.

Coronaviruses are an ideal family of viruses with which to compare and validate predictive models of probable zoonotic reservoirs. Coronaviruses are positive-sense, single-stranded RNA viruses that have been detected in both mammals and birds.¹⁸ They have a broad host range, a high mutation rate, and the largest genomes of any RNA virus; but they have also evolved mechanisms for RNA proofreading and repair to mitigate the deleterious effects of a high recombination rate acting over a large genome.¹⁹ Consequently, coronaviruses fit the profile of viruses with a high potential for being a zoonotic disease. There are eight human coronaviruses (three in the genera alphacoronavirus and five in the genera betacoronavirus), of which three are highly pathogenic in humans: SARS-CoV, MERS-CoV, and SARS-CoV-2. These viruses are zoonotic and widely agreed to have evolutionary origins in bats.20, 21, 22, 23

The challenges caused by both SARS-CoV and MERS-CoV illustrate the difficulty of tracing the specific animal hosts of emerging viruses. During the 2002–03 severe acute respiratory syndrome (SARS) epidemic, SARS-CoV was traced to the masked palm civet (Paguma larvata),²⁴ but the ultimate origin was unknown for several years. Horseshoe bats (family Rhinolophidae, genus Rhinolophus) were implicated as reservoir hosts in 2005, but their SARS-like coronaviruses were not identical to circulating human strains.²¹ Stronger data from 2017 placed the most likely evolutionary origin of SARS-CoV in Rhinolophus ferrumequinum or Rhinolophus sinicus.²⁵ There is even less certainty about the origins of MERS-CoV, although spillover to humans often occurs through contact with dromedary camels (Camelus dromedarius). A virus with 100% nucleotide identity in an approximately 200 base pair region of the MERS-CoV polymerase gene was detected in Taphozous perforatus (family Emballonuridae) in Saudi Arabia;²⁶ however, based on spike gene similarity, other sources treat the HKU4 virus from Tylonycteris pachypus (family Vespertilionidae) in China as the most closely related bat virus to MERS-CoV.27, 28 Several bat coronaviruses have shown close phylogenetic relationships with MERS-CoV, with a surprisingly broad geographical distribution from Mexico to China.29, 30, 31, 32

COVID-19 is caused by SARS-CoV-2, a novel virus with presumed evolutionary origins in bats. Although the earliest cases were linked to a wildlife market,²³ contact tracing was low, and there has been no definitive identification of the wildlife contact that resulted in the spillover nor a true so-called index case. The divergence time between SARS-CoV-2 and two of the closest related bat viruses (RaTG13 from Rhinolophus affinis and RmYN02 from Rhinolophus malayanus) has been estimated to be 40–50 years,³³ suggesting that the main host(s) involved in the spillover are unknown. A viral recombination in pangolins has been suggested but is unconfirmed.³³ In 2020, SARS-like betacoronaviruses were isolated from Sunda pangolins (Manis javanica) traded in wildlife markets,34, 35 and these viruses have a high amino acid identity to SARS-CoV-2, but only show an approximately 90% similarity in nucleotide identity with SARS-CoV-2 or bat coronavirus RaTG13.³⁶ None of these host species are universally accepted as the origin of SARS-CoV-2 nor are any of the viruses a clear SARS-CoV-2 progenitor, and a better fit wildlife reservoir could still be identified. However, substantial gaps in betacoronavirus sampling across wildlife reduce how much actionable inference can be made about plausible reservoir hosts and bridge hosts for SARS-CoV-2.³⁷

Building a predictive ensemble

Here, we use betacoronaviruses in bats as a case study for the data-driven process of comparing and validating predictive models of probable reservoir hosts, with the aim of helping to identify which targets to prioritise for surveillance for known and future zoonotic viruses. We focused on betacoronaviruses rather than SARS-like coronaviruses (subgenus Sarbecovirus) specifically, because SARS-like coronaviruses are only characterised from a small number of bat species in publicly available data. This sparsity makes current modelling methods poorly suited to more precisely infer potential reservoir hosts of Sarbecoviruses specifically. Instead, we used predictive models to firstly identify bats (and other mammals) that might broadly host any betacoronavirus, and secondly to identify species with a high viral sharing probability with the two Rhinolophus species carrying the earliest known close viral relatives of SARS-CoV-2. In mid-2020, in the early stages of the COVID-19 pandemic, we developed a standardised dataset of mammal–virus associations by integrating a previously published edge list³⁸ with a targeted scrape of all GenBank accessions for Coronaviridae and their associated hosts. Our final dataset spanned 710 host species and 359 virus genera, including 107 mammal hosts of betacoronaviruses as well as hundreds of other (non-coronavirus) association records. We integrated our host–virus data with a mammal phylogenetic supertree³⁹ and more than 60 standardised ecological traits of bat species.7, 40, 41

We then used these data to generate an ensemble of predictive models and drew on two popular approaches, network-based and trait-based approaches, as well as a hybrid approach, to identify the candidate bat reservoir hosts of betacoronaviruses (table ). Network-based methods estimate a full set of true unobserved host–virus interactions on the basis of a recorded network of associations (here, pairs of host species and associated viral genera). These methods are increasingly popular to identify latent processes structuring ecological networks,42, 43, 44 but they are often confounded by sampling bias and often can only make predictions for species within the observed network (ie, those that have available virus data; in-sample prediction). In contrast, trait-based methods use observed relationships concerning host traits to identify species that fit the morphological, ecological, or phylogenetic profile of known host species of a given pathogen, and rank the suitability of unknown hosts on the basis of these trait characteristics.8, 45 These methods might be more likely to recapitulate patterns in observed host–pathogen association data (eg, geographical biases in sampling and phylogenetic similarity in host morphology), but they more easily correct for sampling bias and can predict host species without known viral associations (ie, out-of-sample prediction).

Table.

Scope and calibration of different predictive modelling approaches

	Prediction on hosts without known associations (out of sample)	Predictive extent and use of pseudoabsences
Network 1: k-nearest neighbours	No	Only predicts link probabilities among species in the association data
Network 2: linear filter	No	Only predicts link probabilities among species in the association data
Network 3: plug and play	No	Uses pseudoabsences to predict over all mammals in association data, using latent approach
Network 4: scaled phylogeny	No	Only predicts link probabilities among species in the association data
Trait 1: boosted regression trees	Yes	Uses pseudoabsences for all bats in trait data to predict over all species, including those without known associations
Trait 2: Bayesian additive regression trees	Yes	Uses pseudoabsences for all bats in trait data to predict over all species, including those without known associations
Trait 3: neutral phylogeographic	Yes	Trains on a broader network, and predicts sharing probabilities among any mammals in phylogeny and International Union for Conservation of Nature range map data
Hybrid 1: two-step kernel ridge regression	Yes	Uses pseudoabsences for all bats in trait data to predict over all species, including those without known associations

Some methods use pseudoabsences to expand the scale of prediction but still only analyse existing host–virus data, with no out-of-sample inference, whereas other methods can predict onto new data.

In total, we implemented eight different predictive models of host–virus associations, including four network-based approaches, three trait-based approaches, and one hybrid approach using trait and phylogenetic information to make network predictions. These efforts generated eight ranked lists of suspected bat hosts of betacoronaviruses. Each ranked list was then scaled proportionally and consolidated in an ensemble of recommendations for betacoronavirus sampling and broader ecological–evolutionary research. Next, approximately 1 year after our initial model ensemble, we reran our entire analytical pipeline with new bat betacoronavirus detections, taking advantage of the new proliferation of published research on bat coronaviruses. This process provided an unprecedented opportunity to rapidly compare model performance, provide up-to-date predictions of probable but unsampled bat hosts, and assess model accuracy in the context of ongoing sampling for bat coronaviruses.

Predicted bat reservoirs of betacoronaviruses

Our initial ensemble found a wide variation in model performance; individual models explained 0–69% of the variance in betacoronavirus positivity (with a mean of 25%), whereas the ensemble generally had improved predictive capacity (R ²=42%; appendix p 22). The predictions of bat betacoronavirus hosts derived from network-based and trait-based modelling approaches displayed strong inter-model agreement within each group but largely differed between groups (as measured by pairwise Spearman's rank correlations among predicted ranks; figure 1 ). Of the 1037 included bat species not known to be infected by betacoronaviruses during our initial analysis in 2020 (against 79 bat species known to be infected), our models identified between 7 and 723 potential hosts on the basis of a 10% omission threshold (90% sensitivity). Applying this same threshold to our ensemble predictions, our initial models identified 371 bat species as probable undetected hosts. Notably, only 48 suspect hosts were identified in sample, whereas we identified 323 suspect hosts out of sample, highlighting that most undiscovered hosts—and, in turn, undiscovered betacoronaviruses—should be in unsampled bat species.

Figure 1.

Agreement across an ensemble of predictive modelling approaches

Agreement across models identifying hosts with available virus data (in sample) (A) and without known viral associations (out of sample) (B). The pairwise Spearman's rank correlations between models' ranked species-level predictions were generally substantial and positive. Models were arranged in decreasing order of their mean correlation with other models. Models that used trait data made more similar predictions to each other than approaches using network methods with the same data. Network-based models that used some ecological data made more similar predictions than all other models (eg, network 4, which uses phylogeny, and hybrid 1, which uses both phylogeny and trait data). All models that could make out-of-sample predictions used trait data and showed strong agreement.

This multi-model ensemble predicted undiscovered betacoronavirus bat hosts with notable geographical and taxonomic patterning (figure 2 ). In-sample predicted hosts were globally distributed and recapitulated geographical patterns of known bat betacoronavirus hosts in Europe, the Neotropics, and southeast Asia; however, our models also predicted a high richness of probable bat reservoirs in North America. Applying a graph partitioning algorithm (phylogenetic factorisation) to the bat phylogeny,⁴⁶ we similarly found that both betacoronavirus positivity and in-sample predictions were, on average, lowest for the superfamilies Noctilionoidea and Vespertilionoidea in the suborder Yangochiroptera. This finding makes intuitive sense, because these taxa do not include the groups known to harbour most of the betacoronaviruses detected in bats (eg, Rhinolophus and Hipposideridae). In contrast, our out-of-sample predicted hosts were more notably clustered in much of sub-Saharan Africa and southeast Asia (eg, Vietnam, Myanmar, and southern China), with no representation in the western hemisphere. Likewise, out-of-sample predictions were lower in Neotropical bat families (eg, Noctilionidae, Mormoopidae, and Phyllostomidae), most emballonurids, and primarily Neotropical molossids; whereas the Rhinolophus genus and most of the Old World subfamily Pteropodinae were predicted to be more likely to host betacoronaviruses (appendix p 18).

Figure 2.

Initial ensemble predictions of the geographical and evolutionary distribution of known and predicted bat hosts of betacoronaviruses

Known hosts of betacoronaviruses (A,B) are found worldwide, but particularly in southern Asia and southern Europe. Taxonomically, betacoronaviruses are less common in two superfamilies of the suborder Yangochiroptera, Noctilionoidea, and Vespertilionoidea (clade 1). The predicted in-sample bat hosts (ie, those with any viral association records; C,D) tend to recapitulate observed geographical patterns of known hosts but with a higher concentration in the Neotropics. Similarly, taxonomic patterns reflect those of known betacoronavirus hosts. In contrast, the out-of-sample bat host predictions based on phylogeny and ecological traits (E,F) are mostly clustered in Myanmar, Vietnam, and southern China, with none in the Neotropics, and North America. Predicted hosts are likewise more common in the Rhinolophidae (clade 2) and subfamilies of Old World bats (clade 5) and are rare in many Neotropical taxa (clades 1 and 7) and emballanurids (clades 3 and 4). In the phylogenies, bar height indicates betacoronavirus positivity (B) or predicted rank (D,F; higher values indicate lower proportional ranks). Colours indicate likelihood of clades to contain hosts identified through phylogenetic factorisation (red indicates clades more likely to contain hosts, blue indicates less likely hosts; appendix).

Because only trait-based models were capable of out-of-sample prediction, the differences in geographical and taxonomic patterns of our predictions probably reflect distinctions between the network-based and trait-based modelling approaches. We suggest that these should be considered as qualitatively different lines of evidence. Network approaches proportionally upweight species with a high observed viral diversity, recapitulating sampling biases largely unrelated to coronaviruses (eg, frequent screening for rabies lyssaviruses in the common vampire bat Desmodus rotundus, which has been sampled only a few times for coronaviruses31, 47, 48, 49). Highly ranked species might also have been previously sampled without evidence of a betacoronavirus presence; for example, Rhinolophus luctus from China and Macroglossus sobrinus from Thailand tested negative for betacoronaviruses, but the detection probability was limited by small sample sizes.50, 51, 52 In contrast, trait-based approaches are constrained by their reliance on phylogeny, ecological traits, and geographical covariates, all of which made the models more likely to recapitulate existing spatial (ie, clustering in southeast Asia) and taxonomic (ie, the Rhinolophus genus) patterns. However, out-of-sample predictions are, by definition, inclusive of unsampled bat hosts,⁵³ which potentially offer a greater return on viral discovery investment.

Model validation

After this initial 2020 model ensemble, we used broad literature searches to systematically track betacoronavirus-positive bat species that were missed in our initial data compilation (eg, coronavirus sequences that were not annotated to genus on GenBank).52, 54 These searches also tracked the exponential increase in data on bat coronaviruses stemming from the emergence of SARS-CoV-2 that were published after our first model ensemble. This informal non-systematic Review used a combination of a Web of Science and PubMed searches (on Sept 24, 2020) and an ongoing Google Scholar search to update these results (from May 24, 2020, to Sept 30, 2021), including papers in English, with keywords such as “bat” and “betacoronavirus”; all specific search terms and species identified are given in the appendix (p 14). A year after our initial data compilation (in June, 2021), we also reran our initial scrape of GenBank to identify new betacoronavirus-positive bats, limiting our search to matches to betacoronavirus (taxid: 694002) and the order Chiroptera (taxid: 9397); however, this did not recover any additional host species positive for betacoronavirus not already recorded as positives in our updated data. We also mined publicly available metagenomic and transcriptomic datasets for evidence of a betacoronavirus infection.55, 56, 57 However, no published libraries contained evidence of betacoronaviruses (appendix p 15). Lastly, we analysed the wildlife testing data from the US Agency for International Development Emerging Pandemic Threats PREDICT programme, collected from 2009 to 2019, which was publicly released in June, 2021, and includes many betacoronaviruses that were discovered during the programme's run but have only published and identified down to the genus level in the full release.

In total, we uncovered 47 novel bat hosts of betacoronaviruses that were either absent from our original dataset or newly discovered after our initial analyses. This data update resulted in a total of 126 known bat species that host betacoronaviruses, and we continue to collate these records in a public online database. Of these 47 new hosts, the original ensemble correctly predicted only 36 (77% success rate), but some sub-models performed significantly better than others; for instance, three models (trait 3, network 1, and network 4) all correctly identified 100% of novel hosts in their predictive sample. The high performance of all these models, and their high performance on the training data (appendix p 22), suggest that both approaches contributed usefully to the initial ensemble.

The 47 newly discovered hosts also enabled us to develop a new kind of performance metric for machine learning tasks with presence-only validation data (ie, new positives can be collected, whereas negatives are substantially more difficult to prove). If a model makes predictions at random, the predicted prevalence of positives in the training data should be roughly the same as the success rate with novel test data. For example, a so-called coin toss model will estimate that approximately 50% of species are betacoronavirus hosts and would likewise successfully identify approximately 50% of newly discovered hosts. A high-performing model, however, will identify a higher proportion of newly discovered hosts than expected at random. To evaluate how models perform in this regard, we developed a new diagnostic called the training prevalence-test sensitivity curve (TPTSC) that can be applied to modelling problems where the training data are composed of a mix of true positives, true negatives, and false negatives, but test data only include novel true positives (figure 3 ). The TPTSC plots the assumed prevalence in the training data against the sensitivity in the test data at every possible threshold from 0 to 1; these curves can be treated similarly to receiver–operator or precision-recall curves, where a higher area under the curve (AUC) indicates a better-than-random performance. Using the AUC-TPTSC scores, we found that trait-based and hybrid models consistently performed well (trait 1, AUC-TPTSC 0·79; trait 2, 0·78; trait 3, 0·73; hybrid 1, 0·67), whereas network methods performed at random or worse (network 1, 0·56; network 2, 0·42; network 3, 0·50; network 4, 0·52). Accordingly, the ensemble model performed similarly to the trait-based models (0·75).

Figure 3.

Measuring model performance with novel data

Performance is based on the comparison of total predicted prevalence (ie, what proportion of species are predicted hosts of betacoronaviruses) with the sensitivity measured from validation data (ie, how many of the 47 new species are correctly identified). The null expectation for a model with a random performance is that these should be equivalent, whereas a model with strong performance will be more than that null expectation (grey line). (A) The training prevalence–test sensitivity curve is a novel diagnostic that is conceptually similar to the receiver–operator curve, in that the model is evaluated at each possible scaled rank threshold between 0 and 1. (B) The same analysis as shown in (A), but only showing the point estimate of positivity created by each model's internally calibrated threshold. For model-guided sampling, the best model would be one that predicts a low-to-medium positivity rate and has a disproportionately high sensitivity (ie, in the upper left corner). Both (A) and (B) show that the trait-based models (including the hybrid model) perform well, whereas the network-only models perform roughly at-random or worse than random (ie, close to the line); the ensemble model, which includes all eight, performs similarly to the two best trait models and better than six of the eight component models.

These results have two key implications for future efforts in target sampling for putative reservoir hosts. First, ensemble modelling can be useful as a buffer against a variation in model quality, particularly in settings when the underlying drivers of model performance have yet to be identified. Second, and perhaps more importantly, models have been unable to have better-than-random performance without trait data that characterised bat ecology, even when they included phylogenetic data (eg, network 4). Part of this difference might also be attributable to the different scope of prediction: the response variable of trait-based models is betacoronavirus presence, whereas betacoronavirus-relevant predictions were extracted from a broader set of predictions made by the network models. However, this is contraindicated by the results of hybrid 1, which performed similarly to the other trait-based models. Therefore, we conclude that making meaningful predictions about probable zoonotic reservoirs is best accomplished by incorporating detailed information on the host ecology. The substantially greater performance of trait-based models compared with network-based models provides another compelling reason, in addition to other One Health and conservation rationales, to better understand the fundamental ecology and evolution of bats.

Dynamic prediction

Inclusion of these 47 novel bat hosts substantially improved the performance of our predictive models. When revised with new data, our eight individual models explained 7–77% (mean, 33%) of the variance in betacoronavirus positivity, with the ensemble R ² increasing to 62% (appendix p 23). Using our previously applied 90% sensitivity threshold, our revised ensemble identified a narrower set of 318 bat species as probable undetected hosts of betacoronaviruses. Predictions from the initial and revised ensembles were strongly correlated (ρ=0·97). However, after dynamically updating our models, our revised ensemble lost 46 suspected reservoirs and gained 29 new suspected reservoirs (figure 4A ). The predicted reservoir species that were lost from the initial ensemble were dominated by members of the family Vespertillionidae, whereas new suspect hosts were gained in the family Vespertillionidae, Hipposideridae, and Molossidae.

Figure 4.

Comparing bat betacoronavirus host prediction with dynamic model updates

Scatterplots show bat species predictions from our original ensemble in 2020 against the revised predictions after updating models with 47 new hosts (A), and the final predictions from the weighted revised ensemble (B). Species are coloured by their status in the respective revised ensemble: unlikely host, a retained suspected host, a new betacoronavirus-positive host (new host), lost as a suspected host (lost), or a novel suspected host (gained). Trendlines show a linear regression fit between the original and revised predictions against a 1:1 line, whereas dashed lines display the threshold cutoffs from each ensemble. The top ten in-sample and out-of-sample predictions from the original (C) and final (D) ensemble are also listed. *Five of the original top ten in-sample predictions, and one of the top ten out-of-sample predictions, have been empirically confirmed since the first iteration of our study.

Using the 47 newly discovered hosts, we were also able to tailor the updated ensemble responsively to model performance. To do so, we weighted the rank averaging across models based on their AUC-TPTSC score relative to the lowest performing model (network 2). In doing so, we effectively dropped network 2 from the ensemble, a choice supported by the fact the model's predictions were substantially poorer than expected at random. In the original ensemble, this correction would have created a marginal improvement in the model performance (unweighted ensemble: AUC-TPTSC, 0·746; weighted ensemble: AUC-TPTSC, 0·783). Therefore, we applied this weighting to the ensemble of updated predictions in the final copy released with this study.

This weighted, revised ensemble identified 412 suspect bat hosts, substantially expanding the scope of plausible candidates for future virus discovery compared with the two previous unweighted ensembles (figure 4B). Predictions from this final ensemble iteration were slightly less correlated with those from our initial ensemble (ρ=0·92) than those in the unweighted revised ensemble, and these final predictions retained most of the suspected hosts from the original ensemble. The top-ranked undiscovered hosts retained between our model updates included Murina leucogaster, Myotis nattereri, M blythii, Barbastella barbastellus, and Taphozous melanopogon in-sample, and the top out-of-sample hosts consistent between ensembles included Macroglossus sobrinus, Rhinolophus fumigatus, R marshalli, and Sphaerias blanfordi (figure 4C). Only 30 predicted hosts were lost, most of which were from the Pteropodidae and Vespertilionidae families, and the Rhinolophus genus. Of the 107 additional predicted hosts added to our final ensemble, most of these bat species were observed in the families Vespertilionidae (primarily the genus Myotis), Pteropodidae (primarily the genus Pteropus), Molossidae (primarily the genus Mops), and Hipposideridae (all in the genus Hipposideros), although we also identified several new predicted betacoronavirus hosts in the families Nycteridae, Emballonuridae, Rhinolophidae, and Phyllostomidae. The top-ranked novel predicted in-sample hosts included Myotis myotis, Molossus nigricans, Hipposideros bicolor, Pteropus scapulatus, and P vampyrus, and the most likely new out-of-sample hosts included Myotis chinensis, M altarium, Nycticeinops schlieffeni, Pipistrellus rueppellii, and Scotophilus viridis (figure 4D).

For Rhinolophus bats specifically, our final ensemble identified 44 suspected hosts relative to 21 known hosts, suggesting that more than two thirds of potential reservoirs in this genus could still be unidentified. Given the known roles of Rhinolophus bats as hosts of SARS-like coronaviruses,18, 21, 50 it is notable that our results suggest that the diversity of these viruses could be undescribed from approximately three quarters of Rhinolophus species not currently known to be hosts.

As in our initial ensemble, we lastly evaluated the geographical and taxonomic patterns in this finalised set of predicted betacoronavirus hosts. Spatially, undiscovered bat hosts were globally distributed (in not only the eastern but also western hemisphere), especially concentrated within a narrower band of equatorial sub-Saharan Africa, and more starkly in Malaysia and Borneo (figure 5A ). Notably, the geography of these predicted hosts contrasted with the distributions of both known bat hosts and probable hosts from our initial ensemble, each of which instead showed a stronger hotspot in southern China. We also identified distinct clades of bats highly predicted to be hosts by the weighted revised ensemble (figure 5B; appendix p 19). Both the Rhinolophus genus and subclades of the Pteropodidae family again had greater concentrations of predicted betacoronavirus hosts, although the phylogenetic factorisation now identified the Old World Molossidae family (ie, genus Mops and Chaerophon) as particularly likely to host these viruses, even though the Molossidae family as a whole had lower mean probabilities of having betacoronavirus hosts.

Figure 5.

Updated ensemble model predictions of geographical and evolutionary hotspots of bat betacoronavirus hosts

(A) Geographical map of the weighted revised ensemble predictions. Most predicted undiscovered betacoronavirus hosts were found in sub-Saharan Africa and southeast Asia, especially in Malaysia and Borneo (and less so in the high-elevation mainland hotspots where most reservoirs of severe acute respiratory syndrome coronavirus-like viruses are found). (B) Phylogeny of the weighted revised ensemble predictions. Predicted hosts from this final ensemble were also most likely in the Rhinolophus genus (clade 7), several subclades of the Pteropodidae (clades 5 and 6), and the Old World Molossidae (clade 8), even though the Molossidae family as a whole had less likely hosts (clade 3). Bar height in the phylogeny indicates predicted rank, and colours indicate clades identified through phylogenetic factorisation (red indicates clades more likely to contain hosts, blue indicates clades less likely to contain hosts; appendix p 19).

These geographical hotspots and clade-specific patterns of predictions could be particularly applicable for guiding future viral discovery and surveillance. On the one hand, betacoronavirus sampling in southeast Asian bat taxa (especially the genus Rhinolophus) might have a high success of viral detection (and isolation) of sarbecoviruses specifically, but might not substantially improve existing bat sampling gaps.⁵ On the other hand, the discovery of novel betacoronaviruses in pteropodid clades, Old World Molossidae, and bats in the Neotropics could substantially revise our understanding of the bat–virus association network relative to the coevolutionary distribution of bat betacoronaviruses.³⁸ For example, predicted bat hosts in the Neotropics might be unlikely reservoirs of sarbecoviruses (given their known distribution in the eastern hemisphere) but would be expected to carry novel viruses from the subgenus merbecovirus. Such discoveries could be particularly important for global health security, given the surprising identification of MERS-like viruses within the merbecoviruses in Mexican and Belizean bats31, 58 and the likelihood that post-COVID research efforts will focus disproportionately on Asia, despite the near-global presence of bat betacoronavirus hosts.

Insight into SARS-CoV-2's emergence

Our work suggests that more than 400 species of bats might host undiscovered betacoronaviruses and that these species can be prioritised for sampling more efficiently via machine learning. Although our models do not target sarbecoviruses specifically, these efforts might help to find more SARS-like viruses in wildlife and might even uncover the direct progenitor of SARS-CoV-2, particularly given that 44 species of horseshoe bats are predicted to host undiscovered betacoronaviruses. However, our models provide otherwise limited insight into the origins of SARS-CoV-2, given the probable role of non-bat bridge hosts in spillover to humans.59, 60 We, thus, attempted a similar model ensemble in June, 2020, using five of our eight models to predict the broader mammal–virus network with a focus on potential betacoronavirus bridge hosts. At the time, only 30 non-bat hosts of betacoronaviruses were available in our data. Among the five models, we found a poor concordance in predictions (appendix p 24). The predictions were also heavily biased towards well studied and domesticated mammals (eg, Ovis aries, Vulpes vulpes, Capra hircus, Procyon lotor, and Rattus rattus), indicating that the sampling bias dwarfed biological signals. As such, we evaluated these models as having little value or consistency for an ensemble. This finding might be relevant given other studies have also modelled the susceptibility to SARS-CoV-2 across mammals; however, some have used more detailed trait data and thus probably make better predictions on this broader taxonomic scale.61, 62

Instead of further calibrating this mammal-wide ensemble, we focused on the outputs of trait 3, which predicted how species should share viruses in nature based on their evolutionary history and geography. In June, 2020, we predicted the mammals expected to share viruses with R affinis and R malayanus, which hosted the two viruses (RaTG13 and RmYN02) most relevant to SARS-CoV-2's origins known at that time23, 63 (a closer related virus, RpYN06, has since been discovered in Rhinolophus pusillus).⁶⁴ We predicted that these two bat species are disproportionately more likely to share viruses with pangolins (of the order Pholidota) and carnivores (of the order Carnivora), including civets (Viverridae family), mustelids (Mustelidae family), and cats (Felidae family; appendix p 25). These predictions have been broadly validated by the role of the masked palm civet (P larvata) in the original SARS-CoV outbreak,65, 66 the discovery of SARS-CoV-2-like viruses in the Sunda pangolin (Manis javanica),³⁴ and extensive so-called spillback of SARS-CoV-2 into captive big cats, domestic cats, and both farmed and wild mink.67, 68 Notably, only the association between palm civets and SARS-CoV was present in the training data used for generating predictions from trait 3.

Given these successful predictions, we expect there might be potential insights into SARS-CoV-2's emergence when these predictions are paired with data on wildlife supply chains. Of the top 30 species (figure 6 ), two are known to have been traded in wildlife markets in Wuhan, China, immediately before the pandemic (the wild boar, Sus scrofa, and the palm civet, Paguma larvata), as were two species closely related to those in the top predictions (the Siberian weasel, Mustela siberica; the northern hog badger, Arctonyx albogularis).⁷⁰ Another top species, the Burmese ferret badger (Melogale personata), was also reportedly of interest in WHO's origins investigation.⁷¹ Our models indicate that any of these species would be expected to regularly share viruses with relevant Rhinolophus bats in nature. Although bats use habitats differently than most of these probable bridge host species, opportunities for contact exist: one study from Gabon found cohabitation among pangolins, bats, and other mammals in burrows.⁷² Many species potentially implicated in the origins of SARS-CoV-2 could therefore have plausibly acquired a progenitor to SARS-CoV-2 in nature, at some point before contact with, and spillover into, humans.⁷³ We suggest that this shortlist of species (figure 6) might, therefore, be useful for further investigations into the identity of potential bridge hosts, especially in combination with the experimental evaluation of susceptibility.

Figure 6.

Potential bridge hosts involved in SARS-CoV-2's emergence

Each dot represents predicted species-level sharing probabilities with Rhinolophus affinis (A) and R malayanus (B), estimated according to the phylogeographical viral sharing model trait-3.⁶⁹ Each coloured point is a different mammal species. Black points and error bars denote the means and standard errors of viral sharing probability for each order; the mammal orders are arranged according to their mean sharing probability, ascending from left to right. The tables below report the top 15 predicted non-bat species for R affinis and R malayanus; several families are disproportionately represented, including pangolins (order, Pholidota; family, Manidae), mustelids (order, Carnivora; family, Mustelidae), and civets (order, Carnivora; family, Viverridae). Notable species are bolded (ordered based on immediate relevance to possible origins): (a) the wild boar S scrofa and palm civet P larvata were both traded in wildlife markets in Wuhan, China, before the pandemic; as were (b) close relatives of the greater hog badger, A collaris, (the northern hog badger, A albogularis), and of the mountain weasel, M altaica, and Malayan weasel, M nudipes (the Siberian weasel, M siberica). (c) SARS-CoV-2-like viruses have been found in traded Sunda pangolins (M javanica) outside of Wuhan, China, though the species was not reported in Wuhan. (d) The ferret badger (M personata) was also reportedly of interest in WHO's origins investigation, which explored the role of wildlife farm supply chains.

Conclusions

This Review is the first to show, by using predictive validation, that machine learning models could help to optimise wildlife sampling for undiscovered viruses. As such, the growing toolkit of models that predict host–pathogen interactions are likely to aid future efforts both to predict and prevent pandemics and to trace the origins of novel infections after they emerge. However, these tools will work best if they are implemented through a dynamic process of prediction, data collection, validation, and updating, as we have implemented here. Although some previous studies have incidentally tested specific hypotheses (eg, filovirus models and bat surveys,7, 11 henipavirus models and experimental infections,2, 16 and vector–virus models and competence trials13, 14, 15), predictions are almost never subject to systematic verification. More dialogue between modelers and empiricists is necessary to confront this research gap. This improved communication is particularly necessary when establishing a species' role as a viral reservoir rather than incidental hosts; susceptibility is only one aspect of host competence.1, 10 Future work, including the longitudinal tracking of viral shedding over space and time, the isolation of the live virus from wild animals, and the experimental confirmation of viral replication, can support more robust conclusions about whether predicted host species actually play a role in viral maintenance¹⁶ as well as inform related efforts to pinpoint and minimise risk factors for pathogen spillover.74, 75

This Review is also the first to benchmark the performance of a set of differently calibrated and designed statistical models all trained for one host prediction task. We found a range in model performances, even among a set of models that all performed well on training data. This finding underscores the need to incorporate long-term validation into similar studies and suggests there might be key lessons about viral ecology to be learned from this type of process. In our study, we found that network-based models performed mostly at random in validation against new bat hosts, whereas trait-based models were more successful in their predictions. There are two possible explanations for this difference in model performance. First, models that successfully predict the broader mammal–virus network are likely to vary in performance when subset to any given node. This likelihood might seem contradictory to the idea that understanding the broader so-called rules of life underpinning mammal–virus interactions will improve predictions in specific cases. However, there are ways to combine the strengths of both network-based and trait-based approaches. In a similar study, network-based predictions of zoonotic risk performed essentially at random, whereas a hybrid approach that embedded network predictions in targeted, trait-based models performed better than any approach in isolation.⁷⁶ Future work should aim to develop and benchmark these types of hybrid model approaches more extensively and treat exclusively network-driven predictions with caution in the interim.

Second, models that integrate data on host ecology, evolution, and biogeography are likely to make more powerful predictions than those that mostly do not incorporate biology. This finding has many broader implications. Most notably, it suggests that filling gaps in the basic biology of bats is a key step towards zoonotic risk assessment and can benefit both pandemic prevention and bat conservation. High-quality host genomes are crucial to developing better predictive features, including genome composition bias metrics, improved host phylogenetic trees, and immunological traits.62, 77, 78, 79, 80 Whole-genome sequencing through initiatives such as the Bat1K project will expand the sparse available data on bat genomics and can facilitate other insights into the immune pathways used by bats to harbour virulent viruses.81, 82, 83 Targeted sequencing could also identify endogenised viral elements in bat genomes, shedding light on bat virus diversity and the evolution of bat immune systems.84, 85 Large-scale research networks, such as the Global Union of Bat Diversity Networks and its member networks, will further facilitate efficient sample sharing and ensure proper partnerships and equitable access and benefit sharing of knowledge across countries.86, 87 Additionally, museum specimens and historical collections offer opportunities to retrospectively screen samples for betacoronaviruses (thereby testing predictions), sequence tissue for an assembly of host genomes, and enhance understanding of complex host–virus interactions.⁸⁸

Lastly, our iterative modelling of bat betacoronaviruses fits into a broader set of synergies in One Health research on bats, which can create win–win scenarios for conservation and outbreak prevention. For example, North American bats are threatened by an emerging disease, white-nose syndrome,⁸⁹ which has documented synzootic interactions with other bat coronaviruses;⁹⁰ at least seven North American bat species that can be infected by the fungal pathogen (Eptesicus fuscus, Myotis ciliolabrum, Myotis lucifugus, Myotis septentrionalis, Myotis velifer, Myotis volans, and Tadarida brasiliensis) are among the 412 bat species that we predicted could be undiscovered betacoronavirus hosts. Although our predictions do not imply bat susceptibility to SARS-CoV-2 specifically (and experimental infections of E fuscus have been unsuccessful⁹¹), efforts to minimise the risks of SARS-CoV-2 spillback into novel bat reservoirs,92, 93, 94, 95, 96 as well as to understand the dynamics of other bat coronaviruses, will both reduce zoonotic risk and help to understand and counteract disease-related population declines. Similarly, conservationists have expressed concern that the negative framing of bats as the source of SARS-CoV-2 has affected public and governmental attitudes toward bat conservation;⁹⁷ this can fuel negative responses, including indiscriminate culling (ie, the reduction of populations by slaughter), which has already occurred in response to COVID-19 even outside of Asia (where a spillover probably occurred).⁹⁸ Evidence shows that culling has many negative consequences, not only threatening population viability⁹⁹ but also possibly increasing viral transmission within the very species that are targeted.100, 101 Bat conservation programmes and One Health practitioners should continue to work together to find sustainable solutions for humans to live safely alongside wildlife and to communicate with the public about the ecological importance of these highly vulnerable species and the science of pathogen spillover.

For more on the Viral Emergence Research Initiative consortium see https://viralemergence.org

This online publication has been corrected. The corrected version first appeared at thelancet.com/microbe on January 28, 2022

Data sharing

The standardised data on betacoronavirus associations, and all associated predictor data, are available from the Viral Emergence Research Initiative consortium's Github (github.com/viralemergence/virionette). All modelling teams contributed an individual repository with their methods, which are available in the organisational directory (github.com/viralemergence). Code for the 2020 analysis, and a working reproduction of each author's contributions, is available from the study repository (github.com/viralemergence/Fresnel), and code to produce the updated analyses from 2021 is available in a separate repository (github.com/viralemergence/Fresnel_Jun). A complete list of the 412 predicted betacoronavirus bat hosts from our final ensemble is provided in the appendix.

Declaration of interests

We declare no competing interests.