How to quantify immigration from community abundance data using the neutral community model

Significance

Understanding how individuals move between communities is key to developing a more complete theory of community ecology. The neutral community model (NCM) is often used to ask whether observed biodiversity patterns can be explained without invoking fitness differences; it can also estimate the rate at which individuals move from one community to another. We introduce two methods for inferring immigration rates, evaluate the reliability of these inference methods, and determine the sampling requirements necessary for accurate inference. By leveraging agreement between three inference methods, we estimate the immigration rates in empirical community abundance datasets using the NCM. These findings set the stage for the robust estimation of immigration rates between interconnected communities throughout the biosphere.

Abstract

Biological communities are connected through dispersal, which regulates diversity across local and regional scales. However, dispersal is difficult to measure directly, limiting what is known about dispersal’s impact on species composition in complex communities. One method to measure dispersal employs the Neutral Community Model (NCM) to quantify how a local community is influenced by the immigration of individuals from a larger source community. Conveniently, the immigration rate N_Tm of the NCM can be fit from biological sequence abundance datasets, which are plentiful. Yet it is neither known if these estimated values reflect the ground truth, nor what sampling effort is required to yield accurate estimates. In this study, we introduce two inference methods, a variance-based and a Dirichlet-multinomial log-likelihood (DM-LL) method, to complement the established occupancy-based inference method. In simulations of communities that resemble activated sludge microbiomes, all inference methods were capable of estimating N_Tm within 10% of ground-truth, with the variance-based and DM-LL methods requiring less sampling effort. Accurate inferences require read depths greater than N_Tm in each sample. The three methods agree in their inferred N_Tm in simulations of communities experiencing weak non-neutral effects (e.g., selection) and in applications to abundance datasets from wastewater activated sludge, tropical trees, and coral reefs. Based on these findings, we propose practical sampling and methodological guidelines for quantifying immigration between highly diverse, complex communities using the NCM.

A central goal in ecology is to describe how biological communities assemble and vary over time and space. The four fundamental community assembly processes— diversification, dispersal, selection, and drift—determine taxa composition and diversity (i.e., community structure) in any habitat (1). In particular, dispersal is a regional process that connects communities across larger spatial scales through immigration and emigration. Moreover, the rate of dispersal mediates the relative influence of stochastic and deterministic assembly processes: High dispersal rates homogenize communities and reduce the influence of selection and drift, while low dispersal rates increase the impact of local differentiation and taxa turnover on community structure (2). Advances in high-throughput sequencing methods enable researchers to census micro- and macro-organismal communities with unprecedented resolution (3, 4), facilitating the development of models that describe community assembly and community structure at finer spatial and temporal scales. The Neutral Community Model (NCM) has emerged as a powerful and analytically tractable framework for quantifying how dispersal affects community structure in ideal “neutral” communities in which all taxa are assumed to have equal per-capita birth and death rates (5). Though biological communities are obviously not made of species with identical fitness, NCMs nonetheless recapitulate observed ecological patterns like taxa-area and distance–decay relationships (6–9).

In various implementations of the NCM, the abundance of each taxon in a “local” community is determined by stochastic birth–death processes and immigration from either an external “source” community (10) or a larger “metacommunity” (5, 11). We consider both implementations in this study. Dispersal occurring in unidirectional ecosystems (such as wastewater treatment plants, streams, or gut microbiomes) primarily transports individuals from an upstream “source” community to a downstream “local” community, while in other connected ecosystems (such as soils, forests, or coral reefs) dispersal can transport individuals throughout a metacommunity. At each timestep a random individual in the local community is lost to death or emigration, then replaced by i) an immigrant from a larger source community with a probability m or ii) by the reproduction of a random individual in the local community with a probability 1−m (Fig. 1). The long-term taxa abundance distributions predicted by the NCM are approximated by a Dirichlet distribution that depends on the immigration rate N_Tm, the expected number of immigrants colonizing the local community per N_T replacement events (12–14). The larger the immigration rate, the more local-community composition is driven by immigration from external sources, which can be an important process for maintaining the presence of rare (low abundance) taxa that would otherwise go extinct due to drift.

Fig. 1.

Schematic of the Neutral Community Model (NCM), describing the temporal dynamics of a “local” community with N_T sites that are each occupied by some taxon i. At each timestep, one site is cleared and replaced by an individual from a “source” community with probability m or by an individual from the local community with probability 1−m. In the source community, the relative abundance p_i of each taxon is fixed.

In this paper, we set out to examine how sampling affects the reliability of N_Tm inference. In any ecological sampling campaign, both the number of samples taken from a community (e.g., aliquots of biomass or count surveys) and the total number of counts in each sample (i.e., read depth in sequencing parlance) influence the accuracy and reliability of measured taxon abundances. A limited number of samples leads to an imprecise understanding of how community structure varies over space and/or time, while a limited read depth hampers the detection of rare taxa leading to an underestimation of biodiversity (15, 16). However, increasing sampling effort requires additional material and labor investment, highlighting the importance of rational sampling campaigns that optimize the allocation of resources.

In addition to sampling challenges, there are methodological limitations in how N_Tm is inferred. Though the established “occupancy-based” N_Tm inference method has been widely applied to viral (17), microbial (18–23), and macro-organismal (24, 25) datasets, it relies only on the presence/absence of each taxon across samples, a small fraction of the total amount of information contained in abundance datasets. To complement the established occupancy-based method, here we introduce two N_Tm inference methods: a variance-based method that matches the variance in measured taxon relative abundances to the variance predicted by the NCM and a likelihood-maximization method grounded in the Dirichlet-multinomial distribution (DM-LL). With neutral community assembly simulations, we validate these inference methods and measure their accuracy as a function of number of samples and read depth. Last, we apply these inference methods to simulated non-neutral communities and an empirical dataset to evaluate the generality of each method. Our results show that the range of immigration rates that can be inferred is limited by the read depth and that the variance-based and DM-LL methods yield more accurate estimates than the occupancy-based method. Our findings lay out practical guidelines for quantifying immigration in complex communities using the NCM.

Results

Methods to Infer the Immigration Rate N_Tm From Sampled Abundance Data.

Following the NCM update rule (Fig. 1), the relative abundance X_i (of each taxon i out of S taxa) will fluctuate over time and eventually attain a stationary steady state (i.e., a statistical steady state) (14). In the limit that the number of sites in the local community is very large, the steady-state relative abundances X₁,…, X_S of the local community are Dirichlet distributed (26):

$$X_1,\cdots ,X_S\sim \mathrm{Dirichlet}\left(N_{\mathrm{T}}m{p}_1,\cdots ,N_{\mathrm{T}}m{p}_S\right).$$ [1]

Dynamic NCM simulations of communities with discrete sites confirm that long-term taxa abundance distributions approach the Dirichlet distribution (SI Appendix, Fig. S1). Comparing theoretical and empirical relative abundance distributions and finding the value of N_Tm that produces the best match gives rise to the following variance, occupancy, and Dirichlet-multinomial N_Tm inference methods. The closer the empirical distribution is to the theoretical Dirichlet distribution, the more accurate the estimates of N_Tm will be.

Variance-Based Inference.

Marginalizing over the Dirichlet distribution, the relative abundance of each taxon in the local community is Beta distributed with $X_i\sim \mathrm{B}\mathrm{e}\mathrm{t}\mathrm{a}\left(N_{\mathrm{T}}m{p}_i,N_{\mathrm{T}}m\left(1-p_i\right)\right)$. Therefore, the mean relative abundance of a taxon in the local community is equal to its relative abundance in the source community $\mathrm{E}\left[X_i\right]=p_i$, and its variance is inversely proportional to N_Tm, $\mathrm{V}\mathrm{a}\mathrm{r}\left[X_i\right]=p_i\left(1-p_i\right)/(N_{\mathrm{T}}m+1)$. Accordingly, N_Tm can be inferred from the observed variances and average relative abundances of each taxon. To account for sampling noise, the observed variance $\widehat{\text{Var}\left[X_i\right]}$ in each taxon’s relative abundance also includes a contribution from the binomial sampling variance $p_i(1-p_i)/R$, where R is the read depth. The best fit N_Tm is obtained by least-squares regression between observed and predicted variance:

$$\widehat{\mathrm{Var}\left[X_i\right]}=\frac{p_i\left(1-p_i\right)}{N_{\mathrm{T}}m+1}+\frac{p_i\left(1-p_i\right)}{R}.$$ [2]

The larger the parameter N_Tm, the lower the variance of relative abundances of local-community taxa (i.e., the “tighter” the relative abundance distribution across samples).

Occupancy-Based Inference.

A taxon’s sample occupancy O_i is the proportion of samples in which it has nonzero abundance (it is “present”). The occupancy O_i of neutral communities produced by the NCM can be calculated (27) by integrating the probability density function of the beta distribution from the sample detection limit D to 1. The detection limit is the smallest measurable value of the relative abundance, 1/R. The best fit N_Tm is obtained by least-squares regression between observed and predicted sample occupancies:

$$\begin{array}{c}{O}_i={\int }_D^1\mathrm{P}\left[X_i=x_i\right]\mathrm{d}{x}_i=\frac{1}{\mathrm{B}\left(N_{\mathrm{T}}m{p}_i,N_{\mathrm{T}}m\left(1-p_i\right)\right)}{\int }_D^1{x}_i^{N_{\mathrm{T}}m{p}_i-1}{\left(1-x_i\right)}^{N_{\mathrm{T}}m\left(1-p_i\right)-1}\mathrm{d}{x}_i,\hfill \end{array}$$ [3]

where $\mathrm{B}\left(z_1,z_2\right)=\frac{\mathrm{\Gamma }\left(z_1\right)\mathrm{\Gamma }\left(z_2\right)}{\mathrm{\Gamma }(z_1+z_2)}$ is the beta function, and $\mathrm{\Gamma }\left(z\right)={\int }_0^{\infty }{t}^{z-1}{e}^{-t}\mathrm{d}t$ $\mathrm{\Gamma }\left(z\right)={\int }_0^{\infty }{t}^{z-1}{e}^{-t}\mathrm{d}t$ is the gamma function. As N_Tm increases, the occupancy of all observed taxa approaches 1.

Dirichlet-Multinomial Log-Likelihood inference (DM-LL).

Last, the Dirichlet-multinomial log-likelihood (DM-LL) method identifies the N_Tm value that maximizes the likelihood of having observed the measured reads in local-community samples. Formally, this likelihood quantifies the probability that J samples drawn from a multinomial distribution (whose probabilities are themselves drawn from a Dirichlet distribution (Eq. 1)) give the measured reads,

$$\begin{array}{ll}\text{P}\left(y\right)\hfill & ={\displaystyle {\displaystyle \prod _{j=1}^J}\underset{\widehat{p}}{{\displaystyle \int }}\text{Mult}\left(y^{\left(j\right)}|R,\widehat{p}\right)\mathrm{\text{Dir}}\left(\widehat{p}|N_{\text{T}}mp\right)\mathrm{\text{d}}\widehat{p}}\hfill \\ \hfill & {\displaystyle {\displaystyle \prod _{j=1}^J}\frac{R\mathrm{\text{B}}\left(N_{\text{T}}m,R\right)}{{\displaystyle {\prod }_{k:y^{\left(j\right)}>0}{y}_k^{\left(j\right)}\mathrm{\text{B}}\left(N_{\text{T}}m{p}_k,y_k^{\left(j\right)}\right)}}},\hfill \end{array}$$ [4]

where the Dirichlet distribution captures the dependence on the relative abundance of each taxon in the source community p_i and N_Tm, and the multinomial distribution captures the effects of sampling. Each multinomial draw consists of R reads, $\mathbf{p}$ is the vector of source-community relative abundances, ${\mathbf{y}}^{(j)}$ is the vector of local-community abundances (i.e., reads) for sample j, and $\mathit{y}$ is the set of taxon abundances in all samples. The resulting Dirichlet-multinomial distribution is well studied (28) and analytically tractable, as demonstrated by the striking simplification in Eq. 4. The maximum-likelihood estimate of N_Tm, $\mathcal{L}\left(N_{\mathrm{T}}m|\mathit{y}\right)$, maximizes $\mathrm{P}\left(\mathbf{y}\right)$; equivalently, it maximizes the log-likelihood:

$$\begin{array}{c}\log \left[\mathcal{L}\left(N_{\mathrm{T}}m|\mathit{y}\right)\right]\sim {\sum }_{j=1}^J\\ \left\{\log R+\log \mathrm{\Gamma }\left(N_{\mathrm{T}}m\right)+\log \mathrm{\Gamma }\left(R\right)-\log \mathrm{\Gamma }\left(N_{\mathrm{T}}m+R\right)-{\sum }_{k:y_k^{\left(j\right)}>0}\left[\log y_k^{\left(j\right)}+\log \mathrm{\Gamma }\left(N_{\mathrm{T}}m{p}_k\right)+\log \mathrm{\Gamma }\left(y_k^{\left(j\right)}\right)-\log \mathrm{\Gamma }\left(N_{\mathrm{T}}m{p}_k+y_k^{\left(j\right)}\right)\right]\right\}.\end{array}$$ [5]

Inference Improves With Increased Read Depth and Number of Samples.

The NCM posits an inverse relationship between the immigration rate and the variance of the relative abundance of each taxon Var[X_i] in the local community according to $\mathrm{V}\mathrm{a}\mathrm{r}\left[X_i\right]=\frac{p_i\left(1-p_i\right)}{N_{\mathrm{T}}m+1}$. However, when communities are sampled, the estimate of a taxon’s relative abundance X_i itself has a variance $\mathrm{V}\mathrm{a}\mathrm{r}\left[\widehat{X_i}\right]=\frac{p_i\left(1-p_i\right)}{R}$ due to sampling noise, which scales inversely with the read depth R. When R < N_Tm, sampling noise dominates the ecological variance, obscuring the underlying community dynamics. Put simply, sample read depths must be larger than N_Tm for accurate inference. Since N_Tm cannot be known prior to sampling, researchers can avoid situations where inferences are limited by read depth by sequencing samples as deeply as possible (SI Appendix, Figs. S2–S3). Increasing the number of samples improves the resolution for taxon sample occupancies, sampled abundances, and their variances.

Variance-Based and Dirichlet-Multinomial Inference Methods Require Less Sampling.

We first explore how estimated immigration rates depend on local-community sampling intensity, using the inference methods in an idealized sampling scenario. Inferences were performed on simulated abundance datasets of neutrally assembled communities in which source-community relative abundances were exactly known (SI Appendix, Fig. S2) and chosen to resemble a wastewater microbiome (Materials and Methods).

Increasing the read depth and the number of samples led to more accurate and consistent N_Tm estimates (Fig. 2). Increasing the number of local-community samples increases the accuracy of all inference methods, but little improvement is achieved beyond taking 10 samples (SI Appendix, Figs. S3 and S4). The DM-LL inference method provided the most accurate estimates of N_Tm across all combinations of read depth and number of samples, whereas the occupancy method required the largest read depth and number of samples for inference within 10% of the simulated value. Provided that a community is well described by neutral dynamics and read depths are larger than N_Tm, our findings suggest that all methods will estimate immigration rates within 10% of the true value with at least 10 local-community samples.

Fig. 2.

Inference accuracy depends on the sampling effort of the local community. Inferred N_Tm from 3, 10, and 100 local-community samples at read depths of 10²–10⁵ using the occupancy-based (red), variance-based (blue), and DM-LL (pink) methods. Source-community abundances were chosen to resemble a wastewater microbiome (Materials and Methods). The shaded area indicates N_Tm within 10% of the ground-truth value of 500 (dashed line).

Sample a Separate Source Community at Least as Much as the Local Community.

Important ecosystems like wastewater bioreactors and gut microbiomes consist of microbes that at one point immigrated and colonized from an “upstream” community (21). In the NCM, the flow of immigrants “downstream” is proportional to N_Tm, and accurately estimating this quantity requires knowledge of taxa relative abundances in the upstream source community p_i as well as taxa relative abundances in the downstream local community. Given that the source and local communities must be sampled separately, we next investigate how sampling of the source community (which determines p_i in the NCM) affects N_Tm inference (Fig. 3). In the limit that sampling of the source community achieves exact estimates of p_i, inferences achieve the accuracy of the idealized sampling scenario discussed in the previous section. Importantly, N_Tm is accurately inferred only when samples have read depths greater than N_Tm, regardless of the number of samples taken (SI Appendix, Fig. S5). When at least 10 source-community samples are taken, typical read depths of 10³–10⁴ (29, 30) are sufficient to infer the immigration rate within 10% of simulated ground-truth when N_Tm ≤ 5,000 (SI Appendix, Fig. S6).

Fig. 3.

Increasing source-community sampling effort improves inference accuracy, approaching the ideal case that taxon p_i values are exactly known. Inferred N_Tm from 3, 10, and 100 source-community samples for the occupancy-based (red), variance-based (blue), and Dirichlet-multinomial log-likelihood (DM-LL, pink) methods. In all simulations, 10 samples were taken from the local community, and read depths for both source and local communities are varied from 10²–10⁵. The shaded area indicates N_Tm within 10% of the ground-truth value of 500 (dashed line).

Quantifying Immigration in Non-Neutral Communities.

In nature, communities are not strictly neutral. Selection, for example, can favor certain taxa over others, causing local-community abundances to deviate from neutral expectations. Moreover, Allee effects can reduce the colonization odds of rare taxa (31). To investigate how deviations from neutrality influence N_Tm inferences in steady-state populations, we first randomly biased the relative abundance of local-community taxa by an amount proportional to a “non-neutrality” parameter σ. For each of J samples simulated with the NCM, the relative abundance of taxon i in the local community was shifted by an amount α_i drawn from the normal distribution N(0, σ²). After this perturbation, relative abundances were renormalized. The larger the non-neutrality parameter σ, the more taxon relative abundances deviate from their expected value under neutrality (Materials and Methods). These biases perturb the local community composition away from the Dirichlet distribution expected under purely neutral dynamics. Positive biases increase taxon relative abundances and large negative biases can cause extinction, in either case breaking the “neutral” condition that average local-community relative abundances equal source-community relative abundance.

Four values of the non-neutrality parameter σ were tested: 0 (neutral), 5 × 10⁻⁵ (weak), 5 × 10⁻⁴ (moderate), and 5 × 10⁻³ (strong). The variance-based and DM-LL methods converged to accurate estimates with increasing read depth for weak and moderate levels of non-neutrality (Fig. 4 and SI Appendix, Fig. S7). The occupancy-based method was the most sensitive to deviations from neutrality and did not converge to a stable N_Tm estimate with increasing read depth because rare taxa in the source community regularly achieved occupancies of 1, a result at odds with the NCM (SI Appendix, Fig. S8). To correct this, we modified the occupancy method to only consider taxa with sample occupancies less than 1, which yielded better performance at weak non-neutrality (SI Appendix, Fig. S11). Still, none of the inference methods accurately inferred N_Tm from local communities with strong non-neutrality (SI Appendix, Figs. S8–S10).

Fig. 4.

Non-neutrality reduces agreement between inference methods. Inferred N_Tm in neutral (σ = 0) and non-neutral (σ = 5 × 10⁻⁵, 5 × 10⁻⁴, and 5 × 10⁻³) local communities using the occupancy (red), variance (blue), and DM-LL (pink) methods at a simulated N_Tm of 500 (dashed line). Each data point consists of 10 simulations in which selection advantages are randomly drawn, local-community abundances are simulated, the source and local communities are sampled, and inference is performed. Error bars show one SD.

The introduction of this non-neutrality shifted taxon relative abundance distributions away from a Beta distribution, with the degree of this shift increasing with the strength of the non-neutrality parameter. In steady-state simulations with J = 50 when σ = 0 and σ = 5 × 10⁻⁵ nearly every taxon was consistent with a Beta-distributed relative abundance (P > 0.05, KS test) (Materials and Methods). However, for σ = 5 × 10⁻⁴ only 15% of taxa were consistent with Beta-distributed relative abundances, and for σ = 5 × 10⁻³ less than 1% of taxa were.

Next, modified dynamic simulations of the NCM that incorporated Allee effects generated non-neutral local communities, which impacted N_Tm estimates (SI Appendix, Fig. S12). To implement an Allee effect in these simulations, only taxa with a local-community abundance greater than a threshold population size (the “Allee threshold”) were allowed to colonize after site clearance. As a result, rare taxa without a “foothold” in the local community are unable to colonize. Even though the local community still possessed N_T sites and immigration from a source community occurred with probability m, taxa abundances were not well described by the Dirichlet distribution (Eq. 1), and the inferred values of N_Tm differed substantially from the ground-truth value even for weak Allee effects. Interestingly, among the three inference methods, estimates from the variance-based method were least impacted by the simulated levels of Allee thresholds. These simulations highlight the importance of understanding the ecology of sampled communities, since different ecological models (e.g., with or without Allee effects) can lead to different interpretations of inferred dispersal rates.

To infer N_Tm From Real Data: Rarefy, Infer, and Repeat.

Finally, we apply these methods to real community abundance datasets from a full-scale activated sludge system and influent wastewater (18), tropical trees on Barro Colorado Island (BCI) (32), and an aquatic reef ecosystem in Indonesia (33). The total number of individuals observed in each survey of a site, reported in the BCI and aquatic reef datasets, is analogous to the read depth in sequencing terms. For simplicity, throughout we continue using the language of “read depth.” These datasets provide an opportunity to validate our recommendations for sampling and N_Tm inference under natural conditions, where the true N_Tm value is unknown and additional non-neutral ecological complexities influence community assembly. For each dataset, abundance tables with progressively smaller read depths were used for N_Tm inference to assess inference stability and detect potential deviations from neutrality in each method (Materials and Methods).

Inference profiles—inferred N_Tm values at varying rarefied read depths—show agreement between the three inference methods for the activated sludge and wastewater communities at read depths of 10⁵ (N_Tm_occ = 905, N_Tm_var = 1,252, N_Tm_DM = 626) (Fig. 5). Moreover, estimates from all three inference methods were stable across high read depths, confirming that inferences are not read depth limited. Similar results are observed for the BCI (N_Tm_occ = 236, N_Tm_var = 83.4, N_Tm_DM = 126) and coral reef (N_Tm_occ = 66, N_Tm_var = 41, N_Tm_DM = 48) abundance datasets, but not for the fish abundance dataset (SI Appendix, Fig. S13). The application of N_Tm inference methods to microbial, macrofaunal, and macrofloral communities demonstrates that rarefaction, repeated inference, and comparison across methods can provide confidence for estimates of immigration rates and reveal when non-neutral processes substantially impact community assembly.

Fig. 5.

Inferring immigration rates of microbes from wastewater to an activated sludge microbiome in a full-scale wastewater treatment plant (A) and of tropical trees within a forest metacommunity in Barro Colorado Island (B). Error bars show one SD.

A majority of the 196 most-abundant taxa of the activated sludge microbiome and the 15 most-abundant taxa on BCI, cumulatively representing over 50% relative abundance in their respective communities, have abundance distributions that are consistent with the Beta distributions predicted by the NCM (119/196 taxa and 9/15 taxa, respectively; p > 0.05, KS test) (Materials and Methods). The goodness-of-fit for taxon abundances to the Beta distribution, which only depends on a single free parameter N_Tm, is demonstrated in SI Appendix, Fig. S14.

Discussion

Pitfalls abound when attempting to quantify immigration rates using the NCM. Motivated by our findings, we summarize key considerations here.

What Is Your Source Community?

Originally, the NCM was formulated within the context of island biogeography (34), considering a large mainland community that serves as the source of immigrants to a smaller island community. This scale asymmetry creates a directional influence of immigration in which the island (local) community is shaped by immigration from the mainland (source) but not vice versa. On the other hand, many ecological systems are better represented as metacommunities—networks of interacting local communities connected by dispersal without a single external source community (35). If an experimental system possesses upstream (mainland) and downstream (island) communities, both communities must be sampled separately, and the mainland-community abundances serve as the inputs for estimating the immigration rate N_Tm. By contrast, under a metacommunity framework, the composition of the source community is not an independent entity but is instead the average of the local-community samples. Identifying the appropriate spatial model—mainland-island or metacommunity—impacts sampling requirements, NCM estimation, and the interpretation of estimated parameters (36, 37).

Take at Least 10 Samples From the Local and Source Communities and Maximize Sequence Read Depths.

Resource constraints often limit the scale of sampling efforts, making optimized sampling strategies essential for studying community assembly. Our results show that read depth upper-bounds the N_Tm that can be inferred. Increasing the number of samples improves the precision and resolution of observed sample occupancies and improves inferences regardless of the inference method applied, although diminishing returns were observed beyond 10 local-community samples. Though more is always better, based on our simulations, we recommend taking at least 10 samples and prioritizing deep sequencing for robust N_Tm estimation. Similar recommendations apply when a source community must also be independently sampled.

Apply all Three Inference Methods to Build Confidence in Estimates.

The three inference methods each leverage different aspects of abundance data—occupancy, variance in relative abundances, and abundance distributions—to estimate N_Tm. Under neutral conditions, all three methods can infer N_Tm within 10% of its ground-truth value. However, non-neutrality impacts their estimates in distinct ways (Fig. 4 and SI Appendix, Figs. S6 and S13). For empirical datasets, researchers should apply all three methods and evaluate the consensus between their estimates: Provided that read depth is not the limiting factor, non-neutral effects likely dominate if the three methods yield inconsistent estimates. In this case, community assembly is not well described by the NCM.

Make the Most of Your Abundance Data.

In practice, read depths can vary up to 100-fold between individual samples, making normalization techniques such as rarefaction/subsampling a common preprocessing step before constructing relative abundance tables (38). When incorporated as part of a bootstrapping method, rarefaction provides a systematic way to evaluate confidence in N_Tm estimates. With evenly subsampled abundance tables, researchers can evaluate how N_Tm estimates change across read depths and determine if inferences are limited by sampling intensity or unreliable due to non-neutrality. Applying the “rarefy, infer, repeat” approach in the activated sludge, BCI, and coral datasets revealed distinct patterns among the three methods across read depths. The occupancy-based, variance-based, and DM-LL methods converged to stable N_Tm estimates at higher read depths, confirming that read depths were sufficient for reliable inference. Crucially, the similarity between N_Tm estimates lends confidence to inferred immigration rates.

Estimating Immigration Rates in Complex Communities Using the NCM.

Our results demonstrate that the NCM can address not only the qualitative question of whether community assembly is “neutral” but also provide quantitative estimates of immigration rates shaping local communities. Through simulations, we codify practical recommendations for sampling and inference methodologies to guide researchers interested in applying the NCM to their study ecosystems; code that implements these inference methods is freely available. Furthermore, our “rarefy, infer, repeat” approach enables researchers to extract robust immigration estimates from high-resolution sequence datasets. In this work, we focused on inference from steady-state abundance distributions derived from diffusion approximations for large populations. Many stochastic processes of interest in community population dynamics such as extinction, fixation, or immigration are rare events whose probabilities can also be investigated under non-neutral conditions using large-deviation theory (39). Although we found that N_Tm inference methods are resilient to some implementations of non-neutrality, future work should evaluate their robustness to real and simulated datasets exhibiting other forms of non-neutrality. By broadly applying these methods across diverse ecosystems and contexts, researchers can deepen ecological understanding of how dispersal influences community biodiversity.

Materials and Methods

Simulating Neutral and Non-Neutral Community Assembly at Steady State.

The source-community taxa abundances were simulated by a log-series abundance distribution (Θ = 250) with a total of 10¹⁰ individuals and 3453 taxa (SI Appendix, Fig. S1). Parameters used to define the log-series source community were chosen to align with a previously published wastewater dataset (40).

The neutral steady-state local-community taxa abundances were generated using a Dirichlet distribution (Eq. 1) that solves the NCM for the chosen log-series source community and an immigration rate N_Tm of either 5 × 10², 5 × 10³, or 5 × 10⁴. These values were chosen since previous studies have estimated N_Tm in this range (23, 41, 42). The NCM assumes that there are no fitness differences among taxa, so the expected relative abundances of local-community taxa mirror those in the source community.

To simulate violations of the neutrality assumption, an advantage term α_i is sampled from a normal distribution N(0, σ²) and randomly assigned for each local-community taxon i (23, 27, 43). The randomly drawn advantage terms have a 99.7% probability of falling within the range (−3σ, 3σ). Briefly, the α terms are added to Dirichlet-distributed local-community relative abundances—advantaged taxa (α_i > 0) and disadvantaged taxa (α_i < 0) have their local abundances increased and decreased relative to their source-community abundances, respectively. Taxa with relative abundances less than zero after this addition have their abundance set to zero (simulating extinction). Finally, the now “non-neutral” local-community relative abundances are renormalized to sum to one. We simulated three levels of neutrality violation (σ = 5 × 10⁻⁵, 5 × 10⁻⁴, and 5 × 10⁻³) alongside the neutral case (σ = 0).

Three scenarios were used to evaluate the performance of the three N_Tm inference methods under varying ecological and sampling conditions:

• Scenario 1: Inferring N_Tm assuming known source-community abundances. Local communities were simulated as neutral. Simulated source-community abundances were used as input p_i for the inference methods. The number of samples taken from the local community were varied from 2 to 100.

• Scenario 2: Inferring N_Tm in cases where source-community abundances need to be sampled. Local communities were simulated as neutral. The number of samples taken from the source community were varied from 2 to 100 to determine input p_i for the inference methods. Ten samples of the local community were taken.

• Scenario 3: Inferring N_Tm from non-neutral local communities. Local communities were simulated to deviate from neutrality, testing four levels of non-neutrality (σ = 0, 5 × 10⁻⁵, 5 × 10⁻⁴, and 5 × 10⁻³). Ten samples were taken from both source and local communities.

Read depths were varied from 10²−10⁵ in each case. For each combination of i) N_Tm values, ii) number of samples taken, and iii) read depths, ten simulations were run; each simulation generated relative abundance datasets. The reported N_Tm is the average of the N_Tm values inferred from each simulation.

To simulate sampling and sequencing, J samples (each containing R reads) are drawn from simulated community abundances. Formally, each of these J samples is a draw from a multinomial distribution parameterized by the relative abundances of each taxon in the source- or local-community and by the read depth R.

All steady-state simulations, analyses, and visualizations were created using R 4.2.2 (44).

Simulating Dynamic and Discrete Neutral and Non-neutral Communities.

Complementing the previously described simulations of populations at steady state (which obey Dirichlet statistics), we performed dynamic simulations of the NCM in a local community with N_T sites. The relative abundances p_i of taxa in the source community are displayed in SI Appendix, Fig. S1A and were chosen to mimic the relative abundances of tree species on Barro Colorado Island (32).

These dynamic simulations produce relative abundance distributions that approach the Beta distributions predicted by the NCM for taxa across abundance ranks. This agreement is demonstrated for the second-most abundant species in SI Appendix, Fig. S1B. Here, the source community is fixed, as in mainland-island systems. Local communities (islands) were initialized to consist solely of the most abundant taxon, the number of local-community sites N_T = 500, and the immigration rate m = 0.3. To generate the histogram, 5,000 identically initialized simulations were performed that each consisted of 500,000 timesteps (long enough to reach steady state). At steady state, the simulated relative abundance distribution and the theoretical Beta distribution were in strong agreement (SI Appendix, Fig. S1B).

Metacommunity simulations were performed by slightly modifying the dynamic mainland-island NCM simulations. Specifically, the source community was replaced by J–1 other “local” communities so that J communities interact with each other. At each timestep, one of these communities was selected at random and a random site on it was cleared. With probability 1-m this site was replaced by an individual randomly selected from the same community; with probability m, this site was replaced by a random individual from a randomly selected different community. In these simulations N_T = 500, m = 0.3, and J = 50. SI Appendix, Fig. S1C demonstrates that the relative abundance distribution of the second-most abundant taxon in each of the J communities matches the Beta distribution predicted by the NCM. Communities were identically initialized with the expected average number N_Tp_i of each taxon rounded to the nearest integer. Any empty or overallocated sites produced by this initialization procedure were added or deducted, respectively, from the most abundant taxon. After 100,000 timesteps, every single local community approaches the theoretical Beta distribution predicted by the NCM. It is important to keep one nuance of the metacommunity simulations in mind: Once a taxon goes extinct, it will never return (unlike the case of the mainland-island NCM). This means that all local communities will eventually consist of a single taxon. First, though, a quasi-stationary state is reached that matches the stationary state of the NCM and persists longer for larger N_T.

To incorporate non-neutrality, the dynamic mainland-island NCM simulations (i.e., with source and local communities) were modified to incorporate an Allee effect. After a site is cleared, it can only be replaced by a taxon whose abundance in the local community is greater than an “Allee threshold.” The consequence of this is that rare taxa without a “foothold” in the local community cannot colonize, shifting the local community composition toward taxa with higher abundance than the Allee threshold. After these dynamic simulations, the local community was sampled by randomly selecting R individuals. In SI Appendix, Fig. S5 the occupancy, variance, and DM-LL inference methods were applied to these sampled relative abundance datasets with J = 50 samples, N_T = 25,000, and m = 0.02. Sample size (i.e., read depth) R was varied from 100 to 10,000, and Allee thresholds of 0, 1, 5, and 10 were considered. These simulations did not incorporate the occupancy-method correction of excluding taxa that were present in every sample. For an Allee threshold of 0, the N_Tm inference of these dynamic simulations matches the inference profiles of the three methods shown in Fig. 2. In general, Allee effects distort the relative taxa abundance distributions in the local community, causing deviations in N_Tm inference.

All dynamic simulations and subsequent analyses were performed with Python (version 3.13.5).

Inferring N_Tm From Real Abundance Datasets.

The three N_Tm inference methods were applied to abundance data from a full-scale activated sludge reactor microbiome (18), tropical trees in a 50-ha plot in Barro Colorado Island (32), and coral reef and fishes in a reef crest habitat in Indonesia (33).

Abundance tables (S taxa × J samples) for the full-scale activated sludge bioreactor were constructed from 16S rRNA gene amplicon sequence data (V4 region) from a long-term sampling campaign. Raw sequences were downloaded from the NCBI Sequence Read Archive (SRA) using the SRA Toolkit (https://www.ncbi.nlm.nih.gov/sra). Quality control, sequence denoising, pair-end read merging and chimera removal were performed using the dada2 pipeline (45). Microbial taxonomy was resolved up to genus level using the MiDAS database (46). Abundance tables were packaged into a phyloseq (47) object along with sample metadata and used for analysis. The resulting abundance tables consist of 28 samples each from the influent wastewater and activated sludge samples. Sample read depths ranged between 31,309 to 163,733 reads per sample. The influent wastewater was considered the source community (mainland), and the activated sludge system was considered the local community (island). To build inference read depth profiles, the abundance tables used as input for the inference methods were first rarified to different read depths before being converted to relative abundance. To ensure at least 10 samples from both source and local communities are used for inference, the maximum read depth considered was chosen to be 10⁵.

The tree and reef species abundances were surveyed along 1-ha quadrats and 10-m transects, respectively. The BCI abundance tables consisted of 50 samples in total, with 341 to 601 individuals (“reads”) per sample. The coral reef abundance dataset consisted of 12 samples (165 to 375 “reads” per sample) in total, and the fish dataset consisted of 8 samples (72 to 338 “reads” per sample) in total. For these datasets, individual samples corresponding to surveyed sites along quadrats or transects were considered as local community samples, and the average of abundances across all samples were considered the abundances of species in the source community (metacommunity). No preprocessing of these datasets was required as they were already in the necessary form (S species x J samples). Inference read depth profiles were constructed like the microbial abundance dataset.

Evaluating fit of Relative Abundance Distributions to Theoretical Beta Distributions.

The Kolmogorov–Smirnov test was used to evaluate the goodness of fit between empirical abundance distributions and the theoretical Beta distribution that is the solution of the NCM [X_i ~ Beta(N_Tmp_i, N_Tm(1-p_i)]. The difference between the empirical abundance distribution and the theoretical Beta distribution was quantified by the KS statistic, equal to the largest pointwise difference between the two cumulative distribution functions. Small P-values indicate that the KS statistic of the empirical distribution function is in the most extreme Pth-percentile of possible Beta-distributed samples. That is, small P-values (P < 0.05) indicate a low probability that an empirical distribution was drawn from a theoretical Beta distribution. For empirical datasets, N_Tm is the value fit in the main text, and for steady-state simulations, it is the prescribed value of 500. The mean taxon relative abundance p_i is calculated directly from the dataset. The test is implemented using the scipy.stats.kstest function in python.

For steady-state simulations, goodness of fit was only evaluated for taxa that were present in every sample; the number of such taxa depends on sampling noise and non-neutrality.

Supplementary Material

Appendix 01 (PDF)

Data, Materials, and Software Availability

R code that replicates Figs. 2–5 and that can be applied to any sequencing dataset and Python code that performs dynamic discretized NCM simulations are available on GitHub (https://github.com/ramisrafay/NCM-Inference-Methods) (48).