Modeling Persistent Trends in Distributions

Abstract

We present a nonparametric framework to model a short sequence of probability distributions that vary both due to underlying effects of sequential progression and confounding noise. To distinguish between these two types of variation and estimate the sequential-progression effects, our approach leverages an assumption that these effects follow a persistent trend. This work is motivated by the recent rise of single-cell RNA-sequencing experiments over a brief time course, which aim to identify genes relevant to the progression of a particular biological process across diverse cell populations. While classical statistical tools focus on scalar-response regression or order-agnostic differences between distributions, it is desirable in this setting to consider both the full distributions as well as the structure imposed by their ordering. We introduce a new regression model for ordinal covariates where responses are univariate distributions and the underlying relationship reflects consistent changes in the distributions over increasing levels of the covariate. This concept is formalized as a trend in distributions, which we define as an evolution that is linear under the Wasserstein metric. Implemented via a fast alternating projections algorithm, our method exhibits numerous strengths in simulations and analyses of single-cell gene expression data.

1. Introduction

A common type of data in scientific and survey settings consists of real-valued observations sampled in batches, where each batch shares a common label (this numerical/ordinal value is the covariate) whose effects on the observations are the item of interest. When each batch consists of a large number of i.i.d. observations, the empirical distribution of the batch may be a good approximation of the underlying population distribution conditioned on the value of the covariate. A natural goal in this setting is to quantify the covariate’s effect on these conditional distributions, considering changes across all segments of the population. In the case of high-dimensional observations, one can measure this effect separately for each variable to identify which are the most interesting. However, it may often occur that, in addition to random sampling variability, there exist unmeasured confounding variables (unrelated to the covariate) that affect the observations in a possibly dependent manner within the same batch (cf. batch effects in Risso et al. 2014).

The primary focus of this paper is the introduction of the TRENDS (Temporally Regulated Effects on Distribution Sequences) regression model, which infers the magnitude of these covariate-effects across entire distributions. TRENDS is an extension of classic regression with a single covariate (typically of fixed-design), where one realization of our dependent variable is a batch’s entire empirical distribution (rather than a scalar) and the condition that fitted-values are smooth/linear in the covariate is replaced by the condition that fitted distributions follow a trend. Formally defined in §5, a trend describes a sequence of distributions where the p^th quantile evolves monotonically for all p ∈ (0,1), though not necessarily in the same direction for different p, and there are at most two partitions of the quantiles that move in opposite directions. Thus, TRENDS extends scalar-valued regression to full distributions while retaining the ability to distinguish effects of interest from extraneous noise.

Despite the generality of our ideas, we motivate TRENDS with a concrete scientific application: the analysis of single-cell RNA-sequencing time course data (see §S7 for a different application to income data; references preceded by ‘S’ are in the Supplementary Material).

The recent introduction of single-cell RNA-seq (scRNA-seq) techniques to obtain transcriptome-wide gene expression profiles from individual cells has drawn great interest (Geiler-Samerotte et al. 2013). Previously only measurable in aggregate over a whole tissue-sample/culture consisting of thousands of cells, gene-expression at the single-cell level offers insight into biological phenomena at a much finer-grained resolution, and is important to quantify as even cells of the same supposed type exhibit dramatic variation in morphology and function. One promising experimental design made feasible by the advent of this technology involves sampling groups of cells at various times from tissues / cell-cultures undergoing development and applying scRNA-seq to each sampled cell (Trapnell et al. 2014, Buettner et al. 2015). It is hoped that these data can reveal which developmental genes regulate/mark the emergence of new cell types over the course of development.

Current scRNA-seq cost/labor constraints prevent dense sampling of cells continuously across the entire time-continuum. Instead, researchers target a few time-points, simultaneously isolating sets of cells at each time and subsequently generating RNA-seq transcriptome profiles for each individual cell that has been sampled. More concretely, from a cell population undergoing some biological process like development, one samples N_ℓ ⩾ 1 batches of cells from the population at time t_ℓ where ℓ = 1, 2, …, L indexes the time-points in the experiment and $i=1,\dots ,N={\displaystyle {\sum }_{\mathcal{l}=1}^L{N}_{\ell }}$ indexes the batches. Each batch consists of n_i cells sampled and sequenced together. We denote by $x_{i,s}^{\left(g\right)}\in ℝ$ the measured expression of gene g in the sth cell of the ith batch (1 ⩽ s ⩽ n_i), sampled at time $t_{{\mathcal{l}}_i}$.

Because expression profiles are restricted to a sparse set of time points in current scRNA-seq experiments, the underlying rate of biological progression can drastically differ between equidistant times. Thus, changes in the expression of genes regulating different parts of this process may be highly nonuniform over time, invalidating assumptions like linearity or smoothness. One common solution in standard tissue-level RNA-seq time course analysis is time-warping (Bar-Joseph et al. 2003). Since our interest lies not in predicting gene-expression at new time-points, we instead aim for a procedure that respects the sequence of times without being sensitive to their precise values. In fact, researchers commonly disregard the wall-clock time at which sequencing is done, instead recording the experimental chronology as a sequence of stages corresponding overall qualitative states of the biological sample. For example, in Deng et al. (2014): Stage 1 is the oocyte, Stage 2 the zygote, …, Stage 11 the late blastocyst. Attempting to impose a common scale on the stage numbering is difficult because the similarity in expression expected across different pairs of adjacent stages might be highly diverse for different genes. In this work, we circumvent this issue by disregarding the time-scale and t_ℓ values, instead working only with the ordinal levels ℓ (so the only information retained about the times is their order t₁ < t₂ < ⋯ < t_L), as done by Bijleveld et al. (1998) (Section 2.3.2).

Depictions of such data from two genes (where N_ℓ = 1 for each ℓ) are shown in the lefthand panels of Figure 1. Lacking longitudinal measurements, these data differ from those studied in time series analysis: at each time point, one observes a different group of numerous exchangeable samples (no cell is profiled in two time points), and also the number of time points is small (generally L < 10). As a result of falling RNA-seq costs, multiple cell-capture plates (each producing a batch of sampled cells, i.e. N_ℓ > 1) are being used at each time point to observe larger fractions of the cell population (Zeisel et al. 2015). Because the cells in a batch are simultaneously collected and sequenced (independently of other batches), the measured gene-expression values are often biased by batch effects: technical artifacts that perturb observed values in a possibly correlated fashion between cells of the same batch (Risso et al. 2014, Kharchenko et al. 2014). Rather than treating the cells from a single time point identically, it is desirable to retain batch information and account for this nuisance variation. Batch effects are also prevalent in other applications including temporal studies of demographic statistics, where a simultaneously-collected group of survey results may be biased by latent factors like location.

Figure 1:

Violin plots (kernel density estimates) depicting the empirical distribution of known developmental genes’ expression measured in myoblast cells (on left), and the corresponding TRENDS fitted distributions (on right). Each point shows a sampled cell.

Furthermore, cell populations can exhibit enormous heterogeneity, particularly in developmental or in vivo settings (Trapnell et al. 2014, Buettner et al. 2015). A few high-expression cells often bias a population’s average expression, and transcript levels can vary 1,000-fold between seemingly equivalent cells (Geiler-Samerotte et al. 2013)¹. By fitting a TRENDS model (which accounts for both batch effects and the full distribution of expression across cells) to each gene’s expression values, researchers can rank genes based on their presumed developmental relevance or employ hypothesis testing to determine whether observed temporal variation in expression is biologically relevant.

2. Related Work

To better motivate the ideas subsequently presented in this paper, we first describe why existing methods are not suited for scRNA-seq time course experiments and similar ordered-batched data lacking longitudinal measurements. As an alternative to time-series techniques, regression models might be applied in this setting, such as the Tobit generalized linear model of Trapnell et al. (2014). However, these models rely on linearity/smoothness assumptions, which can be inappropriate for sporadic processes such as development. More importantly, classic regression models scalar values such as conditional expectations, for which results must be interpreted as the effects in a hypothetical “average cell”.

Rather than focusing only on (conditional) expectations or a few quantiles, it is often more appropriate to model the full (conditional) distribution of values in a heterogeneous population (Geiler-Samerotte et al. 2013, Buettner et al. 2015). Let P_ℓ denote the underlying distribution of the observations from covariate-level ℓ. An omnibus test for distribution-equality (H₀ : P₁ =⋯= P_L vs. the alternative that they are not all identical, cf. the Komogorov-Smirnov method described in §S3) can capture arbitrary changes, but fails to reflect sequential dynamics. Significance tests also do not quantify the size of effects, only the evidence for their existence. Krishnaswamy et al. (2014) have proposed a mutual-information based measure (DREMI) to quantify effects, which could be applied to our setting. However, under systematic noise caused by batch effects, measures of general statistical dependence between the batch-values and label ℓ (e.g. mutual information or hypothesis testing) become highly susceptible to the spurious variation present in the observed distributions (resulting in false positives). We thus prefer borrowing strength in the sense that a consistent change in distribution should ideally be observed across multiple time points for an effect to be deemed significant.

Instead of these general approaches, we model the P_ℓ as conditional distributions Pr(X | ℓ) which follow some assumed structure as ℓ increases. Work in this vein has focused on modeling only a few particular quantiles of interest (Bondell et al. 2010) or accurate estimation of the conditional distributions using smooth nonparametric regression techniques (Fan et al. 1996, Hall et al. 1999). While such estimators possess nice theoretical properties and good predictive-power, the relationships they describe may be opaque and it is unclear how to quantify the covariate’s effect on the entire distribution. Note that in the case of classic regression, interpretable linear methods remain favored for measuring effects throughout the sciences, despite the availability of flexible nonlinear function families. Our TRENDS framework retains this interpretability while modeling effects across full distributions.

Change-point analysis can also be applied to sequences of distributions, but is designed for detecting the precise locations of change-points over long intervals. However, scRNA-seq experiments only span a brief time-course (typically L ⩽ 10), and the primary analytic goal is rather to quantify how much a gene’s expression has changed in a biologically interesting manner. Many change-point methods require explicit parameterization of the types of distributions, an undesirable necessity given the irregular nature of scRNA-seq expression measurements (Kharchenko et al. 2014). Moreover, some development-related genes exhibit gradual rather than abrupt temporal temporal changes in expression. Requiring few statistical assumptions, TRENDS models changes ordinally rather than only considering effects that are either smooth or instantaneous, and this method can therefore accurately quantify both abrupt or gradual effects.

3. Methods

Formally, TRENDS fits a regression model to an ordered sequence of distributions, or more broadly, sample pairs ${\left\{\left({\mathcal{l}}_i,{\widehat{P}}_i\right)\right\}}_{i=1}^N$ where each ℓ_i ∈{1,…,L} is an ordinal-valued label associated with the ith batch, for which we have univariate empirical distribution $\widehat{P_i}$. Here, it is supposed that for each batch i: a (empirical) quantile function ${\widehat{F}}_i^{-1}$ is estimated from n_i scalar observations ${\left\{X_{i,s}\right\}}_{s=1}^{n_i}~P_i$ sampled from underlying distribution P_i = Pr(X|ℓ_i), which may be contaminated by different batch effects for each i. We assume a fixed-design where each level of the covariate 1,…,L is associated with at least one batch. In scRNA-seq data, $\widehat{P_i}$ is the empirical distribution of one gene’s measured expression values over the cells captured in the same batch and ℓ_i indicates the index of the time point at which the batch was sampled from the population for sequencing.

Unlike the supervised learning framework where one observes samples of X measured at different ℓ and the goal is to infer some property of P_ℓ := Pr(X|ℓ), in our setting, we can easily obtain $\widehat{P_i}$ as an empirical estimate of Pr(X|ℓ_i). We thus neither seek to estimate the distributions P₁,…,P_L, nor test for inequality between them. Rather, the primary goal of TRENDS analysis is to infer how much of the variation in Pr(X | ℓ) across different ℓ may be attributed to changes in ℓ as opposed to the effects of other unmeasured confounding factors. To quantify this variation, we introduce conditional effect-distributions Q_ℓ for which the sequence of transformations Q₁ → Q₂ → ⋯ → Q_L entirely captures the effects of ℓ-progression on Pr(X | ℓ), under the assumption that these underlying forces follow a trend (defined in §5). We emphasize that the Q_ℓ themselves are not our primary inferential interest, rather it is the variation in these conditional-effect distributions that we attribute to increasing-ℓ rather than batch effects.

Thus, the Q_ℓ are not estimators of the sequence of $P_{{\mathcal{l}}_i}$. Rather, the Q_ℓ represent the distributions one would expect see in the absence of exogenous effects and random sampling variability, in the case where the underlying distributions only change due to ℓ-progression and we observe the entire population at each ℓ. Because we do not believe exogenous effects unrelated to ℓ-progression are likely to follow a trend over ℓ, we can identify the sequence of trending distributions which best models the variation in ${\left\{{\widehat{P}}_{{\mathcal{l}}_i}\right\}}_{i=1}^N$ and reasonably conclude that changes in this sequence reflect the ℓ-progression-related forces affecting P_ℓ.

4. Wasserstein Distance

TRENDS employs the Wasserstein distance to measure divergence between distributions. Intuitively interpreted as the minimal amount of “work” that must be done to transform one distribution into the other, this metric has been successfully applied in many domains (Levina & Bickel 2001, Mueller & Jaakkola 2015). The Wasserstein distance is a natural dissimilarity measure of populations because it accounts for the proportion of individuals that are different as well as how different these individuals are. For univariate distributions, the L_q Wasserstein distance is simply the L_q distance between quantile functions given by:

$$d_{L_q}\left(P,Q\right)={\left({\displaystyle {\int }_0^1|F^{-1}\left(p\right)-G^{-1}\left(p\right){|}^{q}dp}\right)}^{1/q}$$ (1)

where F, G are the CDFs of P, Q and F⁻¹,G⁻¹ are the corresponding quantile functions.Slightly abusing notation, we use $d_{L_q}\left(\cdot ,\cdot \right)$ to denote both Wasserstein distances between distributions or the corresponding quantile functions’ L_q-distance (both q = 1, 2 are used in this work). In addition to being easy to compute (in 1-D), the L₂ Wasserstein metric is equipped with a natural space of quantile functions, in which the Fréchet mean takes the simple form stated in Lemma 1. Calling this average the Wasserstein mean, we note its implicit use in the popular quantile normalization technique (Bolstad et al. 2003).

Lemma 1. Let $\mathcal{Q}$ denote the space of all quantile functions. The Wasserstein mean is the Fréchet mean in $\mathcal{Q}$ under the L₂ norm:

$${\overline{F}}^{-1}:=\frac{1}{N}{\displaystyle \sum }_{i=1}^N{F}_i^{-1}{=}_{G^{-1}\in \mathcal{Q}}^{\arg \min }\left\{{\displaystyle \sum }_{i=1}^N\underset{0}{\overset{1}{{\displaystyle \int }}}{\left(F_i^{-1}\left(p\right)-G^{-1}\left(p\right)\right)}^{2}dp\right\}$$ (2)

5. Characterizing trends in distributions

Definition 1. Let $F_{\mathcal{l}}^{-1}\left(p\right)$ denote the pth quantile of distribution P_ℓ with CDF F_ℓ. A sequence of distributions P₁ ,…,P_L follows a trend if:

1. For any p ∈ (0,1), the sequence $\left[F_1^{-1}\left(p\right),\dots ,F_L^{-1}\left(p\right)\right]$ is monotonic.

2. There exists p* ∈ [0,1) and two intervals A,B that partition the unit-interval at p^* (one of A or B equals (0,p*) and the other equals [p*, 1)) such that: for all p ∈ A, the sequences $\left[F_1^{-1}\left(p\right),\dots ,F_L^{-1}\left(p\right)\right]$ are all nonincreasing, and for all q ∈ B, the sequences $\left[F_1^{-1}\left(p\right),\dots ,F_L^{-1}\left(p\right)\right]$ are all nondecreasing. Note that if p* = 0, then all quantiles must change in the same direction as ℓ grows.

Our formal definition of a trend applies to distributions which evolve in a consistent fashion, ensuring that the temporal-forces that drive the transformation from P₁ to P_L do so without reversing their effects or leading to wildly different distributions at intermediate ℓ values. While the second condition of our definition technically subsumes the first, Condition 1 contains our key idea and is therefore separated from Condition 2, a subtler additional assumption that does not significantly alter results in practice. Note that the trend definition employed in this paper is intended for relatively short sequences and does not include cyclic/seasonal patterns studied in time-series modeling.

Lemma 2. If distributions P₁,…,P_L follow a trend, then

$$d_{L_1}\left(P_i,P_j\right)={\displaystyle \sum }_{\ell =i+1}^j{d}_{L_1}\left(P_{\ell -1},P_{\ell }\right)\text{for}\text{all\hspace{0.05em}}i<j\in \left\{1,\dots ,L\right\}$$

Measuring how much the distributions are perturbed between each pair of levels via the L₁ Wasserstein metric, Lemma 2 shows the trend criterion as an instance of Occam’s razor, where the underlying effects of interest are assumed to transform the distribution sequence in the simplest possible manner (recall that the Wasserstein distance is interpreted as the minimal work required for a given transformation). If one views the underlying effects of interest as a literal force acting in the space of distributions, Lemma 2 implies that this force points the same direction for every ℓ (i.e. P₁,…,P_L lie along a line in the L₁ Wasserstein metric space of distributions). A trend is more flexible than a linear restriction in the standard sense, because the magnitude of the force (how far along the line the distributions move) can vary over ℓ. Thus, we have formally extended the colloquial definition of a trend (“a general direction in which something is developing or changing”) to probability distributions.

To further conceptualize the trend idea, one can view quantiles as different segments of a population whose values are distributed according to Pr(X | ℓ) (e.g. for wealth-distributions, it has become popular to highlight the “one percent”). From this perspective, it is reasonable to assume that while the force of sequential progression may have different effects on the groups of individuals corresponding to different segments of the population, its effects on a single segment should be consistent over the sequence. If some segment’s values initially change in one way at lower levels of ℓ and subsequently revert in the opposite direction over larger ℓ (i.e. this quantile is non-monotone), it is natural to conclude there are actually multiple different progression-related forces affecting this homogeneous group of individuals. It is therefore natural to assume a trend if we only wish to measure the effects of a single primary underlying force. Often in settings such as scRNA-seq developmental experiments, the researcher has a priori interest in a specific effect (such as how each gene contributes to a specific stage of the developmental process). Therefore, data are collected over a short ℓ-range such that the primary effects of interest should follow a trend.

The second condition in the trend definition specifies that adjacent quantiles must move in the same direction over ℓ except at most a single p*. This restricts the number of population-segments which can increase over ℓ when a nearby segment of the population is decreasing. Intuitively, Condition 2 forces us to borrow strength across adjacent quantiles when estimating effects that follow a trend. The main effect of the additional restriction imposed by this condition prevents a trend from completely capturing extremely-segmented effects (such as the example depicted in Figure 3C). However, applications involving such complex phenomena are uncommon (it is difficult to imagine a setting where the primary effects-of-interest push more than two adjacent segments of a population in different directions), and such nuanced changes can be reasonably attributed to spurious nuisance variation. We note that a trend can still roughly approximate the major overall effects even when the actual distribution-evolution violates Condition 2 (as seen in Figure 3C). In practice, the results of our method are not significantly affected by this second restriction, but it provides nice theoretical properties ensuring our estimation procedure (presented in §8) efficiently finds a globally optimal solution, as well as additional robustness against spurious quantile-variation in the data (possibly due to estimation-error given limited samples per batch).

Figure 3:

Violin plots depicting sequences of distributions which do not follow a trend (Observed Distributions in lefthand panels). Shown to the right of each example are the corresponding fitted distributions estimated by TRENDS (with the TRENDS R² value).

Figure 2 depicts simple examples of trending distribution-sequences. In each example, it is visually intuitive that the evolution of the distributions proceeds in a single consistent fashion. To highlight the broad spectrum of interesting effects TRENDS can detect, we present three conceptual examples in §S1 of distribution-sequences that follow a trend, which includes consistent changes in location/scale and the growth/disappearance of modes. Despite imposing conditions on every quantile, the trend criterion does not require: explicit parameterization of the distributions, specification of a precise functional form of the ℓ-effects, or reliance on a smooth or constant amount of change between different levels. This generality is desirable for modeling developmental gene expression and other enigmatic phenomena where stronger assumptions may be untenable.

Figure 2:

Violin plots depicting four different sequences of distributions which follow a trend. The pth rectangle in the color bar on the righthand side indicates the monotonicity of the pth quantile over the sequence of distributions (for p = 0.01, 0.02,…, 0.99).

The lefthand panels of Figure 3 depict three examples of sequences which do not follow a trend for different reasons. To the right of each example, we show the “best-fitting” sequence that does follow a trend (formally defined in (4)), each distribution of which corresponds to our estimate of Q_ℓ (introduced in §3). We reiterate that the Q_ℓ are not by themselves of interest, but are merely used to quantify the sequential-progression effects (as will be described in §7). Nonetheless, the visual depiction of the trending Q_ℓ provides insight regarding what sort of changes a trend can accurately approximate. Whereas the evolution of the (trending) fitted distributions in Figure 3A (on right) can intuitively be attributed to one consistent force, multiple are required to explain the variation in the original non-trending sequence of distributions on the left. Identifying a single consistent effect responsible for the changes in the left panel of Figure 3B is far more plausible, and we note that these distributions in fact are much closer to following a trend (while hard to visually discern, the 0.04^th – 0.16^th quantiles of the observed distribution sequence increase between ℓ = 1 to 2 and decrease slightly from ℓ = 2 to 3, thus violating a trend).

During specific stages of development, changes in the observed cellular gene-expression distributions generally stem from the emergence/disappearance of different cell subtypes (plus batch and random sampling effects). Clear subtype distinctions may not exist in early stages where cells remain undifferentiated, and thus not only are the relative proportions of different subtypes changing, but the subtypes themselves may transform as well. Therefore, developmental genes’ underlying expression patterns are likely described by Examples 2 and 3 (of specific conceptual types of trends) in §S1. The trend criterion fits our a priori knowledge well, while remaining flexible with respect to the precise nature of expression changes.

6. TRENDS regression model

Recall that in our setting, even the underlying batch distributions Pi (from which the observations X_i,s are sampled) may be contaminated by latent confounding effects. We assume the quantile functions of each P_i are generated from the model below:

$$F_i^{-1}=G_{{\ell }_i}^{-1}+{Ɛ}_i\text{\hspace{0.17em}such that\hspace{0.17em}}{G}_1^{-1},\dots ,G_L^{-1}\text{\hspace{0.17em}follow a trend},\text{and the following hold}:$$ (3)

(A.1) ${Ɛ}_i:\left(0,1\right)\to ℝ$ is constrained so that $G_{{\ell }_i}^{-1}$ and $F_i^{-1}$ are valid quantile functions.

(A.2) For all p ∈ (0,1) and i: Ɛ_i(p) follows a sub-Gaussian(σ) distribution (Honorio & Jaakkola 2014), so $\mathbb{E}\left[{Ɛ}_i\left(p\right)\right]=0$ and $\mathrm{Pr}\left(\left|{Ɛ}_i\left(p\right)\right|>t\right)\le 2\exp \left(-\frac{t^2}{2{\sigma }^2}\right)$ for any t > 0.

(A.3) For all p ∈ (0,1) and i ≠ j: Ɛ_i(p) is statistically independent of Ɛ_j(p).

In this model, $G_{\ell }^{-1}$ is the quantile function of the conditional effect-distribution Q_ℓ, whose evolution captures the underlying effects of level-progression. The random noise functions Ɛ_i : (0,1) → ℝ can represent measurement-noise or the effects of other unobserved variables which contaminate a batch. Note that the form of Ɛ_i is implicitly constrained to ensure all $F_i^{-1},G_{{\ell }_i}^{-1}$ are valid quantile functions. Because Ɛ_i(p₁) and Ɛ_i(p₂) are allowed to be dependent for p₁ ≠ p₂, the effect of one Ɛ_i may manifest itself in multiple observations X_i,s, even if these observations are drawn i.i.d. from P_i (for example, a batch effect can cause all of the observed values from a batch to be under-measured). In fact, condition (A.1) encourages significant dependence between the noise at different quantiles for the same batch. The assumption of sub-Gaussian noise is quite general, encompassing cases in which the Ɛ_i(p) are either: Gaussian, bounded, of strictly log-concave density, or any finite mixture of sub-Gaussian variables (Honorio & Jaakkola 2014). Although condition (A.3) stringently ensures all dependence between observations from different ℓ arises due to the trend, similar independence assumptions are required in general regression settings where one cannot reasonably a priori specify a functional form of dependence in the noise. Real batch effects are likely to satisfy (A.3) since they typically have the same chance of affecting any given batch in a certain manner (because the same experimental procedure is repeated across batches, as in the case of the cell-capture and library preparation in scRNA-seq). Nonetheless, we note that assumption (A.2) can be immediately generalized (with trivial changes to our proofs) in order to allow heteroscedasticity in the batch effects Ɛ_i (endowing each batch with a different σ_i sub-Gaussian parameter), but we opt for simplicity in this theoretical exposition.

Model (3) is a distribution-valued analog of the usual regression model, which assumes scalars Y_i= f (X_i) + ∊_i where ∊_i ~ sub-Gaussian(σ²) and ∊_i is independent of ∊_j for i ≠ j. In (3), an analogous f maps each ordinal level {1,…, L} to a quantile function, $f\left({\ell }_i\right)=G_{{\ell }_i}^{-1}$, and the class of functions is restricted to those which follow a trend. Our assumption of mean-zero Ɛ_i that are independent between batches is a straightforward extension of the scalar error-model to the batch-setting, and ensures that the exogenous noise is unrelated to ℓ-progression under (3). Just as the Y₁,…,Y_N are rarely expected to exactly lie on the curve f (x) in the classic scalar-response model, we do not presume that the observed distributions $\widehat{P_i}$ will exactly follow a trend (even as n_i → ∞ ∀i so that $\widehat{P_i}\to P_i$). Rather our model simply encodes the assumption that the effects of level-progression on the distributions should be consistent over different ℓ (i.e. the effects follow a trend).

For each ℓ, TRENDS finds a fitted distribution ${\widehat{Q}}_{\ell }$ using the Wasserstein least-squares fit which minimizes the following objective:

$${\widehat{Q}}_1,\dots ,{\widehat{Q}}_L{=}_{Q_1,\dots ,Q_L}^{\arg \min }\left\{{\displaystyle \sum _{\ell =1}^L{\displaystyle \sum _{i\in I_{\ell }}{d}_{L_2}{\left(Q_{\ell },{\widehat{P}}_i\right)}^2}}\right\}\text{where}{Q}_1,\dots ,Q_L\text{follow}\text{a}\text{trend}$$ (4)

where I_ℓ is the set of batch-indices i such that ℓ_i = ℓ, and we require N_ℓ :=|I_ℓ| ⩾ 1 for all ℓ ∈ {1,…,L}. Subsequently, one can inspect changes in the ${\widehat{Q}}_{\ell }$ which should reflect the transformations in the underlying P_ℓ that are likely caused by increasing ℓ. Figure 3 shows some examples of fitted distributions produced by TRENDS regression. The objective in (4) bears great similarity to the usual least-squares loss used in scalar regression, the only differences being: scalars have been replaced by distributions, squared Euclidean distances are now squared Wasserstein distances, and the class of regression functions is defined by a trend rather than linearity/smoothness criteria.

Expression measurements in scRNA-seq are distorted by significant batch effects, so the Ɛ_i are likely to be large. In addition to technical artifacts, Buettner et al. (2015) find biological sources of noise due to processes such as transcriptional bursting and cell-cycle modulation of expression. Unlike development-driven changes in the underlying expression of a developmental gene, other biological/technical sources of variation are unlikely to follow any sort of trend. TRENDS thus provides a tool for modeling full distributions, while remaining robust to the undesirable variation rampant in these applications by leveraging independence of the noise between different batches of simultaneously captured and sequenced cells.

7. Measuring fit, effect size, and statistical significance

Analogous to the coefficient of determination used in classic regression, we define the Wasserstein R² to measure how much of the variation in the observed distributions ${\widehat{P}}_1,\dots ,{\widehat{P}}_N$ is captured by the TRENDS model’s fitted distributions ${\widehat{Q}}_1,\dots ,{\widehat{Q}}_L$:

$$R^2:=1-\left(\frac{1}{N}{\displaystyle \sum }_{i=1}^N{d}_{L_2}{\left({\widehat{Q}}_{{\ell }_i},\widehat{P_i}\right)}^2\right)/\left(\frac{1}{N}{\displaystyle \sum }_{i=1}^N{d}_{L_2}{\left(\widehat{P_i},{\overline{F}}^{-1}\right)}^2\right)\in \left[0,1\right]$$ (5)

Here, squared distances between scalars in the classic R² are replaced by squared Wasserstein distances between distributions, and the quantile function ${\overline{F}}^{-1}=\frac{1}{N}{\displaystyle {\sum }_{i=1}^N{\widehat{F}}_i^{-1}}$ is the Wasserstein mean of all observed distributions. By Lemma 1, the numerator and denominator in (5) are respectively analogous to the residuals and the overall variance from usual scalar regression models.

In classic linear regression, the regression line slope is interpreted as the expected change in the response resulting from a one-unit increase in the covariate. While TRENDS operates on unit-less covariates, we can instead measure the overall expected Wasserstein-change under model (3) in the ${\widehat{P}}_i$ over the full ordinal progression ℓ = 1,…,L using:

$$\Delta :\frac{1}{L}\cdot d_{L_1}\left({\widehat{Q}}_1,{\widehat{Q}}_L\right)$$ (6)

The L₁ Wasserstein distance is a natural choice, since by Lemma 2, it measures the aggregate difference over each pair of adjacent ℓ levels (just as the difference between the largest and smallest fitted-values in linear regression may be decomposed in terms of covariate units to obtain the regression-line slope). Thus, Δ measures the raw magnitude of the inferred trend-effect (depends on the scale of X), while R² quantifies how well the trend-effect explains the variation in the observed distributions (independently of scaling). Note that if the TRENDS model is fit to the distributions from the example in Figure 3B, the TRENDS-inferred effect of sequential-progression is nearly as large as the overall variation in this sequence, which agrees with our visual intuition that the observed distributions already evolve in a fairly consistent fashion.

Finally, we introduce a test to assess statistical significance of the trend-effect. We compare the null hypothesis H₀ : Q₁ = Q₂ = ⋯ = Q_L against the alternative that the Q_i are not all equal and follow a trend. To obtain a p-value, we employ permutation testing on the ℓ_i-labels of our observed distributions ${\widehat{P}}_i$ with test-statistic R² (Good 1994). More specifically, the null distribution is determined by repeatedly executing the following steps: (i)randomly shuffle the ℓ_i so that each ${\widehat{P}}_i$ is paired with a random ${\ell }_i^{Perm}\in \left\{1,\dots ,L\right\}$ value, (ii)fit the TRENDS model to the pairs ${\left\{\left({\ell }_i^{Perm},{\widehat{P}}_i\right)\right\}}_{i=1}^N$ to produce ${\widehat{Q}}_1^{Perm},\dots ,{\widehat{Q}}_L^{Perm}$, (iii) use these estimated distributions to compute $R_{Perm}^2$ using (5). Due to the quantile-noise functions Ɛ_i(⋅) assumed in our model (3), H₀ allows variation in our sampling distributions P_i which stems from non-ℓ-trending forces. Thus the TRENDS test attempts to distinguish whether the effects transforming the P_i follow a trend or not, but does not presume the P_i will look identical under the null hypothesis. By measuring how much further the ${\widehat{P}}_i$ lie from one distribution vs. a sequence of trending distributions in Wasserstein-space, we note that our R² resembles a likelihood-ratio-like test statistic between maximum-likelihood-like estimates ${\overline{F}}^{-1}$ and ${\widehat{Q}}_{\ell }$ (where we operate under the Wasserstein distance rather than Kullback-Leibler which underlies the maximum likelihood framework).

As we do not parametrically treat the distributions, we find permutation testing more suitable than relying on asymptotic approximations. Unfortunately, N and L may be small, undesirably limiting the number of possible label-permutations. In §S2, we overcome the granularity problem that arises in such settings by developing a more intricate permutation procedure akin to the smoothed bootstrap of Silverman & Young (1987).

To determine whether our model is reasonable when working with real data, it is best to rely on prior domain knowledge regarding whether or not the effects of primary interest should follow a trend. When this fact remains uncertain, then (as in the case of classical regression) the question is not properly answered using just our Wasserstein R² values (which we caution tend to be much larger than the familiar R² values from linear regression, due to the heightened flexibility of our TRENDS model). §S6 demonstrates a simple method for model checking based on plotting empirically-estimated residual functions ${\widehat{\mathcal{E}}}_i$ against the sequence-level ℓ. Similar plots of scalar residuals are the most common diagnostic employed in standard regression analysis. While this model-checking procedure is able to clearly delineate simulated deviations from our assumptions, it shows little indication that the TRENDS assumptions are inappropriate for the real scRNA-seq data from major known developmentally-relevant genes. Our simulation in §S6 also empirically demonstrates that despite its restrictive assumptions, the TRENDS model can provide superior estimates of severely-misspecified effects than the initial empirical distributions.

8. Fitting the TRENDS model

We propose the trend-fitting (TF) algorithm which finds distributions satisfying

$${\widehat{Q}}_1,\dots ,{\widehat{Q}}_L=\arg \underset{Q_1,\dots ,Q_L}{\min }\left\{{\displaystyle \sum _{\ell =1}^L{\displaystyle \sum _{i\in I_{\ell }}{w}_i\cdot d_{L_2}{\left(Q_{\ell },{\widehat{P}}_i\right)}^2}}\right\}where{Q}_1,\dots ,Q_{L}followatrend$$ (7)

If ${\widehat{P}}_i$ (the empirical per-batch distributions) are estimated from widely varying sample sizes n_i for different batches i, then it is preferable to replace the objective in (4) with the weighted sum in (7). Given weights w_i chosen based on n_i and N_ℓ, TRENDS can better model the variation in the empirical distributions that are likely more accurate due to larger sample size. As n_i and N_ℓ are fairly homogeneous in scRNA-seq experiments, we use uniform weights here (but provide an algorithm for the general formulation). To fit TRENDS to data ${\left\{\left({\ell }_i,{\widehat{P}}_{{\ell }_i},w_i\right)\right\}}_{i=1}^N$ via our procedure, the user must first specify:

• Numerical quadrature points 0 < p₁ < p₂ < ⋯ < p_P−1 < 1 for evaluating the Wasserstein distance integral in (1), i.e. which P – 1 quantiles to use for each batch

• a quantile estimator ${\widehat{F}}^{-1}\left(p\right)$ for empirical CDF $\widehat{F}$

Given these two specifications, the TF procedure solves a numerical-approximation of the constrained distribution-valued optimization problem in (7). Defining p₀ := 2p₁ – p₂ and pP : = 2pP−1 − pP−2, we employ the following midpoint-approximation of the integral

$$\underset{G_1^{-1},\dots ,G_L^{-1}}{min}\left\{{\displaystyle \sum }_{\ell =1}^L{\displaystyle \sum }_{i\in I_{\ell }}{w}_i{\displaystyle \sum }_{k=1}^{P-1}{\left({\widehat{F}}_i^{-1}\left(p_k\right)-G_{\ell }^{-1}\left(p_k\right)\right)}^2\left[\frac{p_{k+1}-p_{k-1}}{2}\right]\right\}\text{where}{G}_1,\dots ,G_L\text{must follow a trend}$$ (8)

While this problem is unspecified between the p_kth and p_k+1th quantiles, all we require to numerically compute Wasserstein distances (and hence R² or Δ) is the values of the quantile functions at p₁,…,p_P−1, which are uniquely determined by (8). Although our algorithm operates on a discrete set of quantiles like techniques for quantile regression (Bondell et al. 2010), this is only for practical numerical reasons; the goal of our TRENDS framework is to measure effects across an entire distribution. Throughout this work, we use P – 1 uniformly spaced quantiles between $\frac{1}{P}$ and $\frac{P-1}{P}$ (with P = 100) to comprehensively capture the full distributions while ensuring computational efficiency. In settings with limited data per batch, one might alternatively select fewer quadrature points (quantiles), avoiding tail regions of the distributions for increased stability (our results were robust to the precise number of quadrature points employed).

Since no unbiased minimum-variance ∀p ∈ (0,1) quantile estimator is known, we simply use the default setting in R’s quantile function, which provides the best approximation of the mode (Type 7 of Hyndman & Fan (1996)). Other quantile estimators perform similarly in our experiments, and Keen (2010) have found little practical difference between estimation procedures for sample sizes ⩾ 30. Here, we assume the n_i cells sampled in the ith batch are i.i.d. samples (reasonable for cell-capture techniques). If this assumption is untenable in another domain, then the quantile-estimation should be accordingly adjusted (cf. Heidelberger & Lewis 1984).

Our procedure uses the Pool-Adjacent-Violators-Algorithm (PAVA), which given an input sequence $y_1,\dots ,y_L\in ℝ$, finds the least-squares-fitting nondecreasing sequence in only O(L) runtime (de Leeuw 1977). The basic PAVA procedure is extended to weighted observations by performing weighted backaveraging in Step 3. When multiple (ℓ_i,y_i) pairs are observed with identical covariate-levels, i.e. ∃ℓ s.t. N_ℓ : = |I_ℓ| > 1 where I_ℓ := {i : ℓ_i = ℓ}, we adopt the simple tertiary approach for handling predictor-ties (de Leeuw 1977). Here, one defines ${\overline{y}}_{\ell }$ as the (weighted) average of the {y_i : i ∊ I_ℓ} and for each level ℓ all y_i : i ∊ I_ℓ are simply replaced with their mean-value ${\overline{y}}_{\ell }$. Subsequently, PAVA is applied with non-uniform weights to ${\left\{\left(\ell ,{\overline{y}}_{\ell }\right)\right\}}_{\ell =1}^L$ where the ℓth point receives weight N_ℓ (or weight ${\Sigma }_{i\in I\ell }{w}_i$ w_i if the original points are assigned non-uniform weights w₁,…,w_N). By substituting “nonincreasing” in place of “nondecreasing” in Steps 2 and 3, the basic PAVA method can be trivially modified to find the least-squares nonincreasing sequence. From here on, we use PAVA((y₁,w₁),…, (y_N,w_N); δ) to refer to a more general version of basic PAVA, which incorporates observation-weights w_i (for multiple y values at a single ℓ), and a user-specified monotonicity condition δ ∊ {“nonincreasing”, “nondecreasing”} that determines which monotonic best-fitting sequence to find.

Basic PAVA Algorithm: ${\min }_{z\ell }{\displaystyle \sum }_L^{\ell =1}{\left(y_{\ell }-z_{\ell }\right)}^2\text{s.t.}{z}_1⩽\cdots ⩽z_L$
Input:  A sequence of real numbers y1 ,…, yL
Output: The minimizing sequence ${\widehat{y}}_1,\dots ,{\widehat{y}}_L$ which is nondecreasing.
1. Start with the first level ℓ = 1 and set the fitted value ${\widehat{y}}_1=y_1$
2. While the next $y_{\ell }⩾{\widehat{y}}_{\ell -1}$, set ${\widehat{y}}_{\ell }=y_{\ell }$ and increment ℓ
3. If the next ℓ violates the nondecreasing condition, i.e. $y_{\ell }<{\widehat{y}}_{\ell -1}$ then backaverage to restore monotonicity: find the smallest integer k such that replacing ${\widehat{y}}_{\ell },\dots ,{\widehat{y}}_{\ell -k}$ by their average restores the monotonicity of the sequence ${\widehat{y}}_1,\dots ,{\widehat{y}}_{\ell }$. Repeat Steps 2 and 3 until ℓ = L.

Theorem 1. The Trend-Fitting algorithm produces valid quantile-functions ${\widehat{G}}_1^{-1},\dots ,{\widehat{G}}_L^{-1}$ which solve the numerical version of the TRENDS objective given in (8).

Fundamentally, our TF algorithm utilizes Dykstra’s method of alternating projections (Boyle & Dykstra 1986) to project between the set of L-length sequences of vectors which are monotone in each index over ℓ and the set of L-length sequences of vectors where each vector represents a valid quantile function. Despite the iterative nature of alternating projections, we find that the TF algorithm converges extremely quickly in practice. This procedure has overall computational complexity O(TLP² + NP), which is efficient when T (the total number of projections performed) is small, since both P and L are limited. The proof of Theorem 1 provides much intuition on the TF algorithm (all proofs are relegated to §S8). Essentially, once we fix a δ configuration (specifying which quantiles are decreasing over ℓ and which are increasing), our feasible set becomes the intersection of two convex sets between which projection is easy via PAVA. Furthermore, the second statement in our trend definition limits the number of possible δ configurations, so we simply solve one convex subproblem for each possible δ to find the global solution.

Trend-Fitting Algorithm: Numerically solves (7) by optimizing (8)
Input 1: Empirical distributions and associated levels (and optional weights) ${\left\{\left({\ell }_i,{\widehat{F}}_i,w_i\right)\right\}}_{i=1}^N$
Input 2: A grid of quantiles to work with 0 < p1 < ⋯ < pP–1 < 1
Output: The estimated quantiles of each $Q_{\ell }\left\{{\widehat{G}}_{\ell }^{-1}\left(p_k\right):k=1,\dots ,P-1\right\}$ for $\ell \in \left\{1,\dots ,L\right\}$ from which these underlying trending distributions can be reconstructed.
1. ${\widehat{F}}_i^{-1}\left(p_k\right):=\mathbf{\text{quantile}}\left({\widehat{F}}_i,p_k\right)$ for each i ∈ {1,…,N}, k ∈ {1,…,P – 1}
2. $w_{\ell }^{*}:={\displaystyle \sum _{i\in I_{\ell }}{w}_i}$ for each ℓ ∈ {1,…,L}
3. $x_{\ell }\left[k\right]:=\frac{1}{w_{\ell }^{*}}{\displaystyle \sum _{i\in I_{\ell }}{w}_i{\widehat{F}}_i^{-1}\left(p_k\right)}$ for each ℓ ∈ {1,…,L}, k ∈ {1,…,P – 1}
4. for p* = 0,p1, p2, pP-1 :
5.  δ[k] := “nondecreasing” if pk > p*;otherwise δ[k] := “nondecreasing”
6.  y1,…,yL := AlternatingProjections $\left(x_1,\dots ,x_L;\delta ;{\left\{w^{*}\right\}}_{\ell =1}^L,{\left\{p_k\right\}}_{k=1}^{P-1}\right)$
7.  W[δ] := the value of (8) evaluated with $G_{\ell }^{-1}\left(p_k\right)=y_{\ell }\left[k\right]\forall \ell ,k$
8.  Redefine δ[k] := “nonincreasing” if pk > p*; otherwise δ[k] := “nonincreasing” and repeat Steps 6 and 7 with the new δ
9.  Identify min W[δ] and return $G_{\ell }^{-1}\left(p_k\right)=y_{\ell }^{*}\left[k\right]\forall \ell ,k$ where y* was produced at the Step 6 or 8 corresponding to δ* := argmax W[δ].

AlternatingProjections Algorithm: Finds the Wasserstein-least-squares sequence of vectors which represent valid quantile-functions and a trend whose monotonicity is specified by δ.
Input 1:  Initial sequence of vectors $x_1^{\left(0\right)},\dots ,x_L^{\left(0\right)}$
Input 2:  Vector δ whose indices specify directions constraining the quantile-changes over ℓ.
Input 3:  Weights $w_{\ell }^{*}\in ℝ$ and quantiles to work with 0 < p1 < ⋯ < pP–1 < 1
Output:  Sequence of vectors $y_1^{\left(t\right)},\dots ,y_L^{\left(t\right)}$ where $\forall \ell ,k:y_{\ell }^{\left(t\right)}\left[k\right]⩽y_{\ell }^{\left(t\right)}\left[k+1\right]$ and the sequence $y_1^{\left(t\right)}\left[k\right],\dots ,y_L^{\left(t\right)}\left[k\right]$ is monotone nonincreasing/nondecreasing as specified by δ[k], provided that $x_{\ell }^{\left(0\right)}\left[k\right]\le x_{\ell }^{\left(0\right)}\left[k+1\right]$ for each ℓ, k
1. $r_{\ell }^{\left(0\right)}\left[k\right]:=0,s_{\ell }^{\left(0\right)}\left[k\right]:=0$ for each ℓ ∈ {1,…,L}, k ∈ {1,…,P – 1}
2. for t = 0, 1, 2, … until convergence:
3.   $y_1^{\left(t\right)}\left[k\right],\dots ,y_L^{\left(t\right)}\left[k\right]:=\mathrm{PAVA}\left(\left(x_1^{\left(t\right)}\left[k\right]+r_1^{\left(t\right)}\left[k\right],w_1^{*}\right),\dots ,\left(x_L^{\left(t\right)}\left[k\right]+r_L^{\left(t\right)}\left[k\right],w_L^{*}\right);\delta \left[k\right]\right)$ for each k ∈ {1,…,P – 1}.PAVA computes either the least-squares nondecreasing or nonincreasing weighted fit, depending on δ[k].
4.   $r_{\ell }^{\left(t+1\right)}\left[k\right]:=x_{\ell }^{\left(t\right)}\left[k\right]+r_{\ell }^{\left(t\right)}\left[k\right]-y_{\ell }^{\left(t\right)}\left[k\right]$ for each ℓ, k
5.   $\forall \ell \in \left\{1,\dots ,L\right\}:x_{\ell }^{\left(t+1\right)}\left[1\right],\dots ,x_{\ell }^{\left(t+1\right)}\left[P-1\right]:=\mathrm{PAVA}\left(\left(y_{\ell }^{\left(t\right)}\left[1\right]+s_{\ell }^{\left(t\right)}\left[1\right],\frac{p_2-p_0}{2}\right),\dots ,\left(y_{\ell }^{\left(t\right)}\left[P-1\right]+s_{\ell }^{\left(t\right)}\left[P-1\right],\frac{p_P-p_{P-2}}{2}\right);"\text{nondecreasing}"\right)$
6.   $s_{\ell }^{\left(t+1\right)}\left[k\right]:=y_{\ell }^{\left(t\right)}\left[k\right]+s_{\ell }^{\left(t\right)}\left[k\right]-x_{\ell }^{\left(t+1\right)}\left[k\right]$ for each ℓ, k

9. Theoretical results

Under the model given in (3), we establish some results regarding the quality of the ${\widehat{Q}}_1,\dots ,{\widehat{Q}}_L$ estimates produced by the TF algorithm. To develop pragmatic theory, we use finite-sample bounds defined in terms of quantities encountered in practice rather than the true Wasserstein distance (1), which relies on an integral that must be numerically approximated. Thus, in this section, d_W(·,·) is used to refer to the midpoint-approximation of the L₂ Wasserstein integral illustrated in (8). In addition to the conditions of model (3), we make the following simplifications throughout for ease of exposition:

(A.4) The number of batches at each level is the same, i.e. N_ℓ := N₁ = ⋯ = N_L ⩾ 1

(A.5) The same number of samples are drawn per batch, i.e. n := n_i for all 1 ⩿ i ⩿ N

(A.6) For k = 1,…,P — 1: the (k/P)th quantiles of each distribution are considered

(A.7) Uniform weights are employed, i.e. in (7): w_i = 1 for all i

Theorem 2. Under model (3) and additional conditions (A. 4)-(A.7), suppose the TF algorithm is applied directly to the true quantiles of Pi,…, P_N. Then, given any ϵ > 0, the resulting estimates satisfy:$d_W\left({\widehat{G}}_{\ell }^{-1},{\widehat{G}}_{\ell }^{-1}\right)<\in$ for each ℓ ϵ{1,…,L}

$$withprobabilitygreaterthan:1-2PL\exp \left(-\frac{{ϵ}^2{N}_{\ell }}{8{\sigma }^{2}L}\right)$$ (9)

Thus, Theorem 2 implies that our estimators are consistent with asymptotic rate $O_P\left(1/\sqrt{N_{\ell }}\right)$ if we directly observe the true per-batch quantiles P₁,…,P_N (which are contaminated by Ɛ_i under our model). By using the union-bound, our proof does not require any independence assumptions for the noise introduced at different quantiles of the same batch. Because direct quantile-observation is unlikely in practice, we now examine the performance of TRENDS when these quantiles are instead estimated using n samples from each P_i. Here, we additionally assume:

(A.8) For i = 1,…,N : quantiles are estimated from n i.i.d. samples X_1,i,…, X_n,i ~ P_i

(A.9) There is nonzero density at each of the quantiles we estimate, i.e. CDF F_i is strictly increasing around each $F_i^{-1}\left(k/P\right)$ for k = 1,…,P – 1.

(A.10) The simple quantile estimator defined below is used for each k/P, k = 1,…,P – 1

$${\widehat{F}}_i^{-1}\left(p\right):=\inf \left\{x:{\widehat{F}}_i\left(x\right)⩾p\right\}$$

where ${\widehat{F}}_i(\cdot )$ is the empirical CDF computed from X_1,i,…,X_n,i, ~ P_i.

Theorem 3. Under the assumptions of Theorem 2 and (A.8)-(A.10), suppose the TF algorithm is applied to estimated quantiles ${\widehat{F}}_i^{-1}\left(k/P\right)$ for i = 1,…,N,k = 1,…,P – 1. Then, given any ϵ > 0, the resulting estimates satisfy:$d_W\left({\widehat{G}}_{\ell }^{-1},G_{\ell }^{-1}\right)<\in$ for each ℓ ∈ {1,…,L} with probability greater than:

$$1-2PL\left[\exp \left(\frac{-{ϵ}^2{N}_{\ell }}{32{\sigma }^{2}L}\right)+N_{\ell }\exp \left(-2n\cdot R{\left(\frac{ϵ}{4\sqrt{L}}\right)}^2\right)\right]$$ (10)

Where for γ > 0:

$$\begin{array}{r}\hfill R(\gamma ):=\underset{i,k}{\min }\left\{R\left(\gamma ,i,k/p\right):i=1,\dots ,N,k=1,\dots ,P-1\right\}\\ \hfill R\left(\gamma ,i,p\right):\min \left\{F_i\left(F_i^{-1}\left(p\right)+\gamma \right)-p,p-F_i\left(F_i^{-1}\left(p\right)-\gamma \right)\right\}\end{array}$$ (11)

Theorem 3 is our most general result applying to arbitrary distributions P_i that satisfy basic condition (A.9). However, the resulting probability-bound may not converge toward to 1 if $n\cdot R{\left(\frac{ϵ}{4\sqrt{L}}\right)}^2<O\left(\log N_{\ell }\right)$, which occurs if few samples are available per batch (because then the P_i are can be very poorly estimated). Thus, TRENDS is in general only designed for applications with large per-batch sample sizes. The bounds obtained under the extremely broad setting of Theorem 3 may be significantly improved by instead adopting one of the following stronger assumptions:

(A.11) The simple quantile-estimator defined in (A.10) is used, and the support of each P_i is bounded and connected with non-neglible density, i.e. ∃ constants B, c > 0 s.t. ∀_i : f_i(x) =0 ∀x ∉ [–B, B] and f_i(x) ⩾c ∀ x ∊ [–B, B] (f_i is density for CDF F_i).

(A.12) The following is known regarding the quantile-estimation procedure:

1. The quantiles of each P_i are estimated independently of the others.

2. The quantile-estimates converge at a sub-Gaussian rate for each quantile of interest, i.e. there exists c > 0 such that for each k, i and any ϵ > 0:

   $$\mathrm{Pr}\left(|{\widehat{F}}_i^{-1}\left(k/P\right)-F_i^{-1}\left(k/P\right)|>ϵ\right)\le 2\exp \left(-2n{c}^2{ϵ}^2\right)$$

Theorem 4. Under the assumptions of Theorem 2, conditions (A.8), (A.9), and one of either (A.ll) or (A.12), the bound in (10) may be sharpened to ensure that for any ϵ > 0:

$$d_W\left({\widehat{G}}_{\ell }^{-1},G_{\ell }^{-1}\right)<ϵforeach\ell \in \left\{1,\dots ,L\right\}$$

with probability greater than:

$$1-2P\left[L\exp \left(\frac{-{ϵ}^2{N}_{\ell }}{32{\sigma }^{2}L}\right)+\exp \left(-\frac{c^2}{8}{N}_{\ell }n{ϵ}^2\right)\right]$$ (12)

In Theorem 4, the additional assumption of bounded/connected underlying distributions results in a much better finite sample bound that is exponential in both n and N_ℓ (implying asymptotic $O_P\left(N_{\ell }^{-1/2}+n^{-1/2}\right)$ convergence). While this condition and the result of Theorem 3 assume use of the simple quantile-estimator from (A.10), numerous superior procedures have been developed which can likely improve practical convergence rates (Zielinski 2006). Assuming guaranteed bounds for the quantile-estimation error (which may be based on both underlying properties of the P_i as well as the estimation procedure), one can also obtain the same exponential bound. In fact, condition (A.11) is an example of a distribution and quantile-estimator combination which achieves the error required by (A.12). Because the boundedness assumption is undesirably limiting, we also derive a similar result under weaker assumptions:

(A.13) Each P_i has connected support with non-neglible interior density and sub-Gaussian tails, i.e. there are constants B > b > 0,a > 0,c > 0 such that for all i :

• (1)F_i is strictly increasing,
• (2)f_i(x) ⩾ c ∀x ∊ [–B, B] where f_i is the density function of CDF F_i.
• (3)Pr(X_i > x) ⩽ exp (– a [x – (B – b)]²) if x > B

and Pr(X_i < x) ⩽ exp (– a [x – (–B + b)]²) if x < –B

(A.14) Defining $r:=\min \left\{2c^2,\frac{2a{b}^2-1}{4P{B}^2}\right\}$, we have r > 0, or equivalently, 2ab² > 1.

(A.15) We avoid estimating extreme quantiles, i.e. $F_i^{-1}\left(k/P\right)\in \left(-B,B\right)\forall k\in \left\{1,\dots ,P-1\right\}$

Theorem 5. Under the assumptions of Theorems 2 and 3 as well as conditions (A.13)-(A.15), the previous bound in (10) may be sharpened to ensure that for all ϵ > 0:

$$d_W\left({\widehat{G}}_{\ell }^{-1},G_{\ell }^{-1}\right)<ϵforeach\ell \in \left\{1,\dots ,L\right\}$$

with probability greater than:

$$1-2P\left[L\exp \left(\frac{-{ϵ}^2{N}_{\ell }}{32{\sigma }^{2}L}\right)+\exp \left(-\frac{r^2}{16}{N}_{\ell }n{ϵ}^2\right)\right]$$ (13)

Theorem 5 again provides an exponential bound in both n and N_ℓ under a realistic setting where the distributions are small tailed with connected support, and the simple quantile estimator of (A.10) is applied at non-extreme quantiles. Note that while we specified properties of the distributions, noise, and quantile estimation in order to develop this theory, our nonparametric significance tests do not rely on these assumptions.

10. Simulation study

We perform a simulation which realistically reflects various properties of scRNA-seq data, based on assumptions similar to those explicitly relied upon by the model of Kharchenko et al. (2014). Samples are generated from one of the following choices of the underlying trending distribution sequence Q₁,…,Q_L with L = 5 (additional details in §S4):

(S₁) Q_ℓ ~ NB(r_ℓ,p_ℓ) with r_ℓ = 5 and p_ℓ = 0.3, 0.3, 0.4,0.5, 0.8 for ℓ = 1,…,5.

(S₂) Q_ℓ is a mixture of NB(r = 5,p = 0.3) and NB(r = 5,p = 0.7) components, with the mixing proportion of the latter ranging over λ_ℓ = 0.1, 0.4, 0.8,0.8, 0.8 for ℓ = 1,…,5.

(S₃) Q_ℓ ~ NB(r = 5,p = 0.5) for ℓ = 1,…,5.

NB(r, p) denotes the negative binomial distribution parameterized by r (target number of successful trials) and p (probability of success in each trial). To capture various types of noise affecting scRNA-seq measurements (e.g. dropout, PCR amplification bias, transcriptional bursting), noise for the i^th batch is introduced (independently of the other batches) via the following steps: rather than sampling from $Q_{{\ell }_i}$, we instead sample from $P_{{\ell }_i}~NB\left(\widetilde{r_{\ell }},{\tilde{p}}_{\ell }\right)$, where ${\tilde{r}}_{\ell }=r_{\ell }+r_{noise}$ and ${\tilde{p}}_{\ell }=p_{\ell }+p_{noise}$. Here, p_noise, r_noise are independently drawn from centered Gaussian distributions with standard deviations σ, 10 σ respectively (σ thus controls the degree of noise). For the mixture-models in S₂, we sample from $P_{{\ell }_i}$ which is also a mixture of negative binomials (with the same mixing proportions as $Q_{{\ell }_i}$) where the parameters of both mixing components are perturbed by noise variables r_noise,p_noise. To the observations sampled from $P_{{\ell }_i}$, we finally apply a log₁₀(x + 1) transform (also applied to the scRNA-seq data in §11) before proceeding with our analysis.

We first investigate the convergence of TRENDS estimates under each of the models S₁, S₂, and S₃, varying n, N_ℓ, and the amount of noise independently. Figure 4 shows the Wasserstein error (sum over ℓ of the squared Wasserstein distances between the underlying Q_ℓ and estimates thereof) of our TRENDS estimates vs. the error of the empirical distributions. The plot demonstrates rapid convergence of the TRENDS estimator (as guaranteed by our theory in §9) and shows that TRENDS can produce a much better picture of the underlying distributions than the (noisy) observed empirical distributions. As shown in Figure 4A, this may occur even in the absence of noise, thanks to the additional structure of the trend-assumption exploited by our estimator. Thus, when the underlying effects follow a trend, our Δ statistic provides a much more accurate measure of their magnitude than distances between the empirical distributions. These results indicate that the largest benefit of our TRENDS approach is for small to moderate sized samples.

Figure 4:

The Wasserstein error of the TRENDS fitted distributions vs. the observed empirical distributions, under models S₁ - S₃ with various settings of n, σ, and N_ℓ. Depicted is the average error (and standard deviation) over 100 repetitions.

To compare performance, we evaluate TRENDS against alternative methods under our models S_i-S₃ with substantial batch-noise (σ = 0.1). Fixing N_ℓ = 1,n_i = 1000 for all ℓ, i, we generate 400 datasets from the different underlying trending models described above (100 from each of S_i, S₂, and 200 from S3). TRENDS is applied to each dataset to obtain a p-value (via the permutation procedure described in §S2). In this analysis, we also apply the following alternative methods (detailed in §S3): a linear variant of our TRENDS model (where quantiles are restricted to evolve linearly rather than monotonically), an omnibus-testing approach (using the maximal Kolmogorov-Smirnov (KS) statistic between any pair of distributions), and a measure of the (marginally-normalized) mutual information (MI) between ℓ and the values in each batch. The latter two alternative methods make no underlying assumption and capture arbitrary variation in distributions over ℓ. We employ the same approach to ascertain statistical significance (at the 0.05 level) under each method. All p-values are obtained via permutation-testing (with 1000 permutations). To correct these p-values for multiple comparisons, we employ the step-down minP adjustment algorithm of Ge et al. (2003), which cleverly avoids double permutations to remain computationally efficient.

Table 1 demonstrates that methods sensitive to arbitrary differences in distributions are highly susceptible to spurious batch effects (both the KS and MI identify all 400 datasets as statistically significant), whereas our TRENDS method has the lowest false-positive rate, only incorrectly rejecting its null hypothesis for 4 out of the 200 datasets from S₃. TRENDS also exhibits the greatest power in these experiments. To ascertain how well these methods distinguish the trending data from the non-trending samples, we computed area under the ROC curve (AUROC) by generating ROC curves for each method using its p-values (ties broken using test statistics) as a classification-rule for determining which simulated datasets the method would correctly distinguish from constant model S3 at each possible cutoff value. The results of Table 1 show that TRENDS is superior at drawing this distinction in these simulations.

Table 1:

False-positive rate (FPR) and true-positive rate (TPR) produced by different methods, as well as AUROC values. FPR is determined by the fraction of datasets generated under model S₃ deemed statistically significant (or S_i, S₂ for TPR).

Method	FPR	TPR	AUROC
TRENDS	0.02	0.35	0.87
Linear-TRENDS	0.03	0.32	0.85
KS	1.0	1.0	0.44
MI	1.0	1.0	0.53

11. Single cell RNA-seq analysis

To evaluate the practical utility of our method, we analyze two scRNA-seq time course experiments and compare TRENDS against the alternative approaches described in §S3. The first dataset is from Trapnell et al. (2014) who profiled single-cell transcriptome dynamics of skeletal myoblast cells at 4 time-points during differentiation (myoblasts are embryonic progenitor cells which undergo myogenesis to become muscle cells). In a second larger-scale scRNA-seq experiment, Zeisel et al. (2015) isolated 1,691 cells from the somatosensory cortex (the brain’s sensory system) of juvenile CD1 mice aged P22-P32. We treat age (in postnatal days) as our batch-labels, with L = 10 possible levels. §S5 contains detailed descriptions of the data and our analysis.

Assuming that trending temporal-progression effects on expression reflect each gene’s importance in development, we measure the size of these effects using our Δ statistic (6). Fitting a separate TRENDS model to each gene’s measurements, we thus produce a ranking of the genes’ presumed developmental importance. If instead, one’s goal is simply to pinpoint high-confidence candidate genes relevant at all in development (ignoring the degree to which their expression transforms in the developmental progression), then our permutation test can be applied to establish which genes exhibit strong statistical evidence of an underlying nonconstant TREND effect. For all methods, p-values are obtained using the same procedure as in the simulation study (1000 permutations with step-down minP multiple-testing correction). In these analyses, significance testing (which identifies high-confidence effects) and the Δ statistic (which identifies very large effects) both produce informative results.

As the myoblast data only contains four ℓ-levels and one batch from each, the TRENDS permutation test stringently identifies only 20 genes with significant non-constant trend at the 0.05 level (with multiple-testing correction). Terms which are statistically overrepresented in the Gene Ontology (GO) annotations of these significant genes (Kamburov et al. 2011), indicate the known developmental relevance of a large subset (see Figure 5A). Enriched biological process annotations include “anatomical structure development” and “cardiovascular system development” (Table S2A). In contrast, the cortex data are much richer, and TRENDS accordingly finds far stronger statistical evidence of trending genes, identifying 212 as significant (at the 0.05 level with multiple testing correction). A search for GO enriched terms in the annotations of these genes shows a large subset to be developmentally relevant (Figure 5B), with enriched terms such as “neurogenesis” and “nervous system development” (Table S2B). Due to the limited batches in these scRNA-seq data (each of which may be corrupted under our model), the TRENDS significance-tests act conservatively (a desirable property given the pervasive noise in scRNA-seq data), identifying small sets of genes we have high-confidence are primarily developmentally relevant.

Figure 5:

Word clouds of terms significantly enriched (at the 0.01 level) in GO annotations of the genes with significantly trending expression in each analysis (Kamburov et al. 2011).

Ranking the genes by their TRENDS-inferred developmental effects (using Δ), 9 of the top 10 genes in the myoblast experiment have been previously discovered as significant regulators of myogenesis and some are currently employed as standard markers for different stages of differentiation (see Table S3A). Also, 7 of the top 10 genes in the cortex analysis have been previously implicated in brain development, particularly in sensory regions (Table S3B). Thus, TRENDS accurately assigns the largest inferred effects to clearly developmental genes (see also Table S4). Since experiments to probe putative candidates require considerable effort, this is a very desirable feature for studying less well-characterized developmental systems than our cortex/myoblast examples. Figure 1A shows TRENDS predicts that MT2A (the gene with the largest Δ-inferred effect in myoge-nesis and a known regulator of this process) is universally down-regulated in development across the entire cell population. Interestingly, the majority of cells express MT2A at a uniformly high level of ⩾ 3 log FPKM just before differentiation is induced, but almost no cell exhibits this level of expression 24 hours later. MT2A expression becomes much more heterogenous with some cells retaining significant MT2A expression for the remainder of the time course while others have stopped expressing this gene entirely by the end. TRENDS accounts for all of these different changes via the Wasserstein distance which appropriately quantifies these types of effects across the population.

Because any gene previously implicated in muscle development is of interest in the myoblast analysis, we can form a lower-bound approximation of the fraction of “true positives” discovered by different methods by counting the genes with a GO annotation containing both the words “muscle” and “development” (e.g. “skeletal muscle tissue development”). Table S5 contains all GO annotations meeting this criterion. Figure 6A depicts a pseudo-sensitivity plot based on this approximation over the genes with the highest presumed developmental importance inferred under different methods. Here, the Tobit models are censored regressions specifically designed for scRNA-seq data, which solely model conditional expectations rather than the full distribution of expression across the cells (see §S3). A larger fraction of the top genes found by TRENDS and our closely-related Linear TRENDS method have been previously annotated for muscle development than top candidates produced by the other methods.

Figure 6:

Pseudo-sensitivity of various methods based on their ability to identify known developmental genes. (A) the number of genes with a GO annotation containing both “muscle” and “development” found in the top K genes (ranked by the different methods for the myoblast data), over increasing K. (B) similar plot for the cortex data, where developmental genes are now those annotated with a relevant GO term from Table S6.

We repeat this analysis for the cortex data using a different set of “ground truth” GO annotations (listed in Table S6), and again find that TRENDS produces higher sensitivity than the other approaches (Figure 6B) based on this crude measure. As researchers cannot practically probe a large number of genes in greater detail, it is important that a computational method for developmental gene discovery produces many high ranking true positives which can be verified through limited additional experimentation. While TRENDS appears to display greater sensitivity than other methods, we note that it is difficult to evaluate other performance-metrics (e.g. specificity) using the scRNA-seq data, since the complete set of genes involved in relevant developmental processes remains unknown.

The Nestin gene in the myoblast data provides one example demonstrating the importance of treating full expression distributions rather than just mean-effects. Nestin plays an essential role in myogenesis, determining the onset and pace of myoblast differentiation, and its overexpression can also bring differentiation to a halt (Pallari et al. 2011), a process possibly underway in the high-expression cells from the later time points depicted in Figure 1B. TRENDS ranks Nestin 35th in terms of inferred developmental effect-size (with TRENDS p-value = 0.02 before multiple-testing correction and 0.09 after), but this gene is overlooked by the scalar regression methods (only ranking 3,291 and 5,094 in the linear / B-spline Tobit results). Although Figure 1B depicts a clear temporal effect on mean Nestin expression, scalar regression does not prioritize this gene because these methods fail to properly consider the full spectrum of changes affecting different segments of the cell population in the multitude of other genes with similar mean-effects as Nestin.

Although the closely-related Linear TRENDS model appears to do nearly as well as TRENDS in our Figure 6 pseudo-sensitivity analysis, we find the linearity assumption overly restrictive, preventing the Linear TRENDS model from identifying important genes like TSPYL5, a nuclear transcription factor which suppresses levels of well-known myogenesis regulator p53 (Epping et al. 2011, Porrello et al. 2000). Linear TRENDS model only assigns this gene a p-value of 0.2 whereas TRENDS identifies it as significant (p = 0.05), since TSPYL5 expression follows a monotonic trend fairly closely (R² = 0.95) but is not as well approximated by a linear trend (R² = 0.68).

12. Discussion

While established methods exist to quantify change over a sequence of probability distributions, TRENDS addresses the scientific question of how much of the observed change can be attributed to sequential progression rather than nuisance variation. Although the TF algorithm resembles quantile-modeling techniques, our ideas are grounded under the unifying lens of the Wasserstein distance, which we use to measure effects (6), goodness-of-fit (5), and a distribution-based least-squares fit (4). Like linear regression, an immensely popular scientific method despite rarely reflecting true underlying relationships, our TRENDS model is not intended to accurately model/predict the data, which are likely subject to many more effects than our simple trend definition encompasses. Rather, TRENDS quantifies effects of interest, which remain highly interpretable (via our Wasserstein-perspective) despite being considered across fully nonparametric populations.

We recommend our model for data in which the underlying population is heterogeneous (possibly subject to diverse effects), each batch contains many samples (n_i ⩾ 50), and the sequence of levels L ⩾ 3 is short enough that effects of interest should follow persistent trends. When considering TRENDS analysis, it is important to ensure that the primary effects of interest are a priori expected to follow our trend definition. For the developmental scRNA-seq data considered in this work, this is a reasonable assumption because the experiments typically focus on a limited window of the underlying process. Furthermore, the severe prevalence of nuisance variation makes it preferable to identify a high-confidence developmentally-relevant subset of genes (e.g. because they display consistent effects over time), rather than attempting to characterize the complete set of genes displaying interesting effects.

While our trend definition produces good empirical results in these scRNA-seq analyses (and encompasses various conceptually interesting effects discussed in §S1), we emphasize that adopting this assumption narrowly restricts the sort of effects measured by our approach. Our limited definition is unlikely to characterize more complex effects of interest in general settings (particularly for longer sequences), and future work should explore extensions such as allowing change-points in the model. Note that our proposed Wasserstein-least-squares fit objective and Wasserstein-R² measure remain applicable for more general classes of regression functions on distributions. Furthermore, Lemma 2 provides an alternative definition of a trend which also applies to multidimensional distributions, and thus may be useful for applications such as spatiotemporal modeling. Nevertheless, the basic TRENDS methodology presented in this work can produce valuable insights. As simultaneously-profiled cell numbers grow to the many-thousands thanks to technological advances (Macosko et al. 2015), significant discoveries may be made by studying the evolution of population-wide expression distributions, and TRENDS provides a principled framework for this analysis.