LOESS and DE-SWAN can induce artifactual “waves” of molecular aging

Abstract

A growing body of literature has investigated the relationship between age and biomolecular changes, leading to conclusions that aging occurs in discrete molecular “waves.” Data summary tools such as LOESS and sliding window analyses like DE-SWAN are common approaches that have gained acceptance in recent years. We demonstrate via simple simulations that these tools can identify non-linear patterns of aging where they do not exist. Specifically, we show that (i) clustering of molecular trajectories using LOESS can lead to artifactual characteristic patterns of molecular aging, (ii) “waves” of aging identified using the combination of LOESS and DE-SWAN in real data are not robust to changes in the underlying age distribution and are not supported by valid permutation testing, and (iii) DE-SWAN alone can generate pronounced “waves” of nonlinear molecular aging in linear data due to differences in statistical power along the age continuum. Our results specifically challenge the statistical support for discrete aging crests inferred in the literature, but do not rule out nonlinear molecular aging or age-associated transitions that may be detectable using other cohorts and statistical models.

1. Introduction

An increasing number of studies have explored aging at a molecular level with the goal of identifying biological processes that change over time. Such studies typically assay multi-omic abundance data (such as gene expression, protein expression, and metabolomic profiles) in cohorts of individuals spanning a broad range of ages. These data are then analyzed to identify molecules that change along the continuum of age, with an emphasis on finding non-linear trajectories. Molecules showing significant differences are further examined to generate hypotheses about how age-dependent alterations may influence health and disease (15; 24).

Ideally, such studies would use a longitudinal design in which a large cohort of research subjects is followed over years, and individuals are profiled at multiple time points—a design that is often impractical due to time and expense. An alternative would be to use a cross-sectional design in which a relatively large number of individuals, uniformly distributed across a prespecified age range, are profiled once. However, this is also impractical in most cases. In reality, most available aging study cohorts suffer from nonuniformity of important data attributes (number of samples, variance, presence of outliers, etc.) that are associated with age distribution. The resulting variations in statistical power across the age continuum impair valid statistical analysis of the relationship between age and the levels of various biomolecules.

Analyzing these data is challenging; so far, there are no established standard best practices in this context. Many recent studies use locally estimated scatterplot smoothing (LOESS) (3), Differential Expression - Sliding Window ANalysis (DE-SWAN) (15), or a combination of the two.

LOESS (an acronym for LOcally Estimated Scatterplot Smoothing) is a nonparametric regression method used to estimate smooth relationships between variables without assuming a specific functional form. Instead of fitting one global curve (like a straight line or polynomial) to all the data, LOESS fits many small local regressions around each point and assembles them into a smooth curve (3; 10). At each point, the local regression function is estimated using a subset of the data belonging to a local neighborhood. The proportion of the data included in this neighborhood is the “span” tuning parameter, often denoted α and tuned via cross-validation. Typically, each data point is weighted so that points closest to the neighborhood center have a larger influence on the local regression. After the local regressions have been fit at each point in a grid, the fitted values are connected to create a smooth curve (10).

The Differential Expression-Sliding Window ANalysis (DE-SWAN) algorithm uses a sliding window approach to detect points at which molecules are differentially expressed along a quantitative trait, which is typically age (15). Within each window, the method tests for a difference between individuals younger and older than the window center. This method has been deployed across many molecular assays (such as RNA-seq, proteomic, and metabolomic data) so that results from the collection of differentially expressed molecules can be pooled to identify particular ages at which there may be a concerted biomolecular change (24).

DE-SWAN is based on two key input parameters: a set of window centers and a set of “bucket” sizes determining the width of each window. Based on these choices, DE-SWAN sequentially steps through the window centers and applies a statistical test for differential expression between two groups: those samples falling in the lower half of the window (in the case of age, “younger”) and those in the upper half (“older”). Repeating this test for each molecule results in a set of molecules determined to be significantly differentially expressed between the two halves of the given window. The algorithm then shifts to the next window center and repeats the process (Figure 1). The original DE-SWAN algorithm uses a linear regression model to compare expression between the younger and older groups (15). However, DE-SWAN has been adapted to test for differential expression using other statistical tests (24).

Figure 1:

Example of DE-SWAN algorithm using toy data. The graphic shows how, for each molecule, the sliding window analysis traverses the age range. The orange window shows the younger group and the green window shows the older group. At each window midpoint, the difference between these two groups is assessed using a statistical test. The number of molecules assessed as significantly different at each window midpoint is then plotted and interpreted for patterns such as nonlinearity.

Here, we critically analyze how LOESS, DE-SWAN, and LOESS+DE-SWAN (see Section 3) have been used to study the aging process. We demonstrate that these tools can elicit conclusions driven by structural features of the data (such as the age distribution, heteroskedasticity, and outliers) rather than by substantive biological signal (such as true changes in expression). We show that when applied to null datasets consisting of random noise, these tools can identify strong, dramatic patterns that mirror those in published papers. Absent unfeasible assumptions about structural features of the sampled data, the results of these pipelines cannot reliably characterize the aging process.

The remainder of this note is structured as follows. In Section 2, we discuss established weaknesses of LOESS and show how these weaknesses can be problematic for the common “smooth-cluster-interpret” workflow used in recent papers (for example, ^{4; 15; 16; 22; 24; 29; 30}). In Section 3, we show how the combination of LOESS and DE-SWAN can find artifactual “waves” of aging in simulated datasets that were explicitly constructed to have no true signal. We also discuss how LOESS+DE-SWAN has been used to find waves in real datasets and how these identified waves do not survive permutation testing (24). In Section 4, we show that DE-SWAN (without LOESS) can indicate evidence of nonlinearity even when the underlying data-generating process is strictly linear, due to variations in statistical power at different ages. Lastly, in Section 5, we discuss the importance of these findings and provide recommendations for future analyses of molecular aging and other related life course data. All code is open-source and publicly available at https://github.com/QuackenbushLab/artifactual-waves-of-aging.

2. Artifact 1: LOESS can induce clustered trajectories in null data

It has become common in the aging literature to model continuous trajectories of molecular levels as a function of age by using LOESS to generate smooth approximations from discrete measurements of these molecules originally taken from a cross-sectional population. In a typical workflow, these smoothed trajectories are then clustered, and cluster centroids are interpreted as characteristic profiles of aging (for example, molecules that are stable through midlife and then increase in expression or molecules that are highly expressed in younger individuals and then decrease with age). Enrichment analyses are typically conducted on these clusters to identify biological functions that may exhibit these characteristic trajectories through the life course. We refer to this as the “smooth-cluster-interpret” workflow.

The use of this workflow is problematic in the context of the sparse, non-uniform age distributions underlying most aging studies. In fact, our simulations demonstrate that the smooth-cluster-interpret workflow finds “biologically interpretable” aging trajectory shapes in null data. We simulated uniform, Gaussian, and Gaussian mixture age distributions (n = 200 samples in each case) as well as the age distribution underlying the transcriptomic results of Shen et al. (24) (n = 97); these data are part of the Integrated Personal Omics Profiling (iPOP) project Chen et al. (2) and are hereafter referred to as the “iPOP age distribution.” For each sample, we simulated null expression levels for 1000 molecules, where “null” refers to data generated from a standard Gaussian distribution that is independent of age. We then applied LOESS to these data, using a span of 0.6. The resulting trajectories were grouped using hierarchical clustering with Euclidean distance and complete linkage.

The age distributions, raw data, and LOESS-smoothed data are shown in Figure 2A. As expected, no apparent trajectories are observed in the raw data, and clustering yields a dendrogram of nearly equal height for each predictor. After applying LOESS to the data, smooth trajectories appear in the heatmap and a clear clustering structure appears in the dendrogram. Centroids of these clusters exhibit similar trends to those that have previously been presented as true signals in the aging literature (Figure 2B). Furthermore, as the number of clusters increases, the signals often become more “interpretable” – a conclusion that is, in fact, due to overfitting (Supplementary Figure S2).

Figure 2:

Hierarchical clustering of random null data before and after smoothing shows that smoothing induces patterns of aging that follow the age density, even when there is no true signal.

These analyses illustrate the necessity of using null comparisons; without such a benchmark for comparison, it is unclear if a true biological signal is being observed in the clustered data or if we are simply perceiving random artifacts.

3. Artifact 2: LOESS+DE-SWAN can elicit artifactual waves of aging that are not supported by null simulations and permutation testing

Here, we discuss how using LOESS and DE-SWAN in tandem can induce artifactual “waves of aging.” The LOESS+DE-SWAN workflow combines LOESS with DE-SWAN by first applying LOESS to the original data to interpolate a smooth curve of molecular expression and then running DE-SWAN on these interpolated values, using a Wilcoxon test to compare younger and older groups in each window (Figure 3a) (24). Using LOESS and DE-SWAN simultaneously is problematic because the points on a fitted LOESS curve are not independent from one another. Instead, each point along the curve is estimated relative to the same dataset and constrained to follow a smooth trajectory. Therefore, points along this curve will be highly dependent and cannot be used as independent observations supporting the same trend.

Figure 3:

A simulated example of inflated detection of differential expression in LOESS+DE-SWAN.

Using LOESS-interpolated data with DE-SWAN thus yields an immediate concern: a statistical test assuming independent data points is instead being used on highly correlated data. Most tests assume independent data points, so the problem with LOESS+DE-SWAN is not solely due to using the Wilcoxon test; however, it is exacerbated by the Wilcoxon test’s rank-based nature. This is because the smoothness of the LOESS-interpolated curve makes the rank ordering often nearly monotonic from the left to the right side of the sample window. We illustrate this phenomenon in Section 3.1 below.

3.1. Simulation example on a single gene

To demonstrate inflation of significance by LOESS+DE-SWAN, we consider a simple example in which LOESS+DE-SWAN is applied to a single gene with expression levels that have no relationship with age (linear or non-linear). Simulation details can be found in Supplementary Section S2.2.1.

As expected, a scatterplot of expression versus age shows no clear relationship (Figure 3a, i). A LOESS curve fit to these data reflects a classic limitation of LOESS: the curve is sensitive to the density of the data distribution, and its tail behavior differs from the behavior at the center due to the lower data density at the margins (Figure 3a, ii). When the LOESS curve is used to estimate expression according to the original age distribution, an artifactual trend is now observed in the random data (Figure 3a, iii). This artifactual trend is amplified when LOESS is used to interpolate data points across a uniform grid of points positioned every 0.5 years on the age distribution (Figure 3a, iv).

While scatterplot visualization already demonstrates how LOESS+DE-SWAN identifies false signals, direct statistical calculations reveal the magnitude of this issue. Figure 3b shows an example of DE-SWAN without LOESS. In this case, the ranks of the expression data are similarly distributed between the “younger” and “older” groups, as reflected by a non-significant Wilcoxon test result (p=0.75). Figure 3c shows the same calculation performed on the LOESS-interpolated expression values. Because of the tail behavior of the LOESS curve, the ranks are now substantially higher among the younger group than the older group, and the Wilcoxon test indicates high significance of this difference (p = 1.8 × 10⁻⁵).

While we focus on the first window in this example, the inflated statistical significance of LOESS+DE-SWAN appears to be sensitive to the underlying age distribution and persists across many windows in the sliding window analysis (Figure 3d). Windows including data near sparser regions of the age distribution seem to be particularly prone to finding spurious signals in null data.

3.2. Waves of aging appear in null data and depend on the age distribution

To illustrate how LOESS+DE-SWAN can generate spurious results related to the underlying age distribution, we conducted a simple simulation study. We simulated 1000 molecules for the four age distributions described in Section 2; see Supplementary Section S2.2.2 for details.

In every case, LOESS+DE-SWAN detected a majority of the 1000 simulated expression profiles to be significantly changing (controlling FDR at 0.05) at each of the window centers (Figure 4). Because there is no true change in the mean expression, these targets represent hundreds of false discoveries per time point. While this is notable in itself, the primary concern with these results is the observed relationship between the number of significant molecules and the age distribution (Figure 4A-B). It appears that LOESS+DE-SWAN is biased to identify peaks of changing expression in areas where the population density is rapidly changing. In particular, when there are few points in one or both halves of the sliding window, the influence of LOESS-interpolated points is amplified. In contrast, DE-SWAN without LOESS controls the FDR, finding few age-associated molecules in this setting where simulated expression is unassociated with age (Figure 4C).

Figure 4:

Simulation study of LOESS+DE-SWAN and DE-SWAN on randomly generated data with no true association between molecular expression and age.

We posit that this phenomenon could be the cause of the waves of aging in the mid-40s and early 60s reported by Shen et al. (24). To address this hypothesis more specifically, we conducted an additional simulation that used the iPOP age distribution but simulated random noise as the expression values for each gene, with no true change in mean expression as a function of age (Supplementary Section S2.2.2).

Figure 5a shows the results of this analysis, with the locations of the transcriptomic crests observed by Shen et al. (24) indicated. While the null results do not perfectly recapitulate the reported iPOP transcriptomic waves, it is notable that despite the lack of any true signal, two clear crests are apparent in our null study, one in the early 40s and one in the early 60s. Also notable is the correlation between the sample size in the sliding window and the number of changing molecules that are reported (Figure 5B); this phenomenon applies to both LOESS+DE-SWAN and DE-SWAN alone and is discussed further in Section 4. The appearance of two crests at these time points based on random noise calls into question the validity of the conclusion that they represent true biological trends.

Figure 5:

Simulation study repeating the transcriptomic analysis of Shen et al. (24) using the iPOP age distribution but with simulated gene expression values having no true association with age. The dashed lines indicate the locations of crests reported by Shen et al. (24).

3.3. DE-SWAN alone does not detect “waves of aging” in the iPOP data

To further examine the role of LOESS interpolation in these results, we analyzed the transcriptomic data from iPOP using DE-SWAN without LOESS. We downloaded code and data published by Shen et al. (24) and reproduced the results shown in Figure 4(d) of Shen et al. (24); see Figure 6A. To investigate the impact of running LOESS before conducting the DE-SWAN analysis, we next ran DE-SWAN on the transcriptomic data without applying the LOESS interpolation. With DE-SWAN alone, the peaks seen in Figure 6A disappear; no transcripts meet significance after controlling the FDR at 0.05 (Figure 6B).

Figure 6:

Reproducing the transcriptomic analyses of Shen et al. (24) on the original iPOP data using LOESS+DE-SWAN, and comparing with results obtained using DE-SWAN alone.

3.4. Use of permutation testing to validate the procedure

Another way to construct a null is via a permutation test, which preserves the marginal distribution of the expression data but breaks the dependence on age. Shen et al. (24) reported the results of a permutation test intended to validate the robustness of the aging crests, finding that the observed peaks disappeared following permutation of the ages. We attempted to reproduce this result by conducting a permutation test using the method in Algorithm 1 on the iPOP expression data and ages. In contrast to the results of Shen et al. (24), we found that clear peaks remain visible with the permuted data (Figure 7A). Thus, even when we broke the dependence between age and expression in the original data by shuffling the ages, LOESS+DE-SWAN still spuriously detected significant peaks that resembled the results from the unpermuted data. These results provide additional evidence that, under the LOESS+DE-SWAN workflow, the peaks observed in Figure 6A cannot be reliably distinguished from artifacts generated by the interaction of smoothing, interpolation, random variation, and the underlying age distribution.

Figure 7:

Permutation results and analysis.

Algorithm 1 Permutation testing for LOESS+DE-SWAN: Shuffle before LOESS

$$\begin{array}{cc}\hfill & \mathrm{Input}:\text{Subject-level transcriptomic data}X\text{and vector of ages}a.\hfill \\ \hfill & \mathrm{Output}:\text{Number of significant genes at each time}t.\hfill \\ \hfill & \text{For each permutation}i=1,\dots ,N:\hfill \\ \hfill & 1.\text{Permute ages:}{\tilde{a}}_i\leftarrow \text{permute}(a).\hfill \\ \hfill & 2.\text{Fit LOESS for each gene using}(X,{\tilde{a}}_a)\text{to obtain smoothed expression}{\tilde{X}}_i.\hfill \\ \hfill & 3.\text{Run DE-SWAN on}({\tilde{X}}_i,{\tilde{a}}_i)\text{to identify differential-expression time points for each gene}.\hfill \\ \hfill & 4.\text{For each age}t,\text{count the number of genes that have Benjamini}-\text{Hochberg-adjusted}p<0.05.\hfill \end{array}$$

The discrepancy between our permutation results (Figure 7A) and the permutation results reported in Supplementary Figure 4(c) of Shen et al. (24) is that the latter employed a subtly—but importantly—different approach, shown in Algorithm 2. The key difference is that Shen et al. (24) permuted the ages after LOESS interpolation of the data, whereas here we permuted the ages before LOESS. This seemingly small difference in procedure produces an invalid null comparison for the question at hand. The difference between Algorithm 1 and Algorithm 2 is shown for one gene, A1BG, in Figure 7B. In this figure, we see that the green points (permuted data according to Algorithm 1) are much more likely to elicit positive results from DE-SWAN than the orange points (permuted data according to Algorithm 2).

The validity of permutation tests emerges from the assumption of exchangeability (7). Under the null, it is common to assume that subject-level data are exchangeable. However, after applying LOESS to the data, points along the LOESS curve are not exchangeable with respect to age. Valid permutation testing requires that the data are shuffled at the level of exchangeability under the null hypothesis, which will be at the level of the transcriptomic matrix $X$, not the LOESS-interpolated values $\tilde{X}$. Therefore, Algorithm 1 is the appropriate permutation-based approach for assessing the validity of the results shown in Figure 6A. The results in Figure 4(d) of Shen et al. (24) are not adequately supported by the original permutation analysis and strongly influenced by noise captured and amplified by LOESS.

Algorithm 2 Permutation testing for LOESS+DE-SWAN: Shuffle after LOESS

$$\begin{array}{cc}\hfill & \mathrm{Input}:\text{Subject-level transcriptomic data}X\text{and vector of ages}a.\hfill \\ \hfill & \mathrm{Output}:\text{Number of significant genes at each time}t.\hfill \\ \hfill & \text{Run LOESS for each gene using}(X,a)\text{to obtain smoothed expression}\tilde{X}.\hfill \\ \hfill & \text{For each permutation}i=1,\dots ,N:\hfill \\ \hfill & 1.\text{Permute ages:}{\tilde{a}}_i\leftarrow \text{permute}(a).\hfill \\ \hfill & 2.\text{Run DE-SWAN on}(\tilde{X},{\tilde{a}}_i)\text{to identify differential-expression time points for each gene}.\hfill \\ \hfill & 3.\text{For each age}t,\text{count the number of genes with Benjamini}-\text{Hochberg-adjusted}p<0.05.\hfill \end{array}$$

4. Artifact 3: DE-SWAN identifies non-linear patterns of aging in linear data due to age-associated differences in statistical power

DE-SWAN identifies age-specific molecular changes by counting the number of significantly changing molecules at each of a sequence of time points along the age continuum (Figure 1). This count of molecules at each age point is influenced by both the true number of molecules whose expression is changing and the statistical power to detect them from the observed data. It is well-established that power is influenced by several characteristics of the data: the sample size, the significance level ($\alpha$), the noisiness of the data, and the magnitude of the true effect (23). To interpret a DE-SWAN curve as a faithful reflection of age-related molecular changes, we must assume that statistical power depends only on the magnitude of the true effect, which is unlikely if the other factors vary with age.

In practice, such variation is commonplace in aging cohorts. We illustrate this issue in a simulation study in which molecular expression changes linearly with age, so the magnitude of the true effect is constant. We show that non-uniform age distributions (Section 4.1.1), age-dependent heteroskedasticity (Section 4.1.2), and outliers (Supplementary Section S2.3.3) can produce DE-SWAN results that indicate non-linear patterns, even though the true change is linear. Our analyses call into question published conclusions about “waves of aging” and non-linearity of the aging process that are based on the DE-SWAN algorithm.

4.1. Simulations

We simulate the true molecular expression as linearly changing with age. Therefore, any nonlinearity shown in DE-SWAN curves does not result from true nonlinear expression, but rather some other aspect of the data. Specifically, we simulate the expression Y_ij for gene j in subject i by sampling from the model:

$$Y_{i,j}={\alpha }_j+{\beta }_j\times {\text{age}}_i+{\epsilon }_{ij}.$$ (1)

Mechanisms for specifying ${\alpha }_j$, ${\beta }_j$, ${\text{age}}_i$, and ${\epsilon }_{ij}$ vary among simulations (Supplementary Section S2.3).

4.1.1. Sample age distribution

When the DE-SWAN algorithm is applied to a non-uniform age distribution, some age ranges will have more observations than others as the sliding window traverses the age distribution. This change in the number of samples per window results in variation in the power to detect changing molecules at each age. This is illustrated in Figure 8, which shows that different age distributions generate profoundly different DE-SWAN results on simulated linear data. For each distribution and simulation, the data are simulated according to the same procedure (see S2.3). Therefore, the differences in these DE-SWAN results are attributable solely to the differences in age distribution.

Figure 8:

Effects of modulating age distribution on DE-SWAN results. Each dataset was simulated for 1000 observations and 2000 molecules. Results from 100 simulations are shown. The first column shows six randomly drawn molecular expression profiles from a single simulation for each of the population age distributions. The second column shows DE-SWAN curves plotted for 100 simulations for uniform, Gaussian, and two specifications of a bimodal (Gaussian) distribution. The mean of the curves is plotted in blue. The third column shows the mean DE-SWAN curve plotted with a ±1 SD band (solid blue). The population age distributions (from which each simulation’s age distribution is drawn) are plotted in the blue dashed line. The last column shows the same mean DE-SWAN curve plotted against the expected number of samples per group (older vs. younger) for the group with the smallest number of samples for each slide of the window. Simulation details can be found in Supplementary Section S2.3.

One tempting solution to this problem is to use quantile-based window centers to try to minimize variation in age group size; indeed, this is the default setting for the window parameter in the DE-SWAN R package (14). However, this does not ensure that all statistical tests made along the age distribution have the same number of samples in each group. Because a constant window length is still used, the endpoints of the window do not necessarily coincide with the selected quantiles. The choice of the window length and the midpoint locations dictate which samples will be included in each statistical comparison between younger and older groups, and quantile-based window centers alone will not guarantee that each group contains the same number of subjects.

Even if techniques such as quantile-based window centers, variable window lengths, or resampling methods are used to ensure that each window has the same number of samples, additional mechanisms related to the age distribution can impact power. We also observe that the intra-window sample density impacts statistical power (see Figure 9). For all three panels, the linear relationship between age and expression is the same. However, when samples are closer to the center of the window (middle panel), the difference between the younger and older groups is smaller than if the samples are located near the boundaries of the window (last panel). In this way, the distribution of the samples within the sliding window impacts the effect size of the difference between the two groups. When samples are located near the boundaries of the window, there will be higher statistical power to detect this change (p = 2.3 × 10⁻⁵⁶) than when samples are located at the center of the distribution (p = 1.3 × 10⁻¹⁷). This may also help explain the middle inflection point for the bimodal (20-yr) distribution’s DE-SWAN curves in Figure 8. At the location of the middle peak, the samples are densest at the boundaries of the window, inducing a larger effect size.

Figure 9:

Effects of intra-window age distribution on DE-SWAN results for one window centered at 50. All three panels show data generated for one simulated molecule: $Y_i=0.5+0.25\times {\text{age}}_i+{\epsilon }_i$ where ${\epsilon }_i\stackrel{\text{iid}}{\sim }𝒩(0,{0.85}^2)$. For each panel, there are 100 observations with 50 samples falling between the ages of 40 and 50 and 50 samples falling between the ages of 50 and 60. In the left panel, samples are evenly distributed throughout the window. In the middle panel, samples are concentrated at the center of the window. In the right panel, samples are concentrated at the boundaries of the window. The vertical black dashed line shows the window midpoint (50). The horizontal dotted black and gray lines show the means of the younger and older groups (respectively). The black line with arrows shows the difference between these two means. Two different statistical tests were used to evaluate the evidence of differential expression between the younger and older groups: the regression-based test ($Y_i=\alpha +\beta \times 1({\text{Age}}_i\ge 50)$) (15) or the Wilcoxon rank-based test (24). P-values for both of these tests are shown in the top left of each panel.

4.1.2. Heteroskedasticity

Heteroskedasticity refers to the presence of non-constant variance along the range of a predictor—in this case, age—and can be problematic for many statistical analyses (6; 8). It is biologically feasible that age can drive heteroskedasticity in molecular expression; in fact, the existence of this phenomenon has been reported previously (12). The presence of non-constant variance in molecular expression data has important implications for DE-SWAN. As variance changes along the age distribution, so does the statistical power to detect molecular change, yielding nonlinearity in the DE-SWAN curves.

We simulated various types of heteroskedasticity in linearly changing molecules to assess the effect of heteroskedasticity on DE-SWAN (Figure 10). In each case, data are generated following the linear model in Equation 1, with the variance of ${\epsilon }_{ij}$ depending on ${\text{age}}_i$. For some molecules, expression might be more tightly or less tightly regulated at the center of the age distribution (Figure 10, first and second rows). For others, expression in older ages might be more variable (Figure 10; third row). Alternatively, some molecules might exhibit more variable expression in younger ages (Figure 10; fourth row). These different trends in variance along the age distribution result in different DE-SWAN curve shapes. Moreover, it is likely that different molecules will exhibit different variability trends. The overall shape of the DE-SWAN curves will be impacted by the relative contributions of molecules exhibiting a range of patterns such as the ones in Figure 10.

Figure 10:

Effects of non-constant noise on DE-SWAN results. Each dataset was simulated for 1000 observations and 2000 molecules. Results from 100 simulations are shown. The first column shows six randomly drawn molecular expression profiles for one simulation for each variance trend. The second column shows DE-SWAN curves plotted for 100 simulations, with the mean curve plotted in blue. The third column shows the mean DE-SWAN curve plotted with a ±1 SD band (solid blue). The population standard deviations are plotted as blue dashed lines. Simulation details can be found in Supplementary Section S2.3.2.

5. Discussion and recommendations for future work

In this manuscript, we presented critical flaws of three techniques that have been applied in many high-impact aging studies: LOESS, DE-SWAN, and the combination of the two (LOESS+DE-SWAN).

We find that clustering the output of LOESS on simulated null data produces clusters of molecules with trajectories similar to those reported in the aging literature. Such trajectories have been interpreted as indicating biologically meaningful properties; however, our analyses caution against such interpretations since similar trajectories can appear when there is no true association with age.

The LOESS+DE-SWAN pipeline can indicate seemingly strong nonlinear signatures of molecular expression, however, when we rigorously test these findings using null simulations and permutation tests, we cannot confidently attribute these signatures to true biological change. Instead, we find evidence that applying DE-SWAN to smoothed data artificially boosts significance, and that the patterns that emerge appear to be driven by stochastic properties of the data, small numbers of samples, and the underlying age distribution.

We found that DE-SWAN can produce a range of non-linear curves on simulated null data that actually have a linear relationship with age, due to variation in power across age windows. The shape of these curves is strongly affected by non-uniform age distributions, heteroskedasticity, and clusters of outliers, all of which can occur in multi-omic datasets. Although we cannot rule out the possibility that waves of aging may exist in some populations or affect some biomolecules, without accounting for varying statistical power versus age, the shape of a DE-SWAN curve is difficult to interpret. Nonetheless, our simulation studies using random null data and various simulated and experimental age distributions clearly demonstrate that differences in power can induce artifactual waves of aging in DE-SWAN curves when there is no basis for these patterns in the data.

This manuscript has been partially motivated by the rapid proliferation of papers that use LOESS, DE-SWAN, or related analytical methods to identify age-related patterns in molecular profiles (see, for example, Ding et al. (4); Eduthan et al. (5); Lai et al. (13); Lehallier et al. (15); Liu et al. (16, 17, 18); Shen et al. (24, 25); Yin et al. (29); You et al. (30)). As of June 21, 2026, a Google Scholar query using the search term “(loess OR de-swan OR deswan) AND aging lehallier” identified more than 100 papers published between 2019 and 2026 that may use these methods in the analysis of aging-related data (Supplementary Section S1). Fully analyzing such a large number of papers is beyond the scope of this study and would require access to both data and the software used by the authors. Nevertheless, our results presented here demonstrate that when analyzing random noise with “population distributions” similar to those appearing in many aging studies, using LOESS, DE-SWAN, trajectory clustering methods, and combinations thereof, identify patterns similar to those reported when applied to real data. This is most easily seen when the distribution across age-based bins is nonuniform and at the “edges” of the distributions.

We recognize that disentangling results arising from methods that do not adequately discriminate between differential statistical power and true molecular change is a difficult task. Statistical power is an elusive quantity that can be influenced by many aspects of the data and the methods that are used in its analysis. But without a reliable estimate of statistical power along the population distributions, apparent patterns identified in any analysis are difficult to interpret with confidence. Consequently, we recognize a pressing need for best practices and new methods for analyzing molecular aging data.

5.1. Methodology recommendations

There is a great interest in understanding how and why we age, particularly as an increasing number of people are living longer (26). Caring for our aging population requires a deeper understanding of the biological processes that characterize aging and the effects of age on health and wellness. Based on our findings, we offer the following methodological recommendations to maintain the integrity of this research.

Critically evaluate the use of LOESS in aging research.

LOESS is a powerful tool that can be used to better visualize and understand trends in data; the “bin—summarize—smooth” application of the method has been presented as a useful way to visualize trends in large, complex data (Wickham). However, as with other data visualization techniques, there are important limitations to consider when using LOESS. In particular, LOESS requires dense datasets to produce reliable results. This affects not only how we interpret LOESS trends from study to study, but also how we interpret LOESS trends interpolated over a sample distribution. Typically, data are more sparse at the extremes of the predictor distribution. Consequently, with fixed-width local regressions, LOESS curves will be less reliable near the boundaries or in other areas where the population is particularly sparse due to the variation in data density (19). Further, LOESS curves can also be sensitive to outliers in the data (21).

These issues are exacerbated when LOESS curves are used as an input for downstream analyses. Approaches that treat every point along the fitted curve as equally certain may identify apparent changes near the tails of the predictor (age) distribution, even though these patterns are artifacts of overfitting in regions with sparse data. Moreover, LOESS-interpolated values are derived estimates, not independent observations, and should not be analyzed as though they were measured data. Such treatment violates the independence assumptions of most standard statistical tests and can produce spurious statistical significance and misleading biological conclusions.

Most commonly used statistical tests assume that data are independent. Consequently, statistical tests on LOESS outputs can be invalid unless the statistical test accounts for the relationships in the data imposed by LOESS. Furthermore, because LOESS imposes structure on the observed data, statistical tests that are intended to assess the strength of structure in the data will almost always find it in LOESS-interpolated values, resulting in spurious findings.

When used properly, LOESS can be a valuable tool for communicating trends in dense data. To that end, we recommend that LOESS not be used with sparse data where outliers in age bins can induce artifactual trends. Further, analysis should not be conducted on LOESS-interpolated data unless the uncertainty in local LOESS estimates—including their induced correlation structure—is propagated throughout the analytical pipeline. For visualizing trends, LOESS plots should also include some measure of uncertainty, such as 95% confidence intervals. Communicating the level of uncertainty around LOESS estimates will help readers understand how the characteristics of the underlying data may impact the strength of any conclusions drawn.

Employ appropriate hypothesis testing procedures.

To support the claim that a result is significant, we need a null hypothesis that envisions what the results would look like absent the phenomenon of interest. Deviations between the observed results and those obtained under the null hypothesis support the conclusion that the phenomenon causes these differences. Permutation testing, which holds the age distribution fixed while randomly shuffling age labels between samples, is a powerful method for empirically constructing a null distribution from the observed data. As discussed in Section 3.3, permutation tests such as these construct a null that breaks the dependence between exposure and outcome (in the studies here, age and expression). A test statistic must be selected that captures features of the observed data we would like to compare to the data generated under the null. Many replicate shuffled data sets are generated, the test statistic is computed for each, and the empirical distribution of these test statistics is then our null distribution. The observed test statistic is compared to this null distribution to compute a p-value.

A key assumption of this approach is that data shuffling must be done at the level of exchangeability implied by the null hypothesis and study design. Exchangeability is defined as invariance in distribution under finite permutations (11). In this context, for observations to be exchangeable under the null hypothesis, permuting the exposure (age) labels must leave the joint distribution of the data unchanged. In other words, the probability distribution of the entire dataset, including the relationships among observations, should be unaffected by shuffling the age labels if the null hypothesis is true and there is no relationship between exposure and outcome (age and expression). Processing data with tools such as LOESS, where each new data point is informed by other data points, induces an index-dependent structure in the data. Therefore, shuffling after such processing will violate the exchangeability assumption and yield an invalid null distribution.

Investigate methods’ sensitivity in null simulations before use in real data.

Throughout, we have used arguments from simulated null datasets to investigate the behavior of LOESS, DE-SWAN, and LOESS+DE-SWAN; these simulations have demonstrated the key vulnerabilities of these tools. In aging research, statistical inference on trends across the life course is likely to be hampered by data artifacts such as imbalances in the number of samples available at each age, the overall age distribution, uneven noise, and outliers, among other factors. Before using a method, it is important to assess its sensitivity to factors that can induce artifacts in the findings. While it is always valuable to have tools that detect real signals, it is also important to characterize the circumstances under which they can detect false signals. Simulations in null datasets and exploratory data analysis are both valuable for ensuring that analyses are robust and reliable.

Refocus methods to quantify effect sizes with proper uncertainty propagation.

Broadly, statisticians recommend using estimated effect sizes with corresponding measures of uncertainty (such as estimated odds ratios with confidence intervals) over measures of significance (p -values) to analyze data. Reasons for this are varied, although many of them have been discussed here (28). What becomes challenging about this approach is that it requires the careful propagation of uncertainty in the estimated effect sizes. Doubts about how to best propagate uncertainty are likely what has driven the development of methods based on p-values and other related measures of significance. While methods that analyze effect sizes rather than significance are more challenging to develop and implement, such methods will be more robust to many of the structural problems discussed here.

5.2. Concluding remarks

There are large methodological gaps in the study of aging. Several technical gaps have been discussed here. However, even after these problems have been addressed, there are still opportunities for broader methodological improvement. One is that current methods cannot fully leverage longitudinal data. In longitudinal datasets, we have the benefit of analyzing multiple samples from the same subject as they age. In contrast, cross-sectional datasets are static and provide less information about how time affects the variable being measured (such as biomolecule expression) (1). Because aging is a dynamic process, longitudinal data provide a distinct advantage over cross-sectional data. Appropriately controlling for within-subject correlations can be challenging at the scale of large multi-omic datasets. Another methodological consideration is how to adjust for between-molecule correlation when summarizing changes across the life course in such studies. For example, in the context of transcriptomic data, genes that are co-expressed often share some function (20). However, in most analyses, all genes are generally treated as being independent. Although this could be useful for some scientific questions, it is also important to understand whether the “waves” of aging reflect an increase in the underlying biological activity or whether the crests simply align with biological processes that are represented by many measured molecules in the assay. Adjusting for correlation among the measured molecules might help address this question.

There are many published studies that have used LOESS and/or DE-SWAN to study molecular or other changes that occur during the aging process (see, for example, Ding et al. (4); Eduthan et al. (5); Lai et al. (13); Lehallier et al. (15); Liu et al. (16, 17, 18); Shen et al. (24, 25); Yin et al. (29); You et al. (30)). The analysis presented here has not considered each of these studies in detail and, therefore, cannot be taken as a direct refutation of their findings. Our results, however, do cast doubt on their conclusions. Further, although nothing in our analysis disproves the existence of nonlinearities in the aging process, we have shown that patterns similar to those reported can emerge from random noise; we believe this plainly demonstrates that some of the most commonly used tools in molecular aging research cannot reliably identify molecular shifts throughout the aging process. Clearly, developing new ways to overcome the methodological limitations we have identified and establishing best practices will be crucial for characterizing the molecular factors that contribute to the biological process of aging.