Exploring how deep learning decodes anomalous diffusion via Grad-CAM

Abstract

While deep learning has been successfully applied to the data-driven classification of anomalous diffusion mechanisms, how the algorithm achieves the feat remains a mystery. Using a well-known technique aimed at achieving explainable AI, namely the Gradient-weighted Class Activation Map (Grad-CAM), we investigate how deep learning (implemented by ResNets) recognizes the distinctive features of a particular anomalous diffusion model from the raw trajectory data. Our results show that Grad-CAM reveals the portions of the trajectory that hold crucial information about the underlying mechanism of anomalous diffusion, which can be utilized to enhance the robustness of the trained classifier against the measurement noise. Moreover, we observe that deep learning distills unique statistical characteristics of different diffusion mechanisms at various spatiotemporal scales, with larger-scale (smaller-scale) features identified at higher (lower) layers.

Using Grad-CAM with ResNets, this study probes how deep learning classifies anomalous diffusion from raw trajectories. The method reveals trajectory segments and multiscale features driving predictions, improving interpretability and robustness to measurement noise.

Introduction

As our ability to generate, store, and analyze large datasets has dramatically increased, data analysis has become an essential component of modern science, including physics¹. Recent developments of deep learning have revolutionized the way we extract information from the data, allowing us to recognize meaningful patterns from complex, unprocessed empirical data². Besides applications to speech recognition³, computer vision⁴, self-driving cars⁵, and natural language processing⁶, deep learning has demonstrated remarkable efficacy for analyzing various physical data, such as tracking specific events in complex environments⁷, extracting signals from astronomical data⁸, and predicting valid protein configurations⁹.

More recently, deep learning was also applied to analyze anomalous diffusion (AnDi)^10–13. The phenomenon, characterized by nonlinear growth in time of the mean square displacement, is observed in diverse disciplines encompassing biology^14–16, social science¹⁷, and finance¹⁸. Various mathematical models have been proposed to describe the mechanism of AnDi, including continuous-time random walk (CTRW)¹⁹, fractional Brownian motion (FBM)²⁰, Lévy walk (LW)²¹, annealed transient time motion (ATTM)²², and scaled Brownian motion (SBM)²³. However, identifying the correct mechanism underlying a given trajectory is a challenging task, as long-range temporal correlations often make it difficult to identify useful statistical features²⁴. Despite various statistical features proposed for direct identification of the diffusion mechanism^25–29 and for preprocessing data for machine learning^30–32, a consensus on this issue is yet to be reached. To address this long-standing problem, the recent AnDi challenge applied a variety of machine learning techniques to the task³³, confirming the enhanced accuracy and efficiency of deep-learning approaches (for algorithms that excelled in the classification of two-dimensional trajectories, see refs. ^34–38) compared to the conventional statistical methods.

However, the black-box nature of deep learning poses a significant challenge as to how to interpret its outcomes—it is unclear which features of the data are used by the algorithm to perform the given task. Notably, a recent study³⁹ employed Bayesian learning techniques to provide error estimates of the outcomes, whose behaviors as the training data are varied indicate how properties of the underlying diffusion mechanism affect the learning performance. However, this approach does not explicitly reveal which features of the data lead to the observed outcomes.

In this study, we propose a method to highlight regions within the input trajectory that are key to how deep learning classifies the diffusion mechanisms. Similar problems have been addressed in different contexts, especially computer vision, using the techniques aiming to achieve explainable AI^40–53. Among these, noting that many of the deep learning methods in the AnDi challenge were based on convolutional neural networks (CNNs), we employ the gradient-weighted class activation mapping (Grad-CAM) developed for the architecture. The versatility of Grad-CAM has been demonstrated in various problems, which is now widely accepted as a standard technique of explainable AI^48–53. By integrating Grad-CAM with the diffusion model classifier based on deep learning, we aim to identify substructures within trajectories that contain key information about the underlying mechanism, which can be further applied to enhancing the robustness of the learning performance.

Results

Before proceeding, let us clarify what we aim to accomplish with deep learning. The task is to identify the AnDi mechanism underlying a two-dimensional particle trajectory. The trajectory is generated using one of the five standard models of AnDi: ATTM, CTRW, FBM, LW, and SBM. While the original AnDi challenge focused on classifying trajectories into these five categories, we also require the neural network to distinguish between subdiffusive and superdiffusive trajectories. By adding the prefixes “Sub-” and “Sup-” to indicate subdiffusion and superdiffusion, respectively, our classification task involves seven distinct mechanisms of AnDi: SubATTM, SubCTRW, SubFBM, SubSBM, SupFBM, SupLW, and SupSBM. Additionally, we include ordinary Brownian motion (BM) as the eighth mechanism.

In this study, we use the CNN-based architecture of neural networks to perform the task. This is due to the following two reasons. First, they provide a natural way to detect structures of trajectories at various time scales by stacking convolutional layers. Second, they allow application of Grad-CAM in its native setting and straightforward visualization of which subtrajectories are critical for the classification task. Previous works also support the viability of CNN-based architectures for identifying diffusion mechanisms^11,54,55. With these considerations in mind, we introduce ResAnDi, a deep learning algorithm designed to identify the AnDi mechanism underlying an empirical trajectory. This algorithm utilizes a neural network architecture based on the residual neural network (ResNet)⁵⁶. ResNet enhances the standard CNN by incorporating skip connections between layers, allowing the use of deeper networks while mitigating the vanishing gradient problem. Specifically, our model is based on ResNet18, which consists of one convolutional layer, four convolutional blocks (4 convolutional layers with skip connections in each block), and one fully-connected (FC) layer. While ResNet was originally developed for processing RGB images through three channels of two-dimensional arrays, we have modified the architecture to handle time series data of two-dimensional particle trajectories using two channels of one-dimensional arrays. In the end, the network produces a vector with components representing the probabilities that the trajectory belongs to each of the eight mechanisms. See Fig. 1 for a schematic illustration.

Fig. 1. Schematic illustration of model classification via ResAnDi and evaluation of the Grad-CAM score.

Processing a time series representing a two-dimensional particle trajectory via 18 layers, ResAnDi yields a vector whose components indicate the probabilities that the trajectory belongs to each of the eight classes of diffusion mechanisms described in the main text. Moreover, by calculating which nodes of the last convolutional layer contribute more to correct classification, the Grad-CAM score is assigned to each subinterval of the trajectory.

See the “Methods” section for more details regarding trajectory generation and training method.

Relevance of Grad-CAM

Previous studies have shown that Grad-CAM effectively highlights the most relevant features of an image that contribute to its correct classification^48,49. We show that the same holds for the identification of diffusion mechanisms. For this purpose, we employ two different approaches. First, we show that the accuracy of ResAnDi is more adversely affected by targeted erasure of particle trajectories with a higher Grad-CAM score. Second, we show that the accuracy of ResAnDi is more robust against noisy input if the training data are augmented using trajectories with a higher Grad-CAM score.

Grad-CAM

For completeness, we first give a brief description of Grad-CAM. The method was developed with the architecture of CNNs in mind. It focuses on the last convolutional layer, which is expected to possess the highest-level of semantic information while retaining some spatial information^49,57,58. Within this layer, let $A_i^k$ denote the activation of node i of the k-th feature map A^k. If the probability of the CNN correctly classifying the input data is p, then the influence of the k-th feature map on the decision can be quantified as

$$a^k\equiv \frac{1}{\mid {\mathbf{A}}^k\mid }{\sum }_i\frac{\partial p}{\partial A_i^k},$$ 1

where ∣A^k∣ is the size of the k-th feature map.

The Grad-CAM scores G are obtained by averaging the activations of all feature maps, where each feature map is weighted by its influence on the correct classification. This is expressed by the formula

$$G_i\equiv {\sum }_k{a}^k{A}_i^k.$$ 2

Given a trained CNN and an input sample, G can be obtained readily using the standard backpropagation method. In this study, we used the Captum Python package⁵⁹ to implement the calculation.

It should be noted that the Grad-CAM scores defined above are assigned to the nodes in the last convolutional layer, whose activation pattern is a coarse-grained representation of the original input. To reassign these scores to the nodes of the input data, an interpolation scheme is needed. For efficiency, we partition the input nodes into (approximately) equal-length subintervals, with the number of subintervals matching the number of nodes in the final convolutional layer. This creates a one-to-one correspondence between the nodes in the final convolutional layer and the subintervals in the input layer. Through this correspondence, the Grad-CAM score is transferred from the final convolutional layer to the input trajectory.

Targeted erasure based on Grad-CAM

To check whether Grad-CAM captures the relevant features of trajectories, we propose the following test inspired by the occlusion-based perturbation, which was used to test the reliability of Grad-CAM in computer vision⁴⁹. First, we prepare a new set of trajectories not used in the training, whose temporal duration varies from 10 to 1000. Given this test dataset, targeted erasure is implemented as follows. We start with partitioning each trajectory into subintervals so that the Grad-CAM score can be assigned to each of them. By letting ResAnDi classify the trajectory, we obtain the Grad-CAM score of each subinterval. This allows us to modify a trajectory by “erasing” all subintervals whose Grad-CAM score falls within a targeted range. More specifically, erasing a subinterval means that the particle is forced to stay at the origin (x = y = 0) during the whole subinterval. By comparing the effects of targeted erasure with those of random erasure, where randomly chosen subintervals are erased, we can assess how the Grad-CAM score relates to the presence of useful information about diffusion mechanisms. See Fig. 2 for illustrations of targeted and random erasures.

Fig. 2. Examples of particle trajectories modified by two different erasure schemes.

Subintervals of trajectories are erased (top) by targeting the top 10% of the Grad-CAM score (indicated by the color scale) or (bottom) by random choice. Note that the x and the y coordinates are separately shown.

In Fig. 3, we show how the classification accuracy of ResAnDi is affected by targeted erasure of subintervals whose Grad-CAM scores fall between each consecutive pair of deciles (i.e., every tenth percentile). Clearly, erasing subintervals of the higher Grad-CAM score leads to lower accuracy of ResAnDi. Compared to the random erasure of subintervals, erasing subintervals whose Grad-CAM score belongs to the upper 70% has more adverse effects on the classifier. These suggest that certain parts of the particle trajectories contain more information about the underlying diffusion mechanism than others, and that the Grad-CAM score can be used as an indicator of those crucial parts. One may question how the information contents of the trajectory can be inhomogeneous when the generation mechanism itself is homogeneous in time. Even if the mechanism involves homogeneous dynamics, the generated trajectory exhibits inhomogeneities coming from rare fluctuations. That said, subtrajectories representing characteristic rare fluctuations (such as those resembling persistent biased random walks exhibited by SupFBM, see Supplementary Fig. S5) can help the algorithm recognize the underlying generation mechanism, whereby they receive high Grad-CAM scores. In this sense, some subtrajectories can be more informative than others.

Fig. 3. Effects of targeted erasure on the classification accuracy of ResAnDi.

The accuracy is measured after erasing subintervals corresponding to each decile of the Grad-CAM score. Removing subintervals with a higher Grad-CAM score results in lower accuracy. For comparison, the effect of random erasure is also shown by a dashed line.

Dataset augmentation via Grad-CAM

Dataset augmentation aims to enhance machine learning performance by expanding the diversity of training data. This typically involves making variants of certain samples through random cropping, resizing, or rotating, which modifies non-critical aspects of the data while preserving the features crucial for classification tasks⁶⁰. However, these conventional approaches do not evaluate the usefulness of individual samples for training, generating an augmented dataset only through random selection. This prompts the question of whether the procedure could be improved by making informed decisions about which samples to augment.

We hypothesize that if a trajectory has a higher mean value of the Grad-CAM score, it has a higher density of information useful for discerning the underlying diffusion mechanism, which makes it a better candidate for the augmented dataset. To test this, we propose targeted augmentation, which is implemented as follows. First, we select trajectories that ranked in the top 60% based on their mean Grad-CAM scores, calculated by averaging the scores across subintervals of each trajectory. Then, these selected trajectories are rotated by a random angle and added to the original training dataset to form the augmented dataset. See Fig. 4 for a schematic illustration of targeted augmentation as opposed to the conventional random augmentation. Additional details on this procedure can be found in the Supplementary Information Sec. B.

Fig. 4. Schematic illustrations of dataset augmentation method.

(Top) Using targeted augmentation, trajectories with high a mean Grad-CAM score are rotated by random angles to build the augmented dataset. (Bottom) Using random augmentation, the trajectories to be rotated are chosen at random.

The advantage of targeted augmentation over random counterpart becomes evident in the presence of noise in the test trajectory data, which stems from the measurement error present in the experimental data. We simulate this effect by generating the test dataset from the models and then applying Gaussian noise to every point of the trajectories. Since every trajectory has been rescaled so that the standard deviation of particle displacement per time step is unity, the amplitude of the Gaussian noise can be regarded as the inverse signal-to-noise ratio (SNR). In Fig. 5, we show that targeted augmentation consistently achieves more robust performance than random augmentation for a broad range of 1/SNR, with the maximum enhancement ( ≈ 4%pt) exceeding the error range. This demonstrates that Grad-CAM indeed captures the features that contain information about the underlying diffusion mechanism, which can be utilized to mitigate the adverse effects of data contamination by erasure or noise.

Fig. 5. Effects of noise in the unseen test data on the classification accuracy.

Targeted augmentation using trajectories belonging to the top 60% of the mean Grad-CAM score exhibits more robust performance against increased inverse signal-to-noise ratio (SNR). The statistics are obtained from 5 models trained using each augmentation scheme, with the standard errors indicated by shaded regions. (Inset) Enhanced accuracy due to the use of targeted augmentation.

Features indicated by Grad-CAM

We have demonstrated that Grad-CAM, as it does for image classification tasks, reliably indicates the relative importance of different subtrajectories in identifying the diffusion mechanism. Now we address the question of which characteristics of the mechanisms are captured by the Grad-CAM score. We take a two-step approach. First, by visualizing how ResAnDi architecture encodes trajectories generated by different mechanisms at various depths of the network, we construct a set of statistical features which might be useful for the classification task. Second, we calculate the correlations between these features and the Grad-CAM score, quantifying the extent to which Grad-CAM highlights those characteristics. For the ease of statistical analysis, through this section, we use a newly trained ResAnDi, which was trained on a noiseless dataset comprising 8 × 10⁴ × 5 trajectories of temporal duration fixed at 1000. In this case, ResAnDi achieved the classification accuracy of 94.89%, which is higher than 90.36% achieved when the trajectory length was uniformly distributed between 10 and 1000. For the calculation of correlations, a test dataset of the same size is used.

Visualization of the classification process

As previously described, ResAnDi architecture comprises four convolutional blocks, which successively process the trajectory data to identify its underlying diffusion mechanism. For every input trajectory, each block yields a high-dimensional output vector. Since ResAnDi is trained to distinguish between trajectories generated by different mechanisms, the output vectors must be clustered so that trajectories of “similar” mechanisms are close to each other, while those from “dissimilar” mechanisms are farther apart. In shallower layers, trajectories are likely to be clustered according to local features. In deeper layers, the network focuses more upon longer-range features. Hence, examining clustering patterns across different layers can provide us with useful clues as to which statistical features are used by ResAnDi to classify trajectories.

In Fig. 6, we visualize how each convolutional block clusters the training dataset trajectories by embedding the output vectors in two dimensions via t-distributed stochastic neighbor embedding (t-SNE). In the output of Block 1, we can discern four mechanism clusters, namely SubFBM, [SupFBM, SupLW] (FBM/LW), [SubATTM, SubCTRW] (ATTM/CTRW), and [SubSBM, SupSBM, BM] (SBM/BM). Among these, SupFBM, SupLW, SubATTM, and SubCTRW are identified as individual mechanisms in the output of Block 2. Finally, the SBM/BM cluster is fully classified only after Block 3, indicating that long-range features are required to distinguish between these mechanisms. Based on these observations, we propose four statistics that may be utilized by ResAnDi for the classification task.

Fig. 6. Visualization of ResAnDi’s classification process.

The high-dimensional output of each of the four convolutional blocks is embedded into a two-dimensional space using t-SNE. Each scatter plot includes 400,000 trajectories of the training dataset, with every point corresponding to a single trajectory.

Statistics correlated with Grad-CAM

To facilitate further statistical analysis, we begin with partitioning each trajectory of the training dataset into subtrajectories, so that the Grad-CAM score can be assigned to each of them. However, in this case, we let the subtrajectories overlap with each other, so that each subtrajectory is long enough for reliable statistics. See Supplementary Information Sec. D for details.

Now, guided by Fig. 6, let us construct the statistics that would be relevant to the classification task. In the output of Block 1, the SubFBM stands out as a single clearly distinct mechanism. Noting that SubFBM is the only model producing negative correlations between consecutive particle displacements, it is natural to conjecture that Block 1 utilizes the Autocorrelation (AC) defined as

$$\mathrm{AC}\equiv \frac{1}{2}{\sum }_{r\in \left\{x,y\right\}}\frac{⟨\mathrm{\Delta }{r}_t\mathrm{\Delta }{r}_{t+1}⟩-{⟨\mathrm{\Delta }{r}_t⟩}^2}{⟨⟨\mathrm{\Delta }{r}_t^2⟩⟩}.$$ 3

Here Δr_t represents the displacement in the x or y direction during the time interval from t to t + 1, and 〈〈Xⁿ〉〉 denotes the nth cumulant of observable X over a chosen subtrajectory, with 〈X〉 ≡ 〈〈X〉〉.

With AC thus identified, it seems natural that SupFBM and SupLW should be clustered together (as FBM/LW cluster) by Block 1 since they share positive AC. But while SupLW tends to make the particle move in the same direction for a prolonged period of time, SupFBM results in frequent changes in the direction of motion. Thus, to distinguish these two mechanisms, we expect that Block 2 takes advantage of Consistency (CS) defined as

$$\mathrm{CS}\equiv \mathrm{AC}\times \overline{\sqrt{⟨⟨{\left(\mathrm{\Delta }{\theta }_t/\pi \right)}^2⟩⟩}},$$ 4

where

$$\mathrm{\Delta }{\theta }_t\equiv \arccos \frac{\mathrm{\Delta }{x}_{t-1}\mathrm{\Delta }{x}_t+\mathrm{\Delta }{y}_{t-1}\mathrm{\Delta }{y}_t}{\sqrt{\left(\mathrm{\Delta }{x}_{t-1}^2+\mathrm{\Delta }{y}_{t-1}^2\right)\left(\mathrm{\Delta }{x}_t^2+\mathrm{\Delta }{y}_t^2\right)}}$$ 5

is the change in the direction of motion occurring at time t, and the notation $\overline{X}$ indicates that the quantity X is min-max normalized to the interval [0, 1]. Clearly, high (low) CS corresponds to SupFBM (SupLW).

As for the SubATTM and the SBM/BM cluster, the component mechanisms are all characterized by locally Gaussian displacements. This motivates us to divide each subtrajectory into n subintervals and consider the Non-Gaussianity (NG) defined as

$$\mathrm{NG}\equiv \overline{∣\frac{1}{2n}{\sum }_{r\in \left\{x,y\right\}}{\sum }_{i=1}^n\frac{{⟨⟨\mathrm{\Delta }{r}_t^4⟩⟩}_i}{{⟨⟨\mathrm{\Delta }{r}_t^2⟩⟩}_i^2}∣},$$ 6

where 〈〈⋅〉〉_i denotes a cumulant calculated over the ith subinterval. Since the fourth cumulant vanishes for the Gaussian distribution, NG is greater if the subtrajectory deviates farther from the Gaussian statistics at the subinterval level.

Now, we can guess how SubATTM and SubCTRW are established as individual mechanisms after Block 2. Both are distinct from the SBM/BM cluster in that their trajectories feature sudden changes in the magnitude of displacements. They are also distinguished from each other by the locally Gaussian nature of SubATTM and the strong non-Gaussianity of SubCTRW. Thus, we are led to consider Singularity (SG) defined as

$$\mathrm{SG}\equiv \overline{{\max }_{r\in \left\{x,y\right\}}\left[{⟨⟨{\left(\frac{\mathrm{\Delta }{r}_{t+1}+ϵ}{\mathrm{\Delta }{r}_t+ϵ}\right)}^2⟩⟩}^{1/2}\right]}\times \mathrm{NG},$$ 7

where ϵ is a small positive number introduced to prevent the quantity from diverging. Throughout this study, we use ϵ = 10⁻⁶. We note that high (low) SG indicates SubCTRW (SubATTM), which features both higher (lower) probability of abrupt jumps and high (lower) NG.

Finally, there still remains the SBM/BM cluster, which persists up to Block 3. The three mechanisms are similar in that their fluctuations are locally Gaussian without any sudden changes in the magnitude of displacements. Their unique characteristics become apparent only when one observes how diffusivity gradually changes over time. As a measure of this property, we propose Varying Diffusivity (VD) defined as

$$\mathrm{VD}\equiv \frac{1}{2n}{\sum }_{r\in \left\{x,y\right\}}\frac{{⟨⟨\mathrm{\Delta }{r}_t^2⟩⟩}_n^{1/2}-{⟨⟨\mathrm{\Delta }{r}_t^2⟩⟩}_1^{1/2}}{{⟨⟨\mathrm{\Delta }{r}_t^2⟩⟩}^{1/2}}\times \mathrm{NG},$$ 8

which quantifies how diffusivity changes over each subtrajectory. We note that positive (negative) VD indicates SupSBM (SubSBM).

In Fig. 7, we show how the statistics constructed by the above procedure correlate with the Grad-CAM score of the subtrajectories obtained using the output of Block 4. To track how successive blocks characterize the subtrajectories differently, in Fig. 8, we repeat the same analysis for the output of each of the other three blocks. A couple of cautionary remarks are needed regarding this process. First, as indicated by the t-SNE results shown in Fig. 6, the outputs of Blocks 1 and 2 do not fully distinguish between some diffusion mechanisms. Thus, the Grad-CAM scores for these blocks should treat individual clusters of models shown in Fig. 6 as separate labels, not individual models. Second, as previously described, the Grad-CAM score is calculated using the output of the deepest convolution layer. This means that, for Blocks 1, 2, and 3, we need to remove all blocks located deeper than the block of interest, and then append a linear, FC layer to the block’s output. Then, we train only the weights of the FC layer so that its output distinguishes the clusters identified by the block of interest according to the t-SNE analysis in Fig. 6. In this way, the Grad-CAM scores can be assigned to the nodes in the output of the block of interest, which can in turn be reassigned to the subtrajectories of the input data. Additional details on this procedure can be found in the Supplementary Information Sec. B.

Fig. 7. Pearson correlation coefficients between the Grad-CAM score and the four key statistics.

The calculations are based on the Grad-CAM score of the final output. The thick vertical lines distinguish models belonging to different t-SNE clusters according to the output of Block 1 (see Fig. 6). Note that the Grad-CAM score and the statistics are assigned to every overlapping subtrajectories, as illustrated in the inset. Correlations are obtained using a test dataset consisting of 400,000 trajectories.

Fig. 8. Pearson correlation coefficients between the Grad-CAM score and the four key statistics.

The calculations are based on the Grad-CAM score of the output of a Block 1, b Block 2, and c Block 3. The thick vertical lines distinguish models belonging to different t-SNE clusters according to the output of Block 1 (see Fig. 6). Correlations are obtained using a test dataset consisting of 400,000 trajectories.

Keeping these in mind, the results of Figs. 7 and 8 can be interpreted as follows.

• In the final output (shown in Fig. 7), AC is negatively correlated with the Grad-CAM score of SubFBM. The reason is clear: SubFBM features negative AC throughout the trajectories. Meanwhile, AC exhibits positive correlation with the Grad-CAM scores of SupFBM and SupLW, and correlations with the other mechanisms are largely negligible. This trend stays the same starting from Block 1 (see Fig. 8a), which is consistent with how the block’s output already clearly separates the SubFBM and the FBM/LW clusters from the rest in Fig. 6.

• In the final output (shown in Fig. 7), CS exhibits significant correlations of opposite signs with the Grad-CAM scores of SupFBM and SupLW. The distinction between the positive correlation for SupFBM and the negative correlation for SupLW is already observable after Block 2 (see Fig. 8b), which becomes even more conspicuous in the outputs of deeper blocks (see Figs. 7 and 8c). This is consistent with our conjecture that Block 2 utilizes the statistic to distinguish between the two mechanisms that commonly feature positive AC. We note that, starting from Block 1 (see Fig. 8a), CS also exhibits negative correlation with the Grad-CAM score of SubFBM, which might have been inherited from the behaviors of AC.

• In the final output (shown in Fig. 7), SG and the Grad-CAM score exhibit a positive correlation for SubCTRW and a negative correlation for SubATTM, confirming that ResAnDi does capture this difference between the two models. When they become separable after Block 2, Fig. 8b, c show that the correlations differ only in magnitude and not in sign until after Block 3. This may reflect that SG, being a nonlocal measure, requires some depth of the network to be captured properly.

• In the final output (shown in Fig. 7), VD exhibits significant correlations of opposite signs with the Grad-CAM scores of SupSBM and SubSBM, suggesting that ResAnDi does capture this difference between the two models. We also note that the Grad-CAM score of SubCTRW exhibits a negative correlation with VD, which may stem from the strong non-Gaussianity of the SubCTRW trajectories. As shown in Fig. 8a–c, VD exhibits increasingly diverse behaviors for the deeper blocks, reflecting its nonlocal nature.

In the end, Fig. 7 shows that every diffusion mechanism exhibits a distinct correlation profile with the four statistics introduced above, except for SubATTM and BM, which are distinguishable only by the weak negative correlation shown by SG. Indeed, as shown in Fig. S2 of the Supplementary Information, ResAnDi finds it difficult to distinguish SubATTM from BM. These results demonstrate how the Grad-CAM scores can provide some insight into which statistical features are utilized by deep learning to decode the underlying diffusion mechanism of a trajectory.

Ablation study

Feature ablation by subtrajectory replacement

To further demonstrate the relevance of the four features identified by the Grad-CAM scores, we also conduct an ablation study, whereby we examine how the classification performance of ResAnDi changes after the key parts of input trajectories are modified to weaken a target feature. Toward this goal, we propose a subtrajectory replacement scheme, which is implemented using the noiseless dataset as follows.

First, we partition each input trajectory into subtrajectories, just as we did when calculating the Grad-CAM scores (see Supplementary Information Sec. D for details). Then we choose the subtrajectory with the highest (or lowest) value of the selected feature, which is to be replaced with a copy of another subtrajectory from the same trajectory. The replacement is to be done with appropriate position adjustments at the boundaries to preserve the trajectory continuity. The replacement subtrajectory is determined by minimizing the cost function

$$\mathcal{L}=-{\sum }_i{\left(s_i-s_i^{\prime }\right)}^2+{\sum }_r\left[{\sum }_i{\left(r_i-r_i^{\prime }\right)}^2\right],$$ 9

where s_i and $s^{\prime }$ are the values of the selected feature of the ith subtrajectory before and after the replacement, respectively, and r_i and $r^{\prime }$ stand for the other three features of the ith subtrajectory before and after the replacement, respectively. In other words, the replacement subtrajectory is chosen to maximize the impact on the target feature, while minimizing the effects on the other features.

To check how independently our method can perturb each feature, we calculate the ℓ²-norm of the changes of every feature across all subtrajectories after the perturbation of each selected feature, which is shown in Fig. 9 together with standard deviations. Larger ℓ²-norms are indicated with darker red (blue) shades when the subtrajectories with the highest (lowest) selected feature are replaced. To account for differences in the scale of features, all feature values were min-max scaled before calculating the ℓ²-norms. As shown by the figure, the diagonal entries stand out, indicating that our method succeeds in perturbing the intended feature most strongly, well beyond the error range. Still, there are notable side effects on the other features. In particular, changing one among AC, CS, and VD tends to affect the other two features, while changing SG affects all the other features. Such mutual dependence between features should be considered when interpreting the results of our ablation study.

Fig. 9. Independence between ablated features.

The heatmaps show the ℓ²-norms (with their standard deviations) of changes in each feature (rows) caused by ablation of a selected feature (columns). The upper (lower) table shows the results obtained when subtrajectories with the highest (lowest) feature values are replaced. To account for the scale differences, the values of the features are min-max scaled before calculating the ℓ²-norms.

Effects of ablation

Using the method described so far, we examine how perturbing one of the features affects the classification accuracy of trajectories generated by each model. In Fig. 10, we show the percentage of misclassification for each diffusion model (columns) resulting from the ablation of a selected feature (rows), together with the name of the model (shown within the parentheses) accounting for the largest fraction of misclassification results. The colors indicate whether we target subtrajectories with the highest (red) or the lowest (blue) value of the chosen feature, which are darker for features causing a relatively larger accuracy drop compared to the other features.

Fig. 10. Heatmap of misclassification percentage.

Each column corresponds to a diffusion model, and each row corresponds to an ablated feature. The name of the model accounting for the largest portion of the misclassification results is shown within the parentheses. Red (blue) cells correspond to the case where the subtrajectories with the highest (lowest) feature values are replaced. Darker shades are used for features that induce relatively larger accuracy drops.

A comparison between Figs. 7 and 10 reveals many common properties. The features most correlated with the Grad-CAM score in Fig. 7 tend to be the features whose perturbation leads to relatively large accuracy drops in Fig. 9 (AC for SubFBM, SupFBM, and SubCTRW, CS for SupLW, SG for SubCTRW, and VD for SubCTRW, SubSBM, and SupSBM). Moreover, when these features are perturbed, ResAnDi tends to confuse models belonging to the same t-SNE cluster (SupLW tends to be misclassified as SupFBM, SubCTRW as SubATTM, SubATTM as SubCTRW, and SubSBM and SupSBM as BM). The results corroborate our previous observation that ResAnDi uses CS to distinguish SupLW from SupFBM, SG to tell SubCTRW from SubATTM, and VD to distinguish BM, SubSBM, and SupSBM from each other in the deeper layers.

Still, there are two notable exceptions. First, even though the Grad-CAM scores of SupLW are not strongly correlated with SG, the model exhibits a relatively large accuracy drop (4.78%) when subtrajectories with the lowest SG are replaced, with SupFBM accounting for most (79.2%) of the misclassification results. This could be because the “straight” segments of SupLW trajectories, which consistently travel in the same direction, may have very low NG and thus very low SG as well. Replacing such subtrajectories with those having more variations in the travel directions can effectively increase CS, thus making the SupLW trajectories look more like those generated by SupFBM. Indeed, Fig. 9 confirms that perturbing SG also has significant effects on CS. Second, even though the Grad-CAM scores of SubATTM are only weakly correlated with SG, the model displays a relatively large accuracy drop (13.50%) when subtrajectories with the lowest VD are replaced. In this case, the misclassification results are almost evenly accounted for by three different models, namely SupFBM (28.2%), BM (27.6%), and SubCTRW (25.0%). As previously noted, the Grad-CAM scores of SubATTM are not strongly correlated with any of the four features, and ResAnDi does seem to have trouble distinguishing SubATTM from the other models (see Supplementary Information Fig. S2). Since perturbing VD has significant side effects of modifying AC and CS as well (as shown in Fig. 9), the resulting subtrajectories may accidentally attain “strong” features that make them look like SupFBM or SubCTRW trajectories. Moreover, since SubATTM fundamentally resembles BM when variations in the diffusivity happen to be small, it is not surprising that BM accounts for a significant portion of the misclassification results.

To sum up, the results of the ablation study are largely consistent with our interpretation of the Grad-CAM results, with notable exceptions attributable to some fundamental difficulties in independent feature perturbations.

Discussion

In this study, we utilized Grad-CAM, a technique developed for explainable AI, to highlight which parts of particle trajectories are crucial for the deep learning algorithm to identify the underlying diffusion mechanism. By observing how targeted erasure of trajectories based on Grad-CAM impairs the machine learning performance, we found that Grad-CAM indeed captures the informative parts of the trajectories, which can also be utilized to enhance the dataset augmentation method. Furthermore, by measuring correlations between the Grad-CAM score and trajectory statistics of varying nonlocality, we could elucidate the process through which deep learning differentiates between different diffusion mechanisms, step by step. The ablation study, which selectively weakens the target feature of input trajectories, also corroborates our picture.

Our results have twofold implications. First, they demonstrate that Grad-CAM provides a useful measure of which parts of the training dataset are more informative than the rest. This suggests that one may design an active learning algorithm that makes best use of the available trajectory dataset by incorporating Grad-CAM into the training procedure. For example, the methods presented in this study can easily be applied to the detection of change points, where one has to pinpoint when the diffusion mechanism changes, implemented by adapting the ResAnDi architecture⁶¹. Second, our results are consistent with the intuition that deep learning decodes the trajectory data by first focusing on local features and then gradually broadening the scope to nonlocal features. Checking whether the same is true for network architectures beyond CNNs and designing statistical inference methods inspired by such multiscale attention implemented by deep learning would be a worthwhile directions of future research.

Methods

Trajectory generation

Trajectories exhibiting AnDi are generated using the Python package provided by the AnDi challenge^33,62. A dataset, be it for training, validation, or testing, comprises an equal number of trajectories for each of the eight mechanisms. For models exhibiting subdiffusion, the diffusion exponents of the corresponding trajectories are uniformly distributed in the interval [0.1, 0.9]. Similarly, for models exhibiting superdiffusion, the diffusion exponents are uniformly distributed in the interval [1.1, 1.9]. Each trajectory is rescaled so that the displacement per unit time (Δt = 1) has unit variance. Unless specified otherwise, the temporal duration of each trajectory is uniformly distributed between 10 and 1000. Without making any explicit assumptions about the measurement error, we only use noiseless trajectories for the training datasets. So are the trajectories in the test datasets unless specified otherwise.

See the Supplementary Information Sec. A for more details regarding the definition of each AnDi model and the preparation of datasets.

Deep learning algorithm

To train ResAnDi, we use the PyTorch package⁶³ with the categorical cross-entropy loss function and the Adam optimizer⁶⁴, employing an early-stopping method to avoid overfitting. We tested the performance of ResAnDi using a dataset composed of 10⁴ trajectories belonging to each of the eight classes. ResAnDi achieves an overall classification accuracy of 90.36%, which is comparable to the best algorithm³⁴ submitted to the AnDi challenge³³ (which achieved 89.16%), even though our task involves a greater number of classes.

See the Supplementary Information for more details regarding the neural network architecture (Sec. B1), training procedure (Sec. B2–B5), and classification accuracy for each class (Sec. C).

Author contributions

J.B. designed the research, assembled input data, wrote code, ran the model, and analyzed output data; Y.B. and H.J. jointly supervised the research; all authors contributed to the writing of the manuscript.

Peer review

Peer review information

Nature Communications thanks Carlo Manzo and the other anonymous, reviewer(s) for their contribution to the peer review of this work. A peer review file is available.

Data availability

The data that support the plots within this paper and other findings of this study are available from the corresponding author upon reasonable request.

Code availability

The computer codes used to generate the results of this paper are available at the following URL/10.5281/zenodo.18836353.

Competing interests

The authors declare no competing interests.

Supplementary information

The online version contains supplementary material available at 10.1038/s41467-026-71229-x.