Interpretable Fine‐Grained Phenotypes of Subcellular Dynamics via Unsupervised Deep Learning

Abstract

Uncovering fine‐grained phenotypes of live cell dynamics is pivotal for a comprehensive understanding of the heterogeneity in healthy and diseased biological processes. However, this endeavor poses significant technical challenges for unsupervised machine learning, requiring the extraction of features that not only faithfully preserve this heterogeneity but also effectively discriminate between established biological states, all while remaining interpretable. To tackle these challenges, a self‐training deep learning framework designed for fine‐grained and interpretable phenotyping is presented. This framework incorporates an unsupervised teacher model with interpretable features to facilitate feature learning in a student deep neural network (DNN). Significantly, an autoencoder‐based regularizer is designed to encourage the student DNN to maximize the heterogeneity associated with molecular perturbations. This method enables the acquisition of features with enhanced discriminatory power, while simultaneously preserving the heterogeneity associated with molecular perturbations. This study successfully delineated fine‐grained phenotypes within the heterogeneous protrusion dynamics of migrating epithelial cells, revealing specific responses to pharmacological perturbations. Remarkably, this framework adeptly captured a concise set of highly interpretable features uniquely linked to these fine‐grained phenotypes, each corresponding to specific temporal intervals crucial for their manifestation. This unique capability establishes it as a valuable tool for investigating diverse cellular dynamics and their heterogeneity.

An unsupervised deep learning framework is developed to analyze live cell dynamics by combining an unsupervised teacher model with a student deep neural network. This method successfully delineates detailed subcellular protrusion phenotypes and their responses to drugs. This approach preserves cellular heterogeneity while improving feature discrimination and interpretation, making it a valuable tool for studying subcellular dynamics.

1. Introduction

Live cell imaging offers unique insights into dynamic cellular processes and their functional outcomes across various spatiotemporal scales.^{[ 1} , ² , ³ , ^{4 ]} Recent advancements in microscopy technology have enabled the acquisition of extensive live cell image data at high resolutions. However, conventional ensemble‐averaging approaches may overlook critical mechanistic details due to spatiotemporal heterogeneity in live cell data.^{[ 4} , ⁵ , ^{6 ]} This becomes particularly problematic when specific molecular perturbations induce responses in only a limited subcellular region of a rare cellular subpopulation, a phenomenon often missed without the deconvolution of heterogeneity. Consequently, effective data clustering is necessary to address their heterogeneous nature and accurately capture the diversity within subcellular and cellular populations.^{[ 7} , ^{8 ]} Although computational image analysis and machine learning have been extensively utilized for cellular heterogeneity analysis, the majority of these efforts have concentrated on static images.^{[ 9} , ¹⁰ , ^{11 ]} The investigation into phenotypic heterogeneity in live cell images is still in its nascent stages,^{[ 4} , ⁶ , ¹² , ¹³ , ¹⁴ , ^{15 ]}necessitating the extraction of informative features that accurately represent subcellular and cellular temporal dynamics for effective deconvolution of heterogeneous phenotypes.

With the advent of single‐cell RNA‐seq analysis, it has become increasingly clear that fine‐grained cell subpopulations within established cell types encapsulate the complexities of disease mechanisms. A prior investigation revealed that subpopulations within each major brain cell type are associated with Alzheimer's disease, involving abnormal myelination, inflammation, and neuron survival.^{[ 16 ]} Furthermore, novel phenotypes of epithelial cells were identified as significantly abundant in pulmonary fibrosis.^{[ 17 ]} Therefore, uncovering live cell fine‐grained phenotypes, referred to as deep phenotyping for brevity in this work,^{[ 18} , ^{19 ]} is pivotal for establishing the groundwork for a more profound comprehension of cellular dynamics in the context of disease mechanisms. Nonetheless, this endeavor has encountered considerable technical challenges, demanding the identification of subtle differences in cellular dynamics with limited prior knowledge.

In deep phenotyping, handling high‐dimensional datasets requires informative features that effectively reduce dimensionality while retaining crucial information, addressing challenges associated with the curse of dimensionality.^{[ 20} , ^{21 ]} Traditional feature engineering involves domain‐specific hand‐crafted designs, which may not be directly applicable to deep phenotyping. Deep learning has emerged as a promising alternative due to its exceptional ability to learn comprehensive representative features directly from large raw datasets.^{[ 22} , ^{23 ]} However, conventional methods for feature learning encounter challenges when addressing the complexities of deep phenotyping. Conventional neural network training, focusing on improving classification accuracy, often produces features that might not comprehensively capture the entire spectrum of intra‐class heterogeneity, particularly when dealing with limited data prone to overfitting. Conversely, autoencoders^{[ 24 ]} can identify extensive features across samples, but these features may lack the discriminatory power required for recognizing specific cellular or disease states.^{[ 25} , ^{26 ]} Additionally, they might include considerable irrelevant heterogeneity, compromising their effectiveness in unsupervised learning. Moreover, an ongoing challenge lies in making the learned features interpretable to domain experts.^{[ 27} , ^{28 ]} Interpreting these features within the context of cellular dynamics remains an active area of research, as it enables a deeper understanding of the underlying mechanisms.^{[ 29 ]} These limitations underscore the need for more advanced approaches in feature learning capable of preserving relevant heterogeneity and ensuring accurate discrimination while enhancing their interpretability.

To overcome these challenges, we developed DeepHACX (Deep phenotyping of Heterogeneous Activities of subCellular dynamics with eXplanations), an unsupervised self‐training deep learning framework specifically designed for identifying interpretable fine‐grained phenotypes of live cell dynamics. DeepHACX leverages the power of deep neural networks (DNNs) to unravel the phenotypic heterogeneity present in subcellular time‐series datasets. In this self‐training framework, we employ an unsupervised teacher model to guide the learning process of a student DNN. The student DNN is tailored to capture features that faithfully preserve the heterogeneity stemming from the effects of molecular perturbations. Our approach ensures both relevant heterogeneity and interpretability of the learned features, enabling a deeper comprehension of the underlying cellular dynamics.

In this study, we employed DeepHACX to analyze the dynamic behavior of migrating epithelial cells' leading edges, which undergo protrusion and retraction cycles. Cell protrusion is crucial for tissue regeneration,^{[ 30} , ^{31 ]} cancer invasiveness and metastasis,^{[ 32} , ³³ , ^{34 ]} and microenvironmental surveillance of leukocytes.^{[ 35 ]} Coordinated protrusion and retraction are linked to the metastatic potential of cancer cells^{[ 36 ]} and phenotypic switching of cell migration,^{[ 37 ]} making the analysis of morphodynamic biomarkers relevant for diagnosing breast and prostate cancer.^{[ 38 ]} However, dissecting these dynamics has been challenging due to the significant heterogeneity.^{[ 5} , ⁶ , ^{39 ]} DeepHACX successfully identified fine‐grained phenotypes from the highly heterogeneous and non‐stationary edge dynamics of migrating epithelial cells. It revealed specific responses to pharmacological perturbations in the form of “bursting” and “accelerating” protrusions. Intriguingly, DeepHACX generated distinct and concise features associated with these phenotypes, enhancing interpretability by precisely identifying the critical temporal intervals governing their manifestation. Furthermore, we demonstrated the ability to define single‐cell phenotypes based on these subcellular protrusions. This remarkable capability solidifies DeepHACX as a valuable tool for exploring diverse cellular dynamics and understanding their heterogeneous responses to molecular perturbations.

2. Results

2.1. DeepHACX: An Interpretable Self‐Training Framework for Fine‐Grained Phenotyping of Subcellular Dynamics

DNNs are adept at learning comprehensive features from raw datasets, making them well‐suited for the discovery of phenotypes. However, the multitude of features generated by DNNs often lack interpretability and the essential heterogeneity required for their effective application in fine‐grained phenotyping. To address this limitation, we propose a machine learning framework, DeepHACX (Figure  1a), which integrates unsupervised learning with self‐training. This allows a student DNN to acquire a concise and interpretable set of informative features capturing the temporal dynamics, guided by a teacher model. Self‐training, a widely recognized semi‐supervised learning method, involves a teacher model trained on a restricted labeled dataset providing pseudo‐labels for unlabeled data to train a student model.^{[ 40} , ⁴¹ , ⁴² , ^{43 ]} While this approach has proven effective in supervised learning with limited labeled and abundant unlabeled datasets, we employ it here in an unsupervised context. Our teacher model is an unsupervised model utilizing straightforward interpretable features. Then, we introduce a novel regularization strategy for training the student DNN, aiming to extract informative features that preserve heterogeneity while discriminating molecularly perturbed states. This integration of interpretability and feature learning ensures the extraction of a concise set of interpretable features, facilitating the identification of fine‐grained phenotypes with distinct sensitivities to molecular perturbations.

Figure 1.

Schematic representation of the computational pipeline. a) Self‐training framework of DeepHACX. b) Time series data generation and preprocessing. (1‐2) Live cell imaging and local sampling. Time‐lapse movies of the leading edge of migrating cells treated with/without different drugs were captured every 5 seconds, and then probing windows (500 × 500 nm) were generated to track the cell edge movement and local velocities were calculated per window. (3‐4) Event registration and time series selection. The velocity profile per window was divided into protrusion and retraction events based on protrusion onset. Then, protrusion events longer than 250s were selected and the time series were truncated to 250s for further analysis. (5‐6) Random noise padding and velocity transformation. For time series shorter than 250 sec, random noises around the sample mean were added for further analysis. The raw velocity profiles were non‐linearly scaled to [−1, 1] for the purpose of convenient training and eliminating the effect of larger values. c) Pseudo‐labels generated by the teacher model using interpretable features such as autocorrelation function (ACF). ACF‐based clustering pipeline was applied to the collected data to identify the protrusion phenotypes and characterize their sensitivities of molecular perturbations. d) Feature Learning by the student deep model integrating a bi‐LSTM‐autoencoder and multiple‐layer perceptron (MLP) classifier. The regularization parameter to balance the contributions of two branches was chosen to maximize the sensitivity of given molecular perturbations. The deep features from the bi‐LSTM encoder were extracted for further phenotyping. e) Phenotyping. Subcellular phenotypes were identified by clustering analysis using deep features. The identified phenotypes can be divided into deep phenotypes for more precise drug characterization. Subcellular phenotypes were used for single‐cell phenotyping. f) Feature Interpretation. We performed differential feature analysis to identify the deep features significantly associated with deep phenotypes. The SAGE (Shapley Additive Global importancE) analysis was applied to identify the time points critically linked to the significant deep features.

DeepHACX is a comprehensive framework comprising five key components: i) an unsupervised teacher model using a predefined interpretable feature such as Autocorrelation Function (ACF), ii) a student model specialized in deep feature learning, leveraging pseudo‐labels generated by the teacher model, while being regularized to maximize the heterogeneity associated with molecular perturbations, iii) fine‐grained clustering and molecular perturbation analysis, iv) analysis of feature interpretability, and iv) single‐cell phenotyping based on subcellular phenotypes. Our approach incorporates ACF‐based clustering as the teacher model to derive pseudo‐labels, seamlessly integrated into the student DNN for the automatic acquisition of interpretable features in deep phenotyping of live cells. Through the synergy of these components, DeepHACX offers a comprehensive and interpretable framework for uncovering fine‐grained phenotypes and facilitating an in‐depth understanding of complex cellular dynamics.

To prepare our live cell videos, we imaged migrating PtK1 epithelial cells in control and drug‐perturbed conditions (Figure 1b‐1). We then segmented the cell membrane boundary in each frame of each movie and divided the cell edges into small probing windows with a size of 500 by 500 nm.^{[ 5} , ^{44 ]} We acquired time series of protrusion velocities by averaging the velocities of pixels in each probing window (Figure 1b‐2). Next, we detected the protrusion onsets and aligned the time‐series protrusion velocities as a temporal fiduciary (Figure 1b‐3,4).^{[ 5 ]} Following the same procedure as in our previous study,^{[ 6 ]} we applied Empirical Mode Decomposition (EMD) to the protrusion velocity time series to reduce noise.^{[ 45 ]}

To develop an unsupervised teacher model (Figure 1c), we employed ACFs as time series features (ACF‐based clustering, Figure 1c‐2). Our previous work demonstrated that clustering based on the similarity of the ACF features of protrusion velocity time series produced highly interpretable protrusion phenotypes of similar molecular mechanisms.^{[ 6 ]} For DeepHACX, we advanced this method by creating a teacher model to guide a student model in recognizing time series data representing the same biological processes for effective clustering. While subcellular protrusion occurs over various temporal durations, leading to heterogeneous time series lengths, our earlier study focused solely on robust protrusion time series with durations longer than 250s and analyzed these time series of uniform temporal lengths.^{[ 6 ]} To include the remaining time series with durations shorter than 250s in training the student model, it is essential to have a computational framework capable of processing time series of varying temporal lengths.

Evaluating the similarity between samples of varying temporal lengths is inherently complex. Consequently, rather than computing the similarity distance across all samples, we specifically calculated distances among samples with comparable temporal lengths. To compute the ACF‐based distance for time series with similar temporal lengths, we introduced random noises as padding to the latter part of the time series, thereby equalizing their temporal lengths (Figure 1b‐5). We followed our previous ML pipeline, which involved SAX (Symbolic Aggregate approXimation)^{[ 46 ]} (Figure 1c‐1) and Euclidean‐based ACF distance. The distances, indicative of similarity, were aggregated to form a partial similarity distance matrix. Applying a community detection clustering algorithm^{[ 47 ]} to this matrix identified distinct clusters representing potential protrusion phenotypes (Figure 1c‐3). These clusters, which served as the basis for generating pseudo‐labels, were crucial for training a student DNN. Additionally, we quantified the effects of molecular perturbations within each cluster. This quantification identified the clusters that would exhibit the highest sensitivity to these molecular perturbations during subsequent training of the student model.

We proceeded with feature learning through the training of a student DNN model (Figure 1d) utilizing the pseudo‐labels from the teacher model. While standard classification tasks enable DNNs to learn crucial features to discriminate existing labels, they may not adequately preserve the heterogeneity vital to deep phenotyping. Although autoencoder^{[ 24 ]} and its variants^{[ 48} , ⁴⁹ , ⁵⁰ , ^{51 ]} have been widely used for feature learning by minimizing the reconstruction loss, their features are not guaranteed to be discriminative to molecular perturbations of interest, and may contain irrelevant heterogeneity. To address these challenges, we introduced a novel regularizer, named SENSER (SENSitivity‐enhancing autoEncoding Regularizer). Importantly, regularization plays pivotal roles in self‐training.^{[ 52} , ^{53 ]} By injecting significant heterogeneity during this feature learning, SENSER allows our student model to explore the feature space where specific molecular perturbations can be discriminated. To accomplish this, the student model training is regularized to maximize the effect size of specific molecular perturbations within the clusters identified by the teacher model. This novel regularization strategy is designed to capture features that preserve the essential heterogeneity in the context of molecular perturbations. Therefore, this approach markedly differs from supervised autoencoder,^{[ 51 ]} where the primary focus is maximizing the accuracy of a given classification task. Specifically, we produce distinct clustering outcomes by adjusting the regularization parameter assigned to the losses of the autoencoder and the classifier. Subsequently, we compute the effect sizes of the clusters identified by the teacher model as sensitive to molecular perturbation, determining the optimal regularization parameter by maximizing these effect sizes. Through this process, our student model guarantees that the learned features encapsulate essential heterogeneity and exhibit discriminative power concerning specific molecular perturbation. Crucially, our method can also be viewed as a multi‐task learning (MTL) approach, anticipated to yield more robust and generalizable features compared to single‐task learning.^{[ 43} , ⁵¹ , ^{54 ]}

Our student model was based on bidirectional Long‐Short Term Memory (bi‐LSTM) structure^{[ 55} , ^{56 ]} (Figure 1d). In particular, LSTM specializing in time series data can handle variable‐length time series since it does not require a fixed length of time series. Having initially preprocessed the time series by rescaling them to the range [−1, 1] to mitigate the impact of large velocity magnitude (Figure 1b‐6), we trained our student model using our dataset, including more than 30,000 time‐series samples. By maximizing the total loss comprising the loss functions of the autoencoder and the classifier, we extracted “deep features” (features learned from deep learning). Subsequently, we applied Principal Component Analysis (PCA) to reduce the feature dimension and selected the first 15 principal components, explaining more than 95% of the total variance. Following this, we conducted clustering analysis to refine the preliminary clustering results obtained from the teacher model (Figure 1e).

The deep features derived from our student model are anticipated to maintain the heterogeneity of molecular perturbation effects owing to the SENSER. Leveraging these deep features, we conduct sub‐clustering analyses to further partition the identified clusters (parent phenotypes). We analyze the sub‐clusters under control and perturbed conditions to identify those with heightened responsiveness to molecular perturbations. These specific sub‐clusters are designated as “deep phenotypes” (Figure 1e). Subsequently, we utilize the proportions of subcellular protrusion phenotypes in individual cells as single‐cell features for clustering analysis, facilitating the identification of single‐cell protrusion phenotypes.

The interpretation of the identified deep phenotypes involves a two‐step approach (Figure 1f). First, statistical analyses are applied to ascertain the significance of deep features associated with identified deep phenotypes. Following this, SAGE (Shapley Additive Global ImportancE) analysis^{[ 57 ]} is employed to identify the temporal intervals that notably contribute to the expression of these deep features. SAGE assigns values to each time point, elucidating their importance in predicting target feature values. This combined approach not only validates the relevance of identified features but also provides a global understanding of the temporal dynamics, pinpointing specific time points or durations crucial for shaping the deep phenotypes.

2.2. Unsupervised Teacher Model with Interpretable Features for Cell Protrusions

Our unsupervised teacher model builds on our previous study,^{[ 6 ]} where we utilized ACF features of the time series data and the density‐peak clustering algorithm to cluster protrusion velocity time series.^{[ 58 ]} The previous analysis resulted in a poorly characterized heterogeneous cluster of “fluctuating” protrusion and was limited by the constraint of utilizing time series of uniform temporal lengths for clustering. To address these limitations, we employed the community detection method^{[ 59 ]} to uncover more refined protrusion phenotypes. Additionally, this approach enables the utilization of all available data, irrespective of the lengths of the time series, during the training of the student model, thereby aiding in the prevention of overfitting.

First, to know the role of the community detection method in our analysis, we focused on the equal length time series (56 time frames: 5 frames before protrusion onset and 50 frames after protrusion onset) of the protrusion velocity. By quantifying Silhouette values^{[ 60 ]} with varying the number of clusters (Figure S1a, Supporting Information), we chose the optimal number of clusters as six. Figure  2a shows the average velocity profiles in each cluster. The raw velocity map in each cluster (Figure 2b; Figure S1d, Supporting Information), the t‐SNE, and the ordered distance map of clustering results (Figure S1b,c, Supporting Information) visually confirmed the distinctiveness of the clusters. The significant difference from our previous results was that we were able to split the previous “fluctuation” cluster into the clusters of “steady” (Cluster I) and “bursting” (Cluster II) protrusions (Figure 2a,b). Because Cluster II exhibited rapid changes of edge velocity within 100 seconds, we named it “bursting protrusion”. Since Cluster III‐V exhibited periodic edge velocity, we named them “periodic protrusion” and Cluster VI named “accelerating protrusion” as we did in our previous work.^{[ 6 ]}

Figure 2.

Unsupervised teacher model of subcellular protrusion phenotypes. a) Average temporal patterns of protrusion velocity time series in the clusters whose temporal length is larger than 250s with a 95% confidence interval registered at protrusion onset (t = 0). (m: the number of probing windows, the number of cells: 46). b) Raw velocity maps for six identified phenotypes. All time series are registered with respect to protrusion onset (t = 0) c–f) Effects of myosin‐II inhibitor, blebbistatin on the subcellular protrusion phenotypes. Averaged protrusion velocity time series in each cluster registered at protrusion onset in DMSO or blebbistatin‐treated cells (m: the number of probing windows in DMSO; m’: the number of probing windows in blebbistatin‐treated cells) (c). t‐SNE plots of autocorrelation functions of protrusion velocity time series overlaid with the density of data in the cells treated with DMSO (the number of cells: 14) (d) and blebbistatin (the number of cells: 13) (e). Comparison of the proportion for each cluster per cell between DMSO and blebbistatin (20 µm) (f). Solid lines indicate population averages, and shaded error bands indicate 95% confidence intervals of the mean computed by bootstrap resampling. The error bars indicate 95% confidence interval of the mean of the cluster proportions. *p < 0.05, and **p < 0.01 indicate the statistical significance by bootstrap resampling. g–l) Representative bursting protrusion (Cluster II) in DMSO (g–i) and blebbistatin (j–l) treated cells. Raw velocity maps (g, j). Cluster labels (h, k). Edge time‐evolution (I, l). Bars: 5 µm.

The well‐established mechanism of cell protrusion at the leading edge is primarily driven by rapid actin polymerization, a process tightly regulated by various actin regulators, including Arp2/3 and VASP.^{[ 61} , ⁶² , ⁶³ , ⁶⁴ , ^{65 ]} The specific temporal coordination between Arp2/3 and VASP has been characterized, particularly in the context of accelerating protrusion.^{[ 6 ]} Additionally, myosin II plays a multifaceted role in regulating actin dynamics,^{[ 66 ]} contractility,^{[ 67} , ⁶⁸ , ^{69 ]} and adhesion dynamics,^{[ 70} , ^{71 ]} all of which contribute to the modulation of cell protrusion. To discern the specific effects of these regulators in each identified phenotype, we conducted an analysis of live cell movies from drug‐treated experiments. Initially, we performed the analysis of the CK666^{[ 72 ]} (Arp2/3 inhibitor) perturbation to these protrusion phenotypes. Consistently with the previous results, CK666 significantly reduced the proportion of the accelerating protrusion phenotype (Cluster VI) only in comparison to the inactive control (CK689) (p‐value = 0.013 by bootstrap resampling) (Figure S1e–h and Table S1, Supporting Information for details of statistical testing).

Next, we characterized the effects of blebbistatin on each phenotype. First, the velocity profile in Cluster II (bursting protrusion) was substantially elevated by blebbistatin treatment (Figure 2c) and Figure 2g–l demonstrate an example of bursting protrusion of a blebbistatin‐treated PtK1 cell in comparison to that of the control case. Moreover, the t‐SNE visualization revealed that more bursting protrusions (Cluster II) were produced by blebbistatin treatment (Figure 2d,e). Consistently, the quantification of the proportion of the clusters showed that bursting protrusion was significantly increased by the blebbistatin treatment (p‐value = 0.004 by bootstrap resampling) (Figure 2f; Table S2, Supporting Information for details of statistical testing), while CK666 did not show significant effects (Figure S1h, Supporting Information). This result aligns with a previous study suggesting that down‐regulation of myosin II with blebbistatin promotes lamellipodial protrusion.^{[ 70 ]} Nevertheless, our analysis implies that this form of protrusion can coexist with a range of diverse activities within the spectrum of protrusion behaviors.

As distinct subcellular protrusion phenotypes may arise due to diverse spatiotemporal regulation of actin regulators, we investigated the relationship between the velocity profiles of each protrusion phenotype and the dynamics of fluorescence intensities in actin and several actin regulators associated with each protrusion phenotype. This analysis was conducted using time‐lapse movies of PtK1 cells expressing fluorescently tagged actin (SNAP‐tag‐actin), Arp3 (HaloTag‐Arp3), and VASP (HaloTag‐VASP), with a cytoplasmic marker (HaloTag) serving as a control signal. The time‐series fluorescence intensities of each molecule were averaged for each protrusion phenotype (Figure S2, Supporting Information). While accelerating protrusion (Cluster VI) exhibited the same dynamics of actin and actin nucleators as our previous study,^{[ 6 ]} the fluorescence intensities of actin and VASP in Cluster IV‐V exhibited much clearer periodicity (Figure S2a,c, Supporting Information) than the previous study where their intensity periodicity was lost as the frequency of the oscillation increased (Figure 3 in the previous study^{[ 6 ]}). Intriguingly, the intensities of Arp3 still did not show clear periodicity in Cluster IV‐V (Figure S2b, Supporting Information), suggesting that Arp2/3 and VASP may have differential roles in regulating Periodic Protrusions.

After confirming the beneficial effects of the community detection algorithm in our teacher model, we added the time series of variable temporal lengths to the dataset and performed the same clustering analysis using the partial similarity matrix of ACF distances (Figure S3a,b, Supporting Information). The clustering results contained the same types of the clusters (Cluster I–VI) as the equal‐length time series. We also had additional clusters whose temporal lengths are shorter than 250 s (Cluster VII–X). Cluster VII and VIII are short (< 200 s) lived transient protrusions and Cluster IX and X are slightly longer (>200 s) protrusions. In summary, our findings suggest that using the community detection algorithm in this study helped to generate more accurate labels for time‐series data of varying lengths, which will improve the training of our student model.

2.3. Enhanced Discriminatory Power for Protrusion Dynamics by Deep Feature Learning

To train our student DNN (Figure  3a; and Table S3, Supporting Information), we utilized pseudo‐labels created by the teacher model for time‐series data of varying temporal lengths, including both longer and shorter than 250 s. While the primary focus of our study is to investigate robust long protrusions with durations exceeding 250 s, characterized by differential temporal coordination of actin regulators as established in our previous study,^{[ 6 ]} we also included short‐duration time series to create a larger and more comprehensive dataset for training the student model. This inclusion facilitates the capture of more robust features. Once feature learning was completed, we concentrated on analyzing the time series with durations longer than 250 s.

Figure 3.

Deep feature learning from protrusion velocity time‐series. a) Deep feature learning of supervised bi‐LSTM autoencoder. b–d) The training performance of feature learning. Total loss (b), autoencoder loss (c), and classification loss (d). e) Principal component analysis on the deep features. f) Effect sizes of the difference between CK689 and CK666 in Cluster VI with varying weight parameters. g) Effect sizes of the difference between DMSO and blebbistatin in Cluster II with varying weight parameters. h,i) Visual comparison between the input of scaled velocities (h) and reconstructed output from autoencoder (i). j–p) Comparison between ACF and DF‐based clustering. Average protrusion velocity time series registered at protrusion onset (t = 0) (j). Solid lines indicate population averages, and shaded error bands indicate 95% confidence intervals of the mean computed by bootstrap resampling. (the number of cells: 81; m: the number of probing windows in ACF‐based clustering; m’: the number of probing windows in DF‐based clustering). t‐SNE plot of ACF (k) and DF (n). Ordered distance map from ACF‐based clustering (l) and DF‐based clustering (o). Silhouette plots from ACF‐based clustering (m) and DF‐based clustering (p).

As training progressed, we observed a decrease in the reconstruction, classification, and total losses (Figure 3b–d). The visual comparison between the input and the output confirmed that the training was effective (Figure 3h,i). After reducing the dimensionality of the bi‐LSTM encoder features (Deep Features, DF) by PCA (Figure 3e), we applied the community detection algorithm to identify clusters. To determine the regularization parameter of SENSER that balances the losses of the autoencoder and the classifier, we evaluated the clustering results based on the effect sizes of accelerating protrusion (Cluster VI) between CK689 and CK666 (Figure 3f), and bursting protrusion (Cluster II) between DMSO and blebbistatin (Figure 3g). We found that the effect size of accelerating protrusion was highest when the regularization parameter was set to 25 (Figure 3f), while there was no clear pattern for the effect size variations of bursting protrusion (Figure 3g). This means that finding the optimal regularization parameter is critical for the analysis of accelerating protrusion, while the wide range of the regularization parameters can be used for the analysis of bursting protrusion. Hence, we determined the optimal regularization parameter for the student model as 25.

We observed that the average patterns of protrusion velocities resulting from clustering based on both ACF and DF, with equal temporal lengths, exhibit similarities (Figure 3j) (Figure S3c,d, Supporting Information illustrates the results for variable temporal lengths). However, the t‐SNE feature visualization highlighted notable distinctions between the two methods, revealing clearer cluster boundaries when employing DF (Figure 3k,n). The utilization of DF for clustering enhanced the clustering outcomes, providing a distinct order‐distance map and yielding high Silhouette values (Figure 3l,m,o,p).

This enhancement in discriminative power, relative to the teacher model, can be attributed to our self‐training strategy. We anticipate this will enable more sensitive statistical analyses of molecular perturbations. In our CK666 perturbation analysis, where we optimized the deep feature learning, we found similar clustering patterns using the DFs (Figure  4a) as those obtained using the ACF (Figure S1e, Supporting Information). We observed that the significant drug effects of CK666 on Clusters I and VI (Figure 4b–d) are consistent with the results from ACF‐based teacher model (Figure S1f–h, Supporting Information). While both ACF‐based and DF‐based clustering (Figure 4d) showed significant results (ACF‐based clustering, p‐value = 0.045 for Cluster I and 0.013 for Cluster VI, Table S1, Supporting Information; DF‐based clustering, p‐value = 0.016 for Cluster I and 0.0069 for Cluster VI by bootstrap resampling, Table S4, Supporting Information), we found that DF‐based clustering showed smaller p‐values, demonstrating that the DF can provide more sensitive and robust statistical analyses.

Figure 4.

Effects of CK666 and Cytochalasin D on subcellular protrusion phenotypes. a,e) Average velocity time series registered at protrusion onset (t = 0) in each cluster of the cells treated with CK689 or CK666 (m: the number of probing windows in CK689(control)‐treated cells; m’: the number of probing windows in CK666‐treated cells) (a) and DMSO or Cytochalasin D (CyD50: 50 nm, CyD100: 100 nm) (m: the number of probing windows in DMSO‐treated cells; m’: the number of probing windows in CyD50‐treated cells; m’’: the number of probing windows in CyD100‐treated cells) (e). Solid lines indicate population averages, and shaded error bands indicate 95% confidence intervals of the mean computed by bootstrap resampling. b,c,f–h) t‐SNE plot overlaid with the density of the deep features of the cells treated with CK689 or CK666 (b‐c) and DMSO or Cytochalasin D (f‐h). d,i) Comparison of the proportions for each cluster per cell between CK689 and CK666 (d), and DMSO and Cytochalasin D (i). j) Average velocity time series registered at protrusion onset (t = 0) in DF‐based clusters of the cells treated with DMSO or blebbistatin (m: the number of probing windows in DMSO; m’: the number of probing windows in blebbistatin‐treated cells). k,l) t‐SNE plot overlaid with the density of the deep features of the cells treated with DMSO (k) or blebbistatin (l). m) Comparison of the proportions for each cluster per cell between DMSO and blebbistatin. The error bars indicate a 95% confidence interval of the mean of the cluster proportions. *p < 0.05, **p < 0.01 indicate the statistical significance by bootstrap resampling. The numbers of cells: 10 for CK689, 10 for CK666 a–d) and 22 for DMSO, 16 for CyD50, 20 for CyD100 (e‐i), 14 for DMSO and 13 for blebbistatin (j‐m).

To confirm the validity of our results, we used the same DF‐based clustering on two additional drugs: Cytochalasin D (CyD) at concentrations of 50–100 nm (Figure 4e–i) and blebbistatin (Bleb) (Figure 4j–m). Previous research has shown that CyD affects accelerating protrusion by inhibiting VASP‐dependent actin elongation.^{[ 6 ]} We found that CyD significantly decreased the proportion of Cluster VI (CyD at 50 nm: p‐value = 0.027; CyD at 100 nm: p‐value = 0.013 by bootstrap resampling) (Figure 4i; and Table S5, Supporting Information), and significantly increased the proportion of Cluster I (CyD at 50 nm: p‐value = 0.021; CyD100 at 100 nm: p‐value = 0.023 by bootstrap resampling), consistent with the previous finding using CK666. Additionally, blebbistatin significantly elevated the proportion of Cluster II (p‐value = 0.0004; by bootstrap resampling), surpassing the p‐value obtained using the teacher model with ACF‐based clustering (p‐value = 0.004, Figure 4j,m; Table S6, Supporting Information). In contrast, Cluster II was not notably affected by CK666 (Figure 4d and p‐value = 0.4577; bootstrap resampling) or CyD (Figure 4i, CyD50: p‐value = 0.0707, CyD100: p‐value = 0.1455; bootstrap resampling). The t‐SNE plot of the DF distribution displays that the local region of Cluster II was highly elevated by the blebbistatin treatment (Figure 4k,l).

To demonstrate the effectiveness of the guidance provided by a teacher model in our proposed DeepHACX model, we compared it with a conventional autoencoder that operates without a teacher model. In this comparative study, we utilized a bi‐LSTM autoencoder with an identical number of layers and hyperparameters used in our current study, but without the classifier branch. Consequently, the conventional AE learns DFs solely by reconstructing the protrusion velocity time series. The t‐SNE plot of DFs from the conventional autoencoder exhibited significantly less discrimination of potential clusters compared to the results obtained with DeepHACX (Figure S4a,b, Supporting Information). When clustering was applied to these autoencoder features, the distance similarity heatmap (Figure S4c, Supporting Information) and Silhouette plot (Figure S4d, Supporting Information) indicated that the clusters were not clearly separable, unlike the plots generated using the DFs from DeepHACX (Figure 3o,p; Figure S4e, Supporting Information). Additionally, the mean velocity profiles within each cluster (Figure S4f, Supporting Information) lacked the distinct periodic and accelerating protrusion phenotypes identified by DeepHACX, displaying more complex patterns that were not readily interpretable (Cluster IV–VI). Overall, the comparison of discrimination ability, cluster performance, and interpretability highlights the critical role of the teacher model in our DeepHACX pipeline.

To evaluate the model's generalization to a different cell line, we applied the student model trained on PtK1 cells to analyze protrusion velocity time series of MCF10A, human mammary epithelial cells. Clustering results using these DFs demonstrated that MCF10A cells exhibit bursting and accelerating protrusion phenotypes, in addition to steady and periodic phenotypes (Figure S5, Supporting Information). This suggests that our findings extend to other types of epithelial cell lines.

2.4. Deep Phenotypes of Accelerating and Bursting Protrusions

With the validation of the enhanced discriminant analysis provided by DeepHACX's deep features for parent phenotypes, our focus shifts to the exploration of hitherto undiscovered deep phenotypes—fine‐grained phenotypes within the broader parent categories—and their associated drug sensitivities. Our investigation commences with a detailed sub‐clustering analysis, concentrating initially on samples within Cluster VI, characterized as accelerating protrusions. The aim is to pinpoint fine‐grained clusters within this category that exhibit heightened sensitivity to drug perturbations. The t‐SNE analysis of DFs suggested the presence of sub‐clusters that may be more sensitive to CK666 (Figure  5a) and Cytochalasin D (Figure 5b). The Silhouette values indicated that the optimal number of sub‐clusters of accelerating protrusion was three (Figure S6a,b, Supporting Information). The sub‐clusters identified in Cluster VI displayed subtle but distinct behaviors (Figures 5c,e, and Figure S6c, Supporting Information). We identified one weak (Cluster VI‐1) and two strong accelerating clusters (Cluster VI‐2 and 3). Cluster VI‐2 exhibited a brief pause in acceleration before 100s, while Cluster VI‐3 had constant acceleration.

Figure 5.

Deep phenotypes in accelerating protrusion and bursting protrusion. a,b) t‐SNE plots overlaid with the density of the deep features from accelerating protrusion (Cluster VI) for CK689/CK666‐treated (a) or DMSO/Cytochalasin D‐treated cells (b). c,d) Average protrusion velocity time series (m: the number of probing windows) (c) registered at protrusion onset (t = 0) and comparison of the proportions (d) for three deep phenotypes of accelerating protrusion from the cells treated with CK689 or CK666 (m: the number of probing windows in CK689‐treated cells; m’: the number of probing windows in CK666‐treated cells). e,f) Average velocity time series registered at protrusion onset (t = 0) (e) and comparison of the proportions (f) of the deep phenotypes of accelerating protrusion from the cells treated with DMSO/Cytochalasin D (m: the number of probing windows in DMSO‐treated cells; m’: the number of probing windows in CyD50‐treated cells; m’’: the number of probing windows in CyD100‐treated cells). g,h): t‐SNE plots overlaid with the density of the deep features from the bursting protrusion (Cluster II) for DMSO (g) or blebbistatin‐treated cells (h). i,j): Average velocity time series registered at protrusion onset (t = 0) (i) and comparison of the proportions (j) of the deep phenotypes of bursting protrusion from the cells treated with blebbistatin. (m: the number of probing windows in DMSO‐treated cells; m’: the number of probing windows in blebbistatin‐treated cells). Solid lines indicate population averages, and shaded error bands indicate 95% confidence intervals of the mean estimated by bootstrap resampling. The error bars indicate a 95% confidence interval of the mean of the cluster proportions. *p < 0.05, **p < 0.01 indicate the statistical significance by bootstrap resampling. The numbers of cells: (c‐d) CK689: 10, CK666: 10; (e‐f) DMSO: 22, CyD50: 16, CyD100: 20; (i‐j) DMSO: 14, blebbistatin: 13.

Then, we found that Cluster VI‐1 was not notably affected by CK666 (Figure 5c,d; p‐value: 0.0694, bootstrap resampling) and low doses of CyD (Figure 5e,f; CyD at 50 nm: p‐value = 0.0505; CyD at 100 nm: p‐value = 0.2543, bootstrap resampling, Table S7, Supporting Information). However, the proportions of Cluster VI‐2 and 3 were significantly decreased by CK666 (Figure 5d; p‐value = 0.0049 for VI‐2, 0.0012 for VI‐3, bootstrap resampling) and CyD (Figure 5f; CyD at 50 nm: p‐value = 0.3698 for VI‐2, 0.0193 for VI‐3; CyD at 100 nm: p‐value = 0.0165 for VI‐2, 0.0011 for VI‐3, bootstrap resampling, Table S8, Supporting Information). Furthermore, the effects of CK666 and CyD on Cluster VI‐3 were stronger than Cluster VI‐2. The treatment with blebbistatin had no significant effect on any deep phenotypes of accelerating protrusion (Figure S6d,e and Table S9, Supporting Information). We also identified similar deep accelerating phenotypes from MCF10A cells by using the DFs from the student model trained with PtK1 cells (Figure S6f,g, Supporting Information). In particular, Cluster VI‐3 from MCF10A exhibited a more robust acceleration profile than the other subclusters, sharing the same characteristics as PtK1 cells.

We conducted a parallel sub‐clustering analysis on Cluster II (bursting protrusions) to further identify deep phenotypes sensitive to blebbistatin. Sub‐clustering utilizing DFs identified five clusters based on the Silhouette values (Figure S7a,b, Supporting Information). These deep phenotypes (Cluster II‐1–5) identified in Cluster II were shown displayed subtle and visually distinct behaviors as shown in Figures 5i and Figure S7c (Supporting Information). We found that Cluster II‐1 was significantly increased by blebbistatin (p‐value < 0.0001, bootstrap sampling, Table S12, Supporting Information) while the other Clusters (II‐2–5) were not significantly affected (Figure 5g–j). This refined deep phenotyping pinpointed Cluster II‐1 as the sole phenotype susceptible to blebbistatin treatment. These findings underscore the utility of our deep phenotyping approach in isolating specific phenotypes, offering precise quantification of drug actions.

In contrast, CK666 treatment did not affect any deep phenotypes of Cluster II significantly (Figure S7d,e,i,k, and Table S10, Supporting Information). CyD treatment decreased the proportion of Cluster II‐2 and 5 significantly (p‐value < 0.0001 for II‐2 and CyD100, p‐value = 0.0059 for II‐5 and CyD50) and increased the proportion of Cluster II‐3 and 4 significantly (p‐value = 0.004 for Cluster II‐3 and CyD100, p‐value = 0.0149 for Cluster II‐4 and CyD100) (Figure S7f–h,j,l and Table S11, Supporting Information). Due to the opposite effects of CyD on the deep phenotypes of busting protrusion, the original bursting phenotype (Cluster II) was not shown to be notably affected by CyD (Figure 4i). We also identified bursting deep phenotypes from MCF10A cells by using the DFs from the student model trained with PtK1 cells (Figure S7m,n, Supporting Information). Clusters II‐1–3 in MCF10A cells displayed a velocity profile remarkably similar to that of PtK1 cells, whereas Clusters II‐4–5 exhibited distinctive differences from PtK1 cells. Collectively, our deep phenotyping approach facilitated the identification of subtle temporal patterns within our protrusion velocity time series dataset, associated with specific drug perturbations.

2.5. Identification of Concise and Interpretable Deep Features

We employed a comprehensive interpretation approach to elucidate the distinct roles played by deep features in various deep phenotypes identified by DeepHACX. Visualizations of the DF values (without PCA), organized for acceleration protrusion (Figure  6a) and bursting protrusion (Figure 6b) phenotypes, highlighted their individual contributions to each deep phenotype. In the case of accelerating protrusion (Figure 6c), the elevated values of DF7, DF23, and DF10 were significantly associated with Clusters VI‐1, 2, and 3, respectively (Figure 6d and p‐values <10⁻⁴ by Wilcoxon rank sum testing). For bursting protrusion (Figure 6f), the heightened values of DF19, DF3, and DF20 were distinctly linked with Clusters II‐1, 2, and 4, respectively, while the diminished values of DF17 and DF22 were associated with Clusters II‐3 and 5, respectively (Figure 6g and p‐values < 10⁻⁴ by Wilcoxon rank sum testing). The presence of DFs exclusively correlated with each deep phenotype underscores the remarkable efficiency of DeepHACX's features. With a minimal number of DFs, DeepHACX can comprehensively elucidate the heterogeneity characterizing both accelerating and bursting protrusion phenotypes.

Figure 6.

Interpretation of deep features associated with deep phenotypes. a,b) Visualization of deep features in the phenotypes of accelerating (a) and bursting protrusions (b). The rectangular boxes indicate the deep features uniquely linked to deep phenotypes. c,f) Average velocity time series of deep phenotypes of accelerating (c) and bursting (f) protrusion registered at protrusion onset (t = 0) (m: the number of probing windows). d,g) Differential expression analysis of deep features with accelerating (d) and bursting (g) protrusions. **** indicates statistical significance with p < 0.0001. The p values were estimated using two‐sided Wilcoxon rank sum test implemented in presto R package. The sample size for accelerating protrusion: 1085 (VI‐1), 645 (VI‐2), 366 (VI‐3). The sample size for bursting protrusion: 1099 (II‐1), 509 (II‐2), 495 (II‐3), 312 (II‐4), 206 (II‐5). e,h) SAGE analysis of each deep feature in accelerating (e) and bursting (h) protrusions.

The identification of a concise set of DFs, each exclusively associated with a specific deep phenotype, forms the basis for the subsequent SAGE analysis.^{[ 57 ]} This interpretability analysis reveals the individual time points' contribution to predicting distinct deep phenotypes. As each deep phenotype aligns exclusively with a particular DF, capturing essential heterogeneity, discerning each time point's contribution to a specific DF enhances the interpretability of the associated deep phenotype. Thus, the SAGE values for each DF provide insights into crucial time intervals for the manifestation of each deep phenotype. The deep phenotypes of accelerating protrusion displayed highly specific temporal contributions (Figure 6c,e). The importance of protrusion events in Cluster VI‐3 is emphasized by the SAGE values associated with DF10, particularly highlighting events occurring between 100 and 180 s. Conversely, in Cluster VI‐2, the crucial time points are those before 100 s and after 180 s, as indicated by the SAGE values linked to DF23. Similarly, for Cluster VI‐1, DF7's SAGE values highlight the significance of protrusion events occurring after 150 s.

In addition, the SAGE analysis for the deep phenotypes of bursting protrusion revealed precise temporal contributions (Figure 6f,h). Despite the similar dynamic profiles observed in Clusters II‐1, 2, and 4, the SAGE profiles exhibited remarkable specificity for each cluster. In Cluster II‐1 (DF19), pivotal protrusion events were identified ≈100 s when the protrusion velocity peaked and after 200s when protrusion ceased. Conversely, in Cluster II‐2 (DF3), crucial protrusion events occurred ≈80s, during a phase of increasing protrusion velocity. Notably, time points ≈125 s, when protrusion velocity reached its maximum, did not substantially contribute to Cluster II‐2. For Cluster II‐4 (DF20), the significant temporal interval was after 125s, representing the late phase of protrusion. This underscores the distinct and specific occurrence of significant events in these deep phenotypes of bursting protrusion at precise time intervals. Notably, in Clusters II‐3 (DF17) and 5 (DF22), as suggested by their velocity profiles (Figure 6f), multiple time intervals contribute to their phenotypes.

Our interpretation approach unraveled the remarkable properties of DeepHACX's features, showcasing their distinctive efficiency in characterizing the heterogeneity within accelerating and bursting protrusion phenotypes. Through visualizations and statistical analyses of DFs, we identified a concise set of features exclusively correlated with each deep phenotype. This precision laid the foundation for subsequent SAGE analysis, offering insights into the temporal intervals crucial for the manifestation of each fine‐grained phenotype. The interpretability of DeepHACX's features stems from their ability to encapsulate essential heterogeneity, enabling the discernment of individual time points' contributions to the prediction of specific deep phenotypes.

2.6. Subcellular Phenotypes as Single‐Cell Features

Our subcellular phenotyping approach yields valuable insights into the molecular mechanisms governing cellular protrusion at a subcellular level. Furthermore, by leveraging these subcellular phenotypes as features at the single‐cell level, we can unravel a deeper understanding of individual cell behaviors through a bottom‐up analytical approach. The inherently interpretable nature of our subcellular protrusion phenotypes enhances the comprehensibility of single‐cell phenotypes and their relationship with underlying molecular mechanisms. To delineate single‐cell phenotypes related to cell protrusion, we quantified the proportions of subcellular protrusion phenotypes within individual cells, utilizing them as cellular features. Subsequently, we applied the manifold learning technique, UMAP (Uniform Manifold Approximation and Projection),^{[ 73 ]} to visualize and analyze these single‐cell feature distributions. With the feature distribution of single‐cell data in a lower‐dimensional space, we conducted clustering analysis using a community detection method.

The Silhouette plots with the varying number of clusters indicated that the optimal number of cell clusters is nine (Figure S8a, Supporting Information). UMAP 2D visualization (Figure  7a), the proportion plots of each cluster (Figure 7b), the ordered‐distance plot (Figure 7c), and Silhouette plot (Figure S8b, Supporting Information) demonstrated that the identified cell clusters are highly distinct. Each cell cluster was characterized by the mean proportions of subcellular protrusion phenotypes (Figure 7d; and Table S13, Supporting Information for the number of cells in each phenotype). Particularly, Cell Cluster 7 (C7 in Figure 7d) and Cell Cluster 9 (C9 in Figure 7d) have high levels of bursting and accelerating protrusions. Therefore, we call them Bursting Cells and Accelerating Cells, respectively. Cell Cluster 8 (C8 in Figure 7d) or Cell Cluster 6 (C6 in Figure 7d) have high or medium levels of both bursting and accelerating protrusion. Therefore, we call them Strong or Mid‐Bursting/Accelerating Cells. In Table S14 (Supporting Information), we summarized the characteristics of these cell clusters and their phenotype names. Figure 7i–n depicts representative examples of Cell Cluster 7–9.

Figure 7.

Single‐cell phenotyping based on subcellular characterization. a,b) UMAP (a) and heatmap (b) of clustered single‐cell proportions of subcellular protrusion phenotypes (the number of cells: 133). c) The ordered distance map of single‐cell clustering. d,e) Average subcellular protrusion proportions in each cell phenotype (d) and the deep phenotypes of bursting (Cluster II) and accelerating (Cluster VI) protrusions in Cell Phenotype 7–9 (e). The 95% confidence intervals of the mean were estimated by bootstrap resampling. n: the number of cells in each cell cluster. f‐h) Comparison of the proportional differences between Bursting and Steady Cell Groups, and Accelerating and Steady Cell Groups in the conditions of CK689/CK666 (the number of cells in CK689‐treated cells: 10, the number of cells in CK666‐treated cells: 10) (f), DMSO/Cytochalasin D (the number of cells in DMSO cells: 22, the number of cells in Cytochalasin D‐treated cells: 36) g), DMSO/blebbistatin (the number of cells in DMSO cells: 14, the number of cells in blebbistatin‐treated cells: 13) (h). *p < 0.05, ***p < 0.001 indicate the statistical significance by bootstrap resampling. i–n) Representative examples of the space‐time maps of instantaneous edge velocity of the cell edge of Cell Phenotype 7 (i), 8 (k), and 9 (m). Representative examples of edge evolution on 5 s interval of Cell Phenotype 7 (j), 8 (l), and 9 (n). Bars: 5 µm.

We also quantified the proportions of the subcellular deep phenotypes, the blebbistatin‐sensitive Cluster II‐1 (bursting‐1) and CK666/CyD‐sensitive Cluster VI‐2/3 (accelerating 2/3) in each cell cluster (Figure 7e; Figure S8c, Supporting Information). Bursting Cells (Cell Cluster 7) have high levels of Cluster II‐1 and low levels of Cluster VI‐2/3. Conversely, Accelerating Cells (Cell Cluster 9) have a low level of Cluster II‐1 and high levels of Cluster VI‐2/3. Intriguingly, Strong‐Bursting/Accelerating Cells (Cell Cluster 8) have high levels of both Cluster II‐1 and VI‐2/3, while they have fewer Cluster VI‐1 (CK666/CyD‐insensitive) than Accelerating Cells. As summarized in Table S14 (Supporting Information), as cells have more proportions of bursting or accelerating protrusion, they tend to have more corresponding drug‐sensitive deep phenotypes. This suggests that the identified cell phenotypes have differential sensitivities to CK666, CyD, and blebbistatin. To confirm this, we first pooled the cell phenotypes as follows: Cell Cluster #3, 6, 7, and 8 into Bursting Cell Group; Cell Cluster #6, 8, and 9 into Accelerating Cell Group; Cell Cluster #1 and 2 into Steady Cell Group. Since Steady Protrusion was affected by the drugs oppositely to bursting or accelerating protrusion in the previous analysis (Figure 4d,i,m), we quantified the proportional differences between the cells of Bursting and Steady Cell Groups or Accelerating and Steady Cell Groups. We found that CK666 significantly decreased the proportion of Accelerating Cell Group over Steady Cell Group (p‐value = 0.014 by bootstrap resampling) while it did not notably change that of Bursting Cell Group over Steady Cell Group (p‐value = 0.123 by bootstrap resampling) (Figure 7f; Table S15, Supporting Information). Intriguingly, CyD/blebbistatin significantly decreased/increased the proportions of both Bursting and Accelerating Cell Groups over Steady Cell Group (Figure 7g,h; For CyD, Acc vs Steady, p‐value = 0.005 by bootstrap resampling; Bursting vs Steady, p = value<0.001 by bootstrap resampling; For blebbistatin, Acc versus Steady, p‐value<0.001 by bootstrap resampling, Bursting versus Steady p‐value = 0.001 by bootstrap resampling; Supplementary Table 15 for details of statistical testing) even though they did not affect bursting/accelerating protrusion in the previous subcellular analysis where every cell phenotype was considered (Figures 4i and 6c). This demonstrates that both subcellular and cellular phenotyping is necessary to fully understand the heterogeneity of cellular drug responses.

3. Discussion

In this study, we developed DeepHACX, a deep learning‐based unsupervised self‐training framework tailored for the analysis of live cell movies. Our results demonstrate that DeepHACX excels at learning concise and interpretable features that effectively capture the complex heterogeneity induced by molecular perturbations. This unique capability positions DeepHACX as a powerful tool for a comprehensive exploration of molecular mechanisms concealed within subcellular and cellular heterogeneity.

In DeepHACX, the process of unsupervised learning is involved with the transfer of knowledge from an interpretable, unsupervised teacher model to a student model, facilitating the extraction of features that are both robust and generalizable. The teacher‐student framework implemented in DeepHACX acts as a guiding mechanism for the student model, allowing it to capture essential information from the interpretable features of the teacher model. This knowledge transfer process, coupled with the application of our specialized regularizer, SENSER, encourages the student model to maximize the heterogeneity associated with molecular perturbations. Moreover, the identification of a concise set of features contributing to specific deep phenotypes, strongly supports the notion that these features achieved through DeepHACX are closely aligned with disentangled features in the context of deep learning.^{[ 22 ]}

Feature disentanglement implies that each learned feature distinctly represents a particular aspect or factor within the data.^{[ 22 ]} In this context, the deep features identified by DeepHACX are not only concise but also exhibit a specificity that aligns with the unique characteristics of the underlying cellular dynamics and molecular perturbation effects. The concise set of the features extracted by DeepHACX are disentangled in a way that each feature corresponds to specific aspects or deep phenotypes of cell protrusion. The fact that these deep features are concise yet robust enough to capture the heterogeneity suggests a form of disentanglement where each feature isolates and represents a distinct and interpretable aspect of the complex and multifaceted cellular dynamics.

The SAGE analysis conducted in our study provides a crucial layer of validation for the achieved feature disentanglement by DeepHACX. SAGE analysis delves into the temporal contributions of specific features to the overall prediction of deep phenotypes, essentially pinpointing which time points play important roles in shaping each phenotype. DeepHACX's concise set of features, with highly specific temporal contributions to various deep phenotypes, aligns with the essence of disentangled features. This specific temporal attribution implies that each feature distinctly contributes to the manifestation of different phenotypes, thereby reinforcing the notion that DeepHACX has effectively disentangled these features. The precision offered by SAGE analysis in identifying the temporal intervals associated with deep features further enhances the interpretability and specificity of these features, solidifying their disentangled nature and adding another layer of confidence in the reliability and robustness of our findings. This intrinsic disentanglement facilitates a more interpretable and granular understanding of the interplay between cellular behaviors and molecular perturbations, underscoring the efficacy of the DeepHACX framework in achieving this informative feature learning outcome.

Using this approach, we refined the effects of drugs on the previously identified acceleration protrusion phenotype^{[ 6 ]} and discovered a novel protrusion phenotype termed bursting protrusion, specifically potentiated by the myosin inhibitor, blebbistatin. Myosin is known to influence leading edge dynamics through processes such as focal adhesion assembly, cortical contractility enhancement, and actin retrograde flow amplification, and previous reports indicate blebbistatin's promotion of protrusions in both 2D^{[ 70 ]} and 3D^{[ 68} , ^{69 ]} environments. Our analysis indicates that pinpointing such specific subcellular phenotypes enhances our understanding of cell motility and morphodynamics, facilitating the characterization of precise drug effects at subcellular and cellular levels. Moreover, by examining subcellular phenotypes characterized by DeepHACX, we identified nine previously unknown single‐cell dynamic phenotypes of protrusion dynamics, each exhibiting differential drug sensitivities. This underscores the importance of multi‐scale analysis spanning subcellular to cellular scales, offering a more comprehensive understanding of the heterogeneity of drug actions. Our findings suggest an unexpectedly rich phenotypic diversity in cellular dynamics at subcellular and single‐cell levels, emphasizing the potential of DeepHACX in unraveling complex cellular behaviors.

The generalization capability of DeepHACX is supported by its robust performance across various drug treatment conditions. Furthermore, DeepHACX successfully delineated fine‐grained phenotypes within the heterogeneous protrusion dynamics of MCF10A cells, a different epithelial cell line, by applying the pre‐trained model with PtK1 cells to new datasets from independent experiments. The identified phenotypes remained consistent, showcasing the model's adaptability and robustness. This capacity to generalize across different biological conditions and cell types highlights DeepHACX is capable of uncovering complex cellular dynamics and their responses to molecular perturbations across diverse experimental setups.

DeepHACX represents a significant advancement in the field of live cell phenotyping, offering a unique combination of interpretability, heterogeneity‐preserving feature learning, and concise representation. DeepHACX can be easily adapted to investigate a range of subcellular processes, including local retraction dynamics, filopodial dynamics, and cytoskeletal dynamics, with the support of an expanded array of machine learning tools for segmentation,^{[ 74} , ⁷⁵ , ⁷⁶ , ^{77 ]} tracking,^{[ 78} , ^{79 ]} and registration.^{[ 80 ]} Its implications for drug discovery, cell biology, and personalized medicine make it a valuable tool for researchers seeking a deeper understanding of cellular dynamics in response to molecular perturbations.

4. Experimental Section

Data Preparation

The cell culture and live cell imaging procedures were followed according to the previous studies.^{[ 5} , ^{6 ]} For the drug treatment experiments, PtK1 cells were cultured on 27 mm glass‐bottom dishes (Thermo Scientific cat. #150 682) for two days and stained them with 55 µg ml⁻¹ CellMask Deep Red (Invitrogen) following the manufacturer's protocol. The cell was monitored using a spinning‐disk confocal microscope. Cellular morphology, the level of protein expression, and the number of nuclei in each cell movie were visually examined. The analysis was performed using the cells with a flat, minimally ruffling morphology and wide leading edges, the low expression level of fluorescent proteins, and a single nucleus. At this stage, the cluster distribution along the cell edges was not known. Thus, this data selection can be assumed unbiased for the presented analyses.

For Arp2/3 inhibition experiments, cells were incubated with 50 µm of CK666 or CK689 (EMD Millipore) for an hour before imaging. For Cytochalasin D experiments, cells were incubated with DMSO or Cytochalasin D (50 or 100 nm) (Sigma) for half an hour before imaging. For myosin inhibition experiments, cells were incubated with 20 µm blebbistatin (EMD millipore, cat. # 02 3389) for a half‐hour before imaging. In order to prepare a large dataset to train a deep learning model, the data from the previous study was also included in addition to the data generated in this study^{[ 6 ]} which provided more than 24,000 protrusion time series.

Local Sampling and Event Registration

After the threshold‐based segmentation, the cell edge velocity was calculated by tracking the cell edges.^{[ 5} , ³⁹ , ^{44 ]} Then, probing windows with the size of 500 nm by 500 nm were placed along the cell boundary.^{[ 5 ]} The number of probing windows then maintained constant throughout the movie (≈100 windows in each cell). The local protrusion velocity and fluorescence intensity were quantified by averaging the values within probing windows. By repeating this procedure in each frame of the time‐lapse movies, the time series of protrusion velocities and fluorescence intensities were acquired. Then, the time of protrusion onset was determined in each probing window.^{[ 5 ]} The protrusion velocity and fluorescence intensities over time in individual windows were registered by aligning the protrusion onset at t = 0.

Treating Missing Values

In the case where velocity values were missed from a time series due to image noise, these missing values were treated as follows: For each edge velocity or fluorescence intensity sample, the entire time series was discarded if the length of continuous missing values is longer than a threshold (here, the value 8 was used). Otherwise, the average of four values before and after the missing value was used to individually estimate the value for this location.

De‐Noising the Samples by Empirical Mode Decomposition

For each registered sample, the edge displacement was calculated from the edge velocity using the Matlab function trapz(). Empirical Mode Decomposition (EMD)^{[ 81 ]} was then applied to the transformed protrusion edge displacement to remove noise. Finally, the denoised velocity was calculated from the denoised displacement using the Matlab function diff().

Cell movement was highly non‐stationary, and Empirical Mode Decomposition (EMD)^{[ 81 ]} was a local and data‐driven de‐noising method to decompose non‐stationary signals into a series of intrinsic components. The general procedure of EMD could be described as follows:

1. Identify all the extremes (minima and maxima) of sample d(t);

2. Connect the local maximum points and local minimum points separately using an interpolation method to generate the envelope, e(t);

3. Compute the average of envelopes, avg (t) =  (e_min (t) +  e_max (t))/2;

4. Eliminate the average signal of the envelope from the sample d(t) to obtain the residual: avg (t) =  (e_min (t) +  e_max (t))/2;

5. Iterate from steps 1) to 4) on the residual m(t) until the avg(t) equals zero.

After EMD, the original signals could be decomposed into intrinsic mode functions (IMF) without any loss of information, and the residue was called the trend. For each component, a de‐trended fluctuation analysis was used to measure the self‐affinity as the fractal scaling index (α), which estimates the fractal‐like autocorrelation properties. The value of α inversely correlated with the possibility that the component is originated from noise. The code was obtained from the previous publication,^{[ 82 ]} and the value of α was empirically set to 0.33 to balance the maintaining information and reducing noise.

Representing the Velocity by Symbolic Aggregate Approximation (SAX)

The dimensionality of times series samples consisting of 56 frames remained high and should be reduced. To this end, SAX (Symbolic Aggregate approXimation)^{[ 46 ]} was applied to the time series dataset to reduce dimensionality and discretize the data. The general procedure of SAX was summarized as follows:

1. Manually determine the reduced dimension, N, and the symbolic number, M (the number of discretization levels).

2. For each normalized sample, the time series data over the entire time range were pooled together and fitted to a Gaussian distribution. The entire time series was divided into M ranges with equal probabilities of the fitted Gaussian distribution. The values of each range a symbol was represented accordingly.

3. Subsequently, the time series was divided into N intervals over time. The average value was calculated in each interval to represent the raw time series data.

4. Finally, the average value in each interval was represented by the symbol defined in step 2.

5. Iterate from step 2) and 4) until all samples were represented.

After the SAX representation, all time series data were reduced to low‐dimensional (N) symbolic series data, which were used in further analyses. In the current experiment, M and N were both empirically set to four. Here, 4 symbols that range from 0 to 3 would be used to calculate the autocorrelation coefficients. In addition, the symbol representation process in SAX also removes local noise.

Calculating the Sample Dissimilarity

To measure the dissimilarity of two time series, the original description of SAX representation proposed an approximate Euclidean distance of SAX as a dissimilarity measure.^{[ 83 ]} Instead, the dissimilarity measure was used based on estimated autocorrelation functions (ACF).^{[ 84 ]} First, the estimated autocorrelation vector was calculated, and the square Euclidean distance between the autocorrelation coefficients was then used to measure the dissimilarity of two samples as follows:

$$d_{\mathrm{ACF}}\left(X,Y\right)=\sum _{i=1}^L{\left(X_i-Y_i\right)}^2$$ (1)

In the implementation, the ACF distance was calculated using the TSdist R package.^{[ 85 ]}

Unsupervised Teacher Model Using ACF Feature—Calculation of Partial Similarity Matrix

1. After the velocity time series were denoised by Empirical Mode Decomposition (EMD), the time series whose temporal lengths were similar within the threshold (6 frames; 30 s) were pooled.

2. Random noise was padded to the end of each time series to make the length equal to the longest one. The random noise was generated from the Gaussian distribution with the mean estimated by the average value of the last five time points. Then, the missing part would be padded with random noise with the estimated mean and standard deviation.

3. To reduce the dimensionality of time series, the time series data was represented by Symbolic ApproXimate representation (SAX)^{[ 83 ]} as described in the previous study.^{[ 6 ]}

4. The Euclidean distance was calculated based on the autocorrelation coefficient.

5. Repeat from steps 1) to 4) until all the samples were processed.

6. Due to the random noise padding, the distance from sample A to sample B may differ slightly from the distance from sample B to sample A. To ensure the distance matrix is symmetrical, we calculated the average of the original distance similarity matrix and its transpose.

Unsupervised Teacher Model Using ACF feature—Clustering

Community detection method^{[ 59 ]} was applied to the partial similarity matrix. First, a K‐nearest neighbor graph was made based on similarity distance. Then, the adjacency matrix was calculated and identified the communities using the R package igraph. The number of clusters was estimated based on the Silhouette values of clustering results.

Student Model for Deep Feature Learning—Velocity Time Series Preprocessing

In this step, nonlinear scaling of protrusion velocity was performed to reduce the effects of large magnitude of protrusion velocities since they may come from less accurate measurements. The majority of velocity magnitude should be less than 10 µm min⁻¹ based on the experience. Therefore, a sigmoidal mapping function was designed, 2/(1 + e ^−0.3v ) − 1. After this sigmoidal scaling, the range of the velocity becomes [−1, 1].

Student Model for Deep Feature Learning—Model Training

The datasets consist of a total of 27,540 protrusion samples from 10 different drug experiments, with each subset containing a similar proportion of data. For robust and unbiased training, a random splitting method was utilized to divide the dataset into training, validation, and test sets. Specifically, without considering subsets of the data, 13,495 samples (49%) were randomly selected as the training set and 5,783 samples (21%) as the validation set. The remaining samples were reserved for testing. This approach ensures that the training, validation, and test sets were not biased toward any specific subset of the data, thus preventing any bias that could arise from phenotypes of any specific subset.

The training set was used to fit the parameters of the model, while the validation set was used to select the model with the best fit (the lowest value of the loss function). The details of the proposed supervised bi‐LSTM autoencoder are shown in Figure 2a. Mainly, three layers of bidirectional long short‐term memory (bi‐LSTM) were utilized as an encoder to extract the features and combined another three layers of bi‐LSTM as a decoder to reconstruct the input. In order to make the representative features consistent with the clustering results from the ACF‐based clustering, a multilayer perception (MLP) classifier was added to guide the training process. The total loss includes two loss functions as follows.

$$L_{total}=L_{classs}+\lambda L_{recon}$$ (2)

where reconstruction loss (L_recon ) is the mean squared error function from the autoencoder, the classification loss (L_classs ) is the multiple‐categorical cross‐entropy function from the MLP, λ is the balancing weight between L_classs and L_recon . The neural network training was performed using Keras 2.6.0 on NVIDIA RTX Titan GPU.

Student Model for Deep Feature Learning—Model Selection

In order to optimize the balancing weight between cross‐entropy for classifier training and mean squared error for autoencoder training, the model was trained with varying weights from 1 to 50 and selected the best weight that provides the most discriminative features for CK666 perturbation by maximizing the effect size of Cluster VI. A training set was used to fit the parameters with the batch size 128 and 237 epochs. During the training, the loss in the validation set was monitored and we used the model parameters for the best performance with the validation set. The bi‐LSTM encoder was used to extract the features for the subsequent analysis. TensorFlow was used to implement this supervised bi‐LSTM autoencoder in Python 3 in Ubuntu 18.04.

Phenotyping Using Deep Features

The features were extracted from the trained bi‐LSTM encoder, and then Principal Component Analysis (PCA) was applied for the dimensional reduction of the learned features. Based on the percentage of the explained variance, the first 15 principal components were used for clustering analysis.

After the feature reduction, the dataset was split into paired experiments: CK689/CK666, DMSO/CyD50/CyD100, DMSO/Bleb. For each paired experiment, the sample similarity was calculated using Euclidean distance, and then community detection was applied on the selected samples with 51 frames after protrusion onsets. To evaluate the optimal numbers of clusters, Silhouette values were calculated to estimate the optimal numbers in each experiment on the pooled control samples.

Deep Phenotyping Using Deep Features

After the initial clustering, the phenotypes of bursting protrusion (Cluster II) and accelerating protrusion (Cluster VI) were sub‐clustered into deep phenotypes using the deep features. All the samples were first pooled from the target phenotypes from different paired experiments and then applied the community detection to determine the deep phenotypes based on the Euclidean distance. The optimal numbers of clusters were determined by the maximum Silhouette values.

Statistical Testing of Heterogenous Drug Effects

The drug effects on each phenotype were quantified based on its proportion. Specifically, the number of each protrusion phenotype in each cell was counted under control and drug‐treatment conditions. These counts in each cell were then resampled using the bootstrp() function in MATLAB to generate 10,000 different bootstrapped datasets, from which distributions of the proportional differences were created of each phenotype between control and drug‐treatment conditions. Using these distributions, p‐values were calculated by estimating the probability that the phenotype proportion of one condition was greater or less than that of the other condition (one‐sided test). The 95% confidence intervals of the proportions were estimated using the MATLAB bootci() function. This procedure does not assume any particular distribution of the data.

Interpretation of Deep Features

To elucidate contributions of protrusion velocities to deep phenotypes, feature differential expression analysis was conducted followed by SAGE,^{[ 57 ]} a model‐agnostic method designed to quantify predictive power while considering feature interactions. Initially, deep features extracted from the bi‐LSTM encoder, and then we identified the deep features uniquely associated with each specific deep phenotype of accelerating or bursting phenotypes. This selection process utilized the Wilcox rank sum test, implemented in the presto R package. Subsequently, for the identified deep features, the SAGE analysis^{[ 57 ]} was employed to pinpoint time intervals significantly contributing to the prediction of the associated deep phenotypes. This was achieved using the framework of additive importance measures, employing Shapley values. The SAGE functions were implemented in the SAGE python package.

Cluster Visualization

For each paired experiment, t‐SNE (t‐distribution stochastic neighboring embedding) was applied for visualization with the default parameter (PC number:15, perplexity:30). The sample densities on 2D t‐SNE mappings were estimated using the crowdedness of each sample below the radius threshold, which was implemented in scatplot() in MATLAB by Alex Sanchez.

Cellular Phenotyping Based on Subcellular Phenotypes

The proportions of six identified subcellular phenotypes were used as the cellular morphodynamic features, and then we applied the previously described clustering method to identify cellular phenotypes from 133 cell movies. To quantify the drug effect on the cell phenotypes, Cell Cluster 6, 8, 9 were pooled into Accelerating Cell Group, Cell Cluster 3, 6, 7, 8 into Bursting Cell Group, and Cell Cluster 1, 2 into Steady Cell Group. Then, the proportion differences between Accelerating/Bursting and Steady Cell Groups were calculated. Also, the uncertainty of proportion difference was estimated using a bootstrap resampling strategy with replacement for 1,000 times using the boot.ci function with the parameter “norm” in boot R package.

Statistical Analysis

1. Pre‐processing of data: Refers the following subsections in Experimental Section: Treating Missing Values, De‐noising the Samples by Empirical Mode Decomposition, Representing the Velocity by Symbolic Aggregate approximation (SAX), Student Model for Deep Feature Learning ‐ Velocity Time Series Preprocessing.

2. Data presentation: In Figures 2, 4, 5, 6, and 7, error bars: 95% confidence intervals of mean.

3. Sample sizes for each statistical analysis. Also, see the figure legends for details.

   1. Figure 2f (DMSO: 14, blebbistatin: 13).
   2. Figure 4d (CK689: 10, CK666: 10), i (DMSO: 22, CyD50: 16, CyD100: 20), m (DMSO: 14, blebbistatin: 13).
   3. Figure 5d (CK689: 10, CK666: 10), f (DMSO: 22, CyD50: 16, CyD100: 20), j (DMSO: 14, blebbistatin: 13).
   4. Figure 6d (VI‐1: 1085, VI‐2: 645, VI‐3: 366), f (II‐1: 1099, II‐2: 509, II‐3: 495, II‐4: 312, II‐5: 206).
   5. Figure 7a–e (the numbers of cells: 133), f(CK689: 10, CK666: 10), g (DMSO: 22, CyD: 36), h (DMSO: 14, blebbistatin: 13).
   6. Figure S1h (Supporting Information) (CK689: 10, CK666: 10).
   7. Figure S2 (Supporting Information): the numbers of cells: 10 (actin), 11 (Arp2/3), 9 (VASP), and 5 (Halo).
   8. Figure S4e (Supporting Information) (autoencoder: 8702, DeepHACX: 8702)
   9. Figure S5 (Supporting Information): the numbers of cells: 16.
   10. Figure S6e (Supporting Information) (DMSO: 14, blebbistatin: 13).
   11. Figure S7k (Supporting Information) (CK689: 10, CK666: 10), l (DMSO: 22, CyD50: 16, CyD100: 20).
   12. Figure S8 (Supporting Information): the numbers of cells: 133.

4. Statistical methods: For the drug effect evaluation (Figures 2, 4, 5, and 7), a bootstrap resampling strategy (one‐sided) was used to assess the significant difference between the control and drug‐treatment experiments at the cell population level (see the subsection of Statistical Testing of Heterogenous Drug Effects for details). For the deep feature differential analysis (Figure 6), the two‐sided Wilcoxon rank sum test method (see the subsection of Interpretation of Deep Features) was applied. p‐values less than 0.05 indicated statistical significance. This testing methods do not rely on the assumption of normal distribution.

5. Software used for statistical analysis: MATLAB R2019b, Python 3.9.16, Keras 2.6.0, R 4.3.2.

Conflict of Interest

The authors declare no conflict of interest.

Author Contributions

C.W. initiated the project, designed the algorithm of deep learning, performed drug response analyses, and interpretation analyses, and wrote the final version of the manuscript and supplement. H.J.C. designed and performed live cell imaging experiments and drug perturbation experiments; L.W. performed the clustering analyses of the time series of unequal temporal lengths; K.L. coordinated the study and wrote the final version of the manuscript and supplement. All authors discussed the results of the study.

Supporting information

Supporting Information

Wang C., Choi H. J., Woodbury L., Lee K., Interpretable Fine‐Grained Phenotypes of Subcellular Dynamics via Unsupervised Deep Learning. Adv. Sci. 2024, 11, 2403547. 10.1002/advs.202403547

Data Availability Statement

The time series data and the code used in the current study are publicly available at https://github.com/kleelab‐bch/DeepHACX.