Predicting weekly instability in depressive symptoms among individuals diagnosed with Major Depressive Disorder using deep learning and passively-collected movement data

Abstract

Major Depressive Disorder (MDD) is a common mental health condition marked by persistent low mood, reduced motivation, and low energy. Persons with MDD often experience large fluctuations in their symptoms over hours and days, which can offer valuable clinical insights, highlighting potential targets for treatment and intervention. Yet, it remains unclear how accurately these acute symptom changes can be predicted. In this preregistered study, we examined whether deep learning models could predict short-term fluctuations in depressive symptoms among individuals diagnosed with MDD. Passively-collected accelerometry data, a ubiquitous, privacy-preserving data stream with empirical ties to MDD symptomatology, was used to predict instability in depressive symptoms. Our sample contains clinically-depressed individuals per a structured clinical interview (N = 179, Mean Age = 38.2 ± 10.0, Women = 81.6%). Participants completed depressive symptom assessments three times daily for 90 days, resulting in an average of 214.49 EMA questionnaires (median: 237, min: 77, max: 270, SD: 54.27) per person. Although our results suggest a small-moderate correspondence between the predictions and observed outcomes of passively-collected movement alone in predicting future high/low depression instability (AUC_Test = 0.59 ± 0.06, Sensitivity_Test = 0.56 ± 0.21, Specificity_Test = 0.59 ± 0.23), our model performed worse than a random intercept only baseline model (AUC_Test = 0.83) and predicted intraindividual variability of instability in depressive symptoms below chance (AUC_Test = 0.46 ± 0.21). Thus, there is a need to improve predictive performance to enhance clinical utility. This may require the inclusion of additional passive sensing modalities in future research.

Introduction

Major depressive disorder (MDD) is a common and disabling psychological condition characterized by persistent low mood, diminished pleasure or interest, and a lack of motivation and energy (American Psychiatric Association, 2013). As a result of their symptoms, individuals with MDD often experience severe distress and impairments across different areas of their lives (Bica et al., 2017; Kupferberg et al., 2016; Woo et al., 2016). Furthermore, an estimated 8.8% of U.S. adults, or 22 million people, experienced a depressive episode in 2020, with 6.2% suffering from severe impairment (NSDUH, 2022). Given its high prevalence and disabling effects, MDD is a leading cause of global disability and disease burden (Reddy, 2010). Consequently, there is a need to effectively characterize, monitor, and treat MDD. Traditionally, MDD is often conceptualized as static within depressive episodes, overlooking the within-person heterogeneity and fluctuations of depressive symptoms. However, symptoms can vary significantly both within and across days, with some persons experiencing rapid changes from hour to hour, rather than symptoms remaining relatively stable over time (Crowe et al., 2019; Ebrahimi et al., 2021; Lorenz et al., 2020; Nemesure et al., 2024).

Within fluctuations, researchers have investigated instability, or short-term, moment-to-moment fluctuations, as well as variability in symptoms, or overall fluctuations from the mean, often across longer periods of time including weeks or months (Dejonckheere et al., 2019; Pirla et al., 2023). Although research on the instability (i.e., short-term dynamics) of depressive symptoms is still in its early stages, existing studies suggest that emotional instability is linked to poorer psychological well-being and greater psychopathology (Houben et al., 2015). While depressive symptoms and affect are distinct constructs, low arousal negative affect items, such as feeling down or feeling lonely, are partly overlapping with depressive symptoms and thus research on affective instability offers valuable insights into the importance of instability in depressive symptoms. For example, Wichers and colleagues (2020; 2016) assessed participants’ momentary affective states multiple times per day for many months, finding that autocorrelations and variance in feeling down acted as early warning signs for critical transitions, significantly increasing a month before a significant and sudden transition in depression symptoms. Moreover, studies have shown that daily variability in positive and negative affect can significantly predict depression symptom trajectories, with higher variability predicting an increase in depression symptom levels over a 6 month follow up period (Panaite et al., 2020). Taken together, this research underscores the importance of monitoring moment-to-moment instability in depressive symptoms, alongside overall symptom improvement.

Capturing depression symptom instability also offers important practical clinical implications, particularly in terms of personalized treatment and assessment. For example, a patient with high symptom instability would require more frequent assessments to accurately describe the course of their depression. This high instability might also call for more temporally flexible interventions, such as adjusting treatments to match changes in symptom severity, or temporally precise interventions if their effectiveness depends, at least in part, on current symptom instability and severity.

Research on instability in psychopathology and related behaviors, such as mood and emotions, has especially grown in its popularity in the past few years with the rise in intensive longitudinal data methods, including ecological momentary assessments (EMAs). EMAs are brief self-report surveys used to repeatedly assess an individual’s psychological states or symptoms over a short period in a naturalistic environment in real-time and therefore well-suited to capture depression instability. EMAs are sometimes used in tandem with passive sensing methods that can capture behavior on an even more granular time scale throughout the day, are less burdensome for patients, and are thus more practical in clinical settings. Passive sensing methods involve objectively and unobtrusively monitoring of individuals’ behaviors in their natural environments over time, usually via mobile devices or wearables. Previous studies have shown promise in predicting depression severity by using passively-collected movement data together with other data sources, such as passively-collected sleep data (Jacobson et al., 2019a, 2019b; Jacobson and Chung, 2020; Moshe et al., 2021; Price, Heinz, Song, et al., 2023; R. Wang et al., 2018), suggesting the potential of passive measures in predicting depression instability.

In particular, the use of passively collected movement data alone holds promise for predicting depression instability, as it provides a less intrusive and privacy-preserving alternative for participants compared to the use of multiple sensors (Cornet and Holden, 2018). In addition, the movement of individuals has strong empirical ties with major depressive episodes and there is substantive research highlighting the positive association between psychomotor disturbances and depression severity indicating it might be a good predictor (Novick et al., 2005; Parker, 2000; Walther et al., 2019). Due to its strong ties to depression it is likely that movement data can be also linked to depressive symptom instability. Since there is limited literature on short-term depression symptom instability specifically, we can further draw parallels from studies on mood instability. For instance, Morshed and colleagues (2019) used ridge regression to predict mood instability based on various passive sensing data. They found that indoor mobility data (via WiFi) was the best single predictor (although not significant), and that combining it with activity data improved prediction performance. This highlights the relevance of movement data in capturing fluctuations in affective states and make them a strong candidate for inclusion as a predictor.

In the present study, we explore the feasibility of predicting acute fluctuations in depressive symptoms (i.e., high instability in depressive symptoms) under circumstances of limited data using passively-collected movement data. Our clinical sample (n = 179) used a wrist-worn wearable to collect movement data and completed frequent depression symptom questionnaires for 90 days. Depression symptom instability was measured using the root mean square of successive differences (RMSSD, see Houben et al., 2015) over a rolling seven-day window. We utilize deep learning models, which are well-suited for handling the high-density data streams generated by passive sensing and have shown superior performance over traditional machine learning approaches for detecting MDD using passively-collected movement data (Price, Heinz, Collins, et al., 2023). Thus, our aim was to employ deep learning models with passively-collected movement data to predict depression symptom instability over time across and within persons. Being able to monitor and predict when acute fluctuations in symptoms will occur within a person may help determine when interventions to stabilize symptoms are needed, thereby stabilizing symptoms, improving patient outcomes, and potentially preventing relapse.

Methods

Procedure and sample

The present analysis used data from the Tracking Depression Study, an R01 study using passive sensing and intensive longitudinal data collection to characterize the behaviors and symptoms of depressed persons across 3 months. Participants for the Tracking Depression Study were recruited from the United States via online advertisements on Meta Ads and Google Ads. Eligibility requirements included being at least 18 years old, a current diagnosis of MDD (measured via a structured clinical interview for DSM disorders (SCID)), and primary use of an Android smartphone. Participants were excluded if they were considered to be at risk of active suicidal intent based on the Columbia Suicide Severity Rating scale screener or during interaction with the study team. Additional exclusion criteria included current or past psychotic symptoms, bipolar disorder, or not meeting criteria for MDD within the prior 30 days. This study received approval from the Dartmouth College Committee for the Protection of Human Subjects (STUDY00032081).

At the beginning of the study, participants were asked to install the ecological momentary assessment and passive sensing platform (i.e., MLife) on their Android devices (R. Wang et al., 2014). This application prompted participants to complete an ecological momentary assessment (EMA) related to their depressive symptoms three times a day at four-hour intervals: starting in the morning (four hours after self-reported wake-up time), afternoon, and evening. Additional questionnaires were collected but are not included in the current study. Detailed explanations of the study procedures were provided to each participant during an onboarding session prior to the start of the study.

For study purposes, participants were also provided with and asked to wear a Garmin smartwatch (Vivoactive 4 s). This smartwatch collected multiple continuous passive sensing data modalities; however, for the present study, we utilized only passively-collected movement data gathered from the Garmin watches, namely step counts and zero crossing rates (ZCR). ZCR measures the frequency at which the signal crosses zero, reflecting the motion and activity frequency. This signal is derived from accelerometer data (Garmin Health, 2023). Participants were instructed to wear the smartwatch throughout the day and, if needed, to charge it during rest periods during daytime hours to ensure the collection of passive sensing data during periods of sleep.

To enhance compliance, participants received feedback on their study participation around every two weeks. This feedback included information on the number of completed surveys and days of smartwatch usage. Upon completion of the study, participants were also provided with information regarding their symptoms across the 90 days they were in the study. In total, participants had the opportunity to earn rewards of up to $487, contingent upon their completion of all study components.

At the time of data analysis, 198 participants had completed data collection. Sixteen participants were excluded as described in the preregistration because they did not have passive sensing data or because >50% of their passive measurement data were missing. Finally, three participants were excluded due to missing data midway through the study, resulting in a final sample size of N = 179. The 179 participants included in our analysis completed an average of 214.49 EMA questionnaires (79.44%) (median: 237 [87.77%], min: 77 [28.51%], max: 270 [100%], SD: 54.27), resulting in a total of 38,393 EMA questionnaires used for analysis. Baseline characteristics were collected for participants at time of enrollment including age (38.2 ± 10.0), sex at birth (86.6% Female), gender (81.2% women, 13.4% men, 3.4% non-binary, 1,7% prefer to self-describe), ethnicity (9.5% Hispanic or Latino) and race (80.4% White, 6.1% Asian, 5.0% Black or African American, 2.8% American Indian or Alaska Native and White, 2.2% Black or African American and White, 1.7% Other (Prefer to Self Describe), 0.6% Asian and White, 0.6% White and Other (Prefer to Self Describe), and 0.6% Asian and American Indian or Alaska Native.

Measures

Depressive symptoms

To assess depressive symptoms, a modified version of the PHQ-9 questionnaire was used (Torous et al., 2015). Participants used a sliding scale ranging from 0 (not at all) to 100 (constantly) to assess their symptoms over the preceding 4 h (e.g., in the past 4 h, I have had little interest or pleasure in doing things). Participants were asked to envision the sliding scale as representing a spectrum reflecting the least to the greatest degree they had ever experienced the symptom in question. Item 3 on the PHQ-9 (sleep) was only asked in the morning EMAs, therefore, afternoon and evening values for PHQ-9 item 3 were forward imputed based on the morning response for that day. The PHQ-9 scale demonstrated good internal consistency with a Cronbach’s alpha of 0.86 and strong general factor saturation (ω_hierarchical = 0.74; ω_{hierarchical−asymptotic} = 0.84, ω_total = 0.88), indicating reliable measurement. For more information on the reliability and validity see Haddox and Colleagues (2025).

To measure instability in depressive symptoms, we calculated the root mean square of successive differences (RMSSD) of the average PHQ-9 item score, which represents the moment-to-moment variability of the mean PHQ-9 score. We used the mean PHQ-9 score of each questionnaire and employed a rolling window spanning 7 days, equivalent to a maximum of 21 questionnaires (see Fig. 1). A 7-day period for calculating RMSSD was chosen to effectively capture symptom instability over an entire week. A shorter time window would have resulted in fewer data points due to the limited frequency of EMA questionnaires (three times per day), potentially leading to less stable and less reliable RMSSD estimates. Such estimates may be less clinically relevant. Participants completed an average of 15.75 questionnaires per week, which were used to calculate their instability in depressive symptoms, missing questionnaires were excluded from the calculation (median: 17.45, max: 20.04, min: 5.54, SD: 4.19).

Fig. 1.

Illustration of RMSSD calculation.

Note. This figure illustrates the calculation of the outcome variable (i.e., instability in depressive symptoms). We used the mean PHQ-9 score from each questionnaire and applied a rolling window spanning 7 days, as depicted by the colored dots. This window is equivalent to a maximum of 21 questionnaires. Consequently, each predicted outcome (depicted by the orange dot) incorporates the data collected over one week of EMA questionnaires.

To predict high instability in depressive symptoms, the rolling RMSSD PHQ-9 scores were binarized (see section Deviations from Preregistration). We aimed to predict high instability in depressive symptoms, which we defined as more than one standard deviation above the mean (M = 10.03, SD = 5.31). Consequently, any RMSSD of instability in depressive symptoms greater than 15.34 is considered high, which was the case for 13.77% of observations with a mean RMSSD of 19.6. Accordingly, 80.49% of the observations had low instability in depressive symptoms, with a mean RMSSD of 8.39. Any row of participant data where no RMSSD could be calculated was excluded (5.74% of observations). We chose to binarize depression instability using the pooled standard deviations (SDs) across all participants rather than individual (idiographic) SDs because using idiographic SDs would suggest that each participant experienced significant fluctuations in their symptoms at some point during the study, which is not necessarily the case. As illustrated in Fig. 2, some participants exhibited relatively stable symptoms throughout the study period (e.g., Participant 8). This indicates that large instability is not universal across all individuals.

Fig. 2. Example Data from 12 Participants.

Note. The orange line represents the mean PHQ-9 score for each EMA questionnaire (morning, afternoon, evening). The blue line depicts the rolling RMSSD, calculated using a 7-day rolling window (therefore it is repeated for morning, afternoon and evening). Bold dots and horizontal lines indicate periods of significant fluctuation in depressive symptoms, defined as scores greater than the mean plus one standard deviation.

Predictors

Step count and ZCR were streamed from the Garmin Software Development Kit and included as predictors in the modeling approach. The step count data were recorded approximately every 60 s, and therefore, we rounded each data point to the nearest minute. Entries exceeding 182 steps per minute were excluded from the dataset (n = 1476, <0.01%). We considered values above 182 as unlikely, given that even elite ultra-marathon runners typically do not exceed an average of 182 steps per minute (Burns et al., 2019).

The ZCR was recorded approximately every 30 s. Thus, we rounded each entry to the nearest 30-second interval. To address outliers, we established an appropriate threshold for step count based on the literature; however, we were unable to identify a comparable threshold for ZCR. Therefore, assuming that the error would be similar across the metrics measured via the same smartwatch, we removed the equivalent percentile from ZCR, which are entries that exceed 722 counts per 30 s and a percentile above 99.9935% (n = 3499, 0.0001%).

For both measures, we conducted further data cleaning by eliminating entries with invalid dates and any entries with time intervals deviating from the expected 60 s for steps and 30 s for ZCR. In cases where multiple entries occurred within the specified time intervals, we excluded those as well. Furthermore, we used a min-max transformation to shrink our data to a range between 0.5 and 1, which has been shown to increase model performance and efficiency (Shahriyari, 2019). In our approach, we opted not to impute or eliminate missing data, recognizing that it may contain valuable information (e.g., wear compliance and its relationship with depression instability). Thus, each missing value was labeled with a flag value of 0.01 (Chollet, 2018, p. 102, Chapter 4), chosen because it falls outside the transformed feature range (0.5–1), ensuring it is easily distinguishable from valid observations.

To predict instability in depressive symptoms, we exclusively utilized data that occurred before the first EMA measure used to calculate the rolling RMSSD. Fig. 3 illustrates an example using a three-day movement data window (i.e., a three-day lookback period), incorporating 4320 min of step count data and 8640 30-second epochs of ZCR data to forecast depressive symptom instability during the subsequent week. During model development, we varied the number of days of movement data included in the model to predict instability in depressive symptoms (ranging from 1 day to 10 days; further explanation is provided in the Model Development section below).

Fig. 3.

Illustration of predictors being used (the orange dot represents the day that will be predicted).

Note. This figure depicts the relationship between the outcome variable (instability in depressive symptoms) and predictors (movement data). Instability of depressive symptoms is computed over a rolling window spanning 7 days, as shown in the first row. We utilize movement data occurring before the initial EMA measure used for calculating instability in depressive symptoms, which is represented by the dark blue dots. In this specific example, we incorporate three days of movement data (i.e., a lookback window of three days). Throughout the model development process, we experimented with varying the number of included days of predictors.

Prediction model

Prediction task

Our initial objective was to predict instability of depressive symptoms measured with the rolling RMSSD value (see preregistration: https://osf.io/d3×82). However, we were unable to develop a well-performing model using only step data and ZCR data as predictors. Specifically, we could not find a correlation higher than 0.1 in any validation set split for the regression task, and we did not evaluate this approach on the test set.

Given this low performance, we shifted our focus to a binary prediction model aimed at identifying high instability in depressive symptoms (see Deviations from Preregistration). However, during the revision process, we recognized that performance metrics for this binary classification task were likely overestimated. Specifically, the model appeared to capture interindividual differences more effectively than intra-individual variability and most participants exhibited either consistently high or consistently low instability in their depressive symptoms (see https://osf.io/eun7z, Langener and Jacobson, 2026).

To investigate whether ZCR and step data contribute to predicting instability in depressive symptoms, we moved back to the regression task, this time not only using step count and zcr data but also incorporating participant ID as a dummy variable in our deep learning model aimed at predicting the rolling RMSSD value. By including participant ID, we aimed to account for individual differences and improve prediction performance. We compared this model with baseline models to determine whether ZCR data and step data provide any predictive value beyond using participant ID alone.

Thus, we will present the results of two prediction tasks here: 1) the classification problem (predicting high instability in depressive symptoms) using only ZCR and step data as predictors, and 2) the regression problem (predicting the rolling RMSSD) using ZCR data, step data, and participant ID as predictors.

Model development

Classification.

As stated in our preregistration (https://osf.io/d3×82), we explored different deep-learning model architectures and tested them on parts of our validation set. We focused on three approaches that are commonly used for predicting mental health outcomes based on passive sensor data when applying deep learning models (Heinz et al., 2022; Jacobson et al., 2021; Price, Heinz, Collins, et al., 2023). First, we used long short-term memory (LSTM) models. Second, convolutional neural network (CNN) LSTM models were used. Both performed poorly on the validation sets tested, leading to the decision not to further optimize or test them. Finally, a 2D CNN model was employed, which is described in the remainder of this section.

In line with related research leveraging densely-collected movement data to predict depression-related outcomes (Heinz et al., 2022; Price, Heinz, Collins, et al., 2023), the present analysis used a 2D CNN architecture (Krizhevsky et al., 2017), a deep learning approach originally developed for image classification. As such, segments of the minimally processed time series step count and ZCR data were transformed into 2D images using a Gramian Angular Field (GAF). This method allows for univariate data to be transformed into a 2D representation while maintaining the temporal dependency of data points (Z. Wang and Oates, 2014).

To include two predictors (steps and ZCR) in our model, we used a multi-head 2D CNN architecture (similar as Canizo et al., 2019). In this architecture, step data and ZCR data are first processed in separate branches and then concatenated before being passed to the final dense layer, which provides as output the probability of high instability in depressive symptoms. This architecture allows for predictions to be made with both data modalities (i.e., step data and ZCR data).

The model architecture for each branch comprised different layers. For each predictor, the data was passed into a 2D CNN layer that captures the spatial relationship of features from the image input (i.e., our movement time series data transformed into an image), followed by batch normalization and/or maximum pooling. This architecture was repeated three times. Batch normalization was employed to avoid overfitting and speed up training. The maximum pooling layer down samples the input by using a sliding window over the images, which returns the maximum value and thus is assumed to extract the most relevant information. Subsequently, the output was passed through two dense layers, each followed by an additional layer of batch normalization and dropout. The dense layers enable the model to capture complex relationships in the data. The dropout layer served as an additional means to prevent overfitting (see Supplementary Material Figure 1 for an overview of the full model architecture). The layers included in our model architecture were part of our hyperparameter tuning. Additional details on the model development process can be found in the Hyperparameter Search section and Supplementary Material.

As part of the model development, we examined the optimal amount of past movement data for predicting instability in depressive symptoms. To do this, we varied the quantity of included data used for prediction. That is, we utilized a time series of different numbers of days for the GAF transformation, ranging from 1 to 10 days of ZCR and step data (i.e., a lookback window ranging from 1 to 10 days). Model performance across these lookback windows was compared in the validation set. The performance was close, and no lookback window outperformed the others (see Supplementary Table 2). Consequently, we opted for an ensemble model that integrates predictions from each of the 10 different lookback windows. For this purpose, we computed a weighted average of predictions derived from the 10 different multi-head 2D CNN models for each cross-validation split. The weights were assigned based on individual performance in the validation set, utilizing the Area Under the Receiver Operating Curve (AUC) value. As a robustness check, we also calculated a non-weighted average prediction.

To address class imbalance, we applied class weights and sample weights during training. Class weights were computed using a balanced heuristic, which assigns each class a weight inversely proportional to its frequency in the training data, so that underrepresented classes contribute more to the loss (using compute_class_weight from sklearn). Sample weights were manually assigned to further emphasize the underrepresented positive class, with the optimal value (1.9) determined through manual hyperparameter tuning.

Regression.

Our regression model is based on a similar architecture to that used for our classification problem. The primary modification is the activation function in the final dense layer, which has been changed to a linear activation function to suit the regression context. Additionally, the model incorporates a third input: the participant IDs, which is one-hot encoded. The participant IDs are processed through a series of dense layers before being integrated into the final dense layer, allowing the model to account for individual differences in the predictions (see https://osf.io/qhbf7/ for the code of the full model). Our outcome (rolling RMSSD) was scaled between 0 and 1 within each cross validation fold.

Baseline Models.

Based on the comments from an anonymous reviewer, we compared the performance of our deep learning models with linear mixed-effects models that included only random intercepts. To predict the rolling RMSSD value, we used a linear regression model, while to predict high instability in depressive symptoms, we employed a logistic regression model. These models do not include any other predictors and rely solely on a random intercept. Table 1 shows an overview of all models.

Table 1.

Overview of Models.

Task	Name	Architecture
Classification	Deep Learning	Multi-head 2D CNN (ZCR, step)
	Random Intercept Only (Baseline)*	Mixed Logistic Regression
Regression	Deep Learning Full*	Multi-head 2D CNN (ZCR, step, participant ID)
	Random Intercept Only (Baseline)*	Mixed Linear Regression

^*added after first round of review.

Hyperparameter search

We conducted a systematic exploration of model architectures and hyperparameters prior to finalizing the reported model. This process was performed exclusively on validation data to avoid information leakage from the test set. Specifically, we evaluated alternative architectural configurations (e.g., single-feature vs. multi-head inputs, varying numbers of convolutional and dense layers, and inclusion of batch normalization and dropout), as well as a range of optimization and training parameters (including learning rates, batch sizes, dropout rates, and sample/class weights). We additionally explored different parameterizations of the 2D CNN and pooling layers and assessed the impact of preprocessing choices such as Gramian Angular Field image size. Across these experiments, model performance was compared using validation set metrics, and the final configuration reflects the best-performing and most stable setup. A detailed overview of all tested configurations and corresponding hyperparameters is provided in the Supplementary Material.

Model validation

We used an expanding window nested cross-validation strategy, as depicted in Fig. 4. This approach involved a systematic division of the dataset into multiple training, validation, and testing sets. While the training set expanded over time, the validation set remained fixed at seven days and the testing set at one day. Given that we also used a rolling window to compute our outcome variable (i.e., rolling RMSSD), we wanted to avoid the same data points that were used to calculate the RMSSD being present in both the training and testing sets. Therefore, we included a time lag of 6 days between the training and validation sets, as well as between the validation and testing sets. In total, our cross-validation strategy included 39 windows.

Fig. 4.

Example for expanding moving cross-validation set-up.

Note. The figure illustrates the setup for the expanding window cross-validation. The dark blue dots represent the training sets that expand over time. The light blue dots represent the validation sets, which remain fixed at 7 days. Finally, the light dots in the orange square represent the test sets, which remain fixed at 1 day.

Initially, we allocated the earliest data points of all participants to the training set (days 1–14)¹ and the next data points, including a six-day lag, to the validation set (days 21–27). Simultaneously, the test set comprised the next observation after a six-day lag for each participant (day 34). The outcome variable for this day was the binarized RMSSD of the PHQ-9 scores from days 28 to 34 to prevent data leakage between our training and test sets. As the iterations progressed, the testing set and validation set shifted forward by one day (new testing set: day 35, new validation set: days 22–28), while the training set expanded to include additional historical data (days 1–15). This iterative process continued until we reached the end of the dataset. Consequently, the model was exclusively evaluated on data points that were not employed in its development.

Model evaluation

Classification Task.

For the classification task, model performance was reported as AUC, Sensitivity, Specificity, Accuracy, Kappa,F1 Score, Negative Predictive Values, and Positive Predictive Values for the Test Set (see Table 2). Note that Youden’s index was applied to the threshold related to calculating Sensitivity, Specificity, Accuracy, Kappa and F1 Score (Fluss et al., 2005). Statistical significance of the held-out test set was assessed with a 95% confidence interval (Hanley and McNeil, 1982), and a conversion of AUC into cohen’s D was used in order to qualitatively assess the model’s performance. As such, a small-moderate correspondence between the held-out test set predictions and observed binary depression instability values reflected 0.56 ≤ AUC < 0.64 (Salgado, 2018). Performance was compared to a logistic random intercept only model.

Table 2.

Prediction Performance of the Classification Task within the Test Set.

Performance Metric	Ensemble Model	Random Intercept Only (Baseline Model)
	Mean ± S.D.	Mean ± S.D.
AUC	0.59 ± 0.06	0.83 ± (0.06)
Sensitivity	0.56 ± 0.21	0.76 ± (0.12)
Specificity	0.59 ± 0.23	0.78 ± (0.10)
Accuracy	0.59 ± 0.18	0.78 ± (0.08)
Kappa	0.09 ± 0.09	0.36 ± (0.10)
F1 Score	0.26 ± 0.07	0.47 ± (0.08)
Negative predictive values (NPV)	0.89 ± 0.09	0.96 ± (0.02)
Positive predictive values (PPV)	0.18 ± 0.08	0.36 ± (0.10)

Note. The left column shows a summary of the model’s performance (using the weighted average across lookback windows as prediction) for the validation and held-out test set. Reported as AUC, Sensitivity, Specificity, Accuracy, Kappa, and F1 Score. The right column shows the performance of a random intercept only model as baseline comparison.

Regression Task.

For the regression task, we evaluated model performance using several standard metrics: Mean Squared Error (MSE), Mean Absolute Error (MAE), the Coefficient of Determination (R²), and the correlation coefficient. Performance was compared to a linear random intercept only model.

Intraindividual vs. Interindividual variability.

We used an expanding moving window cross-validation strategy, which results in the same participants being present in the training and test sets. Thus, performance metrics calculated on the full test set capture both intra-individual and interindividual variability. However, since the same participants are included in both sets, the model may not generalize well to new, unseen populations. For practical application, it is therefore crucial that the model demonstrates good performance in capturing intraindividual variability within individuals. This would suggest that the model can reliably monitor individual fluctuations, rather than merely differentiating between individuals. We report performance for predicting both intra- and interindividual variability as well as performance for predicting intraindividual variability only.

Deviations from preregistration

Although our study was primarily exploratory, we chose to preregister our research question and cross-validation strategy. Two versions of our preregistration were submitted, each with a different cross-validation setup. The first version described a rolling cross-validation methodology (see previous section). In contrast, the later version described an approach in which each training, validation, and test set was of equal length. We decided to use the approach described in the first version of preregistration because, in the second version, insufficient data prevented effective model training. Importantly, this decision was made before any predictions were made on the test set. A disadvantage of the first cross-validation setup is that if a large lookback window is used (i.e., greater than 7 days), predictors are present in both the training and test sets. However, we do not believe that this limits the applicability of our approach since we are not using data that would not be available in a real-world scenario where we want to make predictions in real-time.

An additional modification we made involved binarizing the outcome, aiming to predict high instability in depressive symptoms compared to predicting the rolling RMSSD value of the PHQ-9 score. This shifts the prediction task to a classification problem instead of a regression problem. The reason for this is that we could not predict the rolling RMSSD value of the PHQ-9 score, i.e., we could not find a correlation higher than 0.1 in any validation set split for the regression task (we did not evaluate this approach on the test set). Nevertheless, we believe that the prediction of high instability in depressive symptoms has clinical utility, serving as an indicator of moments requiring intervention. In practice, this analysis aims to define a clinically meaningful threshold of symptom instability that would warrant such action. Accordingly, discretizing instability into low and high categories reflects this intended clinical application.

Based on the comments of an anonymous reviewer we also added the comparison to baseline models, which were not preregistered. Additionally, we moved back to the regression task, this time incorporating participant ID as a dummy variable in our deep learning model. Our aim was to increase prediction performance and to compare this model with baseline models to determine whether ZCR data and step data provide any predictive value beyond using participant ID alone. This comparison was not preregistered. Based on another reviewer’s comments, we also ran a generalized linear mixed model on the full dataset. The results of this model can be found in the supplementary material.

Lastly, three participants were excluded because they had missing data midway through the study. This was unknown at the time when the preregistration was written.

Transparency and openness

The data used in the current study is from the Tracking Depression Study, an R01 study funded by the National Institute of Mental Health (NIMH) and the National Institute of General Medical Sciences under grant 1 R01 MH123482–01. This research is also supported in part by T32 (T32DA037202–07) and P30 (P30DA029926) grants provided by the National Institute on Drug Abuse (NIDA). The analytic plan for the study was preregistered at https://osf.io/d3×82. The data will be made available as part of the NIMH data archive, but will remain under embargo until the study is completed. The analysis code is available at https://osf.io/w62fe/.

Results

Predictive performance classification task

Intra- and interindividual variability

As summarized in Table 2, the random intercept only model, used as the baseline comparison, demonstrated relatively high performance across all metrics. In the test set, the baseline model achieved an AUC of 0.83, with an accuracy of 0.78. It also showed good sensitivity (0.76) and specificity (0.78). The model’s chance-adjusted agreement, measured by kappa, was 0.36, indicating a moderate level of agreement between the predicted and observed values. The F1 score of 0.47 reflects a moderate balance between precision and recall. Additionally, the negative predictive value (NPV) of 0.96 and positive predictive value (PPV) of 0.36 suggest high accuracy in predicting the absence of instability in depressive symptoms, but lower accuracy in predicting high instability.

The ensemble model achieved a small-moderate correspondence between the predictions and observed outcomes, with an AUC of 0.59 (95% CI [0.56,0.61])² and an accuracy of 0.59. The model demonstrated an ability to identify cases with high instability in depressive symptoms (sensitivity of 0.56) and an ability to correctly identify cases with low instability (specificity of 0.59). However, the chance-adjusted agreement between predicted and observed values remained low, marked by a kappa of 0.09, along with the balance of precision and recall, marked by an F1 score of 0.26. The NPV of 0.89 indicates high accuracy in predicting the absence of high instability in depressive symptoms. The PPV of 0.18 suggests lower accuracy in predicting high instability in depressive symptoms.

As a robustness check, we also used an ensemble model that calculated predictions based on a non-weighted average. The AUC values (and standard deviation) remained approximately the same in both the validation and test sets (e.g., the difference between the AUCs in the test set was 0.0002).

Overall, our ensemble model performed worse than the random intercept-only model across all performance measures in capturing both intra- and interindividual variability in predicting high fluctuations in depressive symptoms.

Intraindividual variability

As a next step, we investigated the model’s performance at the individual level in predicting intraindividual differences. It is important to note that we can only assess how well the model predicts variability within a person if variability is present in the dataset. In our sample, only 76 participants exhibited both high and low instability in depressive symptoms in the test set. The remaining participants showed either consistently high or consistently low instability in depressive symptoms (see https://osf.io/eun7z for plots displaying the data for each participant). Therefore, the remainder of this section will focus on reporting the model’s performance in predicting high versus low instability in depressive symptoms for this subset of 76 participants.

Both the random intercept only model and the ensemble model performed poorly in predicting intraindividual variability and exhibited large fluctuations across participants (see Fig. 5). On average, the AUC within participants is 0.46 ± 0.23 for the random intercept only model and 0.46 ± 0.21 for the ensemble model. Which means that both models predict below chance in predicting intraindividual variability. Similarly, the sensitivity for the random intercept only model is 0.31 ± 0.32, while the ensemble model shows slightly better sensitivity at 0.36 ± 0.30. In terms of specificity, the random intercept only model had a mean of 0.53 ± 0.28, while the ensemble model had a slightly lower mean of 0.49 ± 0.22. For accuracy, the random intercept only model achieved 0.47 ± 0.23, whereas the ensemble model performed similarly, with a mean of 0.46 ± 0.17.

Fig. 5.

Performance Predicting High Instability in Depressive Symptoms Within Person (Intraindividual Variability).

Predictive performance regression task

Intra- and interindividual variability

Table 3 shows the performance of the deep learning model - including step data, zcr data and a dummy variable indicating participant ID - and the random intercept only model in predicting intra- and inter-individual variability within the test set. Both models performed similarly across all metrics, with only minor differences in predictive accuracy.

Table 3.

Prediction Performance of the Regression Task within the Test Set.

Performance Metric	Deep Learning (Dummy, ZCR, Step)	Random Intercept Only (Baseline Model)
	Mean ± S.D.	Mean ± S.D.
MSE	0.0130 ± (0.0005)	0.0129 ± (0.0005)
MAE	0.0836 ± (0.0009)	0.0836 ± (0.0009)
R 2	0.4334 ± (0.0097)	0.4343 ± (0.0097)
Correlation	0.6687 ± (0.0062)	0.6839 ± (0.0062)

Note. The left column shows a summary of the deep learning model’s performance using a lookback window of 8 days and ZCR data, step data, and a dummy variable indicating participant ID as predictor. The outcome was scale from 0 to 1. The right column shows the performance of a random intercept only model as baseline comparison.

Both models demonstrated a low MSE, indicating small prediction errors on average. The deep learning model had an MSE of 0.013, while the Random Intercept Only model had a slightly lower MSE of 0.0129. The MAE was identical for both models at 0.0836. The deep learning model has an R² of 0.4334, while the random intercept only model has an R² of 0.4343 The deep learning model exhibited a correlation of 0.6687 between predicted and observed values, while the random intercept only model had a slightly higher correlation of 0.6839.

Intraindividual variability

We again investigated the model’s performance at the individual level in predicting intraindividual variability of the rolling RMSSD score. As this is a regression task, all participants exhibit some variability in their depressive symptoms and are therefore included in the assessment of within-person predictive performance (as opposed to the classification task, where we were only able to assess performance in predicting intraindividual variability for 76 participants).

Similarly to the classification problem, both the random intercept only model and the deep learning model performed poorly in predicting intraindividual variability with large variability across participants (see Fig. 6). The MSE and MAE were similar for both models, with means of around 0.01 ± 0.02 and 0.09 ± 0.05, respectively. The R² values were negative (random intercept only: −1.36 ± 2.77, deep learning: −1.58 ± 3.36), suggesting poor model fit. Additionally, the correlation between predicted and observed values are 0.01±0.02 for the baseline model and −0.03±0.27 for the deep learning model.

Fig. 6.

Performance Predicting Rolling RMSSD Within Person (Intraindividual Variability).

Discussion

General overview

The primary aim of the present preregistered study was to explore the feasibility of predicting depression symptom instability using exclusively passively-collected movement data. Using a deep learning modeling approach combined with GAF to transform densely collected time series data into 2D images, we were able to predict binary fluctuations in depressive symptoms using step count and zero crossing rate (AUC = 0.59). However, the model performed less well than a random intercept-only baseline model, which achieved an AUC of 0.83. By maintaining temporal precedence across the modeling pipeline between training, validation, and test set data, the present implementation aligns with a real-world implementation of deploying a model to predict future depression symptom instability. It also enabled us to investigate the model’s ability to predict both interindividual and intraindividual variability in depressive symptoms. Importantly, both the random intercept only model and the deep learning model performed poorly when predicting intraindividual variability. Neither model was able to reliably predict fluctuations within individuals, and there was considerable variability in how well intraindividual variability was captured across participants (AUC = 0.46 ± 0.21). Due to the fact that only 76 participants in the test set exhibited both high and low instability in depressive symptoms, while the majority showed consistently high or low instability, we conducted additional analyses using the rolling RMSSD as the outcome, incorporating participant ID as a predictor alongside step count and ZCR data. These analyses yielded similar results; the deep learning model did not outperform the baseline model, and its predictive performance for within-person variability remained low.

Implications for research and clinical utility

Our findings underscore the importance of evaluating and optimizing predictive performance not only between people but specifically at an individual level. We employed an expanding moving window cross-validation approach, which results in the same participants being included in both the training and test sets. While this method captures both intraindividual and interindividual variability, it may limit the generalizability of the model to new, unseen populations. For real-world clinical applications, it is essential that predictive models demonstrate strong performance in capturing intraindividual variability—i.e., predicting fluctuations in symptoms within an individual over time—rather than merely distinguishing between individuals. Our results suggest that the model was not able to predict within-person fluctuations, with large differences in how well intraindividual variability was captured across participants. This indicates that the model was mainly able to distinguish between individuals (see also Langener and Jacobson, 2026) which is limitedly useful for clinical applications that aim to intervene on the individual level (i.e., such as just-in-time interventions). For this, models are needed that can accurately monitor symptom instability in individual patients over time, rather than simply differentiating individuals.

To improve intraindividual performance, individualized models may offer a promising solution, as they have been shown to outperform non-individualized models and are specifically designed to capture intra-individual variability (Abdullah et al., 2016; Benoit et al., 2020; Cai et al., 2018). Previous research has also demonstrated that combining individualized and non-individualized models can effectively capture both intraindividual and interindividual variability in depressed mood (Jacobson and Chung, 2020). However, a key challenge in implementing individualized models is the need for sufficient data to train them. While our study used a relatively long data collection period, we still lacked enough independent data points to build individualized models. As a result, we were unable to take advantage of these models in this paper. Moving forward, future research could benefit from developing methods that better capture intraindividual variability, even with limited available data, to improve predictive accuracy and clinical applicability. A promising avenue for future work would be to develop hybrid modeling approaches that integrate individualized and population-level prediction models, potentially capturing both person-specific dynamics and shared patterns across individuals. Another useful direction may be to identify subgroups of participants exhibiting similar trajectories in behavioral and mental health changes over time, for example through clustering, which could help account for heterogeneity in symptom dynamics and improve predictive performance.

Accelerometer data has many advantages, such as low battery consumption and minimal privacy concerns, and is used in various studies on mental health and general well-being (Trifan et al., 2019). However, relying solely on accelerometer and movement data presents challenges. The data often comes in a large volume, is heterogeneous between and within subjects, and complex to analyze (for an overview see Karas et al., 2019). Heterogeneity within and between participants, as well as factors such as sampling frequency, may impact the results. Therefore, before data collection begins, the research team and experts should carefully discuss the choices that need to be made during data collection and cleaning (Karas et al., 2019), and ideally, these choices should be preregistered (Langener, Siepe, et al., 2024). Another challenge is that movement data alone may not be a strong enough predictor to capture within-person changes. Therefore, future studies could benefit from incorporating other passively collected data or actively assessed variables to improve the prediction of within-person changes. For future research, we recommend that besides focusing on methods that can better capture intra-individual variability to also incorporate other data sources when predicting depression instability.

To effectively integrate passive sensing into clinical practice, researchers must prioritize outcomes with significant clinical utility that are also feasible for implementation. Depression symptom instability fulfills these criteria. Depression symptom instability can capture acute changes in depressive symptoms experienced by individuals with depression that a single time point (Ebrahimi et al., 2021), static outcome would fail to reflect; however, this requires methods of data collection that can also capture such acute changes in behavior, highlighting the utility of passive sensing for modeling depression symptom instability. However, as our results indicate that we were unable to predict instability in depression with high accuracy, predicting instability in depressive symptoms based on movement data alone may be an overly ambitious goal. While there is a theoretical link between depression and movement, depression is influenced by many factors and predicting instability in depressive symptoms may be even more challenging than predicting overall symptom severity. Additionally, self reported measures can introduce biases (Jongs et al., 2020), contain measurement errors, are often not validated, and lack clear conceptualization (Bringmann et al., 2022; Bringmann and Eronen, 2016; Lucas et al., 2021; Roos et al., 2023), which can make accurate prediction even more challenging.

Constraints and generality

The present application of deep learning methods to attempt to predict future depression symptom instability in a clinical sample using strictly movement-related data represents an important step in implementing passive sensing methods in real-world clinical settings. While many passive sensing studies have relied on opportunistic designs with small, unrepresentative samples and short follow-up periods that limit the generalizability of the findings, this study followed a large sample of individuals diagnosed with clinical depression for 90 days (De Angel et al., 2022). This study improved upon previous research in two significant ways. First, by utilizing a clinical sample of individuals with diagnosed depression, the findings can be directly applied to understanding depression in real-world clinical settings. This enhances the external validity of the study, making it more generalizable to those with the condition. Additionally, the long 90-day follow-up window allowed for the emergence of different digital phenotypes over varying time-scales. This is important because, different features may reach their highest predictive performance over varying amounts of time (Langener et al., 2024b). Furthermore, it enables the collection of a greater amount of data over a longer period of an individual’s life, which may further enhance generalizability.

Despite the strengths of our study, some limitations warrant discussion. Although the sample is representative of the U.S. MDD population, the demographic composition, predominantly female, White, and non-Hispanic, and recruited via online advertisement may limit the generalizability of the findings. Given that we were interested in the feasibility of triggering just-in-time interventions, we chose an extending window cross-validation set-up. This approach also offered a degree of personalization, as each participant’s data was included in both the training and testing phases. One drawback, however, is that the model may not generalize well to a different sample and that the computational costs are high. Thus, for some scenarios, a different validation strategy (e.g., external validation using a completely new sample) might be more suitable, particularly when the aim is to develop a model that generalizes to independent datasets. Our outcome variable was calculated as a rolling RMSSD based on the EMA questionnaires completed during the last seven days. As described in the method section, we forward-imputed responses to the third PHQ question, which was collected only once per day as it was asking about sleep. This approach may have artificially reduced within-person variability and future research could consider removing this item as a robustness check. Another limitation is that we ignored missing EMA questionnaires, which means that if participants have a lot of missing data, the instability of symptoms might become less meaningful. However, on average 16 out of 21 questionnaires were completed per week, suggesting that this issue is limited. We acknowledge, nonetheless, that incorporating sample weights to account for differences in the number of completed questionnaires could improve the performance of the model. Missing passive data were flagged with 0.01 to capture potential signal in missingness. However, this may produce artifacts in GAF images. Alternative strategies, such as imputation or models that handle missing data, were not explored and could affect performance. Furthermore, we chose to calculate depression symptom instability based on RMSSD because it is well suited to capture moment-to-moment fluctuations, as opposed to measures like the standard deviation, which tend to reflect more gradual trends. Although autocorrelation is a valuable measure of temporal dynamics, we chose RMSSD because it specifically captures the magnitude of successive symptom changes, which better aligns with our clinical interest in moment-to-moment instability. Autocorrelation, while useful for identifying patterns, may mask clinically relevant instability if these changes occur in regular patterns. Lastly, the decision to binarize a continuous outcome does leave the potential for a loss of information; however, as the intent of this analysis in practice would be to define a clinically significant level of symptom instability that would require intervention, the decision to binarize based on a low/high threshold of depression instability is mirroring that intent. We chose to binarize depression instability by setting a threshold at one standard deviation above the mean because we believe this difference is meaningful for potential interventions. However, alternative thresholds, either higher or lower, could have been explored. Given the scarcity of research on depression instability, identifying an appropriate threshold is challenging. For future research, it would be interesting to determine which thresholds for depression instability are most meaningful for clinical utility.

To confirm that each participant was clinically depressed at the start of the study, we conducted a structured clinical interview. However, this interview was not repeated at the end of the study. Thus, it is possible that some participants may no longer have been depressed by the end. We consider this unlikely, as no significant mean differences in PHQ-9 scores were found between the first and last weeks of the study. Nonetheless, a follow-up interview would be necessary to definitively confirm this. Importantly, we view variability in symptom trajectory within the clinical group as a meaningful feature rather than a limitation, as it reflects the natural course of depressive episodes over time.

One limitation in evaluating the intraindividual predictive performance of the binary classification model was that only 76 participants exhibited both high and low instability in depressive symptoms in the test set. The remaining participants showed either consistently high or consistently low instability, which restricted our ability to assess instability across the full sample. Despite this, the key conclusions of the study remain robust as shown by additional analyses using a continuous outcome that supported the findings. Another limitation is that each participant had only 39 predictions available for assessing intra-individual variability. Consequently, the dataset may be too small to draw definitive conclusions, and a larger dataset with more time points per individual would be necessary to ensure more reliable results. Similarly, it remains an open question how large the sample size needs to be to train deep learning models effectively. It might be the case that our sample size was not large enough to adequately support the level of model complexity used in this study. In particular, it is possible that the combination of a relatively small number of participants and limited within-person observations constrained the ability of the deep learning model to fully leverage its representational capacity.

Conclusions and future directions

The present analysis sought to investigate future depression symptom instability in a clinical sample using only passively-collected movement related information. Our model did not meet the performance necessary for clinical deployment, especially within the context of predicting intraindividual variability, but provide an important benchmark for the field. This highlights the need for better methods that are able to predict intraindividual variability in depressive symptoms to enhance treatment and intervention opportunities for individuals with high instability in their depressive symptoms in real-time. Despite this, a small-to-moderate signal was observed at the between-person level, with movement data differentiating individuals with high versus low symptom instability. However, this relationship is notably weaker than when using a person’s own past instability levels as predictors, indicating that its predictive value is limited. At the within-person level, the relationship between movement and depression instability was absent. This finding may be constrained by the limited amount of within-person data, which prevents firm conclusions. A larger dataset, with more time points per individual, would be necessary to evaluate these within-person results more robustly.

Supplementary Material

Supplementary material associated with this article can be found, in the online version, at doi:10.1016/j.jadr.2026.101102.

General Scientific Summary.

In this study, we aimed to predict instability in depressive symptoms using passively collected movement data alone in individuals diagnosed with major depressive disorder. Our results suggest poor performance, specifically in predicting intraindividual variability of instability in depressive symptoms. This highlights the need for better methods that are able to predict intraindividual variability to enhance treatment and intervention opportunities for individuals with high instability in their depressive symptoms in real-time. This might be possible through the inclusion of additional types of data (such as GPS data).