Not missing at random: Missing data are associated with clinical status and trajectories in an electronic monitoring longitudinal study of bipolar disorder

Abstract

There is limited information on the association between participants’ clinical status or trajectories and missing data in electronic monitoring studies of bipolar disorder (BD). We collected self-ratings scales and sensor data in 145 adults with BD. Using a new metric, Missing Data Ratio (MDR), we assessed missing self-rating data and sensor data monitoring activity and sleep. Missing data were lowest for participants in the midst of a depressive episode, intermediate for participants with subsyndromal symptoms, and highest for participants who were euthymic. Over a mean ± SD follow-up of 246 ± 181 days, missing data remained unchanged for participants whose clinical status did not change throughout the study (i.e., those who entered the study in a depressive episode and did not improve, or those who entered the study euthymic and remained euthymic). Conversely, when participants’ clinical status changed during the study (e.g., those who entered the study euthymic and experienced the occurrence of a depressive episode), missing data for self-rating scales increased, but not for sensor data. Overall missing data were associated with participants’ clinical status and its changes, suggesting that these are not missing at random.

1. Introduction

The wide availability of low-cost wearables and smartphones enables us to measure how the environment and our experiences impact our physiology. Almost a decade ago, the promise of this approach, referred to as “digital phenotyping” was recognized (Torous et al., 2016), comparing its potential impact on psychiatry to the impact of the microscope on biological sciences (Onnela and Rauch, 2016). Based on the synthesis of performance metrics in our systematic review on e-monitoring in bipolar disorder (BD) (Ortiz et al., 2021b), we identified two challenges in its adoption and implementation: high proportion of missing data (Depp et al., 2012; Torous et al., 2020) and inadequate statistical tools to process large amounts of data (Barnett et al., 2018; Depp et al., 2017; Rohani et al., 2018).

Low adherence and high dropout rates (Nwosu et al., 2022) are common challenges in e-monitoring studies. For instance, a recent systematic review reported an average dropout rate of 48% in interventions using smartphones in major depressive disorder (MDD) (Torous et al., 2020). In an insomnia study, dropout rates were 57% in the first 6 weeks and 61% after 6 months (Christensen et al., 2016). Adherence and dropout rates in e-monitoring studies of BD have also been highly variable: Dropout rates have ranged from less than 13% over 18 months (Ortiz et al., 2023) to almost 40% over 15 months (Dominiak et al., 2022). Similarly, the rate of missing data among participants with BD who do not drop out has ranged from 0.3% to 83% for self-rating scales and 22%–89% for sensor data (Beiwinkel et al., 2016; Busk et al., 2020b; Cho et al., 2019). To address these limitations, most studies have assumed these data were missing at random, either excluding them from their analysis or using imputation based on time series stationarity (e.g., forward filling) (Busk et al., 2020a). Neither of these approaches are optimal (Ortiz et al., 2021a), and they can be problematic if missing data are informative, i.e., influenced by the participant’s clinical status and missing not at random (MNAR).

To our knowledge, no published study has investigated the effect of clinical status on missing data in e-monitoring studies of BD. In this context, we analyzed the impact of clinical status (i.e., being depressed or euthymic) and trajectories (i.e., resolution of a depressive episode or experiencing the occurrence of a new depressive episode) on rates of missing data for both self-rating scales and sensor data in participants with BD in a longitudinal study. We hypothesized that the data would be MNAR: they would be associated with clinical status and trajectories.

2. Methods

2.1. Participants

All participants included in this analysis were enrolled in an ongoing study, the details of which have been reported previously (Ortiz et al., 2022). In brief, the study is taking place at two academic hospitals in Canada: The Centre for Addiction and Mental Health (CAMH), Toronto, Ontario; and the Mood Disorders Program, Queen Elizabeth II Health Sciences Centre, Halifax, Nova Scotia. All research procedures were approved by both local REBs and were contactless (i.e., conducted virtually). Participants were referred by their treating psychiatrists and were invited to participate in the study. Inclusion criteria included: men or women, 18 years or older (with no upper limit); able to consent to participate in the study; with a primary diagnosis of BD I or II according to Diagnostic and Statistical Manual of Mental Disorders 5 (DSM-5) criteria American Psychiatric Association (2013) confirmed by the Structured Clinical Interview for DSM-5 (SCID-5) (First et al., 2015), in any phase of the illness (i.e., euthymic, depressed, (hypo)manic, or mixed). Exclusion criteria included active substance use disorder according to DSM-5/SCID-5 criteria, and a diagnosis of mood disorder secondary to a general medical condition or substance use. Participants included in this report were outpatients recruited between April 1, 2021, and August 1, 2023; who were either euthymic or in a depressive episode when they were enrolled in the study. All participants had the capacity to consent.

2.2. Traditional assessments

Baseline assessment:

after providing informed consent, participants completed a comprehensive assessment that included sociodemographic, clinical, and pharmacotherapy information. Upon entry in the study, a psychiatrist or a trained staff member administered the Young Mania Rating Scale (YMRS) (Young et al., 1978) and the Montgomery-Asberg Depression Rating Scale (MADRS) (Montgomery and Asberg, 1979) to assess the presence and severity of current symptoms. All participants were treated by a psychiatrist according to standard local practices.

Self-ratings:

every week, participants received a secure e-mailed link asking them to complete the Patient Health Questionnaire (PHQ-9) (Kroenke et al., 2001) and the Altman Self-Rating Mania Scale (ASRS) (Altman et al., 1997). Participants had to respond to all items of both self-rating scales to submit their ratings. Self-rating scale data were securely managed using REDCap (Research Electronic Data Capture) hosted at CAMH; this web-based platform provided an interface for validated data capture, manipulation, export, and interoperable integration with external sources (Harris et al., 2009). Euthymia was defined as a PHQ-9 score ≤5, subsyndromal depressive symptoms as a PHQ-9 score between 6 and 9, and a major depressive episode (MDE) as a PHQ-9 score ≥10. Only two participants in presented during a manic, hypomanic, or mixed episode (both were hypomanic) and they were excluded from this analysis.

2.3. Passive sensing

Participants were provided with an Oura smart sensor (Oura Health Oy, Generation 2, Oulu, Finland), a waterproof ring that measures activity (e.g., number of steps), sleep (e.g., total sleep duration), and cardiorespiratory sensing (e.g., heart rate). Participants were mailed a sizing kit to select their best ring size to ensure optimal skin-sensor contact and data quality. Participants wore this ring 24 h per day throughout the study duration, except for an average of 30 min of charging per week. Sensor data were securely managed using the Oura cloud platform with access restricted to our research team’s data manager.

2.4. Missing Data Ratio (MDR)

To assess the proportion of missing data points for each participant, we developed a ratio of missing data (“Missing Data Ratio”, MDR). This new metric assumes a fixed data sampling rate per data type: weekly for the PHQ-9 and every minute during the daytime or the nighttime (see Supplementary information to see how daytime and nighttime were defined for each participant based on their sensor data). The denominator of the MDR is expected data size for each participant at full adherence: a participant who is filling the PHQ-9 every week and wearing the sensor every daytime or nighttime minute, except for up to 30 min per week to charge the device, for the duration of the sampling interval (e.g., first two weeks of the study or duration of participation in the study). The numerator of the MDR is the actual data size collected for each participant: the numbers of weekly PHQ-9 filled and the number of minutes the sensor collected data during the daytime or nighttime for the duration of the sampling interval (see Supplementary information for details on the calculation of MDRs).

2.5. Statistical analysis

We conducted two separate statistical analyses, using Python version 3.1.1.For clarity, we describe them in Table 1. See Fig. 1 for a schematic representation of missing data and clinical trajectory of one participant.

Table 1.

Description of statistical analyses.

	ANALYSIS 1: MISSING DATA ACCORDING TO CLINICAL STATUS	ANALYSIS 2: MISSING DATA ACCORDING TO CLINICAL TRAJECTORIES
MDR definition	Overall proportion of missing data points from PHQ-9 and sensor data during the entire study
Objective	To analyze the association between MDRs for PHQ-9 and sensor data and participants’ clinical status throughout the study	To analyze the association between MDRs for PHQ-9 and sensor data and participants’ clinical trajectory (a change in clinical status) throughout the study
MDR: number of categories per participant	Three	Nine
MDR: types of categories	Euthymia: PHQ-9 ≤5; - Experiencing subsyndromal symptoms: PHQ-9 between 6 and 9; - In a depressive episode: PHQ-9 ≥10.	(i) euthymic throughout the study (i.e., all PHQ-9 scores ≤5); - (ii) euthymic upon entry to the study and subsequently experimenting subsyndromal symptoms (i. e., PHQ-9 between 6 and 9) for at least two consecutive weeks; - (iii) euthymic upon entry to the study and subsequently experimenting a depressive episode (i.e., PHQ-9 score ≥10) for at least two consecutive weeks; - (iv) subsyndromal symptoms throughout the study (i.e., all PHQ-9 scores between 6 and 9); - (v) subsyndromal symptoms upon entry to the study and becoming euthymic at some point during the study for at least two consecutive weeks; - (vi) subsyndromal symptoms upon entry to the study and experimenting a depressive episode at some point during the study for at least two consecutive weeks; - (vii) in a depressive episode upon throughout the study (i.e., all PHQ-9 scores ≥10); - (viii) in a depressive episode upon entry to the study and experiencing a partial remission (defined as a PHQ-9 scores between 6 and 9 for at least 2 consecutive weeks) at some point in the study; and - (ix) in a depressive episode upon entry to the study and experiencing a full remission (defined as a PHQ-9 scores ≤5 for at least 2 consecutive weeks) at some point in the study
MDR calculation when a PHQ-9 score was missing	We relied on PHQ-9 scores immediately before and after the missing PHQ-9 score: - Euthymia if both PHQ-9 scores surrounding the missing observation were ≤5; - Subsyndromal depressive symptoms if both PHQ-9 scores surrounding the missing observation were between 6 and 9; - In a depressive episode if both PHQ-9 scores surrounding the missing observation were ≥10. When the two PHQ-9 scores surrounding a missing score did not correspond to the same status, (indicating a possible transition from one status to another) the participants’ data were excluded from this first analysis.	We relied on observations based on the PHQ-9 score immediately before the missing score(s) (i.e., we assumed that no transition took place during the periods PHQ-9 scores were missing). When participants experienced several transitions during the study (e.g., a participant who was euthymic when they entered the study, experienced a depressive episode, and then remission) they were classified based on their first transition .In 4 participants, the first transition was to a manic, hypomanic, or mixed episode and these participants were excluded from the second analysis.
MDR calculation when sensor data was missing	We relied on the PHQ-9 score of the week it fell within
To summarize the data, we used:	Median and interquartile range (IQR) for participant-specific MDRs of both PHQ-9 and sensor data
To compare MDRs:	We compared the MDRs in We compared the MDRs in pairwise fashion for the three pairwise fashion for the nine status groups used a Mann-trajectory groups used a Whitney test and applied the Mann-Whitney test and Benjamini/Hochberg non-applied the Benjamini/negative false discovery rate Hochberg non-negative false (BH-FDR) (Benjamini and discovery rate (BH-FDR) (Hochberg, 1995). Benjamini and Hochberg, 1995).	We compared the MDRs in pairwise fashion for the nine trajectory groups used a Mann-Whitney test and applied the Benjamini/Hochberg non-negative false discovery rate (BH-FDR) (Benjamini and Hochberg, 1995).

Fig. 1.

Schematic representation of missing data and clinical trajectory for one participant.

We also conducted statistical analysis to examine potential confounding demographic factors that could account for differences in MDR: age, sex, and education. Specifically, we conducted an analysis of variance (ANOVA) for categorical variables (i.e., sex and education) and a linear regression analysis for age.

3. Results

After excluding two participants who were hypomanic when they entered the study, 145 participants were followed for a mean ± SD of 246 ± 181 days. Of these 145 participants, 46 (31.7%) entered the study euthymic, 33 (22.8%) with subsyndromal symptoms, and 66 (45.5%) in a depressive episode See Table 2 for their sociodemographic and clinical characteristics.

Table 2.

Sociodemographic and clinical characteristics of the sample (N = 145).

Characteristic	Participants
Age, mean (SD)	38.1 (11.9)
Sex assigned at birth, n (%)
Male	53 (36.6)
Female	92 (63.4)
Gender, n (%)
Man	48 (33.1)
Woman	77 (53.1)
Queer/gender non-conforming	4 (2.8)
Prefer not to disclose	16 (11)
Education, n (%)
Completed high school or less	28 (19.3)
College/diploma	75 (51.7)
University education	42 (29.0)
Marital status, n (%)
Single	74 (51.0)
Married	53 (36.6)
Divorced	16 (11.0)
Widowed	2 (1.4)
Socioeconomic status, n (%)
Work full-time	76 (52.4)
Work part-time	15 (10.3)
Homemaker	1 (0.7)
Social assistance or disabled	17 (11.7)
Retired	3 (2.1)
Student	10 (6.9)
Unemployed and othera	23 (15.9)
Primary Diagnosis, n (%)
Bipolar Disorder I	94 (64.8)
Bipolar Disorder II	51 (35.2)
Rapid cycling, n (%)	30 (20.7)
Clinical Status at study entry, n (%)
Euthymic	46 (31.7)
Subsyndromal symptoms	33 (22.8)
In a depressive episode	66 (45.5)
Pharmacotherapy at the time of entry to the study
On no treatment at the time of entry	6 (4.1)
Lithium monotherapy	7 (4.8)
Anticonvulsant monotherapy	16 (11.0)
Antipsychotic monotherapy	14 (9.7)
Combination treatment	95 (65.5)
Other	7 (4.8)

^aFor example, between two jobs.

Our first analysis (MDRs for the three groups based on clinical status) showed that MDRs both for PHQ-9 scores and sensor data were highest (i.e., reflecting more missing data) during euthymia. See Table 2 and Fig. 2 for details.

Fig. 2.

Missing data ratios based on clinical status upon entry to the study for a) PHQ-9 scores, b) daytime sensor data, and c) nighttime sensor data ns = non-significant; ***: p < 0.001; *: p < 0.05.

Our second analysis (MDRs for nine groups based on clinical trajectories) showed that participants who did not experience a transition had lower MDRs than those who transitioned to subsyndromal symptoms or a full depressive episode. Participants who entered and remained euthymic throughout the study showed the lowest levels of missing data for the PHQ-9 scores than both those who transitioned to subsyndromal symptoms (U = 103.0, p < 0.05) and a full depressive episode (U = 69.0, p < 0.001). However, participants who remained euthymic throughout the study showed significantly higher level of missing nighttime sensor data than those who transitioned to subsyndromal symptoms (U = 115.0, p < 0.05).

Only two participants entered and remained subsyndromal throughout the study course. Participants who entered the study exhibiting subsyndromal symptoms and reached euthymia showed the highest levels of missing data for PHQ-9 across all groups (MDR: 0.19 (0.33)), p < 0.001). There were no significant differences for missing sensor data.

Participants who entered the study in a depressive episode and did not improve, showed the lowest MDR (i.e., reflecting fewer missing data) for nighttime sensor data (p > 0.05). Participants who responded to treatment showed the lowest MDR for daytime sensor data (p > 0.05). See Table 3 and Fig. 3 for details. Finally, our results showed that there were no statistically significant differences that could explain the differences in MDR for sex (F = 0.23; p = 0.79); education (F = 3.05 (p = 0.38), or age (β = −0.0003, p = 0.72), suggesting that none of these variables were a significant predictor of MDRs in our study sample.

Table 3.

MDRs for the nine trajectory groups (Median (IQR).

Trajectory	n (%)	PHQ-9 (S)	Daytime (D) Sensor Data	Nighttime (N) Sensor Data	Significant Pairwise Comparisons Higher missing data in:
Euthymic throughout	17 (11.7)	0.03 (0.16)	0.05 (0.15)	0.13 (0.14)	N > D; N > S
Euthymic to subsyndromal	16 (11.0)	0.08 (0.27)	0.01 (0.13)	0.11 (0.10)	S > D; N > S
Euthymic to depressive episode	13 (9.0)	0.18 (0.30)	0.03 (0.11)	0.13 (0.14)	S > D; N > D
Subsyndromal throughout	2 (1.4)	0.38 (−)	0.02 (−)	0.33 (0.11)	N/A
Subsyndromal to euthymic	7 (4.8)	0.19 (0.33)	0.04 (0.06)	0.20 (0.31)	S > D; N > D
Subsyndromal to depressive episode	24 (16.6)	0.08 (0.16)	0.05 (0.25)	0.17 (0.27)	N > D
Depressive episode throughout	17 (11.7)	0.06 (0.15)	0.01 (0.31)	0.08 (0.34)	S > D; N > D
Depressive episode with response	15 (10.3)	0.18 (0.19)	0 (0.02)	0.13 (0.30)	S > D; N > D
Depressive episode with remission	34 (23.4)	0.10 (0.24)	0.05 (0.17)	0.10 (0.29)	S > D; N > D

IQR: Interquartile range; MDE: Major depressive episode; MDRs: Missing data ratios.

Fig. 3.

Missing Data Ratio box plots based on clinical trajectory for a) PHQ-9 scores, b) daytime sensor data, and c) nighttime sensor data EU: Euthymic; SS: Subsyndromal; MDE: Major depressive episode.

The SS-SS group is not shown in the figure because only two participants were in this trajectory.

4. Discussion

To our knowledge, this is the first study investigating the association between clinical status and trajectories and missing data in e-monitoring studies in BD. Our results show that data are not missing at random and that they add information relevant to clinical outcomes.

The association between missing data and clinical status showed that participants who were euthymic had lower levels of missing data when they developed subsyndromal symptoms, reaching their lowest missing data levels upon relapse. Our findings indicate that clinical status has a major effect on missing data levels.

The association between missing data and clinical trajectories showed that participants who remained in a depressive episode throughout the study (i.e., they did not experience response or remission) showed the lowest levels of missing data. Conversely, those who were in a depressive episode upon entry to the study, but experienced response or remission, became less engaged and exhibited higher levels of missing data.

Our findings also showed that clinical status and trajectories affect different data modalities differently: both clinical status and trajectories had a smaller effect on wearing the sensor than to completing PHQ-9 (i.e., participants tend to wear the sensor more often than filling out the scales, independently of their clinical status). This is relevant, as activity and sleep changes are earlier indicators of change in clinical state than mood (Ortiz et al., personal communication).

While some studies have reported no association between adherence to mobile interventions and baseline symptoms (Depp et al., 2015), our findings are congruent with previous studies showing a correlation between illness burden and lower rates of missing data (i.e., higher engagement) in e-monitoring studies (Ortiz et al., 2023): in a study describing phenotypes of engagement with wearables for heart rhythm monitoring (Lee et al., 2021), more atrial fibrillation episodes were associated with higher engagement with the device and lower missing data. Similarly, in a study in patients with chronic pain (Ross et al., 2020), patients with higher pain intensity and more disability reported higher engagement.

Limitations of this study include a relatively small sample size and a wide range in length of follow-up. While the analysis accounted for data pattern shifts and unequal data lengths, short time series recorded for newly enrolled participants do not provide enough data for statistical generalization. However, we accounted for the range in length of follow-up by using weighted metrics, so the effect of insufficiently long data was insignificant. Participants in this study were referred from two outpatient clinical settings specialized in BD; they all consented to participate in a research study of e-monitoring. Thus, one could expect lower motivation and more missing data in a broader population of patients with BD, limiting the ability to generalize our findings, as would be the case with every research study. Despite these limitations, our analysis introduces a new metric to quantify missing data in e-monitoring studies and shows that missing data are informative.

5. Conclusions

Missing data patterns were associated with clinical status and trajectories, particularly for PHQ-9 and to a lesser extent to sensors. As e-monitoring is being increasingly implemented in mental health studies, the complex relationship between participants’ clinical status and trajectories and their level of engagement must be accounted for in all statistical analyses. We believe that our newly developed metric “MDR” (Missing Data Ratio) can be a useful tool to quantify missing data and incorporate it in statistical analyses. Moreover, future studies could assess the effect of tailored strategies to improve engagement in participants who are transitioning between phases (i.e., from subsyndromal symptoms to euthymia) to improve adherence with data collection.