Did you miss me? Making the most of digital phenotyping data by imputing missingness with point process models: observational study

Abstract

Objectives

Smartphone-based digital phenotyping can provide low-burden behavioural measures for mental disorder monitoring. However, progress in making inferences from these data is challenged by the common occurrence of missing data. We propose a method to impute missingness using non-homogeneous Poisson point process models (PPPMs), where activities (overall phone, social media, communication app usage, outgoing/incoming calls) are modelled as ‘points’.

Methods

We evaluate personalised PPPMs for imputation and investigate their influence on downstream analysis. In a ground truth evaluation (in and out-of-sample), we evaluate time-varying covariates (‘hour of the day’, ‘day of the week’; encoded using one-hot encoding and sine–cosine transformation) to model behavioural patterns in participants from SMARD (depression; n=26). We train a hidden Markov model (HMM) on data simulated by the PPPMs and compare this to a ground truth HMM. We then perform a replication of a prior HMM analysis in PRISM (Alzheimer’s disease, schizophrenia, healthy controls; n=65) and Hersenonderzoek studies (Alzheimer’s disease, memory complaints, healthy controls; n=283).

Results

In the ground truth evaluation, ‘hour’ was consistently significant in in-sample likelihood ratio tests and ‘day’ was less commonly significant. PPPMs including one-hot encoded hour generally provided the highest out-of-sample likelihood. Using this PPPM variant, HMM properties were preserved, and prior findings were replicated.

Discussion

Personalised PPPMs provide behavioural simulations that can be used for temporal imputation. These models capture average patterns and could be extended to include further temporal components.

Conclusion

Non-homogeneous PPPMs are a promising imputation tool that may contribute to improved utility of digital phenotyping by providing realistic temporal imputations.

WHAT IS ALREADY KNOWN ON THIS TOPIC

• Digital phenotyping provides ongoing behavioural monitoring for applications such as symptom tracking and relapse prediction but is susceptible to missing data. So far there have been limited investigations of appropriate methods for imputing missing values, limiting the usability of digital phenotyping in the clinic.

WHAT THIS STUDY ADDS

• We used personalised Poisson point process models (PPPMs) to impute missing values by modelling activities as points with an associated probability of occurring. We identified that PPPMs using the hour of the day as a covariate generally served as the most appropriate models for imputation.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

• Improved imputation of missing periods using PPPMs can be used to increase the statistical power of digital phenotyping studies and broaden the analysis methods available, leading to improved quality of methods available for clinical practice.

Introduction

Digital phenotyping is a promising technique to quantify behaviour or physiology from digital devices, for example, smartphones, enabling the passive monitoring of patients in a naturalistic environment. Key goals include objective behavioural measurement and prediction of relapse in mental illness. Several digital phenotyping apps exist, including ‘Behapp’¹ and ‘RADAR-base pRMT’,² which can measure a range of different behaviours over time, such as app usage, movement/location, calls and steps. An emerging body of literature has produced promising digital phenotyping results, including demonstrating correlations between digital phenotyping features and depression symptoms,^{3 4} and age and diurnal effects.⁵

However, in digital phenotyping, missing data have become a major hurdle.⁶ Missingness may arise from user behaviour, eg, flight mode or technical issues, eg, battery management. In a digital phenotyping review in major depression (MDD),⁶ 20 out of 24 articles referred to missing data. Missing samples were often excluded from analysis, with 10 studies using an exclusion threshold and three studies excluding samples if any variable was missing. However, this can introduce bias⁷ and lead to extreme data loss.⁸ Five studies carried out imputation, including using the mean from preceding data,⁹ applying a resampling method to global positioning system (GPS) data,¹⁰ using linear interpolation on Bluetooth data¹¹ and imputing missing records using similar records.¹²

Imputation is useful as it increases the number of usable samples, therefore increasing the power of statistical analyses and broadening the number of methods available, as researchers are not limited to methods that can handle missingness. However, timeseries imputation is challenging due to temporal dependencies. Often, people follow diurnal rhythms where their behaviour varies with daily cycles (eg, sleep-wake cycle), and ultradian rhythms, where behaviour varies throughout the day (eg, working hours vs breaks). Digital phenotyping imputation needs to capture these variations but is challenging as missingness can occur in long chunks due to technical difficulties. Popular approaches such as using the mode/mean⁹ or last-observation-carried-forward result in the same value being used across the chunk, obscuring behavioural fluctuations. Linear interpolation¹¹ also does not capture these rhythms. Furthermore, missingness often occurs across all variables at the same timepoints. Methods relying on relationships between variables, including multivariate imputation by chained equations,¹³ implementations of support vector machines,⁷ k-nearest neighbours^{7 12 14} and some tailored regression methods,¹⁵ are therefore unsuitable. Some methods are application-specific (eg, GPS,¹⁰ accelerometer¹⁶) or designed for labelled datasets.¹⁷ Additionally, many smartphone-based behaviours follow a zero-inflated distribution.¹⁸ Gaussian assumptions used by some methods (eg, Gaussian processes¹⁹) therefore do not hold, and modifications for modelling these data are needed. In our prior work, we binned hourly activity percentages into activity/no activity bins to facilitate modelling using a binomial distribution.¹⁸

To address these challenges, we aimed to investigate Poisson point process models (PPPMs) as a method to impute missing digital phenotyping data occurring in phone use-related activity channels (including app usage, calls¹⁸). PPPMs model the probability of ‘points’, in our case activities, occurring. While PPPMs do not explicitly model temporal dependence, time-varying covariates can be included to introduce temporal information to the models. PPPMs can therefore be used to impute activities occurring in blocks of missingness appropriate to each timepoint, without relying on information from external channels. We evaluated the imputation of phone usage activities collected using Behapp²⁰ (with our imputation method and validation using downstream analysis outlined in figure 1), although this approach is suitable for any platform. First, we investigated PPPM performance against ground truth data from participants in the Smartphone-based Monitoring and cognition modification Against Recurrence of Depression (SMARD) dataset.²¹ Next, we validated the use of PPPMs for imputation in real missingness from the Psychiatric Ratings using Intermediate Stratified Markers (PRISM)²² and Hersenonderzoek²³ datasets (figure 1E).

Figure 1.

Overview of the imputation method, where the same example time series is shown in (A)–(C). (A) Digital phenotyping can provide measures of behaviour over long time periods, but is susceptible to missingness. (B) Methods like last-value-carried-forward can be used to impute missingness but provide unrealistic values. (C) Poisson point process models (PPPMs) are more suitable for modelling behavioural rhythms when including the hour of the day as a time-varying covariate, providing more realistic imputed values. The PPPM also showed promising results when used in downstream analysis: (D) When compared with a hidden Markov model (HMM) trained on real data, a HMM trained on data simulated by a PPPM learnt comparable model parameters, suggesting the PPPM method preserved properties of the data. (E) A HMM trained on real data where real cases of missingness were imputed using PPPMs remained very similar to the HMM from the original study without imputation, with previous statistical results replicated. AD, Alzheimer’s disease; HCs, healthy controls; SCC, subjective cognitive complaints.

Methods

Datasets

Participants from the SMARD dataset²¹ in the Netherlands (NL) were used for part 1 of this study. SMARD aims to investigate depression in recurrent MDD. Participants were 18–65 years, diagnosed with MDD and classified as in remission. Inclusion criteria are provided in Ikani et al.²¹ Behapp collection lasted for 1.5 years/participant. A subset of participants with excellent data availability (≤1% missingness) were selected to ensure sufficient ground truth data were available for evaluation. Data collection took place between August 2019 and March 2023.

Part 2 used data from the PRISM²⁴ and Hersenonderzoek²³ studies, which collected Behapp data for 42 days/participant. Study details are provided in the original analysis.¹⁸ Data collection took place between August 2017-May 2019 for PRISM and March 2018-January 2020 for Hersenonderzoek. Both datasets contained participants with probable Alzheimer’s disease (AD) and age-matched healthy controls (HCs), PRISM contained participants with schizophrenia (SZ) and Hersenonderzoek contained participants with subjective cognitive complaints (SCC). PRISM investigates social withdrawal in SZ and probable AD.^{25 26} Recruitment took place in NL and Spain. Participants with SZ were 18–45 years, and participants with probable AD were 50–80 years. In Hersenonderzoek, participants in NL with age 45+ years self-reported probable AD, or an absence of neurological or psychiatric diseases with/without memory complaints (SCC and HCs, respectively).²³

Behapp acquisition and smartphone channels

‘Behapp’ was installed on each participant’s smartphone.²⁰ Social media app, communication app and overall phone usage activities were recorded for all studies. Outgoing/incoming calls were recorded for PRISM and Hersenonderzoek. Activity start and end times were recorded. Activities were split into hourly bins, indicating whether an activity was carried out each hour, with missing hours identified (see the supplement and Leaning et al¹⁸ for details). Missingness was seen to occur for a range of time windows, with some missingness very brief (lasting a matter of hours) and other missingness cases being very prolonged (lasting for consecutive days/weeks). Missingness occurred across all channels concurrently.

Poisson point process models

A Poisson distribution models the probability of a point occurring within a given domain, for example, the probability of app usage across time. We treated homogeneous PPPMs, which model a constant probability, as our baseline. We evaluated non-homogeneous PPPMs against this baseline, where the probability of points occurring varies according to the values of covariates. Note that the probability is determined by the covariates, and not by the occurrence of other points. We used the ‘NHPoisson’ R package (V.3.3)²⁷ for training, covariate selection and simulation. Full details are provided in online supplemental material.

Model development: ground truth-based investigations

For each participant, we fit homogeneous and non-homogeneous PPPMs where ‘hour of the day’ and ‘day of the week’ were used as covariates to model behavioural rhythms. We investigated two covariate encoding methods: one-hot encoding (each value is encoded using dummy variables) and sine–cosine transformation (each value is transformed using sine and cosine functions). Each encoding approach has different implications for the ordering of covariate values: one-hot encoding does not imply ordering (eg, midday is as far away from midnight as it is from 13:00), whereas sine–cosine transformation provides cyclical ordering (so midday is closer to 13:00 than midnight).

We used 10-fold cross-validation to investigate PPPM performance, where for each participant at each iteration, their withheld fold was considered ‘missing’. PPPMs were trained for each available channel (overall phone usage, communication app usage, social media app usage).

For each channel, likelihood ratio tests (LRTs) were run for each model, with comparisons made between the homogeneous model and models including one of: one-hot encoded hour, sine–cosine transformed hour, one-hot encoded day, sine–cosine transformed day. The p-value from the LRT was used to assess whether the covariate model provided a significantly better fit (p<0.05). For participants where the day covariate was significant for ≥1 fold (considering each encoding method), models were trained including both day and hour, with LRTs carried out for this model compared with the hour-only model, to verify whether the day was contributing significantly more information than hour alone.

The accuracy of the models was evaluated by calculating the out-of-sample Poisson log-likelihood of the datapoints in each withheld fold, given the corresponding model trained on the other nine folds and calculating the mean across folds per participant for each PPPM type. The models were used to simulate events occurring during the ‘missing periods’, by simulating point processes during a period matching the withheld fold in terms of length and covariate values using NHPoisson’s ‘simNHP’ function. The simulated time series could then be compared with the real values.

After selecting the best-performing PPPM type, the simulated time series for each channel were concatenated and used to train hidden Markov models (HMMs) following the method in Leaning et al¹⁸ using ‘depmixS4’,²⁸ with one-hot encoded hour included as a covariate on the HMM transition probabilities, to investigate the effect of imputation on downstream analysis. We considered ‘unconstrained’ and ‘constrained’ imputation cases (see supplement for details) and compared these HMMs to a HMM trained on the ground truth.

Clinical validation: imputation application to real missingness

We applied the selected method to data from PRISM²⁴ and Hersenonderzoek²³ used in a prior study (where missingness was not imputed)¹⁸ (figure 1E). We sought to replicate the hidden states identified in Leaning et al,¹⁸ training the HMM on an updated time series where missingness was imputed using PPPMs, and to evaluate whether we would still identify significantly lower ‘total dwell time’ (ie, percentage of time spent in the identified ‘socially active’ hidden state) in participants with AD or SCC compared with HCs. This paper involved various HMMs, and we compare to the 2-state HMM trained on the entire dataset. We followed the analysis steps exactly as in the original paper.¹⁸

In the current study, we used the available data per participant to train an individualised non-homogeneous PPPM with one-hot encoded hour as a covariate. For participants whose model failed to converge, the ‘optim’ function with the ‘BFGS’ method was used in place of the default function. For participants who had a channel with <3 activity points, we did not carry out imputation for that channel due to the limited amount of activity data. This threshold was arbitrarily selected as for some channels datapoints could be very sparse. Additionally, if a participant had more than 85% of their time series missing no imputation was carried out as missingness was deemed too high for a representative model to be learnt. Where missingness occurred, we inserted the values simulated by each personalised PPPM, so that we could compare the original HMM to an HMM trained on data containing imputations.

To calculate the total dwell time per participant, missing periods were dropped in the original analysis after hidden state sequence generation, and we keep this step for consistency as we are most interested in ensuring that the updated HMM itself is comparable to the original. As in the prior study, the total dwell times (and age covariate) were used to predict diagnostic group (SZ, AD, SCC) using logistic regression, with HCs as the reference. We present false discovery rate (FDR)-corrected p-values, considering three tests, with results considered significant at p<0.05.

Results

Sample statistics

Demographics are provided in table 1. SMARD participants had experienced a mean of 4.4 (±2.5) depressive episodes at inclusion. Their mean timeseries length was 213 (±214) days. For PRISM, the mean length was 34 (±13) days and for Hersenonderzoek, 35 (±12) days. Further details are provided in Leaning et al.¹⁸

Table 1.

Demographics across diagnostic groups

Diagnostic group	Number	Dataset	Age (mean (SD))	Sex	Country
Major depression	26	SMARD	51 (14)	F=22; M=4	NL=26
Healthy control	247	PRISM=28; Hersenonderzoek=219	59 (13)	F=140; M=107	NL=234; ES=13
Schizophrenia	18	PRISM	31 (6)	F=7; M=11	NL=12; ES=6
Alzheimer’s disease	26	PRISM=19; Hersenonderzoek=7	67 (7)	F=10; M=16	NL=18; ES=8
Subjective cognitive complaints	57	Hersenonderzoek	61 (7)	F=36; M=21	NL=57

ES, Spain; F, female; M, male; NL, the Netherlands.

Ground truth-based investigations

The percentage of models for which each covariate was significant in LRTs is provided in online supplemental table S1. For almost all models, hour was significant. The day covariate was less commonly significant, with the maximum percentage of models for which this was significant at 30% (communication app). For models containing both day and hour, day generally remained significant (online supplemental table S2).

The mean out-of-sample Poisson log-likelihoods are shown in figure 2 (numerical values provided in online supplemental tables S3–S5). The log-likelihoods for PPPMs including hour were higher than for homogeneous PPPMs, indicating a better fit. PPPMs including one-hot encoded hour were also generally higher than sine–cosine transformation. Two participants appeared not to follow this pattern. Participant 3 appears to show similar performance across most models for phone usage, however this is driven by the poor performance of their combined hour+day (one-hot) PPPM. This model type failed for most of their folds, and so the low value reflects the log-likelihood of a single fold (which was of similar magnitude to the other model types for this specific fold). Participant 16, despite meeting the data quality inclusion criteria, had a short study participation (8 days), informally demonstrating a limit to the minimum data required to train a PPPM.

Figure 2.

Mean out-of-sample log-likelihoods for (A) overall phone usage, (B) communication and (C) social media apps. Each row corresponds to one participant. The darkest tile gives the highest mean log-likelihood for a participant.

In combined hour+day models (for the subset of participants for whom day was significant), the log-likelihood rarely showed an improvement on hour-only models: models including both covariates provided the highest log-likelihood for 0 out of 7 participants for phone usage, 5 out of 13 participants for communication app (with 2 of these negligibly higher), and 2 out of 14 participants for social media app (with 1 of these negligibly higher). We therefore chose PPPMs including one-hot encoded hour as our imputation approach for the remaining analyses.

We present simulated visualisations alongside real data in figure 3 and online supplemental figures S1, S2. Figure 3A and B displays participants with clear diurnal rhythms, and 3C shows a participant with elevated nighttime activity. PPPMs including hour provided improved imputation when compared with homogeneous PPPMs, capturing personal diurnal behaviours.

Figure 3.

Real versus imputed time series for the phone usage channel for three participants (A, B and C). Each row corresponds to a single fold from each participant. The left column shows the fold’s original timeseries, the centre shows the simulation provided by the PPPM using one-hot encoded hour, and the right shows the corresponding homogeneous PPPM. For each participant, the hour model captured individual behaviour more accurately than the homogeneous model. PPPM, Poisson point process model.

The simulated SMARD data were concatenated and used to train HMMs, which were compared with an HMM trained entirely on the ground truth (figure 1D). The ground truth HMM has one state with low probabilities of activity across channels, and another state with a high probability of phone usage, high probability of communication app usage and lower probability of social media app usage (online supplemental figure S3A). The HMM trained on unconstrained imputed time series (online supplemental figure S3B) also shows low activity and high activity states, however the high activity state here gives lower probabilities than the ground truth, with phone usage still the most probable channel. Once the imputed time series is constrained (online supplemental figure S3C), the probability of phone usage is more comparable to the ground truth HMM.

For all HMMs, the probability of transitioning into the high activity state, from either state, rapidly increases during the morning, is high throughout the day and decreases in the evening, though varies in magnitude (online supplemental figure S4). For both HMMs trained on imputed data, there was a 1.0 probability of starting the timeseries in the inactive state, whereas for the ground truth HMM this probability was 0.44.

Imputation application to real missingness

After applying the method to missingness in PRISM and Hersenonderzoek (figure 1E; see online supplemental figure S5 for original data availability), the percentage of missing datapoints decreased from 30% overall to 4% for phone usage, 5% for communication app usage, 13% for social media app usage, 7% for incoming calls and 6% for outgoing calls, with the remaining missingness from participants where imputation was deemed not reasonable. Example real timeseries and their corresponding simulations are shown in figure 4. We trained a two-state HMM using the updated timeseries (with missingness imputed).

Figure 4.

Example real timeseries from three participants with their corresponding simulations, used to impute the missing periods. (A) A participant with very low missingness. The simulated timeseries show similar patterns, most obvious in the phone usage channel, but underestimate the number of points; (B) a participant with notable missingness; (C) a participant with moderate missingness.

The emission probabilities from the HMM containing imputation are extremely similar to those from the original HMM, reflecting ‘socially active’ and ‘socially inactive’ states (online supplemental figure S6). The transition probabilities show the same pattern of increased probability of transitioning into the socially active state during the daytime (see online supplemental figure S7).

We successfully replicated the prior logistic regression results, with time spent socially active significantly lower in the AD (FDR-corrected p<0.001) and SCC (FDR-corrected p=0.01) groups relative to HCs (online supplemental figure S8 and online supplemental tables S6, S7). These results were seen for constrained and unconstrained imputation.

Discussion

We demonstrated the utility of PPPMs for imputing missing smartphone activity data. In our ground-truth investigations, we investigated homogeneous PPPMs and PPPMs including covariates. The PPPMs including one-hot encoded hour generally provided the best models out-of-sample, suggesting that accounting for diurnal variation is important. When sine–cosine transformed hour was used, PPPMs overestimated nighttime activity and underestimated daytime activity, suggesting a flattening of the oscillatory behaviour. However, sine–cosine models more reliably converged. HMMs trained on imputed data learnt comparable model parameters to HMMs trained on original data, recovering properties of the original data.

We investigated the influence of our method on a prior analysis using HMMs to investigate group differences between clinical populations and HCs. When imputation was applied, the updated HMM showed minimal differences in emission and transition probabilities compared with the original HMM. We subsequently demonstrated significantly lower time spent socially active for participants with AD and SCC compared with HCs, as in the previous study.

Our work has several limitations. We treated a homogeneous PPPM as our baseline, however it would have been preferable to be able to use an established baseline. Other methods had obvious limitations, using the same value to impute the entire missing chunk (figure 1B), or specific reasons for not being selected as a baseline (eg, relying on relationships between channels for imputation, or being unable to capture behavioural rhythms; see the Introduction section for examples). Our approach does not account for covariation between behavioural channels; future research could investigate imputation using multivariate PPPMs. While our datasets were prone to chunks of missingness, in some participants, missingness occurs periodically (see figure 4C). In these cases, further refinement may be needed, for example, including parameters from similar participants²⁹ without periodic missingness. As our datasets include clinical populations, it is possible their timeseries include behavioural changes attributable to symptoms. It could be evaluated whether to use all data for imputation or selected periods.

PPPMs capture temporal dependence via covariates, and so activity estimates reflect averages for each covariate value. Contrastingly, HMMs base estimation of the hidden state value at time t off the value at t-1. The HMM, therefore, must infer this temporal dependency from the PPPM estimates reflecting average activities. For models with a larger number of hidden states, it may be difficult for the HMM to learn temporal dependence between the hidden states.

We selected the best-performing PPPM across participants for imputation. However, future work could carry out individualised covariate selection. Additional covariates that may affect behaviour, for example, weather or long-term factors, including seasonality, could be explored. Consistent with our prior work, we imputed values indicating activity or inactivity. We applied this binning due to the zero-inflated distribution of the data.¹⁸ However, the suitability of PPPMs for predicting activities with a continuous value could be investigated, for example, using marked PPPMs to impute the percentage of an hour spent carrying out an activity. Furthermore, PPPM imputation could move beyond first-order models and include higher order components. In our ground truth HMM investigations, we compared real data to an extreme missingness case where all data were ‘imputed’. Additionally, our ground truth investigation was carried out in a sample with a longer study duration than in the replication analysis. Future work should seek to investigate the limits to how much data can realistically be imputed using PPPMs and with what minimum length of training data, with the understanding that imputation should not be the sole method for dealing with missing data but used alongside methods that actively work to reduce missingness.³⁰

Our method imputes learnt behavioural averages and so can impute average clustered behaviours (eg, activity is more likely to occur during daytime) and average windows of persistent inactivity (eg, nighttime), but not sporadic clusters. Future research could evaluate additional temporal dependencies, including those captured by history-dependent terms (eg, time since last activity) or self-excitatory processes (eg, Hawkes). However, it may be necessary to incorporate inhibitory processes to prevent ongoing prediction during inappropriate periods (eg, sleep).

Conclusions

Digital phenotyping has clinical potential but is prone to missingness. We investigated non-homogeneous PPPMs for imputing missing temporal activity. Non-homogeneous PPPMs using one-hot encoded hour provided the highest out-of-sample Poisson likelihood, so were selected for imputation. Using this method, we replicated results from a previous study. Non-homogeneous PPPMs for imputation are a promising avenue for improving the analysis and utility of digital phenotyping.

Data availability statement

Data are available upon reasonable request.

Ethics statements

Patient consent for publication

Not applicable.

Ethics approval

SMARD was approved by the Radboudumc ethics review board (2016-3009; NL60033.091.16). PRISM was approved by the Ethical Review Board University Medical Centre of Utrecht (17-021/D) for the Dutch research centres, and the Comité Ético de Investigación Clínica Hospital General Universitario Gregorio Marañón (59359) for the Spanish research centres. Hersenonderzoek was approved by the Ethical Review Board VU University Medical Centre (2017.254). Participants gave informed consent to participate in the respective study before taking part.