Recurrent Neural Networks in Mobile Sampling and Intervention

Abstract

The rapid rise and now widespread distribution of handheld and wearable devices, such as smartphones, fitness trackers, or smartwatches, has opened a new universe of possibilities for monitoring emotion and cognition in everyday-life context, and for applying experience- and context-specific interventions in psychosis. These devices are equipped with multiple sensors, recording channels, and app-based opportunities for assessment using experience sampling methodology (ESM), which enables to collect vast amounts of temporally highly resolved and ecologically valid personal data from various domains in daily life. In psychosis, this allows to elucidate intermediate and clinical phenotypes, psychological processes and mechanisms, and their interplay with socioenvironmental factors, as well as to evaluate the effects of treatments for psychosis on important clinical and social outcomes. Although these data offer immense opportunities, they also pose tremendous challenges for data analysis. These challenges include the sheer amount of time series data generated and the many different data modalities and their specific properties and sampling rates. After a brief review of studies and approaches to ESM and ecological momentary interventions in psychosis, we will discuss recurrent neural networks (RNNs) as a powerful statistical machine learning approach for time series analysis and prediction in this context. RNNs can be trained on multiple data modalities simultaneously to learn a dynamical model that could be used to forecast individual trajectories and schedule online feedback and intervention accordingly. Future research using this approach is likely going to offer new avenues to further our understanding and treatments of psychosis.

Introduction

In recent years, the use of behavioral, physiological, and other digital data collected in the context of daily life using wearable technologies to improve understanding of various mental health outcomes has received increasing attention. Recent work applying machine learning methods to experience sampling methodology (ESM) data showed that patterns differentiating patients with psychosis spectrum disorder from controls could be recognized with up to 82% accuracy.¹ This article explores the potential of (deep) recurrent neural networks (RNNs), a machine learning method, which has been successfully applied for many types of time series data with sequential structure, like language and text processing² or motion data,³ for harnessing digital data from wearable devices to further our understanding and optimizing treatment of psychosis.

Understanding Psychosis in Context

Over the past decade, there has been a growing number of studies using ESM⁴ (or synonymously, ecological momentary assessment^5–7) to investigate the phenomenology, onset, course, and outcome of psychosis in daily life, outside the research laboratory. ESM is a structured diary technique that allows us to assess variation in thoughts, feelings, and behaviors from one moment to the next using repeated, naturalistic sampling several times a day over a number of (often consecutive) days.^4–7 Data collected using the ESM are now often referred to as active or explicit data, more generally,⁸ and there is a long tradition of collecting data other than self-report requiring active participation of the participant in experience sampling research such as taking cortisol samples.⁹ ESM is more than just a data collection method. As pointed out in a recent review by Myin-Germeys et al,⁷ ESM has its origins in ecological psychology and emerged from the idea that experience and behavior are situated in context and vary over time, and thus, should always be assessed and investigated in relation to this context, and in the moment.⁷ This is nowadays typically achieved through using mobile devices, primarily applications on smartphones, and, although a number of methodological challenges and ethical issues remain,⁷ over 2 decades of experience sampling research in psychosis^10,11 make ESM the most widely used and methodologically grounded mobile Health (mHealth) assessment method in the field. The most promising use of ESM in psychosis research has been in identifying intermediate and clinical phenotypes, investigating psychological processes and mechanisms (including cognitive variables using experimental ESM tasks¹²), and studying the interplay with socioenvironmental contexts in daily life, as well as evaluating treatment effects for psychosis on important clinical and social outcomes.

Further, although ESM has long entailed collecting data in the context of daily life requiring no active participation of the participant (eg, data on time, allowing to examine timing and calculate time budgets), triangulation with additional, so-called passive or implicit data has become more common. This includes the use of GPS tracking (as another proxy for context), accelerometers, physiological sensors, keyboard interaction (on mobile devices), and data from other smartphone applications (eg, social media) and wearable technologies. This offers an opportunity to elucidate more fully how experience and behavior of people with psychosis interact with socioenvironmental context, physiological parameters, and other proxies over time using data from various modalities. This may help to approximate even better ecological psychology’s central posit of experience and behavior being situated in context.

Delivering Treatments for Psychosis in Context

Another recent development has extended the principles of ESM to the delivery of treatments for psychosis using mobile devices as part of what is now commonly referred to as ecological momentary interventions (EMIs).^5,13,14 EMIs are mHealth interventions that extend beyond previous ESM research by assuming that, if experience and behavior are situated in context and vary over time, they are best targeted and most amenable to change in a given moment and context.¹³ More generally speaking, EMIs therefore are consistent with, and broaden the scope of, community mental health service delivery models, as they aim to translate treatments for psychosis beyond clinical settings to patients’ daily life. Examples of EMIs for psychosis are FOCUS, an automated intervention to provide illness management support for psychosis;^15,16 Mobile Assessment and Treatment for Schizophrenia, a text-messaging approach to targeting maladaptive beliefs corresponding to outcomes of socialization, medication adherence, and voices in people with psychosis;¹⁷ and Acceptance and Commitment Therapy in Daily Life (ACT-DL), which complements ACT sessions with 3 days of exercises in daily life using an ESM app.

Despite these promising developments, the challenge that remains at this point is to identify key parameters in ESM and other digital data that reflect (1) risk markers for individuals in a given moment and context, and (2) targets for interventions (that may or may not coincide with risk markers) that optimize timing and effect on outcomes in the context of daily life.

Time Series Models and Mobile Data

Data from mobile and wearable devices all come as time series (or longitudinal data). Time series, ie, the consecutive repeated sampling of data points in time, need special attention and treatment from a statistical and machine learning perspective as they are usually (highly) temporally dependent and violate the common statistical assumption of independent samples. Time series models often express temporal dependencies in the data by some time-recursive function of the general form $x_t=F_{\text{θ}}(x_{t-1}\dots x_{t-\Delta },u_t,{ϵ}_t)$, ie, the current observation $x_t$ is assumed to depend on the values $x_{t-1}\dots x_{t-\Delta }$ of preceding observations up to some time lag $\Delta$, as well as on current external inputs (regressor variables) $u_t$.$\theta$ denotes parameters of the system (like regression weights), and ${ϵ}_t$ is a noise term (ie, a random variable). To make this concrete, we can consider, eg, the process of sensitization. Here, repeated exposure to an environmental risk factor results in a progressive increase of the stress response, such that individuals experience a strong stress response even to minor stressors in daily life. It has been hypothesized that this enhanced stress response may facilitate the transition to psychosis.¹⁸ In this example, the $x_t$ may constitute consecutive stress responses (frequently operationalized as stronger emotional reactions assessed as negative affect ESM ratings), which are related to previous stress responses $x_{t-1}\dots x_{t-\Delta }$ and to minor stressors $u_t$ (eg, ratings of unpleasant and taxing events, activities, and social situations) by a set of regression weights inferred from the data. Once such a model has been trained on the data, it can be used to statistically test various assumptions about the influence of external regressors $u_t$ or the nature of the dependencies in time, and most importantly perhaps, to produce predictions about future states by running it forward in time.

The most popular and commonly employed class of time series models is “auto-regressive moving average (ARMA)” models, where $F_{\text{θ}}$ is assumed to be a linear function for simplicity, and temporal relationships are expressed directly between observations $x_t$ (ie, in our example the individual’s future stress response is directly predicted from previous stress responses and external stressors via the estimated regression weights).

More powerful and perhaps interesting are so-called latent variable (“generative” or “state space”) models, which assume that there is some underlying but itself unobserved process $z_t=F_{\text{θ}}\left(z_{t-1},u_t,{ϵ}_t\right)$ that generates the observations $x_t$ according to some probability distribution$p\left(x_t|z_t\right)$.¹⁹ Time series are usually generated by some underlying dynamical system that evolves in time,²⁰ and it is this underlying system that we are often ultimately interested in. Coming back to our example, we may not be so much interested in the subjective ESM ratings per se, but only because these hint to some underlying psychological or biological dynamical process we would like to tap into. In this specific case, we would be interested in the underlying sensitization process (or even concomitant changes in the dopaminergic transmitter system posited to be involved in this process in the development of psychosis¹⁸). Thus, $z_t$ would model some not directly observed, underlying affective, cognitive, or neural process such as sensitization that gives rise to the changes in subjective stress reactivity ratings $x_t$. Hence latent variable models enable us, to some degree, to reveal the true processes of interest from some “surface measurements” that are a reflection of this underlying process. Although we have outlined this for just a single observation modality (stress reactivity ratings), generative models, if designed properly, can combine and integrate information from many diverse sources, such as different sensor readings on top of ESM ratings, or different classes of environmental factors.

RNNs may be seen or cast as such latent variable time series models where the transition function$F_{\text{θ}}$ is highly nonlinear,^21,22 a decisive difference to the former more common statistical models.^22,23 It is well known that purely linear systems like ARMA models can only capture or produce a very limited class of dynamical phenomena,²⁴ eg, may not be able to properly model the pattern of reoccurring episodes of psychotic symptoms with relatively sudden onsets and slower offsets. RNNs, on the other hand, which, in theory, can be used to approximate (almost) any other dynamical system,^25,26 are very powerful devices for modeling and predicting even complex multivariate time series, as they arise from mobile sampling (see figure 1 for details).

Fig. 1.

(A) Schema of a state space model, unwrapped in time, where the latent process is represented by a recurrent neural network (RNN). The latent states z_t represent the activations of “neural units” that are connected and interact through “synaptic” weights (connecting black lines). “Recurrent” means that both forward and backward connections among units are present, in contrast to the much more common pure feedforward networks. These recurrent connections are what make these models true time series or dynamical system models that express recursive relationships in time. In the graphical representation here, stimulus inputs u_t (in this case minor stressors in daily life and ecological momentary interventions) exert their effects directly onto the latent states (the underlying dynamical model of a person). These in turn generate observations x_t, in this case stress response ratings and psychotic experiences, as assessed through ESM and other sensors on mobile and wearable devices. (B) Particular RNN architectures, or particular forms of the nonlinearities $F_{\text{θ}}$, enable the network to detect, represent, and predict very long-term (“deep”) temporal dependencies. By simulating the RNN model forward in time, longer-term predictions on future observations can be produced, as for instance depicted here for the example of symptom severity as a function of fictive potential stressors and an EMI at future time points.

Opportunities and Challenges of RNNs for Digital Data in Psychosis Research

RNNs as latent variable models, when trained on a set of time series data, come to represent the underlying dynamical process. In this sense, they build a dynamical systems model of the person that could be used for predicting the individual’s behavior and for suggesting suitable interventions ahead of time, to change that predicted behavioral course. For instance, the models could warn individuals about upcoming risks and signal critical periods for EMIs and other interventions. Vice versa, by incorporating those EMIs as model inputs, the same RNN could be leveraged for predicting treatment response, paving the way for context-dependent and customized interventions. Ultimately, a feedback loop that optimizes ESMs and EMIs iteratively and subject-specifically could be realized.

This generative aspect of RNNs also enables us to predict the influence of environmental factors by simulation. The model can “generate” new behavior when forwarded in time such that model inputs for instance could be emulated to assess the effect of specific interventions and their (hypothetical) interaction with other variables and the dynamical process itself. RNNs may also give insights into aberrant mechanisms underlying psychosis or other conditions, and by integrating many different features and inputs, unravel new types of relations between environmental factors and behavior that were previously not known or hypothesized to exist. As these relations may be rather complex and involve long-term temporal dependencies,²¹ hard to assess intuitively, merely raising awareness by feedback could already prove useful from a psychoeducative perspective.

Of course, although very powerful, such models also come with major challenges. These include the computational and data aspects of model training, ethical and data safety issues,²⁷ as well as the selection of appropriate model inputs and features (both of which may be sampled at different frequencies and may follow different distributions). For instance, efficient RNN models may often require large amounts of data for deriving their many parameters. A potential solution to this is “transfer learning”^22,28 where one uses data obtained from a larger group of individuals for model pretraining, and data from the single individual for fine-tuning the model.²⁹ Especially in such ecological contexts where each individual shapes their very own and unique experiences, integrating data across many subjects may be particularly beneficial, as it may enable to piece together separate bits of the same puzzle. This would also give us more powerful ways to study rare events such as when several risk factors come together in a specific combination.

Conclusion

Recent years have seen rapid progress in the use of behavioral, physiological, and other mobile data collected in context of daily life using wearable technologies to improve understanding of psychosis. We have argued here that RNNs, a powerful statistical machine learning approach for time series analysis and prediction, can be trained on multiple data modalities simultaneously to learn a dynamical model to forecast individual trajectories, and schedule online feedback and intervention accordingly. Future research using this approach is likely going to offer new avenues to further our understanding of, and treatments for, psychosis.

Funding

S.G. was supported by the European Community’s Seventh Framework Program (grant agreement no. HEALTH-F2-2009–241909) (project EU-GEI). U.R. was supported by a Heisenberg professorship from the German Research Foundation (grant no. 389624707). D.D. was supported by grants from the German Federal Ministry of Education and Research within the e:Med program (01ZX1311A [SP7] and 01ZX1314G [SP10]) and the German Science Foundation (Du 354/8-2).