Experience sampling methodology in mental health research: new insights and technical developments

Abstract

In the mental health field, there is a growing awareness that the study of psychiatric symptoms in the context of everyday life, using experience sampling methodology (ESM), may provide a powerful and necessary addition to more conventional research approaches. ESM, a structured self‐report diary technique, allows the investigation of experiences within, and in interaction with, the real‐world context. This paper provides an overview of how zooming in on the micro‐level of experience and behaviour using ESM adds new insights and additional perspectives to standard approaches. More specifically, it discusses how ESM: a) contributes to a deeper understanding of psychopathological phenomena, b) allows to capture variability over time, c) aids in identifying internal and situational determinants of variability in symptomatology, and d) enables a thorough investigation of the interaction between the person and his/her environment and of real‐life social interactions. Next to improving assessment of psychopathology and its underlying mechanisms, ESM contributes to advancing and changing clinical practice by allowing a more fine‐grained evaluation of treatment effects as well as by providing the opportunity for extending treatment beyond the clinical setting into real life with the development of ecological momentary interventions. Furthermore, this paper provides an overview of the technical details of setting up an ESM study in terms of design, questionnaire development and statistical approaches. Overall, although a number of considerations and challenges remain, ESM offers one of the best opportunities for personalized medicine in psychiatry, from both a research and a clinical perspective.

The essence of psychiatric symptoms is that they are natural experiences emerging in the realm of ordinary daily life, often in interaction with contextual factors.

In the mental health field, there is a growing awareness that the study of psychiatric symptoms in the context of everyday life may provide a powerful and necessary addition to more conventional research approaches1. Increasingly, studies are using techniques such as experience sampling methodology (ESM)2, 3 or ecological momentary assessment (EMA)4, 5 to study psychopathology in daily life.

This paper provides a comprehensive review of the principles and applications of ESM, and an update on its design and techniques in the mental health field.

PRINCIPLES OF ESM

ESM is a structured self‐report diary technique assessing mood, symptoms, context and appraisals thereof as they occur in daily life1, 3. It typically requires participants to complete a momentary questionnaire several times a day over a number of days.

ESM is rooted in ecological psychology, which states that behaviour is radically situated, i.e., it can only be understood in relation to the context in which it occurs6. In order to fully understand experience and behaviour, they need to be investigated in the real‐world context, outside the laboratory.

The use of ESM to investigate experiences within, and in interaction with, the real‐world context is also consistent with a more recent emphasis on embodiment and embeddedness in the cognitive sciences7. This approach claims that “the way humans (and other organisms) perceive, think and act is determined by the kinds of bodies they have and the kind of actions they perform or are capable of”8. Therefore, in order to understand or explain experiences, one must examine them in interaction with the context. ESM directly addresses this, in that it allows us to study the actual experience as it occurs in everyday environments, rather than assessing people's self‐reflections on who they are or how they usually behave.

As experience and behaviour are at the core of psychopathology, ESM was quickly adopted by mental health researchers3. In addition to its benefits in terms of examining experience and behaviour in interaction with the real‐life context, ESM also allows us to study these in the moment, overcoming the problem of retrospective recall bias, and prospectively, allowing us to investigate temporal variability and associations. ESM may therefore fundamentally strengthen the behavioural study of mental health problems and contribute to a contextual approach to personalized medicine in psychiatry.

APPLICATIONS OF ESM IN THE MENTAL HEALTH RESEARCH FIELD

In this section, we review how zooming in on the micro‐level of experience and behaviour using ESM can help in improving our understanding of the phenomenology and aetiology of psychopathology and in strengthening and changing clinical practice.

Improving understanding of symptoms

Although psychiatry has developed a common terminology to describe different aspects of psychopathology (e.g., as reflected in features described in the ICD or DSM), our understanding of the nature of these phenomena is still fairly limited, partly due to the biases introduced by the retrospective recall of symptoms. ESM addresses this issue by capturing symptoms as they occur. Indeed, studies that compared retrospective and ESM assessment of symptoms reported that the former assessment tends to under‐ or over‐estimate depressive symptoms in patients with depression9. Furthermore, only moderate correlations were found between retrospective and momentary assessment of affect in patients with psychosis10, 11.

Besides providing a more accurate assessment, ESM may be instrumental for obtaining a deeper understanding of how symptoms unfold in daily life over time. Good examples are anhedonia and avolition, both of which have long been reported to form part of the phenomenology of psychotic disorders, depression and bipolar disorder12.

Anhedonia is generally described as a diminished capacity to experience pleasure. However, what does this mean for our experience and behaviour in daily life? A decreased level of positive affect in daily life – which has been found in some studies in patients with psychosis13, 14 – may reflect a diminished capacity to experience pleasure. Yet, decreased levels of positive affect may also result from patients' lives being less enjoyable. Indeed, patients with psychosis do report, on average, a lower number of pleasant events in their daily life than healthy controls13. In order to disentangle this, positive affect has been examined in moments when people do report pleasant events. ESM research in individuals with psychosis revealed an intact ability to generate positive affect upon experiencing pleasant events in daily life15, 16, 17, which does not support the widely held assumption that anhedonia reflects a general incapacity to experience pleasure.

So, what does anhedonia then relate to? Gard et al18 distinguished experiencing positive affect in the moment (consummatory pleasure) from pleasure related to future activities (anticipatory pleasure), and found the latter to be particularly reduced in patients with psychosis. This distinction may partly explain why patients with psychosis and students with social anhedonia (assessed with observer‐rated measures) reported higher levels of positive affect when in the company of others compared to when alone (i.e., suggesting higher consummatory pleasure in social situations), but still spent more time on their own13, 19.

This brings us to avolition, commonly defined as a lack of motivation or, put differently, an incapacity to translate positive emotional experience into productive goal‐directed behaviour. The longitudinal design as well as the assessment of both mood and activities in ESM allows us to directly relate emotional experience to subsequent activities and behaviour. For example, it has been shown that positive affect experienced in social contexts or during physical activity increases the odds of engaging in these behaviours at the next measurement moment in healthy women20. Interestingly, this reward‐oriented behaviour of positive affect in one moment driving future behaviour was absent in the everyday behaviour of individuals with anhedonia16, supporting the idea that a lack of anticipatory pleasure (i.e., anhedonia) may result in a reduced drive for seeking out these activities (i.e., avolition).

An emerging trend in affective neuroscience has been to increase functional relevance of experimental findings by investigating hedonic and goal‐directed behaviour using laboratory paradigms in conjunction with ESM. Moran et al21 linked blunted daily‐life experience of pleasure and motivation to poorer performance on effort and reward learning tasks in individuals with schizophrenia. Our group combined ESM with neuromolecular imaging in healthy individuals with increased familial risk for psychosis. We found that intact striatal dopaminergic modulation of reward learning predicted daily‐life reward‐oriented behaviour in both groups, which may point to a neurochemical and behavioural mechanism of resilience in those predisposed to psychosis22, 23.

In sum, the accumulating ESM accounts of hedonic and volition function in clinical populations have contributed to a more fine‐grained understanding of the affective and behavioural dynamics compromising productive goal‐directed behaviour, thus opening promising avenues for equally fine‐grained prevention and treatment strategies.

Capturing emotional variability

Many mental disorders are characterized by dynamic fluctuations in emotions. The resolution of traditional self‐report measures for capturing these fluctuations is limited. Multiple measurements within one person in ESM may help to assess affective variability in more detail, as well as to identify the context in which (mal)adaptive emotion regulation strategies are used24, 25.

A meta‐analysis investigating dynamic fluctuations in emotions showed that lower levels of psychological well‐being were associated with greater intensity of emotions, larger moment‐to‐moment fluctuations and a slower recovery to a normative state (i.e., inertia), and this was particularly true for negative affect26. Indeed, studies in large samples of patients revealed intense and variable negative and positive affect in patients with psychotic disorder13, 14. The slower recovery to a normative state possibly reflects inadequate emotion regulation strategies.

A recent ESM study suggested that patients with schizophrenia demonstrated adequate effort to control their emotions in their daily life, but these efforts were unsuccessful27. Another study in bipolar disorder examined coping mechanisms or response styles to both positive and negative mood. Low mood predicted elevated rumination, which subsequently dampened mood further. High positive affect, on the other hand, predicted high‐risk taking, which resulted in even higher positive affect, demonstrating a vicious cycle of escalating mood levels28.

Capturing variability may not just be helpful in identifying a worse psychological state or an inadequate emotion regulation strategy. It may also predict future course of symptoms. Affective variability at baseline predicted the recurrence of depressive symptoms over a three‐year period in remitted depressed patients29. Similarly, elevated temporal auto‐correlation and variance, as well as increased correlation between emotions, increased the probability for an upcoming shift between a normal and a depressed state30.

Identifying internal and situational determinants of psychopathology

Variability does not only pertain to affect. Most symptoms observed in patients with severe psychiatric disorder show meaningful and widespread variation over time. For example, intensity of visual and auditory hallucinations or delusions is highly variable over time31, 32. Identifying what drives this variation, either internally or contextually, may be very helpful in detecting targets for treatment. At the same time, it may help patients to become aware of their own patterns of behaviour.

The longitudinal nature of ESM data makes it excellently suited to examine temporal associations between context, experience and behaviour. It has, for example, been shown that increases in paranoia are preceded by increases in anxiety, reductions in self‐esteem and engagement in experiential avoidance33, 34, 35. Similarly, an ESM study into non‐suicidal self‐injury (NSSI) – the non‐accidental damaging of one's own bodily tissue without suicidal intent36 – found increased feelings of numbness and rejection to predict NSSI behaviour. These studies thus identify potential targets for treatment; improving self‐esteem or diminishing feelings of rejection may help reduce levels of paranoia or NSSI behaviour.

ESM is not restricted to inner mental states as a possible predictor of symptoms. Contextual variables can also be taken into account. Collip et al37 found paranoid thinking to be context‐dependent in individuals with low or medium levels of trait paranoia. Paranoid thoughts increased when people were in the company of strangers. Yet, for those with high levels of trait paranoia, momentary paranoia became autonomous and independent of social reality. With respect to NSSI in adolescents, it was found that they were more often alone when they started thinking about NSSI, and being alone was also a significant predictor of engaging in NSSI38. Making people aware of these behavioural patterns may, again, be very instrumental for treatment.

Examining the consequences of events may also improve our understanding of real‐life dynamics. In NSSI research, it has been shown that negative affect rises substantially prior to an episode of NSSI and then decreases directly after, underlining the mood‐regulating function of the NSSI behaviour39. Similarly, an ESM study examining the association between cannabis use and psychosis found higher increases in both positive affect and psychotic symptoms following cannabis use in patients compared to controls, possibly explaining the vicious cycle of deleterious use in these patients40.

Importantly, these patterns of association may substantially differ within persons. Our group examined the individual data of 64 persons with psychotic disorder and found clear inter‐individual differences in the temporal order of mood and paranoia, with findings for each case deviating from the overall group findings41. ESM is highly attuned to individual patterns of associations, which may lead to person‐tailored psychoeducation and identification of individual targets for treatment, thus providing opportunities for personalized medicine.

Zooming in on person‐environment interaction: sensitivity to stress

An important putative psychological mechanism for many psychiatric disorders is sensitivity to minor stressors in daily life. While most epidemiological studies have focused only on the association between stressors and presence of psychopathology, ESM allows us to investigate the way an individual reacts to a stressor and how this is associated with psychopathology.

Several ESM studies have shown altered reactivity to daily‐life stressors in patients with psychopathology. In major depressive disorder, patients showed increased negative affective reactivity to daily stressors compared to non‐depressed individuals42 and remitted patients43. Increased affective reactivity to stressors has also been reported across the psychosis continuum44, 45, 46, 47, 48, 49, 50. Interestingly, the most pronounced increases in stress sensitivity have been found in individuals with a clinical high risk state for psychosis48, 49, 51, in part since psychotic symptoms may particularly add to the experience of stress in this group51. Moreover, stress is also associated with momentary increases in psychotic experiences in both patients and their family members52.

As affective disturbances mediate the effect of daily hassles on psychotic experiences53, an affective pathway to psychosis has been proposed54. In line with this, ESM studies have shown that exposure to social adversity, such as childhood trauma50, 55, 56, 57, 58, 59 and major life events60, 61, is associated with increased reactivity even to minor stressors in daily life, suggesting a process of behavioural sensitization62. In addition, these traumatic experiences were also linked to stronger psychotic reactivity to threat anticipation in daily life in individuals at the more severe end of the psychosis continuum50. Interestingly, mediation analyses revealed that the effect of daily stressors on psychotic experiences was not only mediated by affective reactivity, but also by threat anticipation in first‐episode psychosis patients53.

Increased reactivity to daily‐life stressors also has prognostic value, as it has been found to predict persistence of psychotic symptoms63, and onset of depressive disorder64 one year later in adults. One study even found an increased likelihood of reporting an affective disorder ten years later65. Reactivity to the smallest of daily hassles has been shown to be predictive of persisting psychopathology in adolescents and young adults66.

As ESM provides multiple assessments over time within one individual, more complex analysis techniques have been adopted to study the temporal dynamics of stress, affect and symptoms. Using a network analysis, Klippel et al67 showed that risk for psychosis is associated with changes in contextual networks where daily stressors have a central position and predict psychotic experiences while reducing physical activity. In depression, a network approach indicated an important role for the experience of social pressure68. Automatically generated personalized models that require intensive sampling have revealed different patterns of affect dynamics, including stress reactivity, between subclinically depressed individuals with and without anhedonic complaints69.

Finally, an evolving field in ESM stress research incorporates biological measures. Previous studies have looked at associations between such measures and daily‐life stress. For instance, structural changes in the pituitary and hippocampus have been directly associated with increased daily‐life stress reactivity in psychosis70, 71. Similarly, several studies were able to predict daily‐life stress reactivity using functional neuroimaging72, 73, 74. More recent studies have embedded physiological monitoring in the ambulatory setting, measuring heart rate, blood pressure, cortisol and α‐amylase in real life. Initial results indicated increased cardiovascular stress reactivity in post‐traumatic stress disorder75, 76, blunted cortisol responsivity in depression77 and psychosis78, and increased cortisol reactivity in people at familial risk for psychosis44. In this evolving field, technological developments allow for novel passive monitoring approaches for continuous measurement of physiology, which can provide unique insight into the role of stress in the aetiology of mental disorders.

Examining real‐world social interactions

ESM assessments also provide an opportunity for gaining more insight in activities and social interactions of people in daily life. One study found that patients with psychosis spend more time alone and at home, and are more often doing nothing compared to a healthy control sample79. This was also the case for patients meeting criteria for symptomatic recovery80: despite the reduction of symptoms, they were still more isolated and less engaged in goal‐directed activities compared to healthy controls. Another study detected that individuals with psychosis set more pleasure‐based and fewer effort‐based goals, and, similarly, engage in more pleasurable and less effortful activities throughout their daily lives18. In a general population sample of women, it was found that avoiding social contact after appraising company more negatively increased the risk for the development of major depressive disorder in the following 20 months81.

When comparing a standard social functioning questionnaire, the Social Functioning Scale (SFS), with ESM measures of social functioning, the SFS did show some degree of ecological validity for assessing the broad aspects of social functioning, but ESM measures offered a much more detailed and rich alternative79. This may be helpful for clinical practice, but can also further our theoretical knowledge. It has, for example, been shown that different social cognition tasks such as emotion recognition and theory of mind are not related to fine‐grained measures of interaction in daily life82, 83. Similarly, subjective quality of life measured in the moment was more consistently associated with affect, social interaction and activity compared to self‐reported quality of life as assessed with a retrospective questionnaire84.

Evaluation of treatment

As ESM provides a fine‐grained picture of mental state and functioning, it may be much more sensitive to capturing change and, thereby, significantly improve assessment of outcomes in studies investigating therapeutic effects of biological, psychological and social interventions in psychiatry85. Moore et al86, in a study evaluating the effect of a mindfulness intervention in depressed individuals, reported that ESM measures were much more sensitive to change, particularly for depressive symptoms and mindfulness, for which the number‐needed‐to‐treat was 25 to 50% lower than for traditional questionnaire.

In addition, ESM expands outcome and process measures beyond the reach of conventional assessments. One study in patients with major depressive disorder found clear dose‐response effects in increases of positive affect and enhanced responsiveness to pleasant daily‐life activities over 18 weeks of antidepressant treatment87. This effect was also observed in patients participating in mindfulness‐based cognitive therapy88 and in cognitive behavioural therapy89. The latter group also endorsed increased resilience to stress in daily life. Another study in psychosis found different dimensions of delusions changing at different rates over time in response to antipsychotic treatment90.

ESM's sensitivity to change may also allow earlier detection of side effects. One study investigating the association between the dosage of antipsychotic medication and affect in daily life found significant decreases in positive affect at a much lower medication dose than was needed for the occurrence of more pronounced extrapyramidal symptoms91.

A more detailed baseline assessment, that can be achieved using ESM, may also improve prediction of treatment outcome. Forbes et al92 found lower negative affect and higher positive affect at baseline to be predictive of better treatment response in children and adolescents with affective problems. There is also evidence of diminished emotional reactivity to be associated with a lower likelihood of recovery over a period of 18 months in adults suffering from major depressive disorder93.

In addition, early processes of change can be identified that predict later outcome. Early changes in positive rather than negative affect during the first week of an antidepressant treatment predicted treatment response at six weeks94. Furthermore, responding to treatment was associated with an increase in reward experience, suggesting that response to treatment in depression may be conditional on a recovery of hedonic capacity95. With an even more sophisticated analysis, the same authors found that stronger reductions in negative affect following a peak of positive affect during the day was associated with a more favourable course of depression96.

Opportunity for new interventions

Over the last years, ESM has also been used to deliver treatment in real life. Ecological momentary interventions (EMIs) use mobile devices to deliver treatment in the daily life of patients, thus extending the therapy beyond the clinical setting and into daily life97.

The content of these interventions is highly variable. Some are developed to augment face‐to‐face contacts with EMI components, such as the recently developed EMI Acceptance and Commitment in Daily Life, where therapeutic sessions are followed by three days of real‐life exercises using an ESM app98, 99. An example of a fully automated EMI is FOCUS, which has been specifically developed to provide automated real‐time and real‐world illness management support for psychosis100, 101.

Some EMIs integrate assessment of symptoms using ESM with a real‐world delivery of treatment. PRISM (Personalized Real‐time Intervention for Stabilizing Mood), for example, prompts individuals with bipolar disorder to fill out a survey on current context and mood state twice a day, which then triggers predefined and personalized action steps102. Another study in depression provided personalized feedback based on aggregated ESM information to increase awareness and induce behavioural change103, 104, thus reducing depressive symptoms over time.

Although the field of EMI is still in its early stages, recent systematic reviews suggest a high acceptance and feasibility in individuals with severe mental illness97, 105, 106. With regard to efficacy, there is only a limited amount of research available to date. Overall, the limited evidence supports the efficacy of EMIs in mental health101, 102, 104, 107, 108, 109, 110, 111. Evidence seems to point towards greater efficacy when EMI is integrated with real‐life assessment using ESM, preferentially tailoring the intervention to the specific needs of the individual as well as to those moments when intervention is most needed97, 112. Evidence from exploratory and definitive randomized controlled trials is now required to further elucidate the efficacy and effectiveness of EMIs.

Summary

In summary, ESM has many advantages. It increases patient empowerment by identifying the individual as the expert of his/her experiences. Focusing on the micro‐level dynamics of symptoms113, it improves our understanding of their nature, their variability over time and their patterns of associations, both at group level and at the level of the individual. Furthermore, ESM can improve evaluation of treatment due to its sensitivity to capture change, as well as provide opportunities for the development of new interventions.

DESIGNS, METHODOLOGY AND STATISTICS

In this section, we provide updated information on the technical details of setting up an ESM study.

Designs

ESM uses event‐contingent, time‐contingent or hybrid designs.

In event‐contingent designs, the sampling units are predefined events, i.e., ESM assessments are triggered by certain events such as panic attacks, social interactions or cannabis use. These designs offer the advantage of a comprehensive sample of the events under investigation. The disadvantage, however, is that it is sometimes difficult to define discrete events in a way that is readily accessible to, and recalled by, participants. It is also difficult to establish compliance, i.e., whether participants completed assessments for all predefined events. Finally, while a comprehensive sample of predefined events may be obtained (assuming optimal compliance), other relevant aspects of experience and behaviour may be missed. Therefore, most studies use time‐contingent designs as an alternative approach.

In time‐contingent designs, time is used as the sampling unit, i.e., participants are asked to complete a questionnaire contingent on time instead of an event. Sampling schedules of time‐contingent designs can either be fixed or random.

A fixed sampling schedule requires participants to complete questionnaires at equal time intervals (e.g., every two hours). Although fixed sampling schedules ensure that target constructs are comprehensively assessed and allow for longitudinal statistical analyses, they have two major limitations. First, fixed sampling schedules may increase reactivity to the method, as participants know when they have to complete an assessment, which may lead them to adapt their daily routines. Second, they do not allow calculation of time budgets, as assessments at equal intervals are not necessarily representative of other moments that are not sampled.

A random sampling schedule signals the participant at random intervals during the day, making use of random blocks to ensure that time is evenly sampled across the day. A significant advantage of random sampling schedules is that they provide a representative sample of the target construct. They significantly decrease reactivity to the method, as participants do not know when the next signal will occur. Also, as random sampling schedules allow for calculating time budgets, they offer the advantage of providing estimates of the average time that people spend in certain contexts.

Hybrid designs combine event‐contingent and time‐contingent designs. For example, measuring mood using a time‐contingent design with a blocked random time schedule, and substance use as a discrete event in an event‐contingent design, or time‐contingent designs with different (fixed or random) time schedules.

Sampling frequency and period determines the resolution required for assessing the target construct. This primarily depends on the known or expected variability of the target construct over time. Many experience sampling studies in psychiatry conducted to date have used a sampling frequency and period of ten assessments per day over a period of six consecutive days, given the resolution required for assessing highly variable constructs (e.g., mood). However, this needs to be considered in detail for each individual study and balanced with assessment burden for participants.

ESM designs were, for a long time, implemented using pen and paper diaries to assess target constructs and wristwatches to implement time‐contingent designs. After a brief period of using personal digital assistants or other handheld devices, most, if not all, studies now use smartphone applications for implementing ESM designs114.

Questionnaire development

The development of questionnaires is an important aspect of ESM research, as targeting momentary experiences is very different from the global and retrospective approach in cross‐sectional questionnaires. ESM questionnaires, therefore, follow their own rules and logic. Overall, completing a questionnaire should not take longer than 2 minutes. Questionnaires contain on average 30‐60 items, depending on the item format. Including more items may minimize reactivity to the method by diverting attention away from specific items of interest, but increases burden. On the other hand, when the explicit goal is to make people aware of their patterns of behaviour, such as in clinical therapy, it may be helpful to reduce the number of items.

ESM questionnaires in psychiatry often include questions on current mental states (e.g., thoughts, mood and symptoms), behaviour, context, and appraisals of these contexts. Items are preferentially presented in this order, moving from more to less transient items. Sometimes “between moment” questions are included at the end of the questionnaire, inquiring about the time between the previous and the current report. Although it may be of interest to include ratings of these in‐between periods (e.g., drug use or daily hassles), these questions should be limited, as they may again be subject to recall bias, even if minimal, and do not directly reflect interactions in context.

When developing a questionnaire, the first imperative is that the questions should inquire about momentary states. This may seem obvious, yet is not always easy to achieve. For example, including “right now” before a global statement is not transferring this statement into an item of a momentary state. “Right now, I have a number of good qualities” remains a global statement and, therefore, will result in little variation over time.

Another imperative is that the language should reflect how people think about and describe their own behaviour and experience. Lexicon commonly used by professionals, such as attribution, coping or dissociation, is best avoided. ESM questionnaires could still substantially improve in this respect. We are currently conducting focus groups with people with experience of psychosis to more accurately grasp their actual experiences, and thus improve the assessment of psychosis using ESM, but also the way we assess their social interactions.

Third, it is conceptually important to focus on processes that pertain to common situations in daily life. “Did you initiate the contact?” may be informative in very specific situations but, when you are in the kitchen having breakfast with your husband, it is difficult to answer.

Finally, ESM optimally aims to capture patterns of behaviour that people are not necessarily aware of. Therefore, it is important to avoid using reflective questions, such as “How do you feel in this company?”, but rather to inquire purely about the momentary states, e.g. “How do you feel right now?” and “With whom are you right now?”, which can later be correlated, either cross‐sectionally or over time, to establish behavioural patterns.

Statistical approaches

ESM studies typically yield a substantial amount of data. A study with 100 subjects, each assessed ten times per day over the course of six days using a random time sampling schedule, yields a dataset with 6,000 rows of data, where each row corresponds to a particular assessment moment for a given subject. Data of this type can be used at a more descriptive level, e.g. to estimate the mean level of a particular variable and its variability within and between study participants, or to study group differences, e.g. whether patients report on average elevated levels of negative affect compared to controls. However, the full strength of ESM comes into play once we start to examine the within‐person relationship between some outcome of interest (e.g., negative affect) and some time‐varying predictor (e.g., stress) and how the strength of such a relationship may differ across groups (e.g., whether the relationship between negative affect and stress differs for patients versus controls).

Although an ESM study is essentially a repeated measures design, classical analysis procedures such as repeated measures analysis of variance (ANOVA) or multivariate analysis of variance (MANOVA) are not directly applicable, as they cannot easily handle the complexities involved (e.g., missing data, unequally spaced time points, time‐varying covariates, auto‐correlated observations). Instead, given that ESM data adhere to a multilevel structure (with repeated assessments nested within days, which in turn are nested within subjects), multilevel/mixed‐effects models are typically the method of choice for their analysis115.

Mixed‐effects models extend the standard regression model by allowing for the inclusion of additional “random effects”, which can be used to account for person‐ and day‐level differences in the model coefficients (i.e., intercepts and slopes). Such models also allow us to disaggregate between‐ and within‐person relationships116. For example, the degree to which those who experience higher levels of stress overall also tend to report higher levels of negative affect (between‐person) may be quite different from the degree to which negative affect varies within a subject in relation to the perceived stressfulness of particular situations (within‐person).

Standard mixed‐effects models used for the analysis of ESM data117 can be extended in various ways: for example, by allowing mixture distributions for the random effects118, by adding predictors for the amount of within‐ and between‐subject variability119, and by allowing model coefficients to change smoothly over time using splines120, 121. Vector autoregressive models using ESM data of single subjects122 or groups of subjects123 can provide further insights into the dynamics of psychopathology from a network perspective124. Recently, the use of mixture latent Markov models has also been suggested as an alternative approach to analysing ESM data125. Given the increasing use of ESM in research, we expect to see further developments in the analysis approaches in the coming years.

CONSIDERATIONS AND FUTURE PROSPECTS

While there has been a rapidly increasing number of studies using ESM to investigate highly important areas of research in psychiatry, it is important to take stock and critically appraise what has been achieved in order to move on and build upon what can be gleaned from studies using this method.

One important consideration is the replicability and consistency of findings that have emerged from previous ESM studies. Most experience sampling studies to date have focused on novel questions using different designs and questionnaires, resulting in heterogeneity in terms of conceptual definitions as well as findings across different populations. Therefore, as in many other areas of psychiatric research, there is a need for direct replications of findings for the same study population. In order to achieve this, there is a pressing need for greater consistency in the definition and operationalization of target constructs, as well as robust psychometric research on the structural validity of ESM measures of key constructs as a basis for deriving composite scores and reducing heterogeneity in findings. Similar to other areas of research where small samples are common, such as neuroimaging research, careful sampling of participants is required to minimize selection bias.

One way of moving the field forward is to develop questionnaires with good psychometric properties (e.g., the Maastricht Mood Questionnaire126) that can be used in ESM studies. Additionally, more methodological research is needed providing robust evidence on design issues for key constructs to achieve standardization and replicability. This will enable ESM researchers to work in larger networks and consortia, as has been the case in other areas of psychiatric research127, to generate consistent and generalizable findings across countries.

A further consideration is that ESM data collection is time‐intensive and may be associated with assessment burden for participants, which raises the question of whether this method can be used in all populations and, in particular, in vulnerable populations. However, there is strong evidence in support of the feasibility of using ESM in vulnerable populations, including individuals with (severe) mental health problems3, which may be due to the nature of ESM as a structured inquiry about current mental states with clear ecological appeal. However, further developments are needed for the use of ESM in children and in older populations, e.g. people in the early stages of dementia. An interesting prospect here is the development of dyadic approaches, combining self‐report within one person with observational and context data provided by an informant.

Furthermore, numerous researchers have raised the question of whether being repeatedly asked about particular thoughts and behaviours may, in fact, induce those thoughts and behaviours or may cause participants to alter their behaviour3, 5, 128. Whilst participants may feel positively about repeated questioning during ESM, this may nevertheless result in them consciously or unconsciously altering their behaviour. Measurement reactivity is a key challenge for ESM research, yet remains an under‐researched phenomenon129. However, as outlined above, there are ways of minimizing reactivity by selecting appropriate ESM designs and measures.

Finally, most findings to date relate to evidence of associations based on cross‐sectional modelling of ESM data. This now needs to be probed further to investigate the strength of the evidence in support of other important criteria for establishing causality, such as temporal order or experimental evidence using ecological interventionist causal models130. Also, the further development and implementation of new statistical techniques is crucial here. As ESM is collecting a large amount of data from each individual person, this may be linked to other sources, such as big data, to monitor and recognize an individual's state108 and context to eventually provide person‐tailored contextualized interventions (although this may require an even larger number of observations than are usually collected).

There is further immense emerging potential for combining ESM with physical remote monitoring technologies. Combining ESM with wearables assessing, for example, physical activity, heart rate variability or sleep may provide even richer and more detailed insights at various levels of causality (biological, psychological, social)131, 132. Another step is to include data from context‐aware systems using sensor data that automatically provide input on relevant context variables108. A large ongoing study in Europe, RADAR‐CNS (https://www.radar-cns.org), set up to examine the relevance of both active and passive remote monitoring approaches in predicting and understanding the clinical course of central nervous system disorders, including depression, may be an important step in this respect.

In addition, one of the most important, but also most challenging, next steps is to bridge the gap between research and clinical care that would allow the implementation of ESM in routine monitoring and outcome measurement in mental health services. ESM has enormous potential to contribute to, and improve upon, clinical care. Yet, to date, it has hardly been implemented, due to issues related to data safety, data ownership, privacy and consent, access to technology, as well as integration of data management systems across mental health services. More implementation initiatives are needed to bridge this gap.

CONCLUSIONS

In summary, we have shown that ESM is an indispensable methodology in psychiatry research. It adds new insights and additional perspectives to standard approaches, enriches our understanding of psychopathological phenomena and their associated mechanisms, and offers clear opportunities for improving and changing clinical practice.

A number of considerations and challenges remain and, with the growing body of research in this field, there is a pressing need for methodological advances. However, as using ESM creates the possibility to study and analyze temporal associations in everyday social contexts, as well as tailor treatment to individual needs, it offers one of the best opportunities for personalized medicine in psychiatry, from both a research and a clinical perspective.