Practitioner and researcher perspectives on the utility of ecological momentary assessment in mental health care: A survey study

Abstract

Background

Ecological momentary assessment (EMA) is a scientific self-monitoring method to capture individuals' daily life experiences. Early on, EMA has been suggested to have the potential to improve mental health care. However, it remains unclear if and how EMA should be implemented. This requires an in-depth investigation of how practitioners and researchers view the implementation of EMA.

Objective

Explore the perspectives of mental health practitioners and EMA researchers on the utility of EMA for mental health care.

Methods

Practitioners (n = 89; psychiatrists, psychologists, psychiatric nurses) and EMA researchers (n = 62) completed a survey about EMA in clinical practice. This survey addressed EMA goals for practitioner and patient, requirements regarding clinical use of EMA, and (dis)advantages of EMA compared to treatment-as-usual. t-Tests were used to determine agreement with each statement and whether practitioners' and researchers' views differed significantly. Linear regression was used to explore predictors of goals and preferences (e.g., EMA experience).

Results

Practitioners and researchers considered EMA to be a useful clinical tool for diverse stages of care. They indicated EMA to be most useful for gaining insight into the context specificity of symptoms (55.0 %), whereas receiving alerts when symptoms increase was rated the least useful (11.3 %, alerts is in 95 % of bootstrap iterations between rank 8 and 10). Compared to treatment-as-usual, EMA was considered easier to use (M = 4.87, t = 5.30, p < .001) and interpret (M = 4.52, t = 3.61, p < .001), but also more burdensome for the patient (M = 4.48, t = 3.17, p < .001). Although participants preferred personalization of the EMA diary, they also suggested that EMA should cost practitioners and patients limited time. The preference for creating personalized EMA was related to the level of experience with EMA. Finally, they highlighted the need for practitioner training and patient full-time access to the EMA feedback.

Conclusions

This survey study demonstrated that practitioners and researchers expect EMA to have added value for mental health care. Concrete recommendations for implementation of EMA are formulated. This may inform the development of specific clinical applications and user-friendly EMA software.

Highlights

• •Ecological momentary assessment (EMA) is an intensive self-monitoring method.
• •EMA has clinical added value for practitioners and researchers.
• •EMA may be useful for gaining insight in context specificity of symptoms.
• •Personalization of EMA is an important requirement for implementation.
• •EMA software should minimize time and burden.

1. Introduction

Ecological momentary assessment1 (EMA) is frequently used in research to collect information about daily life experiences from individuals. It involves prompting individuals to answer a limited number of questions multiple times a day for several days or weeks on their smartphone. EMA questionnaires often include questions on mood, symptoms of mental disorders (e.g., psychosis, depression, anxiety, eating disorders), and contextual factors, such as (social) activities and events. By repeatedly asking individuals to rate the same questions, EMA captures and quantifies temporal dynamics of people's momentary behavior, thoughts, feelings, and context as they unfold in daily life (Bolger et al., 2003). The collected EMA data can be visualized in EMA feedback, which could demonstrate, for example, the fluctuations in symptoms over time, or how mood changes during activities. EMA established itself as a popular research method in recent years, and many researchers also propose EMA can have benefits for mental health care.

Mental health researchers have proposed that EMA allows practitioners to assess individuals in their natural context (Bell et al., 2017) throughout the psychotherapeutic process, aiding diagnosis, intervention, and relapse prevention. Momentary self-monitoring allows patients to frequently reflect on their experiences, which has been shown to enhance their sense of empowerment (Simons et al., 2015), improve depressive symptoms (Kramer et al., 2014a) and instigate behavioral change (Snippe et al., 2016). Relatively few large-scale quantitative studies into the effectiveness of EMA as a mental health tool have been conducted so far. Although one randomized controlled trial showed beneficial effects of EMA on symptom improvement in depression (Kramer et al., 2014b), two others did not (Bastiaansen et al., 2020a; van Roekel et al., 2017). As such, the effectiveness of EMA in mental health care is yet to be determined.

Nonetheless, qualitative research indicates that practitioners and patients with diverse mental health problems believe that EMA increases patients' engagement in the therapeutic process (Folkersma et al., 2021; Palmier-Claus et al., 2013; Saunders et al., 2017; Moore et al., 2020). Involving the patient during the EMA diary construction and analyzing the data together is expected to improve shared decision-making. Other interesting applications of EMA include the monitoring of treatment effects (Bos et al., 2015) and providing alerts when patients relapse (Smit et al., 2019). However, practitioners and patients also mentioned several potential downsides, including high assessment burden and symptom worsening (Bos et al., 2020; Bos et al., 2019). Frumkin and colleagues (Frumkin et al., 2020) conducted a three-week EMA study with twelve patients regarding their psychological symptoms and presented ideographic models as EMA feedback to practitioners and patients. Practitioners demonstrated a more tempered response in the utility of these models compared to their patients, indicating the difficulty of incorporating EMA feedback in treatment. These qualitative studies provide insight into the receptiveness of practitioners and patients to EMA, but also give clues about how EMA should be implemented in mental health care.

One such important requirement is to include relevant stakeholders, such as patients, practitioners, and researchers experienced in the design of EMA diaries and the statistical analyses needed to construct the EMA feedback. By incorporating their perspectives, research into clinical applications of EMA is offered directions for requirements of EMA tools and their place in mental health care. Indeed, to accommodate to the clinical use of EMA, several technological applications have been developed together with stakeholders (PETRA, n.d.; Bos et al., 2022; m-Path, n.d.; Verhagen et al., 2017). These tools aid practitioners in using personalized EMA diaries in treatment, by offering diverse EMA diary options and visualizing EMA feedback. In addition to drawing from qualitative studies, development may benefit from larger scale quantitative studies incorporating perspectives from patients and practitioners. For example, a recent large (N = 375) survey study (Ellison, 2020) among mental health psychologists suggested that EMA might be especially relevant for scientist-practitioners and psychologists already familiar with outcome monitoring tools. Taken together, these studies suggest the importance of incorporating stakeholder perspectives in the implementation of EMA.

The present study aimed to gain insight into practitioner and researcher perspectives on when and how EMA might be applied in mental health care. To that end, we send out an online survey to practitioners and EMA researchers that addressed three main questions: (1) what are the goals for using EMA for practitioners and patients, (2) what is the perceived importance of several requirements regarding EMA diary construction, EMA feedback, and the use of EMA in mental health care, and (3) how do the advantages and disadvantages of EMA compare with treatment-as-usual. We used convenience sampling to recruit Belgian and Dutch practitioners and EMA researchers. Answers to these questions could inform further research into the clinical utility of EMA, as well as further developments needed for its implementation in mental health care.

2. Methods

2.1. Participants

The study used convenience sampling to recruit Belgian and Dutch practitioners and EMA researchers via e-mail in April 2021. We approached practitioners from student guidance centers, EMA app users, smaller to medium-sized group practices, psychiatric hospitals, mental health care facilities, and governmental centers focusing on social and mental health problems via newsletters and online community groups. Practitioners were not required to have experience with or knowledge of EMA. Researchers were invited when they had experience with EMA methodology and/or analysis. The survey was broadly distributed via newsletters and online community groups within and outside the authors' personal network. We asked practitioners and researchers to fill in a survey about the utility of EMA in clinical practice.

89 practitioners and 62 researchers completed this online survey between April and June 2021. Practitioners were primarily psychologists and psychiatrists (n’s = 58 and 19), but psychological assistants, psychiatric nurses, and social workers also participated in this study (see Table 1). Participants who completed the survey could win a voucher in exchange for participating. Study procedures were approved by the medical ethics committee of the University Medical Center Groningen (no. 202100219) and the social and societal ethics committee of the KU Leuven (no. G-2021-3306). All participants signed an electronic informed consent.

Table 1.

Details participant characteristics.

Characteristics	Practitioners n = 89	Researchers n = 62
Gender, n [%]
Male	25 [28.1]	21 [33.9]
Female	63 [70.8]	41 [66.1]
Doesn't want to say	1 [1.1]	0 [0.0]
Age, years
Mean (SD)	39.9 (12.5)	33.5 (9.2)
Country, n [%]
Belgium	49 [55.1]	23 [37.1]
The Netherlands	40 [44.9]	39 [62.9]
Education level, n [%]
Secondary school 1	1 [1.1]	0 [0.0]
Bachelor or Master	59 [66.29]	27 [43.6]
PhD	29 [32.6]	35 [56.4]
Profession, n [%]
Psychologist	58 [65.1]
Psychiatrist	19 [21.3]
PhD Student		25 [40.3]
Postdoctoral researcher		12 [19.4]
Professor		3 [4.8]
Master student		4 [6.5]
Academic staff		13 [21.0]
Othera	12 [13.48]	5 [8.1]
Theoretical orientationb, n [%]
Cognitive Behavioral (CBT)	64 [71.9]
Acceptance and Commitment (ACT)	23 [25.8]
Eclectic	21 [23.6]
Mindfulness	8 [9.0]
Psychopharmacological treatment	22 [24.7]
Systemic	14 [15.7]
Psychodynamic (PDT)	12 [13.5]
Interpersonal (IPT)	9 [10.1]
Client Centered	9 [10.1]
Other	17 [19.1]
Level of health care, n [%]
Primary	20 [22.5]
Secondary	52 [58.4]
Tertiary or higher	17 [19.1]
Experience with EMAc, [Mean (SD)]
Gathering EMA data		4.32 (1.96)
Analyzing EMA data		4.42 (2.00)
General experience with EMA	3.00 (1.72)	4.37 (1.68)
Years of experience in treatment of psychiatric patients [Mean (SD)]	12.1 (11.24)

Abbreviations: SD = standard deviation.

^aOther professions were: psychological assistants to the general practitioner, psychiatric nurses or social workers

^bMultiple responses were possible.

^cRated from 1 (“No experience”) to 7 (“Much experience”).

2.2. Surveys

To assess the perception of practitioners and researchers on the utility of EMA in clinical practice, two online surveys were constructed, one for practitioners and one for researchers (see Supplementary Materials S1 and S2). Questions were identical for both practitioners and researchers, but researchers received a smaller subset without questions regarding clinical work and the comparison of EMA to treatment-as-usual. The average completion time was 15–20 min. Before starting the survey, EMA was explained as an intensive data collection technique by which patients can monitor their mood, complaints, thoughts, and contextual experiences (e.g., activities and companionship) multiple times a day on their smartphone (Myin-Germeys et al., 2009) (see Supplementary Materials S1). Furthermore, the explanation addressed how EMA data can be visualized in EMA feedback, which demonstrates, for instance, the fluctuations in mood or how mood differs during various activities as shown in Fig. 1. After viewing the provided figures, the survey started.

Fig. 1.

EMA and feedback. Left: patient answers questions about context, thoughts, and feelings five times a day (e.g., at noon, in the evening). Right: patient and practitioner receive feedback about the answers given by the patient.

Both surveys covered the following topics: (1) potential goals of EMA for the practitioner and/or patient according to the participant, (2) requirements regarding EMA diary construction, feedback, and use, and (3) participant characteristics. Practitioners (but not researchers) were also asked to compare their most-used instruments with the (dis)advantages of EMA compared to treatment-as-usual. The surveys were specifically designed for this study to cover the most relevant themes emerging from qualitative and quantitative research (Kramer et al., 2014b; Saunders et al., 2017; Bos et al., 2020; Bos et al., 2019; Bastiaansen et al., 2018). The surveys were pilot-tested with five EMA research experts and ten practitioners to test the understanding of the survey items, sequence of survey items, completion time, and lay out. Practitioners and researchers were asked to rate potential goals for practitioners and patients (e.g., obtaining insight into the context specificity of symptoms) on 7-point Likert scales from 1 (“Strongly disagree”) to 7 (“Strongly agree”). They also selected a top three out of ten EMA goals they regarded as most useful (see Supplementary Materials S3). Next, participants rated the importance of several requirements regarding EMA diary construction on 7-point Likert scales: (1) standardized versus personalized diaries, (2) item formulation, (3) measurement schedule, and (4) time constraints. Similarly, requirements for feedback and practical use were rated on 7-point Likert scales. In addition, practitioners (but not researchers) were asked to report on their most-used instruments (i.e., treatment-as-usual) and to compare these with the (dis)advantages of EMA on 7-point Likert scales. Such instruments could include: semi-structured diagnostic interviews, validated screening surveys, personality/intellectual surveys, quantitative interviews, or paper-and-pencil registration. Finally, participants provided demographic and professional information (e.g., age, sex, country, profession), including their current experience with EMA and whether they would recommend it to others.

2.3. Data analysis

Survey responses were extracted from Google Forms and analyses were performed in MATLAB on the entire sample, including assessments of both practitioners and researchers. Most 7-point Likert statements indicated participant agreement on certain topics. A t-test was then used to test whether the average score was higher than 4, which was taken to reflect agreement with the statements. If the test was not significant, opinions on this statement were regarded as inconclusive. We further tested whether there was a significant difference between practitioners or researchers, and whether the level of self-reported experience with EMA influenced a certain statement. Both variables were used as predictors (using a dummy variable to indicate practitioner/researcher status and a continuous variable to indicate the level of self-reported EMA experience) in a linear regression model using the Likert score of the statement as the outcome variable.

For categorical statements such as the top three of EMA goals, we used z-tests to test for differences in proportions of practitioners and researchers that endorsed a particular statement. To facilitate interpretation, we grouped goals that appeared similar in categories in the Figures (based on group consensus by the authors). The 95 % confidence interval of the bootstrapped order was calculated for each goal. Paired-samples t-tests were used to gain insight into the relative importance of requirements for EMA implementation. We corrected for multiple testing with the Benjamini-Hochberg procedure (Benjamini and Hochberg, 1995). This resulted in a new alpha, a = 0.024, below which p-values were considered statistically significant.

3. Results

3.1. Participant characteristics

Practitioners were primarily psychologists and psychiatrists (resp. 65.1 % and 21.3 %) and were mainly specialized in cognitive behavioral therapy (CBT, 71.9 %), followed by acceptance and commitment therapy (ACT, 25.8 %), and psychopharmacological treatment (24.7 %). Researchers were mostly PhD candidates (master level). They were younger (two-sample t = 3.44, p < .001) and more experienced with EMA (two-sample t = 4.87, p < .001) than practitioners. Demographic information is summarized in Table 1. Both practitioners and researchers indicated that EMA in clinical practice is useful (79.0 %) and would recommend it to others (75.5 %), with no significant difference between practitioners and researchers (|z| < 2, p > .046). The self-reported level of experience with EMA positively predicted opinions on the usefulness of EMA (β = 0.23, t = 2.42, p = .017), and whether they would recommend it to others (β = 0.3, t = 3.20, p = .002).

• 1.Potential goals of EMA for the practitioner

Practitioners and researchers indicated the utility of ten EMA goals for the practitioner on 7-point Likert scales, after which they provided a top three of goals they regarded as most useful (see Fig. 2). The EMA goals were rated similarly useful when comparing the Likert scales (means ranged 4.91–5.81). Goals that were most often selected in the top three were: gaining insight into the context specificity of symptoms (55.0 %), followed by identifying triggers (40.4 %), and elucidating the severity and frequency of symptoms (39.1 %). Goals that were picked least often were: receiving a notification in case of increasing complaints (i.e., alerts; 11.3 %), followed by receiving advice in the moment (12.6 %), and monitoring the effects of treatment (17.2 %). Researchers differed from practitioners in that they more often picked the alert goal (z = 3.67, p < .001). Other differences were not statistically significant (all |z| < 1.67, p's ≥ 0.096). Although the alert goal was one of the least picked goals as described above, when asked directly how often practitioners wanted to receive notifications about patients' wellbeing, 25.8 % of the practitioners indicated weekly, 7.9 % monthly, 46.1 % only when the client's complaints are too alarming, and 20.2 % never wanted to receive notifications.

Fig. 2.

Difference between practitioners and researchers in the top three most useful goals for practitioners (in %). Data on goals were grouped in improving (diagnostic) insights, process monitoring, and real-time interventions. The 95 % confidence interval of the bootstrapped order is shown between parentheses (e.g., identify triggers is in 95 % of bootstrap iterations between rank 2 and 5).

Goals could be grouped within three categories, which we qualitatively labeled: (1) improving (diagnostic) insights, (2) process monitoring, and (3) real-time interventions. Following this distinction, the most frequently selected goals could be categorized in improving insights (see Fig. 2), whereas real-time intervention goals were selected the least.

• 2.Potential goals of EMA for the patient

Similar to the goals for the practitioner, Likert scales indicated similar utility of EMA goals for the patient (means ranged from 4.84 to 5.61). When asked to pick the top three goals for the patient, practitioners and researchers indicated on average the context specificity goal as most useful (55.0 %), followed by learning the effects of behavior and events on symptoms (55.0 %), and identifying triggers (53.0 %). Goals that were picked least often were gaining insight into the severity and frequency of symptoms (25.8 %), regaining control over one's life and complaints (30.5 %), and elucidating the associations between symptoms (37.7 %). Researchers differed from practitioners in that they more often picked the context specificity goal (z = 3.96, p < .001), and less often picked the goal of learning the effects of behavior and events on symptoms (z = 2.35, p = .019). No other differences between practitioners and researchers were found (all |z| ≤1.76, p's ≥ 0.079). Data on goals were grouped in associations between symptoms or behavior, and patients' insight and self-management (see Fig. 3).

• 3.EMA compared to treatment-as-usual

Fig. 3.

Difference between practitioners and researchers in the reported top three most useful goals for patients (in %). Data on goals were grouped in associations between symptoms or behavior, and patients' insight and self-management. The 95 % confidence interval of the bootstrapped order is shown between parentheses.

Practitioners and researchers were asked to choose in which treatment phase (i.e., diagnostic, intervention, and evaluation phase) EMA would be most useful. No significant differences in the proportion of practitioners and researchers were found, indicating that using EMA in all three phases was viewed as equally useful. Furthermore, practitioners (but not researchers) were asked to compare their treatment-as-usual (e.g., semi-structured diagnostic interviews, validated screening surveys, quantitative interviews and/or paper-and-pencil diaries) to EMA. Practitioners indicated that, compared to treatment-as-usual, EMA is more helpful for the diagnostic (M = 4.78, t = 5.47, p < .001), intervention (M = 5.20, t = 9.11, p < .001), and relapse prevention (M = 4.65, t = 4.94, p < .001) phases.

Practitioners were also asked to compare treatment-as-usual with perceived advantages and disadvantages of EMA (see Table 2). Regarding advantages, practitioners indicated that EMA provides additional information that cannot be collected with their current instruments, and that EMA provides patients more insight into their problems. Furthermore, practitioners indicated that EMA is easier to use, more reliable, and yields results that are easier to interpret compared to traditional instruments.

Table 2.

Practitioners' reported advantages and disadvantages of EMA compared to treatment-as-usual.

Characteristics	Mean	t	pa
Advantages
Provides additional information that cannot be collected with current instruments	5.52	11.34	<0.001
Provides the patient more insight into his/her problems	5.48	12.50	<0.001
Is easier to use	4.87	5.30	<0.001
Is more reliable	4.54	3.84	<0.001
Results are easier to interpret	4.52	3.61	<0.001
Helps me better with diagnosis	4.78	5.47	<0.001
Helps me better during treatment	5.20	9.11	<0.001
Helps me better in relapse prevention	4.65	4.94	<0.001
Disadvantages
Requires more effort for the patient (e.g., more time)	4.48	3.17	0.002
Requires more resources (e.g., infrastructure, software)	4.91	6.05	<0.001
Is more expensive	4.22	1.29	0.200
Contains more risks for the practitioner (e.g., not noticing a crisis)	3.01	−6.00	<0.001
Contains more risks for the patient (e.g., worsening of complaints)	2.89	−7.91	<0.001
Is less suited for treatmentb	2.67	−10.47	<0.001
Is more difficult to understandb	3.15	−7.86	<0.001

^aBonferroni-Holm corrected alpha = 0.024. A t-test was then used to test whether the average score was higher than 4, which was taken to reflect agreement with the statements.

^bThe items were reverse coded.

Regarding disadvantages, practitioners considered EMA to require the patient more effort and to require more resources (e.g., infrastructure, software) than treatment-as-usual. Practitioners were undecided on whether EMA is more expensive than treatment-as -usual. The other disadvantages were not endorsed: EMA was not considered to yield more risks for the practitioner (e.g., not noticing a crisis), nor for the patient (e.g., worsening of complaints). Furthermore, EMA was not considered less suitable for treatment, or more difficult to understand for the practitioner.

• 4.Requirements regarding EMA diary construction

Practitioners and researchers rated several requirements regarding the construction of an EMA diary on four themes, which were: (1) standardized versus personalized diaries, (2) item formulation, (3) measurement schedule, and (4) time constraints.

In general, practitioners and researchers preferred personalized to standardized diaries (M = 4.97, t = 9.54, p < .001). In addition, both groups preferred to create their own measurement schedule (e.g., the start/end point, timing, and frequency of assessments), instead of a standardized schedule based on patient diagnosis (resp. M's = 5.10 and 2.75, paired t = 12.2, p < .001). Although participants indicated that personalization is important for EMA diary construction, they also suggested that EMA should not cost practitioners (M = 5.79, t = 18.0, p < .001) nor patients (M = 5.93, t = 21.4, p < .001) much time.

Linear regression analysis demonstrated that practitioners were more pronounced in their preference for personalized diaries than researchers (β = −0.51, t = 2.3, p = .021). The level of experience with EMA was not a significant predictor for personalized diaries (β = −0.06, t = 0.58, p = .56). Practitioners did not differ from researchers in their preference for personalized measurement schedules (β = 0.17, t = 0.70, p = .48), but the level of experience with EMA increased their preference for personalized measurement schedules (β = 0.29, t = 2.40, p = .017). Furthermore, practitioners and researchers did not differ in their preference for low-time investment for creating the EMA diary (β = −0.67, t = 2.26, p = .032), nor was EMA experience a significant predictor (β = −0.072, t = 0.69, p = .49). Finally, compared to the practitioners, researchers found short EMA diaries less important for patients (β = −0.67, t = 3.52, p < .001). The level of experience with EMA was not predictive of their preference for short EMA diaries (β = 0.04, t = 0.4, p = .69).

• 5.Requirements regarding EMA feedback

Practitioners and researchers were inconclusive on whether they wanted to obtain concrete advice about the meaning of EMA feedback (e.g., advice about diagnosis or treatment selection) instead of just visualizing the collected EMA data (resp. M's = 4.36 and 4.00, paired t = 2.28, p = .024). This was unrelated to experience, nor was there a significant difference between practitioners and researchers (|t| < 2.1, p > .042). Both agreed that the patient should have access to a summary of all data (e.g., notes or graphs of all sessions; M = 5.90, t = 22.5, p < .001), as well as feedback in-between treatment sessions (e.g., a comparison with last session; M = 5.23, t = 10.27, p < .001). Practitioners and researchers further wanted to enable patients to analyze the data themselves (M = 4.42, t = 3.08, p = .002) and that after treatment, the patient should be able to continue to collect data about themselves and receive feedback on this data (M = 4.89, t = 6.5631, p < .001).

• 6.Technical requirements regarding the use of EMA

Practitioners and researchers rated several technical requirements regarding the use of EMA on three topics, which were: (1) integration of EMA in electronic health records (EHR), (2) payment and support options (e.g., training for practitioners and patients), and (3) EMA data preservation.

In general, both practitioners and researchers preferred to integrate EMA in electronic health records (M = 5.13, t = 8.2, p < .001). Post hoc linear regression analyses highlighted that Dutch participants were more likely to prefer this integration compared to Belgian participants (β = 0.79, t = 2.91, p = .004). The level of experience with EMA was not meaningfully related to the integration of EMA in electronic health records (|t| < 1.79, p > .076). Practitioners and researchers were undecided on whether EMA software should cost money (M = 4.02, t = 0.152, p = .879), but researchers were more willing to pay for software costs (β = 0.83, t = 3.04, p = .003). The level of experience with EMA was not meaningfully related to the willingness to pay for software costs (β = 0.12, t = 0.91, p = .37). Nonetheless, both practitioners and researchers required availability of technical support, such as a fast and responsive helpdesk (M = 5.42, t = 12.64, p < .001). Practitioners and researchers further indicated that the practitioner should receive training on the use of EMA (M = 5.77, t = 16.7, p < .001), whereas they were inconclusive with regards to the necessity of patient training (M = 4.12, t = 0.79, p = .43). No consensus was found on how long the EMA-data should be stored: some indicated that the data should be deleted after a year (30 %), while others indicated that the patient (28 %) or practitioner (12 %) should delete the EMA data at a particular moment in time.

4. Discussion

The present quantitative study explored the perspectives of practitioners and researchers on the utility of the intensive self-monitoring method EMA for mental health care. In particular, we examined potential goals of EMA for diverse treatment phases (i.e., diagnostic, intervention, or relapse prevention phase), as well as the requirements regarding diary construction, feedback, and technical requirements of EMA. In addition, we compared practitioners' most-used instruments with the advantages and disadvantages of EMA. Both practitioners and researchers indicated that EMA is a useful mental health tool and would recommend it to others. EMA was considered especially useful for gaining insight into the context specificity of symptoms, regardless of treatment phase. Furthermore, compared to treatment-as-usual, EMA was considered easier to use and interpret, but also more burdensome for the patient. Notably, both practitioners and researchers indicated the importance of personalization of EMA, user-friendliness of the software, and asked for training and support when integrating EMA in clinical practice. Both also highlighted that EMA demands more resources and should not require the practitioner or patient too much time or money. Table 3 summarizes the main recommendations for the clinical use of EMA, as brought forward by the findings of this study.

Table 3.

Recommendations on the clinical use of EMA based on the findings of this study.

Place of EMA in care	-EMA can be useful any phase of care: diagnostics, intervention and follow-up care -Most useful goal is gaining insight in context specificity of symptoms for both practitioner and patient -Compared to TAU, EMA is considered easier to use and interpret -However, EMA was also considered more burdensome for patients than TAU
Diary construction	-EMA diary should be personalized, not standardized based on diagnosis -Measurement schedule should be personalized, not standardized based on diagnosis -EMA diary construction should cost practitioner and patient minimal time -Completing the EMA diary should cost patient minimal time
Feedback	-Feedback should consist of a mix between freely exploring the EMA data and offering concrete advice on diagnosis and treatment -Patients should have access to a summary of the data (e.g., notes or graphs of all sessions) and feedback in between treatment sessions -Patients should be able to analyze the data themselves -Patients should be able to continue EMA after treatment has finished -Even though receiving alerts was the least-picked goal, most practitioners want the option to receive alerts on the well-being of the patient at a self-chosen timing
Practical points	-Software is preferably integrated with personal health record systems -Technical support options: helpdesk required -EMA software should not cost too much money -Practitioners should be trained in EMA diary construction and interpreting EMA feedback -Necessity for patient training is considered inconclusive – perhaps practitioner training is sufficient -Preferred duration of data storage is considered inconclusive

Note. Abbreviations: TAU = treatment-as-usual.

Our findings provide larger-scale quantitative support for qualitative findings showing that the potential of EMA for mental health care is recognized by practitioners (Bos et al., 2020; Bos et al., 2019). Similarly, more and more qualitative studies show patient enthusiasm for the use of EMA technology in their treatment (Folkersma et al., 2021; Palmier-Claus et al., 2013; Saunders et al., 2017). Practitioners reported that EMA yields additional information, is easier to use, and more reliable compared to currently used instruments in treatment-as-usual, such as diagnostic instruments, questionnaires, and paper-and-pencil monitoring. Our positive results contrast to previous work detailing practitioners' hesitance to work with EMA in treatment (Frumkin et al., 2020; Ellison, 2020). In a large (N = 375) survey study in the United States (U.S.), clinical psychologists perceived EMA-based tools to be less useful compared to existing instruments. The differing results may be due to differences in the American versus Belgian/Dutch health care systems. In the U.S., mental health care is often not insured (Chung, 2017), while the health insurance systems in Belgium and The Netherlands are part of the social security system (Health Insurance, 2021). The U.S. National Council for Mental Wellbeing reported that cost and poor insurance coverage are the top barriers for accessible mental health care (Paul et al., 2018). Speculatively, the Dutch and Belgian health care systems may offer practitioners more room to experiment with novel treatment innovations. Another possible explanation may lie in the different respondent sample. Most practitioners in our sample were psychologists and psychiatrists who worked in specialist mental health care (77.5 %), in contrast to the study by Ellison, which focused on U.S.-based psychologists working in primary care. In more specialist care, there is more need as well as available resources for detailed self-monitoring instruments that improve patient self-management (Murnane et al., 2016). This could tentatively suggest that EMA may be more useful in specialist, long-term care (Riese et al., 2021).

Regarding the goals of EMA, practitioners and researchers both indicated that EMA could be especially insightful in the context specificity of symptoms. This advantage of EMA has been highlighted in numerous publications (Myin-Germeys, 2012; van Os et al., 2017), and is an important part of mental health care (Beck, 2011). Interestingly, although frequently highlighted by researchers as a promising clinical application of EMA data (Epskamp et al., 2018; Kroeze et al., 2017; Bak et al., 2016), the elucidation of associations between symptoms was less frequently selected. Associations between symptoms can be computed and visualized using statistical techniques in person-specific networks (von Klipstein et al., 2020). However, these analytic techniques have important limitations, such as the reliability of the insights (Bastiaansen et al., 2020b) and the potential for misinterpreting causality (Bringmann et al., 2015). Notably, practitioners find them difficult to understand (Frumkin et al., 2020). Some researchers have therefore concluded that person-specific networks of symptoms or (affective) experiences in psychotherapy should not be used in clinical practice (Wichers et al., 2021), or should be used only exploratively (von Klipstein et al., 2020). Our present findings show that EMA feedback should focus on visualizing how symptoms and experiences vary according to context, in order to be useful to practitioners. Furthermore, whereas some practitioners and researchers in our sample preferred concrete advice on the meaning of EMA feedback for diagnosis and treatment, others opted for more freely exploring the visualizations of the EMA data and drawing their own conclusions. As such, more research is needed into the exact shape of the feedback to meet the expectations of practitioners and patients.

Interestingly, practitioners and researchers were less interested in providing real-time alerts or advice if EMA data indicated elevated symptoms. This contrasts to previous work showing evidence that ecological momentary interventions (EMI) can be successfully used for a variety of symptoms (Heron and Smyth, 2010). One reason for this may be that the practitioners in our sample were less familiar with this application of EMA, and feared to be overburdened by setting up or using an EMA tool (Bos et al., 2019). When the tool notifies the practitioner too often, they may experience it as too burdensome.

Additionally, our results point towards an intriguing inconsistency in practitioners' preferences for the practical implementation of EMA. On the one hand, and in line with previous qualitative work (Bos et al., 2019), practitioners and researchers preferred personalized to standardized EMA diaries. This means that practitioners and patients should be enabled to decide together on the content of the EMA diary, as well as the measurement schedule. Interestingly, researchers were less pronounced in this preference, indicating that the clinical application of EMA may differ from the use of EMA in research. At the same time, however, practitioners and researchers highlighted that EMA diary creation should take as little time and effort as possible, and they expected EMA to be more burdensome and require more resources. This means that an important factor in the development of EMA software is to strike a balance between personalization and efficiency. Furthermore, the finding that the preference for personalization was related to more experience with EMA suggests that EMA cannot be simply offered ‘as is’ to practitioners, but that they will need to be trained and offered support options (Daniëls et al., 2019). Another way to lessen the burden for practitioners would be to integrate EMA software in EHR systems, which was preferred significantly more by Dutch participants than their Belgian counterparts. The differing results can be explained by the Euro Health Consumer Index (EHCI), a comparison of European health care systems based on waiting times, results, and generosity (Euro health consumer index, 2018). The Netherlands has a higher index compared to Belgium and is therefore a more consumer friendly healthcare system. Taken together, these findings suggest that clinical EMA tools should be user-friendly, offer personalization options, be embedded in clinical software systems, and offer training for their use in clinical practice.

Strengths of our study include the in-depth investigation into practitioner and researcher perspectives on the clinical utility of EMA, and practitioners' attitudes towards EMA compared with other assessment techniques. However, our study findings should be interpreted in light of several limitations. First, although we broadly distributed the survey intending to reach a diverse group of participants, we may have mostly reached practitioners that already had some knowledge of EMA. As such, we may have overestimated its uptake in clinical practice. However, it is likely that, when implemented, EMA will first be adopted by practitioners with some experience with EMA (Grol and Grimshaw, 2003). Second, we focused our survey on Dutch and Belgian practitioners and researchers. Our results may be of limited generalizability to other countries with different mental health care systems. Third, we did not ask practitioners about their experience in gathering and analyzing EMA data. Future research might address the proportion of practitioners with this kind of experience, as those who already have some hands-on experience are likely to be early adopters if EMA would be rolled out in clinical practice.

To conclude, this study adds to accumulating evidence from qualitative studies that practitioners and researchers consider EMA to be helpful as a mental health tool. In particular, EMA may improve insight into context specificity of symptoms in diverse phases of care. Practitioners saw numerous advantages of EMA, such as improved insight into patient's problems, increased reliability of patient's insights, and better ease of use compared to treatment-as-usual. However, they also recognized that EMA demands more resources and effort of the patient. Both practitioners and researchers highlighted the need for personalization of the EMA diary. Future research should incorporate the views of practitioners on clinically relevant data visualizations and more user-friendly EMA software.

Abbreviations

EMA

ecological momentary assessment

ESM

experience sampling method

EHR

electronic health record

TAU

treatment-as-usual

Funding

This study was financially supported by Innovation Fund “Stichting De Friesland” (grant no. DS81 awarded to Dr. Riese and prof. Dr. Wichers), the charitable foundation “Stichting tot Steun VCVGZ” (grant no. 239 awarded to Dr. Riese and Dr. Bastiaansen), the “KU Leuven technology transfer grant” (C3/3H200318 awarded to Dr. Mestdagh and Dr. Kuppens), and the “KU Leuven research council grant” (C14/19/054 awarded to Dr. Kuppens). Dr. Mestdagh and Dr. Dejonckheere are post-doctoral research fellows supported by the Fund for Scientific Research, Flanders (FWO; grant no.'s 1256221N and 1210621N). The funding agencies had no role in the interpretation of the results and the drafting of this manuscript.

CRediT authorship contribution statement

Conceptualization: MP, FMB, MM, and ED.

Data acquisition: MP and JB.

Funding acquisition: HR and MM.

Formal analysis: MP and MM.

Project coordination: FMB.

Supervision: FMB and MM.

Writing – original draft: MP.

Writing – review and editing: MP, FMB, MM, JW, JB, HR, PK, and ED.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix A. Supplementary data

Supplementary material