Patients' experience of an ecological momentary intervention involving self-monitoring and personalized feedback for depression

Abstract

Experts in clinical mental health research count on personalized approaches based on self-monitoring and self-management to improve treatment efficacy in psychiatry. Among other things, researchers expect that Ecological Momentary Interventions (EMI) based on self-monitoring and personalized feedback will reduce depressive symptoms. Clinical trial findings have, however, been conflicting. A recent trial (ZELF-i) investigated whether depression treatment might be enhanced by an add-on EMI with self-monitoring items and feedback focused on positive affect and activities (Do-module) or on negative affect and thinking patterns (Think-module). There was no statistical evidence that this EMI impacted clinical or functional outcomes beyond the effects of regular care, regardless of module content. In apparent contrast, 86% of the participants who completed the intervention indicated they would recommend it to others. In the present study, we used in-depth interviews (n = 20) to better understand the EMI's personal and clinical benefits and downsides. A thematic analysis of the interviews generated six areas of impact with various subthemes. In line with the trial results, few participants reported behavioral changes or symptom improvement over time; the self-assessments mainly amplified momentary mood, in either direction. The most often mentioned benefits were an increase in self-awareness, insight, and self-management (e.g., a stronger sense of control over complaints). Consistently, these domains received the highest ratings in our evaluation questionnaire (n = 89). Furthermore, the EMI instilled a routine into the days of individuals without regular jobs or other activities. Participants reported few downsides. The experiences were rather similar between the two modules. This study suggests that EMI might contribute to health by helping individuals deal with their symptoms, rather than reducing them. Measures on self-awareness, insight, and self-management should be more emphatically involved in future EMI research.

Highlights

• •Experiences of a self-monitoring intervention for depression were assessed.
• •Contrary to previous concerns, participants reported little downsides.
• •Few participants experienced behavioral changes or symptom improvement over time.
• •Many participants described benefits in self-awareness, insight and self-management.

1. Background

Enabled by advances in mobile technology, experience sampling and ecological momentary assessment techniques (Larson and Csikszentmihalyi, 1983; Shiffman et al., 2008) are increasingly being used in clinical settings. By repeatedly sampling symptoms, behaviors and experiences during their daily life, patients gather fine-grained, real-world information that could potentially benefit their diagnosis and treatment (Van Os et al., 2013; Wichers, 2014). Researchers and experts in clinical mental health research have touted self-monitoring as a promising way to personalize treatments and optimize outcomes (Colombo et al., 2019; Elfeddali et al., 2014; Myin-Germeys et al., 2018). Clinicians and clients seem to embrace it as an opportunity to increase an individual's self-awareness, illness insight, and self-management (Bos et al., 2019).

There are many conceivable applications of these monitoring techniques in clinical care. Ecological Momentary Interventions (EMI) comprising systematic self-monitoring and personalized feedback might, for instance, reduce depressive symptoms by increasing self-awareness (Kauer et al., 2012) and inducing behavioral change (Myin-Germeys et al., 2018). Research on the actual efficacy of such EMIs in clinical depression is, however, limited (Colombo et al., 2019; Myin-Germeys et al., 2016) and findings are contradictory. In one randomized controlled trial, the EMI improved depressive symptoms compared to pharmacological treatment alone (Kramer et al., 2014), and these changes in depressive symptoms were related to changes in daily life behaviors such as physical activity and talking (Snippe et al., 2015). However, a more recent trial (Bastiaansen et al., 2020) did not find improvements in symptoms or social functioning beyond the effects of regular depression treatment (in this case, psychotherapy alone or in combination with pharmacotherapy). Furthermore, in neither study were empowerment increases in the experimental groups significantly different from the control group (Bastiaansen et al., 2020; Simons et al., 2015). Hence, clinical trial findings currently do not seem to match EMI's promise of improved efficacy.

When evaluating an intervention, the quantitative methods used in clinical trials are necessary but not sufficient; qualitative methods focusing on experiences and opinions of patients are an essential component of health care research as well (Forchuk et al., 2015; Pope et al., 2000). Patients' perspectives seem particularly relevant when evaluating self-management interventions. Despite this, very little is known about how people with depression perceive the clinical usefulness of EMIs. In a rare qualitative study, the majority of participants (16/22, among which 10 depressed individuals) had no actual experience with the technology (Bos et al., 2019). In a personal case study, Peter Groot (2010) described how self-monitoring supported his recovery from depression by establishing a process of conscious self-awareness of his ‘aberrant’ responses to the environment. He depicted feeling “better ‘protected’ and more ‘empowered’ against the vulnerability with which I have apparently been cursed” (p. 354). It is not known whether these experiences are shared by other individuals.

The main aim of the current study was to develop an understanding of the personal and clinical benefits and downsides of using an EMI, as perceived by people with depression. We examined participants' experiences in the previously mentioned trial on the efficacy of an EMI for depression in routine clinical practice (Bastiaansen et al., 2020) by means of an add-on qualitative study (complemented with a questionnaire). The clinical trial did not find statistical evidence that the EMI impacted clinical or functional outcomes beyond the effects of regular care. Nonetheless, 86% of the participants who completed the intervention reported that they would recommend it to others. This raises the question what the EMI did bring participants. A qualitative interview about the experiences with and impact of the EMI, which was conducted shortly after the intervention, might help to elucidate this.

In the trial, two experimental groups engaged in 28 days of systematic self-monitoring (5 times per day), and received weekly feedback on either positive affect and activities (Do-module) or negative affect and thinking patterns (Think-module). These two modules were based on common behavioral and cognitive strategies that have been found effective in the treatment of depression (Cuijpers et al., 2019). A secondary aim of this study was to compare patients' experiences with these two methodologically similar EMI modules with a different focus.

2. Material and methods

The ZELF-i study (‘Zelf’ is Dutch for self, i stands for intervention) is a pragmatic randomized controlled trial with three arms, which evaluates the efficacy of two different EMI modules (see Section 2.2.1 The Intervention) as an add-on for regular depression treatment (Bastiaansen et al., 2020). Outpatients starting depression treatment at secondary mental health services were enrolled in the study as soon as possible after clinical intake. Regular treatment was not adapted and started upon availability (almost all participants received a form of psychotherapy at one point during the study period). The control group received no additional intervention. Participants completed questionnaires on depressive symptoms (primary outcome), social functioning, and empowerment before and after the intervention period, and at four measurements during a 6-month follow-up period. Data acquisition for the trial took place between May 2016 (first study intake) and March 2019 (last follow-up). More detailed information on the inclusion criteria and research procedures have been described elsewhere (Bastiaansen et al., 2018). The interviews for the current article were conducted shortly after participants completed the EMI, while data acquisition for the trial was ongoing (and hence trial results were still unknown). Reporting of this study was done according to the consolidated criteria for reporting qualitative research (COREQ: Tong et al., 2007, checklist in Appendix A). The institutional review board of the University Medical Center Groningen (UMCG, no. 2015/530) approved the trial and qualitative add-on study.

2.1. Participants

For the clinical trial, 161 adults with depressive complaints were recruited, who were referred to outpatient secondary mental health services in the Netherlands. The main inclusion criteria were a clinical diagnosis of depression and primary indication for depression treatment by the practitioner. Individuals were excluded if a crisis intervention was warranted and in case of psychotic or manic symptoms, or incapability of following the research procedures. All participants provided written informed consent.

For the evaluation questionnaire, we included ‘completers’ of the treatment arms (n = 90), that is, participants who completed the baseline, the 28-day intervention period, and the feedback session (Bastiaansen et al., 2020). Dropouts (n = 20) were excluded.

For the qualitative study, we invited completers by telephone for an interview, until data saturation occurred (Glaser and Strauss, 1967). Participants were selected purposively using a maximum variation strategy (Coyne, 1997; Marshall, 1996) to ensure diversity in demographic and clinical characteristics that might influence experiences with the intervention: gender, age, educational level, daytime activity, concurrent psychotherapy, intervention module (Do/Think), and response compliance for the self-assessments. That is, we created a sampling grid (Marshall, 1996) based on these variables and recruited participants that reflect various combinations of these variables. The resulting sample (n = 20) was diverse, but with a dominance of younger participants with mid-level education and moderate levels of depression severity. The majority had moderate or high compliance levels, due to the exclusion of dropouts and the scarcity of completers with low compliance in the trial (i.e., only 6 had compliance ≤50%). Thirteen participants were initially selected but not interviewed, because they could not be reached (n = 5), declined participation (n = 4), or for other reasons (n = 4) such as no-show. See Table 1 for characteristics of the completers (n = 90) and the qualitative subsample (n = 20).

Table 1.

Participant characteristics.

Characteristic	Qualitative subsample (n = 20)	Intervention group (n = 90)
Module
Do	10	48
Think	10	42
Gender
Male	13	41
Female	7	49
Age
18-30	10	45
31-45	4	24
46-65	6	21
Educational levela
Low	3	20
Middle	14	49
High	3	21
Daytime activity
Study	7	20
Paid work	6	37
Household	1	3
No employment	6	30
Depression severityb
Normal/no (0−13)	1	2
Mild (14-25)	1	11
Moderate (26-38)	15	45
Severe (39-48)	3	25
Very severe (49 >)	0	7
Compliance (self-assessments)
≤50%	2	6
51-74%	5	30
≥ 75%	13	54

Note. Characteristics of the intervention group, which comprises all completers of both treatment arms, and the qualitative subsample.

^aEducational level – low: no/primary/low secondary, middle: high school/low vocational, high: higher vocational/university.

^bScores on the self-report version of the Inventory of Depressive Symptomatology (IDS-SR) at baseline (http://www.ids-qids.org/interpretation.html [last accessed: March 2021], derived from Rush et al., 2003).

2.2. Material and procedure

2.2.1. The intervention

The EMI comprised 28 consecutive days of systematic self-monitoring (i.e., five brief questionnaires per day at fixed time points taking approximately 2 min to complete) in combination with four weekly digital feedback reports and one face-to-face feedback session to discuss the fourth and final feedback report. The two EMI modules had identical procedures, but differed in the focus of self-monitoring items and feedback: positive affect and activities in the Do-module, and negative affect, thoughts, and daily events in the Think-module. Examples of self-monitoring items and personalized feedback graphs are provided in Fig. 1.

Fig. 1.

Examples of self-monitoring items and personalized feedback graphs. (a) Smartphone display of two exemplary items with sliders. (b) Examples of feedback graphs for the Do-module: mean day-level of positive affect (PA) across the 28-day intervention period, and amount of PA experienced per type of activity. (c) Examples of feedback graphs for the Think-module: mean day-level of negative affect (NA) across the 28-day intervention period, and amount of NA experienced after a negative or positive event. Images were adapted from feedback reports used in the ZELF-i trial (for example reports see https://osf.io/m6hvg/, and for more information on the reports and feedback session see (Bastiaansen et al., 2020).

In both modules, self-assessments started with questions on momentary well-being, momentary affect, and momentary physical state, and ended with a question on measurement burden. In between, participants in the Do-module retrospectively recorded experienced pleasure, motivation, physical activity, busyness, time spent at home, in pleasant social contexts, and outdoors, and performed activities; and prospectively recorded anticipatory pleasure and motivation. Items deliberately focused on positive contexts and activities to help participants monitor changes in their behavioral patterns, and ultimately increase their activity level, especially in pleasurable activities. Participants in the Think-module retrospectively recorded how much they focused on feelings, the amount of brooding, the occurrences of specific negative and positive events, and the presence of both negative and positive thoughts; and prospectively recorded worrying. Items were chosen to increase personal insights in daily events and participants' reactions to them with the ultimate goal of reducing negative affect (Bastiaansen et al., 2020, pg. 3).

Feedback reports mainly comprised graphs on descriptive statistics (e.g., average daily mood) and contained increasingly rich information across the intervention period (e.g., by showing participants' changes in activities and mood from week to week). For those participants who filled out at least 75% of the measurements, the fourth report additionally included feedback on temporal relationships between sets of variables (e.g., positive affect and physical activity (Do-module), or negative affect and rumination (Think-module)). A more detailed explanation of the intervention has been described elsewhere (Bastiaansen et al., 2018, Bastiaansen et al., 2020).

2.2.2. The evaluation questionnaire

Participants filled out an evaluation questionnaire directly after the feedback session. Similar to the previous trial (Kramer et al., 2014), this survey included questions on the feasibility and usability of the EMI (Bastiaansen et al., 2020, Appendix C). We added several questions (Table 2) to obtain a rough estimation on how the intervention impacted participants in the treatment arms. The study is, however, primarily a qualitative analysis in a purposively diverse subsample to obtain a rich understanding of participants' experiences with the EMI.

Table 2.

Patient-perceived impact of the intervention (post-intervention evaluation questionnaire).

	Total (n = 89)a	Do-module (n = 47)	Think-module (n = 42)	t/χ2	df	p
Was participation in the study a positive experience?	66.0 ± 16.6	64.2 ± 16.9	68.1 ± 16.2	−1.1	87	0.27
Did you gain anything by participating?	60.1 ± 20.1	58.0 ± 22.0	62.4 ± 17.8	−1.0	87	0.31
Would you recommend the intervention to others? (% Yes)b	87%	85%	88%	0.0b	1	0.92
Did you find the structure (5 measurements per day for 28 days) pleasant?	70.0 ± 20.7	67.6 ± 22.4	72.7 ± 18.6	−1.2	87	0.24
Did you have to adjust your normal activities or daily routine due to the measurements?	28.0 ± 25.7	25.2 ± 24.4	31.1 ± 26.9	−1.1	87	0.28
Did the interim reports make you think about what you can do to influence your mood?c	47.4 ± 29.9	46.8 ± 30.6	48.3 ± 29.4	−0.3	84	0.79
Did using the EMI make you think differently about things or do things differently?	46.0 ± 26.5	44.1 ± 28.0	48.2 ± 25.0	−0.7	87	0.47
Did using the EMI make you feel you were actively dealing with your complaints?	62.6 ± 25.7	63.0 ± 27.3	62.2 ± 24.1	0.2	87	0.88
Did the EMI help you gain more insight in your mood?	65.5 ± 22.5	65.0 ± 22.5	66.1 ± 22.8	−0.2	87	0.82
Did you become more aware of how your mood related to your activities?	59.2 ± 25.2	58.9 ± 25.4	59.5 ± 25.3	−0.1	87	0.90
Did you become more aware of how your mood related to your thoughts/worrying?	57.7 ± 23.0	54.5 ± 22.9	61.2 ± 22.9	−1.4	87	0.17

Note. Numbers represent mean ± standard deviation for the questions rated on visual analogue scales (ranging from not at all (0) to very much (100)).

^aOne completer of the Think-module did not fill out the evaluation questionnaire, bringing the total to 89.

^bFor this one dichotomous variable, numbers represent percentages and outcomes of chi-squared testing.

^cThree Do participants indicated not to have read any interim reports and did not receive this question.

2.2.3. Interviews

Participants who were selected for the semi-structured interviews were contacted within two weeks after the feedback session to provide more information and schedule an appointment, which usually took place within three weeks of the invitation at their treatment location. At the time of the interview 12 out of 20 participants reported they had started psychotherapy at one of the study locations. Participation in the interview was optional and involved a separate informed consent form. Interviews were audio-recorded and lasted 45 to 60 min. They were conducted by experienced psychologists (WF and VV: female BIG-registered healthcare psychologists, MSc), who were trained in conducting and executing qualitative research. The interviewers did not share their personal goals and had no therapeutic relations with the participants, but were aware of the basic demographic and clinical characteristics used for participant selection; this did not affect how they conducted the interviews. Besides the participant and psychologist, no one else was present during the interview. The interview guide (Appendix B) comprised mainly open-ended questions with subsequent probes to gain as rich information as possible. It was prepared to cover impact domains suggested by research and/or theory (e.g., empowerment, depressed mood). The interview guide was pilot tested and updated after each interview when new topics arose (Boeije, 2002; Pope et al., 2000).

2.3. Data analysis

We used standard statistical approaches to summarize participants' responses on the evaluation questionnaire, and t-tests and chi-squared tests (for continuous and categorical variables, respectively) to evaluate differences between the EMI groups, using R (R Core Team, 2019). We focused on questions relating to intervention impact; answers to feasibility and usability questions have been published elsewhere (Bastiaansen et al., 2020).

A thematic content analysis was conducted as part of an iterative process according to the Qualitative Analysis Guide of Leuven (QUAGOL, Dierckx de Casterlé et al., 2012). Analyses were mainly inductive, based on empirical findings (i.e., participants' answers), but to a certain extent also guided by the interview guide, which was based on prior research. Field notes were made during the interviews and audiotaped interviews were transcribed verbatim, anonymized, and summarized in narrative reports, which participants were invited to comment on. Next, a concept code list was constructed based on themes identified in the data. WF and VV used this list to independently code transcripts. New codes were added when previously unidentified themes were encountered, and existing codes were more clearly defined in consensus meetings with MAA (PhD, health scientist and qualitative methods expert) and JAB (PhD, psychologist researcher). When no new information emerged to add to the code list, we carried out two more interviews to confirm that data saturation had indeed been reached. Finally, WF and VV grouped the codes in overarching coding categories or central themes. These central themes were verified against all transcripts and discussed with MAA and JAB. Participants' quotations, which are presented to illustrate our (sub)themes, were translated from Dutch by a native English speaking editor. Qualitative data coding, management and analysis were performed using ATLAS.ti version 8.3.1 (ATLAS.ti Scientific Software Development GmbH). All statistical and qualitative analysis codes (e.g., codebook and coding tree) that could be openly shared without privacy concerns can be found online (https://osf.io/ypkhv/).

3. Results

3.1. Evaluation questionnaire

One completer of the Think-module did not fill out the evaluation questionnaire, bringing the total to 89 (Table 2). In both modules, more than 85% would recommend the intervention to other people with depression. Overall, participants were moderately positive about whether they gained something from the intervention (i.e., a 60-score on a scale ranging from not at all (0) to very much (100), SD = 20.1). On average, the intervention helped participants (somewhat) in gaining insights into their own mood (M = 65.5, SD = 22.5) and the dynamic interplay with activities (M = 59.2, SD = 25.2) and thoughts (M = 57.7, SD = 23.0). Whereas the EMI seemed to help participants in feeling they were actively dealing with their complaints (M = 62.6, SD = 25.7), they were less positive about whether it gave them handles to make actual changes (M = 46.0, SD = 26.5). Furthermore, the daily structure of the measurements was generally experienced as pleasant (M = 70.0, SD = 20.7). There were no statistically significant differences between the two EMI modules (all p > 0.17). The large standard deviations suggest there were substantial differences in participants' experiences.

3.2. In-depth interviews

As evident from Table 3, the analysis generated 6 areas of impact with various subthemes, which were addressed by more (in black) or less (in gray) individuals.

Table 3.

Participants' themes and subthemes with exemplary quotes.

Note. Topics that were described by many participants are shown in black font, whereas those that were covered by some participants are shown in gray. Participants mostly described benefits (+); (additional) downsides are marked by a hyphen (–).

3.2.1. Self-awareness

3.2.1.1. Mindfulness

Most participants (in both modules) reported that the EMI made them focus on the present moment more and take notice of how they were feeling. The self-assessments were described as a time to pause in the now. Many participants indicated that this could be confronting at difficult times: “You try not to think about it and then you have to bring it up again to fill it in” (Do, male, 31-45 yrs.). Various participants indicated it could also be a pleasant experience at times. Participants, for instance, reported becoming more conscious of internally experienced changes (e.g., shifts towards more positive mood states) as well as of good things happening in the outside world (e.g., a pleasant encounter): “You stop to think about it more. That nice things are really nice whereas you normally just take them for granted” (Think, female, 46-65 yrs.).

3.2.1.2. Gaining perspective

The majority of participants (in both modules) reported that completing the self-assessments made them gain more perspective with regard to their complaints. This is illustrated by the following quote: “You also learn to take a bit of distance from it. In the beginning it is very uncertain as the sliders go up and down and all, but after a while you can put a little more distance between yourself and the measurement so you can look at yourself more abstractly” (Think, male, 31-45 yrs.). Some participants described how looking at their situation from a more distanced standpoint made them more aware of how they experienced certain problems and whether that related to reality. Some also mentioned it made them realize their problems were actually smaller (or in some cases worse) than they initially thought.

3.2.2. Insight

3.2.2.1. Mood dynamics

Most participants (in both modules) indicated that the EMI helped them gain more insight into the relationship between their feelings and the situations they were in. Participants, for instance, gained more insight into the context in which positive emotions were experienced: “That is also a question of awareness, that you think: oh yes, this is nice, that was nice ... [that person] has responded nicely and I have had a nice conversation with so and so, and now you see, it's all actually not so heavy and so dismal as you thought” (Think, female, 46-65 yrs.). Participants also reported that a better understanding helped them accept negative mood states: “It helped with making connections. Like ‘gosh, today I've been on the road all day so I'm probably tired so the chance that I feel worse now… oh, that's what's wrong.’ Then it's okay” (Do, female, 18-30 yrs.).

Many participants specifically stated that it was instructive to see a summary of their mood measurements in the reports. For some, the graphs provided insight on how strongly their mood states fluctuate throughout the day. For instance: “So I did have changes, I did notice that in the emotions, the moments of worrying. [...] So yes, those reports made me aware that it can have lots of ups and downs” (Think, female, 46-65 yrs.). For others, it was reassuring to note there were few extremes: “What you notice is that there were often not very great extremes and in a way I found that nice to notice” (Do, male, 31-45 yrs.).

3.2.2.2. Confirmation

For almost all participants (in both modules) the EMI, and particularly the feedback reports, confirmed what participants thought they knew about themselves. Some participants described that the reports made their complaints seem more ‘true’ or ‘tangible’. One participant (Do, male, 31-45) described that the reports made him feel recognized in his illness, which facilitated talking about his depression with others.

3.2.2.3. Retrospective bias

A considerable number of participants (in both modules) said that the momentary assessments gave a more accurate reflection of their mood, compared to situations in which they had to rate or describe their mood retrospectively. Retrospective assessments of mood can be biased by the most recent or intense experiences, or distorted by memory problems, as the following quote illustrates: “No, that also helps, because at the end of the whole day I sometimes have trouble remembering how I felt in the morning. It was easier that way, because then I was usually still in that moment” (Do, female, 18-30 yrs.).

3.2.2.4. Emotion differentiation

Several participants described that completing the measurements helped them become better at differentiating their emotions (i.e., noticing subtle differences in mood). For instance: “I started to think about it more. For example, ‘are you stressed?’ [...] and then I started to think for a moment, ‘is this really stress that I'm feeling right now?’” (Do, male, 18-30 yrs.).

3.2.3. Mood

3.2.3.1. Mood shift

Some participants said their mood improved during the intervention period. A few participants explicitly stated they did not experience clear mood shifts. None of the participants indicated that their mood deteriorated over time.

3.2.3.2. Mood amplification

Many participants (in both modules) indicated that completing the measurements amplified how they felt in that moment. One participant, for instance, said: “It reinforced the feeling; it was a bit like a catalyst for the whole thing” (Think, male, 31-45 yrs.). This measurement reactivity could be experienced negatively as well as positively, depending on the participant's current mood. On the one hand, participants mentioned that having to report on negative emotions when in a negative mood could induce frustration. On the other hand, participants indicated that they enjoyed filling in the measurements during a positive mood, because it emphasized the things they had undertaken and how good that made them feel.

3.2.3.3. Breaking the mood

Some participants (in both modules) noted that completing the measurements sometimes helped breaking a negative mood, as this quote illustrates: “At any rate you are taken out of the moment by the test.... So if I was pretty negative at the time because something wasn't going well, and then that thing comes along, you're dealing with it and it takes you out of the moment. Then I often scored myself higher than I had just been. In itself, it's good to notice that; that I can get out of such a state of mind more easily, or at least get taken out of it” (Do, male, 31-45 yrs.). A few participants in the Do-module stated that the measurements sometimes also interrupted their positive mood; they reminded them of why they were doing them in the first place, while they were just now feeling cheerful.

3.2.4. Self-management

3.2.4.1. Sense of control

Many participants (in both modules) described that the EMI induced a stronger sense of control over their complaints. Many explicitly mentioned experiencing more options for self-management as a result of increased self-awareness and insight: “How I was doing and why I felt that way. (...) if that's clear to me that means I should at least be able to do something about what caused it” (Do, male, 18-30 yrs.). Furthermore, various participants, particularly in the Do-module, indicated that doing the self-assessments was satisfying, because it made them feel they were actively dealing with their complaints.

3.2.4.2. Attitude to regular treatment

Various participants described how the EMI strengthened their position in regular treatment. One participant, for instance, disclosed that the intervention allowed him to draw his own conclusions, which he could support for the full 100% – as opposed to having others draw conclusions about him (Think, male, 18-30). Other aspects that were mentioned: the EMI helped determine the course of subsequent treatment, confirmed the importance of behavioral activation (Do-module) and the importance of challenging negative thoughts (Think-module) in their regular treatment.

3.2.5. Behavioral change

3.2.5.1. Motivation

A minority indicated that the intervention (and particularly the enhanced awareness by the repeated self-assessments) made them want to make behavioral changes. Particularly participants of the Do-module (which asks about performed activities) reported they wanted to commit themselves to doing at least something in the hours before the next measurement, even if it was only a small activity.

3.2.5.2. Activities

The majority, however, did not notice any actual impact of the EMI on their behavior. Some participants (particularly in the Think-module) reported that the measurements did encourage them to reach out to others more, for example, to do something together or to check the reality of their negative thoughts. A few participants mentioned they wished the EMI would have steered them more explicitly towards behavioral change, for instance, by questioning participants in more detail about what activities could affect their negative mood state (Think-module) or by encouraging them to get active after a few idle periods (Do-module).

3.2.6. Daily rhythm

3.2.6.1. Instilled structure

The interviews went into detail about whether the EMI (particularly the self-assessments at fixed time points) aided participants in structuring their day. Most participants (in both modules) experienced no impact on their daily structure. They described that their days were already structured around regular activities (e.g., their jobs). For a minority, the intervention did instill more structure. They mentioned, for example, that in the absence of other activities, the measurements set a rhythm for the day. For example, one participant stated: “I didn't have a lot of structure in my day while I was completing it, so for me it was helpful to have the structure it gave me” (Think, male, 18-30 yrs.).

3.2.6.2. Disturbance

Most participants generally did not experience fitting the self-assessments into their daily lives as a burden. Self-assessments were occasionally annoying when participants were busy doing something else (e.g., work, doctor's visit) or when they were in company (being busy with other things or other people were also the most frequently cited reasons for missing measurements). A few participants described feeling antisocial being on their phones or withdrawing themselves from a conversation, and one individual felt uncomfortable out of shame about their depression. A couple of participants made small adjustments to better fit the measurements into their schedule (e.g., by setting an alarm to wake up promptly or park the car in time for a new measurement). Note that many participants did state they would rather have had fewer measurements per day, but this was not so much burden-related; most of them simply thought that three (instead of five) daily measurements would have provided them with just as much insight.

3.2.7. Future directions

3.2.7.1. More personalization

Almost all participants gave tips for improvement that were related to enhancing the EMI's personal fit. Various participants, for instance, indicated they would have liked to be able to add their own comments to make measurements better searchable and interpretable (e.g., “If you're just really sick for a day, then of course those figures crash down too” (Do, female, 18-30 yrs.)). Personal comments could thus act as “a kind of bookmark” (Think, male, 46-65 yrs.). In a similar vein, various participants in the Think-module (but not the Do-module) said they would have preferred more detailed feedback, for example on specific events rather than global categories of positive versus negative events. Other opportunities for customization that were mentioned several times included the possibility to omit personally irrelevant questions and add personally relevant questions, changing the wording and/or the tone of the questions (e.g., adopting a more informal tone and addressing a participant directly by his/her name), and being able to repeat the intervention in a different period (as comparison) or continue using the EMI in adapted form. One participant, for instance, appreciated the standard module as a starting point, but would have liked to personalize the module with the practitioner over time (Think, male, 31-45 yrs.).

3.2.7.2. Integration of EMI into regular treatment

Participants' descriptions on how the EMI could complement regular treatment and/or how blended care could benefit patients suggest increased self-insight is the driving force. Various participants, for instance, foresee a role for EMIs as a complementary diagnostic tool and a way of increasing personal insights in the run-up to face-to-face therapy, as this quote illustrates: “It's a good way to gain insight and then to use this insight in further therapy” (Think, male, 18-30 yrs.). Furthermore, many participants describe how an EMI could provide support in-between therapy sessions and deepen those sessions by providing detailed and accurate information about their daily functioning, or as one participant puts it: “If you can connect the data to a thought, feeling or event, then you can immediately do more with it” (Think, male, 31-45 yrs.).

4. Discussion

Researchers expect EMIs involving self-monitoring and personalized feedback to improve treatment efficacy in depression, but clinical trial findings on their ability to reduce depressive symptoms have been conflicting. In this study, we used in-depth interviews (n = 20) to better understand such an EMI's personal and clinical benefits and downsides. A thematic analysis of the interviews generated six areas of impact: self-awareness, insight, mood, self-management, behavioral change, and daily rhythm. In line with the trial results (Bastiaansen et al., 2020), few participants reported behavioral changes or mood improvement over time. Yet the vast majority would recommend the intervention to others. In line with work on patients' and clinicians' expectations (Bos et al., 2019), the most often mentioned benefits of the EMI were an increase in self-awareness and insight, and a stronger sense of control over complaints as a result; questions related to these domains also received the highest average scores in the evaluation questionnaire (n = 89). An EMI might not only have added value by generating personal and “novel information” (Bos et al., 2019). Most participants in our study indicated that the EMI was helpful by confirming what they already knew at some level; the EMI made this knowledge more explicit or ‘measurable’. Furthermore, the EMI instilled a routine into the days of individuals without regular jobs or other activities. Participants' experiences were rather similar between the two modules.

Self-awareness and illness insight are rarely examined in intervention studies, although insight is often mentioned by people with depression as an outcome that matters (Chevance et al., 2020) and essential from a perspective on health as the ability to adapt and to self manage (Huber et al., 2011). In studies on self-management strategies, people with mood disorders often mention structured attention to oneself (van Grieken et al., 2015) and the development of a better illness understanding (Villaggi et al., 2015) as important in their recovery. Moreover, they describe how analyzing their thoughts, emotions, and behaviors, and gaining perspective on situations helps them in managing daily symptoms (Villaggi et al., 2015). The EMI seems to support exactly these strategies.

Increased self-insight is not necessarily mood-enhancing in itself. Many participants in our study described it could be confronting at times, but did experience more options for dealing with their symptoms as a result. Thus, an EMI might contribute to mental health by helping individuals optimize their well-being. Based on our one study, we cannot be sure whether self-awareness and insight are endpoints or intermediate stations on the way to symptom improvement. To examine this, self-monitoring studies would need to include these measures at pretest, posttest, and at follow-up. In one such rare study, Kauer et al. (2012) found that changes in depressive symptoms were mediated by increases in emotional self-awareness. More research is needed to investigate mechanisms of change.

Self-awareness nor insight3 were included as measures in the two existing EMI depression trials (Bastiaansen et al., 2020; Kramer et al., 2014). Both did include the Netherlands Empowerment List (NEL: Boevink et al., 2008), but found that the EMI did not increase empowerment significantly. In contrast, most participants in the qualitative study did mention that the EMI made them feel better equipped to act upon their complaints themselves and to take a more proactive attitude in treatment. It is possible that control participants experienced the same increase in personal strength. Alternatively, these specific intrapersonal changes might have gone lost in the overall score on the NEL, which predominantly covers interpersonal aspects of empowerment (e.g., social support). Perhaps other instruments are better suited to monitor the efficacy of self-management interventions.

According to a systematic review on continuous self-monitoring (Dogan et al., 2017), potential risks and adverse effects have been neglected in the literature. The authors speculate, for instance, that for people with depression self-assessments mean a daily confrontation with their perceived shortcomings, which might sustain or even worsen their symptoms. Patients who received an introduction to what an EMI entails, also expressed concern about potential symptom worsening and high assessment burden (Bos et al., 2019). However, in our study, participants who engaged in four weeks of self-monitoring had rather high compliance and reported few downsides. Self-assessments were not considered very burdensome, only occasionally inconvenient. Moreover, none of the participants indicated that their mood deteriorated over time. Although measurements could —at times— amplify negative mood, this effect was transient (and positive reactivity occurred as well).

In previous work, a focus on negative rather than positive self-aspects has been associated with increased negative affect (Mor and Winquist, 2002). In both modules the first part of each questionnaire measured momentary affect, but the second part and weekly feedback reports of the Think-module can be considered to have a more negative focus (thoughts, events, and negative affect) than the Do-module (activities and positive affect). Yet, even with regards to affect reactivity, participants' experiences did not really differ between the two modules. This is consistent with a related study, which did not find differential effects of the two modules on momentary affect throughout the intervention period (Ornée et al., 2021). The comparable findings for the two modules suggest that monitoring affect alone, or in combination with activities or thoughts, triggers generic changes in self-awareness, insight and self-management.

Our findings should be interpreted in the context of qualitative research: our aim was to provide an in-depth and contextualized understanding of patients' experience with an existing EMI. Therefore, caution is advised when generalizing these results. For one, experiences might vary based on the specific EMI application and treatment phase. Moreover, experiences might not be representative for all people with depression. Our sample comprised only those individuals who completed the EMI (while waiting for or starting regular treatment). Refusers and intervention dropouts might hold more negative views, particularly concerning burden and negative reactivity: ten participants quit the intervention due to practical or time constraints and four due to negative effects from completing the measurements (Bastiaansen et al., 2020). Future research should reveal whether the reasons people stop or refuse are inherent to certain patient characteristics or can be overcome, for instance, by tailoring the intervention more to the individual. The latter desire was also expressed by the completers: with more personalization, the EMI could become a reference book that patients can consult in the run-up to or during regular treatment. Lastly, we cannot exclude that other therapies could impact similar domains as the EMI. We can, however, conclude from participants' specific reports on the EMI that it brought them benefits beyond those of their regular treatment. This is further supported by the fact that participants who had already started regular treatment at the time of their interview reported similar benefits from the EMI as participants who did not start regular treatment until after their interview.

In conclusion, the EMI brought participants benefits that were not adequately covered by the trial's main outcome domains, and are often overlooked in intervention research. According to patients' experience, an EMI involving self-monitoring and personalized feedback contributes positively to self-awareness, insight, and self-management, regardless of module content. Our study suggests an EMI could contribute to mental health by helping participants deal with their symptoms, rather than reducing them. Measures on self-awareness, insight, and self-management should be included as outcome measures in future EMI trials (e.g., Therap-i: Riese et al., 2021) to better understand intervention impact and underlying mechanisms of change.

CRediT authorship contribution statement

WF, VV, AJO, and JAB conceived the study and its design. WF and VV conducted the interviews, coding and analyses with supervision by MAA and JAB. DAO worked with the re-analyses. JAB did the final write-up. All authors contributed to, read and approved the final manuscript.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix A. COREQ (COnsolidated criteria for REporting Qualitative research) checklist

COREQ (COnsolidated criteria for REporting Qualitative research) checklist
Topic	Item No.	Guide questions/description	Reported on page no.
Domain 1: research team and reflexivity
Personal characteristics
Section Data collection	1	Which author/s conducted the interview or focus group?	Interviews section (2.2.3)
Section Data collection	2	What were the researcher's credentials? E.g. PhD, MD	Interviews section (2.2.3)
Section Data collection	3	What was their occupation at the time of the study?	Interviews section (2.2.3)
Section Data collection	4	Was the researcher male or female?	Interviews section (2.2.3)
Section Data collection	5	What experience or training did the researcher have?	Interviews section (2.2.3)
Relationship with participants
Section Data collection	6	Was a relationship established prior to study commencement?	Interviews section (2.2.3)
Section Data collection	7	What did the participants know about the researcher? e.g. personal goals, reasons for doing the research	Interviews section (2.2.3)
Section Data collection	8	What characteristics were reported about the inter viewer/facilitator? e.g. Bias, assumptions, reasons and interests in the research topic	Interviews section (2.2.3)
Domain 2: study design
Theoretical framework
Methodological orientation and Theory	9	What methodological orientation was stated to underpin the study? e.g. grounded theory, discourse analysis, ethnography, phenomenology, content analysis	Data analysis section (2.3)
Sampling	10	How were participants selected? e.g. purposive, convenience, consecutive, snowball	Participants section (2.1)
Method of approach	11	How were participants approached? e.g. face-to-face, telephone, mail, email	Participants section (2.1)
Sample size	12	How many participants were in the study?	Participants section (2.1)
Non-participation	13	How many people refused to participate or dropped out? Reasons?	Participants section (2.1)
Setting of data collection	14	Where was the data collected? e.g. home, clinic, workplace	Interviews section (2.2.3)
Presence of non-participants	15	Was anyone else present besides the participants and researchers?	Interviews section (2.2.3)
Description of sample	16	What are the important characteristics of the sample? e.g. demographic data, date	Participants section (2.1) and Table 1
Interview guide	17	Were questions, prompts, guides provided by the authors? Was it pilot tested?	Interviews section (2.2.3)
Repeat interviews	18	Were repeat inter views carried out? If yes, how many?	Interviews section (2.2.3)
Audio/visual recording	19	Did the research use audio or visual recording to collect the data?	Interviews section (2.2.3)
Field notes	20	Were field notes made during and/or after the interview or focus group?	Data analysis section (2.3)
Duration	21	What was the duration of the interviews or focus group?	Interviews section (2.2.3)
Data saturation	22	Was data saturation discussed?	Participants section (2.1) and Data analysis section (2.3)
Transcripts returned	23	Were transcripts returned to participants for comment and/or correction?	Data analysis section (2.3)
Domain 3: analysis and findings
Data analysis
Number of data coders	24	How many data coders coded the data?	Data analysis section (2.3)
Description of the coding tree	25	Did authors provide a description of the coding tree?	Data analysis section (2.3) and online material (https://osf.io/ypkhv/)
Derivation of themes	26	Were themes identified in advance or derived from the data?	Data analysis section (2.3)
Software	27	What software, if applicable, was used to manage the data?	Data analysis section (2.3)
Participant checking	28	Did participants provide feedback on the findings?	Data analysis section (2.3)
Quotations presented	29	Were participant quotations presented to illustrate the themes/findings? Was each quotation identified? e.g. participant number	Data analysis section (2.3) and Results section (3.2) including Table 3
Data and findings consistent	30	Was there consistency between the data presented and the findings?	Results section (3.2)
Clarity of major themes	31	Were major themes clearly presented in the findings?	Results section (3.2) and Table 3
Clarity of minor themes	32	Is there a description of diverse cases or discussion of minor themes?	Results section (3.2) and Table 3

Developed from: Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. International Journal for Quality in Health Care. 2007. Volume 19, Number 6: 349–357.

Appendix B. Interview guide

General

• How did you experience participation in the study?

• What was the content of your module, and how did this affect you?

Before treatment

• What made you agree to participate?

• What were your expectations before the treatment?

• To what extent were your expectations met?

Impact during treatment

• Can you tell something about how the intervention affected your complaints?

Probes:

• -New insights
• -Changes
• -Day/week schedule
• -Awareness of thoughts/feelings/actions
• -Understanding thoughts/feelings/actions
• -Social interactions

Self-assessments

• How did you experience filling out the self-assessments in your daily life?

• How did the number/frequency of self-assessments influence you?

• What impact did filling in the self-assessments have on you?

Feedback reports

• What was it like to receive feedback on data that you collected yourself?

• What did you think of the feedback reports?

• And what did you do with them?

After treatment

• How did you get along afterwards?

• Probes: gains from the intervention.

• What would you have missed if you had not taken part?

Factors that help or hinder change

• To what degree did the method (self-assessments and personal feedback reports) suit you?

• Which elements have you missed?

• What were downsides of using the intervention?

• What do you think of the duration of the intervention?

• How did the intervention fit in with your regular treatment?

• What do you think, should this intervention be used for treatment of depression?

  • Which elements should be used, and which not? Why?

• To what degree did your module match to you being a doer or a thinker?

• How could the intervention have benefitted you more?