Patient and therapist experiences of using a smartphone application monitoring anxiety symptoms

ABSTRACT

Purpose

A smartphone application (app) from the company Monsenso was developed to monitor anxiety symptoms in the treatment of anxiety disorders as an alternative to paper registrations. The aim of the study was to explore patient and therapist experiences of using the app in conjunction with standard treatment for anxiety disorder in a developmental and implementation phase.

Method

The study design was qualitative. Semi-structured interviews were conducted with three therapists and seven patients from an outpatient clinic. The interviews were analysed using thematic analysis.

Findings

Three main themes emerged for both patients and therapists. The patient themes were usability (it was easier to use the app and remember daily mood registrations), insight in own disorder (awareness of symptom progress), and support to use the app (support from the therapist was wanted). The therapist themes were therapeutic quality (app registrations made it easier to prepare sessions), the role of the therapist (enthusiasm and technical assistance affected the patient), and implementation challenges (time allocation is important).

Conclusion

The anxiety monitoring app is recommended in standard treatment as an alternative to paper registrations. However, a successful development and implementation process include ready available technical support, time allocation, and therapist effort and enthusiasm.

Introduction

Anxiety disorders are characterized in the International Statistical Classification of Diseases and Related Health Problems 10th edition (ICD-10; World Health Organization [WHO], 1992) by physiological and psychological symptoms as a response towards perceived threatening stimuli. Anxiety disorders affect millions of people worldwide with an estimated prevalence of more than 274 million people (Vos et al., 2017). It poses great personal costs in terms of years lived with disability (Fox-Rushby & Hanson, 2001) and it represents a significant cost to society with regard to healthcare expenditures and loss of productivity (Konnopka & Konig, 2020). Substantial gains could therefore be obtained by improving the treatment for this patient group.

Digital technology is a way to offer accessible, flexible, and personalized care (Hollis et al., 2015) but needs to be proven effective both at the individual and societal level. Worldwide, there has been a considerable increase in the number of smartphone owners (https://www.statista.com/statistics/330695/number-of-smartphone-users-worldwide/), and two-thirds of adults would be willing to use their smartphone to improve their health and well-being (Makovsky). Thus, smartphone-delivered psychological interventions for anxiety disorders seem to be highly relevant to integrate into therapy in the mental health services.

However, there is a lack of research into the effectiveness of mental health smartphone applications (apps; Marshall et al., 2020) and solutions for Danish patients with anxiety are requested. Qualitative research is the first step to explore the development and usability in a mental health setting (Crosby & Bonnington, 2020). The majority of existing apps have been developed as stand-alone interventions as opposed to integrated tools in face-to-face therapy (Lan et al., 2018). Thus, we need digital health tools that can be part of a treatment programme, using the advantages of technology. This may support communication and increase the ability to offer patient-centered treatments tailored to individual needs, which again may increase treatment adherence (Mackie et al., 2017). Apps enable “Ecological Momentary Assessments” that involves repeated assessments in real-time of behaviours or emotions in a subject’s ecological environment (Shiffman et al., 2008), and this helps patients reduce anxiety symptoms (Loo Gee et al., 2016). These real-time assessments can be an effective tool to understand the symptoms during everyday life of patients with anxiety, while simultaneously enabling personalized treatment and prevention of dropout (Walz et al., 2014).

Cognitive Behavioural Therapy (CBT) has already been successfully translated into digital formats and has proven effective in the treatment of anxiety disorders (DiMauro et al., 2013; Otto et al., 2004). It addresses thoughts, emotions, bodily sensations, and behaviour, and includes exposure therapy (Barlow & Craske, 2006; Otte, 2011) where the patient is exposed to situations that trigger anxiety. In traditional CBT, the patient registers the exposure situations and the anxiety symptoms on a paper, but the paper is not always available in situations where anxiety occurs (e.g., in a bus or in a shopping centre), the patients may lose their papers, or simply forget registering. Thus, an app may be a useful alternative to paper and pen.

In interdisciplinary fields such as eHealth, the need for collaboration between end-users and developers is important (Clemensen et al., 2007; Pagliari, 2007). It is imperative to include therapists in the development and research of mental health apps in conjunction with face-to-face therapy to increase greater legitimacy of the apps, since the successful implementation of digital tools into a patient’s treatment seems to be dependent on therapist attitudes towards the tools and recommendations from other therapists (Marshall et al., 2020).

Aim of the study

The aim of this qualitative study was to explore therapist and patient experiences with an anxiety monitoring app used in conjunction with standard treatment for anxiety disorders.

Based on semi-structured interviews, the study investigated patient and therapist experiences with using the app as an alternative to paper and pencil registrations of emotions and behaviours in a developmental and implementation phase. This is important because we need evidence-based digital health tools to improve the mental well-being of anxiety sufferers.

Method

The smartphone application intervention

The app used in this study was developed by the Danish company Monsenso. The app was designed to collect self-rating data from patients as well as provide feedback via graphs that show patient progress. Originally, the Monsenso app was developed for patients with bipolar disorder (Frost, 2011). During the present study, the app was re-designed and developed as an anxiety monitoring app by Monsenso system developers and programmers in collaboration with a researcher and two therapists specialized in anxiety disorders from an outpatient clinic for affective disorders in The Mental Health Services in the Region of Southern Denmark.

The purpose of the app was to help patients register anxiety symptoms to increase awareness and insight in the patient’s symptom course. Parameters that could be monitored included anxiety intensity, exposure tasks (discomfort before, under, and after exposure), alternative thoughts and cognitive restructuring, as well as other parameters relevant for the patient group (see, Figure 1). The patient received a daily pop-up notification scheduled at an optional time a day reminding them to register on the parameters they had activated. The registrations automatically generated a graph, where it was possible to get an overview of symptom progress (see, Figure 2). It was also possible for the patients to take notes of things that had happened during the day (see, Figure 3). Combining the overview of symptom-changes with notes could help determine potential anxiety triggers.

Figure 1.

Parameters that can be activated in the Monsenso app.

Figure 2.

A screenshot from the app displaying a graph of symptom progress.

Figure 3.

A screenshot from the app displaying the function of taking notes.

Registered data could be viewed in the app on the patients’ smartphone and through a web portal where the therapist could gain insight in the patient’s registrations.

After six months development, the app was introduced to patients in the clinic in 2017. The study period lasted for one year additional. During this period, the app was further developed several times. The patients used the app to supplement their CBT-treatment that were carried out by mental health therapists according to national clinical guidelines. Patients were encouraged to use the app daily during their treatment programme.

Study design

A qualitative design based on semi-structured interviews was used to investigate therapist and patient experiences with development and implementation of the Monsenso app. The qualitative approach was chosen to explore personal experiences from the main users before implementing in large scale. Collaborating with people with “lived experiences” at various intervention development stages can offer interventions that are more personally relevant and engaging (Cus et al., 2021).

Recruitment

Since therapists working in the developmental phase could have different experiences with the app compared to therapists working only in the operational phase (Helweg-Joergensen et al., 2019), therapists from both phases were invited. The two therapists who participated in the developmental phase were invited three months after the introduction of the app into clinical practice. These therapists were asked to recommend other therapists to participate which led to a third therapist agreeing to be interviewed five months after the introduction.

The patients were introduced to the app by researchers or by their therapist in group or individual sessions. Patient inclusion criteria were having any anxiety disorder diagnosis according to the ICD-10 (WHO, 1992). Exclusion criteria were having a diagnosis of psychosis, schizophrenia, and mental retardation. In addition, patients were excluded if they did not have a smartphone. All patients who were introduced to the app were also invited to participate in the study. At the end of their planned treatment course, patients were e-mailed an invitation to participate in an interview, and if they responded, they were phoned to schedule a time.

Participants

In total, 49 patients were invited to participate in the study. Initially, 24 patients signed a consent form. However, 17 dropped out due to various reasons: non-responsive (n = 10), written comment (n = 1), refused participation (n = 1), and ceased contact after initially agreeing (n = 5). The patient inclusion process can be seen in Figure 4.

Figure 4.

Patient inclusion process.

Ultimately, the sample of the present study included 7 patients (PT) aged between 23 and 45 years of whom 4 were women and 3 were men, and 3 therapists (TP) of whom all were women aged between 37 and 52 years. See Table I.

Table I.

Interview participants

Interview participants	n	Mean (range)
Patients (n = 7) Sex, female Age Diagnosis: Anxiety disorder not otherwise specified Generalized anxiety disorder Panic disorder Panic disorder and generalized anxiety disorder Social phobia Social phobia and agoraphobia	4 1 1 1 1 2 1	32 (23–45)
Therapists (n = 3) Sex, female Age Training: Psychiatric nurse Cognitive psychotherapist and social worker Psychologist specialized in psychotherapy Experience, years	3 1 1 1	45 (37–52) 11 (1–22)

TP 1 and TP 2 took part in the development and adaptation phase. They treated patients with different anxiety disorders in group settings and individually. TP 3 took part in the adaptation phase and treated patients with obsessive compulsive disorders in individual treatment courses.

Data collection

Data was collected by the means of individual semi-structured interviews using interview guides. For the patients, an interview-guide was developed based on three themes: 1) implementation process, 2) usefulness and acceptability, and 3) improvement potential (see Appendix 1). For the therapists, an interview-guide was developed based on four themes: 1) implementation process, 2) acceptability and working procedures, 3) usefulness and quality of therapy, and 4) future perspectives (see Appendix 2). All interviews were conducted by the second author who is a psychologist and a research assistant. None of the authors knew any of the participants before enrolment, and none were involved in the participants’ treatment courses. All participants were interviewed individually face to face in a private room at the outpatient clinic, sequential to other appointments. All interviews lasted between 20 and 45 minutes, were audio-recorded, and transcribed. The transcribed therapist interviews consisted of 42 pages in total, with an average of 14, and a range between 9 and 19 pages per interview. The transcribed patient interviews consisted of 85 pages in total, with an average of 12, and a range between 8 and 15 pages per interview.

Data analysis

Qualitative analyses of interviews with therapists and patients were conducted by the means of a thematic analysis approach (Braun & Clarke, 2006). Thematic analysis is a widely used method for identifying and describing patterns or themes within data, without being bound to any pre-existing theoretical framework. In the present study, thematic analysis was used as an essentialist or realist method, reporting participant experiences. Themes or patterns within data were identified inductively, meaning that the identified themes only bear little relation to the specific questions that were asked. Therapist and patient experiences were analysed independently. The first three authors independently read the transcripts multiple times, coded interesting features into categories, and collated categories into potential themes. Identified categories were based on representation or relevance to the aim of the present study, and themes identified with a semantic approach. Themes were then discussed, and an agreement was reached. Themes were then refined, and the final analysis written with relevant quotations added. The last author was a clinical expert peer reviewer, validating the relevance of the findings from a clinical perspective.

Findings

Patient experiences

From the analysis of patient interviews, seven categories across patients emerged, which were assembled into three main themes. See Table II. The main findings were that the app was easy to use, graphical illustrations were beneficial, but therapist support was needed.

Table II.

Themes and categories from the patient interviews

Main themes	Categories
Usability	It is easier to use than a paper version It requires a mental surplus to reflect over your day It is easier to remember to register with a reminder
Insight in own disorder	The graphs increased awareness of symptom progress It is useful to combine “notes” and “self-evaluation”
Therapist support	Various introductions to the app The therapist needs to support the app use

Usability

During the trial, the app was under development and, therefore, all of the patients mentioned different improvements for the layout. Yet, they all saw the potential of using the app to support their in-person treatment. During the patients’ treatment course, they became aware of the therapeutic benefits registering daily measures of their anxiety. Even though registration on paper was similar, six out of seven patients found it easier to use the app because they had to register in a readily available place and because they received a daily reminder helping them to comply. One participant explained:

PT 3: Eh … since it (the app) forces you to write down things and … and … and then it is a little easier when it is on the smartphone, so you don’t run around with a lot of papers, right? That is, all around where … eh … where they then get lost and all that.

Most of the patients had their smartphone available anytime, except one of them, who, therefore, did not find it easier to remember to register on the app than on paper.

The smartphone enabled real time assessment of the anxiety, however, all of the patients only registered at a convenient time once a day or more infrequently. In general, they thought it was hard to reflect on how their day was, especially in hard times, and one participant explained that he would probably not have the mental surplus to register right after a situation with anxiety:

Interviewer: Can you describe a typical situation where you used the app, if you really try to imagine yourself right there in the moment when you use it?

PT 4: Well, then it’s been when I remembered that I have it, and then I would sit down and then I often kind of typed in for two days in a row, because I don’t always just remember it, eh, but it has been when I kind of have a mental surplus to do it anyway, where there sort of haven’t been anything. I have been, in that way, very relaxed in the situation. I haven’t been … and that … I don’t know if I would find it (symptom registering) difficult to do right after a situation with anxiety, eh, but it has always been where I have been, well, relaxed and comfortable and wanting to use it (the app), I think.

Patients need a mental surplus to evaluate their anxiety and they, therefore, typically register when they are at a quiet place where it is possible to use paper as well. However, the app helped patients to register their symptoms typically once a day due to the reminder and because the phone still was easier to keep track of than papers.

Insight in own disorder

Six of the seven patients highlighted the ability to create a visual graph that gave a better overview of their symptom course than the registration in the paper version could. This was highly rated by the patients since they thought it increased their insight into their therapeutic progress. A participant explained:

PT 6: (…) but overall, I find that it was a really fine app. Eh. It has been nice to be able to be attentive to how the days actually were, when one may find that it has been a tough period in regard to … eh … well, for instance, in comparison to where one started, and then find that it is a tough period, then you can go back and see that maybe it wasn’t quite the same anyway, and one has actually moved. Eh, then I also find there has been this, eh, what’s it called where you write, such … how the day went with notes, but where you also had some things you could tick off on a scale from one to five, for example. Eh, that I thought worked really well. Eh, also because you gained an insight into how big variations there were. Eh, and where it used to be, also.

In combination with the registrations, all of the patients to some degree used the function “notes” where they would put some words on how they felt and what had happened during the day. Over time, they could go back and see what had happened during the days where they felt low, and this gave them a better understanding of their course of illness. A participant explained:

PT 1: And then, it has helped me a lot with going back to see how my days have been. And I used notes, eh, in there. To write, enlighten myself a little about … whether there was a reason why I had had a stupid day.

All of the patients held positive views about the visualization option and saw a great potential in combining it with the notes. However, they all reported that the “notes” function still needed to be adjusted, and, therefore, it varied how much the “notes” actually were used.

Therapist support

Five out of the seven patients were having their therapy in group sessions, and they all explained that they did not go through their app registrations with their therapist. The two other patients did go through some elements with their therapist in individual sessions but only scarcely. The patients were, therefore, mainly using the app as an individual supplement to their regular therapy. However, two of the patients in group therapy specifically mentioned that it would make better sense if they had used the app actively with their therapist. One believed that it would give the therapist a better understanding of the patient’s symptoms, and in that way, qualify the therapy:

PT 2: Sometimes in retrospective, then it’s a, a little … well, it might feel very intense right there (when the anxiety happens), and then when one talks about it (later in therapy), then it might not be that bad, or the other way around, or something.

Interviewer: Yes

PT 2: So, it’s also like that, a little snapshot (of the symptoms) they (the therapists) can get a better insight into.

The other patient needed the therapist’s guidance in how to choose parameters and what to focus on:

Interviewer: If you were to use the app with your therapist or together with your therapy group, what would you … could it be something with comparing, or?

PT 5: Yes, it … it … could have been, maybe, to … to also find out, well, where should my focus be in relation to what … what anxiety problems one has, because, during the (psycho)education and in the course here with … with the group, then you get more knowledge about what is at stake in one’s anxiety. Eh … I could have used a single session, where we would look some things through, and where … where someone maybe guided me and said: “you know what? I actually believe that it is this you need to focus on” or “we need a goal about doing this instead” or “if you find that this focus on sleep has a too strong presence, then we might not need you to register this” or … so you had like … you kind of had to find your own form, your own answer book.

This patient had chosen parameters that he initially thought was important, but over time had come to worry if the daily registrations he did were sustaining his anxiety. He would have liked to elaborate on this with a therapist.

Involving the therapist in the evaluation of the patients’ app registrations would, therefore, improve communication between the patient and therapist since the ecological momentary assessment enabled the therapist to get a picture of the patients’ experience of anxiety when it happens. However, at the same time, it is important that the therapist assists and guide the patient to avoid that the registrations hamper the treatment course or potentially worsens it.

Therapist experiences

From the analysis of therapist interviews, seven categories emerged that were assembled into three main themes. See Table III. Main findings show that when the patients monitored their anxiety in the app, it could improve the therapeutic process. However, the therapists have an active role in engaging patients in its use, and smooth implementation depends on how ready the app is.

Table III.

Themes and categories from the therapist interviews

Main themes	Categories
Therapeutic quality	Access to patients’ symptom registering made it easier to prepare therapy sessions App registrations could be used to enhance therapeutic conversations
The therapists’ role	The therapists’ enthusiasm affected the patients’ interest in the app The therapists’ needed available technical assistance to support patients
Implementation challenges	Involvement of end-users in the developmental phase is preferable Reliable technology at the point of implementation is important Implementation takes time

Therapeutic quality

All the therapists highlighted therapeutically benefits when the app was used with the patients. First of all, they were able to access patients’ symptom registrations before a session which could help them prepare what was going to be the focus of the forthcoming session. Further, they did not rely on whether the patient had remembered their paper registrations and, therefore, it was possible to get a more reliable picture of the patients’ therapeutic progress. One therapist explained:

Interviewer: How did it work in that situation (when the app was used)?

TP 3: Well, I think it works well because we nearly always use the beginning of a conversation (with patients) discussing: “how have you been since last time?” That is difficult to remember. Then you have to use paper. Paper, you forget that, eh, and in that situation it’s been really fine, and, eh, having it on the screen. And personally, or as a professional, I actually think it is a little fun. I like to enter (the monitoring site) and, and just kind of see, ah, that’s how the stress level, the mood, and sleep lay. And how did they assess it themselves? And, eh, and you easily get that overview, I find. And that is, even before they come here, you have an idea of, eh, is the person coming here and have felt really, really bad since last time, or does it look reasonable?

Secondly, when the patients added notes to their registrations, it was possible to investigate the causal relations between symptoms and what might have triggered them together with the patients. The therapists believed this could improve the quality of the treatment by helping patients to understand their course of illness and, thus, support psychoeducational aspects of the therapy. A therapist elaborated:

Interviewer: Eh, did you notice changes in the quality or frequency of your consultations, eh, since you started to use the system?

TP 1: No, that is to early … it’s too early to say yet, eh … the quality, eh … yes the quality in that sense, maybe, that you in a more focused way can say, eh, if for instance, it is bad sleep quality, then you can focus your attention on: “well, I can see there November 29 that you slept bad,” then often, many of those who use it in a way where you get pleased, right, they wrote notes: “well, I actually also was a little worried about some kind of … eh, event I was going to the next day”. Then you can use it in the conversation psycho educative, right? “How can that be interconnected? What do you think about that?” Okay, and then you can try to look at it together with the patient and say: “ah, what do you think? Is there a pattern in this?” And so forth. Then we get a dialogue about it, and that was not possible, not always possible (without the app) because … my experience is that, when you ask: “how did you do during last week?” then you get an answer of how they are doing right now.

The therapists believed that when the app would be fully developed and used by patients, it would increase the number of registrations patients would do since it was easier than registering on paper. Their experience was that the patients who worked therapeutically in between sessions, something that the number of registrations represents, would have better clinical outcomes, and, therefore, they believed that introducing the app would increase the quality of the treatment.

The therapists’ role

The therapists experienced that the success of implementing the app depended on their effort and commitment in helping and encouraging the patients to use the app. The therapists’ learned that their genuine interest and enthusiasm in the app affected the patients’ interest in the app. A therapist explained:

TP 1: If a patient had failed to use the app then we talked about what the reason was and tried to solve it together; thus, I realized that my enthusiasm might influence the patients without manipulation or me trying to convince or persuade them because they are the ones who shall have the motivation for using the app.

The therapists got help with the introduction of the app to their patients from an appointed therapist at the clinic. Over time, they needed less help but kept emphasizing that it was important to continue having technical support to maintain their skills in supporting the patients with their app use since the app continued developing during the study period. The therapists’ ability to support the patients were directly related to the patients continued use:

TP 1: In the beginning, the patients stopped using the app if there were too many difficulties, but as I gradually learned more, I could be more persistent towards the patients’ experiences with the technology, explaining that it is a process with improvement possibilities where the patient has a say in it about what works and what doesn’t.

This effort was an additional task that would take time from regular sessions. A therapist explained:

TP 1: Even though the patients have their smartphones, it requires that they are able to navigate in the app which is our job to show them how it works — which again requires time — but it depends on how scared they are of technology, because if they are, it may be a hurdle from their perspective. There are many psychosocial challenges in their lives, and if there are too many other things and it is too confusing and compressed or boring, they are more likely to stop using the app.

The therapists highlighted that they needed to support the patients with regard to making the app meaningful for individual patients and that they themselves needed technical support as long as the app continued to be adapted.

Implementation challenges

Besides therapist support, it was also important implementing a useable app, that is, one that is easy to use, reliable and makes sense. The two therapists who participated in the development of the app believed that a close cooperation between the clinic and the system developers was necessary. They emphasized the importance of testing the app in practice before adaptation, as testing would reveal more nuances of what the registering needs in a naturalistic setting.

All the therapists held positive attitudes towards using technology in their workday and they explained that they prioritized using the app. However, in the beginning of the launch into clinical practice, they encountered some technical errors in the app. They emphasized the importance of reliable and stable technology at the point of implementation since it was difficult to maintain patience for problematic technology in a pressured workday. They also worried that technical challenges would prevent patients from using the app:

TP 2: Our patients are a vulnerable group, if they are new and anxious to be with others in a group then you are extra vulnerable when something is not working, and then it quickly comes down to that it does not work and then they use the data papers instead because there is not much patience in the group.

Finally, they emphasized that they needed allocated time at the beginning of the implementation process to 1) gain knowledge about and experience with how the app works, 2) handling technological challenges, 3) introducing and setting up the app with patients, and 4) facilitating patients who use the app. It was important to have enough time in the beginning so that they had the opportunity to build routines and gain ownership over the new tasks. If this was not achieved, it was easy for the therapists to fall back to their previous routines, and thereby diminish the chance of a successful implementation. However, they stressed that time spent in the beginning could be time won at the end because the app could contribute to making the treatment more efficient.

Discussion

The aim of the present study was to explore patient and therapist experiences with the use of an app registering anxiety symptoms in conjunction with face-to-face therapy in an adaptation phase. Findings reveal that patients found it easier to register on an app compared to paper when the smartphone was readily available, and it was easier to remember registering due to a reminder. The patients preferred that the registering from the app was evaluated together with a therapist. Therapists similarly found that it could improve the therapeutic process if the patients’ registering was used actively in therapy sessions. Since the app was introduced to patients in an adaptation phase, they highlighted that sufficient time and resources should be allocated to this phase if the app should be successfully implemented.

Crosby and Bonnington (Crosby & Bonnington, 2020) found benefits of using apps for anxiety in terms of flexibility, empowering, and coping, but also pointed at the risk of biological reductionism and isolation from interpersonal support. In our study, the app did not replace human interaction. It was only used as an assessment tool alternative to paper and pen registration. However, both patients and therapists underlined the importance of therapist support to make the registrations meaningful for the individual and to avoid that the app would increase users’ stress level.

Our study showed that the app could support treatment due to a combination of factors. Compared to paper registrations, the app presented a symptom overview that served as a visual aid, it allowed nuances to be explored that increased understanding of the anxiety, and allowed instant monitoring concerning the chosen parameters. Patients used this as a motivational reinforcement that they could see their progress, and therapists used it to better remember the specific treatment courses, improve preparation for the sessions, and optimize their dialogues with patients. These findings are corroborated by findings from similar studies about how utilizing real-time assessments and providing visual feedback (Garrido et al., 2019; Myin-Germeys et al., 2016) as well as monitoring participants’ use of the application, sending reminders, and using an app as a communication and connection tool (Mackie et al., 2017) can be important remedies in the treatment of mental disorders.

However, we also discovered that several of the patients did not use the app together with their therapist. Getting feedback on data entries can be especially important because validation is an important aspect of the therapeutic alliance, and not acknowledging the patient’s hard work can be detrimental for compliance. Cus et al. (2021) evaluated the use of a smartphone intervention to manage non-suicidal self-injury and found that users experiencing improvement in their symptoms become more engaged in the app, while users with psychological distress not supported by the app may disengage.

Moreover, we found that when the app was used actively as an integrated part of the treatment course, it seemed that it contributed to more advanced reflections on the anxiety and decreased distorted thinking, which is part of the CBT treatment regimen. The graphs that were generated in the app gave patients a more realistic evaluation of number of “bad days”. This finding supports previous research focusing on self-monitoring, suggesting that simple self-monitoring techniques may effectively increase self-awareness of one’s own emotions and, thus, lead to behavioural change (Kauer et al., 2012). Another possibility was to explore causal relations between symptoms and events. Furthermore, patients thought it was easy to register on the app, potentially leading to more solved assignments between sessions. Thus, monitoring anxiety symptoms on an app appeared to be useful for both patients and therapists.

The analysis of the interviews with the therapists revealed that involving various users in the design phase would improve the creation of a useful app. They expressed that in the implementation process it was important to have sufficient amount of time to acquaint themselves with the app if they were to use it together with patients, as they felt inexperienced in introducing the app to the patients, handling technical issues, and taking care of unexpected events. They further experienced that their enthusiasm functioned as support for the patients and therefore affected whether the patients would use it.

The need for collaboration between end-users and developers is important in interdisciplinary fields such as eHealth (Clemensen et al., 2007; Pagliari, 2007), which also appeared in our study. The therapists were able to use their insight knowledge of anxiety treatments to readjust the app for a different patient group. Garrido et al. 2019 also found that mental health apps need to be developed by including end-users in early phases, especially for young people that seek personalized solutions (Garrido et al., 2019). In the present study, including patients in the design phase could have resulted in the presentation of features in the app being more straightforward, and easing the patients’ concerns about choosing the right features.

When the app is ready for use, the foundation of use will depend on implementation success. We found several crucial aspects that influenced the implementation process. First, we found that a stable technical performance was a prerequisite for the utilization of the app; thus, implementation of the app should first happen when the technology is reliable and stable. Second, we found time to be an imperative factor in the matter of implementing new technology successfully. Department managers should allocate dedicated therapist time in the beginning of an implementation phase as it gets easier with time and experience, making utilization of the app a normalization process. In accordance with the theory on normalization processes, investment of time is needed for a process to become normalized (Bracher & May, 2019). Third, the app may not be intuitively easy to use, so therapists and patients need a proper introduction to the app. A good introduction highlights both why and how to use it. Fourth, we found that a combination of factors, such as therapist effort, commitment, enthusiasm, and increased knowledge about the app influenced the therapists’ ability to motivate the patients. Patients wanted feedback on their entries from therapists to increase their motivation to register (other people also care about your data) and to increase understanding of how monitoring could help the patient. To make the anxiety monitoring app useful for patients, it has to be used with dedicated therapists.

Strengths and limitations

It is a limitation that the patient sample in the present study solely consists of people who would like to use an app in treatment and, therefore, may have favoured the utilization of digital tools during their treatment course. Thus, the findings are not directly transferable to other settings, where the patient group has diverging preferences for app use.

A high number of non-responders and patients who declined as well as limited time from therapists made it difficult to set up interviews. Therefore, the findings may be biased as the sample consists of patients with a mental surplus and a possible interest in helping others.

It is a strength that all interviews were conducted by the same researcher, the second author of this article. Data were coded by three independent raters, which enhances the credibility of the analysis (O’Brien et al., 2014; Thomas, 2006) and increases reliability and internal validity (Malterud, 2001).

Conclusion

The present study found that monitoring anxiety symptoms on an app was useful for patients and therapists in the treatment of anxiety disorders in an outpatient mental health service. The app made it easy for patients, in general, to register their symptoms, potentially leading to more registrations, which is an important part of CBT for anxiety disorders. Moreover, it facilitated easy access to registrations in therapy sessions since they were available via the therapist’s computer. Therapists could inform themselves of symptom progress even before meeting the patient, and they could engage in a therapeutic dialogue about potential causal relations based on the app registrations. Therefore, the app is recommended in standard treatment as an alternative to paper registrations.

The present study also found that the development and implementation phases possess several challenges. An important factor is that in an adaptation phase, extensive time allocation and technical support must be available to ensure that therapists are capable of facilitating the app and keeping them engaged in the process. Therapists’ engagement and enthusiasm appeared to directly affect whether the app was used among patients.

Future research should focus on exploring what features an anxiety monitoring app should contain, besides daily evaluations of mood symptoms, to make it most useful as a digital tool to support standard therapy. Further, research should also measure the effectiveness of using an app for registering anxiety symptoms, which includes whether the app increases the number of registrations and with that symptom progress.

Biographies

Kristine Tarp is originally trained as an anthropologist, holds a PhD in Health Sciences, and works as a Postdoc. Her research concentrates primarily on implementation of digital e-mental health solutions and qualitative methods.

Trine Theresa Holmberg is a clinical psychologist and works as a Research Assistant. Her work concentrates on digital psychological treatment offers for patients suffering from, e.g., anxiety, binge-eating disorder, and sleep disturbances.

Anne Marie Moeller holds a Master in public health science and is a PhD Student in Health Sciences. Her research interests evolves around human well-being and qualitative methods, e.g., Content Analysis and Grounded Theory.

Mia Lichtenstein is a clinical psychologist, holds a PhD in Health Sciences, and works as an Associate Professor and Head of Research. Her research focuses on improving people's health behaviors, especially within the areas of disturbed eating, exercise addiction and e-mental health.

Appendices.

Appendix 1

Interview guide for patient interviews

Consent form: Please read and sign the consent form. Ask participant if they have any questions, make sure they are informed and willing to participate before you continue.

Ask if they filled out the survey or give them a printed version after the interview.

Presentation of interviewer and framing the interview

Hello, my name is XYZ. My job is to gather interview data for the project to better understand if the anxiety-monitoring tool is useful as a tool to assist patients and therapists in treating anxiety in people who experience mild to moderate anxiety symptoms.

The interview will be recorded and transcribed. The information will be analysed in combination with other interviews, which are part of this research project.

Your answers will be anonymized, and you will not be recognized in the final presentation.

I have prepared some questions and your answers will help us better understand the subject we are investigating. You are welcome to comment and elaborate on everything in the process.

You can withdraw consent at any point in this process including after the interview.

Are you ready to continue the interview?—If yes, I will start the recording now.

Introduction

Will you present yourself?

Age

Work

Disorder

About technology in general

Do you like technology?

Do you have many apps? Which are your favourites?

1. General impression

Can you describe your experience of using the app?

• Which parts did you like best?

• Which parts did you not like as much?

How did you hear about this app?

How were you introduced to the app?

Which functions did you like, which didn’t you like as much?

Have you received therapy without an app?

- If yes, can you compare the two?

• (2) Usage

Can you describe a typical situation where you used the app?

How did you use the app together with your therapist?

Did you like using the app?

• (3) Future use

Would you use technological aid in the future if it was an option?

Do you have any suggestions to improve the app or is there anything you think could have been done differently?

• (4) Compliance

How often did you complete the self-monitoring?

- What affected when you did not?

How did you feel about monitoring yourself each day?

Finally

Is there anything else you would like to mention or clarify, that we have not talked about?

Finish

This concludes the interview; I will turn of the microphone.

Thank you so much for your participation.

Appendix 2.

Interview guide for therapist interviews

Consent form: Please read and sign the consent form. Ask participants if they have any questions, make sure they are informed and willing to participate before continuation.

Hello, my name is XYZ. My job is to gather interview data for the project to better understand if the anxiety-monitoring tool is useful as a tool to assist patients and therapists in treating anxiety in people who experience mild to moderate anxiety symptoms.

The interview will be recorded and transcribed. The information will be analysed in combination with other interviews, which are part of this research project. Your answers will be anonymized.

I have prepared some questions. Your answers will help us better understand the subject we are investigating. You are welcome to comment and elaborate on everything you want. You can withdraw your consent at any time including after the interview.

Are you ready to continue the interview?

- If yes—I will start the recording now.

I have four overall questions for you. The two first are regarding your experience with using the app, and the final two questions are about your impression of the usability of technology in treatment of anxiety disorders. Lastly, I will ask you to fill in a short survey.

Presentation of therapist

Will you introduce yourself? Name, function and educational background as well as your general knowledge about technology—which technology do you have yourself?

1. General impression

Can you describe your experience of using the monitoring app in the clinic?

(For example, how did you become aware of the app, how did you find working with the app with your patients?

• (2) Workflow

Did you notice a change in workflow since you started using the app?

If yes, can you comment on which type of change you noticed and how they have influenced your work life? I am interested in both negative and positive experiences.

• (3) Quality of therapy

Did you notice any changes in quality or frequency since you started using the system with your patients?

How is this in comparison with your usual interaction with your patients without the app?

• (4) Future use

What is your professional opinion regarding incorporating technology in a psychiatric setting?

Do you think technology can assist you in the future?

If yes, how?

What do you think would be useful for developers to know to develop technology that is meaningful for both therapist and patient?

Lastly—do you have anything else you would like to share with me regarding your experience of using this app with your patients?

I will turn of the microphone now. Thank you for your participation.

Funding Statement

This work was supported by Innovation Fund Denmark, grant number 5159-00002B.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Ethical statement

The study was approved by the Regional Committees on Health Research Ethics for Southern Denmark and the Danish Data Protection Agency. As per the declaration of Helsinki, participants were informed about the study and had to sign an informed consent if they agreed to participate. They had the opportunity to withdraw at any time without consequences for their further treatment.