A qualitative study of a blended therapy using problem solving therapy with a customised smartphone app in men who present to hospital with intentional self-harm

Abstract

Background

Blended therapy describes the use of computerised therapy combined with face-to-face therapy to extend the depth, range and nature of the face-to-face therapy. We wanted to develop a treatment manual for a randomised trial of blended therapy combining face-to-face problem solving and a smartphone app in men who present to hospital with self-harm.

Objective

To develop a treatment manual and to describe the experience of receiving and delivering a blended therapy.

Methods

After completion of the blended therapy, semistructured qualitative interviews were conducted with participants to describe their experience of the treatment. Two independent coders analysed the material using a thematic, grounded theory approach.

Findings

Seven men were enrolled in the study, and six completed the qualitative interviews. The two main themes identified were of trust and connection. Participants attended 85% of their appointments.

Conclusions

In the treatment manual, we emphasised the themes of trust and connection by allowing time to discuss the app in the face-to-face to sessions, ensuring that therapists are familiar with the app and know how to respond to technical queries. Identification of trust and connection generates novel questions about the importance of the therapeutic alliance with technology rather than with people.

Clinical implications

Clinicians and app developers need to pay attention to the therapeutic relationship with technology as trust and good communication can be easily damaged, resulting in disengagement with the app. Blended therapy may result in increased adherence to face-to-face sessions.

Trial registration number

NCT02718248

Background

Blended therapy is the integration of internet-based treatment and psychotherapy in the treatment of psychiatric disorders. It has been described as ‘any possible combination of regular face-to-face treatments and web-based interventions’.¹ Several systematic reviews have examined the partial replacement of face-to-face therapy with web-based technologies.^2–4 Yet there have been few descriptions of instances where internet-based therapies supplement or augment face-to-face therapy to create a new treatment rather than simply replacing face-to-face treatment with variable degrees of inperson support. This approach to blended therapy attempts to increase the depth, reach and nature of traditional face-to-face treatment.

A 2016 review of mental health smartphone applications recommended the integration of cognitive behavioural therapy (CBT)-based features when developing applications.⁵ The smartphone’s facility for establishing, rewarding and reinforcing behavioural change is also cited as a significant advantage of this technology in the context of mental health.⁶ Tang and Kreindler suggest that mobile applications may support homework compliance for CBT treatment when the application is congruent with therapy, fosters learning, incorporates guided therapy, inspires connection building, emphasises completion and is population specific.⁷ This is especially important in the care of men with mental illness, with evidence suggesting that males may be particularly responsive to therapies that reward planned behaviours.⁸

The current study is part of a larger trial, the BEACON study, which is a cluster randomised trial of a smartphone-assisted therapy to improve outcomes in men who present to emergency departments with intentional self-harm in Ontario. As part of the preparation for the larger trial, we conducted this qualitative study to develop a treatment manual for the blended therapy and to get users’ feedback on the blended therapy. Since the objective of our research is the prevention of suicide, we decided to focus on men, as more than 75% of individuals who die by suicide are male. Self-harm has a strong association with suicide as about 1%–2% of people who present to emergency departments with self-harm will kill themselves within 1 year with the rate being almost double in males compared with females (2.7% vs 1.2%).⁹ This is a suicide rate that is 100–200 times the general population rate. We also chose to focus on men, as previous treatments that have delivered the same treatment after self-harm to everyone, regardless of gender, have not been particularly effective.¹⁰

Objective

To inform the production of a treatment manual for a larger cluster randomised trial of a smartphone-assisted therapy for men who present to hospital after intentional self-harm. Second, we wanted to describe the experience of receiving and delivering a novel blended therapy combining a customised smartphone application with problem solving therapy (PST) for this population.

Methods

Study design

The study was a qualitative study of blended therapy with men who presented to the emergency department with intentional self-harm in a major Canadian urban centre.

Study participants

Emergency department clinical staff at The Ottawa Hospital (TOH) were asked to refer adult men to the research team who met the following eligibility criteria: (1) patient presented to the emergency department with intentional self-harm and (2) had access to a smartphone with a data plan. Patients were then contacted over telephone by the research coordinator with more information about the study. Interested patients were then invited to TOH General Campus to complete a baseline intake appointment, which included the Informed Consent discussion and participant introduction to the app.

We defined self-harm according to the National Institute for Health and Care Excellence guidelines, referring to any act of self-poisoning or self-injury carried out by a person, irrespective of their motivation.¹¹

Study intervention

Patients who consented to take part in this study received six sessions of individual face-to-face PST over 6–10 weeks provided by a psychiatrist combined with a smartphone application specifically adapted for this population.

The PST had previously been used in patients who presented to hospital with self-harm in New Zealand (refer to the training site at www.problemsolvingtherapy.ac.nz). PST sessions took place every 1–2 weeks, with some variation in frequency based on patient need.

The smartphone application adapted for this study, the Addiction Comprehensive Health Enhancement Support System (ACHESS), was originally developed to treat harmful substance use among veterans in the USA. A randomised controlled trial (RCT) found that it increased the number of substance-free days compared with controls.¹² In adapting the application, features were added to make it more relevant to suicidal men receiving PST. Of particular note is the BEACON button, which was designed to remind users about their safety plan that was mapped during the first face-to-face session. The BEACON button accesses the safety plan and was populated by a preidentified list of emergency supports while also sending an email alert to the psychiatrist. It was also linked to a range of suicide and self-harm resources included as part of the ‘Discover’ section. The therapist also had access to a ‘Clinician Dashboard’, which allowed for interaction with participants between sessions, allowed the clinician to monitor progress and displayed automated alerts if participant survey responses suggested that they were experiencing higher levels of distress or suicidal ideas. The main interface (figure 1) allows for users to Gather, Connect, Discover and Plan, as described below.

Figure 1.

ACHESS smartphone application interface.

Gather

Gather provides users with a social network of supporters. (Supports do not need to have the ACHESS Mobile Health application.) From here, users can create a personalised profile as well as view My Support Team profiles.

Connect

Connect allows users to message individual members, which facilitates instant and time-delayed communication between the patient and individuals in their healthcare network.

Discover

Discover provides users with various patient education resources about self-harm and suicide prevention. This includes access to web, video and audio content. It also includes a section for users to upload their own motivations for recovery.

Plan

Plan is linked to the PST and focuses on the implementation of agreed goals. It also provides access to surveys, which can be personalised and appear on the clinician’s dashboard.

The user can also program high-risk addresses through the application’s Global Positioning System (GPS). If the participant got close to a high-risk address, the app would send a personalised warning to the user. The clinician had no access to information about the more general movements of participants and would only receive GPS-based alert on the clinician’s dashboard with the preinformed and explicit consent of users.

Data collection and analysis

Semistructured qualitative interviews were completed up to 6 months after the last face-to-face session. The interviews were completed individually, and we obtained consent to record the interviews from all participants. Following transcription, two independent coders analysed the material using a thematic, grounded theory approach. The coding took place in two stages, with coders meeting during the first stage to discuss emergent codes, reconcile definitions and compare coding rationales. The two coders had not been involved with delivering the treatment or any of the previous study contacts.

While not being primarily interested in effectiveness in this study, we also monitored changes in Patient Health Questionnaire (PHQ-9) scores¹³ and recorded any adverse events including further episodes of self-harm. The PHQ-9 is a nine-item questionnaire that assesses the severity of depression symptoms experienced within the previous 2 weeks. Scores of less than 10 indicate no or mild depression; 10–14 is moderate depression; 15 and above is moderate or severe depression. Participants were asked to complete this questionnaire, when attending their face-to-face PST sessions. While this questionnaire does not address self-harm specifically, the final question does make explicit reference to self-harm and suicide. Other considerations for using the PHQ-9 included its brevity, its widespread use and validation, and the frequency of depression as a comorbid symptom in individuals at risk of suicide.

A written Participant Exit Questionnaire was completed at the last face-to-face session, which collected data on user satisfaction with the smartphone-assisted PST intervention. This questionnaire solicited quantitative and qualitative data from participants on the usability and applicability of the study intervention using various Likert scales and open-ended questions.

Findings

A total of seven men were enrolled into the pilot study from October 2016 to February 2017, and six completed the qualitative interviews. Participants varied in age from 19 to 41, with a mean age of 27 (SD=8). They presented with various methods of self-harm, including cutting and overdose of medication. At the time of their index presentation, their assessing clinicians diagnosed participants with major depressive disorder, generalised anxiety disorder and substance abuse. Engagement with the PST was high, with 85.7% of participants completing all follow-up appointments and 100% of participants completing five out of six sessions.

Results of qualitative interviews

The coders identified two main themes of trust and connection.

Trust

Trust in the functioning and effectiveness of the application was associated with overall participant engagement. Less obvious is how sensitive this relationship becomes when the application in question is related to mental health. One participant described it in the following way:

Interviewer: “Was the therapy sufficient to meet your emotional needs?”

Participant: “It was. I did feel supported. It was sufficient.”

Interviewer: “During the times that you were away, not directly in the face-to-face therapy, there was the BEACON button, which… potentially would be sort of like an emergency contact. Did that give you any feeling of security? Or was it, because of the bugs, kind of a non-issue?”

Participant: “It was a non-issue. It didn’t give me any security because it didn’t work.” (P4)

Perspectives such as these seemed to form early and resulted in poor engagement with the relevant feature of the application. Of the four participants who reported low use of the application, one attributed this to a feeling that it was not appropriate for them and the remaining three to a belief that the application was not technically trustworthy. One participant who reported habitual use (three to four times per day on average) overlooked these problems, attributing malfunctions to the beta stage of the application’s development, and requested permission to keep the application on his phone after the study period had ended. As can be seen in the quotation above, the theme of trust also appeared frequently in discussions of the BEACON button, particularly with regard to its trustworthiness during moments of crisis.

Discussions of the BEACON button often led to conversations about the use of the phone application during times of crisis. Participants described how their ability to use the application, and thus their needs as users, would change with the transition from a state of everyday use to that of use during an emergency. As one participant described it:

Participant: “The BEACON function itself was enough to give me a nudge to get my safety plan going. Because my problem coming into this was that I pretty much lost situational awareness and won’t realize that I’m getting into such a mental state. But the BEACON button was able to overcome that.”

Interviewer: “Okay, so, what was your actual functional use of the BEACON button after hitting it?”

Participant: “Well it certainly was dual purpose, ‘A’ it reminded me that there was help out there, and ‘B’ It, for me personally, it directed me towards my safety plan.” (P2)

He later elaborated:

Participant: “Well, the BEACON function on that night did, uh, did actually prompt, uh, did actually push me to start my safety plan.”

Interviewer: “Okay, so that’s good. So the BEACON prompted you to activate your safety plan.”

Participant: “Um, well it’s more of an association thing, and pressing the BEACON, does remind me that help is there. And then, of course, it reminded me that Dr. Hatcher and I had worked out a safety plan.” (P2)

This sentiment was echoed by another participant, who also described the difficulties of decision-making and reaching out during a crisis:

Participant: “Because I’ve been in moments where, it’s like you don’t want, necessarily, [to] speak to someone, or explain yourself, or, you know, you’re sounding crazy or whatever, in your head, right? There are so many decisions and things spinning through your head in these moments of crisis, that, anything that can stop you from reaching out [is a problem], which, most of the time, like, 99% of the time reaching out is better than not… I think the rule of thumb there would be, every extra click, every extra decision, every moment you’re asking the user to make an action, makes it harder for them to get the help that they need, but you still don’t want them to be nervous of accidentally reaching out and do the ‘boy cried wolf’ thing, right?” (P3)

The BEACON button also elicited a certain amount of anxiety in some participants, who were concerned that they would accidentally trigger an emergency response:

“And even having the BEACON button right at the bottom of the screen, really massive and big, it’s good to have it there for emergencies, but I was kind of nervous that I would tap it (laughs).”(P3)

Another participant described feeling embarrassment after accidentally activating the BEACON button, which elicited a follow-up call from the primary investigator. Some suggestions were made to improve this function, including a slide-to-unlock feature, similar to those used to access the homepage on certain smartphones.

There was also some concern expressed that the supports underlying the BEACON button would be insufficiently resourced once it had been released for public use. As one participant put it:

“[My favourite thing about the app] was probably the button to contact [support during] an emergency. Because there are different circumstances where you [might reach out], like, it’s not a 9-1-1 situation, but it’s also not, like, a wait six weeks for an appointment situation. Then again, I don’t know if there are the resources for people to be able to access a psychiatrist, ah, just by pressing a button (laughing).” (P1)

Though the majority of the participants developed a safety plan, and three reported using it successfully, integration with the application was limited. For example, of the three men who reported using their safety plan effectually, two mentioned that the application did not facilitate the plan at all.

Despite the feature’s perceived limitations, there was significant interest in the BEACON button, with one participant describing it as his favourite thing about the study, even though he did not use the feature. Other participants expressed hope for greater customisation of the feature, integration with a crisis line or extra monitoring by the therapist.

Connection

Connection between the user, the application and therapist was another dominant theme identified during analysis and often overlapped with the theme of trust. Indeed, trust in the application’s effectiveness and function, and trust in the therapeutic alliance, appeared to influence participant feelings of connectedness. This desire for connection was clearly seen in participant’s comments about personalisation, preferences and recommendations.

Personalisation

Participants made numerous recommendations as to how the application could be improved and identified components of the application they found particularly useful. In particular, a desire for more personalised features was consistently expressed. As one participant described it:

“I don’t know how hard it would be, but if it was sort of like a branched wizard, like, branched questions so… Are you looking to solve a problem? Or are you looking for safety planning? And then it would ask you what sort of situation [you are in,] you know, to guide you to what you were looking for, because I’m guessing depending on who it is, you’re going to be looking for a different function.” (P1)

A similar comment was made by a different participant in the context of encouragement and motivational messages, he suggested:

Participant: “just like, interesting, encouraging things…”

Interviewer: “They had the Daily Thoughts, did you have that active?”

Participant: “Yeah, but even making it more personal… [drawing on] values that are currently in the app that the user is using, [so that] the app feels like it’s helping them personally. If it’s just those general thoughts or those general motivations, people can get to the point where they just ignore them… But if it’s something that’s interesting and specifically geared to them, so, I’m thinking, something random, like, you made it 35 days, and then, you know, maybe some General somewhere battled for 35 days, and you [have] an interesting fact, you did this for 35 days and the battle in China lasted for 35 days.” (P3)

Other participants expressed a desire for greater customisation of the ‘Discover’ section of the smartphone application, especially in relation to mental health reference documents, to better reflect the mental health challenges that they faced.

This theme, the uniqueness of an individual’s mental health needs versus the fit of a structured application or program, was also evident in discussions of mental health literacy, technical literacy and in discussions of disidentification, as described below. Though such observations are preliminary, and limited by the sample size of the study, it did seem that there was a relationship between a participant’s relative technical literacy and the participant’s expectations regarding the application, with higher levels of mental health literacy acting as deterrent to engagement. For example, one respondent described his use of the calendar function:

“Was more just to keep [the primary investigator] in the loop than useful for me… I have all these different apps that maybe [already] do one of the things. Like I have a calendar app, I have a note writing app… I like that you were trying to put them all in one place, and that the emergency stuff is there and what not, but any way to make it more smooth…” (P3)

Half of the participants also explicitly mentioned the relaxation audio files included as part of the ‘Discover’ function as being particularly useful.

Use, fit and engagement

Engagement with the application was variable, though the principal reason for lack of engagement with the application was a result of technical complaints. Especially common were descriptions of the interface as ‘buggy’ or ‘clunky’, with one participant expressing a belief that the application ‘didn’t really work’ in the sense of basic technical function and another who wondered if the application was deliberately designed to malfunction as part of a deception-based study.

Also frequent were reports of disidentification, a personal rejection of the application and/or expression of the belief that the ‘app was not for me.’ As one participant described on first being introduced to it, “Well, because it was clear, looking at it, I was like, this clearly is not for me, like, it was not made for me.” (P5) Despite efforts made to adapt the application for use by our participants, residual components of the application’s original design contributed to this difficulty. For example, the legacy of the ACHESS application as a tool for sobriety was also cited as a reason for disidentification. One participant found that the frequent references to a ‘higher power’ in the Daily Thoughts sequences—a supplement of Alcoholics Anonymous inspirational quotations—caused him concern and prompted unwelcome reminders. Another participant reported that his initial experience with the application was, “awkward because it was more geared toward sobriety,” (P2) and a third participant described that, for him, “the legacy of the alcoholism thing on it, kind of interfered with the usability… it kept congratulating me for another day [of sobriety] (laughs).” (P1)

Satisfaction with therapy and study procedures

Participants reported a high degree of satisfaction with the therapy provided and likewise expressed satisfaction with study procedures. Four out of the six respondents reported that the PST provided met or exceeded their expectations, and the majority of participants reported that they felt sufficiently emotionally supported during the course of the therapeutic relationship. PST itself was frequently described as ‘useful’, ‘valuable’ and/or ‘helpful’. All respondents replied ‘no’ when asked if they felt any inappropriate topics were broached during therapeutic sessions.

Changes in PHQ-9 scores and adverse events

Participants presented to their baseline intake appointment with a mean PHQ-9 score of 18.3 (SD=5.5), indicating moderately severe depression. At their final face-to-face PST session, participants reported a mean PHQ-9 score of 13.7 (SD=10.0). At the final study visit, three participants had improved by 50% or more, or scored less than 10 on the PHQ-9. There were no repeat presentations to hospital with self-harm during the study.

Participant exit questionnaire

All participants enrolled in the study completed the Participant Exit Questionnaire after their final PST session. Five out of seven participants reported that the ACHESS smartphone application was user friendly and easy to use (response of Strongly Agree or Agree on the Likert scale), but only two out of seven reported that they would use the smartphone application again in the future. Five out of seven participants reported that PST was a good fit for them.

Participants reported that the smartphone-assisted PST intervention would be more useful if it was personalised to their situation and that the bugs in the smartphone application should be resolved to make it easier to use. Participants also reported that the features of the smartphone application that worked well were the communication with their therapist through the messaging feature, the relaxation and breathing exercises in times of stress in the Discovery part of the app and the 24/7 crisis support provided through accessing the BEACON button feature.

Therapist feedback

The feedback from the therapist (SH) was that using the smartphone application was more time consuming as both the therapist and participant referred to the application and used it in the session—it was ‘something else to talk about’ and became like a ‘third person’ in the room. The therapist also used the Clinician Dashboard between sessions, to monitor the participant’s use of the application and as a communication tool. The therapist found the application useful to send reminders about agreed upon tasks and appointments. It also seemed that this increased the sense of connection between the patients and therapist, which resulted in a high rate of attendance at appointments. One of the therapist’s critiques of the application was that the beta version did not fully integrate with the face-to-face therapy, although the therapist was able to refer to links within the application to an online PST training module. An important advantage of the application was that adding extra resources relevant to discussion points raised during face-to-face treatment, such as information on moral injury (what happens when a person perpetrates something that transgresses their own ethical or moral values) or information for friends and family on how to manage someone who was cutting, was a straightforward process.

Discussion

Men who participated in this study spoke at length about trust and connectedness, but these themes were not primarily about confidence in immediate human relationships or even about relationships with people. Instead, the smartphone appears as the locus of these comments and, in acting as the facilitator of links between therapist, emergency services and service user, in many ways becomes the emblem of the therapeutic relationship and the focus of confidence in that relationship. What was noteworthy was the high attendance at follow-up, which suggests that using the app in therapy did have some influence on connection to the therapist.

Because the smartphone was the primary source of connection to available resources (including, for example, online supports, contact with the therapist via text, 9-1-1 and videos), its effectiveness in prompting, displaying and accessing these resources correspondingly became a significant factor in the participant’s experience of security. If a given feature was ‘buggy’ or clumsy, a pronounced, and speedy, loss of participant confidence was often the result, often leading to non-engagement with that feature of the application for the remainder of the intervention. This suggests that a particularly rigorous debugging process is required in the development of any computer-assisted therapeutic programme, to avoid losing user engagement.

For the development of the training manual for the multicentre RCT, this study underlined the importance of active check-ins by clinicians for quality control, the need for timeliness in responding to problems and the need for operational fluency when providing explanations. Accordingly, we have added sections to our manual on the specific process for addressing application technical maintenance, emphasising speed or response time, and an introductory training phase emphasising comprehensive and repeated personal use. In the ‘Getting Started’ section of our manual, we recommend that clinicians undertake an extended period of ‘playing’ with the application (15–30 min per day for 2 weeks), to ensure familiarity with the program’s various components, and have added a test of operational fluency at the end of the training period. It has also prompted us to allow enough time within each session to address using the app; to ensure consistency between the content of the face-to-face sessions and the app; and to specify therapists’ tasks on the Clinician Dashboard, which need to be done between face-to-face appointments.

The limitations of the study include the small sample size, which impacts on generalisability. However, we chose not to recruit more men as themes rapidly became repetitive. The study is also limited as it refers to the use of a beta version of the app so bugs in the program have more prominence than would be expected in a final version. However, the nature of technological interventions is that they are in a constant state of development so upgrading apps is to be expected with the potential for new bugs with each upgrade.

The therapeutic alliance is usually assumed to exist between a patient and their therapist. Yet this study suggests that supplemental technologies can also become an important feature of this relationship, requiring explicit consideration when a therapist decides to adopt a blended therapy approach. This alliance can easily be damaged by bugs in the application or by a feeling that the application was not sufficiently personalised. Further studies that explore the nature of the therapeutic alliance in the context of computer assistance are needed, in particular the factors that promote or discourage such an alliance.

Clinical implications

It is feasible to deliver blended therapy to a high-risk population, and it generated new ways to provide support in this group. Clinicians and application developers need to pay attention to the therapeutic relationship with technology, as trust and good communication can be easily damaged resulting in disengagement with the application. Using a blended therapy may lead to increased adherence with face-to-face sessions.