Transdiagnostic internet-delivered cognitive behaviour therapy: Feasibility of a motivational interviewing resource

Abstract

Background

Despite growing evidence for the effectiveness of internet-delivered cognitive behaviour therapy (ICBT), engagement and treatment outcomes are lower for some clients. Online motivational interviewing (MI) has been investigated prior to offering ICBT to facilitate engagement and outcomes, but only appears to improve engagement.

Purpose

This feasibility study investigated the potential use of a brief MI resource offered during ICBT rather than before, by examining: (1) use of the resource; (2) client and treatment variables associated with use; (3) whether use of the resource was associated with improved engagement and outcomes; and (4) how those who used the resource evaluated it.

Method

This study used data collected from 763 clients enrolled in an ICBT course. Symptoms related to depression, anxiety and disability were assessed at pre- and post-treatment. The website tracked treatment engagement. Clients completed an MI resource evaluation measure at post-treatment.

Results

Approximately 15% of clients used the resource. Clients who were older, had higher education, scored in the clinical range on depression, and scored lower on anxiety at pre-treatment were more likely to use the resource. Those who reported using the resource had higher engagement (i.e., more lessons and messages) in ICBT, but lower improvement in disability post-treatment. Positive feedback on the MI resource outweighed negative feedback, with 94 % of clients identifying a positive aspect of the resource and 66 % of clients reporting making changes in response to the resource. Overall, the MI resource appears to be used by and perceived as beneficial by a small portion of clients who complete ICBT. The study provides insight into use of the resource and directions for future research related to MI and ICBT.

Highlights

• •How to offer motivational interviewing in internet therapy requires research
• •Brief motivational interviewing resource was developed and evaluated.
• •The motivational interviewing resource was used by 15 % of clients.
• •Resource reviewers had higher age, education and depression and lower anxiety.
• •94 % said the motivational interviewing resource was worth their time.

1. Introduction

Internet-delivered cognitive behavioural therapy (ICBT) is a modern form of treatment that uses technology to provide cognitive behavioural therapy (CBT) treatment materials online. ICBT has many beneficial features including increased flexibility and accessibility (Andersson et al., 2019) and has proven to be effective in treating many mental health disorders such as social anxiety disorder, generalized anxiety disorder, panic disorder, posttraumatic stress disorder, major depression and alcohol use disorder (Andersson et al., 2019; Carlbring et al., 2017; Hadjistavropoulos et al., 2019a). Despite the favorable features and results of this form of treatment, high levels of program non-completion remain a concern (van Ballegooijen et al., 2014; Webb et al., 2017). This may be due to challenges that clinicians face helping their clients online enhance motivation (Gandy et al., 2016). Of note, there is also evidence that dropout happens more gradually in ICBT compared to face-to-face CBT (van Ballegooijen et al., 2014), which may suggest there is a need to design ways to aid with program engagement that are available more on demand.

Motivational interviewing (MI) has been used as a possible solution to increase engagement within the face-to-face therapy. MI is a goal-oriented and client-focused style of communication used to address client's ambivalence about making a behavioural change (Miller and Rollnick, 2013). It involves asking clients open-ended questions, reflective listening by the clinician, and the strengthening of clients' change talk (Miller and Rollnick, 2013). Within the face-to-face literature, MI in conjunction with CBT is found to improve treatment initiation, adherence, and treatment response overall (Randall and McNeil, 2017).

MI research in relation to ICBT has been rather limited, and further trials are needed to explore its incorporation in ICBT. In one past study, Titov et al. (2010) examined whether online MI questions presented at pre-treatment improved treatment completion rates and clinical outcomes. The questions posed were related to participants' goals in their participation, obstacles that may affect their participation, possible techniques that could be used to overcome these obstacles, and reflection on their personal strengths. The pre-treatment MI questions were found to increase completion rates (75 % versus 56 %, respectively), but did not improve clinical outcomes or acceptability. A second study by Soucy et al. (2021) examined a more interactive online MI intervention (e.g., videos, feedback provided based on client response) at pre-treatment and compared outcomes of ICBT with or without the online MI at pre-treatment. In this study, online MI was associated with clients making more motivational statements in emails to therapists and they stayed enrolled in ICBT for a longer period than those who did not receive online MI. Nevertheless, outcomes did not differ between those who participated in online MI and those who did not with both groups showing comparable improvements in anxiety, depression, mental health related disability, and distress.

An alternative to offering MI before ICBT is to offer an optional MI resource during treatment, whereby, as desired, clients can review a resource to enhance motivation. This approach would allow clients to self-select utilization of the resource when their motivation diminishes. To date, no research has been conducted to determine the feasibility of taking such an approach to offering MI during ICBT. In this feasibility trial, clients who were enrolled in an 8- week ICBT program were given open access to an online MI resource (consisting of motivational questions) and we investigated the uptake, correlates of and satisfaction with the resource. We specifically aimed to answer: 1) what will be the use of the resource among those enrolled in ICBT?; 2) how will those who use the MI resource differ from those who do not on demographic variables, treatment engagement, treatment satisfaction, as well as outcomes (depression, anxiety, and mental health related disability); and 3) how will those who use the resource evaluate the resource?

This study was exploratory in nature as we identified no previous research to determine how frequently the resource would be used, what demographic/clinical characteristics would be associated with use and outcomes, and how the resource would be used and evaluated. Although there are no existing studies evaluating the integration of an MI resource throughout ICBT, it was hypothesized that use of the MI resource would be associated with better engagement in ICBT overall (e.g., number of lessons completed, log-ins, messages to therapist, and days between first and last log-in), and improved outcomes (e.g., greater reductions in depression, anxiety, disability) from pre-treatment to post-treatment, which is consistent with the theory of MI (Randall and McNeil, 2017; Soucy et al., 2021).

2. Methods

2.1. Participants and recruitment

Data obtained as part of routine service delivery at the Online Therapy Unit was used in this study. Clients discovered the Online Therapy Unit via a number of sources, including referral by mental health and medical providers, word of mouth and presentations, internet searches and email announcements, and other referral sources including media and posters. All clients started by applying for ICBT through the Online Therapy Unit website (www.onlinetherapyuser.ca). Institutional ethics approval was secured.

2.1.1. Sample size

All clients (N = 763) who began ICBT between January 4, 2021 and July 5, 2021 (screening for treatment took place between December 17, 2020 and June 29, 2021) were included in the study. A 6-month time frame was selected to allow for an initial assessment of the MI resource in a large sample.

2.1.2. Eligibility criteria

Clients were eligible to participate if they declared in their screening that they were: (a) at least 18 years old or more; (b) experiencing at least mild symptoms on measures of anxiety and depression (see below); (c) equipped with secure access to the internet; (d) residing in Saskatchewan for the 8-week treatment; (e) able to provide a medical contact in event of an emergency; and (f) interested in ICBT.

Clients were excluded from ICBT if they: (a) had severe drug or alcohol problems; (b) had unmanaged mania or psychosis; (c) were at high risk of suicide; or (d) were receiving mental health services from another provider more than twice per month. See Fig. 1 for client flowchart.

Fig. 1.

Client flowchart.

2.2. Measures

Demographic variables were collected at pre-treatment. The measures of depression, anxiety and disability were administered at screening as well as at post-treatment1(8 weeks after starting treatment). MI evaluation questions and treatment satisfaction questionnaires were administered at post-treatment.

2.2.1. Demographics

Client demographic variables recorded during screening included age, gender, marital status, education, employment status, ethnicity, location, and mental health characteristics.

2.2.2. Patient Health Questionnaire (PHQ-9)

The PHQ-9, is a validated self-report questionnaire consisting of nine items intended to measure depressive symptom severity (Kroenke et al., 2001). Answers were rated on a scale from 0 to 3, with a total score ranging from 0 to 27 (Kroenke et al., 2001). A score 4 or lower suggests minimal depression (Manea et al., 2012), while a score of 10 or more suggests a potential diagnosis of major depressive disorder (Manea et al., 2012). Cronbach's alpha for the PHQ-9 ranged from 0.84 (pre-treatment) to 0.88 (post-treatment).

2.2.3. Generalized Anxiety Disorder (GAD-7)

The GAD-7 is a self-report questionnaire that measures symptoms of anxiety (Spitzer et al., 2006). There are seven items generating a total score ranging from 0 to 21. Scores of 4 or below are indicative of minimal anxiety, whereas scores of 10 or above suggest clinically significant symptoms of anxiety (Spitzer et al., 2006). Cronbach's alpha for the GAD-7 ranged from 0.87 (pre-treatment) to 0.90 (post-treatment).

2.2.4. Sheehan Disability Scale (SDS)

Three items from the SDS were administered to assess impairment in work, school, and social/family functioning (Sheehan, 1983). In this online administration, items were rated from 0 to 10, resulting in a total score ranging from 0 to 30. Past research has shown the SDS to improve over the course of ICBT (e.g., Hadjistavropoulos et al., 2019b; Soucy et al., 2021; Titov et al., 2015). Cronbach's alpha for the SDS was 0.81 at pre-treatment and 0.89 at post-treatment.

2.2.5. MI Resource evaluation survey

Clients were asked to complete a post-treatment survey related to the Building Motivation Resource. The survey began by asking clients whether they had reviewed the Building Motivation Resource during the Wellbeing Course (“Yes” or “No”) and how much effort they put into working on it (1- “None at all” to 7- “A great deal”). Clients were subsequently asked how understandable the resource was (1- “Not at all” to 7- “Very”), if something new was learned from accessing it (1- “Not at all” to 7- “Very”), how helpful they found it to be (1- “Not at all” to 7- “Very”), and whether it was worth their time (“Yes” or “No”). Clients were also asked open-ended questions about what they liked and disliked about the resource and whether and what changes they had made as a result of reviewing the resource.

2.2.6. Treatment satisfaction

At post-treatment, clients were asked to rate how satisfied they were with ICBT overall on a scale of 1 to 5 (1- “very dissatisfied”, 5- “very satisfied”). They also rated whether the treatment was worth their time (“Yes” or “No”), if they would feel confident in recommending the treatment to a friend (“Yes” or “No”), if the course impacted their confidence in managing their symptoms, and their motivation to seek treatment in the future (1- “greatly reduced”, 5- “greatly increased”).

2.2.7. Engagement

The treatment platform automatically tracked engagement by counting the number of lessons each client accessed, emails sent to the therapist, emails received from the therapist, phone calls with the therapist, and days between first and last log-in.

2.3. Intervention

2.3.1. The Wellbeing Course

All clients in this study received the same ICBT course, called the Wellbeing Course, which was developed by the eCentreClinic at Macquarie University in Sydney, Australia (Titov et al., 2015) and included five lessons related to (1) the cognitive behavioural model and symptom identification; (2) thought monitoring and challenging; (3) de-arousal strategies and pleasant activity scheduling; (4) graded exposure; and (5) relapse prevention and goal setting. Each lesson included a psychoeducational slideshow, stories from other clients, and downloadable materials and activities to facilitate skill acquisition. Additional resources, including the Building Motivation Resource, on various topics were provided for all clients to review at any point during treatment (i.e., assertiveness, communication skills, managing beliefs, mental skills, managing panic attacks, managing PTSD, sleep, structured problem solving, managing alcohol use, workplace management, and motivation).

2.3.2. Building Motivation Resource

All clients could access the Building Motivation Resource at any point during the ICBT course. Additionally, therapists could recommend the resource as clinically indicated (e.g., if a client mentioned struggling with motivation). The Building Motivation Resource consisted of 10 pages, including 8 sections to help clients increase and maintain their motivation, and was adapted from Soucy et al. (2021). The first section defined motivation as something that “drives us towards accomplishing tasks, such as working on our emotional wellbeing” and emphasized the ever-changing nature of motivation (e.g., “Motivation does not continue in a straight line. Sometimes motivation is high, sometimes it is moderate, and sometimes it is low”). Clients were then asked to think about what they were trying to build motivation to do. They were prompted to identify a specific goal for something they were hoping to change or work on, why this change/goal was important to them, and how motivated they felt to make this change or to work on this goal on a scale of 0 to 100, with 100 being the most motivated possible. Following this, they were presented with a section on clarifying values including questions to generate ideas about what is important to them in various domains (e.g., family/parenting, intimacy, health/physical wellbeing). The next section contained an exercise in which the clients were prompted to imagine that they were at their 85th birthday and were being honoured by three important people in their life. This exercise was intended to encourage clients to think about their long-term goals, related to who they are and what they want to accomplish. The next section encouraged clients to examine their thoughts and behaviours and notice negative self-talk which can be a barrier to maintaining motivation. This was achieved by asking clients to consider how their thoughts and behaviours may conflict with their previously stated values, which can intentionally create discrepancy and be used as a tool for increasing motivation (Miller and Rollnick, 2013). Furthermore, clients were encouraged to identify ways in which their thoughts and behaviours had an adverse influence on their motivation. The resource also included a “looking back’ and “looking forward” section, both of which were intended to elicit client change talk. The questions in these sections support client confidence to change by having clients consider how they have handled difficult situations in the past and what strengths and resources they can use to handle current or future struggles. The “looking forward” section also serves the purpose of resolving client ambivalence to change by prompting the client to consider how their life can look different if they work towards their specific goal. The resource concluded with a summary of key points including the fluctuating nature of motivation, the importance of practicing skills from the course when motivation fluctuates, and how clarifying values, looking back to previous accomplishments, and looking forward to the future can help improve motivation.

2.3.3. Therapist support

Clients were informed that they could email their therapist at any point during the ICBT course and that their therapist would respond to any messages received on a designated day each week. Clients scoring in the clinical range (score of 10 or above) on both the PHQ-9 and GAD-7 were further told that they could pick from regular once-weekly therapist support or optional therapist support, while clients who were in the non-clinical range (score of 9 or lower on the PHQ-9 and GAD-7) were offered optional therapist support (for information on PHQ-9 cut-off scores see Manea et al., 2012 and for GAD-7 cut-off scores see Spitzer et al., 2006). In regular weekly support, therapists always sent an email to clients regardless of whether the client had emailed. In optional therapist support, the therapist only emailed if the client had sent an email to the therapist or clients exhibited increased symptoms. In emails, therapists were encouraged to: convey warmth/concern for clients, engage clients with the course material, highlight lesson content, provide feedback on symptom questionnaires, answer client questions, reinforce progress, manage risks, and remind clients of course timelines and the day of their next check-in. Phone calls were rare and were typically made only if the client and therapist deemed a phone conversation clinically necessary based on an increase of five or more points on the PHQ-9 or GAD-7, increased suicide risk, or online communication with the client. The therapist would also call the client if they had not logged in for at least 7 days during regular weekly support. Past research shows that about 20 % of clients prefer optional over regular once-weekly therapist support and both approaches result in similar large improvements in depression and anxiety (Hadjistavropoulos et al., 2019b). In terms of the resources, the therapist encouraged clients to review additional resources as clients had time or interest in their first message and then again during week five. Additionally, therapists would recommend resources as clinically indicated throughout the course (e.g., if a client mentioned struggling with motivation, clients would be referred to the Building Motivation Resource).

2.4. Data analysis

In reviewing the data, three groups were identified including those who reported accessing the MI resource, those who reported they did not review the resource, and those who did not respond to the questionnaire. A series of one-way ANOVAS and chi-square tests were performed to understand how the three groups differed at pre-treatment on both categorical and continuous demographic variables. If the findings were significant, post hoc analyses were conducted to explore the nature of the group differences. Next, we used one-way ANOVAs to compare the groups who reported using or not using the MI resource on anxiety, depression, and disability change scores where negative change scores indicate symptom improvement. We also calculated Cohen's d effect sizes for each of the outcome measures. In a sub-analysis, we excluded clients who reported mild symptoms of depression (PHQ-9 < 10) and anxiety (GAD-7 < 10) and compared change scores and Cohen's d effect sizes between resource reviewers and non-reviewers. Furthermore, we utilized chi-square tests and independent samples t-tests at post-treatment to compare treatment engagement and satisfaction between those who used or did not the MI resource.

Descriptive statistics were used to examine ratings of the MI resource. In addition, we examined responses to open-ended questions using a conventional content analysis technique (Hsieh and Shannon, 2005), which allows for different categories of responses to be quantified (Vaismoradi et al., 2013). Three coders (S·H., V.P., and H.H.) met prior to coding to go through all of the comments and develop overall codes and sub-codes regarding what clients liked and disliked about the resource, as well as any changes clients made as a result of the resource. Using the computer program Excel, the original coder S.H. coded each answer into one or more of the predetermined codes or sub-codes. S.H. made certain that the classification was drawn directly from the clients' statements (Hsieh and Shannon, 2005). The second coder, V.P., then evaluated the previously given codes by S.H and offered input on their own coding of the responses. This process was repeated until there were no inconsistencies in the categorization of the responses by the two coders and they agreed on all resulting codes.

3. Results

3.1. Client characteristics at pre-treatment

Of the 763 participants in the sample, 102 clients (13.4 %) reported using the MI resource, 378 did not (49.5 %), and 283 did not reply to the questionnaire (37.1 %). Table 1 summarizes the pre-treatment characteristics of clients, both overall and by these three groups. Overall, the mean age was 37.67 years (SD = 13.98) and the majority of clients (n = 589; 77.2 %) were female. Over half of the clients (n = 452, 59.2 %) were either in a common-law relationship or married and, approximately half of the clients had less than a post high school certificate/college diploma/university degree (n = 398; 52.2 %). Half of the clients (n = 383, 50.2 %) worked part-time or full-time, and the majority (n = 661, 86.6 %) were White and resided in a large city with over 100,000 inhabitants (n = 440, 57.7 %). Clients most often received optional therapist support (n = 477, 62.5 %) and had pre-treatment PHQ-9 (n = 559, 73.3 %) and GAD-7 (n = 549, 72.0 %) scores in the clinical range.

Table 1.

Client characteristics at pre-treatment.

Variable	All clients (N = 763)		Accessed MI resource (n = 102)		Did not access MI resource (n = 378)		Questionnaire non-responders (n = 283)		Statistical Significance
	n	%	n	%	n	%	n	%
Age
Mean (SD)	37.67 (13.98)	–	41.95 (15.07)	–	38.93 (14.48)	–	34.46 (12.13)	–	F(2, 762) = 14.96, p < .001
Range	18–82	–	19–78	–	18–82	–	18–77	–
Gender
Male	163	21.4	22	21.6	74	19.6	67	23.7	χ2(2, 752) = 1.54; p = .46
Female	589	77.2	78	76.5	298	78.8	213	75.3
Othera	11	1.4	2	2.0	6	1.6	3	1.1
Marital status
Single/never married	251	32.9	34	33.3	108	28.6	109	38.5	χ2(2, 763) = 8.76; p = .07
Married/common-law	452	59.2	57	55.9	240	63.5	155	54.8
Separated/divorced/widoweda	60	7.9	11	10.8	30	7.9	19	6.7
Education
Less than high school/ high school diploma/some college or university	398	52.2	49	48.0	175	46.3	174	61.5	χ2(2, 763) = 15.76; p < .001
Post high school certificate/diploma university education	365	47.8	53	52.0	203	53.7	109	38.5
Employment status
Employed (part-time/full-time)	383	50.2	45	44.1	187	49.5	151	53.4	χ2(2, 763) = 2.72; p = .26
Other (homemaker, student, disability, retired)	380	49.8	57	55.9	191	50.5	132	46.6
Ethnicity
White	661	86.6	96	94.1	331	87.6	234	82.7	χ2(2, 763) = 9.03; p = .01
Other	102	13.4	6	5.9	47	12.4	49	17.3
Location
Large city (over 100,000)	440	57.7	65	63.7	212	56.1	163	57.6	χ2(4763) = 5.63; p = .23
Small to medium city	113	14.8	14	13.7	50	13.2	49	17.3
Small rural location (under 10,000)	210	27.5	23	22.5	116	30.7	71	25.1
Support
Optional	477	62.5	61	59.8	220	58.2	196	69.3	χ2(4763) = 8.81; p = .01
Once-weekly	286	37.5	41	40.2	158	41.8	87	30.7
Mental health characteristics
Pre-treatment PHQ-9 ≥ 10	559	73.3	79	77.5	253	66.9	227	80.2	χ2(2, 763) = 15.63; p < .001
Pre-treatment GAD-7 ≥ 10	549	72.0	62	60.8	270	71.4	217	76.7	χ2(2, 763) = 9.49; p = .01
No clinical scores	114	14.9	16	15.7	67	17.7	31	11.0	χ2(2, 763) = 5.89; p = .05
Psychotropic medication in the past 3 months	419	54.9	55	53.9	194	51.3	170	60.1	χ2(2, 547) = 0.23; p = .90
Pre-treatment credibility Mean (SD)	20.70 (4.53)	–	20.28 (4.09)	–	20.73 (4.53)	–	20.81 (4.69)	–	F(2, 762) = 0.52, p = .60

Note: MI = Motivational Interviewing; PHQ-9 = Patient Health Questionnaire-9; GAD-7 = Generalized Anxiety Disorder-7.

^aNot included in Chi-square analysis due to insufficient number of clients in group.

Age, education, ethnicity, level of therapist support, and whether pre-treatment GAD-7 and PHQ-9 scores were in the clinical range all had statistically significant group differences. Both individuals who did, and did not, use the MI resource were older (M = 41.95; t₃₈₃ = 5.00, p < .001 and M = 38.93; t₆₅₉ = 4.20, p < .001 years respectively) than the non-responder group (M = 34.46 years). Furthermore, clients who used the MI resource had greater levels of education than questionnaire non-responders (52.0 % vs 38.5 %), X²(1, n = 661) = 14.98, p < .01. More resource reviewers identified as White than the non-responder group (94.1 % vs 82.7 %), X²(1, N = 763) = 8.00, p = .01. Clients who did not use the MI resource chose once-weekly therapist support more frequently than those who did not respond to the questionnaire (41.8 % vs 30.7 %), X²(1, n = 661) = 8.48, p = .004. Those who accessed the MI resource X²(1, n = 480) = 4.17, p = .04, and the non-responder group X²(1, n = 661) = 14.36, p < .01 had significantly more clients who scored in the clinical range on the PHQ-9 at pre-treatment (77.5 % and 80.2 %) than those who did not access the MI resource (66.9 %). In contrast, the group who accessed the MI resource (60.8 %) had fewer individuals with GAD-7 scores in the clinical range, than the group who did not make use of the MI resource (71.4 %), X²(1, n = 480) = 4.267, p = .04, and the non-responder group (76.7 %), X²(1, n = 385) = 9.49, p = .002.

3.2. Symptom scores

Table 2 details the pre-treatment, post-treatment, change scores and Cohen's d effect sizes on measures of depression (PHQ-9), anxiety (GAD-7), and disability (SDS). There were no significant differences at pre-treatment on mean PHQ-9 (F (1,478) = 3.88, p = .05), GAD-7 (F (1, 478) = 1.46, p = .23), or SDS (F (1, 478) = 0.44, p = .51) scores between those who accessed the MI resource and those who did not. Those who accessed the resource displayed substantially smaller decreases on the SDS than the group who did not access the MI resource, F (1, 477) = 6.17, p = .01, d = 0.26, 95 % CI [0.9, 1.4] vs. d = 0.52, 95 % CI [0.4, 0.6]. There were no significant group differences in PHQ-9 (F (1, 477) = 0.70, p = .40) or GAD-7 (F (1, 477) = 0.25, p = .62) change scores. In a secondary analysis, we explored if results would differ if we excluded clients who had non-clinical scores on the PHQ-9 and GAD-7. Similarly, there were no significant differences at pre-treatment on mean PHQ-9 (F (1,266) = 2.31, p = .13), GAD-7 (F (1, 266) = 0.18, p = .67), or SDS (F (1, 266) = 0.58, p = .81) scores between clients with pre-treatment PHQ-9 > 9 and GAD-7 > 9 who accessed the MI resource and those who did not. There were also no significant group differences in decreases on the PHQ-9 (F (1, 265) = 0.01, p = .90), GAD-7 (F (1, 265) = 0.31, p = .58) or SDS (F (1, 266) = 1.11, p = .29) change scores. We found comparable effect sizes between those who reviewed and did not review the resource on the PHQ-9 (reviewers: Cohen's d = 1.36, 95 % CI: 0.99–1.73; non-reviewers: Cohen's d = 1.29, 95 % CI: 1.11–1.47) and GAD-7 (reviewers: Cohen's d = 1.37, 95 % CI: 0.99–1.73; non-reviewers: Cohen's d = 1.26, 95 % CI: 1.08–1.44), and SDS (reviewers: Cohen's d = 0.38, 95 % CI: 0.10–0.65; non-reviewers: Cohen's d = 0.54, 95 % CI: 0.39–0.68).

Table 2.

Pre- and post-treatment symptom scores, change scores, and effect sizes.

Variable	All clients with pre and post scores (n = 514)			Accessed MI resource (n = 102)				Did not access MI resource (n = 378)				Significance (pre-treatment)	Significance (change scores)
	Pre-treatment	Post-treatment	Change score	Pre-treatment	Post-treatment	Change score	Cohen's d	Pre-treatment	Post-treatment	Change score	Cohen's d
PHQ-9	13.53 (5.61)	6.97 (5.37)	−5.83 (5.41)	13.65 (5.17)	7.34 (5.43)	−6.30 (5.30)	1.19 [0.93–1.44]	12.44 (5.58)	6.62 (5.18)	−5.80 (5.43)	1.07 [0.94–1.19]	F(1,478) = 3.88, p = .05	F(1, 477) = 0.70, p = .40
GAD-7	12.67 (5.04)	6.31 (4.99)	−5.84 (5.24)	11.58 (5.28)	5.89 (5.19)	−5.69 (5.34)	1.07 [0.82–1.31]	12.25 (4.89)	6.25 (4.85)	−5.98 (5.21)	1.15 [1.02–1.28]	F(1, 478) = 1.46, p = .23	F(1, 477) = 0.25, p = .62
SDS	18.24 (7.20)	13.98 (8.20)	−3.60 (7.93)	17.01 (8.03)	15.08 (8.21)	−1.93 (7.37)	0.26 [0.64–0.46]	17.54 (7.06)	13.45 (8.08)	−4.10 (7.92)	0.52 [0.41–0.62]	F(1, 478) = 0.44, p = .51	F(1, 477) = 6.17, p = .01

Note: MI = Motivational Interviewing; PHQ-9 = Patient Health Questionnaire-9; GAD-7 = Generalized Anxiety Disorder-7; SDS = Sheehan Disability Score. Change scores and effect sizes were not calculated for questionnaire non-responders as only 12 % completed post-treatment measures.

3.3. Treatment engagement and satisfaction

Table 3 includes treatment engagement and satisfaction variables, as well as comparisons between individuals who accessed the MI resource and those who did not. Those who used the MI resource were more likely to access all five lessons than those who did not use the MI resource (96.1 % vs 85.4 %), X²(1, n = 480) = 8.42, p = .004. Furthermore, clients who used the MI resource sent significantly more messages to their therapist on average, than those who did not use the MI resource, (M = 3.95, SD = 3.17 vs M = 3.28, SD = 2.96; t(478) = −2.01, p = .05). There were no significant differences found in treatment satisfaction between the two groups (p = .10–0.93). Most clients reported being satisfied or very satisfied with the course overall (n = 405, 81.3 %). The majority also believed that the course was worth their time (n = 479, 96.2 %) and that they would feel confident recommending this treatment to a friend (n = 478, 96.0 %). Finally, the majority of clients reported that their confidence had either grown, or significantly increased (n = 416, 83.7 %), as had their motivation for additional treatment (n = 385, 77.5 %).

Table 3.

Treatment engagement and satisfaction.

Variable	Accessed MI resource (n = 102)		Did not access MI resource (n = 378)		Significance
	n	%	n	%
Engagement
Accessed lesson 4	99	97.1	353	93.4	χ2(1, 480) = 1.97, p = .16
Accessed lesson 5	98	96.1	323	85.4	χ2(1, 480) = 8.42, p = .004
Mean written messages received from therapist (SD)	7.23 (2.51)	–	7.04 (2.70)	–	t (478) = −0.06 p = .53
Mean written messages sent to therapist (SD)	3.95 (3.17)	–	3.28 (2.96)	–	t (478) = −2.01 p = .05
Mean number of log-ins (SD)	27.70 (15.44)	–	26.28 (24.13)	–	t (478) = −0.56 p = .57
Mean number of phone calls with therapist (SD)	0.95 (1.32)	–	0.94 (1.30)	–	t (478) = −0.01 p = .92
Mean days between first and last log-in (SD)	85.34 (30.12)	–	80.75 (33.18)	–	t (173) = −1.34 p = .21
Satisfaction	n = 102		n = 378
Satisfied/very satisfied overall	83	81.4	309	81.7	χ2(1, 480) = 0.01; p = .93
Course was worth the time (%)	99	97.1	364	95.3	χ2(1, 480) = 0.14.; p = .71
Would recommend course to friend (%)	98	96.1	364	96.3	χ2(1480) = 0.01; p = .92
Increased/greatly increased confidence	91	89.2	312	82.5	χ2(1, 480) = 2.66, p = .10
Increased/greatly increased motivation for other treatment	78	76.5	296	78.3	χ2(1, 480) = 0.16, p = .69

3.4. Evaluation of the MI Resource

The majority (n = 93, 94.1 %) of the 102 clients who evaluated the MI resource stated that it was worth their time. Clients indicated a mean of 4.78 (SD = 1.40) to represent the amount of effort that they put into the resource where 1 is “none” and 7 is “a great deal”. They also rated a mean of 6.10 (SD = 1.00) out of 7 for understandability. Clients generally reported acquiring new knowledge from the resource (M = 5.04, SD = 1.45) and judged it as being moderately helpful (M = 5.21, SD = 1.40).

3.4.1. Liked about MI Resource

Eight codes were identified when examining what clients liked about the resource (see Table 4). Among clients who responded to the question, the feature that the majority of clients liked the most was how informative the resource was (n = 35, 45.5 %). Some clients appreciated the emphasis placed on establishing goals (n = 16, 20.8 %) or clarifying values (n = 16, 20.8 %). Others noted the resource assisted them in understanding how their symptoms affected their motivation (n = 11, 14.3 %). Furthermore, some clients said they liked the format and structure of the worksheets (n = 9, 11.7 %), and that the resource helped them improve their motivation (n = 8, 10.4 %). Finally, some individuals recalled the material as being easy to understand (n = 4, 5.2 %) or liking everything about the resource (n = 2, 2.6 %). Notably, 19.8 % (n = 19 clients) of clients who said they reviewed the resource did not offer a comment in response to the question about what they liked about the resource.

Table 4.

Most liked about the MI resource.

Most liked about the resource	Example	Client ID	n = 77	%
Informative	“It is helpful information. I am fairly certain I will refer back to it.”	34190	35	45.5
Establishing goals	“It helped me find ways to get up and start my day. Set goals that are achievable.”	33829	16	20.8
Clarifying values	“The resource is great. It covered a lot of things I've done before. It inspired me to get back to the values and personal “compass” I developed some years ago.”	34180	16	20.8
Understanding how symptoms impact motivation	“Made sense of how my behaviours effect my motivation and how setting goals and determining my values can help me stay motivated to achieve what I want.”	34684	11	14.3
Format/structure of worksheet	“I liked the DIY sections and prompts. It was actively engaging.”	34880	9	11.7
Resource helped improve motivation	“This resource helped me the most at the beginning of the course when I really didn't have the energy to do anything”	33956	8	10.4
Easy to understand	“Easy to follow, understand and read”	34014	4	5.2
Everything	“I liked all of this course”	34204	2	2.6

Note: MI = Motivational Interviewing; clients could provide more than one comment.

3.4.2. Disliked about MI Resource

Nine codes were established when examining what clients said they disliked about the MI resource. Over half of the clients stated that they disliked “nothing” about the resource and this question did not apply to them (n = 58, 65.9 %). Some clients even took this opportunity to make additional positive remarks such as, “all information is helpful and makes a person reflect³⁵⁰⁸⁵” (n = 6, 6.8 %). In other situations, clients reported that their symptoms had interfered with their ability to use the resource (n = 5, 5.7 %). Some clients also noted that they needed to review the resource again in order to appropriately answer this question (n = 4, 4.5 %).

As seen in Table 5, only 14 (15.9 %) negative comments were made about the resource. These opinions were divergent, ranging from too much information (n = 3, 3.4 %), to not enough (n = 3, 3.4 %). Some shared that the content was not relevant to their experiences (n = 3, 3.4 %) or that the resource provided no new knowledge (n = 3, 3.4 %). Two clients (2.3 %) also remarked on the format and structure, reporting that they “learn best when the information is given in an interactive manner³⁴³¹³” and that they would have preferred if they were able to “fill it in on the computer³⁴⁸²⁹”.

Table 5.

Disliked about the MI resource.

Disliked about the resource	Example	Client ID	n = 88	%
Nothing/Not applicable	“Nothing of note.”	34658	58	65.9
Positive comment	“I liked this resource and read through it more than once”	33956	6	6.8
Symptoms interfered with ability to engage with resource	“I understand that according to the resource I have to be active in order to gain motivation, however without reason to do things I feel like my brain is blocked. Even if I get myself into position to draw, play a video game, or whatever else, since I don't have a reason I just have no interest at all in doing said activity, and like I will literally sit and stare at it, knowing I want to do it, but I feel like physically, I just cannot.”	34796	5	5.7
Need to review again	“Again I felt as if I was pressed for time I wish I could have put more time into this. I plan on looking over these exercises again once this course is over so I can absorb more of the details.”	34794	4	4.5
Too much work/overwhelming	“There is a lot of work I have to do for that lesson. It makes me feel overwhelmed.”	33961	3	3.4
Content was not relevant to their experience	“I don't think it really applied to my situation that much”	34854	3	3.4
Lack of new information	“I don't think I learned much from it as it was things I had already learned.”	33995	3	3.4
Not enough information	“Compared to some of the other resources this was one was shorter.”	34791	3	3.4
Format/structure of the worksheet	“That I couldn't fill it in on the computer”	34829	2	2.3

Note. MI = Motivational Interviewing; clients could make more than one comment.

3.4.3. Changes made as a result of the MI Resource

As shown in Table 6, the majority of clients who reported using the MI resource reported making a change as a result of the resource (n = 61, 65.6 %). Most commonly clients stated that after reviewing the MI resource, they started to use skills from the course to address symptoms (n = 18, 19.4 %). Other clients noted making changes such as breaking down goals into smaller steps (n = 12, 12.9 %), or made a generic comment about working on exercises from the resource (n = 11, 11.8 %), increasing their focus on their values (n = 7, 7.5 %), or increased awareness of how their symptoms impact motivation (n = 6, 6.5 %). Nevertheless, some reported making no changes as a result of the resource (n = 17, 18.3 %), that they were still attempting to make changes but had not yet done so (n = 8, 8.6 %), or expressed uncertainty about whether they had made changes (n = 7, 7.5 %).

Table 6.

Changes made as a result of the MI resource.

Changes made as a result of MI resource	Example	Client ID	n = 93	%
Yes	“I'm making changes.”	34368	61	65.6
Started to use skills from the course to address symptoms	“focusing more on goals and thought challenging.”	34140	18	19.4
Working on breaking down goals into smaller steps	“Broke goals down into smaller steps and reached for those smaller goals to build to bigger ones, increasing my motivation and well-being.”	34424	12	12.9
Generic comment about working on the exercises from the resource	“Worked through the papers”	34956	11	11.8
Increased focus on values	“Yes, I reincorporated my values and personal “compass” as part of my morning meditation routine.”	34180	7	7.5
Greater awareness of how symptoms impact motivation	“It helped me to see how things connected and how it could be a result of the negative self talk I had and the focussing on problems I used to do.”	34491	6	6.5
None	“I attempted to, but employment demands (including frustrations) have prevented me to dedicate the time I need to fully utilize what the resources provides.”	34412	17	18.3
Trying	“I will try.”	34684	8	8.6
I don't know/I need to review resource again	“Need to read again”	35068	7	7.5

Note. MI = Motivational Interviewing; clients could provide more than one comment.

4. Discussion

ICBT is an accepted form of treatment for anxiety and depression (Carlbring et al., 2017). However, some clients have difficulty maintaining motivation and commitment to treatment, and more research is needed to identify how to best support clients during ICBT (van Ballegooijen et al., 2014). Online MI has been offered before ICBT in an attempt to improve adherence to ICBT and appears to improve some indicators of treatment engagement, but has not improved treatment outcomes (Soucy et al., 2021; Titov et al., 2010). Soucy et al. (2021) suggested that motivational deterioration may be more likely to occur during, rather than before treatment and recommended offering an MI resource during ICBT.

The current study followed up on this recommendation and investigated the feasibility of an MI resource made available to clients at any point during ICBT. We used data from 763 clients enrolled in ICBT to explore the potential use and evaluation of this resource. Overall, use of the MI resource was quite minimal with only 13.4 % of clients reporting they accessed the resource as compared to 49.5 % (n = 378) of clients who reported not using the resource and 37.1 % (n = 283) who failed to respond to questions about use of the resource. Of importance, while use was low, there was some evidence to suggest that the resource was used by those most in need. Specifically, clients who viewed the resource were more likely to score in the clinical range for depression. Moreover, those who did not have as much improvement on mental health disability scores at post-treatment also reported greater use of the resource, although this was only the case when the larger sample was examined. No group differences were found in improvements on mental health related disability scores in the sub-analyses that excluded clients with mild symptoms of depression and anxiety. These findings are encouraging and consistent with the tenant proposed by Hettema et al. (2005) that MI should only be administered to those low on motivation so as to not contradict the motivation already experienced by those who are high on motivation. A further trend identified in the use of the MI resource is that it appears that those who used the resource were more engaged with ICBT generally. Given that this is an observational trial, however it is important not to draw causal conclusions about this finding (e.g., we can't say that using the resource, increased engagement). It is quite possible that clients who were engaged with ICBT generally, were more likely to review the resource, more likely to complete the course, and more likely to engage with therapists. Nevertheless, it is notable that Titov et al. (2010) and Soucy et al. (2021) also found that those who utilized MI were enrolled for a significantly longer time in treatment than those who did not (Titov et al., 2010; Soucy et al., 2021). In general, the association represents valuable clinical knowledge suggesting that clients who use the resource are more likely to complete the program.

In general, among those who used the resource, engagement with the resource and satisfaction with the resource was promising. Notably, those who endorsed using the resource, reported putting in a moderate degree of effort into the resource and favorably rated the resource. The resource's strengths were related to the information provided and offering client's assistance with goal setting and clarifying values. The majority of clients (65.6 %) who used the resource reported making a positive change as a result of this resource. In general, negative comments from clients who used the resource were quite rare and diverse in nature such that they did not provide clear direction for improvement of the MI resource. For example, some clients reported finding the resource provided either too much information, while other clients said the resource was not detailed enough or insufficiently novel.

4.1. Study strengths

While the current study is preliminary, it serves to advance the literature on use of MI as an adjunct to ICBT and sheds light on this method of assisting clients with motivation. Using a large observational sample of 763 clients, we gathered useful information about how frequently such a resource would be used, who would use the resource, and the correlates of using the resource. Furthermore, we acquired useful qualitative and quantitative information about how the resource was used as well as its strengths and weaknesses. These findings have the potential to guide future implementation studies, suggesting that while <15 % will use this resource, those that do will be satisfied with the resource and rate it as being worth their time. It is also helpful for therapists to know that use of the resource is associated with higher depression as well as higher course completion rates.

4.2. Study limitations and future directions

One of the study's significant weaknesses was that 37.1 % of clients did not respond to the MI resource questionnaire. We cannot draw many conclusions from this non-responder group, other than that they tended to be younger and more often chose optional therapist support, than the other two groups. They were also less educated and more diverse in terms of ethnocultural background than those who completed the MI resource, as well as more depressed and anxious compared to those who did not complete the resource. As a result, it is unclear if more clients who did not complete the post-treatment MI resource survey meaningfully interacted with the resource.

Another source of missing data was related to timing, as we did not record when clients opened the resource. This data would have been useful in determining whether clients opened the resource towards the beginning, middle or end of treatment. Therapists would then have a better understanding of when client's motivation waned during therapy, which might dictate future schedules for offering MI. Another limitation is the lack of measures related to motivation and readiness to engage in behavioural changes. This limitation prevents us from knowing if the MI resource is addressing motivation or readiness to change. These measures would have allowed us to investigate if motivation and readiness to change decreased over treatment or triggered use of the resource. In the future, it would be beneficial to examine if change in motivation precedes opening the resource and whether motivation improves after the resource is accessed.

It was difficult to determine how much of the resource clients recalled since the survey was administered at the end of treatment and an unknown period of time since using the resource, with some clients stating that they had forgotten and needed to examine the resource again. That being so, in the future it would be beneficial to administer the resource evaluation questionnaire soon after clients view the MI resource. Since this study is correlational, it is also not possible to determine whether the MI resource increased engagement or motivation. It could be that those who accessed the MI resource were more engaged in treatment to begin with.

In terms of outcomes, counterintuitively, reviewing the MI resource was not associated with improved outcomes. This could reflect that using the MI resource does not improve outcomes or could be related to other factors such as small sample size and lack of data on questionnaire non-responders.

To better understand the influence of the MI resource on engagement and outcomes, a randomized controlled study comparing individuals who are offered the MI resource to those who are not would be required. However, such a study, would need a very large sample size as <15 % of all clients reported using the resource. To further investigate the advantages of the resource, follow-up measures beyond 8-weeks would be preferable in order to gain a better understanding of the long-term effect of the MI resource that were not explored in this study.

There are several directions for future research that may help advance how to best incorporate MI into ICBT. It may be helpful to explore options for alternative MI delivery modalities that are more similar to face-to-face MI, such as conducting MI phone or video calls. This would allow therapists to retain the spirit of MI by using both verbal and non-verbal prompts and maintaining the “way of being” with clients that is central to MI (Miller and Rollnick, 2013). It could also be beneficial to further explore what leads clients to open an MI resource, which would add to the research into the mechanisms underlying motivational decline and enhancements.

Additionally, conducting further research into offering intermittent MI would be another possible method of integrating MI into ICBT. This would involve administering brief MI sessions over the course of ICBT to ensure that the benefits of MI do not fade over time while also ensuring that the length of treatment is not extended (Marker et al., 2020). At this time, it is unknown how many clients actually need MI and thus further assessment of motivation during ICBT would be valuable. It would be worthwhile to have clients submit responses to the MI resource to therapists for further discussion. This would provide therapists with more information, allowing them to better support their clients.

4.3. Conclusion

Previous research on MI has shown that it is a successful method for improving motivation in face-to-face therapeutic settings (Randall and McNeil, 2017). For the first time, we examined how clients might use an optional MI resource during ICBT. We discovered that approximately 15% of clients used the MI resource with the majority evaluating it favorably. Furthermore, those with higher levels of depression were more inclined to use it, as were clients who showed lower improvement in mental health-related disability. Future studies should build on this preliminary evaluation of MI integration in ICBT, especially by measuring motivation over the course of ICBT and having therapists support clients in their use of the MI resource.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.