Predicting response to transdiagnostic iCBT for emotional disorders from patient and therapist involvement

Abstract

Background

Transdiagnostic iCBT has been shown to be effective for the treatment of emotional disorders. Less is known about the optimal level of therapist and patient involvement in these interventions. Specific characteristics of Internet-delivered interventions include treatment adherence (e.g., amount of review of the materials) and guidance (e.g., amount of therapist support). Exploring the importance of these elements in treatment outcome may help to maximize the efficiency of Internet-delivered psychological interventions.

Aim

In this study, we aimed to analyze the relationship between patient and therapist involvement (i.e., platform usage and amount of therapist guidance) in a sample of patients with emotional disorders who received transdiagnostic iCBT in Spanish public specialized mental healthcare services.

Method

This is a secondary analysis of a randomized controlled trial. The sample included 63 patients who completed transdiagnostic iCBT for emotional disorders. Platform usage metrics included number of logins into the platform and number of times the participants reviewed the modules. Therapist guidance was measured as the number of support phone calls with a therapist and their total duration (minutes). Logistic regressions and ROC analyses were performed to explore the predictive value of platform usage and therapist guidance in symptom reduction. Clinical outcomes included depressive and anxiety symptoms assessed at baseline and post-intervention. The bivariate relationship between the platform usage and therapist guidance variables was also explored.

Results

Overall, platform usage and therapist guidance were not associated with symptom improvement. However, the patient and therapist involvement parameters were intercorrelated. Specifically, the number of calls and their duration were associated with a greater number of logins (r = 0.61; p < .001) and more frequent reviews of the modules (0.46 ≤ r ≤ 0.60; p < .001). Higher baseline depression and anxiety were, respectively, associated with greater improvements in depression (r = −0.37, p = .003) and anxiety after treatment completion (r = −0.48, p < .001).

Discussion

The results suggest that there is no reliable cut-off point for platform usage and therapist guidance in predicting optimal symptom reduction. However, significant associations were found between platform usage and guidance variables that warrant additional research. More research on this topic is necessary to further clarify the role of these and other platform usage and guidance variables in Internet-delivered iCBT outcomes.

Highlights

• •Optimal levels of platform usage and guidance in transdiagnostic iCBT were explored.
• •The amount of platform usage and guidance did not predict symptom improvement.
• •The selection of the most adequate metrics for platform usage is a key issue in iCBT.
• •Specific cut-offs for optimal levels of platform usage and guidance might be difficult to establish.
• •Newer research strategies might contribute to understand the relationship between platform usage, guidance and outcome in iCBT.

1. Introduction

Anxiety and depressive disorders are the most common emotional disorders (Bullis et al., 2019) and represent a global mental health concern due to their alarming prevalence rates and associated consequences in terms of economic costs and emotional suffering (Baxter et al., 2014; Lim et al., 2018). In this scenario, for decades, researchers have claimed that efforts should be made to find more accessible, evidence-based psychotherapies for these disorders (Kazdin, 2015). As a result, more automated and easy-to-disseminate ways to deliver psychological treatments have emerged in the past two decades. These treatments include, for example, psychological treatments that are partly or completely delivered through the telephone, the Internet, an app, or a combination of media, although most of the effort has been dedicated to Internet-delivered treatments and, increasingly, app-based interventions (Mohr et al., 2017).

Of the range of Internet-delivered psychological treatments, most of the research has focused on the study of Internet-delivered Cognitive Behavior Therapy (iCBT). A large number of studies have demonstrated the efficacy and effectiveness of iCBT (Andrews et al., 2018; Carlbring et al., 2018). Although the effectiveness of iCBT has been relatively well established, less is known about the characteristics that moderate or predict treatment success or failure in iCBT. The study of predictors and moderators of treatment outcomes plays an important role in psychotherapy research because the same treatment is not likely to work for everyone in the same way (Kazdin, 2014). To date, predictors and moderators of treatment outcomes have generally included patients' sociodemographic characteristics (e.g., age, sex, educational level) and clinical status (e.g., symptom severity, diagnosis) (El Alaoui et al., 2013; Webb et al., 2017). However, in recent years, and especially with the advent of self-administered interventions, researchers have also shown interest in adherence and compliance parameters as potential predictors and moderators of Internet-delivered psychotherapy. The relationship between treatment adherence and treatment outcomes has been highlighted as a critical aspect in the literature on Internet-delivered interventions (Christensen et al., 2009; Van Ballegooijen et al., 2014). However, findings on this relationship remain unclear due to the heterogeneity in the adherence indicators used in these studies (Donkin et al., 2011; Sieverink et al., 2017). Adherence is often defined as a function of the dropout rate (the number of participants who do not complete the treatment) or treatment compliance (i.e., whether the patient completes the treatment in accordance with the study goals) (Donkin et al., 2013). The importance of treatment compliance in terms of adherence to the prescribed use of the platform (e.g., number of modules completed and use of the review content) is less clear. This is a key issue because finishing an Internet-delivered treatment (i.e., completing all the treatment modules) does not necessarily mean that the patient is actively using the intervention. For instance, research has found low or no usage, even in patients who do not drop out of Internet interventions (Christensen and Mackinnon, 2006). Therefore, a wider range of adherence aspects, such as the patients' level of activity or engagement with the program, deserve more research attention in iCBT. An advantage of iCBT is that it facilitates data collection (Andersson and Titov, 2014). For instance, Internet-delivered interventions can easily collect objective data related to platform usage, such as the number of logins, the number of modules completed, or the time spent using the program (for a review of measures of usage and adherence, see Donkin et al., 2011). This information may help to analyze whether there is a dose-response relationship between usage and treatment outcomes (Sieverink et al., 2017).

To date, the relationship between platform usage and treatment outcomes has been examined in a number of studies (Manwaring et al., 2008; Enrique et al., 2019; Couper et al., 2010; Donkin et al., 2013; Fuhr et al., 2018). For example, Donkin et al. (2013) found that the number of activities completed per login was associated with outcome improvement. In another study, Enrique et al. (2019) showed that patients who achieved clinically meaningful changes after completing iCBT for depression were, in general, more engaged with the program than those who did not experience these changes (including time spent using the platform, number of logins, percentage of treatment completion, and number of activities performed). In sum, the literature suggests that there is a positive relationship between platform usage and treatment outcomes, and that these metrics might be used to establish a dose-response relationship. For example, Enrique et al. (2019) showed that at least 7 h of platform usage distributed over 15 sessions and the completion of 30 activities during a period of 12 weeks predicted a clinically significant change in patients with emotional disorders.

Another important feature of iCBT that may have a considerable impact on treatment adherence and treatment outcomes is the nature and degree of the guidance provided to participants (Andersson, 2018). It is generally assumed that some guidance in Internet interventions is more beneficial than completely self-guided interventions. It is important to note, however, that some studies have failed to find an association between guidance and outcomes in iCBT research. For example, a study by Mira et al. (2017) did not show significant differences between guided and unguided psychotherapy in an Internet-delivered treatment for mild to moderate depression. Similarly, a study that implemented an iCBT intervention for tinnitus found similar levels of treatment efficacy in patients with and without guidance (Rheker et al., 2015). Nevertheless, most of the available evidence suggests that there is a significant relationship between guidance and treatment outcomes (Baumeister et al., 2014; Palmqvist et al., 2007). Additionally, the amount of guidance in iCBT seems to be associated with the magnitude of treatment outcomes in a linear way, that is, patients who receive more guidance appear to show greater improvement (Richards and Richardson, 2012). Research also suggests that clinician-guided iCBT (i.e., guidance includes the delivery of some clinical content) can be as effective as other types of guided iCBT where a clinical background is not necessary, such as technician-guided iCBT, which may include responding to technical questions or making general recommendations to guide a patient through an Internet-delivered intervention (Andersson et al., 2019; Richards and Richardson, 2012). However, even though the literature has shown the relationship between guidance and treatment outcomes, it is unclear whether there is an optimal amount of guidance that could predict the outcome response to iCBT (e.g., time spent on the phone with a therapist).

In the current study, we conceptualized participants' level of engagement as patient involvement and treatment guidance as therapist involvement. The aim of the present investigation was to explore the relationship between patient and therapist involvement and the response to an iCBT intervention delivered in Spanish public mental health units (i.e., predicting changes in anxiety and depression symptoms in a sample of individuals with emotional disorders). Patient involvement variables included the number of times they logged into the program and the frequency with which they reviewed the treatment contents. Therapist involvement variables were measured as a function of the time spent during phone calls to provide guidance to the patient. Specifically, we aimed to explore whether the amount of patient involvement (i.e., platform usage) and clinician involvement (i.e., amount of guidance) predicted a clinically significant change in depressive and anxiety symptoms after iCBT. In doing so, we explored whether a recommended cut-off point can be obtained for both patient and therapist involvement. In general terms, the literature suggests that guided iCBT frequently leads to better outcomes than unguided iCBT (Baumeister et al., 2014; Palmqvist et al., 2007). Based on these findings, we expected to find a positive relationship between patient involvement and reductions in symptomatology following iCBT. However, although there is some research that links the degree of guidance with the magnitude of the outcomes (e.g., Richards and Richardson, 2012), the available evidence is still limited, and so the study of this relationship warrants further investigation.

2. Methods

2.1. Design

This study is a secondary analysis of data from a randomized controlled trial (RCT) that compared transdiagnostic iCBT for emotional disorders with treatment as usual in public specialized mental healthcare services (González-Robles et al., 2015). Transdiagnostic iCBT for emotional disorders has been successfully tested in a large number of studies, showing that it is superior to waitlist conditions (Newby et al., 2016) and just as effective as face-to-face CBT (Newby et al., 2015). With regard to our RCT, the results supported the greater effectiveness of a transdiagnostic Internet-delivered protocol compared to specialized mental health care on measures of anxiety, depression, and quality of life (González-Robles et al., 2020).

The trial was approved by the Ethics Committee of Universitat Jaume I (Castellón, Spain) and the Clinical Research Ethics Committees of the three participating hospitals (Consorcio Hospitalario Provincial de Castellón, Hospital Universitario de la Ribera, and Hospital Universitario Vall d'Hebron). The study protocol was registered at ClinicalTrials.gov as NCT02345668 on July 27, 2015. More details about the study protocol and the clinical trial can be found elsewhere (González-Robles et al., 2015, González-Robles et al., 2020).

2.2. Participants and procedure

The sample was composed of adult outpatients who sought treatment for depression or anxiety problems in Spanish public specialized mental health settings (mental health units). Potential candidates were identified by the clinicians working in these centers and assessed for eligibility by the study researchers. A detailed description of the recruitment process has been described elsewhere (González-Robles et al., 2020). To participate, individuals had to meet the following inclusion criteria: (1) be at least 18 years old; (2) be able to understand and read Spanish; (3) have an email address and access to the Internet at home; (4) meet Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV) diagnostic criteria [74] for an emotional disorder (i.e., major depressive disorder, dysthymic disorder, depression not otherwise specified, panic disorder, agoraphobia, social anxiety disorder, generalized anxiety disorder, anxiety not otherwise specified, and obsessive-compulsive disorder); (5) provide written informed consent; (6) not suffer from a severe mental disorder (schizophrenia, bipolar disorder, and alcohol and/or substance dependence disorder); (7) not present a high suicide risk; (8) not suffer from a disabling medical disease that would keep the participant from carrying out the psychological treatment; and (9) not be receiving another psychological treatment during the study. Pharmacological treatment was allowed, but participants had to be taking the same dose during the two months prior to enrolling in the study. In addition, changes or increases in the medication during the study period led to the participant's exclusion from the trial. All participants provided written, informed consent to participate in the study.

In all, 326 patients were interested in the study. Of them, 281 were assessed for eligibility, and 67 were excluded for eligibility reasons. The final sample consisted of 214 participants who were randomized to transdiagnostic iCBT (n = 106) or treatment as usual (n = 108). Reasons for exclusion in the transdiagnostic iCBT group included not meeting the diagnostic criteria for an emotional disorder (n = 37), the presence of substance and alcohol dependence disorders (n = 9), suicide risk (n = 7), and the presence of severe disorders such as bipolar disorder (n = 5) and psychotic disorders (n = 2). Finally, seven additional patients were excluded for a variety of reasons (n = 7).

For the purposes of the present study, we only included the sample of patients who completed the Internet-delivered intervention. Specifically, to be able to study the relationship between patient/therapist involvement and outcomes, participants in this study were included if they had completed at least 75% of the intervention (9 out of 12 modules). Most participants completed all modules (n = 54, 86%), one patient (1,6%) completed 11 modules, two patients (3%) completed 10 modules, and six patients (9,5%) completed 9 modules. In this condition, 35 patients (33%) dropped out, seven patients were excluded because they withdrew before completing the baseline assessment, and data from one additional patient could not be retrieved. Therefore, the sample for this study consisted of 63 participants.

2.3. Intervention

The intervention was a 12-module, transdiagnostic guided Internet-delivered protocol for the treatment of emotional disorders. The protocol was delivered through a web platform (https://psicologiaytecnologia.com/) designed by our research group. The main core components are derived from the Unified Protocol (Barlow et al., 2011a, Barlow et al., 2011b), but it also incorporates treatment strategies from Dialectical Behavioral Therapy (Linehan, 1993). The treatment is focused on teaching the individuals adaptive emotion regulation strategies through the following core components: present-focused emotional awareness, cognitive flexibility, emotional avoidance and emotion-driven behaviors, and exposure procedures (interoceptive and situational) (Modules 4 to 11). Most of the content in modules 4 to 11 was adapted from the Unified Protocol. However, modules 4 and 5 also incorporated Dialectical Behavioral Therapy techniques, such as mindfulness “what skills” (observing, describing, and participating) and “how techniques” (non-judgmentally, one-mindfully, and effectively). The protocol contains three additional modules (Modules 1 to 3). They include an introductory module, a module to promote the patient's engagement with the therapy (i.e., motivation to change), and a module with psychoeducation about emotions (e.g., nature and role of emotions). The protocol ends with a relapse prevention module (Module 12). The 12 treatment modules are preceded by a “Welcome module” with general information about the protocol and its goals, as well as recommendations for how to use the program. The objectives of the modules are described in Table 1. Regarding the program structure, the modules are presented sequentially (i.e., a module cannot be accessed until the previous one has been completed), and participants are encouraged to review the modules as often as they wish. Fig. 1 displays a screenshot of this functionality.

Table 1.

Objectives of the treatment modules.

Module	Objective
Module 1. Introduction to treatment	Provides a framework about the role of emotion regulation in emotional disorders.
Module 2. Motivation for change and goal setting	Analyzes pros and cons of changing, emphasizes the importance of being motivated, and helps to establish significant life goals.
Module 3. Understanding the role of emotions	Provides psychoeducation about the roles and functions of emotions and trains the patient to track the three components of emotional experiences.
Module 4. Non-judgmental emotional awareness and acceptance of emotional experiences	Teaches the patient non-judgmental emotional awareness (i.e., mindfulness “what” and “how” skills) and the acceptance of emotional experiences.
Module 5. Practicing present-focused awareness	Continues to practice the acceptance of emotional experiences and increase awareness of physical sensations, thoughts, emotions, and daily activities.
Module 6. Learning to be flexible	Focuses on identifying maladaptive ways of thinking (i.e. thinking traps).
Module 7. Practicing cognitive flexibility	Teaches patients strategies to modify thinking traps (i.e., cognitive reappraisal). Provides information about intrusive thoughts and how to deal with them.
Module 8. Emotional avoidance	Teaches patients to identify the emotion avoidance strategies that contribute to the maintenance of their emotional problems.
Module 9. Emotion-Driven Behaviors	Teaches the concept of emotion-driven behaviors and how to replace them with more adaptive behaviors.
Module 10. Accepting and facing physical sensations	Teaches the role of physical sensations in the emotional response and provides training in interoceptive exposure.
Module 11. Facing emotions in the contexts in which they occur	Builds exposure hierarchies to help patients begin to face situation-elicited avoided emotions.
Module 12. Relapse prevention	Reviews what patients have learned throughout the program. Schedules future practice of the learned strategies and teaches patients how to identify and cope with future high-risk situations.

Fig. 1.

Screenshot of the treatment platform. Patients have the option to review (“repasar”) the modules.

2.3.1. Guidance

All the participants received therapist and automated guidance. Therapist guidance included one weekly brief phone call with a maximum duration of 10 min. These weekly phone calls were previously scheduled with the patients and were conducted by two of the researchers involved in the trial. Phone calls were not intended to address clinical content. The objectives of these phone calls were: 1) to explore difficulties or concerns the participants might have encountered while using the treatment and help them to solve these problems, 2) to remind them to review the treatment contents as often as necessary, 3) to emphasize the importance of completing the homework tasks, 4) to encourage the participants to keep using the protocol, as well as to reinforce them for engaging in the treatment, and 5) to recommend that they complete one module per week.

Automated guidance consisted of a) 2 weekly brief text messages and b) guidance delivered throughout the treatment modules. Automated guidance included: a) reminders about the importance of reviewing the contents as needed, b) messages encouraging the user to complete the proposed activities in each module, c) messages with positive reinforcement after completing a module and when starting a new one, and d) messages to encourage the user to complete the treatment at an adequate pace (one module per week).

For the purpose of this study, and because all the patients received the same amount of automated support, we only included therapist support (i.e., number of phone calls and their duration) in the analyses.

2.4. Measures

2.4.1. Primary outcome measures

The Beck Anxiety Inventory (BAI) (Beck and Steer, 1993; Magán et al., 2008) is a 21-item self-report questionnaire that assesses anxiety. Scores can range from 0 to 63 points. Patients are asked to report on the severity of their symptoms during the previous week on a 4-point Likert scale (from not at all to severely). The BAI has shown good to excellent internal consistency in previous validation studies (alpha between 0.85 and 0.94), as well as convergent and divergent validity. The Spanish version has demonstrated excellent internal consistency (α = 0.93; Magán et al., 2008). The Cronbach's α for the BAI in this study was 0.93.

The Beck Depression Inventory, second edition (BDI-II) (Beck et al., 1990; Sanz et al., 2005) is one of the most widely used instruments for the assessment of depressive symptoms. The scale contains 21 items about the different symptoms characterized in major depressive disorders. Scores are added together to obtain a total score that can range between 0 and 63. The BDI-II has demonstrated good internal consistency in previous research (α = 0.76–0.95). The Spanish version showed good internal consistency for both the general and clinical populations (alphas between 0.87 and 0.89; Sanz et al., 2005). The Cronbach's α for the BDI-II in this study was 0.91.

2.4.2. Patient involvement (platform usage) metrics from the baseline assessment to post-treatment

Number of sessions: This metric was established to register the times the patient logged in or accessed the treatment platform. When a specific session had inactivity periods longer than 60 min, the next moment of activity was counted as a new login.

Number of reviews: This metric included the number of times the patients reviewed the modules, that is, the number of times they reopened a given module after the first login. Therefore, this count did not include the first time the patient accessed each module.

2.4.3. Therapist involvement (guidance)

All patients received a weekly phone call with a maximum duration of 10 min. Because some of the scheduled phone calls were unanswered, the analysis of therapist involvement included a) the number of phone calls that were successfully completed and b) the total duration of all the calls received during the treatment.

2.5. Analytic procedures

First, the sample characteristics were described (means and standard deviations of study variables). Next, the bivariate associations between usage (logins and reviews made), guidance (number and duration of phone calls), and clinical variables (baseline depression and anxiety and their change scores after treatment) were calculated by means of Pearson correlations. To calculate changes in depression and anxiety to include them in the Pearson correlations, we subtracted post scores from pre scores (e.g., change in depression = post-treatment depression – pretreatment depression). Therefore, negative scores reflect a reduction in symptomatology (i.e., improvement).

Finally, two types of analyses were performed to explore whether clinically significant changes (CSC) in outcomes were predicted by patient platform usage and guidance. First, a logistic regression was computed to explore the extent to which more practice and guidance helped to classify those who presented a clinically significant reduction in symptomatology (depression and anxiety). Second, a receiver operating characteristic (ROC) curve was calculated to obtain sensitivity and specificity scores that would reveal optimal levels of practice and guidance (cut-offs).

A clinically significant change in outcomes was considered to occur when a person moved at least two standard deviations from the mean of the group (Jacobson et al., 1984). Because the sample included patients presenting quite diverse symptom severity (i.e., mild, moderate, or severe), for each outcome, the analysis of the clinically significant change was carried out with each of the three subgroups according to symptomatology severity. For example, for depression, cut-offs have been proposed to be 14–19 for mild depression, 20–28 for moderate depression, and above 28 for severe depression (Sanz et al., 2014). For each of these subgroups, a separate mean and standard deviation was calculated, and these values were used to decide whether a patient in that subgroup presented a clinically significant change. The same procedure was followed for anxiety scores using the recommended cut-offs for mild (between 8 and 15), moderate (between 16 and 25), and severe (above 26) anxiety (Sanz et al., 2012).

All analyses were performed with SPSS version 22 (IBM Corp, 2013). Alpha levels were set at 0.01 to reduce the risk of unimportant effects and false positives when conducting multiple comparisons.

3. Results

3.1. Sample characteristics

Table 2 shows the baseline sociodemographic and clinical characteristics of the sample.

Table 2.

Baseline and sociodemographic characteristics of the sample (N = 63).

Age (years), mean (SD)		38.64 (10.61)
Sex, n (%)	Female	42 (66.7)
	Male	21 (33.3)
Marital status, n (%)	Single	13 (20.6)
	Married/partnered	41 (65.1)
	Divorced/widowed	9 (14.3)
Education, n (%)	Basic studies	16 (25.4)
	Secondary studies	22 (34.9)
	University studies	25 (39.7)
Occupation, n (%)	Student	3 (4.8)
	Housekeeper	4 (6.3)
	Employed	34 (54)
	Unemployed	13 (20.6)
	Off work	8 (12.7)
	Retired	1 (1.6)
Monthly income (€), n (%)	None	13 (20.6)
	<641.40	9 (14.3)
	641.40–1282.80	25 (39.7)
	1282.81–2565.60	15 (23.8)
	>2565.60	1 (1.6)
Principal diagnosis, n (%)	GAD	15 (23.8)
	AG	5 (7.9)
	PD	10 (15.9)
	SAD	3 (4.8)
	OCD	3 (4.8)
	MDD	11 (17.5)
	DD	6 (9.5)
	Anxiety NOS	10 (15.9)
Comorbid diagnoses, n	GAD	5
	PD	5
	AG	12
	SAD	5
	OCD	1
	MDD	7
	DD	9
	Anxiety/depression NOS	2
	Alcohol abuse	1
Number of comorbid disorders, n (%)	0	36 (57.1)
	1	15 (23.8)
	2	7 (11.1)
	≥3	5 (7.9)
Medication, n (%)	None	16 (25.4)
	Antidepressant	14 (22.2)
	Anxiolytic	9 (14.3)
	Both	24 (38.1)

GAD: Generalized anxiety disorder; PD: Panic disorder; AG: Agoraphobia; SAD; Social anxiety disorder; OCD: Obsessive-compulsive disorder; MDD: Major depressive disorder; DD: Dysthymic disorder; NOS: Not otherwise specified.

3.2. Patient and therapist involvement results (platform usage and guidance)

In terms of practice, study participants logged into the treatment and accessed the review content a median of 28 (mean = 30.4, SD = 26.0) and 2 times (mean = 12.6, SD = 18.6), respectively. Regarding guidance, participants received a median of 8 calls (mean = 7.2, SD = 4.1). The phone calls to the participants had a total median duration of 48.9 min (mean = 37.2, SD = 41.0). Table 3 shows the bivariate association between platform usage (logins and reviews of content), guidance variables (minutes during phone calls), and the clinical characteristics of the sample (baseline depressive and anxiety symptoms and changes in both outcomes after the treatment). The Pearson correlations revealed a significant and positive relationship between the number of phone calls received and the number of times participants accessed the platform (r = 0.61; p < .001) and reviewed the content (r = 0.46; p < .001). Similarly, a significant and positive relationship was observed between the time spent during the phone calls and the number of times patients logged into the platform (r = 0.61, p < .001) and reviewed the modules during the treatment (r = 0.60; p < .001). There was also a significant relationship between the number of phone calls and their total duration at the end of the study (r = 0.80; p < .001), as well as between the number of logins to the platform and the number of times they reviewed the content (r = 0.84, p < .001).

Table 3.

Correlations between platform usage, guidance, and clinical outcomes.

	Mean (SD)	Reviews	Number of calls	Duration of calls	Change BDI-II	Change BAI	BDI-II (baseline)	BAI (baseline)
Logins	30.39 (25.99)	0.84**	0.61**	0.61**	0.10	0.15	0.03	0.04
Reviews	12.55 (18.63)		0.46**	0.60**	0.01	0.14	−0.02	−0.06
Number of calls	7.21 (4.12)			0.80**	−0.19	−0.01	0.10	0.10
Duration of calls	48.94 (40.98)				−0.06	0.06	0.21	0.09
Change BDI-II	−9.03 (9.82)					0.42**	−0.37*	−0.08
Change BAI	−6.41 (9.77)						−0.05	−0.48**
Baseline BDI-II	23.70 (11.03)							0.39**
Baseline BAI	20.14 (11.96)

* p < .01, ** p < .001. Duration of calls represented in minutes. Change scores are calculated as post scores – pre scores, so that negative scores indicate a reduction in symptomatology (i.e., improvement). BDI-II: Beck Depression Inventory (2nd edition); BAI: Beck Anxiety Inventory.

Regarding clinical variables, patients who were more depressed at baseline were more likely to improve on depression after completing the treatment (r = −0.37, p = .003). The same relationship was found between baseline anxiety and anxiety improvement at post-treatment (r = −0.48, p < .001).

Neither baseline levels of depression and anxiety nor changes in these outcomes after treatment were linearly associated with platform usage and guidance variables (all p > .01). Establishing a less restrictive alpha level, the results would show that patients who were more depressed at baseline received more guidance, that is, longer phone calls (r = 0.22, p = .034).

3.3. Percentage of individuals presenting a CSC in symptoms

A CSC in depressive and anxiety symptoms was obtained by 61.7% and 41.5% of participants, respectively.

In the mild (n = 9), moderate (n = 17), and severe depression groups (n = 22), a CSC was obtained by 77.8%, 64.7%, and 54.5% of participants, respectively. In the mild (n = 19), moderate (n = 12), and severe depression groups (n = 22), a CSC was reported by 47.4%, 66.7%, and 22.7% of participants, respectively.

3.4. Predicting CSC classification based on patient and therapist involvement: logistic regression and ROC analysis

Neither patient nor therapist involvement predicted the classification of CSCs in depressive or anxiety symptoms (Table 4).

Table 4.

Binary logistic regression predicting the classification of CSCs in depressive and anxiety symptoms.

	CSC in depressive symptoms		CSC in anxiety symptoms
	Beta	P	Beta	P
Logins	−0.03	0.254	−0.03	0.165
Reviews	0.04	0.200	0.02	0.510
Number of calls	−0.16	0.299	0.12	0.447
Duration of calls	−0.02	0.095	−0.01	0.501

CSC: Clinically significant change.

Nagelkerke's R2 for depressive and anxiety symptoms were 0.12 and 0.08, respectively.

The results of the ROC analyses, reported in Table 5, revealed that none of the predictions had sufficient classification capacity (all areas under the curve were below 0.5 and non-significant). Therefore, no value of platform usage and guidance had an adequate sensitivity and specificity ratio.

Table 5.

Receiver operating characteristic curve analyses.

	CSC in depressive symptoms		CSC in anxiety symptoms
	AUC	P	AUC	P
Logins	0.45	0.597	0.37	0.119
Reviews	0.44	0.485	0.41	0.284
Number calls	0.47	0.768	0.49	0.914
Duration calls	0.41	0.322	0.42	0.354

CSC: clinically significant change; AUC: area under the curve.

4. Discussion

The current study analyzed the role of patient and therapist involvement in the effectiveness of transdiagnostic iCBT for patients with emotional disorders attending Spanish specialized mental healthcare services. Patient involvement variables included platform usage factors, that is, how many times the patients logged into the program and how often they reviewed the treatment contents. Therapist involvement was evaluated as the number of phone calls with a therapist throughout the treatment, as well as their duration. Overall, the results do not show that the amount of patient and therapist involvement predicted symptom reduction after a transdiagnostic iCBT. In this line, no cut-off point was found that could differentiate those who presented a clinically significant reduction in symptoms from those who did not.

In terms of patient involvement, the results obtained in this study suggest that the amount of platform usage, defined as the number of logins and the number of module reviews, does not predict the patients' response to transdiagnostic iCBT. Our results differ from other studies that found a relationship between platform usage and outcomes in iCBT (e.g., Manwaring et al., 2008; Enrique et al., 2019; Donkin et al., 2013; Fuhr et al., 2018). However, it is important to note that the relationship between usage and outcomes in the existing literature is not consistent across measures, and it is related to the type of usage metric selected. For instance, Manwaring et al. (2008) found that the amount of platform usage predicted specific eating-disorder symptom improvement in a sample with eating disorders but no other related clinical symptoms. In another study that tested Internet-delivered psychotherapy for depression, the association between various usage metrics and treatment outcomes was analyzed. The results showed that only one usage metric was associated with the treatment outcome (i.e., the number of activities completed per login), whereas other measures, such as the number of logins or the number of activities completed, were not related to symptom reduction (Donkin et al., 2013). In our study, the fact that we failed to find a significant relationship between usage and symptom improvement might be explained by the metrics selected in the present investigation. For example, it is possible that the number of logins and reviews does not provide information about what the participants do when they log into the platform or review the treatment modules. Specifically, a patient might login quite frequently, but the number of logins per se might not provide information about how the patient is actually using the treatment (e.g., how much the patient is using the treatment strategies in real contexts or whether the patient performs the suggested activities). In other words, the patient might be using the program passively rather than actively (Enrique et al., 2019).

With regard to guidance, no relationship was observed between the amount of guidance received (number and duration of phone calls) and the treatment outcome. To date, the iCBT literature has generally been interested in comparing therapy with and without therapist guidance, generally supporting the idea that some therapist guidance is preferable (Aardoom et al., 2016; Baumeister et al., 2014; Palmqvist et al., 2007). What is less clear is how much guidance is necessary or optimal and which sources of guidance are preferable (e.g., phone calls, emails, text messages, etc.). This is important in terms of costs (e.g., texts can be more automatized and require less therapist involvement). Our results do not support the idea that certain levels of support are better than others, which suggests that the amount of therapist involvement will require some flexibility depending on the patient's response to iCBT.

An interesting finding was that the patients' severity before treatment onset did not predict prospective patient and therapist involvement. One might assume that patients experiencing more severe symptomatology would require more guidance (i.e., more frequent and longer phone calls) and have to check the treatment content more often (more logins into the platform and more reviews of the content). Interestingly, however, the severity of anxiety and depressive symptoms was unrelated to patient and clinical involvement, which suggests that additional factors should be investigated to explore individual differences in patient and therapist involvement in iCBT. In terms of patient involvement, several factors might influence platform usage, such as personality characteristics (e.g., intrinsic vs. extrinsic motivation) (Arnold et al., 2019) or contextual variables (e.g., time available to revise the content and quality of the technology used to visualize the treatment) (Fernández-Álvarez et al., 2017). Similarly, personality characteristics (e.g., neuroticism) and contextual factors (e.g., previous experience with Internet-delivered treatments) (Wallin et al., 2016) could predict the amount of clinician involvement patients need. However, it is worth to note that, when a less restrictive alpha was considered (i.e., p < .05), baseline depression was significantly associated with therapist involvement (longer phone calls). Taken together, these findings suggest that the extent to which these and other variables predict patient and therapist involvement in iCBT requires further investigation. Along the same lines, the lack of relationship between guidance and outcomes in this study might be partly accounted for by the bond between the online platform and the patient, in other words, the participants' therapeutic alliance with the online program (Herrero et al., 2020). Research has shown that, as in face-to-face psychotherapy, patients in Internet-delivered interventions are able to develop a bond with the online program (Berry et al., 2018), and this relationship is positively associated with treatment outcomes and satisfaction with the treatment (Gómez-Penedo et al., 2020; Pihlaja et al., 2018). Thus, it is important to note that, in the current study, guidance (phone calls) did not include clinical content. In other words, phone calls were focused on aspects such as resolving specific questions about the use of the platform, reminders about the importance of practicing, positive reinforcement for completing the modules, and so on, aspects that were also present in the treatment modules as “automated support” (see the Intervention section). It is possible that participants in this study had high levels of therapeutic alliance with the online program (i.e., patients may have benefited from automated guidance) and, therefore, that the specific type of therapist guidance provided in this study did not add anything to the intervention in terms of symptom improvement. Moreover, because both therapist and automated guidance had similar content, the specific influence of each guidance modality (therapist vs. automated) on treatment outcomes could not be ascertained. In any case, the literature suggests that the presence of some type of human contact in an Internet-delivered treatment seems necessary for some patients to foster their participation and adherence (Fernández-Álvarez et al., 2017; Schueller et al., 2017), which are core aspects in research on Internet-delivered interventions.

In the present study, the relationship between patient involvement (e.g., platform usage) and therapist involvement (i.e., guidance) was also investigated. On the one hand, the number of sessions (logins) was strongly associated with the number of reviews. That is, patients who used the program more frequently also tended to review the treatment modules more often. On the other hand, time spent on the phone was associated with higher levels of platform usage (number of logins and number of module reviews). At a glance, therapist involvement seems to increase the levels of patient involvement with the platform. That is, patients who spent more time on the phone may have presented greater platform usage because they received more positive reinforcement and had more reminders about the importance of practicing. Another possibility is that this relationship reflects the different level of engagement shown by the participants. Specifically, it is possible that patients with greater availability or interest in using the platform were also more willing to spend time on the phone. In any case, we only measured the number of logins and the number of module reviews, but we did not collect data about the specific behaviors that are assumed to cause clinical change. These might include, for example, to what degree participants understand the different components or whether participants actively and correctly completed the proposed strategies (e.g., exposure procedures, cognitive flexibility techniques, and so on). Given the undeniable importance of these variables, future research should strive to integrate assessment strategies (e.g., Ecological Momentary Assessment, EMA) (Colombo et al., 2020) to analyze how these aspects are related to treatment outcomes in iCBT. For example, specific EMAs might include questions about whether a specific treatment strategy (e.g., an interoceptive exposure task) has been practiced at a certain moment, or how many times specific techniques have been practiced throughout the day. Similarly, Ecological Momentary Interventions (EMI) (Heron and Smyth, 2010) might be used to monitor whether the homework is actively carried out by participants (e.g., a notification in the mobile phone that encourages the patients to do the homework and asks them to provide data about their emotions and thoughts after an exposure task).

Our study has limitations. First and foremost, some relevant metrics, such as the time spent using the different modules or the time spent reviewing the modules, could not be analyzed. This limitation was due to technical characteristics of the treatment platform rather than to a study design flaw. Second, because only completers were included in the analyses, the sample size was small, which limits the generalizability of the results. Third, analyses could not be separated according to the diagnosis because of sample size limitations and frequent comorbidities. Finally, the design of the current study does not allow separating the specific effects of therapist guidance vs. automated guidance in treatment outcomes.

To conclude, our study found no relationship between patient involvement (i.e., platform usage) and therapist involvement (i.e., guidance) in predicting the response to transdiagnostic iCBT for reducing depressive and anxiety symptoms in patients with emotional disorders. Thus, our findings do not support the idea that patients should be given specific recommendations in terms of platform usage, at least in terms of the variables analyzed in this study (i.e., number of logins and number of reviews of the contents). Moreover, therapist support should be provided flexibly depending on the patient and the situation. These results shed new light on the role of patient and therapist involvement in iCBT. Although patient and therapist involvement may be important, as reported in the literature, specific cut-offs might be difficult to establish (or unnecessary), at least for some usage metrics. More research is warranted to explore this specific aspect. Moreover, our findings suggest that therapist involvement is important for patient involvement in iCBT. Based on our findings, we believe that the interrelationship between the therapist and patient involvement variables, as well as the role of this relationship in treatment outcomes, is an important area of research that should be further explored. We encourage researchers to replicate the current study findings and include additional patient and therapist involvement metrics to provide a more comprehensive view of patient and therapist factors that predict response to iCBT, which is crucial for enhancing the effectiveness of these interventions. Finally, we suggest that future studies move beyond the traditional assessment and treatment approaches by integrating EMAs and EMIs to examine the relationship between therapist and patient involvement factors and outcomes in Internet-delivered interventions.

Funding

Funding for this study was provided by grants P1-1B2014-43 (Universitat Jaume I, Castellón), a PhD grant from the Ministry of Education, Culture and Sports (FPU13/00576), and CIBER Fisiopatología de la Obesidad y Nutrición-ISCIII CB06/03/0052. The funders had no role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.