Pragmatic clinical trial of two bilingual therapist-guided transdiagnostic iCBT programs for anxiety and depression in outpatient clinics in Canada

Highlights

• •French and English iCBT programs were assessed in routine care in eastern Canada.
• •Moderate to large improvements on all treatment outcomes were obtained.
• •Adherence was lower and attrition higher than in previous studies.
• •Satisfaction was high among study completers.
• •Therapist-guided iCBT may help to increase access to treatment for some patients.

1. Introduction

Anxiety and depression are leading causes of disability worldwide and are often comorbid (World Health Organization, 2022). Since 1990, >200 randomized controlled trials (RCTs) of Internet-delivered cognitive behavioral therapy (iCBT) have been published among adults primarily seeking treatment of anxiety and depression (Carlbring et al., 2018). Recent meta-analyses of iCBT for anxiety and or depression revealed moderate to large effect sizes on symptoms and good acceptability (e.g., Eilert et al., 2021; Mamukashvili-Delau et al., 2022). A literature review indicated deterioration rates lower than control groups and comparable to the ones reported in face-to-face therapy (Andersson et al., 2019). While the efficacy of guided iCBT for anxiety and depression is well established, less is known on the generalizability of these results to routine care. Only a limited number of pragmatic trials have been conducted so far.

In a descriptive analysis, the successful implementation of guided iCBT in routine care in five countries revealed large within group effect sizes on primary outcome measures (d ≥ 0.8) as well as low deterioration rates, and high treatment satisfaction (Titov et al., 2018). Effective monitoring of patients' progress and safety, as well as the utilization of evidence-based iCBT programs were identified as key factors for success. A few meta-analyses comprised of effectiveness trials of Internet-based interventions for anxiety and depression have been published (Etzelmueller et al., 2020; Moshe et al., 2021; Romijn et al., 2019). In a meta-analysis of digital interventions for depression comprised of 21 effectiveness trials out of a total of 83 studies (Moshe et al., 2021), an overall moderate effect size was found compared with control conditions (g = 0.73). Of interest, a significant interaction effect was obtained between study setting (efficacy vs. effectiveness) and guidance (unguided vs. guided) highlighting that guidance may be especially important in routine care settings. A systematic review and meta-analysis have been conducted exclusively on the acceptability and effectiveness of guided iCBT in routine care for the treatment of anxiety and depression (k = 17, N = 12,096; Etzelmueller et al., 2020). Despite the high heterogeneity between studies, effect sizes were at least moderate, the majority (61.3 %) of participants completed the intervention, and satisfaction was high. The average deterioration rate was 2.9 %.

Most trials to date have been conducted in specialized clinics focusing exclusively on the delivery of iCBT. The UK offers an example of a successful nationwide implementation of guided iCBT in non-specialized settings as part of a stepped-care approach, the NHS Talking Therapies (Richards et al., 2020). An RCT was conducted among 361 participants for the treatment of anxiety and depression. Symptom improvement was significantly higher in the iCBT group than the waitlist group with additional gains during the 12-month follow-up. Long-term cost-effectiveness was also supported. A secondary analysis of the data showed that 70 % of patients remained in remission at a 9-month follow-up while about 30 % relapsed, which is in line with findings reported for face-to-face CBT (Palacios et al., 2022). In another stepped-care trial, therapist-guided iCBT was assessed as a prequel to face-to-face therapy for the treatment of severe clinical symptoms of anxiety and depression among outpatients referred to the UK's NHS talking therapies program (Duffy et al., 2020). An open trial was conducted among 124 outpatients who were on a waiting list for face-to-face psychological treatment. Analyses revealed significant improvements of anxiety, depression, and functioning from baseline to iCBT exit, and from iCBT exit to services exit. Waiting times were reduced by 30 %. These findings support the feasibility of guided iCBT as a prequel to face-to-face therapy even for more severe presentations.

The delivery of iCBT as part of a stepped-care model is of particular interest in Canada as it aligns with the Mental Health Commission of Canada's partnership with Stepped Care Solutions, a Canadian model based on NHS talking therapies, which is being gradually implemented (Mental Health Commission of Canada, 2021).While the implementation of iCBT as part of Canadian publicly funded mental health services has been limited, guided iCBT for anxiety and depression has been implemented with success in Saskatchewan (Hadjistavropoulos et al., 2014, Hadjistavropoulos et al., 2016, Hadjistavropoulos et al., 2022). A transdiagnostic iCBT program, the Wellbeing Course, developed, tested, and implemented at the MindSpot Clinic in Australia, has been offered at a specialized iCBT unit in Regina, Saskatchewan. Following initial supportive data (Hadjistavropoulos et al., 2014), the clinical effectiveness of the Wellbeing Course when delivered at the specialized online unit (n = 260; 56.8 %) versus non-specialized community clinics (n = 198; 43.2 %) was compared (Hadjistavropoulos et al., 2016). Significant and large reductions (d = 1.17–1.31) were found on outcome measures. Completion rates, satisfaction levels and outcomes did not differ between the specialized online clinic and non-specialized community clinics. In a subsequent large observational study, large improvements in depression and anxiety symptoms were found and maintained at follow-ups, which was consistent over the course of six years since the Wellbeing Course has been offered (Hadjistavropoulos et al., 2022). More than 5000 patients were screened (n = 5321), started (n = 4283) and completed all treatment material (66 %; n = 2831). Satisfaction was high. Over 50 % of patients were referred by physicians or other providers. Although more patients were likely to complete four lessons out of 5, post-treatment measures, and follow-up measures; overall, the reported trends are comparable to the ones observed in Australia over the course of seven years (Titov et al., 2020). These findings are supporting the potential effectiveness of guided iCBT in Canada in the context of publicly funded services.

Few studies have assessed the efficacy and effectiveness of iCBT among young adults. Young adults are commonly included in adult studies but are grouped under the broad “adult” category, which includes a wide age spectrum that can range from 16 years old to 80 and over (Babajide et al., 2020). To our knowledge, only two iCBT studies have been conducted among young adults in clinical settings, both supporting its effectiveness (Hobbs et al., 2017; Staples et al., 2019). Previous trials have supported the use of the Mood Mechanic Course (Dear et al., 2018; Johnston et al., 2014; Staples et al., 2019), an adaptation of the Wellbeing Course for young adults aged 18 to 25 years old. However, adherence was lower, and attrition higher in routine care compared to the Wellbeing Course (Staples et al., 2019). Also, compared to other age groups, another study found that young adults showed higher levels of distress at pre- treatment (Hobbs et al., 2017). Data on young adults is of relevance since they have the highest prevalence of depression (Statistics Canada, 2018) and a preference to solve their problems on their own, possibly because of a need for autonomy, which may influence engagement and treatment outcomes (Gulliver et al., 2010; Rickwood et al., 2007). A treatment gap for this population has also been identified leaving their mental health needs unmet (Institute of Medicine and National Research Council, 2015). ICBT may help to fill this gap while giving them the opportunity to self-manage their problems with minimal guidance.

Overall, a limited number of studies have assessed the effectiveness of therapist-guided iCBT in non iCBT specialized clinics. Few studies have also offered iCBT to patients on a waiting list in clinical settings or have assessed the helpfulness of iCBT while distinguishing among young adults and adults. iCBT programs are also rarely offered in consideration of language of preference. This is of importance as it can impact process and outcomes in psychotherapy (Verkerk et al., 2021). The current study aims to assess the effectiveness in both English and French of the Wellbeing Course (Cours Mieux-Etre; age 26 and over) and Mood Mechanic Course (Cours Mécanique de l'humeur; age 18 to 25) when provided in a therapist-guided format to waitlist and intake patients in their language of preference in routine care in New Brunswick, Canada, for the management of anxiety and depression. Based on the results obtained in routine care by Hadjistavropoulos et al., 2016, Hadjistavropoulos et al., 2022, Titov et al. (2017), and Staples et al. (2019), it was hypothesized that (1) a majority of patients would agree to undergo iCBT while on the waiting list; (2) significant improvements would be found on measures of depression, anxiety, distress, resilience, and life satisfaction; (3) a majority of study completers would report being satisfied with the programs; (4) a majority of patients would complete the programs within eight weeks; (5) gains would be maintained at a 3-month follow-up.

2. Method

2.1. Design and ethics

An uncontrolled open trial design was used with measures completed online at pre- treatment, post-treatment, and 3-month follow-up. The OQ-30.2 was administered weekly throughout the program to monitor symptoms. The study was approved by the research ethics board of the Vitalité Health Network and the Université de Moncton.

2.2. Participant recruitment and screening

Using GPower 3.1 to achieve power at 80 % (alpha = 0.05), n = 34 participants were required to detect a medium effect size for a paired t-test. A medium effect size is consistent with previous meta-analyses of therapist-assisted ICBT (Etzelmueller et al., 2020). A total of 53 participants were recruited to account for participant attrition. In Canada, mental health care within publicly funded healthcare institutions is provided at no cost (e.g., health centres, community mental health clinics, hospitals). New Brunswick, Canada, where the current study was conducted, is the only bilingual province in the country. As a result, there are two health authorities, the Vitalité Health Network and the Horizon Health Network which operate in French and English respectively (Government of New Brunswick, n.d). However, while each health authority operates in its official language, each must deliver health services to patients in their preferred official language. Recruitment in the present study was conducted within the Vitalité Health Network in one community mental health clinic and two health centres providing primary health care. These institutions are funded by the New Brunswick government to offer outpatient mental health services to patients in specific parts of the province. They were targeted for the study by the Vitalité Health Network based on limited access to mental health services and as a result longer waitlists.

Recruitment was conducted from February 1, 2021 to February 28, 2022. Patients on waitlists for face-to-face psychotherapy, who presented at the time of the initial intake assessment symptoms of a moderate severity, were first recruited followed by intake patients. The eligibility criteria were the same as the ones used in Saskatchewan so that the findings could be compared. An initial triage was first conducted by two administrative managers in consideration of the eligibility criteria described below that were not based on questionnaires scores. After potential candidates from the waitlists had been screened, iCBT was then offered on site by the study clinicians at the time of the intake assessment. Promotional material (posters and pamphlets) was displayed in all participating sites. Eligible participants were referred to the secure platform of the Telepsychotherapy Research Unit, https://www.etherapies.ca/, managed by Markanyx Solutions Inc. in Alberta, Canada. They completed online screening questionnaires about their background (age, sex, gender, level of education, employment status, income, marital status, language preferences, and city), symptoms, and treatment (past or current diagnoses, hospitalization, medication, and self- harm). The inclusion criteria were (1) 18 years or older; (2) residents of New Brunswick, Canada; (3) self-reporting symptoms of depression and/or anxiety; (4) comfortable reading and writing in French or English; (5) able to access and comfortably use computers and the Internet, and (6) available to participate in the iCBT program for 8 weeks. The exclusion criteria were (1) experiencing unmanaged symptoms of psychosis, hypomania or mania, or active suicidal ideation assessed by a score ≥ 2 to item 9 on the PHQ-9 (Kroenke et al., 2001) or a total score ≥ 7 on the SBQ-R (Osman et al., 2001); (2) suicide attempt or self-harm in the last 12 months; (3) significant problems with alcohol (score ≥ 20 on the AUDIT; Gache et al., 2005) or drugs (score ≥ 25 on the DUDIT; Berman et al., 2003) and evidence upon reviewing the results over the phone with the patients of a significant problem, and (4) currently following another psychotherapy. Of the 63 patients that were allocated to the intervention, 10 patients did not begin the lessons, therefore a total of 53 patients were included in the analyses. Of these, 69.8 % (n = 37/53) were recruited from the two participating health centres and 30.2 % (n = 16/53) from the community mental health care clinic. Participant flow is shown in Fig. 1.

Fig. 1.

Flowchart of participants.

2.3. Interventions

Two transdiagnostic iCBT programs were offered in French and English, the Wellbeing Course for adults (see Titov et al., 2015 for details) and the Mood Mechanic Course, which is an adaptation of the Wellbeing Course, for young adults (see Dear et al., 2018). The feasibility and efficacy of a self-guided French-Canadian version of the Wellbeing Course has been supported (DaPonte et al., 2018; Robichaud et al., 2020). Regarding the French-Canadian version of the Mood Mechanic Course, a feasibility trial conducted among the general population revealed significant challenges with recruitment, adherence, and attrition but satisfactory treatment outcomes (LeBlanc et al., 2022). A higher level of engagement was observed when the program was imbedded in undergraduate university classes (LeBlanc et al., 2022) with findings like the ones obtained in Australia (Dear et al., 2018).

Both programs include 5 lessons, released gradually over the course of 8 weeks based on a fixed timeframe regardless of the completion of earlier modules. They were developed using a case enhanced learning approach which helps to encourage people and standardizes the intervention process (Titov et al., 2013). Throughout the program, patients follow the progress of two individuals using the techniques taught. Each lesson is comprised of reading material on practical CBT skills to manage anxiety and depression, homework assignments, case stories, and additional resources (i.e., good sleep guide, structured problem solving and worry time, and communication skills). Core skills that are taught include the cycle of symptoms, thought challenging, controlled breathing, behavioral activation, graduated exposure, and relapse prevention. While these core skills are the same in both programs, the Mood Mechanic Course was adapted to the young adult population in several ways. Images and stories of young adults are provided and examples of CBT techniques to manage some of the difficulties commonly encountered by this population are given (e.g., worrying about the future once college is over). In addition, the vignettes used as part of case-enhanced learning are young adults as well and the content of the vignettes (e.g., unhelpful thoughts) differ from the one of the Wellbeing Course. Offering a program adapted for this age group may promote engagement which tends to be more challenging than among adults (Wood et al., 2018). Patients received weekly secured messages from their assigned clinicians as well as automated emails informing them of the availability and content of the upcoming lessons, normalizing difficulties and offering encouragement. Reminders to complete lessons or questionnaires were also provided. The programs were offered through the secure platform etherapies.ca.

2.4. Clinicians

Clinicians worked either in a community mental health centre (n = 2 registered social workers) or in the family medicine clinics (n = 3 registered social workers and one psychologist). Each demonstration site delivered face-to-face outpatient mental health care. One of the social workers from a family medicine clinic left her job mid-study and was not replaced. Prior to the study, the six clinicians participated in a two half-day didactic and experiential workshop on iCBT and therapeutic email writing offered by an experienced iCBT clinician (as described in Hadjistavropoulos et al., 2012). Clinicians were instructed to spend 15 to 20 min per week per patient to go over progress, review, and compose messages. They had the option to call patients as well if they felt that this would facilitate treatment. In their weekly messages, clinicians were instructed to use the following guidelines: 1) summarize the lesson content; 2) answer questions; 3) encourage and assist patients to use problem solving skills; 4) highlight improvement and practice of skills; 5) offer support and normalize difficulties; and 6) promote engagement with the program.

2.5. Measures

Outcome measures were brief self-reported questionnaires administered online in French or English. Patients filled out primary and secondary measures at pre-treatment, post-treatment, and at 3-month follow-up. Treatment satisfaction was administered at post-treatment only. Cronbach's alphas obtained in the present study are reported for each measure.

2.5.1. Primary outcome measures

2.5.1.1. Patient health questionnaire 9—item scale (PHQ-9; Kroenke et al., 2001, translation by MAPI research institute)

The PHQ-9 is a nine-item measure assessing the presence and severity of depressive symptoms using a scale from 0 (not at all) to 3 (nearly every day). The total score ranges from 0 to 27 with 10 or greater being the cut-off score identifying possibility of major depression (Hadjistavropoulos et al., 2022). (α = 0.83).

2.5.1.2. Generalized anxiety disorder 7-item scale (GAD-7; Spitzer et al., 2006, translation by MAPI research institute)

The GAD-7 includes seven items assessing symptoms and severity of general anxiety based on the DSM-IV diagnostic criteria for GAD. The same Likert scale as the PHQ-9 is used with total scores ranging from 0 to 21. Cut-off scores of 8 or greater suggest the possibility of generalized anxiety disorder (Hadjistavropoulos et al., 2022). (α = 0.86).

2.5.1.3. Outcome questionnaire 30.2 (OQ-30.2; Lambert et al., 2005, translation by OQ measures)

The OQ-30.2 is a 30-item measure assessing psychological distress including subjective discomfort and distress related to interpersonal relationships and social role performance. Items are answered on a scale from 0 (never) to 4 (almost always) with total scores ranging from 0 to 120. A cut-off score of 44 or greater is indicative of higher levels of distress. The total score was used to monitor patient progress. Sensitivity to therapeutic change of the OQ-30.2 has been supported (Lambert, 2015). A report is provided including a graphic display of patient weekly progress and normative comparisons of expected change over the course of therapy. (α = 0.78).

2.5.2. Secondary outcome measures

2.5.2.1. Brief resilience scale (BRS; Smith et al., 2008; French adaptation by Jacobs and Horsch, 2019)

This is a 6-item measure of resilience defined as the ability to bounce back from challenges. Items are answered on a scale ranging from 1 (strongly disagree) to 5 (strongly agree) with total scores ranging from 6 to 30 and higher scores indicating greater resilience. (α = 0.73).

2.5.2.2. Satisfaction with life scale (SWLS; Diener et al., 1985; French-Canadien adaptation by Blais et al., 1989)

The SWLS is a 5-item measure assessing overall subjective well-being and quality of life scored using a scale from 1 (strongly disagree) to 7 (strongly agree) with total scores ranging from 5 to 35 and higher scores reflecting greater life satisfaction. (α = 0.90).

2.6. Statistical analyses and clinical significance of the results

Chi-square tests and paired two-tailed t-tests were used to compare pre-treatment differences between study completers and non-completers, as well as between treatment completers and non-completers. Study completion was defined as completing the post-treatment questionnaires and treatment completion as completion of at least 4 lessons, the last lesson providing primarily a summary of content rather than new content (Hadjistavropoulos et al., 2016). Missing data were treated using Markov chain Monte Carlo multiple imputation (MI). Before doing so, descriptive statistics, including measures of skewness and kurtosis, were used to assess the distribution of the variables as well as Shapiro-Wilk tests on change scores before imputing missing data. Age as well as the outcome variables were normally distributed except for GAD-7 change scores for participants who used the English iCBT programs. Two extreme outliers were excluded which produced a non-significant Shapiro-Wilk test as well as acceptable levels of kurtosis and skewness. Boxplots also revealed cases classified as extreme outliers for the AUDIT and DUDIT measures. They were excluded from the analysis to prevent sample distribution skewness. Given the high attrition at follow-up, no statistical analyses were conducted using these data. Multiple imputations (MI) were performed based on Manly and Wells (2015)’s recommendations to replace missing data with plausible post-treatment values. Little's test supported the assumption of missingness by random (χ2 (747) = 629.236, p = .999). Pretreatment scores were included in the MI models. Missing data were imputed separately for primary and secondary measures. For the OQ-30.2, given the proportion of missing data at post-treatment (64.2 %), the last score provided in Week 7 to 9 was used as the post-treatment score. The maximum percentage of missing data determined the number of imputations (Manly and Wells, 2015). The primary and secondary outcome measures variables at pre-treatment were added as auxiliary variables to improve the MI estimates. A maximum of 58 % of cases had missing data for the primary psychological outcome measures; therefore 58 datasets were imputed with the pretreatment primary outcome measures as auxiliary variables. A maximum of 56 % of data were missing for the secondary psychological outcome measures so 56 datasets were imputed for those measures. MI are recommended for studies with high rates of dropouts and small sample sizes to help increase statistical power (Barns et al., 2006).

The programs' effectiveness was evaluated using intention-to-treat two-tailed paired t-tests. Since the number of complete case for the Mood Mechanic Course was limited (n = 5), pre-post statistical analyses were not conducted by program but by age groups with young adults defined as being 18 to 29 years old as done in previous studies (i.e., Hobbs et al., 2017). The complete case results were similar to the pooled results, therefore only the latter ones are reported. To assess treatment dosage, paired t-tests using the OQ-30.2 completed on a weekly basis were also conducted with patients who completed one or two lessons only. Effect sizes (Cohen's d) were calculated based on the estimated means. Indices of remission and reliable recovery were calculated to assess the clinical significance of the findings. The remission index was defined as the proportion of patients who initially scored at or above clinical cut-offs on primary outcome measures and then subsequently below the clinical cut-offs (GAD-7 ≥ 8, PHQ-9 ≥ 10, and OQ-30.2 ≥ 44; Kroenke et al., 2010; Spitzer et al., 2006; Lambert et al., 2005). For comparison purposes with the implementation study in Canada from Hadjistavropoulos et al. (2022), the same reliable change criteria were used in the present study (Jacobson and Truax, 1991). Clinical recovery was defined as PHQ-9 decreasing ⩾6, GAD-7 decreasing ⩾5, and OQ-30.2 resulting in a patient moving from the clinical to the non-clinical range. On the other hand, patients reporting increases in their total scores of ⩾6, ⩾5, ⩾10 points on the PHQ-9, GAD-7, and OQ-30.2 respectively were classified as having Reliably Deteriorated. Results are reported for the complete sample only given the small sample size. All analyses were performed using SPSS, version 29 (SPSS Inc., Chicago, IL).

3. Results

3.1. Patient characteristics

As shown in Table 1, the mean age of patients was 37.5 years (SD = 12.9); 69.8 % (n = 37/53) were women, 60.4 % (n = 32/53) spoke English as their first language, 52.8 % (n = 28/53) were married or in a civil union, 66 % (n = 35/53) reported having a university or college degree, 49.1 % (n = 26/53) reported working a full-time or part-time job, and 64.2 % (n = 34/53) reported an income of <49,999$. Psychotropic medication was taken by 67.9 % (n = 36/53) of patients. Most patients (n = 43/53; 81.1 %) reported previously received psychotherapy and 50.9 % (n = 27/53) were diagnosed with a mental health disorder in the past.

Table 1.

Sociodemographic and mental health characteristics of the patients.

Variable Age	n	%
Mean (SD)	37.5 (12.9)
Range	19 to 64
18 to 29	21	39.6 %
30 and older	32	60.4 %
Gender Woman	37	69.8 %
Man	16	30.2 %
Preferred language
English	32	60.4 %
French	21	39.6 %
Marital status Married/civil union	28	52.8 %
Single	23	43.4 %
Chose not to disclose	2	3.8 %
Education (completed)
Primary/secondary	14	26.4 %
College/university	35	66 %
Chose not to disclose	4	7.5 %
Occupation Full-time or part-time work	26	49.1 %
Absence from work/retired/student	24	45.3 %
Chose not to disclose	3	5.7 %
Income <49,999$	34	64.2 %
50,000$ and above	16	30.2 %
Chose not to disclose	3	5.7 %
Previously diagnosed with a mental health	27	50.9 %
disorder
Previously received psychotherapy	43	81.1 %
Currently taking psychotropic medication	36	67.9 %

3.2. Intervention usage and satisfaction

Data on uptake were captured from February to September 2021, administrative time constraints in the participating settings preventing further documentation. Of the patients deemed eligible to participate and who could be reached during that period (n = 157), slightly less than half agreed to participate in the study (n = 70/157; 44.59 %). Reasons of refusal from patients from whom feedback was documented (n = 21/76; 27.63 %) included not having access or being uncomfortable with technology or the Internet (n = 8/21; 38.1 %), a preference for face-to-face therapy (n = 7/21; 33.3 %) or a lack of time (n = 4/21; 19 %). One patient indicated that iCBT was not recommended by his family physician while another was hospitalized.

On average, clinicians sent 8.3 (SD = 2.9) messages over the course of the study compared to an average of 1.9 (SD = 2.1) sent by patients. Of the messages sent by patients, 56.9 % (n = 58/102) were regarding symptoms or material in lessons; 28.4 % (n = 29/102) were requests for technical support, and 13.7 % (n = 14/102) were brief answers to clinicians' question or thank-you notes. As can be seen in Fig. 1, slightly less than half of the participants completed all 5 lessons (n = 24/53; 45.3 %). There were 48.65 % (n = 18/37) patients from the family medicine clinics and 37.5 % (n = 6/16) from the mental health centre who completed all lessons. Rates of lesson competition were similar for the Wellbeing Course and the Mood Mechanic Course (45.24 % and 45.45 % respectively). The average number of lessons completed for the total sample was 3.5 (SD = 1.58). Of the 37 patients who completed 3 to 5 lessons, the majority (n = 26/37; 70.27 %) did not ask for additional services. A similar proportion of participants who completed treatment provided post-treatment data (45.28 % and 43.4 % respectively). Few patients completed the 3-month follow-up (n = 7/53; 13 %). Four patients formally withdrew from the study because of a lack of time, lack of energy or concentration, other priorities (separation, sick child, etc.), increase in anxiety and change of medication. No significant differences were found between completers and non-completers (see Appendix). Effect sizes nevertheless revealed that a younger age, alcohol use and symptom severity had a small adverse effect on engagement including higher levels of anxiety and depression as well as lower levels of resilience and life satisfaction.

All but one of the 22 patients who filled out the treatment satisfaction questionnaire, reported that they would recommend the course to a friend with anxiety or depression, and that the course was worth their time (n = 21/22; 95.5 %). Most patients reported finding the most useful the didactic material (n = 13/22; 59.1 %), while 22.7 % referred more specifically to the skills taught (n = 5/22). In relation to what they found less helpful, several study completers (n = 6/22; 27 %) found the content either complex, heavy, or a lot to remember given their mental state. Of interest, a patient suggested inserting quizzes to facilitate the integration of the material. A few patients (n = 3/22; 13.64 %) reported that they would have liked to talk to someone in person and that an individualized treatment would have been more beneficial.

3.3. Primary outcome measures

Observed and estimated means at pre- and post-treatment are shown in Table 2. Complete case results were the same as pooled results so only the latter are reported. Paired t-tests revealed a significant decrease in PHQ-9 scores from pre- to post-treatment (t (185) = 5.031, p < .001) as well as on the GAD-7 (t (352) = 4.37, p < .001), and on the OQ-30.2 (t (216) = 6.261, p < .001). Large effect sizes were found for the PHQ-9 and OQ-30.2, and a medium effect size for the GAD-7. Effect sizes as well as the clinical significance of results are shown in Table 2.In terms of age groups, paired t-tests indicated a significant decrease in scores from pre- to post-treatment for young adults on the PHQ-9 (t (217) = 4.113, p < .001), the GAD-7 (t (441)3.39, p < .001), and the OQ-30.2 (t (260) = 4.077, p < .001). Significant decreases in scores were also found among adults on the PHQ-9 (t (258) = 3.674, p < .001), the GAD-7 (t (460) = 3.093, p = .002), and the OQ-30.2 (t (305) = 5236, p < .001). Effect sizes ranged from moderate to large for both age groups (see Table 3).

Table 2.

Means, estimated means, standard deviation, effect sizes (Cohen's d) s, and clinical significance.

Variable	Observed mean (SD) (n = 23)		Estimated mean (SD) (n = 53)		% change	Effect sizes (based on estimated means, 95 % CI)	Clinical significance (based on observed mean)
	Pre- treatment	Post- treatment	Pre- treatment	Post- treatment	Pre-Post	Pre-Post	Remission	Pre-Post Recovery	Deterioration
All patients PHQ-9	13.09	8.26	13.83	9.07	34.4 %	0.83	50 %	37.5 %	–
	(5.16)	(6.25)	(5.52)	(7.67)		[−0.41, 1.01]	(8/16)	(6/16)
GAD-7	11.32	6.64	11.75	7.68	34.6 %	0.66	62.5 %	56.3 %	4.3 %
	(5.04)	(6.01)	(5.08)	(6.93)		[0.36, 0.97]	(10/16)	(9/16)	(1/23)
OQ-30.2	62.07	41.14	61.73	42.53	31.1 %	0.94	53.8 %	50 %	–
	(14.22)	(21.48)	(13.33)	(26.04)		[0.57, 1.21]	(14/26)	(13/26)
BRS	13.87	18.64	14.55	18.74	28.8 %	0.52	–	–	–
	(4.63)	(6.36)	(3.84)	(9.25)		[0.26, 0.92]
SWLS	19.46	24.24	18.24	23.01	26.2 %	0.78	–	–	–
	(6.91)	(6.69)	(6.78)	(8.8)		[0.36, 0.84]

Note. PHQ-9 = Patient Health Questionnaire-9; GAD-7 = Generalized Anxiety Disorder-7; OQ-30.2 = Outcome Questionnaire 30.2; BRS = Brief Resilience Scale; SWLS = Satisfaction With Life Scale.

Table 3.

Means, estimated means, standard deviation, effect sizes (Cohen's d) s for age groups and language of preference.

Variable	Observed means (n, SD)		Estimated means (n, SD)		% change	Effect sizes 95 % CI (based on estimated means)
	Pre-treatment	Post-treatment	Pre-treatment	Post-treatment	Pre-Post	Pre-Post
Age 18 to 29
PHQ-9	12.11 (9. 3.82)	6.33 (9, 3.24)	13.86 (21, 5.02)	8.34 (21, 6.58)	39.8 %	1.09 [0.47, 1.41]
GAD-7	10.75 (8, 4.71)	4.38 (8, 1.85)	11.86 (21, 4.89)	7.08 (21, 6.02)	40.3 %	0.80 [0.32, 1.41]
OQ-30.2	55.89 (10, 12.36)	32.78 (10, 7.78)	60.82 (21, 13.94)	42.43 (21, 23.64)	30.2 %	0.98 [0.43, 1.45]
BRS	14.89 (9, 4.14)	19.95 (9, 6.05)	15.62 (21, 3.74)	19.72 (21, 8.40)	26.2 %	0.55 [0.10, 1.15]
SWLS	21.64 (10, 7.03)	25.54 (10, 5.27)	20.61 (21, 7.01)	24.50 (21, 7.90)	18.9 %	0.743 [0.18, 0.86]
Age 30 and over
PHQ-9	13.71 (14, 5.92)	9.50 (14, 7.44)	13.81 (32, 5.84)	9.56 (32, 7.67)	30.8 %	0.76 [0.29, 0.95]
GAD-7	11.64 (14, 5.36)	7.93 (14, 7.18)	11.69 (32, 5.28)	8.08 (32, 7.11)	30.9 %	0.60 [0.24, 0.90]
OQ-30.2	64.26 (19, 14.54)	43.11 (19, 23.92)	62.53 (32, 13.25)	42.61 (32, 25.88)	32.0 %	1.01 [0.55, 1.38]
BRS	13.17 (13, 4.98)	17.74 (13, 6.68)	13.86 (32, 3.82)	18.10 (32, 8.61)	30.6 %	0.56 [0.21, 1/05]
SWLS	18.07 (14, 6.71)	23.40 (14, 7.5)	16.68 (32, 6.25)	22.03 (32, 8.57)	32.1 %	0.88 [0.37, 1.04]
Language English
PHQ-9	13.23 (13, 5.09)	7.77 (13, 5.96)	14.10 (31, 5.59)	8.88 (31, 7.41)	37.0 %	0.86 [0.41, 1.17]
GAD-7	10.50 (10, 5.23)	7.20 (10, 5.47)	11.74 (31, 5.51)	7.94 (31, 6.50)	32.4 %	0.66 [0.26, 0.99]
OQ-30.2	61.39 (18, 12.74)	37.71 (18, 17.35)	62.16 (31, 12.23)	41.27 (31, 23.80)	33.6 %	1.06 [0.65, 1.55]
BRS	14.92 (12, 5.02)	20.63 (12, 6.86)	14.82 (31, 3.95)	19.41 (31, 8.74)	23.5 %	0.57 [0.27, 1.07]
SWLS	19.31 (13, 6.43)	25.21 (13, 6.13)	17.35 (31, 6.17)	22.93 (31, 8.10)	24.3 %	0.961 [0.43, 1.11]
Language French
PHQ-9	12.90 (10, 5.53)	8.90 (10, 6.87)	13.45 (22, 5.40)	9.33 (22, 7.15)	30.6 %	0.83 [0.21, 1.08]
GAD-7	10.60 (10, 4.17)	6.80 (10, 7.04)	11.77 (22, 4.41)	7.48 (22, 6.69)	36.4 %	0.81 [0/24, 1.25]
OQ-30.2	63.18 (11, 16.96)	46.73 (11, 26.93)	61.44 (22, 14.89)	44.19 (22, 25.56)	28.1 %	0.95 [0.37, 1.27]
BRS	12.62 (10, 4.00)	16.27 (10, 5.04)	14.19 (22, 3.81)	17.80 (22, 7.71)	20.3 %	0.48 [0.03, 1.15]
SWLS	19.67 (10, 7.85)	22.98 (10, 7,49)	19.49 (22, 7.51)	23.12 (22, 8.35	15.7 %	0.68 [0.13, 0.77]

Note. PHQ-9 = Patient Health Questionnaire-9; GAD-7 = Generalized Anxiety Disorder-7; OQ-30.2 = Outcome Questionnaire 30.2; BRS = Brief Resilience Scale; SWLS = Satisfaction With Life Scale.

In relation to language of preference, paired t-tests showed a significant decrease in PHQ-9 scores from pre- to post-test for the English iCBT programs (t (262) = 4.139, p < .001) and French ones (t (223) = 3.501, p < .001). A significant improvement was also found on the GAD-7 f for the English programs (t (394) = 3.274, p = .001) and French programs (t (806) = 3.136, p = .002). Similarly, there was a significant improvement on the OQ-30.2 for both the English (t (277) = 5.442, p < .001) and French programs (t (372) = 3.956, p < .001). Effect sizes ranged from moderate to large for both languages of preference (see Table 3).

Patients who completed one to two lessons (n = 12) did not show significant improvements on the OQ-30.2 from pre-treatment (M = 60.58, SD = 14.789) to the last available measure up to Week 3 (M = 54.50, SD = 20.664; t (11) = 1.535, p = .153), but a small to moderate effect size was found (d = 0.443). Patients who completed 3 lessons (n = 21) showed significant reductions on the OQ-30.2 from pre-treatment (M = 63.23, SD = 12.688) to the last available measure up to Week 4 (M = 54.90, SD = 17.587; t (20) = 2.928, p = .008) and the change was of a moderate size (d = 0.639).

3.4. Secondary outcome measures

Means for the complete cases of the BRS and SWLS are also shown in Table 2. Paired t-tests revealed a significant increase from the pre- to post-treatment on the BRS (t (133) = −3.443, p < .001), and the SWLS (t (156) = −4.511, p < .001). Effect sizes for secondary outcome measures were large. There was a significant increase on scores for the young adults on the BRS (t (164) = −2.333, p = .021), and SWLS (t (178) = −2.554, p = .011), as well as for the adults on the BRS (t (12) = −2.86, p = .014), and the SWLS (t (13) = −3.69, p = .003). The increases in scores were of moderate to large sizes for both groups. Significant increases were also observed in relation to language preference. For English programs, there was a significant increase from the pre- to post-treatment on the BRS (t (152) = −3.227, p = .002), and the SWLS (t (195) = −4.401, p < .001). Significant improvements were also observed on the BRS (t (261) = −2.164, p = .031) and the SWLS (t (258) = −2.544, p = .012) for French programs. Effect sizes were moderate to large.

3.5. Clinical significance

Half of the study completers (n = 8/16) showed remission on the PHQ-9, 62.5 % (n = 10/16) on the GAD-7, and 53.8 % (n = 14/26) on the OQ-30.2 (see Fig. 2). Reliable improvement rates of 37.5 % (n = 6/16), 56.3 % (n = 9/16), and 50 % (n = 13/26) on the PHQ-9, GAD-7, and OQ-30.2 respectively were obtained. Only one of the participants (4.3 %) showed reliable deterioration on one measure, the GAD-7. Of the few patients who completed the 3-month follow-up, remission rates from pre-treatment to follow-up were slightly lower than the rates obtained post-treatment on the PHQ-9 (n = 3/7; 42.86 %), GAD-7 (n = 3/7; 42.86 %), and OQ-30.2 (n = 1/3; 33.33 %).

Fig. 2.

Percentage of participants with scores on primary outcome measures in the clinical range over time.

4. Discussion

This study aimed to assess the effectiveness of two transdiagnostic iCBT programs for anxiety or depression in publicly funded outpatient clinics in New Brunswick, Canada. Moderate to large improvements were observed on all primary outcome measures of depression, anxiety, and distress. A third to a half of patients achieved remission on these measures, while approximately a third to slightly over half showed clinical recovery. Deterioration rates were lower than 5 %, with only one participant showing reliable deterioration score on the GAD-7. The majority of patients who completed three or more lessons did not request further intervention after receiving the iCBT programs. Moderate improvements were also found on the secondary outcome measures assessing resilience and life satisfaction suggesting that the iCBT programs may have an impact not only on mental illness but also on mental health, two correlated but distinct dimensions (Westerhof and Keyes, 2010). A high rate of satisfaction among study completers was reported.

The effect sizes obtained in this study are consistent with the ones reported in clinical trials of iCBT for depression and anxiety in routine care (Etzelmueller et al., 2020). Compared with the Wellbeing Course in Saskatchewan (Hadjistavropoulos et al., 2022) and Australia (Titov et al., 2017), effect sizes were large as opposed to moderate to large in the present study. This is despite the use of the same inclusion and exclusion criteria as in Saskatchewan. The sample characteristics are like the Saskatchewan sample, but baseline scores on the PHQ-9 were slightly more elevated in our sample (13.09 vs. 12.05), and a higher percentage of patients were taking medication (67.9 % vs. 57.3 %). This may suggest a higher level of pre-treatment symptomatology and may have contributed, at least in part, to the lower rate of adherence and higher rate of attrition observed in our study, higher PHQ-9 scores being associated with both variables (Romijn et al., 2019). This in turn may have impacted effect sizes.

With regards to the Mood Mechanic Course, the small number of participants assigned to this program (n = 11) limits the comparisons that can be made with benchmarking data from Australia (Staples et al., 2019). Albeit not formally documented, fewer young adults than adults were seeking care in the participating clinics which is reflective of the fact that young adults are the least likely to seek care for mental health problems compared to the other age groups (Rickwood et al., 2007; Statistics Canada, 2018). However, when comparing treatment outcomes between age groups, our findings suggest that iCBT can be helpful to both young adults and adults recruited in clinical settings similarly to the results found by Hobbs et al. (2017) for different age groups. Our findings also suggest that participants took advantage of being able to follow iCBT using their language of preference.

The proportion of participants who chose French over English (n = 21/53, 39.6 %) is higher than the proportion of the population in New Brunswick who reported speaking French regularly at home at the time of the study (30.4 %, Statistics Canada, 2023). This might be explained by the fact that recruiting took place in regions where French is more predominant. It would be informative in a future study to assess outcomes when iCBT programs are offered in people's preferred language compared to when they are not.

In relation to engagement, adherence was lower and attrition higher than hypothesized for both iCBT programs. Adherence (≥ 4 lessons) was lower (52.38 %) for the Wellbeing Course than in Saskatchewan and Australia (69.4 % to 80.4 %). The attrition rate (56.7 %) was noticeably higher than in Saskatchewan (33.78 %) or Australia (35.4 %) albeit still in the range of dropout rates reported in blended care studies where iCBT was offered as a prequel to face-to-face therapy (30 %–59 %; Duffy et al., 2020). In relation with the Mood Mechanic Course, adherence and attrition rates were, however, comparable to the ones observed in routine care in Australia (43 % and 54 % respectively; Staples et al., 2019). A younger age and higher levels of distress, but not anxiety or depression, have been identified among a large community sample as predictors of lower adherence to the Wellbeing Course in routine care (Edmonds et al., 2018). In the current study, no significant differences could be identified between non-completers and completers. Different predictors may play a role depending on the setting (i.e., community versus outpatient clinics).

What may seem to be problematic engagement, however, may be that patients are dropping out after having received a sufficient dosage of treatment. This has been referred to as effective engagement (Yardley et al., 2016) and was suggested as a possibility in previous research of ours using the Mood Mechanic Course (Leblanc et al., 2022). While the current sample was too small to assess week-to-week symptomatic change in relation to the number of lessons completed, a significant decrease in psychological distress was found among patients who completed three lessons. About 25 % of the sample (n = 13/53) dropped out after this lesson. It could be hypothesized that some patients might not have completed treatment because they were satisfied with the improvements made. It is nevertheless unlikely to have been the only factor to impact adherence and attrition given that satisfactory engagement was found in Saskatchewan and Australia. Additional potential explanatory factors that may be helpful to consider in future effectiveness trials include the recruitment method or receiving iCBT while on the waiting list.

Clinical service recruitment (recruitment through outpatient clinics) may produce different results than recruitment from the community (open recruitment). A meta-analysis of iCBT for anxiety disorders, primarily therapist-guided, examined this possibility (k = 42; N = 3714; Romijn et al., 2019) revealed that recruitment from the community produced significantly larger decreases in anxiety than clinical service recruitment which could partly be explained by higher treatment adherence in open recruitment trials and the severity of depressive symptoms being lower at pre-treatment. This may at least in part explain the higher rates of adherence obtained in Saskatchewan. In that province, recruitment relies on a digital entry point and people reaching out are motivated to follow an iCBT program, similar to self-referrals. In our study, we used a telephone-based contact from a clinician as an entry point for about half of our patients, our waitlist patients.

The current study also included a higher proportion of outpatients from waitlists than in Saskatchewan (50.9 % vs. 11.9 %; Hadjistavropoulos et al., 2022), which may have impacted motivation. Longer waiting periods to begin the intervention have been associated with greater drop-out rates, disengagement, and reduced motivation to participate (Toso Salman et al., 2023). However, in the current study, no significant differences were found in terms of adherence and attrition between waitlist and intake participants. Another potential explanation is that in our study, all outpatients were looking for face-to-face therapy and could receive it after iCBT. Motivation among patients who receive iCBT only may differ. Since it is the only treatment received, motivation to persist may be higher (e.g., Kok et al., 2017). A lack of engagement may then reflect the extent to which iCBT is integrated into treatment service pathways. The assessment of patients' motivation and expectancies would provide helpful information in this regard.

4.1. Limitations

A high percentage of post-treatment and follow-up data were missing although pre-post findings based on multiple imputation of missing data are consistent with complete case analyses. The long-term effects of the programs are unknown for most patients. However, results from the studies on the Mood Mechanic Course and Wellbeing Course in clinical settings suggest that treatment gains can be maintained up to 3 months (Hadjistavropoulos et al., 2016; Staples et al., 2019; Titov et al., 2015). This study offers preliminary data on the use of iCBT with waitlist patients. However, given the small sample, the impact of the programs on waitlists and treatment capacity could not be captured. Another limitation is that the acceptability of the programs may be overstated since only patients who completed the study offered input on the intervention. No control group was included, but the effect sizes using the same outcome measures are consistent with the ones reported for the iCBT group (d = 0.36–0.46) in a controlled NHS talking therapies effectiveness study (Richards et al., 2020), Moreover, the iCBT programs have previously been empirically supported (Dear et al., 2015; Hadjistavropoulos et al., 2016; Robichaud et al., 2020; Staples et al., 2019; Titov, Dear, Johnston and Terides, 2012, Titov et al., 2013, Titov et al., 2014). Many participants were women, which limit the generalizability of the findings, although this is representative of what is found in clinical practice (Bushnik, 2016).

In terms of implementation, offering iCBT in non-specialized settings was met with a few challenges. Therapists had to balance iCBT, including training, recruiting, and providing two iCBT programs in French and English, with the demands of face-to-face therapy. Balancing iCBT with traditional therapy has been reported before as being difficult (Hadjistavropoulos et al., 2017). Recruiting among waitlist patients as opposed to intake patients also turned out to be more time consuming than expected. Repeated phone calls to reach patients on the waitlist were made and those were often outside of the therapists' dedicated time for iCBTThe implementation of iCBT on a larger scale would likely require additional administrative assistance. Having designated therapists to exclusively provide iCBT or a centralized and specialized iCBT clinic as done in Saskatchewan and Australia may be a more efficient model of delivery. Another challenge in terms of implementation is that several patients experienced technical difficulties. Patients' level of comfort with technology was assessed as an inclusion criterion, but some were less comfortable than others (e.g., difficulty to reset password if forgotten). It would be worth documenting in future effectiveness studies, namely when patients are not self-referred as self- referred participants may be more comfortable with technology. For patients who may be less comfortable with technology, it may be best to give them the option of completing the program in a private room in a mental health setting with a trained administrator on site to help with technical and other tasks such as printing handouts as done by Learmonth et al. (2008). Beside completing the program on site, another option would be to provide bibliotherapy instead of iCBT. Although few trials have compared iCBT to bibliotherapy in routine care, preliminary data from a few studies included in a meta-analysis suggest that it would be as efficient (Andrews et al., 2018).

4.2. Conclusion

The preliminary findings of this study add to the literature on the effectiveness of transdiagnostic iCBT programs in outpatient clinics. Treatment outcomes were consistent with the ones obtained in Saskatchewan and Australia (Hadjistavropoulos et al., 2022; Titov et al., 2017). The provision of iCBT in non-specialized settings may, however, present additional challenges in relation to implementation and engagement. The provision of iCBT in routine care is new in New Brunswick. It remains to be seen if more optimal results would be obtained if iCBT were to be offered as part of a stepped- care model as found in the UK (Richards et al., 2022). This study provided valuable information on factors that may influence engagement including the recruitment method and people experiencing technical difficulties. Minor revisions to the programs may also be considered to reduce content and facilitate learning for patients with higher levels of symptomatology. Despite the challenges encountered, these initial findings are encouraging and suggest that guided iCBT may be worth pursuing to increase access to treatment in the Canadian public health sector.

Funding

This work was supported by the Saint John Regional Hospital Foundation, through generous donations including a substantial gift from the Royal Bank of Canada, and MindCare New Brunswick.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix A. Supplementary data

Supplementary material