Improving internet-delivered cognitive behaviour therapy for alcohol misuse: Patient perspectives following program completion

Abstract

Although Internet-delivered cognitive behaviour therapy (ICBT) for alcohol misuse is efficacious in research trials, it is not routinely available in practice. Moreover, there is considerable variability in engagement and outcomes of ICBT for alcohol misuse across studies. The Alcohol Change Course (ACC) is an ICBT program that is offered free of charge by an online clinic in Saskatchewan, Canada, which seeks to fill this service gap, while also conducting research to direct future improvements of ICBT. As there is limited qualitative patient-oriented research designed to improve ICBT for alcohol misuse, in this study, we describe patient perceptions of the ACC post-treatment. Specifically, post-treatment feedback was obtained from 191 of 312 patients who enrolled in the ACC. Qualitative thematic analysis was used to examine post-treatment written comments related to what patients liked and disliked about the course, which skills were most helpful for them, and their suggestions for future patients. The majority of patients endorsed being very satisfied or satisfied with the course (n = 133, 69.6%) and 94.2% (n = 180) perceived the course as being worth their time. Worksheets (n = 61, 31.9%) and reflections of others (n = 40, 20.9%) received the most praise. Coping with cravings (n = 63, 33.0%), and identifying and managing risky situations (n = 46, 24.1%) were reported as the most helpful skills. Several suggestions for refining the course were provided with the most frequent recommendation being a desire for increased personal interaction (n = 24, 12.6%) followed by a desire for wanting more information (n = 22, 11.5%). Many patients offered advice for future ACC patients, including suggestions to make a commitment (n = 47, 24.6%), do all of the work (n = 29, 15.2%), and keep a consistent approach to the course (n = 24, 12.6%). The results provide valuable patient-oriented directions for improving ICBT for alcohol misuse.

Highlights

• •Internet-delivered Cognitive Behaviour Therapy (ICBT) for alcohol misuse is efficacious in research trials.
• •There is a significant gap between those in need of treatment for alcohol misuse and those who receive it.
• •Consideration of patient preferences on ICBT for alcohol misuse may improve uptake of these programs.
• •Patients provide insight on which components of ICBT for alcohol misuse are most valuable and which require improvement.

1. Introduction

Alcohol misuse, also known as harmful or problem drinking, refers to a spectrum of difficulties ranging from individuals who consume more than public health guidelines to those who meet diagnostic criteria for an alcohol use disorder (Moyer, 2013; Riper et al., 2014). Globally, alcohol misuse accounts for more than 5% of the burden of injury and disease (Connor et al., 2016; World Health Organization, 2018). Despite the magnitude of the problem, alcohol misuse is frequently untreated (Schmidt, 2015). It is estimated that 80% of those with problematic alcohol misuse do not seek treatment (Kohn et al., 2004), with women less likely to receive treatment than men (McCrady et al., 2020). There are various reasons for this treatment gap, including not seeking treatment due to embarrassment, time constraints, and rural and remote locations where access to treatment is not feasible (Probst et al., 2015; Saunders et al., 2006; Schomerus et al., 2014). In recent years, internet-delivered cognitive behavioural therapy (ICBT) has emerged as an effective alternative to face-to-face therapy that overcomes barriers of traditional treatment (Andersson et al., 2019). As alcohol misuse is one of the most stigmatized psychiatric conditions (Kilian et al., 2021), the anonymous nature of ICBT may be particularly suited for this population.

Several ICBT programs have been developed to treat alcohol misuse, with substantial evidence of their effectiveness (Hadjistavropoulos et al., 2020c). These programs commonly incorporate a relapse prevention model that helps patients identify high-risk situations, develop cognitive and behavioural coping skills, and manage lapses (Larimer et al., 1999). In ICBT, patients read online treatment materials and complete relevant exercises. The treatment can either be self-directed or offered with the guidance of a provider via weekly emails and/or phone calls (Hadjistavropoulos et al., 2020c). Research thus far suggests the guidance of a provider is predictive of greater alcohol reductions (Riper et al., 2018).

Although evidence related to ICBT for alcohol misuse is promising, attrition is a concern and outcomes vary across studies (Hadjistavropoulos et al., 2020c). Moreover, in research trials, there is also significant variability observed among patients who receive ICBT, with not all patients showing benefit (e.g., Sinadinovic et al., 2014; Sundström et al., 2020a). A further issue is that ICBT for alcohol misuse has not yet been broadly implemented in clinical settings (Kiluk et al., 2018). Qualitative patient-oriented research has the potential to inform improvements to ICBT for alcohol misuse, which could enhance engagement and outcomes in clinical settings by identifying patient perspectives related to what is most valued and how to improve care (Brett et al., 2014). Previous research has shown that integrating patient feedback into programs can improve the quality of care by ensuring programs are designed so that they are acceptable to patients, and therefore, are more likely to be used (Lyle et al., 2017). Qualitative research designed to better understand patients' experiences with ICBT has been conducted for other conditions, such as depression and anxiety, identifying what patients like (e.g., format and flexibility, therapist support, and CBT techniques) and dislike (e.g., difficulty navigating websites, frustrations with therapist feedback) about ICBT (Earley et al., 2017; Hadjistavropoulos et al., 2018; Richards et al., 2016). A recent study of ICBT found that common suggestions for improvements provided by patients included changing the timeline of the course (e.g., allowing for flexibility), matching therapist availability to patient needs, and improving case stories (Hadjistavropoulos et al., 2018). In a meta-synthesis of common themes in user experiences of eight qualitative studies involving ICBT programs for depression and anxiety disorders, improving feelings of connection and collaboration through enhanced personalization of content to the individual user was found to be beneficial (Knowles et al., 2014).

Specific to alcohol misuse, one study gathered qualitative feedback from patients who enrolled, but did not complete an ICBT program for problem drinking (Postel et al., 2011). Patients provided suggestions for improvement of the program, including frequent email notifications of incoming messages or assignments from their therapist, increased flexibility in the treatment protocol (e.g., having the possibility to skip lessons or start immediately on a new assignment without completing the prior), increased contact with their therapist and other patients, and general suggestions regarding the program layout (Postel et al., 2011). In another qualitative study undertaken after we began our research described below, Ekström and Johansson (2020) interviewed 38 of 1169 former users of an ICBT program in part to understand what components of the treatment program contributed to changes in alcohol use. Significant variability was observed in what patients found helpful, with program content, therapist support and discussion forums described as helpful by some, but not all, patients. A limitation of this latter study is that feedback was sought more than 2 years after individuals participated in the study, and many participants were not able to recall the nature of the program. Overall, while the foregoing information is valuable, the literature examining ICBT for alcohol misuse remains sparse.

The current study examines patient perceptions of the Alcohol Change Course (ACC), an ICBT program for alcohol misuse offered within an online therapy clinic that delivers ICBT on a routine basis at no cost. The overarching goal was to understand patient experiences and preferences related to ICBT for alcohol misuse, as patient preferences have been found to impact intervention engagement and outcomes (Preference Collaborative Review Group, 2008) and there is growing recognition that patients can make important contributions to the design and delivery of health services (Bradshaw, 2008). The study used a post-treatment survey, with both closed- and open-ended questions, to gather salient information about patient preferences. Patients were asked questions about treatment satisfaction to gauge a general understanding of patient satisfaction with the intervention. To understand what should be retained and what potentially could be improved, patients were asked what was liked and disliked about the ACC. Patients were also asked about which skills they most appreciated learning in order to identify their perspective on the most helpful behaviour change techniques (Michie et al., 2012). In addition, patients were asked about what practical advice they would offer to others starting the ACC that might help prepare them better for ICBT. The latter question was formulated to generate patient-oriented treatment advice that has the potential to make the intervention more persuasive, which is consistent with past research showing messages are more persuasive when coming from individuals who are perceived to have relevant expertise (Wilson and Sherrell, 1993). Overall, findings from this study provide insight into what existing components of the ACC were viewed as helpful and should be maintained from the patient perspective, while also identifying opportunities to improve ICBT for alcohol misuse by incorporating patient preferences into the delivery of such programs. Given the qualitative nature of the study, no hypotheses were formulated in advance.

2. Method

2.1. Design and setting

The Online Therapy Unit (OTU), based at the University of Regina, is funded by the Government of Saskatchewan to deliver ICBT to patients, including funding to develop, implement, and evaluate the ACC. Additional grant funding was secured to form a Patient-Oriented Research Steering Committee (PORSC) to further support the development, evaluation, and improvement of the ACC, with specific attention to partnering with patients, providers, and managers in this process. The PORSC consisted of the first author, four patient partners (individuals who reported personal, family, or work-related experience with alcohol misuse), two guides, two healthcare managers (one from primary care and one from the online unit), two trainees, and two facilitators. Two patient partners have a long-standing involvement with the OTU while the others were recommended for the committee given their known interest in mental health and addictions. This committee collaborated on the development of the ACC, advertisements, trial design, selection of outcome measures, interpretation of patient findings, and manuscript preparation (a video regarding the team is available at https://www.scpor.ca/videos-1). The PORSC held a total of 13 meetings between March 2019 and April 2021, prior to the preparation of this paper.

The committee's work resulted in the ACC (described below), which was piloted and then offered on an ongoing basis beginning June 26, 2019 when the OTU initiated a randomized factorial trial. In this trial, patients with alcohol misuse were randomized to: 1) structured diagnostic interview or no diagnostic interview; and 2) health educator guidance or no guidance. See published protocol for details (Sundström et al., 2020b). Clinical Trial Identifier: NCT03984786. The current study focuses on feedback from all patients enrolled in the trial who completed post-treatment questionnaires regardless of condition. See Fig. 1. Outcomes of the trial will be reported at a later date.

Fig. 1.

ACC flowchart qualitative.

2.2. Treatment materials

Content for the ACC was initially drawn from an ICBT program for alcohol misuse developed by a team at the Swiss Research Institute of Public Health and Addiction and made available by Michael Schaub (see Baumgartner et al., 2021). The program was then translated into English under the supervision of Matthew Keough at the University of Manitoba for use with young adults (Frohlich et al., 2018). This particular program was chosen because review of the content by our team showed it covered topics of interest, the program was available in English, at no cost, and with permission to adapt to our context. The program was adapted by the PORSC to suit an adult population, and information about drinking guidelines in Canada, abstinence, and the effects of alcohol on the body was added to the first lesson of the program. Further, worksheets with relevant quizzes and exercises were created to accompany each lesson (Fig. 2). Each lesson included brief reflections from six patients with diverse backgrounds in terms of age, gender, and vocations (Fig. 3). The final version of the course consisted of 12 online lessons that were released gradually over 8 weeks (see Table 1).

Fig. 2.

Worksheet example.

Fig. 3.

Reflections of others example.

Table 1.

The alcohol change course.

Week	Module/lesson title	Lesson summary
Week 1	Lesson 1 – Introduction to the alcohol change course	Drinking guidelines in Canada, abstinence, and adverse effects of alcohol on the body. Examples of benefits of changing drinking Recommendation to set a drinking goal
	Lesson 2 – Strategies for meeting your goals	Discovering your resources Strategies for changing drinking habits Common excuses for not changing drinking habits Creating a personal strategies list
Week 2	Lesson 3 – Identifying risk situations	Identifying specific risky situations Understanding “seemingly unimportant situations”
	Lesson 4 – Say yes to positive activities	The importance of engaging in positive activities while changing drinking habits How to overcome barriers to carrying out activity plans
Week 3	Lesson 5 – Learning to say no to alcohol	Preparing to say no to alcohol Methods of resisting social pressures to drink
	Lesson 6 – Coping with cravings	Defining cravings Identifying personal triggers Strategies to deal with cravings
Week 4	Lesson 7 – Problem solving	Learning how everyday problems, stress, and alcohol are related 5-step problem solving plan Solutions to problems that are hard to solve
Week 5	Lesson 8 – Challenge your thought patterns	How negative emotions and alcohol uses are related The importance of identifying automatic thoughts Thinking traps and thought challenging
Week 6	Lesson 9 – Meeting your needs	Specific information regarding sleep and its' importance Dealing with excessive worrying The importance of having a social network
	Lesson 10 – Progressive Muscle Relaxation	Discover a relaxation exercise that is helpful when experiencing cravings
Week 7	Lesson 11 – Dealing with slips and relapses	Explanation of slips and relapses How to deal with risky life events where slips and relapse are more likely to occur
Week 8	Lesson 12 – Preserve your success	Reflection on the course Identifying a future alcohol goal Identifying the specific skills that were most helpful

2.3. Patients

To be eligible for the course, patients had to: 1) be a Canadian resident; 2) be over the age of 18; 3) have access to a computer and the internet; 4) have a score of 8 or more on the Alcohol Use Disorder Identification Test [AUDIT (Saunders et al., 1993)]; 5) report having consumed 14 or more drinks in the past week; and 6) not endorse severe psychiatric illness (e.g., psychosis), high risk of suicide, or severe problems with drugs as assessed using a combination of online questionnaires and a telephone interview. Online questionnaires used to initially identify patients to be excluded from the trial included scoring >24 on the Patient Health Questionnaire-9 (PHQ-9; Kroenke et al., 2001), endorsing suicidal thoughts nearly every day on the PHQ-9 (Kroenke et al., 2001), and scoring >24 on the Drug Use Disorder Identification Test (Berman et al., 2005). Individuals were further assessed for severe psychiatric illness, high risk of suicide or severe problems with drugs in the telephone interview and referred to more appropriate services as needed (e.g., local face-to-face mental health and addiction services, family physician). Individuals reporting minimal difficulties were offered self-directed ICBT.

2.4. Procedure

Recruitment took place from June 2019 to November 2020 after receiving approval from the University of Regina Research Ethics Board. We advertised the availability of the ACC through Google Ads and posters sent to physicians and Saskatchewan Liquor and Gaming locations throughout the province. All interested individuals were directed to the OTU website (www.onlinetherapyuser.ca). On the study website, patients began by completing an online screening questionnaire where they were presented with a consent form explaining the screening protocol. After consent was given, patients were assessed for eligibility. Following the online screen, a brief telephone call was conducted to further assess eligibility for the ACC, and patients were randomly assigned to their treatment condition (structured interview or not; guidance or not). After being admitted to the study, patients could then log in to the ACC website and begin the course. After the 8-week treatment period, all patients were asked to respond to a post-treatment questionnaire, regardless of how much of the course material they had accessed.

2.5. Measures

Sociodemographic data was collected from patients as part of the screening process. Information collected included age, gender, marital status, education level, employment status, ethnicity, residence location, years with alcohol misuse, background with treatments, and baseline AUDIT (Saunders et al., 1993) scores.

2.5.1. Treatment satisfaction

Following past research on ICBT (Dear et al., 2011, Hadjistavropoulos et al., 2020b, Hadjistavropoulos et al., 2020a), patients responded to the following questions using a 1 (“very dissatisfied”) to 5 (“very satisfied”) scale: (1) “Overall, how satisfied were you with treatment?”; and (2) “How satisfied were you with the quality of the Lessons and Worksheets?” Patients were also asked if the treatment had been worth their time (Yes/No) and if they would recommend the course to a friend (Yes/No). Additionally, patients were asked how the course affected their confidence to manage symptoms (1 “greatly reduced” to 5 “greatly increased”) and their motivation to seek treatment in the future (1 “greatly reduced” to 5 “greatly increased”).

2.5.2. Open-ended questions

Patients responded to four open-ended questions: (a) What they liked about the course; (b) what they did not like about the course and felt should be improved; (c) what skill they found most helpful; and (d) what advice they would offer to someone beginning the ACC. The first two questions have been used in past research to assist with understanding patient preferences and generate ideas for program improvement (Hadjistavropoulos et al., 2018). The latter questions were asked to understand patient perspectives on helpful behaviour change techniques (Michie et al., 2012) and patient-oriented treatment advice, which has potential to improve the persuasiveness of the intervention (Wilson and Sherrell, 1993).

2.6. Analysis

Quantitative data analysis was completed using SPSS version 25. Descriptive analyses were conducted to investigate whether patients who completed post-treatment questionnaires (“completers”) differed from those who did not (“non-completers”) on demographic and pre-treatment variables, with t-tests and chi-square tests used to assess any potential bias in the sample. Descriptive analyses were also used to examine responses to rating scales to understand overall perceptions of the ACC and to contextualize the qualitative analysis.

A conventional qualitative content analysis approach was used to examine open-ended responses, as this approach is considered the most appropriate approach when existing research literature is limited (Hseigh and Shannon, 2005). Responses to the open-ended questions were on average 15.19 words (SD = 4.07). The qualitative data analysis program NVivo 12 was used and qualitative methods of coding were applied (Creswell and Clark, 2011) by three employees of the OTU, one with a BA in psychology (co-author C.C. associated with the Unit for two years), and two with MA level training in psychology (researcher S.A.T and co-author C.D·B, both associated with the Unit for four years). The original analysis followed these three steps:

• 1.)First, after one year of offering the ACC, we exported data and analyzed post-treatment responses from the first 90 patients to complete the program. This initial coding was conducted by researchers C.C. and S.A.T., who read patient responses closely and independently divided them into categories derived directly from text.
• 2.)Second, researchers C.C. and S.A.T met to discuss initial impressions. They then created a coding guide of categories with code descriptions. Responses were then classified according to the coding guide.
• 3.)Third, an expert coder and co-author (H.D.H.) approached the data to confirm categories and resolve any coding inconsistencies between coders.

This process was repeated with minor modifications to the coding guide when additional patients were included in the sample. A fourth and fifth step was added to the process:

• 4.)We hired an expert in qualitative analysis to review our work (J.W.) and identify opportunities to improve the descriptions of themes.
• 5.)Descriptions of themes were polished and minor adjustments to coding were made. Comments that could not be coded because the response was unclear were identified as “non-coded” responses.

The PI and researchers then met with the PORSC for two meetings to discuss this analysis so as to ensure clarity and consensus on interpretations of the data.

3. Results

3.1. Background characteristics

We examined feedback from the 191 of 312 patients who completed post-treatment questionnaires (61.2% participation rate), comparing background characteristics (i.e. demographic and clinical variables) between those who did and did not complete the post-treatment questionnaires (see Table 2). Completers were somewhat more likely to be female, X²(1, N=309) = 4.06, p = 0.04, and were also significantly older than non-completers, t(309) = 4.081, p < 0.01. Completers and non-completers were not significantly different in any other background characteristic.

Table 2.

Pre-treatment patient characteristics by group.

				Completers (n = 191)		Non-Completers (n = 121)
		n	%	n	%	n	%
Age mean (SD)		46.1 (11.2)	–	48.09 (11.4)	–	42.9 (10.1)
Gender*
	Male	106	34.0	57	29.8	49	40.5
	Female	203	65.1	133	69.6	70	57.9
	Non-binary	2	0.6	1	0.5	1	0.8
	Prefer not to disclose	1	0.3	0	0.0	1	0.8
Marital status
	Single/never married	46	14.7	27	14.1	19	15.7
	Married/common-law	198	63.5	126	66.0	72	59.5
	Separated/divorced/widowed	68	21.8	38	19.9	30	24.8
Education
	Less than high school	9	2.9	4	2.1	5	4.1
	High school diploma	37	11.9	26	13.6	11	9.1
	Post high school certificate/diploma	128	410	73	38.2	55	45.5
	University education	138	44.2	88	46.1	50	41.3
Employment status
	Employed part-time/full-time	220	70.5	135	70.7	85	70.2
	Unemployed	21	6.7	10	5.2	11	9.1
	Homemaker	25	8.0	13	6.8	12	9.9
	Student	3	1.0	1	0.5	2	1.7
	Disability	16	5.1	10	5.2	6	5.0
	Retired	27	8.7	22	11.5	5	4.1
Ethnicity
	White/Caucasian	288	92.3	180	94.2	108	89.3
	Spanish/Hispanic/Latin	2	0.6	2	1.0	0	0.0
	Indigenous	9	2.9	4	2.1	5	4.1
	Other	13	4.2	5	26	8	6.6
Location
	Large city (over 200,000)	143	45.8	90	47.1	53	43.8
	Small to medium city	85	27.2	51	26.7	34	28.1
	Small rural location (under 10,000)	82	26.3	48	25.2	34	28.1
	Reserve	2	0.6	2	1.0	0	0.0
Years with alcohol problems
	Less than one year	11	3.5	7	3.7	4	3.3
	1–2 years	28	9.0	19	9.9	9	7.4
	3–5 years	74	23.7	43	22.5	31	25.6
	6–10 years	52	16.7	29	15.2	23	19.0
	More than 10 years	146	47.1	93	48.7	54	44.6
Previously received treatment
	Yes	101	32.4	64	33.5	37	30.6
	No	211	67.6	127	66.5	84	69.4
AUDIT mean (SD)
		24.3 (5.8)	–	23.9 (5.9)	–	25.0 (5.7)	–

Note. AUDIT = Alcohol Use Disorders Identification Test.

3.2. Treatment satisfaction

The majority of patients endorsed being very satisfied or satisfied with the course (n = 133, 69.6%), with very few reporting being dissatisfied or very dissatisfied (n = 9, 4.7%), and the remainder reporting neutral satisfaction (n = 49, 25.7%). The vast majority of patients perceived the course as being worth their time (n = 180, 94.2%) and most patients reported that the course increased or greatly increased their confidence in their ability to manage their symptoms (n = 156, 81.7%). See Table 3.

Table 3.

Treatment satisfaction as rated by patients who completed post-treatment measures (n = 191).

Treatment ratings	n	%
Satisfied/very satisfied overall	133	69.6
Satisfied/very satisfied with materials	154	80.6
Increased/greatly increased confidence	156	81.7
Increased/greatly increased motivation for other treatment	149	78.0
Course was worth the time (%)	180	94.2
Would recommend the course to a friend (%)	179	93.7

3.3. Most liked course aspects

Patients reported enjoying several course components, with worksheets (n = 61, 31.9%) and reflections of others (n = 40, 20.9%) receiving the most praise. Of note, 6.8% (n = 13) reported enjoying everything, however, two patients reported not liking anything in the course. One of these patients reported in response to another question that the material simply was not relevant to them, while the other patient did not provide further explanation, answering “Non-applicable” to most prompts. See Table 4.

Table 4.

Patient response to question: what did you like most about the alcohol change course?

Most Liked	Example	Patient ID	n	%
Content			140	73.3
Worksheets	“Worksheets were very good for organizing thoughts and giving true value to attempts to control alcohol.”	20321	61	31.9
-Worksheets were action-oriented and thought-provoking	“I like being able to think and work out what I would actually do.”	20238	11	5.8
- Worksheets provided the opportunity to review	“I like being able to go back and work through the process as needed.”	20161	7	3.7
Reflections of others	“I found the personal reflections very helpful.”	20179	40	20.9
Lessons overall	“The lessons progressively helped me learn new ways to think positively.”	21164	26	13.6
Improved my self-awareness	“It made me really think about myself and why I drink.”	20174	13	6.8
Relatable and comprehensive	“I liked the pragmatic approach – very tactical but content was easy to understand and felt it was tailored to the realities of drinking and issues involving alcohol.”	20885	8	4.2
Alcohol education	“I didn't know 1 bottle of wine was 6 servings. What I am learning is that everything we learn is one more tool for our toolkit.”	20845	3	1.6
Learning about cravings (L6) and coping in risky situations (L3)	“Learning that cravings are a normal part of the process, and about seemingly unimportant decisions.”	20569	3	1.6
Identifying positive activities (L4)	“Pleasurable activities.”	20585	3	1.6
Tracking consumption	“Having to be accountable every week having to say how many drinks I had.”	20675	3	1.6
Goal setting (L2)	“Setting goals.”	20920	3	1.6
Format			63	33.0
Online aspect	“I could work on it at my leisure instead of adding yet another busy meeting to my schedule.”	20355	21	11.0
- Online aspect allows course to be completed on my own time	“Flexible time to study.”	20235	13	6.8
- Online aspect allows for privacy	“I liked that I could do it privately at home.”	20095	5	2.6
Provider contact	“I really enjoyed emailing with Lee. It really made a difference having someone check in on you weekly.”	21043	22	11.5
Module design	“Information was well laid out in each course.”	20240	15	7.9
Time-span	“I liked that it was several weeks, forced you to look at the issue over a long period of time.”	20383	6	3.1
Non-judgmental and hopeful tone	“The material is respectful of the patient.”	20363	9	4.7
Everything	“Everything. Thank you very much, I was happy with the course.”	20958	13	6.8
Non-coded	“Read above.”	20764	9	4.7
Have not yet completed the course	“I haven't completed all the lessons at this time. Therefore, I cannot respond.”	21031	7	3.7
Nothing	“Nothing useful for me.”	21073	2	1.0

3.4. Disliked course aspects

The umbrella category, format, was identified as a primary issue (n = 91, 47.6%), with patients most commonly reporting a desire for increased personal interaction (n = 24, 12.6%) and reporting that the design of the course could be improved to allow for increased ease of use (n = 12, 6.3%). Content of the program represented the second broad category issue (n = 52, 27.2%), with patients most often reporting wanting more information (n = 22, 11.5%), or finding that the content was not related to them (n = 14, 7.3%). Many patients opted not to provide suggestions for improvement (n = 54, 28.3%) or reported enjoying the course in its entirety (n = 25, 13.1%). See Table 5 for patient perspectives on least liked aspects of the course and also subsequent changes made to the ACC in response to patient feedback.

Table 5.

Patient report of “least-liked component of the ACC” and response to feedback.

Least liked	Example	Patient ID	n	%	Online therapy unit response to feedback
Format			91	47.6
Limited personal interaction or follow-up	“Lack of interaction with someone while working through the course. I did get a call that I could contact the therapy group if needed, but maybe at least one call with someone similar to the intake”	21020	24	12.6	Guidance is now offered once weekly to all patients receiving ACC who reside in Saskatchewan
Design of course	“A more user friendly program/dashboard would be helpful. It would've been great if it was clear where you left off in the program. That wasn't clear to me and I spent a lot of time re-doing lessons.”	20763	12	6.3	Improved formatting to enhance readability and instruction
Surveys	“The questions where you are given the option of never – several days a week etc. I think the responses could be more exact. Several seem too broad.”	20459	12	6.3	Surveys are standardized measures that cannot be changed
Technical problems and general errors	“It would be good to get weekly reminders about the course over email.”	20799	10	5.2	Review was conducted and errors were corrected with newest version of our website
Course requires motivation and accountability	“I'm not sure if you aren't determined to work on it, that it would be much benefit.”	20095	8	4.2	Guidance is now offered to all patients in Saskatchewan.
Limited time	“More time than a week between lessons.”	20151	8	4.2	Time changes not feasible due to funding, however, patients can continue with the course on their own
Repetitive or too long	“Some of the lessons and the worksheets very much overlapped. I get that they should, but it felt in some cases I did not need to read the lesson to fill in the sheet.”	20238	6	3.1	Adjustments made so each lesson is comparable in length
Wanted a printable workbook	“Perhaps a practical workbook that you could keep after, that includes lessons etc. that you can customize to keep with you and carry around.”	20162	6	3.1	Lessons have been made printable
Worksheets	“Make the worksheets more engaging and require submittal.”	21134	5	2.6
No suggestions	“There is nothing that I know of”	20226	54	28.3	N/A
Content			52	27.2
Wanted more information	“Perhaps provide additional resources to strengthen such as books or online support groups.”	21215	22	11.5	Addressed comments regarding using alcohol as a coping mechanism to deal with anxiety and chronic pain.
Content not related to me	“For me I have never had a problem with social settings or social pressure. I always drink alone so that part was not helpful to me.”	20421	14	7.3	Course description refined so patients know what to expect upon enrolment
Reflections of others not useful	“I did not find the stories very useful.”	20190	9	4.7	Reflections have been revised to enhance relatability
Wanted more engaging features such as video or audio	“Audio or video presentations for the lessons would increase the ease at which they are done as well as the sense of interaction and therefor increase interest and persistence.”	21087	7	3.7
Satisfied with the course	“Nothing, everything was great.”	20958	25	13.1	N/A

Further analysis of the 24 individuals who felt there was not enough personal interaction in the ACC was undertaken. Feedback on insufficient personal interaction came from 45.8% (n = 11) of patients who were randomized to receive health educator guidance, while the remaining feedback came from those who were assigned to the self-directed condition. Of the 11 patients who received health educator guidance, three felt their interactions with their health educator were not frequent enough, three desired correspondence with the health educator to continue after treatment was complete, two specifically desired video sessions, two found it difficult conversing over email rather than in “real-time”, and one stressed the importance of having additional support outside of the course. The remaining patients, who were randomized to the self-directed condition, reported a general desire for more personal contact and two specifically requested an online chat to converse with providers and other patients.

3.5. Most valuable skill taught

Coping with cravings (n = 63, 33.0%) and identifying and managing risky situations (n = 46, 24.1%) were identified as the most helpful skills. The same two participants (1.0%) who had reported not liking the course provided an answer of “none” or “not applicable.” Table 6 summarizes what patients reported as the most valuable skills taught.

Table 6.

Patient response to question: what skill did you find most helpful in the ACC?

Skill	Example	Patient ID	n	%
Coping with cravings (L6)	“Anticipating the cravings and having good strategies to manage them.”	21090	63	33.0
Identifying and managing risky situations (L3)	“Identifying risk situations and planning ahead to avoid those.”	21190	46	24.1
Goal setting (L2)	“Making realistic goals for myself.”	21208	33	17.3
Challenging thoughts (L8)	“Recognizing and dealing with automatic thoughts is very important.”	20459	31	16.2
Improved self-awareness and insight into problem	“Awareness. Simply being confronted with your situation.”	21167	18	9.4
Identifying reasons to make a change and weighing the pros & cons (L1)	“Reflecting on the effects of alcohol, both good and bad, and weighing the difference.”	20376	16	8.4
Problem solving (L7)	“Problem solving, through the thought process of this lesson I learnt to identify my real problems.”	21144	14	7.3
Identifying positive activities that do not involve alcohol (L4)	“Planning to integrate positive things into my life.”	21063	10	5.2
Managing slips (L11)	“Learning how slips are part of the process and not indicative of failure.”	20569	8	4.2
Progressive muscle relaxation (L10)	“Progressive relaxation.”	20153	8	4.2
Overall positive reflection	“Actually, all skills in this course have value.”	20152	5	2.6
Saying no to alcohol (L5)	“How to decline an offer of alcohol.”	20985	4	2.1
Accountability	“Making me be honest and accountable”	20573	3	1.6
Reflections of others	“Stories of others.”	20157	2	1.0
Non-coded	“.”	20775	11	5.8
Have not yet completed the course	“I did not complete the lessons.”	21156	8	4.2
None	“None”	21073	2	1.0

3.6. Patient advice

Table 7 provides a summary of what patients would recommend to future patients who take the ACC. The most common recommendation was to “Set yourself up for success through your actions” (n = 116, 60.7%), which consisted of subcategories such as make a commitment (n = 47, 24.6%), to do all of the work (n = 29, 15.2%), and keep a consistent approach to the program (n = 24, 12.6%).

Table 7.

Patient response to question: what advice would you offer to someone who is starting the course?

Advice	Example	Patient ID	n	%
Set yourself up for success through your actions			116	60.7
Make a commitment	“They need to be committed to changing their lives and focus on the information provided to achieve success.”	20240	47	24.6
Do the work	“Read everything and do the worksheets!”	20675	29	15.2
Keep a consistent approach to the program	“Have a dedicated time and space to complete it.”	20258	24	12.6
Review closely and work on tasks often	“Keep going back to the lessons and worksheets. I think reviewing the lessons and work sheets helps to control impulsive drinking.”	20129	10	5.2
Get organized	“Print out the worksheets! I did mine on the computer and they are not handy enough for quick review.”	20569	8	4.2
Make specific goals	“Make sure you start with a specific goal in mind, clarity on what you want to get out of this course will be key.”	20547	4	2.1
Set yourself up for success through your attitude			40	20.9
Be honest, truthful, and engage in self-reflection	“Be honest with yourself and you will get the most value.”	21164	21	11.0
Know that the course is valuable	“Give it a chance. Where I am now versus where I was the day I started this course are two different people. There is so much value to be found in each lesson.”	20588	18	9.4
Don't give up	“Try their best and don't get mad if you have a slip up.”	20260	7	3.7
Maintain a positive mindset	“I think you need to be in a positive headspace.”	20060	4	2.1
Set yourself up for success through getting support			17	8.9
Build a network of support	“It is helpful to work through the material with a partner. It helps to discuss the material to better internalize the concepts.”	20363	9	4.7
Communicate with your provider and reach out for support	“Take advantage of the opportunity for accountability and support.”	20247	8	4.2
Set yourself up for success through how you approach alcohol			15	7.9
Consider abstaining from alcohol through the duration of this course	“Commit to not drinking at all during the whole course to get the full benefit of the information and the opportunity to properly put the lessons into practice in real life.”	20202	5	2.6
Keep alcohol out of the home	“Removing alcohol from my home was the most helpful. Without it being here it was easier to not worry about having a drink as there was none to be had.”	21151	3	1.6
Track your alcohol consumption	“I created a consumption log which helped me a lot.”	20603	3	1.6
Know the ACC may not be for you if…			15	7.9
It is a bad time to start	“COVID greatly affected me and not being able to get as much as I had hoped out of it. Just terrible timing.”	20491	6	3.1
You want more support	“This is a self-help course with very little support along the lessons. Be prepared to self-manage, if you are unable to, this course is difficult to complete.”	21031	5	2.6
You have high severity of symptoms or alcohol use	“If you are a heavy long time user this would not be the appropriate course, but it would help those who are just entering a dependent stage.”	20861	4	2.1
Non-coded	“None. Started taking naltrexone which stopped my cravings.”	20088	16	8.4
I don't know or I am unwilling to offer advice	“I couldn't, sorry.”	20465	14	7.3

4. Discussion

This study examined patient perceptions of an ICBT program for alcohol misuse (ACC). The goal was to better understand what was most liked and disliked about ICBT, what skills were most valued, and generate patient-oriented treatment advice. The overall aim was to identify features of the course to be maintained and improved. Overall, patient satisfaction was high, with 94.2% (n = 180) of patients reporting they felt the course was worth their time, and a majority of patients endorsing being very satisfied or satisfied with the course (n = 133, 69.6%). Most patients reported they felt the ACC increased or greatly increased confidence in their ability to manage their symptoms (n = 156, 81.7%). These same questions have been used to assess satisfaction with the Wellbeing Course, a well-established ICBT course for depression and anxiety offered in the same clinic (e.g., Hadjistavropoulos et al., 2020a). Ratings of the ACC being worth the patients' time compared favourably to the Wellbeing Course (96.5%, n = 466), but ratings of satisfaction with materials (82.4%, n = 402) and confidence in ability to manage symptoms (90%, n = 430) were lower for the ACC compared to the Wellbeing Course. Overall, the ratings suggest there is room to improve the program beyond current levels. Qualitative content analysis of patient feedback revealed several important themes that provide direction for future development of the ACC and ICBT programs for alcohol misuse in general. The aspects of the course that patients most frequently reported liking were the worksheets (see Fig. 2) and personal reflections of others (see Fig. 3). Patients described the worksheets as action-oriented (i.e., helping to guide behaviour), as a helpful resource to refer to later, and as helpful for organizing thoughts related to alcohol. Personal reflections were described as thought-provoking. The finding that patients described their most-liked aspects of the course in terms of the changes in their behaviour and thinking is consistent with findings from an earlier qualitative study of patients who received face-to-face interventions for alcohol misuse, which found that patients who had completed treatment described activities that led to thinking and acting differently as contributing the most to change in drinking behaviours (Orford et al., 2006). Of interest, while patients most frequently reported liking these aspects of the course, some patients identified that these same aspects required improvement (i.e., some patients reported finding worksheets overlapped with other content, could be more engaging, and identified that reflections were not relatable). In general, the results highlight that even the most valued aspects of a program require refinement to ensure they do not interfere with patient engagement and outcomes.

Similar to previous qualitative research on ICBT for other disorders (Earley et al., 2017; Hadjistavropoulos et al., 2018; Richards et al., 2016) perceptions of provider guidance were prominent in patient responses. On the one hand, 11.5% of patients indicated this was the aspect of the course they liked the most, and on the other hand, 12.6% of patients identified that increased provider contact could improve the course. Of note, this dislike of limited interaction was highlighted both by those randomized to guided-ICBT and those who were randomized to self-guided ICBT. Patients had divergent ideas as to how guidance could be improved, ranging from increasing contact, extending guidance beyond 8 weeks, to changing the method of guidance (e.g., video, chat). Overall, the suggestion for improving ICBT by offering greater guidance aligns with a qualitative meta-synthesis of user experience of computerised therapy for depression and anxiety (Knowles et al., 2014). Previous studies of patient feedback from in-person alcohol interventions have also identified social support as something patients felt was important for their recovery (Orford et al., 2006). When combined with existing research showing that guidance from a provider is related to better patient outcomes (Riper et al., 2018), findings of the current study further builds the case for consistently offering guidance to patients who engage in ICBT for alcohol misuse. Nevertheless, it must be acknowledged that the research literature is not fully consistent on the benefits of guidance, as some recent studies have found no significant differences between guided and self-directed programs (Sundström et al., 2020a; Johansson et al., 2021). Ultimately, this puts program developers/clinics in a challenging position – balancing client preferences with mixed literature. One potential option that has not been systematically studied with alcohol misuse is the use of optional provider guidance, whereby guidance is only offered if patients request it. This would serve to meet the needs of patients who desire guidance and would also allow patients who prefer to work independently on ICBT this option as well. Past research on optional therapist support provided to patients enrolled in ICBT for depression and anxiety showed that outcomes of optional therapist support are similar to when offering once-weekly therapist support (Hadjistavropoulos et al., 2017).

A second area where mixed feedback was obtained from patients related to the content of the course. Overall, 73.3% of patients generated a positive comment about the content of the course. Nevertheless, 27.2% of patients also generated comments about how the content of the course could be improved. In accordance with previous qualitative research on ICBT for other disorders (Hadjistavropoulos et al., 2018), 11.5% of patients wanted more in-depth information with suggestions to incorporate additional resources and readings. Finding the content difficult to relate to was an additional concern raised by 7.3% of patients. In particular, three of these patients felt the content was geared towards a younger population managing drinking in social settings, rather than towards those who have a long-standing problematic relationship with alcohol who tend to drink at home alone. In general, like the point made above, even aspects of a program that are most valued have potential to be refined to facilitate patient engagement and outcomes.

On the note of improvements, Table 5 summarizes how patient feedback from this study was used to improve the ACC format (e.g., merged 12 lessons into 8 lessons each with similar length, improved stories), content (e.g., increased information on anxiety and pain as they relate to alcohol misuse) and delivery (e.g., optional guidance is now consistently offered to patients in Saskatchewan; weekly automated reminders, printable materials). Only two areas remain largely unchanged; not changing standardized questionnaires (so we can continue to use psychometrically sound measures), and not changing the timeline for the course (given current resources available in the unit).

As for the most helpful skills for managing alcohol problems, one-third of patients identified coping with cravings as the most helpful skill, while a quarter of patients described managing risky situations (n = 46, 24.1%) as the most helpful skill. The feedback from patients that behavioural strategies are most helpful for them is consistent with findings from a recent systematic review that implicate changes in coping skills as a key causal mechanism of cognitive behaviour therapy for alcohol use disorders (Magill et al., 2020). Many of the most valuable skills identified by patients in our study are those commonly included in ICBT for alcohol misuse (Hadjistavropoulos et al., 2020c). All the skills identified by patients as helpful were aligned with content from the ACC, except for two general areas. Some patients reported that the most helpful skill they learned was an improved sense of self-awareness and insight into the problem they face, while others described the most helpful skill as an improved sense of accountability over their drinking. The feedback provided by patients about the most helpful skills and most/least liked components provided in this study has implications for clinicians and researchers interested in developing an ICBT course of their own, highlighting components that are important and most valuable to retain.

Our patients also provided various advice for future patients. The most common advice included encouraging future patients to commit to the course, to do the work, and to be consistent in their approach to the course. Patient feedback showed similarity to advice that patients give other patients in online alcohol support groups (Sanger et al., 2019). The advice from past patients to fully engage in the course is also consistent with guidance normally provided by therapists at the outset of any treatment course, but has the potential to be more persuasive if patients know the advice was provided by past patients with lived experience. Past researchers have noted that messages are regarded as more persuasive if they are viewed as coming from individuals with relevant expertise (Wilson and Sherrell, 1993). Providers can also draw on this advice when communicating with patients and incorporate past-patient advice into treatment content. Emphasizing past-patient advice as a way to improve treatment credibility is consistent with the persuasive systems design framework, which highlights the importance of persuasive design elements that increase the credibility of the intervention as one of four key principles of persuasive design (Oinas-Kukkonen and Harjumaa, 2009). This information was ultimately used by our group to co-create ACC graphics with the PORSC and was placed at the beginning, middle, and end of the course. The graphics were meant to provide encouraging advice to patients to complete the ACC. In the future, it would be valuable to explore the extent to which program improvements based on incorporating patient treatment-advice serve to enhance patient engagement and outcomes to a significant degree.

4.1. Limitations and strengths

One limitation of this study is that all data collection was based on self-report measures, which may be subject to biases (Del Boca and Noll, 2000). The open-ended questions resulted in brief responses, and it is also possible that the way questions were formulated put pressure on patients to respond. It is possible that the statements offered may not reflect a strong opinion among patients, and that patients may hold positive and negative opinions that were not expressed. Although patient feedback was positive, it should be noted that 121 of the patients in this study did not complete post-treatment questionnaires. Our analysis revealed that younger or male patients were somewhat less likely to complete post-treatment measures, and a limitation of this study is therefore that young men may be underrepresented in our results. Although other patient demographics were not associated with completion rates, it should be noted that this sample had limited ethnic diversity, with the vast majority of patients reporting as Caucasian. Despite the aforementioned limitations, the current study is important for several reasons. Alcohol-related harm and alcohol misuse are on the rise in Canada (Spithoff, 2019), and the need for efficacious interventions is evident. ICBT represents a promising intervention for patients with alcohol misuse (Hadjistavropoulos et al., 2020c); however, recent research emphasizes the need to increase their clinical implementation and uptake (Strudwick et al., 2020). Numerous patients who provided feedback in this study reported experiencing alcohol problems for many years and never previously seeking treatment. The sample in this study therefore, represents a critical population with both longstanding problems with alcohol use and unmet treatment needs. Additionally, this study is one of the first to qualitatively examine patient feedback of an ICBT program for alcohol misuse, which offers insight into how providers can best serve this population.

The insight gained from qualitative feedback also stimulates multiple future research directions. Asking patients to rate all components of the course (e.g., reflections, provider guidance, worksheets) and each skill to determine whether they found it beneficial or not, would lead to a greater understanding of the need for program improvements. It is also possible with additional rating scales, it would be possible to tease apart why certain features, such as worksheets, were valued (e.g., stimulated thinking, stimulated action). With respect to skill development, we asked which skills patients found most helpful, but it would also be beneficial to explore which skills they found least helpful. Such ratings may assist in uncovering behaviour change techniques that are of greatest and least benefit to patients, as recommended by Michie et al. (2012). Moreover, it may be valuable to examine if perceptions of course components and skills change from pre-, to mid-, to post-treatment, and whether the ratings relate to engagement and outcomes. It would also be valuable to explore if patient perspectives of the program and skills vary by first-time treatment seekers versus those with long-term alcohol misuse and those who have previously received treatment.

With respect to guidance, additional research is needed on the benefits of optional guidance relative to self-guided and regular weekly guidance. Some patients in the study reported a desire for additional interaction. It may be worthwhile to explore if those who wanted additional interaction were ultimately those who completed fewer modules and obtained lower levels of improvement. There may also be value in examining possible correlations between the level of contact engaged in by the patient and the level of improvement obtained. On the note of guidance, the question of whether guidance from a peer is similar to guidance from a professional requires further study. Another noteworthy detail is that 78% (See Table 3) of patients who completed the ACC reported the program either increased or greatly increased their motivation to seek other treatment. Examining the motivational influence to seek additional treatment is another direction for future research. Broadening the scope of material available in the program to address comorbidities, such as anxiety and pain, as well as patient reflections, and then examining rates of uptake and outcomes pertaining to the new materials and reflections would be beneficial.

In terms of methodology, it is possible that exit interviews would provide a deeper understanding of patient preferences, and may be helpful for understanding perceptions of those who do not complete ICBT or who may have had a negative treatment response. It would also be beneficial to explore research results across ICBT programs for alcohol misuse to identify similarities and differences in the results, especially in the case of differing inclusion and exclusion criteria (i.e., accepting more severe patients or patients with lower alcohol use than this study). In the future, it could also be beneficial to use a different satisfaction questionnaire used in other programs to compare satisfaction across different programs (e.g., Client Satisfaction Questionniare-8; Larsen et al., 1979).

4.2. Implications

As ICBT becomes increasingly accepted as an effective method for treating alcohol misuse, patient perspectives on these programs can inform changes to the programs, which could improve adherence and outcomes (Bombard et al., 2018). This study accomplished this by examining our patients' experience of the ACC using a qualitative approach to analyze patient feedback. Although the purpose of this study was not to demonstrate efficacy, the finding that patients were generally satisfied with the course offers encouragement for the ICBT approach to treating alcohol misuse. The high level of satisfaction identified likely reflects the considerable effort that went into the development of the course materials, which involved input from three research groups operating in different regions. Despite the considerable attention already invested in the development of the ACC, the feedback gathered from patients as part of this study offers the opportunity for continuous improvement of the course materials. The results of this study will be beneficial for providers developing or managing an internet-based intervention for alcohol misuse, highlighting skills (e.g., coping with cravings, identifying and managing risky situations) and components (e.g., worksheets, personal reflections) that are most likely to be valued. It also highlights that, even in a program that has undergone thorough revisions, many opportunities exist for addressing patient feedback to ensure programs are fully meeting the needs of patients. Tailoring the degree and nature of guidance to patient needs represents the most common opportunity for improving ICBT, and there is also opportunity to improve the content of the course, expanding information on comorbid conditions. Patients themselves have considerable advice to offer to other patients, and this advice can be incorporated into treatment materials to improve credibility and persuasiveness of the intervention.

Declaration of competing interest

Heather Hadjistavropoulos is Executive Director of the Online Therapy Unit at the University of Regina, funded by the Saskatchewan Ministry of Health, to deliver internet-delivered cognitive behaviour therapy to the residents of Saskatchewan. Cynthia Beck and Michael Edmonds are trainees in this Unit. Christopher Sundström is a former postdoctoral fellow. Wendy Edwards, Dianne Ouellette, and Justin Waldrop are patient partners. Carly Chadwick, Kelly Adlam, Lee Bourgeault, and Marcie Nugent are employees of the Unit.