Initial Field Trial of a Coach-Supported Web-Based Depression Treatment

Abstract

Early web-based depression treatments were often self-guided and included few interactive elements, instead focusing mostly on delivering informational content online. Newer programs include many more types of features. As such, trials should analyze the ways in which people use these sites in order to inform the design of subsequent sites and models of support. The current study describes of a field trial consisting of 9 patients with major depressive disorder who completed a 12-week program including weekly coach calls. Patients usage varied widely, however, patients who formed regular patterns tended to persist with the program for the longest. Future sites might be able to facilitate user engagement by designing features to support regular use and to use coaches to help establish patterns to increase long-term use and benefit.

I. Introduction

Major depressive disorder is a common affliction, with a one-year prevalence rate estimated to be between 6.6-10.3%[1]. Furthermore, depression imposes a very high societal burden in terms of cost, morbidity, suffering, and mortality[2]. It is the leading cause of disability in the United States, and the fourth leading cause of disability worldwide[2]. Although efficacious treatments exist, very few people with depression actually receive them[3]. Barriers to treatment include stigma, time, transportation, and cost, as well as a limited supply of therapists that are overwhelmingly concentrated in urban areas. As such, new modes of treatment are needed that overcome barriers to treatment and reduce the demands on therapists’ time.

To meet this need, Internet sites and mobile applications aimed to reduce depression have been developed. Most of these draw from the principles of cognitive-behavioral therapy (CBT) and Internet-CBT (ICBT) has demonstrated efficacy [4]. ICBT can be self-guided (users are simply provided access to a website or app) or supported by a coach who communicates regularly with the patient via phone calls or messaging. Self-guided ICBT programs tend to have very low use rates compared to coach support programs. A recent review of ICBT found rates of completion of site material ranged from 12-100%, with a median of 56%[5]. Coach supported programs, however, are somewhat better, with one study finding rates of 80%[6]. Supported programs also show significantly larger benefits than self-guided programs[7].

The mechanism through which supported programs lead to greater benefits, however, is not yet understood. Thus, there is little information to guide what coaches should do to promote effective use of ICBT, or how to design coach interfaces to support effective coaching. The current study explored user’s interactions with a web-based depression treatment delivered with coach support. The model of coach support draws from a theoretical model deemed supportive accountability.

II. THEORETICAL BASIS

Supportive accountability is a theoretical model that argues that adherence and engagement is promoted through human support through three aspects: the bond between the user and supporting individual, sense of accountability, and the user’s perception of the supporting individual as possessing legitimacy and expertise[8]. This sense of accountability can be strengthened through several aspects including social presence, expectations, and performance monitoring. Some of these aspects can be made obvious through the technological platform with which the user interacts. For example, users can be made aware that another individual will view data entered in the site. Although automated features may accomplish this, in this study interactions between the user and the coach facilitated this awareness.

Incorporating supportive accountability helps increase the pervasiveness of these platforms, as platforms that leverage this model require both patient and coach-facing features. Platforms that open up patient’s use of the site to a coach allows the coach a glimpse into that patient’s daily life and mental health issues in real-time and allows coaches to tailor calls based on this information rather than a patient’s self-report.

Mohr and colleagues[9] tested whether coach support based on the theory of supportive accountability would increase the use and efficacy of a web-based depression intervention. All patients received a web-based depression intervention, however, patients in the supported group also received weekly 5-10 minute phone calls from a coach. The coaches in this study did not provide clinical support but rather followed a protocol designed to make their monitoring of the patients’ usage more apparent. The coach-supported condition produced significantly greater adherence to the intervention website, however, it did not produce significantly greater reduction in depressive symptoms relative to the self-guided group, departing from past findings on supported programs[7]. In light of this, it is important to better understand how coaching can help achieve the goals of a web-based intervention through coach interactions and site features.

The aim of this study was to conduct a field trial of a web-based depression treatment that included brief, weekly phone calls from a coach drawing on the theory of supportive accountability. We were specifically interested in understanding how and when patients would use site features and whether features were useful and liked by the patients. This would allow us to optimize features and the coaching protocol prior to a randomized trial and guide the design of future sites.

III. FIELD TRIAL

3.1 Patient Recruitment

Patients were recruited mainly online with a few coming from referrals from university primary care clinics. Inclusion criteria were being 19 years of age or older, being able to speak and read English, meeting a diagnosis of major depressive disorder, having a score of 10 or greater on the Patient Health Questionnaire (PHQ-9) [10], and having access and basic literacy to the Internet. Exclusion criteria included impairments that would prevent completion or clinical diagnoses or suicidality for which participation in the trial would be inappropriate or dangerous.

3.2 Study Procedures

After screening eligible, patients were assigned to one of four coaches. Three coaches were doctorate-level psychologists, and one was a clinical psychology doctoral student. Upon enrolling in the study, all patients received access to the website. The site was based on principles of CBT and designed to provide information and skills to overcome depression. The major tenet of CBT is that one’s thoughts, behaviors, and feelings are connected and that thoughts play a casual role in the onset and maintenance of depression[11]. Thus, CBT involves educating patients about this relationship and providing skills that help increase pleasant activities and modify maladaptive cognitions.

Our platform consists of a patient-facing website containing lessons and summaries, tools, and messaging features and a coach-facing dashboard containing data summaries of patient’s site use. Lessons give information about depression and how to cope with it. They consist of several slides and were designed to take 5-10 minutes to read. Summaries accompany most lessons which are a single slide highlighting the main points. Patients were enrolled in the field trial for 12 weeks and lessons and tools were gradually made available to patients on the website. Lessons were made available approximately every three days, and summaries followed two days after the corresponding lesson. Three interactive tools (mood, activity, and thoughts) facilitated the practice of CBT skills. The mood tool allowed patients to enter their mood on a 1-10 rating scale and provided a graph of these scores overtime. The activity tool allowed patients to either enter a past or future activity and to rate their predicted pleasure and accomplished during the activity (for a future activity) or their experienced pleasure and accomplishment during the activity (for a past activity). The thought tool allowed entry of a situation, negative thought, emotional response, cognitive distortion, and alternative thought. The mood and activity tracking tools were available from the start and the thought tracking tool became available at the beginning of the second week of participation. Lastly, patients could send secure messages to their coach through the website. The coach dashboard provided information regarding a patient’s use including what a patient accessed and when (e.g., which lesson was read associated with a date and time stamp), all content entered into the interactive tools (e.g., the specific activities a patient reported doing), and data summaries related to clinical outcomes (e.g., mood, depression). Coaches called the patient each week for a 10-15 minute coach call.

IV. RESULTS

4.1 Study Sample

This field trial consisted of 9 patients (4 men, 5 women). The average age was 42.8. The average level of depressive symptoms at baseline on the PHQ-9 was 16.8 (SD = 4.94) indicating moderately severe levels of depressive symptoms.

4.2 Usage

Patients made an average of 26.11 visits (SD = 28.37) to the website over 13.67 unique days (SD = 15.17), the average length of participation in the study was 35.67 days (SD = 31.11). Throughout the length of the study, patients spent an average of 788.77 minutes on the website (SD = 310.44). Thus visits were relatively brief (30 minutes per session) and time spent on the site was spread over several visits. The most frequent activity on the website was viewing or entering data into the mood tool (M = 29.00, SD = 40.92) and patients entered an average of 16.44 (SD = 21.07) ratings into this tool. The next most common activity was viewing a lesson (M = 24.89, SD = 34.06), followed by using the activity tracking tool (17.00, SD = 21.08), and then the thought tracking tool (M = 9.56, SD = 14.11). Patients entered an average of 14.00 activities (SD = 23.24) and 2.89 thoughts (SD = 3.92) into those respective tools.

We were interested in how the coach calls impacted patients’ use of the site. Due to the sequential aspects of log data, it has been suggested that data visualization is a promising mode of data analysis [12]. Fig. 1 presents a stripe graph displaying the days the site was visited and coach calls were completed throughout participation in the trial. Each column represents a day during the patient’s involvement with the trial, black cells represent a site visit during that day and grey cells represent a coach call. As can be seen in the Figure, visits to the site tend to cluster around the coach calls. Patients are sorted according to their relative use of the site.

Fig. 1.

Days Site Visited and Coach Calls Completed

We were interested in what patterns of site use might predict persistence with the site. Understanding use patterns that are associated with sustained engagement could help coaches in guiding patients toward successful use of the website. As patterns are established early with one’s use of a new technology, we examined site usage within the first week and second week as predictors of the length of one’s use of the site (the number of days from one’s start date in which the last interaction with either the site or a coach was made) and the total number of days the site was visited. In addition to these variables, we also calculated a measure of periodicity of site use. This periodicity value determines what percentage of each patient’s activities (no matter which) is performed within a circadian (24-hour) period. In short, this is an indicator of how regular one’s use of the site is within a daily pattern. Periodicity was calculated for the entire trial period as calculating it for shorter periods is less feasible. Table I displays results from linear regressions predicting length of use from various usage metrics. Although, many variables were not significant due to the small sample, it is worth noting that the magnitude of relations differed widely both within and between each outcome. For length of use, the number of sessions was a stronger predictor than the amount of time in week 1. For week 2, however, the relationship was reversed, with time being the stronger predictor. For number of days the site was visited, however, the total number of sessions was a stronger predictor. This might not be surprising as number of sessions contributes more directly to that outcome. For both outcomes, however, the strongest predictor was periodicity. Thus, people whose visits to the site were more consistent were most likely to persist.

TABLE I.

Regressions of Usage Variables Predicting Engagement

	β	t	p
Length of Use
Time (Week 1)	-.28	-0.81	.48
Sessions (Week 1)	.46	1.33	.28
Time (Week 2)	.55	2.65	.08
Sessions (Week 2)	.14	0.74	.51
Periodicity	.40	2.14	.12
Days Visited
Time (Week 1)	.10	0.24	.82
Sessions (Week 1)	.28	0.71	.53
Time (Week 2)	.07	0.27	.81
Sessions (Week 2)	.16	0.76	.50
Periodicity	.70	3.29	.04

4.3 User Experiences

Patients completed usability interviews at weeks 3, 6, and 12. Five patients completed a total of 10 usability interviews. These interviews consisted of both open-ended questions as well as general statements regarding use of the website modified from the USE questionnaire[13] (e.g., “I like it”, “It was easy to use”, “I learned how to use it quickly”, “It was useful/helpful”) rated on 7-point Likert-type scales. For patients who provided feedback at multiple time points, ratings were averaged to provide one score for each question. Overall, all aspects of the site were rated higher than the midpoint of the scale (see Table II). The lessons were the highest rated aspect of the site and the thought tracking tool was the lowest. Qualitative feedback was also collected, and although the discussion of this feedback is beyond the scope of this report, it was consistent with these quantitative ratings.

TABLE II.

Mean Usability Ratings of Site

	Like	Use	Learned	Useful/Helpful	Average
Overall	5.9	6.2	5.9	6.3	6.1
Lessons	6.3	6.3	6.6	6.8	6.5
Mood tool	6.0	6.7	6.7	5.9	6.3
Activity tool	5.7	5.8	5.6	5.9	5.8
Thought tool	4.7	5.7	5.9	4.4	5.2

V. IMPLICATIONS FOR INTERVENTION DESIGN

The results of this field trial provide insights into how people use web-based treatments for depression and characteristics of use that might correspond to long-term engagement. We will discuss the implications of these findings for the design of future websites and creation of models of support that might facilitate greater use and benefit.

Time spent on the site in the second week of the program corresponded to patients persisting with the program for a longer period of time. The second week might be particularly critical as at this point all tools (mood, activity, and thought trackers) for the website are available and the patients have completed initial coach calls which detail functionality of the website. Increased time on the site during this time might indicate high motivation on the part of the patient, in which case it is unsurprising that these are the patients who tend to persist. However, it is also possible that time spent on the site during the first week allows patients to better explore the site’s features and to make visiting the site part of their life routine. Features of the site could be designed to better facilitate regular, repeated use, such as releasing brief lesson content more regularly and more quickly. Some web-based interventions, have tied lesson release to site use[14]. For example, a new lesson could become available when the previous lesson is completed. This ties content release to user behavior, thus allowing a user to progress at his of hers own pace and reinforce the user for interacting with the site.

The relationship between temporal patterns of site usage, as measured by periodicity, is a novel, though not overly surprising finding in this area. Habit formation is a strong predictor of long-term behavior change and using a previously formed habit or time of day is a useful strategy to create new habits[15]. Features of intervention technologies could be better constructed and framed in ways to create habits, such as allowing patients to schedule reminders, and encouraging them to engage with the intervention during or adjacent to a routine activity such as one’s morning coffee or bedtime. Allowing patients to label their tools (e.g. “My morning log” or “My bedtime journal” for Activity or Thought tools) may facilitate habit formation around the use of tools.

It is also worth noting that coach calls tended to increase activity as many of the site visits clustered around these calls. Automated reminders to visit the site could be helpful, but should facilitate regular, periodic visits. For example, a patient could schedule a notification to visit the site at the same time everyday tied to their preferred time of use (e.g., first thing in the morning, during a lunch break, before they go to bed).

Our platform was constructed to facilitate monitoring, in line with supportive accountability. As such, coaches could view what a patient was doing on the site and when, as well as all content entered into the interactive tools. Using this information, coaches could comment on use during calls. This differs widely from ways that professionals usually support ICBT programs in which they still must rely on patient’s self-report of what they did and learned that week. A patient’s self-report is useful, as it highlights their perception and recall of their week and activities, but usage data allows coaches an unbiased report and allows direct comments on a patient’s use. Coaches could support pattern formation (like establishing routine) or intervene early with patients who show site use patterns that are associated with dropout (like little minimal time on the site during the second week).

More attention should be devoted to understanding how early use of websites might facilitate continued use and eventual benefit. In face-to-face CBT, early change predicts eventual benefit. In psychotherapy research, it is difficult to measure use of skills outside of sessions, however, in web-based research, site usage is a reasonable proxy.

Despite these findings, this study was not without limitations. For example, this study used a small sample of 9 patients, many of whom recorded very little usage of the site. This is not uncommon among studies of ICBT, one study found that log-ins ranged from 1 to 65 with an average of 18.72 for those who provided outcome data and 7.18 for those who did not[16]. In our study, we found that patients could be divided into three roughly equal subsets, those who recorded none or almost no use of the site, those who recorded very minimal use of the site, and those who recorded heavy use of the site. Similar patterns have been found in other trials[6].

Nevertheless, this trial provides useful lessons to the burgeoning field of research in ICBT. As many studies focus on the gross metric of symptom improvement, more investigations such as this could be useful to illustrate how people use the site. This information may be more useful to help create protocols for models of support. This trial also represents the first steps in efforts to create a multiplatform, multiuser tool, accessible by web and mobile device. Ideally, other patients could serve similar roles to that of the support coach through engaging with patient generated material to reinforce concepts included in supportive accountability.

VI. CONCLUSION

Our work highlights aspects of use of ICBT platforms. First, usage tends to be brief and sporadic, but regular, periodic usage tends to predict persistence with the program overtime. Second, time invested early in the intervention might be critical to predicting long-term use. Future ICBT platforms should be designed in ways to allow easy and frequent usage of features and improve the ability of patient’s to tailor features to facilitate habitual and regular use.