Randomized Controlled Trial of a Computerized Interactive Media-Based Problem Solving Treatment for Depression

Abstract

This study evaluated the efficacy of an interactive media-based, computer-delivered depression treatment program (imbPST) compared to a no-treatment control condition (NTC) in a parallel-group, randomized, controlled trial conducted in an outpatient psychiatric research clinic. 45 adult participants with major depressive disorder or dysthymia were randomized to receive either 6 weekly sessions of imbPST or no treatment (No Treatment Control; NTC). The primary outcome measure was the Beck Depression Inventory II (BDI-II). There was a significant Group x Time interaction effect [F (1.73, 43)= 58.78; p <.001; η2 = .58, Cohens d = 1.94], such that the patients receiving imbPST had a significantly greater reduction in depressive symptoms compared to the patients in the NTC condition. Participants in the imbPST group improved their depression symptoms significantly from moderate (BDI-II = 21.9 ±4.20) to mild levels of depression (BDI-II = 17.9 ± 4.0) after receiving 3 weekly sessions of imbPST (p <0.001), and progressed to still milder levels of depression after six weekly sessions (BDI-II = 14.5 ± 3.7, p <0.001). NTC participants showed no significant reduction in BDI-II scores (BDI-II =21.8 ± 4.2 pre, BDI-II = 21.5 ± 5.2 post, N.S.). Additionally, 40% of the imbPST group showed a clinically significant and reliable change in depression levels while none of the NTC group met this criterion. imbPST participants rated the program highly usable on the system usability scale (SUS) after the first session (SUS Session 1 = 74.6 ± 7.2) and usability scores increased significantly by the last session (SUS Session 6 = 85.4 ± 5.6). We conclude that imbPST is an effective, engaging, and easily used depression treatment program that warrants further evaluation with heterogeneous depressed populations in a stand-alone, self-administered fashion.

Approximately 1 in 10 adults in the U.S. report experiencing depression, but only about 50% of the depressed population receive effective treatment (Kessler, Chiu, Demler, Merikangas, & Walters, 2005). Depressed individuals often don't seek treatment for a variety of reasons including cost, lack of privacy, possible stigma, inconvenient clinic hours, and difficulty accessing evidenced-based treatments (particularly psychotherapy) (Hirschfeld et al., 1997).

Computer-based therapy (CB) can be effective for treating depression, with small to moderate effect sizes (Andersson & Cuijpers, 2009). Proudfoot et al. (2003), showed that patients who used the Beating the Blues program (BtB) had significant improvements in depression and anxiety compared to a usual-care control condition by the end of treatment and at 6 months follow-up. Christensen, Griffiths, and Jorm (2004) demonstrated that depressed adults using a CBT therapy website (MoodGYM) showed a reduction in depressive symptoms when compared to a matched control intervention (d = 0.30).

Another RCT from this group of investigators found that participants with increased depressive symptoms showed improvements after implementing the MoodGYM program (d = 0.20 to 0.40) compared to controls (Christensen, Leach, Barney, Mackinnon, & Griffiths, 2006). Similarly, individuals with mild to moderate depression receiving a CBT-based Internet psychotherapy experienced a greater reduction in depression and anxiety symptoms posttreatment compared to controls (d = 0.94; Andersson et al., 2005). Additionally, Perini, Titov, and Andrews (2009) demonstrated that clinically depressed adults in Australia and New Zealand again showed a large reduction in depressive symptoms after using a CBT-based Internet intervention in comparison to the control group (d = 0.89).

The rapid proliferation of CB and Internet-based studies during the 2000s allowed a number of meta-analyses to be conducted. These have provided further support for the effectiveness of CB and Internet therapies for depression. For example, Barak, Hen, Boniel-Nissim, and Shapira (2008) analyzed 92 studies that examined the success of Internet-based interventions. Their analysis revealed that Internet-based interventions in general shared medium effect sizes (d 0.53),similar to face-to-face therapies, and that depression interventions in general produce small to medium effect sizes (d 0.32). Interventions based on a cognitive behavioral therapy model fared better, demonstrating a large effect size (d 0.83). A meta-analysis by Andersson and Cuijpers (2009) also assessed the efficacy of CB therapies to treat depression. They showed a medium effect size for Internet and CB treatments in comparison to controls (d 0.41). Furthermore, larger effect sizes were seen in studies using support (d 0.61) (e.g., therapist, automated, telephone, email) compared to no support (d 0.25).

Compared to the 20% average dropout rate in controlled studies of face-to-face psychotherapy (Swift & Greenberg, 2012), the dropout rate for unguided, self-help CB depression programs is approximately 57% (Richards & Richardson, 2012). The high dropout rate may be caused by the program's difficulties engaging and developing a therapeutic alliance with the users. This may decrease acceptability or efficacy. Many CB therapies have limited interactivity, don't provide tailored feedback, and are text-dominated (e.g., MoodGym, (Christensen et al., 2004); Worry and Sadness Program (Newby, et al., 2013). As a result, the affective bond between therapist and client that underpins the therapeutic alliance may not develop (Martin, Garske, & Davis, 2000; Ormrod, Kennedy, Scott, & Cavanagh, 2010; Zuroff, et al., 2000). The fact that CB depression treatments lacking human support have higher dropout rates than those that don't supports this premise (Richards & Richardson, 2012).

However, a recent large-scale pragmatic trial that added human support did not guarantee effective outcomes either, or even adherence to using the program. The trial included weekly telephone support calls from study “technicians” to encourage use of two frequently used Internet CB programs (BtB and MoodGYM) and to assist with resolving technical difficulties. No significant differences in depression outcomes were found for the CB programs compared to a usual-care control condition in primary care (Gilbody et al., 2015). Furthermore, nearly one-quarter of participants assigned to use the programs never accessed the programs, and for those that did access the programs the median number of sessions completed was one (of eight sessions offered).

In the current study, we evaluated an interactive multimedia-based, immersive, computer based Problem Solving Treatment program (imbPST) that responds flexibly to users' experience over time (Carter, Buckey, Greenhalgh, Holland, & Hegel, 2005; Cartreine, Locke, Buckey, Sandoval, & Hegel, 2012) and is designed to facilitate the therapeutic alliance (Martin, et al., 2000; Ormrod, et al., 2010; Zuroff, et al., 2000). Recent meta-analyses of studies of face-to-face problem-solving therapy conducted by Cuijpers, van Straten, and Warmerdam (2007) and Malouff, Thorsteinsson, and Schutte (2007) found that problem-solving treatment (PST) is superior to no treatment, treatment as usual, and attention placebo for treating major depressive disorder.

To enhance the therapeutic alliance, imbPST uses an onscreen virtual therapist (MTH). He appears throughout the users' engagement with imbPST, and interacts with the users through branching algorithms personalized for the users based on user choices in their problem-solving efforts, and their scores on the depression questionnaire. When users encounter difficulties, the imbPST program asks targeted questions about the issue and provides flexible troubleshooting assistance. Through targeted questioning about progress, effort, solutions, action plans, and other parts of problem solving, the program identifies areas where the user is having trouble and offers suggestions, additional training, and encouragement. This feature is intended to overcome a weakness noted for other CB depression programs—the ability to develop and maintain the therapeutic alliance over time (Barazzone, Cavanagh, & Richards, 2012).

We previously conducted an open trial (Berman et al., 2014) comparing imbPST to a previously completed study of face-to-face PST (Oxman, Hegel, Hull, & Dietrich, 2008) and found similar patterns of improvement in depressive symptoms. We also compared imbPST to studies of face-to-face psychotherapy (Agnew-Davies, Stiles, Hardy, Barkham, & Shapiro, 1998) and the CB therapy program BtB (Ormrod et al., 2010) on their ability to develop therapeutic alliance. We found that imbPST creates a strong therapeutic alliance equivalent to face-to-face psychotherapy (Berman et al., 2014), consistent with previous CB therapy literature (Andersson et al., 2012; Kiropoulos et al., 2008; Knaevelsrud & Maercker, 2006, 2007; Pugh, Hadjistavropoulos, & Dirkse, 2016; Sucala et al., 2012). Despite these promising findings, to date imbPST has not been subjected to a randomized controlled trial.

For the trial presented here, participants were randomly assigned to two treatment conditions (imbPST and a no-treatment control; NTC). We hypothesized that compared to the NTC condition patients using the imbPST program would show significantly greater improvements in depression as measured by the Beck Depression Inventory-II (BDI-II; Beck, Steer, & Brown, 1996; Beck, Steer, & Garbin, 1988).

Methods

The Beth Israel Deaconess Medical Center Institutional Review Board approved this study.

Participant Recruitment and Randomization

Participants were recruited through announcements in the local media, university health centers, hospitals, churches, workplaces, and community center bulletins in the greater Boston area. Participants were randomized to treatment condition based on an online true random-number service in blocks of six. The study was ended early due to funding and logistical reasons (without an interim analysis), leaving the study conditions unbalanced in size and gender distribution.

Screening Of Participants

Persons 18 years of age and older who reported interest in the clinical trial were contacted for a prescreening interview over the phone to explain the study and to prescreen for eligibility. If potential participants met the prescreening criteria, an initial consent was obtained verbally over the phone. They were then invited for an in-person screening visit. This visit included five parts: (a) explaining and describing the study; (b) signing the informed consent form; (c) evaluating inclusion and exclusion criteria; (d) administering the baseline assessment; and (e) assigning the participant randomly to the treatment or control conditions. Persons not eligible for the study were provided with local referral numbers to contact other area services. Participant characteristics are shown in Table 1.

Table 1. Baseline Characteristics of the Groups.

		Control		Treatment
		(n = 20)		(n = 25)
		n	%	n	%	p value
Gender						p < 0.001
	Male	13	65	4	16
	Female	7	35	21	84
Age						N.S.
	Male	24.5	SD = 10.01	29.23	SD = 10.75
	Female	29.86	SD = 7.09	30.9	SD = 8.73
Race/Ethnicity						p = 0.10
	White/Caucasian	10	50	19	76
	Black/African American	4	20	-	-
	Hispanic/Latino	5	25	5	20
	Asian	1	5	1	4
	Other	-	-	-	-
Educational Level						p = 0.37
	Some high School	3	15	-	-
	High school graduate	1	5	1	4
	Some college or Associates degree	8	40	7	28
	College graduate	5	25	9	36
	Masters level graduate	3	15	7	28
	Doctorate level Graduate	-	-	1	4
	Did not respond	-	-	-	-
Computer Skill						p = 0.32
	> 1 hr. a day	7	15	3	12
	< 3 hrs. a day but less than 5 hrs.	7	35	12	48
	< 6 hrs. a day but less than 8 hrs.	2	10	4	16
	< than 8 hrs. a day	4	20	6	24

Note. p values represent between group differences.

Study Inclusion Criteria

Potential participants had to meet the following criteria: (a) be 18 years of age or older; (b) meet criteria for a current major depressive disorder or dysthymic disorder based on the DSM-IV-TR (American Psychiatric Association, 2000); (c) a score of 10 or greater on the Patient Health Questionnaire-9 item Depression scale (PHQ-9; moderate or greater depressive symptom severity; Kroenke, Spitzer, & Williams, 2001); (d) major depressive disorder or dysthymic disorder as the primary diagnosis and not secondary to any other diagnosis, such as an anxiety disorder or complicated bereavement as determined by the judgment of the clinical interviewer (advanced clinical psychology graduate student, first author LS); (e) be able to write and speak English according to the Rapid Estimate of Adult Literacy in Medicine test (REALM; Murphy, Davis, Long, Jackson, & Decker, 1993).

Participants who were taking antidepressants were included but were questioned about changes in medication or in dosage at each study assessment.

Study Exclusion Criteria

Potential participants were excluded from the study if they had: (a) current suicidal ideation (i.e., having current thoughts of hurting him/herself whether with or without concrete plans) or a history of suicide attempts or self-injurious behavior in the year prior to study participation; (b) been diagnosed with schizophrenia, bipolar I disorder with psychosis, other disorders with psychotic symptoms; (c) history of brain injury that included loss of consciousness greater than 15 minutes and/or posttraumatic amnesia of any duration, (d) history of treatment with antipsychotic medication, (e) a felony conviction; (f) current or recent (i.e., within the previous 6 months) substance abuse/dependence diagnosis (other than nicotine or caffeine); (g) undergoing psychological treatment (e.g., face-to-face psychotherapy).

A standardized method was set for establishing significant reliable worsening of depression during the study. imbPST participants completed the PHQ-9 at each session. The PHQ-9 stratifies depression on five levels (Minimal, Mild, Moderate, Moderate–Severe, and Severe). Individuals scoring in the severely depressed range (21 to 27) for two consecutive sessions were to be withdrawn from the study. Similarly, participants were to be withdrawn if their PHQ-9 score increased two levels within three consecutive sessions.

Procedures

Participants assigned to the imbPST group received the computerized intervention via flash drive, which they used on a designated computer onsite. The first session occurred within 1 week of enrollment, and Sessions 2-6 were scheduled at the outset as close to weekly as possible. Participants in the NTC group were scheduled to come for a mid-point assessment (4 weeks after baseline) and for a final assessment (7 weeks after baseline). Participants randomized to NTC were offered use of imbPST off study after completion of their study participation. Both treatment groups were compensated $100 for their time to complete the three study assessments.

Intervention (Imbpst)

imbPST uses interactive media in a flexible, computer-delivered format, to offer “virtual therapy” that feels more like interacting with another individual than with a software program. The program provides a simulated therapy session based on a PST treatment manual used in face-to-face depression clinical trials (Hegel & Arean, 2003). A “virtual” therapist (author MTH presented via audio and video) provides programmed instructions on the steps and skills of problem solving, responds empathically to the users' input, and offers tailored feedback. Tailoring was driven by PHQ-9 scores for the session (e.g., providing information on the meaning of the score, comparison to prior scores indicating improvement or deterioration, and recommendations for treatment options), quality of problem-solving steps (e.g., providing instructions on improving and guiding revision of problem and goal statements, brainstorming solutions, evaluating pros and cons), and trends of progress (e.g., commenting on trends of success and providing encouragement and empathic statements if problem resolution was not going well). The imbPST program requires minimal reading skills because the therapist guides the user through the problem-solving process via audio, video, and graphical feedback.

The first session includes six components: (a) a welcome to imbPST by the virtual therapist; (b) a self-assessment of depressive symptoms using the PHQ-9; (c) feedback on the PHQ-9 results; (d) psychoeducation about depression; (e) introduction to the components of PST (i.e., defining the problem in a way that is potentially solvable, stating an objective and achievable goal, brainstorming possible solutions, choosing effective solutions, creating an action plan, and scheduling pleasant activities); and (f) structured guidance through the PST components addressing a personal problem of the participant's choosing. At the end of the session, a printout of the session's work is generated for the participant. This report includes the problem chosen, the goal, the step-by-step action plan, and the pleasant activities planned for the week.

Subsequent sessions (Sessions 2 to 6) include the following:

• A welcome to the current session;

• A depression self-assessment using the PHQ-9;

• A comparison between previous and current depressive symptoms and feedback on results;

• A check on how the action plan unfolded and the level of participant satisfaction with their effort;

• Troubleshooting of difficulties with the problem statement, goal, solutions, or action plan;

• Suggestions for improvement;

• Development of an action plan for the previous problem for the coming week if indicated/ desired;

• An option to work on an additional problem;

• Scheduling pleasant activities;

• Presentation of sample PST cases.

The imbPST program did not allow participants to skip ahead without completing the previous step, thus assuring that all imbPST participants were exposed to the same intervention material. The initial session, which includes a substantial amount of education on the steps of problem solving, lasted approximately 70 minutes. Sessions 2-6 lasted approximately 40 minutes. The total amount of exposure to the imbPST program averaged 4.9 hours per participant.

As a safety measure, the imbPST software created a printed report of the patient's depressive symptoms after they completed the PHQ-9 at the start of each session. This printout was reviewed by the study staff, and if the participant endorsed suicidal ideation, the individual was further assessed for safety. Although participants in our study had moderate to severe levels of depression, nobody became suicidal during the study.

Ntc

Participants in the NTC group were asked to attend the intake session, and then returned for a midpoint depression assessment 4 weeks after baseline and for a final depression assessment at Week 7.

Assessment Time Points

To monitor participants' depression levels and progress, three assessments were used: baseline (Week 0), midpoint (Week 4), and posttreatment (Week 7). During the baseline assessment participants completed a series of self-assessment measures and structured diagnostic interview (described below), including a technology usability assessment for the imbPST group: the System Usability Scale (Bangor, Kortum, & Miller, 2008). Midpoint measures were identical to baseline measures, excluding the technology usability assessment. Posttreatment measures were identical to the baseline assessment.

Study Measures

Demographic Questionnaire

Information was collected at baseline on participants' age, race/ethnicity, gender, education level, and computer usage.

Mini International Neuropsychiatric Interview (MINI)

The MINI is a brief, structured diagnostic interview that assists in the diagnosis of psychiatric disorders, based on DSM-IV and ICD-10 diagnostic criteria. The MINI demonstrates strong diagnostic concordance with the Structured Clinical Interview for DSM-III-R diagnosis of major depression, with a Cohen's kappa of 0.84. In addition, the MINI has demonstrated strong interrater reliability (0.75 to 0.80) and test–retest reliability (above 0.75) (Sheehan et al., 1998; Sheehan et al., 1997). The MINI was administered by an advanced clinical psychology graduate student (first author LS) at the enrollment interview and at the posttreatment assessment to establish depression diagnoses. The assessor was unaware of the assigned treatment condition at the posttreatment assessment.

Beck Depression Inventory–II

The Beck Depression Inventory Second Edition (BDI-II) (Beck et al., 1996; Beck et al., 1988) is a 21-item self-report instrument used to assess the severity of depressive symptoms on a 4-point scale (0–3), with a total score ranging from 0 to 63. Cutoff scores are: 0–13, minimal depression; 14–19, mild depression; 20–28, moderate depression; and 29–63, severe depression (Beck et al., 1988). The BDI-II has a coefficient alpha of 0.92 with depressed outpatients and 0.93 with nonclinical samples, and a test–retest reliability coefficient across the period of 1 week of 0 .93 (Beck et al., 1988; Dozois, Dobson, & Ahnberg, 1998). Scale reliabilities in this sample for the BDI-II ranged from α = 0.65 to 0.87 depending on assessment time point and condition. The BDI-II was the primary outcome measure of depressive symptoms.

Hopkins Symptom Checklist 20-Item Depression Scale (HSCL-20-d)

The HSCL-20-d is a validated self-reported measure of depressive symptoms. This 20-item depression scale (Katon et al., 1996; Katon et al., 1995) is derived from the 90-item HSCL (Lipman, Covi, & Shapiro, 1979). Items are rated on a 5-point scale (0–4) according to how much the symptom has been experienced during the past week. Scale scores are determined by dividing the sum of the items by the total number of items, yielding a range of 0–4. A score of 1.72 has been shown to be associated with a high positive predictive value for a diagnosis of major depression in adult primary care patients (Mulrow et al., 1995). The HSCL-20-d has also been shown to be a valid measure of depressive symptom improvement (Katon et al., 1996; Katon etal., 1995). The HSCL-20-d with depressed adults has a coefficient alpha of 0.86 (Williams, Stellato, Cornell, & Barrett, 2004). The HSCL-20-d item scale was used in our previous studies of CB and face-to-face PST (Berman et al., 2014; Oxman et al., 2008).

Patient Health Questionnaire 9-Item Depression Scale (PHQ-9)

The PHQ-9 was used for participant selection (score ≥ 10). The PHQ-9 was designed for depression screening, maps directly onto DSM-IV diagnostic criteria for major depressive disorder, and has been validated with 3000 primary care patients (Kroenke et al., 2001). Responses for these 9 items range from 0 (not at all) to 3 (nearly every day). A total score is calculated by adding the scores for each of the 9 symptoms. Higher scores indicate greater depressive symptom severity, with scores ranging from 0 to 27. The PHQ-9 possesses excellent sensitivity (0.97), specificity (0.97), and positive predictive value (0.75) for detecting major depression (Kroenke & Spitzer, 2002).

System Usability Scale (SUS)

The SUS is a 10-item self-report measure of the ease of using computer programs. Items are scored on a 5-point scale (0–4) measuring the strength of agreement with each of 10 statements (e.g., “I found the system unnecessarily complex” “I felt very confident using the program”). Cronbach's alpha for inter-item agreement is a robust 0.91 (Bangor et al., 2008). Factor analysis shows only one significant factor, suggesting that the overall score is the best measure of usability. The sum of the individual items (range 0'40) is multiplied by 2.5 to obtain the total score, ranging from 0 to 100. SUS scores that are 68 or higher are at least average, while scores below 68 are below average (Sauro, 2011). The SUS scale was programmed into the imbPST software and was administered to the participants at the end of Sessions 1 and 6.

The Rapid Estimate of Adult Literacy in Medicine-Short Form (REALM-SF)

The REALM has been demonstrated to be a brief valid measure of literacy (Arozullah et al., 2007). Participants read 7 words aloud. Reading all 7 correctly corresponds to a 12th grade reading level—which was set as being sufficient to use the imbPST computer program.

Statistical Analysis

To test the main hypothesis, a between-group repeated-measures analysis of variance (ANOVA) was used for the depression measures. Partial eta-squared (η²) was computed as a measure of effect size within a group. There were only two levels of factor A (i.e., imbPST and NTC). Alpha level was set at 0.05. Pairwise mean comparisons were performed for the time factor using the Bonferroni procedure. The familywise significance level was set at 0.05.

The Mauchly Test of Sphericity and Greenhouse-Geisser tests were used to test for homogeneity of covariance and variance. Preliminary analyses were conducted prior to testing the study hypotheses. The descriptive analyses of the data were inspected (including frequencies, means, standard deviations, skewness, and ranges of values) to determine accuracy of entry and to look for outliers. The analyses were based on the intention-to-treat principle (i.e., those who provided follow-up data irrespective of treatment adherence), although, in this instance, all participants completed the trial. Proportions of participants reaching particular endpoints (e.g., reliable change vs. no reliable change) were analyzed using chi-square tests. Between-group Cohen's d effect sizes were calculated by dividing the difference in change scores between groups by the pooled SD of the change score.

The SUS data were analyzed using the Kruskal-Wallis test, examining both the effects of time (Session 1 to Session 6), as well as the effects of gender, ethnicity, education level, and computer skills. The education level data included six levels. These were analyzed both using all six levels, and also using just two levels (college graduate vs. non-college graduate). The computer skills question included four levels. This was reduced to two levels for some analyses (daily use vs. less-than-daily use). SPSS (version 21.1 for Windows) and Matlab (version 2015b, Math-works) were used to conduct the analyses.

Results

Enrollment And Demographics Of The Study Sample

Of the 100 responders to the advertisements, 97 were prescreened (Figure 1). In the prescreening phone interview, 33 participants were excluded. An additional 19 individuals were excluded during the in-person intake screening (18 male and 1 female). Among them, 11 (75%) were excluded because they had suicidal ideation and attempts in the past, 4 (6%) were excluded because they did not want to wait 6 weeks to start treatment, and another 4 (6%) were excluded because the clinical study did not meet their expectations (i.e., they were interested in receiving antidepressant medication in addition to the computer intervention).

Figure 1.

Flow of participants through the study.

Table 1 shows the characteristics of the study subjects. Forty-five participants were randomized to study conditions (imbPST n = 25; NTC n = 20) and all completed the trial. Based on the MINI structured clinical interview, 95% of all participants met the criteria for a diagnosis of major depressive episode and 5% met criteria for a diagnosis of dysthymia. Significantly more women were assigned to imbPST than to the NTC condition, but adjusting for gender in the analyses did not alter the findings.

Imbpst User Satisfaction And Usability

SUS scores were examined to determine whether the imbPST participants found the program to be user-friendly. Participants (n = 25) rated imbPST at Week 1 (M = 74.6, SD = 7.17) and Week 6 (M = 85.4, SD = 5.62). The increase in SUS scores was significant (Session 1 vs. Session 6, Kruskal Wallis chi-square 20.7, p < 0.001). Those with lower reported computer use (daily vs. less than daily) had significantly lower SUS scores at Session 1 (less than daily use = 68.8 vs. daily use = 76.4, chi-square 5.1, p = 0.02) but this difference was not present at Session 6 (less than daily use = 82.1 vs. daily use = 84.4, chi-square 2.8, p = 0.09). Similarly, men showed statistically significant lower SUS scores than females at Session 1 (men 67.5, vs. women 76.0, chi-square 4.2, p = 0.04), but this difference was also gone by Session 6 (men 87.5 vs. women 85.0, chi-square 0.5, p = 0.5).

Depression Outcomes

No participants were withdrawn from the study for clinical instability or worsening. Of the 45 randomized participants, 100% completed all three time point assessments. For the BDI-II, there was a significant Group × Time interaction effect: F(1.73, 43) = 58.78; p < .001. The patients receiving imbPST had a significantly greater reduction in depressive symptoms compared to the patients in the NTC group (Figure 2; Table 2). When the individual time points were compared, BDI-II levels at the midpoint were significantly different from baseline (MD = 4.00, SD = 0.50; p < 0.001, 95% CI 2.6–5.4). The BDI-II continued to decrease significantly with time, and the difference between midpoint and posttreatment BDI-II scores was also significant (MD = 3.40, SD = − 0.04; p < 0.001, 95% CI 2.5–4.3). The NTC group did not show any differences in BDI-II scores over time. Within the imbPST group no significant interactions were found for gender, race/ethnicity, education level, or computer skills and depressive symptoms.

Figure 2.

BDI-II results for the trial by group. Symbols represent the mean. The error bars show the standard error of the mean. * refers to group difference p < 0.01.

Table 2. Within and Between Group Treatment Effects for imbPST Across Assessment Time Points.

	Baseline Mean	Baseline SD	Midpoint Mean	Midpoint SD	Post Mean	Post SD	F	P-value	Within group Eta-squared	Between group Cohen's d
BDI imbPST	21.9	4.2	17.9	4.0	14.5	3.7	58.8	<0.001	0.81 (CI 0.7–0.9)	1.94 (CI 1.6–2.3)
BDI Control	21.8	4.2	22.4	5.5	23.2	5.2	-	-	0.11 (CI 0.0–0.4)	-
HSCL imbPST	1.5	0.5	1.4	0.4	1.2	0.4	24.9	<0.001	0.54 (CI 0.4–0.7)	0.94 (CI 0.5–1.3)
HSCL Control	1.5	0.5	1.5	0.4	1.6	0.4	-	-	0.18 (CI 0.1–0.4)	-
	Baseline	Baseline			Post	Post	Chi-sq	P-value
	Mean	SD			Mean	SD
SUS imbPST	74.6	7.2	-	-	85.4	5.6	20.7	<0.001	-	-

Note. For the BDI and HSCL results, the F and p-values are the results from the two-way repeated measures ANOVA considering both the effects of time and group. The within-group eta-squared values are from one-way ANOVAs examining just the effects of time individually. For the SUS results, the chi-square statistic for the Kruskal-Wallis test is shown. Confidence intervals for the Cohen's d and eta-squared values were calculated using bootstrap resampling.

For the HSCL-20-d, there was a significant Time × Group interaction effect: F(1.56, 43) = 24.91; p < .001. The patients receiving imbPST had a significantly greater reduction in depressive symptoms compared to the patients in the NTC group (Figure 3; Table 2). In the imbPST group there was a reduction in the HSCL-20-d scores between baseline and the midpoint (MD = 0.11, SD = 0.10; p = 0.1; 95% CI −0.02-0.23). The HSCL-20-d scores continued to decrease significantly with time, and the difference between midpoint and posttreatment scores was also significant (MD = 0.20, SD = 0.02; p < 0.001, 95% CI 0.14–0.27). Finally, changes on the HSCL in the control group were nonsignificant.

Figure 3.

HSCL results for the trial by group. Symbols represent the mean. The error bars show the standard error of the mean. * refers to group difference p < 0.01.

Clinical Significance Of Findings

The clinical significance of treatment effects was assessed in three ways. First, we calculated the percentage of people meeting the criteria for major depression before and after the treatment. The imbPST group went from 96% to 76%, while the control group was 95% at both time points. Second, we calculated clinically significant change as recommended by Jacobson and Truax (Jacobson & Truax, 1991) by calculating the number of participants who had a change in the BDI-II that was greater than two standard deviations from the mean of the baseline value. Using this approach, within the imbPST group, 40% met the criteria for a clinically significant change, while none of the NTC group met this criterion.

Third, reliable change was assessed using the methods described in Jacobson and Truax (1991). The standard error of measurement (SEM) for the BDI was obtained from the meta-analysis performed by Yin and Fan (2000). If the absolute value of the difference between the pre- and posttreatment BDI-II values divided by the SEM for the BDI were greater or equal to 1.96, then this was recorded as a reliable change. Eighty-eight percent of the participants in the imbPST group (n = 22) met this criteria for reliable change. With the cutoff for clinical significance set as a BDI < 14 (minimal depression), 40% of the imbPST (n = 10) met criteria for both clinically significant and reliable change. None of the participants in the NTC condition met these criteria.

Change In Antidepressant Use

Over this 6-week trial, none of the participants reported changing their use of antidepressants.

Discussion

Depression Outcomes

The current study was a randomized controlled trial of a computerized interactive media-based problem-solving treatment to reduce symptoms of depression in depressed adults. This program had been evaluated previously in a smaller uncontrolled trial (Berman et al., 2014), which showed that the program had high scores both for usability and therapeutic alliance as well as a strong suggestion of efficacy. Participants in the treatment group (imbPST) in this study received six sessions of computer-delivered PST, while participants in the NTC group were assessed but not treated. In the current study depression scores decreased significantly more in the imbPST group compared to the NTC group.

Depressive symptoms in the imbPST group improved significantly by the midpoint assessment and continued to improve at the posttreatment assessment. On the other hand, depressive symptoms remained in the same range (moderate to high) for the NTC group over the same time assessments. The efficacy of imbPST to reduce symptoms of depression after three sessions is consistent with other studies where adults with depressive disorders improved their symptoms after three sessions of face-to-face problem-solving therapy (L. Mynors-Wallis, 1996; L. M. Mynors-Wallis, Gath, Lloyd-Thomas, & Tomlinson, 1995; Oxman et al., 2008). The between-group effect sizes observed in this study using imbPST as a stand-alone treatment in a structured setting (BDI-II d = 1.94, HSCL d = 0.94) are comparable to or better than the findings from other CB and internet therapy studies and meta-analyses (Christensen et al., 2004, d = 0.30; Christensen et al., 2006, d = 0.20–0.40; Andersson et al., 2005, d = 0.94; Perini et al., 2009, d = 0.89; Barak et al., 2008, d = 0.32–0.83; Andersson & Cuijpers, 2009, d = 0.41–0.61).

Imbpst Usability

The participants' assessment of the program's usability improved significantly over time. This is encouraging, since poor usability would likely reduce the likelihood that individuals would complete a treatment course when left to themselves outside of a research trial. The usability data suggest that the subjects learned how to use the program effectively and were satisfied with it. However, at the initial usability assessment, men and individuals with less than daily computer use gave the program significantly lower usability scores. This difference, however, disappeared by the Session 6 assessment. These data suggest that as users with less frequent computer use become more familiar with the program they find it more usable and easier to navigate. According to Bangor et al. (2008) the average system usability score is 70 points, while Sauro (2011) suggested 68 points to be considered as average. In either case, the present scores (74.6 at Session 1 and 85.4 at Session 6) put imbPST in the good-to-excellent range. Furthermore, these scores were consistent with another two studies where imbPST was evaluated for usability (Berman et al., 2014; Cartreine et al., 2012).

These findings suggest that the imbPST program may be poised to become a valuable addition to the growing number of CB and Internet-based therapies for depression. Although in need of evaluation, the intensive use of audio and video elements and interactive and tailoring features may improve engagement with the user, reduce attrition, and ultimately enhance treatment outcomes under real-world conditions. Given the tremendous obstacles to moving evidence-based psychotherapies into routine practice, the availability of this type of technology could have a meaningful impact on reducing the burden of depression in the general population.

Limitations

Our retention rate of 100% could have been aided by the fact that our clinical trial had three study coordinators dedicated to the recruiting and monitoring of participant appointments. Therefore, these results cannot be generalized to a situation where the program is fully self-administered. For instance, in a systematic review of CBs for depression, Kaltenthaler et al. (Kaltenthaler, Parry, Beverley, & Ferriter, 2008) reported dropout rates ranging from 0% to 75%. A recent review of technology-assisted self-help tools by Newman, Szkodny, Llera, and Przeworski (2011) showed that programs administered as they were here (i.e., self-administered within a structured setting) are the most effective. The imbPST program was administered in an unguided, automated self-help manner, and the very high retention rates perhaps suggest improved engagement of the user compared to prior such programs, but nonetheless, the next step in research with imbPST is to determine if the high retention rates and significant improvements can be achieved in a fully stand-alone mode without research staff or therapist-guided support.

The majority of the participants who completed the study were college students (i.e., undergraduate or higher level) and professional workers. Having a college-educated study sample, as occurred in this study, has been a common observation in computerized and Internet trials (Andersson & Titov, 2014). These factors could skew the results, as well as the attrition and dropout rates, in favor of the technology. It is likely that the population in this study was more highly educated and more experienced with computers than those persons accessing CBs in real life (Andersson & Titov, 2014). imbPST, like other computerized interventions, needs to be specifically tested with populations that have lower educational backgrounds.

Although a standardized semistructured interview was used to detect changes in diagnosis, another potential limitation is that we mostly relied upon a self-reported measure of depressive symptom severity (BDI-II) as the primary outcome measure. Although such scales are widely used in clinical trials and are practical tools to detect the severity of symptoms of depression, the accuracy and usefulness of the instruments depends on the individual's ability to rate him or herself accurately (Andersson & Titov, 2014).

Participants in the imbPST group received an active intervention for depression in an unblinded fashion, while participants in the NTC group did not receive any treatment. Even though this is a commonly used methodology (i.e., treatment vs. no-treatment or waitlist controls) the participants in the imbPST condition knew that they were receiving a treatment, and so may have experienced a placebo response. Also, in spite of using a random treatment assignment method, there was an unequal distribution of males and females in the two conditions such that the imbPST condition had a significantly greater number of females than the NCT condition. Although analyses by gender did not show a differential effect, it is possible that the treatment effect observed for imbPST was influenced by the gender difference between groups. Also, we did not collect information on prior history of depression or prior treatment. Chronic depression or treatment-resistant depression could have exerted uncontrolled effects on the study results, such as the primary findings of differences between groups, and the relatively low rate of clinically significant improvement. We also did not assess the sample for comorbid psychiatric conditions and therefore we cannot rule out the possible effect of these disorders on the outcomes. Likewise, pleasant event scheduling is an embedded component in the imbPST program. Pleasant event scheduling may have accounted for the observed treatment effects independently of specific problem solving and we cannot rule this out as the main therapeutic mechanism.

Also, a 7-week study might be considered a relatively short time to evaluate the efficacy of the intervention. A recent meta-analysis on computerized interventions conducted by Warmerdam, van Straten, Twisk, Riper, and Cuijpers (2008) and another study by van Straten, Cuijpers, and Smits (2008) have highlighted that many participants have shown similar rapid improvement within the first 5 weeks of treatment. A similar study conducted by Meyer et al. (2009) observed lasting therapeutic effects for patients receiving a fewer number of sessions. Finally, the studies conducted by Van Straten et al. (2008) and by Warmerdam et al. (2008) showed that treatments with less than 8 sessions showed similar treatment effects to those studies that have 8 or more sessions. Specifically regarding our face-to-face PST studies, Oxman et al. (2008) found the significant improvements were maintained or continued to improve over a 6-month follow-up period, and Arean, Hegel, Vannoy, Fan, and Unuzter (2008) found the difference between PST and usual care held up over 12 months. Nonetheless, we did not follow the participants over the long term and cannot say whether the observed treatment effects would hold up over time. Also, in spite of clinically significant improvements on the self-report measures, a substantial number of imbPST participants continued to meet diagnostic criteria for major depression (76%), suggesting that the majority were only partial responders to the treatment and therefore were vulnerable to relapse.

Another major limitation of the study consists of the amount of interaction that participants had with the study personnel. Although we made efforts to have minimal contact with participants (i.e., less than 3 minutes per session), direct contact (e.g., meeting with study staff during the assessments, and meeting with professional mental health providers for safety checks) and indirect contacts (e.g., receiving emails to remind them about appointments) should be considered as confounding factors that could have caused the treatment outcomes to be much more positive than would have occurred in a completely stand-alone trial without contact with research staff.

Conclusions

This randomized controlled trial provides support for the effectiveness of imbPST as an intervention based on PST to treat depression in adults. Studies to date also show that the program is rated highly for usability and for therapeutic alliance, suggesting that it is engaging of the user and therefore well suited for self-administration. The extensive use of video and audio also minimizes demands on those who may have lower reading abilities or minimal education. Further evaluation of the imbPST program should include longer follow-up assessment periods with larger and more heterogeneous patient samples. An Internet version of imbPST is currently being developed, which will allow us to evaluate its effects under real-world conditions, as well as to investigate the moderators and mechanisms of its effects to inform further refinements.