Key Points
Question
Is a mobile intervention platform composed of a suite of simple-to-use apps, supported by brief coaching, efficacious for treating depression and anxiety among primary care patients?
Findings
In this randomized clinical trial of 146 patients with depression and anxiety, a mobile platform achieved greater reductions in depression and anxiety and higher odds of recovery compared with treatment-as-usual wait list control individuals, and effects were sustained at follow-up. Engagement with apps was high throughout the intervention period.
Meaning
Results support the efficacy of a platform approach to mobile intervention using apps designed to fit into the fabric of users’ lives for treating patients with depression and anxiety in primary care.
Abstract
Importance
Depression and anxiety are common and disabling. Primary care is the de facto site for treating these mental health problems but is typically underresourced to meet the burden of these demands.
Objective
To evaluate the efficacy of a mobile intervention platform, IntelliCare, for addressing depression and anxiety among primary care patients.
Design, Setting, and Participants
Two-arm randomized clinical trial at internal medicine clinics at the University of Arkansas for Medical Sciences. Adult primary care patients (N = 146) who screened positive for depression on the Patient Health Questionnaire-8 (PHQ; score ≥ 10) or anxiety on the Generalized Anxiety Disorder-7 (GAD-7; score ≥ 8) were recruited between July 17, 2018, and December 14, 2018.
Interventions
The coach-supported platform composed of a suite of apps, was delivered over 8 weeks. Wait list control participants received treatment as usual for 8 weeks, then the mobile platform.
Main Outcomes and Measures
Primary outcomes were changes in depression (PHQ-9) and anxiety (GAD-7) during the intervention period. Secondary outcomes were differences in the proportion of patients who achieved recovery (PHQ-9/GAD-7 <5 or 50% improvement from baseline), sustainment of intervention effects during 2-month follow-up, and app use during the intervention period.
Results
One hundred forty-six patients were included (119 of 146 were women [81.5%]; mean [SD] age, 42.3 [13.8] years). Of the 146 patients, 122 (83.6%) were diagnosed as having depression and 131 (89.7%) were diagnosed as having anxiety. A greater proportion of intervention vs wait list control participants achieved recovery from depression (n = 38 of 64 [59%] vs n = 18 of 58 [31%]; odds ratio, 3.25; 95% CI, 1.54-6.86) and anxiety (n = 37 of 65 [57%] vs n = 25 of 66 [38%]; odds ratio, 2.17; 95% CI, 1.08-4.36). Sustained effects were observed for depression (slope, 0.01; 95% CI, –0.09 to 0.10; P = .92) and anxiety scores (slope, 0.02; 95% CI, –0.08 to 0.12; P = .67) during follow-up. App use was high, with a median of 93 and 98 sessions among participants with depression and anxiety, respectively.
Conclusions and Relevance
In this trial, a mobile intervention app was effective for depression and anxiety among primary care patients. Findings also support designing digital mental health interventions as platforms containing simple, brief apps that can be bundled by users to meet their needs.
Trial Registration
ClinicalTrials.gov Identifier: NCT03500536.
This randomized clinical trial evaluates the efficacy of a mobile intervention platform, IntelliCare, for addressing depression and anxiety among primary care patients
Introduction
Depression and anxiety are prevalent, impairing, and costly health problems, most often treated in primary care.1,2,3 The US Preventive Services Task Force recommends routine depression screening in primary care4; however, without additional assistance to support treatment, increased identification provides no benefit.5 Unfortunately, demand for mental health services outweighs workforce capacity and resources to meet these needs. Digital mental health interventions (DMHIs), using Internet-based and mobile tools, can be cost-effective and have potential to improve capacity of health care settings to address mental health problems.6 However, implementation in primary care has often failed, primarily because patients do not use the tools.7
Computer-delivered DMHIs are primarily based on evidence-based therapies, rely heavily on psychoeducation, and typically require use for 30 to 45 minutes per week for 6 or more weeks. Despite good efficacy in randomized clinical trials (RCTs),8 real-world engagement is challenging.7,9,10 In contrast to computers, people tend to use smartphone apps for a single purpose, in short bursts of time, sometimes frequently throughout the day.11,12 Thus, app-based DMHIs often focus on 1 to 2 psychological strategies and more on actions than psychoeducation.13 App-based DMHIs have demonstrated efficacy in RCTs,14,15 are increasingly used by the public, and produce reasonable early engagement, but adherence drops dramatically over the first 2 weeks.13
App-based DMHIs may have greater success if they accommodate app use conventions and overcome 2 major challenges for delivering mental health interventions. First, the many effective behavioral strategies may appeal differentially to different people. Second, people do not tend to use mental health apps for long periods. This may be because people seek the novelty of new apps16 or have acquired the app’s benefit and no longer find it useful. Thus, a single app approach for a mental health condition may be less effective.17
The IntelliCare platform was designed to address these challenges. Rather than a single app, the platform contains multiple apps, each of which is brief, simple, and supports a single clinical target. For example, the Daily Feats app targets goal-setting; the sole activity is setting and monitoring completion of daily goals, with scaffolding to increase complexity. Because depression and anxiety are highly comorbid1,2 and respond to similar psychological strategies,18 the platform was designed to address both conditions. This mobile platform has been evaluated in field trials, the first of which assessed symptom changes over 8 weeks and the second of which tested 2 engagement strategies.19,20 In both trials, the platform produced significant reductions in depression and anxiety19,20 and consistent engagement.16,21
The purpose of this RCT was to evaluate the efficacy of the platform for the treatment of depression and anxiety in primary care patients. We hypothesized that individuals randomized to receive the mobile platform would report greater reductions in depression or anxiety symptoms compared with treatment-as-usual wait list control individuals (WLC) during the 2-month intervention period (primary outcome) and have a higher proportion of patients achieve recovery from depression or anxiety (secondary outcome). Secondary outcomes also included evaluating sustainment of effects over follow-up among individuals in the condition and engagement metrics. To our knowledge, this is the first evaluation of an app platform for the treatment of a mental health condition.
Methods
Design
This RCT compared IntelliCare with treatment as usual WLC for depression and anxiety. The University of Arkansas for Medical Sciences (UAMS) institutional review board approved the study. A data safety monitoring board provided oversight. Supplement 1 provides the trial protocol.
Participants
The trial protocol in Supplement 1 details entry criteria. In brief, adult UAMS primary care patients were eligible if they had a compatible smartphone (Android or Apple) and elevated symptoms of depression or anxiety at screening (score ≥10 on the Patient Health Questionnaire-8 [PHQ-8]22 or ≥8 on the Generalized Anxiety Disorder-7 scale [GAD-7]23). Individuals were excluded, among others, if they were acutely suicidal,24 had a diagnosis for which the IntelliCare study might not be appropriate, and were currently in or scheduled to initiate psychotherapy to avoid treatment interference. Psychotropic medication was permitted if dose was stable over the past 2 weeks to eliminate medication placebo effects.
Procedure
Participants were recruited between July 17, 2018, and December 14, 2018, from UAMS internal medicine clinics on the main medical center campus and surrounding communities. These clinics serve a diverse, often low-income population. All clinics provided routine depression screening; some had on-site integrated behavioral health resources. Participants were referred by a clinician or responded to outreach methods. Recruitment strategies have been described25 and are in the protocol in Supplement 1.
Interested individuals completed an online screener; those whose results indicated potential eligibility were automatically invited to provide online written informed consent. After medical record review by the research team, individuals who remained potentially eligible were invited to complete an online baseline assessment.
Following randomization, participants in the intervention condition began IntelliCare via an onboarding telephone call with their assigned coach. Wait list control participants were informed they would have access to IntelliCare after an 8-week delay. All participants were prompted to complete assessments every 4 weeks for 16 weeks, administered online and independent of the intervention to mask outcomes to coaches. Participants could receive up to $100 for completing the research assessments. Data collection ended on April 17, 2019.
Randomization
An independent statistician created a computer-generated randomization sequence with a 1-to-1 ratio in randomly permutated blocks of 4 and 6. Randomization assignment was concealed until after entry criteria were confirmed.
Intervention Conditions
IntelliCare
The intervention was delivered over 8 weeks using the coach-supported IntelliCare platform. IntelliCare is a suite of mobile apps available on iPhone and Android operating systems. The Hub app organized the user experience by supporting access to clinically focused IntelliCare apps, providing a library of psychoeducational material, and administering a weekly symptom assessment.19 In this study, participants received access to 5 clinically focused apps (described in the protocol in Supplement 1).19 Each week, a coach recommended a new app to download and try, based on the participant’s preferences and a recommendation protocol. Participants were encouraged to try the newly recommended app but could download any app at any time and use or discontinue apps as preferred.
Coaches were 2 bachelor’s degree–level individuals who received training in the coaching manual26 and weekly supervision with a clinical psychologist. Coaching was delivered primarily via short message service text messaging; coaches typically initiated 2 contacts per week and responded to participants’ messages. Participants also received a welcome packet via email (developed with health literacy experts), 1 onboarding telephone call (approximately 30-45 minutes), and the offer for an optional midtreatment call (approximately 10-15 minutes). Further details are described in the protocol in Supplement 1.
Coaches managed participants using an online dashboard, which provided information on each participant’s app use and weekly symptom assessment score and enabled sending and receiving text messages. Apps remained available to participants after treatment ended.
Wait List Control
The WLC allowed participants to continue treatment-as-usual through primary care. Following the 8-week delay, WLC individuals received the coached intervention.
Outcome Assessments
Primary outcomes (PHQ-927 and GAD-723) were measured at baseline and each follow-up assessment. App use metrics were captured automatically and stored on a secure server. App use was defined using 3 commonly used metrics20,28: number of app sessions, number of days the intervention was used, and time to last use (ie, number of days between the first and last launch) during the treatment period. An app session was defined as a user-initiated action or event after 5 minutes of no activity. Engagement with coaching was defined as number of messages sent by coaches and participants.
Analyses
Assuming 80% power and α = .05, a medium effect size of 0.5 between IntelliCare and WLC could be expected to be detected with 128 participants (64 per arm). Eighteen additional participants enrolled before recruitment procedures closed. All analyses were performed at a 2-sided type 1 error rate of .05 in SAS, version 9.4 (SAS Institute Inc).
Analyses followed intention to treat. Five participants did not have any follow-up data during the first phase of the study, evenly split between study arms (IntelliCare, 3; WLC, 2). No evidence indicated these participants were other than missing at random; hence, multiple imputation methods were used to impute follow-up data for these participants.
Primary outcomes were analyzed separately for participants who screened positive for depression and for anxiety to assess change in depression and anxiety, respectively. General linear mixed models were fit for PHQ-9 scores among those with depression and GAD-7 scores among those with anxiety for each imputed data set. Results were combined for those analyses to present valid inferences for the effect of the intervention over time. Primary analyses were unadjusted; secondary analyses adjusted for age, sex, and race.
Secondary outcome analyses were conducted. Univariable logistic regression models were fit to determine whether there was an association between study condition and recovery at the postintervention point, defined as PHQ-9/GAD-7 scores less than 5 or 50% reduction from baseline; additional models were fit adjusting for age, sex, and race. To assess sustained response between posttreatment and 16-week follow-up (among only those in the IntelliCare condition), general linear mixed models were fit for PHQ-9 among those with depression and GAD-7 among those with anxiety; models did not use imputed data. Correlations were used to assess the association between app use metrics and outcomes and engagement with coaching and outcomes at the postintervention point. Because these were not planned comparisons, only confidence intervals are reported.29
Results
Participants
Figure 1 presents the CONSORT diagram. Of 435 individuals screened, 146 participants (34%) were enrolled and randomized. At screening, 122 participants presented with depression (64 IntelliCare; 58 WLC) and 131 presented with anxiety (65 IntelliCare; 66 WLC); most (n = 107; 73%) had comorbid depression and anxiety. Table 1 shows pretreatment characteristics of the sample and 2 study conditions among the subsets with depression or anxiety. The lost-to-follow-up rate was low, with 141 of 146 participants (96.6%) completing at least 1 follow-up assessment.
Figure 1. CONSORT Flow Diagram.
Table 1. Baseline Demographics of the Complete Sample and by Intervention Condition.
| Characteristic | No. (%) | ||||
|---|---|---|---|---|---|
| Complete sample | Depression | Anxiety | |||
| Mobile app | Wait list | Mobile app | Wait list | ||
| No. | 146 | 64 | 58 | 65 | 66 |
| Age, mean (SD), y | 42.3 (13.8) | 39.7 (12.2) | 45.3 (14.9) | 41.0 (13.1) | 43.5 (14.2) |
| PHQ-9 at baseline,a median (IQR) [range] | 12 (9-16) [0-27] | 13 (10-18) [4-24] | 13 (10-17) [4-27] | NA | NA |
| GAD-7 at baseline,a median (IQR) [range] | 10 (7-15) [1-21] | NA | NA | 12 (8-15) [2-21] | 10 (8-15) [4-27] |
| Female | 119 (82) | 51 (80) | 49 (84) | 50 (77) | 58 (88) |
| Race/ethnicity (multiple) | |||||
| American Indian | 4 (3) | 0 | 4 (7) | 0 | 4 (6) |
| Asian | 2 (1) | 2 (3) | 0 | 1 (2) | 0 |
| Black/African American | 47 (32) | 24 (38) | 18 (31) | 23 (35) | 18 (27) |
| White | 95 (65) | 34 (53) | 40 (69) | 38 (58) | 49 (74) |
| Other/NA | 5 (4) | 3 (5) | 2 (3) | 3 (5) | 2 (3) |
| Hispanic | 10 (7) | 5 (8) | 2 (3) | 5 (8) | 5 (8) |
| Marital status | |||||
| Single | 42 (29) | 17 (27) | 19 (33) | 19 (29) | 20 (30) |
| Married/domestic partner | 57 (39) | 26 (41) | 19 (33) | 28 (43) | 22 (33) |
| Separated/divorced | 27 (18) | 11 (17) | 12 (21) | 10 (15) | 14 (21) |
| Widowed | 5 (3) | 3 (5) | 2 (3) | 3 (5) | 2 (3) |
| Living with significant other | 15 (10) | 7 (11) | 6 (10) | 5 (8) | 8 (12) |
| Education | |||||
| High school or less | 13 (9) | 7 (11) | 5 (9) | 6 (9) | 5 (8) |
| College/associate degree | 59 (41) | 27 (42) | 25 (44) | 28 (43) | 28 (43) |
| Bachelor’s degree | 43 (30) | 22 (34) | 14 (25) | 20 (31) | 17 (26) |
| Master’s degree or higher | 30 (21) | 8 (13) | 13 (23) | 11 (17) | 15 (23) |
| Employment | |||||
| Employed | 107 (73) | 44 (69) | 42 (72) | 44 (68) | 53 (80) |
| Unemployed | 12 (8) | 7 (11) | 5 (9) | 7 (11) | 4 (6) |
| Disability/retired/other | 27 (18) | 13 (20) | 11 (19) | 14 (22) | 9 (14) |
| Household income, $1K; median (IQR) [range] | 42 (29-80) [0-342] | 40 (24-71) [0-164] | 42 (28-80) [1.4-230] | 40 (25-71) [0-200] | 43 (29-80) [3-342] |
| Insurance (multiple) | |||||
| None | 4 (3) | 2 (3) | 1 (2) | 3 (5) | 1 (2) |
| Private | 109 (75) | 46 (72) | 41 (71) | 45 (69) | 51 (77) |
| Medicaid | 27 (18) | 15 (23) | 10 (17) | 9 (14) | 8 (12) |
| Medicare | 19 (13) | 7 (11) | 10 (17) | 31 (48) | 7 (11) |
| Mental health history | |||||
| MDD/depression | 69 (48) | 34 (53) | 28 (49) | 31 (48) | 30 (46) |
| Anxiety disorder | 70 (48) | 31 (48) | 26 (45) | 35 (54) | 30 (45) |
| OCD | 8 (6) | 3 (5) | 4 (7) | 3 (5) | 5 (8) |
| PTSD | 18 (12) | 11 (17) | 5 (9) | 10 (15) | 6 (9) |
| Current psychotropic medication | 82 (56) | 36 (56) | 31 (53) | 37 (57) | 37 (56) |
Abbreviations: GAD-7, Generalized Anxiety Disorder-7; IQR, interquartile range; MDD, major depressive disorder; NA, not applicable; OCD, obsessive-compulsive disorder; PHQ-9, Patient Health Questionnaire-9; PTSD, posttraumatic stress disorder.
PHQ-8 and GAD-7 scores at screening were used to determine eligibility but not used for outcome evaluation.
Primary Outcome: Changes in Depression or Anxiety
Figure 2A and B depict the least square means (LSM) of outcomes over time for depression and anxiety, with imputed data from 5 multiply imputed data sets. eTable 1 in Supplement 2 provides model parameter estimates. There was a significant group-by-time effect over 8 weeks (F2,33 316 = 15.9; P < .001), meaning those assigned to the mobile platform experienced a faster rate of improvement than WLC. The LSM difference in depression scores between the mobile platform and WLC at week 4 was 2.88 (SE = 0.82; Cohen d effect size [ES] = 0.52) and at week 8 was 4.33 (SE = 0.83; d ES = 0.78). The LSM difference in anxiety scores at week 4 was 2.38 (SE = 0.77; d ES = 0.47) and at week 8 was 3.19 (SE = 0.76; d ES = 0.64).
Figure 2. Change in Least Square Means Scores.
A, Change in depression least square mean scores by intervention condition over the intervention period (with imputed data). B, Change in anxiety least square mean scores by intervention condition over the intervention period (with imputed data).
After adjusting for age, sex, and race, the effect of group by treatment interaction remained significant for depression (F2,34 315 = 15.3; P < .001). The LSM difference in depression scores at week 4 was 2.91 (SE = 0.83; ESd = 0.43) and at week 8 was 4.37 (SE = 0.83; d ES = 0.64). Similarly, the group by treatment interaction remained significant for anxiety (F2,3228.4 = 11.9; P < .001). The LSM difference in anxiety scores at week 4 was 2.51 (SE = 0.78; d ES = 0.41) and at week 8 was 3.33 (SE = 0.76; d ES = 0.55).
Secondary Outcomes
Recovery at Postintervention Point
The rate of recovery from depression was 59.4% in IntelliCare and 31.0% in WLC. The rate of recovery from anxiety was 56.9% in IntelliCare and 37.9% in WLC. The odds of recovery for depression were 3.25 (95% CI, 1.54-6.86) times greater and for anxiety were 2.17 (95% CI, 1.08-4.36) times greater for IntelliCare compared with WLC. After adjusting for age, sex, and race, odds of recovery increased slightly to 3.42 (95% CI, 1.57-7.48) for depression and 2.24 (95% CI, 1.07-4.68) for anxiety.
Sustained Response at Follow-up
Among those randomized to IntelliCare, effects were sustained from posttreatment to 16-week follow-up. There was no evidence of a linear change in slope for change in depression scores (slope, 0.01; 95% CI, –0.09 to 0.10; P = .92), nor were the means different over time (F2,119= 0.02; P = .98). Similarly, there was no evidence of a linear change in slope for change in anxiety scores (slope, 0.02; 95% CI, –0.08 to 0.12; P = .67), nor were the means different over time (F2,118= 0.09; P = .92). Means and standard errors for the study conditions by depression and anxiety scores at each point are presented in Table 2, using observed data for ease of interpretation.
Table 2. Means and Standard Errors for Intervention Conditions at Each Time (Observed Data)a.
| Time | Depression (PHQ-9), mean (SE) | Anxiety (GAD-7), mean (SE) | ||
|---|---|---|---|---|
| Mobile app | Wait list | Mobile app | Wait list | |
| Baseline | 14.0 (5.0) | 13.6 (5.2) | 11.6 (4.6) | 11.2 (4.7) |
| Week | ||||
| 4 | 9.0 (5.0) | 11.9 (5.6) | 8.4 (4.6) | 10.7 (5.2) |
| 8 | 7.2 (5.5) | 11.4 (6.6) | 6.8 (5.1) | 9.8 (5.5) |
| 12 | 7.3 (5.9) | 8.0 (5.9) | 6.6 (5.3) | 7.0 (4.5) |
| 16 | 7.2 (5.6) | 7.0 (5.4) | 6.8 (5.6) | 6.0 (4.8) |
Abbreviations: GAD-7, Generalized Anxiety Disorder-7; PHQ-9, Patient Health Questionnaire 9.
Individuals in the IntelliCare condition received the intervention between baseline and week 8. Individuals in the wait list condition received the intervention after week 8.
App Use
Table 3 presents app use metrics for the full suite and each app. Neither app sessions (r, −0.03; 95% CI, −0.22 to 0.15), time to last use (r, −0.14; 95% CI, −0.31 to 0.05), nor days used (r, −0.05; 95% CI, −0.23 to 0.14) were strongly associated with changes in depression. Neither app sessions (r, 0.01; 95% CI, −0.17 to 0.19), time to last use (r, −0.04; 95% CI, −0.22 to 0.13)], nor days used (r, 0.02; 95% CI, −0.15 to 0.20) were strongly associated with changes in anxiety. eTable 2 in Supplement 2 shows correlations with outcomes for each app.
Table 3. App Use Metrics for the Full IntelliCare Platform and Each Individual App.
| Group | Mobile appa | Median (IQR) [range] | ||
|---|---|---|---|---|
| Sessions | Days used | Time to last use | ||
| Depression (n = 122) | Suite (all apps) | 93 (51-133) [0-333] | 27 (17-35) [0-50] | 43 (32-48) [0-56] |
| Hub | 20 (11-30) [0-102] | 12 (7-17) [0-39] | 40 (28-45) [0-56] | |
| Daily Feats | 26 (11-44) [0-94] | 17 (8-26) [0-50] | 36 (19-45) [0-53] | |
| Day-to-Day | 16 (6-25) [0-90] | 11 (5-16) [0-40] | 29 (9-41) [0-52] | |
| MyMantra | 9 (2-19) [0-59] | 6 (2-10) [0-37] | 15 (2-28) [0-50] | |
| Thought Challenger | 8 (3-16) [0-74] | 6 (3-12) [0-41] | 16 (6-29) [0-51] | |
| WorryKnot | 5 (0-11) [0-29] | 4 (0-8) [0-23] | 9 (0-19) [0-46] | |
| Anxiety (n = 131) | Suite (all apps) | 98 (46-146) [0-321] | 27 (16-35) [0-50] | 42 (32-48) [0-56] |
| Hub | 21 (11-30) [0-102] | 11 (7-16) [0-39] | 39 (28-45) [0-56] | |
| Daily Feats | 25 (10-44) [0-94] | 17 (6-26) [0-50] | 36 (11-44) [0-53] | |
| Day-to-Day | 17 (7-29) [0-90] | 11 (5-18) [0-40] | 29 (8-41) [0-52] | |
| MyMantra | 10 (2-22) [0-59] | 6 (1-11) [0-37] | 15 (0-28) [0-50] | |
| Thought Challenger | 8 (3-16) [0-53] | 6 (3-12) [0-30] | 16 (4-30) [0-50] | |
| WorryKnot | 5 (0-10) [0-29] | 4 (0-7) [0-23] | 8 (0-17) [0-50] | |
Abbreviation: IQR, interquartile range.
App recommendations occurred weekly. Daily Feats and Day-to-Day were recommended in the first 2 weeks (order determined by coach and participant) and therefore were likely on the phone longest. The remaining apps were recommended based on participant need and preference (with MyMantra and Thought Challenger typically recommended before WorryKnot). Because app use tends to be skewed, median, interquartile range (IQR) and range are reported.
At 8 weeks following treatment, 119 participants (81.5%) had some app use. For all participants, postintervention median time to last app use was 28 days (range, 0-212 days) and median days used was 7 (range, 0-102 days).
Engagement With Coaching
Among those with depression, coaches sent a mean of 32 messages (SD, 9; range, 7-61); participants sent a mean of 19 messages (SD, 12; range, 1-68). There was a negligible correlation between change in PHQ-9 and number of messages from coaches (r, −0.07; 95% CI, −0.25 to 0.13) or participants (r, −0.03; −0.22 to 0.16). Among those with anxiety, coaches sent a mean of 32 messages (SD, 10; range, 7-61); participants sent a mean of 19 messages (SD, 12; range, 1-68). The correlation between change in GAD-7 and number of messages from coaches (r, −0.01; 95% CI, −0.19 to 0.17) or participants (r, −0.02; 95% CI, −0.20 to 0.17) was not meaningful.
Adverse Events
There were no study-related adverse events.
Discussion
This is, to our knowledge, the first RCT of an app-based platform for the treatment of depression and anxiety and of any app-based mental health treatment among primary care patients. Individuals who received the mobile platform had a greater reduction in depression and anxiety symptoms compared with treatment-as-usual WLC individuals, and changes were sustained over 2-month follow-up. Although analysis of the treatment period for the WLC was not planned or performed, once these participants received treatment, their depression and anxiety scores dropped to levels similar to those in the treatment group, further strengthening the conclusion that IntelliCare is effective for depression and anxiety among primary care patients.
Effect sizes of 0.78 and 0.64 for depression and anxiety in this trial are in the range of effect sizes in meta-analyses of face-to-face psychotherapy (0.69 and 0.84, respectively)30,31 and compare favorably with effect sizes for app-based treatments (0.38 and 0.33, respectively)14,15 and web-based interventions evaluated in primary care (0.31 and 0.26, respectively).32,33
Digital mental health interventions have been championed as a solution to access barriers and high dropout rates for psychotherapy among primary care patients.34,35 However, engagement has proven challenging. An effectiveness trial of a web-based intervention found participants logged in for approximately half of the 8 sessions32; in 2 implementation trials of web-based interventions in English primary care settings, median uses were 1-2, with only 10% to 19% completing their intervention.7,9 While we are unaware of mental health app adherence data in primary care, the median number of mental health app sessions is 3 to 9 among active public users.13 In contrast, we found high app use, with a median of 93 to 98 app sessions across all apps over 8 weeks. Our findings may be because the platform approach matches how people use apps: participants could use or ignore apps as preferred, and the novelty of receiving new apps is likely engaging.16,21,36 The sample also comprised a group rarely represented in technology research: in contrast to past research, which has tended to attract those drawn to technology,33 participants were from a diverse, traditionally underserved community seeking help for depression and anxiety. Thus, findings suggest this service appealed to patients more generally and underscore the importance of designing DMHIs to match patients’ app use preferences.33,37,38 App use was not strongly associated with outcomes, consistent with much of the DMHI literature.16,39,40 The association between use and outcomes is complex. Clinically meaningful app engagement may be more important than raw quantity.21 Furthermore, people tend to use DMHIs less as their depression improves.41 Taken together, the relative high engagement suggests that platform approaches, such as IntelliCare, may be more acceptable to patients and likely produce better outcomes than disorder-focused single apps; however, this remains to be tested in a trial.
Results also inform potential for scale. Although routine screening has increased the identification of patients with mental health problems in primary care, this has not necessarily led to improved outcomes.5 Further, despite the success of collaborative care, which embeds behavioral health specialists in primary care,42,43 demand for such services frequently exceeds health care organizations’ capacity. Digital mental health intervention app platforms have potential to enhance behavioral health treatment effectiveness by improving clinicians’ efficiency while maintaining cost-effectiveness. Indeed, in this trial, coaches spent less than 1 hour on the telephone with participants, communicating primarily through text messages. Text messaging is simple, unobtrusive, acceptable to patients, and overcomes the often high rates of disengagement frequently seen in phone-based coaching.7,9
Limitations
Study limitations should be considered in interpreting results. Although previous studies20 of the mobile platform have shown maintenance of gains over 6 months,20 this trial did not have a follow-up period sufficient to replicate those findings. By using self-report depression and anxiety measures to align with screening/monitoring methods in primary care,44 findings may not generalize to mental health clinics where thorough clinical assessments are performed. Because 56% of the sample was receiving antidepressant medication, it is possible some of the within-treatment effects were owing to medication; however, these effects should be equivalent across treatment arms and do not affect the validity of the between-treatment effects. We also did not capture data on patients’ use of other apps during the intervention period. Finally, although the research was conducted in partnership with UAMS clinics, study team members delivered the intervention or assessments (performed by different individuals to prevent coaches from knowing outcomes). Future work will be required to design and evaluate implementation strategies that integrate DMHI app platforms into clinic workflows. Further, design work to support translation to adolescent populations and care settings may be beneficial given the prevalence of mental health problems and unmet treatment need among youths.45
Conclusions
The mobile app is effective for depression and anxiety among primary care patients, which represents an important innovation for alleviating the burden of these mental health problems in this population. Findings support the movement of DMHIs from single apps for depression or anxiety to platform approaches containing multiple brief apps that patients can bundle to meet their needs and fit into the fabric of their lives.
Trial Protocol
eTable 1. GLM model parameter estimates comparing treatment conditions on changes in the primary outcomes among individuals with depression or anxiety
eTable 2. Correlations and 95% confidence intervals between individual app use and outcome
Data Sharing Statement.
References
- 1.Ruscio AM, Hallion LS, Lim CCW, et al. Cross-sectional comparison of the epidemiology of DSM-5 generalized anxiety disorder across the globe. JAMA Psychiatry. 2017;74(5):465-475. doi: 10.1001/jamapsychiatry.2017.0056 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Hasin DS, Sarvet AL, Meyers JL, et al. Epidemiology of adult DSM-5 major depressive disorder and its specifiers in the United States. JAMA Psychiatry. 2018;75(4):336-346. doi: 10.1001/jamapsychiatry.2017.4602 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Olfson M, Blanco C, Marcus SC. Treatment of adult depression in the United States. JAMA Intern Med. 2016;176(10):1482-1491. doi: 10.1001/jamainternmed.2016.5057 [DOI] [PubMed] [Google Scholar]
- 4.Siu AL, Bibbins-Domingo K, Grossman DC, et al. ; US Preventive Services Task Force (USPSTF) . Screening for depression in adults: US Preventive Services Task Force Recommendation Statement. JAMA. 2016;315(4):380-387. doi: 10.1001/jama.2015.18392 [DOI] [PubMed] [Google Scholar]
- 5.O’Connor EA, Whitlock EP, Beil TL, Gaynes BN. Screening for depression in adult patients in primary care settings: a systematic evidence review. Ann Intern Med. 2009;151(11):793-803. doi: 10.7326/0003-4819-151-11-200912010-00007 [DOI] [PubMed] [Google Scholar]
- 6.McCrone P, Knapp M, Proudfoot J, et al. Cost-effectiveness of computerised cognitive-behavioural therapy for anxiety and depression in primary care: randomised controlled trial. Br J Psychiatry. 2004;185:55-62. doi: 10.1192/bjp.185.1.55 [DOI] [PubMed] [Google Scholar]
- 7.Gilbody S, Littlewood E, Hewitt C, et al. ; REEACT Team . Computerised cognitive behaviour therapy (cCBT) as treatment for depression in primary care (REEACT trial): large scale pragmatic randomised controlled trial. BMJ. 2015;351:h5627. doi: 10.1136/bmj.h5627 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Andrews G, Basu A, Cuijpers P, et al. Computer therapy for the anxiety and depression disorders is effective, acceptable and practical health care: an updated meta-analysis. J Anxiety Disord. 2018;55:70-78. doi: 10.1016/j.janxdis.2018.01.001 [DOI] [PubMed] [Google Scholar]
- 9.Gilbody S, Brabyn S, Lovell K, et al. ; REEACT collaborative . Telephone-supported computerised cognitive-behavioural therapy: REEACT-2 large-scale pragmatic randomised controlled trial. Br J Psychiatry. 2017;210(5):362-367. doi: 10.1192/bjp.bp.116.192435 [DOI] [PubMed] [Google Scholar]
- 10.Christensen H, Griffiths KM, Farrer L. Adherence in internet interventions for anxiety and depression. J Med Internet Res. 2009;11(2):e13. doi: 10.2196/jmir.1194 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Vaish R, Wyngarden K, Chen J, Cheung B, Bernstein MS Twitch crowdsourcing: crowd contributions in short bursts of time. Presented at: SIGCHI Conference on Human Factors in Computing Systems; 2014; Toronto, ON, Canada. [Google Scholar]
- 12.Andrews S, Ellis DA, Shaw H, Piwek L. Beyond self-report: tools to compare estimated and real-world smartphone use. PLoS One. 2015;10(10):e0139004. doi: 10.1371/journal.pone.0139004 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Baumel A, Muench F, Edan S, Kane JM. Objective user engagement with mental health apps: systematic search and panel-based usage analysis. J Med Internet Res. 2019;21(9):e14567. doi: 10.2196/14567 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Firth J, Torous J, Nicholas J, et al. The efficacy of smartphone-based mental health interventions for depressive symptoms: a meta-analysis of randomized controlled trials. World Psychiatry. 2017;16(3):287-298. doi: 10.1002/wps.20472 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Firth J, Torous J, Nicholas J, Carney R, Rosenbaum S, Sarris J. Can smartphone mental health interventions reduce symptoms of anxiety? a meta-analysis of randomized controlled trials. J Affect Disord. 2017;218:15-22. doi: 10.1016/j.jad.2017.04.046 [DOI] [PubMed] [Google Scholar]
- 16.Kwasny MJ, Schueller SM, Lattie E, Gray EL, Mohr DC. Exploring the use of multiple mental health apps within a platform: secondary analysis of the IntelliCare Field Trial. JMIR Ment Health. 2019;6(3):e11572. doi: 10.2196/11572 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Chiauzzi E, Newell A. Mental health apps in psychiatric treatment: a patient perspective on real world technology usage. JMIR Ment Health. 2019;6(4):e12292. doi: 10.2196/12292 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Barlow DH, Farchione TJ, Bullis JR, et al. The unified protocol for transdiagnostic treatment of emotional disorders compared with diagnosis-specific protocols for anxiety disorders: a randomized clinical trial. JAMA Psychiatry. 2017;74(9):875-884. doi: 10.1001/jamapsychiatry.2017.2164 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Mohr DC, Tomasino KN, Lattie EG, et al. IntelliCare: an eclectic, skills-based app suite for the treatment of depression and anxiety. J Med Internet Res. 2017;19(1):e10. doi: 10.2196/jmir.6645 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Mohr DC, Schueller SM, Tomasino KN, et al. Comparison of the effects of coaching and receipt of app recommendations on depression, anxiety, and engagement in the IntelliCare platform: factorial randomized controlled trial. J Med Internet Res. 2019;21(8):e13609. doi: 10.2196/13609 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Zhang R, Nicholas J, Knapp AA, et al. Clinically meaningful use of mental health apps and its effects on depression: mixed methods study. J Med Internet Res. 2019;21(12):e15644. doi: 10.2196/15644 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Kroenke K, Strine TW, Spitzer RL, Williams JB, Berry JT, Mokdad AH. The PHQ-8 as a measure of current depression in the general population. J Affect Disord. 2009;114(1-3):163-173. doi: 10.1016/j.jad.2008.06.026 [DOI] [PubMed] [Google Scholar]
- 23.Spitzer RL, Kroenke K, Williams JB, Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med. 2006;166(10):1092-1097. doi: 10.1001/archinte.166.10.1092 [DOI] [PubMed] [Google Scholar]
- 24.Posner K, Brown GK, Stanley B, et al. The Columbia-Suicide Severity Rating Scale: initial validity and internal consistency findings from three multisite studies with adolescents and adults. Am J Psychiatry. 2011;168(12):1266-1277. doi: 10.1176/appi.ajp.2011.10111704 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Graham AK, Greene CJ, Powell T, et al. Lessons learned from service design of a trial of a digital mental health service: informing implementation in primary care clinics. Transl Behav Med. In press. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Tomasino KN, Noth F, Bardsley LR, Lattie EG, Mohr DC IntelliCare Study Coaching Manual. Northwestern University: DigitalHub. Published 2016. Accessed April 13, 2020. https://digitalhub.northwestern.edu/files/00fa4294-5b9f-4afc-897a-7fffceae8f3f
- 27.Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med. 2001;16(9):606-613. doi: 10.1046/j.1525-1497.2001.016009606.x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Pham Q, Graham G, Carrion C, et al. A library of analytic indicators to evaluate effective engagement with consumer mhealth apps for chronic conditions: scoping review. JMIR Mhealth Uhealth. 2019;7(1):e11941. doi: 10.2196/11941 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Kraemer HC. Is it time to ban the P value? JAMA Psychiatry. 2019. doi: 10.1001/jamapsychiatry.2019.1965 [DOI] [PubMed] [Google Scholar]
- 30.Cuijpers P, Van Straten A, Warmerdam L, Smits N. Characteristics of effective psychological treatments of depression: a metaregression analysis. Psychother Res. 2008;18(2):225-236. doi: 10.1080/10503300701442027 [DOI] [PubMed] [Google Scholar]
- 31.Cuijpers P, Sijbrandij M, Koole S, Huibers M, Berking M, Andersson G. Psychological treatment of generalized anxiety disorder: a meta-analysis. Clin Psychol Rev. 2014;34(2):130-140. doi: 10.1016/j.cpr.2014.01.002 [DOI] [PubMed] [Google Scholar]
- 32.Rollman BL, Herbeck Belnap B, Abebe KZ, et al. Effectiveness of online collaborative care for treating mood and anxiety disorders in primary care: a randomized clinical trial. JAMA Psychiatry. 2018;75(1):56-64. doi: 10.1001/jamapsychiatry.2017.3379 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Mohr DC, Weingardt KR, Reddy M, Schueller SM. Three problems with current digital mental health research . . . and three things we can do about them. Psychiatr Serv. 2017;68(5):427-429. doi: 10.1176/appi.ps.201600541 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Mohr DC, Ho J, Duffecy J, et al. Perceived barriers to psychological treatments and their relationship to depression. J Clin Psychol. 2010;66(4):394-409. doi: 10.1002/jclp.20659 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Mohr DC, Hart SL, Howard I, et al. Barriers to psychotherapy among depressed and nondepressed primary care patients. Ann Behav Med. 2006;32(3):254-258. doi: 10.1207/s15324796abm3203_12 [DOI] [PubMed] [Google Scholar]
- 36.Graham AK, Lattie EG, Mohr DC. Experimental therapeutics for digital mental health. JAMA Psychiatry. 2019. doi: 10.1001/jamapsychiatry.2019.2075 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Mohr DC, Riper H, Schueller SM. A solution-focused research approach to achieve an implementable revolution in digital mental health. JAMA Psychiatry. 2018;75(2):113-114. doi: 10.1001/jamapsychiatry.2017.3838 [DOI] [PubMed] [Google Scholar]
- 38.Mohr DC, Lyon AR, Lattie EG, Reddy M, Schueller SM. Accelerating digital mental health research from early design and creation to successful implementation and sustainment. J Med Internet Res. 2017;19(5):e153. doi: 10.2196/jmir.7725 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Donkin L, Christensen H, Naismith SL, Neal B, Hickie IB, Glozier N. A systematic review of the impact of adherence on the effectiveness of e-therapies. J Med Internet Res. 2011;13(3):e52. doi: 10.2196/jmir.1772 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Fuhr K, Schröder J, Berger T, et al. The association between adherence and outcome in an Internet intervention for depression. J Affect Disord. 2018;229:443-449. doi: 10.1016/j.jad.2017.12.028 [DOI] [PubMed] [Google Scholar]
- 41.Mohr DC, Duffecy J, Jin L, et al. Multimodal e-mental health treatment for depression: a feasibility trial. J Med Internet Res. 2010;12(5):e48. doi: 10.2196/jmir.1370 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Gilbody S, Bower P, Fletcher J, Richards D, Sutton AJ. Collaborative care for depression: a cumulative meta-analysis and review of longer-term outcomes. Arch Intern Med. 2006;166(21):2314-2321. doi: 10.1001/archinte.166.21.2314 [DOI] [PubMed] [Google Scholar]
- 43.Muntingh AD, van der Feltz-Cornelis CM, van Marwijk HW, Spinhoven P, van Balkom AJ. Collaborative care for anxiety disorders in primary care: a systematic review and meta-analysis. BMC Fam Pract. 2016;17:62. doi: 10.1186/s12875-016-0466-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.The MacArthur Foundation’s Initiative on Depression and Primary Care The MacArthur Initiative on Depression and Primary Care at Dartmouth and Duke: Depression Management Toolkit. Hanover, NH: Dartmouth;2004. [Google Scholar]
- 45.Whitney DG, Peterson MD. US national and state-level prevalence of mental health disorders and disparities of mental health care use in children. JAMA Pediatr. 2019;173(4):389-391. doi: 10.1001/jamapediatrics.2018.5399 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Trial Protocol
eTable 1. GLM model parameter estimates comparing treatment conditions on changes in the primary outcomes among individuals with depression or anxiety
eTable 2. Correlations and 95% confidence intervals between individual app use and outcome
Data Sharing Statement.
