Abstract
Background
The COVID-19 pandemic created an unprecedented need for mental health services that can be remotely delivered. Digital mental health services that offer personalized care recommendations hold promise to efficiently expand service, but evidence of the effectiveness of digitally delivered mental health care in real-world settings remains limited.
Methods
A retrospective cohort of adults (N = 1,852) receiving care through a digital mental health platform with elevated depressive symptoms during the COVID-19 pandemic was analyzed to estimate changes in subjective well-being and clinical improvement in depressive symptoms (using the World Health Organization-Five [WHO-5] Well-Being Index), as well as compare the relative effectiveness and cost of different care utilization patterns.
Results
The average improvement in WHO-5 score was 10.1 points (CI: 9.3–10.9, p<0.001) at follow-up, which constituted a medium effect size (d = 0.73). The odds of clinical improvement in depressive symptoms were significantly greater among those who utilized telecoaching (aOR = 2.45, 95%CI: 1.91–3.15, p < .001), teletherapy (aOR = 2.01, 95%CI: 1.57–2.57, p < .001), and both services (aOR = 2.28, 95%CI: 1.67–3.11, p < .001) compared to those who only utilized assessments, adjusting for baseline WHO-5 score, age, sex, and number of days between baseline and follow-up assessments. The average estimated cost of care for telecoaching was $124 per individual, which was significantly less than teletherapy ($413) or both services ($559).
Conclusion
Digitally delivered care with a therapist and/or coach was effective in improving subjective well-being and clinical improvement in depressive symptoms. Although clinical outcomes were similar across utilization patterns, the cost of care was lowest among those utilizing telecoaching.
Introduction
The prevalence of mental health conditions continues to outpace the provision of timely, affordable, and evidence-based mental health services in the United States and globally [1, 2]. More than 264 million people around the world were affected by depression in 2017 and the rate of depressive symptoms has continued to rise dramatically since the onset of the COVID-19 pandemic [3, 4]. The alarming gap between the number of individuals who need care and the supply of providers in the current behavioral health workforce has resulted in a large unmet need for mental health treatment [5]. The lack of face-to-face treatment options available during the pandemic further exacerbated existing barriers to care, such as cost, stigma, and an overreliance on one-on-one therapy with licensed providers [6]. Collectively, this has created an overwhelming need to expand and better allocate evidence-based mental health services to fill serious gaps in population-level health care access.
Digitally delivered mental health services that leverage videoconferencing technology have emerged as a scalable option to meet this demand [7]. Technology-enabled mental health care that delivers virtual one-on-one support appears to produce similar outcomes as in-person treatment [8, 9] and offers several promising solutions to enhance care. Digital platforms that collect information before treatment can use patient characteristics and preferences to identify and deliver the most effective yet least burdensome and least costly intervention from a range of care modalities, including psychotherapy with paraprofessionals or licensed therapists [10]. Research has shown that this initial allocation of services is both clinically and cost-effective [11–14] and given global behavioral health workforce gaps [5], connecting people for whom it is indicated with providers like certified professional coaches will help expand access to high-quality, affordable care [15, 16]. Considering these potential benefits, more research is warranted to investigate digital mental health services that leverage technology to offer personalized recommendations and deliver care with different types of providers.
In the present study, we examined clinical and cost outcomes from a digital platform that recommends care and delivers one-on-one virtual mental health services. The platform uses brief psychological assessments to recommend adults with symptoms of depression to telecoaching or teletherapy services. We analyzed data from participants who registered for care during the first year of the COVID-19 pandemic (March 2020—March 2021) and screened positive for depressive symptoms. We had three aims: 1) evaluate changes in subjective well-being and rates of clinical improvement from depressive symptoms overall and among the different utilization patterns, 2) investigate associations between utilization patterns and the likelihood of clinical improvement in depressive symptoms, and 3) compare estimated costs between utilization patterns.
Method
Design and participants
We analyzed retrospective de-identified data from participants who registered for mental health services through a digital platform (Modern Health, Inc., San Francisco, CA) during the COVID-19 pandemic (3/11/2020–3/11/2021). Eligible participants were: 18 years or older; had access to a smartphone, tablet, or computer; screened positive for depressive symptoms (World Health Organization-Five [WHO-5] Well-Being Index ≤ 28) at registration; and completed a follow-up assessment at least 14 days after registering. Western Clinical Group IRB reviewed this study and determined it to be exempt from IRB oversight.
Procedures
Eligible adults registered on the web or mobile application using a device (smartphone, tablet, or computer). As part of their registration, participants completed a baseline questionnaire, where they selected topics and symptoms of concern from five areas (emotional, professional, physical, social, and financial health), indicated their preferences for care modality, and completed the WHO-5. Participants were recommended a care plan based on their initial clinical acuity and care preferences. Though a care plan was recommended, participants could self-refer and utilize combinations of teletherapy and telecoaching, as well as digital resources and assessments. There were no prescribed number of teletherapy or telecoaching visits to complete and participants could utilize visits, digital resources, and assessments at their discretion within the limits of the plan offered by their employer. Participants who utilized teletherapy or telecoaching could message with their provider between sessions and could rate their satisfaction with the provider after each visit. Participants were prompted to voluntarily complete a follow-up assessment at least 14 days after registering through Modern Health’s secure platform.
Evidence-based digital mental health services
The platform offers several evidence-based modalities of care, including one-on-one care with mental health professionals. In this study, participants had access to teletherapy, telecoaching, or both. Below are descriptions of the teletherapy and telecoaching programs.
Teletherapy
The platform has a global network of high-quality providers who practice evidence-based care. Therapists were licensed providers with advanced degrees in clinical psychology or a related field (e.g., Doctor of Philosophy, Doctor of Psychology, Licensed Clinical Social Worker, Licensed Marriage and Family Therapist, Licensed Professional Counselor), with advanced training and practice in evidence-based care (e.g., CBT, Acceptance and Commitment Therapy [ACT], Dialectical Behavior Therapy, Interpersonal Therapy), and demonstrated alignment with short-term, evidence-based, culturally responsive, and ethical care. Therapy sessions lasted approximately 50 minutes each, and were most often scheduled to occur weekly, but could be spaced out according to the individual’s needs. Therapy visits were provided via videoconferencing through the digital platform. The number of therapy sessions an individual utilized could vary based on the number of covered sessions offered by their employer, their clinical needs, and their engagement with treatment.
Telecoaching
All coaches were certified by an International Coaching Federation (ICF) accredited program and screened and vetted to ensure they were trained in and offered evidence-based approaches (primarily non-clinical techniques that draw from the principles of CBT, ACT, Motivational Interviewing, and Mindfulness-Based Approaches). Coaching sessions lasted approximately 30 minutes each and were provided via videoconferencing. Coaches identified goals the participant wanted to work towards, used evidence-based principles like CBT or ACT to help the participant think about self-beliefs or behaviors impeding on those goals, and explored action plans for the participant to work towards desired outcomes. To ensure ongoing quality, clinical care managers collected and reviewed aggregate feedback, and offered ongoing training and case consultation to support all coaches and therapists.
Measures
Standardized and validated symptom questionnaires are administered to participants at the baseline assessment and throughout treatment. The WHO-5 questionnaire is the primary treatment outcome measurement [17].
Utilization pattern
We characterized utilization patterns by the types of services an individual had engaged over the study period. In this study, we classified individuals as having utilized: 1) assessments only, 2) telecoaching only (at least 1 session), 3) teletherapy only (at least 1 session), or 4) both telecoaching and teletherapy (at least 1 session of each type of care).
Subjective well-being and depressive symptoms
We used the WHO-5 to assess well-being and depressive symptoms within the past two weeks on a six-point scale (0 = at no time, 5 = all of the time), which is a unidimensional assessment of well-being with high clinimetric validity as a screening tool for depression [17]. Scores are summed and multiplied by 4, with higher scores indicating greater well-being and lower depressive symptomatology. Clinical improvement in depressive symptoms is defined as an increase of at least 10 points and clinical deterioration is defined as a decrease of at least 10 points [17, 18].
Cost of care
We conservatively estimated an hourly rate for coaches and therapists based on the national 2021 Medicare reimbursement rate of $103.28 for a 50 minute psychotherapy session [19]. Extrapolating from this rate, we modeled the adjusted cost to $61.97 per telecoaching session since coaching sessions are approximately 40% shorter on average (30 vs. 50 minutes). The cost of care for each individual was estimated as the sum of the product of the cost of the actual telecoaching and/or teletherapy sessions utilized during the study period.
Satisfaction with care
Satisfaction ratings can serve as a proxy for treatment acceptability and therapeutic alliance with providers. For participants who utilized teletherapy and/or telecoaching, we assessed satisfaction with a 5-star rating. Modern Health prompts individuals to rate their satisfaction after every visit.
Data analysis
We conducted analyses using R version 4.0.4. We used two-sided paired t-tests to evaluate differences in subjective well-being between baseline and follow-up assessments in our overall sample, as well as stratified by utilization patterns. Due to a strongly skewed distribution, we performed two-sided Kruskal-Wallis tests to evaluate differences in the cost of care between telecoaching, teletherapy, and having utilized both services. Post hoc Dunn tests were additionally performed to evaluate which utilization patterns differed significantly in terms of cost of care. We constructed logistic regression models to estimate the association between utilization patterns and clinical improvement given the binary nature of clinical improvement in depressive symptoms. We focused on clinical improvement in depressive symptoms (i.e., a 10-point increase at follow-up) as opposed to clinical recovery (i.e., moving above the clinical cut-off of 28) because it is a more robust outcome and a more challenging metric to achieve in the overall sample (e.g., some participants starting care with a WHO-5 score of 28 only need to increase one point to achieve recovery). The adjusted regression model included age, sex, baseline WHO-5 score, and number of days between baseline and follow-up assessments (to control for time) as covariates. We considered hypothesis tests statistically significant using an alpha level of 0.05.
Results
Study participants
Of the 7,476 adults who initially screened positive for depression at baseline during the study period, 1,852 (24.8%) had a follow-up assessment at least 14 days after baseline available for analysis. The average age of participants was 35.1 years (SD = 8.5, range = 20–71), 57.2% identified as female and 26.6% identified as male. We were missing sex from 16.1% of the sample due to optional reporting from employers. The mean WHO-5 score of participants at baseline was 21.1 (SD = 6.5, range = 0–28) and 31.2 (SD = 17.7, range = 0–100) at their last available follow-up assessment. The average time between baseline and follow-up assessments was 158 days (SD = 111, range = 14–467). Among those who had utilized care with a provider, the average time spent in care was 69 days (SD = 87, range = 0–449) and the average time between their last session and follow-up assessment was 75 days (SD = 95, range = 0–452). See Table 1 for the demographic and clinical factors in the overall sample and utilization-stratified samples.
Table 1. Descriptives of demographic and clinical factors among registrants of a digital mental health platform.
| Utilization | |||||
|---|---|---|---|---|---|
| Assessments Only (n = 758) | Telecoaching (n = 401) | Teletherapy (n = 455) | Telecoaching and Teletherapy (n = 238) | Total (n = 1852) | |
| Demographic Factors | |||||
| Age | |||||
| Mean | 35.78 | 35.20 | 34.38 | 34.24 | 35.11 |
| SD | 9.13 | 8.38 | 8.06 | 7.03 | 8.48 |
| Sex | |||||
| Female | 403 (53.2%) | 251 (62.6%) | 252 (55.4%) | 154 (64.7%) | 1060 (57.2%) |
| Male | 230 (30.3%) | 94 (23.4%) | 123 (27.0%) | 46 (19.3%) | 493 (26.6%) |
| Unknown | 125 (16.5%) | 56 (14.0%) | 80 (17.6%) | 38 (16.0%) | 299 (16.1%) |
| Clinical Factors | |||||
| Baseline Subjective Well-being (WHO-5) | |||||
| Mean | 21.59 | 22.26 | 19.46 | 20.69 | 21.09 |
| SD | 6.16 | 6.34 | 6.98 | 6.40 | 6.52 |
| Follow-up Subjective Well-being (WHO-5) | |||||
| Mean | 27.43 | 35.60 | 31.93 | 34.40 | 31.20 |
| SD | 16.12 | 17.76 | 18.33 | 18.58 | 17.68 |
| Follow-up Period (Number of days between baseline and follow-up assessment) | |||||
| Mean | 128.86 | 146.10 | 179.99 | 227.23 | 157.80 |
| SD | 97.61 | 104.12 | 113.44 | 115.53 | 110.54 |
| Difference in Baseline and Follow-up WHO-5 | |||||
| Mean | 5.84 | 13.34 | 12.48 | 13.71 | 10.10 |
| SD | 15.92 | 17.85 | 18.42 | 19.00 | 17.74 |
| Clinical Improvement | 241 (31.8%) | 210 (52.4%) | 225 (49.5%) | 123 (51.7%) | 799 (43.1%) |
Note. WHO-5 = World Health Organization-Five Well-Being Index.
Utilization
The most common utilization pattern among those with a positive depression screening at baseline was assessments only (40.9%), followed by teletherapy (24.6%), telecoaching (21.7%) and both services (12.9%). Individuals who utilized telecoaching alone completed an average of 3.2 sessions (SD = 3.3, range = 1–26), while those who utilized teletherapy alone completed an average of 5 sessions (SD = 5.1, range = 1–40). Individuals who utilized both telecoaching and teletherapy completed an average of 3.5 telecoaching sessions (SD = 3.2, range = 1–24) and 5.1 teletherapy sessions (SD = 4.1, range = 1–22). The majority of those utilizing both services started with telecoaching (73.1%), while 25.6% started with teletherapy and 1.3% started both services on the same day.
Subjective well-being
Regarding changes in subjective well-being between baseline and follow-up in our overall sample (see Table 2), paired t-tests found that participants reported an average improvement of 10.1 points on the WHO-5, which represents a 48% increase and medium effect size (d = 0.73). Those who utilized assessments only reported an average increase of 5.8 points, representing a 27% improvement and small effect size (d = 0.46). Those who utilized telecoaching alone reported an average increase of 13.3 points, which represents a 60% improvement and a large effect size (d = 0.97). Those who utilized teletherapy alone reported an average increase of 12.5 points, which represents a 64% improvement and a large effect size (d = 0.87). Those who utilized both telecoaching and teletherapy reported an average increase of 13.7 points, representing a 66% improvement and a large effect size (d = 0.97).
Table 2. Two-tailed paired t test results for subjective well-being outcomes.
| Baseline | Follow-up | Paired Pre-Post Difference | CIs of Pre-Post Difference | ||||
|---|---|---|---|---|---|---|---|
| Sample | n (%) | M (SD) | M (SD) | M | 95% CIs | t (df) | p |
| Overall Sample | |||||||
| Full | 1852 (100%) | 21.09 (6.52) | 31.20 (17.68) | 10.10 | 9.3–10.9 | 24.51 (1851) | < .001 |
| Utilization Type | |||||||
| Assessments Only | 758 (40.9%) | 21.59 (6.16) | 27.43 (16.12) | 5.84 | 4.7–6.97 | 10.09 (757) | < .001 |
| Telecoaching | 401 (21.7%) | 22.26 (6.34) | 35.60 (17.76) | 13.34 | 11.6–15.1 | 14.96 (400) | < .001 |
| Teletherapy | 455 (24.6%) | 19.46 (6.98) | 31.93 (18.33) | 12.47 | 10.8–14.2 | 14.45 (454) | < .001 |
| Telecoaching & Teletherapy | 238 (12.9%) | 20.69 (6.40) | 34.40 (18.58) | 13.71 | 11.3–16.1 | 11.13 (237) | < .001 |
Note. SD = standard deviation, CI = 95% confidence intervals.
Clinical improvement
As shown in Table 3, unadjusted logistic regression results found that participants who utilized any provider care were significantly more likely to achieve clinical improvement in depressive symptoms at follow-up. After adjusting the model for age, sex, baseline WHO-5 score, and number of days between baseline and follow-up assessments, those who utilized telecoaching (aOR = 2.45, 95% CI: 1.91–3.15, p < 0.001), teletherapy (aOR = 2.01, 95% CI: 1.57–2.57, p < 0.001), or both services (aOR = 2.28, 95% CI: 1.67–3.11, p < 0.001) were significantly more likely to achieve clinical improvement than those who only used assessments. In the overall sample, 9.3% (n = 173) of participants clinically deteriorated. The majority (51%) of those who deteriorated were participants who utilized assessments only (n = 88). Participants who clinically deteriorated (M = 24.53, SD = 4.11) had higher average baseline WHO-5 scores than those who did not clinically deteriorate (M = 20.74, SD = 6.62), t(1850) = -7.39, p < .001, Cohen d = 0.59.
Table 3. Unadjusted and adjusted logistic regression results for clinical improvement by utilization type.
| Clinical Improvement | |||||||
|---|---|---|---|---|---|---|---|
| Unadjusted model | Adjusted Model (a) | ||||||
| n (%) | OR | 95% CI | p | aOR | 95% CI | p | |
| (Intercept) | 0.47 | 0.40–0.54 | < .001 | 0.60 | 0.34–1.05 | .07 | |
| Age | -- | -- | -- | -- | 1.00 | 0.99–1.02 | .43 |
| Sex | -- | -- | -- | -- | -- | -- | -- |
| Female (reference) | -- | -- | -- | -- | -- | -- | -- |
| Male | -- | -- | -- | -- | 1.16 | 0.93–1.45 | .20 |
| Unknown | -- | -- | -- | -- | 1.21 | 0.93–1.58 | .15 |
| Baseline WHO-5 Score | -- | -- | -- | -- | 0.98 | 0.96–0.99 | .001 |
| Follow-Up Time | -- | -- | -- | -- | 1.00 | 0.99–1.00 | .71 |
| Utilization Pattern | -- | -- | -- | -- | -- | -- | -- |
| Assessments Only (reference) | 241 (31.8%) | -- | -- | -- | -- | -- | -- |
| Telecoaching | 210 (52.4%) | 2.36 | 1.84–3.03 | <0.001 | 2.45 | 1.91–3.15 | <0.001 |
| Teletherapy | 225 (49.5%) | 2.1 | 1.65–2.67 | <0.001 | 2.01 | 1.57–2.57 | <0.001 |
| Telecoaching & Teletherapy | 123 (51.7%) | 2.29 | 1.71–3.09 | <0.001 | 2.28 | 1.67–3.11 | <0.001 |
Note. n = 1,852. WHO-5 = World Health Organization-Five Well-Being Index. OR = Odds ratio; CI = Confidence interval; Follow-Up Time = Number of days between baseline and follow-up assessments.
(a) Adjusted model additionally included covariates.
Unadjusted model: Goodness of Fit (Hosmer and Lemeshow): X-squared: 2.4094e-23, df = 8, p-value = 1. Adjusted Model: Goodness of Fit (Hosmer and Lemeshow): X-squared: 6.3826, df = 8, p-value = .60.
Cost of care
The median total cost of care based on Medicare reimbursement rates was $124, $413, and $559 among those utilizing telecoaching, teletherapy, and both teletherapy and telecoaching, respectively. There was a significant difference in the median cost of care by utilization pattern (Kruskal-Wallis chi-square = 393.17, p < .001), and a post hoc Dunn test found that the cost of all utilization patterns differed significantly from each other. Specifically, telecoaching was less costly than teletherapy (Z = -14.14, p < .001) and both services (Z = 18.26, p < .001), and teletherapy was less costly than both services (Z = 6.57, p < .001).
Satisfaction with care
Among those who had utilized care with a provider, 53.6% (n = 586) completed at least 1 satisfaction rating following a session. Average satisfaction was 4.86 (SD = 0.48) and similar among those who utilized telecoaching (M = 4.87, SD = 0.48), teletherapy (M = 4.84, SD = 0.58), and both telecoaching and teletherapy (M = 4.86, SD = 0.35).
Discussion
We examined relative changes in subjective well-being and depressive symptoms and compared the costs between utilization patterns from a technology-enabled platform that delivers digital mental health services. We found that individuals who had originally reported elevated depressive symptoms at intake reported significant improvements in subjective well-being and clinically meaningful changes in depressive symptoms at follow-up. Utilization of telecoaching, teletherapy, or both services were similarly associated with greater clinical outcomes than having only utilized assessments during the study period and telecoaching was found to be the most economical utilization pattern.
In our sample of individuals who had registered for the platform during the COVID-19 pandemic, subjective well-being increased by an average of 10 points at follow-up, which represents both a statistically and clinically significant improvement. Larger effective sizes were observed among those who had utilized telecoaching, teletherapy, or both services compared to those utilizing assessments only. In a recent systematic review and meta-analysis evaluating the effectiveness of interventions on subjective well-being among working populations, psychosocial interventions were found to be the most effective in improving subjective well-being among working populations [20]. Although subjective well-being scores significantly increased post-intervention, the average well-being score was still relatively low (31/100) yet was above the threshold for moderate to severe depressive symptoms. A commonly used cut-off indicating mild or no depressive symptoms on the WHO-5 is a score greater than or equal to 50 [17]. Achieving this level of average improvement may occur with more time as more participants utilize the mental health services and future interventions should focus on increasing utilization and uptake. Nevertheless, our findings suggest that psychosocial interventions for subjective well-being remain effective under real world conditions outside of randomized controlled trial conditions.
Additionally, 43.1% of those who initially reported depressive symptoms showed clinical improvement at follow-up. Previous research has similarly found that digitally delivered mental health services can be effective in fostering clinical improvement from depression. A recent systematic review that included both controlled and uncontrolled studies of mental health systems that recommended care via pre-defined decision criteria found that recovery rates for depression in working age adults was between 40–60% [11]. Our results suggest that digital mental health services delivering care in this way may be similarly effective, especially given the more robust threshold of clinical improvement we tracked in our study (i.e., increasing 10 points) compared to clinical recovery (i.e., moving above a cut-off).
We also found that those utilizing telecoaching, teletherapy, or both services were significantly more likely to report clinical improvement in depressive symptoms at follow-up than individuals who had only utilized assessments. The observed effect sizes were similar across telecoaching, teletherapy, and having utilized both types of services. Previous research has similarly found low intensity psychosocial interventions may produce similar clinical benefits, [21] even for those with higher initial severity in depressive symptoms [22]. England’s Improving Access to Psychological Therapies programme that similarly recommends and delivers mental health treatment using patient characteristics recently reported that progressive delivery of services beginning with low intensity psychological well-being practitioners was associated with better clinical outcomes than stratified care that began with higher intensity therapists [23]. Finally, we found that 11.6% of participants who utilized assessments only clinically deteriorated and 7.8% of participants who utilized any one-on-one care deteriorated. Although few studies (around 6%) report deterioration rates [24], some prior work has found that rates vary between 0–25% in psychotherapy treatment groups and 11–44% in comparison groups [24–27]. The deterioration rates we observed are on the low end of these ranges and appear reasonable given this study was not a controlled trial and took place during the COVID-19 pandemic when mental health concerns have risen.
While the observed clinical outcomes and satisfaction were similar across utilization patterns that involved a provider, the average cost of care per individual varied significantly, such that telecoaching was the most economical service followed by teletherapy and the combination of telecoaching and teletherapy. The incorporation of telecoaching into digitally delivered mental health care models may improve their cost-effectiveness compared to dominant treatment models that only provide in-person psychotherapy. A previous randomized trial in a primary care setting similarly suggests that recommending services as a function of patient clinical symptoms may be more cost-effective for the treatment of depression than treatment as usual [13]. Future research is needed to evaluate the cost-effectiveness of digital mental health services to optimize use of limited resources and ultimately improve access to care.
In our sample, we found that 40.9% of registrants had not initiated digital care with either a coach or therapist by the follow-up assessment. Previous research has found similar rates of engagement with care in both digital and in-person mental health services as the majority of those with depression remain untreated. The national MindSpot Clinic in Australia, which provides digital mental health services, found the majority (67%) of patients reported their main purpose of using Mindspot was for assessments and information, while only 25.9% reported seeking treatment [14]. A large-scale retrospective analysis of electronic medical records in the United States found that only 35.7% of patients newly diagnosed with depression in primary care initiated treatment [28], which suggests that the delivery of multiple digital mental health services in this study may have helped to improve treatment initiation rates compared to usual care settings. Furthermore, emerging evidence suggests that digital modalities of care, such as iCBT that may or may not involve a provider, can be effective treatments and may be an important modality for reducing the proportion of individuals with mental health conditions that do not initiate treatment [29].
There are several important limitations to consider when interpreting the results of our study. Most notably, our analysis was limited to individuals who had completed a follow-up assessment during the study period. Although missing data is common in routine cohorts outside of trial environments, the exclusion of individuals without follow-up data introduces the potential that our results may not generalize to all individuals who screened positive for depressive symptoms at baseline [30]. Our study was also observational in nature and our findings require experimental confirmation to directly attribute clinical outcomes with the intervention or directly estimate the relative effectiveness of telecoaching, teletherapy, and having utilized both services. Despite these limitations, our study also has notable strengths. The longitudinal data analyzed in this study was collected under real world conditions, which may improve the generalizability of results compared to data collected in controlled trial conditions. Additionally, the average follow-up assessment occurred more than 2 months after the final visit for those who had utilized care with a provider, suggesting that benefits may be sustained.
Conclusions
This study demonstrates that digital mental health services including telecoaching and teletherapy were effective in improving subjective well-being and clinical improvement in depressive symptoms. While the likelihood of clinical improvement was similar across services, the average cost of care was significantly less for telecoaching. Digitally delivered mental health services may be a promising means of achieving clinical and cost-effective outcomes for those experiencing depressive symptoms.
Acknowledgments
We would like to acknowledge Emily Wang for her assistance with formatting and editing the manuscript. Additionally, we’d like to thank the Modern Health members, coaches, therapists, and team for their contributions to this work.
Data Availability
There are legal and ethical limitations on sharing a de-identified data set for this manuscript. In accordance with the Modern Health Privacy Notice and Terms of Use, Modern Health does not represent to their members that their data would be shared in the manner requested by PLOS ONE. These documents are binding promises that Modern Health has made to their users about how they use their data and are legally enforceable. Additionally, Modern Health members expect that the sensitive information about their health and wellness is closely guarded, and we need to honor that. Individual de-identified data that underlie the results reported in this manuscript can be shared privately for research purposes upon receipt of a methodologically sound proposal, and whose proposed use of the data from the study related to this article are approved by the authors. To gain access, requesters will need to sign a data access agreement that includes a commitment: (1) to using the data only for research purposes; (2) to not attempt to, or actually, re-identify any individual; (3) to securing the data using appropriate safeguards; and (4) to destroying or returning the data after analyses are completed. Please send data requests to research@joinmodernhealth.
Funding Statement
This study was funded by Modern Health. The funder had no role in study design, data collection, analysis, decision to publish, or preparation of the manuscript. Modern Health provided support in the form of salaries and stock options for MRP, SJSH, CEWC, CCS, and MA. The specific roles of these authors are articulated in the ‘author contributions’ section.
References
- 1.Agency for Healthcare Research and Quality. 2019 National Healthcare Quality and Disparities Report. {Agency for Healthcare Research and Quality}; 2020. Dec. Report No.: 20(21)-0045-EF. [PubMed] [Google Scholar]
- 2.Vos T, Flaxman AD, Naghavi M, Lozano R, Michaud C, Ezzati M, et al. Years lived with disability (YLDs) for 1160 sequelae of 289 diseases and injuries 1990–2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet. 2012. Dec 15;380(9859):2163–96. doi: 10.1016/S0140-6736(12)61729-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.James SL, Abate D, Abate KH, Abay SM, Abbafati C, Abbasi N, et al. Global, regional, and national incidence, prevalence, and years lived with disability for 354 diseases and injuries for 195 countries and territories, 1990–2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet. 2018. Nov 10;392(10159):1789–858. doi: 10.1016/S0140-6736(18)32279-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Ettman CK, Abdalla SM, Cohen GH, Sampson L, Vivier PM, Galea S. Prevalence of Depression Symptoms in US Adults Before and During the COVID-19 Pandemic. JAMA Netw Open. 2020. Sep 1;3(9):e2019686. doi: 10.1001/jamanetworkopen.2020.19686 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Resources Health and Administration Services. National Projections of Supply and Demand for Behavioral Health Practitioners: 2013–2025. 2016. Nov. [Google Scholar]
- 6.Andrade LH, Alonso J, Mneimneh Z, Wells JE, Al-Hamzawi A, Borges G, et al. Barriers to mental health treatment: results from the WHO World Mental Health surveys. Psychol Med. 2014. Apr;44(6):1303–17. doi: 10.1017/S0033291713001943 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.O’Brien M, McNicholas F. The use of telepsychiatry during COVID-19 and beyond. Ir J Psychol Med. 2020. Dec;37(4):250–5. doi: 10.1017/ipm.2020.54 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Krzyzaniak N, Greenwood H, Scott AM, Peiris R, Cardona M, Clark J, et al. The effectiveness of telehealth versus face-to face interventions for anxiety disorders: A systematic review and meta-analysis. J Telemed Telecare. 2021. Dec 3;1357633X211053738. doi: 10.1177/1357633X211053738 [DOI] [PubMed] [Google Scholar]
- 9.Thomas N, McDonald C, de Boer K, Brand RM, Nedeljkovic M, Seabrook L. Review of the current empirical literature on using videoconferencing to deliver individual psychotherapies to adults with mental health problems. Psychol Psychother. 2021. Sep;94(3):854–83. doi: 10.1111/papt.12332 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Haaga DA. Introduction to the special section on stepped care models in psychotherapy. J Consult Clin Psychol. 2000. Aug;68(4):547–8. [PubMed] [Google Scholar]
- 11.Firth N, Barkham M, Kellett S. The clinical effectiveness of stepped care systems for depression in working age adults: a systematic review. J Affect Disord. 2015. Jan 1;170:119–30. doi: 10.1016/j.jad.2014.08.030 [DOI] [PubMed] [Google Scholar]
- 12.van Straten A, Hill J, Richards DA, Cuijpers P. Stepped care treatment delivery for depression: a systematic review and meta-analysis. Psychol Med. 2015. Jan;45(2):231–46. doi: 10.1017/S0033291714000701 [DOI] [PubMed] [Google Scholar]
- 13.Yan C, Rittenbach K, Souri S, Silverstone PH. Cost-effectiveness analysis of a randomized study of depression treatment options in primary care suggests stepped-care treatment may have economic benefits. BMC Psychiatry. 2019. Aug 5;19(1):240. doi: 10.1186/s12888-019-2223-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Titov N, Dear BF, Nielssen O, Wootton B, Kayrouz R, Karin E, et al. User characteristics and outcomes from a national digital mental health service: an observational study of registrants of the Australian MindSpot Clinic. Lancet Digit Health. 2020. Nov;2(11):e582–93. doi: 10.1016/S2589-7500(20)30224-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Clark DM, Layard R, Smithies R, Richards DA, Suckling R, Wright B. Improving access to psychological therapy: Initial evaluation of two UK demonstration sites. Behav Res Ther. 2009. Nov;47(11):910–20. doi: 10.1016/j.brat.2009.07.010 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Theeboom T, Beersma B, van Vianen AEM. Does coaching work? A meta-analysis on the effects of coaching on individual level outcomes in an organizational context. J Posit Psychol. 2014. Jan 2;9(1):1–18. [Google Scholar]
- 17.Topp CW, Østergaard SD, Søndergaard S, Bech P. The WHO-5 Well-Being Index: A Systematic Review of the Literature. Psychother Psychosom. 2015;84(3):167–76. doi: 10.1159/000376585 [DOI] [PubMed] [Google Scholar]
- 18.Bech P, Lunde M, Bech-Andersen G, Lindberg L, Martiny K. Psychiatric outcome studies (POS): does treatment help the patients? A Popperian approach to research in clinical psychiatry. Nord J Psychiatry. 2007;61 Suppl 46:4–34. [DOI] [PubMed] [Google Scholar]
- 19.License for use of current procedural terminology, fourth edition (“CPT”) [Internet]. [cited 2021 Aug 13]. Available from: https://www.cms.gov/medicare/physician-fee-schedule/search?Y=0&T=4&HT=0&CT=0&H1=90834&M=5
- 20.Sakuraya A, Imamura K, Watanabe K, Asai Y, Ando E, Eguchi H, et al. What Kind of Intervention Is Effective for Improving Subjective Well-Being Among Workers? A Systematic Review and Meta-Analysis of Randomized Controlled Trials. Front Psychol. 2020;11:2884. doi: 10.3389/fpsyg.2020.528656 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Kunkle S, Yip M, Ξ W, Hunt J. Evaluation of an On-Demand Mental Health System for Depression Symptoms: Retrospective Observational Study. J Med Internet Res. 2020. Jun 18;22(6):e17902. doi: 10.2196/17902 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Bower P, Kontopantelis E, Sutton A, Kendrick T, Richards DA, Gilbody S, et al. Influence of initial severity of depression on effectiveness of low intensity interventions: meta-analysis of individual patient data. BMJ. 2013. Feb 26;346:f540. doi: 10.1136/bmj.f540 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Boyd L, Baker E, Reilly J. Impact of a progressive stepped care approach in an improving access to psychological therapies service: An observational study. PLoS One. 2019. Apr 9;14(4):e0214715. doi: 10.1371/journal.pone.0214715 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Cuijpers P, Reijnders M, Karyotaki E, de Wit L, Ebert DD. Negative effects of psychotherapies for adult depression: A meta-analysis of deterioration rates. J Affect Disord. 2018. Oct 15;239:138–45. doi: 10.1016/j.jad.2018.05.050 [DOI] [PubMed] [Google Scholar]
- 25.Hansen NB, Lambert MJ, Forman EM. The psychotherapy dose-response effect and its implications for treatment delivery services. Clinical Psychology: science and practice. 2002;9(3):329. [Google Scholar]
- 26.Schmitt A, Kulzer B, Reimer A, Herder C, Roden M, Haak T, et al. Evaluation of a Stepped Care Approach to Manage Depression and Diabetes Distress in Patients with Type 1 Diabetes and Type 2 Diabetes: Results of a Randomized Controlled Trial (ECCE HOMO Study). Psychother Psychosom. 2021. Dec 7;1–16. [DOI] [PubMed] [Google Scholar]
- 27.Rozental A, Magnusson K, Boettcher J, Andersson G, Carlbring P. For better or worse: An individual patient data meta-analysis of deterioration among participants receiving Internet-based cognitive behavior therapy. J Consult Clin Psychol. 2017. Feb;85(2):160–77. doi: 10.1037/ccp0000158 [DOI] [PubMed] [Google Scholar]
- 28.Waitzfelder B, Stewart C, Coleman KJ, Rossom R, Ahmedani BK, Beck A, et al. Treatment Initiation for New Episodes of Depression in Primary Care Settings. J Gen Intern Med. 2018. Aug;33(8):1283–91. doi: 10.1007/s11606-017-4297-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Andrews G, Basu A, Cuijpers P, Craske MG, McEvoy P, English CL, et al. Computer therapy for the anxiety and depression disorders is effective, acceptable and practical health care: An updated meta-analysis. J Anxiety Disord. 2018. Apr;55:70–8. doi: 10.1016/j.janxdis.2018.01.001 [DOI] [PubMed] [Google Scholar]
- 30.Callahan A, Shah NH, Chen JH. Research and Reporting Considerations for Observational Studies Using Electronic Health Record Data. Ann Intern Med. 2020. Jun 2;172(11 Suppl):S79–84. doi: 10.7326/M19-0873 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
There are legal and ethical limitations on sharing a de-identified data set for this manuscript. In accordance with the Modern Health Privacy Notice and Terms of Use, Modern Health does not represent to their members that their data would be shared in the manner requested by PLOS ONE. These documents are binding promises that Modern Health has made to their users about how they use their data and are legally enforceable. Additionally, Modern Health members expect that the sensitive information about their health and wellness is closely guarded, and we need to honor that. Individual de-identified data that underlie the results reported in this manuscript can be shared privately for research purposes upon receipt of a methodologically sound proposal, and whose proposed use of the data from the study related to this article are approved by the authors. To gain access, requesters will need to sign a data access agreement that includes a commitment: (1) to using the data only for research purposes; (2) to not attempt to, or actually, re-identify any individual; (3) to securing the data using appropriate safeguards; and (4) to destroying or returning the data after analyses are completed. Please send data requests to research@joinmodernhealth.