Abstract
Background
Technology-enhanced mental health platforms may serve as a pathway to accessible and scalable mental health care; specifically, those that leverage stepped care models have the potential to address many barriers to patient care, including low mental health literacy, mental health provider shortages, perceived acceptability of care, and equitable access to evidence-based treatment. Driving meaningful engagement in care through these platforms remains a challenge.
Objective
This study aimed to examine predictors of engagement in self-directed digital mental health services offered as part of an employer-based mental health benefit that uses a technology-enabled care platform.
Methods
Using a prospective, longitudinal design, we examined usage data from employees who had access to an employer-sponsored mental health care benefit. Participants had access to a digital library of mental health resources, which they could use at any time, including daily exercises, interactive programs, podcasts, and mindfulness exercises. Coaching and teletherapy were also available to. The outcome was engagement with the self-directed digital mental health resources, measured by the number of interactions. Poisson regression models included sociodemographic characteristics, patient activation, mental health literacy, well-being, PHQ-9 and GAD-7 scores at baseline, primary concern for engaging in treatment, and the use of coaching or teletherapy sessions.
Results
In total 950 individuals enrolled in the study, with 38% using any self-directed digital mental health resources. Approximately 44% of the sample did not use the app during the study period. Those using both self-directed digital and 1:1 modalities made up about one-quarter of the sample (235/950, 24.7%). Those using only coaching or therapy (170/950, 17.9%) and those using only self-directed digital mental health resources (126/950, 13.3%) make up the rest. At baseline, these groups statistically significantly differed on age, PHQ-9, GAD-7, MHLS, and primary concern. Receipt of coaching and teletherapy was associated with the number of self-directed digital mental health resources interactions in adjusted Poisson regression modeling. Use of any coach visit was associated with 82% (rate ratio [RR] 1.82, 95% CI 1.63-2.03) more self-directed digital mental health resource interactions while use of any teletherapy session was associated with 80% (RR 1.80, 95% CI 1.55-2.07) more digital mental health resources interactions (both P<.001). Each additional year of age was associated with increased digital mental health resources interactions (RR 1.04, 95% CI (1.03-1.05), and women had 23% more self-directed digital resources interactions than men (RR 1.23, 95% CI 1.09-1.39).
Conclusions
Our key finding was that the use of coaching or teletherapy was associated with increased self-directed digital mental health resource use. Higher self-directed digital resource engagement among those receiving coaching or therapy may be a result of provider encouragement. On the other hand, when a participant engages with 1 modality in the platform, they may be more likely to begin engaging with others, becoming “super users” of all resources.
Keywords: digital health, mental health, health care benefit, prediction, technology, digital mental health, employer-based, teletherapy, coaching, utilization, mobile phone
Introduction
Mental health and substance use disorders are leading causes of disease burden in the United States [1]. However, while nearly 25% of adults experience a mental disorder at any given time [2], less than a quarter (21.7%) of individuals with a mental disorder receives treatment from a licensed mental health clinician [3]. This discrepancy between the prevalence of mental health disorders and rates of mental health service use is largely attributed to barriers to health care access, such as not being able to schedule an appointment in a timely manner [4] and persistent shortages in the mental health workforce—especially in rural communities and for people with serious mental illness [5,6]. As the demand for psychological services outpaces the supply of mental health providers, innovative mental health care delivery models are increasingly needed.
Digital mental health (digital health) platforms represent a promising approach to deliver accessible, scalable, and timely mental health care. These platforms often provide care through multiple modalities with varying intensities (eg, traditional one-on-one telepsychotherapy or telecoaching, asynchronous self-guided treatments, and telegroup-based treatments). As digital health platforms can be equipped to provide a range of treatments to patients, they have the potential to dismantle key barriers to patient care, including mental health provider shortages and inequitable access to mental health treatment [7-9].
However, digital health platforms can be challenging to implement, largely because user adoption and continued engagement are relatively low [10-12]. While traditional psychotherapy also struggles with early dropout rates ranging between 8% and 20% [13,14], digital health historically has seen much higher rates of discontinuation [11]. As a result, while they hold the potential to promote increased, equitable access to mental health care, additional research is needed to determine how to facilitate increased engagement in mental health care from digital health platforms. Recent research investigating engagement in digital health applications after initial adoption suggests that several factors, including user attributes (eg severity of depressive symptoms), technology issues (eg concerns about anonymity and privacy), components of user experience (eg, interface design of a digital health application) [15] and motivation to engage in the health goal of interest, are associated with continued use [16]. Other research indicates that users’ beliefs about the severity of their health conditions may also be associated with digital health adoption and continued use [17] and that improvement in psychological outcomes of interest during early treatment may predict continued engagement [18]. Finally, it is important to understand the appropriate level of engagement in different services given a user’s needs and preferences. Not all users will need long-term unlimited engagement with a digital health platform, and engagement likely involves evaluating the need for continued use [19,20].
As the number of digital mental health platforms continues to grow and more modalities of digital mental health care (eg, podcasts, online courses, meditations, discussion boards, etc.) become available to consumers, it is critical to understand predictors of engagement in digital mental health care. Elucidating the range of factors that predict initial and continued use of digital health services for mental health will support these platforms in further customizing care approaches to maximize user engagement, and ultimately, optimize personalized mental health treatment. The purpose of this study was to identify predictors of engagement in multiple modalities of mental health treatments offered on an employer-sponsored digital mental health care benefit.
Methods
Design and Participants
This investigation was part of a prospective, longitudinal, observational study of individuals who received services through an employer-sponsored digital mental health benefits platform (Modern Health Inc). The study timeframe was from September 20, 2021, to May 31, 2022.
Eligible participants were 18 years or older, based in the United States, registered for the benefit, and had access to a smartphone, tablet, or computer.
Procedures
Eligible employees registered for a Modern Health account through a mobile app or website and completed several onboarding assessments, which asked about areas of focus, care modality preferences, and clinical symptoms. The platform’s proprietary algorithm recommended an initial care pathway to participants based on this information. Recommendations used a stratified stepped care model, meaning that patients with higher acuity and with certain areas of focus were more likely to be recommended to more intensive care options (eg, one-on-one teletherapy) (more details in Figure 1). Participants could use their recommended care or self-refer or be referred by a provider to any combination of care.
Figure 1.
Stratified stepped care model incorporating areas of support, care preferences, and clinical assessments into personalized care recommendations. PHQ-9: 9-item Patient Health Questionnaire; GAD-7: 7-item Generalized Anxiety Disorder Questionnaire; WHO-5: 5-item World Health Organization Well-Being Index.
Members who matched with a provider or used at least 1 digital resource after onboarding were invited by email to complete a screening questionnaire within 2 weeks of onboarding, which contained questions about demographics. We used the demographic screening information for purposeful recruitment to ensure representation in racial and ethnic identities, gender identities, socioeconomic status, age groups, and mental health symptoms at baseline. After the screening questionnaire, eligible participants were directed to a consent form to provide informed consent. Consenting participants were sent the baseline survey asking questions about their mental, social, and physical health by email along with instructions and compensation. Participants were sent a follow-up survey at 12 weeks post baseline that contained the same mental, social, and physical health questions. All forms and surveys were hosted on Qualtrics’ online survey platform.
Digital Mental Health Services
Individuals in this study were able to engage in all of the following digital mental health services, within the limits of their employer-covered sessions for 1:1 care modalities. Self-directed digital resources had no limits.
Digital Resources
Participants had access to a digital library of programs and resources they could use in unlimited quantities at any time. These included daily meditative exercises, interactive programs and podcasts, mindfulness exercises like meditations and breathing exercises, and self-paced structured educational lessons (similar to self-help workbooks). All digital materials were developed and designed by an in-house team of clinical psychologists. Digital health programs were designed to cover topics such as emotions, relationships, professional life, healthy lifestyles, and finances. Engagement across all digital resources was combined in analyses to represent total digital program engagement.
Telecoaching
Coaches were certified by an International Coaching Federation accredited program and underwent a careful screening process by Modern Health. They provided 30-minute telecoaching services to eligible employees through a videoconferencing platform (Zoom; Zoom Video Communications). Coaches and participants were also able to communicate by messaging within the app. All coaches had at least 150 hours of coaching experience in addition to completing additional training in evidence-based techniques and Modern Health’s proprietary model of care. Rather than being confined to any specific protocol, coaches were encouraged to implement evidence-based techniques where appropriate. The number of coaching sessions attended by participants depended on the number of sessions covered by their employer, as well as on personal preferences and their level of need.
Teletherapy
Furthermore, 50 minutes one-on-one teletherapy sessions were provided through a videoconferencing platform (Zoom) by licensed and vetted therapists who had an advanced degree in clinical psychology or a related field (eg, PhD, PsyD, LCSW, LMFT, or LPC). Similar to telecoaching, therapists and participants were able to communicate through messaging in-app. All therapists were skilled in evidence-based practices, such as cognitive behavioral therapy, acceptance and commitment therapy, and dialectical behavior therapy and had completed additional training in Modern Health’s proprietary model of care. Similar to telecoaching, the number of therapy sessions attended by participants depended on the number of sessions covered by their employer, as well as on personal preferences and their level of need [21].
Outcome Measures
Engagement in digital mental health services: We created 4 outcome variables to capture engagement in platform services. First, we created a 4-level categorical variable to represent use of the digital health services: (1) only self-directed digital resource use, (2) only telecoach or teletherapist use, (3) use of both digital and telecoach or teletherapist modalities, and (4) no use of any modality. Second, we created a binary variable (0/1) indicating whether participants used at least 1 service within the platform (inclusive of self-directed digital resources, one-on-one telecoaching visits, and one-on-one teletherapy visits). Third, we created a binary variable (0/1) indicating whether participants used at least one of the self-directed digital resources collapsed across resource types: meditative exercises, programs, meditations, and podcasts. Fourth, we created a count of the number of self-directed digital resources each participant used. The use of both self-directed digital resources and 1:1 modalities was collected automatically by the digital mental health platform. Engagement data during the study period (between the baseline and follow-up surveys) was extracted from the platform for analysis.
Predictor Measures
Demographics
Participants self-identified their age, race and ethnicity (American Indian or Alaskan Native, Asian, Black or African American, Hawaiian or Pacific Islander, Hispanic, Latino, or Spanish origin, non-Hispanic white, prefer to self-describe, multiracial; collapsed to non-Hispanic White vs non-White), gender identity (man, woman, non-binary), and educational attainment (less than a bachelors’ degree, bachelors’ degree, and more than a bachelors’ degree).
Depression
The 9-item Patient Health Questionnaire (PHQ-9) [22] was used to assess the presence of depression symptoms over the past 2 weeks. Participants responded on a 4-point scale (0= “Not at all,” 3= “Nearly every day”), and total scores were categorized into 1 of 5 severity ratings: none, minimal, mild, moderate, or moderately severe. Higher scores indicate higher severity of depression symptomatology.
Anxiety Symptoms
The 7-item Generalized Anxiety Disorder Questionnaire (GAD-7) [23] was used to assess the presence of anxiety symptoms over the past 2 weeks. Participants responded on a 4-point scale (0=“Not at all,” 3=“Nearly every day”) and total scores were categorized into 1 of 4 severity ratings: minimal, mild, moderate, or severe. Higher scores indicate higher severity of anxiety symptomatology.
Well-Being
The 5-item World Health Organization Well-Being Index (WHO-5) [24] was used to assess well-being over the past 2 weeks. This construct was assessed during the onboarding process, approximately 2 weeks before the baseline assessment. Participants responded on a 6-point scale (0=“At no time,” 5=“All of the time”). Higher scores indicate greater subjective well-being.
Patient Activation
The 13-item Patient Activation Measure for Mental Health (PAM-MH) was used [25]. Participants chose from 5 possible response options ranging from 1=Strongly Disagree to 4=Strongly Agree to 5=Not Applicable. Higher scores indicate higher patient activation.
Mental Health Literacy
Mental health literacy was measured using 13 items from the Mental Health Literacy Scale (MHLS) [26]. Response options ranged from 1=Strongly Disagree to 5=Strongly Agree.
Area of Focus
Participants were able to choose from over 40 topics of focus, or reasons for registering for the platform, during the onboarding process. These topics were wide-ranging and organized by the following well-being dimensions: “my emotions,” “my professional life,” “my physical well-being,” “my relationships,” and “my finances.”
One-on-One Care Engagement
Variables indicating whether participants had used at least telecoaching (0/1) or teletherapy (0/1) sessions within the platform were also included as predictors in analyses predicting the use of self-directed digital resources.
Statistical Analyses
We conducted 4 sets of analyses examining the association between all the predictors and each engagement in digital mental health services outcome. First, we used descriptive analyses to investigate differences in predictor variables across the 4-level categorical variable of use, with ANOVA performed for continuous predictors and chi-square tests for categorical predictors.
Second, we examined predictors of any digital health service use in the app using logistic regression. The outcome variable was the binary variable representing if participants had used at least 1 service. In this regression model, we included age, race and ethnicity (non-White as reference group), gender identity (man as reference group), educational attainment (less than bachelor’s degree completion as reference group), patient activation, mental health literacy, well-being, area of focus (“my emotions” as reference group), baseline depression (entered as a linear predictor), and baseline anxiety (entered as a linear predictor) as predictors.
Third, we examined predictors of any digital health self-directed digital resource use in the app using logistic regression. The outcome variable was the binary variable representing if participants had at least one self-directed digital resource in the 3-month follow-up period. In this regression, we included age, race and ethnicity, gender identity, educational attainment, patient activation, mental health literacy, well-being, primary concern, baseline depression (entered as a linear predictor), and baseline anxiety (entered as a linear predictor) as predictors, with the same reference groups as in the previous analysis.
Fourth, using Poisson regression, we examined if the use of any telecoaching or teletherapy visits was associated with the number of self-directed digital resources used. We created binary variables for both telecoaching and teletherapy, reflecting any use of each modality in the follow-up period. We used both unadjusted and adjusted approaches to investigate this association. In the unadjusted model, we only included the telecoaching and teletherapy visit variables as predictors. In the adjusted model, we additionally included age, race and ethnicity, gender identity, educational attainment, patient activation, mental health literacy, well-being, and area of focus, with reference groups mirroring the previous analyses. We also performed the same adjusted Poisson regression analyses on a subgroup of our sample: those with elevated GAD-7 or PHQ-9 scores (scores on either scale above 9 indicating moderate symptoms [27,28]) at baseline.
Ethical Considerations
All participants provided informed consent, and this study protocol was reviewed and approved by the Western Clinical Group institutional review board (protocol #1316167). Compensation for this study was a US $25 digital gift card per completed survey. All data were deidentified.
Results
In total, 950 individuals were enrolled in this study. More than half of participants (n=531, 55.9%) engaged with at least 1 service within the platform (eg, self-directed digital resource, telecoach, or teletherapist session); 44.1% (n=419) did not engage in any service. Among engagers, 44.3% (n=235) used both self-directed digital resources and 1:1 care (teletherapy or telecoaching), 32% (n=170) used only 1:1 care, and 23.7% (n=126) used only self-directed digital resources. At baseline, these engagement groups statistically significantly differed in age, depressive symptoms, anxiety symptoms, health literacy, and area of focus (Table 1). Among participants who only used self-directed digital resources, the average number of digital engagements was 2.0 (median 1.0). Among those using both 1:1 care and self-directed digital resources, the average number of digital engagements was 3.4 (median 2.0).
Table 1.
Demographics and baseline characteristics, overall and by modality of app use. P values are derived from chi-square tests for categorical variables and from ANOVA tests for continuous variables.
|
|
Overall (N=950) | No use (N=419) | Therapy or coach only (N=170) | Self-directed digital resources only (N=126) | Both modalities (N=235) | P value | ||||||||
| Age | <.001 | |||||||||||||
|
|
Mean (SD) | 33.9 (8.70) | 33.9 (8.68) | 31.5 (7.02) | 36.8 (9.79) | 34.0 (8.76) |
|
|||||||
|
|
Median (range) | 32 (19-65) | 32 (19-63) | 30 (22-57) | 36.5 (22-60) | 32 (21-65) |
|
|||||||
| Race and ethnicity (White vs non-White), n (%) | .74 | |||||||||||||
|
|
Non-White | 399 (42) | 184 (43.9) | 73 (42.9) | 49 (38.9) | 93 (39.6) |
|
|||||||
|
|
White | 548 (57.7) | 232 (55.4) | 97 (57.1) | 77 (61.1) | 142 (60.4) |
|
|||||||
|
|
Missing | 3 (0.3) | 3 (0.7) | 0 (0) | 0 (0) | 0 (0) |
|
|||||||
| Gender identity, n (%) | .66 | |||||||||||||
|
|
Gender-nonbinary | 46 (4.8) | 23 (5.5) | 8 (4.7) | 4 (3.2) | 11 (4.7) |
|
|||||||
|
|
Man | 343 (36.1) | 164 (39.1) | 59 (34.7) | 48 (38.1) | 72 (30.6) |
|
|||||||
|
|
Woman | 559 (58.8) | 230 (54.9) | 103 (60.6) | 74 (58.7) | 152 (64.7) |
|
|||||||
|
|
Prefer not to say | 2 (0.2) | 2 (0.5) | 0 (0) | 0 (0) | 0 (0) |
|
|||||||
| Educational attainment, n (%) | .40 | |||||||||||||
|
|
<Bachelors | 140 (14.7) | 75 (17.9) | 23 (13.5) | 17 (13.5) | 25 (10.6) |
|
|||||||
|
|
Bachelors | 545 (57.4) | 230 (54.9) | 105 (61.8) | 71 (56.3) | 139 (59.1) |
|
|||||||
|
|
>Bachelors | 265 (27.9) | 114 (27.2) | 42 (24.7) | 38 (30.2) | 71 (30.2) |
|
|||||||
| Baseline PHQ-9a | .03 | |||||||||||||
|
|
Mean (SD) | 8.58 (5.96) | 9.13 (6.51) | 8.51 (5.41) | 7.11 (5.52) | 8.54 (5.51) |
|
|||||||
|
|
Median (range) | 8 (0-27) | 8 (0-27) | 8 (0-23) | 6 (0-23) | 8 (0-25) |
|
|||||||
|
|
Missing, n (%) | 35 (3.7) | 35 (8.4) | 0 (0) | 0 (0) | 0 (0) |
|
|||||||
| Baseline GAD-7b | .01 | |||||||||||||
|
|
Mean (SD) | 7.73 (5.51) | 8.03 (5.75) | 7.86 (5.21) | 6.10 (5.09) | 8.01 (5.44) |
|
|||||||
|
|
Median (range) | 7 (0-21) | 7 (0-21) | 7 (0-21) | 5 (0-20) | 7 (0-21) |
|
|||||||
|
|
Missing, n (%) | 34 (3.6) | 34 (8.1) | 0 (0) | 0 (0) | 0 (0) |
|
|||||||
| Baseline PAMc | .85 | |||||||||||||
|
|
Mean (SD) | 58.3 (13.0) | 57.9 (13.1) | 59.3 (13.2) | 58.3 (12.8) | 58.2 (12.7) |
|
|||||||
|
|
Median (range) | 55.6 (17.9-100) | 53.2 (17.9-100) | 55.6 (34.2-100) | 55.6 (30.4-100) | 55.6 (31.7-100) |
|
|||||||
|
|
Missing, n (%) | 35 (3.7) | 34 (8.1) | 1 (0.6) | 0 (0) | 0 (0) |
|
|||||||
| Baseline MHLSd | .02 | |||||||||||||
|
|
Mean (SD) | 55.9 (6.43) | 55.2 (7.13) | 56.9 (6.01) | 56.9 (6.16) | 55.8 (5.46) |
|
|||||||
|
|
Median (range) | 57 (29-65) | 57 (29-65) | 58.5 (34-65) | 58 (29-65) | 57 (37-65) |
|
|||||||
|
|
Missing, n (%) | 26 (2.7) | 26 (6.2) | 0 (0) | 0 (0) | 0 (0) |
|
|||||||
| Baseline WHO-5e | .16 | |||||||||||||
|
|
Mean (SD) | 43.3 (18.3) | 42.1 (17.7) | 42.8 (19.4) | 46.7 (18.7) | 44.1 (18.2) |
|
|||||||
|
|
Median (range) | 44.0 (0-100) | 40 (0-100) | 40 (4-88) | 48 (8-84) | 44 (0-100) |
|
|||||||
| Any past mental health care, n (%) | .88 | |||||||||||||
|
|
No | 233 (24.5) | 112 (26.7) | 43 (25.3) | 33 (26.2) | 45 (19.1) |
|
|||||||
|
|
Yes | 244 (25.7) | 111 (26.5) | 42 (24.7) | 34 (27) | 57 (24.3) |
|
|||||||
|
|
Missing | 473 (49.8) | 196 (46.8) | 85 (50) | 59 (46.8) | 133 (56.6) |
|
|||||||
| Any current mental health care, n (%) | .66 | |||||||||||||
|
|
No | 464 (48.8) | 217 (51.8) | 82 (48.2) | 62 (49.2) | 103 (43.8) |
|
|||||||
|
|
Yes | 275 (28.9) | 114 (27.2) | 49 (28.8) | 44 (34.9) | 68 (28.9) |
|
|||||||
|
|
Missing | 211 (22.2) | 88 (21) | 39 (22.9) | 20 (15.9) | 64 (27.2) |
|
|||||||
| Primary concern, n (%) | .02 | |||||||||||||
|
|
My emotions | 466 (49.1) | 198 (47.3) | 102 (60) | 44 (34.9) | 122 (51.9) |
|
|||||||
|
|
My finances | 33 (3.5) | 15 (3.6) | 5 (2.9) | 5 (4) | 8 (3.4) |
|
|||||||
|
|
My physical well-being | 95 (10) | 40 (9.5) | 9 (5.3) | 23 (18.3) | 23 (9.8) |
|
|||||||
|
|
My professional life | 141 (14.8) | 65 (15.5) | 17 (10) | 27 (21.4) | 32 (13.6) |
|
|||||||
|
|
My relationship | 214 (22.5) | 100 (23.9) | 37 (21.8) | 27 (21.4) | 50 (21.3) |
|
|||||||
|
|
Missing | 1 (0.1) | 1 (0.2) | 0 (0) | 0 (0) | 0 (0) |
|
|||||||
aPHQ-9: 9-item Patient Health Questionnaire.
bGAD-7: 7-item Generalized Anxiety Disorder Questionnaire.
cPAM: Patient Activation Measure.
dMHLS: Mental Health Literacy Scale.
eWHO-5: 5-item World Health Organization Well-Being Index.
In the logistic regression analysis predicting engagement in any service, only higher educational attainment was associated with increased odds of engaging in at least 1 service (Table 2). Relative to participants without a bachelor’s degree, those with a bachelor’s had 54.5% (P=.03) higher odds of engaging, and those with education greater than a bachelor’s had 56.2% (P=.05) higher odds of engaging. No other variables in the model were statistically significant.
Table 2.
Adjusted logistic regression analysis predicting any app use (N=908).
| Variables | Odds ratio (95% CI) | P value | ||
| Age | 1.002 (0.986-1.019) | .78 | ||
| Race and ethnicity | ||||
|
|
White | 1.074 (0.813-1.419) | .61 | |
|
|
Non-White | REFa | —b | |
| Gender identity | ||||
|
|
Nonbinary | 1.084 (0.559-2.125) | .81 | |
|
|
Woman | 1.310 (0.977-1.756) | .07 | |
|
|
Man | REF | — | |
| Educational attainment | ||||
|
|
Greater than bachelors | 1.562 (1.01-2.422) | .05 | |
|
|
Bachelors | 1.545 (1.037-2.305) | .03 | |
|
|
Less than bachelors | REF | — | |
| Patient activation | ||||
|
|
PAMc,d | 0.999 (0.987-1.01) | .82 | |
| Mental health literacy | ||||
|
|
MHLSc,e | 1.022 (0.999-1.045) | .07 | |
| Well-being | ||||
|
|
WHO-5c,f | 1.003 (0.994-1.013) | .51 | |
| Primary concern | ||||
|
|
My emotions | REF | — | |
|
|
My finances | 0.864 (0.409-1.871) | .70 | |
|
|
My physical well-being | 0.926 (0.577-1.495) | .75 | |
|
|
My professional life | 0.779 (0.52-1.171) | .23 | |
|
|
My relationships | 0.808 (0.568-1.151) | .24 | |
| Mental health at baseline | ||||
|
|
PHQ-9c,g | 0.975 (0.938-1.013) | .19 | |
|
|
GAD-7c,h | 1.005 (0.968-1.044) | .79 | |
aREF: reference range.
bNot applicable.
cPAM, MHLS, WHO-5, PHQ-9, and GAD-7 scores were entered as linear predictors.
dPAM: Patient Activation Measure.
eMHLS: Mental Health Literacy Scale.
fWHO-5: 5-item World Health Organization Well-Being Index.
gPHQ-9: 9-item Patient Health Questionnaire.
hGAD-7: 7-item Generalized Anxiety Disorder Questionnaire.
In our logistic regression analysis predicting engagement in any self-directed digital resources, certain demographic characteristics were associated with an increased likelihood of engagement. For every year of increasing age, subjects were 2.6% more likely to have used any self-directed resources (P=.003). Relative to men, women had 45.4% higher odds of using any self-directed resources (P=.01). Finally, educational attainment was also significantly associated with self-directed resource use, such that those with bachelor’s degrees (odds ratio 1.56 95% CI 1.026-2.414; P=.04) and with education higher than a bachelor’s degree (odds ratio 1.65 95% CI 1.045-2.625; P=.03) had higher odds of self-directed resource engagement than those with less than a bachelor’s degree (Table 3).
Table 3.
Adjusted logistic regression analysis predicting any self-directed digital resources use (N=908).
| Variable | Odds ratio (95% CI) | P value | |
| Age | 1.026 (1.009-1.044) | .003 | |
| Race and ethnicity | |||
|
|
White | 1.115 (0.841-1.48) | .45 |
|
|
Non-White | REFa | —b |
| Gender identity | |||
|
|
Non-binary | 1.122 (0.55-2.22) | .75 |
|
|
Woman | 1.454 (1.079-1.965) | .01 |
|
|
Man | REF | — |
| Educational attainment | |||
|
|
Greater than bachelors | 1.648 (1.045-2.625) | .03 |
|
|
Bachelors | 1.564 (1.026-2.414) | .04 |
|
|
Less than bachelors | REF | — |
| Patient activation | |||
|
|
PAMc,d | 0.995 (0.983-1.007) | .40 |
| Mental health literacy | |||
|
|
MHLSc,e | 1.003 (0.98-1.027) | .82 |
| Well-being | |||
|
|
WHO-5c,f | 1.008 (0.998-1.017) | .11 |
| Primary concern | |||
|
|
My emotions | REF | — |
|
|
My finances | 1.143 (0.524-2.43) | .73 |
|
|
My physical well-being | 1.432 (0.894-2.292) | .13 |
|
|
My professional life | 1.158 (0.769-1.737) | .48 |
|
|
My relationships | 0.989 (0.689-1.414) | .95 |
| Mental health at baseline | |||
|
|
PHQ-9c,g | 0.992 (0.954-1.031) | .68 |
|
|
GAD-7c,h | 1.004 (0.967-1.043) | .83 |
aREF: reference.
bNot applicable.
cPAM, MHLS, WHO-5, PHQ-9, and GAD-7 scores were entered as linear predictors.
dPAM: Patient Activation Measure.
eMHLS: Mental Health Literacy Scale.
fWHO-5: 5-item World Health Organization Well-Being Index.
gPHQ-9: 9-item Patient Health Questionnaire.
hGAD-7: 7-item Generalized Anxiety Disorder Questionnaire.
In an unadjusted Poisson regression examining if engagement in at least 1 telecoaching or teletherapy was associated with self-directed digital resources use, engagement in any telecoach sessions was associated with a 71.6% increase in the number of self-directed digital resources used, while engagement in any teletherapy sessions was associated with a 41.4% increase in the number of digital resources used (Table 4). After adjusting for the additional predictor variables described above, engagement in any telecoaching sessions was associated with an 82% increase in the number of self-directed resources used, and engagement in any teletherapy sessions was associated with a 79.5% increase in the number of digital resources used. In the adjusted analysis, age was a significant predictor of engagement, with each additional year of age associated with a 3.9% increase in the number of self-directed digital resources used. Relative to men, those identifying as women used 23% more digital resources and non-binary individuals did not statistically significantly differ. The primary topic of focus was also a significant predictor of self-directed resource engagement, with finances (relative risk [RR] 1.46, P=.02), physical well-being (RR 2.28 P<.001), professional life (RR 1.71, P<.001), and relationships (RR 1.37, P<.001) all being associated with increased digital resource use as compared with those who chose emotions. In the subgroup analysis conducted in participants with an elevated GAD-7 or PHQ-9 score, engagement in any telecoaching was associated with a 101% increase in the number of self-directed digital resources used (P<.001), and engagement in any teletherapist visit was associated with a 77.8% increase in the number of digital resources used (P<.001) (Table 5).
Table 4.
Unadjusted and adjusted Poisson regression models predicted the number of self-directed digital resources used.
| Variable | Unadjusted model (N=950) | Adjusted model (N=911) | |||||||
|
|
RRa (95% CI) | P value | RR (95% CI) | P value | |||||
| Coaching or therapy | |||||||||
|
|
Any coach sessions | 1.716 (1.542-1.908) | <.001 | 1.820 (1.629-2.033) | <.001 | ||||
|
|
Any therapy sessions | 1.414 (1.247-1.6) | <.001 | 1.795 (1.554-2.068) | <.001 | ||||
| Age | —b | — | 1.039 (1.033-1.045) | <.001 | |||||
| Race and ethnicity | |||||||||
|
|
White | — | — | 1.120 (0.998-1.258) | .06 | ||||
|
|
Non-white | — | — | REFc | — | ||||
| Gender identity | |||||||||
|
|
Nonbinary | — | — | 1.794 (1.425-2.236) | .95 | ||||
|
|
Woman | — | — | 1.230 (1.091-1.388) | <.001 | ||||
|
|
Man | — | — | REF | — | ||||
| Educational attainment | |||||||||
|
|
Less than bachelors | — | — | REF | — | ||||
|
|
Greater than bachelors | — | — | 0.991 (0.83-1.186) | .92 | ||||
|
|
Bachelors | — | — | 1.147 (0.975-1.356) | .10 | ||||
| Patient activation | |||||||||
|
|
PAMd,e score | — | — | 0.991 (0.987-0.996) | <.001 | ||||
| Mental health literacy | |||||||||
|
|
MHLSd,f score | — | — | 1.000 (0.991-1.01) | .97 | ||||
| Well-being | |||||||||
|
|
WHO-5d,g score | — | — | 1.001 (0.998-1.004) | .52 | ||||
| Primary concern | |||||||||
|
|
My emotions | — | — | REF | — | ||||
|
|
My finances | — | — | 1.456 (1.058-1.954) | .02 | ||||
|
|
My physical well-being | — | — | 2.257 (1.925-2.64) | <.001 | ||||
|
|
My professional life | — | — | 1.713 (1.457-2.009) | <.001 | ||||
|
|
My relationships | — | — | 1.365 (1.174-1.584) | <.001 | ||||
aRR: rate ratio.
bNot available.
cREF: reference.
dPAM, MHLS, and WHO-5 scores were entered as linear predictors.
ePAM: Patient Activation Measure.
fMHLS: Mental Health Literacy Scale.
gWHO-5: 5-item World Health Organization Well-Being Index.
Table 5.
Subgroup Poisson regression analysis conditional on elevated GAD-7 or PHQ-9 score (≥ 10) at baseline, predicting number of self-directed resources used (n=400).
| Variable | RRa (95% CI) | P value | |
| Use of therapy or coach | |||
|
|
Any coach visits | 2.099 (1.76-2.503) | <.001 |
|
|
Any therapist visits | 1.778 (1.455-2.165) | <.001 |
| Age | 1.036 (1.025-1.047) | <.001 | |
| Race and ethnicity | |||
|
|
Non-White | REFb | —c |
|
|
White | 1.218 (1.014-1.467) | .04 |
| Gender identity | |||
|
|
Man | REF | — |
|
|
Woman | 0.864 (0.71-1.053) | .14 |
|
|
Non-binary | 1.266 (0.859-1.819) | .22 |
| Educational attainment | |||
|
|
<Bachelors | REF | — |
|
|
>Bachelors | 1.005 (0.758-1.339) | .97 |
|
|
Bachelors | 1.203 (0.943-1.553) | .15 |
| Patient activation | |||
|
|
PAMd,e | 0.985 (0.977-0.993) | <.001 |
| Mental health literacy | |||
|
|
MHLSd,f | 1.008 (0.993-1.023) | .31 |
| Well-being | |||
|
|
WHO-5d,g | 0.993 (0.987-0.999) | .03 |
| Primary concern | |||
|
|
My emotions | REF | — |
|
|
My finances | 2.015 (1.202-3.191) | .005 |
|
|
My physical well-being | 2.411 (1.865-3.095) | <.001 |
|
|
My professional life | 2.318 (1.814-2.945) | <.001 |
|
|
My relationships | 1.350 (1.04-1.735) | .02 |
aRR: rate ratio.
bREF: reference range.
cNot applicable.
dPAM, MHLS, and WHO-5 scores were entered as linear predictors.
ePAM: Patient Activation Measure.
fMHLS: Mental Health Literacy Scale.
gWHO-5: 5-item World Health Organization Well-Being Index.
Discussion
Principal Findings
Our study aimed to evaluate predictive factors of engagement in multiple digital health modalities offered by a digital mental health employer-based benefit. Over half of the participants (55.9%, 531/950) used at least 1 component of the digital mental health platform at least once. It is important to note that due to the nature of employer benefits programs, employers often choose to market their mental health services benefit at specific time points, such as open enrollment periods, which leads to an increase in sign-ups. However, not everyone needs or wants mental health care at the time they receive the email advertising the benefit. Thus, it is possible that people initiate with a mental health digital platform before they need or are ready to engage in care. Or, if they do not sign up at the time of the email, they may forget about the program at a time when they do need care. These findings underscore the need to continue adapting digital health and other digital resources in ways that engage broader populations of users curious about mental health resources and ensure that people know how to access care if they need it. Our study did identify several factors that were associated with increased use of the platform, including the use of a coach or therapist, older age, identifying as a woman, baseline symptom severity, and presenting primary concerns of finances, physical well-being, professional life, and relationships.
A key finding of our study was that the use of coaching or therapy was associated with increased self-directed digital resource use. This relationship was even stronger for individuals with elevated GAD-7 and PHQ-9 scores. Higher self-directed use in this population may be the result of provider encouragement. Coaches and therapists seeing participants with elevated GAD-7 and PHQ-9 scores might be inclined to recommend additional use of digital health resources outside of one-on-one sessions, resulting in more self-directed use for those groups. Thus, additional education for platform mental health coaches and therapists about available digital support tools and their quality and effectiveness might facilitate greater user engagement in app-based resources. Training for the network of platform providers could also include information on the barriers and facilitators that sustain engagement in digital resources. Indeed, previous research has highlighted the impact clinicians can have on digital health interest, use, and engagement [29-31]. While clinicians may be recommending self-directed strategies and driving use, alternatively, it may be that when a participant engages in 1 modality of the platform, they are more likely to become “super users” across multiple modalities. Future research is needed to disentangle the relationship between live support and self-directed resource engagement.
Furthermore, certain topics of focus were more associated with self-directed resource use, including finances, physical well-being, professional life, and relationships. Participants whose primary chosen topic was “[their] emotions” were less likely to engage with self-directed digital resources, though almost half (48.1%, n=224) of this group used one-on-one modalities. Given that higher baseline depression was also associated with less self-directed digital engagement, perhaps features of depression, such as lack of motivation, energy, or concentration, could decrease the likelihood of using these resources, even when the need for them is higher. This result is consistent with findings from a systematic review that human factors, such as severe mental health symptoms, can be a barrier to engagement [15]. This may also be a feature of the personalized care recommendations, which were more likely to suggest users work with a provider one-on-one if they were experiencing higher levels of distress. Future research may be helpful in determining how to make digital health self-directed resources more appealing for individuals struggling with low mood who are not already engaging in therapy or coaching. Research is also needed to better determine the circumstances when self-guided care is a clinically effective stand-alone approach for people with depressive symptoms.
Several sociodemographic characteristics, including identifying as a woman, higher education, and older age, were also associated with greater self-guided resource use. The extant literature has demonstrated that women are more likely to receive mental health treatment in general [32,33] and to use digital mental health resources specifically [34]. Our finding extends previous literature by revealing that at the intersection of digital health and mental health treatment, women users are more likely to engage with the technology than men. Higher education was also related to increased use of self-directed resources, as well as overall increased use of the platform more generally. Some research indicates that education is a significant predictor of mental health service use [35-37]. Finally, older age was associated with more self-directed resource use, though given the restricted age range of this sample, this result may be more challenging to interpret. It is possible that people in their 30s and 40s are more time-constrained by career and caregiving responsibilities and may be more likely to engage in mental health services they can access when it is convenient to them.
This study had several limitations with regard to the sample and restrictions of data. First, participants were eligible for the study if they had matched with a provider or used at least 1 digital mental health resource. This inclusion criteria likely selected participants who were more predisposed to engaging with digital mental health resources, possibly biasing our sample to more resource use. In addition, the digital mental health resource use that made participants eligible for inclusion in the study is included in the total number of times a participant used these resources, likely inflating the count of times used. Next, our sample may not be generalizable to other samples, given that we used participants from companies that were contracted with 1 mental health platform. Thus, the sample was limited to working adults with benefits-eligible positions, which skews towards a higher level of socioeconomic status.
Another limitation of this study was our inability to determine the sequence of events for each participant, for example, whether they engaged in a coaching session and then self-directed resource use or vice versa. This information would be helpful in further breaking down predictors of digital health use. For example, do sessions with a coach then lead to subsequent self-directed resource use, or does an attempt at self-directed resources lead to a request for additional live support by coaching or therapy? Future research can disentangle the timing of the use of various modalities. Third, the platform design and user experience may have driven engagement to some extent, and future research is needed to explore intervention-level factors (eg, ease of use, perceived effectiveness) that may affect treatment engagement.
Next, we used a definition of engagement based on the number of times a participant used the self-directed digital mental health resources. In addition to the volume of care used, it is important to understand whether users are engaging in the right level of care based on their needs. Although we found alignment between self-guided resource use and user needs (eg, users with lower depression symptoms and less mental health-focused topics of concern engaged in resources on their own, while users with higher depression and anxiety were more likely to engage with resources in combination with one-on-one care), additional data could provide a more nuanced look at engagement. For example, future studies could examine whether users engaged in their recommended care pathway and if they saw improved mental health outcomes as a result of engaging.
Conclusion
Digital mental health platforms, including those using a stepped-care model, have the potential to deliver appropriate and evidence-based care to large populations of individuals in need. However, these platforms are affected by similar challenges related to engagement that plague many of the high-quality, evidence-based psychological treatment modalities currently available. This study revealed that users were more likely to use self-directed digital mental health resources if they were also engaged in at least 1 coaching or therapy session, which suggests that the key to successful engagement with digital therapeutics may lie in additional live support.
Acknowledgments
We would like to thank Camille Welcome Chamberlain for contributing to this project. We would also like to thank the platform members and providers for their contributions to this work, as well as the employers who made this mental health benefit available to their employees. The authors received research support from Modern Health to conduct this work. The funder had no role in the design of the study, in the collection, analyses, or interpretation of data, in the writing of the manuscript, or in the decision to publish the results.
Abbreviations
- GAD-7
7-item Generalized Anxiety Disorder Questionnaire
- MHLS
Mental Health Literacy Scale
- PAM-MH
Patient Activation Measure for Mental Health
- PHQ-9
9-item Patient Health Questionnaire
- RR
rate ratio
- WHO-5
5-item World Health Organization Well-Being Index
Data Availability
The data set analyzed in this study is not publicly available as it is proprietary information from Modern Health, but are available from the corresponding author upon reasonable request.
Footnotes
Authors' Contributions: MN contributed to conceptualization, formal analysis, methodology, and writing-original draft. MM contributed to conceptualization, methodology, and writing–original draft. LB contributed to writing the original draft. EK contributed to supervision, writing-review and editing. SSH contributed to investigation, writing–review and editing. CCS contributed to investigation, writing–review and editing. EK contributed to conceptualization, methodology, supervision, and writing–original draft.
Conflicts of Interest: SSH and CSS were employed by Modern Health and received salary and stock options at the time this study was conducted.
References
- 1.Kamal R, Cox C, Rousseau D, Kaiser Family Foundation Costs and outcomes of mental health and substance use disorders in the US. JAMA. 2017 Aug 01;318(5):415. doi: 10.1001/jama.2017.8558.2646703 [DOI] [PubMed] [Google Scholar]
- 2.Reeves WC, Strine TW, Pratt LA, Thompson W, Ahluwalia I, Dhingra SS, McKnight-Eily LR, Harrison L, D'Angelo DV, Williams L, Morrow B, Gould D, Safran MA, Centers for Disease Control Prevention (CDC) Mental illness surveillance among adults in the United States. MMWR Suppl. 2011 Sep 02;60(3):1–29. https://www.cdc.gov/mmwr/preview/mmwrhtml/su6003a1.htm .su6003a1 [PubMed] [Google Scholar]
- 3.Wang PS, Lane M, Olfson M, Pincus HA, Wells KB, Kessler RC. Twelve-month use of mental health services in the United States: results from the National Comorbidity Survey Replication. Arch Gen Psychiatry. 2005 Jun;62(6):629–40. doi: 10.1001/archpsyc.62.6.629. https://jamanetwork.com/journals/jamapsychiatry/fullarticle/208673 .62/6/629 [DOI] [PubMed] [Google Scholar]
- 4.Coombs NC, Meriwether WE, Caringi J, Newcomer SR. Barriers to healthcare access among U.S. adults with mental health challenges: A population-based study. SSM Popul Health. 2021 Sep;15:100847. doi: 10.1016/j.ssmph.2021.100847. https://linkinghub.elsevier.com/retrieve/pii/S2352-8273(21)00122-1 .S2352-8273(21)00122-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Olfson M. Building the mental health workforce capacity needed to treat adults with serious mental illnesses. Health Aff (Millwood) 2016 Jun 01;35(6):983–90. doi: 10.1377/hlthaff.2015.1619. https://www.healthaffairs.org/doi/10.1377/hlthaff.2015.1619 .35/6/983 [DOI] [PubMed] [Google Scholar]
- 6.Thomas D, Macdowell M, Glasser M. Rural mental health workforce needs assessment - a national survey. Rural Remote Health. 2012;12:2176. https://www.rrh.org.au/articles/subviewnew.asp?ArticleID=2176 .2176 [PubMed] [Google Scholar]
- 7.Agarwal S. Jalan M. Wilcox HC. Sharma R. Hill R. Pantalone E. Thrul J. Rainey JC. Robinson KA Evaluation of Mental Health Mobile Applications. Agency for Healthcare Research and Quality (US) technical brief, No. 41. 2022. May, [2023-07-02]. https://www.ncbi.nlm.nih.gov/books/NBK580948/ [PubMed]
- 8.Andrade LH, Alonso J, Mneimneh Z, Wells JE, Al-Hamzawi A, Borges G, Bromet E, Bruffaerts R, de Girolamo G, de Graaf R, Florescu S, Gureje O, Hinkov HR, Hu C, Huang Y, Hwang I, Jin R, Karam EG, Kovess-Masfety V, Levinson D, Matschinger H, O'Neill S, Posada-Villa J, Sagar R, Sampson NA, Sasu C, Stein DJ, Takeshima T, Viana MC, Xavier M, Kessler RC. Barriers to mental health treatment: results from the WHO World Mental Health surveys. Psychol Med. 2014 Apr;44(6):1303–17. doi: 10.1017/S0033291713001943. https://europepmc.org/abstract/MED/23931656 .S0033291713001943 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Mojtabai R, Olfson M, Sampson NA, Jin R, Druss B, Wang PS, Wells KB, Pincus HA, Kessler RC. Barriers to mental health treatment: results from the National Comorbidity Survey Replication. Psychol Med. 2011 Aug;41(8):1751–61. doi: 10.1017/S0033291710002291. https://europepmc.org/abstract/MED/21134315 .S0033291710002291 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Akter S, D’Ambra J, Ray P. Service quality of mHealth platforms: development and validation of a hierarchical model using PLS. Electron Markets. 2010 Nov 18;20(3-4):209–227. doi: 10.1007/s12525-010-0043-x. https://www.researchgate.net/publication/225654619_Service_quality_of_mHealth_platforms_Development_and_validation_of_a_hierarchical_model_using_PLS . [DOI] [Google Scholar]
- 11.Krebs P, Duncan DT. Health app use among us mobile phone owners: a national survey. JMIR Mhealth Uhealth. 2015 Nov 04;3(4):e101. doi: 10.2196/mhealth.4924. https://mhealth.jmir.org/2015/4/e101/ v3i4e101 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Kayyali R, Peletidi A, Ismail M, Hashim Z, Bandeira P, Bonnah J. Awareness and use of mHealth Apps: A study from England. Pharmacy (Basel) 2017 Jun 14;5(2):33. doi: 10.3390/pharmacy5020033. https://www.mdpi.com/resolver?pii=pharmacy5020033 .pharmacy5020033 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Kullgard N, Holmqvist R, Andersson G. Premature dropout from psychotherapy: prevalence, perceived reasons and consequences as rated by clinicians. Clin Psychol Eur. 2022 Jun;4(2):e6695. doi: 10.32872/cpe.6695. https://europepmc.org/abstract/MED/36397946 .cpe.6695 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Leichsenring F, Sarrar L, Steinert C. Drop-outs in psychotherapy: a change of perspective. World Psychiatry. 2019 Feb;18(1):32–33. doi: 10.1002/wps.20588. https://europepmc.org/abstract/MED/30600632 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Borghouts J, Eikey E, Mark G, De Leon C, Schueller SM, Schneider M, Stadnick N, Zheng K, Mukamel D, Sorkin DH. Barriers to and facilitators of user engagement with digital mental health interventions: systematic review. J Med Internet Res. 2021 Mar 24;23(3):e24387. doi: 10.2196/24387. https://www.jmir.org/2021/3/e24387/ v23i3e24387 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Vaghefi I, Tulu B. The continued use of mobile health apps: insights from a longitudinal study. JMIR Mhealth Uhealth. 2019 Aug 29;7(8):e12983. doi: 10.2196/12983. https://mhealth.jmir.org/2019/8/e12983/ v7i8e12983 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Rai A, Chen L, Pye J, Baird A. Understanding determinants of consumer mobile health usage intentions, assimilation, and channel preferences. J Med Internet Res. 2013 Aug 02;15(8):e149. doi: 10.2196/jmir.2635. https://www.jmir.org/2013/8/e149/ v15i8e149 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Kim M, Kim C, Kim E, Choi M. Effectiveness of mobile health-based exercise interventions for patients with peripheral artery disease: systematic review and meta-analysis. JMIR Mhealth Uhealth. 2021 Feb 15;9(2):e24080. doi: 10.2196/24080. https://mhealth.jmir.org/2021/2/e24080/ v9i2e24080 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Doherty K, Doherty G. Engagement in HCI: Conception, theory and measurement. ACM Comput. Surv. 2018 Nov 19;51(5):1–39. doi: 10.1145/3234149. [DOI] [Google Scholar]
- 20.Sidner CL, Kidd CD, Lee C, Lesh N. Where to look: a study of human-robot engagement. Proceedings of the 9th international conference on Intelligent user interfaces; January 13, 2004; Funchal, Madeira , Portugal. ACM; 2004. pp. 78–84. [DOI] [Google Scholar]
- 21.Kozlov E, Bantum E, Pagano I, Walser R, Ramsey K, Taylor K, Jaworski B, Owen J. The reach, use, and impact of a free mhealth mindfulness app in the general population: mobile data analysis. JMIR Ment Health. 2020 Nov 27;7(11):e23377. doi: 10.2196/23377. https://mental.jmir.org/2020/11/e23377/ v7i11e23377 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Kroenke K, Spitzer RL, Williams JBW. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med. 2001 Sep;16(9):606–13. doi: 10.1046/j.1525-1497.2001.016009606.x. https://europepmc.org/abstract/MED/11556941 .jgi01114 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Spitzer RL, Kroenke K, Williams JBW, Löwe Bernd. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med. 2006 May 22;166(10):1092–7. doi: 10.1001/archinte.166.10.1092.166/10/1092 [DOI] [PubMed] [Google Scholar]
- 24.Topp CW, Østergaard Søren Dinesen, Søndergaard Susan, Bech P. The WHO-5 Well-Being Index: a systematic review of the literature. Psychother Psychosom. 2015;84(3):167–76. doi: 10.1159/000376585. doi: 10.1159/000376585.000376585 [DOI] [PubMed] [Google Scholar]
- 25.Green CA, Perrin NA, Polen MR, Leo MC, Hibbard JH, Tusler M. Development of the Patient Activation Measure for mental health. Adm Policy Ment Health. 2010 Jul;37(4):327–33. doi: 10.1007/s10488-009-0239-6. https://europepmc.org/abstract/MED/19728074 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.O'Connor M, Casey L. The Mental Health Literacy Scale (MHLS): A new scale-based measure of mental health literacy. Psychiatry Res. 2015 Sep 30;229(1-2):511–6. doi: 10.1016/j.psychres.2015.05.064.S0165-1781(15)00369-8 [DOI] [PubMed] [Google Scholar]
- 27.Giebel C, Lord K, Cooper C, Shenton J, Cannon J, Pulford D, Shaw L, Gaughan A, Tetlow H, Butchard S, Limbert S, Callaghan S, Whittington R, Rogers C, Komuravelli A, Rajagopal M, Eley R, Watkins C, Downs M, Reilly S, Ward K, Corcoran R, Bennett K, Gabbay M. A UK survey of COVID-19 related social support closures and their effects on older people, people with dementia, and carers. Int J Geriatr Psychiatry. 2021 Mar;36(3):393–402. doi: 10.1002/gps.5434. https://europepmc.org/abstract/MED/32946619 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Shiwaku H, Doi S, Miyajima M, Matsumoto Y, Fujino J, Hirai N, Jitoku D, Takagi S, Tamura T, Maruo T, Shidei Y, Kobayashi N, Ichihashi M, Noguchi S, Oohashi K, Takeuchi T, Sugihara G, Okada T, Fujiwara T, Takahashi H. Novel brief screening scale, Tokyo Metropolitan Distress Scale for Pandemic (TMDP), for assessing mental and social stress of medical personnel in COVID-19 pandemic. Psychiatry Clin Neurosci. 2021 Jan;75(1):24–25. doi: 10.1111/pcn.13168. https://europepmc.org/abstract/MED/33225480 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Kozlov E, McDarby M, Pagano I, Llaneza D, Owen J, Duberstein P. The feasibility, acceptability, and preliminary efficacy of an mHealth mindfulness therapy for caregivers of adults with cognitive impairment. Aging Ment Health. 2022 Oct;26(10):1963–1970. doi: 10.1080/13607863.2021.1963949. https://europepmc.org/abstract/MED/34428993 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Yang WE, Shah LM, Spaulding EM, Wang J, Xun H, Weng D, Shan R, Wongvibulsin S, Marvel FA, Martin SS. The role of a clinician amid the rise of mobile health technology. J Am Med Inform Assoc. 2019 Nov 01;26(11):1385–1388. doi: 10.1093/jamia/ocz131. https://europepmc.org/abstract/MED/31373364 .5542898 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Simblett S, Pennington M, Quaife M, Theochari E, Burke P, Brichetto G, Devonshire J, Lees S, Little A, Pullen A, Stoneman A, Thorpe S, Weyer J, Polhemus A, Novak J, Dawe-Lane E, Morris D, Mutepua M, Odoi C, Wilson E, Wykes T. Key drivers and facilitators of the choice to use mhealth technology in people with neurological conditions: observational study. JMIR Form Res. 2022 May 23;6(5):e29509. doi: 10.2196/29509. https://formative.jmir.org/2022/5/e29509/ v6i5e29509 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Cunningham R, Crowe M, Stanley J, Haitana T, Pitama S, Porter R, Baxter J, Huria T, Mulder R, Clark MTR, Lacey C. Gender and mental health service use in bipolar disorder: national cohort study. BJPsych Open. 2020 Nov 06;6(6):e138. doi: 10.1192/bjo.2020.117. https://europepmc.org/abstract/MED/33153508 .S2056472420001179 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Terlizzi E, Zablotsky B. Mental health treatment among adults: United States, 2019. Hyattsville, MD: National Center for Health Statistics; 2020. [2024-10-11]. https://www.cdc.gov/nchs/data/databriefs/db380-H.pdf#:~:text=In%202019%2C%2019.2%25%20of%20adults%20had%20received%20any,men%20to%20have%20received%20any%20mental%20health%20treatment . [PubMed] [Google Scholar]
- 34.Smail-Crevier R, Powers G, Noel C, Wang J. Health-related internet usage and design feature preference for e-mental health programs among men and women. J Med Internet Res. 2019 Mar 18;21(3):e11224. doi: 10.2196/11224. https://www.jmir.org/2019/3/e11224/ v21i3e11224 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Johnson EM, Possemato K. Correlates and predictors of mental health care utilization for veterans with PTSD: A systematic review. Psychol Trauma. 2019 Nov;11(8):851–860. doi: 10.1037/tra0000461. https://europepmc.org/abstract/MED/30998061 .2019-21140-001 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Smith LD, McGovern RJ, Peck PL. Factors contributing to the utilization of mental health services in a rural setting. Psychol Rep. 2004 Oct;95(2):435–42. doi: 10.2466/pr0.95.2.435-442. [DOI] [PubMed] [Google Scholar]
- 37.Steele LS, Dewa CS, Lin E, Lee KLK. Education level, income level and mental health services use in Canada: associations and policy implications. Healthc Policy. 2007 Aug;3(1):96–106. https://europepmc.org/abstract/MED/19305758 . [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
The data set analyzed in this study is not publicly available as it is proprietary information from Modern Health, but are available from the corresponding author upon reasonable request.