The efficacy of app‐supported smartphone interventions for mental health problems: a meta‐analysis of randomized controlled trials

Abstract

Although impressive progress has been made toward developing empirically‐supported psychological treatments, the reality remains that a significant proportion of people with mental health problems do not receive these treatments. Finding ways to reduce this treatment gap is crucial. Since app‐supported smartphone interventions are touted as a possible solution, access to up‐to‐date guidance around the evidence base and clinical utility of these interventions is needed. We conducted a meta‐analysis of 66 randomized controlled trials of app‐supported smartphone interventions for mental health problems. Smartphone interventions significantly outperformed control conditions in improving depressive (g=0.28, n=54) and generalized anxiety (g=0.30, n=39) symptoms, stress levels (g=0.35, n=27), quality of life (g=0.35, n=43), general psychiatric distress (g=0.40, n=12), social anxiety symptoms (g=0.58, n=6), and positive affect (g=0.44, n=6), with most effects being robust even after adjusting for various possible biasing factors (type of control condition, risk of bias rating). Smartphone interventions conferred no significant benefit over control conditions on panic symptoms (g=–0.05, n=3), post‐traumatic stress symptoms (g=0.18, n=4), and negative affect (g=–0.08, n=5). Studies that delivered a cognitive behavior therapy (CBT)‐based app and offered professional guidance and reminders to engage produced larger effects on multiple outcomes. Smartphone interventions did not differ significantly from active interventions (face‐to‐face, computerized treatment), although the number of studies was low (n≤13). The efficacy of app‐supported smartphone interventions for common mental health problems was thus confirmed. Although mental health apps are not intended to replace professional clinical services, the present findings highlight the potential of apps to serve as a cost‐effective, easily accessible, and low intensity intervention for those who cannot receive standard psychological treatment.

The treatment of mental health problems is expected to change considerably over the next few decades as a result of the widespread availability of Internet and mobile‐device applications, and their use to deliver psychological interventions1, 2. This change is predicted to alleviate many barriers that stand in the way of people seeking or receiving treatment under the current model of health care delivery (e.g., insufficient number of trained professionals, geographical constraints, lack of anonymity), thereby vastly increasing the availability of psychological therapies3, 4.

Smartphone interventions, in particular, offer many advantages over other digital interventions (e.g., computer‐based), including their ability to allow users to engage in exercises and monitor symptoms in situ, in real time, and immediately before and after pivotal events, as well as their capacity to be accessed in private and at a time and location of choice5. However, some have noted possible risks with app‐based smartphone interventions, a crucial one being the ease with which users may have access to potentially ineffective or harmful interventions6. Thus, practitioners and the general population need up‐to‐date guidance on the evidence base and clinical utility of app‐supported smartphone interventions.

The efficacy of smartphone interventions for common and costly mental health problems, such as depressive and anxiety symptoms, was preliminarily documented in two recent meta‐analyses. Firth et al7, 8 identified a small number of randomized controlled trials (RCTs) that examined the efficacy of app‐based smartphone interventions on symptoms of anxiety (n=9) and depression (n=18) in both clinical and non‐clinical samples. Smartphone interventions were found to be significantly more efficacious in reducing symptoms of anxiety and depression than both waitlist (g=0.45 and g=0.56) and active control (mainly attention/placebo‐based) groups (g=0.19 and g=0.21). The authors found no evidence that various intervention features (e.g., in‐app feedback, mood monitoring features, theoretical orientation) were significantly associated with effect sizes, although larger effects were observed when in‐person feedback was not provided7. Preliminary findings from these meta‐analyses suggest that app‐supported smartphone interventions have potential in treating and preventing certain common and debilitating mental health problems.

Since the publication of those two meta‐analyses, which included data from RCTs published until May 2017, nearly 50 RCTs evaluating the efficacy of smartphone apps on various mental health outcomes have been conducted. Since most RCTs of smartphone apps have been published within the last couple of years, and interventions delivered through smartphone devices are attracting enormous public, scientific and media attention9, we expect that a significant number of additional RCTs will be conducted and published in the near future.

It is therefore timely, pertinent and necessary to conduct an updated meta‐analysis examining the efficacy of app‐supported smartphone interventions not only on symptoms of anxiety and depression, but also on other prevalent, costly and important mental health outcomes not examined in prior meta‐analyses, including stress levels, specific anxiety symptoms (e.g., social anxiety, post‐traumatic stress) and well‐being/quality of life.

The aims of the present meta‐analysis of RCTs were to evaluate the efficacy of app‐supported smartphone interventions on a range of mental health outcomes, and to examine whether various features related to the intervention (theoretical orientation, whether professional guidance was offered, whether reminders to engage were sent) and sample (degree of mental health problem) moderated the observed effect sizes.

METHODS

Search strategy and study selection

We searched four major online databases (Medline, PsycINFO, Cochrane databases, Web of Science) in December 2018, using the search terms (“smartphone*” OR “mobile phone” OR “cell phone” OR “mobile app*” OR “iphone” OR “android” OR “mhealth” OR “m‐health” OR “cellular phone” OR “mobile device*” OR “mobile‐based” OR "mobile health" OR “tablet‐based”) AND (“random*” OR “trial*” OR “allocat*”) AND (“anxiety” OR “agoraphobia” OR “phobia*” OR “panic” OR “post‐traumatic stress” OR “mental health” OR “mental illness*” OR “depress*” OR “affective disorder*” OR “bipolar” OR “mood disorder*” OR “psychosis” OR “psychotic” OR “schizophre*” OR “well‐being” OR “wellbeing” OR “quality of life” OR “self‐harm” or “self‐injury” OR “stress*” OR “distress*” OR “mood” OR “body image” OR “eating disorder*”). Reference lists of included studies and previous reviews were also hand‐searched to identify any further eligible studies.

A protocol for this review was registered via PROSPERO (CRD42019122136). There were three small deviations to the original protocol. First, we made a post‐hoc decision to include rather than exclude studies that incorporated an app‐supported smartphone intervention within a broader treatment program (e.g., additive or adjunctive designs). Second, we did not conduct meta‐analyses of head‐to‐head comparisons of CBT vs. non‐CBT‐based apps, as there was an insufficient number of relevant studies. Third, we included an additional moderator, i.e. whether the smartphone intervention was directly aimed at targeting the specific symptom of interest.

Included studies were English language RCTs that examined the effects of an app‐supported smartphone intervention, compared to either a control condition or an active intervention, and provided the outcome data required to calculate an effect size.

Both published and unpublished RCTs were eligible for inclusion. Provided the smartphone intervention was designed to improve mental health or general well‐being, no restrictions on the samples were applied. Trials of interventions delivered only in part via smartphone devices were also included, such as adjunctive designs (smartphone app + standard therapy vs. standard therapy alone) or blended intervention programs (when participants could access the app‐based intervention via smartphones or computers).

Control conditions were categorized as waitlist, assessment only, treatment as usual, informational and educational resources (e.g., website links, health tips), or attention/placebo controls (e.g., gaming apps, music‐listening conditions). Active interventions were categorized as standard face‐to‐face therapy, web‐based or computerized interventions, pharmacotherapy, and self‐monitoring conditions.

Studies were excluded if: a) the smartphone intervention did not address mental health or well‐being (e.g., interventions focusing on weight loss, physical activity, diabetes management, smoking cessation or alcohol use were excluded); b) a computerized intervention, a virtual reality exposure treatment, or a text messaging‐only intervention was delivered; and c) there was no relevant comparison condition (e.g., a two‐arm trial comparing two apps was excluded) or no outcome measure was reported. If a study did not include data for effect size calculation, the authors were contacted, and the study was excluded if they failed to provide the data.

JL screened all records, and full texts were obtained for potentially eligible RCTs. Two independent assessors (JL and MM) examined the full texts and selected eligible RCTs.

Quality assessment and data extraction

The quality of trials was assessed using four criteria from the Cochrane Risk of Bias tool10: adequate generation of allocation sequence, concealment of allocation to conditions; blinding of outcome assessors or the use of self‐report questionnaires; and dealing with incomplete outcome data (assessed as low risk when outcome data used to calculate effect size were based on intention‐to‐treat analyses). JL conducted the quality assessments and MM coded a random 40% of studies, with good agreement observed between raters (kappa = 0.77, 0.69, 1.00 and 0.91, respectively). Disagreements were resolved through in‐depth discussion.

We also coded the participant characteristics (target sample, mean age); the characteristics of the smartphone intervention (name, theoretical orientation, whether the app contained mindfulness components); the comparison condition; the outcome measures; and other trial characteristics (sample size, whether guided support or reminders to engage were offered, length of post‐assessment).

Meta‐analysis

For each comparison between a smartphone intervention and a control or active intervention condition, the effect size was calculated by dividing the difference between the two group means by the pooled standard deviation at post‐test. The standardized mean difference (d) was then converted to Hedges' g to correct for small sample bias11. If means and standard deviations were not reported, effect sizes were calculated using conversion equations from significance tests (e.g., t statistics).

To calculate a pooled effect size, each study's effect size was weighted by its inverse variance. A positive g indicates that the smartphone condition had better outcomes than the comparison condition. Effect sizes of 0.8 can be assumed to be large, while effect sizes of 0.5 are moderate, and effect sizes of 0.2 are small12. If data from both intention‐to‐treat and completer analyses were presented, the former were extracted and analyzed.

We selected and analyzed the following mental health outcomes, as a sufficient number of trials (≥3) reported these outcomes and allowed for a meta‐analysis: depressive symptoms; generalized anxiety symptoms; specific anxiety symptoms (social anxiety symptoms, panic symptoms, post‐traumatic stress symptoms); stress levels; quality of life/well‐being; general psychological distress; and positive and negative affect. If multiple measures of a given outcome variable were used, the mean of the effect sizes from each measure within the study was calculated, before the effect sizes were pooled.

Comprehensive Meta‐Analysis Version 3.0 was used for the analyses13. Since we expected considerable heterogeneity among the studies, random effects models were employed. Heterogeneity was examined by calculating the I² statistic, which quantifies heterogeneity revealed by the Q statistic and reports how much overall variance (0‐100%) is attributed to between‐study variance14. The 95% confidence intervals (CIs) for the I² statistic were also calculated.

Subgroup analyses were conducted to explore sources of heterogeneity under a mixed effects model, which pools studies within a subgroup using a random effects model, but tests for significant differences between subgroups using fixed effects models.

Publication bias was examined through the trim‐and‐fill procedure15, as well as Begg and Mazumdar rank correlation test.

RESULTS

Characteristics of included studies

A flow chart of the literature search is presented in Figure 1. Out of a total of 3,136 screened abstracts, 66 RCTs with 77 smartphone intervention conditions were included. A variety of smartphone apps were tested, most of which were based on cognitive and/or behavioral principles (n=35) and/or acceptance‐ or mindfulness‐based principles (n=38).

Figure 1.

PRISMA flow chart of literature search. RCT – randomized controlled trial

Numerous trials used some indication of mental health problems as an inclusion criterion for study entry (n=38), which most frequently included those presenting with elevated levels (either at a diagnostic or subthreshold level) of depression (n=14), anxiety (n=9), or stress (n=8). Several trials (n=28) did not use any indication of mental health problems as an inclusion criterion (e.g., general community sample, student samples), but rather targeted general well‐being in these samples.

The quality of RCTs varied. Fifty trials (75.7%) reported an adequate sequence generation; 24 (36.4%) used adequate allocation concealment; four (6.1%) reported blinding of outcome assessors and 62 (93.9%) used self‐report questionnaires (so that direct interaction with an assessor was not required); and 37 studies (56.1%) reported data needed to calculate an effect size based on the intention‐to‐treat principle. Seventeen trials (25.7%) met all four criteria, 16 (24.2%) met three criteria, 27 (40.9%) met two criteria, and six trials (9.1%) met one of the criteria.

Efficacy of smartphone interventions on depressive symptoms

Smartphone interventions vs. controls

Depressive symptoms were assessed as an outcome in 47 trials (71.2%), and 11 trials (16 comparisons) delivered an app that was specifically designed to target depressive symptoms.

The pooled effect size for the 54 comparisons between smartphone interventions and control conditions on depressive symptoms was g=0.28 (95% CI: 0.21‐0.36), with moderate heterogeneity (see Table 1 and Figure 2). The pooled effect size was somewhat larger when adjusting for potential publication bias (g=0.41, Begg and Mazumdar test: p=0.087) and when analyzing only low risk of bias trials (g=0.43).

Table 1.

Meta‐analysis of efficacy of mental health smartphone apps on depressive and generalized anxiety symptoms

	Depressive symptoms				Generalized anxiety symptoms
	N	g (95% CI)	I2 (95% CI)	Q	N	g (95% CI)	I2 (95% CI)	Q
Smartphone vs. controls
Overall effect	54	0.28 (0.21‐0.36) ***	54% (38‐66)		39	0.30 (0.20‐0.40) ***	63% (48‐73)
Adjusted for publication bias	37	0.41 (0.32‐0.49)			31	0.39 (0.28‐0.49)
Sensitivity analysis
One effect size per study (smallest)	41	0.28 (0.18‐0.37) ***	58% (40‐70)		28	0.31 (0.18‐0.43) ***	69% (54‐78)
One effect size per study (largest)	41	0.37 (0.29‐0.44) ***	41% (15‐59)		28	0.38 (0.27‐0.49) ***	64% (46‐75)
Low risk of bias only (all criteria met)	13	0.43 (0.31‐0.55) ***	41% (0‐68)		7	0.56 (0.39‐0.74) ***	56% (5‐80)
Control condition type
Waitlist	34	0.32 (0.22‐0.42) ***	52% (28‐67)		28	0.32 (0.19‐0.44) ***	63% (45‐75)
Informational resources	8	0.39 (0.21‐0.58) ***	60% (17‐80)		3	0.51 (0.14‐0.88) **	72% (17‐90)
Attentional/placebo control	11	0.12 (0.01‐0.23) *	6% (0‐31)		8	0.18 (0.07‐0.29) **	7% (0‐20)
Subgroup analyses
Target sample				0.151				0.358
Elevated symptoms entry criteria	13	0.38 (0.23‐0.52) ***	61% (30‐78)		10	0.36 (0.25‐0.47) ***	0% (0‐58)
Elevated symptoms not entry criteria	41	0.24 (0.16‐0.33) ***	50% (28‐64)		29	0.28 (0.15‐0.41) ***	70% (57‐79)
Smartphone intervention target				0.728				0.319
Directly aimed at targeting this outcome	16	0.26 (0.11‐0.41) ***	71% (52‐82)		12	0.24 (0.09‐0.38) **	44% (0‐69)
Not directly aimed at targeting this outcome	38	0.29 (0.21‐0.38) ***	43% (16‐61)		27	0.33 (0.21‐0.46) ***	68% (53‐78)
CBT‐based app				0.125				0.011
Yes	26	0.34 (0.23‐0.46) ***	64% (46‐76)		16	0.42 (0.26‐0.57) ***	76% (57‐100)
No	27	0.23 (0.14‐0.32) ***	22% (0‐50)		23	0.19 (0.11‐0.27) ***	0% (0‐43)
Contains mindfulness components				0.359				0.952
Yes	28	0.33 (0.24‐0.41) ***	24% (0‐50)		24	0.30 (0.20‐0.41) ***	43% (8‐64)
No	25	0.25 (0.12‐0.39) ***	68% (51‐78)		15	0.29 (0.10‐0.49) **	77% (63‐86)
ACT‐based app				0.903				0.967
Yes	9	0.30 (0.08‐0.53) **	33% (0‐66)		8	0.30 (0.11‐0.49) **	1% (0‐10)
No	44	0.28 (0.20‐0.37) ***	57% (39‐68)		31	0.30 (0.19‐0.41) ***	69% (55‐78)
Reminders to engage provided				0.065				0.004
Yes	34	0.32 (0.22‐0.42) ***	61% (43‐72)		23	0.39 (0.27‐0.52) ***	63% (42‐76)
No	20	0.18 (0.08‐0.29) **	16% (0‐45)		16	0.15 (0.04‐0.26) *	18% (0‐50)
Professional guidance provided				0.002				0.001
Yes	15	0.48 (0.34‐0.62) ***	46% (4‐69)		12	0.53 (0.36‐0.70) ***	60% (26‐78)
No	37	0.23 (0.15‐0.31) ***	32% (0‐54)		27	0.21 (0.12‐0.30) *	36% (0‐59)
Duration of post‐assessment				<0.001				<0.001
2‐6 weeks	33	0.17 (0.08‐0.26) ***	30% (0‐53)		24	0.11 (0.02‐0.21) *	12% (0‐41)
7‐11 weeks	18	0.46 (0.36‐0.55) ***	45% (3‐66)		15	0.52 (0.41‐0.63) ***	44% (0‐68)
12+ weeks	3	0.09 (–0.23 to 0.42)	49% (0‐80)		0	‐
Apps vs. active comparison
Overall effect	12	0.13 (–0.07 to 0.34)	60% (27‐78)		4	0.09 (–0.21 to 0.39)	32% (0‐68)
Low risk of bias trials only	4	–0.00 (–0.36 to 0.35)	77% (41‐90)		0	‐	‐
Smartphones as an adjunct intervention	4	0.26 (–0.09 to 0.61)	71% (26‐89)		1	0.05 (–0.27 to 0.38)	0%

N – number of comparisons, CBT– cognitive behavior therapy, ACT – acceptance and commitment therapy

^*p<0.05,

^**p<0.01,

^***p<0.001.

Bold prints indicate significant differences

Figure 2.

Effect of smartphone apps vs. control conditions on depressive symptoms

The pooled effect size was small but still statistically significant when attention/placebo control conditions were used (g=0.12), and larger when waitlist (g=0.32) or informational resources (g=0.39) were used as control conditions.

In the previous analyses, we included a few trials in which more than one intervention condition was compared with the same control condition (or vice versa). These comparisons were not independent from each other, which may have artificially reduced the heterogeneity estimate and affected the pooled effect size. To deal with this, we ran sensitivity analyses in which the comparison with the smallest effect size was only included in the analysis, and then repeated this again for the comparison with the largest effect size. These sensitivity analyses ensured that only one comparison per study was included in the meta‐analysis. These sensitivity analyses yielded a pooled effect size highly similar to the overall effect (Table 1).

Subgroup analyses

Subgroup analyses were conducted to test whether various participant or trial characteristics were significantly associated with the pooled effect size (Table 1).

Studies that offered professional guidance (e.g., regular supportive text messages, phone calls, or personalized feedback from therapists or research staff) produced larger effect sizes than studies that did not offer guidance. Studies with a follow‐up length between 7 and 11 weeks produced larger effect sizes than studies with a follow‐up length of 2‐6 or ≥12 weeks. No other study characteristics were significantly associated with effect sizes.

Smartphone interventions vs. active comparisons

The pooled effect size for the 12 comparisons between smartphone interventions and active comparisons was g=0.13 (95% CI: –0.07 to 0.34), with moderate heterogeneity. Non‐significant effect sizes were observed for low risk of bias trials.

Additive effects of smartphone interventions

Four trials examined whether adding a smartphone intervention to a standard intervention (face‐to‐face, computerized, pharmacotherapy) was superior to a standard intervention‐only condition. The pooled effect size for the four comparisons between smartphone intervention + standard intervention vs. the standard intervention‐only arm was g=0.26 (95% CI: –0.09 to 0.61).

Efficacy of smartphone interventions on generalized anxiety symptoms

Smartphone interventions vs. controls

Generalized anxiety symptoms were assessed as an outcome in 29 studies (43.9%), and eight studies (12 comparisons) delivered an app that was specifically designed to target generalized anxiety symptoms.

The pooled effect size for the 39 comparisons was g=0.30 (95% CI: 0.20‐0.40), with high heterogeneity (see Table 1 and Figure 3). It remained statistically significant across all sensitivity analyses. Begg and Mazumdar test was non‐significant (p=0.217).

Figure 3.

Effect of smartphone apps vs. control conditions on generalized anxiety symptoms

Subgroup analyses

Four statistically significant moderation effects were observed. Larger effect sizes were found by studies that used a CBT‐based app, that reminded participants to engage in the app, that offered professional guidance, and that had a longer post‐assessment duration (7‐11 weeks) compared to those with a shorter duration (2‐6 weeks).

Smartphone interventions vs. active comparisons

The pooled effect size for the four comparisons was g=0.09 (95% CI: –0.21 to 0.39), with moderate heterogeneity.

Efficacy of smartphone interventions on stress levels

Stress levels were assessed in 22 trials (33.3%). The pooled effect size for the 27 comparisons was g=0.35 (95% CI: 0.21‐0.48), with high heterogeneity (Table 2). The pooled effect size remained statistically significant across the sensitivity analyses. Begg and Mazumdar test was non‐significant (p=0.392).

Table 2.

Efficacy of mental health smartphone apps on stress levels and quality of life outcomes

	Stress levels				Quality of life
	N	g (95% CI)	I2 (95% CI)	Q	N	g (95% CI)	I2 (95% CI)	Q
Smartphone vs. control conditions
Overall effect	27	0.35 (0.21‐0.48) ***	69% (55‐79)		43	0.35 (0.29‐0.42) ***	24% (0‐47)
Adjusted for publication bias	22	0.44 (0.30‐0.57)	‐		37	0.39 (0.32‐0.46)
Sensitivity analysis
One effect size per study (smallest)	22	0.38 (0.22‐0.54) ***	72% (58‐82)		34	0.36 (0.29‐0.44) ***	29% (0‐53)
One effect size per study (largest)	22	0.42 (0.28‐0.57) ***	65% (46‐77)		34	0.41 (0.35‐0.47) ***	0% (0‐36)
Low risk of bias only	4	0.78 (0.63‐0.04) ***	0% (0‐22)		10	0.46 (0.31‐0.61) ***	50% (1‐74)
Control type
Waitlist	20	0.47 (0.33‐0.62) ***	60% (36‐75)		37	0.35 (0.28‐0.43) ***	29% (0‐52)
Informational sources	1	0.06 (–0.32 to 0.44)	0		4	0.41 (0.21‐0.61) ***	0% (0‐84)
Attentional/placebo control	6	0.09 (–0.05 to 0.24)	0% (0‐74)		2	0.23 (0.03‐0.42) *	0%
Subgroup analyses
Target sample				0.010				0.084
Clinical sample	‐	‐	‐		6	0.24 (0.07‐0.41) **	0% (0‐57)
Symptomatic/at‐risk sample	7	0.59 (0.35‐0.83) ***	80% (61‐90)		12	0.44 (0.33‐0.56) ***	45% (0‐70)
Non‐clinical/non‐symptomatic sample	20	0.24 (0.09‐0.37) **	45% (9‐67)		25	0.31 (0.23‐0.39) ***	4% (0‐46)
CBT‐based app				0.003				0.823
Yes	8	0.61 (0.39‐0.83) ***	77% (56‐88)		19	0.37 (0.26‐0.48) ***	45% (8‐67)
No	19	0.21 (0.07‐0.35) **	45% (7‐67)		23	0.35 (0.27‐0.44) ***	0% (0‐39)
Contains mindfulness components				0.371				0.968
Yes	23	0.31 (0.19‐0.44) ***	58% (33‐73)		29	0.36 (0.29‐0.43) ***	7% (0‐33)
No	4	0.52 (0.09‐0.95) *	80% (52‐92)		13	0.36 (0.22‐0.49) ***	48% (4‐71)
ACT‐based app				0.252				0.305
Yes	5	0.16 (–0.17 to 0.49)	30% (0‐66)		13	0.29 (0.13‐0.44) ***	15% (0‐62)
No	22	0.38 (0.23‐0.52) ***	72% (58‐82)		29	0.38 (0.31‐0.45) ***	25% (0‐51)
Reminders to engage provided				0.066				0.025
Yes	20	0.41 (0.25‐0.57) ***	73% (57‐82)		29	0.39 (0.32‐0.47) ***	28% (0‐53)
No	7	0.19 (0.03‐0.35) *	1% (0‐71)		14	0.24 (0.14‐0.35) ***	0% (0‐41)
Professional guidance provided				0.010				0.001
Yes	10	0.57 (0.35‐0.79) ***	63% (29‐80)		13	0.52 (0.39‐0.64) ***	24% (0‐57)
No	17	0.24 (0.12‐0.36) ***	42% (0‐66)		30	0.29 (0.22‐0.35) ***	0% (0‐21)
Duration of post‐assessment				<0.001				0.971
2‐6 weeks	19	0.18 (0.06‐0.28) **	13% (0‐44)		31	0.35 (0.26‐0.44) ***	30% (0‐54)
7‐11 weeks	6	0.63 (0.38‐0.88) ***	83% (65‐91)		11	0.36 (0.26‐0.46) ***	17% (0‐51)
12+ weeks	2	0.59 (0.35‐0.83) ***	20% (0‐33)		1	0.31 (–0.13 to 0.74)	0%
Apps vs. active comparisons
Overall effect	2	0.21 (–0.46 to 0.88)	72% (27‐88)		6	0.02 (–0.14 to 0.17)	0% (0‐57)
Low risk of bias trials only	‐	‐	‐		3	–0.08 (–0.27 to 0.12)	0% (0‐66)

N – number of comparisons, CBT– cognitive behavior therapy, ACT – acceptance and commitment therapy

^*p<0.05,

^**p<0.01,

^***p<0.001.

Bold prints indicate significant differences

Four significant moderation effects were observed in subgroup analyses. Larger effect sizes were found by studies that used elevated stress levels as an entry criterion for trial inclusion, that used a CBT‐based app, that offered professional guidance, and that had a longer post‐assessment duration (≥ 7 weeks) compared to those with a shorter duration.

The pooled effect size for the two comparisons of smartphone vs. active interventions was g=0.21 (95% CI: –0.46 to 0.88), with high heterogeneity.

Efficacy of smartphone interventions on well‐being/quality of life

Measures of well‐being/quality of life were assessed in 36 studies (54.5%). The pooled effect size for the 43 comparisons was g=0.35 (95% CI: 0.29‐0.42), with low heterogeneity (Table 2). The pooled effect size remained statistically significant across all sensitivity analyses. Begg and Mazumdar test was non‐significant (p=0.622).

Two significant moderators were observed in subgroup analyses. Larger effect sizes were found by studies that reminded participants to engage, and by those that offered professional guidance.

The pooled effect size for the six comparisons of smartphone vs. active interventions was g=0.02 (95% CI: –0.14 to 0.17). A negative, non‐significant effect size was observed when restricting these analyses to low risk of bias trials.

Efficacy of smartphone interventions on other outcomes

Table 3 presents the meta‐analyses comparing smartphone interventions to control conditions on “other” outcomes.

Table 3.

Meta‐analysis comparing the effect of mental health smartphone apps vs. control conditions on other outcomes

Outcome measure	Analysis	N	g (95% CI)	I2 (95% CI)
General distress
	Overall effect	12	0.40 (0.24‐0.56) ***	60% (24‐77)
	Low risk of bias trials only	3	0.47 (0.08‐0.87) *	70% (15‐89)
Social anxiety symptoms
	Overall effect	6	0.58 (0.25‐0.90) ***	78% (53‐89)
	Low risk of bias trials only	3	0.76 (0.51‐1.03) ***	0% (0‐77)
Panic symptoms
	Overall effect	3	–0.05 (–0.41 to 0.31)	0% (0‐92)
	Low risk of bias trials only	2	0.12 (–0.41 to 0.65)	0%
Post‐traumatic stress symptoms
	Overall effect	4	0.18 (–0.04 to 0.41)	0% (0‐86)
	Low risk of bias trials only	0	‐	‐
Positive affect
	Overall effect	6	0.44 (0.15‐0.73) **	67% (24‐85)
	Low risk of bias trials only	1	–0.05 (–0.46 to 0.35)	0%
Negative affect
	Overall effect	5	–0.08 (–0.48 to 0.32)	76% (45‐89)
	Low risk of bias trials only	1	0.26 (–0.14 to 0.67)	0%

^*p<0.05;

^**p<0.01;

^***p<0.001

Smartphone interventions were significantly more effective than control conditions in improving general psychological distress (g=0.40), social anxiety symptoms (g=0.58), and positive affect (g=0.44). No significant group differences were observed for panic symptoms (g=–0.05), post‐traumatic stress symptoms (g=0.18), and negative affect (g=–0.08), although the number of studies contributing to these analyses was low.

DISCUSSION

This meta‐analysis examined the efficacy of app‐supported smartphone interventions for a range of mental health problems. Our search identified 66 RCTs that tested several smartphone interventions on numerous distinct clinical and non‐clinical populations. Importantly, the majority of RCTs were published in the last two years, highlighting that this area of research is gaining significant momentum and is growing exponentially.

We found evidence that app‐supported smartphone interventions are efficacious for several common mental health problems. They significantly outperformed control conditions in improving depressive symptoms, anxiety symptoms (generalized anxiety and social anxiety), stress levels, general psychiatric distress, quality of life, and positive affect, with effect sizes ranging from g=0.28 to g=0.58. Crucially, these effects were robust even after performing various sensitivity analyses that adjusted for common biasing factors in RCTs, including the type of control condition, trial risk of bias rating, and publication bias57, 58.

The statistically significant effect sizes were observed in both symptomatic (e.g., people meeting diagnostic criteria or reporting elevated mental health symptoms) and non‐symptomatic (e.g., university students, general population) samples, further highlighting the potential that smartphone apps could bring within current models of mental health care. For instance, smartphone interventions could eventually serve as a low‐cost, easily accessible, and user‐friendly option for universal, selective or indicated preventive programs59. Smartphone interventions could also fit within the stepped‐care model, in which low intensity interventions are offered as a first step in treatment, with more intensive resources reserved for those who fail to respond60.

Studies that offered professional guidance (e.g., supportive text messages, personalized feedback, telephone calls) and engagement reminders were consistently associated with larger effect sizes on several mental health outcomes, although smartphone interventions still significantly outperformed control conditions even in the subset of studies that did not offer guidance or reminders. That therapist guidance and engagement reminders bolster the effectiveness of smartphone interventions is consistent with what has been observed in a series of meta‐analyses of Internet‐based and computerized psychological treatments61, 62, 63, 64. However, the involvement of a therapist can be costly and may thus restrict the capacity of smartphone apps to reach the millions of people around the world in need of (and who cannot gain access to) treatment.

It has been suggested that digital interventions may benefit from peer or automated support rather than human support systems38, 65. The development of automated support systems may be guided by machine learning principles, so that users could receive guidance or prompts that are customized to their own needs and in real time66. Automated (and personalized) support has been shown to produce equivalent clinical outcomes to human support in RCTs of computerized treatments67, which suggests that developing and testing automated support systems for smartphone apps may be an important avenue for future research.

We also investigated whether the theoretical orientation of the smartphone intervention was associated with effect sizes. While interventions containing mindfulness‐ or acceptance‐based components were not associated with effect sizes, CBT‐based interventions produced larger effects for anxiety and stress. However, conclusions concerning the relative efficacy of different theoretical orientations would be premature at this stage, as too few head‐to‐head comparisons of different smartphone interventions have been performed, and those that compared CBT vs. non‐CBT‐based smartphone interventions reported no differences in level of symptom improvement68, 69, 70.

Smartphone interventions did not significantly differ from active interventions on any outcome. These findings, although preliminary, are in line with reports regarding Internet‐based treatments71, and point further toward the clinical utility of mental health apps. However, we note that few studies contributed to these head‐to‐head comparisons, so these analyses may have been underpowered. Power may have also been an issue for the other outcomes in which smartphone interventions conferred no benefit over control conditions (panic, post‐traumatic stress, and negative affect). Alternatively, it could be that the content quality of smartphone apps for these specific symptoms needs to improve72.

Limitations to the present meta‐analysis must be considered. First, possible negative effects of smartphone interventions (e.g., deterioration rates)73 were not assessed, since they were not reported in the included studies. Future studies should examine these possible negative effects74. Second, we did not analyze the long‐term effects of smartphone interventions, due to large differences in follow‐up times and since drop‐outs were dealt with inconsistently across studies. Thus, it is unclear whether improvements in mental health are sustained after the period of the study. Assessing the long‐term efficacy of smartphone interventions is an important future goal, particularly since promising long‐term effects have been noted in Internet‐based trials75. Third, nearly all included studies assessed outcomes via self‐report questionnaires. A previous meta‐analysis demonstrated that clinician‐rated instruments yield significantly larger effect sizes in psychotherapy trials than self‐reported measures76. So, it is possible that our effect size estimates were slightly underestimated.

In summary, we found evidence for the efficacy of app‐supported smartphone interventions. They significantly outperformed (with small to moderate effect sizes) control conditions in improving a range of mental health outcomes, with effects remaining robust even after adjusting for various biasing factors in RCTs. Studies that offered professional guidance and engagement reminders were shown to produce the largest effects, and smartphone interventions did not differ significantly from active intervention comparisons on any outcome.

Although mental health apps are not here to replace professional clinical services, the present findings highlight the potential of apps to serve as a cost‐effective, easily accessible, and low intensity intervention for the millions of people worldwide who cannot receive standard psychological treatment.