Skip to main content
Journal of the American Medical Informatics Association : JAMIA logoLink to Journal of the American Medical Informatics Association : JAMIA
. 2015 May 21;23(1):221–229. doi: 10.1093/jamia/ocv043

Effectiveness of mobile technologies delivering Ecological Momentary Interventions for stress and anxiety: a systematic review

Brendan Loo Gee 1,1,, Kathleen M Griffiths 1, Amelia Gulliver 1
PMCID: PMC7814930  PMID: 25997643

Abstract

Objectives Mobile technologies may be suitable for delivering Ecological Momentary Interventions (EMI) to treat anxiety in real-time. This review aims to synthesize evidence on the effectiveness of EMI for treating anxiety conditions.

Materials and Methods Four databases and the reference lists of previous studies were searched. A total of 1949 abstracts were double screened for inclusion. Sufficient studies were available to undertake a quantitative meta-analysis on EMIs on generalized anxiety symptoms.

Results The 15 randomized trials and randomized controlled trials examined anxiety ( n = 7), stress ( n = 3), anxiety and stress ( n = 2), panic disorder ( n = 2), and social phobia ( n = 1). Eight EMIs comprised self-monitoring integrated with therapy modules, seven comprised multimedia content, and three comprised self-monitoring only. The quality of studies presented high risk of biases. Meta-analysis ( n = 7) demonstrated that EMIs reduced generalized anxiety compared to control and/or comparison groups (Effect Size (ES) = 0.32, 95% CI, 0.12-0.53). Most EMIs targeting stress were reported effective relative to control as were the two EMIs targeting panic disorders. The EMI targeting social phobia was not effective.

Discussion EMIs have potential in treating both anxiety and stress. However, few high-quality trials have been conducted for specific anxiety disorders. Further trials are needed to assess the value of EMI technologies for anxiety in enhancing existing treatments.

Conclusion This study found a small significant effect of EMI studies on reducing generalized anxiety. Studies on stress demonstrated EMI was effective compared to control, with the small number of studies on panic and social phobia demonstrating mixed results.

Keywords: telemedicine, mobile health, anxiety, anxiety disorders, psychological stress

BACKGROUND AND SIGNIFICANCE

Anxiety disorders are among the most common mental disorders in adults, 1–3 with a 12 month prevalence of 14% among Australians aged 16–85 4 and an estimated annual and lifetime prevalence of 18–29% in the United States 2,3 and 13–14% in Europe. 1

Although effective psychological treatments for anxiety disorders can be delivered face-to-face in a clinical setting, 5 or online through a desktop computer, 6 changes in anxiety symptoms often arise in specific moment-to-moment real-life situations that only emerge outside the therapist’s office within a real-world natural setting. 7 Thus, the optimal treatment of anxiety symptoms may require innovative approaches in which a person’s anxiety symptoms are directly targeted in real time.

A possible solution to treating anxiety symptoms in real-time is the employment of Ecological Momentary Interventions (EMIs). EMIs are “momentary health treatments provided via hand-held mobile technologies that deliver psychological interventions while people are engaged in their typical routines in their everyday life.” 8 EMIs can be used as an adjunct to existing psychological therapies delivered by a therapist or they can be implemented as a stand-alone intervention. 8,9 When delivered as an adjunct, EMIs have the potential to reduce the amount of clinician contact required to treat anxiety conditions or to improve the effectiveness of clinician-delivered therapy. There is also the potential for EMIs to improve outcomes when used as a stand-alone intervention, although a priori it might be anticipated that EMIs would be most effective when delivered as an adjunct to clinical care. 10

Carter et al.11 described the differing complexities of EMIs through a multilevel classification framework. First, EMIs can be a “simple low-level” informative intervention, providing either health information on a specific health issue or consumer support material at momentary periods. Second, EMIs can be “interactive” interventions allowing individuals to record their psychological, behavioral, and contextual states at momentary periods, and to display this information on request. Third, EMIs can incorporate a high-level “integrative” feature that computationally detects and interprets patterns of an individual’s momentary input, and uses the resulting information to tailor the intervention to the individual. The three types of EMIs can be deployed on hand-held devices such as mobile phones, portable device assistants (PDAs), palmtop computers, ambulatory biofeedback devices, and portable digital media players.

Historically, pocket-sized ambulatory biofeedback and hand-held computers have been used in studies delivering EMIs for anxiety. Regular ambulatory biofeedback devices have shown promise in the context of panic disorders, especially in directing an individual’s attention to physiological changes through continuous self-monitoring. 12 Additionally, hand-held computers such as PDAs and palmtop computers have enabled programmable capabilities for the ability to track symptoms and to deliver relaxation training and cognitive behavioral therapy (CBT) material for generalized anxiety. 13,14 Since the early 2000s, however, and the introduction of modern mobile phones, technologies such as PDAs and palmtop computers have become obsolete. 15

Mobile phone technologies include regular mobile phones and smartphones. In the past decade, the technical capabilities of mobile phones have advanced with the emergence of smartphones. 16 These portable devices incorporate a specific operating system platform that enables greater computing power and extra network connectively to various electronic devices and the internet. 15,16 Furthermore, smartphone technologies can wirelessly connect to wearable sensors to detect changes in physiological factors (such as heart rate) and to provide real-time feedback of sensor information or tailored therapeutic content. 17 Given these advances, such technologies have significant potential to improve health outcomes including mental health, and are of particular utility for stress and anxiety disorders, given the strong physiological component of these conditions.

A recent paper by Klasnja and Pratt 18 provided a framework specifically for mapping features of modern mobile phones onto different health intervention strategies. They identified five mobile phone features including text messaging, cameras, automated sensing, Internet access, and native applications (such as programming libraries for global positioning system, accelerometer, graphic and audio, notification, contact list, and calendar functions.) They mapped these onto five health intervention strategies, which included tracking health information, involving the healthcare team, leveraging social influence, increasing access to health information, and using entertainment. Some mobile phone features were employed in each health strategy (e.g., text messaging); others were restricted to fewer strategies (e.g., cameras for tracking health information and involving the health care team). Although they provided a review of studies on mobile health interventions for various health conditions, Klasnja and Pratt 18 did not specifically report on the effects of interventions for mental health problems.

There have been a small number of reviews of studies on the effectiveness of EMIs for mental health problems. 8,19,20 However, to our knowledge there have been no comprehensive systematic reviews of the effectiveness of EMIs for anxiety conditions. One previous narrative review assessed EMI for anxiety disorders 12 but it focused on panic disorders and phobia, and it did not employ systematic identification and synthesis techniques. Accordingly, this study reports on a systematic review of the effectiveness of EMIs for stress, anxiety symptoms, and anxiety disorders. Furthermore, this study examines the specific technical features of mobile technologies that deliver different EMIs for these conditions.

METHOD

This systematic review conforms to the Preferred Reporting Items for Systematic Review and Meta-Analysis statement. 21

Search strategy

The Cochrane Library, PUBMED, OvidSP (including MEDLINE and PsycInfo), and Science Direct databases were searched in January 2014 using search terms for a combination of the following three main concepts: “ecological momentary intervention,” “anxiety,” and “mobile technologies” (a list of specific search terms are available in online supplement 1 ). MeSH and subject heading keywords from relevant databases were included. The search was restricted through limit functions on the databases to “clinical trials” and “randomized controlled trials (RCTs).” No restriction was applied on publication date. Additional studies were identified by manually searching the reference lists of relevant studies that were not identified by the database search.

Inclusion criteria

Studies were included if they 1) evaluated the effectiveness of an ecological momentary intervention (as defined above); 2) examined anxiety symptoms or anxiety disorder or anxiety related outcomes such as stress or tension; 3) were published in English language in a peer-reviewed journal; 4) employed a RCT or randomized trial methodology and included at least one control group or second experimental/comparison; and 5) examined an EMI with or without human support.

Exclusion criteria

We excluded studies that 1) only measured the adherence to or usability of an intervention; 2) did not measure an anxiety or anxiety related outcome; 3) examined chemical and biological stress, such as oxidative stress and starvation-stress responses but not anxiety-related psychological symptoms; 4) examined interventions that were not undertaken within real-time and/or a real world setting (this included studies using biofeedback treatment confined to be laboratory as such interventions are likely to have low ecological validity); 5) used ecological momentary assessment with the sole aim of monitoring a particular psychological phenomenon over time; or 6) did not employ a mobile or hand-held electronic computer as part of the intervention or treatment.

Selection of studies

As shown in Figure 1 , a total of 3145 records were retrieved from the database search and 35 references from two review papers on EMI for mental health, 8,20 in which 1231 records were duplicate abstracts, leaving 1949 unique records. Of these records, three raters (BLG and two research assistants) judged relevant records based on the titles and/or abstracts. From these, the full-text of 51 records were screened which yielded a total of 15 relevant papers.

Figure 1:

Figure 1:

Flowchart of systematic review.

Data extraction

Each of the 15 relevant papers was individually coded using a pre-formulated rating sheet by BLG and two other PhD students (RR, KA). The following information was extracted: participant characteristics and recruitment method, demographics, description of study design, intervention details (including the intervention type as defined by Mrazek and Haggerty’s 22 framework, EMI type as defined by Carter et al. , 11 the features of the mobile phone intervention as defined by Klasnja and Pratt’s framework, 18 and level of human contact coded based on categories defined by Newman et al.10 ), data analysis details, and qualitative information of treatment effects. EMIs were considered to be on-site or combined if the participant was required to travel to a physical location to receive face-to-face human support. Self-administered (SA) EMIs that required minimal travel for face-to-face human support were considered to be distal. Study quality was assessed using Cochrane Effective Practice and Organization of Care Group criteria, 23 ) for each study, and the totals and percentage of studies that were of high risk or unclear were calculated across studies.

Data analysis

Study characteristics were summarized using descriptive statistics. Effect size was calculated based on the available intention-to-treat and/or completer analyses results. Cohen’s d effect sizes were calculated to determine the between group effects at post-test to provide an estimate of the intervention’s effect. Where data was available, the intervention effect size was calculated based on the post-test mean and standard deviations from the study.

Given the heterogeneity of the included studies, an overall quantitative meta-analysis was not conducted. However, sufficient data were available to conduct a meta-analysis of the sub-group of studies examining generalized anxiety as an outcome measure. This meta-analysis was undertaken using the random effects model of the Comprehensive Meta-Analysis software program (Version 2.2.064). 24 The between-group effect size (standardized mean difference) was calculated at post-test except for one study wherein the effect size was based on F for difference in change 25 and a second study wherein the odds ratio was computed from reported improvement at post-test. 26 Effect sizes were combined across conditions and measures within a study such that only one data point was incorporated into the meta-analysis for each study. The I2 statistic was employed as a measure of heterogeneity between studies. 27 Publication bias was investigated using visual inspection of funnel plots and the Tweedie trim and fill procedure. 28

RESULTS

Study characteristics

The characteristics of each study are summarized in online Supplement 2 . The 15 studies examined anxiety ( n = 4), 25,26,29,30 generalized anxiety disorder ( n = 3), 31–33 stress ( n = 3), 34–36 anxiety and stress ( n = 2), 37,38 panic disorder ( n = 2), 39,40 and social phobia ( n = 1). 41 Three studies specifically targeted work-related stress. 34–36

Studies were conducted in a number of countries including Italy ( n = 6), 26,29,30,32,33,35 United States ( n = 2), 31,41 Australia ( n = 2), 37,38 Australia and United States ( n = 1), 40 Australia and Scotland ( n = 1), 39 Canada ( n = 1), 34 Finland ( n = 1), 36 and Mexico ( n = 1). 25

The mean age of participants across studies ranged from 17 to 55 years. Seven studies examined adults (aged 18 and over), 31–33,36,37,39,40 and one study focused on adolescents to young adults (aged 14–24 years). 38 The remaining seven studies provided the average age of the sample but did not specify the age group targeted. 25,26,29,30,34,35,41 No studies targeted older people. Females represented the majority of participants in 12 (80%) of the studies; only three studies composed more males than female participants. 34,36,41

The majority of studies recruited participants through patients at a health care facility. 25,32,33,38,39 Three studies recruited participants from the general community using newspaper advertisement and online media. 36,37,41 Two studies recruited staff of a health care facility. 34,35 A further two studies recruited participants from the general community at a public transport station, 29,30 and one study recruited participants from the general community and a mental health care facility. 31 Only one study exclusively recruited participants from a university setting. 40 Finally, one study did not state where participants were recruited. 26

Study quality

The sample size of the studies ranged from 13 to 720 (median 33.5). All included studies employed either an RCT or randomized trial design. Studies compared EMI groups with a wait-list control group, 32,33,36,39,41 a group that deployed no intervention, 25,29,30 an attention control group, 34,35,38 no control group, 31,40 both an attention and waitlist control groups, 37 or an unspecified control group. 26 Nine studies compared EMIs with another intervention. 26,29–33,39–41Table 1 presents the risk of bias assessment for each individual study. Most of the studies were unclear of high risk in study sequence generation, allocation concealment, the appropriate handling of incomplete data, and blinding.

Table 1:

Risk of bias assessed by Cochrane Quality Rating Criteria

Studies Sequence generation Allocation concealment Baseline outcomes Baseline factors Incomplete data Blinding Contamination protection Selection biases Other biases
Newman et al. (1997) 40 ? ? x x x ? x x x
Gruber et al. (2001) 41 ? ? x - x - x x x
Kenardy et al. (2003) 39 ? ? x ? x ? x x x
Riva et al. (2006) 30 ? ? ? ? - ? x - ?
Riva et al. (2007) 26 ? ? ? ? - ? x x ?
Grassi et al. (2009) 29 ? ? x x - ? x x ?
Mosso et al. (2009) 25 ? ? x x - - x x x
Pallavicini et al. (2009) 32 x x x x - ? x x x
Gorini et al. (2010) 33 ? ? ? ? - ? x x ?
Lemaire et al. (2011) 34 x x x - - - x x x
Reid et al. (2011) 38 x x x x x - x x x
Newman et al. (2013) 31 ? ? x - x - x x x
Villani et al. (2013) 35 ? ? x ? - ? x x ?
Proudfoot et al. (2013) 37 x x x - x ? x x x
Lappalainen et al. (2013) 36 ? ? x - x ? x x x
Unclear and high-risk studies, n (%) 11 (73.3) 11 (73.3) 3 (20) 10 (66.6) 8 (53.3) 15 (100) 0 (0) 1 (6.66) 5 (33.3)

x = low risk; ? = unclear information; - = high risk

Intervention characteristics

Six studies evaluated treatment interventions, 31–33,39–41 and a further five studies involved universal interventions delivered regardless of risk status and symptom levels. 25,26,29,30,34 Four studies examined indicated interventions in those with high levels of symptoms. 35–38 No studies evaluated selective interventions in those determined to be at risk for a disorder. There were a total of 18 EMIs evaluated in the 15 studies; 8 of these were integrative EMIs, 31–33,36,37,39–41 7 were simple EMIs, 25,26,29,30,32,33,35 and 3 were interactive EMIs. 34,38,40

Generalized anxiety

Of the seven studies examining EMIs targeting generalized anxiety symptoms, 25,26,29–33 four studies evaluated universal interventions, 25,26,29,30 and three studies examined treatment interventions. 31–33 Six of the seven studies investigated simple EMIs, which employed relaxation training that adopted narrative and distraction techniques. 25,26,29,30,32,33 The remaining study examined an integrative EMI using CBT and self-monitoring for treating generalized anxiety disorder. 31 There were no indicated interventions for generalized anxiety. None assessed an interactive EMI.

Stress

Three studies evaluated interventions aimed at managing stress 34–36 ; two of these examined indicated interventions, 35,36 and one evaluated a universal intervention. 36 Of these studies, one comprised a simple EMI employing relaxation training. 35 one investigated an interactive EMI using heart-rate self-monitoring plus relaxation training sessions. 34 Finally, one study examined an integrative EMI that included CBT, Acceptance and Commitment Therapy, relaxation, and self-monitoring. 36 No studies examined treatment interventions for stress alone.

Anxiety and stress

Two separate studies examined indicated interventions aimed at reducing anxiety and stress. 37,38 Of these studies, one study examined an interactive EMI that included self-monitoring, 38 and one study examined an integrative EMI that included CBT and self-monitoring. 37 No studies evaluated a universal intervention. None assessed a simple EMI.

Panic disorders

Two studies examined treatment interventions for panic disorder. 39,40 Both of these examined integrative and interactive EMIs which included computerized CBT and individualized face-to-face CBT, in conjunction with self-monitoring of panic symptoms. 39,40 One study examined the above treatment components with the addition of relaxation training. 40 No studies evaluated a universal or an indicated intervention nor used a simple EMI.

Social phobia

Only one study examined a treatment intervention for social phobia and it comprised an integrative EMI. 41 The intervention consisted of computerized CBT and group face-to-face CBT, in conjunction with symptom self-monitoring.

Types of EMI technologies

The 15 studies examined EMIs that deployed regular mobile phones, 26,29,30,35,37,38 smartphones with or without sensors, 25,32,33,36 hand-held computers, 31,39–41 and regular ambulatory biofeedback. 34 The table presented in online Supplement 3 show the details of the technical features of mobile technologies (including nonmobile phones) and related health intervention strategies as described by Klasnja and Pratt. 18 Native software features of the palmtop computer for diagnosed panic disorders and social phobia were used in a customized application that tracked health information via a diary module, and to assist in the clinician’s care through guided CBT modules via palmtop software. 31,39–41 One study examined regular ambulatory biofeedback. 34 This study monitored the heart rhythm pattern of individuals with stress while using this information to assist researchers in monitoring and documenting the participant’s adherence to the intervention.

Seven studies examined EMIs using mobile phones, smartphones, and other portable devices that utilized graphic, audio, and video features of the device to deliver relaxation training. 25,26,29,30,32,33,35 All of those studies examined interventional information targeting generalized anxiety or stress through an engaging virtual world simulation delivered on various mobile devices. Of these, one study used a smartphone and a portable virtual reality device to deliver relaxation content. 25 A further, two studies involved the delivery of similar relaxation content on touchscreen smartphones, gaming hardware, and wearable sensors. 32,33 These interventions tracked psychophysiological symptoms via wearable sensors and smartphones to provide the therapist with the ability to monitor symptoms during the delivery of the entertaining relaxation content. Lastly, two studies examined portable digital media players that delivered relaxation training content via MP3 audio with no video capabilities. 26,29

Three studies examined EMIs that tracked symptoms and delivered personalized therapeutic content via a mobile device. 36–38 One study that examined an intervention targeting stress used native features of smartphones, internet access, and wearable sensors to track symptoms, and to deliver fitness and relaxation training content. 36 Lastly, two studies that examined EMIs for anxiety and stress utilized text messaging, native application features, and internet access. 37,38 Both interventions used text messaging for self-monitor psychological symptoms, and for sending motivational reminders to encourage adherence to the intervention. However, one of the latter interventions used native application features of a mobile phone and a website to further facilitate doctor-client communication, 38 whereas the other intervention used these features to deliver guided online CBT information. 37 No studies examined interventions using cameras on mobile phones.

Intervention delivery

Seven studies delivered interventions both on-site and distally, 25,31–34,36,38 five were distal, 26,29,30,35,37 and three involved on-site and combined on-site and distal delivery. 31,39,41

Automation and human support

Six of the 18 EMIs were SA and fully automated using a standalone electronic intervention with low levels of human support. 25,26,29,30,35,37 Another four EMIs involved predominately self-help interventions with no >1.5 h of human support 32–34,38 while four EMIs were predominately therapist administered (TA), and required regular or high levels of human support. 31,39–41 The remaining EMIs were minimal-contact therapies, which involved >1.5 h of human support. 36,40

Outcome measures

Half of all studies used the State Trait Anxiety Inventory as the primary outcome measure for either state or trait anxiety. 26,29–33,35,39 Two studies targeting anxiety, stress and depression used the Depression, Anxiety, and Stress Scale. 37,38 Other measures of anxiety included the Hamilton Anxiety Rating Scale ( n = 3), the Penn State Worry Questionnaire ( n = 3), the Beck Anxiety Inventory ( n = 2), and a behavioral and cognitive assessment test on anxiety in social interactions ( n = 1). 31–33,41 Three studies targeting work-related stress used specific scales to measure distress induced by job demands, work burnouts, and organizational pressures. 34–36 The two panic disorder studies used the Mobility Inventory, the Agoraphobic Cognitions Questionnaire, the Fear Questionnaire, the Body Sensations Questionnaire, the Agoraphobia subscale, and the phobia subscales. 39,40 The social phobia trial used the Social Phobia Scale and Social Phobia and Anxiety Inventory. 41

EMI effectiveness on anxiety outcomes

Generalized anxiety ( n = 9)

Most ( n = 7) of the nine studies that evaluated EMIs specifically targeted generalized anxiety and used an anxiety measure. 25,26,29–33,37,38 These data were included in the meta-analysis. 25,26,29,31,32,37,38 All but one of these studies incorporated a control group. This study instead employed two active comparison groups, 31 comparing 6 weeks of group CBT plus EMI with (i) 6 weeks of group CBT; and (ii) 12 weeks of group CBT. Since the 12-week comparison intervention was not suitable for determining the effect of adding EMI to the 6-week group CBT, only the 6-week group CBT comparative data were incorporated into the analysis. The meta-analysis was run both with and without the inclusion of this study.

Table 2 and Figure 2 display the findings and forest plot of the meta-analyses of the seven studies reporting generalized anxiety outcomes. Overall, EMI interventions were effective in reducing anxiety symptoms. The pooled effect size was 0.32 (95% CI, 0.12- 0.53, P = .002) and heterogeneity among studies was low ( I2 = 8%). It is unlikely that publication bias influenced the conclusions. The fail-safe N was 16 for the meta-analysis of all studies and after adjusting for publication bias using Duval and Tweedie trim and fill procedure the pooled standardized effect size for the combined interventions remained statistically significant (adjusted effect size = 0.27; 95% CI, −0.04 to 0.46).

Table 2:

Meta-analysis of studies comparing the effects of EMI on generalized anxiety

N d (95% CI) Z P Q P I 2 Fail safe N
All Conditions
All EMIs (FEM) 7 0.32 (0.15-0.50) 3.57 <.001 6.52 0.37 7.96 16
All EMIs (REM) 0.32 (0.12-0.53) 3.12 .002
Excluding study with active control
All EMIs (FEM) 6 0.31 (0.13-0.49) 3.34 .001 6.08 0.30 17.78 10
All EMIs (REM) 0.31 (0.07-0.55) 2.54 .011

Note: FEM = fixed-effects model, REM = random-effects model.

Figure 2:

Figure 2:

Forest plot showing the effect of EMI on generalized anxiety (random effects model; a positive effect signifies a decrease in anxiety symptoms).

There was little change in the pooled effect size after excluding data from the Newman et al.31 study, which employed an active comparison group (pooled standardized effect size: 0.31; 95% CI, 0.07-0.55). Heterogeneity across studies was low ( I2 = 17.8%). Further, the fail safe N was 10 and after imputing potentially missing studies using the Duval and Tweedie procedure, the effect size remained statistically significant (adjusted effect size = 0.27, 95% CI, 0.006-0.50).

Stress ( n = 5)

Studies examining EMIs in reducing stress demonstrated positive effects for EMI relative to controls. 34,36,37 A study evaluating a transdiagnostic EMI comprising online CBT modules and SMS reminders for treating stress and anxiety 37 was associated with lower stress levels than a waitlist control ( d = 0.34 at post-test). Another study reported a reduction in work stress relative to waitlist control for an EMI that delivered CBT and Acceptance and Commitment Therapy therapies ( d = 0.21–0.63 at post-test). 36 Further, a study evaluating an EMI consisting of multimedia relaxation training 35 reported a reduction in work anxiety and stress relative to an attention control. Additionally, a study evaluating an EMI consisting of portable heart rate monitoring accompanied by breathing and positive emotion exercises 34 was associated with decreased work stress compared with attention control ( d = 0.19 at post-test). However, an EMI using self-monitoring on mobile phones 38 was not effective relative to an attention control in reducing stress ( d = −0.08 at post-test).

Panic disorders ( n = 2)

The two studies on panic disorders yielded inconsistent findings. One study compared 6 weeks of individual face-to-face CBT with EMI including psychotherapy and self-monitoring with 6 or 12 weeks of individual face-to-face CBT without EMI, and a waitlist control condition. 39 Although the 6 weeks of CBT (with EMI) was effective in reducing panic symptoms relative to a waitlist control ( d = 2.00 at post-test), the 6 weeks of CBT with EMI was not significantly superior to 6 weeks ( d = 0.43) or 12 weeks ( d = −0.23) of CBT without EMI. At follow up, the three active treatments did not statistically differ in reducing symptoms relative to control, but treatment gains were sustained. The only other study of panic disorder compared 4 weeks of individual face-to-face CBT with EMI including therapy and self-monitoring with 12 weeks of individual face-to-face CBT with EMI including self-monitoring only. 40 The combined 4 weeks of CBT with EMI using therapy and monitoring was significantly superior to the 12 weeks of CBT with EMI using monitoring only ( d = −0.92 to 0.81 at post-test). At follow up, there was no statistical difference between the two treatment conditions.

Social Phobia ( n = 1)

The single study on social phobia reported 12 sessions of group face-to-face CBT without EMI improved symptoms relative to the waitlist control. 41 However, the eight sessions of group face-to-face CBT with EMI including psychotherapy and self-monitoring did not significantly reduce symptoms relative to the waitlist control. Furthermore, the comparison of 8 sessions of group CBT with EMI and 12 sessions of group CBT without EMI found no significant difference. Lastly, there were no comparisons done between the EMI intervention and waitlist control at follow up.

Outcomes for different levels of automation and human support

Four of the six studies that examined SA interventions found EMIs to significantly reduce anxiety and stress relative to controls. 25,26,30,37 Similarly, three of the four studies that examined predominately self-help interventions found EMI to significantly reduce anxiety or stress relative to controls. 32–34 Finally, one of the four studies that examined TA interventions found EMIs to significantly reduce anxiety symptoms relative to controls. 39 However, another study that compared a TA intervention against a control reported no significant reduction in anxiety symptoms. 41 Only one study that compared a TA against a minimal-contact intervention found the TA intervention to be significantly superior in reducing panic symptoms. 40

DISCUSSION

The systematic review findings identified 15 RCTs and randomized trials of the effectiveness of EMIs for generalized anxiety, stress, panic disorders, and social phobia. The majority of these studies targeted generalized anxiety. The meta-analysis results demonstrated that EMIs are associated with a small, but significant reduction in generalized anxiety symptoms ( d = 0.31). There was also some encouraging evidence that EMIs targeting stress may be effective. However, only a few studies have examined EMIs targeting other anxiety-related conditions with all of those studies showing mixed results. Overall, a majority of studies evaluated integrative EMIs that involved self-monitoring of symptoms, and the delivery of automated or therapist-delivered psychotherapy content.

To our knowledge this is the first quantitative meta-analysis of the effectiveness of EMIs for generalized anxiety. The review findings suggest that EMIs may be a promising treatment for generalized anxiety. The effect size appears lower than reported previously for Internet-delivered cognitive behavioral therapy for anxiety. 42 Given many of the EMIs included in the meta-analysis were predominately SA and unguided by a therapist, it is possible EMIs with therapist support would have yielded larger effect sizes. 43,44 Further, the EMIs included in the meta-analysis were on average briefer and less intensive than Internet-delivered cognitive behavioral therapy. Overall, the present results are comparable to findings of a recent systematic review of mental health interventions delivered using mobile phones. 19 This review concluded that mobile application treatments targeting anxiety are effective. Similar findings have been identified in EMI studies for depression, smoking, and other additive behaviors. 8,45

The current review findings also suggest that EMIs may be effective for reducing stress. All but one study targeting stress found EMIs demonstrated significant reductions in stress compared with controls. In the present research, the study conducted by Reid et al.38 was the only study evaluating an interactive EMI using mobile phones which failed to yield significant effects in reducing stress relative to an attention control. However, the negative findings of this study may have been due to nature of the attention control, which closely resembled the intervention. 19 Given the scarcity of studies on stress, more research is needed to examine the effects of EMI in various stressful environments that may induce high levels of anxiety.

Currently, there is no convincing evidence that EMIs are effective for other anxiety disorders. Two of three studies did not find EMIs to be associated with a reduction in the required number of face-to-face psychotherapy sessions for panic disorders or social phobia. 39,41 Although EMI and face-to-face CBT combined was more effective than receiving no treatment for panic disorders, EMI did not enhance face-to-face CBT, there being no significant difference between 6 weeks of CBT with and without EMI. 39 Clearly, further research is required to investigate EMIs for panic and social phobias and whether EMI can enhance the effectiveness of face-to-face treatment. Other broader reviews have previously noted that more trials are needed on the effectiveness of EMI. 8,20 Despite this, no new studies targeting social phobia or panic disorder were located in the present study since the review conducted by Ehrenreich et al.20 in June 2010.

The majority of the studies in the current review assessed integrative EMIs consisting of features that allow individuals to self-monitor symptoms and to receive appropriate in-situ electronic psychotherapy. Many of these integrative EMIs included therapy delivered through programmed computer modules. It is possible that mobile phones delivering therapy exercises and self-monitoring will prove useful as a technology adjunct to enhance the efficacy of existing web-based psychotherapies. 46 Current psychotherapies for anxiety can be delivered online using a website, 47,48 and evidence suggests that psychotherapies for anxiety disorders delivered through the Internet demonstrate comparable effects to treatments delivered by a therapist. 49 Further research is warranted to investigate whether EMIs can enhance existing psychotherapy treatments such as Internet-based interventions.

In the present review, only two studies examined the effects of the individual features of mobile technologies that delivered EMIs for anxiety. 32,33 Neither study found that adding automated sensors to tailored therapeutic content delivered via smartphones significantly reduced anxiety symptoms. Further, no study examined EMIs using in-built cameras. However, many of the recent advances in smartphones have enabled the use of automated sensors and in-built cameras. 50 These features can be accessed through native operating system programming libraries, which can be used to develop smartphone applications (apps). 15 Generally, smartphone apps are designed to deliver specific therapeutic activities, such as self-monitoring, facilitating skills acquisition, or to provide information about a particular health condition. 51–53 In comparison to other delivery modalities (such as face-to-face and desktop apps), smartphone apps have the advantage of delivering media-rich, personalized therapeutic content directly on the mobile device at all times of the day. 15 However, further work is required to determine if and how EMIs can take full advantage of the smartphone app platform to optimize the delivery of particular therapeutic components of anxiety interventions. For example, more research is needed to evaluate the effectiveness of specific Human-Computer Interaction mechanisms of smartphones and other portable devices in alleviating anxiety-related symptoms. 54

Overall, it is difficult to draw reliable conclusions on the effects of EMIs on reducing anxiety symptoms and stress. Many of the included studies yielded low effective practice and organization of care group criteria quality scores. In particular, a number of studies failed to treat missing data appropriately, and provided inadequate documentation of randomization procedures, sequence generation, and allocation concealment. These issues are also coupled with the small sample sizes in many of the trials. It is important to improve the quality and reporting of prospective RCTs using EMI for anxiety and stress to allow accurate evaluation of the effectiveness of EMIs for these conditions.

Limitations

Several limitations are presented in the current review. First, the review included studies that examined EMIs with varying levels of therapist support. There are very few studies directly investigating the relative effectiveness of EMIs with and without human support or if EMI adds to face-to-face therapy or online human support. Further research is required to investigate these questions. Second, the main search concept “ecological momentary intervention” was used in the search strategy. It is possible that the terms used to identify papers relevant to this concept did not capture all relevant studies. Third, studies were only included if they measured an anxiety-related outcome. Thus, studies observing other related measures such as cost-efficiency and protocol adherence were not incorporated into our analysis. These studies may have provided further insights into the value of EMIs. Finally, a more standardized definition for EMI would have reduced the ambiguity of included interventions. The current definitions for EMI are broad and there are insufficient models describing these types of interventions in detail. It is difficult to meaningfully compare different EMIs that deploy a wide range of mobile and self-tracking technologies.

CONCLUSION

The current review found a small significant positive effect of EMIs on generalized anxiety. Studies targeting stress found EMIs to be significantly superior to controls. EMIs for panic disorders and social anxiety demonstrated mixed results. Although overall the findings are promising, more high quality RCTs are required, particularly to examine the effectiveness of EMIs for anxiety disorders such as panic and social anxiety disorder and the utility of EMIs as an adjunct to face-to-face and online web-based programs.

CONTRIBUTORS

All the authors contributed to the design of the study. BLG, RR, and KA rated all studies. KMG undertook the quantitative meta-analysis. All authors contributed to the interpretation of the data. The paper was drafted by BLG (KMG assisted in drafting the meta-analysis sections) and critically edited and reviewed by KMG and AG. All authors approved the final version of the paper.

FUNDING

KMG is supported by a National Health and Medical Research Council Fellowship (No. 1059620).

COMPETING INTERESTS

None.

PROVENANCE AND PEER REVIEW

Externally peer reviewed.

Supplementary Material

Supplementary Data

ACKNOWLEDGEMENTS

The authors would like to thank Rebecca Randall (RR) and Kathina Ali (KA) for rating the included studies for risk of bias.

SUPPLEMENTARY MATERIAL

Supplementary material is available online at http://jamia.oxfordjournals.org/ .

REFERENCES

  • 1. Alonso J, Angermeyer MC, Bernert S, et al. . Prevalence of mental disorders in Europe: results from the European Study of the Epidemiology of Mental Disorders (ESEMeD) project . Acta Psychiatrica Scandinavica. 2004. ; 109 ( s420 ): 21 – 27 . [DOI] [PubMed] [Google Scholar]
  • 2. Kessler RC, Berglund P, Demler O, et al. . Lifetime prevalence and age-of-onset distributions of DSM-IV Disorders in the national comorbidity survey replication . Arch Gen Psychiatr 2005. ; 62 ( 6 ): 593 – 602 . [DOI] [PubMed] [Google Scholar]
  • 3. Kessler RC, Chiu WT, Demler O, et al. . Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication . Arch Gen Psychiatr 2005. ; 62 ( 6 ): 617 – 627 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4. Australian Bureau of Statistics ABS. National Survey of Mental Health and Wellbeing: Summary of Results. In: Australian Bureau of Statistics ABS, ed. Canberrra; 2008.
  • 5. Høifødt RS, Strøm C, Kolstrup N, et al. . Effectiveness of cognitive behavioural therapy in primary health care: a review . Fam Pract. 2011. ; 28 ( 5 ): 489 – 504 . [DOI] [PubMed] [Google Scholar]
  • 6. Reger MA, Gahm GA . A meta-analysis of the effects of internet- and computer-based cognitive-behavioral treatments for anxiety . J Clin Psychol 2009. ; 65 ( 1 ): 53 – 75 . [DOI] [PubMed] [Google Scholar]
  • 7. Wilhelm FH, Grossman P . Emotions beyond the laboratory: theoretical fundaments, study design, and analytic strategies for advanced ambulatory assessment . Biol Psychol. 2010. ; 84 ( 3 ): 552 – 569 . [DOI] [PubMed] [Google Scholar]
  • 8. Heron KE, Smyth JM . Ecological momentary interventions: incorporating mobile technology into psychosocial and health behaviour treatments . Brit J Health Psychol. 2010. ; 15 ( Pt 1 ): 1 – 39 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9. Shiffman S, Stone AA, Hufford MR . Ecological momentary assessment . Ann Rev Clin Psychol. 2008. ; 4 ( 1 ): 1 – 32 . [DOI] [PubMed] [Google Scholar]
  • 10. Newman MG, Szkodny LE, Llera SJ, et al. . A review of technology-assisted self-help and minimal contact therapies for anxiety and depression: Is human contact necessary for therapeutic efficacy? Clin Psychol Rev . 2011 ; 31 ( 1 ): 89 – 103 . [DOI] [PubMed] [Google Scholar]
  • 11. Carter BL, Day SX, Cinciripini PM, et al. . Momentary health interventions: where are we and where are we going? In: Stone AA , ed. The Science of Real-time Data Capture: Self-reports in Health Research . Oxford: : Oxford University Press; ; 2007. : 289 – 307 . [Google Scholar]
  • 12. Alpers GW . Ambulatory assessment in panic disorder and specific phobia . Psychol Assess. 2009. ; 21 ( 4 ): 476 – 485 . [DOI] [PubMed] [Google Scholar]
  • 13. Newman MG . The clinical use of palmtop computers in the treatment of generalized anxiety disorder . Cogn Behav Pract. 1999. ; 6 ( 3 ): 222 – 234 . [Google Scholar]
  • 14. Dumont IP, Olson AL . Primary care, depression, and anxiety: exploring somatic and emotional predictors of mental health status in adolescents . J Am Board Fam Med. 2012. ; 25 ( 3 ): 291 – 299 . [DOI] [PubMed] [Google Scholar]
  • 15. Liu C, Zhu Q, Holroyd KA, et al. . Status and trends of mobile-health applications for iOS devices: A developer's perspective . J Syst Software. 2011. ; 84 ( 11 ): 2022 – 2033 . [Google Scholar]
  • 16. Fiordelli M, Diviani N, Schulz PJ . Mapping mHealth research: a decade of evolution . J Med Internet Res. 2013. ; 15 ( 5 ): e95 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17. Morris M, Guilak FYA-J . Mobile heart health: project highlight T2 . Pervasive Comput, IEEE. 2009. ; 9 ( 2 ): 57 – 61 . [Google Scholar]
  • 18. Klasnja P, Pratt W . Healthcare in the pocket: Mapping the space of mobile-phone health interventions . J Biomed Inform. 2012. ; 45 ( 1 ): 184 – 198 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19. Donker T, Petrie K, Proudfoot J, et al. . Smartphones for smarter delivery of mental health programs: a systematic review . J Med Internet Res. 2013. ; 15 ( 11 ): e247 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20. Ehrenreich B, Righter B, Rocke DA, et al. . Are mobile phones and handheld computers being used to enhance delivery of psychiatric treatment?: a systematic review . J Nervous Ment Dis. 2011. ; 199 ( 11 ): 886 – 891 . [DOI] [PubMed] [Google Scholar]
  • 21. Moher D, Liberati A, Tetzlaff J, et al. . Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement . BMJ. 2009. ; 339 ( 7716 ): 332 – 336 . [PMC free article] [PubMed] [Google Scholar]
  • 22. Mrazek PB, Haggerty RJ . Reducing Risks for Mental Disorders: Frontiers for Preventive Intervention Research . Washington, DC: : National Academy Press; ; 1994. . [PubMed] [Google Scholar]
  • 23. Cochrane Effective Practice and Organisation of Care Group EPOC. Risk of Bias - EPOC Specific. Secondary Risk of Bias - EPOC Specific 2012. http://www.webcitation.org/6CSHcMZR2 . Accessed May 6, 2015.
  • 24. Borenstein M, Hedges L, Higgins J, et al. . Comprehensive meta-anlaysis version 2 [program]. 2 version. Englewood: Biostat; 2005. [Google Scholar]
  • 25. Mosso JL, Gorini A, Cerda G, et al. . Virtual reality on mobile phones to reduce anxiety in outpatient surgery . Studies in health technology and informatics 2009; 142. http://onlinelibrary.wiley.com/o/cochrane/clcentral/articles/195/CN-00700195/frame.html . Accessed May 6, 2015 . [PubMed] [Google Scholar]
  • 26. Riva G, Grassi A, Villani D, et al. . Managing exam stress using UMTS phones: the advantage of portable audio/video support . Studies in health technology and informatics 2007; 125. http://onlinelibrary.wiley.com/o/cochrane/clcentral/articles/419/CN-00579419/frame.html . Accessed May 6, 2015 . [PubMed] [Google Scholar]
  • 27. Higgins J, Green S . Cochrane handbook for systematic reviews of interventions . Secondary Cochrane handbook for systematic reviews of interventions 2011. http://handbook.cochrane.org/ . Accessed May 6, 2015 . [Google Scholar]
  • 28. Duval S, Tweedie R . Trim and fill: a simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis . Biometrics. 2000. ; 56 : 455 – 463 . [DOI] [PubMed] [Google Scholar]
  • 29. Grassi A, Gaggioli A, Riva G . The green valley: the use of mobile narratives for reducing stress in commuters . Cyberpsychol Behav. 2009. ; 12 ( 2 ). http://onlinelibrary.wiley.com/o/cochrane/clcentral/articles/748/CN-00702748/frame.html . Accessed May 6, 2015 . [DOI] [PubMed] [Google Scholar]
  • 30. Riva G, Preziosa A, Grassi A, et al. . Stress management using UMTS cellular phones: a controlled trial . Stud Health Technol Inform. 2006;119. http://onlinelibrary.wiley.com/o/cochrane/clcentral/articles/173/CN-00554173/frame.html . Accessed May 6, 2015 . [PubMed] [Google Scholar]
  • 31. Newman MG, Przeworski A, Consoli AJ, et al. . A randomized controlled trial of ecological momentary intervention plus brief group therapy for generalized anxiety disorder . Psychotherapy (Chicago, Ill.) . 2013. ; 51 ( 2 ): 198 – 206 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32. Pallavicini F, Algeri D, Repetto C, et al. . Biofeedback, virtual reality and mobile phones in the treatment of generalized anxiety disorder (gad): a phase-2 controlled clinical trial . J Cyber Ther Rehabil. 2009. ; 2 ( 4 ). http://onlinelibrary.wiley.com/o/cochrane/clcentral/articles/429/CN-00889429/frame.html . Accessed May 6, 2015 . [Google Scholar]
  • 33. Gorini A, Pallavicini F, Algeri D, et al. . Virtual reality in the treatment of generalized anxiety disorders . Stud Health Technol Inform. 2010. ; 154 . http://onlinelibrary.wiley.com/o/cochrane/clcentral/articles/724/CN-00760724/frame.html . Accessed May 6, 2015 . [PubMed] [Google Scholar]
  • 34. Lemaire JB, Wallace JE, Lewin AM, et al. . The effect of a biofeedback-based stress management tool on physician stress: a randomized controlled clinical trial . Open Med. 2011. ; 5 ( 4 ). http://onlinelibrary.wiley.com/o/cochrane/clcentral/articles/590/CN-00843590/frame.html . Accessed May 6, 2015 . [PMC free article] [PubMed] [Google Scholar]
  • 35. Villani D, Grassi A, Cognetta C, et al. . Self-help stress management training through mobile phones: an experience with oncology nurses . Psychol Services. 2013. ; 10 ( 3 ): 315 – 322 . [DOI] [PubMed] [Google Scholar]
  • 36. Lappalainen P, Kaipainen K, Lappalainen R, et al. . Feasibility of a personal health technology-based psychological intervention for men with stress and mood problems: randomized controlled pilot trial . JMIR Res Protoc. 2013. ; 2 ( 1 ): e1 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37. Proudfoot J, Clarke J, Birch MR, et al. . Impact of a mobile phone and web program on symptom and functional outcomes for people with mild-to-moderate depression, anxiety and stress: a randomised controlled trial . BMC Psychiatr. 2013. ; 13 : 312 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38. Reid SC, Kauer SD, Hearps SJC, et al. . A mobile phone application for the assessment and management of youth mental health problems in primary care: a randomised controlled trial . BMC Fam Pract. 2011. ; 12 : 131 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39. Kenardy JA, Dow MGT, Johnston DW, et al. . A Comparison of delivery methods of cognitive-behavioral therapy for panic disorder: an international multicenter trial . J Consult Clin Psychol. 2003. ; 71 ( 6 ): 1068 – 1075 . [DOI] [PubMed] [Google Scholar]
  • 40. Newman MG, Kenardy J, Herman S, et al. . Comparison of palmtop-computer-assisted brief cognitive-behavioral treatment to cognitive-behavioral treatment for panic disorder . J Consult Clin Psychol. 1997. ; 65 ( 1 ). http://onlinelibrary.wiley.com/o/cochrane/clcentral/articles/591/CN-00138591/frame.html . Accessed May 6, 2015 . [DOI] [PubMed] [Google Scholar]
  • 41. Gruber K, Moran PJ, Roth WT, et al. . Computer-assisted cognitive behavioral group therapy for social phobia . Behav Ther. 2001. ; 32 ( 1 ): 155 – 165 . [Google Scholar]
  • 42. Mewton L, Smith J, Rossouw P, et al. . Current perspectives on Internet-delivered cognitive behavioral therapy for adults with anxiety and related disorders . Psychol Res Behav Manag. 2014. ; 7 : 37 – 46 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43. Spek V, Cuijpers P, Nyklicek I, et al. . Internet-based cognitive behaviour therapy for symptoms of depression and anxiety: a meta-analysis . Psychol Med. 2007. ; 37 ( 3 ): 319 – 328 . [DOI] [PubMed] [Google Scholar]
  • 44. Palmqvist B, Carlbring P, Andersson G . Internet-delivered treatments with or without therapist input: does the therapist factor have implications for efficacy and cost? Expert Rev Pharmacoecon Outcomes Res. 2007. ; 7 ( 3 ): 291 – 297 . [DOI] [PubMed] [Google Scholar]
  • 45. Telford C, McCarthy-Jones S, Corcoran R, et al. . Experience Sampling Methodology studies of depression: the state of the art . Psychol Med. 2012. ; 42 ( 6 ): 1119 – 1129 . [DOI] [PubMed] [Google Scholar]
  • 46. Clough BA, Casey LM . Technological adjuncts to increase adherence to therapy: A review . Clin Psychol Rev. 2011. ; 31 ( 5 ): 697 – 710 . [DOI] [PubMed] [Google Scholar]
  • 47. Cuijpers P, Marks IM, van Straten A, et al. . Computer-aided psychotherapy for anxiety disorders: a meta-analytic review . Cogn Behav Ther. 2009. ; 38 ( 2 ): 66 – 82 . [DOI] [PubMed] [Google Scholar]
  • 48. Griffiths KM, Farrer L, Christensen H . The efficacy of internet interventions for depression and anxiety disorders: a review of randomised controlled trials . Med J Aust. 2010. ; 192 ( 11 Suppl ): S4 . [DOI] [PubMed] [Google Scholar]
  • 49. Christensen H, Batterham P, Calear A . Online interventions for anxiety disorders . Curr Opin Psychiatr. 2014. ; 27 ( 1 ): 7 – 13 . [DOI] [PubMed] [Google Scholar]
  • 50. Luxton DD, McCann RA, Bush NE, et al. . mHealth for mental health: Integrating smartphone technology in behavioral healthcare . Prof Psychol Res Pr. 2011. ; 42 ( 6 ): 505 – 512 . [Google Scholar]
  • 51. West JH, Hall PC, Hanson CL, et al. . There’s an App for that: content analysis of paid health and fitness apps . J Med Internet Res. 2012. ; 14 ( 3 ): e72 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 52. Martínez-Pérez B, de la Torre-Díez I, López-Coronado M . Mobile health applications for the most prevalent conditions by the World Health Organization: review and analysis . J Med Internet Res. 2013. ; 15 ( 6 ): e120 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53. Harrison AM, Goozee R . Psych-related iPhone apps . J Mental Health. 2014. ; 23 ( 1 ): 48 – 50 . [DOI] [PubMed] [Google Scholar]
  • 54. Klasnja P, Consolvo S, Pratt W . How to evaluate technologies for health behavior change in HCI research . Proceedings of the SIGCHI Conference on Human Factors in Computing Systems . Vancouver, BC, Canada: ACM: ; 2011. : 3063 – 3072 . [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary Data

Articles from Journal of the American Medical Informatics Association : JAMIA are provided here courtesy of Oxford University Press

RESOURCES