Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2019 Mar 1.
Published in final edited form as: Behav Ther. 2017 Jul 29;49(2):249–261. doi: 10.1016/j.beth.2017.07.008

Consumer smartphone apps marketed for child and adolescent anxiety: A systematic review and content analysis

Laura Jane Bry 1, Tommy Chou 1, Elizabeth Miguel 1, Jonathan S Comer 1
PMCID: PMC6082393  NIHMSID: NIHMS896370  PMID: 29530263

Abstract

Anxiety disorders are collectively the most prevalent mental health problems affecting youth. To increase the reach of mental healthcare, recent years have seen increasing enthusiasm surrounding mobile platforms for expanding treatment delivery options. Apps developed in academia and supported in clinical trials are slow to reach the consumer marketplace. Meanwhile, proliferation of industry-developed apps on consumer marketplaces has been high. The present study analyzed content within mobile products prominently marketed toward consumers for anxiety in youth. Systematic inventory of the Google Play Store and Apple Store using keyword searches for child and adolescent anxiety yielded 121 apps, which were evaluated on the basis of their descriptive characteristics, mobile functionalities, and adherence to evidence-based treatment principles. Findings revealed that evidence-based treatment content within the sample is scant and few comprehensive anxiety self-management apps were identified. Advanced features that leverage the broader functionalities of smartphone capabilities (e.g., sensors, ecological momentary assessments) were rarely present. Findings underscore the need to increase the prominence and accessibility of quality child anxiety intervention products for consumers. Strategies for improving marketing of supported apps to better penetrate consumer markets are discussed.

Childhood anxiety disorders constitute an enormous public health concern. In the U.S. alone, estimates indicate that anxiety disorders are collectively the most prevalent class of mental disorders and are among the earliest disorders to establish themselves, with median age of onset occurring at age 6 (Merikangas et al., 2010). Indeed, almost one in ten preschoolers suffer from an anxiety disorder before age 5 (Egger & Angold, 2006) and nearly one-third of adolescents experience anxiety disorder onset prior to age 18 (Merikangas et al., 2010). Furthermore, anxiety is related to a high degree of comorbidity with other mental health problems (Costello, Egger, & Angold, 2005) and a stable course across the lifespan. Clinical levels of anxiety during adolescence have been linked to a number of poor outcomes in adulthood, including increased risk for anxiety, depression, substance use, suicidal behavior, educational underachievement, early parenthood, and overall reduced quality of life (Comer et al., 2011; Woodward & Fergusson, 2001). Moreover, daily interference and impairment in those experiencing anxiety is high, and the nature of anxiety symptoms is linked to deficits in multiple domains of functioning, such as school absenteeism (Kearney, 2008), poorer social skills (Crawford & Manassis, 2011), sleep problems (Weiner, Elkins, Pincus & Comer, 2015), lower levels of peer acceptance (Greco & Morris, 2005), elevated substance use (Wu, Goodwin, Comer, Hoven & Cohen, 2010), and greater peer victimization (Crawford & Manassis, 2011; Siegel, La Greca, & Harrison, 2009). Early intervention for anxiety is critical.

The past several decades have witnessed tremendous advances in the development and evaluation of evidence-based treatments for anxious youth (Higa-McMillan, Francis, Rith-Najarian, & Chorpita, 2016). Cognitive-behavioral therapy (CBT) has received the strongest support in clinical trials and is considered the “gold-standard” psychosocial intervention for anxiety disorders in children and adolescents (e.g., Compton et al., 2004; Higa-McMillan et al., 2016; Walkup et al., 2008). Exposure exercises—widely accepted to be among the most essential ingredients of effective CBT for anxious youth (Kendall et al., 2005)—have children develop a hierarchy of situations and objects that cause them fear, as they systematically confront these feared experiences in a graduated fashion. Other key CBT ingredients for child anxiety include: psychoeducation about the harmless and normative nature of anxiety, self-monitoring (child or parent observes and keeps logs of their anxious patterns, behaviors, and thoughts), contingency management (child is rewarded and reinforced for brave behaviors, rewards are removed for behavioral avoidance), problem solving (child learns to identify problems and generate, select, and evaluate solutions), and thought challenging/cognitive restructuring (i.e., child learns to identify and modify anxious “self-talk”). Although relaxation training was once considered to be an important element of anxiety treatment, it is now considered less essential and potentially counterproductive, and is rarely included in modern CBTs for youth anxiety (Ehrenreich-May et al., 2016).

Despite advances in the identification of efficacious treatment practices for child and adolescent anxiety, significant barriers prevent those in need from accessing supported care. Indeed, epidemiological data show that rates of treatment among those with mental disorders is low, with less than half of affected individuals seeking any type of mental health treatment in a 12-month period (Wang et al., 2005). Treatment rates are lowest among vulnerable and traditionally underserved groups (e.g., economically disadvantaged, rural, racial/ethnic minorities, elderly, those lacking insurance; Wang et al., 2005). Moreover, when traditional clinic-based services are accessed, dropout rates are high (Wierzbicki & Pekarik, 1993). Several patient-level barriers have been identified, including attitudinal barriers (e.g., low perceived need and stigma; Mojtabai et al., 2011), financial limitations (Mojtabai, 2005) and structural barriers (e.g., geographically underserved regions; scheduling difficulties; transportation obstacles; Sareen et al., 2007). Organization-level and systemic barriers to traditional clinic-based services have also been identified, including mental health workforce shortages (Thomas, Ellis, Konrad, Holzer, & Morrissey, 2009) and English language proficiency (Derose & Baker, 2000). Moreover, individuals experiencing psychological distress perceive greater barriers to treatment than their non-impaired counterparts (Mohr et al., 2010). Thus, the obstacles to providing evidence-based mental health treatment to those in need are highly entrenched in the American healthcare system and individual attitudes. Overcoming such barriers to quality mental healthcare will require a paradigm shift in the manner in which treatments are delivered (Comer & Barlow, 2014; Kazdin & Blase, 2011).

Over the last two decades, there has been increasing enthusiasm about the potential for leveraging technology to better meet population-level mental healthcare needs (Chou, Bry, & Comer, in press; Comer & Barlow, 2014; Kazdin & Blase, 2011). For example, technology-based or -assisted mental health interventions hold promise to provide treatment at a lower cost (Muñoz, 2010; Washington State Institute for Public Policy, 2015), though more research is needed to demonstrate cost-effectiveness at a systems-level (Schweitzer & Synowiec, 2012). Technology also affords the potential to overcome geographical obstacles and expand access to care to more individuals, including children and adolescents in geographically remote areas (e.g., Bunnell, Davidson, Dewey, Price, & Ruggiero, 2016; Comer & Barlow, 2014; Hilty, Cobb, Neufeld, Bourgeois, & Yellowlees, 2008; Price, Yuen, Davidson, Hubel, & Ruggiero, 2015). With about 75% of Americans accessing healthcare information online (Pew Research Center, 2015), the feasibility and acceptability of technology-based treatments appears to be on the rise. Moreover, recent randomized controlled trials have supported the initial efficacy of technology-enhanced interventions in improving children’s mental health outcomes (e.g., Hedman et al., 2014; Jones et al., 2014; March, Spence, & Donovan, 2009; Spence, Holmes, March, & Lipp, 2006).

The platforms of behavioral intervention technologies (BITs) have increasingly shifted towards applications addressing healthcare needs via a range of mobile communication devices -referenced broadly as mHealth applications. While mHealth platforms range across a number of consumer products (e.g., smartphones, tablets, wearable sensors), the present evaluation focuses specifically on those made available for smartphones; Jones et al., 2015). Given the increasing prevalence and usage rates of smartphones, in particular, across a wide range of demographics (e.g., youth, racial/ethnic minorities, those dwelling in rural communities; Pew Research Center, 2017), smartphone applications are being developed as a novel method of extending access and delivering mental health treatment to traditionally underserved groups (Anton et al., 2016). Preliminary studies point to evidence for the acceptability of using smartphones to monitor mental health (Torous, Friedman, & Keshavan, 2014) as well as the feasibility and efficacy of delivering interventions on smartphones (Donker et al., 2013; Jones et al., 2014), however this body of literature is inchoate, precluding firm conclusions.

Innovative research groups are increasingly drawing on the expanding technological capabilities and functionalities of smartphone platforms which afford unique opportunities for patient monitoring and treatment. For example, a phone’s sensor data, (i.e., its use of the innate GPS, accelerometer, and/or microphone) can allow for passive data collection on user sleep patterns, heart rate, social context, physical location, mood and stress levels (Mohr, Zhang, & Schueller, 2017), each providing clinical information that can relate to a user’s physiological anxiety symptoms and the environmental factors producing such. Furthermore, smartphone platforms provide features that allow for ecological momentary assessment, “just-in-time” intervention, and the delivery of interactive, tailored treatment content aligned with the precision medicine movement (Schueller, Muñoz, & Mohr, 2013) in ways face-to-face treatment may not.

The unique opportunities brought about by smartphones and related mobile communication technologies have not gone unnoticed by the private sector, evidenced by the proliferation of industry-funded mental health apps available on the consumer market place. As of January 2014, there were roughly 2,000 mobile apps marketed for anxiety alone that were available to consumers (Chan, Torous, Hinton, & Yellowlees, 2014) and a national US survey showed that more than half of Americans had downloaded at least one health-related app (Krebs & Duncan, 2015). Despite growing mHealth development and research within academic settings, research-to-practice gaps and poor dissemination of science-based healthcare innovations to the general public persist (Westfall, Mold, & Fagnan, 2007). Amidst rapid expansion of industry-sponsored mHealth apps marketed for consumers, it is not clear whether supported mHealth apps developed in academic settings are actually reaching consumers, and whether mHealth apps that are readily available on the consumer marketplace are grounded in evidence-based principles.

Despite the high prevalence and burdens of child and adolescent anxiety disorders, and despite advances in academic settings in the development and examination of mobile apps for youth anxiety, to date no studies have evaluated the scope and quality of mobile apps actually dominating the consumer marketplace for child or adolescent anxiety. The present study entailed a systematic content analysis of readily searchable apps on leading consumer marketplaces for mobile communication technologies. Given concerns about the quality of mobile apps targeting other mental health problems (e.g., depression, Shen et al., 2015; eating disorders, Juarascio, Manasse, Goldstein, Forman, & Butryn, 2015), it was hypothesized that the majority of mobile apps marketed to consumers for youth anxiety would not include the critical evidence-based components supported in research trials for treating youth anxiety (e.g., exposures, psychoeducation about anxiety, self-monitoring, contingency management, development of fear hierarchies, problem solving, and thought challenging). To further examine the research-practice gap, apps were also evaluated on the extent to which they incorporated advanced technological components and functionalities highlighted in the academic mHealth literature (e.g., ecological momentary assessment, data collection through sensors). Lastly, comparisons were made between the content and functionalities of free apps in the consumer marketplace to the content and functionalities of apps that cost money to explore whether financial considerations might be associated with the quality of apps for child and adolescent anxiety.

Methods

App Search Criteria and Selection Procedures

Screening and assessment of apps for study inclusion was guided by a systematic review process and restricted to the two most widely used marketplaces for mobile apps: Android’s Google Play store and Apple’s iOS App store (Boulos, Wheeler, Taveres & Jones, 2011). Search terms were identified for anxiety-related apps and included two technical terms (i.e., anxiety, phobia) as well as two common synonyms and layperson alternatives (i.e., fear, stress) in effort to identify consumer-driven products that take into account varying levels of mental health literacy. These four anxiety search terms were crossed with a second set of four search terms to restrict results to apps specific to youth problems (i.e., child, kid, teen, and adolescent). Combining each of the four search terms for anxiety with each of the four search terms for youth resulted in 16 unique searches in each of the two app marketplaces, for a total of 32 searches.

Given research suggesting that only a small number of users view apps past the first 10 ranked apps in their search query, and most downloaded apps are listed within the first 5 ranked apps per those search terms (Dogruel, Joeckel, & Bowman, 2015), not all apps that were generated in each search were evaluated. As a conservative approximation of potential consumer search patterns, the study team included up to the top 30 results from each of the 32 unique searches.

The Google Play store and Apple App store were inventoried using the aforementioned search terms during a single week in February, 2016 resulting in the identification of 755 total apps, as some search terms yielded fewer than 30 results. 182 of these apps were identified as duplicates (either on the basis of appearing in both the Android and Apple marketplaces, or appearing in the results of multiple search term combinations) and excluded. The remaining 573 unique mobile apps were further scrutinized for study inclusion. Apps meeting the following inclusion criteria were retained for analysis: (1) marketed for the treatment or management of anxiety-related symptoms, (2) intended for use by children, teens, or their parents (i.e., apps were not specifically listed for adult use only), (3) available to the public for download, (4) available in English, and (5) evidenced stability in the marketplace by remaining available throughout the entirety of the 6 month coding window.

Two study authors (LB and TC) used the information provided in the apps’ titles, app store descriptions, and available screenshots to identify those apps meeting the above inclusion criteria. The most common reason for exclusion were apps that, despite appearing in search results for anxiety-related apps, did not make any mention of treating or managing anxiety-related symptoms in their description pages. Most of these apps were games or other multimedia content marketed toward youth. An ultimate sample of 121 apps met inclusion criteria and was submitted to content analysis.

App Coding

A codebook was created to organize content analysis and optimize intercoder reliability. Content analysis consisted of noting the presence or absence of all study codes except reading level, which was treated as a continuous variable. Reading level of each app was approximated by uploading the full text included in the app’s description page into a Flesch-Kincaid grade level calculator. Other study codes included descriptive characteristics of the apps: (1) age category (the app rating that the app platform, not the developer, has assigned via the app’s description page.), (2) price category (free versus for-purchase) (3) confidentiality (the app offers the option of logging in via a set password and/or the app directly states its security, privacy, or data storage policy), and (4) risk-management (the app includes language regarding the management of suicidal ideation, mental health emergencies, and/or indicates the app is not intended to replace a therapist). For each app, coders also recorded the presence versus absence of three specific mobile app functionality components: (1) sensors (data is imported to the app via sensors innate to the smartphone that capture motion, orientation, or environment), (2) interactivity (the app’s content is individualized and influenced by user behavior, input, or context, and/or the app provides customizable options to users based on preference), and (3) ecological momentary assessment (assessment and/or intervention, in real time, of the user’s mood, thoughts, behaviors or distress level).

Consensus guidelines on research-supported practices for youth anxiety (Chorpita & Daleiden, 2009; Higa-McMillan et al., 2016; Silverman, Pina & Viswesvaran, 2008) were consulted, and a panel of three doctoral-level youth anxiety experts provided additional input to identify the key evidence-based practice components in the treatment of child anxiety. For each app, coders recorded the presence or absence of six identified evidence-based treatment components: (1) Psychoeducation, (2) Self-monitoring, (3) Cognitions and Thought Challenging, (4) Problem Solving, (5) Contingency Management, and (6) Exposures. See Figure 1 for definitions of how these components were operationalized for study coders. In addition, given historical recommendations for relaxation training in the treatment of youth anxiety, despite waning enthusiasm for the practice in modern CBTs for youth anxiety, coders also assessed apps for Relaxation Training, see Figure 1.

Figure 1.

Figure 1

Open in a new tab

Study definitions of evidenced-based treatment components within study apps

Three coders (LB, TC, and EM) were trained to reliability on several rounds of sample mental health apps. Coders had to achieve an inter-rater reliability criterion of k= 0.80 on a series of 8 independently coded apps before initiating coding on the study apps. The final round of independent practice coding revealed high concordance and high inter-rater reliability k= 0.93.

The 121 study apps were downloaded from their respective app stores and each was assessed and coded by at least one study coder. To ensure inter-rater reliability, 20% of the overall app sample was independently double-coded by two reviewers. Inter-rater reliability was calculated overall, as well as on individual codes. Overall Kappa reliability coefficients ranged from k=0.71–1.00 on individual codes and an overall reliability of k=0.94 indicating strong reliability (Fleiss, 1971).

Data analysis

Cohen’s kappa values and descriptive statistics were calculated using SPSS version 23. Chi square tests were computed to make comparisons between free and for-purchase apps on all categorical variables. T-tests measured differences between free and for-purchase apps for all continuous variables.

Results

Sample characteristics

Table 1 presents characteristics of the sampled youth anxiety apps available on the consumer marketplace. Over three-quarters of the sample were Android apps found in the Google Play Store. The majority of sampled apps were available free of cost. Among apps that were not available for free, prices ranged from $0.99–$6.99 (M= $1.21, SD=1.74). The majority of apps in the sample had a formal age classification designated through the app store as being appropriate for either “Everyone” (Google Play Store) or those ages “4+” (Apple iTunes Store). The mean Flesch-Kincaid Grade Level score of the overall sample indicated that the content on study apps were written, on average, at a 9th grade reading level. The mean Flesch-Kincaid reading level of apps specifically categorized as “Everyone” or age “4+” indicated that content on these apps’ description pages was also written at a 9th grade reading level. Lastly, features to maintain user safety were scant within the sample. Specifically, features related to user confidentiality (e.g., passwords, statements about data security) were found in less than 5% of apps in the sample. Statements regarding acute risk, crisis or suicidality aimed at directing users to more appropriate risk management outlets were more frequent within the sample, but found in only 1 out of every 6 apps.

Table 1.

Characteristics of sampled youth anxiety apps on the marketplace (N=121)

N %
Marketplace
 Google Play Store 83 68.6
 iOS App Store 38 31.4
Cost
 Free 70 57.9
 For Purchase 51 42.1
Designated Age Classification
 Everyone/Ages 4+a 91 75.2
 Not for everyone/Ages 4+a 30 24.8
Confidentiality
 Yes 5 4.1
 No 116 95.9
Risk Management
 Yes 18 14.9
 No 102 85.1
M SD
Flesch-Kincaid Reading Grade Level 9.3 3.4
Open in a new tab
a

Google Play Store and iOS App Store have different age classification levels. The lowest age classification level on the Google Play Store is “Everyone,” whereas the lowest age classification level on the iOS App Store is “Ages 4+”.

Functionality components

Advanced functionalities often included or touted in the academic literature on the potential of mHealth were rare among sampled apps for child anxiety on the marketplace (see Table 2). No apps gathered data passively through sensors and less than 5% of the sampled apps used ecological momentary assessment to evaluate or “ping” user mood, emotional or behavioral states in real time. Lastly, apps that allowed users to customize content to their preferences or those containing interactive, logic-driven content reacting to user input were similarly rare.

Table 2.

Prevalence of advanced functionalities across sampled youth anxiety apps on the marketplace (N=121)

N %
Individual Functionalities
 Sensors
  Yes 0 0.0
  No 121 100.0
 Interactive Content
  Yes 3 2.5
  No 118 97.5
 Ecological Momentary Assessment
  Yes 4 3.3
  No 117 96.7
Any Advanced Functionalities
  Yes 5 4.1
  No 116 95.9
Open in a new tab

Evidence-based treatment components

Half of the sampled apps for child anxiety on the marketplace included any evidence-based treatment component (see Table 3), and 23% of sampled apps contained two or more evidence-based components. The most common evidence-based treatment component found across sampled apps on the marketplace was “exposure” which was found in a fifth of sampled apps. Similarly, roughly one-fifth of sampled apps on the marketplace contained mention of cognitive biases associated with anxiety or had instruction in thought challenging or cognitive restructuring, and nearly one-fifth of the sampled apps referenced self-monitoring, reflection, or tracking of one’s thoughts, emotions or behaviors. General psychoeducational information related to anxiety, its definition and symptoms, and how it is maintained or treated was scant among sampled apps, with only 1 in every 6 apps referencing such information. Mention of rewards or positive reinforcement for use of the app or brave/approach behavior toward anxiety-related situations was present in roughly 1 out of 10 apps, and problem-solving content was even more rare.

Table 3.

Prevalence of evidence-based treatment components across sampled youth anxiety apps on the marketplace (N=121)

Cost
Total sample Free For-purchase Significance test
N % N % N %
Individual Evidence-Based Components
 Exposures
  Yes 26 21.5 16 13.2 10 8.3 χ2 (1, 121) = 0.19, p= 0.67
  No 95 78.5 54 44.6 41 33.9
 Cognitive Restructuring/Thought
 Challenging
  Yes 25 20.7 10 8.3 15 12.4 χ2 (1, 121) = 4.12, p= 0.04
  No 96 79.3 60 49.6 36 29.8
 Self-Monitoring
  Yes 24 19.8 11 9.1 13 10.7 χ2 (1, 121) = 1.77, p= 0.18
  No 97 80.2 59 48.8 38 31.4
 Psychoeducation
  Yes 20 16.5 9 7.4 11 9.1 χ2 (1, 121) = 1.62, p= 0.20
  No 101 83.5 61 50.4 40 33.1
 Contingency Management
  Yes 14 11.6 10 8.3 4 3.3 p=0.39, Fisher’s Exact Test
  No 107 88.4 60 49.6 47 38.8
 Problem Solving
  Yes 6 5.0 2 1.7 4 3.3 p=0.24, Fisher’s Exact Test
  No 115 95.0 68 56.2 47 38.8
Any Evidence-Based Component(s)
  Yes 60 49.6 32 26.4 28 23.1 χ2 (1, 121) =1.00, p=.32
  No 61 50.4 38 31.4 23 19.0
Open in a new tab

The majority of sampled apps marketed for youth anxiety that lacked any evidence-based treatment components were largely distraction tools, such as games (21%), coloring activities (9%), or other audio or visual distraction activities (9%). Roughly half of all study apps (53%) included relaxation exercises (e.g., breathing exercises, guided imagery, progressive muscle relaxation) which was once considered to be an important element of anxiety treatment, but is now considered less essential and potentially counter-productive, and thus rarely included in modern CBTs for youth anxiety.

Cost-related differences in evidence-based content

Table 3 also includes data on the presence of evidence-based treatment components, broken down by app cost (i.e., free versus for-purchase). Among specific evidence-based components, content related to cognitions and thought challenging was significantly more common among apps costing money than among free apps.

Discussion

Despite increased scholarly attention to the potential of leveraging smartphones to extend the reach of evidence-based treatments (e.g., Chou, Bry, & Comer, in press; Chou, Bry, Comer, 2017; Donker et al., 2013; Jones et al., 2015; Whiteside, 2016), and despite considerable industry activity in the development of smartphone apps for youth anxiety, this is the first investigation to systematically assess the overall quality of readily searchable smartphone apps for youth anxiety on leading consumer marketplaces. Of great concern, evidence-based components for the treatment of youth anxiety were quite scant among sampled apps on the consumer marketplace. The sampling strategy yielded very few standalone, comprehensive treatment or anxiety-management products. Although roughly half of the sample had at least one evidence-based treatment component, only a quarter of apps incorporated more than one evidence-based treatment component. Given that consensus treatment guidelines (e.g., Higa-McMillan et al., 2016) emphasize the need for a multi-component approach to treating youth anxiety (e.g., simultaneously including psychoeducation, thought challenging, contingency management, and exposure exercises into a course of treatment), there is a clear need to increase the development and accessibility of more comprehensive self-management apps that leverage multiple evidence-based treatment strategies. Indeed, without any formal body assessing and recommending self-management apps to consumers (FDA, 2015; Huckvale et al., 2015), the present findings suggest the possibility for consumers to encounter spurious or ill-equipped treatment apps for youth anxiety is high. Encouragingly, at the time of this report, the American Psychiatric Association and the United Kingdom’s National Health Service in partnership with their National Information Board have both set out to endorse a range of self-management apps across mental health disorders to better guide consumers towards quality digital treatment options (APA Smartphone App Evaluation Task Force, 2016; National Information Board, 2015).

Evidence-based treatment of anxiety is generally regarded as a multicomponent procedure consisting largely of the content areas upon which apps were assessed (Chorpita & Daleiden, 2009). In addition to improving the flow of consumers toward quality comprehensive self-management apps, consumers may also benefit from using a single quality app dedicated to one practice element (e.g., thought restructuring) or a group of singularly focused treatment apps (i.e., pairing an app with exposure-based content with one geared toward thought restructuring). This more modular approach may better meet unique patient needs and symptom profiles, similar to in-clinic modular treatment approaches (Chorpita, Taylor, Francis, Moffitt, & Austin, 2004). Such a modular approach may also be more feasible to implement, as comprehensive apps incorporating a range of evidence-based features are highly complex and expensive for researchers and programmers to create and maintain (Watts et al., 2013). Similarly, usability research suggests that most individuals are accustomed to using mobile phone apps in short bursts, and comprehensive, content-heavy mHealth apps may not align with end-user habits (Vaish, Wyngarden, Chen, Cheung, & Bernstein, 2014). Recent findings from a field trial piloting a modular mHealth approach for adults consisting of a suite of 14 distinct mental health apps coordinated by a central app “hub” showed favorable reductions in depressive and anxious symptomatology (Mohr et al., 2017) and favorable engagement (Lattie et al., 2016). Future research examining modular mHealth approaches for youth problems is needed to inform the extent to which single, comprehensive treatment apps versus suites of modular, individual practice element apps best promote user engagement and symptom reduction.

Moreover, endorsements for quality treatment apps focused on a single practice element may additionally benefit clinicians and patients already accessing treatment. Apps can offer innovative and user-friendly representations of traditional in-clinic concepts and out-of-session homework tasks such as mood and thought tracking. These apps may be useful to clinicians who recommend them as adjuncts to in-clinic treatment and may increase out-of-session homework compliance and overall engagement in treatment (see Jones et al., 2015). Relatedly, it is important to focus on within-app features that can improve engagement to mHealth interventions. Recent work on practice elements related to engagement in traditional child and adolescent treatments shows five prominent elements: assessment, psychoeducation, accessibility, attention to barriers to care, and goal setting (Becker, Boustani, Gellatly, & Chorpita, 2017). Using these as a guide, the present sample of apps would yield poor results on assessment, psychoeducation and goal setting, although given the mobile platform and rising rates of access to smartphones, this sample might score well on the accessibility and attention to barrier to care. Future work examining engagement factors in mHealth interventions is needed to fully consider the viability of mobile phones as a treatment platform. Further, research focusing on whether engagement-promoting factors differ across mHealth and more traditional face-to-face treatment delivery formats is needed to inform mHealth adherence and the extent to which mHealth platforms may ultimately increase access to care. For example, researchers focusing specifically on self-administered eHealth and mHealth engagement have noted high rates of attrition from these interventions (Christensen, Griffiths, & Farrer, 2009). However, using the model of “supportive accountability,” or pairing a treatment app or website with a human coach, researchers have found that just 10 minutes per week of coaching on engagement to the intervention with no other delivery of therapeutic content by the coach can significantly reduce attrition and improve mental health outcomes (Mohr, Cuijpers, & Lehman, 2013). mHealth apps available to consumers are beginning to incorporate human support strategies (e.g., Joyable, Coach.me), though none of the apps in this sample employed a human coaching model. Future research on how a human coaching model, or supportive accountability, relates to children and adolescents or families utilizing mHealth interventions will be important for considering engagement to these treatment models.

The present analysis found that less than 5% of the sampled apps marketed for youth anxiety addressed confidentiality. Research examining user attitudes and perceptions toward mHealth data security and patient privacy shows that patients are indeed concerned about the storage of personal health-related data in digital spaces and potential secondary use of such data (Atienza et al., 2015). With only 4% of sampled apps containing a password protection or a data security notice, the majority of apps for youth anxiety may go against consumer comfort and preferences. Despite recent proliferation of apps available to consumers on public platforms, these apps may be met with consumer distrust and concerns over privacy.

Scholarly attention has frequently touted the great potential for mHealth platforms to leverage advanced mobile features and innovations to broaden intervention opportunities. One such area of interest relates to the measurement and monitoring of physiological indices of anxiety (e.g., arousal, heart rate, sleep patterns). The importance of in situ data for improving clinical assessment has been emphasized (Aldao & De los Reyes, 2015; Davis, May & Whiting, 2011; Silverman & Ollendick, 2005), and smartphones offer the ability to capture in situ data via sensors and ecological momentary assessment and integrate this information into the intervention itself to provide “just-in-time,” “in-the-moment” treatment content to users. Indeed, ecological momentary interventions for anxiety are gaining empirical support and scholarly attention (Schueller, Aguilera & Mohr, 2017). Yet, the present findings suggest that the vast majority of readily accessible smartphone apps for youth anxiety on the consumer marketplace lag behind scholarly enthusiasm in this area. Research focusing on EMA and sensors is a burgeoning area although to date, the majority of studies relating these to mental health outcomes have utilized only small samples (Mohr et al., 2017b). However, with continued advances, and as “wearable” devices and “tracking” personal data gain traction in the consumer market, a key future direction will be integrating objective sensor data and evidence-based assessment of physiological anxiety responses with interventions that are both accessible to consumers and make use of these data through actionable intervention.

Moreover, apps incorporating such advanced functionalities (often developed in academia) do not appear to be meaningfully penetrating the consumer marketplace. Despite opportunities for smartphone apps to bridge gaps between research and practice that have been observed with regard to traditional clinic-based care, researchers must be cautious about replicating a similar research-to-practice gap in the arena of mHealth. Traditionally, research shows an 18-year gap from the point of intervention development to widespread dissemination (Westfall et al., 2007). Technology has the potential to serve as an agile platform through which supported interventions may be more quickly disseminated and through which larger proportions of individuals in need of care may access quality care (Kazdin & Blasé, 2011). However, the present study suggests that currently, mHealth interventions developed in academic settings for youth anxiety are not yet being disseminated or made readily accessible to consumers. The findings from this report underscore a need for strategies that help researchers better break into the consumer marketplace to increase the availability of evidence-based mHealth interventions. Without such efforts, the field of mHealth is poised to replicate the well-documented research-to-practice gap associated with traditional care models (Weisz, Jensen-Doss, & Hawley, 2006).

The present findings also summon questions related to a replication of other known barriers to treatment. For example, significant differences were found between for-purchase apps and free apps in their incorporation of cognitive restructuring and thought challenging. Often, financial considerations can regrettably impact the quality of care received in traditional office-based care, but technology may offer needed opportunities to balance the accessibility of evidence-based treatment. As the field of mHealth continues to evolve, caution must be taken to avoid perpetuating disparities in the quality of available services.

Limitations and future directions

Several study limitations warrant comment. First, although the present methodology attempted to replicate common consumer search strategies to identify youth anxiety apps, the present findings may not generalize to apps identified via other search strategies. However, little is presently known about the precise search patterns of consumers seeking to identify youth anxiety apps. For example, the present use of the search term “stress” as a hypothesized layperson synonym for “anxiety” may have influenced the search results. Many apps meeting selection criteria encouraged user download of non-evidence-based distraction tools like coloring books and games through statements that claimed these activities could decrease user stress. It is possible that restricting study search terms to just the more clinical and technical term “anxiety” may have yielded a sample with more favorable incorporation of evidence-based treatment components. Further work engaging consumers in generating search terms and examining the apps naturally selected by consumers may provide a more generalizable view of digital consumption.

Importantly, some high quality treatment apps did not make it into the present analysis based on this consumer-based search strategy. For example, the app SmartCAT (Pramana, Parmanto, Kendall, & Silk, 2014), an mHealth platform delivering CBT to children with anxiety, is an intervention developed in academia that incorporates EMA, a therapist portal and monitoring system, and all 7 evidence-based treatment components included for content analysis in the present study. However, at the time of the study’s app search and selection, and at the time of this report, the SmartCAT app is still not included in the top 30 search results of any of the 16 search terms used, despite its presence on the Google Play Store. Little is known about how search results are generated within app stores (e.g., what determines an app’s position as third in a search, versus thirtieth in that same search) and whether quality apps would have appeared using the set of search terms if apps had been culled beyond the top 30. Big data mining of app marketplaces could yield important information related to both consumer search patterns and search result rankings which could then inform consumer health literacy as well as researcher efforts to make quality, evidence-based interventions more accessible to consumers. Unfortunately, this data is privately owned through the Google and Apple corporations and would require specific permissions and access in order to be utilized for these purposes. Ultimately, it is important to keep in mind the scope of this current analysis as a consumer driven framework. Thus, these are the apps that appear to consumers who may be searching for self-management apps using common search terms, and these are the apps marketing themselves toward symptom management, regardless of the actual scope of content therein.

Second, the present focus on evidence-based treatment components predominantly emphasized cognitive-behavioral strategies. While CBT is considered the current gold-standard psychosocial intervention for youth anxiety (Walkup et al., 2008) and relevance mapping of randomized-controlled trials for youth anxiety formed the foundation of treatment component focus, it is possible that this lens failed to capture other potentially useful intervention methods. Indeed, some have cautioned developers away from attempting to directly mimic in-clinic sessions and tools in behavioral intervention technologies given limitations associated with traditional face-to-face treatments as well as the potential to restrict design, innovation and creativity (Schueller et al., 2013).

Third, the present findings indicate that the most readily accessible apps that are marketed for youth anxiety are not strongly grounded in evidence-based treatment content. That said, formal investigations have not been conducted testing the efficacy of leading consumer apps on the marketplace. Despite the absence of evidence-based treatment components, it is nonetheless possible that sampled apps are effective and include other active elements. Randomized evaluations formally examining the effectiveness of leading consumer products for reducing youth anxiety would complement the present analysis and help determine the state of readiness for private-sector mHealth products to meet consumer mental health needs.

Further, this analysis represents a snapshot of accessible consumer-marketed anxiety apps from 2016, and it is possible that conducting the same study at a different time could yield different results. To partially account for potential fluctuations on the consumer marketplace, apps were only included for content review if they demonstrated stable accessibility by maintaining their presence on the marketplace across a 6-month study window. Relative stability of the youth anxiety app marketplaces was observed, with over 90% of the initially identified apps still available on the marketplace 6 months later.

Finally, given considerable variability in the number of screens and functionalities across apps, we standardly selected the app description pages (which all apps include) to assess app reading level. Importantly, the reading level of an app’s description page may not be representative of the reading level of the actual app. That said, in this consumer-focused review, a description page may act as a “gatekeeper” to the actual app itself, with users judging app content based on what is marketed and how it is marketed in the description page. Reading level of this content, therefore, may impact the extent to which apps are eventually downloaded by users and ultimately their accessibility to consumers.

Conclusion

Despite limitations, the present study adds to a growing body of literature indicating that relatively little evidence-based treatment content is currently included in the majority of consumer-marketed mental health apps for a range of clinical targets (Shen et al., 2015; Juarascio et al., 2015). Given the high prevalence, early onset, stability and impairment associated with youth anxiety disorders, and given that today’s children, teens, and families are more digitally connected than any previous generation (Pew, 2015), self-management of anxiety symptoms through mobile platforms may represent a developmentally appropriate treatment opportunity with significant public health potential. However, the present findings raise concerns about the quality of the majority of accessible smartphone apps marketed for youth anxiety. The present findings also highlight how, at present, smartphone apps that are most readily available to consumers contain relatively simplistic functionality and do not yet incorporate the more advanced data collection and intervention strategies touted in the mHealth literature. The present examination also shows that despite the existence of higher quality apps developed and evaluated in research settings and in the private sector, such comprehensive self-management options that incorporate evidence-based treatment components are not highly visible to consumers and therefore will expectedly have limited reach. Future mHealth efforts would do well to explore strategies for researchers to better break into the consumer marketplace to increase the visibility and therefore accessibility of quality mHealth interventions.

Highlights.

  • We analyzed the quality and scope of consumer-marketed apps for youth anxiety

  • We found that evidence-based content within a systematic sample of apps was scant

  • Features that engage advanced functionalities of smartphones were rarely present

  • Findings underscore the need to better disseminate quality child anxiety apps

Acknowledgments

Funding: This work was supported in part by the National Institutes of Health (NIH) K23 MH090247 (PI: Comer).

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Financial disclosures: No authors have relevant financial interests to declare.

References

  1. Aldao A, De Los Reyes A. Introduction to the special section: Toward implementing physiological measures in clinical assessments of adult mental health. Journal of Psychopathology and Behavioral Assessment. 2016;38(1):1–4. doi: 10.1007/s10862-015-9521-y. [DOI] [Google Scholar]
  2. Anton MT, Jones DJ, Cuellar J, Forehand R, Gonzalez M, Honeycutt A, Khavjou O, Newey G, Edwards A, Jacobs M, Pitmman S. Caregiver use of the core components of technology-enhanced Helping the Noncompliant Child: A case series analysis of low-income families. Cognitive and Behavioral Practice. 2016;23:194–204. doi: 10.1016/j.cbpra.2015.04.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. APA Smartphone App Evaluation Task Force. APA Task Force to Develop Guidelines to Assess Apps, Wearable Sensors. 2016 Retrieved from: http://psychnews.psychiatryonline.org/doi/full/10.1176%2Fappi.pn.2016.2a10.
  4. Atienza AA, Zarcadoolas C, Vaughon W, Hughes P, Patel V, Chou WYS, Pritts J. Consumer attitudes and perceptions on mhealth privacy and security: findings from a mixed-methods study. Journal of Health Communication. 2015;20(6):673–679. doi: 10.1080/10810730.2015.1018560. [DOI] [PubMed] [Google Scholar]
  5. Becker KD, Boustani M, Gellatly R, Chorpita BF. Forty Years of Engagement Research in Children’s Mental Health Services: Multidimensional Measurement and Practice Elements. Journal of Clinical Child & Adolescent Psychology. 2017:1–23. doi: 10.1080/15374416.2017.1326121. [DOI] [PubMed] [Google Scholar]
  6. Boulos MNK, Wheeler S, Tavares C, Jones R. How smartphones are changing the face of mobile and participatory healthcare: An overview, with example from eCAALYX. Biomedical Engineering Online. 2011;10:24–38. doi: 10.1186/1475-925X-10-24. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Bunnell BE, Davidson TM, Dewey D, Price M, Ruggiero KJ. Rural and Urban/Suburban Families’ Use of a Web-Based Mental Health Intervention. Telemedicine ande-Health. 2016;23(5):390–396. doi: 10.1089/tmj.2016.0153. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Chan S, Torous J, Hinton L, Yellowlees P. Mobile tele-mental health: increasing applications and a move to hybrid models of care. Healthcare. 2014;2:220–33. doi: 10.3390/healthcare2020220. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Chorpita BF, Daleiden EL. Mapping evidence-based treatments for children and adolescents: application of the distillation and matching model to 615 treatments from 322 randomized trials. Journal of Consulting and Clinical Psychology. 2009;77(3):566–579. doi: 10.1037/a0014565. [DOI] [PubMed] [Google Scholar]
  10. Chorpita BF, Taylor AA, Francis SE, Moffitt C, Austin AA. Efficacy of modular cognitive behavior therapy for childhood anxiety disorders. Behavior Therapy. 2004;35(2):263–287. doi: 10.1016/S0005-7894(04)80039-X. [DOI] [Google Scholar]
  11. Chou T, Bry LJ, Comer JS. Multimedia field test: Evaluating the creative ambitions of SuperBetter and its quest to gamify mental health. Cognitive and Behavioral Practice. 2017;24:115–120. doi: 10.1016/j.cbpra.2016.10.002. [DOI] [Google Scholar]
  12. Chou T, Bry LJ, Comer JS. Overcoming traditional barriers only to encounter new ones: Doses of caution and direction as technology-enhanced treatments begin to “go live.”. Clinical Psychology: Science and Practice. 2017 doi: 10.1111/cpsp.12196. [DOI] [Google Scholar]
  13. Christensen H, Griffiths KM, Farrer L. Adherence in internet interventions for anxiety and depression: systematic review. Journal of Medical Internet Research. 2009;11(2):e13. doi: 10.2196/jmir.1194. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Comer JS, Barlow DH. The occasional case against broad dissemination and implementation: Retaining a role for specialty care in the delivery of psychological treatments. American Psychologist. 2014;69:1–18. doi: 10.1037/a0033582. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Comer JS, Blanco C, Grant B, Hasin D, Liu SM, Turner JB, Olfson M. Health-related quality of life across the anxiety disorders: Results from the National Epidemiologic Survey on Alcohol and Related Conditions. Journal of Clinical Psychiatry. 2011;72:43–50. doi: 10.4088/JCP.09m05094blu. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Compton SN, March JS, Brent D, Albano AM, Weersing R, Curry J. Cognitive-behavioral psychotherapy for anxiety and depressive disorders in children and adolescents: An evidence-based medicine review. Journal of the American Academy of Child and Adolescent Psychiatry. 2004;43(8):930–959. doi: 10.1097/01.chi.0000127589.57468.bf. [DOI] [PubMed] [Google Scholar]
  17. Costello EJ, Egger HL, Angold A. The developmental epidemiology of anxiety disorders: Phenomenology, prevalence and comorbidity. Child and Adolescent Psychiatric Clinics of North America. 2005;14(4):631–648. doi: 10.1016/j.chc.2005.06.003. [DOI] [PubMed] [Google Scholar]
  18. Crawford AM, Manassis K. Anxiety, social skills, friendship quality, and peer victimization: An integrated model. Journal of Anxiety Disorders. 2011;25(7):924–931. doi: 10.1016/jjanxdis.2011.05.005. [DOI] [PubMed] [Google Scholar]
  19. Davis TE, May A, Whiting SE. Evidence-based treatment of anxiety and phobia in children and adolescents: Current status and effects on the emotional response. Clinical Psychology Review. 2011;31(4):592–602. doi: 10.1016/j.cpr.2011.01.001. [DOI] [PubMed] [Google Scholar]
  20. Dogruel L, Joeckel S, Bowman ND. Choosing the right app: An exploratory perspective on heuristic decision processes for smartphone app selection. Mobile Media & Communication. 2015;3(1):125–144. doi: 10.1177/2050157914557509. [DOI] [Google Scholar]
  21. Derose KP, Baker DW. Limited English proficiency and Latinos’ use of physician services. Medical Care Research and Review. 2000;57(1):76–91. doi: 10.1177/107755870005700105. [DOI] [PubMed] [Google Scholar]
  22. Donker T, Petrie K, Proudfoot J, Clarke J, Birch MR, Christensen H. Smartphones for Smarter Delivery of Mental Health Programs: A Systematic Review. Journal of Medical Internet Research. 2013;15(11):e247–e260. doi: 10.2196/jmir.2791. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Egger HL, Angold A. Common emotional and behavioral disorders in preschool children: presentation, nosology, and epidemiology. Journal of Child Psychology and Psychiatry. 2006;47(3):313–337. doi: 10.1111/j.1469-7610.2006.01618.x. [DOI] [PubMed] [Google Scholar]
  24. Ehrenreich-May J, Rosenfield D, Queen AH, Kennedy SM, Remmes CS, Barlow DH. An initial waitlist-controlled trial of the unified protocol for the treatment of emotional disorders in adolescents. Journal of Anxiety Disorders. 2017;46:46–55. doi: 10.1016/jjanxdis.2016.10.006. [DOI] [PubMed] [Google Scholar]
  25. Fleiss JL. Measuring nominal scale agreement among many raters. Psychological Bulletin. 1971;76(5):378–382. doi: 10.1037/h0031619. [DOI] [Google Scholar]
  26. Food and Drug Administration. Policy for low risk devices. 2015 Retrieved from: http://www.fda.gov/downloads/medicaldevices/deviceregulationandguidance/guidancedocuments/ucm429674.pdf.
  27. Greco LA, Morris TL. Factors influencing the link between social anxiety and peer acceptance: Contributions of social skills and close friendships during middle childhood. Behavior Therapy. 2005;36(2):197–205. doi: 10.1016/S0005-7894(05)80068-1. [DOI] [Google Scholar]
  28. Hedman E, El Alaoui S, Lindefors N, Andersson E, Rück C, Ghaderi A, Ljótsson B. Clinical effectiveness and cost-effectiveness of Internet-vs. group-based cognitive behavior therapy for social anxiety disorder: 4-year follow-up of a randomized trial. Behaviour Research and Therapy. 2014;59:20–29. doi: 10.1016/j.brat.2014.05.010. [DOI] [PubMed] [Google Scholar]
  29. Higa-McMillan CK, Francis SE, Rith-Najarian L, Chorpita BF. Evidence base update: 50 years of research on treatment for child and adolescent anxiety. Journal of Clinical Child and Adolescent Psychology. 2016;45:91–113. doi: 10.1080/15374416.2015.1046177. [DOI] [PubMed] [Google Scholar]
  30. Hilty DM, Cobb HC, Neufeld JD, Bourgeois JA, Yellowlees PM. Telepsychiatry reduces geographic physician disparity in rural settings, but is it financially feasible because of reimbursement? Psychiatric Clinics of North America. 2008;31(1):85–94. doi: 10.1016/j.psc.2007.11.010. [DOI] [PubMed] [Google Scholar]
  31. Huckvale K, Prieto JT, Tilney M, Benghozi PJ, Car J. Unaddressed privacy risks in accredited health and wellness apps: a cross-sectional systematic assessment. BMC medicine. 2015;13(1):214–227. doi: 10.1186/s12916-015-0444-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Jones DJ, Anton M, Gonzalez M, Honeycutt A, Khavjou O, Forehand R, Parent J. Incorporating mobile phone technologies to expand evidence-based care. Cognitive and Behavioral Practice. 2015;22:281–290. doi: 10.1016/j.cbpra.2014.06.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Jones DJ, Forehand R, Cuellar J, Parent J, Honeycutt A, Khavjou O, Gonzalez M, Anton M, Newey GA. Technology-enhanced program for child disruptive behavior disorders: Development and pilot randomized control trial. Journal of Clinical Child and Adolescent Psychology. 2014;43:88–101. doi: 10.1080/15374416.2013.822308. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Juarascio AS, Manasse SM, Goldstein SP, Forman EM, Butryn ML. Review of smartphone applications for the treatment of eating disorders. European Eating Disorders Review. 2015;23(1):1–11. doi: 10.1002/erv.2327. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Kazdin AE, Blase SL. Rebooting psychotherapy research and practice to reduce the burden of mental illness. Perspectives on Psychological Science. 2011;6(1):21–37. doi: 10.1177/1745691610393527. [DOI] [PubMed] [Google Scholar]
  36. Kearney CA. School absenteeism and school refusal behavior in youth: A contemporary review. Clinical Psychology Review. 2008;28(3):451–471. doi: 10.1016/j.cpr.2007.07.012. [DOI] [PubMed] [Google Scholar]
  37. Kendall PC, Robin JA, Hedtke KA, Suveg C, Flannery-Schroeder E, Gosch E. Considering CBT with anxious youth? Think exposures. Cognitive and Behavioral Practice. 2005;12:136–150. doi: 10.1016/S1077-7229(05)80048-3. [DOI] [Google Scholar]
  38. Krebs P, Duncan DT. Health app use among US mobile phone owners: a national survey. JMIR mHealth and uHealth. 2015;3(4):e101–e113. doi: 10.2196/mhealth.4924. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. March S, Spence SH, Donovan CL. The efficacy of an internet-based cognitive-behavioral therapy intervention for child anxiety disorders. Journal of Pediatric Psychology. 2009;34(5):474–487. doi: 10.1093/jpepsy/jsn099. [DOI] [PubMed] [Google Scholar]
  40. Merikangas KR, He JP, Burstein M, Swanson SA, Avenevoli S, Cui L, Benjet C, Georgiades K, Swendsen J. Lifetime prevalence of mental disorders in US adolescents: Results from the national comorbidity study-adolescent supplement (NCS-A) Journal of the American Academy of Child and Adolescent Psychiatry. 2010;49(10):980–989. doi: 10.1016/jjaac.2010.05.017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Mohr D, Cuijpers P, Lehman K. Supportive accountability: a model for providing human support to enhance adherence to eHealth interventions. Journal of Medical Internet Research. 2011;13(1):e30–e41. doi: 10.2196/jmir.1602. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Mohr DC, Ho J, Duffecy J, Baron KG, Lehman KA, Jin L, Reifler D. Perceived barriers to psychological treatments and their relationship to depression. Journal of Clinical Psychology. 2010;66(4):394–409. doi: 10.1002/jclp.20659. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Mohr DC, Tomasino KN, Lattie EG, Palac HL, Kwasny MJ, Weingardt K, Caccamo L. IntelliCare: An Eclectic, Skills-Based App Suite for the Treatment of Depression and Anxiety. Journal of Medical Internet Research. 2017a;19(1):e10–e24. doi: 10.2196/jmir.6645. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Mohr DC, Zhang M, Schueller S. Personal sensing: Understanding mental health using ubiquitous sensors and machine learning. Annual Review of Clinical Psychology. 2017b;13(1):23–47. doi: 10.1146/annurev-clinpsy-032816-044949. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Mojtabai R. Trends in contacts with mental health professionals and cost barriers to mental health care among adults with significant psychological distress in the United States: 1997–2002. American Journal of Public Health. 2005;95(11):2009–2014. doi: 10.2105/AJPH.2003.037630. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Mojtabai R, Olfson M, Sampson NA, Jin R, Druss B, Wang PS, Kessler RC. Barriers to mental health treatment: Results from the national comorbidity survey replication (NCS-R) Psychological Medicine. 2011;41(8):1751–1761. doi: 10.1017/S0033291710002291. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Muñoz RF. Using evidence-based internet interventions to reduce health disparities worldwide. Journal of Medical Internet Research. 2010;12(5):e60–e70. doi: 10.2196/jmir.1463. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. National Information Board. Policy paper: NIB workstreams explain how we plan to improve digital services by making it easier for the public to access health and care information. 2015 Retrieved from: https://www.gov.uk/government/publications/national-information-boards-workstreams.
  49. Pew Research Center. Health fact sheet. 2017 Retrieved from: http://www.pewinternet.org/fact-sheets/health-fact-sheet/
  50. Pew Research Center. Technology device ownership. 2015 Retrieved from: http://www.pewinternet.org/2015/10/29/technology-device-ownership-2015/
  51. Pramana G, Parmanto B, Kendall PC, Silk JS. The SmartCAT: an m-health platform for ecological momentary intervention in child anxiety treatment. Telemedicine and e-Health. 2014;20(5):419–427. doi: 10.1089/tmj.2013.0214. [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Price M, Yuen EK, Davidson TM, Hubel G, Ruggiero KJ. Access and completion of a web-based treatment in a population-based sample of tornado-affected adolescents. Psychological Services. 2015;12(3):283–290. doi: 10.1037/ser0000017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Sareen J, Jagdeo A, Cox BJ, Clara I, ten Have M, Belik SL, de Graaf R, Stein MB. Perceived barriers to mental health service utilization in the United States, Ontario, and the Netherlands. Psychiatric Services. 2007;58(3):357–364. doi: 10.1176/ps.2007.58.3.357. [DOI] [PubMed] [Google Scholar]
  54. Schueller SM, Aguilera A, Mohr DC. Ecological momentary interventions for depression and anxiety. Depression and Anxiety. 2017;34(6):540–545. doi: 10.1002/da.22649. [DOI] [PubMed] [Google Scholar]
  55. Schueller SM, Muñoz RF, Mohr DC. Realizing the potential of behavioral intervention technologies. Current Directions in Psychological Science. 2013;22(6):478–483. doi: 10.1177/0963721413495872. [DOI] [Google Scholar]
  56. Schweitzer J, Synowiec C. The economics of eHealth and mHealth. Journal of Health Communication. 2012;17(sup1):73–81. doi: 10.1080/10810730.2011.649158. [DOI] [PubMed] [Google Scholar]
  57. Shen N, Levitan MJ, Johnson A, Bender JL, Hamilton-Page M, Jadad AR, Wiljer D. Finding a Depression App: A Review and Content Analysis of the Depression App Marketplace. Journal of Medical Internet Research: mHealth and uHealth. 2015;3(1):e16–e34. doi: 10.2196/mhealth.3713. [DOI] [PMC free article] [PubMed] [Google Scholar]
  58. Siegel RS, La Greca AM, Harrison HM. Peer victimization and social anxiety in adolescents: Prospective and reciprocal relationships. Journal of Youth and Adolescence. 2009;38(8):1096–1109. doi: 10.1007/s10964-009-9392-1. [DOI] [PubMed] [Google Scholar]
  59. Silverman WK, Ollendick TH. Evidence-based assessment of anxiety and its disorders. Journal of Clinical Child and Adolescent Psychology. 2005;34:380–411. doi: 10.1080/15374410701817907. [DOI] [PubMed] [Google Scholar]
  60. Silverman WK, Pina AA, Viswesvaran C. Evidence-based psychosocial treatments for phobic and anxiety disorders in children and adolescents. Journal of Clinical Child and Adolescent Psychology. 2008;37:104–130. doi: 10.1080/15374410701817907. [DOI] [PubMed] [Google Scholar]
  61. Spence SH, Holmes JM, March S, Lipp OV. The feasibility and outcome of clinic plus internet delivery of cognitive-behavior therapy for childhood anxiety. Journal of Consulting and Clinical Psychology. 2006;74(3):614–621. doi: 10.1037/0022-006X.74.3.614. [DOI] [PubMed] [Google Scholar]
  62. Thomas KC, Ellis AR, Konrad TR, Holzer CE, Morrissey JP. County-level estimates of mental health professional shortage in the United States. Psychiatric Services. 2009;60(10):1323–1328. doi: 10.1176/ps.2009.60.10.1323. [DOI] [PubMed] [Google Scholar]
  63. Torous J, Friedman R, Keshavan M. Smartphone ownership and interest in mobile applications to monitor symptoms of mental health conditions. JMIR mHealth and uHealth. 2014;2(1):e2–e10. doi: 10.2196/mhealth.2994. [DOI] [PMC free article] [PubMed] [Google Scholar]
  64. Vaish R, Wyngarden K, Chen J, Cheung B, Bernstein MS. Twitch crowdsourcing: crowd contributions in short bursts of time. Proceedings of the 32nd annual ACM conference on Human factors in computing systems. 2014 Apr;:3645–3654. [Google Scholar]
  65. Walkup JT, Albano AM, Piacentini J, Birmaher B, Compton SN, Sherrill JT, Kendall PC. Cognitive behavioral therapy, sertraline, or a combination in childhood anxiety. The New England Journal of Medicine. 2008;359(26):2753–2766. doi: 10.1056/NEJMoa0804633. [DOI] [PMC free article] [PubMed] [Google Scholar]
  66. Wang PS, Lane M, Olfson M, Pincus HA, Wells KB, Kessler RC. Twelve-month use of mental health services in the United States: Results from the national comorbidity survey replication. Archives of General Psychiatry. 2005;62(6):629–640. doi: 10.1001/archpsyc.62.6.629. [DOI] [PubMed] [Google Scholar]
  67. Washington State Institute for Public Policy. Remote cognitive behavioral therapy (CBT) for anxious children. 2015 Retrieved from: http://wsipp.wa.gov/BenefitCost/Program/64.
  68. Watts S, Mackenzie A, Thomas C, Griskaitis A, Mewton L, Williams A, Andrews G. CBT for depression: a pilot RCT comparing mobile phone vs. computer. BMC Psychiatry. 2013;13(1):49–58. doi: 10.1186/1471-244X-13-49. [DOI] [PMC free article] [PubMed] [Google Scholar]
  69. Weiner CL, Elkins RM, Pincus DB, Comer JS. Anxiety sensitivity and sleep-related problems in anxious youth. Journal of Anxiety Disorders. 2015;32:66–72. doi: 10.1016/jjanxdis.2015.03.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  70. Weisz JR, Jensen-Doss A, Hawley KM. Evidence-based youth psychotherapies versus usual clinical care: a meta-analysis of direct comparisons. American Psychologist. 2006;61(7):671–689. doi: 10.1037/0003-066X.6L7.671. [DOI] [PubMed] [Google Scholar]
  71. Westfall J, Mold J, Fagnan L. Practice-based research – “Blue Highways” on the NIH roadmap. Journal of the American Medical Association. 2007;297(4):403–406. doi: 10.1001/jama.297.4.403. [DOI] [PubMed] [Google Scholar]
  72. Whiteside SPH. Mobile device-based applications for childhood anxiety disorders. Journal of Child and Adolescent Psychopharmacology. 2016;26:246–251. doi: 10.1089/cap.2015.0010. [DOI] [PubMed] [Google Scholar]
  73. Wierzbicki M, Pekarik G. A meta-analysis of psychotherapy dropout. Professional Psychology: Research and Practice. 1993;24(2):190–195. doi: 10.1037/0735-7028.24.2.190. [DOI] [Google Scholar]
  74. Woodward LJ, Fergusson DM. Life course outcomes of young people with anxiety disorders in adolescence. Journal of the American Academy of Child and Adolescent Psychiatry. 2001;40(9):1086–1093. doi: 10.1097/00004583-200109000-00018. [DOI] [PubMed] [Google Scholar]
  75. Wu P, Goodwin R, Comer JS, Hoven C, Cohen P. The relationship between anxiety disorders and substance use among adolescents in the community: Specificity and gender differences. Journal of Youth and Adolescence. 2010;39:177–188. doi: 10.1007/s10964-008-9385-5. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES