Abstract
Ownership of mobile phones is on the rise, a trend in uptake that transcends age, region, race, and ethnicity, as well as income. It is precisely the emerging ubiquity of mobile phones that has sparked enthusiasm regarding their capacity to increase the reach and impact of health care, including mental health care. Community-based clinicians charged with transporting evidence-based interventions beyond research and training clinics are in turn, ideally and uniquely situated to capitalize on mobile phone uptake and functionality to bridge the efficacy to effectiveness gap. As such, this article delineates key considerations to guide these frontline clinicians in mobile phone-enhanced clinical practice, including an overview of industry data on the uptake of and evolution in the functionality of mobile phone platforms, conceptual considerations relevant to the integration of mobile phones into practice, representative empirical illustrations of mobile-phone enhanced assessment and treatment, and practical considerations relevant to ensuring the feasibility and sustainability of such an approach.
Keywords: mobile phone, smartphone, technology, clinical practice
The disparity between mental health need and the availability and impact of state-of-the field services in frontline service settings has been discussed for decades (American Psychological Association Task Force on Evidence-Based Practice for Children and Adolescents, 2008; Beidas & Kendall, 2010; National Institute of Mental Health, 1999; Sanders & Turner, 2005; Schoenwald et al., 2008). Although increasing access to and use of efficacious mental health services has at times seemed an unobtainable ideal, technology appears to be offering renewed enthusiasm and a promising, yet relatively untapped, resource for mental health providers (e.g., Clough & Casey, 2011; Kazdin & Blasé, 2011; Nelson, Bui, & Valazquez, 2011). At the forefront of enthusiasm regarding technology is the potential to leverage the availability and functionality of mobile phones, a prospect referred to elsewhere as “therapeutic gold” (Aguilera & Muench, 2012, p. 70). Yet, the swell of attention to the potential capacity of mobile phones in both the popular and academic press makes it a challenge to disentangle the extent to which there is evidence to support a mobile phone-enhanced practice approach. Accordingly, this article aims to update clinicians practicing in frontline, community-based mental health service settings on the state of the field in mobile phone-enhanced practice. As such, this article is not intended as an exhaustive review of the literature, but rather attempts to offer a lens through which clinicians may approach mobile phone-enhanced practice via both conceptual and empirical illustrations. Practical considerations that are key to ensuring the feasibility and sustainability of a mobile phone-enhanced practice approach are also considered.
Leveraging the Ubiquity and Functionality of Mobile Phones
The goal that technologies “weave themselves into the fabric of everyday life until they are indistinguishable from it” (Weiser, 1991, p. 94) generally refers to the human-technology interface, rather than a specific platform per se. Yet, it would be a challenge at this point in time to think of a platform that is more intricately woven into the personal, professional, and social contexts of consumers than the mobile phone. For the purposes of this review, the term “mobile phone” will be used broadly to refer to devices that range from more traditional platforms (i.e., functionality limited to making and receiving calls and text messaging) to “smartphones.” Although the category of smartphones is diverse, a smartphone is essentially a mobile phone with an operating system that allows more advanced connectivity and capability analogous to a handheld computer (e.g., built-in software applications, digital voice service, e-mail and text messages, Internet access).
In contrast to many other technologies, the diverse functionality afforded by mobile phones is relatively cost-effective for even the lowest-income consumers. For example, numerous phones and service plans are marketed through “big-box” retail outlets, although it is true that such programs have received some criticism regarding the actual cost savings afforded to consumers (e.g., WalMart's “Straight Talk”). Major wireless carriers, however, including AT&T (e.g., “GoPhone”), Verizon (e.g., “Pay as You Go”), and T-Mobile (e.g., “Pay by the Day”), also offer options such as prepaid phones with no annual contract (i.e., more feasible for lower-income consumers with no or poor credit histories). In turn, the affordability of mobile phones has led to an upsurge in popularity among a range of consumers.
In the United States, for example, the vast majority (estimates range from 85 to 91%) of American adults (18 and over) own a mobile phone, with the percentage of smartphones steeply rising (estimates range from 55 to 58%; Duggan & Smith, 2013; Fox & Rainie, 2014; Nielsen Wire, 2012; Smith, 2013). Affluent and higher-educated consumers are more likely to own mobile phones (Duggan & Smith, 2013; Smith, 2013; Zickuhr, 2013; Zickuhr & Smith, 2012); however, increased uptake and use cuts across sociodemographics. Ethnic and racial minorities, as well as the low income and less educated, for example, are not only purchasing smartphones at a higher rate than is typical of the digital divide but they are more likely than other sociodemographic groups to rely on smartphones as their primary, if not only, technology (Duggan & Smith, 2013; Rainie & Fox, 2012; Smith, 2013). Trends in the uptake and use of mobile phones extend internationally as well. Smartphone ownership, for example, increased internationally from 5% in 2009 to 22% by the end of 2013, reflecting an increase of nearly 1.3 billion smartphones worldwide in less than 5 years (Heggestuen, 2013). In turn, mobile phones offer the promise of a handheld and relatively cost-effective way to connect a diverse range of clients with mental health services and, in turn, potentially enhance assessment and treatment process and outcome (Aguilera & Muench, 2012; Boschen & Casey, 2008; Clough & Casey, 2011; Eonta et al., 2011; Jones, 2014; Jones et al., 2013; Luxton, McCann, Bush, Mishkind, & Reger, 2011).
Conceptual Considerations Guiding Mobile Phone-Enhanced Practice
Theory has guided, if not defined, the rich history and evolution of the evidence base guiding effective mental health services (see Kazak et al., 2010; Youngstrom, 2013, for reviews). Although there is much discussion in the literature regarding the empirical questions and hypotheses guiding technology-enhanced services work (Jones et al., 2013; Proudfoot et al., 2010; Riley et al., 2011; Ritterband, Thorndike, Cox, Kovatchev, & Gonder-Frederick, 2009), advancements in this area have progressed with relatively less regard for conceptual considerations. To this point, a myriad of smartphone applications (i.e.,“apps”) targeting a range of mental health issues are currently available via direct sale to potential mental health consumers, applications that target the assessment and treatment of specific diagnoses (e.g., bipolar disorder, depression, eating disorders) and more general clinical symptomatology (e.g., sleep, exercise, coping). Many of these applications appear to rely on the core elements of effective practices, including mood tracking, pleasant activities scheduling, and even more interactive options that target social information processing and social skills training (see Luxton et al., 2011, for a more exhaustive list of examples). However, a review of these applications suggests that the developers may not fully comprehend the conceptual underpinnings guiding the use of these skills in a mental health context or the likely challenges therapists and clients will experience using these skills in real-world practice settings and beyond (see Luxton et al., 2011, for a discussion of quality standards and safety).
As an example of the disconnect between the conceptual model underlying the core elements of treatment program and applications designed for mobile phones, behavioral parent training (BPT) is the standard of care for the treatment of early onset (3 to 8 years old) disruptive behavior disorders in children (Chorpita et al., 2011; Dretzke et al., 2009; Eyberg, Nelson, & Boggs, 2008; Forehand, Jones, & Parent, 2013). One core element of BPT, “time-out,” is the primary strategy for implementing effective consequences for child noncompliance and other problem behavior (McMahon & Forehand, 2003). The theoretical rationale guiding the use of time-out is that removal of attention—both positive and negative—in cases of child noncompliance and other problem behavior, is critical to effecting change in the parent–child relationship and, in turn, the child's behavior. Given the centrality of time-out for child behavior change, it is perhaps not surprising that there are numerous smartphone applications available to parents related to the time-out skill. The primary role of these applications as designed, however, is to tell parents how long the child should be in time-out (i.e., based on age) and/or tracking the elapsed time while the child is in time-out (e.g., 3 minutes).
Those who have implemented BPT with a family of a young child with oppositional defiant or conduct disorder, however, know that tracking time is rarely, if ever, the biggest challenge to parents' effective use of the time-out skill in session or at home (Jones, 2014; Jones et al., 2013, 2014; Jones, Forehand, McKee, Cuellar, & Kincaid, 2010). Rather, parents more typically deal with challenges such as the child refusing to go to the time-out chair, refusal to leave the time-out chair, and/or continued or escalating problem behavior in the time-out chair (McMahon & Forehand, 2003). Such responses to time-out from the child often elicit attention from the parent, the very cycle of interaction that BPT is designed to ameliorate and, ultimately, prevent. As such, more conceptually relevant and useful applications to enhance BPT would likely aim to help parents deal with these more realistic challenges both in the context of the treatment setting and at home.
The case of BPT, however, is but one example of the disconnection between conceptual considerations driving evidence-based practice and mobile phone-enhanced services. At the broadest and most basic level, the fundamental questions guiding clinical advances in this area must be Through what processes is it hypothesized that mobile phone enhancements will strengthen the reach and/or impact of mental health services? and What functionality is necessary to achieve these intended processes and, in turn, effects? Most basic to the discussion of the conceptual framework guiding mobile phone-enhanced practice is the extent to which therapist involvement is hypothesized to impact treatment outcomes. Research to date suggests that those seeking services for issues more likely targeted by prevention programs may have success with programming in which technology is the primary or only delivery vehicle (e.g., a Web-delivered program for weight loss); however, therapist involvement may be optimal, if not necessary, for the presenting issues more typically seen in community-based clinical practice (e.g., psychopathology; see Barak & Proudfoot, 2009; Clough & Casey, 2011; Jones et al., 2013; Mohr, Cujpers, & Lehman, 2011; Tate & Zabinski, 2004, for reviews). As such, this review will proceed with the general assumption that mobile phones will most likely be theorized to function as an enhancement or “adjunct” to—rather than replacement of—the current, face-to-face, standard of care in clinical practice (Clough & Casey, 2011).
Next, therapists should consider the diverse functionalities of mobile phones and, most important, the processes by which client use of the intended functionality is hypothesized to enhance the reach and impact of service delivery (see Jones et al., 2013, 2014; Riley et al., 2011; Ritterband et al., 2009; Venkatesh, Morris, Davis, & Davis, 2003, for reviews). This point relates to the possible range of functionalities afforded by the client's mobile phone platform (e.g., traditional mobile phone vs. smartphone), as well as the extent to which the functionalities allow the therapist to connect with the client in real time (e.g., videoconferencing) and/or asynchronously (e.g., text messages, email; Lovell, 2010; Tate & Zabinski, 2004; Titov, 2010). For example, self-determination theory has been used as a framework in technology-enhanced services work more broadly to highlight that technology has the capacity to increase the user's connection to the intervention, support for the skills taught in the intervention, and autonomy with using the skills beyond the context of the intervention (Williams, Lynch, & Glasgow, 2007). In the case of mobile phone-enhanced clinical practice in particular, a therapist may hypothesize that using the mobile phone for real-time connection with the client between sessions is critical to client progress and that face-to-face check-ins may have more of an impact than voice only (e.g., videoconference check-ins between sessions). In contrast, the therapist may expect that more general support for skill building and progress can be effective as long as it is timely, although not necessarily in real time (e.g., text messages, e-mail). Rather than relying on hypothetical examples alone, however, we turn next to empirical illustrations that highlight the intersection of hypotheses, mobile phone technology, and enhanced service delivery.
Empirical Illustrations of Mobile Phone-Enhanced Practice
Rather than provide an exhaustive review, our goal in this section is to highlight examples of pilot and feasibility work with clear attention to conceptual considerations that underlie the use of mobile phone enhancements to established, evidence-based assessment and treatment. Accordingly, we review representative research in the areas of both assessment and treatment, as well as incorporate illustrations of mobile phones into services for a diversity of presenting issues, in both individual and family treatment contexts, and in the treatment of children/adolescents and adults.
Mobile Phone-Enhanced Practice With Children and Adolescents
The Pew Internet and American Life Project (Lenhart, 2012) reports that 54 percent of American youth have a mobile phone, with rates increasing as children age into and through adolescence. Of the 54 percent of youth with a mobile phone, 23 percent of them have a smartphone. In turn, investigators are turning to mobile phones as a platform for increasing the reach and impact of assessment and treatment services for youth (Southam-Gerow, McLeod, Brown, Quinoy, & Avny, in press).
With regard to assessment, the availability and popularity of mobile phone use among youth has sparked interest in the potential for real-time mobile assessment opportunities. In contrast with the challenges associated with measuring nuanced and potentially labile clinical constructs (e.g., mood) inherent in more traditional assessment methods (e.g., laboratory tasks, retrospective reports), mobile phones provide a portable, handheld vehicle for in vivo assessment (Pine et al., 2004; Stone et al., 1998). To this end, work with mobile technologies generally suggests that mobile assessment may be a more efficacious strategy for tracking mood and behavior across a range of disorders and issues than traditional paper diaries (Burke et al., 2012; Shapiro et al., 2008). As an example of such work with youth, Silk and colleagues (2011) developed a mobile phone assessment designed to capture youth (7–11 years) emotions in vivo (also see Cole, Martin, & Dennis, 2004). Preliminary testing in a pilot and feasibility study (N = 79) included providing all youth with a mobile phone and calling youth at 12 random times throughout the week for 5 weeks. Calls included questions about current emotion, the broader social context in which the mood was occurring, and the behavior that linked to the emotion. Findings revealed the use of mobile phone calls to assess momentary emotionality was feasible. That is, calls were completed 92% of the time and were, on average, less than 4 minutes. Such work suggests that mobile phones may, in turn, provide a vehicle for clinicians to effectively monitor relevant clinical processes and outcomes in youth between sessions without substantially increasing time for the client or clinician.
There are also preliminary examples of mobile phone-enhanced interventions with youth. Researchers in the autism community, for example, are increasingly interested in the feasibility of using mobile technology in classrooms to support the social and life-functioning skills of students with autism spectrum disorders. Drawing on the persuasive technology design in social psychology, or the concept that technology can mimic human interaction and increase motivation, Mintz, Branch, March, and Lerman (2012) developed a cognitive support application for the classroom setting designed for interactive use by both teachers and students. The smartphone application has interactive functionality for both teachers and students and allows teachers to flexibly tailor the content of interventions via prompts (e.g., reminders for the child to pay attention to other people's perspectives), social stories (e.g., narratives about specific situations), daily diaries (e.g., logs of child's social interactions), and a “personal trainer” (i.e., specific support and intervention pieces tailored for an individual child). Qualitative results from a pilot study of four schools for children with autism spectrum disorders suggest promise. For example, parents and teachers agreed that the smartphone-enhanced, interactive approach to learning helped students to reach their goals and maintain the results, as well as improve the overall quality of the learning experience.
Turning from mobile phone-enhanced interventions for children to an example with adolescents, Whittaker and colleagues (2012) utilized mobile phone technology to increase the reach of intervention services to rural New Zealand adolescents, who the authors report are more vulnerable to depression due to sociocultural factors associated with ethnic discrimination and related stressors. Building on teenagers' reports that messaging is the most frequently used feature of their mobile phones, youth in the treatment arm of the intervention (n = 835) received psychoeducation about cognitive-behavioral skill building and support for using skills using both text (e.g., “You can take control of this” and “We can deal with negative thoughts”) and video (e.g., videos helping youth to identify cognitive distortions and problem-solving strategies) messages. Youth in the control group (n = 418) also received daily text messages; however, the content included topics such as healthy eating. Preliminary findings suggest that it is feasible to deliver the key messages of cognitive-behavioral therapy (CBT) via mobile phone to underserved youth in rural areas (i.e., more than 75% of the participants in the treatment group viewed at least half the sent messages) and that adolescents find these messages helpful (e.g., the majority of adolescents in the intervention group reported that the messages improved their mood and indicated that they would recommend the program to a friend).
Finally, mobile phones have also been used to address the challenges of engagement in family focused interventions for youth. For example, low-income families are more likely to have a child with an early onset disruptive behavior disorder, yet, less likely to engage in treatment than other sociodemographic groups. Accordingly, Jones and colleagues (2010, 2013, 2014) developed a smartphone-enhanced program that aimed to supplement BPT via increased connection between the family and the therapist and treatment program (e.g., between session video-call check-in, text message reminders about appointments and home practice), as well as increased support for skill building between sessions (e.g., modeling of skills via a skills video series, video recording home practices for therapist review and feedback). The pilot randomized controlled trial (n = 10 families in standard BPT; n = 9 in smartphone-enhanced BPT) suggested promise for the smartphone-enhanced BPT program to increase family engagement (e.g., smartphone-enhanced BPT families were more likely to come to sessions than standard BPT families) and, in turn, to enhance treatment outcomes (e.g., smartphone-enhanced BPT families evidenced greater effect sizes for treatment than standard BPT families; Jones et al., 2014). Moreover, the smartphone-enhanced BPT families required fewer sessions to complete the mastery-based BPT program than the standard BPT families, suggesting the potential cost-effectiveness of a smartphone-enhanced approach.
Mobile Phone-Enhanced Practice With Adults
In addition to the incorporation of mobile phones in the assessment and treatment of children, adolescents, and families, mobile phones have also been used to augment clinical practice with adults. For example, Aguilera and Muñoz (2011) conducted a usability and feasibility pilot study to test an automated text-messaging enhancement to CBT for depression in a community clinic serving low-income clients (N = 12). Consistent with the aforementioned work by both Mintz et al. (2012) and Jones et al. (2014) focusing on engaging underserved youth, the text-messaging feature in Aquilera and Muñoz's (2011) study was specifically aimed at increasing homework adherence, improving self-awareness, and helping track client progress in a difficult-to-engage low-income adult sample. As such, clients received weekly group therapy for depression (Muñoz, Ippen, Rao, Le, & Dwyer, 2000), as well as two to three daily text messages that inquired about their mood (on a scale ranging from 1 to 10), number of positive thoughts, and number of pleasant activities. Participants responded at a rate of 65% to text messages and reported overall positive experiences with the text-messaging enhancements to the program.
Shapiro and colleagues (2010) also used text messages in their intervention work with individuals with bulimia nervosa. Self-monitoring is considered one of the core components of cognitive-behavioral treatment for bulimia nervosa; however, clients with eating disorders seldom adhere to traditional methods of self-monitoring (i.e., paper diaries; Stone, Shiffman, Schwartz, Broderick, & Hufford, 2003). With the aim of increasing adherence, all clients in the Shapiro et al. (2010) study received treatment as usual (i.e., weekly treatment sessions), but were also asked to submit a nightly text message to the program indicating (a) frequency of binge-eating episodes, (b) frequency of purging episodes, and (c) peak urge to binge and purge (0 = no urge, 8 = extreme urge). Upon sending their nightly text message, participants immediately received a feedback message based on algorithms that included (a) how many goals were met (e.g., abstinence from binge eating and purging) and (b) clinical improvement or deterioration from the previous day. Within-group analysis indicated that 87% of participants adhered to self-monitoring and the number of binge-eating and purging episodes, as well as symptoms of depression and night eating, decreased significantly from baseline through posttreatment and follow-up. Of note in this study in particular, some data do suggest that the clinical benefit of mobile technologies is enhanced by the provision of personalized feedback to clients (Burke et al., 2012; Fjeldsoe, Marshall, & Miller, 2009).
In addition to text messaging, a broader range of mobile phone functionalities have been examined in research with adults as well. Rizvi, Dimeff, Skutch, Carroll, and Linehan (2011), for example, piloted a smartphone enhancement to dialectical behavioral therapy (DBT; Linehan, 1993) among women with comorbid borderline personality disorder (BPD) and substance use disorder (N = 22), a common comorbidity (Kosten, Kosten, & Rounsaville, 1989; Skodol, Oldham, & Gallaher, 1999). DBT (Linehan, 1993) is a well-established evidence-based treatment for BPD and involves individual therapy, skills training, a therapist consultation team, and as-needed phone consultation. Therapeutic progress in DBT is contingent upon individuals' ability to generalize DPT skills to their natural environment; however, the traditional parameters of the treatment setting limit the opportunities for therapists to provide feedback on in vivo skill use. In turn, a smartphone application, the “DBT Coach,” was designed to enhance the generalization of “opposite to emotion action” by providing skills coaching during a crisis, before individuals engage in dysfunctional behavior, as opposed to after a crisis, when individuals may have already engaged in dysfunctional behavior. The DBT Coach asked participants, for example, to identify the emotion they were currently experiencing and whether they were willing to work on changing the emotion. Consistent with the earlier point regarding the importance of tailored feedback (Burke et al., 2012; Fjeldsoe et al., 2009), the client's response to the second question regarding willingness to work on changing the emotion prompted the DBT to respond accordingly: (a) If the client responded “Yes,” then the DBT Coach directed the client to a list of emotion-specific opposite action behaviors (DBT component); or (b) If the client responded “No,” the DBT Coach directed the client to evaluate the pros and cons of changing the emotion and instructions to call the therapist if the application was not helpful. Within-group results indicated that clients used the DBT Coach 15 times on average during the trial period, which lasted an average of 13 days. Use of the DBT Coach was associated with decreased emotion intensity, urges to use substances, depression, and general distress.
Summary and Conclusions
Mobile phone-enhanced clinical practice is being examined in research with children, adolescents, and adults and across a range of disorders and presenting issues. Findings examining a range of functionalities inherent in mobile phone platforms (e.g., text messages), as well as software designed for mobile platforms (e.g., applications), suggest promise for improving assessment, enhancing treatment outcomes, and improving engagement and retention in services more generally. Yet, the promise of mobile applications must be interpreted with caution given that the research is still in a relative infancy, relying largely on pilot and feasibility work, including designs that fail to include random assignment or control groups. Accordingly, as we turn our attention in the next section to the feasibility and sustainability of this approach, it is our view that these considerations should be incorporated into both research and practice as mobile phone-enhanced approaches to clinical work evolve.
The Feasibility and Sustainability of Mobile Phone-Enhanced Practice
At the core of a discussion on mobile phone-enhanced practice must be feasibility and sustainability in real-world, community-based practice settings. Issues affecting the dissemination and implementation of the evidence base informing mobile phone-enhanced practice are myriad, but primary ones to consider include therapist training in evidence-based practice, acceptance of technology, cost, and ethics and safety.
Acceptance and Training
The implementation of evidence-based treatments into clinical practice settings has been hindered by a relative lack of acceptance for manualized approaches to assessment and treatment, as well as inadequate training opportunities even when acceptance and interest are high (American Psychological Association Task Force on Evidence-Based Practice for Children and Adolescents, 2008; Beidas & Kendall, 2010; Sanders & Turner, 2005; Schoenwald et al., 2008; United States Department of Health and Human Services, 2013). The uptake of and benefit from mobile phone-enhanced practice will rest partially on training in and acceptance of the tenets of evidence-based practice in general. However, this does not mean that research on the use of mobile phone technology or preliminary efforts to enhance interventions in clinical practice via mobile phone technology should be delayed—it is critical to be ready when the opportunity to deliver evidence-based mobile phone-enhanced practice presents itself!
Related to training and acceptance is comfort using mobile phone enhancements. We are not aware of research examining comfort with and, in turn, uptake of mobile phone-enhanced practice in particular; however, research on technology in services work more generally suggests that providers have primarily favorable attitudes toward technology as a delivery vehicle in mental health, particularly when clients are considered less vulnerable (e.g., Comer, Elkins, Chan, & Jones, in press; Stallard, Richardson, & Velleman, 2010). Research from related fields may also begin to help us identify factors that may predict variability in comfort and uptake among clinicians. For example, work by Venkatesh and colleagues (2003) in the area of business and management information technology examined correlates of use of technology among employees at four organizations over a 6-month period. One factor that they examined that seems especially relevant to consider regarding uptake of mobile phone-enhanced practice among frontline providers is “facilitating conditions” or the extent to which workers believed that the knowledge, resources, and support was available to facilitate the use of the technology. Not surprisingly, those workers who believed that there were higher levels of knowledge, resources, and support were more likely to use the technology than those who believed there were lower levels. Moreover, these findings were moderated by age and experience, such that having the necessary support and infrastructure for using the technology was especially important for older workers and those with more experience at the organization.
Generalizing from this research to the integration of mobile phone enhancements into clinical practice suggests that providing an infrastructure to support mobile phone-enhanced practice is likely critical to the successful uptake of this approach among frontline clinicians, perhaps particularly those who are older and, in turn, practicing in more traditional face-to-face therapy models for longer. That said, although it is true that older (age 65 and over) adults are less likely to own a mobile phone than younger adults, mobile phone ownership is rising among older adults (65%) as it is in every other demographic, suggesting that age may become less of a factor with time (Zickuhr & Madden, 2012). Moreover, the importance of infrastructure is a primary, if not the central, theme in the literature on dissemination and implementation of evidence-based practice more generally (Chorpita & Daleidan, 2014; Sanders & Turner, 2005; Wandersman, Duffy, Flaspohler, Noonan, & Lubell, 2008). Structural knowledge, support, and resources for mobile phone-enhanced practice in particular could then perhaps be folded into the more established foundation of dissemination and implementation efforts.
Costs
Another set of factors that must be considered with regard to feasibility and sustainability are potential costs, as well as cost-effectiveness. We are not aware of a review on the cost-effectiveness of mobile phone-enhanced treatment in particular; however, analyses of the costs of other technologies in health services research provide an indicator of the state of the field. For example, Tate, Finkelstein, Khavjou, and Gustafson (2009) highlight that cost-effectiveness is given as a primary rationale for developing service-based, Internet interventions; however, only 8 of the 420 studies published on Internet interventions from 1995 to 2008 actually reported economic data. Of these, the authors noted that “many were lacking comprehensive analyses” (Tate et al., 2009, p. 40), leaving relatively little to be said regarding the cost-effectiveness of technology-delivered interventions. Yet, an understanding of the cost and cost-effectiveness of mobile phone-enhanced practice is critical to guide policy makers and funding sources with increasingly limited mental health financial resources. One framework to guide such analysis breaks costs into two categories: development (i.e., “sunk”) and implementation (see Tate et al., 2009, for a more thorough discussion of cost-effectiveness analysis).
With regard to mobile phone-enhanced practice, development costs would generally be the costs to develop the functionality. For example, if the mobile phone enhancement to a particular treatment relies primarily or entirely on standard functionality of a mobile phone, such as making and receiving calls and/or text messages, then the development costs are negligible (if not zero dollars). If the requirement is for functionality beyond that in the traditional mobile phone platform, however, then development costs may vary widely. For example, the development costs will be more substantial for an agency that wants to create an application tailored to the needs of a specific target population and treatment (e.g., the DBT Coach; Rizvi et al., 2011) than for functionality that can be achieved via a less integrated, although potentially still effective, approach (e.g., using multiple, existing mood and behavior tracking applications in combination with video calls and text messaging). Importantly, these options may not be mutually exclusive. The latter less integrated, but less costly, approach may be more practical at least until more data are collected to assess whether the mobile phone enhancements are having the hypothesized effect (e.g., providing opportunities for in vivo assessment, enhancing adherence to a proscribed assessment protocol, increasing engagement in treatment). Then, if data do support the mobile phone-enhanced approach, integrating or bundling the enhancements into a tailored application may yield cost savings over time, which brings us to implementation costs and cost savings.
The greatest cost of delivery for the users (i.e., therapists and clients) is typically implementation costs (Tate et al., 2009). Implementation costs may include the cost of the platform (i.e., traditional mobile or smartphone) and any related user fees, which in the case of mobile phones are primarily service plans (e.g., data plan), as well as any additional “cost” of therapist time to utilize the mobile phone-enhanced aspects of the treatment program (e.g., How much additional “time” is the therapist spending with the client between sessions via the technology?). As noted earlier, however, service plans for mobile phones, including smartphones, continue to decline and increasingly include subsidized options. Moreover, providers can consider the extent to which the desired functionality must by necessity include implementation costs (e.g., increased cost of data usage on a smartphone) or whether more creative and less expensive options can be considered (e.g., skills demonstration videos can be loaded and housed on a smartphone without increasing data usage or costs). Although preliminary, some work also does suggest that mobile phone enhancements to treatment may yield implementation cost savings with regard to therapist time (i.e., fewer sessions; Jones et al., 2014). Finally, mobile phones are the operating systems for a range of innovations in technology, including the evolution in wearable technology (e.g., Google Glass uses a smartphone platform as the operating system), suggesting that forethought in the development of mobile phone applications may allow mobile phone-enhanced practice to cost-effectively evolve with time.
By considering and tracking both development and implementation costs, therapists and agencies can better understand the extent to which the functionality of the mobile phone-enhanced approach will impact the hypothesized processes and/or outcomes of practice and, if so, whether this boost is worth the additional cost associated with the platform and functionality. For example, if mobile phone technology enhances the reach and impact of evidence-based practice, then any resulting mental health outcomes may be well worth the cost (Drummond, O'Brien, Stoddart, & Torrance, 1997; Haddix, Teutsch, & Corso, 2003; Jones et al., 2014).
Ethics and Safety
A final set of issues that must be considered in thinking about the feasibility and sustainability of mobile phone-enhanced practice is the ethics and safety of such an approach. As with other aspects of the use of technology in services work, the field is progressing far more quickly than advances in relevant ethical and practice guidelines (Jones et al., 2013; Nelson et al., 2011; Nelson & Velasquez, 2011; Novotney, 2011; Reed, McLaughlin, & Milholland, 2000; Richardson et al., 2009). Some of the potential ethical issues related to mobile phone-enhanced services delivery include, but are not limited to, cross-state licensure (e.g., therapist conducting some aspect of treatment with a client who is traveling or living in another state), standard of care (e.g., emergency protocols when a client conveys an indicator of risk during a video call, e-mail, or text), and privacy and security (e.g., use of secure networks to store sensitive data retrieved via the mobile phone). Using issues of risk assessment and response as an example, how will the potential for self-harm or harm to others be assessed via mobile phone-enhanced interventions? Will a reliance on mobile phones increase the probability that signs of risk will be overlooked or missed? Finally, will opportunities to intervene and thus protect the client, family member, or some identified “other” be reduced in a mobile phone-enhanced practice approach?
Importantly, leaders in mental health and across our governing and guiding agencies are beginning to address these very complicated issues, which must continue to be resolved as new technologies emerge (American Psychological Association, 2012; American Telemedicine Association, 2009; Nelson & Velasquez, 2011). In fact, the issue of digital traces or the security of data captured by technology is now at the forefront of some of the most heated and sensitive public debates of our time. Clinicians must in turn pay attention to and ideally be involved in these discussions if they are going to safely and effectively utilize mobile phone technology to enhance the reach and impact of services for their clients. Notably, the opportunities for breaches of confidentiality proliferate when we think of even the most basic mobile phone functionalities many use to communicate with clients (e.g., text messages, e-mail), let alone those that may arise if we are not careful with more advanced functionalities (e.g., using mobile phones to capture sensitive data, such as surveys and videos that may include identifiable information). Most believe that the potential benefits outweigh the potential risks, but this is only true if frontline providers remain mindful of the risks associated with emerging technologies, including mobile phones.
Conclusions
In summary, this is an exciting time. The evolution in ubiquitous technology offers and will continue to offer innovative mobile platforms for extending the reach and impact of clinical practice. This frontier is one that can be guided by the extent to which and how we harness the capacity of mobile phones in particular, given the uptake in, use of, and even reliance on mobile phones across diverse sociodemographic groups. Yet, with opportunity comes responsibility, responsibilities that include remaining true to the established pillars of evidence-based practice, while also being practically and fiscally responsible in the ways in which we envision and assess the clinical advancements afforded by such an approach. Perhaps even more important than presenting the current state of the field, this article intends to guide and foster further discussions at the intersection of technology and frontline service delivery. Such discussions, of course, must continue to evolve with advances in mobile phone technology and technology more broadly, as well as changes that occur as our field responds to emerging and evolving issues such as privacy and data security.
Highlights.
Mobile phones show capacity to increase the reach and impact of health care and mental health care
Community-based clinicians capitalize on mobile phone uptake and functionality to bridge efficacy to effectiveness gap
Clinical practice integrates data on mobile phone platforms and conceptual, empirical, and practical considerations
Acknowledgments
Author Note: Funding for this project was provided by R34MH082956 (ClinicalTrials.gov Identifier: NCT01367847) and R01MH100377 from the National Institute of Mental Health. We would like to extend our special thanks to Dr. Joel Sherrill, program chief, Child and Adolescent Psychosocial Services Division, National Institute of Mental Health, for his continued guidance throughout the grant application and administration process. Our additional appreciation to our industry partner colleagues (Greg Newey at Research Technology Solutions and Sean Doherty at CrossComm Inc.) who have been instrumental in our own research on mobile phone-enhanced service delivery. Finally, we are grateful to our Advisory Panel of clinicians who greatly influenced our formative work in this area: April Harris-Britt, Ph.D., AHB Psychological Associates; Kathryn J. Smith, MSW, Center for Child and Family Health; Jennifer Youngstrom, Ph.D., UNC Child and Family Clinic; Sarah Stearns, Ph.D., Dartmouth-Hitchcock Psychiatry Associates; and Timothy Verduin, Ph.D., NYU Child Study Center.
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Contributor Information
Deborah J. Jones, University of North Carolina at Chapel Hill
Margaret Anton, University of North Carolina at Chapel Hill.
Michelle Gonzalez, University of North Carolina at Chapel Hill.
Amanda Honeycutt, Research Triangle International.
Olga Khavjou, Research Triangle International.
Rex Forehand, University of Vermont.
Justin Parent, University of Vermont.
References
- Aguilera A, Muench F. There's an app for that: Information technology applications for cognitive behavioral practitioners. The Behavior Therapist. 2012;35:65–73. [PMC free article] [PubMed] [Google Scholar]
 - Aguilera A, Muñoz RF. Text messaging as an adjunct to CBT in low-income populations: A usability and feasibility pilot study. Professional Psychology: Research and Practice. 2011;42(6):472–478. doi: 10.1037/a0025499. [DOI] [PMC free article] [PubMed] [Google Scholar]
 - American Psychological Association. Guidelines for the practice of telepsychology. 2012 Retrieved from http://apacustomout.apa.org/commentCentral/default.aspx?site=26.
 - American Psychological Association. Disseminating evidence-based practice for children And adolescents: A systems approach to enhancing care. Washington, DC: American Psychological Association; 2008. Task Force on Evidence-Based Practice for Children and Adolescents. [Google Scholar]
 - American Telemedicine Association. Practice guidelines for videoconferencing-based telemental health. 2009 doi: 10.1089/tmj.2010.0148. Retrieved from http://www.americantelemed.org/files/public/standards/PracticeGuidelinesforVideoconfrencing-Based%20TelementalHealth.pdf. [DOI] [PubMed]
 - Barak A, Proudfoot JG. Defining internet-supported therapeutic interventions. Annals of Behavioral Medicine. 2009;38(1):4–17. doi: 10.1007/s12160-009-9130-7. [DOI] [PubMed] [Google Scholar]
 - Beidas RS, Kendall PC. Training therapists in evidence-based practice: A critical review of studies from a systems-contextual perspective. Clinical Psychology: Science and Practice. 2010;17(1):1–30. doi: 10.1111/j.1468-2850.2009.01187.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
 - Boschen MJ, Casey LM. The use of mobile telephones as adjuncts to cognitive behavioral psychotherapy. Professional Psychology: Research and Practice. 2008;39:546–552. doi: 10.1037/0735-7028.39.5.546. [DOI] [Google Scholar]
 - Burke LE, Styn MA, Sereika SM, Conroy MB, Ye L, Glanz K, Ewing LJ. Using mHealth technology to enhance self-monitoring for weight loss: A randomized trial. American Journal of Preventive Medicine. 2012;43(1):20–26. doi: 10.1016/j.amepre.2012.03.016. doi:http://dx.doi.org/10.1016/j.amepre.2012.03.016. [DOI] [PMC free article] [PubMed] [Google Scholar]
 - Chorpita BF, Daleiden EL. Doing more with what we know: Introduction to the special section. Journal of Clinical Child and Adolescent Psychology. 2014;43:143–144. doi: 10.1080/15374416.2013.869751. doi:0.1080/15374416.2013.869751. [DOI] [PubMed] [Google Scholar]
 - Chorpita BF, Daleiden EL, Ebesutani C, Young J, Becker KD, Nakamura BJ, Starace N. Evidence-based treatments for children and adolescents: An updated review of indicators of efficacy and effectiveness. Clinical Psychology: Science and Practice. 2011;18:154–172. doi: 10.1111/j.1468-2850.2011.01247.x. [DOI] [Google Scholar]
 - Clough BA, Casey LM. Technological adjuncts to increase adherence to therapy: A review. Clinical Psychology Review. 2011;31:697–710. doi: 10.1016/j.cpr.2011.03.006. [DOI] [PubMed] [Google Scholar]
 - Cole PM, Martin SE, Dennis TA. Emotion regulation as a scientific construct: Methodological challenges and directions for child development research. Child Development. 2004;75(2):317–333. doi: 10.1111/j.1467-8624.2004.00673.x. doi:http://dx.doi.org/10.1111/j.1467-8624.2004.00673.x. [DOI] [PubMed] [Google Scholar]
 - Comer JS, Elkins RM, Chan PT, Jones DJ. New methods of service delivery for children's mental health care. In: Alfano C, Beidel D, editors. Handbook of evidence-based treatments for childhood disorders. in press. [Google Scholar]
 - Dretzke J, Davenport C, Frew E, Barlow J, Stewart-Brown S, Bayliss S, et al. Hyde C. The clinical effectiveness of different parenting programmes for children with conduct problems: A systematic review of randomised controlled trials. Child and Adolescent Psychiatry and Mental Health. 2009;3 doi: 10.1186/1753-2000-3-7. doi:http://dx.doi.org/10.1186/1753-2000-3-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
 - Drummond MF, O'Brien B, Stoddart GL, Torrance GW. Methods for the economic evaluation of health care programmes. 2nd. New York, NY: Oxford University Press; 1997. [Google Scholar]
 - Duggan M, Smith A. Cell internet use 2013. Washington, DC: Pew Research Center's Internet and American Life Project; 2013. Report released September 26, 2013. Retrieved October 15, 2013, from http://pewinternet.org/Reports/2013/Cell-Internet/Main-Findings/Cell-Internet.aspx. [Google Scholar]
 - Eonta AM, Christon LM, Hourigan SE, Ravindran N, Vrana SR, Southam-Gerow M. Using everyday technology to enhance evidence-based treatments. Professional Psychology: Research and Practice. 2011;42(6):513–520. doi:http://dx.doi.org/10.1037/a0025825. [Google Scholar]
 - Eyberg SM, Nelson MM, Boggs SR. Evidence-based psychosocial treatments for children and adolescents with disruptive behavior. Journal of Clinical Child and Adolescent Psychology. 2008;37(1):215–237. doi: 10.1080/15374410701820117. [DOI] [PubMed] [Google Scholar]
 - Fjeldsoe BS, Marshall AL, Miller YD. Behavior change interventions delivered by mobile telephone short-message service. American Journal of Preventive Medicine. 2009;36(2):165–173. doi: 10.1016/j.amepre.2008.09.040. doi:http://dx.doi.org.libproxy.lib.unc.edu/10.1016/j.amepre.2008.09.040. [DOI] [PubMed] [Google Scholar]
 - Forehand R, Jones DJ, Parent J. Behavioral parenting interventions for child disruptive behaviors and anxiety: What's different and what's the same? Clinical Psychology Review. 2013;33:133–145. doi: 10.1016/j.cpr.2012.10.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
 - Fox S, Rainie L. Part 1: How the internet has woven itself into American life. Washington, DC: Pew Research Internet Project; 2014. Report released February 27, 2014. Retrieved February 28, 2014, from http://www.pewinternet.org/2014/02/27/part-1-how-the-internet-has-woven-itself-into-american-life/ [Google Scholar]
 - Haddix AC, Teutsch SM, Corso PS. Prevention effectiveness. New York, NY: Oxford University Press; 2003. [Google Scholar]
 - Heggestuen J. One in every 5 people in the world own a smartphone, one in every 17 own a tablet. Business Insider 2013 . Retrieved from http://www.businessinsider.com/smartphone-and-tablet-penetration-2013-10.
 - Jones DJ. Future directions in the treatment of children, adolescents, and families: Leveraging emerging technologies toward improved service delivery. Journal of Clinical Child and Adolescent Psychology. 2014;43(1):128–142. doi: 10.1080/15374416.2013.859082. [DOI] [PMC free article] [PubMed] [Google Scholar]
 - Jones DJ, Forehand R, Cuellar J, Kincaid C, Parent J, Fenton N, Goodrum N. Harnessing innovative technologies to advance children's mental health: Behavioral parent training as an example. Clinical Psychology Review. 2013;32:241–252. doi: 10.1016/j.cpr.2012.11.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
 - Jones DJ, Forehand R, Cuellar J, Parent J, Honeycutt A, Khavjou O, Newey G. Technology-enhanced program for child disruptive behavior disorders: Development and pilot randomized control trial. Journal of Clinical Child and Adolescent Psychology. 2014;43(1):88–101. doi: 10.1080/15374416.2013.822308. [DOI] [PMC free article] [PubMed] [Google Scholar]
 - Jones DJ, Forehand R, McKee L, Cuellar J, Kincaid C. Behavioral parent training: Is there an “app” for that? The Behavior Therapist. 2010;33:72–77. [PMC free article] [PubMed] [Google Scholar]
 - Kazak AE, Hoagwood K, Weisz JR, Hood K, Kratochwill TR, Vargas LA, Banez GA. A meta-systems approach to evidence-based practice for children and adolescents. American Psychologist. 2010;65(2):85–97. doi: 10.1037/a0017784. [DOI] [PubMed] [Google Scholar]
 - Kazdin AE, Blasé SL. Rebooting psychotherapy research and practice to reduce the burden of mental illness. Perspectives on Psychological Science. 2011;6:21–37. doi: 10.1177/1745691610393527. [DOI] [PubMed] [Google Scholar]
 - Kosten TA, Kosten TR, Rounsaville BJ. Personality disorders in opiate addicts show prognostic specificity. Journal of Substance Abuse Treatment. 1989;6(3):163–168. doi: 10.1016/0740-5472(89)90003-2. [DOI] [PubMed] [Google Scholar]
 - Lenhart A. Teens, smartphones, and texting. Pew Internet and American Life Project; 2012. Retrieved from http://pewinternet.org/Reports/2012/Teens-and-smartphones/Cell-phone-ownership/Smartphones.aspx. [Google Scholar]
 - Linehan MM. Cognitive behavioral treatment of borderline personality disorder. New York, NY: Guilford Press; 1993. [Google Scholar]
 - Lovell K. Supporting low intensity interventions using the telephone. In: Bennett-Levy J, Richards DA, Farrand P, Christensen H, Griffiths KM, Kavanagh DJ, et al.Williams C, editors. Oxford guide to low intensity CBT interventions. Oxford, UK: Oxford University Press; 2010. pp. 275–280. [Google Scholar]
 - Luxton DD, McCann RA, Bush NE, Mishkind MC, Reger GM. mHealth for mental health: Integrating smartphone technology in behavioral healthcare. Professional Psychology: Research and Practice. 2011;42(6):505–512. doi:http://dx.doi.org/10.1037/a0024485. [Google Scholar]
 - McMahon RJ, Forehand R. Helping the noncompliant child: A clinician's guide to effective parent training. 2nd. New York, NY: Guilford Press; 2003. [Google Scholar]
 - Mintz J, Branch C, March C, Lerman S. Key factors mediating the use of a mobile technology tool designed to develop social and life skills in children with autistic spectrum disorders. Computers and Education. 2012;58(1):53–62. [Google Scholar]
 - Mohr DC, Cuijpers P, Lehman J. Supportive accountability: A model for providing human support to enhance adherence to eHealth interventions. Journal of Medical Internet Research. 2011;13:136–146. doi: 10.2196/jmir.1602. [DOI] [PMC free article] [PubMed] [Google Scholar]
 - Muñoz RF, Ippen CG, Rao S, Le H, Dwyer EV. Manual for group cognitive behavioral therapy of major depression. San Francisco, CA: San Francisco General Hospital Depression Clinic; 2000. [Google Scholar]
 - National Institute of Mental Health. Mental health: A report of the surgeon general. 1999 Available from http://www.surgeongeneral.gov/library/mentalhealth/home.html.
 - Nelson EL, Bui T, Velasquez SE. Telepsychology: Research and practice overview. Child Adolescent Psychiatric Clinics of North America. 2011;20(1):67–79. doi: 10.1016/j.chc.2010.08.005. [DOI] [PubMed] [Google Scholar]
 - Nelson EL, Velasquez SE. Implementing psychological services over telemedicine. Professional Psychology Research and Practice. 2011;42:535–542. [Google Scholar]
 - Nielsen Wire. America's new mobile majority: A look at smartphone owners in the U S. 2012 Available at http://blog.nielsen.com/nielsenwire/online_mobile/who-owns-smartphones-in-the-us/
 - Novotney A. A new emphasis on telehealth: How can psychologists stay ahead of the curve—and keep patients safe? Monitor on Psychology. 2011;42(6):40–45. [Google Scholar]
 - Pine DS, Lissek S, Klein RG, Mannuzza S, Moulton JL, Guardino M, Woldehawariat G. Face-memory and emotion: Associations with major depression in children and adolescents. Journal of Child Psychology and Psychiatry. 2004;45(7):1199–1208. doi: 10.1111/j.1469-7610.2004.00311.x. [DOI] [PubMed] [Google Scholar]
 - Proudfoot J, Parker G, Pavlovic DH, Manicavasagar V, Adler E, Whitton A. Community attitudes to the appropriation of mobile phones for monitoring and managing depression, anxiety, and stress. Journal of Medical Internet Research. 2010;12(5) doi: 10.2196/jmir.1475. [DOI] [PMC free article] [PubMed] [Google Scholar]
 - Rainie L, Fox S. Just-in-time information through mobile connections. Pew Internet & American Life Project; 2012. Internet. [Google Scholar]
 - Reed GM, McLaughlin CH, Milholland K. Ten interdisciplinary principles for professional practice in telehealth: Implications for psychology. Professional Psychology: Research and Practice. 2000;31:170–178. doi: 10.103//0735-7028.31.2.170. [DOI] [Google Scholar]
 - Richardson LK, Frueh BC, Grubaugh AL, Johnston RH, Egede L, Johnson RH, Elhai JD. Current directions in videoconferencing tele-mental health research. Clinical Psychology: Research and Practice. 2009;16:323–338. doi: 10.1111/j.1468-2850.2009.01170.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
 - Riley WT, Rivera DE, Atienza AA, Nilsen W, Allison SM, Mermelstein R. Health behavior models in the age of mobile interventions: Are our theories up to the task? Translational Behavioral Medicine. 2011;1(1):53–71. doi: 10.1007/s13142-011-0021-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
 - Ritterband LM, Thorndike FP, Cox DJ, Kovatchev BP, Gonder-Frederick LA. A behavior change model for internet interventions. Annals of Behavioral Medicine. 2009;38(1):18–27. doi: 10.1007/s12160-009-9133-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
 - Rizvi SL, Dimeff LA, Skutch J, Carroll D, Linehan MM. A pilot study of the DBT coach: An interactive mobile phone application for individuals with borderline personality disorder and substance use disorder. Behavior Therapy. 2011;42(4):589–600. doi: 10.1016/j.beth.2011.01.003. [DOI] [PubMed] [Google Scholar]
 - Sanders MR, Turner KMT. Reflections on the challenges of effective dissemination of behavioral family intervention: Our experiences with the Triple P Positive Parenting Program. Child and Adolescent Mental Health. 2005;10:158–169. doi: 10.1111/j.1475-3588.2005.00367.x. [DOI] [PubMed] [Google Scholar]
 - Schoenwald SK, Chapman JE, Kelleher K, Hoagwood KE, Landsverk J, Stevens J, et al. Rolls-Reutz J. A survey of the infrastructure for children's mental health services: Implications for the implementation of empirically supported treatments (ESTs) Administration and Policy in Mental Health and Mental Health Services Research. 2008;35(1-2):84–97. doi: 10.1007/s10488-007-0147-6. doi:http://dx.doi.org/10.1007/s10488-007-0147-6. [DOI] [PubMed] [Google Scholar]
 - Shapiro JR, Bauer S, Andrews E, Pisetsky E, Bulik-Sullivan B, Hamer RM, Bulik CM. Mobile therapy: Use of text-messaging in the treatment of bulimia nervosa. International Journal of Eating Disorders. 2010;43(6):513–519. doi: 10.1002/eat.20744. [DOI] [PubMed] [Google Scholar]
 - Shapiro JR, Bauer S, Hamer RM, Kordy H, Ward D, Bulik CM. Use of text messaging for monitoring sugar-sweetened beverages, physical activity, and screen time in children: A pilot study. Journal of Nutrition ducation and Behavior. 2008;40(6):385–391. doi: 10.1016/j.jneb.2007.09.014. [DOI] [PMC free article] [PubMed] [Google Scholar]
 - Silk JS, Forbes EE, Whalen DJ, Jakubcak JL, Thompson WK, Ryan ND, et al. Dahl RE. Daily emotional dynamics in depressed youth: A cell phone ecological momentary assessment study. Journal of Experimental Child Psychology. 2011;110(2):241–257. doi: 10.1016/j.jecp.2010.10.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
 - Skodol AE, Oldham JM, Gallaher PE. Axis II comorbidity of substance use disorders among patients referred for treatment. American Journal of Psychiatry. 1999;156:733–738. doi: 10.1176/ajp.156.5.733. [DOI] [PubMed] [Google Scholar]
 - Smith A. Smartphone ownership—2013 update. Washington, DC: Pew Research Center's Internet and American Life Project; 2013. Report released June 5, 2013. Retrieved October 15, 2013, from http://pewinternet.org/Reports/2013/Smartphone-Ownership-2013.aspx. [Google Scholar]
 - Southam-Gerow MA, McLeod BD, Brown RC, Quinoy AM, Avny SB. Cognitive behavioral therapy for adolescents. In: Brown BB, Prinstein MJ, editors. Encyclopedia of adolescence. Vol. 3. New York, NY: Elsevier; in press. [Google Scholar]
 - Stallard P, Richardson T, Velleman S. Clinician's attitudes towards the use of computerized cognitive behavior therapy (cCBT) with children and adolescents. Behavioural and Cognitive Psychotherapy. 2010;38(5):545–560. doi: 10.1017/S1352465810000421. [DOI] [PubMed] [Google Scholar]
 - Stone AA, Schwartz JE, Neale JM, Shiffman S, Marco CA, Hickcox M, et al. Cruise LJ. A comparison of coping assessed by ecological momentary assessment and retrospective recall. Journal of Personality and Social Psychology. 1998;74(6):1670–1680. doi: 10.1037//0022-3514.74.6.1670. [DOI] [PubMed] [Google Scholar]
 - Stone AA, Shiffman S, Schwartz JE, Broderick JE, Hufford MR. Patient non-complicance with paper diaries. British Medical Journal. 2002;324:1193–1194. doi: 10.1136/bmj.324.7347.1193. doi:http://dx.doi.org/10.1136/bmj.324.7347.1193. [DOI] [PMC free article] [PubMed] [Google Scholar]
 - Tate DF, Finkelstein EA, Khavjou O, Alison Gustafson RD. Cost effectiveness of internet interventions: Review and recommendations. Annals of Behavioral Medicine. 2009;38(1):40–45. doi: 10.1007/s12160-009-9131-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
 - Tate DF, Zabinski MF. Computer and internet applications for psychological treatment: Update for clinicians. Journal of Clinical Psychology. 2004;60:209–220. doi: 10.1002/jclp.10247. [DOI] [PubMed] [Google Scholar]
 - Titov N. Email in low intensity CBT interventions. In: Bennett-Levy J, Richards DA, Farrand P, Christensen H, Griffiths KH, Kavanagh DJ, et al.Williams C, editors. Oxford guide to low intensity CBT interventions. Oxford, UK: Oxford University Press; 2010. pp. 287–294. [Google Scholar]
 - United States Department of Health and Human Services. Shortage designation: Health professional shortage areas and medically underserved areas/populations. 2013 Retrieved from http://www.hrsa.gov/shortage/
 - Venkatesh V, Morris MG, Davis GB, Davis FD. User acceptance of information technology: Toward a unified view. MIS Quarterly. 2003;27(3):425–478. [Google Scholar]
 - Wandersman A, Duffy J, Flaspohler P, Noonan R, Lubell K. Bridging the gap between prevention research and practice: The interactive systems framework for dissemination and implementation. American Journal of Community Psychology. 2008;41:171–181. doi: 10.1007/s10464-008-9174-z. [DOI] [PubMed] [Google Scholar]
 - Weiser M. The computer for the twenty-first century. Scientific American. 1991:94–104. [Google Scholar]
 - Whittaker R, Merry S, Stasiak K, McDowell H, Doherty I, Shepherd M, et al. Rodgers A. MEMO—A mobile phone depression prevention intervention for adolescents: Development process and postprogram findings on acceptability from a randomized controlled trial. Journal of Medical Internet Research. 2012;14:13. doi: 10.2196/jmir.1857. [DOI] [PMC free article] [PubMed] [Google Scholar]
 - Williams GC, Lynch MF, Glasgow RE. Computer-assisted intervention improves patient-centered diabetes care by increasing autonomy support. Health Psychology. 2007;26:728–734. doi: 10.1037/0278-6133.26.6.728. [DOI] [PubMed] [Google Scholar]
 - Youngstrom EA. Future directions in psychological assessment: Combining evidence-based medicine innovations with psychology's historical strengths to enhance utility. Journal of Clinical Child and Adolescent Psychology. 2013;42:139–159. doi: 10.1080/15374416.2012.73658. [DOI] [PubMed] [Google Scholar]
 - Zickuhr K. Who's not online and why? Washington, DC: Pew Research Center's Internet and American Life Project; 2013. Report released September 25, 2013. Retrieved October 15, 2013, from http://pewinternet.org/Reports/2013/Non-internet-users.aspx. [Google Scholar]
 - Zickuhr K, Madden M. Older adults and internet use. Washington, DC: Pew Research Center's Internet and American Life Project; 2012. Report released June 6, 2012. Retrieved February 19, 2014, from http://www.pewinternet.org/2012/06/06/older-adults-and-internet-use/ [Google Scholar]
 - Zickuhr K, Smith A. Digital differences. Washington, DC: Pew Research Center's Internet and American Life Project; 2012. Report released April 13, 2012. Retrieved October 15, 2013, from http://pewinternet.org/Reports/2012/Digital-differences.aspx. [Google Scholar]
 
