Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2014 Oct 3.
Published in final edited form as: Clin Psychol Psychother. 2013 Aug 5;21(5):427–436. doi: 10.1002/cpp.1855

mHealth: A Mechanism to Deliver More Accessible, More Effective Mental Health Care

Matthew Price 1,*, Erica K Yuen 2, Elizabeth M Goetter 3,4, James D Herbert 3, Evan M Forman 3, Ron Acierno 5,6, Kenneth J Ruggiero 5,6
PMCID: PMC3926903  NIHMSID: NIHMS518462  PMID: 23918764

Abstract

The increased popularity and functionality of mobile devices has a number of implications for the delivery of mental health services. Effective use of mobile applications has the potential to (a) increase access to evidence-based care; (b) better inform consumers of care and more actively engage them in treatment; (c) increase the use of evidence-based practices; and (d) enhance care after formal treatment has concluded. The current paper presents an overview of the many potential uses of mobile applications as a means to facilitate ongoing care at various stages of treatment. Examples of current mobile applications in behavioural treatment and research are described, and the implications of such uses are discussed. Finally, we provide recommendations for methods to include mobile applications into current treatment and outline future directions for evaluation.

Keywords: mHealth, mobile phone, technology-based care, access to care

Technology solutions have provided a means to overcome many of the barriers associated with delivering mental health care. For example, Televideo services are an effective means of delivering evidence-based psychological and psychiatric treatment to patients that are unable to meet in person with a provider (Tuerk, Yoder, Ruggiero, Gros, & Acierno, 2010; Yuen, Goetter, Herbert, & Forman, 2012). Similarly, Web-assisted therapy is effective at addressing mood and anxiety disorders by providing easily accessible support outside of session (Cuijpers et al., 2009; Litz, Engel, Bryant, & Papa, 2007), and Web-based self-help approaches have steadily accumulated support (Amstadter, Broman-Fulks, Zinzow, Ruggiero, & Cercone, 2009; Andrews, Cuijpers, Craske, McEvoy, & Titov, 2010). Technology has been applied in session, with virtual reality exposure used to engage patients with feared stimuli that are not easily accessible, such as heights (Coelho, Waters, Hine, & Wallis, 2009; Emmelkamp et al., 2002), planes (Rothbaum et al., 2006), combat scenarios (Rizzo et al., 2011) and crowded public speaking venues (Anderson, Zimand, Hodges, & Rothbaum, 2005; Price & Anderson, 2011a).

mHealth (mobile health care), a rapidly growing area that relies heavily on mobile applications deployed to cell phones and handheld devices, represents a new frontier for delivering mental health treatment (Kazdin & Blase, 2011). This wave of care has been driven by the rapid proliferation of smartphones—mobile phones with increased computing power and the ability to transmit and receive data over wireless connections. Smartphone ownership has steadily risen such that the majority (52%) of mobile devices owned by American adults are smartphones (Rainie, 2012). Furthermore, 55% of American adults access the Internet by mobile phone (Rainie, 2012), and analysts predict that the mobile phone will be the primary tool for connecting to the Internet by 2020 (Anderson & Rainie, 2008). The ubiquity of personally owned devices suggests that it will be highly feasible to include them into mental health care.

The current paper presents an overview of potential solutions and enhancements that mHealth can bring to mental health care. First, we discuss potential ways in which to capitalize on mHealth technologies to increase patient engagement in treatment. Second, we discuss methods that mHealth applications can be used to facilitate the treatment process. Third, we discuss opportunities to develop post-intervention resources to improve long-term outcomes and maintain treatment gains. Fourth, we briefly review aspects of development of mHealth applications from the perspective of a mental health researcher or provider. Finally, we conclude with next steps to be addressed as mHealth continues to grow within mental health treatment. Such an overview is beneficial given the rapidly advancing pace of technology in health care (Heron & Smyth, 2010) and the need for psychologists to become familiar with such approaches to remain on the cutting edge of clinical practice (Prinstein, Fox, Guan, Arthur, & Gur, 2012).

APPS AS A MEANS FOR ENGAGEMENT

Mobile applications present a unique opportunity to prepare and engage patients in treatment. Those at high risk for mental health problems often have significant difficulty accessing quality mental health care (Burgess et al., 2009; Kreyenbuhl, Nossel, & Dixon, 2009), especially when symptoms first manifest (Trusz, Wagner, Russo, Love, & Zatzick, 2011). Furthermore, patients who enter treatment often have a limited understanding about the therapy process. They are most commonly referred from primary care after screen-based detection of mental health issues but are unsure about what mental health treatment entails (Verhaak, Van de Lisdonk, Bor, & Hutschemaekers, 2000). Poor preparation for care often translates to reduced expectations for successful treatment, which is associated with poorer overall response (Greenberg, Constantino, & Bruce, 2006; Price & Anderson, 2011b).

Mobile apps provide an excellent way to educate patients about the treatment process prior to enrolment. Education mechanisms integrated into mobile symptom tracking features can offer feedback about the best available practices for their specific concerns. For example, a patient with significant panic symptoms could be informed about the rationale for interoceptive exposure therapy and examples about what this entails shortly after experiencing a panic attack. They could then be directed to local providers who are able to administer that specific therapy. Such initial information could be especially relevant because patients who have searched online for mental health information then enter the market seeking a specific type of intervention.

Mobile devices also offer a method to overcome the structural barrier of distance. Many of the existing televideo technologies have sought to overcome challenges associated with accessing mental health care. Televideo can be used when the primary barrier is distance—the patient and provider are sufficiently far apart that in-person treatment is prohibitive. The televideo capabilities of smartphones, a feature that has become standard on recent devices, has the potential to further improve access for those without access to standard Internet in their homes by allowing patients to access assessments and additional care via their device.

Another structural barrier that often prevents patients from accessing care is that of reimbursement (Olfson et al., 2000). Many patients encounter difficulty locating a provider who will accept a specific form of coverage. Recently, electronic systems administered in acute care settings have helped patients navigate this barrier by identifying local providers who accept a given insurance plan (Boudreaux et al., 2009). Such a system was implemented via a dedicated computer system that was only available in the acute care setting, however. Adapting such systems to a mobile platform could prove a useful mechanism for connecting patients across clinics and helping those who are not prepared to identify a provider in a given environment. For example, patients reluctant to engage in mental health care after exposure to a traumatic event can be offered a mobile app that will help them identify a care provider when they are ready for treatment.

For others, attitudes towards treatment are the primary barrier, especially for multicultural populations (Corrigan, 2004). Recent work demonstrated that underserved groups who are less likely to access in-person services are just as likely to access and complete Web-based interventions, highlighting the potential for such approaches to have broad reach (Price, Davidson, Andrews, & Ruggiero, 2013). Preliminary interventions that are administered via mobile device may hold similar promise for these groups, given high rates of ownership of mobile devices among underserved groups (Smith, 2010). Mobile applications that are designed to address the stigma associated with receiving mental health care have the potential to further reduce health disparities and improve the likelihood of engagement among those who would not have previously sought treatment.

Finally, data that are obtained from the period prior to entering treatment can inform the direction of the treatment process. Measures of distress, functioning and perceptions about treatment could be made available to a provider at the start of the first session. The therapist could use this information to inform their assessment of the patient and expedite the development of a treatment plan.

APPS AS A MEANS TO FACILITATE THE TREATMENT PROCESS

Apps designed for mobile devices present an opportunity to extend the reach of the therapist beyond that of the standard session in ways that are far more complex than the original SMS ‘text message’ approaches wherein useful, encouraging or educational phrases were sent to patients (Aguilera & Muñoz, 2011; Luxton, McCann, Bush, Mishkind, & Reger, 2011; Prociow & Crowe, 2010). The majority of text message-based approaches relied on one-way communication—sending messages from the provider to the patient without obtaining feedback from patients. However, apps afford a plethora of additional options.

mHealth applications can be constructed to play central roles in evidence-based therapies. For example, an app can utilize the geolocation categorization of ‘risk’ areas for substance abusers followed by reminder ‘warnings’ when near these areas in which contact with a therapist or coach is then initiated (Vahabzadeh, Mezghanni, Lin, Epstein, & Preston, 2010). Recently, Rizvi et al. (2011) described an app called ‘DBT Field Coach’ that provided instructions, exercises, reminders and other components to help borderline patients cope with emotional crises (e.g., access to video/audio messages from the individual’s therapist, games designed to distract from intense emotions and motivational images). Results of their study suggested that the application was used when needed, reduced intense emotions, reduced substance use cravings and assisted in improving symptoms of depression and distress during the treatment period. Applications that allow patients to access essential components of skills learned in treatment independent of the therapist can be valuable to the recovery process.

Homework is another key area where mobile applications offer a opportunity in mental health care. Homework adherence closely correlates with overall treatment response, but increasing adherence can be challenging (Mausbach, Moore, Roesch, Cardenas, & Patterson, 2010). For example, many exposure-based treatments for anxiety involve completing homework that involves a range of activities such as engaging in a feared activity, listening to a recording of a traumatic event or going to a feared location while monitoring their symptoms (e.g., Foa, Hembree, & Rothbaum, 2007). To this end, an application for adults with severe depression, CBT MobilWork, prompts users to complete basic homework assignments and coaches them through the process in real time (Trudeau, 2010). It is not uncommon for patients to poorly monitor their symptoms during such assignments or to fail to provide accurate records to their therapist. Completing such activities via a mobile application offers multiple methods to promote adherence and the collection of real-time data through a prompt for assessment data and even provide helpful feedback when the patient engages in an activity or completes an assessment. For example, a patient who prematurely discontinues an exposure may receive a series of motivational messages or suggestions on how to complete the exercise.

Symptom monitoring during the treatment process can also be significantly enhanced via mobile applications. First, mobile devices are less cumbersome and conspicuous relative to paper forms. Participants report a preference for mobile devices in the completion of research studies because of the ability of the device to maintain privacy (Matthews, Doherty, Coyle, & Sharry, 2008). Using a phone in public is a behaviour that is more socially accepted than completing a paper form. Second, adherence to real-time self-monitoring may be enhanced when conducted via mobile phone or a similar handheld device. Many patients who complete paper self-monitoring do so retrospectively (Matthews, Doherty, Coyle, & Sharry, 2008), and this process is more prone to error (Shiffman et al., 1997; Shiffman, Stone, & Hufford, 2008). Conversely, mobile phone monitoring encourages patients to complete monitoring forms as activities are completed, as well as enable interventionists to see precisely when the ratings were taken. More advanced applications allow these data to be instantly transmitted to therapists, so therapists can see homework being completed as it happens. In some applications supporting guided in vivo exposure, therapists may even initiate a televideo call to help patients successfully complete exposure trials, thereby demonstrating an integration of real time assessment, treatment and outcome enhancement. Not surprisingly, evidence also suggests that these mobile methods are preferred, particularly among younger patients (Goldstein, Wilson, & VanDenKerkhof, 2007; Matthews et al., 2008; VanDenKerkhof, Goldstein, Blaine, & Rimmer, 2005).

Alternatively, mobile applications may allow patients and providers to complete treatment in fewer face-to-face sessions in that real-time interactions with therapists during crucial therapeutic events outside of a typical session yield greater and faster improvements. This has the potential to increase satisfaction and engagement, reduce barriers (e.g., scheduling, transportation, parking and waiting time) and reduce costs. A previous study used a similar strategy to treat panic disorder with handheld computers (Newman, Kenardy, Herman, & Taylor, 1997). The devices provided assistance with homework activities and did not communicate directly with the therapist. The findings suggested that those who received the adjunctive intervention obtained comparable symptom reduction in four to eight sessions as compared with those who did not have an adjunctive intervention in 12.

A hybrid or semi-automated use of technology to use applications to allow contextual therapist assistance in real time and use applications to enhance treatment gains without therapist involvement is now possible. Patients would meet with therapists on a semi-regular basis, between which the app would be used as a resource for homework adherence, behaviour tracking, skill development and education. In-person sessions would focus on reviewing progress, problem solving for challenging areas, skill development and updating goals. By contrast, automated app directed periods would help to sustain behavioural goals established in session. For example, patients undergoing behavioural activation for depression (Hopko, Lejuez, Ruggiero, & Eifert, 2003) would be presented with the rationale and basic skills for scheduling while meeting with their therapist in person. They would then continue to adhere to their schedule via the application, with periodic televideo meetings with the therapist to problem solve specific challenges that they were unable to address. In-person meetings could be used as milestone events to teach new, complex skills or refocus therapeutic goals. Such an approach could facilitate increased self-efficacy on behalf of the patient, as they are able to make meaningful changes with successively greater levels of independence.

The televideo technology on mobile devices deserves additional comment. In addition to serving as a therapeutic enhancement strategy for ‘in context’ events, this technology also allows patients who otherwise could not obtain treatment due to geographic or health barriers to receive remote therapy sessions (or adjunctive contact with therapists) while being physically located in a place of their choosing, saving the time and monetary cost of travelling to and from the therapist’s office. This would be especially beneficial for patients who are physically disabled, lack transportation, have busy schedules, live in non-metropolitan areas or areas where no expert in their needed area exists or who are unwilling to seek therapy in person due to anxiety or stigmatization concerns (e.g., active duty military personnel).

APPS TO SUSTAIN GAINS AFTER TREATMENT HAS ENDED

Another area where mobile applications have potential to help patients is after formal treatment ends. Many, but not all, patients maintain gains after treatment terminates, but most do not experience meaningful symptom improvement (Price, Anderson, Henrich, & Rothbaum, 2008). This can be challenging for patients who may not have achieved a satisfactory level of symptom reduction but must end treatment for other reasons, such as financial concerns (Westmacott & Hunsley, 2010). Mobile applications present a means to provide patients with continued access to interventions that they began with a therapist, which could allow for continued progress. For example, an individual with social anxiety can use an in vivo exposure application for coaching and encouragement on how to systematically approach feared situations using the methods learned in treatment. They could also use the application to help overcome anxiety-provoking situations that were previously not encountered. Research on individuals who were fearful of flying prior to 2001 suggested that those who drew upon skills they learned in therapy were able to reduce specific phobia symptoms after the September 11 attack (Kim et al., 2008). As such, mobile applications may provide a method to continue treatment gains after contact with a therapist has ended or provide a cost-efficient method to provide care to those who are unable to continue with traditional in-person treatment.

Such methodologies also offer opportunities to step patients down from intensive treatment to reduced care. The end of treatment is often marked by increased distress and uncertainty, and has the potential for relapse (Kupfer et al., 1992). An app that enables the patient to either continue treatment components, or to maintain limited connectivity with a care centre, can reduce the likelihood of relapse and promote the use of skills obtained in treatment. For example, Bauer et al. (2003) demonstrated that text messaging provided support to individuals with bulimia who were recently discharged from an inpatient programme for bulimic symptoms. Patients provided information about their symptoms for 6 months after treatment ended and received a mix of pre-programmed and individualized feedback. Results from a larger randomized controlled trial (RCT) suggested that the programme was both acceptable and effective at continuing to reduce bingeing and compensatory behaviours (Bauer, Okon, Meermann, & Kordy, 2012). Mobile devices have the unique potential to gradually end the therapeutic process after in-person treatment has formally ended, in a manner that is both helpful and cost efficient.

Prior work has demonstrated the utility of ‘booster sessions’ after completing treatment to maintain gains (McWhirter, McWhirter, & Bundy, 2011; Raue, Schulberg, Heo, Klimstra, & Bruce, 2009; Schlup, Munsch, Meyer, Margraf, & Wilhelm, 2009). Such sessions are designed to reinforce progress made during treatment, remind the patients of their progress during treatment and give the patient continued support. These sessions are often spaced further apart than regular sessions and may come at inopportune times. That is, booster sessions may not be scheduled when the patient needs them most. Mobile interventions are a method to provide ‘booster’ sessions in timely manner. Such boosters would coincide with the acute needs of a patient. For example, a depressed patient may access a thought record application after having a negative experience as a means to prevent a depressive episode.

DEVELOPMENT OF APPLICATIONS FOR MENTAL HEALTH PROVIDERS

Members of the mental health community may be interested in developing their own apps for use in treatment. Developing mobile apps requires consideration of a number of important factors that are somewhat distinct from other types of mental health treatments. The following discussion provides an overview of some of the issues that should be considered by those seeking to become involved in the development of mobile applications.

Technical Knowledge

The development of apps requires multidisciplinary collaboration with partners that have a background in software development. Such a team should include members capable of content design, software coding and usability testing of the product (e.g., alpha testing). Although it is possible for a mobile app to be developed by a single individual, it is unlikely that a mental health provider will be able to individually develop a comprehensive app independently, especially given the advanced technical and programming skills required. The mental health provider may find it easier to collaborate with qualified teams of software developers to facilitate the technical components. The provider would offer expertise in content to guide development and communicate user needs, whereas the programmers would complete the technical components. However, it would be helpful for the mental health provider to possess, or involve other mental health professionals who possess, basic training in software design to facilitate communication with technical staff.

Usability Testing

During application development, it is important to consider the concept of usability, which refers to the user experience of interacting with the application. Formal usability testing, which involves systematically recording and analyzing user interactions with the application in order to uncover common usability issues, is an essential component of any such evaluation of mHealth strategies. Usability issues include anything that prevents task completion, takes someone off course, causes frustration or creates confusion. Poor usability is a primary cause for failed adoption of health technologies. Prior work has shown that patients will not engage with technology that is challenging to use or perceived as irrelevant to their needs (Chiu & Eysenbach, 2010; Lenert et al., 2003). This has resulted in limited engagement and completion of other technology-based strategies for care delivery such as the Internet delivered interventions (Christensen, Griffiths, & Farrer, 2009; Donkin et al., 2011; Price, Gros, McCauley, Gros, & Ruggiero, 2012). Research has found that some of the main usability hurdles for mobile devices include small screens that display a limited amount of information at a time, touch activation screens that are error prone and small keyboards that result in slow text entry and typographical errors (Gong & Tarasewich, 2004).

The evaluation of usability in the development process should involve all relevant stakeholders. This includes incorporating the feedback from providers as well as patients. This work should include evaluating provider willingness towards using a given application in treatment as well as determining what features providers are interested in integrating into their care. Prior work examining provider perspectives in the use of mobile devices in care for chronic health conditions identified several areas of concern including an increased workload in managing data, restrictions on the increased contact that such approaches afford and a clarification of the malpractice liabilities associated with the use of a mobile devices (Seto et al., 2010). Similar mixed method (qualitative and quantitative) studies should be conducted to identify issues that may attenuate provider interest mHealth approaches.

App Maintenance

New mobile technologies and evidence-based mental health care continues to evolve. As such, app developers and mental health providers should identify methods to maintain, modify and improve existing apps to incorporate these advances in technology and psychological research. These methods should include strategies to fix technical problems that arise as hardware and software systems are updated. Developers are encouraged to create a plan to maintain the app after the initial development is completed.

Maintaining and updating an app can be challenging due to limitations in funding. Traditional research funding mechanisms, including multi-year research projects, are conducive to initial development, usability testing and preliminary evaluation but are limited in their ability to support continued revision. Fortunately, apps themselves are products and can be integrated into a self-sustaining business model. This approach would also promote the development of high quality apps that are more likely to be used in order to generate such financial support. However, the need for sustained funding leads to challenges in identifying ethical and manageable revenue streams. The most common approach is to charge for the initial download of the app itself. Additional models include offering an initial free download and then selling supplemental content for the app. For example, a mood monitoring app could offer a supplemental feature that communicates with an external heart rate monitor for an additional fee. Alternatively, arrangements with a specific institution (e.g., a university or hospital) or health insurance agencies could be made to allocate funding to successfully sustain an app. Such an agreement is likely to succeed when the app offers direct benefit to the larger organization, such as coordinating care amongst disparate service centres. Although no one strategy has emerged as consistently successful, there is a continued need to identify approaches that balance costs of development and continued maintenance with wide-scale dissemination of such software.

NEXT STEPS FOR mHEALTH IN MENTAL HEALTH CARE

As with the development of any new modality, mHealth faces a number of challenges as it moves towards more widespread and seamless adoption by patients and providers. Current challenges include security and privacy, establishing an evidence base for the use of different apps, sustainability of apps over time and remaining current with an ever-changing market.

Security and Privacy Issues

The increased use of mobile devices in mental health care also brings with it new challenges around the security of the clinical data collected via these devices. Patients should be made aware of any explicit security and privacy risks associated with inputting potentially confidential information into a mobile device. Although the majority of security issues associated with using such software for mental health is similar to those of using an app for other processes, they should still be reviewed with patients at the start of treatment.

The most common security issue that occurs with mobile devices is loss of the device due to theft or the misplacement on behalf of the owner. Several phone-specific security measures can be implemented to minimize the risks associated with this scenario. Typically, devices can be secured with a personal password, which affords some protection of the patient’s privacy. Furthermore, device manufacturers have allowed owners to locate their service in the event it is stolen and to remotely remove all of the data from the device. For additional protection, developers should consider adding a feature requiring users to input a personal password to access the information stored within the app.

More sophisticated security risks have also been documented, such as taking information from the device without the patient’s knowledge. Such threats include keystroke loggers, phishing messages and Trojan software (DHS, 2012). A comprehensive discussion of these methods is beyond the scope of this paper, but the quantity of risks highlights the need for comprehensive protection efforts to ensure that mobile devices are used safely in treatment. It should be noted that security risks are likely to continue to evolve and will present a continuous issue in treatment. It is unlikely that a therapist or patient will have comprehensive knowledge to navigate all types of security risks associated with devices. Instead, it is recommended that clinicians obtain a base level of knowledge of the security risks associated with using these devices and work with clients to mitigate such risks. Such practices are common in other healthcare settings in which employees are warned of malicious emails, suspicious websites and other common methods of breaching security. When a mobile device is introduced into treatment, clinicians should have open discussions with the patient about the security risks and how to take proper precautions. Such conversations are likely to facilitate overall rapport, particularly for patient populations that have expressed concerns over confidentiality of information when using mobile devices (George, Hamilton, & Baker, 2009; Price, Williamson et al., 2013). These discussions should include developing a plan to implement basic security measures such as activating a remote tracking feature or implementing a lock screen password on a patient’s mobile device. Such a joint decision between patient and therapist is likely to facilitate engagement in therapy, improve use of the device and improve overall outcomes.

Establishment of an Evidence Base for Mobile Apps

There have been few published research studies examining the efficacy of mHealth applications for the treatment of mental health problems. Recent reviews of studies that have used handheld computing devices such as personal digital assistants (PDA) and text messages hold promise to facilitate behaviour change in physical and mental health conditions (Ehrenreich, Righter, Rocke, Dixon, & Himelhoch, 2011; Heron & Smyth, 2010). These authors highlight the rapidly evolving pace of technology and the need for continuous evaluation and discussion of new methods that become available. Indeed, all of the reviewed studies were conducted prior to 2007 when rates of mobile phone ownership amongst adults and adolescents were lower and smartphones were primarily used for business purposes. As such, the lack of current mHealth research is to be expected given the nascency of the field, and much of the published literature on mHealth approaches for mental health has been primarily theoretical.

The growing number of mental health applications and their ease of access highlight the need for formal evaluation (Fox, 2010). There are a considerable number of mental-health-related applications that are available on the Google Play Store and iTunes App store, the platforms for the two dominant mobile platforms, as well as the new Veterans Affairs Application Center (see the Presidentially recognized ‘PTSD COACH’ application for Apple and Android) (Sloan, Marx, & Keane, 2011). Just as untested, unregulated health supplements represent potential negative outcomes for physical health care (e.g., using an untested supplement in lieu of effective treatment), the large number of mental health applications available in the marketplace that claim to provide benefit is also potentially problematic. It is unknown what proportion of these applications actually utilizes evidence-based principles and techniques. For example, an application may claim to use cognitive–behavioural techniques in its description, but it is not guaranteed that this claim is true or that these techniques are implemented effectively. Similarly, an application may provide psychoeducation about a certain disorder, but there is no guarantee that the information is accurate and based on the latest scientific findings. Furthermore, there is a dearth of research investigating the efficacy of specific applications, and many of the applications that do have preliminary research support for their efficacy are not yet available for use by the general public.

Apps that are ineffective or offer detrimental experiences can have significant costs to patient care. Such apps may deter individuals from seeking further treatment. A patient who has an unsatisfactory experience with an ineffective app may be less likely to seek further treatment that stands to alleviate their symptoms. Alternatively, a significantly impaired individual may not seek additional care because they believe that using an ineffective app is sufficient treatment for their condition. Within the context of treatment, apps may potentially disrupt the therapeutic alliance between patient and provider. Specific individuals may not want to use apps in care or have minimal benefit from their inclusion in treatment. Thus, the inclusion of this additional component of treatment may be viewed as burdensome or contribute to a lack of efficacy in treatment. To address these issues, clinicians are encouraged to talk with their patients about the applications that they are using and to evaluate their utility on a case-by-case basis.

These concerns highlight the need for research on the acceptability and effectiveness of applications for use as primary, secondary and tertiary treatment options. However, there are significant challenges associated with the evaluation of mobile applications. The speed with which applications can be developed rapidly outpaces the evaluation process. Application are often developed and disseminated via a download service within months. By comparison, the completion and dissemination of the findings of an RCT takes approximately 5 years. In order to rapidly build such an evidence base, experts have advocated that researchers should use innovative designs that will produce results in a more timely fashion than the traditional RCT (NIH, 2011). Such strategies, including Sequential Multiple Assignment Randomized Trials (Murphy, 2005) and Multiphase Optimization Strategy (Collins et al., 2011), can allow applications to be submitted to the same rigorous evaluation that is consistent with other treatment approaches in a timely manner.

Recent work has suggested that a strong evidence base is being established for mHealth methods (Labrique, Vasudevan, Chang, & Mehl, 2012). As the evidence base grows, there is a need for tools to facilitate the targeted dissemination of apps to patients and providers. Although apps have built in dissemination mechanisms via an app store, it is unlikely that patients and providers will be able to discern high-quality and evidence-based apps from those without support. Such dissemination systems should rely on a common set of standards for different levels of support (e.g., RCT studies, case–control studies and pilot studies), different theoretical orientations (behavioural, cognitive and systemic) and potentially different user preferences (e.g., frequency of use). It is unclear if a central rating system is ideal, such as the Cochrane Review for health care, or if individualized standards, such as approval by specific organizations that subscribe to a particular modality, is preferred. Such a system should leverage the various app stores by including ratings within the descriptions, advertise through social media, promote their use through professional organizations and publish their research in high-quality journals.

Mobile Device Trends and Mental Health Care

As described, mobile devices have numerous functions that are of direct utility to mental health treatment. The present discussion has highlighted the ability of such devices to asynchronously communicate with patients in specific contexts at all stages of the treatment process. Such functions rely on the expanded communication functionality of the mobile device (e.g., messaging, telephone conversations and video chat). Additionally, such devices offer increased functionality through integrated hardware (e.g., global positioning systems and camera) and supplemental hardware that connects to the device directly or wirelessly through Bluetooth.

Mobile devices contain a number of embedded sensors that can be used to improve the timeliness of a given intervention. These sensors include global positioning systems capable of determining locations as well as movement in a given area, accelerometers capable of measuring movement of the device, microphones capable of recording sounds in the area, camera capable of video and pictures and Bluetooth sensors capable of identifying and communicating with nearby devices. These sensors can be used in conjunction with one another to obtain a comprehensive understanding of an individual’s behaviour at a given time and have the potential to predict subsequent behaviour (Burns et al., 2011).

The future of mobile devices will involve advances to functionality within the device such as improving the breadth and precision of sensor capabilities, as well as enhancing interactivity with other devices. This interoper-ability includes seamless communication with other mobile devices (e.g., Bluetooth and near field communication), software systems (e.g., electronic medical records) and other hardware (e.g., actigraph). These features are likely to improve opportunities for passive data collection, which requires minimal input from the user and reduces the overall burden of usage. The ability of the mobile device to communicate with other devices and systems around an individual represents a potential shift in the functionality of the device from that of an additional tool to be used in treatment to the central device that manages the process of treatment outside of care. This feature will be increasingly important as high-powered wearable sensors become more prominent.

CONCLUSIONS

Mobile applications that focus on mental health treatment can be used for a variety of purposes. They show great promise in promoting healthy behaviour changes, increasing adherence to treatment programmes, providing immediate psychological support, facilitating self-monitoring and reducing the demand for clinician time (Spurgeon & Wright, 2010). As mobile applications grow in popularity among the general public, so does the potential to increase the quality of care and access to evidence-based treatments through this technology.

Mental health providers and researchers are encouraged to thoughtfully develop and evaluate mobile applications for use in clinical practice and research. As the mental health field continues to adopt new technologies to increase access and quality of care, it is imperative to investigate their effectiveness, feasibility, usability and acceptance in an effort to reduce the burden of mental illness, increase access to high-quality evidence-based treatments, bridge the gap between science and clinical practice and ensure that the mental health field remains at the cutting edge of new developments in health care.

Key Practitioner Message.

  • Mobile devices are becoming increasingly common among the adult population and have tremendous potential to advance clinical care.

  • Mobile applications have the potential to enhance clinical care at stages of treatment—from engaging patients in clinical care to facilitating adherence to practices and in maintaining treatment gains.

  • Research is needed to validate the efficacy and effectiveness of mobile applications in clinical practice.

  • Research on such devices must incorporate assessments of usability and adherence in addition to their incremental benefit to treatment.

Acknowledgments

Dr Price is supported by NIMH Grant T32 MH018869. Drs Acierno and Yuen are funded by W81XWH-11-1-0073 (PI: Acierno). Dr Ruggiero is supported by NIMH Grants R01 MH81056 and R34 MH096907 (PI: Ruggiero), and Drs Ruggiero and Acierno are supported by VA HSR&D Grant MHI 08-105-2 (PI: Ruggiero). Drs Forman and Herbert are supported by NIDDK Grant R21 DK08043 (PI: Forman). Views expressed in this article do not necessarily reflect those of the funding agencies.

References

  1. Aguilera A, Muñoz RF. Text messaging as an adjunct to CBT in low-income populations: A usability and feasibility pilot study. Professional Psychology: Research and Practice. 2011;42(6):472. doi: 10.1037/a0025499. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Amstadter AB, Broman-Fulks J, Zinzow H, Ruggiero KJ, Cercone J. Internet based interventions for traumatic stress-related mental health problems: A review and suggestion for future research. Clinical Psychology Review. 2009;29(5):410–420. doi: 10.1016/j.cpr.2009.04.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Anderson JQ, Rainie L. The future of the internet III. Pew Internet & American Life Project; 2008. Retrieved from http://www.pewinternet.org/~/media//Files/Reports/2008/PIP_FutureInternet3.pdf.pdf. [Google Scholar]
  4. Anderson PL, Zimand E, Hodges LF, Rothbaum BO. Cognitive behavioral therapy for public-speaking anxiety using virtual reality for exposure. Depression and Anxiety. 2005;22(3):156–158. doi: 10.1002/da.20090. [DOI] [PubMed] [Google Scholar]
  5. Andrews G, Cuijpers P, Craske MG, McEvoy P, Titov N. Computer therapy for the anxiety and depressive disorders is effective, acceptable and practical health care: A meta-analysis. PloS One. 2010;5(10):e13196. doi: 10.1371/journal.pone.0013196. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Bauer S, Percevic R, Okon E, Meermann R, Kordy H. Use of text messaging in the aftercare of patients with bulimia nervosa. European Eating Disorders Review. 2003;11:279–290. [Google Scholar]
  7. Bauer S, Okon E, Meermann R, Kordy H. Technology-enhanced maintenance of treatment gains in eating disorders: Efficacy of an intervention delivered via text messaging. Journal of Consulting and Clinical Psychology. 2012;80(4):700–706. doi: 10.1037/a0028030. [DOI] [PubMed] [Google Scholar]
  8. Boudreaux ED, Bedek KL, Gilles D, Baumann BM, Hollenberg S, Lord SA, Grissom G. The Dynamic Assessment and Referral System for Substance Abuse (DARSSA): Development, functionality, and end-user satisfaction. Drug and Alcohol Dependence. 2009;99:37–46. doi: 10.1016/j.drugalcdep.2008.06.015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Burgess PM, Pirkis JE, Slade TN, Johnston AK, Meadows GN, Gunn JM. Service use for mental health problems: Findings from the 2007 National Survey of Mental Health and Wellbeing. Australasian Psychiatry. 2009;43:615–623. doi: 10.1080/00048670902970858. [DOI] [PubMed] [Google Scholar]
  10. Burns MN, Begale M, Duffecy J, Gergle D, Karr CJ, Giangrande E, Mohr DC. Harnessing context sensing to develop a mobile intervention for depression. Journal of Medical Internet Research. 2011;13(3):e55. doi: 10.2196/jmir.1838. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Chiu T, Eysenbach G. Stages of use: Consideration, initiation, utilization, and outcomes of an internet-mediated intervention. BMC Medical Informatics and Decision Making. 2010;10(1):73–84. doi: 10.1186/1472-6947-10-73. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Christensen H, Griffiths KM, Farrer L. Adherence in Internet interventions for anxiety and depression: Systematic review. Journal of Medical Internet Research. 2009;11(2):e13. doi: 10.2196/jmir.1194. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Coelho CM, Waters AM, Hine TJ, Wallis G. The use of virtual reality in acrophobia research and treatment. Journal of Anxiety Disorders. 2009;23:563–574. doi: 10.1016/j.janxdis.2009.01.014. [DOI] [PubMed] [Google Scholar]
  14. Collins LM, Baker TB, Mermelstein RJ, Piper ME, Jorenby DE, Smith SS, Christiansen BA, Schlam TR, Cook JW, Fiore MC. The Multiphase Optimization Strategy for Engineering Effective Tobacco Use Interventions. Annals of behavioral medicine. 2011;41(2):208–226. doi: 10.1007/s12160-010-9253-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Corrigan P. How stigma interferes with mental health care. The American Psychologist. 2004;59(7):614–625. doi: 10.1037/0003-066X.59.7.614. [DOI] [PubMed] [Google Scholar]
  16. Cuijpers P, Marks IM, Van Straten A, Cavanagh K, Gega L, Andersson G. Computer-aided psychotherapy for anxiety disorders: A meta-analytic review. Cognitive Behavior Therapy. 2009;38(2):66–82. doi: 10.1080/16506070802694776. [DOI] [PubMed] [Google Scholar]
  17. DHS. Attack surface: Healthcare and public health sector (Bulletin No. 201205040900) National cybersecurity and communications integration center. 2012:11. Retrieved from http://info.publicintelligence.net/NCCIC-MedicalDevices.pdf.
  18. Donkin L, Christensen H, Naismith LS, Neal B, Hickie BI, Glozier N. A systematic review of the impact of adherence on the effectiveness of e-therapies. Journal of Medical Internet Research. 2011;13(3):e52. doi: 10.2196/jmir.1772. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Ehrenreich B, Righter B, Rocke DA, Dixon L, Himelhoch S. Are mobile phones and handheld computers being used to enhance delivery of psychiatric treatment? The Journal of Nervous and Mental Disease. 2011;199(11):886–891. doi: 10.1097/NMD.0b013e3182349e90. [DOI] [PubMed] [Google Scholar]
  20. Emmelkamp PM, Krijn M, Hulsbosch A, De Vries S, Schuemie M, Van der Mast CAP. Virtual reality treatment versus exposure in vivo: A comparative evaluation in acrophobia. Behaviour Research and Therapy. 2002;40(5):509–516. doi: 10.1016/S0005-7967(01)00023-7. [DOI] [PubMed] [Google Scholar]
  21. Foa E, Hembree E, Rothbaum BO. Prolonged exposure therapy for PTSD: Emotional processing of traumatic experiences therapist guide. 1. USA: Oxford University Press; 2007. [Google Scholar]
  22. Fox S. Mobile health 2010. Pew Internet & American Life Project; 2010. Retrieved from http://www.pewinternet.org/Reports/2010/Mobile-Health-2010/Report.aspx. [Google Scholar]
  23. George SM, Hamilton A, Baker R. Pre-experience perceptions about telemedicine among African Americans and Latinos in South Central Los Angeles. Telemedicine and e-Health. 2009;15(6):525–530. doi: 10.1089/tmj.2008.0152. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Goldstein DH, Wilson R, VanDenKerkhof EG. Electronic monitoring in an acute pain management service. Pain Medicine. 2007;8:S94–S100. [Google Scholar]
  25. Gong J, Tarasewich P. Guidelines for handheld mobile device interface design. Presented at the DSI Annual Meeting; Boston, MA: CiteSeerX; 2004. Retrieved from http://citeseerx.ist.psu.edu/viewdoc/summary? [Google Scholar]
  26. Greenberg RP, Constantino MJ, Bruce N. Are patient expectations still relevant for psychotherapy process and outcome? Clinical Psychology Review. 2006;26:657–678. doi: 10.1016/j.cpr.2005.03.002. [DOI] [PubMed] [Google Scholar]
  27. Heron KE, Smyth JM. Ecological momentary interventions: Incorporating mobile technology into psychosocial and health behaviour treatments. British Journal of Health Psychology. 2010;15(1):1–39. doi: 10.1348/135910709X466063. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Hopko DR, Lejuez CW, Ruggiero KJ, Eifert GH. Contemporary behavioral activation treatments for depression: Procedures, principles, and progress. Clinical Psychology Review. 2003;23(5):699–717. doi: 10.1016/S0272-7358(03)00070-9. [DOI] [PubMed] [Google Scholar]
  29. Kazdin AE, Blase SL. Rebooting psychotherapy research and practice to reduce the burden of mental illness. Perspectives on Psychological Science. 2011;6:21–37. doi: 10.1177/1745691610393527. [DOI] [PubMed] [Google Scholar]
  30. Kim S, Palin F, Anderson P, Edwards S, Lindner G, Rothbaum BO. Use of skills learned in CBT for fear of flying: Managing flying anxiety after September 11th. Journal of Anxiety Disorders. 2008;22(2):301–309. doi: 10.1016/j.janxdis.2007.02.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Kreyenbuhl J, Nossel IR, Dixon LB. Disengagement from mental health treatment among individuals with schizophrenia and strategies for facilitating connections to care: A review of the literature. Schizophrenia Bulletin. 2009;35(4):696–703. doi: 10.1093/schbul/sbp046. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Kupfer DJ, Frank E, Perel JM, Cornes C, Mallinger AG, Thase ME, McEachran AB, Grochocinski VJ. Five-year outcome for maintenance therapies in recurrent depression. Archives of General Psychiatry. 1992;49:769–773. doi: 10.1001/archpsyc.1992.01820100013002. [DOI] [PubMed] [Google Scholar]
  33. Labrique A, Vasudevan L, Chang LW, Mehl G. H_pe for mHealth: More “y” or “o” on the horizon? International Journal of Medical Informatics. 2012 doi: 10.1016/j.ijmedinf.2012.11.016. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Lenert L, Muñoz RF, Stoddard J, Delucchi K, Bansod A, Skoczen S, Pérez-Stable EJ. Design and pilot evaluation of an internet smoking cessation program. Journal of the American Medics Informatics Association. 2003;10:16–20. doi: 10.1197/jamia.M1128. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Litz BT, Engel CC, Bryant RA, Papa A. A randomized, controlled proof-of-concept trial of an Internet-based, therapist-assisted self-management treatment for posttraumatic stress disorder. The American Journal of Psychiatry. 2007;164(11):1676–1684. doi: 10.1176/appi.ajp.2007.06122057. [DOI] [PubMed] [Google Scholar]
  36. Luxton DD, McCann RA, Bush NE, Mishkind MC, Reger GM. mHealth for mental health: Integrating smartphone technology in behavioral healthcare. Professional Psychology: Research and Practice. 2011;42:505. [Google Scholar]
  37. Matthews M, Doherty G, Coyle D, Sharry J. Designing mobile applications to support mental health interventions. In: Lumsden J, editor. Handbook of research on user interface design and evaluation for mobile technology, IGI. Information Science Reference; 2008. pp. 635–656. [Google Scholar]
  38. Mausbach BT, Moore R, Roesch S, Cardenas V, Patterson TL. The relationship between homework compliance and therapy outcomes: An updated meta-analysis. Cognitive Therapy and Research. 2010;34:429–438. doi: 10.1007/s10608-010-9297-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. McWhirter PT, McWhirter JJ, Bundy A. Anger and violence prevention: Enhancing treatment effects through booster sessions. Education and Treatment of Children. 2011;34(1):1–14. [Google Scholar]
  40. Murphy SA. An experimental design for the development of adaptive treatment strategies. Statistics in Medicine. 2005;24(10):1455–1481. doi: 10.1002/sim.2022. [DOI] [PubMed] [Google Scholar]
  41. Newman MG, Kenardy J, Herman S, Taylor CB. Comparison of palmtop-computer-assisted brief cognitive-behavioral treatment to cognitive-behavioral treatment for panic disorder. Journal of Consulting and Clinical Psychology. 1997;65(1):178–183. doi: 10.1037//0022-006x.65.1.178. [DOI] [PubMed] [Google Scholar]
  42. NIH. mHealth evidence workshop. 2011 Retrieved from http://videocast.nih.gov/summary.asp?file=16799.
  43. Olfson M, Guardino M, Struening E, Schneier FR, Hellman F, Klein DF. Barriers to the treatment of social anxiety. The American Journal of Psychiatry. 2000;157:521–527. doi: 10.1176/appi.ajp.157.4.521. [DOI] [PubMed] [Google Scholar]
  44. Price M, Anderson PL. The impact of cognitive behavioral therapy on post event processing among those with social anxiety disorder. Behaviour Research and Therapy. 2011a;49(2):132–137. doi: 10.1016/j.brat.2010.11.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Price M, Anderson PL. Outcome expectancy as a predictor of treatment response in cognitive behavioral therapy for public speaking fears within social anxiety disorder. Psychotherapy. 2011b;49(2):173–179. doi: 10.1037/a0024734. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Price M, Anderson P, Henrich CC, Rothbaum BO. Greater expectations: Using hierarchical linear modeling to examine expectancy for treatment outcome as a predictor of treatment response. Behavior Therapy. 2008;39(4):398–405. doi: 10.1016/j.beth.2007.12.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Price M, Gros DF, McCauley JL, Gros KS, Ruggiero KJ. Nonuse and dropout attrition for a web-based mental health intervention delivered in a post-disaster context. Psychiatry. 2012;75(3):267–284. doi: 10.1521/psyc.2012.75.3.267. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Price M, Davidson T, Andrews J, Ruggiero KJ. Access, use and completion of a brief disaster mental health intervention among Hispanics, African-Americans and Whites affected by Hurricane Ike. Journal of Telemedicine and Telecare. 2013:1–5. doi: 10.1177/1357633X13476230. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Price M, Williamson D, McCandless R, Mueller M, Gregoski MJ, Brunner-Jackson BM, Treiber E, Davidson L, Treiber FA. Attitudes of Hispanic migrant farm workers towards mobile phone based telehealth to manage chronic health conditions. Journal of Medical Internet Research. 2013 doi: 10.2196/jmir.2500. [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Prinstein MJ, Fox KR, Guan K, Arthur MS, Gur S. Psychologists’ and trainees’ perceptions of the future of clinical psychology. The Behavior Therapist. 2012;6(35):101–106. [Google Scholar]
  51. Prociow PA, Crowe JA. Towards personalised ambient monitoring of mental health via mobile technologies. Technology and Health Care. 2010;18:275–284. doi: 10.3233/THC-2010-0590. [DOI] [PubMed] [Google Scholar]
  52. Rainie L. Smartphone ownership update: September 2012. Pew Internet & American Life Project. 2012 Retrieved October 30, 2012, from http://pewinternet.org/Reports/2012/Smartphone-Update-Sept-2012.aspx.
  53. Raue PJ, Schulberg HC, Heo M, Klimstra S, Bruce ML. Patients’ depression treatment preferences and initiation, adherence, and outcome: A randomized primary care study. Psychiatric Services. 2009;60(3):337–343. doi: 10.1176/appi.ps.60.3.337. [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. Rizvi SL, Dimeff LA, Skutch J, Carroll D, Linehan MM. A pilot study of the DBT coach: An interactive mobile phone application for individuals with borderline personality disorder and substance use disorder. Behavior Therapy. 2011;42:589–600. doi: 10.1016/j.beth.2011.01.003. [DOI] [PubMed] [Google Scholar]
  55. Rizzo A, Parsons TD, Lange B, Kenny P, Buckwalter JG, Rothbaum B, Difede JA, Frazier J, Newman B, Williams J. Virtual reality goes to war: A brief review of the future of military behavioral healthcare. Journal of Clinical Psychology in Medical Settings. 2011;18(2):1–12. doi: 10.1007/s10880-011-9247-2. [DOI] [PubMed] [Google Scholar]
  56. Rothbaum BO, Anderson P, Zimand E, Hodges L, Lang D, Wilson J. Virtual reality exposure therapy and standard (in vivo) exposure therapy in the treatment of fear of flying. Behavior Therapy. 2006;37:80–90. doi: 10.1016/j.beth.2005.04.004. [DOI] [PubMed] [Google Scholar]
  57. Schlup B, Munsch S, Meyer AH, Margraf J, Wilhelm FH. The efficacy of a short version of a cognitive-behavioral treatment followed by booster sessions for binge eating disorder. Behaviour Research and Therapy. 2009;47(7):628–635. doi: 10.1016/j.brat.2009.04.003. [DOI] [PubMed] [Google Scholar]
  58. Seto E, Leonard KJ, Masino C, Cafazzo JA, Barnsley J, Ross HJ. Attitudes of heart failure patients and health care providers towards mobile phone-based remote monitoring. Journal of Medical Internet Research. 2010;12(4):e55. doi: 10.2196/jmir.1627. [DOI] [PMC free article] [PubMed] [Google Scholar]
  59. Shiffman S, Hufford M, Hickcox M, Paty JA, Gnys M, Kassel JD. Remember that? A comparison of real-time versus retrospective recall of smoking lapses. Journal of Consulting and Clinical Psychology. 1997;65:292. doi: 10.1037/0022-006x.65.2.292.a. [DOI] [PubMed] [Google Scholar]
  60. Shiffman S, Stone AA, Hufford MR. Ecological momentary assessment. Annual Review of Clinical Psychology. 2008;4:1–32. doi: 10.1146/annurev.clinpsy.3.022806.091415. [DOI] [PubMed] [Google Scholar]
  61. Sloan DM, Marx BP, Keane TM. Reducing the burden of mental illness in military veterans commentary on Kazdin and Blase (2011) Perspectives on Psychological Science. 2011;6(5):503–506. doi: 10.1177/1745691611416995. [DOI] [PubMed] [Google Scholar]
  62. Smith A. Mobile access 2010. Pew Internet & American Life Project, July. 2010;7:2010. Retrieved March 22, 2012, from http://www.pewinternet.org/Reports/2010/Mobile-Access-2010.aspx. [Google Scholar]
  63. Spurgeon JA, Wright JH. Computer-assisted cognitive-behavioral therapy. Current Psychiatry Reports. 2010;12:547–552. doi: 10.1007/s11920-010-0152-4. [DOI] [PubMed] [Google Scholar]
  64. Trudeau M. Mental health apps: Like a “therapist in your pocket”. National Public Radio; 2010. Retrieved from http://www.npr.org/templates/story/story.php?storyId=127081326. [Google Scholar]
  65. Trusz SG, Wagner AW, Russo J, Love J, Zatzick DF. Assessing barriers to care and readiness for cognitive behavioral therapy in early acute care PTSD interventions. Psychiatry: Interpersonal and Biological Processes. 2011;74(3):207–223. doi: 10.1521/psyc.2011.74.3.207. [DOI] [PubMed] [Google Scholar]
  66. Tuerk PW, Yoder M, Ruggiero KJ, Gros DF, Acierno R. A pilot study of prolonged exposure therapy for posttraumatic stress disorder delivered via telehealth technology. Journal of Traumatic Stress. 2010;23:116–123. doi: 10.1002/jts.20494. [DOI] [PMC free article] [PubMed] [Google Scholar]
  67. Vahabzadeh M, Mezghanni M, Lin JL, Epstein DH, Preston KL. PGIS: Electronic diary data integration with GPS data initial application in substance-abuse patients. Presented at the Computer-Based Medical Systems; IEEE. 2010. pp. 474–479. [Google Scholar]
  68. VanDenKerkhof EG, Goldstein DH, Blaine WC, Rimmer MJ. A comparison of paper with electronic patient-completed questionnaires in a preoperative clinic. Anesthesia and Analgesia. 2005;101:1075. doi: 10.1213/01.ane.0000168449.32159.7b. [DOI] [PubMed] [Google Scholar]
  69. Verhaak P, Van de Lisdonk E, Bor J, Hutschemaekers G. GPs’ referral to mental health care during the past 25 years. The British Journal of General Practice. 2000;50:307–308. [PMC free article] [PubMed] [Google Scholar]
  70. Westmacott R, Hunsley J. Reasons for terminating psychotherapy: A general population study. Journal of Clinical Psychology. 2010;66:965–977. doi: 10.1002/jclp.20702. [DOI] [PubMed] [Google Scholar]
  71. Yuen EK, Goetter EM, Herbert JD, Forman EM. Challenges and opportunities in internet-mediated telemental health. Professional Psychology: Research and Practice. 2012;43(1):1–8. doi: 10.1037/a0025524. [DOI] [Google Scholar]

RESOURCES