Abstract
Purpose of Review
Interest in digital mental health, especially smartphone apps, has expanded in light of limited access to mental health services and the need for remote care during COVID-19. Digital clinics, in which apps are blended into routine care, offer a potential solution to common implementation challenges including low user engagement and lack of clinical integration of apps.
Recent Findings
While the number of mental health apps available in commercial marketplaces continues to rise, there are few examples of successful implementation of these apps into care settings. We review one example of a digital clinic created within an academic medical center and another within the Department of Veterans Affairs. We then discuss how implementation science can inform new efforts to effectively integrate mental health technologies across diverse use cases.
Summary
Integrating mental health apps into care settings is feasible but requires careful attention to multiple domains that will influence implementation success, including characteristics of the innovation (e.g., utility and complexity of the app), the recipients of the technology (e.g., patients and clinicians), and context (e.g., healthcare system buy-in, reimbursement, and regulatory policies). Examples of effective facilitation strategies that can be utilized to improve implementation efforts include co-production of technology involving all end users, specialized trainings for staff and patients, creation of new team members to aid in app usage (e.g., digital navigators), and re-design of clinical workflows.
Keywords: Apps, Smartphone, Clinics, Technology, mHealth
Introduction
In-person care restrictions due to the COVID-19 pandemic have led to an increased interest in and utilization of digital mental health tools. These tools, such as smartphone applications (apps), can aid clinicians in providing high-quality, continuous mental health care from a distance. However, the process of integrating these digital tools into care is not as simple as merely downloading an app. In this article, we will review the theory, motivations, real-world implementation, and challenges of digitally integrated mental health care. Rather than offering a cross-sectional perspective on this evolving topic, this article will instead highlight two recent examples of app implementation and provide actionable next steps needed to establish digital tools as a core component of mental health care.
The theory for a digital clinic is grounded in measurement-based care, shared decision making, and personalized medicine. While there is no standard definition of a digital clinic, in this paper, we will use the term to refer to a clinic that continues to offer synchronous mental health visits (including in-person, by video, or by phone) but augments and extends these visits via digital health tools such as smartphone apps. This model of practice can also be referred to as hybrid care given that it represents a blended approach towards the use of technology in the care setting. This model can support evidence-based care, as apps can be used to capture longitudinal patient data between visits, ranging from simple surveys about symptoms of depression to smartwatch-based estimates of daily sleep duration [1–3]. These new data streams can help support patient recovery and shared decision making. They can inform discussions between clinicians and patients and ensure that clinical decisions are made based on the most relevant and accurate information regarding patients’ current level of functioning. Finally, the digital clinic can support personalized medicine. Mental health apps allow for precise monitoring of individual patients’ symptoms and behaviors [4–6]. As such, app-based interventions can be customized to best match a patient’s unique characteristics and needs at various stages of their treatment journey.
Motivations
Previous barriers to implementing digital clinics have lessened during COVID-19, creating a more hospitable environment in which to establish this new model of care. Significant changes have occurred in the domains of digital health regulation, payment models, and patient and clinician experience with and acceptance of digital tools [7]. Prior to COVID-19, various regulations around the world limited the scope of practice of digital health. In the USA, regulations limiting prescription of controlled substances and the practice of synchronous telehealth across state lines were relaxed, while in countries such as India, national digital health policies were formed for the first time [8]. Changes in regulation around payment parity for telehealth services were also announced in many countries, making digital care more financially sustainable [7]. While these regulatory and payment changes are currently temporary, they will likely endure in some fashion. For example, temporary measures around FDA approval of digital mental health software announced during the height of COVID-19 in April 2020 are being proposed as permanent changes as of January 2021 [9]. Technological approaches to mental health care have been labeled as “likely… crucial” by the director of the National Institute of Mental Health, suggesting considerable interest in research and innovation from the US government [10]. Similarly, the German government passed the Digital Health Act in October 2020, allowing providers to prescribe an insured app for anxiety for the first time [11].
Equally important, COVID-19 has forced both patients and clinicians to overcome prior resistance to engaging in mental health care from a distance. This is particularly true with regard to synchronous telehealth services provided via video or phone. Survey data collected from 3700 psychiatry outpatients in a national US sample found 80% reported excellent or good experiences with receiving care via video or telephone during COVID-19, and 64% noted they would consider continued use after the pandemic ends [12]. Similarly, mental health clinicians have reported generally high satisfaction providing care via telehealth, with many expressing the sentiment that it was better than they had initially expected [13, 14, 15, 16]. This increase in openness towards and acceptance of remote mental health care bodes well for the integration of apps and other digital tools into clinical practice.
Finally, technology has advanced to such a state that patients and clinicians have access to a wealth of digital health resources, with 81% of patients today already owning a smartphone [17]. In addition, a variety of evidence-based mental health apps are available at low to no cost to patients. For example, the US Department of Veterans Affairs (VA) offers a wide range of mental health–related apps [18••] that are free for public use and have broad applicability to civilian populations as well as Veterans. Even the apps that are native to any smartphone such as Apple Health and Google Fit offer a wealth of clinically actionable data that may not require the patient to download any software. For example, the step count feature in most smartphones can support basic interventions to help patients be less sedentary [19], and the calendar and clock apps can provide reminders regarding times to take medication, exercise, or go to sleep.
Implementation: Two Real-world Examples
Altogether, the successful implementation of the digital clinic requires effective facilitation strategies that take into account features of the innovation itself (e.g., the utility and complexity of a given app), the recipients of that innovation (e.g., the attitudes and beliefs of patients and providers engaging with the apps), and the context in which it is deployed (e.g., a community mental health center, hospital, or private practice) [20••, 21]. Below, we describe two examples of digital clinic implementation to illustrate how apps can be effectively integrated into mental health care in real-world settings which are illustrated further in Table 1 and Fig. 1 below.
Table 1.
Challenges of implementing digital clinics | Example solutions |
---|---|
Stakeholder buy-in and support | Input from key stakeholders (i.e., patients, clinicians, clinic managers) from app design to clinical implementation |
Clinical workflow | Redesigned clinics, Digital Navigator support, designing apps that integrate within existing provider workflow |
Provider time and training | Digital Navigator support, provider training initiatives, and online resources (including sharing data regarding the effectiveness and acceptability of digital mental health care) |
Digital data policy | Clear clinical policies, attention to safety planning, and following best practices for app use |
Patient digital literacy and access to devices | Lifeline Program for low-income consumers, Digital Opportunities for Outcomes in Recovery Services (DOORS), Digital Navigator support, VA’s Digital Divide Consult, and Internet-Connected iPad Loaner Program |
Selection of apps | Mobile Health and Index Navigation Database (MIND), Digital Navigator support, collection of free, evidence-based VA apps |
The Beth Israel Deaconess Medical Center Digital Clinic
The Digital Clinic at Beth Israel Deaconess Medical Center seeks to integrate digital tools into mental health care for patients with mood and anxiety disorders. The clinic utilizes a smartphone app called mindLAMP that was created to support dual use by both clinicians and patients in allowing for customization of content, assessments, and skills training [22]. Clinicians and patients can tailor app features to current clinical needs, ensuring that care is driven by shared decision making rather than a “one-size-fits-all” approach. This customization process is facilitated by a new team member called the digital navigator [23, 24]. The concept of a digital navigator is not novel; the role of clinical technology specialists [25, 26] has been well outlined and refers to patient-facing team members who help patients with technology needs. The digital navigator role is related and also includes interfacing with the clinical team and involves three core roles: (1) assisting in evidence-based app selection to match each clinical case, (2) helping patients troubleshoot technology needs, and (3) distilling patient app data into clinically meaningful summaries [24]. Training for digital navigators is offered through a 10-h curriculum that covers five core modules including the following: (1) core smartphone skills, (2) basic technology troubleshooting, (3) app evaluation, (4) clinical terminology and data, and (5) engagement techniques [23].
By delegating these app-related tasks to the digital navigator, clinical visits can focus on mental health treatment instead of on troubleshooting technology. In turn, clinical visits have been redesigned to ensure that data captured outside of the visit via the mindLAMP app (e.g., mood and step count) is always reviewed at the start of the session. This data is used to inform clinical considerations and shared decision making, and the visit concludes with the creation of a technology plan (e.g., using a certain app to practice a skill or taking a new symptom survey within an app to monitor medication response). This plan is then supported by a check-in with the digital navigator after the visit. Finally, the Digital Clinic is supported by several additional facilitation efforts to improve integration of the mindLAMP app into care. Digital navigator training is offered to all clinicians and staff who wish to become more proficient around technology, and the clinic also conducts a digital literacy course for patients who are interested in using their phone as a tool towards recovery [27]. These educational efforts are buttressed by a library of online resources including a database of mental health apps, office hours, and new note templates to help with documentation.
Integrating the PTSD Coach App Into VA Primary Care
The Digital Clinic at Beth Israel Deaconess Medical Center is an example of a deliberately developed general digital mental health clinic to serve patients with mood and anxiety disorders. Other types of digital clinics will also be needed to bring mental health apps into routine care to achieve their full promise. These include efforts to integrate apps into established care structures where they can be delivered by existing treatment staff. One such example of this approach is the VA Clinician-Supported Posttraumatic Stress Disorder Coach (CS PTSD Coach), which is a specialty digital clinic for VA primary care patients with PTSD [28]. CS PTSD Coach includes the self-management mobile app, PTSD Coach [29] coupled with four brief protocol-driven sessions delivered over 8 weeks in-person or by telephone, per patient preference, by a primary care mental health care provider. Given the high prevalence of PTSD among VA primary care patients [30] and the lack of brief, problem-focused, evidence-based PTSD treatments suitable for delivery in primary care settings, CS PTSD Coach fills a care gap by providing a low-intensity, limited-duration, stepped-care intervention.
The Consolidated Framework for Implementation Research (CFIR) was used to guide the CS PTSD Coach implementation process [31, 32]. CFIR outlines a variety of domains that must be considered to ensure successful implementation, including characteristics of the intervention, the context in which it is being implemented, and characteristics of individuals involved in the implementation process. In the first phase of developing CS PTSD Coach, nine key organizational stakeholders, including a primary care lead physician, a PTSD clinic team leader, a mental health leader in primary care, and additional care providers provided formative feedback. A CS PTSD Coach clinician manual was then created which was informed by their responses. The manual provided information and instructions regarding collaborative goal setting, symptom management recommendations, supporting patient use of PTSD Coach, and if needed, helping patients consider additional PTSD treatment. In the second phase, nine veterans with PTSD participated in CS PTSD Coach and afterwards were interviewed about what they found helpful about the intervention and changes they would recommend to it. Overall, veterans had positive views of the intervention, including the following: (1) the brief session length, (2) the choice of completing sessions in-person or by telephone, and (3) receiving guidance from the clinician in tailoring their app use. Finally, in the third phase, three clinicians who delivered CS PTSD Coach provided feedback on their experiences delivering the protocol, which informed further modifications of the manual.
A preliminary randomized control trial (RCT) comparing CS PTSD Coach (n = 10) to PTSD Coach alone (n = 10) suggested that it may be more helpful than using the app alone (d = .54, n.s.) among veterans with PTSD seen in VA primary care [28]. In fact, the within-condition PTSD symptom improvement was large (d = 1.4, p < .01) with 70% of patients reporting clinically significant improvement of symptoms (versus 30% for PTSD Coach alone). Likewise, 90% of the CS PTSD participants accepted a referral for specialty mental health treatment versus 25% of the PTSD Coach only condition. Although the small sample size precludes drawing strong conclusions from these findings, a full-scale RCT is currently underway testing CS PTSD compared to primary care mental health treatment as usual [28].
Emerging and Related Examples
Additional examples of emerging digital clinics include a flipped clinic model proposed by an Australian research group that would offer “web-based interventions with the support of a junior clinician” [33]. In a similar vein, Kaiser Permanente in the USA describes the creation of a “mental health and wellness digital ecosystem,” in which apps are introduced in “the context of a trusting patient–provider relationship”; they report that 1678 trained clinicians made 44,277 referrals to apps in May 2020 [34]. We expect similar examples of digital clinics to continue to emerge as attitudes towards digital health integration have become more favorable. Current temporary changes in regulation and payment models may lead to more permanent modifications. Clinician exposure to and success with digital health during COVID-19, combined with positive patient experiences and outcomes, may further encourage use.
Challenges and Next Steps
Implementation of digital mental health tools is not an easy process, and recent reviews have reported inconsistent success among the published literature [20••, 35]. It is critical to gain digital clinic buy-in at the level of the recipients (e.g., patients and providers) as well as the organization (e.g., healthcare system) in order to achieve optimal outcomes. Indeed, lack of support has been demonstrated as a major source of implementation failure in previous attempts at digital integration [36]. It will be critical to rethink the clinical workflow in order to make the integration of digital tools as seamless and simple as possible. The addition of a new team member in the form of a digital navigator may be an essential component of this process, in ensuring that the introduction of apps into care does not become burdensome to patients and providers. Indeed, providers have reported not having training in how to use apps with their patients [37] and note not having enough time or understanding of apps to recommend them during visits [38]. The need for digital navigators has become especially apparent during COVID-19, as grassroots efforts to train volunteers to support digital care have rapidly emerged to fill the current void in digital literacy education and resources [39–43]. Similar roles such as mental health technology specialists have also demonstrated promising preliminary outcomes in supporting patients and clinicians [44]. These real-world and research efforts are encouraging in demonstrating the feasibility of community members or peers potentially serving in a digital navigator role.
In addition to these challenges, fears that integrating technology into mental health care will fracture the therapeutic alliance are widespread [14, 45]. Yet evidence to date does not support this fear, as alliance ratings are high for care offered through digital channels [46]. An additional challenge involves the amorphous world of digital data. As some apps allow for a constant stream of patient data into online provider portals 24 h a day, it is unclear where to set the boundaries of the digital clinic; where does a clinician’s duty and responsibility stop? Careful establishment of clinical policies, attention to safety planning, and following of clinical best practices for app use [47••] will help ensure that the scope of the digital clinic is firmly defined.
A more subtle but urgent challenge is to ensure that digital clinics offer equal access to care, which requires attention to digital literacy and the availability of internet-enabled devices and adequate wireless connectivity. Emerging evidence suggests that populations with limited access to devices or limited digital health literacy skills may not seek out digital care [48]. It is also critical that digital tools are culturally appropriate [49], available in languages other than English [50], and accessible to those with disabilities [51], to prevent further deepening the digital divide in who can take advantage of these novel treatment modalities. Increasing access to devices may be possible through federal programs in the USA (Lifeline Program for Low-Income Consumers) and resources like those shared at https://www.digitalinclusion.org/free-low-cost-internet-plans/. The VA has demonstrated success in providing internet-enabled tablets to Veterans without access to a device via its Digital Divide Consult and Internet-Connected iPad Loaner Program [52, 53]. Increasing digital literacy and ensuring that patients have the knowledge, skills, and confidence to partake in all forms of digital care will require expanding programs focused on teaching and training a new generation of support staff to assist patients in need [24, 54]. If these barriers can be overcome, the digital clinic has the potential to significantly enhance and reimagine the future of mental health care delivery.
Conclusion
Successful implementation of digital clinics requires effective facilitation strategies that account for features of the technology itself, its end users, and the clinical context of deployment. With COVID-19 restrictions increasingly favoring digital delivery of care and a plethora of impressive technology solutions already available today, there lies an ideal opportunity to support the creation of digital clinics and in turn to realize the full potential of digital mental health care.
Footnotes
This article is part of the Topical Collection on Psychiatry in the Digital Age
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
Papers of particular interest, published recently, have been highlighted as: •• Of major importance
- 1.Huckvale K, Nicholas J, Torous J, Larsen ME. Smartphone apps for the treatment of mental health conditions: status and considerations. Curr Opin Psychol. 2020;4. [DOI] [PubMed]
- 2.Bauer M, Glenn T, Geddes J, Gitlin M, Grof P, Kessing LV, Monteith S, Faurholt-Jepsen M, Severus E, Whybrow PC. Smartphones in mental health: a critical review of background issues, current status and future concerns. Int J Bipolar Disorders. 2020;8(1):1–9. doi: 10.1186/2194-7511-1-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Figueroa CA, Aguilera A. The need for a mental health technology revolution in the COVID-19 pandemic. Front Psych. 11. [DOI] [PMC free article] [PubMed]
- 4.Targum SD, Sauder C, Evans M, Saber JN, Harvey PD. Ecological momentary assessment as a measurement tool in depression trials. J Psychiatr Res. 2021. [DOI] [PubMed]
- 5.Henson P, D’Mello R, Vaidyam A, Keshavan M, Torous J. Anomaly detection to predict relapse risk in schizophrenia. Transl Psychiatry. 2021;11(1):1–6. doi: 10.1038/s41398-020-01123-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Andrea A, Agulia A, Serafini G, Amore M. Digital biomarkers and digital phenotyping in mental health care and prevention. Eur J Pub Health. 2020;30(Supplement_5):ckaa165–ckaa1080. doi: 10.1093/eurpub/ckaa165.1080. [DOI] [Google Scholar]
- 7.Kinoshita S, Cortright K, Crawford A, Mizuno Y, Yoshida K, Hilty D, Guinart D, Torous J, Correll CU, Castle DJ, Rocha D. Changes in telepsychiatry regulations during the COVID-19 pandemic: 17 countries and regions’ approaches to an evolving healthcare landscape. Psychol Med. 2020;27:1–8. doi: 10.1017/S0033291720004584. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Bhatia R. Telehealth and COVID-19: using technology to accelerate the curve on access and quality healthcare for citizens in India. Technol Soc. 2021;64:101465. doi: 10.1016/j.techsoc.2020.101465. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.https://www.govinfo.gov/app/details/FR-2021-01-15/2021-00787
- 10.Gordon JA, Borja SE. The COVID-19 pandemic: setting the mental health research agenda. Biol Psychiatry. 2020;18. [DOI] [PMC free article] [PubMed]
- 11.Mageit, S. October 19, 2020. Two healthcare apps available for prescription in Germany for first time. Healthcare IT News. https://www.healthcareitnews.com/news/emea/two-healthcare-apps-available-prescription-germany-first-time
- 12.Guinart D, Marcy P, Hauser M, Dwyer M, Kane JM. Patient attitudes toward telepsychiatry during the COVID-19 pandemic: a nationwide, multisite survey. JMIR Mental Health. 2020;7(12):e24761. doi: 10.2196/24761. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Chen JA, Chung WJ, Young SK, Tuttle MC, Collins MB, Darghouth SL, Longley R, Levy R, Razafsha M, Kerner JC, Wozniak J. COVID-19 and telepsychiatry: early outpatient experiences and implications for the future. Gen Hosp Psychiatry. 2020;66:89–95. doi: 10.1016/j.genhosppsych.2020.07.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Uscher-Pines et al. Suddenly becoming a “virtual doctor”: experience is a psychiatrist transitioning to telemedicine during the COVID-19 pandemic. Psychiatric Services. 2020 [DOI] [PMC free article] [PubMed]
- 15.Connolly SL, Miller CJ, Lindsay JA, Bauer MS. A systematic review of providers’ attitudes toward telemental health via videoconferencing. Clin Psychol Sci Pract. 2020;27(2):e12311. [DOI] [PMC free article] [PubMed]
- 16.Humer E, Stippl P, Pieh C, Pryss R, Probst T. Experiences of psychotherapists with remote psychotherapy during the COVID-19 pandemic: cross-sectional web-based survey study. J Med Internet Res. 2020;22(11):e20246. doi: 10.2196/20246. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.https://www.pewresearch.org/internet/fact-sheet/mobile/
- 18.••.Owen JE, Kuhn E, Jaworski BK, McGee-Vincent P, Juhasz K, Hoffman JE, et al. VA mobile apps for PTSD and related problems: public health resources for veterans and those who care for them. Mhealth. 2018;4 This paper outlines the history of apps offered by the VA, reviews key offerings, and explores the evidence and rigor behind this publicly accessible app library. [DOI] [PMC free article] [PubMed]
- 19.Diamond R, Byrd E. Standing up for health–improving mental wellbeing during COVID-19 isolation by reducing sedentary behaviour. J Affect Disord. 2020;277:232–234. doi: 10.1016/j.jad.2020.07.137. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.•• Connolly SL, Hogan TP, Shimada SL, Miller CJ. Leveraging implementation science to understand factors influencing sustained use of mental health apps: a narrative review. J Technol Behav Sci. 2020:1–13. This 2020 review offers a thorough overview of implementation efforts around digital mental health as well as how theory can be applied to clinical practice. [DOI] [PMC free article] [PubMed]
- 21.Harvey G, Kitson A. PARIHS revisited: from heuristic to integrated framework for the successful implementation of knowledge into practice. Implement Sci. 2015;11(1):1–13. doi: 10.1186/s13012-016-0398-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Torous J, Wisniewski H, Bird B, Carpenter E, David G, Elejalde E, Fulford D, Guimond S, Hays R, Henson P, Hoffman L. Creating a digital health smartphone app and digital phenotyping platform for mental health and diverse healthcare needs: an interdisciplinary and collaborative approach. J Technol Behav Sci. 2019;4(2):73–85. doi: 10.1007/s41347-019-00095-w. [DOI] [Google Scholar]
- 23.Wisniewski H, Gorrindo T, Rauseo-Ricupero N, Hilty D, Torous J. The role of digital navigators in promoting clinical care and technology integration into practice. Digital biomark. 2020;4(Suppl 1):119. doi: 10.1159/000510144. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Wisniewski H, Torous J. Digital navigators to implement smartphone and digital tools in care. Acta Psychiatr Scand. 2020;141(4):350–355. doi: 10.1111/acps.13149. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Ben-Zeev D, Drake R, Marsch L. Clinical technology specialists. BMJ. 2015;350:h945. doi: 10.1136/bmj.h945. [DOI] [PubMed] [Google Scholar]
- 26.Noel VA, Carpenter-Song E, Acquilano SC, Torous J, Drake RE. The technology specialist: a 21st century support role in clinical care. NPJ Digital Med. 2019;2(1):1–3. doi: 10.1038/s41746-019-0137-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Hoffman L, Wisniewski H, Hays R, Henson P, Vaidyam A, Hendel V, Keshavan M, Torous J. Digital Opportunities for Outcomes in Recovery Services (DOORS): a pragmatic hands-on group approach toward increasing digital health and smartphone competencies, autonomy, relatedness, and alliance for those with serious mental illness. J Psychiatr Pract. 2020;26(2):80–88. doi: 10.1097/PRA.0000000000000450. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Possemato K, Kuhn E, Johnson E, Hoffman JE, Owen JE, Kanuri N, De Stefano L, Brooks E. Using PTSD Coach in primary care with and without clinician support: a pilot randomized controlled trial. Gen Hosp Psychiatry. 2016;38:94–98. doi: 10.1016/j.genhosppsych.2015.09.005. [DOI] [PubMed] [Google Scholar]
- 29.Kuhn E, Greene C, Hoffman J, Nguyen T, Wald L, Schmidt J, Ramsey KM, Ruzek J. A preliminary evaluation of PTSD Coach, a smartphone app for posttraumatic stress symptoms. Mil Med. 2014;179:12–18. doi: 10.7205/MILMED-D-13-00271. [DOI] [PubMed] [Google Scholar]
- 30.Magruder KM, Frueh BC, Knapp RG, Davis L, Hamner MB, Martin RH, Gold PB, Arana GW. Prevalence of posttraumatic stress disorder in Veterans Affairs primary care clinics. Gen Hosp Psychiatry. 2005;27(3):169–179. doi: 10.1016/j.genhosppsych.2004.11.001. [DOI] [PubMed] [Google Scholar]
- 31.Possemato K, Kuhn E, Johnson E, Hoffman JE, Brooks E. Clinician-supported PTSD Coach: provider and patient input on acceptability and barriers and facilitators to implementation. Trans Behav Med Prac Policy, Res. 2017;7:116–126. doi: 10.1007/s13142-016-0393-9. [DOI] [Google Scholar]
- 32.Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4(1):1–15. doi: 10.1186/1748-5908-4-50. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Davenport TA, Cheng VW, Iorfino F, Hamilton B, Castaldi E, Burton A, Scott EM, Hickie IB. Flip the clinic: a digital health approach to youth mental health service delivery during the COVID-19 pandemic and beyond. JMIR Mental Health. 2020;7(12):e24578. doi: 10.2196/24578. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Mordecai D, Histon T, Neuwirth E, Heisler WS, Kraft A, Bang Y, Franchino K, Taillac C, Nixon JP. How Kaiser Permanente created a mental health and wellness digital ecosystem. NEJM Catalyst Innov Care Deliv. 2021;1:2(1). [Google Scholar]
- 35.Graham AK, Lattie EG, Powell BJ, Lyon AR, Smith JD, Schueller SM, Stadnick NA, Brown CH, Mohr DC. Implementation strategies for digital mental health interventions in health care settings. Am Psychol. 2020;75(8):1080. doi: 10.1037/amp0000686. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Greenhalgh T, Wherton J, Papoutsi C, Lynch J, Hughes G, Hinder S, Fahy N, Procter R, Shaw S. Beyond adoption: a new framework for theorizing and evaluating nonadoption, abandonment, and challenges to the scale-up, spread, and sustainability of health and care technologies. J Med Internet Res. 2017;19(11):e367. doi: 10.2196/jmir.8775. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Jacob C, Sanchez-Vazquez A, Ivory C. Social, organizational, and technological factors impacting clinicians’ adoption of mobile health tools: systematic literature review. JMIR mHealth uHealth. 2020;8(2):e15935. doi: 10.2196/15935. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Hoffman L, Benedetto E, Huang H, Grossman E, Kaluma D, Mann Z, Torous J. Augmenting mental health in primary care: a 1-year study of deploying smartphone apps in a multi-site primary care/behavioral health integration program. Front Psych. 2019;10:94. doi: 10.3389/fpsyt.2019.00094. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Triana AJ, Gusdorf RE, Shah KP, Horst SN. Technology literacy as a barrier to telehealth during COVID-19. telemedicine and e-health. 2020 May 19. [DOI] [PubMed]
- 40.Kim et al. How the rapid shift to telehealth leaves many community health centers behind during the COVID-19 pandemic. Health Affairs. 2020
- 41.Byambasuren O, Beller E, Hoffmann T, Glasziou P. mHealth app prescription in Australian general practice: pre-post study. JMIR mHealth uHealth. 2020;8(6):e16497. doi: 10.2196/16497. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Torous J, Wykes T. Opportunities from the coronavirus disease 2019 pandemic for transforming psychiatric care with telehealth. JAMA Psych. 2020;11. [DOI] [PubMed]
- 43.Moreno C, Wykes T, Galderisi S, Nordentoft M, Crossley N, Jones N, et al. How mental health care should change as a consequence of the COVID-19 pandemic. Lancet Psychiatry. 2020;16. [DOI] [PMC free article] [PubMed]
- 44.Carpenter-Song E, Acquilano SC, Noel V, Al-Abdulmunem M, Torous J, Drake RE. Individualized intervention to support mental health recovery through implementation of digital tools into clinical care: feasibility study. Community Ment Health J. 2021;21:1–2. doi: 10.1007/s10597-021-00798-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Cowan KE, McKean AJ, Gentry MT, Hilty DM. Barriers to use of telepsychiatry: clinicians as gatekeepers. InMayo Clinic Proceedings 2019 Dec 1 (Vol. 94, No. 12, pp. 2510–2523). Elsevier. [DOI] [PubMed]
- 46.Simpson S, Richardson L, Pietrabissa G, Castelnuovo G, Reid C. Videotherapy and therapeutic alliance in the age of COVID-19. Clin Psychol Psychother. 2020;10. [DOI] [PMC free article] [PubMed]
- 47.••.Yellowlees P, Shore JH. Telepsychiatry and health technologies: a guide for mental health professionals: American Psychiatric Pub; 2018. This book offers a comprehensive primer for telehealth and digital mental health that is a useful resource for those wishing to understand all aspects of the field.
- 48.Nouri S, Khoong EC, Lyles CR, Karliner L. Addressing equity in telemedicine for chronic disease management during the Covid-19 pandemic. NEJM Catalyst Innov Care Deliv. 2020;4:1(3). [Google Scholar]
- 49.Brooks E, Spargo G, Yellowlees P, O’Neill P, Shore JH. Telemental Health: Clinical, Technical, and Administrative Foundations for Evidenced-Based Practice. 2013. Integrating culturally appropriate care into telemental health practice; pp. 63–79. [Google Scholar]
- 50.Munoz AO, Camacho E, Torous J. Marketplace and literature review of Spanish language mental health apps. Front Digital Health. 2021;3:7. doi: 10.3389/fdgth.2021.615366. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.Valdez RS, Rogers CC, Claypool H, Trieshmann L, Frye O, Wellbeloved-Stone C, et al. Ensuring full participation of people with disabilities in an era of telehealth. J Am Med Inform Assoc. 2020. [DOI] [PMC free article] [PubMed]
- 52.Heyworth L, Kirsh S, Zulman D, Ferguson JM, Kizer KW. Expanding access through virtual care: the VA’s early experience with Covid-19. NEJM Catalyst Innov Care Deliv. 2020;1(4).
- 53.Zulman DM, Wong EP, Slightam C, Gregory A, Jacobs JC, Kimerling R, et al. Making connections: nationwide implementation of video telehealth tablets to address access barriers in veterans. JAMIA open. 2019;2(3):323–329. doi: 10.1093/jamiaopen/ooz024. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 54.Rodriguez-Villa E, Camacho E, Torous J. Psychiatric rehabilitation through teaching smartphone skills to improve functional outcomes in serious mental illness. Internet Interv. 2021;23:100366. doi: 10.1016/j.invent.2021.100366. [DOI] [PMC free article] [PubMed] [Google Scholar]