Abstract
Increasing both access to and quality of mental healthcare is a global priority. One solution is to integrate technologies such as smartphone apps and sensors directly into care. Acknowledging many prior attempts and barriers, we introduce the Digital Clinic which is an already functioning clinic using smartphone apps to augment and extend care today at Beth Israel Deaconess Medical Center (BIDMC) in Boston, Massachusetts. In this piece, we outline the theoretical foundation of the Digital Clinic and its emphasis on the therapeutic alliance, measurement-based care, and shared decision making. We explore both workflow and engagement challenges as well as solutions including a new care team member, the Digital Navigator, and the customization of technology. Acknowledging that the Digital Clinic is an evolving program, we offer details on our implementation in order to allow others to replicate, expand on, and improve these initial efforts.
Keywords: Digital clinic, Mental health, Digital phenotyping, Therapeutic alliance, Program model, Individualized interventions
1. Introduction
The growing burden of mental health and the limitations of the clinical workforce to meet new demand [1] have widened the treatment gap. The COVID-19 pandemic has highlighted the imminent need for new and innovative avenues to increase both access to and quality of care. Yet efforts around computerized therapy and recently emerging smartphone apps have not yet transformed mental health despite their clear potential to offer scalable, accessible, and affordable solutions. The challenges underlying the realization of digital solutions are not technological. They are instead rooted in the need to create trust and clinical systems that ethically and effectively utilize new digital tools. In this paper, we describe the concept and early implementation of a digital clinic, which integrates digital innovations into traditional care models for assessment, treatment and care management of patients with mental health disorders. The digital clinic described in this paper offers a conceptual and feasible model for care delivery that can be implemented in diverse settings.
The goals of the digital clinic are to improve both access to and quality of care through the ethical integration of digital technologies into clinical workflows, evidence-based care, and shared decision making in a sustainable and scalable manner. Distinct from telepsychiatry which offers synchronous telehealth, we define digital health and clinics as using asynchronous technologies such as apps and sensors to collect comprehensive data and inform care. Digital care and digital clinics can supplement telehealth (and in person care) by bringing new data into the virtual visit and offering increased support for patients between those visits. The concept of the digital clinic has many names and draws inspiration from ongoing digital mental health research [2], models of hybrid care [3], implementation science efforts [4], and healthcare technology development work [5]. The cross disciplinary nature of the digital clinic underscores its unique potential to offer impactful care as well as some of the operational challenges it faces towards successful implementation.
Addressing the needs of both patients and clinicians to utilize technology effectively, the digital clinic model is not a new concept. In early 2003 when CD-ROMS were novel, a brick and mortar digital clinic was established in London and offered access to computer-based treatments supported by a psychologist [6]. While the rise of the internet made this clinic less relevant, newer efforts to introduce hybrid care have emerged. Often employing remote coaches who are not licensed clinicians, these platforms seek to engage patients through personalized contact. Yet failure of mental health startups, such as Lantern, that offer this type of hybrid care and mixed published results on the efficacy of coaches [7] illustrate a complicated picture. Several reasons elucidate why companies have struggled to gain footing including insufficient funding and dissonance between business models and routine clinical care. Reports from many users of technology related mental health services have noted such experiences “feel different from real life” [8]. Lack of transformative clinical outcomes from technology and a concomitant lack of connection felt through a strong therapeutic alliance appear as common challenges.
Challenges to digital clinics are understandable in light of past and present overvaluation of technology combined with undervaluation of clinical staff. While technologies such as smartphone apps are themselves scalable and available for immediate download to billions of people today, their meaningful use and clinical impact is low. A 2020 study examining the use of over 100,000 health apps in clinical studies found that apps were used for an average total of 5.5 days [9]. Another study of popular mental health apps with over 100,000 downloads found that the average engagement rate after 10 days was just under 4% [10]. Emerging evidence and clinical intuition suggests that sustained engagement with smartphones and digital tools is likely to be enhanced when they are used in conjunction with support from mental health care providers in clinical settings [11].
Yet today many clinicians are not able to support digital tools or care. While COVID-19 has forced clinicians to adopt telehealth, a study showed that 33% of psychiatrists preferred voice calls to videoconferencing visits because they lacked confidence in how to use telehealth technologies with patients [12]. There is less evidence of integrating digital health tools like apps and sensors into care. A 2020 systematic review of factors impacting clinicians' use of technology highlighted that social and organizational factors led by workflow issues prevented adoption [13]. Other barriers include concerns from clinicians regarding privacy, liability, and a lack of confidence in the technology [14,15]. The popular non-adoption, abandonment, scale-up, spread, and sustainability framework for digital health notes that clinician champions are often the most critical contributor in enabling clinical adoption [16].
Drawing from early experiences and literature on digital health, hybrid care, engagement models, and technology adoption – we propose that a new digital clinic can address the current gaps in technology engagement and clinical workflow to maximize their synergy. The foundation of the digital clinic is based on patient centered care—incorporation of patients' perspectives, support systems, and values [17]—that draws on the therapeutic alliance and its clinical benefits through facilitating shared decision making and understanding [18,19]. The digital clinic model is intended to be scaled and built on with the goal of increasing access to high quality, evidence-based care. As such, we outline our experience of establishing a digital clinic at Beth Israel Medical Center (BIDMC). While our implementation is made unique by our care team, patients, and site, the models and concepts are global in their application.
2. The development of the digital clinic
The digital clinic requires efficient technology and a structured workflow to build therapeutic trust and deliver effective evidence base care. Realizing that today's electronic medical records often slow care and detract from the therapeutic alliance, we want to ensure that technology improves clinical operation without impeding care. A clinical visit follows standard processes of information gathering, clinical decision making, and treatment planning. The digital clinic focuses treatment by reducing the need for information gathering, and makes clinical decision making and treatment planning a collaborative, iterative conversation. We leverage mindLAMP, a digital platform built and designed with patient input, to aggregate patient data, guide reflection, and help inform treatment. mindLAMP, detailed in Appendix 1, is adaptable and offers the care team flexibility to modify the platform's language, content, and specific functionalities [20,21].
2.1. Framework
In order to ensure the implementation process for the digital clinic can be used at other sites and with other teams, we utilized the Replication Effective Programs (REP) framework [22]. Fig. 1 outlines our implementation model and the unique considerations as well as challenges across the implementation cycle. With the ultimate goal of creating a replicable model that can be implemented in diverse settings among different populations, the REP framework offers a practical approach to explaining the process of establishing and maintaining a digital clinic. (See Table 1 .)
Fig. 1.
The mindLAMP (Learn – Assess – Manage – Prevent) app can be customized to track symptoms and activity, collect data, and offer mindfulness resources. It is also possible to use a tool box of several independent apps to serve each of these features.
Table 1.
The implementation process for creating a digital clinic is adapted from the REP framework. Time estimates are based on our personal experience at Beth Israel Deaconess Medical Center and will vary based on setting.
2.2. Preconditions: needs and resource assessment
Core barriers that sites may face in establishing a digital clinic include digital health literacy, patient and clinician willingness to adopt digital tools, workflow issues, and regular engagement. Since the digital clinic requires participants and clinicians to use smart phone technology, access to a smart phone is a prerequisite. Direct feedback from patients and the onsite clinical team— formally structured into the pre-implementation stage — should be incorporated into the program's implementation plan during this precondition stage as well.
Our solutions to core barriers include identifying the right technology and clinical models to leverage. We utilize mindLAMP, a digital care platform detailed in Appendix 2, to track and aggregate patient data. Our team created the mindLamp app specifically for use in a digital clinic. Our team offers the app and its underlying code as free, open source software so that others can utilize and expand upon it. However, other applications and digital resources are useful in the digital clinic. The goal is not to link the clinic to a single technology but rather to work with patients and utilize the ones that best support their treatment. There are many digital health programs and apps available today and we recommend resources, such as the American Psychiatric Association's app evaluation framework, to assist in informed decision making [23].
Engagement, from both patients and clinicians, is a considerable barrier that hinges on comfort using selected technologies. The Digital Opportunities for Outcomes in Recovery Services (DOORS) program offers a clinical model for teaching digital literacy skills to patients with serious mental illness [24]. We utilize the DOORS manual in Appendix 2 to guide digital skills building with patients. We also offer specialized training for clinicians that we refer to as Digital Navigator training that is outlined later in this paper and is available as Appendix 3. DOORS and Digital Navigator training were created by our team in response to a lack of resources that teach patients and clinicians, respectively, digital health skills, competencies, and confidences.
The clinical workflow model also emphasizes the importance of creating a physical space for team technology training, defining roles and responsibilities, mapping changes to workflow, and identifying additional support that may be accessed or required. It is critical that all participants (patients and clients) have access to appropriate technology compatible devices. With smartphone ownership in the US up from 35% in 2011 to 81% in 2018 [25], smartphone access may be less of a barrier with time. Nonetheless, the care team assists patients with reviewing options for affordable or free smartphone plans when applicable (in the US often through Projection Lifeline) and is aware of which apps and programs can run on which phones.
2.3. Pre-implementation
Pre-implementation barriers include creating an engaged and committed feedback committee, determining the role of selected technologies, and ensuring that the clinical team is prepared to integrate these technologies into care.
A feedback committee engages diverse stakeholders including the care team, patients and their family members, and data scientists. We facilitated small, informal focus groups to understand various comfort levels using mobile apps, collect input on mindLAMP, and discuss ways technology can enhance care. We also drew on feedback from previous clinical experiences to design visits that integrate digital tools into therapy safety and effectively. Incorporating relevant feedback – at every stage – allows the digital clinic to continuously improve and adapt to meet the needs of its patients.
mindLAMP supports the digital clinic at Beth Israel Deaconess Medical Center. We are able customize the app at every visit to offer each patient unique survey assessments, data captures, health information, and activities that are relevant to their own illness and recovery. Some of the activities mindLAMP can be customized to track and offer patients are outlined in the Fig. 1. The clinical team works with the patient to determine how the platform is used and can best support the patient. Thus not all features of the app are used at all times, and use adapts to reflect shared decision making and changing clinical needs. App use is not a set protocol. This practice is in contrast to many health apps that guide care and push the clinician into a support role [26] and addresses a chief reason for dissatisfaction with apps among both patients and clinicians. Other practices to support mindLAMP integration include the Digital Navigator offering clinicians technical support, providing sufficient time for participants to adjust to the new intervention (incorporating technology with treatment), and gathering and incorporating constructive feedback from all members of the care team.
Technology should be versatile enough to support diverse use cases and permit customized information flow. One advantage of mobile technology is the ability to rapidly share and access information that the patient is at the center of. Which information mindLAMP collects from a patient, who receives it, and how it is shared with and analyzed by the care team are important considerations. Fig. 2 shows how the care team appears to the patient mindLAMP. The potential of virtual team meetings that include any combinations of the team – in which graphs that show symptom progression or medication adherence patterns can be shared (with consent) by screensharing – offers a means to provide inclusive and responsive team-based care. Clear parameters and guidelines around communication with patients outside digital clinic visits are considered here as well. Policies and informed consent to ensure use and response to any messages sent through the messaging feature on mindLAMP are well outlined and emergency plans are put in place. Some clinics may be able to monitor messaging 24/7 and while others may not. Use of backend analytical software to identify changes in app data that may warrant a response are also considered during pre-implementation. Actions plans that respond to an event of unusual patient data are established to ensure safety.
Fig. 2.
The mindLAMP platform includes a visual representation of the patient's care team. The role of technology in capturing data as well as offering learning and interventions is also highlighted, as are at least three areas where a digital navigator is able to offer assistance.
To assess the impact of the digital clinic and allow for continuous process improvements, an assessment battery was created. The multi-level assessment process offers a number of different scales, with the goal of assessing different dimensions related to the clinic including perspectives from both staff and patients, clinical outcomes, engagement, and other metrics of success. The conceptual framework for this selection of measures is based on Berwick's “triple aim” of health care improvement: optimizing care delivery by improving health outcomes, decreasing costs per capita, and improving patient (and clinician) experience [27]. Table 2 outlines the digital clinic's assessment battery and the frequency with which scales are administered. These assessments have been selected to collect diverse data that produce primary outcome measures (e.g. depression response/remission rates from PHQ-9 scales, clinic costs and healthcare utilization, indices of daily function/wellbeing), process measures (e.g. therapeutic alliance scales, clinical dropout, app usage and adherence), and balancing measures (e.g. clinician burnout, patient satisfaction, total cost of digital navigator utilization). Baseline clinical factors such as age, functional disability, motivation to change, comorbid substance use, and clinical history of trauma are also collected to identify potential moderators of digital impact. Combined, these assessments help us measure our impact overtime and capture a more complete patient profile to contextualize progress and feedback.
Table 2.
The digital clinic uses an assessment battery to measure clinical impact and collect feedback.
| Measurement scale | Frequency administered |
|---|---|
| Technology scale for patients | |
| App/technology assessment | 1× (upon enrollment) |
| Clinical scales for patients | |
| PHQ-9 | 3× monthly |
| GAD - 7 | 3× monthly |
| Sheehan disability scale | 3× monthly |
| Audit - C | 2× monthly |
| PC - PTSD - 5 | 1× monthly |
| Basis - 24 | 2× monthly |
| Satisfaction scales for patients | |
| WAI-SR | 2× monthly |
| WHO-5 | 2× monthly |
| Motivation | 2× monthly |
| Patient satisfaction | 2× monthly |
| Satisfaction scale for clinical team | |
| Burnout inventory | 1× monthly |
| Other quality metrics | |
| App usage | 1× monthly |
| Cost/utilization | 1× monthly |
2.4. Implementation
The success of the digital clinic is in allowing both the patient and clinician the flexibility to use and customize technology that supports decision making and therapeutic alliance. As such, the care team works together to ensure that the functionalities and capabilities of the technology, in our case mindLAMP, are understood by the care team.
The care team for the digital clinic includes a psychiatrist, clinician, Digital Navigator, and patient. The clinician (social worker, psychiatrist, mental health counselor, therapist) will act as the primary clinical point person for clients and assist with building the therapeutic alliance. For the purpose of this article, Bordin's interpretation of the therapeutic alliance will be used with an emphasis on collaboration to set goals, achieve tasks, and maintain the therapeutic relationship [28]. The Digital Navigator, explained in greater detail below and in Appendix 3, supports the clinician by working with the patient to ensure they are able to understand and utilize mindLAMP for their benefit. The patient is an equally essential part of the care team. Their responsibilities include attending therapeutic sessions, engaging with mindLAMP regularly, and helping develop best-individual-clinical practices for themselves.
The Digital Navigator is critical to the care team through helping both clinicians and patients troubleshoot, set up, customize, and monitor technology as noted above in Fig. 2 [29]. They ensure that when the patient arrives for their visit, their unique digital data is accessible and prepared for display. In the sessions, they help explain the displayed graphs and maximize the patient's time with clinicians by doing so. The Digital Navigator can customize mindLAMP in real time to reflect changes in a clinical plan decided in the middle of a session. The Digital Navigator also ensures that when it is useful to select and recommend other apps, for example if there is a need for an app offering DBT skills, informed recommendations can be offered.
2.5. Maintenance and evolution
Maintenance and evolution of the digital clinic is an ongoing stage. Based on our early experience since the first patient visit in Fall 2019, barriers in this stage may include maintaining updated technology, executing effective evaluation protocols, and ensuring ongoing quality improvement efforts. The longevity and replicability of the clinic will be tied to achieving positive clinical outcomes in a financially sustainable manner. Our hope is to contribute positive, measurable outcomes to what limited research there is on the efficacy of digital health app interventions in mental health [30] and their economic endpoints. Overall health savings from improved mental health are well documented but ensuring that the evaluation timeframe of the digital clinic is suitable to realize and prove these outcomes will be important.
Transparent data sharing is critical for the success of the digital clinic. As depicted in Fig. 1, Fig. 2, all data captured by mindLAMP is directly shared and made available to the patient. Capturing quality improvement metrics that can be used to ensure treatment is effective and helping patients realize their goals is also important to share. Below in Fig. 3 we offer an example our team's internal clinical dashboard, showing 3 recent patients seen on a certain day. The digital clinic lends itself to easy quality metrics reporting and tracking, which facilitates ongoing quality improvement.
Fig. 3.
Quality improvement dashboards are accessible to the clinical team. Because of the custom nature of each patient's care, no two dashboards will be collecting the exact same types of outcomes data. Thus, as outlined in Table 2, it is important to have a standardized battery to ensure digital clinic metrics and overall outcomes can be tracked.
2.6. Clinical sessions
Session specific adjustments based on unique program structures are needed in order to provide optimal care, support, and outcomes. However, standard clinical goals for the first several sessions are consistent with clinical assessments at the digital clinic. These include rapport building, obtaining relevant biopsychosocial information, identifying treatment goals, reviewing confidentiality practices, and outlining a flexible treatment trajectory. At the end of the initial meeting, patients meet with the Digital Navigator briefly connect a face to the name of their technology point-person. If the patient requires extensive technology training and assistance, the patient and Digital Navigator can schedule a digital skilling training session independent of the next clinical meeting. The time it takes each patient and their clinical team customize mindLAMP varies. However, since tech support and setup is offered outside of clinic sessions, and the clinical care can continue treatment without utilizing technology, any additional time patients need adjusting mindLAMP does not impede on care delivery. Because technology is customized to meet the clinical needs of each patient, relevant data is reviewed at the beginning of each session as a means to inform conversations and shared decision making. The course of clinical appointments evolves and depends on the clinical situation and needs at hand, not unlike therapy in a traditional clinical context. At the end of the visit, the clinician and patient can invite the Digital Navigator into the session and work with them to adjust any app customizations or features relevant for care between sessions.
2.7. Logic model
Program logic models are generally used as a visualization tool to outline how a program framework will be executed overtime. These models include program theories, assumptions, activities, and outcomes [31]. In the context of the broad implementation model for the digital clinic, a more detailed implementation guide is was created and is depicted in Fig. 4 .
Fig. 4.
The logic model details the foundation, objectives, and long term goals of the digital clinic.
3. Discussion
The strengths of the digital clinic are best viewed in the context of barriers which have historically impeded the impact of digital technologies in care: comfort with technology among both clinicians and patients, workflow integration, and support of shared decision making and therapeutic alliance. By offering digital clinic specific training in the form of DOORS and Digital Navigator courses, our model focuses on ensuring that anyone using technology is able to utilize it to their benefit. The use of technology in the clinic does not replace face to face care or telehealth but rather supplements and extends it through customization of relevant data capture, interventions, and education. Recognizing that such customization takes time, the role of a Digital Navigator is critical. The Digital Navigator can be the clinician themselves or a new care team member that can to support many clinicians and their patients at once. The use of technology, like mindLAMP highlighted in our examples, focuses less on its independent functionalities and instead on how it can be adapted and customized to support each patient and clinician dyad.
The model of the digital clinic introduced in this paper is successful to date and we hope that other implementations in different contexts and distinct settings will be equally successful. A focus on pre-conditions will identify the relevant challenges and solutions that require consideration before implementation. At some sites, it may not be feasible to support all aspects of digital care such as real time messaging because of safety or legal concerns. Different technologies and digital platforms may be more beneficial at different sites. While we offer mindLAMP as free software, we recognize that no one piece of software can meet all needs across all use cases. It is possible to imagine many new offerings and roles of such software beyond those identified in Fig. 1. The digital clinic model embraces the fact that software will continue to evolve and change. As it does, relevant training and clinical use cases must too.
The nascency of the digital clinic raises questions around its ongoing maintenance and sustainability. The Digital Navigator is a new care team member and expense to account for. While it is possible that future technologies will be able to use artificial intelligence to automatically customize and meet the demands of each patient without human support, that is not the reality today. mindLAMP is a key component of our realization of the digital clinic and as such, the Digital Navigator develops deep expertise related to supporting it. In a different configuration of the digital clinic, the Digital Navigator may instead focus on recommending different apps and creating a unique toolkit for each patient. The role of the Digital Navigator fits into many population health and integrated care models that support care coordination.
Although the digital clinic provides a promising model for mental health care treatment, the model alone cannot solve larger structural problems that may impede treatment such as mental health staff shortages [32], insurance coverage [33], culturally and ethnically appropriate care options [34], and societal and cultural stigmas related to mental illness [35,36]. However, the digital clinic will evolve in step with technology and overtime may help mitigate these long-standing barriers in treating mental illness. The digital literacy training that is part of the digital clinic model offers a practical first step towards ensuring this type of care evolves in a manner that reduces health inequities. As technologies such as smartphones become increasingly affordable, it is critical that a new digital divide rooted in ability to effectively utilize and access digital care does not evolve.
The digital clinic offers a unique design to improve treatment outcomes without taxing an already burdened mental health care system. In recent years numerous studies have shown an increase in access and use of mobile technology within underserved populations [[37], [38], [39]]. This new model for delivering care draws on access to mobile technology to increase access to mental health care. The digital clinic has the potential to reach and engage marginalized populations and achieve clinical outcomes that meet needs of local clinical and cultural demands. Through synthesizing digital technology and clinical workflow, the digital clinic can increase access to and also quality and specificity of care.
The following are the supplementary data related to this article.
Technical details of mindLAMP
Digital Opportunities for Outcomes in Recovery Services (DOORS) Manual for Teaching Digital Health Literacy
Digital Navigator Training Details
Acknowledgments
Acknowledgement
The authors would like to thank Honor Hsin PHD MD (Kaiser Permanente) for guidance and feedback throughout this manuscript. JT is supported by a career development award from the NIMH: 1K23MH116130-03.
Declaration of competing interest
JT reports unrelated research support from Otsuka.
References
- 1.Patel V., Maj M., Flisher A.J. Reducing the treatment gap for mental disorders: a WPA survey. World Psychiatry. 2010;9(3):169–176. doi: 10.1002/j.2051-5545.2010.tb00305.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Aboujaoude E., Gega L., Parish M.B., Hilty D.M. Digital interventions in mental health: current status and future directions. Front Psych. 2020:11. doi: 10.3389/fpsyt.2020.00111. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Strudwick G., Zhang T., Inglis F., Sockalingam S., Munnery M., Lo B. Delivery of compassionate mental health care in a digital technology-driven age: protocol for a scoping review. BMJ Open. 2019;9(7):e027989. doi: 10.1136/bmjopen-2018-027989. Jul 1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Mohr D.C., Weingardt K.R., Reddy M., Schueller S.M. Three problems with current digital mental health research… and three things we can do about them. Psychiatr Serv. 2017;68(5):427–429. doi: 10.1176/appi.ps.201600541. May 1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Vaidyam A., Halamka J., Torous J. Actionable digital phenotyping: a framework for the delivery of just-in-time and longitudinal interventions in clinical healthcare. mHealth. 2019;5 doi: 10.21037/mhealth.2019.07.04. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Gega L., Marks I., Mataix-Cols D. Computer-aided CBT self-help for anxiety and depressive disorders: experience of a London clinic and future directions. J Clin Psychol. 2004;60(2):147–157. doi: 10.1002/jclp.10241. [DOI] [PubMed] [Google Scholar]
- 7.Lipschitz J., Miller C.J., Hogan T.P. Adoption of mobile apps for depression and anxiety: cross-sectional survey study on patient interest and barriers to engagement. JMIR Ment Health. 2019;6(1) doi: 10.2196/11334. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.de Alva FEM, Wadley G, Lederman R. It feels different from real life: users' opinions of mobile applications for mental health. In: Proceedings of the annual meeting of the Australian special interest group for computer human interaction on - OzCHI '15. Parkville, VIC, Australia: ACM Press; vol. 2015:598–602. doi: 10.1145/2838739.2838806. [DOI]
- 9.Pratap A., Neto E.C., Snyder P. Indicators of retention in remote digital health studies: a cross-study evaluation of 100,000 participants. npj Digit Med. 2020;3(1):21. doi: 10.1038/s41746-020-0224-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Baumel A., Muench F., Edan S., Kane J.M. Objective user engagement with mental health apps: systematic search and panel-based usage analysis. J Med Internet Res. 2019;21(9) doi: 10.2196/14567. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Sandoval L.R., Torous J., Keshavan M.S. Smartphones for smarter care? Self-management in schizophrenia. AJP. 2017;174(8):725–728. doi: 10.1176/appi.ajp.2017.16090990. [DOI] [PubMed] [Google Scholar]
- 12.Uscher-Pines Suddenly becoming a “virtual doctor:” experience is a psychiatrist transitioning to telemedicine during the COVID-19 pandemic. Psychiatr Serv. 2020 doi: 10.1176/appi.ps.202000250. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Jacob C., Sanchez-Vazquez A., Ivory C. Social, organizational, and technological factors impacting Clinicians’ adoption of Mobile health tools: systematic literature review. JMIR Mhealth Uhealth. 2020;8(2) doi: 10.2196/15935. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Gagnon M.-P., Ngangue P., Payne-Gagnon J., Desmartis M. M-health adoption by healthcare professionals: a systematic review. J Am Med Inform Assoc. 2016;23(1):212–220. doi: 10.1093/jamia/ocv052. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Brewster L., Mountain G., Wessels B., Kelly C., Hawley M. Factors affecting front line staff acceptance of telehealth technologies: a mixed-method systematic review. J Adv Nurs. 2014;70(1):21–33. doi: 10.1111/jan.12196. [DOI] [PubMed] [Google Scholar]
- 16.Greenhalgh T., Wherton J., Papoutsi C. Beyond adoption: a new framework for theorizing and evaluating nonadoption, abandonment, and challenges to the scale-up, spread, and sustainability of health and care technologies. J Med Internet Res. 2017;19(11):e367. doi: 10.2196/jmir.8775. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Barry M.J., Edgman-Levitan S. Shared decision making — the pinnacle of patient-centered care. N Engl J Med. 2012;366(9):780–781. doi: 10.1056/NEJMp1109283. [DOI] [PubMed] [Google Scholar]
- 18.Torous J., Hsin H. Empowering the digital therapeutic relationship: virtual clinics for digital health interventions. npj Digital Med. 2018;1(1):16. doi: 10.1038/s41746-018-0028-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Henson P., Wisniewski H., Hollis C., Keshavan M., Torous J. Digital mental health apps and the therapeutic alliance: initial review. BJPsych open. 2019;5(1):e15. doi: 10.1192/bjo.2018.86. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Torous J., Vaidyam A. Multiple uses of app instead of using multiple apps – a case for rethinking the digital health technology toolbox. Epidemiol Psychiatr Sci. 2020;29:e100. doi: 10.1017/S2045796020000013. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Torous J., Wisniewski H., Bird B. Creating a digital health smartphone app and digital Phenotyping platform for mental health and diverse healthcare needs: an interdisciplinary and collaborative approach. Journal of Technology in Behavioral Science. 2019;4(2):73–85. doi: 10.1007/s4. [DOI] [Google Scholar]
- 22.Kilbourne A.M., Neumann M.S., Pincus H.A., Bauer M.S., Stall R. Implementing evidence-based interventions in health care: application of the replicating effective programs framework. Implementation Sci. 2007;2(1):42. doi: 10.1186/1748-5908-2-42. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Henson P., David G., Albright K., Torous J. Deriving a practical framework for the evaluation of health apps. The Lancet Digital Health. 2019;1(2):e52–e54. doi: 10.1016/S2589-7500(19)30013-5. Jun 1. [DOI] [PubMed] [Google Scholar]
- 24.Hoffman L., Camacho E., Rodriguez-Villa E., Vaidyam A., Henson P., Wisniewski H. Division of Digital Psychiatry, Beth Israel Deaconess Medical Center; Boston, Ma: 2019. Digital opportunities for outcomes in recovery services: group facilitator manual, level 1. [Google Scholar]
- 25.Pew Research Center: Internet, science & tech. Demographics of mobile device owernship and adoption in the United States. https://www.pewresearch.org/internet/fact-sheet/mobile/ Published June 12, 2019. Accessed February 2020.
- 26.Lattie E.G., Nicholas J., Knapp A.A., Skerl J.J., Kaiser S.M., Mohr D.C. Opportunities for and tensions surrounding the use of technology-enabled mental health services in community mental health care. Adm Policy Ment Health. 2020;47(1):138–149. doi: 10.1007/s10488-019-00979-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Berwick D.M., Nolan T.W., Whittington J. The triple aim: care, health, and cost. Health Aff. 2008;27(3):759–769. doi: 10.1377/hlthaff.27.3.759. [DOI] [PubMed] [Google Scholar]
- 28.Bordin E.S. Theory and research on the therapeutic working alliance: new directions. In: Hovarth A.O., Greenberg L.S., editors. The working alliance: theory, research, and practice. John Wiley & Sons; 1994. pp. 13–37. [Google Scholar]
- 29.Wisniewski H., Torous J. Digital navigators to implement smartphone and digital tools in care. Acta Psychiatr Scand. January 2020:acps.13149. doi: 10.1111/acps.13149. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Powell A.C., Torous J.B., Firth J., Kaufman K.R. Generating value with mental health apps. BJPsych open. 2020;6(2):e16. doi: 10.1192/bjo.2019.98. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.W.K. Kellogg Foundation Logic Model Development Guide. W.K. Kellogg Foundation. https://www.who.int/whr/2001/en/. Published 2004. Accessed February 2020.
- 32.Thomas K.C., Ellis A.R., Konrad T.R., Holzer C.E., Morrissey J.P. County-level estimates of mental health professional shortage in the United States. PS. 2009;60(10):1323–1328. doi: 10.1176/ps.2009.60.10.1323. [DOI] [PubMed] [Google Scholar]
- 33.Substance Abuse and Mental Health Services Administration . Substance Abuse and Mental Health Services Administration; Rockville, MD: 2015. Racial/ethnic differences in mental health service use among adults. HHS publication no. SMA-15-4906. [Google Scholar]
- 34.Alegría M., Canino G., Ríos R. Mental health care for Latinos: inequalities in use of specialty mental health services among Latinos, African Americans, and non-Latino Whites. PS. 2002;53(12):1547–1555. doi: 10.1176/appi.ps.53.12.1547. [DOI] [PubMed] [Google Scholar]
- 35.The World Health Report 2001 - Mental health: new understanding, new hope. World Health Organization. https://www.who.int/whr/2001/en/. Published 2001. Accessed February 2020.
- 36.Mackenzie C.S., Erickson J., Deane F.P., Wright M. Changes in attitudes toward seeking mental health services: a 40-year cross-temporal meta-analysis. Clin Psychol Rev. 2014;34(2):99–106. doi: 10.1016/j.cpr.2013.12.001. [DOI] [PubMed] [Google Scholar]
- 37.Firth J., Cotter J., Torous J., Bucci S., Firth J.A., Yung A.R. Mobile phone ownership and endorsement of “mhealth” among people with psychosis: a meta-analysis of cross-sectional studies. SCHBUL. 2016;42(2):448–455. doi: 10.1093/schbul/sbv132. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Ray R., Sewell A.A., Gilbert K.L., Roberts J.D. Missed opportunity? Leveraging mobile technology to reduce racial health disparities. J Health Polit Policy Law. 2017;42(5):901–924. doi: 10.1215/03616878-3940477. [DOI] [PubMed] [Google Scholar]
- 39.Lopez MH., Gonzalez-Barrera A., Patten E., Closing the digital divide: Latinos and technology adoption. Pew Hispanic Center. https://assets.pewresearch.org/wp-content/uploads/sites/7/2013/03/Latinos_Social_Media_and_Mobile_Tech_03-2013_final.pdf Published March 2, 2013. Accessed February 2020.
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Technical details of mindLAMP
Digital Opportunities for Outcomes in Recovery Services (DOORS) Manual for Teaching Digital Health Literacy
Digital Navigator Training Details