Abstract
Background
Telemental health (delivering mental health care via video calls, telephone calls, or SMS text messages) is becoming increasingly widespread. Telemental health appears to be useful and effective in providing care to some service users in some settings, especially during an emergency restricting face-to-face contact, such as the COVID-19 pandemic. However, important limitations have been reported, and telemental health implementation risks the reinforcement of pre-existing inequalities in service provision. If it is to be widely incorporated into routine care, a clear understanding is needed of when and for whom it is an acceptable and effective approach and when face-to-face care is needed.
Objective
This rapid realist review aims to develop a theory about which telemental health approaches work (or do not work), for whom, in which contexts, and through what mechanisms.
Methods
Rapid realist reviewing involves synthesizing relevant evidence and stakeholder expertise to allow timely development of context-mechanism-outcome (CMO) configurations in areas where evidence is urgently needed to inform policy and practice. The CMO configurations encapsulate theories about what works for whom and by what mechanisms. Sources included eligible papers from 2 previous systematic reviews conducted by our team on telemental health; an updated search using the strategy from these reviews; a call for relevant evidence, including “gray literature,” to the public and key experts; and website searches of relevant voluntary and statutory organizations. CMO configurations formulated from these sources were iteratively refined, including through discussions with an expert reference group, including researchers with relevant lived experience and frontline clinicians, and consultation with experts focused on three priority groups: children and young people, users of inpatient and crisis care services, and digitally excluded groups.
Results
A total of 108 scientific and gray literature sources were included. From our initial CMO configurations, we derived 30 overarching CMO configurations within four domains: connecting effectively; flexibility and personalization; safety, privacy, and confidentiality; and therapeutic quality and relationship. Reports and stakeholder input emphasized the importance of personal choice, privacy and safety, and therapeutic relationships in telemental health care. The review also identified particular service users likely to be disadvantaged by telemental health implementation and a need to ensure that face-to-face care of equivalent timeliness remains available. Mechanisms underlying the successful and unsuccessful application of telemental health are discussed.
Conclusions
Service user choice, privacy and safety, the ability to connect effectively, and fostering strong therapeutic relationships need to be prioritized in delivering telemental health care. Guidelines and strategies coproduced with service users and frontline staff are needed to optimize telemental health implementation in real-world settings.
Trial Registration
International Prospective Register of Systematic Reviews (PROSPERO); CRD42021260910; https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42021260910
Keywords: telemental health, remote care, telemedicine, mental health, COVID-19, digital exclusion, realist review, telemedicine, virtual care, rapid realist review, gray literature, therapy, health care staff, digital consultation, frontline staff, children, inpatient, mobile phone
Introduction
Background
Telehealth is defined as “the delivery of health-related services and information via telecommunications technologies in the support of patient care, administrative activities, and health education” [1]. Telemental health refers to such approaches within mental health care settings. It can include care delivered by means such as SMS text messaging and chat functions but most commonly refers to telephone calls and video calls, which are central to telemental health care.
Before the COVID-19 pandemic, there was interest in many countries and settings in integrating new technologies, including telemental health approaches, more widely and effectively in mental health care services. This was of particular interest in countries where face-to-face (ie, in-person) mental health care was largely inaccessible to remote communities [2]. Research has demonstrated that telemental health can be successful in various contexts, although studies before the pandemic tended to relate to relatively small-scale and well-planned applications of telemental health with volunteer participants, rather than large-scale implementations across whole service systems. Telemental health has been found to be effective in reducing treatment gaps and improving access to mental health care for some service users [3-5]. This includes those who live far from services or where caring responsibilities affect their ability to travel [6-8]. Positive outcomes and experiences have been reported across a range of populations (including adult, child and adolescent, older people, and ethnic minority groups) and settings (including hospital, primary care, and community) [9-11]. Some evidence has suggested that telemental health modalities such as videoconferencing are equivalent to, or even better than, face-to-face modalities for some service users in terms of quality of care, reliability of clinical assessments, treatment outcomes, or adherence [9,10,12,13]. High levels of service user acceptance and satisfaction with telemental health services have also been reported in research samples [4] and for certain populations, including those with physical mobility difficulties, social anxiety, or severe anxiety disorders [6,7]. However, conversely, telemental health services are not appropriate for or favored by all service users, and there is no one-size-fits-all approach. In particular, service users experiencing social and economic disadvantages, cognitive difficulties, auditory or visual impairments, or severe mental health problems (such as psychosis) have benefited less from telemental health interventions [14,15]. Digitally excluded service users tend to be people who are already experiencing other forms of disadvantage and are already at risk of poorer access to services and less good quality care; thus, a switch to telemental health may exacerbate existing inequalities [16,17]. In addition, concerns have been raised around the impacts of telemental health on privacy and confidentiality of clinical contacts, especially for the many service users who do not have the appropriate space and facilities for its use, as well as its appropriateness for certain purposes, such as conducting assessments or risk management [14].
Encouraging evidence of telemental health acceptability and effectiveness from prepandemic research tended to relate to limited populations who had opted into well-planned remote services [18]. However, during the COVID-19 pandemic, the use of telemental health around the world greatly accelerated, and telemental health became a routine approach for maintaining and delivering mental health services. Telemental health initiatives were central to delivering mental health services in the context of this emergency. Technological initiatives have also helped to address social isolation, which worsened throughout the pandemic [6,19]. In the United Kingdom, there were large increases in remote consultations in National Health Service (NHS) primary care [20], and national data reported that most contacts in NHS mental health settings were delivered remotely in 2020 [21], particularly during the first UK lockdown (March to July 2020).
Following the rapid adoption of telemental health at the start of the crisis, service planners, clinicians, and service users have expressed interest in the greater use of telemental health in the long term [14,19,22,23]. However, several challenges have been identified as arising from this widespread implementation [14,16,19,24,25]. These include (1) reaching digitally excluded populations, who may, for example, have limited technological access or expertise or both, thus compounding existing inequalities experienced by disadvantaged groups; (2) a lack of staff competence in using telemental health devices and confidence in delivering telemental health care; (3) a lack of technological infrastructure within health services; (4) challenges in managing clinical and technological risks in remotely delivered care; (5) developing and maintaining strong therapeutic relationships online, especially when the first contact is remote rather than face-to-face; (6) maintaining service user safety and privacy; and (7) delivering high-quality mental health assessments without being able to see or speak to the service user face-to-face. It is also more difficult to undertake physical assessments, including for physical signs linked to mental health, and side effect monitoring.
Both for future emergency responses and to establish a basis for the integration of telemental health into routine service delivery (where appropriate) beyond the pandemic, evidence is needed on how to optimize telemental health care, given the unique relational challenges associated with mental health care, and identify what works best and for whom in telemental health care delivery and in which contexts. It is also important to identify contexts in which telemental health is unlikely to be safe and effective, where face-to-face delivery should remain the default.
A methodological approach developed to address questions of which interventions work, for whom, and in which contexts in a timely way is the rapid realist review (RRR) [26]. This methodology has been developed to rapidly produce policy-relevant and actionable recommendations through a synthesis of peer-reviewed evidence and stakeholder consultation. A key characteristic of realist methodology is the focus on interactions between contextual factors (eg, a certain population, geographical location, service setting, or situation) and relevant mechanisms (eg, behavioral reactions, participants’ reasoning, or resources), which affect the outcomes of interest, such as intervention adherence or service user satisfaction [26-28]. Together, these are used to develop context-mechanism-outcome (CMO) configurations, which comprise the fundamental building blocks of realist synthesis approaches. Evidence from the wider literature is also drawn upon to develop midrange theories. Midrange theories are program theories that aim to describe how certain mechanisms in specific contexts result in specific outcomes [29]; the use of the wider literature to develop midrange theories helps to elaborate and refine the developed CMO configurations by shedding further light on how their mechanisms operate [26,30-32]. Additional information on realist terminology can be found in Multimedia Appendix 1.
Objective
This is a unique opportunity to establish the characteristics of high-quality telemental health services and use these findings to identify key mechanisms for acceptable, effective, and efficient integration of telemental health services into routine mental health care. Using a realist methodology, in this RRR, we aimed to answer the question of what telemental health approaches work, for whom, in which contexts, and how? Specifically, we investigated the following questions in this review:
What factors or interventions improve or reduce adoption, reach, quality, acceptability, or other relevant outcomes in the use of telemental health in any setting?
Which approaches to telemental health work best for which staff and service users in which contexts?
In what contexts are phone calls, video calls, or SMS text messages preferable, and in which contexts should mental health care be delivered face-to-face instead?
We focus particularly on groups and contexts identified as high priority by policy makers (the process is described in detail in the Methods section), including children and young people, crisis care and inpatient settings, and groups at high risk of digital exclusion; examples from these groups are included wherever possible.
Methods
Overview
The RRR was conducted by the National Institute for Health Research Mental Health Policy Research Unit (MHPRU), a team established to deliver evidence rapidly to inform policy making, especially by the Department of Health and Social Care in England, associated government departments, and NHS policy leadership bodies. The project constitutes the final stage in a program of work on telemental health delivery conducted to meet urgent policy needs, which included an umbrella review of pre–COVID-19 evidence [18], a qualitative investigation of service user experiences of telemental health [24], a systematic review of literature on telemental health adoption conducted during the early phase of the pandemic [14], and a systematic review on the cost-effectiveness of telemental health approaches (personal communication by Clark et al, 2022). This RRR was registered on PROSPERO (International Prospective Register of Systematic Reviews; CRD42021260910).
We conducted the RRR during the COVID-19 pandemic, with videoconferencing via Zoom (Zoom Video Communications) as the primary means of communication among the research team.
Study Design
An expert reference group of 28 people, including 16 (57%) university-employed academics, 7 (25%) experts by experience (lived experience researchers from the MHPRU Lived Experience Working Group (LEWG) with personal experiences of using mental health services or supporting others or both), and 8 (29%) experts by profession (including frontline clinicians), guided and contributed to the RRR throughout. Some members belonged to multiple groups and therefore worked from several (academic, clinical, or lived experience or multiple) perspectives.
The group met weekly throughout this process from July to November 2021. The expert reference group meetings served to develop and refine the study protocol; plan the searches for evidence (particularly the targeted additional searches supplementing the initially planned strategy); iteratively examine, refine, and validate the CMO configurations derived from our evidence synthesis, with reference to their expertise by experience or profession or both; and plan wider consultation on our emerging findings. Members of the expert reference group also contributed to the literature searches, data extraction, synthesis, and interpretation of data.
The stages of our RRR were based on the following five steps, variations of which have been described and used in previous studies [26,31,32]:
Developing and refining research questions
Literature searching and retrieving information (data/stakeholder views)
Screening and extracting information/data
Synthesizing information/data
Interpreting information/data
Our approach to these steps was iterative rather than linear, particularly for steps 3, 4, and 5, where there were multiple phases of extraction, synthesis, and interpretation. This is described in detail in the following section.
Developing and Refining the Research Question
We formulated the research question in response to policy maker needs. We reviewed findings from earlier stages of the MHPRU’s program on COVID-19 impact on mental health care and on telemental health [6,18,19,24,33] with policy makers, including senior officials and mental health teams in the Department for Health and Social Care, NHS England, and Public Health England. We then identified questions to be addressed from their perspective to plan for future implementation and delivery of telemental health. This included considering how best to incorporate telemental health in routine practice once the need for its emergency deployment passes. Early in these discussions, three priority groups regarding the evidence that is currently lacking were identified as especially important for policy and planning: children and young people, users of inpatient and crisis care services, and digitally excluded groups. Digitally excluded groups include those who have no or reduced access to the digital world because of a lack of digital skills (eg, using computers or smartphones), connectivity (eg, access to the internet or phone signal), and accessibility (eg, those who may require assistive technology or do not have access to digital devices or connectivity because of, eg, costs). Our primary question of which telemental health approaches work, for whom, and in what context originated in these discussions and was further refined by the MHPRU core research team who identified a RRR methodology as appropriate and further refined the primary and secondary questions and methodology with the expert reference group before registering the protocol.
Selection Criteria
Sources were included if they met the criteria described in the following sections.
Participants
Participants included staff working in the field of mental health, people receiving care from mental health services, family members, and other supporters of people receiving mental health care.
Interventions
Interventions included any form of remote (spoken or written) communication between mental health professionals, or among mental health professionals and service users, family members, and other supporters, using video calls, telephone calls, SMS text messaging services, or hybrid approaches combining face-to-face and web-based modalities. Peer support communications were also included alongside any strategies or training programs to support the implementation of the abovementioned interventions. Self-guided web-based support and therapy programs were excluded.
Types of Evidence
This included qualitative or quantitative evidence on (1) what improves or reduces adoption, reach, quality, acceptability, or clinical outcomes in the use of telemental health; (2) impacts of introducing interventions or strategies intended to improve adoption, reach, quality, acceptability, or clinical outcomes; (3) interventions or strategies intended to help mental health staff make more effective use of telemental health technologies; (4) impacts of telemental health on specific service user groups and settings, including people who are digitally excluded, users of inpatient and crisis care services, and children and young people; and (5) the appropriateness of the use of telemental health versus face-to-face care in particular contexts. In addition to outcomes, sources were required to include information on mechanisms (ie, what works, for whom, and how).
Study Design
This included any qualitative, quantitative, or mixed methods study design, including relevant service evaluations, audits, and case series. Gray literature and other sources, such as websites, stakeholder feedback, and testimonies from provider organizations and service user and carer groups were also included. Sources were also included if the focus was not solely on remote working, but the results contained substantial data relevant to our research questions. Substantial data had to provide relevant and sufficient information on context, mechanisms, and outcomes and contribute to the development of overarching CMO configurations. Editorials, commentaries, letters, conference abstracts, and theoretical studies were excluded.
Search Strategy
The search strategy was in accordance with PRISMA (Preferred Reporting Item for Systematic Reviews and Meta-Analyses) guidelines (Figure 1) [34]. Resources and literature were identified through the sources described in the following paragraphs.
Figure 1.
PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) diagram. *Inpatient and crisis service users, children and young people, people who are digitally excluded, and items from a service user viewpoint [14,18]. CMO: context-mechanism-outcome.
First, we screened peer-reviewed studies included in 2 previous reviews on telemental health conducted by the MHPRU. The umbrella review by Barnett et al [18] included systematic reviews, realist reviews, and qualitative meta-syntheses on remote working before the onset of the COVID-19 pandemic [18]. The systematic review by Appleton et al [14] synthesized primary research on the adoption and impacts of telemental health approaches during the pandemic [14]. An updated search of the latter review was conducted on May 19, 2021.
Second, we worked with our expert reference group to identify additional peer-reviewed and gray literature. Searches were conducted on the websites of relevant national and international voluntary and statutory organizations identified by the expert reference group and by internet searches (eg, Mind and the Royal College of Psychiatrists). Identified literature was noted on a shared Microsoft Excel spreadsheet. This process was supported using Slack, a web-based messaging application, to coordinate this complex and rapidly changing process.
Finally, the MHPRU disseminated a call for evidence via Twitter and email to relevant organizations and individuals (such as charities supporting digital inclusion, chief information officers, and telehealth leads within NHS Trusts) inviting them to submit relevant evidence, including evaluations, audits, surveys, stakeholder feedback, and testimonies from provider organizations and service user and carer groups.
Study Selection
References included in the umbrella and systematic review were downloaded and screened for inclusion in the RRR using EPPI-Reviewer (version 4.0) [35]. One of the reviewers screened abstracts and titles of the references identified through the updated searches of the umbrella review and systematic review. Full texts were reviewed for inclusion, with included and unsure sources checked by another reviewer. Sources were included in the final review if they met our inclusion criteria and provided relevant information for the development of CMO configurations. Disagreements were resolved through discussion with the wider research team.
Process of Data Extraction, Synthesis, and Interpretation
Data Extraction
The source characteristics were extracted and inputted on EPPI-Reviewer (version 4.0) [35]. Extracted characteristics included study aim and design (if applicable), type of service, telemental health modalities used, mental health diagnosis of service users, and staff occupation. MHPRU researchers screened each included source for information that could be assembled into CMO configurations related to telemental health (ie, information on contexts, outcomes, and underlying mechanisms). Underlying CMO configurations were extracted by MHPRU researchers and LEWG members. Underlying CMO extraction involved reading each source before identifying contexts, mechanisms, and outcome configurations that were either (1) identified by the authors in the paper or (2) identified by extractors by linking them together from the data, descriptions, and discussion in the paper. Each week, samples of the extracted CMO configurations were reviewed by the expert reference group to ensure coherence, relevance, validity, and format consistency.
Data Synthesis
The research team then began the process of synthesizing the underlying CMO configurations by reviewing the extracted CMO configurations and identifying emerging themes. We developed four domains to encapsulate key aspects of the evidence: (1) connecting effectively; (2) flexibility and personalization; (3) safety, privacy, and confidentiality; and (4) therapeutic quality and relationship. Each of the 4 identified domains was allocated to an MHPRU researcher to lead the synthesis, with input from LEWG members, clinicians, and MHPRU senior researchers.
To develop content for each of these 4 domains, underlying CMO configurations were, in essence, synthesized based on similarities to create a single overarching CMO (discussed in full detail in Multimedia Appendix 1). Underlying CMO configurations extracted from individual sources were reviewed in terms of their similarities and differences (eg, CMO configurations related to the convenience of telemental health) and then grouped together based on similar mechanisms (eg, flexibility, which reduces practical barriers to accessing mental health care) and outcomes (eg, increasing attendance and reducing missed appointments). Each similar group of underlying CMO configurations was then synthesized and refined to create a single overarching CMO, which reflected key content across the underlying CMO configurations, as well as input from the expert reference group. Each overarching CMO was assigned to 1 of the 4 domains.
Realist work does not conduct a traditional quality appraisal, as it values evidence from all sources in a nonhierarchical manner [27,29]. Our overarching CMO configurations were also significantly developed throughout the synthesis process based on stakeholder input, and thus, quality appraisal of individual study sources would not have reflected the quality of the final overarching CMO configurations.
Interpretation
An iterative process of revising and refining overarching CMO configurations from the perspective of stakeholder experience followed. Revisions, refinements, and additions were first made through discussion with the expert reference group. Summaries were then also discussed at three 2-hour stakeholder webinars, each focusing on one of our priority groups: children and young people, inpatient and crisis care services, and digitally excluded groups. The webinars were primarily attended by groups representing these constituencies and services who work with them, including experts and stakeholder representatives from research, policy, and clinical settings (nationally and internationally); the voluntary sector; lived experience groups and community organizations working with marginalized groups; and telehealth technology initiatives. There were between 30 and 40 participants at each webinar. During the webinars, participants were divided into breakout rooms, with a facilitator and a note taker from the core research team. High-level summaries of preliminary data were presented by domain, and attendees were asked to discuss the following questions: (1) whether the preliminary summaries captured their own knowledge and experience of telemental health, (2) whether the summaries applied to and how the summaries were relevant for the priority group at hand, and whether they were aware of any additional challenges or recommendations related to delivering telemental health to the priority group.
On the basis of the feedback from these webinars, the overarching CMO configurations within each of the 4 domains were then further revised and refined. We actively sought additional information related to each overarching CMO, including relevant contexts, further detail about mechanisms, real-life examples of strategies and solutions (such as for overcoming barriers identified within the CMO), and points of particular importance or concern, from the webinar notes, the expert reference group meetings, and related literature. We noted this information alongside the relevant overarching CMO and used it to refine the CMO configurations. In addition, we drew upon midrange theories (evidence-based theories derived from the wider literature) to provide more theoretically informed explanations of mechanisms (eg, the digital inverse care law [16], which theorizes that those most in need of care via telemental health are least likely to engage with it and existing inequalities will widen, helped to strengthen the mechanisms around digital exclusion). Throughout this process, the core research team and the expert reference group were iteratively consulted, and their feedback was integrated into the overarching CMO configurations. The revised theories were shared for a final email consultation with the stakeholders who were invited to our webinars. Their feedback was incorporated and resulted in the final overarching CMO models presented under each domain in this paper.
Results
Overview
Underlying CMO configurations were extracted from 17 studies included in the previous umbrella review [18] and from 45 studies included in the systematic review [14]. The updated search yielded 44 potentially relevant studies, of which 21 (48%) were excluded (either as they did not contain data on context, mechanisms, and outcomes or they added no additional information as data saturation had been reached). CMO configurations were extracted from 52% (12/23) of the remaining studies that met our inclusion criteria and were included in the realist synthesis. Through consultations with our expert reference group, we identified 28 sources, of which 16 (57%) met our inclusion criteria and provided additional information related to our priority groups, and 14 (50%) yielded CMO configurations that were included in the synthesis. We received 68 potential sources through the call for evidence, of which 22 (32%) met our inclusion criteria and provided relevant information on our priority groups; CMO configurations were extracted from 13 (19%) of these. Finally, website searches identified 52 potentially relevant sources, of which 13 (25%) met our inclusion criteria and 7 (13%) provided information relevant to CMO configurations. The realist synthesis includes a total of 108 sources.
Of the 108 included sources with primary data or detailed accounts of what works, for whom, and in what context, most were primary research studies (72/108, 66.7%), followed by service descriptions/evaluations/audits (19/108, 17.6%), guidance documents (4/108, 3.7%) and briefing papers (3/108, 2.8%), commentaries/editorials/discussions (4/108, 3.7%), and letters (2/108, 1.9%), as well as a review (1/108, 0.9%), news article (1/108, 0.9%), webpage (1/108, 0.9%), and a service user–led report (1/108, 0.9%). Of the 84 sources that included primary research data, 32 (38%) used quantitative methods, 19 (23%) used qualitative methods, and 33 (39%) used mixed methods (including n=2, 2% case studies).
Most sources were published in the United States (41/108, 38%) and the United Kingdom (34/108, 31.5%). The remaining sources collected data in Canada (7/108, 6.5%), the Dominican Republic (1/108, 0.9%), Australia (7/108, 6.5%), China (2/108, 1.9%), India (3/108, 2.8%), Egypt (1/108, 0.9%), and Nigeria (1/108, 0.9%), as well as 10 European countries, including Austria (1/108, 0.9%), France (1/108, 0.9%), Germany (1/108, 0.9%), Ireland (1/108, 0.9%), Italy (2/108, 1.9%), Netherlands (1/108, 0.9%), Portugal (1/108, 0.9%), Spain (1/108, 0.9%), Sweden (1/108, 0.9%), and Switzerland (1/108, 0.9%). Details of the included sources are presented in Multimedia Appendix 2 [12,19,24,36-153].
Overarching CMO configurations for each domain are summarized in Tables 1-4, and details of the underlying CMO configurations and summary notes on stakeholder discussions, which shaped the overarching CMO configurations, are presented in Multimedia Appendix 3 [12,19,24,36-153]. For each overarching CMO, we included examples of key contexts that are relevant for the CMO and examples of strategies and solutions addressing the challenges or opportunities identified in the CMO: these were drawn from underlying CMO configurations and stakeholder discussions. In the text outlining each domain, we also identify major midrange theories that elucidate mechanisms and outcomes for overarching CMO configurations.
Table 1.
Domain 1: Connecting Effectively.
| CMOa title | References | Overarching CMO | Key contexts | Example strategies and solutions | |
| CMO 1.1: Providing service users with access to digital devices | [19,39-49, 59] | When service users who do not have access to digital devices are given access to up-to-date devices (and chargers), paid for or loaned to them (context), this results in improved access to and implementation of telemental health services (especially via video platforms; outcome 1), some inequalities in accessing a digital device are addressed (outcome 2), exacerbation of existing inequalities is less likely (outcome 3), and service users are more able to maintain personal contact with family and friends if they wish and access a range of web-based services (outcome 4), as this reduces the burden of having to purchase a device for the service users and provides more financially viable access to devices required for web-based connections (mechanism). | Lack of access to devices particularly affects people living in poverty or unstable living circumstances (such as homeless people or refugees), as well as other groups at risk of exclusion not only from telemental health but from a range of services and networks (such as people who are cognitively impaired or with psychosis or substance abuse disorders). It can also particularly affect inpatients, who may not have access to devices or charging facilities or both on the ward, and children and young people, who may not have access to their own devices. |
|
|
| CMO 1.2: Lack of access to stable, secure, and adequate internet connection | |||||
|
|
Staff | [19, 39, 41, 42, 44, 47, 50-60] | When the staff deliver telemental health via video from workplaces or homes with an unstable and poor internet connection (context), teleconsultations are difficult (or impossible) to conduct with service users (outcome 1), fewer teleconsultations are conducted (outcome 2), and telemental health is viewed less positively (outcome 3), as staff experience frustration, and there is reduced motivation to arrange web-based appointments (mechanism). | This is particularly relevant for staff working within health care providers that frequently have insufficient Wi-Fi or internet connection to deliver sessions smoothly. It also affects staff who do not have adequate Wi-Fi connectivity in their own homes or when working in the community. |
|
|
|
Service users | [19, 39, 41, 42, 44, 47, 50-60] | When service users only have access to an insecure, unstable, and poor-quality internet connection or consistent technological problems (context), it is difficult for telemental health to be viewed positively (outcome 1), they are able or willing to accept fewer web-based consultations (outcome 2), they may continue to struggle with their mental health (if face-to-face consultations are unavailable; outcome 3), and this may result in digital exclusion that could exacerbate existing inequalities (outcome 4), as the service users struggle to engage in sessions with sufficient clarity and mutual comprehension and experience frustration (mechanism). | This is especially relevant for service users on low incomes or from socially marginalized groups (such as homeless people), those living in multiple occupancy households where Wi-Fi is overstretched, and people from low-to-middle income countries. Lack of access to reliable internet or even electricity also differentially affects people in rural and remote areas. People in marginalized groups may also lack the means to pay for a telephone service. |
|
| CMO 1.3: Benefits of providing support and guidance for using technology | |||||
|
|
Staff | [19, 24, 38, 42-44, 46, 53, 59, 61-75] | When staff who lack the confidence or knowledge to deliver mental health care online (particularly via video calls) receive practical instruction and guidance on how to use technology to deliver mental health services (including clear information about how to operate within local policies, procedures, and platforms; troubleshoot issues during telemental health sessions; and formulate and implement backup plans; context), they feel an increased sense of confidence in managing and delivering telemental health services (mechanism), which leads to increased use of telemental health services (outcome 1) and fewer delays, resulting in more appointments being completed on time (outcome 2). | This is especially relevant for staff who are new to delivering mental health care remotely or who are unclear or unfamiliar with using locally recommended platforms and procedures. |
|
|
|
Service users | [19, 24, 38, 42-44, 46, 53, 61-75] | When service users with access to a technology device who struggle with the confidence, knowledge, or the ability to use telemental health receive guidance, reassurance, and instruction (tailored to their health care provider and their language, reading ability, and any sensory disability) on how to use technology (particularly video calls) to access mental health care, engage with backup plans, and receive timely technical support and troubleshooting during treatment sessions (context), they feel an increased sense of confidence in accessing telemental health (mechanism), which reduces anxiety in using telemental health and digital technologies (including in their personal lives; outcome 1), facilitates the adoption of and adherence to telemental health (outcome 2), improves service users’ ability to adjust to remote care (outcome 3), reduces interruptions in care delivered via telemental health (outcome 4), and increases satisfaction with telemental health (outcome 5). | Receiving guidance tailored to local policies and procedures for telemental health access is relevant to all service users. It is likely to be particularly relevant to groups identified as at high risk of digital exclusion through lack of confidence in using technology, including older people, people with severe mental health problems, and people with intellectual disabilities or cognitive impairments. Production of accessible guidance is especially relevant for people with cognitive impairments or intellectual disabilities, children, people with sensory disabilities, and people who do not understand English well. |
|
| CMO 1.4: Impact of technology-related disruptions | [36, 45, 56, 76-82] | When technological issues (including connection problems and device issues) lead to disruptions in web-based sessions and there is no prearranged backup method of contact (eg, a plan to connect by telephone instead of video call if needed; context), the quality of the intervention is diminished (outcome 1), there is a loss of empathic connection between client and therapist (outcome 2), and the sessions may not be able to continue (outcome 3), as the flow of the conversation is interrupted and session time reduced, for example, when having to ask the other person to repeat what has been said or when cut off completely, leaving staff and service users potentially feeling distracted, frustrated, awkward, and upset (particularly if there is a threat of therapy withdrawal because of missed sessions; mechanism). | This is particularly relevant for the use of videoconferencing, where phone backup can reduce the risk of abandoning appointments because of failure to make a stable connection. |
|
|
| CMO 1.5: Preparing service users for telemental health | [83-85] | When staff prepare service users for telemental health appointments and communicate clearly with service users about what to expect (context), this leads to more accepted calls and fewer missed service user contacts (outcome), as service users have more relevant knowledge of the process, including when to expect contacts and from what number, and feel more comfortable engaging with telemental health services (mechanism). | This is relevant across telemental health contexts but may be particularly relevant for service users who have not used telemental health before or are anxious or worried about telemental health. |
|
|
| CMO 1.6: Service users’ familiarity with the platform and ease of use | [41,66,86-90] | Where service users already use remote technologies for social, educational, or work purposes, or where the web-based platforms are relatively easy to use (context), offering a choice of familiar and accessible technology platforms that may be less difficult and time-consuming for staff and service users to understand or learn (mechanism) may increase the likelihood of engagement with services via telemental health, especially video calls (outcome). | This is especially relevant to service users who are already making some use of technology, for example, for social, educational, or work purposes. |
|
|
| CMO 1.7: Telemental health may be a better alternative to receiving no care for service users during an emergency, such as the COVID-19 pandemic | [36, 43, 60, 90-92] | When service users are offered telemental health appointments as face-to-face appointments are restricted (eg, because of COVID-19 or another emergency; context); these appointments are likely to be accepted by some service users on the basis that they are the main way by which mental health care can continue (outcome 1), and there is a reduced risk of infection from COVID-19 (outcome 2), as telemental health is seen as preferable to receiving no support at all and as an alternative to canceling appointments entirely (mechanism). | This is applicable across mental health services in the context of any emergency which restricts face-to-face meetings, especially in services where all face-to-face contacts have been discontinued or where they are limited to immediate crises. It is especially relevant for service users who need to self-isolate because of high personal risk, or where clinicians need to self-isolate after contact with others. |
|
|
aCMO: context-mechanism-outcome.
Table 4.
Domain 4: Therapeutic quality and relationship.
| CMOa title | References | Overarching CMO | Key contexts | Example strategies and solutions | |
| CMO 4.1: Change in nonverbal cues and informal chat, affecting the therapeutic relationship | [12, 19, 24, 36, 37, 40, 45, 49, 54, 55, 60, 67, 76, 77, 79, 81, 90-92, 96, 103, 137 - 142] | When switching from face-to-face to telemental health care (context), staff and some service users perceived the relationship between staff and service users (and other service user group members) to be negatively affected or found it more difficult to develop a therapeutic relationship (outcome 1) and, thus, were less willing to take up or use telemental health (outcome 2), more likely to be dissatisfied (outcome 3), and viewed care as less effective compared with previously received face-to-face care (outcome 4). This was because they perceived telemental health to be impersonal and found it more difficult to discuss personal information because of a lack of nonverbal feedback, eye contact, and social cues, as well as informal chat before, after, and during sessions (mechanism). | This may be relevant during rapid switches to telemental health because of emergency situations such as the COVID-19 pandemic in which staff training and structured telemental health implementation are limited because of time constraints; for staff with limited training and experience generally and those with limited experience of using telemental health specifically, who may lack the confidence to navigate the change in visual cues, which, in turn, can affect the therapeutic relationship; for staff and service users who are new to a specific service, staff/service user, or to mental health care generally; for service users who are apprehensive of technology use or who are concerned about the violation of their privacy; and in telemental health group sessions in which the flow of conversation is affected or people find it less easy to establish relationships and be at ease with the whole group. |
|
|
| CMO 4.2: Assessment via telemental health versus face-to-face (staff) | [40, 82, 92, 111, 121, 133, 137, 138, 143 - 145] | When using telemental health for assessments (context), staff report finding it more difficult to assess mental health problems, care needs, and risk, and make diagnoses (outcome), as they are less able to observe nonverbal and visual cues (depending on the telemental health modality used), and some service users may find it more difficult to have in-depth conversations about their problems and experiences (mechanism). | Cues can include extrapyramidal symptoms from antipsychotics, hygiene, gait, direct eye contact, mannerism, and linguistic nuances. Conducting assessments might be particularly difficult over the phone because of the lack of visual cues; with service users who experience domestic violence and abuse and thus cannot be honest about their well-being and current situation in the presence of their abuser; with young children; with service users who find it difficult to speak directly about their difficulties and experiences; and when the staff make incorrect assumptions about service users’ mental states based on behavioral indicators and without considering service user reports, especially of neurodivergent service users. |
|
|
| CMO 4.3: Staff support and training | [49, 63, 102, 146] | When staff receive specific instructions and training, for example, on how to build rapport using telemental health and support from colleagues with prior telemental health experience (context), it facilitates quality of care (outcome 1), building therapeutic relationships (outcome 2), and increased engagement (outcome 3), as staff are able to ask questions and acquire new skills and knowledge about the interventions and thus build confidence in delivering telemental health (mechanism). | This is likely to be especially relevant to staff who have little or no previous experience of delivering telemental health. |
|
|
| CMO 4.4: Service users who find it easier to establish a therapeutic relationship | |||||
|
|
On the web | [39, 60, 79, 96, 99, 119, 136, 137, 147, 148] | When delivering telemental health to some services users who feel uncomfortable in clinical settings and social situations (context), these service users find it easier to build a therapeutic relationship and are more willing to use telemental health (outcome), as they feel safer, are more relaxed and less anxious being in their own environment or outside of clinical settings and in-person social situations, or both, and, thus, feel more empowered and comfortable to open up and speak freely (mechanism). | This may be especially relevant for some children and young people, including those with special needs and neurodivergent children, who find clinical settings and having to travel upsetting, and some service users with social anxiety. However, it is important that using telemental health does not reinforce potentially detrimental safety behaviors that may maintain and potentially exacerbate their social anxiety. |
|
|
|
Via video versus phone | [24, 36, 50, 52, 55, 78, 105] | When service users and staff who prefer video calls use them (instead of telephone calls or text-based chats) for telemental health (context), it can facilitate a stronger therapeutic relationship (outcome 1), satisfaction (outcome 2), and engagement (outcome 3), as it is easier to see visual and nonverbal cues, gauge the therapist’s reaction, and connect with the service user/staff member than with other telemental health modalities (mechanism). | This applies to service users across age groups and may be especially the case for new service users. |
|
|
|
Via the phone versus face-to-face or video calls | [50, 65, 80, 111] | When services offer phone calls and SMS text messages instead of video calls (context), some service users are more satisfied with their care (outcome) as they do not have to sit still and see themselves on screen, are less conscious of their body language and facial gestures, are less distracted by the clinician’s nonverbal cues, are able to move around freely, and are thus less inhibited and able to open up more quickly (mechanism). | This might be especially relevant for service users who are neurodivergent, socially anxious, and self-conscious about their appearance. |
|
| CMO 4.5: More frequent telemental health sessions plus SMS text messages | [39, 40, 99, 104, 149] | When services adapt flexibly to service users’ preferences regarding the pattern and frequency of telemental health sessions, including offering more frequent, shorter rather than infrequent, long sessions, and additional asynchronous SMS text messages and calls to check in between sessions (context), it may lead to stronger therapeutic relationships (outcome 1), increased engagement (outcome 2), and improved quality of care (outcome 3), as service users receive regular and more frequent support depending on their preference (mechanism). | Lack of a need to travel means that more frequent shorter sessions may be particularly feasible with telemental health: they are potentially less tiring and thus might better maintain concentration and engagement, especially for children. Frequent sessions might help new service users to build trust and reduce anxiety around the treatment. Frequent sessions may also help support and monitor less stable service users, for example, following a crisis. |
|
|
| CMO 4.6: Enhancing the quality of care through the use of telemental health enhancements | [24, 74, 101, 150-152] | When clinicians make appropriate and personalized use of enhancements and extensions of telemental health (such as using chat, voice activation to instruct phones, SMS text messaging and other text-based messaging, web-based appointment schedules, screen sharing, and apps accessed during sessions; context); it can lead to success engaging in telemental health (outcome 1) and broadening the range of strategies and interventions available during clinical meetings (outcome 2), as these features make engaging with services easier and provide a functional method useful for exchanging practical information, such as reminding service users about the date and purpose of an appointment, with less room for ambiguity and more creative methods of engagement (mechanism). | Additional telemental health features might be particularly helpful for young children (who overall find it difficult to engage on the web). Adolescents who experience social anxiety or are autistic may benefit from and prefer the chat function. |
|
|
| CMO 4.7: Staff well-being and quality of care | [51-53, 80, 88, 91, 101, 107, 112, 153] | When the staff use the time saved on travel to take breaks in between telemental health sessions (context), it may increase staff well-being (outcome 1) and improve quality of care (outcome 2), as it provides the opportunity to reflect and recharge after telemental health sessions, which are often experienced as tiring and thus reduces fatigue, tension, and anxiety among staff (mechanism 1), and staff can use some of the time on clinical work, catch up on administrative tasks, or engage in professional developmental activities (mechanism 2). | This may be relevant for clinicians who can work wholly or partly at home and teams working across different sites or who visit service users at home or in other community settings. |
|
|
aCMO: context-mechanism-outcome.
Domain 1: Connecting Effectively
The content of this domain relates to establishing a good web-based connection to join a video call of sufficient quality or to engage in telemental health via phone or message, with a particular focus on digital exclusion. Table 1 outlines 7 overarching CMO configurations identified in relation to this domain, addressing issues concerning device and internet access (CMO 1.1 and CMO 1.2), technology training (CMO 1.3), the impact of preparation and technological disruptions (CMO 1.4 and CMO 1.5), the familiarity and usability of the platforms (CMO 1.6), and the acceptability of telemental health as an alternative to receiving no care during emergency situations (CMO 1.7). Three of the CMO configurations were related to trying to resolve three main challenges: (1) access to a charged, up-to-date device that enables internet access (CMO 1.1); (2) an internet (Wi-Fi or data) or signal connection (CMO 1.2); and (3) the knowledge, ability, and confidence to engage on the web (CMO 1.3). Much of the content relates to challenges service users encounter in engaging with telemental health; however, the literature and stakeholder discussion also yielded significant challenges for staff and service providers in the practicalities of connecting on the web.
Theories regarding the relationship between digital exclusion and other forms of exclusion and deprivation, as well as the potential of digital exclusion to amplify inequalities, contribute to our understanding of key mechanisms and outcomes in this domain. Widening inequalities have been described as an inevitable consequence of the expansion of the role of technology in health care, with loss of access to community facilities, such as libraries, making this a still greater risk during the pandemic [154]. The “digital inverse care law” [16] describes a tendency for groups in most need of care (eg, older people or people experiencing social deprivation) to be least likely to engage with technological forms of health care. This is highly salient in mental health care, given the strong associations between experiencing mental health challenges and experiencing one or, often, many forms of disadvantage [154].
Access to devices (CMO 1.1) is a contributor to digital exclusion, and groups who are especially likely to be affected include homeless individuals and people living in poverty, those receiving inpatient or crisis care, and young children who may not have their own devices. The type of device may be important for accessing telemental health. For example, smartphones may be less suitable for video therapy because of their small screens [155], although this may be less relevant for young people who are familiar with and consistently use smartphones for connecting on the web [156]. This raises future research questions around which types of digital devices work for whom and in what context when it comes to continuing telemental health treatment. It may also have implications for the provision of suitable equipment to certain populations.
Our consultations and the wider literature revealed that lack of access to good quality Wi-Fi, including poor Wi-Fi in hospitals and offices, was a further key barrier to the successful and equitable delivery of telemental health (CMO 1.2). It was emphasized that modernization of software and hardware, particularly within the NHS, is needed in many health care sites to allow for the requirements of telemental health. Service users also reported relying on their own mobile phone data to connect to telemental health services, often depleting their data completely after or during just one video call or consultation, which is expensive to replenish and may also deter engagement. This could amplify existing inequalities, leaving some service users at risk of digital exclusion and unable to access the internet and mental health support. Disruptions to telemental health appointments because of poor connections are a significant barrier to engagement (CMO 1.4). Our consultations and the existing literature highlighted the importance of having an alternate form of communication (eg, a telephone call) as a backup plan in case of a technology or connection failure [36,37,157,158].
We identified the importance of technology training and sustained formal and informal support for service users (CMO 1.3). Variations in the ability to use telemental health are likely to disproportionately affect certain service user populations, often groups who also experience high levels of need for mental health care and inequalities in its provision. This includes people living in deprived circumstances, people with cognitive difficulties, people with paranoia, or those who do not speak the same language as service providers. Understanding how to use technology is also important for service users’ social engagement and connection, which is relevant for wider recovery and citizenship [159-161]. Young children may also be disproportionately affected as they may not be able to resolve difficulties they experience during telemental health sessions without the help of their parents or other supporters.
For staff (CMO 1.3), our evidence suggests that staff training provided more widely and accessibly on using technology for telehealth would be helpful to ensure high-quality service provision and overcome barriers around staff not having time allocated to training or being reluctant to ask for support. Evidence suggests that there are also benefits of having access to technical support to troubleshoot issues during sessions [38], as well as of practicing new skills and learning with colleagues and peers [162].
The importance of the familiarity and usability of platforms was highlighted throughout our stakeholder consultations and weekly reference group meetings, as well as in the published literature (CMO 1.6), in keeping with previous research on the acceptability of telemedicine by service users [163]. Many of the platforms and devices commonly used for telemental health services, for example, during the COVID-19 pandemic, were not designed for use in health care settings and, therefore, may be less user-friendly. The importance of usability is emphasized by Nielsen and Landauer [164]; 3 of 5 main usability attributes of a program are that it should be easy to learn, efficient to use, and easy to remember. This is in keeping with our findings related to familiarity and usability in telemental health.
Finally, preparing service users for telemental health sessions was key (CMO 1.5). This was relevant across telemental health contexts; however, information tailored to individual communication needs may be especially helpful for service users who may experience additional challenges connecting on the web (eg, those who are inexperienced with technology or anxious about using telemental health, young children, older people, and people with cognitive difficulties). For people with significant sensory impairments, specialist adaptations will need to be available if telemental health is to be a viable modality (eg, mobile phones that flash when receiving a call or providing guidance in Braille or sign languages).
Domain 2: Flexibility and Personalization
Table 2 presents the CMO configurations, key contexts, and example strategies and solutions for the domain of flexibility and personalization. The need for flexibility and personalization was a key theme identified in both the literature and stakeholder consultations when considering using telemental health in place of (or in conjunction with) face-to-face mental health support. A total of eight overarching CMO configurations were identified in this domain, which can be divided into three main categories: taking individual preferences into account (CMO 2.1, CMO 2.5, and CMO 2.7), convenience (CMO 2.2), and allowing for more collaborative and potentially specialized care (eg, involving specialists, family, or friends in care; CMO 2.3, CMO 2.4, CMO 2.6, and CMO 2.8).
Table 2.
Domain 2: Flexibility and personalization.
| CMOa title | References | Overarching CMO | Key contexts | Example strategies and solutions |
| CMO 2.1: Taking service users’ individual preferences into account—offering alternatives | [24, 36, 48, 50, 65, 67, 75, 77, 90, 91, 93-104, Eagle et al (email, August 31, 2022)] | When services using remote mental health care allow service users to choose the modality of telemental health and a choice of remote versus face-to-face care and regularly check their preferences (context), this allows service users to have greater autonomy and choice (mechanism), leading to them feeling more satisfied and able to engage with the type of care received (outcome 1), leading to improved uptake (outcome 2) and improved therapeutic relationships with their clinician (outcome 3). | Allowing service user choice and delivering services flexibly is a key principle across settings and populations, with the overall aim that care of equivalent quality should be available in a timely way whatever modality is chosen. Hybrid care, with a flexible mixture of face-to-face and telemental health care based on the purpose or function of appointment (eg, prescription review versus the first visit to see a clinician), preference, and circumstances, is especially relevant to service users receiving relatively complex care with multiple types of appointments, for example, from multidisciplinary community teams. Children and young people may particularly benefit from being offered a choice as it increases their feelings of autonomy and improves engagement in care. |
|
| CMO 2.2: Removing barriers—greater convenience for service users and family/friends | [19, 24, 36, 38, 40, 42, 45, 54, 55, 67, 70, 75, 76, 79, 81, 82, 84, 90-92, 95, 97, 105-112] | Among some service users, family, and other supporters experiencing specific practical barriers to attending face-to-face services (childcare or other caring responsibilities; location, work, and mobility limitations; travel difficulties/costs, and work commitments) and those who have good access to telemental health (context), telemental health may provide increased flexibility that addresses individual practical barriers (mechanism), which can lead to telemental health being viewed by some service users and carers as more convenient and accessible than face-to-face care (outcome 1), easing attendance (outcome 2), increasing uptake (outcome 3), and reducing missed appointments (outcome 4). | This may be relevant for parents with young children, people with caring responsibilities, and people who struggle to travel because of work commitments/disability/costs; children and young people in school or higher education (so they can access mental health care without having to leave their place of education); people who live in remote areas or a long distance away from a specialist service; and people for whom travel is challenging because of impaired mobility or sensory impairments or mental health difficulties such as agoraphobia. There may be more advantages to treatments that involve the support of family and friends. |
|
| CMO 2.3: Involvement and support for family and friends | [79, 91, 113-115] | When family and other supporters are invited (with service user agreement) to join telemental health sessions (context), this may result in more holistic treatment planning and greater engagement of family and others in supporting service users (outcome 1); may help improve therapeutic relationships and treatment success (outcome 2), increase engagement (outcome 3), and reduce some uncertainty and anxiety around treatment (outcome 4); and may increase the satisfaction of and support for family and friends (outcome 5), as family and other supporters may be able to participate in care planning meetings and assessments that they would have found difficult to attend face-to-face, increasing their engagement in supporting service users and their understanding of their difficulties and care plans (mechanism). | This is especially helpful for those living in locations different from their family and friends or where family and friends have caring or work commitments preventing them from attending meetings face-to-face, children and young people (as this may allow their parents to be more involved in their care), and service users in inpatient settings where family and friends cannot visit (eg, because of epidemic-related restrictions) or as the hospital is in a remote location. |
|
| CMO 2.4: Widening the range of available mental health services and treatments for service users via telemental health | [49, 116-118] | For service users who may benefit from services that they cannot readily access locally and that provide specialized forms of treatment and support regionally or nationally (context), telemental health can widen the range of specialist assessment, treatment, and support available (mechanism), which potentially leads to improved access to services tailored to individual needs and culturally appropriate or specialist services (outcome 1) and improved satisfaction and treatment outcomes (outcome 2), although an impoverished range of local face-to-face provision may be a risk if referral to distant specialist care via telemental health becomes routine (outcome 3). | People to whom this is relevant may include people who have complex clinical needs or rarer conditions such that they would potentially benefit from assessment, treatment, and support from specialist services provided at regional and national rather than local levels; people who may be able to access distant therapists who speak their own language or interpreters of rare languages not available locally; people who would benefit from support from voluntary organizations that meet specific needs that are not catered for locally (eg, that support particular cultural groups; lesbian, gay, bisexual, transgender, and queer groups; or people with sensory impairments); and people who would benefit from a wider choice of therapies and support (including peer support) than is available locally. |
|
| CMO 2.5: Adaptations for service users with sensory or psychological barriers to telemental health | [40, 50, 77, 119] | Offering face-to-face (or telephone) appointments to people who struggle to cope with sensory (visual or auditory) aspects of telemental health or have symptoms that are exacerbated by it (context) may help to improve engagement with mental health care (outcome) as the adverse effects of the switch to telemental health for these symptoms and sensory or cognitive impairments may be avoided and service users are able to access their preferred modality of care (mechanism). | This may be relevant for people with symptoms that may interfere with or be exacerbated by engaging with telemental health, such as persecutory ideas or hearing voices; autism; sensory or cognitive impairments; and migraines. |
|
| CMO 2.6: Inclusion of multidisciplinary and interagency teams in service users’ care | [89,115] | When mental health consultations are conducted using telemental health (context), it enables the inclusion of staff in appointments who are based geographically far away or who have schedules that would not have allowed them to join a face-to-face session (outcome 1), meaning care and support has potential to be more holistic and integrated (outcome 2), as it is possible for staff from different services and sectors to provide perspectives and contribute to plans (mechanism). | Key contexts include hospital inpatients, where telemental health may enable staff who work with them in community settings to join reviews and ward rounds (especially in pandemic conditions where they cannot attend in person), and people with complex treatment and support, who are receiving support from >1 team or sector. |
|
| CMO 2.7: Continuing to offer face-to-face care to service users | [53, 81, 92, 116, 120] | When service providers offer care of equivalent quality and timeliness face-to-face (including home visits where needed) rather than via telemental health to service users who do not wish or do not feel able to receive their care remotely (context), it ensures that care can continue and that inequalities in provision are not created or exacerbated (outcome), as it provides a choice to service users and avoids the negative impacts of digital exclusion (mechanism). | People for whom face-to-face options may be preferable, and choice is especially important, include those who do not have access to private spaces, live with people they do not wish to be overheard by in their appointments (including perpetrators of domestic abuse), do not feel comfortable communicating via remote means, and do not want therapy to intrude on their private lives should be included. In addition, some service users who value the time spent traveling to and from face-to-face appointments to process emotions may find face-to-face options particularly useful. |
|
| CMO 2.8: Communication between staff | [19, 85, 121] | When remote technology platforms are used to facilitate real-time communication between staff members, including managers or clinicians working in different teams (context), it can lead to improved efficiency, more convenient working and staff management (outcome 1), improved communication and collaborative planning (outcome 2), and process improvement opportunities (outcome 3), as staff have the ability to rapidly share information, keep track of evolving telemental health procedures (eg, during emergencies), and make collaborative decisions (mechanism). | Contexts in which this is relevant include multidisciplinary teams who are not working on the same site; complex provider organizations with management teams and clinicians working on multiple sites; situations in which people may be receiving care from multiple teams, for example, from an inpatient or crisis service, as well as a continuing care service. |
|
aCMO: context-mechanism-outcome.
Our findings emphasize the importance of taking individual service user preferences into account when deciding whether to make use of telemental health, in selecting the modality of digital communication used (including the type of technology platform), and in decisions about involving others (clinicians or family members) in care. This finding underpins all other CMO configurations in this theme and coheres with theories regarding the importance of shared decision-making, collaborative care planning, and personalization in mental health care [165-167]. Involving service users and carers in decisions and care planning as part of a collaborative approach to mental health care has been identified as central to best practice [168]; for example, a review of collaborative care for depression and anxiety found this approach to be more effective than usual care in improving treatment outcomes [169].
Flexible use of telemental health was also identified as being beneficial in reducing barriers to accessing mental health support for some service users, particularly those who may struggle to access face-to-face services for reasons such as caring or work commitments, problems traveling (eg, because of a physical disability, anxiety, or lack of transport) or a reluctance to attend the stigmatizing places. Telemental health can also facilitate connections between clinicians, especially across different services or specialties, which can improve multidisciplinary working and collaboration across teams and agencies and provide service users with a wider range of specialists or support for specific groups. This approach has been identified as having salience in a mental health setting [170-172]. In some cases, telemental health was viewed by both service users and clinicians as more convenient, as it reduced the need for (and cost of) traveling to face-to-face appointments.
Telemental health was also seen as an important tool in inpatient wards, especially during the COVID-19 pandemic when visiting was restricted, as it allowed service users to stay in touch with family and friends and for them to be involved in their care. It also allowed staff supporting inpatients in the community to remain involved.
However, instances were identified where telemental health was not appropriate, and face-to-face care needs to be available. For example, some service users do not wish or feel able to receive care by remote means or do not wish to have all appointments by this means, whereas others may struggle with telemental health because of sensory or psychological factors or a lack of access to appropriate technology and internet connectivity.
Domain 3: Safety, Privacy, and Confidentiality
Table 3 presents the four overarching CMO configurations, key contexts, and example strategies and solutions for the domain of safety, privacy, and confidentiality. Key messages were the importance of ensuring the availability of a private space for both service users and clinicians (CMO 3.1), the potential for telemental health to provide privacy to some service users experiencing stigma (CMO 3.2), the importance of considering how to manage risk when using telemental health and the limits to how far this is possible (CMO 3.3), and data security and staff training (CMO 3.4).
Table 3.
Domain 3: Safety, privacy, and confidentiality.
| CMOa title | References | Overarching CMO | Key contexts | Example strategies and solutions |
| CMO 3.1: Lack of privacy | [39, 42, 45, 48, 54, 55, 58, 60, 77, 79-82, 90, 91, 97, 101, 102, 104, 111, 120, 122-127] | When accessing telemental health sessions without access to a private space or secure private connection (context), service users and staff are at an increased risk of being overheard (mechanism 1), potentially leading to breaches of privacy and confidentiality (outcome 1), risk of harm to those in unsafe domestic situations (outcome 2), and reluctance to speak openly about sensitive topics (outcome 3). It may also cause some service users to experience frustration, distress, and anxiety (mechanism 2), leading to impacts on service user engagement and interactions (outcome 4) and reduced willingness to use telemental health and continue therapy (outcome 5). | Issues related to lack of privacy at home are especially relevant for young people who are distracted by their home environment, may not feel safe in their own home, or have siblings/parents/other family members unexpectedly appearing in the room; parents with children at home; those experiencing domestic abuse who are not able to be honest about symptoms, risk, or violence experienced; people who may be living with/caring for extended family or in households that are crowded; inpatients who may not have a space where they feel psychologically safe; people living in houses of multiple occupation; staff members who are not able to work in a private environment when providing remote therapy; and service users who experience cultural stigma in the home from their families relating to their mental health. |
|
| CMO 3.2: Privacy, anonymity, and reduced stigma (service users) | [40, 43, 92, 103, 105, 128-131] | For some service users who feel stigmatized when attending a mental health service in person and who have access to a private and secure space to receive therapy remotely (context), being provided with the option of telemental health as an alternative means there is an option to receive care with more anonymity (mechanism), which helps ensure their privacy and safety (outcome 1), thereby increasing the accessibility of services (outcome 2). | Some groups may be more likely to feel there is a stigma associated with attending mental health premises or reluctant to have contact with others doing so, for example, young people not previously in contact with services. |
|
| CMO 3.3: Managing risk | [24, 43, 85, 91, 111, 112, 119, 124, 132-136] | When services incorporate tailored risk management procedures in the delivery of remote care (context), it encourages consideration of the risks associated with remote care specific to each individual, including the risk of self-harm or suicide and risk from others in situations of domestic abuse, and ensures the staff are aware of the procedures to try to assess and respond to risk or safeguarding concerns despite challenges associated with remote care (mechanism), which has the potential to improve the safety and well-being of service users and others (outcome 1). However, a disadvantage of telemental health is that real-time risk assessment limits an immediate response to be organized when someone is at imminent risk of harm and some distance away (outcome 2). | This may be relevant to people who are currently unwell or in a crisis, situations where someone is remote from the assessing clinician or at a location unknown to them, situations where technological difficulties occur during an assessment of someone who is at high risk, people with eating disorders or who are physically unwell and where there are practical impediments to assessing risk remotely, and when a service user suddenly exits during a telemental health consultation and it is not clear why. In substance misuse services, it may be harder to detect whether someone is under the influence of drugs or alcohol. |
|
| CMO 3.4: Technological support and information security | [85] | When services provide technology support, software with appropriate security, and devices (including mobile phones and headphones) to staff specifically for work use (context), it helps ensure privacy and confidentiality for both service users and staff (outcome), as staff can store information securely on devices that are not shared with others (mechanism 1) and are able to ensure that service users are aware of when they will have access to their work devices (mechanism 2). | This may be relevant in services where staff share office space and devices, or where a shortage of devices may lead to the use of personal devices, for example, for home working; when balancing service user preference with risk from using less secure software or software with which the staff are less familiar; where software has a particular set of settings that must be enabled to ensure secure, private connections. |
|
aCMO: context-mechanism-outcome.
With the most supporting literature, CMO 3.1 highlights the need for appropriate private space to receive telemental health and that many service users may not have consistent access to such a space. As a lack of privacy can risk breaches in confidentiality and safety for some, a key message was that alternatives such as face-to-face modalities or alternative times/locations to receive telemental health should be provided. The importance of privacy for effective mental health care has been frequently cited in the literature and is likely to be especially important in ensuring high-quality telemental health [91]. Although some literature indicated that some service users feel an increased sense of privacy and a reduction in stigma when not having to attend mental health clinics in person (CMO 3.2), a key message lies in providing a choice so that each individual can work with their clinicians to find ways of receiving care that they are happy with, a message highlighted in CMO configurations throughout this paper.
CMO configurations in this domain also make it clear that telemental health can result in greater risks, both directly as it may be more difficult for clinicians to assess and respond to risks (CMO 3.3) and indirectly if data security is impaired (CMO 3.4). In both cases, proactive steps to assess and limit risk before the use of telemental health, as well as preplanned strategies to respond to events that threaten safety, are important. Data security knowledge should not be assumed, and training to help staff keep service users’ personal information secure will also mitigate telemental health–specific risks. However, evidence from both the literature and the stakeholder consultations made it clear that it is difficult to fully overcome the obstacles to effective risk assessment and management that result from staff and service users being in different places, meaning that the continuing availability of an in-person community crisis response is also important.
Domain 4: Therapeutic Quality and Relationships
Table 4 displays the overarching CMO configurations, key contexts, strategies, and solutions for the final domain of therapeutic quality and relationships. Therapeutic relationships have been identified as pivotal for the successful delivery of telemental health across the literature and stakeholder consultations. The domain addresses barriers (CMO 4.1 and CMO 4.2) to and facilitators (CMO 4.3, CMO 4.4, CMO 4.5, and CMO 4.6) to the development of therapeutic relationships and delivery of quality care and discusses the impact of telemental health on staff well-being (CMO 4.7).
Trust and therapeutic relationships are important across health care, and relational aspects of care are especially crucial in mental health [173-178]. However, the reliance on telemental health platforms, particularly telephone and text-based communication, may affect communication and subsequently therapeutic relationships (CMO 4.1). Our CMO configurations, particularly their mechanisms, were informed by general theories regarding the role and development of therapeutic relationships in mental health care.
CMO 4.1. highlights that telemental health is likely to lead to a change or reduction in visual and nonverbal cues, including active listening and back channels, facial expressions, gestures, posture, and eye contact, which makes aspects of communication, such as pauses, difficult to interpret. In addition, time delays in video calls may create silences and lead to talking over each other and delayed visual responses, which negatively affect communication and nonverbal synchrony [179,180]. As a result, not only therapeutic relationships but also the staff’s ability to conduct accurate assessments are compromised (CMO 4.1 and CMO 4.2). However, conversely, good quality video contacts with well-trained clinicians may mitigate some of the therapeutic challenges when delivering telemental health. Those making first contact with mental health services appear to be particularly affected by the potentially impersonal nature of telemental health and thus benefit not only from an initial face-to-face session but also from more frequent subsequent telemental health sessions to establish stability and trust (CMO 4.1 and CMO 4.5). In addition, our findings indicate that staff confidence and ability to deliver good quality care and develop therapeutic relationships via telemental health can be fostered through training sessions provided by services (CMO 4.3).
The literature and stakeholder consultations identified no telemental health modality that is consistently superior for developing therapeutic relationships (CMO 4.4). Rather, whether telephone calls, video calls, or face-to-face meetings are most appropriate seems to depend on the purpose of the sessions and on an individual’s preferences based on their personal experiences and circumstances and whether they are new to the service, as well as the nature of their mental or physical health problems. Video calls seem to be preferred for more substantial and in-depth sessions than other telemental health modalities [24]. Providing service users with choice regarding the frequency, duration, and telemental health modality is crucial for therapeutic relationships and quality of care.
Despite its limitations, flexible use of different telemental health modalities can provide significant opportunities to foster therapeutic relationships and increase the quality of care, such as checking in, sending reminders via SMS text messages, and using features such as chat functions to increase engagement among service users (CMO 4.6).
Finally, taking breaks in between telemental health sessions and fostering positive telemental health working environments is key for staff well-being and the delivery of high-quality care (CMO 4.7).
Discussion
Principal Findings
Our RRR identified CMO configurations within four key domains, each with a range of practical implications regarding what works and for whom in telemental health: connecting effectively; flexibility and personalization; safety, privacy, and confidentiality; and therapeutic quality and relationship. Potentially, the most important finding of this realist review is the significance of personal choice and that one size does not fit all for telemental health. This includes choice of modality (eg, video, telephone, and text-based chat functions), platform, frequency or duration of sessions, and the option to revert to face-to-face sessions if preferred or required by the service user based on their current context or to vary modality from contact to contact. This review has highlighted that there are many contexts where face-to-face care is preferred or needed by service users, and this should be accessible and available to them and should be of equivalent timeliness to remote care (especially when delivered as part of routine care rather than as a response to a national emergency). However, the use of telemental health is a convenient and potentially advantageous option for some people in many contexts; thus, it is beneficial for mental health clinicians to have the skills and resources to offer telemental health as an option. When service users’ choices about what works for them are respected and decisions about care planning are made collaboratively, it is likely to be conducive to a stronger therapeutic relationship where the service user feels heard and respected [178].
Access to a device with a stable internet connection, as well as the confidence and ability to use a device to access telemental health, were identified as minimum requirements for both staff and service users to access telemental health, without which face-to-face appointments would be necessary. The devices and platforms used for delivering telemental health needed to be user-friendly [163,164], and preferably familiar, to easily facilitate sessions. Telemental health seemed to reduce some barriers to receiving mental health support experienced by some service users, such as those who were unable to travel or in inpatient wards, making it an acceptable alternative to face-to-face sessions for some people under these circumstances. It may also potentially allow service users greater access to out-of-area specialist services and support that is focused on specific groups (eg, cultural or lesbian, gay, bisexual, transgender, and queer groups). Issues of privacy, including data protection and confidentiality, or staff and service user access to a private space were emphasized throughout the literature and our consultations; this is likely to disproportionately affect disadvantaged groups of service users, such as those experiencing poverty, those in multi-occupancy households, children and young people, or people living with controlling or abusive partners or other family members. Already disadvantaged groups are similarly at particularly high risk of inequalities being exacerbated through digital exclusion. The “inverse digital care law,” stating that the use of digital technologies can make health inequalities worse [154], may well apply to the widespread implementation of telemental health [16]. Service planning and delivery must be based on a strong awareness of these risks and the need to overcome these barriers. There is also a duty to ensure that in-person care of equivalent quality remains readily available.
The impact on therapeutic relationships for both staff and service users has also been highlighted, with difficulties interpreting visual or nonverbal cues being cited as a barrier to establishing a good therapeutic relationship that enables service users to disclose sensitive information and staff are able to conduct valid clinical assessments. Adapting to service user preferences flexibly and giving weight to self-reports during assessments is likely to increase the quality of care and foster strong therapeutic relationships.
Strengths and Limitations
The use of the RRR methodology to rapidly establish a set of theories about what works, for whom, and in which circumstances in telemental health has several strengths. The breadth of screened written evidence extends beyond published academic literature to nonacademic (including policy, third sector, and lived experience) sources. The targeted call for evidence sent directly to expert stakeholders from research, policy, and clinical settings (nationally and internationally), the voluntary sector, lived experience groups, minority groups, and representatives from health tech initiatives identified resources that would otherwise have been missed. Through these procedures, we rapidly identified literature from a wide range of key perspectives to contribute to the development of the CMO configurations.
The analysis process was rigorous and valued both published literature and stakeholder views, with the use of rapid realist methods allowing a range of stakeholder perspectives to be incorporated beyond what is normally possible in reviews. Our expert reference group (including clinical, academic, and lived experience experts) fed into the review process and theory development throughout, iteratively reviewing the plausibility, relevance, and usefulness of our individual and overarching CMO configurations. A wider group of expert stakeholders provided further input to identifying sources and reviewing overarching CMO configurations, especially regarding our priority groups: children and young people, users of inpatient and crisis care services, and digitally excluded populations. Continuous detailed feedback from the lived experience researchers and frontline clinicians helped to reduce bias toward academic perspectives; ensured the inclusion of a breadth of real-life experiences; and supported the iterative development of our methods, results, and interpretation of the findings.
A final key benefit of the RRR methodology is that we could rapidly investigate not only outcomes of telemental health use but also the mechanisms underlying what works and for whom, which most methodologies do not allow. We could also explore the contexts in which telemental health was implemented, as well as the telemental health resources that are available. This approach should be considered for future evaluation of telemental health.
Some limitations should be noted. The first relates to generalizations made in the process of developing overarching CMO configurations. These tended to combine underlying CMO configurations that related to a range of service user and clinician groups, service settings and types, and social and national contexts. We looked for important themes that appeared of general relevance and were validated through stakeholder consultation. However, it is likely that in some areas, we lost a more nuanced understanding of the relationship between particular CMO configurations and particular contexts. We also used a broad definition of telemental health to capture as much richness (and data) as possible from the available sources. However, we have merged heterogeneous forms of telemental health within most of our overarching CMO configurations, which may differ in their effectiveness and underlying mechanisms. Therefore, the conclusions are limited regarding mechanisms; outcomes for specific types of telemental health; and the impact on service users, staff, and carers. We also included literature that draws on the experiences of service users and clinicians both before and during the pandemic. However, technologies and approaches to implementation have changed substantially, with prepandemic evidence tending to focus on the planned and relatively small-scale implementation of tools specifically designed for mental health. Studies from the pandemic tend to relate to a range of phone and video call technologies implemented at scale with limited strategic planning. During the pandemic, staff and service users may also have been more willing to trial telemental health, given the extraordinary circumstances. The available technologies, and clinicians’ and service users’ skills in applying them, are also likely to have changed over time and are likely to continue to change.
The nature and strength of evidence drawn on for the review also need to be noted. Most sources were qualitative studies, service evaluations, or cross-sectional studies of associations; we found few relevant trials or longitudinal studies. We tried to maximize the value of this body of evidence by combining findings from multiple studies with expert stakeholder input to obtain theories with multiple sources of support about what works and for whom, illustrating them with example contexts and strategies. However, the lack of testing through traditionally robust methods in testing intervention strategies, such as trials and other longitudinal forms of evaluation, still needs to be noted, as discussed further in the Implications for Research section. In line with realist methodology [27,29], we did not appraise the sources we identified using traditional methods. However, sources were only included if they provided sufficient information on context, mechanism, and outcomes and contributed to the development of our overarching CMO configurations. Our extracted data were reviewed for their validity and coherence by our expert reference group.
Despite the inclusive search strategy and specific efforts to gain a wide range of perspectives, digitally excluded groups remain underrepresented in this study. This is partly because of the lack of literature focused on digital exclusion and the web-based methods used to conduct our review during the COVID-19 pandemic. Efforts were made to gain perspectives on digitally excluded groups by involving charities and staff and service user advocates working with people in such groups, including in projects aimed at addressing digital exclusion, in our stakeholder consultation. However, people experiencing severe digital exclusion did not participate in our web-based consultations, and the extent to which others can advocate for them is limited. Similarly, this study identified a lack of evidence in the literature about how to make telemental health engaging and effective for young children, nor were we able to find many people with relevant expertise to participate in our consultations. Data were also limited on group therapy and the role or experiences of families and other supporters of service users. Most available literature focused exclusively on staff perspectives of telemental health and crucially neglected to include the views or experiences of service users and their families or other supporters. Therefore, we were unable to incorporate these groups and their perspectives in our analysis and synthesis.
This study was initially planned and commissioned through discussions between policy makers in the Department of Health and Social Care and the MHPRU leads; lived experience researchers did not have the opportunity to contribute during the early stages of formulating research questions and identifying the methodology to be used.
Implications
Implications for Clinical Practice
A range of implications for clinical practice and service planning can be drawn from our CMO configurations. The challenge for the future will be to find sustainable ways of implementing them in clinical practice and finding an appropriate balance between telemental health and traditional face-to-face care in future service delivery. In the context of a recent emergency (the COVID-19 pandemic), telemental health has been used with some degree of success to maintain care for at least some service users. Evidence and experiences from this widespread emergency implementation are helpful, both to inform future responses to such emergencies and allow a preliminary assessment of potential opportunities and pitfalls in implementing telemental health beyond an emergency context.
Some clear principles to guide practice emerge from our CMO configurations. Offering choice, planning care collaboratively, and listening to personal preferences regarding whether to use telemental health need to be embedded within services in which there is continuous use of telemental health as we move through and out of the COVID-19 pandemic. How choice is negotiated, enabled, and communicated is crucial. For choice to be real, options need to be clearly explained and discussed at every stage; face-to-face care of equal quality should be delivered as promptly as telemental health; and choice should be seen as dynamic, especially when a service user is in crisis. Preferences should be reconfirmed regularly and hybrid forms of care made available if appropriate. Choices may also be different after the COVID-19 pandemic, when the risk of infection traveling to and at appointments may no longer be a concern and consultations are no longer masked; mask-wearing at most face-to-face appointments during the pandemic may undermine some advantages of in-person care. Ideally, service user and clinician choices and resources should be balanced through shared decision-making. Use of telemental health cannot be assumed to be a permanent switch; thus, preferences should be revisited regularly. In planning services, it may be easier to switch from in-person appointments to digital appointments than vice versa, and this needs to be considered in staffing and working space arrangements. Traditional inpatient and community services are limited in their ability to collaborate and provide choice in their established processes, such as care planning and risk assessment or management [177,178,181-183]. Therefore, it may be unrealistic to expect improvements in these areas when delivering telemental health.
Lack of access to digital devices or data, or of expertise in connecting to telemental health services, is a problem that service providers may be able to address for some people. For example, opportunities to develop skills and clear guidance and opportunities to practice may be relatively straightforward ways of alleviating problems with connecting effectively for some people who may find telemental health a convenient way of receiving care if they are supported to engage. At best, obtaining access to telemental health may be a skill acquired along with developing the skills to access a variety of other significant parts of the digital world. In other instances, clinicians and service providers should be aware that digital exclusion tends to be rooted in other forms of disadvantage and that they can most readily avoid exacerbating such disadvantages by offering face-to-face care. Persevering with telemental health when service users do not want to receive care by this means and are not in the habit of using digital technologies may prove futile in everyday clinical care. There is also likely to be scope for improving the extent to which service providers have the capacity to connect effectively, for example, through better infrastructure, clear guidance and training for staff, and clarity and flexibility regarding platforms.
Developing a therapeutic relationship is key for the quality and success of care, and offering initial appointments face-to-face may facilitate this, subject to service user choice. In addition, services and staff may need to consider how to adapt telemental health care to account for the change in visual cues, including body language and facial expressions (although visual cues in face-to-face sessions may, in any case, be compromised while infection control considerations mean most sessions are masked).
In addition, the domain of privacy, safety, and confidentiality has implications for clinical practice. Clinician awareness of potential risks associated with using telemental health is important and may steer them away from conducting some consultations in this way. Maintaining privacy and safety, for example, for people at risk within their homes, is also a significant reason to prioritize service user choice, especially choices not to accept telemental health appointments, as they may not readily be able to explain the basis for their choice. Clinicians need also to be aware of the challenges of assessing and responding to risk when using telemental health, including the advantages of face-to-face meetings; the need to give weight to service user reports where visual or verbal cues may be obscured; and the importance of backup plans, such as for disconnection or when an urgent response is needed. Clinicians and care coordinators could also helpfully ensure that service users have access to and can use telemental health care adequately before the agreed web-based sessions begin, although this may be affected by staffing issues and limited resources [184,185].
Implications for Policy
Digital poverty does not exist in isolation, and the experience of poverty may be the root cause of their digital exclusion. Providing service users with access to devices and the internet, for example, serves as an adequate short-term fix but does not address the systemic welfare issues experienced by many service users [186]. Strategies to mitigate digital exclusion could include the provision of good national Wi-Fi coverage, free broadband [187], and investment in accessible and connected community hubs; implementing these would require action from the government rather than health services.
Telemental health services seem to be a viable alternative to face-to-face care for some service users, including in emergency situations, such as COVID-19. To provide good telemental health services, investment is needed, for example, in providing telemental health–specific training and guidance, high-quality infrastructure, and potentially technological devices to staff and service users. Preregistration education and training should include skills in telemental health. Further investment is likely to be needed in updating this as evidence and technologies change (eg, to cover the ongoing costs of keeping hardware and software up to date). This may need to be balanced against any savings anticipated from implementing telemental health. This study has also highlighted the importance of service user and frontline staff involvement in the planning of all telemental health services and provision.
Implications for Research
Much of the included research was based on explorations of views and experiences of people participating in telemental health in various settings. We found few studies involving systematic evaluation of planned strategies to achieve high-quality implementation of telemental health in routine settings. Primary studies of this form would be valuable, potentially using implementation research and participatory action research models to explore outcomes and experiences of strategies aimed at good quality implementation of telemental health in varying real-world settings during and after the pandemic. Our CMO configurations have the potential to inform such a primary research study: it would be helpful to develop and test coproduced strategies for implementing principles encapsulated in the CMO configurations in real-world settings. Similarly, our understanding of what works and for whom in telemental health would be improved by conducting primary research with specific groups, particularly those excluded from previous studies, such as digitally excluded groups or peer support networks, and in specific contexts, such as in group therapy sessions. Identifying methods of reaching digitally excluded populations in research studies, as well as identifying groups for whom telemental health is not appropriate, would be helpful. This is likely to be labor and time intensive and needs appropriate funding. Future research could also usefully explore the use of different telemental health modalities individually and in more depth.
Choice has been emphasized as crucial in the use of telemental health. The mechanisms behind choices and collaborative decision-making would benefit from further investigation, potentially using realist methods and drawing on principles from shared decision-making research. Investigation is warranted of the best approaches to providing the information needed to make an informed choice, holding collaborative discussions on how to personalize care for each individual, and providing staff and service users with guidance and training needed to participate effectively in telemental health. At a provider level, evidence is needed on what makes a good telemental health platform, how to balance data security with the flexibility that service users and clinicians may value in choosing platforms and using familiar tools if possible, and how to adapt risk management to a telemental health context. However, trusts may compromise their ability to offer choice and flexibility to service users when they specify the platforms that can and cannot be used to deliver telemental health services (although this may have advantages, including increasing staff familiarity). Future work could helpfully investigate whether certain combinations of features, platforms, or modalities are optimal for service users, their carers, and clinicians.
Researchers conducting evaluations of telemental health should consider that satisfaction tends to be evaluated as one component. However, satisfaction with telemental health comprises several components that need to be individually considered. For example, the skills of therapists may be rated highly, whereas telemental health platforms themselves may cause significant frustration and, if they were scored separately, would be poorly rated. Therefore, telemental health needs to be evaluated as several elements rather than as a singularity.
The impact of telemental health delivery on staff is a further key area of investigation. Some staff in research studies and in our stakeholder consultations reported finding prolonged screen use draining and perceived it as a contributor to burnout. The impact of telemental health on staff and ways of ensuring that it does not increase burnout or physical or psychological stress requires investigation. It may be pertinent to investigate whether services function better, and staff and service users are more satisfied, when certain staff become telemental health specialists, as opposed to asking all users to engage with it for some appointments.
A final key consideration is that any future research into telemental health should include lived experience knowledge, expertise, and views, including those from digitally excluded groups. Working remotely as a research team can facilitate the inclusion of a range of key stakeholders who might otherwise not be able to attend face-to-face meetings, promote ongoing communication, and simplify the coordination of tasks. However, similarly to telemental health care, it can also lead to the exclusion of already marginalized groups, such as digitally excluded people. There is a need to fund research designed and led by people with lived experience of mental health service use.
Lived Experience Commentary
The lived experience commentary written by KM, RRO, and PS is shown in Textbox 1.
Lived experience commentary.
Commentary
We welcome the question “what works for whom, in what circumstances, and how?” At its heart, a realist review understands that each person has different needs from services, including telemental health services. The challenge is the reliance on existing knowledge and the potential to overlook gaps, especially where the world has changed rapidly because of the COVID-19 pandemic.
The digital methods used to consult a wider audience also further marginalize everyone who does not have, or want, such access. Including people who do not use telemental health would produce different research questions and answers. Similarly, including technology experts might provide some reassurance, for example, about regulation, risk, and ethics raised by practices such as the recent sharing of free text data from a UK crisis text line with third-party researchers to develop artificial intelligence–driven tools [188].
Digital technology has increased restrictive practice in mental health via surveillance [189], sometimes based on poor-quality research conducted with financial incentives from manufacturers [190]. Health data have been shared with police in programs such as Serenity Integrated Mentoring on shaky legal and ethical grounds [191]. Although these are not telemental health per se, they provide a context. In that context, we would have liked the question “What works for whom?” to consider political and financial interests.
This study’s methods encouraged a discussion of choice, personalization, and flexibility, which we welcome. We highlight two reflections.
First, choice is not only about preferring one option over another: it can be life or death. Within mental health, service users are often expected to bare our souls to get our choices respected. With telemental health, this is dangerous. If the criteria for accessing a face-to-face service are harm-based, we might be forced to put ourselves at risk to obtain what we need. When someone is being abused by their partner, they may need face-to-face services but not explain why at a first assessment. We must be taken at our word without being required to explain ourselves to clinicians who have not yet earned our trust.
Second, choice is limited by the available options, which are constrained by material circumstances and power. Service users generally have relatively little power in their relationships with an overstretched system. If a wheelchair user’s choice is to travel to a building with an unreliable lift versus telemental health, it is not a meaningful choice. If you have to wait 6 months for a face-to-face appointment but you can have telemental health the next week, it is not a meaningful choice. If you cannot afford to connect to the internet, you do not have a meaningful choice. The option of telemental health must not become an excuse to allow face-to-face services to become harder to access.
Many of the actions for telemental health implementation are specific applications of general principles of good care: informed consent to make meaningful choices; clarity about the use of our health data, breeding trust; and understanding and responding to the contexts in which we live our lives.
Within such contexts, we welcome a focus on digital poverty as poverty. The policy solutions to poverty lie well beyond mental health—a broader overhaul of the punitive welfare system and a society in which workers are empowered to negotiate livable wages.
Acknowledgments
The authors would like to acknowledge the contributions of the webinar attendees. This paper presents independent research commissioned and funded by the National Institute for Health Research (NIHR) Policy Research Program, conducted by the NIHR Policy Research Unit in Mental Health (grant number PR-PRU-0916-22003). The views expressed are those of the authors and not necessarily those of the NIHR, the Department of Health and Social Care or its arm’s 591-length bodies, or other government departments. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Abbreviations
- CMO
context-mechanism-outcome
- LEWG
Lived Experience Working Group
- MHPRU
Mental Health Policy Research Unit
- NHS
National Health Service
- NIHR
National Institute for Health Research
- PRISMA
Preferred Reporting Items for Systematic Reviews and Meta-Analyses
- PROSPERO
International Prospective Register of Systematic Reviews
- RRR
rapid realist review
Glossary.
Study characteristics.
Overarching context-mechanism-outcome configurations.
Footnotes
Authors' Contributions: MS, KRKS, RA, and PB contributed to the design of the manuscript; acquisition, analysis, and interpretation of data; and drafting and revision of the manuscript.
UF, RRO, KM, PS, BC, KP, MB, and KT contributed to the design of the manuscript; acquisition, analysis, and interpretation of data; and revision of the manuscript.
NL and CT contributed to the acquisition and analysis of data and revised the manuscript.
C-AB, JS, SR, TG, JG, FP, RG, and TJ contributed to the design of the manuscript; acquisition, analysis, and interpretation of data; and revision of the manuscript.
NVSJ, RMC, BL-E, AS, JJN, and SJ contributed to the conception and design of the manuscript; acquisition, analysis, and interpretation of data; and revision of the manuscript.
All authors have approved the final version of the manuscript.
Conflicts of Interest: RRO is a member of King’s Improvement Science, which offers co-funding to the National Institute for Health Research Applied Research Collaboration South London, and is funded by King’s Health Partners (Guy’s and St Thomas’ National Health Service [NHS] Foundation Trust, King’s College Hospital; NHS Foundation Trust, King’s College London; and South London and Maudsley NHS Foundation Trust) and Guy’s and St Thomas’ Foundation.
References
- 1.Telehealth. Agency for Healthcare Research and Quality. [2022-01-31]. https://digital.ahrq.gov/key-topics/telehealth .
- 2.Hensel J, Graham R, Isaak C, Ahmed N, Sareen J, Bolton J. A novel emergency telepsychiatry program in a Canadian urban setting: identifying and addressing perceived barriers for successful implementation: un nouveau programme de télépsychiatrie d'urgence en milieu urbain Canadien: identifier et aborder les obstacles perçus d'une mise en œuvre réussie. Can J Psychiatry. 2020 Aug;65(8):559–67. doi: 10.1177/0706743719900465. http://europepmc.org/abstract/MED/31969011 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Bashshur RL, Shannon GW, Bashshur N, Yellowlees PM. The empirical evidence for telemedicine interventions in mental disorders. Telemed J E Health. 2016 Feb;22(2):87–113. doi: 10.1089/tmj.2015.0206. http://europepmc.org/abstract/MED/26624248 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Salmoiraghi A, Hussain S. A systematic review of the use of telepsychiatry in acute settings. J Psychiatr Pract. 2015 Sep;21(5):389–93. doi: 10.1097/PRA.0000000000000103.00131746-201509000-00009 [DOI] [PubMed] [Google Scholar]
- 5.Telehealth bibliography. Telebehavioural Health Institute. [2022-01-31]. https://telehealth.org/bibliography/
- 6.Sheridan Rains L, Johnson S, Barnett P, Steare T, Needle JJ, Carr S, Lever Taylor B, Bentivegna F, Edbrooke-Childs J, Scott HR, Rees J, Shah P, Lomani J, Chipp B, Barber N, Dedat Z, Oram S, Morant N, Simpson A, COVID-19 Mental Health Policy Research Unit Group Early impacts of the COVID-19 pandemic on mental health care and on people with mental health conditions: framework synthesis of international experiences and responses. Soc Psychiatry Psychiatr Epidemiol. 2021 Jan;56(1):13–24. doi: 10.1007/s00127-020-01924-7. http://europepmc.org/abstract/MED/32804258 .10.1007/s00127-020-01924-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Seritan AL, Heiry M, Iosif AM, Dodge M, Ostrem JL. Telepsychiatry for patients with movement disorders: a feasibility and patient satisfaction study. J Clin Mov Disord. 2019 Jun 6;6:1. doi: 10.1186/s40734-019-0077-y. https://clinicalmovementdisorders.biomedcentral.com/articles/10.1186/s40734-019-0077-y .77 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Telepsychiatry toolkit. American Psychiatric Association Work Group. [2021-09-23]. https://www.psychiatry.org/psychiatrists/practice/telepsychiatry/toolkit .
- 9.Hilty DM, Ferrer DC, Parish MB, Johnston B, Callahan EJ, Yellowlees PM. The effectiveness of telemental health: a 2013 review. Telemed J E Health. 2013 Jun;19(6):444–54. doi: 10.1089/tmj.2013.0075. http://europepmc.org/abstract/MED/23697504 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Varker T, Brand RM, Ward J, Terhaag S, Phelps A. Efficacy of synchronous telepsychology interventions for people with anxiety, depression, posttraumatic stress disorder, and adjustment disorder: a rapid evidence assessment. Psychol Serv. 2019 Nov;16(4):621–35. doi: 10.1037/ser0000239.2018-24930-001 [DOI] [PubMed] [Google Scholar]
- 11.Gloff NE, LeNoue SR, Novins DK, Myers K. Telemental health for children and adolescents. Int Rev Psychiatry. 2015;27(6):513–24. doi: 10.3109/09540261.2015.1086322. [DOI] [PubMed] [Google Scholar]
- 12.Hubley S, Lynch SB, Schneck C, Thomas M, Shore J. Review of key telepsychiatry outcomes. World J Psychiatry. 2016 Jun 22;6(2):269–82. doi: 10.5498/wjp.v6.i2.269. https://www.wjgnet.com/2220-3206/full/v6/i2/269.htm . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Fletcher TL, Hogan JB, Keegan F, Davis ML, Wassef M, Day S, Lindsay JA. Recent advances in delivering mental health treatment via video to home. Curr Psychiatry Rep. 2018 Jul 21;20(8):56. doi: 10.1007/s11920-018-0922-y.10.1007/s11920-018-0922-y [DOI] [PubMed] [Google Scholar]
- 14.Appleton R, Williams J, Vera San Juan N, Needle JJ, Schlief M, Jordan H, Sheridan Rains L, Goulding L, Badhan M, Roxburgh E, Barnett P, Spyridonidis S, Tomaskova M, Mo J, Harju-Seppänen J, Haime Z, Casetta C, Papamichail A, Lloyd-Evans B, Simpson A, Sevdalis N, Gaughran F, Johnson S. Implementation, adoption, and perceptions of telemental health during the COVID-19 pandemic: systematic review. J Med Internet Res. 2021 Dec 09;23(12):e31746. doi: 10.2196/31746. https://www.jmir.org/2021/12/e31746/ v23i12e31746 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Santesteban-Echarri O, Piskulic D, Nyman RK, Addington J. Telehealth interventions for schizophrenia-spectrum disorders and clinical high-risk for psychosis individuals: a scoping review. J Telemed Telecare. 2020;26(1-2):14–20. doi: 10.1177/1357633X18794100. [DOI] [PubMed] [Google Scholar]
- 16.Davies AR, Honeyman M, Gann B. Addressing the digital inverse care law in the time of COVID-19: potential for digital technology to exacerbate or mitigate health inequalities. J Med Internet Res. 2021 Apr 07;23(4):e21726. doi: 10.2196/21726. https://www.jmir.org/2021/4/e21726/ v23i4e21726 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Spanakis P, Peckham E, Mathers A, Shiers D, Gilbody S. The digital divide: amplifying health inequalities for people with severe mental illness in the time of COVID-19. Br J Psychiatry. 2021 Oct;219(4):529–31. doi: 10.1192/bjp.2021.56. http://europepmc.org/abstract/MED/35048887 .S0007125021000568 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Barnett P, Goulding L, Casetta C, Jordan H, Sheridan-Rains L, Steare T, Williams J, Wood L, Gaughran F, Johnson S. Implementation of telemental health services before COVID-19: rapid umbrella review of systematic reviews. J Med Internet Res. 2021 Jul 20;23(7):e26492. doi: 10.2196/26492. https://www.jmir.org/2021/7/e26492/ v23i7e26492 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Johnson S, Dalton-Locke C, Vera San Juan N, Foye U, Oram S, Papamichail A, Landau S, Rowan Olive R, Jeynes T, Shah P, Sheridan Rains L, Lloyd-Evans B, Carr S, Killaspy H, Gillard S, Simpson A, COVID-19 Mental Health Policy Research Unit Group Impact on mental health care and on mental health service users of the COVID-19 pandemic: a mixed methods survey of UK mental health care staff. Soc Psychiatry Psychiatr Epidemiol. 2021 Jan;56(1):25–37. doi: 10.1007/s00127-020-01927-4. http://europepmc.org/abstract/MED/32857218 .10.1007/s00127-020-01927-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Hutchings R. The impact of Covid-19 on the use of digital technology in the NHS. Nuffield Trust. 2020. Aug 27, [2022-01-31]. https://tinyurl.com/2hkr9kt6 .
- 21.Watkins S. Community care - method of contacts. Twitter. 2020. Sep 11, [2021-01-21]. https://twitter.com/SteveWatkinsNHS/status/1304467986556891136 .
- 22.Fisk M, Livingstone A, Pit SW. Telehealth in the context of COVID-19: changing perspectives in Australia, the United Kingdom, and the United States. J Med Internet Res. 2020 Jun 09;22(6):e19264. doi: 10.2196/19264. https://www.jmir.org/2020/6/e19264/ v22i6e19264 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Whaibeh E, Mahmoud H, Naal H. Telemental health in the context of a pandemic: the COVID-19 experience. Curr Treat Options Psychiatry. 2020;7(2):198–202. doi: 10.1007/s40501-020-00210-2. http://europepmc.org/abstract/MED/32292687 .210 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Vera San Juan N, Shah P, Schlief M, Appleton R, Nyikavaranda P, Birken M, Foye U, Lloyd-Evans B, Morant N, Needle JJ, Simpson A, Lyons N, Rains LS, Dedat Z, Johnson S. Service user experiences and views regarding telemental health during the COVID-19 pandemic: a co-produced framework analysis. PLoS One. 2021 Sep 16;16(9):e0257270. doi: 10.1371/journal.pone.0257270. https://dx.plos.org/10.1371/journal.pone.0257270 .PONE-D-21-05340 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Madigan S, Racine N, Cooke JE, Korczak DJ. COVID-19 and telemental health: benefits, challenges, and future directions. Can Psychol. 2021 Feb 26;62(1):5–11. doi: 10.1037/cap0000259. [DOI] [Google Scholar]
- 26.Saul JE, Willis CD, Bitz J, Best A. A time-responsive tool for informing policy making: rapid realist review. Implement Sci. 2013 Sep 05;8:103. doi: 10.1186/1748-5908-8-103. https://implementationscience.biomedcentral.com/articles/10.1186/1748-5908-8-103 .1748-5908-8-103 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Pawson R, Tilley N. Realistic Evaluation. Thousand Oaks, CA, USA: Sage Publications; 1997. [Google Scholar]
- 28.Lemire S, Kwako A, Nielsen SB, Christie CA, Donaldson SI, Leeuw FL. What is this thing called a mechanism? Findings from a review of realist evaluations. New Dir Eval. 2020 Oct 06;2020(167):73–86. doi: 10.1002/ev.20428. [DOI] [Google Scholar]
- 29.Pawson R, Greenhalgh T, Harvey G, Walshe K. Realist review-a new method of systematic review designed for complex policy interventions. J Health Serv Res Policy. 2005 Jul;10 Suppl 1:21–34. doi: 10.1258/1355819054308530. [DOI] [PubMed] [Google Scholar]
- 30.Khangura S, Konnyu K, Cushman R, Grimshaw J, Moher D. Evidence summaries: the evolution of a rapid review approach. Syst Rev. 2012 Feb 10;1:10. doi: 10.1186/2046-4053-1-10. https://systematicreviewsjournal.biomedcentral.com/articles/10.1186/2046-4053-1-10 .2046-4053-1-10 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Stolee P, Elliott J, McNeil H, Boscart V, Heckman GA, Hutchinson R, Hedley M, Glouberman S, Judd M. Choosing Healthcare Options by Involving Canada's Elderly: a protocol for the CHOICE realist synthesis project on engaging older persons in healthcare decision-making. BMJ Open. 2015 Nov 03;5(11):e008190. doi: 10.1136/bmjopen-2015-008190. https://bmjopen.bmj.com/lookup/pmidlookup?view=long&pmid=26534730 .bmjopen-2015-008190 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Willis CD, Saul JE, Bitz J, Pompu K, Best A, Jackson B. Improving organizational capacity to address health literacy in public health: a rapid realist review. Public Health. 2014 Jun;128(6):515–24. doi: 10.1016/j.puhe.2014.01.014.S0033-3506(14)00031-6 [DOI] [PubMed] [Google Scholar]
- 33.Gillard S, Dare C, Hardy J, Nyikavaranda P, Rowan Olive R, Shah P, Birken M, Foye U, Ocloo J, Pearce E, Stefanidou T, Pitman A, Simpson A, Johnson S, Lloyd-Evans B, NIHR Mental Health Policy Research Unit Covid coproduction research group Experiences of living with mental health problems during the COVID-19 pandemic in the UK: a coproduced, participatory qualitative interview study. Soc Psychiatry Psychiatr Epidemiol. 2021 Aug;56(8):1447–57. doi: 10.1007/s00127-021-02051-7. http://europepmc.org/abstract/MED/33665680 .10.1007/s00127-021-02051-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, Shamseer L, Tetzlaff JM, Akl EA, Brennan SE, Chou R, Glanville J, Grimshaw JM, Hróbjartsson A, Lalu MM, Li T, Loder EW, Mayo-Wilson E, McDonald S, McGuinness LA, Stewart LA, Thomas J, Tricco AC, Welch VA, Whiting P, Moher D. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Int J Surg. 2021 Apr;88:105906. doi: 10.1016/j.ijsu.2021.105906.S1743-9191(21)00040-6 [DOI] [PubMed] [Google Scholar]
- 35.Thomas J, Brunton J, Graziosi S. EPPI-Reviewer 4.0: software for research synthesis. Evidence for Policy and Practice Information Centre. 2010. [2022-03-18]. https://eppi.ioe.ac.uk/cms/er4/Features/tabid/3396/Default.aspx .
- 36.Severe J, Tang R, Horbatch F, Onishchenko R, Naini V, Blazek MC. Factors influencing patients' initial decisions regarding telepsychiatry participation during the COVID-19 pandemic: telephone-based survey. JMIR Form Res. 2020 Dec 22;4(12):e25469. doi: 10.2196/25469. https://formative.jmir.org/2020/12/e25469/ v4i12e25469 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Lopez A, Schwenk S, Schneck CD, Griffin RJ, Mishkind MC. Technology-based mental health treatment and the impact on the therapeutic alliance. Curr Psychiatry Rep. 2019 Jul 08;21(8):76. doi: 10.1007/s11920-019-1055-7.10.1007/s11920-019-1055-7 [DOI] [PubMed] [Google Scholar]
- 38.Shore P, Goranson A, Ward MF, Lu MW. Meeting veterans where they're @: a VA Home-Based Telemental Health (HBTMH) pilot program. Int J Psychiatry Med. 2014;48(1):5–17. doi: 10.2190/PM.48.1.b.1N62181211343020 [DOI] [PubMed] [Google Scholar]
- 39.Bierbooms JJ, van Haaren M, IJsselsteijn WA, de Kort YA, Feijt M, Bongers IM. Integration of online treatment into the "new normal" in mental health care in post-COVID-19 times: exploratory qualitative study. JMIR Form Res. 2020 Oct 08;4(10):e21344. doi: 10.2196/21344. https://formative.jmir.org/2020/10/e21344/ v4i10e21344 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Chen JA, Chung W, Young SK, Tuttle MC, Collins MB, Darghouth SL, Longley R, Levy R, Razafsha M, Kerner JC, Wozniak J, Huffman JC. COVID-19 and telepsychiatry: early outpatient experiences and implications for the future. Gen Hosp Psychiatry. 2020;66:89–95. doi: 10.1016/j.genhosppsych.2020.07.002. http://europepmc.org/abstract/MED/32750604 .S0163-8343(20)30103-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Ghaneirad E, Groba S, Bleich S, Szycik GR. [Use of outpatient psychotherapy via video consultation] Psychotherapeut (Berl) 2021;66(3):240–6. doi: 10.1007/s00278-021-00497-3. http://europepmc.org/abstract/MED/33642699 .497 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Ghosh A, Mahintamani T, Pillai RR, Mattoo SK, Basu D. Telemedicine-assisted stepwise approach of service delivery for substance use disorders in India. Asian J Psychiatr. 2021 Apr;58:102582. doi: 10.1016/j.ajp.2021.102582.S1876-2018(21)00038-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Kanellopoulos D, Bueno Castellano C, McGlynn L, Gerber S, Francois D, Rosenblum L, Runge M, Sanchez-Barranco P, Alexopoulos GS. Implementation of telehealth services for inpatient psychiatric Covid-19 positive patients: a blueprint for adapting the milieu. Gen Hosp Psychiatry. 2021;68:113–4. doi: 10.1016/j.genhosppsych.2020.08.011. http://europepmc.org/abstract/MED/32972728 .S0163-8343(20)30130-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Lakeman R, Crighton J. The impact of social distancing on people with borderline personality disorder: the views of dialectical behavioural therapists. Issues Ment Health Nurs. 2021 May;42(5):410–6. doi: 10.1080/01612840.2020.1817208. [DOI] [PubMed] [Google Scholar]
- 45.Trying to connect: the importance of choice in remote mental health services. Mind. 2021. [2022-01-31]. https://www.mind.org.uk/media/8575/mind-20582-trying-to-connect-report-aw2-welsh-recommendations-lr.pdf .
- 46.Digital inclusion in mental health: a guide to help increase choice and improve access to digital mental health services. Mental Health Network, NHS Confederation. 2020. Dec 8, [2022-01-21]. https://www.nhsconfed.org/publications/digital-inclusion-mental-health .
- 47.Simpson J, Doze S, Urness D, Hailey D, Jacobs P. Evaluation of a routine telepsychiatry service. J Telemed Telecare. 2001;7(2):90–8. doi: 10.1258/1357633011936219. [DOI] [PubMed] [Google Scholar]
- 48.Watson A, Mellotte H, Hardy A, Peters E, Keen N, Kane F. The digital divide: factors impacting on uptake of remote therapy in a South London psychological therapy service for people with psychosis. J Ment Health (forthcoming) 2021 Jul 28;:1–8. doi: 10.1080/09638237.2021.1952955. [DOI] [PubMed] [Google Scholar]
- 49.Disney L, Mowbray O, Evans D. Telemental health use and refugee mental health providers following COVID-19 pandemic. Clin Soc Work J. 2021;49(4):463–70. doi: 10.1007/s10615-021-00808-w. http://europepmc.org/abstract/MED/33967352 .808 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50.Costa M, Reis G, Pavlo A, Bellamy C, Ponte K, Davidson L. Tele-mental health utilization among people with mental illness to access care during the COVID-19 pandemic. Community Ment Health J. 2021 May;57(4):720–6. doi: 10.1007/s10597-021-00789-7. http://europepmc.org/abstract/MED/33566269 .10.1007/s10597-021-00789-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.Bommersbach T, Dube L, Li L. Mental health staff perceptions of improvement opportunities around COVID-19: a mixed-methods analysis. Psychiatr Q. 2021 Sep;92(3):1079–92. doi: 10.1007/s11126-021-09890-2. http://europepmc.org/abstract/MED/33575954 .10.1007/s11126-021-09890-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52.Buckman JE, Saunders R, Leibowitz J, Minton R. The barriers, benefits and training needs of clinicians delivering psychological therapy via video. Behav Cogn Psychother. 2021 May 10;49(6):696–720. doi: 10.1017/S1352465821000187.S1352465821000187 [DOI] [PubMed] [Google Scholar]
- 53.Foye U, Dalton-Locke C, Harju-Seppänen J, Lane R, Beames L, Vera San Juan N, Johnson S, Simpson A. How has COVID-19 affected mental health nurses and the delivery of mental health nursing care in the UK? Results of a mixed-methods study. J Psychiatr Ment Health Nurs. 2021 Apr;28(2):126–37. doi: 10.1111/jpm.12745. http://europepmc.org/abstract/MED/33608956 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 54.Frayn M, Fojtu C, Juarascio A. COVID-19 and binge eating: patient perceptions of eating disorder symptoms, tele-therapy, and treatment implications. Curr Psychol. 2021;40(12):6249–58. doi: 10.1007/s12144-021-01494-0. http://europepmc.org/abstract/MED/33623352 .1494 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 55.Hawke LD, Sheikhan NY, MacCon K, Henderson J. Going virtual: youth attitudes toward and experiences of virtual mental health and substance use services during the COVID-19 pandemic. BMC Health Serv Res. 2021 Apr 14;21(1):340. doi: 10.1186/s12913-021-06321-7. https://bmchealthservres.biomedcentral.com/articles/10.1186/s12913-021-06321-7 .10.1186/s12913-021-06321-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 56.He Z, Chen J, Pan K, Yue Y, Cheung T, Yuan Y, Du N, Zhao Y, Feng Y, Zhou D, Zhou Y, Lu F, Chen Y, He M, Xiang YT. The development of the 'COVID-19 Psychological Resilience Model' and its efficacy during the COVID-19 pandemic in China. Int J Biol Sci. 2020 Aug 27;16(15):2828–34. doi: 10.7150/ijbs.50127. https://www.ijbs.com/v16p2828.htm .ijbsv16p2828 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 57.Lecomte T, Abdel-Baki A, Francoeur A, Cloutier B, Leboeuf A, Abadie P, Villeneuve M, Guay S. Group therapy via videoconferencing for individuals with early psychosis: a pilot study. Early Interv Psychiatry. 2021 Dec;15(6):1595–601. doi: 10.1111/eip.13099. [DOI] [PubMed] [Google Scholar]
- 58.Medalia A, Lynch DA, Herlands T. Telehealth conversion of serious mental illness recovery services during the COVID-19 crisis. Psychiatr Serv. 2020 Aug 01;71(8):872. doi: 10.1176/appi.ps.71705. [DOI] [PubMed] [Google Scholar]
- 59.MHIN Team Inclusive messaging: supporting refugee mental health in the United States during COVID-19. The Mental Health Innovation Network. 2020. Apr 19, [2022-01-31]. https://www.mhinnovation.net/blog/2020/apr/19/inclusive-messaging-supporting-refugee-mental-health-united-states-during-covid-19 .
- 60.Ogueji IA, Amusa AO, Olofe OJ, Omotoso EB. Willingness and barriers to utilizing e-therapy services: a Nigerian general population qualitative study. J Human Behav Soc Environ. 2022;32(2):214–28. doi: 10.1080/10911359.2021.1894300. [DOI] [Google Scholar]
- 61.Arighi A, Fumagalli GG, Carandini T, Pietroboni AM, De Riz MA, Galimberti D, Scarpini E. Facing the digital divide into a dementia clinic during COVID-19 pandemic: caregiver age matters. Neurol Sci. 2021 Apr;42(4):1247–51. doi: 10.1007/s10072-020-05009-w. http://europepmc.org/abstract/MED/33459891 .10.1007/s10072-020-05009-w [DOI] [PMC free article] [PubMed] [Google Scholar]
- 62.Castillo M, Conte B, Hinkes S, Mathew M, Na CJ, Norindr A, Serota DP, Forrest DW, Deshpande AR, Bartholomew TS, Tookes HE. Implementation of a medical student-run telemedicine program for medications for opioid use disorder during the COVID-19 pandemic. Harm Reduct J. 2020 Nov 17;17(1):88. doi: 10.1186/s12954-020-00438-4. https://harmreductionjournal.biomedcentral.com/articles/10.1186/s12954-020-00438-4 .10.1186/s12954-020-00438-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 63.Conn DK, Madan R, Lam J, Patterson T, Skirten S. Program evaluation of a telepsychiatry service for older adults connecting a university-affiliated geriatric center to a rural psychogeriatric outreach service in Northwest Ontario, Canada. Int Psychogeriatr. 2013 Nov;25(11):1795–800. doi: 10.1017/S104161021300118X.S104161021300118X [DOI] [PubMed] [Google Scholar]
- 64.Godleski L, Cervone D, Vogel D, Rooney M. Home telemental health implementation and outcomes using electronic messaging. J Telemed Telecare. 2012 Jan;18(1):17–9. doi: 10.1258/jtt.2011.100919.jtt.2011.100919 [DOI] [PubMed] [Google Scholar]
- 65.Greenwood J, Chamberlain C, Parker G. Evaluation of a rural telepsychiatry service. Australas Psychiatry. 2004 Sep;12(3):268–72. doi: 10.1080/j.1039-8562.2004.02097.x.APY2097 [DOI] [PubMed] [Google Scholar]
- 66.Humer E, Stippl P, Pieh C, Schimböck W, Probst T. Psychotherapy via the Internet: what programs do psychotherapists use, how well-informed do they feel, and what are their wishes for continuous education? Int J Environ Res Public Health. 2020 Nov 05;17(21):8182. doi: 10.3390/ijerph17218182. https://www.mdpi.com/resolver?pii=ijerph17218182 .ijerph17218182 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 67.Juarez-Reyes M, Mui HZ, Kling SM, Brown-Johnson C. Accessing behavioral health care during COVID: rapid transition from in-person to teleconferencing medical group visits. Ther Adv Chronic Dis. 2021 Feb 12;12:2040622321990269. doi: 10.1177/2040622321990269. https://journals.sagepub.com/doi/10.1177/2040622321990269?url_ver=Z39.88-2003&rfr_id=ori:rid:crossref.org&rfr_dat=cr_pub%3dpubmed .10.1177_2040622321990269 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 68.Lin X, Swift J, Cheng Y, An Q, Liang H, Wang Y, Jia X. The psychological hotline services quality survey during the pandemic of COVID-19 in mainland China. Int J Mental Health Promotion. 2020;22(3):109–13. doi: 10.32604/ijmhp.2020.011128. [DOI] [Google Scholar]
- 69.Mental Health Commission of Canada. Ottawa, ON, Canada: Mental Health Commission of Canada; 2014. [2022-01-31]. E-Mental health in Canada: Transforming the mental health system using technology. https://mentalhealthcommission.ca/wp-content/uploads/2021/09/MHCC_E-Mental_Health-Briefing_Document_ENG_0.pdf . [Google Scholar]
- 70.Hearing Voices USA groups go virtual in response to COVID-19. Open Excellence. 2020. [2022-01-31]. https://openexcellence.org/hearing-voices-usa-groups-go-virtual-in-response-to-covid-19/
- 71.Pote H, Rees A, Holloway-Biddle C, Griffith E. Workforce challenges in digital health implementation: how are clinical psychology training programmes developing digital competences? Digit Health. 2021 Feb 11;7:2055207620985396. doi: 10.1177/2055207620985396. https://journals.sagepub.com/doi/10.1177/2055207620985396?url_ver=Z39.88-2003&rfr_id=ori:rid:crossref.org&rfr_dat=cr_pub%3dpubmed .10.1177_2055207620985396 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 72.Rabinowitz T, Murphy KM, Amour JL, Ricci MA, Caputo MP, Newhouse PA. Benefits of a telepsychiatry consultation service for rural nursing home residents. Telemed J E Health. 2010;16(1):34–40. doi: 10.1089/tmj.2009.0088. [DOI] [PubMed] [Google Scholar]
- 73.Scharff A, Breiner CE, Ueno LF, Underwood SB, Merritt EC, Welch LM, Fonda C, Weil Malatras J, Lin B, Hormes JM, Pieterse AL, Gordis EB, Halpern LF, Pazienza R, Litchford GB. Shifting a training clinic to teletherapy during the COVID-19 pandemic: a trainee perspective. Couns Psychol Q. 2020 Jun 28;34(3-4):676–86. doi: 10.1080/09515070.2020.1786668. [DOI] [Google Scholar]
- 74.Williams H, Whelan A, The RADAR Team An investigation into access to digital inclusion for healthcare for the homeless population. Seaview. 2017. Apr, [2022-03-18]. https://amhp.org.uk/app/uploads/2018/11/Digital-Inclusion-and-Homeless-People.pdf .
- 75.Yellowlees P, Nakagawa K, Pakyurek M, Hanson A, Elder J, Kales HC. Rapid conversion of an outpatient psychiatric clinic to a 100% virtual telepsychiatry clinic in response to COVID-19. Psychiatr Serv. 2020 Jul 01;71(7):749–52. doi: 10.1176/appi.ps.202000230. [DOI] [PubMed] [Google Scholar]
- 76.Fogler JM, Normand S, O'Dea N, Mautone JA, Featherston M, Power TJ, Nissley-Tsiopinis J. Implementing group parent training in telepsychology: lessons learned during the COVID-19 pandemic. J Pediatr Psychol. 2020 Oct 01;45(9):983–9. doi: 10.1093/jpepsy/jsaa085. http://europepmc.org/abstract/MED/32940702 .5907492 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 77.Ready, set, connect!: exploring young people's perceived barriers to accessing video/phone mental health appointments - a solution-focused consultation. Healthwatch. 2021. [2022-03-18]. https://www.healthwatchbrightonandhove.co.uk/sites/healthwatchbrightonandhove.co.uk/files/RSC_report_FINAL_1.pdf .
- 78.Lichstein KL, Scogin F, Thomas SJ, DiNapoli EA, Dillon HR, McFadden A. Telehealth cognitive behavior therapy for co-occurring insomnia and depression symptoms in older adults. J Clin Psychol. 2013 Oct;69(10):1056–65. doi: 10.1002/jclp.22030. http://europepmc.org/abstract/MED/24014056 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 79.Lodder A, Papadopoulos C, Randhawa G. Using a blended format (videoconference and face-to-face) to deliver a group psychosocial intervention to parents of autistic children. Internet Interv. 2020 Aug 26;21:100336. doi: 10.1016/j.invent.2020.100336. https://linkinghub.elsevier.com/retrieve/pii/S2214-7829(20)30102-0 .S2214-7829(20)30102-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 80.McBeath AG, du Plock S, Bager-Charleson S. The challenges and experiences of psychotherapists working remotely during the coronavirus pandemic. Couns Psychother Res. 2020 Sep;20(3):394–405. doi: 10.1002/capr.12326. http://europepmc.org/abstract/MED/32837329 .CAPR12326 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 81.Pugh M, Bell T, Dixon A. Delivering tele-chairwork: a qualitative survey of expert therapists. Psychother Res. 2021 Sep;31(7):843–58. doi: 10.1080/10503307.2020.1854486. [DOI] [PubMed] [Google Scholar]
- 82.Uscher-Pines L, Sousa J, Raja P, Mehrotra A, Barnett M, Huskamp HA. Treatment of opioid use disorder during COVID-19: experiences of clinicians transitioning to telemedicine. J Subst Abuse Treat. 2020 Nov;118:108124. doi: 10.1016/j.jsat.2020.108124. http://europepmc.org/abstract/MED/32893047 .S0740-5472(20)30380-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 83.Aafjes-van Doorn K, Békés V, Prout TA. Grappling with our therapeutic relationship and professional self-doubt during COVID-19: will we use video therapy again? Couns Psychol Q. 2021;34(3-4):473–84. doi: 10.1080/09515070.2020.1773404. [DOI] [Google Scholar]
- 84.Newbronner E, Spanakis P, Wadman R, Crosland S, Heron P, Johnston G, Walker L, Gilbody S, Peckham E. Exploring access to mental health and primary care services for people with severe mental illness during the COVID-19 restrictions. Front Psychiatry. 2021 Jan 21;12:799885. doi: 10.3389/fpsyt.2021.799885. doi: 10.3389/fpsyt.2021.799885. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 85.O'Dell SM, Hosterman SJ, Parikh MR, Winnick JB, Meadows TJ. Chasing the curve: program description of the Geisinger primary care behavioral health virtual first response to COVID-19. J Rural Mental Health. 2021 Apr;45(2):95–106. doi: 10.1037/rmh0000180. [DOI] [Google Scholar]
- 86.Barney A, Buckelew S, Mesheriakova V, Raymond-Flesch M. The COVID-19 pandemic and rapid implementation of adolescent and young adult telemedicine: challenges and opportunities for innovation. J Adolesc Health. 2020 Aug;67(2):164–71. doi: 10.1016/j.jadohealth.2020.05.006. http://europepmc.org/abstract/MED/32410810 .S1054-139X(20)30225-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 87.Connolly SL, Stolzmann KL, Heyworth L, Weaver KR, Bauer MS, Miller CJ. Rapid increase in telemental health within the Department of Veterans Affairs during the COVID-19 pandemic. Telemed J E Health. 2021 Apr;27(4):454–8. doi: 10.1089/tmj.2020.0233. [DOI] [PubMed] [Google Scholar]
- 88.Hopkins L, Pedwell G. The COVID PIVOT - re-orienting child and youth mental health care in the light of pandemic restrictions. Psychiatr Q. 2021 Sep;92(3):1259–70. doi: 10.1007/s11126-021-09909-8. http://europepmc.org/abstract/MED/33754240 .10.1007/s11126-021-09909-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 89.Peralta EA, Taveras M. Effectiveness of teleconsultation use in access to mental health services during the coronavirus disease 2019 pandemic in the Dominican Republic. Indian J Psychiatry. 2020 Sep;62(Suppl 3):S492–4. doi: 10.4103/psychiatry.IndianJPsychiatry_1047_20. http://europepmc.org/abstract/MED/33227069 .IJPsy-62-492 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 90.Hensel JM, Yang R, Vigod SN, Desveaux L. Videoconferencing at home for psychotherapy in the postpartum period: identifying drivers of successful engagement and important therapeutic conditions for meaningful use. Couns Psychother Res. 2020 Dec 14;21(3):535–44. doi: 10.1002/capr.12372. [DOI] [Google Scholar]
- 91.Liberati E, Richards N, Parker J, Willars J, Scott D, Boydell N, Pinfold V, Martin G, Dixon-Woods M, Jones P. Remote care for mental health: qualitative study with service users, carers and staff during the COVID-19 pandemic. BMJ Open. 2021 Apr 22;11(4):e049210. doi: 10.1136/bmjopen-2021-049210. https://bmjopen.bmj.com/lookup/pmidlookup?view=long&pmid=33888531 .bmjopen-2021-049210 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 92.Sehlo MG, Youssef UM, Elshami MI, Elrafey DS, Elgohari HM. Telepsychiatry versus face-to-face consultation in COVID-19 era from the patients' perspective. Asian J Psychiatr. 2021 May;59:102641. doi: 10.1016/j.ajp.2021.102641. http://europepmc.org/abstract/MED/33838435 .S1876-2018(21)00097-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 93.Adamou M, Jones SL, Fullen T, Galab N, Abbott K, Yasmeen S. Remote assessment in adults with autism or ADHD: a service user satisfaction survey. PLoS One. 2021 Mar 25;16(3):e0249237. doi: 10.1371/journal.pone.0249237. https://dx.plos.org/10.1371/journal.pone.0249237 .PONE-D-20-39665 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 94.Schueller SM, Hunter JF, Figueroa C, Aguilera A. Use of digital mental health for marginalized and underserved populations. Curr Treat Options Psych. 2019 Jul 5;6(3):243–55. doi: 10.1007/s40501-019-00181-z. [DOI] [Google Scholar]
- 95.Guinart D, Marcy P, Hauser M, Dwyer M, Kane JM. Patient attitudes toward telepsychiatry during the COVID-19 pandemic: a nationwide, multisite survey. JMIR Ment Health. 2020 Dec 22;7(12):e24761. doi: 10.2196/24761. https://mental.jmir.org/2020/12/e24761/ v7i12e24761 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 96.King R, Bambling M, Lloyd C, Gomurra R, Smith S, Reid W, Wegner K. Online counselling: the motives and experiences of young people who choose the Internet instead of face-to-face or telephone counselling. Couns Psychother Res. 2006 Sep;6(3):169–74. doi: 10.1080/14733140600848179. [DOI] [Google Scholar]
- 97.Trying to Connect: the importance of choice in remote mental health services (Young people data cut) Mind. 2021. [2022-03-18]. https://www.mind.org.uk/media/9149/trying-to-connect-yp-briefing-september-2021.pdf .
- 98.24/7 urgent mental health helplines available across the country. National Health Service England. 2021. [2022-01-31]. https://www.england.nhs.uk/mental-health/case-studies/24-7-urgent-mental-health-helplines-available-across-the-country/
- 99.Nissling L, Fahlke C, Lilja JL, Skoglund I, Weineland S. Primary care peer-supported Internet-mediated psychological treatment for adults with anxiety disorders: mixed methods study. JMIR Form Res. 2020 Aug 20;4(8):e19226. doi: 10.2196/19226. https://formative.jmir.org/2020/8/e19226/ v4i8e19226 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 100.Orlowski S, Lawn S, Matthews B, Venning A, Wyld K, Jones G, Winsall M, Antezana G, Schrader G, Bidargaddi N. The promise and the reality: a mental health workforce perspective on technology-enhanced youth mental health service delivery. BMC Health Serv Res. 2016 Oct 10;16(1):562. doi: 10.1186/s12913-016-1790-y. https://bmchealthservres.biomedcentral.com/articles/10.1186/s12913-016-1790-y .10.1186/s12913-016-1790-y [DOI] [PMC free article] [PubMed] [Google Scholar]
- 101.Shklarski L, Abrams A, Bakst E. Navigating changes in the physical and psychological spaces of psychotherapists during Covid-19: when home becomes the office. Pract Innov. 2021 Mar;6(1):55–66. doi: 10.1037/pri0000138. [DOI] [Google Scholar]
- 102.Simon N, Ploszajski M, Lewis C, Smallman K, Roberts NP, Kitchiner NJ, Brookes-Howell L, Bisson JI. Internet-based psychological therapies: a qualitative study of National Health Service commissioners and managers views. Psychol Psychother. 2021 Dec;94(4):994–1014. doi: 10.1111/papt.12341. [DOI] [PubMed] [Google Scholar]
- 103.YoungMinds submission to the COVID Committee's inquiry into 'Living Online: The Long Term Impact of Wellbeing'. London, UK: YoungMinds; 2020. [Google Scholar]
- 104.Considerations for psychologists working with children and young people using online video platforms. The British Psychological Society. 2020. [2022-03-18]. https://www.bps.org.uk/guideline/considerations-psychologists-working-children-and-young-people-using-online-video .
- 105.Choi NG, Hegel MT, Marti CN, Marinucci ML, Sirrianni L, Bruce ML. Telehealth problem-solving therapy for depressed low-income homebound older adults. Am J Geriatr Psychiatry. 2014 Mar;22(3):263–71. doi: 10.1097/JGP.0b013e318266b356. http://europepmc.org/abstract/MED/23567376 .S1064-7481(13)00042-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 106.Gaddy S, Gallardo R, McCluskey S, Moore L, Peuser A, Rotert R, Stypulkoski C, LaGasse AB. COVID-19 and music therapists’ employment, service delivery, perceived stress, and hope: a descriptive study. Music Ther Perspect. 2020 Sep 8;38(2):157–66. doi: 10.1093/mtp/miaa018. [DOI] [Google Scholar]
- 107.CWTCH: Connecting with Telehealth to Healthcare and Communities. The Health Foundation. 2020. [2022-03-18]. https://www.health.org.uk/cwtch-connecting-with-telehealth-to-healthcare-and-communities .
- 108.Hernandez-Tejada MA, Zoller JS, Ruggiero KJ, Kazley AS, Acierno R. Early treatment withdrawal from evidence-based psychotherapy for PTSD: telemedicine and in-person parameters. Int J Psychiatry Med. 2014;48(1):33–55. doi: 10.2190/PM.48.1.d.A8Q3122G67L43013 [DOI] [PubMed] [Google Scholar]
- 109.Sheehan R, Dalton-Locke C, Ali A, Totsika V, San Juan NV, Hassiotis A. Mental healthcare and service user impact of the COVID-19 pandemic: results of a UK survey of staff working with people with intellectual disability and developmental disorders. medRxiv. 2020 doi: 10.1101/2020.09.01.20178848. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 110.Simpson J, Doze S, Urness D, Hailey D, Jacobs P. Telepsychiatry as a routine service-the perspective of the patient. J Telemed Telecare. 2001;7(3):155–60. doi: 10.1258/1357633011936318. [DOI] [PubMed] [Google Scholar]
- 111.Uscher-Pines L, Sousa J, Raja P, Mehrotra A, Barnett ML, Huskamp HA. Suddenly becoming a "virtual doctor": experiences of psychiatrists transitioning to telemedicine during the COVID-19 pandemic. Psychiatr Serv. 2020 Nov 01;71(11):1143–50. doi: 10.1176/appi.ps.202000250. http://europepmc.org/abstract/MED/32933411 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 112.Wilson CA, Dalton-Locke C, Johnson S, Simpson A, Oram S, Howard LM. Challenges and opportunities of the COVID-19 pandemic for perinatal mental health care: a mixed-methods study of mental health care staff. Arch Womens Ment Health. 2021 Oct;24(5):749–57. doi: 10.1007/s00737-021-01108-5. http://europepmc.org/abstract/MED/33830374 .10.1007/s00737-021-01108-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 113.Brooks E, Novins DK, Noe T, Bair B, Dailey N, Lowe J, Richardson Jr WJ, Hawthorne K, Shore JH. Reaching rural communities with culturally appropriate care: a model for adapting remote monitoring to American Indian veterans with posttraumatic stress disorder. Telemed J E Health. 2013 Apr;19(4):272–7. doi: 10.1089/tmj.2012.0117. [DOI] [PubMed] [Google Scholar]
- 114.Moslehi S, Aubrey-Jones D, Knowles M, Obeney-Williams J, Leveson S, Aref-Adib G. Cyberpsychiatry versus COVID-19: using video consultation to improve clinical care in an in-patient psychiatric unit. BJPsych Int. 2021 Aug;18(3):E10. doi: 10.1192/bji.2021.19. http://europepmc.org/abstract/MED/34382960 .S2056474021000192 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 115.Whistance B, Wright P, Brind J, Johns G, Khalil S, Ahuja A. 'Ask us about dementia' pilot support service independent evaluation. Technology Enabled Care (TEC) CYMRU. 2021. [2022-03-18]. https://tinyurl.com/2p9u56aj .
- 116.Colle R, Ait Tayeb AE, de Larminat D, Commery L, Boniface B, Lasica PA, Gressier F, Mecifi R, Rotenberg S, Rigal A, Zitoun S, Mezzacappa A, Nicolicea C, Chaneac E, Martin S, Choucha W, Hardy P, Schouman-Claeys E, Corruble E. Short-term acceptability by patients and psychiatrists of the turn to psychiatric teleconsultation in the context of the COVID-19 pandemic. Psychiatry Clin Neurosci. 2020 Aug;74(8):443–4. doi: 10.1111/pcn.13081. doi: 10.1111/pcn.13081. [DOI] [PubMed] [Google Scholar]
- 117.Crowe T, Jani S, Jani S, Jani N, Jani R. A pilot program in rural telepsychiatry for deaf and hard of hearing populations. Heliyon. 2016 Mar;2(3):e00077. doi: 10.1016/j.heliyon.2016.e00077. https://linkinghub.elsevier.com/retrieve/pii/S2405-8440(15)30432-1 .S2405-8440(15)30432-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 118.Zheng P, Gray MJ. Telehealth-based therapy connecting rural Mandarin-speaking traumatized clients with a Mandarin-speaking therapist. Clin Case Stud. 2014 Apr 13;13(6):514–27. doi: 10.1177/1534650114529182. [DOI] [Google Scholar]
- 119.Holland M, Smith V, Robertson N, Heafield G. Staying connected decision making tools, learning and case studies. Health Innovation Network. 2020. [2022-03-18]. https://healthinnovationnetwork.com/wp-content/uploads/2020/12/MH-Remote-Consultations-Staying-Connected-Report.pdf .
- 120.Dores AR, Geraldo A, Carvalho IP, Barbosa F. The use of new digital information and communication technologies in psychological counseling during the COVID-19 pandemic. Int J Environ Res Public Health. 2020 Oct 21;17(20):7663. doi: 10.3390/ijerph17207663. https://www.mdpi.com/resolver?pii=ijerph17207663 .ijerph17207663 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 121.Khanna R, Murnane T, Kumar S, Rolfe T, Dimitrieski S, McKeown M, Ejareh Dar M, Gavson L, Gandhi C. Making working from home work: reflections on adapting to change. Australas Psychiatry. 2020 Oct;28(5):504–7. doi: 10.1177/1039856220953701. [DOI] [PubMed] [Google Scholar]
- 122.Boldrini T, Schiano Lomoriello A, Del Corno F, Lingiardi V, Salcuni S. Psychotherapy during COVID-19: how the clinical practice of Italian psychotherapists changed during the pandemic. Front Psychol. 2020 Oct 21;11:591170. doi: 10.3389/fpsyg.2020.591170. doi: 10.3389/fpsyg.2020.591170. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 123.Moore J. Rethink Mental Illness. 2020. Jun 17, [2022-03-18]. https://www.rethink.org/get-involved/campaign-with-us/resources-and-reports/severe-mental-illness-and-covid-19-service-support-and-digital-solutions/
- 124.Delivering mental health services digitally: group sessions. Mental Health Network NHS Confederation. 2020. Jul, [2022-03-18]. https://www.nhsconfed.org/sites/default/files/media/Delivering-mental-health-services-digitally.pdf .
- 125.Barratt C. Mental health and houses in multiple occupation: the challenge of turning research into practice. University of Essex. [2022-01-31]. https://www.emcouncils.gov.uk/write/Mental_Health_and_HMOs_Caroline__Barratt.pdf .
- 126.De Witte NA, Carlbring P, Etzelmueller A, Nordgreen T, Karekla M, Haddouk L, Belmont A, Øverland S, Abi-Habib R, Bernaerts S, Brugnera A, Compare A, Duque A, Ebert DD, Eimontas J, Kassianos AP, Salgado J, Schwerdtfeger A, Tohme P, Van Assche E, Van Daele T. Online consultations in mental healthcare during the COVID-19 outbreak: an international survey study on professionals' motivations and perceived barriers. Internet Interv. 2021 Sep;25:100405. doi: 10.1016/j.invent.2021.100405. https://linkinghub.elsevier.com/retrieve/pii/S2214-7829(21)00045-2 .S2214-7829(21)00045-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 127.Mad Covid. Edwards B, Grundy A, Bradford E, Aitch N. COVID-19: Recommendations for Mental Health Services. Mad Covid. 2020. [2022-03-18]. http://www.infocop.es/pdf/Recommendations%20for%20Mental%20Health%20Services.pdf .
- 128.Mental health COVID-19 children and young people case studies. National Health Service England. 2021. [2022-01-31]. https://tinyurl.com/yc666ed4 .
- 129.Remote mental health survey. Healthwatch. 2021. [2022-03-18]. https://www.healthwatchbucks.co.uk/2021/07/remote-mental-health-survey-report/
- 130.James K. Remote mental health interventions for young people: a rapid review of the evidence. Youth Access. 2020. [2022-03-18]. https://www.youthaccess.org.uk/downloads/remote-mental-health-interventions-for-young-people---rapid-review.pdf .
- 131.Lannin DG, Vogel DL, Brenner RE, Abraham WT, Heath PJ. Does self-stigma reduce the probability of seeking mental health information? J Couns Psychol. 2016 Apr;63(3):351–8. doi: 10.1037/cou0000108.2015-39652-001 [DOI] [PubMed] [Google Scholar]
- 132.Benaque A, Gurruchaga MJ, Abdelnour C, Hernández I, Cañabate P, Alegret M, Rodríguez I, Rosende-Roca M, Tartari JP, Esteban E, López R, Gil S, Vargas L, Mauleón A, Espinosa A, Ortega G, Sanabria A, Pérez A, Alarcón E, González-Pérez A, Marquié M, Valero S, Tárraga L, Ruiz A, Boada M, Research Center and Memory Clinic‚ Fundació ACE Dementia care in times of COVID-19: experience at Fundació ACE in Barcelona, Spain. J Alzheimers Dis. 2020;76(1):33–40. doi: 10.3233/JAD-200547. http://europepmc.org/abstract/MED/32538856 .JAD200547 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 133.Martin R, Sturt J, Griffiths F. The impact of digital communication on adolescent to adult mental health service transitions. J Res Nurs. 2020 May;25(3):277–88. doi: 10.1177/1744987120915746. http://europepmc.org/abstract/MED/34394636 .10.1177_1744987120915746 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 134.Pierce BS, Perrin PB, Tyler CM, McKee GB, Watson JD. The COVID-19 telepsychology revolution: a national study of pandemic-based changes in U.S. mental health care delivery. Am Psychol. 2021 Jan;76(1):14–25. doi: 10.1037/amp0000722.2020-61592-001 [DOI] [PubMed] [Google Scholar]
- 135.Rosen CS, Morland LA, Glassman LH, Marx BP, Weaver K, Smith CA, Pollack S, Schnurr PP. Virtual mental health care in the Veterans Health Administration's immediate response to coronavirus disease-19. Am Psychol. 2021 Jan;76(1):26–38. doi: 10.1037/amp0000751.2020-81589-001 [DOI] [PubMed] [Google Scholar]
- 136.Smith J, Gillon E. Therapists' experiences of providing online counselling: a qualitative study. Couns Psychother Res. 2021 May 04;21(3):545–54. doi: 10.1002/capr.12408. [DOI] [Google Scholar]
- 137.Bambling M, King R, Reid W, Wegner K. Online counselling: the experience of counsellors providing synchronous single-session counselling to young people. Couns Psychother Res. 2008 Jun;8(2):110–6. doi: 10.1080/14733140802055011. [DOI] [Google Scholar]
- 138.Grover S, Mehra A, Sahoo S, Avasthi A, Tripathi A, D'Souza A, Saha G, Jagadhisha A, Gowda M, Vaishnav M, Singh O, Dalal PK, Kumar P. State of mental health services in various training centers in India during the lockdown and COVID-19 pandemic. Indian J Psychiatry. 2020;62(4):363–9. doi: 10.4103/psychiatry.IndianJPsychiatry_567_20. http://europepmc.org/abstract/MED/33165355 .IJPsy-62-363 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 139.Olwill C, Mc Nally D, Douglas L. Psychiatrist experience of remote consultations by telephone in an outpatient psychiatric department during the COVID-19 pandemic. Ir J Psychol Med. 2021 Jun;38(2):132–9. doi: 10.1017/ipm.2020.51. http://europepmc.org/abstract/MED/32438945 .S0790966720000518 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 140.Sasangohar F, Bradshaw MR, Carlson MM, Flack JN, Fowler JC, Freeland D, Head J, Marder K, Orme W, Weinstein B, Kolman JM, Kash B, Madan A. Adapting an outpatient psychiatric clinic to telehealth during the COVID-19 pandemic: a practice perspective. J Med Internet Res. 2020 Oct 01;22(10):e22523. doi: 10.2196/22523. https://www.jmir.org/2020/10/e22523/ v22i10e22523 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 141.LeRouge CM, Garfield MJ, Hevner AR. Patient perspectives of telemedicine quality. Patient Prefer Adherence. 2015 Dec 24;9:25–40. doi: 10.2147/PPA.S67506. doi: 10.2147/PPA.S67506.ppa-9-025 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 142.Pinto RZ, Ferreira ML, Oliveira VC, Franco MR, Adams R, Maher CG, Ferreira PH. Patient-centred communication is associated with positive therapeutic alliance: a systematic review. J Physiother. 2012;58(2):77–87. doi: 10.1016/S1836-9553(12)70087-5. https://linkinghub.elsevier.com/retrieve/pii/S1836-9553(12)70087-5 .S1836-9553(12)70087-5 [DOI] [PubMed] [Google Scholar]
- 143.Wyler H, Liebrenz M, Ajdacic-Gross V, Seifritz E, Young S, Burger P, Buadze A. Treatment provision for adults with ADHD during the COVID-19 pandemic: an exploratory study on patient and therapist experience with on-site sessions using face masks vs. telepsychiatric sessions. BMC Psychiatry. 2021 May 05;21(1):237. doi: 10.1186/s12888-021-03236-9. https://bmcpsychiatry.biomedcentral.com/articles/10.1186/s12888-021-03236-9 .10.1186/s12888-021-03236-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 144.Milton DE. On the ontological status of autism: the ‘double empathy problem’. Disabil Soc. 2012 Oct;27(6):883–7. doi: 10.1080/09687599.2012.710008. [DOI] [Google Scholar]
- 145.Grover S, Mehra A, Sahoo S, Avasthi A, Tripathi A, D'Souza A, Saha G, Jagadhisha A, Gowda M, Vaishnav M, Singh O, Dalal P, Kumar P. Impact of COVID-19 pandemic and lockdown on the state of mental health services in the private sector in India. Indian J Psychiatry. 2020;62(5):488–93. doi: 10.4103/psychiatry.IndianJPsychiatry_568_20. http://europepmc.org/abstract/MED/33678828 .IJPsy-62-488 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 146.Lindsay JA, Kauth MR, Hudson S, Martin LA, Ramsey DJ, Daily L, Rader J. Implementation of video telehealth to improve access to evidence-based psychotherapy for posttraumatic stress disorder. Telemed J E Health. 2015 Jun;21(6):467–72. doi: 10.1089/tmj.2014.0114. http://europepmc.org/abstract/MED/25714664 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 147.McManus F, Sacadura C, Clark DM. Why social anxiety persists: an experimental investigation of the role of safety behaviours as a maintaining factor. J Behav Ther Exp Psychiatry. 2008 Jun;39(2):147–61. doi: 10.1016/j.jbtep.2006.12.002.S0005-7916(07)00005-5 [DOI] [PubMed] [Google Scholar]
- 148.Rapee RM, Heimberg RG. A cognitive-behavioral model of anxiety in social phobia. Behav Res Ther. 1997 Aug;35(8):741–56. doi: 10.1016/s0005-7967(97)00022-3.S0005-7967(97)00022-3 [DOI] [PubMed] [Google Scholar]
- 149.Chong J, Moreno F. Feasibility and acceptability of clinic-based telepsychiatry for low-income Hispanic primary care patients. Telemed J E Health. 2012 May;18(4):297–304. doi: 10.1089/tmj.2011.0126. [DOI] [PubMed] [Google Scholar]
- 150.Working therapeutically with parents and their infants during pregnancy and postpartum using remote delivery platforms. The British Psychological Society. 2020. [2022-03-18]. https://www.bps.org.uk/guideline/working-therapeutically-parents-and-their-infants-during-pregnancy-and-postpartum-using .
- 151.Jones DJ, Forehand R, Cuellar J, Parent J, Honeycutt A, Khavjou O, Gonzalez M, Anton M, Newey GA. Technology-enhanced program for child disruptive behavior disorders: development and pilot randomized control trial. J Clin Child Adolesc Psychol. 2014;43(1):88–101. doi: 10.1080/15374416.2013.822308. http://europepmc.org/abstract/MED/23924046 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 152.Riches S, Azevedo L, Steer S, Nicholson S, Vasile R, Lyles S, Csehi R, Fialho C, Waheed S, Lokhande M. Brief videoconference-based dialectical behaviour therapy skills training for COVID-19-related stress in acute and crisis psychiatric staff. Clinical Psychology Forum. 2021. [2022-03-18]. https://shop.bps.org.uk/clinical-psychology-forum-no-337-january-2021 .
- 153.Mahmoud H, Naal H, Cerda S. Planning and implementing telepsychiatry in a community mental health setting: a case study report. Community Ment Health J. 2021 Jan;57(1):35–41. doi: 10.1007/s10597-020-00709-1. http://europepmc.org/abstract/MED/32897476 .10.1007/s10597-020-00709-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 154.Watts G. COVID-19 and the digital divide in the UK. Lancet Digit Health. 2020 Aug;2(8):e395–6. doi: 10.1016/S2589-7500(20)30169-2. https://linkinghub.elsevier.com/retrieve/pii/S2589-7500(20)30169-2 .S2589-7500(20)30169-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 155.Kim KJ, Sundar SS. Mobile persuasion: can screen size and presentation mode make a difference to trust? Hum Commun Res. 2016 Jan;42(1):45–70. doi: 10.1111/hcre.12064. [DOI] [Google Scholar]
- 156.Kortum P, Johnson M. The relationship between levels of user experience with a product and perceived system usability. Proc Hum Factors Ergon Soc Annu Meet. 2013 Sep 30;57(1):197–201. doi: 10.1177/1541931213571044. [DOI] [Google Scholar]
- 157.Shore JH, Yellowlees P, Caudill R, Johnston B, Turvey C, Mishkind M, Krupinski E, Myers K, Shore P, Kaftarian E, Hilty D. Best practices in videoconferencing-based telemental health April 2018. Telemed J E Health. 2018 Nov;24(11):827–32. doi: 10.1089/tmj.2018.0237. [DOI] [PubMed] [Google Scholar]
- 158.Nelson EL, Bui T, Sharp S. Telemental health competencies: training examples from a youth depression telemedicine clinic. In: Gregerson MB, editor. Technology Innovations for Behavioral Education. New York, NY, USA: Springer; 2011. pp. 41–7. [Google Scholar]
- 159.MacIntyre G, Cogan NA, Stewart AE, Quinn N, Rowe M, O’Connell M. What’s citizenship got to do with mental health? Rationale for inclusion of citizenship as part of a mental health strategy. J Public Mental Health. 2019 Sep 05;18(3):157–61. doi: 10.1108/jpmh-04-2019-0040. [DOI] [Google Scholar]
- 160.Eiroa-Orosa FJ, Rowe M. Taking the concept of citizenship in mental health across countries. Reflections on transferring principles and practice to different sociocultural contexts. Front Psychol. 2017 Jun 21;8:1020. doi: 10.3389/fpsyg.2017.01020. doi: 10.3389/fpsyg.2017.01020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 161.Cogan NA, MacIntyre G, Stewart A, Tofts A, Quinn N, Johnston G, Hamill L, Robinson J, Igoe M, Easton D, McFadden AM, Rowe M. "The biggest barrier is to inclusion itself": the experience of citizenship for adults with mental health problems. J Ment Health. 2021 Jun;30(3):358–65. doi: 10.1080/09638237.2020.1803491. [DOI] [PubMed] [Google Scholar]
- 162.Greener S, Wakefield C. Developing confidence in the use of digital tools in teaching. Electron J E Learn. 2015;13(4):260–7. [Google Scholar]
- 163.Harst L, Lantzsch H, Scheibe M. Theories predicting end-user acceptance of telemedicine use: systematic review. J Med Internet Res. 2019 May 21;21(5):e13117. doi: 10.2196/13117. https://www.jmir.org/2019/5/e13117/ v21i5e13117 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 164.Nielsen J, Landauer TK. A mathematical model of the finding of usability problems. Proceedings of the INTERACT '93 and CHI '93 Conference on Human Factors in Computing Systems; CHI '93; April 24-29, 1993; Amsterdam, The Netherlands. 1993. pp. 206–13. [DOI] [Google Scholar]
- 165.Alguera-Lara V, Dowsey MM, Ride J, Kinder S, Castle D. Shared decision making in mental health: the importance for current clinical practice. Australas Psychiatry. 2017 Dec;25(6):578–82. doi: 10.1177/1039856217734711. [DOI] [PubMed] [Google Scholar]
- 166.Carr S. Personalisation: a rough guide. Social Care Institute for Excellence. 2008. Oct, [2022-01-31]. https://www.scie.org.uk/publications/reports/report20.asp .
- 167.Rodenburg-Vandenbussche S, Carlier I, van Vliet I, van Hemert A, Stiggelbout A, Zitman F. Patients' and clinicians' perspectives on shared decision-making regarding treatment decisions for depression, anxiety disorders, and obsessive-compulsive disorder in specialized psychiatric care. J Eval Clin Pract. 2020 Apr;26(2):645–58. doi: 10.1111/jep.13285. [DOI] [PubMed] [Google Scholar]
- 168.Wallcraft J, Amering M, Freidin J, Davar B, Froggatt D, Jafri H, Javed A, Katontoka S, Raja S, Rataemane S, Steffen S, Tyano S, Underhill C, Wahlberg H, Warner R, Herrman H. Partnerships for better mental health worldwide: WPA recommendations on best practices in working with service users and family carers. World Psychiatry. 2011 Oct;10(3):229–36. doi: 10.1002/j.2051-5545.2011.tb00062.x. https://onlinelibrary.wiley.com/resolve/openurl?genre=article&sid=nlm:pubmed&issn=1723-8617&date=2011&volume=10&issue=3&spage=229 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 169.Archer J, Bower P, Gilbody S, Lovell K, Richards D, Gask L, Dickens C, Coventry P. Collaborative care for depression and anxiety problems. Cochrane Database Syst Rev. 2012 Oct 17;10:CD006525. doi: 10.1002/14651858.CD006525.pub2. [DOI] [PubMed] [Google Scholar]
- 170.Vinokur-Kaplan D. Treatment teams that work (and those that don't): an application of Hackman's group effectiveness model to interdisciplinary teams in psychiatric hospitals. J Appl Behav Sci. 1995 Sep 1;31(3):303–27. doi: 10.1177/0021886395313005. [DOI] [Google Scholar]
- 171.Kutash K, Acri M, Pollock M, Armusewicz K, Serene Olin SC, Hoagwood KE. Quality indicators for multidisciplinary team functioning in community-based children's mental health services. Adm Policy Ment Health. 2014 Jan;41(1):55–68. doi: 10.1007/s10488-013-0508-2. http://europepmc.org/abstract/MED/23873037 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 172.Miles P. National Wraparound Initiative. Portland, OR, USA: National Wraparound Initiative, Research and Training Center for Family Support and Children's Mental Health; 2008. [2022-03-18]. Family partners and the wraparound process. https://nwi.pdx.edu/NWI-book/Chapters/Miles-4b.3-(fam-part-models).pdf . [Google Scholar]
- 173.Birkhäuer J, Gaab J, Kossowsky J, Hasler S, Krummenacher P, Werner C, Gerger H. Trust in the health care professional and health outcome: a meta-analysis. PLoS One. 2017 Feb 7;12(2):e0170988. doi: 10.1371/journal.pone.0170988. https://dx.plos.org/10.1371/journal.pone.0170988 .PONE-D-16-37935 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 174.Flückiger C, Del Re AC, Wampold BE, Symonds D, Horvath AO. How central is the alliance in psychotherapy? A multilevel longitudinal meta-analysis. J Couns Psychol. 2012 Jan;59(1):10–7. doi: 10.1037/a0025749.2011-23015-001 [DOI] [PubMed] [Google Scholar]
- 175.Flückiger C, Del Re AC, Wampold BE, Horvath AO. The alliance in adult psychotherapy: a meta-analytic synthesis. Psychotherapy (Chic) 2018 Dec;55(4):316–40. doi: 10.1037/pst0000172.2018-23951-001 [DOI] [PubMed] [Google Scholar]
- 176.Kaiser J, Hanschmidt F, Kersting A. The association between therapeutic alliance and outcome in Internet-based psychological interventions: a meta-analysis. Comput Human Behav. 2021 Jan;114:106512. doi: 10.1016/j.chb.2020.106512. [DOI] [Google Scholar]
- 177.Coffey M, Cohen R, Faulkner A, Hannigan B, Simpson A, Barlow S. Ordinary risks and accepted fictions: how contrasting and competing priorities work in risk assessment and mental health care planning. Health Expect. 2017 Jun;20(3):471–83. doi: 10.1111/hex.12474. doi: 10.1111/hex.12474. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 178.Simpson A, Hannigan B, Coffey M, Barlow S, Cohen R, Jones A, Všetečková J, Faulkner A, Thornton A, Cartwright M. Recovery-focused care planning and coordination in England and Wales: a cross-national mixed methods comparative case study. BMC Psychiatry. 2016 May 16;16:147. doi: 10.1186/s12888-016-0858-x. https://bmcpsychiatry.biomedcentral.com/articles/10.1186/s12888-016-0858-x .10.1186/s12888-016-0858-x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 179.Paulick J, Deisenhofer AK, Ramseyer F, Tschacher W, Boyle K, Rubel J, Lutz W. Nonverbal synchrony: a new approach to better understand psychotherapeutic processes and drop-out. J Psychother Integr. 2018 Sep;28(3):367–84. doi: 10.1037/int0000099. [DOI] [Google Scholar]
- 180.Seuren LM, Wherton J, Greenhalgh T, Shaw SE. Whose turn is it anyway? Latency and the organization of turn-taking in video-mediated interaction. J Pragmat. 2021 Jan;172:63–78. doi: 10.1016/j.pragma.2020.11.005. http://europepmc.org/abstract/MED/33519050 .S0378-2166(20)30278-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 181.Coffey M, Hannigan B, Barlow S, Cartwright M, Cohen R, Faulkner A, Jones A, Simpson A. Recovery-focused mental health care planning and co-ordination in acute inpatient mental health settings: a cross national comparative mixed methods study. BMC Psychiatry. 2019 Apr 16;19(1):115. doi: 10.1186/s12888-019-2094-7. https://bmcpsychiatry.biomedcentral.com/articles/10.1186/s12888-019-2094-7 .10.1186/s12888-019-2094-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 182.Ahmed N, Barlow S, Reynolds L, Drey N, Begum F, Tuudah E, Simpson A. Mental health professionals' perceived barriers and enablers to shared decision-making in risk assessment and risk management: a qualitative systematic review. BMC Psychiatry. 2021 Nov 25;21(1):594. doi: 10.1186/s12888-021-03304-0. https://bmcpsychiatry.biomedcentral.com/articles/10.1186/s12888-021-03304-0 .10.1186/s12888-021-03304-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 183.Zelle H, Kemp K, Bonnie RJ. Advance directives in mental health care: evidence, challenges and promise. World Psychiatry. 2015 Oct;14(3):278–80. doi: 10.1002/wps.20268. doi: 10.1002/wps.20268. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 184.Measuring progress: Commitments to support and expand the mental health workforce in England. The British Medical Association. 2019. [2022-03-18]. https://www.bma.org.uk/media/2405/bma-measuring-progress-of-commitments-for-mental-health-workforce-jan-2020.pdf .
- 185.Gilburt H. Securing money to improve mental health care... but no staff to spend it on. The King's Fund. 2019. [2022-01-31]. https://www.kingsfund.org.uk/blog/2019/10/mental-health-staff-shortage .
- 186.The impact of the UK recession and welfare reform on mental health. The Faculty of Public Health. [2022-01-31]. https://tinyurl.com/r7w5fd2s .
- 187.Corbyn J. Full text of Jeremy Corbyn’s Speech on Labour’s British Broadband announcement. Labour. 2019. Nov 15, [2022-01-31]. https://labour.org.uk/press/full-text-of-jeremy-corbyns-speech-on-labours-british-broadband-announcement/
- 188.Das S. Mental health helpline funded by royals shared users? conversations. The Guardian. 2022. Feb 19, [2022-03-18]. https://www.theguardian.com/society/2022/feb/19/mental-health-helpline-funded-by-royals-shared-users-conversations .
- 189.Surveillance in mental health settings – NSUN response to JCHR inquiry. National Survivor User Network. 2021. [2022-03-18]. https://www.nsun.org.uk/news/surveillance-in-mental-health-settings-jchr/
- 190.Wilson K, Eaton J, Foye U, Ellis M, Thomas E, Simpson A. What evidence supports the use of Body Worn Cameras in mental health inpatient wards? A systematic review and narrative synthesis of the effects of Body Worn Cameras in public sector services. Int J Ment Health Nurs. 2022 Apr;31(2):260–77. doi: 10.1111/inm.12954. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 191.Concerns Regarding Privacy And Data Protection Within The High Intensity Network (HIN) And Serenity Integrated Mentoring (SIM) STOPSIM. 2021. Apr 26, [2022-03-18]. https://tinyurl.com/mrymjatb .
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Glossary.
Study characteristics.
Overarching context-mechanism-outcome configurations.