Healthcare stakeholders’ perceptions and experiences of factors affecting the implementation of critical care telemedicine (CCT): qualitative evidence synthesis

Abstract

Background

Critical care telemedicine (CCT) has long been advocated for enabling access to scarce critical care expertise in geographically‐distant areas. Additional advantages of CCT include the potential for reduced variability in treatment and care through clinical decision support enabled by the analysis of large data sets and the use of predictive tools. Evidence points to health systems investing in telemedicine appearing better prepared to respond to sudden increases in demand, such as during pandemics. However, challenges with how new technologies such as CCT are implemented still remain, and must be carefully considered.

Objectives

This synthesis links to and complements another Cochrane Review assessing the effects of interactive telemedicine in healthcare, by examining the implementation of telemedicine specifically in critical care. Our aim was to identify, appraise and synthesise qualitative research evidence on healthcare stakeholders’ perceptions and experiences of factors affecting the implementation of CCT, and to identify factors that are more likely to ensure successful implementation of CCT for subsequent consideration and assessment in telemedicine effectiveness reviews.

Search methods

We searched MEDLINE, Embase, CINAHL, and Web of Science for eligible studies from inception to 14 October 2019; alongside 'grey' and other literature searches. There were no language, date or geographic restrictions.

Selection criteria

We included studies that used qualitative methods for data collection and analysis. Studies included views from healthcare stakeholders including bedside and CCT hub critical care personnel, as well as administrative, technical, information technology, and managerial staff, and family members.

Data collection and analysis

We extracted data using a predetermined extraction sheet. We used the Critical Appraisal Skills Programme (CASP) qualitative checklist to assess the methodological rigour of individual studies. We followed the Best‐fit framework approach using the Consolidated Framework for Implementation Research (CFIR) to inform our data synthesis.  We classified additional themes not captured by CFIR under a separate theme. We used the GRADE CERQual approach to assess confidence in the findings.

Main results

We found 13 relevant studies. Twelve were from the USA and one was from Canada. Where we judged the North American focus of the studies to be a concern for a finding’s relevance, we have reflected this in our assessment of confidence in the finding. The studies explored the views and experiences of bedside and hub critical care personnel; administrative, technical, information technology, and managerial staff; and family members. The intensive care units (ICUs) were from tertiary hospitals in urban and rural areas.

We identified several factors that could influence the implementation of CCT. We had high confidence in the following findings:

Hospital staff and family members described several advantages of CCT. Bedside and hub staff strongly believed that the main advantage of CCT was having access to experts when bedside physicians were not available. Families also valued having access to critical care experts. In addition, hospital staff described how CCT could support clinical decision‐making and mentoring of junior staff. 

Hospital staff greatly valued the nature and quality of social networks between the bedside and CCT hub teams. Key issues for them were trust, acceptance, teamness, familiarity and effective communication between the two teams.

Interactions between some bedside and CCT hub staff were featured with tension, frustration and conflict. Staff on both sides commonly described disrespect of their expertise, resistance and animosity.

Hospital staff thought it was important to promote and offer training in the use of CCT before its implementation. This included rehearsing every step in the process, offering staff opportunities to ask questions and disseminating learning resources. Some also complained that experienced staff were taken away from bedside care and re‐allocated to the CCT hub team.

Hospital staff's attitudes towards, knowledge about and value placed on CCT influenced acceptance of CCT. Staff were positive towards CCT because of its several advantages. But some were concerned that the CCT hub staff were not able to understand the patient’s situation through the camera. Some were also concerned about confidentiality of patient data.

We also identified other factors that could influence the implementation of CCT, although our confidence in these findings is moderate or low. These factors included the extent to which telemedicine software was adaptable to local needs, and hub staff were aware of local norms; concerns about additional administrative work and cost; patients' and families’ desire to stay close to their local community; the type of hospital setting; the extent to which there was support from senior leadership; staff access to information about policies and procedures; individuals' stage of change; staff motivation, competence and values; clear strategies for staff engagement; feedback about progress; and the impact of CCT on staffing levels.

Authors' conclusions

Our review identified several factors that could influence the acceptance and use of telemedicine in critical care. These include the value that hospital staff and family members place on having access to critical care experts, staff access to sufficient training, and the extent to which healthcare providers at the bedside and the critical care experts supporting them from a distance acknowledge and respect each other’s expertise. Further research, especially in contexts other than North America, with different cultures, norms and practices will strengthen the evidence base for the implementation of CCT internationally and our confidence in these findings. Implementation of CCT appears to be growing in importance in the context of global pandemic management, especially in countries with wide geographical dispersion and limited access to critical care expertise. For successful implementation, policymakers and other stakeholders should consider pre‐empting and addressing factors that may affect implementation, including strengthening teamness between bedside and hub teams; engaging and supporting frontline staff; training ICU clinicians on the use of CCT prior to its implementation; and ensuring staff have access to information and knowledge about when, why and how to use CCT for maximum benefit.

Plain language summary

What are healthcare stakeholders’ perceptions and experiences of factors affecting the implementation of critical care telemedicine?

What is the aim of this review?

The aim of this review was to identify factors that affect the acceptance and use of health care from a distance (known as telemedicine) for patients in intensive care units (also known as critical care). To answer this question, we searched for and analysed qualitative studies about the perceptions and experiences of clinical staff, managers and administrators, as well as patients and family members. This review links to another Cochrane Review assessing the effects of telemedicine. 

Key messages

Our review identified several factors that could influence the acceptance and use of telemedicine in critical care. These included the value that hospital staff and family members place on having access to critical care experts, staff access to sufficient training, and the extent to which healthcare providers at the bedside and the critical care experts supporting them from a distance acknowledge and respect each other’s expertise.

What was studied in this synthesis?

In critical care telemedicine (CCT), patients in intensive care units (ICUs) are monitored by critical care experts based at a ‘hub’ outside the hospital. By monitoring patients, hub staff are able to warn staff at the bedside of potential problems and offer them decision support. The use of CCT means that patients and staff in rural or small hospitals have access to critical care experts. But there may still be challenges when implementing CCT. In this review, we assessed studies that looked at the perceptions and experiences of healthcare workers, family members and others to find factors that could influence the acceptance and use of CCT.

What are the main findings of the synthesis?

We included 13 relevant studies. Twelve were from the USA and one was from Canada. Where we judged the North American focus of the studies to be a concern for a finding’s relevance, we have reflected this in our assessment of confidence in the finding. The studies explored the views and experiences of bedside and hub critical care personnel; administrative, technical, information technology, and managerial staff, and family members. The ICUs were from hospitals in both urban and rural areas.

We identified several factors that could influence the acceptance and use of CCT. We had high confidence in the following findings:

Hospital staff and family members described several advantages of CCT. Bedside and hub staff strongly believed that the main advantage of CCT was having access to experts when bedside doctors were not available. Families also valued having access to critical care experts. Hospital staff also described how CCT could support clinical decision‐making and mentoring of junior staff. 

Hospital staff greatly valued the nature and quality of social networks between the bedside and CCT hub teams. Key issues for them were trust, acceptance, being part of a team, familiarity and effective communication between the two teams.

Interactions between some bedside and CCT hub staff were featured with tension, frustration and conflict. Staff on both sides commonly described disrespect of expertise, resistance and animosity.

Hospital staff thought it was important to promote and offer training in the use of CCT before its implementation. This included rehearsing every step in the process, offering staff opportunities to ask questions and disseminating learning resources. Some also complained that experienced staff were taken away from bedside care and re‐allocated to the CCT hub team.

Hospital staff's attitudes towards, knowledge about and value placed on CCT influenced acceptance of CCT. Staff were positive towards CCT because of its several advantages. But some were concerned that the hub staff were not able to understand the patient’s situation through the camera. Some were also concerned about confidentiality of patient data.

We also identified other factors that could influence the acceptance and use of CCT, although our confidence in these findings is moderate or low. These factors include the extent to which telemedicine software was adaptable to local needs, and hub staff were aware of local norms; concerns about additional administrative work and cost; patients' and families’ desire to stay close to their local community; the type of hospital setting; the extent to which there was support from senior leadership; staff access to information about policies and procedures; individuals' readiness to change; staff motivation, competence and values; clear strategies for staff engagement; feedback about progress; and the impact of CCT on staffing levels.

How up‐to‐date is this review?

We searched for studies that had been published up to October 2019.

Summary of findings

Summary of findings 1. Summary of qualitative findings.

Summary of review findings	Studies contributing to the review finding	GRADE‐CERQual assessment of confidence in the evidence	Explanation of GRADE‐CERQual assessment
CFIR Domain I: Factors affecting implementation related to intervention characteristics
Finding 1: Hospital staff’s personal experience, and anecdotes from colleagues, supported their belief that CCT has positive effects on patient care. Specifically, these effects were for patient safety and quality of care, support at the bedside by critical care experts, and standardisation of practice	Khunlertkit 2013; Moeckli 2013; Shahpori 2011a; Stafford 2008a; Ward 2015; Wilkes 2016	Moderate confidence	Downgraded to moderate confidence because of minor concerns about methodological limitations, coherence, and adequacy; and moderate concerns about relevance
Finding 2: Hospital staff and family members described several advantages of CCT. Bedside and hub staff strongly believed that the main advantage of CCT was having access to experts when bedside doctors were not available. Families also valued having access to critical care experts. In addition, hospital staff described how CCT could support clinical decision‐making and mentoring of junior staff	Jahrsdoerfer 2013; Kahn 2019; Khunlertkit 2013; Moeckli 2013; Shahpori 2011a; Stafford 2008a; Thomas 2017	High confidence	Graded as high confidence because of no or very minor concerns about methodological limitations, relevance, coherence, and adequacy
Finding 3: Bedside staff valued the potential adaptability of CCT to speak to local needs and practices. However, this was not always evident, with reported examples being mainly around developing camera usage etiquette and integration with local protocols	Moeckli 2013; Stafford 2008a; Thomas 2017	Low confidence	Downgraded to low confidence because of minor concerns about coherence; moderate concerns about relevance; and serious concerns about adequacy
Finding 4: Both bedside and hub clinicians expressed difficulties with the implementation of CCT. Key barriers related to implementation were perceptions of additional workload, need for more co‐ordination work, and concern around the presence of cameras	Moeckli 2013; Mullen‐Fortino 2012; Shahpori 2011a; Stafford 2008a; Ward 2015	Moderate confidence	Downgraded to moderate confidence because of minor concerns about coherence; moderate concerns about methodological limitations; and moderate concerns about adequacy
Finding 5: Cost considerations featured as an influencing factor in a limited way, with only a few examples noting the high cost of implementing CCT, especially compared to the cost of recruiting additional ICU staff	Shahpori 2011a; Stafford 2008a	Low confidence	Downgraded to low confIdence because of moderate concerns about methodological limitations; and serious concerns about relevance, and adequacy
CFIR Domain II: Factors affecting implementation related to outer setting
Finding 6: Hospital staff as well as family members perceived CCT to be providing a community benefit, specifically for patients' and families' desire to stay close to their local community without requiring transfer to specialist centres to access critical care expertise	Goedken 2017; Moeckli 2013; Shahpori 2011a; Ward 2015; Wilkes 2016	Moderate confidence	Downgraded to moderate confidence because of minor concerns about adequacy; moderate concerns about methodological limitations; and serious concerns about relevance
CFIR Domain III: Factors affecting implementation related to inner setting
Finding 7: Hospital staff greatly valued the nature and quality of social networks between the bedside and CCT hub teams. Key issues for them were trust, acceptance, teamness, familiarity and effective communication between the two teams	Hoonakker 2018; Jahrsdoerfer 2013; Kahn 2019; Khunlertkit 2013; Moeckli 2013; Mullen‐Fortino 2012; Stafford 2008a; Wilkes 2016	High confidence	Graded as high confidence because of no or very minor concerns about relevance, coherence, and adequacy; and minor concerns about methodological limitations
Finding 8: Hospital bedside staff were concerned over the hub team not being aware of local unit norms, values, and culture. This led local bedside teams to feel that CCT intruded on their practice	Kahn 2019; Moeckli 2013; Mullen‐Fortino 2012; Stafford 2008a; Ward 2015; Wilkes 2016	Moderate confidence	Downgraded to moderate confidence because of moderate concerns about methodological limitations, relevance, and adequacy
Finding 9: Bedside clinicians were reluctant to use CCT because they lacked clarity about its purpose, were concerned that their decision‐making skills would be weakened through remote supervision, and did not consider hub clinicians an equal counterpart in patient management. Hub clinicians were disengaged due to lack of role clarity and limited integration with patient care	Kahn 2019; Moeckli 2013; Shahpori 2011a; Stafford 2008a	Moderate confidence	Downgraded to moderate confidence because of minor concerns about methodological limitations, and adequacy; and moderate concerns about relevance.
Finding 10: Hospital locale shaped prioritisation of CCT, with staff in rural centres noting that CCT was of greater benefit to them considering their staff shortage and lack of critical care resources	Kahn 2019; Shahpori 2011a; Ward 2015; Wilkes 2016	Low confidence	Downgraded to low confidence because of moderate concerns about methodological limitations, relevance, and coherence; and serious concerns about adequacy
Finding 11: Bedside and hub clinicians perceived the absence of support from, and lack of engagement in dialogue with leaders and senior administrators during the implementation of CCT as major barriers. Listening to staff needs, and creating groundwork connections with them from the outset were perceived as facilitating factors to implementation	Kahn 2019; Wilkes 2016	Low confidence	Downgraded to low confidence because of minor concerns about methodological limitations; moderate concerns about relevance; and serious concerns about adequacy
Finding 12: Hospital staff expressed it was important to promote and offer training in the use of CCT before its implementation. This included rehearsing every step in the process, offering staff opportunities to ask questions and disseminating learning resources. Some also complained that experienced staff were taken away from bedside care and re‐allocated to the CCT hub team	Kahn 2019; Moeckli 2013; Shahpori 2011a; Stafford 2008a; Ward 2015	High confidence	Graded as high confidence because we had minor concerns about relevance, coherence, and adequacy; and moderate concerns about methodological limitations
Finding 13: Hospital staff reported the lack of access to information about how CCT staff, policies and procedures can be incorporated into the bedside workflow as a barrier to implementation	Moeckli 2013	Low confidence	Downgraded to low confidence because of minor concerns about methodological limitations; and serious concerns about relevance, and adequacy
CFIR Domain IV: Factors affecting implementation related to characteristics of individuals
Finding 14: Hospital staff's attitudes towards, knowledge about and value placed on CCT influenced acceptance of CCT. Staff were positive towards CCT because of its several advantages. But, some were concerned that the CCT hub staff were not able to understand the patient’s situation through the camera. Some were also concerned about confidentiality of patient data	Kahn 2019; Khunlertkit 2013; Moeckli 2013; Mullen‐Fortino 2012; Shahpori 2011a; Stafford 2008a; Thomas 2017	High confidence	Graded as high confidence because of minor concerns about methodological limitations, relevance, coherence, and adequacy
Finding 15: Hospital staff noted that acceptance and normalisation of CCT in their daily work took time; progressing through different stages of change did not occur at the same pace for everyone, with some remaining resistant to change	Kahn 2019; Khunlertkit 2013	Low confidence	Downgraded to low confidence because of minor concerns about coherence; moderate concerns about relevance; and serious concerns about adequacy
Finding 16: Hub nurses’ personal attributes, specifically about their motivation, multitasking competence and values, were noted as important enablers for implementation of CCT	Hoonakker 2013	Low confidence	Downgraded to low confidence because of minor concerns about methodological limitations; moderate concerns about adequacy; and serious concerns about relevance
CFIR Domain V: Factors affecting implementation related to process
Finding 17: Hospital staff were frustrated due to lacking a clear strategy for engagement; specifically lack of consistent training, the orientation of new and resistant staff to the hub facility, and timely co‐ordination for CCT implementation	Kahn 2019; Moeckli 2013	Low confidence	Downgraded to low confidence because of minor concerns about methodological limitations, and coherence; moderate concerns about relevance; and serious concerns about adequacy
Finding 18: Hospital staff were encouraged by the visibility of the intended benefits of CCT. They valued both quantitative feedback through auditing, as well as qualitative feedback through reflective accounts	Kahn 2019; Khunlertkit 2013; Thomas 2017	Moderate confidence	Downgraded to moderate confidence of minor concerns about coherence, and adequacy; and moderate concerns about relevance
Other factors affecting implementation
Finding 19: Hospital staff highlighted that CCT can support ICUs to overcome challenges associated with staff shortages especially during nights and weekends, and in rural hospitals where ICU nurses are assigned to different departments; and with retaining physicians and nurses. Some concerns over the potential negative impact of CCT on overall staffing levels were also expressed	Goedken 2017; Hoonakker 2013; Kahn 2019; Shahpori 2011a	Moderate confidence	Downgraded to moderate confidence because of minor concerns about relevance; moderate concerns about methodological limitations, and adequacy
Finding 20: Interactions between some bedside and CCT hub staff were featured with tension, frustration and conflict. Staff on both sides commonly described disrespect of expertise, resistance and animosity	Hoonakker 2013; Kahn 2019; Khunlertkit 2013; Moeckli 2013; Mullen‐Fortino 2012; Stafford 2008a; Wilkes 2016	High confidence	Graded as high confidence because of no or very minor concerns about coherence and adequacy; minor concerns about relevance; and moderate concerns about methodological limitations

Background

International interest in the benefits and implementation of telemedicine in a variety of settings and for different conditions is growing fast, as evidenced by the recently published Cochrane intervention review (Flodgren 2015) and Cochrane qualitative evidence synthesis protocol (Odendaal 2020). This is especially the case in the care of critically‐ill people. The burden of critical illness is higher than is generally appreciated, and is expected to increase as a result of global population ageing (Adhikari 2010; Vincent 2014). Consequently, critical care services in major hospitals are stretched, while smaller hospitals and rural areas have limited access to relevant expertise (Wunsch 2008). In addition, critical care is challenged by inconsistent application of evidence‐based guidelines, variation in staffing levels and clinical outcomes, higher rates of medication errors and adverse drug events (Pronovost 2004; Rothchild 2005), all of which are aggravated by the unpredictable nature of patient conditions, the urgent nature of many admissions to critical care and the need for out‐of‐hours decision‐making. For the purposes of this review, we define critical care as the concentration of healthcare staff and equipment in a distinct area of the hospital in order to care for people whose conditions are life‐threatening and who need constant and close monitoring and support.

Description of the topic

Telemedicine has been broadly defined by the World Health Organization (WHO) as: “the delivery of health care services, where distance is a critical factor, by all health care professionals using information and communication technologies for the exchange of valid information for the diagnosis, treatment and prevention of disease and injuries, research and evaluation, and for the continuing education of health care providers, all in the interests of advancing the health of individuals and their communities” (WHO 2010). Critical care telemedicine (CCT) in particular enables a team of critical care doctors and nurses to provide 24‐hour remote support to clinicians using audio‐visual communication and computer systems. In 2014, it was estimated that 8% of total intensive care unit (ICU) beds in the USA were covered by CCT, with an average growth rate of 8% a year (Khan 2014). CCT offers minute‐by‐minute monitoring and recording of vital organ function, making use of electronic records and remote surveillance in order to facilitate early detection and response to physiological deterioration. In addition, the integration of decision‐support tools and early‐warning systems supports adherence to clinical guidelines, which can level out variations in quality of care. Further advantages of CCT for stakeholders may include additional support for junior staff, with patients and families feeling looked after. Consequently, CCT has potential to improve clinical outcomes beyond the confines of the ICU for people who may benefit from critical care expertise but are not based in specialist units; for example, they may be in an emergency department, generic ICU or medical/surgical ward. This is possible by extending the availability and reach of critical care expertise through a hub‐and‐spoke model, adding a safety net to ward‐based and non‐specialist bedside providers.

The hub‐and‐spoke model of CCT is used in the context of multi‐location delivery of critical care services. A remotely‐based team of senior and experienced critical care clinicians ‐ called the hub ‐ is networked through audio‐visual communication and telemonitoring systems with a number of bedside terminals, clinicians and patients. The hub acts as a single point of contact for critical care advice and support, while through seamless extensions ‐ called spokes ‐ hands‐on patient care is provided across multiple locations. In a wider role, the hub can also take on co‐ordinating responsibilities, including patient flow through ICUs, brokering admission and discharge of patients, as well as quality, risk and performance management through early‐warning capabilities, rounding tools to monitor at‐risk patients, inbuilt clinical decision support and prompts for adherence to best practice. In summary, CCT includes the following functionality: synchronous, interactive client‐to‐provider telemedicine; telemonitoring; client health records; provider‐to‐provider telemedicine; provider‐based decision support; laboratory and diagnostics management; data collection, management and use.

CCT is designed as a continuous form of clinical support to bedside practice, enabling clinical oversight and interactions between providers. In this way, it is distinct from other telemedicine models that mainly offer an interface for sporadic consultation between providers and patients in remote locations, or between generalist and specialist clinicians. Critical care patients’ condition can be unstable, can deteriorate unexpectedly and quite rapidly, requiring close monitoring and prompt reaction by a multidisciplinary team of expert clinicians, there and then. As a consequence, critical care services tend to have increased organisational autonomy, resources and staffing levels compared to other areas of the hospital. These unique features of critical care practice can influence professionals’ perceptions, experience and use of CCT, all of which can affect successful implementation.

How the intervention might work

The implementation of new technologies in healthcare settings is beset with multiple challenges. Reports on the failure of widely‐accepted and seemingly diffused health technologies to become embedded in daily practise are commonplace in the literature, even where these have support by both clinicians and politicians (May 2000). To understand where the implementation of such technologies fail, a strong theoretical foundation is needed to guide the evaluation of such programmes. Use of implementation theory can help generate explanatory models and hypotheses about factors influencing implementation of health technologies, leading to the identification of approaches more likely to result in successful implementation.

For the purpose of this review, we will use the Consolidated Framework for Implementation Research (CFIR) (Damschroder 2009) to theoretically conceptualise data from the included studies and to guide the data analysis. CFIR is a 'meta‐theoretical' model, made up of constructs generated out of a synthesis of existing theories; one of its strengths and unique features is that it does not depict rigid interrelationships, specific ecological levels, or specific hypotheses. This allows for theory development guided by exploratory questions such as what works, where and why across different contexts. The CFIR has been used successfully in reviews of eHealth and is found to offer great theoretical and explanatory capabilities (Ross 2016).

CFIR is composed of five key constructs, each made up of different factors that affect the implementation of innovations into practice (see Appendix 1). In summary, the five key constructs of the CFIR are:

• Intervention characteristics;

• Inner settings;

• Outer settings;

• Characteristics of individuals; and

• Process of implementation.

The first construct, Intervention characteristics , refers to features of the intervention including its source, evidence base, advantage over other interventions, the extent of its adaptability, 'trialability' and complexity, as well as its quality and cost. The second and third constructs, Inner and Outer Settings , relate to the internal and external environment in which implementation occurs. For example, the inner setting is about features of the structural, political and cultural organisation contexts through which the implementation process takes place; while the outer setting relates to the economic, political and social context within which the organisation resides. The fourth construct refers to the Characteristics of the Individuals who engage with the intervention or the implementation process. Individuals’ knowledge and beliefs about the intervention, their self‐efficacy, personal attributes and identification with the organisation play a key part in the success or failure of the implementation process. The final construct relates to the Implementation process itself, which includes elements of planning, engaging with leaders, champions and change agents, carrying out the implementation plan and evaluating the process and experience.

Operationalising the CFIR as an organising framework in the context of this qualitative evidence synthesis allows for a theoretically informed approach to data extraction, analysis and synthesis; helps with the interpretation of results; and strengthens the theoretical transferability and comparability of conclusions. At the same time, it allows for testing of the CFIR and consequent elaboration in the context of telemedicine in general, and CCT in particular.

Why it is important to do this review

Cochrane Reviews (Flodgren 2015) on the use of telemedicine indicate that answering questions about its efficacy requires attention to the contextual features of its application, including participants and settings. Effectiveness reviews of CCT in particular report a great degree of variability in effectiveness (e.g. Young 2011a), likely related to challenges with successful implementation (Thomas 2009). For example, Wilcox 2012 concluded that "the impact of telemedicine likely depends on characteristics of the environment in which it is deployed, including ICU organisation"; however, existing quantitative studies report limited contextual details. Currently, adoption of CCT appears haphazard and unplanned, and decision‐making about this lies hidden; this risks patient safety, quality of care and resource waste. Before such complex interventions are to be further developed and implemented, a more complete understanding of the factors that influence successful implementation is necessary (Glenton 2013). These include the perceptions, experiences and values of relevant stakeholders, as well as usability and applicability in different contexts.

It is therefore important to complement existing effectiveness reviews on CCT with a qualitative evidence synthesis that enables understanding of the factors affecting successful implementation, and illuminates the unintended consequences, acceptability and feasibility of CCT. This is especially important given that, despite a lack of conclusive evidence, there has been a rapid uptake of CCT in North America; and considering that the 24/7 hub‐and‐spoke model of CCT may have reach beyond critical care – Critical Care Outreach and Emergency Departments, for example – and in this way has great potential to transform the provision, quality and safety of acute care across hospital settings in the future.

How this review might inform or supplement what is already known in this area

This qualitative evidence synthesis addresses a subset of the Flodgren 2015 effectiveness review on interactive telemedicine. By looking at CCT in particular; it will complement Flodgren 2015 by providing an added layer of knowledge that can enable a more nuanced understanding of the factors influencing implementation of CCT. It also complements the Cochrane qualitative evidence synthesis of experiences of mHealth technologies in primary health care (Odendaal 2020), since critical care represents the acute far end of the health system and the opposite pole to primary care. In addition, CCT is distinct as an application from the traditional models of mHealth, which rely on mobile technology, used in primary care, since it uses a hub‐and‐spoke model to provide a 24/7 continuous form of clinical support to bedside practice rather than just being an interface for sporadic communication between patients and providers.

Objectives

To identify, appraise and synthesise qualitative research evidence on healthcare stakeholders’ perceptions and experiences of factors affecting the implementation of CCT, and to identify factors that are more likely to ensure successful implementation of CCT for subsequent consideration and assessment in telemedicine effectiveness reviews.

Methods

Criteria for considering studies for this review

Type of studies

We included empirical studies that used qualitative designs and methods for data collection and analysis. These included ethnographic studies using participant observation and phenomenological studies using interviews. We considered studies using mixed designs where the qualitative component and findings could be discerned; we also considered qualitative process evaluations as well as formative studies used to inform the design of CCT where the previous statement applied. We included studies regardless of whether these were linked to effectiveness studies of CCT. We excluded studies that used qualitative data‐collection methods but performed quantitative data analysis (e.g. using descriptive statistics). We considered both published and unpublished studies and studies published in any language. We did not exclude studies based on our assessment of methodological limitations, but used this information to assess our confidence in the review findings.

Topic of interest

Study participants

We considered all relevant stakeholders with a part to play in the implementation of CCT, including:

• All kinds of critical care workers (i.e. professionals, paraprofessionals and lay health workers) who make use of telemedicine to support or provide care to patients or family members, or both. Critical care workers are the main users of CCT and/or are the ones whose daily work is influenced to various degrees by the introduction of CCT. Their views about acceptance, resistance to or rejection of CCT are likely to be a contributing factor to implementation success or failure.

• Any other individuals or groups involved in the commissioning, evaluation, design and implementation of CCT. These individuals or groups can include administrative staff, information technology staff, managerial and supervisory staff, and industry partners who may or may not be based in a critical care facility, but must be involved in the use or implementation of CCT. We also considered participants identified as the technical staff who develop and maintain the CCT architecture used, since it is their logic and understanding of critical care services that underpin the final product at the point of use.

• Critical care patients and family members who have been the consumers or been involved in the development of CCT. As the recipients of care mediated by CCT, their views are likely to hold insight into factors influencing successful implementation.

Study settings

We included studies of telemedicine programmes implemented in critical care services, irrespective of specialisation (e.g. general, cardiothoracic, liver), or country. For the purposes of this review, we define critical care as the concentration of healthcare staff and equipment in a distinct area of the hospital in order to care for people whose conditions are life‐threatening and who need constant and close monitoring and support. Critical care services provide intensive 24‐hour monitoring and support of threatened or failing vital functions in people who have illnesses with the potential to endanger life.

CCT interventions

This review focuses on healthcare stakeholders' perceptions and experiences of factors affecting the implementation of CCT; we considered studies that looked at either the initiation or ongoing delivery of CCT. For the purposes of this review, CCT consists of the following combination:

• laboratory and diagnostics management, and patient health records including the continuous electronic recording of patients' vital signs at the bedside linked to a computer system enabling display of real‐time data;

• provider‐based decision support, in the form of clinical decision‐making algorithms and electronic alerts; and

• synchronous, interactive provider/client to provider telemedicine, using a remotely‐located team of critical care specialists, including doctors and nurses, who monitor the patients.

We required the presence of all three features to identify an intervention as CCT. We did not consider CCT applications that excluded clinical decision‐making as in some forms of plain remote screening.

Search methods for the identification of studies

Electronic searches

The EPOC Information specialist helped develop the MEDLINE search strategy in consultation with the review authors. We used the following databases to identify primary research studies for inclusion.

• MEDLINE 1946 to October Week 3 2019, Ovid (searched 14 October 2019)

• Embase 1974 to October Week 3 2019, Ovid (searched 14 October 2019)

• CINAHL 1937 to October Week 3 2019, EbscoHost (searched 14 October 2019)

• Web of Science Core Collection 1900 to October Week 3 2019, Clarivate Analytics (searched 14 October 2019)

The search strategies are given in Appendix 2; we tailored the MEDLINE search as necessary for each database following the Cochrane Qualitative Research Methods Group's guidelines (Booth 2011). We did not apply any limits on language or publication date. We searched all databases from inception to the date of search (14 October 2019). We included a methodological filter for qualitative studies. 

Searching other resources

We sought related reviews through PDQ‐Evidence (www.pdq-evidence.org, searched 14 October 2019), the reference lists of which we scanned for relevant studies. We also searched the reference lists of all included studies.

Grey literature

We searched for grey literature through The Grey Literature Report (www.greylit.org, searched 14 October 2019) and OpenGrey (www.opengrey.eu, searched 14 October 2019). We used GoogleScholar to search for references to the included studies.

Selection of studies

We collated all titles and abstracts identified through the search strategy into one reference management database (Covidence). After removing duplicate records, four review authors (AX, KI, SB, MT) independently screened the corpus of identified literature for relevant studies using a predetermined tool based on the SPIDER framework (Cooke 2012; Appendix 3) to evaluate eligibility. Following title and abstract review, we excluded irrelevant citations. We retrieved the full text of all the papers identified as potentially relevant by two review authors. Two review authors then assessed these papers independently, resolving disagreements through discussion, or by involving a third member of the team. 

We include a table listing studies that we excluded from our review at the full‐text stage, and the main reasons for exclusion. We include a PRISMA flow diagram to show our search results and the process of screening and selecting studies for inclusion.

Language translation

Relevant studies published in a language other than English would have been translated following the approach proposed by Downe 2019. However, this was not necessary, since we did not find studies in languages other than English.

Sampling of studies

We acknowledge that qualitative evidence synthesis aims for variation in concepts rather than an exhaustive sample, and large amounts of study data can impair the quality of the analysis. Therefore, once we identified all the studies eligible for inclusion, we assessed whether their number or data richness were likely to represent a problem for the analysis, and considered selecting a sample of studies. For the purposes of this review, and given the limited literature on the topic of CCT, we decided against sampling and instead included all the eligible articles.

Data extraction

At least two review authors extracted key features of the included papers independently, using a predetermined table to include: author(s), year, country, hospital type, ICU model and staffing, CCT system and vendor, study design, data collection and participants. We also extracted data on stakeholders' perceptions and experiences of factors affecting the implementation of CCT; this included authors' interpretations as well as actual data in the form of quotes or field‐note extracts. We considered data presented in either the Results or Discussion sections of the articles.

Appraisal of the methodological limitations of included studies

Two review authors (AX, KI) independently applied a predetermined set of quality criteria to each of the included studies, based on the Critical Appraisal Skills Programme (CASP) quality assessment tool for qualitative studies (CASP 2013). We considered all eligible studies, irrespective of quality. In cases of disagreement between the two review authors, a third member of the team (JP) was invited to adjudicate. We assessed methodological limitations according to the following questions:

• Was there a clear statement of the aims of the research?

• Is a qualitative methodology appropriate?

• Was the research design appropriate to address the aims of the research?

• Was the recruitment strategy appropriate to the aims of the research?

• Were the data collected in a way that addressed the research issue?

• Has the relationship between the researcher and participants been adequately considered?

• Have ethical issues been taken into consideration?

• Was the data analysis sufficiently rigorous?

• Is there a clear statement of findings?

We report our assessment in a Methodological Limitations table (Table 2, Additional Tables).

1. Methodological limitations of included studies.

Study	Was there a clear statement of the aims of the research?	Is a qualitative methodology appropriate?	Was the research design appropriate to address the aims of the research?	Was the recruitment strategy appropriate to the aims of the research?	Were the data collected in a way that addressed the research issue?	Has the relationship between researcher and participants been adequately considered?	Have ethical issues been taken into consideration?	Was the data analysis sufficiently rigorous?	Is there a clear statement of findings?
Goedken 2017	Yes	Yes	Yes	Insufficient	Insufficient	No	Yes	Insufficient	Yes
	The aim of the study is clearly expressed in the abstract and introduction of the paper	The study sought to illuminate ways in which Tele‐ICU can optimise its benefits, therefore a qualitative approach was the appropriate methodology to address the study's aim	Although the qualitative approach was appropriate to address study's aim, the researchers did not discuss or justify why they decided to employ the methods they used	The researchers did not discuss how participants were selected. However, all types of end‐users were included in the sample. There was also no reporting around the recruitment strategy. In their supplement they reported that they approached participants through an email and no participant declined to participate	Data saturation was not reported. The number of participants who were interviewed at the 3 time periods of data collection was not clearly presented. Field notes were not included in the analysis. Focus groups were only reported in the data collection section of the paper	The researchers' critical examination of their own role, potential bias and influence during data collection, sample recruitment and choice of location was lacking	The study was approved by the relevant institutions locally and nationally. Participants consented to participate in the study. However, no information was provided on whether participants had sufficient explanation about the study and whether confidentiality and anonymity would be maintained during and after data collection	An in‐depth description of the analysis process was lacking. Reporting of how the themes were derived from the data was not included, although the authors did report that they applied a coding tree. The researchers’ role, potential bias or influence during analysis and selection of data were not reported	Findings were clearly described, and adequate discussion about these was included. Findings were discussed in relation to the research aims. The 3 authors analysed only 10% of transcripts collectively; the remainder were analysed independently
Hoonakker 2013	Yes	Yes	No	No	Yes	No	Insufficient	Yes	Yes
	The aim of the study was clearly stated. Researchers argued that it was important to examine the satisfaction and motivation of highly skilled and expert ICU nurses for working in the Tele‐ICU working environment, compared to a clinical ICU environment	A qualitative approach was reasonable, considering the research aims. No explanation was provided about the chosen qualitative approach and type of interviews	No overall qualitative design identified. The authors did not provide any justification about how they decided to use a qualitative approach and interviews	The researchers did not report any inclusion and exclusion criteria for sampling the Tele‐ICU nurses. Nor did they explain how the 10 Tele‐ICU nurses interviewed from each unit differed from those who were not interviewed	The data collection method was clearly described, although the choice of method was not justified. The setting where interviews were conducted was not reported. Although an interview guide was used, the link given to this was inactive. Saturation of data was not discussed, but the number of responses corresponding to each category is reasonable	The role of the researchers (bias or influence) during data collection, choice of location, and sample recruitment were lacking	The study was granted approval by the institutional research committees. Transcripts were kept anonymous. But no information was provided about participants' explanations about the purpose, benefits, and harms from the study. It was not clear if interviewees were asked to return a signed consent form	The data analysis process was described sufficiently. The researchers explained how data were selected from the original sample. Enough data extracts were used to support the study's findings. But the researchers did not discuss their own role and potential bias during analysis and selection of data for presentation	The findings were presented clearly. The research team read the transcripts and the interviews were coded by 2 researchers, thus enhancing rigour in the analysis. The findings were discussed in relation to the study's original aim
Hoonakker 2018	Yes	Yes	Yes	No	Yes	No	Insufficient	Yes	Yes
	A clear statement of the aim of the research was reported. The researchers explained why the study was important to undertake	A qualitative approach was an appropriate methodology to address the study's aims, considering that the researchers aimed to explore Tele‐ICU nurses' experiences	The researchers used a case‐study research design by employing multiple data collection methods. This paper focused on the findings from the interviews. The researchers explained why they chose to use this research design	The study's participants accepted to be interviewed voluntarily. However, information about participants’, non‐participants’ characteristics, and whether any differences affected the quality of data was lacking. No discussion was provided about the process of recruitment	Interviews and other data collection methods were employed in the study. However, only interview data were reported in this paper. Interviews were recorded and analysed using a qualitative data analysis software. Saturation of data was not discussed. The structure of the participating setting was described	The researchers did not provide information about their background or relationship with the organisation and study participants	Transcribed interviews were anonymous, and the study was approved by the Institutional Review Board. It was not reported if the study was explained to participants or they provided written consent	A clear description of the data analysis process was reported. Data were initially coded, before assembled into categories, and developed into matrices. The research team collectively completed the analysis over several meetings. The role of researchers during the data analysis process was not discussed	The findings were discussed in relation to the aim of the research. More than 1 researcher analysed the transcripts, and more than 1 method was used to collect data. Data extracts were not adequately used to support the researchers’ interpretations
Jahrsdoerfer 2013	Yes	No	No	Yes	No	No	Yes	No	No
	The aims of the research were clearly stated informing the reader about the study's participants and research design	The study used a questionnaire, which included 2 open‐ended questions. But the phrasing of those 2 questions along with low responses to them, and unclear data analysis, render the qualitative aspect of the study's research design weak	The researchers did not explain why they decided to employ a survey over other approaches (e.g. interviews)	There was a clear explanation of how the participants were selected, and why those selected were the most appropriate. The researchers provided possible explanations as to why the response rate in 1 site was low	Data were not collected in an optimal way to address the qualitative aspect of the study (understanding family members' perceptions)	Trained volunteers explained the study, distributed and collected the completed questionnaires from participants. In 1 site the volunteers did not fully adhere to the study protocol for recruitment	Anonymity of participants' responses was assured, and informed consent was provided. Completed questionnaires were placed in sealed envelopes. The study was approved by the institutional review boards at each of the participating hospitals	Identified themes were broken down into categories, but it was not clear how the themes were derived, and limited evidence was provided to support development of these themes. Moreover, the researchers did not critically discuss their role, potential bias or influence during data analysis	Only findings from 1 of the 2 open‐ended questions were reported. 2 researchers identified recurring themes from comments. Inadequate data extracts were used to support interpretations
Kahn 2019	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
	The goal of the study was clearly stated. Researchers presented clearly why the study was important and what its relevance was at general and individual level	The qualitative approach was appropriate to meet the study’s objective	The researchers employed a focused ethnography to address the study's aims. They explicitly justified why they chose to use in‐depth site visits and in‐person interviews	The researchers clearly described the process of ICU site selection in detail, based on set characteristics and eligibility criteria	The researchers justified their data collection choices. Interview and focus group guides were used, which were digitally audio‐recorded. Saturation of data was discussed. Details of the site visits were not provided. It is not clear when and for how long observations were held in each unit	The paper includes an online supplement which reports on the study methods. The researchers’ relationship with participants was reported in a previously published protocol	Written consent was provided by the participants and the study has been approved by a University Review Board. The study protocol discussed ethical considerations in detail	A rigorous data analysis process was described in full details in the study. A constant comparative approach was used. Interpretation of data were cross‐checked with participants. A thematic codebook was developed. The researchers' role during data analysis has been discussed	The findings were clearly described and discussed in relation to the research aims
Khunlertkit 2013	Yes	Yes	Insufficient	Yes	Yes	Insufficient	Insufficient	Yes	Yes
	The aim of the research was clearly stated. The researchers explained why the study was important, and its relevance within Tele‐ICU research field	The qualitative approach was appropriate to shed light to existing knowledge about Tele‐ICU care processes and patient outcomes	No overall design identified. They did not provide any justification for the study's methodology	The researchers explained why they used a purposeful sampling strategy. Moreover, they justified why the study's sample was the most appropriate to provide the type of knowledge relevant to the study's aim. Inclusion criteria were presented	Information about the location of interviews was provided. It was reported how data were collected, and what predefined questions and probes were asked during the interviews. The link to the interview guide is no longer active. Saturation of data was discussed	The issue of selection bias, and the actions employed to mitigate its impact on the study's results, were examined. The issue of interviewer's bias was addressed	Ethical issues were taken into consideration. The study was approved by 3 institutional review boards. All participants consented to be interviewed. IRBs waived the need for informed consent; it is unclear if participants gave written consent	A detailed description of the data analysis process was included. 2 analysts were involved in data analysis and theme identification. Sufficient data were presented to support the study's findings. Both positive and negative outcomes were taken into consideration in the presentation of findings	The findings were explicitly described. Extracts from the interviews provided rich insights into the identified themes. A sufficient discussion of the findings about the original research aim was included
Moeckli 2013	Yes	Insufficient	Insufficient	No	Insufficient	No	Yes	Insufficient	Yes
	The aim of the study was clearly stated. The researchers justified why the study was important, and reported what gaps in research they attempted to address	Given the focus of the study to evaluate the impact of a Tele‐ICU programme, the choice of qualitative methodology, without a quantitative component in the study, was not sufficiently justified	The researchers employed interviews and observation to address the study's aims. Although a qualitative approach was appropriate, the researchers did not discuss or justify their choice of methods	Participant selection was not explained. No information was provided on why some participants were interviewed in the pre‐ and others in the post‐implementation phase	Data collection methods were not adequately justified. Not enough information was provided about the observational data. The rationale for using observation was not explained, and it was not clear how observational data corroborated interview data. Saturation of data was not discussed	There was no critical examination of the researchers’ role during data collection, sample recruitment and choice of research site. The role of observers – for example, participant or non‐participant observation, establishment of rapport, maintenance of role‐boundaries and Hawthorn effect – was not discussed	Approval was granted by the national and local institutional review boards. Informed consent was obtained by participants, but little is reported about how the study was provided to participants, whether participants’ identification was concealed	Limited information was provided about the codebook development. Thematic analysis was performed, but reporting of the process through which the themes were identified was lacking. Sufficient data were presented to support the findings, while contradictory data were also considered	Development of the findings was not adequately explained. The trustworthiness of the findings was enhanced with a consensus coding by 3 researchers, but this process was only applied to 10% of the data
Mullen‐Fortino 2012	Yes	No	No	Yes	No	No	Yes	No	No
	A clear statement of the study's aim and relevance was reported	An internet survey was used to address the study's aims. Participants were also given space to report their opinions about the Tele‐ICU programme. The nature of the survey does not allow for in‐depth understanding of attitudes toward, and perceptions about, the use of telemedicine	The cross‐sectional research design was not ideal to examine in depth the nurses' perceptions about the Tele‐medicine programme	The researchers explained how the participants were selected, and why they were appropriate to provide the type of data sought by the study	The open‐ended space allowed on the questionnaire for gaining nurses' perceptions of tele‐medicine did not allow for collection of in‐depth and rich data	The researchers did not critically examine their own role, potential bias or influence during data collection, sample recruitment and choice of location	The survey was administered anonymously to the participants. No information was provided whether the study was adequately explained to participants	Analysis of answers to the open‐ended questions was performed by reviewing the participants' responses and summarising key themes. Participant quotes were reported in a table to support the identified key themes	The identified themes and corresponding quotes were presented in a table. The qualitative findings were inadequately discussed. There was no mention of credibility, respondent validation or use of more than 1 analyst
Shaphori 2011	Yes	No	No	Yes	No	No	Insufficient	No	Insufficient
	There was clear reporting of the study aims. The researchers discussed why they considered the study significant in the wider context of Tele‐ICU research	The study used a survey questionnaire to address the study's aims. For each survey question, a free‐text section was provided to capture participants' concerns and suggestions before implementation of a Tele‐ICU intervention	The researchers aimed to capture staff concerns and suggestions about the implementation of ICU‐Telemedicine. For this purpose, free‐text sections were added within each survey question which captured data in a limited way	The researchers explained how and why the participants were selected. No characteristics were provided about the non‐respondents to the survey	The researchers justified the choice of setting for the data collection. Data were collected through survey questionnaires. Free‐text sections for each survey question formed the basis of the qualitative data in the study. Saturation of qualitative data was not discussed	The relationship between the researchers and participants was not reported in the study	The study was approved by ethics and research review boards. An educational preparation for the study was provided prior to the survey. The survey was administered anonymously, but issues around maintaining confidentiality of qualitative data were not reported in the study	An in‐depth description of the analysis process was lacking, and no explanation provided about how the data presented were selected from the original sample to demonstrate the analysis process	Summary commentary data were presented in the study, although actual quotes were missing. The credibility of the qualitative findings was not discussed. An adequate discussion of the findings about the study's aims was provided
Stafford 2008	Yes	Yes	Yes	Yes	Yes	No	Yes	Yes	Yes
	The goal of the study was to describe the experiences of healthcare workers in the eICU and to examine how they functioned in this setting. The researchers justified the significance of this study	An ethnographic approach was used in this study. This approach was appropriate for addressing the study's aims, that is, to describe staff experiences and interactions	The researchers used ethnography, with semi‐structured interviews and participant observation. The researchers explained their choice of design and data collection methods	A purposeful sample was targeted, including all eICU physicians and eICU nurses. The researchers reported how many participated in the semi‐structured interviews and how many participated in the field study. Nobody declined to participate in the study	The setting was clearly described. The researchers clearly explained and justified their choice of data collection methods. But no information was provided about the interview topic guide and observation schedule. Saturation of data was not discussed	The researchers’ role during the formulation of the research aims, data collection methods, sample recruitment and choice of the research setting was not adequately discussed	The study was approved by the participating institutions' review boards. Participants provided informed consent, while identifiers or names in the transcriptions were avoided. But there was insufficient information about how the research was explained to participants	The analysis process was described sufficiently. Coding, memos and typologies were used as part of the analysis. An audit trail was used to assure confirmability. Credibility, transferability, and dependability were also considered	The findings were explicitly presented, and adequate discussion of these was provided. The number of data analysts was not reported. Credibility of the findings was established through data triangulation
Thomas 2017	Yes	Yes	Insufficient	Insufficient	Yes	No	Yes	Yes	Yes
	The aim and significance of the study were clearly stated	A qualitative approach was appropriate to address the study's aims. The researchers aimed to understand how ICU physicians and nurses understood and practised Telemedicine	Individual and group semi‐structured interviews were conducted. The researchers supported their choice by justifying the strengths of open‐ended qualitative interviewing, but details and justification for the focus groups is missing. An overall qualitative design is not identified	It was noted that sample might not have been representative of the population. No information was provided whether more participants were needed to generate data. Little was known why some chose not to take part, and whether a clear recruitment strategy was used	An adequate explanation of how interviews were conducted was provided by the researchers. Saturation of data was not discussed. More details about the characteristics of the setting, interview guide and interview transcripts were previously described	The researchers did not discuss their own role, potential bias and influence during data collection. Little was noted about where the interviews took place and whether any interruptions occurred, and if so how these were managed	The study was approved by Institutional and research boards. Informed consent was obtained by all participants. However, little is known about whether clear explanations of the study was provided; what were the study's potential benefits and harms; and how anonymity of transcripts was ensured	Information about the coding process was missing. Information about the researchers' role during analysis was lacking. No information was provided about their education, background, and perspective. A thematic analysis was used by applying inductive and deductive coding	Findings were explicitly described in the study with an adequate discussion of these in relation to the original aim of the study. Independent analysis of the data by more than 1 researcher was performed. Verbatim quotes were presented
Ward 2015	Yes	Yes	Yes	No	No	No	Insufficient	No	Insufficient
	A clear statement of the aim of the research was reported in the study, which was to assess staff acceptance at multiple hospitals that had implemented a Tele‐ICU system. The researchers also reported why they thought their study was important	The study used both quantitative and qualitative methodology to address the aims of the study. Qualitative approach was appropriate, given the aim to gain a deeper understanding of factors affecting staff perceptions of Tele‐ICU services	Although the researchers provided a discussion about using a qualitative approach alongside a survey, they did not fully explain or justify this design choice	It was not explained how participants were selected from the total sample; whether they had also completed the survey or not; and why they were the most appropriate to provide the data sought by the study. Participant characteristics were not reported	Data were collected through phone interviews and site visits. No information was provided on how interviews were conducted or if a topic guide was used. Interviews were recorded and transcribed. Saturation of data was not discussed. Interview process and purpose of site visits was unclear	The study provided no critical account of the researchers' role, potential bias and influence during data collection and choice of research location	The review boards approved the protocols for the interviews. Transcripts from the interviews were anonymised. It was not clear whether a detailed explanation of the study's aims, potential benefits or harms, and issues of confidentiality were provided to participants	No information was reported about the coding process; how themes were derived from the data; the role of the researchers; and potential bias or influence. Data were reported as narrative text of the authors’ interpretations instead of direct participant quotes	There was a clear description of the identified themes. A summary of the interviewees' responses supported each theme. An adequate discussion on the qualitative findings related to the study's aims was presented. 3 researchers coded and analysed the qualitative data into themes. No direct quotes provided
Wilkes 2016	Insufficient	Yes	Insufficient	Yes	Insufficient	No	Insufficient	Insufficient	No
	The aim of the research was phrased in slightly different ways throughout the text. The significance of the study was discussed	The qualitative methodology used in the study was appropriate, considering that the aim to gain insight into the organisational culture of Tele‐ICUs.	The study used semi‐structured interviews, which were appropriate to address the aims of the research. The overall study design was unclear	There was adequate explanation of the research settings and participants. Justifications were provided about why some participants were not interviewed. Purposive sampling was used, but interviews were recommended by 'administrative leaders' without a clear indication of selection criteria	The researchers justified their choice of method. Interview guides were developed. Saturation of data was not discussed. Handwritten notes were taken during the interviews, increasing the risk of inaccuracies. The researchers discussed their efforts to assure accuracy of notes, but whether this was achieved remains questionable	The study did not report the researchers' role, potential bias and influence during data collection, recruitment or choice of research sites	The study was sufficiently explained to the participants. An informed consent form was signed by the participants, and their anonymity was ensured. Approval was sought by the research company review board. Approval from a university review board, or from the hospital sites is not mentioned	Inter‐analyst agreement was mentioned, but a clear indicator for this was not reported. Much of the findings were from the researchers’ interpretation, not always supported by participant quotes. The researchers' role, potential bias and influence were not reported	The study's findings were not reported in a clear and explicit way, resulting in difficulty in tracking the identified themes. Some discussion about the existing literature was included. Credibility of the study was enhanced by including more than 1 analyst

Data management, analysis and synthesis

We imported all the included papers into the NVivo qualitative data analysis software (QSR International). Data synthesis drew from the CFIR framework (Appendix 1) to examine the available evidence on factors affecting the implementation of CCT. As noted in the Background, the CFIR is a 'meta‐theoretical' model, made up of five constructs: I. Intervention characteristics, II. Inner and III. Outer Settings, IV. Characteristics of individuals, and V. the Process of implementation. CFIR informed but did not restrict data synthesis, with additional themes not captured by CFIR used to challenge and add to previously‐held assumptions. This approach led to a more refined understanding of implementation in the context of CCT, building on and extending the propositions of CFIR, thus strengthening the theoretical generalisability of the review findings.

We followed the Best‐fit framework approach (Carroll 2013), since this allows examination of the alignment of identified themes with an existing framework, as well as conceptual revisions as necessary. Our approach consisted of four main analysis stages completed by two review authors (AX, KI): First, we developed a coding tree in NVivo based on the CFIR framework and coded data from the included studies against this. Second, themes not accounted for by CFIR were noted, coded and classified under separate constructs. Third, following a consensus approach, we used additional constructs to supplement CFIR; had the framework changed substantially, the papers would be re‐coded based on the new framework, but this was not required. Fourth, we revisited the data to explore relationships between themes and constructs in order to develop concise review findings statements that capture the coded data.

Assessing our confidence in the review findings

Two review authors (AX, KI) used the GRADE‐CERQual (Confidence in the Evidence from Reviews of Qualitative research) approach to assess our confidence in each finding (Lewin 2018). CERQual assesses confidence in the evidence, based on the following four key components.

• Methodological limitations of included studies: the extent to which there are concerns about the design or conduct of the primary studies that contributed evidence to an individual review finding

• Coherence of the review finding: an assessment of how clear and cogent the fit is between the data from the primary studies and a review finding that synthesises those data. By cogent, we mean well‐supported or compelling

• Adequacy of the data contributing to a review finding: an overall determination of the degree of richness and quantity of data supporting a review finding

• Relevance of the included studies to the review question: the extent to which the body of evidence from the primary studies supporting a review finding is applicable to the context (perspective or population, phenomenon of interest, setting) specified in the review question

After assessing each of the four components, we made a judgement about the overall confidence in the evidence supporting the review finding. We judge confidence as high, moderate, low, or very low. The final assessment was based on consensus among the review authors. All findings started as high confidence and were then downgraded if there were important concerns about any of the CERQual components. The starting point of high confidence reflected a view that each review finding should be seen as a reasonable representation of the phenomenon of interest, unless there were factors that weakened this assumption.

Summary of qualitative findings table and Evidence Profiles

We present summaries of the findings and our assessments of confidence in these findings in Table 1. We present detailed descriptions of our confidence assessment in a two‐part Evidence Profile (Table 3; Table 4, Additional Tables).

2. Evidence profile (findings 1‐10).

Finding 1: Hospital staff’s personal experience, and anecdotes from colleagues, supported their belief that CCT has positive effects on patient care. Specifically, these effects were in terms of patient safety and quality of care, support at the bedside by critical care experts, and standardisation of practice.
Assessment for each GRADE‐CERQual component
Methodological limitations	6 studies contributed data to this finding. None of the studies discussed researcher reflexivity. 2 studies were assessed as having methodological limitations related to data analysis and collection, of which 1 study was assessed as having methodological limitations related to research design and the other was assessed as having methodological limitations related to recruitment.  A third study was also assessed as having methodological limitations related to recruitment. The body of evidence contributing to this review finding was assessed as having minor concerns about methodological limitations
Coherence	No or very minor concerns about coherence
Relevance	Moderate concerns about relevance, because while the studies covered different ICU settings from different countries these were all North American; and the value staff placed on their experiences and anecdotes is likely to differ across world regions
Adequacy	Minor concerns about adequacy, because the 6 studies together offer only moderately rich data
Overall GRADE‐CERQual assessment and explanation
Moderate confidence	Downgraded to moderate confidence because we had minor concerns about methodological limitations, coherence, and adequacy; and moderate concerns about relevance
Contributing studies
Khunlertkit 2013; Moeckli 2013; Shahpori 2011a; Stafford 2008a; Ward 2015; Wilkes 2016
Finding 2: Hospital staff and family members described several advantages of CCT. Bedside and hub staff strongly believed that the main advantage of CCT was having access to experts when bedside doctors were not available. Families also valued having access to experts. In addition, hospital staff described how CCT could support clinical decision making and mentoring of junior staff
Assessment for each GRADE‐CERQual component
Methodological limitations	7 studies contributed data to this finding. 2 studies discussed researcher reflexivity. 2 studies were assessed as having methodological limitations related to data analysis, research design and data collection. 1 study was assessed as having methodological limitations related to recruitment. The body of evidence contributing to this review finding was assessed as having  minor concerns about methodological limitations
Coherence	No or very minor concerns about coherence
Relevance	Minor concerns about relevance, because the studies covered different ICU settings from different countries, and even though these were all North American the focus of the finding is on standard features of CCT technology that are unlikely to differ across world regions
Adequacy	No or very minor concerns about adequacy
Overall GRADE‐CERQual assessment and explanation
High confidence	Graded as high confidence because we had minor concerns about methodological limitations,  relevance, coherence, and adequacy
Contributing studies
Jahrsdoerfer 2013; Kahn 2019; Khunlertkit 2013; Moeckli 2013; Shahpori 2011a; Stafford 2008a; Thomas 2017
Finding 3: Bedside staff valued the potential adaptability of CCT to speak to local needs and practices. However, this was not always evident, with reported examples being mainly around developing camera usage etiquette and integration with local protocols.
Assessment for each GRADE‐CERQual component
Methodological limitations	3 studies contributed data to this finding. 1 study discussed researcher reflexivity. 1 study was assessed as having methodological limitations related to recruitment. The body of evidence contributing to this review finding was assessed as having no or very minor concerns about methodological limitations
Coherence	Minor concerns about coherence, because the data contributing to the review finding were only reasonably consistent within studies
Relevance	Moderate concerns about relevance, because while the finding is focused on a technological feature of CCT the studies only covered a small range of settings from only 1 country; and local practices are likely to differ across settings and countries
Adequacy	Serious concerns about adequacy due to only 3 studies offering only thin data
Overall GRADE‐CERQual assessment and explanation
Low confidence	Downgraded to low confidence because we had no or very minor concerns about methodological limitations, minor concerns about coherence, moderate concerns about relevance and serious concerns about adequacy
Contributing studies
Moeckli 2013; Stafford 2008a; Thomas 2017
Finding 4: Both bedside and hub clinicians reported difficulties with the implementation of CCT. Key barriers related to implementation were perceptions of additional workload, need for more coordination work, and concern around the use of cameras
Assessment for each GRADE‐CERQual component
Methodological limitations	5 studies contributed data to this finding. None of the studies discussed researcher reflexivity. 3 studies were assessed as having methodological limitations related to data analysis and collection, of which 2 were also assessed as having methodological limitations related to research design; and 1 of the 3 was assessed as having methodological limitations related to recruitment. The body of evidence contributing to this review finding was assessed as having moderate concerns about methodological limitations
Coherence	No or very minor concerns about coherence
Relevance	Minor concerns about relevance, because the studies covered different ICU settings from different countries and even though these were all North American the focus of the finding is on standard features of CCT technology that are unlikely to differ significantly across world regions
Adequacy	Moderate concerns about adequacy, because the 5 contributing studies together offer only moderately thin data.
Overall GRADE‐CERQual assessment and explanation
Moderate confidence	Downgraded as moderate confidence because we had no or very minor concerns about relevance, minor concerns about coherence, and moderate concerns about methodological limitations and adequacy
Contributing studies
Moeckli 2013; Mullen‐Fortino 2012; Shahpori 2011a; Stafford 2008a; Ward 2015
Finding 5: Cost considerations featured as an influencing factor in a limited way, with only a few examples noting the high cost of implementing CCT especially compared to the cost of recruiting additional ICU staff
Assessment for each GRADE‐CERQual component
Methodological limitations	2 studies contributed to this finding. Neither study discussed researcher reflexivity. 1 study was assessed as having methodological limitations related to data analysis, research design, and data collection. The body of evidence contributing to this review finding was assessed as having moderate concerns about methodological limitations
Coherence	No or very minor concerns about coherence.
Relevance	Serious concerns about relevance, because the studies only covered a small range of settings from only 1 region; these were conducted several years ago and it is likely the health resource allocation model used then is no longer current
Adequacy	Serious concerns about adequacy, because the 2  contributing studies together only offer seriously thin data.
Overall GRADE‐CERQual assessment and explanation
Low confidence	Downgraded to low confIdence because we had no or very minor concerns about coherence, moderate concerns about methodological limitations, and serious concerns about relevance, and adequacy
Contributing studies
Shahpori 2011a; Stafford 2008a
Finding 6: Hospital staff as well as family members perceived CCT to be providing a community benefit, specifically relating to patients' and families' desire to stay close to their local community without requiring transfer to specialist centres to access critical care expertise
Assessment for each GRADE‐CERQual component
Methodological limitations	5 studies contributed data to this finding. None of the studies discussed researcher reflexivity. 2 studies were assessed as having methodological limitations related to data collection and analysis, of which 1 study was assessed as having methodological limitations related to research design; and the other was assessed as having methodological limitations related to recruitment. A third study was also assessed as having methodological limitations related to recruitment. The body of evidence contributing to this review finding was assessed as having moderate concerns about methodological limitations
Coherence	No or very minor concerns about coherence
Relevance	Serious concerns about relevance, because while the studies covered different ICU settings from different  countries, these were all from North America where decentralisation of health care is a strong feature. The review finding refers to factors highly influenced by social norms (values and perceptions of patients about community benefit in health care), which are highly likely to differ across world regions
Adequacy	Minor concerns about adequacy, because the 5 studies together offer only moderately rich data
Overall GRADE‐CERQual assessment and explanation
Moderate confidence	Downgraded to moderate confidence because we had no or very minor concerns about coherence, minor concerns about adequacy, moderate concerns about methodological limitations, and serious concerns about relevance
Contributing studies
Goedken 2017; Moeckli 2013; Shahpori 2011a; Ward 2015; Wilkes 2016
Finding 7: Hospital staff greatly valued the nature and quality of social networks between the bedside and CCT hub teams. Key issues for them were trust, acceptance, teamness, familiarity and effective communication between the 2 teams
Assessment for each GRADE‐CERQual component
Methodological limitations	8 studies contributed data to this finding. Only 1 study discussed researcher reflexivity. 1 study was assessed as having methodological limitations related to research design, data analysis and collection. 3 studies were assessed as having methodological limitations related to recruitment strategy, of which 1 was also assessed as having methodological limitations related to research design. The body of evidence contributing to this review finding was assessed as having minor concerns about methodological limitations
Coherence	No or very minor concerns about coherence
Relevance	Minor concerns about relevance, because the studies covered different ICU settings and even though these were all North American the focus of the finding is on standard features of health professional practice that are unlikely to differ significantly across world regions
Adequacy	No or very minor concerns about adequacy
Overall GRADE‐CERQual assessment and explanation
High confidence	Graded as high confidence because we had  no or very minor concerns about coherence, and adequacy; and minor concerns about methodological limitations, and relevance
Contributing studies
Hoonakker 2018; Jahrsdoerfer 2013; Kahn 2019; Khunlertkit 2013; Moeckli 2013; Mullen‐Fortino 2012; Stafford 2008a; Wilkes 2016
Finding 8: Hospital bedside staff were concerned over the hub team not being aware of local unit norms, values, and culture. This led local bedside teams to feel that CCT intruded on their practice
Assessment for each GRADE‐CERQual component
Methodological limitations	6 studies contributed data to this finding. Only 1 study discussed researcher reflexivity. 2 studies were assessed as having methodological limitations related to data analysis and collection, of which 1 study was also assessed as having methodological limitations related to research design, and the other study was assessed as having methodological limitations related to recruitment. The body of evidence contributing to this review finding was assessed as having moderate concerns about methodological limitations
Coherence	No or very minor concerns about coherence
Relevance	Moderate concerns about relevance, because while the studies covered different settings these were only from 1 country; and local unit norms, values and culture are likely to differ across countries
Adequacy	Moderate concerns about adequacy, because the 6 studies together only offer moderately thin data
Overall GRADE‐CERQual assessment and explanation
Moderate confidence	Downgraded to moderate confidence because we had no or very minor concerns about coherence, and moderate concerns about methodological limitations, relevance, and adequacy
Contributing studies
Kahn 2019; Moeckli 2013; Mullen‐Fortino 2012; Stafford 2008a; Ward 2015; Wilkes 2016
Finding 9: Bedside clinicians were reluctant to use CCT because they lacked clarity about its purpose, were concerned that their decision‐making skills would be weakened through remote supervision, and did not consider hub clinicians an equal counterpart in patient management. Hub clinicians were disengaged due to lack of role clarity and limited integration with patient care
Assessment for each GRADE‐CERQual component
Methodological limitations	4 studies contributed data to this finding. 1 study discussed researcher reflexivity. 1 study was assessed as having methodological limitations related to data analysis, research design and data collection. 1 study was assessed as having methodological limitations related to recruitment. The body of evidence contributing to this review finding was assessed as having minor concerns about methodological limitations
Coherence	No or very minor concerns about coherence
Relevance	Moderate concerns about relevance, because the studies only covered a small range of settings and aspects of professional autonomy may be influenced by cultural factors and professional norms that are likely to differ across world regions
Adequacy	Minor concerns about adequacy, because the 4 studies together offer only moderately rich data
Overall GRADE‐CERQual assessment and explanation
Moderate confidence	Downgraded to moderate confidence because we had no or very minor concerns about coherence, minor concerns about methodological limitations, and adequacy, and moderate concerns about relevance
Contributing studies
Kahn 2019; Moeckli 2013; Shahpori 2011a; Stafford 2008a
Finding 10: Hospital locale shaped prioritisation of CCT, with staff in rural centres noting that CCT was of greater benefit to them considering their staff shortage and lack of critical care resources
Assessment for each GRADE‐CERQual component
Methodological limitations	4 studies contributed data to this finding. 1 study discussed researcher reflexivity. 2 studies were assessed as having methodological limitations related to data analysis and data collection,  of which 1 study was also assessed as having methodological limitations related to research design; and the other was assessed as having methodological limitations related to recruitment strategy. The body of evidence contributing to this review finding was assessed as having moderate concerns about methodological limitations
Coherence	Minor concerns about coherence, because the data were only reasonably consistent within studies
Relevance	Moderate concerns about relevance, because while the studies were from different settings and countries these were all North American; and perceptions about prioritisation and availability of critical care resources are likely to differ across world regions
Adequacy	Serious concerns about adequacy, because the 4 contributing studies together offered only thin data
Overall GRADE‐CERQual assessment and explanation
Low confidence	Downgraded to low confidence because we had moderate concerns about methodological limitations, relevance, and coherence; and serious concerns about adequacy
Contributing studies
Kahn 2019; Shahpori 2011a; Ward 2015; Wilkes 2016

3. Evidence profile (findings 11‐20).

Finding 11: Bedside and hub clinicians perceived the absence of support from, and lack of engagement in dialogue with, leaders and senior administrators during the implementation of CCT as major barriers. Listening to staff needs, and creating groundwork connections with them from the outset were perceived as facilitating factors to implementation
Assessment for each GRADE‐CERQual component
Methodological limitations	2 studies contributed to this finding. 1 study discussed  researcher reflexivity. The body of evidence contributing to this review finding was assessed as having no or very minor concerns about methodological limitations
Coherence	Minor concerns about coherence, because data were only reasonably consistent within studies
Relevance	Moderate concerns about relevance, because the studies were from only 2 settings from only 1 country, and perceptions concerning leadership are likely to differ across settings and countries
Adequacy	Serious concerns about adequacy, because the 2 contributing studies together offer only thin data
Overall GRADE‐CERQual assessment and explanation
Low confidence	Downgraded to low confidence because we had minor concerns about methodological limitations and coherence, moderate concerns about relevance, and serious concerns about adequacy
Contributing studies
Kahn 2019; Wilkes 2016
Finding 12: Hospital staff thought it was important to promote and offer training in the use of CCT before its implementation. This included rehearsing every step in the process, offering staff opportunities to ask questions and disseminating learning resources. Some also complained that experienced staff were taken away from bedside care and re‐allocated to the CCT hub team
Assessment for each GRADE‐CERQual component
Methodological limitations	5 studies contributed to this finding. 1 study discussed researcher reflexivity. 2 studies were assessed as having methodological limitations related to data analysis and data collection, of which 1 study was also assessed as having methodological limitations related to research design; and the other study was assessed as having methodological limitations related to recruitment. Another study was also assessed as having methodological limitations related to recruitment. The body of evidence contributing to this review finding was assessed as having moderate concerns about methodological limitations
Coherence	Minor concerns about coherence, because the data were only reasonably consistent within studies
Relevance	Minor concerns about relevance, because while the studies were all North American these were from different settings and countries; and perceptions about the need for training staff on new technology are unlikely to differ significantly across world regions
Adequacy	Minor concerns about adequacy, because the 5e studies together offered only reasonably rich data
Overall GRADE‐CERQual assessment and explanation
High confidence	Graded as high confidence because we had only minor concerns about relevance, coherence and adequacy, and moderate concerns about methodological limitations
Contributing studies
Kahn 2019; Moeckli 2013; Shahpori 2011a; Stafford 2008a; Ward 2015
Finding 13: Hospital staff reported the lack of access to information about how CCT staff, policies and procedures can be incorporated into the bedside workflow as a barrier to implementation
Assessment for each GRADE‐CERQual component
Methodological limitations	1 study contributed data to this finding. It was assessed as having methodological limitations related to recruitment and researcher reflexivity. The body of evidence contributing to this review was assessed as having minor methodological limitations
Coherence	No or very minor concerns about coherence
Relevance	Serious concerns about relevance, because the study was from a single setting in 1 country; and local policies, procedures and workflows are likely to differ across settings and countries
Adequacy	Serious concerns about adequacy, because of only 1 study offering only thin data
Overall GRADE‐CERQual assessment and explanation
Low confidence	Downgraded to low confidence because we had no or very minor concerns about coherence, minor concerns about methodological limitations, and serious concerns about relevance, and adequacy
Contributing studies
Moeckli 2013
Finding 14: Hospital staff's attitudes towards, knowledge about and value placed on CCT influenced acceptance of CCT. Staff were positive towards CCT because of its several advantages. But some were concerned that the hub staff were not able to understand the patient’s situation through the camera. Some were also concerned about confidentiality of patient data
Assessment for each GRADE‐CERQual component
Methodological limitations	7 studies contributed data to this finding. 2 studies discussed researcher reflexivity. 2 studies were assessed as having methodological limitations related to research design, data collection, and data analysis. 1 study was assessed as having methodological limitations related to recruitment. The body of evidence contributing to this review finding was assessed as having minor concerns about methodological limitations
Coherence	Minor concerns about coherence, because the data were only reasonably consistent within studies
Relevance	Minor concerns about relevance, because while the studies were all North American these were from multiple settings and different countries; and the focus of the finding is on technological features of CCT that are unlikely to differ across world regions
Adequacy	Minor concerns about adequacy, because the 7 contributing studies together offer only moderately rich data
Overall GRADE‐CERQual assessment and explanation
High confidence	Graded as high confidence because we only had minor concerns about methodological limitations, relevance, coherence, and adequacy
Contributing studies
Kahn 2019; Khunlertkit 2013; Moeckli 2013; Mullen‐Fortino 2012; Shahpori 2011a; Stafford 2008a; Thomas 2017
Finding 15: Hospital staff noted that acceptance and normalisation of CCT in their daily work took time; progressing through different stages of change did not occur with the same pace for everyone, with some remaining resistant to change
Assessment for each GRADE‐CERQual component
Methodological limitations	2 studies contributed data to this finding. 1 study assessed researcher reflexivity.  The body of evidence contributing to this review finding was assessed as having no or very minor concerns about methodological limitations
Coherence	Minor concerns about coherence, because the data were only reasonably consistent within studies
Relevance	Moderate concerns, because the studies were from a limited range of setting from only 1 country; and the finding's focus on pace of acceptance of new technology is likely to differ across settings and countries
Adequacy	Serious concerns about adequacy, because the 2 contributing studies only offer thin data
Overall GRADE‐CERQual assessment and explanation
Low confidence	Downgraded to low confidence because we had no or very minor concerns about methodological limitations, minor concerns about coherence, moderate concerns about relevance, and serious concerns about adequacy
Contributing studies
Kahn 2019; Khunlertkit 2013
Finding 16: Hub nurses’ personal attributes, specifically with regard to their motivation, multitasking competence and values were noted as important enablers for implementation of CCT
Assessment for each GRADE‐CERQual component
Methodological limitations	1 study contributed data to this finding. It was assessed as having methodological limitations related to research design, recruitment strategy, and researcher reflexivity. The body of evidence contributing to this review finding was assessed as having minor concerns about methodological limitations
Coherence	No or very minor concerns about coherence
Relevance	Serious concerns about relevance, because the data came from a single setting in 1 country; and the issue of nurses' personal attributes and motivation is highly likely to vary significantly across settings and countries
Adequacy	Moderate concerns about adequacy, because the 1 study offered only moderately rich data
Overall GRADE‐CERQual assessment and explanation
Low confidence	Downgraded to low confidence because we had no or very minor concerns about coherence, minor concerns about methodological limitations, moderate concerns about adequacy, and serious concerns about relevance
Contributing studies
Hoonakker 2013
Finding 17: Hospital staff were frustrated due to lacking a clear strategy for engagement; specifically lack of consistent training, the orientation of new and resistant staff to the hub facility, and timely co‐ordination for CCT implementation
Assessment for each GRADE‐CERQual component
Methodological limitations	2 studies contributed data to this finding. 1 study assessed researcher reflexivity. 1 study was assessed as having methodological limitations related to recruitment. The body of evidence contributing to this review finding was assessed as having minor concerns about methodological limitations
Coherence	Minor concerns about coherence, because the data were only reasonably consistent within  studies
Relevance	Moderate concerns about relevance, because the studies were from a limited range of settings in a single country, and the issue of staff engagement is likely to differ across settings and countries
Adequacy	Serious concerns about adequacy, because the 2 studies together offer only thin data
Overall GRADE‐CERQual assessment and explanation
Low confidence	Downgraded to low confidence because we had minor concerns about methodological limitations and coherence, moderate concerns about relevance, and  serious concerns about adequacy
Contributing studies
Kahn 2019; Moeckli 2013
Finding 18: Hospital staff were encouraged by the visibility of the intended benefits of CCT. They valued both quantitative feedback through auditing, as well as qualitative feedback through reflective accounts
Assessment for each GRADE‐CERQual component
Methodological limitations	3 studies contributed data to this finding. 2 studies discussed researcher reflexivity. The body of evidence contributing to this review finding was assessed as having no or very minor concerns about methodological limitations
Coherence	Minor concerns about coherence, because the data were only reasonably consistent within studies
Relevance	Moderate concerns about relevance, because the data came from a limited range of setting in a single country, and the value staff place on different kinds of feedback, and on reflection in particular, is likely to differ across countries and world regions
Adequacy	Minor concerns about adequacy, because the 3 studies together offer only reasonably rich data
Overall GRADE‐CERQual assessment and explanation
Moderate confidence	Downgraded to moderate confidence because we had no or very minor concerns about methodological limitations, minor concerns about coherence, and adequacy; and moderate concerns about relevance
Contributing studies
Kahn 2019; Khunlertkit 2013; Thomas 2017
Finding 19: Hospital staff highlighted that CCT can support ICUs to overcome challenges associated with staff shortages especially during nights and weekends, and in rural hospitals where ICU nurses are assigned to different departments, and with retaining physicians and nurses. Some concerns over the potential negative impact of CCT on overall staffing levels were also expressed
Assessment for each GRADE‐CERQual component
Methodological limitations	5 studies contributed data to this finding. 1 study discussed researcher reflexivity. 2 studies were assessed as having methodological limitations related to data analysis and collection, of which 1 was also assessed as having methodological limitations related to recruitment; and the other was assessed as having methodological limitations related to research design. Another study was also assessed as having methodological limitations related to research design and recruitment. The body of evidence contributing to this review finding was assessed as having moderate concerns about methodological limitations
Coherence	No or very minor concerns about coherence
Relevance	Minor concerns about relevance, because while issues concerning retention of physicians and nurses are unlikely to differ significantly across world regions, the studies supporting this finding were only from North America
Adequacy	Moderate concerns about adequacy, because the 5 contributing studies together offered only moderately thin data
Overall GRADE‐CERQual assessment and explanation
Moderate confidence	Downgraded to moderate confidence because we had no or very minor concerns about coherence, minor concerns about relevance, and moderate concerns about methodological limitations and adequacy
Contributing studies
Goedken 2017; Hoonakker 2013; Kahn 2019; Shahpori 2011a
Finding 20: Interactions between some bedside and CCT hub staff were featured with tension, frustration and conflict. Staff on both sides commonly described disrespect of expertise, resistance and animosity
Assessment for each GRADE‐CERQual component
Methodological limitations	7 studies contributed to this finding. 1 study discussed researcher reflexivity. 2 studies were assessed as having methodological limitations related to research design, of which 1 was also assessed as having methodological limitations related to data analysis and collection; and the other was assessed as having methodological limitations related to recruitment. Another study was also assessed as having methodological limitations related to recruitment. The body of evidence contributing to this review finding was assessed as having moderate concerns about methodological limitations
Coherence	No or very minor concerns about coherence
Relevance	Minor concerns about relevance, because while issues concerning friction across different teams of health professionals have been noted in research worldwide, the studies supporting the current finding were only from North America
Adequacy	No or very minor concerns about adequacy
Overall GRADE‐CERQual assessment and explanation
High confidence	Graded as high confidence because we had no or very minor concerns about coherence and adequacy, only minor concerns about relevance, and moderate concerns about methodological limitations.
Contributing studies
Hoonakker 2013; Kahn 2019; Khunlertkit 2013; Moeckli 2013; Mullen‐Fortino 2012; Stafford 2008a; Wilkes 2016

Integrating the review findings with Cochrane intervention review

Findings are used to complement and contextualise the conclusions of the Flodgren 2015 Cochrane intervention review on interactive telemedicine by looking at implementation of CCT in particular. Using a narrative approach, we explored how the CCT implementation factors identified in our findings could explain or influence the outcomes of interactive telemedicine as identified by Flodgren 2015. To achieve this, we initially listed our findings alongside outcomes by Flodgren 2015 and considered how these might interact. Then, informed by standard implementation and improvement science approaches (IHI 2012), we grouped our implementation factors under process, structure and balancing measures that could have a role to play in influencing CCT outcomes. Finally, we presented these in a model with a view to informing hypothesis‐testing in future CCT research. At least two review authors worked together on this (AX, KI). The resulting model helps to partly explain the variability seen in the effectiveness of telemedicine in general, and CCT in particular, identifying factors that need to be considered in future trials and informing the development of future CCT interventions and evaluations.

Review author reflexivity

We maintained a reflexive stance throughout the stages of the review process, from study selection to data synthesis. The review authors discussed progress and made decisions critically. As a review team, we all have clinical backgrounds: in nursing (AX, NM, SB, JP, KI), medicine (MT) and midwifery (JS). In addition, four review authors have received advanced training in implementation science (NM, SB, JS, KI) and are well‐versed in relevant theory. NM, AX, MT and JS have been part of a project examining the implementation of CCT at a UK site, but SB, JP and KI are independent of that research. Based on our collective and individual experiences (as clinicians, academics and researchers), we anticipated the findings of our review would reveal a combination of organisational, professional and individual factors influencing the implementation of CCT. Moreover, while the evidence of effectiveness for CCT is uncertain, we believe that CCT could be effective if properly implemented. We have remained mindful of our presuppositions, and supported each other to minimise the risk of these skewing our analysis or the interpretation of our findings. As the lead author, AX kept a reflexive journal throughout the review process in which he documented and reflected on progress and decisions made.

Results

Results of the search

We included 13 studies in this synthesis. These were published between December 2008 and April 2019, and were the only studies that met our inclusion criteria (Figure 1).

1.

Flow diagram

Description of the included studies

Study designs

The included studies used different qualitative approaches for data collection and analysis. Two studies used ethnography (Kahn 2019; Stafford 2008a), two used mixed‐methods research (Moeckli 2013; Ward 2015), four used qualitative interviews (Hoonakker 2013; Khunlertkit 2013; Thomas 2017; Wilkes 2016) and one used a case study design (Hoonakker 2018). One study used a longitudinal qualitative evaluation (Goedken 2017). Three studies (Jahrsdoerfer 2013; Mullen‐Fortino 2012; Shahpori 2011a) used a cross‐sectional questionnaire survey design, but were still included in this synthesis because these also included open‐ended questions with qualitative responses analysed thematically.

Study participants

Participants in the included studies were mainly critical care nurses, physicians, respiratory therapists, managers and administrators working at both bedside and CCT facilities. Three studies reported experiences from nurses (Hoonakker 2013; Hoonakker 2018; Mullen‐Fortino 2012). Hub personnel (physicians, nurses, managers) reported their perspectives in two studies (Khunlertkit 2013; Stafford 2008a). One study reported the perceptions of both bedside and hub personnel (Wilkes 2016). Family members' and significant others' perceptions were reported in one study (Jahrsdoerfer 2013). The rest of the studies reported on a mix of perspectives from bedside ICU physicians and nurses, non‐ICU physicians, medical and surgical physicians, fellows and residents, clinical administrators, respiratory therapists, and IT systems analysts (Goedken 2017; Kahn 2019; Moeckli 2013; Shahpori 2011a; Thomas 2017; Ward 2015). None of the studies reported perspectives from ICU patients.

Setting

Type of hospitals

Twelve studies (Goedken 2017; Hoonakker 2013; Hoonakker 2018; Jahrsdoerfer 2013; Kahn 2019; Khunlertkit 2013; Moeckli 2013; Mullen‐Fortino 2012; Stafford 2008a; Thomas 2017; Ward 2015; Wilkes 2016) were conducted in the USA. One study was conducted in Canada (Shahpori 2011a).

One study was conducted in a metropolitan hospital (Stafford 2008a). One study was conducted in a tertiary hospital (Shahpori 2011a). Three studies were conducted in tertiary‐care medical centres, small urban medical centres and rural hospitals (Goedken 2017; Moeckli 2013; Thomas 2017).

One study was conducted in one academic tertiary referral hospital, and two community hospitals (Mullen‐Fortino 2012). One study was conducted in 10 prospective payment system (PPS) hospitals, and 18 critical access hospitals (CAHs), 13 of which were rural hospitals (Ward 2015).  One study was conducted in three rural affiliated hospitals with a mix of critical access and prospective payment system hospitals (Wilkes 2016). Five studies did not specify the type of hospital (Hoonakker 2013; Hoonakker 2018; Jahrsdoerfer 2013; Kahn 2019; Khunlertkit 2013).

ICU model and staffing

Information about ICU model and staffing was provided in four studies (Moeckli 2013; Mullen‐Fortino 2012; Shahpori 2011a; Thomas 2017). Closed ICU units with physician coverage for evenings, nights, and weekends were described in three studies (Moeckli 2013; Mullen‐Fortino 2012; Shahpori 2011a). Nurse‐to‐patient ratio as 1:1 (75%) or 1:2 (25%) was reported in one study (Shahpori 2011a).

Five studies reported on CCT coverage: from noon to 7 am (Wilkes 2016); two critical care nurses and one intensivist staffed the CCT hub 24 hours and 21 hours a day, respectively (Moeckli 2013; Thomas 2017); CCT offered night‐time intensivist consultation to bedside staff (Mullen‐Fortino 2012); hub nurses worked around the clock, seven days a week, while hub‐intensivists were not included in the staffing matrix for the entire 24 hours of every day (Stafford 2008a).

Four studies reported staffing data about the CCT hubs (Hoonakker 2013; Hoonakker 2018; Khunlertkit 2013; Stafford 2008a). A number of hub nurses (Hoonakker 2013; Khunlertkit 2013; Stafford 2008a) and hub physicians (Khunlertkit 2013) also did ICU shifts working at the bedside. On average, hub nurses monitored four to five ICUs and 38 beds during a shift (Hoonakker 2013); in one study a hub nurse monitored 45 ICU beds (Hoonakker 2018); in another study the CCT hub was responsible for monitoring 180 ICU beds (Stafford 2008a).

CCT system and vendor

Eight studies specified the CCT system and vendor they had in place (Goedken 2017; Khunlertkit 2013; Moeckli 2013; Mullen‐Fortino 2012; Shahpori 2011a; Stafford 2008a; Ward 2015; Wilkes 2016). These were: VISICU, |nc., Baltimore, MD (Goedken 2017; Khunlertkit 2013; Mullen‐Fortino 2012; Stafford 2008a; Wilkes 2016) ; Avera eCare (Ward 2015); Philips IntelliVue and Philips VISICU eCareManager (v3.7.1) (Moeckli 2013); and Quantitative Sentinel, GE Marquette Medical Systems, Milwaukee, WI, USA (Shahpori 2011a).

Methodological limitations of the studies

Most studies used semi‐structured interviews (n = 6) (Goedken 2017; Hoonakker 2013; Khunlertkit 2013; Thomas 2017; Ward 2015; Wilkes 2016), with four also employing ethnographic observations (Hoonakker 2018; Kahn 2019; Moeckli 2013; Stafford 2008a). Three other studies used questionnaire surveys (Jahrsdoerfer 2013; Mullen‐Fortino 2012; Shahpori 2011a), which included open‐ended questions through which to collect qualitative data; these studies generally reported poorly on the contribution of the qualitative data, lacked a detailed description of their thematic analysis process and did not fully consider qualitative research rigour. Across all the included studies, we identified poor reporting on researcher reflexivity. Overall, most studies reported adequately about their recruitment strategy, data collection and analysis process. The methodological limitations of the included studies are presented in detail in Table 2 (Additional Tables).

Confidence in the review findings

Out of  20 review findings, we graded five as high confidence, seven as moderate confidence, and the remaining findings as low confidence, using the CERQual approach (See Table 1). All of the studies were from North America, which was an issue we judged to be of concern for the relevance of several of the findings and have therefore downgraded our CERQual assessment to reflect this concern. Even though ICU practice is a highly standardised field, and CCT is rigidly defined, we could not discount the possibility of variations across world regions. The North American focus of the evidence meant we could not assess any of our findings as having no or very minor concerns for relevance. We judged there to be serious concerns for relevance in those findings more likely to be influenced by social norms and local culture, such as those concerned with staff personal attributes or patient and family attitudes towards health care. The issue led to less serious (minor or moderate) concerns for relevance in those findings concerned with standard aspects of ICU organisation and practice governed by internationally‐agreed guidelines, such as ICU equipment use, team composition and staffing levels, as well as standard features of CCT technology. Our explanation of the CERQual assessment for each review finding is available in the Evidence profiles Table 3; Table 4.

Review findings

We mapped CFIR (Damschroder 2009) against the evidence from the included studies leading to the identification of 20 review findings, under six overall domains, reporting on factors affecting the implementation of CCT. We explore each review finding under these domains in greater depth in the following sections. We point to 'hub' teams to refer to telemedicine staff based remotely in the CCT hub, and 'bedside' teams to refer to those staff based in ICUs providing bedside patient care.

CFIR Domain I: Factors related to intervention (CCT) characteristics

Finding 1: Hospital staff’s personal experience, and anecdotes from colleagues, supported their belief that CCT has positive effects on patient care. Specifically, these effects were about patient safety and quality of care, support at the bedside by critical care experts, and standardisation of practice (moderate confidence).

Hub nurses shared positive experiences, and recalled anecdotes from colleagues, of timely support at the bedside by critical care experts made possible through CCT (Khunlertkit 2013; Shahpori 2011a). Based on such experiences, and even in the absence of other evidence (e.g. from research), administrators and hub clinicians felt confident that CCT could impact positively on their patients’ quality of care (Moeckli 2013; Stafford 2008a). Some administrators and hub clinicians perceived that CCT ensured standardised and up‐to‐date care, which could positively impact patient safety and outcomes (Moeckli 2013; Stafford 2008a). [“Standardized care is going to be huge, just because we are a smaller facility. … Previously we've shared staff among the [ICUs], but having a set way of how we're treating our veterans and caring, I think will help us ensure that we are up‐to‐date on the best practice in how we manage things." ICU administrator] (Moeckli 2013).

Finding 2: Hospital staff and family members described several advantages of CCT. Bedside and hub staff strongly believed that the main advantage of CCT was having access to experts when bedside doctors were not available. Families also valued having access to experts. In addition, hospital staff described how CCT could support clinical decision‐making and mentoring of junior staff (high confidence).

Bedside and hub staff strongly believed that the key advantage of CCT was a timely intervention by CCT experts, when bedside physicians were not available (Kahn 2019; Khunlertkit 2013; Stafford 2008a; Thomas 2017) ["If we weren't here and there were critical decisions that needed to be made, you'd have to wait on a time delay from phone call to physicians, and physicians’ at home making a phone call back", Hub physician] (Khunlertkit 2013). Moreover, both bedside and hub teams perceived that through remote patient monitoring and continuous data collection the CCT hub could offer a kind of backup ["a second set of eyes"] in terms of patient care (Moeckli 2013; Stafford 2008a). Some respiratory therapists saw CCT’s greatest advantage in its ability to provide access to critical care expertise, when this was not available locally (Thomas 2017); and an opportunity for junior physicians to receive mentoring via virtual means (Moeckli 2013). The availability of critical care expertise locally was also valued by patients’ family members (Jahrsdoerfer 2013).

Finding 3: Bedside staff valued the potential adaptability of CCT to speak to local needs and practices. However, this was not always evident, with reported examples being mainly around developing camera usage etiquette and integration with local protocols (low confidence).

Some bedside nurses viewed CCT as having the potential to be implemented effectively within various healthcare contexts, but only if local practices and user needs were taken adequately into consideration (Moeckli 2013). Although the adaptability of CCT was not a feature mentioned by all stakeholders, some bedside nurses did report it facilitated and enabled the integration of local protocols (Thomas 2017). ["some degree of tailoring is important to speak to perceived needs at different ICUs and among different populations of users." Researcher interpretation] (Moeckli 2013). Moreover, even though the presence of the camera caused feelings of discomfort for some bedside teams, hub teams were able to adapt their camera usage etiquette to overcome this (Stafford 2008a).

Finding 4: Both bedside and hub clinicians reported difficulties with the implementation of CCT. Key barriers related to implementation were perceptions of additional workload, need for more coordination work, and concern around the use of cameras (Moderate confidence).

Bedside nurses complained that CCT charged them with additional workload, mainly administrative work, on top of their already busy schedule (Moeckli 2013; Mullen‐Fortino 2012; Ward 2015) [“Frequently what happens is when your patient is the sickest and you have the most to do, you have more to do because (chuckles slightly) you have to explain everything to them (CCT clinicians)." Bedside nurse] (Moeckli 2013). On the other hand, hub nurses described feeling frustrated because patients’ charts available on CCT, written by bedside nurses, provided them with only limited information about patient care (Moeckli 2013). Bedside physicians, nurses and respiratory therapists were unsure about the utility of the bedside cameras (Moeckli 2013; Shahpori 2011a; Stafford 2008a). ["The camera is what scares people most of all, including talking over the camera." Hub nurse](Stafford 2008a).

Finding 5: Cost considerations featured as an influencing factor in a limited way, with only a few examples noting the high cost of implementing CCT, especially compared to the cost of recruiting additional ICU staff (low confidence).

Some bedside nurses and respiratory therapists were concerned that CCT would be used by organisations to make savings on staff costs by reducing the number of ICU staff (Shahpori 2011a). ["A sense of mistrust and the perception that Tele‐ICU might eventually be used as a tool to reduce the number of bedside staff were strongly raised in particular by RNs and RTs." Researcher interpretation] (Shahpori 2011a). Some hub physicians believed that the use of CCT could be a cost‐effective option, although this view was not based on evidence (Stafford 2008a).

CFIR Domain II: Factors related to the outer setting

Finding 6: Hospital staff as well as family members perceived CCT to be providing a community benefit, specifically relating to patients and families' desire to stay close to their local community without requiring transfer to specialist centres to access critical care expertise (moderate confidence).

Bedside clinicians and administrators shared experiences of CCT allowing patients in rural hospitals to receive expert critical care attention in their community, avoiding the need for patient transfer to regional centres (Moeckli 2013; Shahpori 2011a; Ward 2015; Wilkes 2016). [“[A patient] was very concerned that we were going to have to ship him to [the community hospital] and when he and his family found out that they could stay here now because of the [CCT], he just couldn't say enough about, ‘You guys were able to meet my needs and, and [sic] keep me where I was comfortable to get my care.’" CCT administrator] (Moeckli 2013).

CFIR Domain III: Factors related to inner setting

Finding 7: Hospital staff greatly valued the nature and quality of social networks between the bedside and CCT hub teams. Key issues for them were trust, acceptance, teamness, familiarity and effective communication between the two teams (high confidence).

Hub nurses viewed trust and acceptance as a requirement for effective communication with their bedside colleagues (Hoonakker 2013; Hoonakker 2018; Khunlertkit 2013; Moeckli 2013). [“It's uncomfortable if there's a hospital or if there's a nurse that doesn't want anything to do with you and you still have to make contact with that person. You know, that's an awkward time.” Hub nurse] (Hoonakker 2013). On the other hand, bedside nurses vigorously supported the issue of familiarity with their hub co‐workers as important to their communication (Kahn 2019; Moeckli 2013; Mullen‐Fortino 2012). ["My comfort level increases exponentially when I am familiar with the doc in the box." Bedside nurse] (Mullen‐Fortino 2012). Both teams appreciated built‐in relationships to enhance team working (Kahn 2019; Wilkes 2016). ["A clear message that emerged from the interviews (administrators, ICU nurses, and physicians) was that teamwork, communication, and cooperatively established tele‐ICU program standards were core issues that affected the overall success of the tele‐ICU program". Researcher interpretation] (Wilkes 2016).

Finding 8: Hospital bedside staff were concerned over the hub team not being aware of local unit norms, values, and culture. This led local bedside teams to feel that CCT intruded on their practice (moderate confidence).

Bedside physicians and nurses, especially in rural hospitals, felt that hub physicians were not aware of their local culture and practices (Kahn 2019; Moeckli 2013; Mullen‐Fortino 2012; Wilkes 2016). ["Rural clinical staff felt that the hub site staff did not understand rural health issues and culture, and did not know the patients, families, or social context of the illness." Researcher interpretation] (Wilkes 2016). This then meant that intervention by the hub team was not welcomed, which led to the hub team finding the interaction process with bedside staff challenging (Kahn 2019; Moeckli 2013; Mullen‐Fortino 2012; Wilkes 2016). ["The eTeam [CCT] described the interaction process with the bedside team as “walking on eggshells." Researcher interpretation] (Stafford 2008a).

Finding 9: Bedside clinicians were reluctant to use CCT because they lacked clarity about its purpose, were concerned that their decision‐making skills would be weakened through remote supervision, and did not consider hub clinicians an equal counterpart in patient management. Hub clinicians were disengaged due to lack of role clarity and limited integration with patient care (moderate confidence).

Some bedside physicians reported a lack of clarity about the purpose of CCT, which made them disinclined to integrate CCT in their daily practice (Kahn 2019). Others felt that remote supervision would compromise residents’ ability to make decisions (Moeckli 2013) [“You need to make those decisions, and not punt it off to someone else who will make 'em for you.” Bedside resident physician] (Moeckli 2013). On the other hand, hub physicians were perceived as disengaged and distanced from patient care and clinical decision‐making (Kahn 2019; Shahpori 2011a) ["The [CCT] staff are present for multidisciplinary rounds, but they’re just present for it. I mean they’re collecting the data they need to continue to follow the measures that we follow. But they don’t participate. They’re really just observing’’. Bedside nurse] (Kahn 2019).

Finding 10: Hospital locale shaped prioritisation of CCT, with staff in rural centres noting that CCT was of greater benefit to them considering their staff shortage and lack of critical care resources (low confidence).

Some bedside and hub physicians in urban hospitals perceived CCT to be of less utility and value to them because they already had in‐house expertise (Kahn 2019). [“In our [urban] units where [intensivists are] in‐house, I think it’s very secondary and very underutilized.” Hub physician] (Kahn 2019). On the contrary, in rural hospitals, where critical care expertise was scarce, clinicians believed that CCT could help them deal with staff shortages and lack of expertise (Shahpori 2011a). Nevertheless, some clinical staff in rural hospitals assigned less value to CCT because they lacked the appropriate equipment to provide the critical care therapies suggested by the CCT hub (Ward 2015).

Finding 11: Bedside and hub clinicians perceived the absence of support from, and lack of engagement in dialogue with, leaders and senior administrators during the implementation of CCT as major barriers. Listening to staff needs, and creating groundwork connections with them from the outset were perceived as facilitating factors to implementation (low confidence).

Bedside and hub physicians and nurses believed that their senior leaders and administrators should have explained every step of the CCT implementation process more thoroughly, supported and engaged with all the relevant stakeholders early on in the process (Kahn 2019; Wilkes 2016) [“If you don’t have complete and absolute buy‐in from the administration, and willing to back those who are going to be implementing the [CCT] you can’t be as effective." Hub physician] (Kahn 2019). Moreover, they regarded their leadership team listening to staff needs and concerns, and building groundwork connections through meeting and discussing with clinical staff directly from the outset as facilitating implementation (Wilkes 2016). ["At first the nurses were against it. There was too much documentation and they did not like someone telling us what to do. After we explained how an eICU works and [hub site administrator] came up here for a staff meeting—she has a dynamic personality—it was fine. Having [hub site administrator] meet the nurses created a connection that was very helpful." Researcher field notes] (Wilkes 2016).

Finding 12: Hospital staff thought it was important to promote and offer training in the use of CCT before its implementation. This included rehearsing every step in the process, offering staff opportunities to ask questions and disseminating learning resources. Some also complained that experienced staff were taken away from bedside care and re‐allocated to the CCT hub team (high confidence).

Bedside and hub physicians, nurses and administrators fully supported training on using CCT through Q&A sessions and on‐hand demonstration prior to implementation (Kahn 2019; Moeckli 2013) [“Before going live, get in and interact with that staff. Dial into the hub with the camera without a patient, go through introductions, just like you would introduce a new staff member… Let them ask the questions that they would ask you. Because to me, it's a better learning experience for the staff if they get to ask the questions versus going through me.” Hub co‐ordinator] (Moeckli 2013). Some hospital staff expressed concerns that taking experienced nurses from the ICU bedside to staff the CCT hub was not the optimal and most cost‐effective way to allocate resources (Shahpori 2011a; Stafford 2008a). ["When nurses were recruited to work on the (CCT) it left some units short‐staffed. One of the (hub physicians) recalled that some felt that the lCUs were being 'raided' of their most experienced and best ICU nurses to staff the (CCT) department." Researcher field notes] (Stafford 2008a).

Finding 13: Hospital staff reported the lack of access to information about how CCT staff, policies and procedures can be incorporated into the bedside workflow as a barrier to implementation (low confidence).

Bedside and hub physicians, administrators and CCT coordinators felt inadequately informed about how CCT hub staff, policies and procedures can be incorporated into the bedside workflow. [“You know, that was never really expressed to us, who and what actually we should be doing, for the formal training (of staff).” CCT coordinator. "I just kind of kept expecting to see something … saying, ‘This is how we're gonna operate, here's how our policies and procedures are going to work.’” Administrator] (Moeckli 2013).

CFIR Domain IV: Factors affecting implementation related to characteristics of individuals

Finding 14: Hospital staff's attitudes towards, knowledge about and value placed on CCT influenced acceptance of CCT. Staff were positive towards CCT because of its several advantages. But some were concerned that the hub staff were not able to understand the patient’s situation through the camera. Some were also concerned about confidentiality of patient data (high confidence).

Some bedside staff acknowledged that CCT was great support for patients at risk since intensivists at the hub were always available; ["I love telemedicine…. I feel they do an excellent job and are always available to lend a second opinion." Bedside nurse] (Mullen‐Fortino 2012). In contrast, others believed that hub physicians were not able to understand the patient situation because they could only see patients remotely through the camera (Kahn 2019; Moeckli 2013; Mullen‐Fortino 2012; Shahpori 2011a; Thomas 2017). [“I’ve seen the [CCT] get involved looking at stuff through a vacuum, not really understanding the patient’s situation or clinically what the patient looks like.” Bedside clinician] (Kahn 2019). Some bedside staff were concerned whether CCT systems endanger patient confidentiality, while others celebrated the ability to hold virtual meetings with patients' relatives through CCT (Shahpori 2011a). Hospital staff valued CCT’s contribution to educating and supporting junior physicians and nurses in real‐time (Khunlertkit 2013; Shahpori 2011a; Stafford 2008a). ["Donna expressed that one of her greatest satisfactions as a (CCT nurse) is when she can support a new ICU (bedside) nurse who would normally have to rely on access to the ICU charge nurse for guidance." Researcher field notes] (Stafford 2008a).

Finding 15: Hospital staff noted that acceptance and normalisation of CCT in their daily work took time; progressing through different stages of change did not occur with the same pace for everyone, with some remaining resistant to change (low confidence).

Some bedside staff went through a transition period before CCT was fully accepted and embedded into their daily practice. ["It’s [CCT] been embedded. So I think it works because over time the bedside folks have accepted this as another tool to the care of the patient and not as an intrusion which, yes, it was seen that way in the beginning...” CCT facility director] (Kahn 2019). For others, change of attitude towards CCT took longer with some remaining resistant to recommendations by hub physicians about patient care (Khunlertkit 2013).

Finding 16: Hub nurses’ personal attributes, specifically with regard to their motivation, multitasking competence and values were noted as important enablers for implementation of CCT (low confidence).

Some hub nurses identified different sources of satisfaction arising from them working in the CCT hub. These included professional development and advancement opportunities, helping staff and hospitals with less critical care expertise, interacting with and supporting staff in other hospitals, and learning to use innovative technology. [“So I think it is the wave of the future, and I wanted to be involved in that.” “I like, when you have good interaction with the other hospitals, you know, younger, less experienced nurses, where you feel that you can mentor and help, it was very satisfying.” Hub nurses] (Hoonakker 2013). Others enjoyed the challenge of learning a new system, especially as CCT needed skills in multitasking, prioritising and triage. [“Well, what I like about it is that it's a constant challenge. It's a challenge of just dealing with lots of different systems, lots of different people. You're having to triage a lot of things in your mind and, you know, multitask, and, you know, prioritize and triage”. Hub nurse] (Hoonakker 2013).

CFIR Domain V: Factors affecting implementation related to process

Finding 17: Hospital staff were frustrated due to lacking a clear strategy for engagement; specifically lack of consistent training, the orientation of new and resistant staff to the hub facility, and timely co‐ordination for CCT implementation (low confidence).

Lacking an adequate engagement strategy was a source of great frustration for some hospital staff (Kahn 2019; Moeckli 2013). ["I don't think we've quite worked out, what I think are satisfactory—, … parameters or rules of engagement for how we do things so [we're] still too standoffish in terms of what we're supposed to do versus what [bedside staff are] doing at the hospitals.” Hub physician] (Moeckli 2013). Some hub physicians argued that consistent introduction and education of new staff, and those with negative attitudes, at the hub facility, was missing and this would be effective in fostering acceptance of CCT. ["Having people come here and see it makes a big difference, and I always say, Bring the negatives. Those people that are resistant, bring them!” “I work pretty closely with the nurse educator up there, and I say, ‘If you have a new staff member, give me fifteen minutes with them.’" Hub clinicians] (Kahn 2019). Some CCT managers gave the initiative to bedside physicians to decide about their level of expertise and involvement with CCT, while in other hospitals managers decided on clinicians' behalf. ["The rollout was, ‘We’re going to consider that you all want to be (involved at) the lowest level unless you come and tell us you want to be a higher level.’” “In [another hospital] the rollout was, ‘Here, Doctor. Here’s your sheet of your paper. Fill out your level, and return it to us.’” CCT facility manager] (Kahn 2019). Some hub physicians and bedside nurses were concerned with the lack of co‐ordination during the implementation of CCT (Moeckli 2013). ["It's like a week before the start of the construction contract, and nobody even knew if the stuff was here, and, you know, some of that kind of stuff [is] where a coordinator would have been handy.” Bedside nurse] (Moeckli 2013).

Finding 18: Hospital staff were encouraged by the visibility of the intended benefits of CCT. They valued both quantitative feedback through auditing, as well as qualitative feedback through reflective accounts (moderate confidence).

Some bedside nurses and hub staff reported that over the course of CCT implementation they noticed improvements in patient outcomes, reduction in rates of medical errors, improved compliance with guidelines, and reduction in the number of healthcare‐associated infections; these were mainly evident through clinical audits (Kahn 2019; Khunlertkit 2013). ["We could see all guideline compliance and we have time to track it, we went from 83% to 98‐99%, and our number of pneumonias went way down…" CCT manager] (Khunlertkit 2013). Moreover, staff reflected on their experience of working with the CCT hub and noted benefits to patient care. [‘‘On a busy night, I like that I can call [the CCT hub] and say, ‘Can you just keep an eye on everybody and round through just to make sure everybody is okay?’ And I did see that one night [the CCT hub] caught my patient about to self‐extubate.’’ Bedside nurse] (Thomas 2017).

Other factors affecting implementation

Finding 19: Hospital staff highlighted that CCT can support ICUs to overcome challenges associated with staff shortages especially during nights and weekends, and in rural hospitals where ICU nurses are assigned to different departments; and with retaining physicians and nurses. Some concerns over the potential negative impact of CCT on overall staffing levels were also expressed (moderate confidence).

Some bedside and hub nurses pointed out that CCT was a mechanism to provide cover to ICUs when there was staff shortage (Goedken 2017; Hoonakker 2013; Kahn 2019; Ward 2015 ). [“[During nights and weekends], our providers are stretched pretty thin. That’s why at our facility, we’re really excited about the (CCT) project because it’ll give us more assistance [after hours]. It’s hard to recruit to where we can have enough providers in‐house.” Bedside nurse] (Goedken 2017). However, others suggested that additional staff should not be used only to support the CCT facility but be deployed flexibly at the bedside as well (Shahpori 2011a).

Finding 20: Interactions between some bedside and CCT hub staff were featured with tension, frustration and conflict. Staff on both sides commonly described disrespect of expertise, resistance and animosity (high confidence).

Reports of tension, conflict and animosity between the bedside and hub teams hindered implementation of CCT. ["If they (bedside nurses) used us (CCT facility) the physicians would come in the next morning and chastise them to the point that the nurses were scared to use us." CCT director. "You get this kind of like animosity towards them because they’re really taking (ICU nurses) away from the bedside." Bedside nurse] (Kahn 2019). Some bedside nurses felt frustrated that some hub physicians sidelined them in patients’ care planning and decision making, which they interpreted as disrespecting their expertise; ["The doc communicated via cell phone with the on‐site physician and I didn’t know anything that was said, and I felt uneasy as a result because I didn’t know what the plan was." Bedside nurse] (Mullen‐Fortino 2012). In contrast, some hub nurses felt satisfied with the team spirit that existed between them and hub physicians; ["...when I work with a physician here (CCT), they have a different level of respect for us, and I think it sort of just transcends to nursing in general. We've never worked like this with a physician as a team this closely, so I think that that's made a difference at the bedside as well, and I think it's a different level of acceptance for nursing, actually." Hub nurse] (Hoonakker 2013). Moreover, some hub nurses believed that their bedside colleagues were annoyed with them due to a perception they were more focused on watching the bedside staff rather than the patients (Moeckli 2013; Stafford 2008a) ["...(Bedside) nurses feel like they know what they’re doing and they don’t need somebody watching them. What they don't realize is that we’re not really watching them, we're watching the patient." Hub nurse (Stafford 2008a)]; [“I think I still feel that somebody's looking over my shoulder…” Bedside nurse] (Moeckli 2013). Some bedside staff, especially in rural areas, felt unnecessarily criticised by hub staff, while some hub staff felt frustrated with the rural bedside teams for not engaging with CCT (Moeckli 2013; Wilkes 2016). ["The interaction between doctors at the rural site and doctors at the hub was immediately strained and hostile. The rural site physicians stopped using (CCT) in retaliation to the God‐like orders and rude treatment from the hub." Researcher field notes] (Wilkes 2016).

Results of integrating the review findings with the Cochrane intervention review

We explored the extent to which our review findings help explain the conclusions of the Flodgren 2015 review on interactive telemedicine, with a view to identifying avenues for future research and review updates. We concluded that the interventions considered within Flodgren 2015 were highly variable and too dissimilar from the focus of this synthesis. CCT is distinct as an application from the traditional models of telemedicine since it uses a hub‐and‐spoke model to provide a continuous form of clinical support to bedside practice 24/7; rather than just being an interface for sporadic communication between patients and providers. Interventions considered in Flodgren 2015 mostly used just one of the CCT features (e.g. laboratory and diagnostics management, or continuous electronic recording of vital signs, or display of real‐time data, or provider‐based decision support, or provider‐to‐provider video conferencing), but not a combination of all of the features. Most interventions in Flodgren 2015 used provider‐to‐patient video conferencing, with some supplementing of this with remote access to patient data. Despite differences in the nature of telemedicine interventions, the geographic location of included studies in both this review and Flodgren 2015 originated mainly in high‐income countries. This points to a significant gap in knowledge of implementation, use and effectiveness of telemedicine in lower‐ and middle‐income settings.

The main outcomes examined in Flodgren 2015 were: patient mortality, adverse events, healthcare resource use and cost. While certainty in the evidence for these outcomes varied, a common feature was heterogeneity in the meta‐analysis, with variability across study results. We argue that a key issue hindering the investigation of this variability lies in the limited consideration and description of the implementation stage, as well as the different nature of the various telemedicine interventions. Furthermore, the opportunity for sensitivity analysis is limited because a consistent programme theory about the implementation and effectiveness of telemedicine is missing from previous reviews. Following a narrative approach, as described earlier (Methods), we grouped our findings under key process, structure and balancing measures of CCT implementation which could be examined in combination with patient outcomes in future telemedicine research and review updates. While outcome measures reflect the impact of CCT on patients, process measures reflect the way CCT works to deliver such impact. In addition, structure measures are important in identifying significant attributes of the wider service in which CCT is implemented; and balancing measures reflect unanticipated consequences of CCT that can influence outcomes either positively or negatively. With a view to informing future development of, and research into, telemedicine, we propose the following model based on our review findings:

• Outcome measures: we propose service use, as well as patient and family satisfaction with care as key outcome measures for consideration in future telemedicine research and reviews, in addition to conventional outcomes such as mortality and adverse events. We also recommend more systematic documentation of costs incurred to include recruitment or redeployment of staff, as well as staff training costs (Findings 1, 2, 5, 6);

• Process measures: we identify the provision of staff training and information/education resources as important process measures for effective implementation. Staff engagement through consultation meetings prior to implementation and regular feedback from audits on outcomes and adherence to evidence‐based guidelines should also be considered. Failure‐to‐rescue indicators, including early identification of deterioration, escalation of care, time to consultation and remedial action may act as moderators to patient outcomes (Findings 4, 12, 13, 14, 17, 18);

• Structure measures: organisational and unit/clinic culture, especially for supportive leadership and collaborative approaches are further suggested as important structure measures to consider. Evidence of adjustments to local practices, protocols and workflows to accommodate and integrate telemedicine in daily practice could also be sought. Investment in human resources, by numbers as well as skill‐mix, also appears critical (Findings 3, 8, 10, 11, 12, 16, 19);

• Balancing measures: aspects of teamness, including communication, professional respect, autonomy and role clarity may be inadvertently affected through the introduction of telemedicine, and should not be overlooked in future studies (Findings 7, 9, 15, 20).

Structure measures	Process measures	Balancing measures	Outcome measures
Culture Leadership Integration Staffing levels Skill mix Workload  IT support	Failure to rescue Adherence to evidence‐based guidelines Staff training & availability of information Staff engagement Audit & feedback	Teamness Communication Respect Autonomy Role clarity	Mortality Adverse events Cost Service utilisation Patient satisfaction Family satisfaction

Review author reflexivity

We describe our initial positioning earlier (see Methods, Review author reflexivity). While our views did not shift significantly in the process of developing the synthesis, the final stages were completed during the COVID‐19 pandemic which inevitably coloured our experience. This relates specifically to the two review authors who worked closely on developing the initial review conclusions and implications for practice (AX, KI). The pandemic shone a bright light on the added value of telemedicine in health care generally, and critical care in particular, with reports of CCT enabling clinical teams to respond better at the height of the pandemic. While our review findings were already decided by this time, showing both advantages and pitfalls with implementing CCT, it would be remiss of us to not acknowledge a sense of pressure to deliver not only evidence‐based but also practical implications for policymakers and other stakeholders. During this stage, the two review authors held regular meetings (online) to discuss progress and reflect on the extent to which the pandemic may have influenced our views about CCT, and in turn our review implications and conclusions. A third review author (JP) checked our conclusions alongside the evidence to confirm a close link, before consulting with other members of the wider review team. By remaining close to the evidence from our included studies, and reflecting continuously throughout the process, we remain confident our conclusions and implications present an honest account of the state of the evidence for factors affecting the implementation of CCT.

Discussion

Summary of the main findings

For a summary of the main findings, please see the Plain Language Summary.

Comparison with other reviews and implications for the field

Past reviews on the effectiveness of CCT reported great variability and uncertainty, with their included studies providing a limited understanding of the contextual factors contributing to this (Wilcox 2012). Attention to contextual features has also been highlighted as important in two other Cochrane Reviews of telemedicine and eHealth (Flodgren 2015; Ross 2016). Findings from this qualitative evidence synthesis provide a more complete understanding of the contextual and process factors affecting implementation, potentially moderating the effectiveness of CCT. Evidence for this review was derived from different stakeholder (providers, administrators, service users) experiences, perceptions and values about the factors enabling or hindering successful implementation. 

Our synthesis drew from the CFIR framework to theoretically inform data synthesis, but not to restrict it, thus adding to the literature about the implementation of telemedicine. We also identified two additional themes (see Findings 19 and 20) not classified under the CFIR domains. Consequently, factors including structure, process and balancing measures have been identified and synthesised in a model (see section Results of integrating the review findings) to aid future research and review updates on telemedicine in different contexts. 

Our work complements the Flodgren 2015 effectiveness review on interactive telemedicine, which considered the acceptability of telemedicine by patients and healthcare professionals. Evidence of acceptability in that review was limited, mainly identifying costs and difficulties with operating the technology. The included studies in Flodgren 2015 recruited heterogeneous groups and interventions mainly in primary‐care settings, which hindered issuing of specific guidance. Our review extends this work by identifying insights on perceived factors that influence implementation (including acceptance, adoption and use) of an advanced form of telemedicine by a range of stakeholders and within a well‐defined patient population.

Finally, our findings complement and reinforce findings of the qualitative synthesis by Odendaal 2020, which looked into health workers’ perceptions and experiences of using mobile health (mHealth) technologies to deliver primary healthcare services. In both syntheses, CCT and mHealth were perceived by healthcare workers to influence traditional ways of working, either positively or negatively. Negative perceptions in both syntheses related to features of the technology that increased the workload, while positive perceptions related to the potential to improve quality of care. The impact of technology on provider‐to‐provider collaboration was another shared finding. Collaboration was strengthened through improved and faster peer feedback and expert advice, but hindered when technology was seen as interfering with professional autonomy. While Odendaal 2020 focused on ongoing use of mHealth, our synthesis additionally captured factors that influenced acceptance and adoption of CCT, such as staff training and engagement, leadership and teamness.

Overall completeness and applicability of the evidence

Most of our included studies examined the perspectives and experiences of bedside and hub hospital staff about factors affecting the implementation of CCT, with only one study including the perceptions of patients’ families and significant others. The importance of family involvement has been gaining attention over recent years (Bench 2020; Xyrichis 2019), especially so during the COVID‐19 pandemic when physical family visits were suspended (Ning 2020). The inclusion of family members and/or significant others’ perspectives in research could have added an important and additional layer of insight about family‐centred care in the ICU. Such inclusion could be achieved through using approaches such as Design Research Methodology (DRM, Blessing 2009) and Participatory Action Research (PAR, Kindon 2007). Moreover, understanding the perspectives of families could strengthen policymakers’ decision‐making about further integration of CCT across settings.

Twelve out of the 13 studies included in this qualitative synthesis were completed in the USA, and one was conducted in Canada. Research in middle‐ and lower‐income countries is urgently needed. Especially in the context of the COVID‐19 outbreak, declared as a pandemic in more than 203 countries, telemedicine implementation and integration within health systems across the higher‐, middle‐ and lower‐income countries is quickly becoming a global imperative. Further qualitative research to understand the experiences and perspectives of relevant stakeholders in different contexts, with different norms, practices and values is needed to inform future implementation, use and development of CCT.

Issues of health‐system infrastructure, workforce capacity, and costs are not fully addressed in the available literature, but are arguably important for those considering investment in CCT. Telemedicine in general has been identified as influenced by such issues as payment and regulatory structures, state licensing and credentialising across hospitals (Hollander 2020). Based on our included studies we are unable to confirm the extent to which such issues also hinder implementation of CCT in particular, which limits our ability to make suggestions about overcoming these.

The included studies did not distinguish results between pre‐, peri‐ and post‐implementation of CCT. Even though some studies did include data collection at different implementation stages, their analysis and reporting conflated their results. While it is conceivable that different factors may have more relevance at different stages of implementation, we are currently unable to distinguish these from the available body of evidence.

Lastly, we have low confidence in the evidence contributing to eight out of the 20 review findings. We downgraded our confidence in these findings mainly for a lack of adequate data (limited richness within studies and breadth across studies) supporting each review finding, as well as for concerns over relevance, given the North American focus of the dataset.
 

Limitations of the review

We only included studies published in the English language, although no language limit was imposed during the electronic database search. Nevertheless, we cannot discount the possibility that we may have missed relevant studies written and published in non‐English‐language journals. The included studies were all from North America (the USA and Canada), which is an issue dealt with in the GRADE‐CERQual assessment. We opted for the CFIR (Damschroder 2009) as our implementation framework to guide our data synthesis, which provided structure and transparency to our approach, although limiting the potential for the chance discovery of themes arguably amplified when using a purely inductive approach. We are confident we have not missed any significant themes, given that two additional factors influencing implementation of the CCT, not adequately captured by CFIR, were identified and included in our findings. Moreover, a limitation of using CIFR is that it is a rather linear and reductionist framework, which hinders pulling out relationships and trade‐offs between its components. Our approach adopted an implementation lens, which shaped the way in which we approached the literature. Adopting a different lens and theoretical stance, arguably, may have led us to emphasise different findings. 

Authors' conclusions

Implications for practice

Below is a set of questions drawn from the findings of this synthesis that may be helpful to health system, programme managers and other ICU stakeholders when planning and managing the implementation of CCT. These questions were drawn from the review findings in which we had high or moderate confidence. 

Patient safety, quality of care and confidentiality

• Have you considered whether it might be useful to monitor the progress and impact of CCT, for instance through both quantitative (e.g. audits) and qualitative (e.g. reflective accounts) approaches?

• Can you assure patient privacy and confidentiality in the context of CCT?

• Have you thought about how to integrate CCT tools (e.g. decision support) in ICU clinicians' daily practice? 

• Can you tailor CCT to the needs of different ICUs, specifically in relation to local protocols and practices?

Training and mentoring of users 

• Is there a detailed, all‐steps‐included, hands‐on training programme on CCT use for all relevant stakeholders?

• Have you thought about how you will raise awareness and encourage uptake of training resources among staff?

• Can you include a mentoring component for junior ICU bedside staff, linking them with experienced staff in the CCT hub?

• Do the ICUs actively encourage bedside teams to seek and share feedback from and with their hub colleagues?

Raising awareness

• What strategies are in place to raise awareness among clinical staff about the strengths and challenges of using CCT, before it is implemented?

• Have you informed staff about the potential advantages of CCT, for patient safety, quality of care and family satisfaction?

• How are family members informed about the strengths and challenges of CCT, for example, the potential it offers for patients in rural communities to avoid transfer to regional centres?

Building teamness

• Have you considered how you can encourage teamness, trust, communication, familiarisation and collaboration between hub teams and local bedside teams?

• Have you clarified the  purpose of CCT to both bedside and hub teams? 

• Have you identified the distinct roles and workflows of bedside and hub teams, and have you communicated these to them?

• Have you considered how hub clinicians can participate equally and engage with their bedside colleagues during ICU ward rounds?

Camera usage etiquette

• Have you consulted with bedside and hub teams to develop an acceptable camera usage etiquette?

• Have you discussed with the bedside team the presence of the camera, and how it can be used to help them in their daily work (e.g. by being a second pair of eyes, watching restless patients)?

Sustainability and ongoing usage

• Have you ensured ongoing maintenance of the equipment?

• Can you ensure 24/7 availability of IT support to bedside and hub teams.

Resource allocation

• Have you identified optimal and safe staffing levels for the CCT hub facilities?

• Bedside nurses may feel concerned that their experienced colleagues are taken to staff the CCT hub. Have you considered what measures can be taken to ensure staffing the CCT hub does not negatively influence staffing levels at the bedside?

• Have you considered ways of minimising the potential for the additional workload on bedside teams?

• Have you considered offering ICU nurses and physicians the opportunity to work across bedside and hub teams, to strengthen knowledge‐sharing and skill development?

Implications for future research

Future research on the implementation and impact of CCT should also consider, and be designed to examine the questions noted in Implications for practice. Additional implications for research have been identified based on the overview of 13 studies included in this qualitative synthesis, and our GRADE‐CERQual assessments of 20 review findings.

More detailed reporting is needed in qualitative studies of CCT implementation, especially around researcher reflexivity, sampling and data analysis methods. Future qualitative studies on this topic should transparently report their research methods, including on the researchers’ roles and how these may influence the conduct and results of the study.

Further research is needed on implementing CCT outside North America, especially in lower‐ and middle‐income countries, to better understand how different norms, cultural practices and health infrastructure may foster or hinder acceptance of CCT.

Research on CCT that includes the perspectives and experiences of family members, and significant others, is missing and urgently needed. Understanding how families perceive the role and usefulness of CCT towards the care of their loved ones, especially in times of pandemics and hospital‐visiting restrictions, would provide valuable evidence about the value, contribution and acceptance of CCT among all relevant stakeholders towards meeting the goal of family‐centred care.

Further qualitative research is needed on this review topic to strengthen the body of evidence contributing to the review findings, since lack of adequate data was a key reason for downgrading our confidence in many of the review findings.

History

Protocol first published: Issue 11, 2017
Review first published: Issue 2, 2021

Appendices

Appendix 1. CFIR Table of Constructs

Topic/Description	Short Description
I. INTERVENTION CHARACTERISTICS
A ‐ Intervention Source	Perception of key stakeholders about whether the intervention is externally or internally developed.
B ‐ Evidence Strength & Quality	Stakeholders’ perceptions of the quality and validity of evidence supporting the belief that the intervention will have desired outcomes.
C ‐ Relative advantage	Stakeholders’ perception of the advantage of implementing the intervention versus an alternative solution.
D ‐ Adaptability	The degree to which an intervention can be adapted, tailored, refined, or reinvented to meet local needs.
E ‐ Trialability	The ability to test the intervention on a small scale in the organization, and to be able to reverse course (undo implementation) if warranted.
F ‐ Complexity	Perceived difficulty of implementation, reflected by duration, scope, radicalness, disruptiveness, centrality, and intricacy and number of steps required to implement.
G ‐ Design Quality and Packaging	Perceived excellence in how the intervention is bundled, presented, and assembled.
H ‐ Cost	Costs of the intervention and costs associated with implementing that intervention including investment, supply, and opportunity costs.
II. OUTER SETTING
A ‐ Patient Needs & Resources	The extent to which patient needs, as well as barriers and facilitators to meet those needs are accurately known and prioritized by the organization.
B ‐ Cosmopolitanism	The degree to which an organization is networked with other external organizations.
C ‐ Peer Pressure	Mimetic or competitive pressure to implement an intervention; typically because most or other key peer or competing organizations have already implemented or in a bid for a competitive edge.
D ‐ External Policy & Incentives	A broad construct that includes external strategies to spread interventions including policy and regulations (governmental or other central entity), external mandates, recommendations and guidelines, pay‐for‐performance, collaboratives, and public or benchmark reporting.
III. INNER SETTING
A ‐ Structural Characteristics	The social architecture, age, maturity, and size of an organization.
B ‐ Networks & Communications	The nature and quality of webs of social networks and the nature and quality of formal and informal communications within an organization.
C ‐ Culture	Norms, values, and basic assumptions of a given organization.
D ‐ Implementation Climate	The absorptive capacity for change, shared receptivity of involved individuals to an intervention and the extent to which use of that intervention will be rewarded, supported, and expected within their organization.
1 ‐ Tension for Change	The degree to which stakeholders perceive the current situation as intolerable or needing change.
2 ‐ Compatibility	The degree of tangible fit between meaning and values attached to the intervention by involved individuals, how those align with individuals’ own norms, values, and perceived risks and needs, and how the intervention fits with existing workflows and systems.
3 ‐ Relative Priority	Individuals’ shared perception of the importance of the implementation within the organization.
4 ‐ Organizational Incentives & Rewards	Extrinsic incentives such as goal‐sharing awards, performance reviews, promotions, and raises in salary and less tangible incentives such as increased stature or respect.
5 ‐ Goals and Feedback	The degree to which goals are clearly communicated, acted upon, and fed back to staff and alignment of that feedback with goals.
6 ‐ Learning Climate	A climate in which: a) leaders express their own fallibility and need for team members’ assistance and input; b) team members feel that they are essential, valued, and knowledgeable partners in the change process; c) individuals feel psychologically safe to try new methods; and d) there is sufficient time and space for reflective thinking and evaluation.
E ‐ Readiness for Implementation	Tangible and immediate indicators of organizational commitment to its decision to implement an intervention.
1 ‐ Leadership Engagement	Commitment, involvement, and accountability of leaders and managers with the implementation.
2 ‐ Available Resources	The level of resources dedicated for implementation and on‐going operations including money, training, education, physical space, and time.
3 ‐ Access to knowledge and information	Ease of access to digestible information and knowledge about the intervention and how to incorporate it into work tasks.
IV. CHARACTERISTICS OF INDIVIDUALS
A ‐ Knowledge & Beliefs about the Intervention	Individuals’ attitudes toward and value placed on the intervention as well as familiarity with facts, truths, and principles related to the intervention.
B ‐ Self‐efficacy	Individual belief in their own capabilities to execute courses of action to achieve implementation goals.
C ‐ Individual Stage of Change	Characterization of the phase an individual is in, as he or she progresses toward skilled, enthusiastic, and sustained use of the intervention.
D ‐ Individual Identification with Organization	A broad construct related to how individuals perceive the organization and their relationship and degree of commitment with that organization.
E ‐ Other Personal Attributes	A broad construct to include other personal traits such as tolerance of ambiguity, intellectual ability, motivation, values, competence, capacity, and learning style.
V. PROCESS
A ‐ Planning	The degree to which a scheme or method of behavior and tasks for implementing an intervention are developed in advance and the quality of those schemes or methods.
B ‐ Engaging	Attracting and involving appropriate individuals in the implementation and use of the intervention through a combined strategy of social marketing, education, role modeling, training, and other similar activities.
1 ‐ Opinion Leaders	Individuals in an organization who have formal or informal influence on the attitudes and beliefs of their colleagues with respect to implementing the intervention.
2 ‐ Formally appointed internal implementation leaders	Individuals from within the organization who have been formally appointed with responsibility for implementing an intervention as coordinator, project manager, team leader, or other similar role.
3 ‐ Champions	Individuals who dedicate themselves to supporting, marketing, and ‘driving through’ an implementation, overcoming indifference or resistance that the intervention may provoke in an organization.
4 ‐ External Change Agents	Individuals who are affiliated with an outside entity who formally influence or facilitate intervention decisions in a desirable direction.
C ‐ Executing	Carrying out or accomplishing the implementation according to plan.
D ‐ Reflecting & Evaluating	Quantitative and qualitative feedback about the progress and quality of implementation accompanied with regular personal and team debriefing about progress and experience.

Appendix 2. Search strategies

Database: Ovid Medline 1946 to October Week 3 2019
#	Searches
1	exp telemedicine/
2	exp telenursing/
3	telemedicine.ti,ab,kw.
4	tele medicine.mp.
5	telehealth.mp.
6	tele health.mp.
7	telecare.mp.
8	tele care.mp.
9	telemonitoring.mp.
10	tele monitoring.mp.
11	medical informatics applications/
12	medical informatics application?.mp.
13	electronic health.mp.
14	electronic care.mp.
15	ehealth.mp.
16	e health.mp.
17	((patient? adj1 monitor*) and (device? or tele* or electronic*)).ti,ab,kw.
18	decision support.mp.
19	electronic alert?.mp.
20	(hub and spoke?).mp.
21	(remote support or remote surveillance or remote monitoring or remote counseling or remote counselling).ti,ab,kw.
22	Computer Communication Networks/
23	Telecommunications/
24	or/1‐23
25	exp Intensive Care Units/
26	Critical care/
27	Intensive Care, Neonatal/
28	critical care.mp.
29	intensive care.mp.
30	intensive therap*.mp.
31	intensive treatment?.mp.
32	ICU?.mp.
33	ITU?.mp.
34	(critical* ill* or sever* ill* or serious* ill* or at risk patient? or trauma patient?).ti,ab,kw.
35	or/25‐34
36	24 and 35
37	eICU?.mp.
38	tele ICU?.mp.
39	tele IC?.mp.
40	teleIC?.mp.
41	or/37‐40
42	36 or 41
43	Qualitative Research/
44	interview:.mp.
45	experience:.mp.
46	qualitative.tw.
47	or/43‐46
48	42 and 47

Database: Embase 1974 to October Week 3 2019
#	Searches
1	exp telemedicine/
2	exp telenursing/
3	telemedicine.ti,ab,kw.
4	tele medicine.mp.
5	telehealth.mp.
6	tele health.mp.
7	telecare.mp.
8	tele care.mp.
9	telemonitoring.mp.
10	tele monitoring.mp.
11	medical informatics applications/
12	medical informatics application?.mp.
13	electronic health.mp.
14	electronic care.mp.
15	ehealth.mp.
16	e health.mp.
17	((patient? adj1 monitor*) and (device? or tele* or electronic*)).ti,ab,kw.
18	decision support.mp.
19	electronic alert?.mp.
20	(hub and spoke?).mp.
21	(remote support or remote surveillance or remote monitoring or remote counseling or remote counselling).ti,ab,kw.
22	Computer Communication Networks/
23	Telecommunications/
24	or/1‐23
25	exp Intensive Care Units/
26	Critical care/
27	Intensive Care, Neonatal/
28	critical care.mp.
29	intensive care.mp.
30	intensive therap*.mp.
31	intensive treatment?.mp.
32	ICU?.mp.
33	ITU?.mp.
34	(critical* ill* or sever* ill* or serious* ill* or at risk patient? or trauma patient?).ti,ab,kw.
35	or/25‐34
36	24 and 35
37	eICU?.mp.
38	tele ICU?.mp.
39	tele IC?.mp.
40	teleIC?.mp.
41	or/37‐40
42	36 or 41
43	Qualitative Research/
44	interview:.mp.
45	experience:.mp.
46	qualitative.tw.
47	or/43‐46
48	42 and 47

Database: CINAHL 1937 to October Week 3 2019
#	Searches
S1	(MH "Telemedicine+")
S2	(MH "Telenursing")
S3	telemedicine
S4	telemedicine
S5	telehealth
S6	tele health
S7	telecare
S8	telecare
S9	telemonitoring
S10	telemonitoring
S11	(MH "Medical Informatics")
S12	medical informatics applications
S13	electronic health
S14	electronic cars
S15	ehealth
S16	e health
S17	((patient? n1 monitor*) and (device? or tele* or electronic*))
S18	decision support
S19	electronic alerts
S20	(hub and spoke?)
S21	(remote support or remote surveillance or remote monitoring or remote counseling or remote counselling)
S22	(MH "Computer Communication Networks")
S23	(MH "Telecommunications")
S24	S1 OR S2 OR S3 OR S4 OR S5 OR S6 OR S7 OR S8 OR S9 OR S10 OR S11 OR S12 OR S13 OR S14 OR S15 OR S16 OR S17 OR S18 OR S19 OR S20 OR S21 OR S22 OR S23
S25	(MH "Intensive Care Units+")
S26	(MH "Critical Care")
S27	(MH "Intensive Care, Neonatal")
S28	critical care
S29	intensive care
S30	intensive therap*
S31	intensive treatment
S32	icu
S33	itu
S34	TI (critical* ill* or sever* ill* or serious* ill* or at risk patient or trauma patient)
S35	AB (critical* ill* or sever* ill* or serious* ill* or at risk patient or trauma patient)
S36	S25 OR S26 OR S27 OR S28 OR S29 OR S30 OR S31 OR S32 OR S33 OR S34 OR S35
S37	S24 AND S36
S38	eICU
S39	tele‐icu
S40	S38 OR S39
S41	S37 OR S40
S42	(MH "Qualitative Studies")
S43	interview
S44	experience
S45	qualitative
S46	S42 OR S43 OR S44 OR S45
S47	S41 AND S46

Database: Web of Science Core Collection 1900 to October Week 3 2019
#	Searches
# 1	TS=(telemedicine or telenursing or "tele medicine" or telehealth or "tele health" or telecare or "tele care" or telemonitoring or "tele monitoring")
# 2	TS=("medical informatics" or "electronic health" or "electronic care" or eHealth or "e health")
# 3	TS=("decision support" or "electronic alert")
# 4	TS=("hub and spoke")
# 5	TS=("remote support" or "remote surveillance" or "remote monitoring" or "remote counselling" or "remote counselling")
# 6	TS=("computer communication networks" or telecommunications)
# 7	#6 OR #5 OR #4 OR #3 OR #2 OR #1
# 8	TS=("intensive care unit*" or "critical care" or "intensive care" or "intensive therap*" or "intensive treatment" or ICU or ITU)
# 9	TI=("critical* ill*" or "sever* ill*" or "serious* ill*" or "at risk patient*" or "trauma patient*")
# 10	#9 OR #8
# 11	#10 AND #7
# 12	TS=(eICU or tele ICU or teleICU)
# 13	#12 OR #11
# 14	TS=("qualitative research" or interview or experience or qualitative)
# 15	#14 AND #13

Appendix 3. Screening tool

Sample	All kinds of critical care workers (i.e. professionals, paraprofessionals and lay health workers) who make use of telemedicine to support or provide care to patients and/or family members. Any other individuals or groups involved in the commissioning, evaluation, design and implementation of CCT. These individuals or groups could include administrative staff, information technology staff, managerial and supervisory staff, technical staff and industry partners who may or may not be based in a critical care facility but must be involved in the utilisation or implementation of CCT. Critical care patients and family members who have been the consumers or been involved in the development of CCT.
Phenomenon of Interest	Healthcare stakeholders' perceptions and experiences of factors affecting the implementation – either initiation or implementation – of CCT. CCT consists of the following combination: continuous electronic recording of patients' vital signs at the bedside, linked to a computer system enabling display of real‐time data; use of clinical decision‐making algorithms and electronic alerts; and Do not consider CCT applications that exclude clinical decision making as in some forms of plain remote screening.
Design	All empirical studies that use qualitative designs and methods for data collection and analysis. These will include, for example, ethnographic studies utilising participant observation; or, phenomenological studies using interviews. Studies utilising mixed designs to be considered only where the qualitative component and findings can be discerned; qualitative process evaluations to be considered where the previous statement applies. Studies to be considered for inclusion regardless of whether these were linked to effectiveness studies of CCT. Studies that use qualitative data collection methods but perform quantitative data analysis (e.g. using descriptive statistics) to be excluded.
Evaluation	Experiences and perceptions
Research type	Qualitative

Characteristics of studies

Characteristics of included studies [ordered by study ID]

Goedken 2017.

Study characteristics
Country	USA
Type of hospital	Rural hospitals (n = 3)
Staffing model	Open unit, mixed‐level patients, 20% of nurses had critical care training
CCT system and vendor	VISN 23
Notes	CCT was implemented in 2011. Interviews were conducted 1 ‐ 3 weeks before, 6 ‐ 12 weeks after, and 12 months after CCT activation. Data were collected between July 2011 and May 2013

Hoonakker 2013.

Study characteristics
Country	USA
Type of hospital	Not specified. Differed by region, size and number of years in existence
Staffing model	Not specified. Nurses moved across the bedside and hub teams and worked 12‐hour shifts. Nearly half (44%) of the nurses had a part‐time job in the CCT. 42% also worked as ICU bedside nurses
CCT system and vendor	Not specified
Notes	The data collection timeframe was not reported

Hoonakker 2018.

Study characteristics
Country	USA
Type of hospital	Not specified, different hospital sites (n = 6)
Staffing model	Varied across sites. The CCT hub employed 42 nurses and 20 physicians. On average a hub nurse monitored 45 beds
CCT system and vendor	Not specified
Notes	The participating CCT was in existence since 2003. The timeframe of the data collection was not reported

Jahrsdoerfer 2013.

Study characteristics
Country	USA
Type of hospital	Tertiary care centre (n = 2), Academic medical centre (n = 1), Community hospitals (n = 3)
Staffing model	Not specified
CCT system and vendor	Not specified
Notes	The survey was distributed while patients were still in the ICU. The timeframe of data collection was not reported

Kahn 2019.

Study characteristics
Country	USA
Type of hospital	Not specified
Staffing model	Not specified
CCT system and vendor	Not specified
Notes	The timeframe of the data collection was not reported

Khunlertkit 2013.

Study characteristics
Country	USA
Type of hospital	Not specified. 5 tele–intensive care units across the USA
Staffing model	In Tele‐ICUs 1 ‐ 3: 1  physician and 2 ‐ 3 nurses per shift. In Tele‐ICU 4 ‐ 5: 1 ‐ 2 physicians and 4 ‐ 5 nurses per shift
CCT system and vendor	VISICU software (Baltimore, MD)
Notes	The exact timeframe of data collection was not reported

Moeckli 2013.

Study characteristics
Country	USA
Type of hospital	Tertiary hospital (n = 3), Urban hospital (n = 1), Rural hospital (n = 3)
Staffing model	2 critical care nurses and 1 intensivist staffed the tele‐ICU
CCT system and vendor	Philips IntelliVue Clinical Information Portfolio, Philips VISICU eCareManager (v3.7.1),  Philips Healthcare, Andover, MA
Notes	The tele‐ICU came online in 2011, and all ICU beds were active in February 2012. The study was conducted between July 2011 and May 2012

Mullen‐Fortino 2012.

Study characteristics
Country	USA
Type of hospital	University hospital (n = 1), Community hospital (n = 1)
Staffing model	University hospital: closed unit. Residents and critical care fellows used for night coverage Community hospital: Residents used for night coverage Tele‐ICU: 1 ‐ 2 specially trained telemedicine nurses continuously monitored ICU patients 7 days a week. A board‐certified intensivist physician and a critical care nurse staffed the tele‐ICU between 7 pm and 7 am
CCT system and vendor	VISICU eICU remote monitoring system (Philips Electronics, Amsterdam, the Netherlands)
Notes	The CCT programme was activated independently for each hospital, with a duration of 1 month between each activation. The first email was sent on 24 October 2008, and the last completed survey was received on 30 December 2008

Shahpori 2011a.

Study characteristics
Country	Canada
Type of hospital	Tertiary hospitals (n = 3)
Staffing model	Closed ICUs. Nurse‐to‐patient ratio was 1:1 or 1:2. Staffed by Fellowship‐trained intensivists during the day and in‐house residents and fellows during the night, and supplemented by on‐call intensivists
CCT system and vendor	Quantitative Sentinel; GE Marquette Medical Systems, Milwaukee, WI, USA, GE and Solar 8000 GE Health care, Milwaukee, WI, USA
Notes	The survey was made accessible to participants from 22 January 2009 to 17 February 2009

Stafford 2008a.

Study characteristics
Country	USA
Type of hospital	Metropolitan hospital
Staffing model	CCT hub: staffed by 44 nurses, 26 physicians, and 2 IT staff
CCT system and vendor	VISICU, Inc., Baltimore, MD
Notes	All hub clinicians worked at the bedside in lCUs before working in the hub unit. Years of clinical experience and experience in critical care ranged from 5 years to more than 30 years. Some of the hub nurses continued to work at least 1 shift a week at the bedside in an lCU. Educational preparation for the nurses ranged from associate's degree to master's degree, with several pursuing advanced degrees

Thomas 2017.

Study characteristics
Country	USA
Type of hospital	Tertiary academic medical centres in medium to large urban settings (n = 3), Small urban medical centre (n = 1), Rural hospitals (n = 2)
Staffing model	CCT hub: an intensivist and 2 board‐certified critical care nurses  provided cover 21 hours and 24 hours a day, respectively Bedside ICU staff: 40 physicians and nurses at day shift, 30 at night shift, and 11 allied health professionals
CCT system and vendor	Not specified
Notes	Interviews were conducted between July 2011 and March 2013, at 2 and 12 months post‐implementation

Ward 2015.

Study characteristics
Country	USA
Type of hospital	Rural hospitals (n = 13), combination of prospective payment system hospitals (PPS) and critical access hospitals (CAHs)
Staffing model	Not specified
CCT system and vendor	Avera eCARE
Notes	The timeframe of the data collection was not reported

Wilkes 2016.

Study characteristics
Country	USA
Type of hospital	Rural hospitals. 3 CCT hub sites and 8 rural ICU sites
Staffing model	CCT provided coverage daily from 12 pm to 7.00 am
CCT system and vendor	Philips VISICU
Notes	In‐person meetings were conducted between August 2011 and March 2012

Characteristics of excluded studies [ordered by study ID]

Study	Reason for exclusion
Asiedu 2019	Ineligible intervention
Karfonta 1999	Ineligible intervention
Leslie 2017	Ineligible intervention
Morse 2014	Ineligible study design
Nadig 2018	Meeting abstract
Rak 2017	Wrong study design
Rak 2018	Meeting abstract
Ramnath 2014	Ineligible study design
Shahpori 2011b	Ineligible study design
Stafford 2008b	A more recent report was available, and included
Young 2011b	Ineligible study design

Differences between protocol and review

In the protocol we outlined our approach to translating studies published in languages other than English and how to sample relevant studies to include in our synthesis. However, we only found a small number of eligible studies (n = 13), all of which were published in English. We therefore did not need to implement our translation or sampling approaches for the final review.

Contributions of authors

Led by AX, the review was initially conceptualised by NM, AX, MT and JS.
JP completed the database searches.
SB, KI, MT, NM, JS and AX participated in the initial screening stages, with AX, KI and JP deciding on the final inclusion of the full papers.
AX, KI and SB contributed to the data extraction.
AX and KI quality‐assessed the included studies, coded the data, drafted and CERQualised the review findings; JP checked these and acted as an arbitrator.
All the review authors reviewed and commented on the review manuscript.
AX is the guarantor of the review.

Sources of support

Internal sources

• King's College London, UK

  Employed AX, JP and JS; and supported a postdoctoral placement for KI.

External sources

• Guy's & St Thomas' Charity, UK

  Financial support.

Declarations of interest

The authors declare no financial conflicts of interest.

New