Abstract
Objective:
The purpose of this study is to understand the communication among care teams during telemedicine-enabled stroke consults in an ambulance.
Background:
Telemedicine can have a significant impact on acute stroke care by enabling timely intervention in an ambulance before a patient reaches the hospital. However, limited research has been conducted on understanding and supporting team communication during the care delivery process for telemedicine-enabled stroke care in an ambulance.
Method:
Video recordings of 13 simulated stroke telemedicine consults conducted in an ambulance were coded to document the tasks, communication events, and flow disruptions during the telemedicine-enabled stroke care delivery process.
Results:
The majority (82%) of all team interactions in telemedicine-enabled stroke care involved verbal interactions among team members. The neurologist, patient and paramedic were almost equally involved in team interactions during stroke care, though the neurologist initiated 48% of all verbal interactions. Disruptions were observed in 8% of interactions, and communication-related issues contributed to 44%, with interruptions and environmental hazards being other reasons for disruptions in interactions during telemedicine-enabled stroke care.
Conclusion:
Successful telemedicine-enabled stroke care involves supporting both verbal and non-verbal communication among all team members using video and audio systems to provide effective coverage of the patient for the clinicians as well as vice versa.
Application:
This study provides the deeper understanding of team interactions during telemedicine-enabled stroke care that is essential for designing effective systems to support team work.
Keywords: communication and teamwork in healthcare, telemedicine, computer-supported collaborations, distractions and interruptions, team cognition
Précis:
Using interactive team cognition theory, this quantitative observational study examines how stroke care delivery teams communicate information in a telemedicine-integrated, ambulance-based setting. Separate videos of remotely located team members providing stroke care were obtained for 13 simulated sessions and coded for interactions, communication pairs, stroke care tasks and disruptions.
Introduction
Telemedicine, the use of communications technology to connect remotely located patients to medical professionals, is quickly emerging as a viable and efficient approach to care delivery where patients and care providers who are physically remote communicate virtually using various technology platforms to enable safe and effective care delivery (Rogers et al., 2017). In some situations, such as emergency care for stroke patients, timely assessment and intervention are critical for preventing death and improving patient outcomes (Saver, 2006; Saver et al., 2013). Telemedicine consultations allow a remotely located neurologist to consult with a patient and paramedic in the ambulance as well as a nurse located at a remote hospital location. The care providers involved in these telemedicine consults are required to communicate with one another and the patient using technology platforms and technology-supported verbal and non-verbal methods (Rogers et al., 2017). While some ambulances may be specifically designed to support telemedicine consultations, most are retrofitted within existing spatial constraints by incorporating cameras, audio systems and computers. This design results in a very complex system where effective communication and teamwork are dependent on the dynamic interactions among the people involved and their tasks, the physical environment where the care is provided, and the technology that supports information flow and interaction (Rogers et al., 2017).
Teamwork has been identified as a critical human factor, and its importance continues to increase as society becomes more collaborative and socio-technically advanced (Salas, Reyes, & McDaniel, 2018). In particular, teamwork is prevalent in many critical and dynamic situations in the healthcare context. As a result, much recent research in both the teamwork and healthcare domains has focused on such aspects as communication (Mills, Neily, & Dunn, 2008; Sacks et al., 2015), team cognition (McNeese, 2016), and training (Hughes et al., 2016; Weaver, Dy, & Rosen, 2014; Alsabri et al., 2020), leading to significant advancements in our understanding in how to improve teamwork and increase overall team performance in healthcare (Buljac-Samardzic et al., 2010). For example, a recent meta-analysis conducted by Schmutz and colleagues (2019) including articles involving 1,390 teams found that teamwork is positively related to performance, and effective communication is considered a hallmark of teamwork, leading to multiple studies seeking to identify its conceptual and applied importance (Marlow et al., 2017; Marlow et al, 2018; Salas, Cooke, & Rosen, 2008). The healthcare context is highly dynamic with much information embedded throughout, meaning effective communication using verbal and non-verbal means is essential to respond to the evolving situation and to share information critical for appropriate care of the patient.
The importance of communication and teamwork in high-risk stressful healthcare environments such as operating rooms (Healey, Sevdalis, & Vincent, 2006; Mills, Neily, & Dunn, 2008; Wahr et al., 2013), trauma rooms and intensive care units (Dietz et al., 2014; Lazzara et al., 2015; Lin, Chaboyer, & Wallis, 2009) have been highlighted by many researchers. However, few studies have examined how physically remote team members communicate with one another to provide effective patient care in telemedicine consults in the emergency department and other hospital-based settings (Butler et al., 2019; Pappas & Seale, 2009; Whitten & Allen, 1995). The studies that have evaluated the feasibility of implementing telemedicine-based care in ambulances focused on the satisfaction of care providers (Liman et al., 2012; Pavlopoulos, Kyriacou, Berler, Dembeyiotis, & Koutsouris, 1998; Yperzeele et al., 2014) and the evaluation of door-to-needle time in ambulances using telemedicine for ischemic stroke patients (Belt, Felberg, Rubin, & Halperin, 2016), with no studies being found that comprehensively examined how team members communicated with one another during telemedicine-based stroke care. There is a need to understand this communication in order to design systems that effectively support the care delivery process for this critical condition.
Closely related to communication is the concept of team cognition, which is broadly defined as cognition occurring at the team level where extensive information is shared to develop a common understanding (Salas and Fiore, 2004; McNeese et al, 2016). Historically, team cognition has been studied from the perspective of shared knowledge, meaning that each team member shares information with the rest, resulting in team level knowledge, often referred to as a team mental model (Mohammed, Ferzandi, & Hamilton, 2010). In order to share this knowledge, communication is required. Thus, the theory of interactive team cognition was posited to emphasize the importance of communication in the overall process of team cognition (Cooke, Gorman, Myers, & Duran, 2013). Specifically, interactive team cognition stipulates that “(1) team cognition is an activity, not a property or a product; (2) team cognition should be measured and studied at the team level; and (3) team cognition is inextricably tied to context” (Cooke, Gorman, Myers, & Duran, 2013). Several studies have found real-time interaction-based measures such as communication among team members to be effective predictors of team cognition (Gorman et al., 2020; Grimm et al., 2017). While traditionally, team cognition has been viewed as either shared knowledge or interactive team cognition, McNeese and colleagues (2016) have suggested that these two approaches should be seen as integrated rather than separate.
Interactive team cognition has been applied in previous research to characterize team cognition in similar distributed complex task settings in which specialized and interdependent team members complete asynchronous or synchronous tasks (Demir, McNeese, & Cooke, 2017; Gorman, Cooke, & Winner, 2006; Gorman, Martin, Dunbar, Stevens, & Galloway, 2013). Thus, it is an appropriate theory for use in this study to analyze real-time communication-based measures to understand aspects of team cognition as a geographically distributed stroke team works together to provide care to a patient in an ambulance using telemedicine.
Research Questions
The following research questions address key aspects of the interactive team cognition theory, namely (1) that team cognition is an activity, (2) that it should be studied at the team level, (3) that it is inextricably tied to context and (4) that shared mental models about care delivery are created among team members through communication. The specific research questions investigated in this study are the following:
How do team members communicate with one another using verbal and non-verbal means during telemedicine-enabled stroke evaluations in an ambulance? (1, 2)
How do team members obtain the information required to complete specific tasks associated with stroke care evaluation? (4)
How do disruptions during telemedicine-enabled stroke evaluations impact communication among team members? (3)
Method
This research complied with the American Psychological Association Code of Ethics and was approved by the Institutional Review Board at Clemson University (IRB 2018-465). Informed consent was obtained from each participant.
Participants
Thirteen participants of each caregiving type -- nurse, neurologist, and paramedic--were recruited to conduct simulated stroke telemedicine consults, 39 participants total. The nurses were recruited from the emergency department at the secondary hospital that serves as the receiving emergency department for stroke patients in a rural county in the southeastern United States. The paramedics were recruited from the Fire and Emergency Medical Services (EMS) in the same county. The neurologists were recruited from a major hospital system in the Southeast with connections to the secondary hospital. In addition to clinicians, 10 standardized patients from the secondary hospital and an Emergency Medical Technician (EMT) from the Fire and EMS department were recruited to assist with the simulations and to create approximate lifelike conditions for the stroke consult.
Apparatus
Simulated stroke care delivery sessions were observed in the three locations where the team members were located -- the emergency department at the secondary hospital, a tertiary hospital within the major hospital system, and an ambulance provided by the rural county Fire and EMS. Figure 1 shows the telemedicine setup at each of three locations.
The nurse participants used a rolling computer cart, called the REACH cart, equipped with a 24” monitor, keyboard, and a connected camera and overhead speaker to complete the telemedicine consult. The neurologist participants completed the telemedicine consult and any further data collection on a Dell 13” laptop. The paramedic’s telemedicine system consisted of a laptop similar to the one used by the neurologists, an overhead speaker, a camera mounted at the rear of the ambulance for a patient view and a wireless router. At each study location a GoPro Hero 7 was used to record the participants’ actions in their environment, with the exception of the ambulance which had two GoPro cameras as shown in Figure 1, one located at the back of the ambulance and one at the front to capture all movements within the ambulance. In addition, the Xbox Game Bar installed on all Windows 10 computers was used to record the screens of the neurologist and nurse to capture their interactions with the telemedicine interface.
Task
Stroke telemedicine consultations begin with notification from the paramedics to the nurse at a receiving emergency department in the area of an incoming patient’s demographics and possible acute stroke status. The patient is subsequently transported to the emergency department in the ambulance accompanied by a paramedic and driven by an EMT along a route determined by the latter. Using the information from the paramedics, the nurse creates a patient case in the telemedicine system, which is then assigned to a neurologist on call. Once the neurologist is connected to the ambulance with the patient and paramedic and the nurse at the receiving emergency department, the team obtains the patient history and vitals before conducting a stroke evaluation using the National Institutes of Health Stroke Scale (Kwah & Diong, 2014). During the evaluation, the paramedic assists the neurologist by interacting with the patient, confirming patient communication, or updating current patient vitals, and the neurologist communicates with the paramedic and nurse to coordinate the care plan for this patient. Once the ambulance arrives at the emergency department, the patient is transferred to the hospital and is given a computed tomography (CT) scan before being admitted to a room in the emergency department. A graphic of the care delivery process (Rogers et al., 2017; Rogers, Ponathil, et al., 2020) can be seen in Figure 2.
Procedure
Participants were instructed to complete their normal tasks for a stroke evaluation completely within the ambulance before the patient arrived at the emergency department, including any documentation required in the telemedicine system. Paramedic participants were also given basic patient information before the start of each study. Researchers started the recording equipment, and participants then logged into the telemedicine system and began their normal tasks. After the patient arrived at the emergency department, researchers stopped the recordings and collected the videos. This study protocol was pilot tested for four consultations, with no changes being suggested, so the study continued with the remaining sessions.
Measures and Analysis.
To analyze the communication structure, tasks in telemedicine stroke care delivery, and disruptions to team communications, recordings of the interior of the ambulance, nursing station, nursing computer screen, neurologist workspace, and neurologist computer screen were created for each simulated stroke observation. These video recordings, which were coded from the time when all clinicians were connected to the telemedicine system to the time when the ambulance arrived at the emergency department, were used as inputs in the Noldus Observer XT program that was used to code the tasks of the paramedic, the flow disruptions, the communication type, and the communication-pairs.
Communication events were coded based on an existing taxonomy (Tiferes et al., 2019) to identify the communication-pair and the direction of the communication (e.g. neurologist to nurse) as well as the communication mode (verbal and non-verbal) and type of communication (e.g. give information, receive information) as shown in Table 1. Each communication event was coded individually; for instance, if an action was requested multiple times, each instance was coded separately to identify related issues and disruptions.
Table 1.
Communication types | ||
---|---|---|
Communication mode |
Types of communication | Definition |
Verbal communication | Request action | Requesting/suggesting someone to do something (e.g. neurologist requesting paramedic to conduct specific evaluation tasks on patient eye movement, leg movement, arm movement, etc.) |
Request acknowledgment/repetition | Asking for repetition or confirmation if a previously given message was heard or a previously requested action was completed (e.g. neurologist asking paramedic to repeat their response to a previous question) | |
Request information | Asking to for new/extra information about something, other than confirmation/repetition (e.g. neurologist directly asking paramedic or patient (if able) to provide information; neurologist asking paramedic for their observations of the patient during evaluation) | |
Give acknowledgment/repetition | Giving acknowledgment that a previously received message was heard or that a previously requested action was completed; repeating/restating previously given information/request (e.g. paramedic repeating a previous sentence, or confirming a statement made by the neurologist) | |
Give information | Giving new/extra information about something, other than confirmation/repetition (e.g. paramedic/patient giving information regarding demographic and history.) | |
Give instruction | Providing specific information that would apply to other cases (e.g. neurologist providing paramedic with general instructions for conducting patient evaluation) | |
Request instruction | Asking for specific information that would apply to other cases (e.g. paramedic asking neurologist to provide specific instructions for conducting evaluation) | |
Non-verbal communication | Instrument movement on screen | Interaction via instrument movements visible on the shared screen (e.g., pointing with an instrument) |
Camera view change | Interaction via changes or adjustments on camera position, zoom, focus, or angle | |
Display indicators on screen | Interaction via changes of display indicators (e.g., when the vitals are changed by a nurse, the area of the screen with the vital sign flashes red for all other participants to indicate that the value was updated) | |
Face-to-face interaction | Interaction via body, head, or hand position and movements (e.g. patient following paramedic’s instructions for eye, arm, leg movement were coded as face-to-face communication between patient and neurologist; or if the paramedic used thumbs up or nodded his head to indicate “yes” when communicating with the neurologist while looking at the ambulance camera) |
The tasks performed as part of telemedicine-enabled stroke care were coded based on the set of tasks identified in a previous study (Rogers, Chalil Madathil, et al., 2020; Rogers, Ponathil, et al., 2020) as shown in Table 2. Flow disruptions, defined as deviations from the natural progression of a procedure that potentially compromise safety or efficiency (Palmer et al., 2013), were coded based on an existing taxonomy developed for measuring flow disruptions in operating rooms that was adapted for this study (Palmer et al., 2013). Specifically, they were categorized as those related to layout, environmental hazards, interruptions, communication-related, equipment failure or usability as shown in Table 3.
Table 2.
Telemedicine stroke care task categories |
Definition |
---|---|
Update demographics | Ask patient for name, date of birth, communicate with nurse |
Update patient history | Ask patient for medical conditions, surgical history, family conditions, medications, communicate with nurse |
Level of consciousness | Ask patient for their level of consciousness |
Eye movement | Move to stand beside patient, hold one or two fingers on the right side of the patient’s peripheral vision, hold up one to two fingers on the left side of the patents peripheral vision |
Facial palsy | Check for weakness of facial muscles |
Arm movement | Ask patient to move right and left arm |
Leg movement | Ask patient to move right and left leg |
Visual field | Move to stand beside patient, hold one or two fingers on the right side of the patient’s peripheral vision, hold up one to two fingers on the left side of the patent’s peripheral vision |
Ataxia | Check for slurred speech |
Sensation | Pinprick patient’s right and left cheek; right and left arm, and right and left leg |
NIH Stroke Scale cards | Show patient object cards, point to an object on the card, confirm object with the MD. Show patient scene cards, show patient sentence cards, point to sentence, and confirm sentence with MD |
Check for tPA contraindications | Check if patient awoke with stroke |
Update vitals | Get blood pressure (affix blood pressure cuff to patient’s upper arm) press "NIBP" button, read results, get heart rate, count breaths taken by the patient for 30 seconds, affix pulse ox reader to patient finger opposite from BP cuff, read Sp02 (%) on reader |
Waiting/other | Not actively engaged in any assignment |
Table 3.
Type of flow disruption |
Definition |
---|---|
Layout related | Spatial organization or positioning of certain items in the ambulance that hinder the team during their task by blocking their route or impeding visibility. These items include > Furniture or equipment positioning to clear the movement path > Inadequate use of space causing clutter or route blockage > Impeded visibility due to objects obstructing the team's view |
Environmental hazards | Incidents involving the interaction of the team with the ambulance environment such as > Slips, trips and/or falls by going over an element on or above the floor > Collison/bumping into people, objects, equipment > Excessive reach needed to access patient, objects, or equipment > Team's interaction with sharp objects and contaminated needles |
Interruptions | Incidents not related to the assessment procedure causing distractions for the team during task performance such as > Phone calls or pagers unrelated to the assessment procedure > External personnel who are not a member of the core assessment team > Spilling, dropping, or picking up something from the floor > Searching for missing items > Looking at personal phones |
Communication related | Incidents interfering with effective communication between the team members during the assessment procedure in the ambulance, including > Poor communication among the team members that does not achieve its desired goal > Lack of response or confirmation > Confusion and lack of understanding associated with the team communication > Simultaneous communication among team members resulting in miscommunication or repetition of information > Nonessential communication (e.g., sports talk, jokes, personal inquiries) > Environmental noise caused by increasing sound levels in the ambulance |
Equipment failure | Incidents where the subject is distracted by malfunctioning or broken equipment during assessment. This includes medical equipment in the ambulance. |
Usability related | Subject is distracted by problems associated with telemedicine related equipment: Computers (operating software, programs, and utilities), camera, speaker, pointing devices, monitors. |
Two researchers with human factors training coded the videos. They received extensive training in the protocol and coding program and coded sessions in parallel until there was consensus. Percentage agreement over 85% was obtained for flow disruption, task and communication codes using an index of concordance. The event-based data extracted from Noldus Observer XT was converted into time-based data using Python such that task, communication and disruption data were available for each second of the observation.
Results
All 13 simulated consults involved a remote neurologist, remote nurse, a paramedic and patient in the ambulance. The average duration of the telemedicine stroke consults from the time the clinicians logged into the system to the time the ambulance reached the emergency department was 12.45 minutes, standard deviation 2.72 minutes, with sessions ranging from 6.73 minutes to 16.73 minutes.
A total of 1,897 communication events were recorded across all observations. Flow disruptions were recorded in 156 instances during the interactions observed. Thus, disruptions occurred during 8% of the interactions observed during the telemedicine stroke care delivery process.
Communication Mode and Types.
Verbal communication constituted the majority (82%) of the interactions in the 13 stroke care evaluations observed. The most frequent type of verbal communication (40%) observed involved one team member giving information to another. Twenty-six percent of communication events involved requests for information and 23% requests for action. Verbal communication was also used for giving acknowledgement or repeating information (5%) and for requesting acknowledgement/repetition (5%). Thus, there were 153 instances (10%) where team members either requested or gave acknowledgement/repeated information. There were very few instances of team members providing or asking for general information that could apply to other communication types (instruction or requesting instruction).
On the other hand, the non-verbal communication in the telemedicine stroke evaluations observed almost always involved face-to-face interaction (99%). There were no instances observed of team members communicating with one another using display indicators on the computer screen or instruments or using movements on the screen. Figure 3 shows the distribution of verbal and non-verbal interactions during the telemedicine stroke care evaluations observed.
Team member interactions.
The neurologist (35%), paramedic (29%) and patient (32%) were almost equally involved in the interactions that occurred during the telemedicine stroke care delivery process (Table 4). The nurse was involved in only 4% of the interactions, when she/he interacted with either the neurologist or paramedic. The nurse had no interactions with the patient. Forty percent of all communication originated from the neurologist, while the paramedic and patient were responsible for 31% and 27% of the initiations, respectively.
Table 4.
Verbal Communication events |
Non-Verbal Communication events |
Total | |
---|---|---|---|
Communication direction | |||
PM to MD | 244 (16%) | 15 (4%) | 259 (14%) |
PM to Patient | 176 (11%) | 137 (41%) | 313 (17%) |
MD to all | 76 (5%) | 4 (1%) | 80 (4%) |
MD to PM | 290 (19%) | 6 (2%) | 296 (16%) |
MD to Nurse | 29 (2%) | 0 (0%) | 29 (2%) |
MD to Patient | 345 (22%) | 2 (1%) | 347 (18%) |
Nurse to PM | 20 (1%) | 0 (0%) | 20 (1%) |
Nurse to MD | 44 (3%) | 1 (0%) | 45 (2%) |
Patient to PM | 115 (7%) | 70 (21%) | 185 (10%) |
Patient to MD | 221 (14%) | 101 (30%) | 322 (17%) |
Total | 1560 (100%) | 337 (100%) | 1897 (100%) |
Team member involved in communication | |||
PM involved | 921 (28%) | 232 (34%) | 1153 (29%) |
MD involved | 1249 (38%) | 129 (19%) | 1378 (35%) |
Nurse involved | 169 (5%) | 5 (1%) | 174 (4%) |
Patient involved | 933 (29%) | 314 (46%) | 1247 (32%) |
Total | 3272 (100%) | 680 (100%) | 3952 (100%) |
Communication-pair | |||
PM-MD | 534 (35%) | 21 (6%) | 555 (29%) |
PM-Patient | 291 (19%) | 207 (61%) | 498 (26%) |
PM-Nurse | 20 (1%) | 0 (0%) | 20 (1%) |
MD- Patient | 566 (36%) | 103 (31%) | 669 (35%) |
MD-Nurse | 73 (5%) | 1 (0%) | 74 (4%) |
Nurse- Patient | 0 (0%) | 0 (0%) | 0 (0%) |
MD-all | 76 (5%) | 4 (1%) | 80 (4%) |
PM= paramedic and MD= neurologist
The neurologist was involved (either talking or being addressed directly) in 38% of all verbal interactions, while the paramedic was involved in 28% and the patient in 29%. The nurse was involved in only 5% of all verbal communication. The patient and neurologist were involved in 46% and 34%, respectively.
Verbal communication.
When communication-pairs are considered, it was observed that the neurologist addressed others in the team (paramedic, nurse, patient or all together) in 48% of all verbal communication events (Table 4). The paramedic talked during 27%, most frequently addressing the neurologist and the patient. The patient talked directly to the neurologist in 14% of all verbal communication events observed and to the paramedic for 7%. The neurologist-paramedic pair and the neurologist-patient pair accounted for 35% and 36%, respectively, of all verbal interactions, and the paramedic-patient pair accounted for 19%.
Non-verbal communication.
Much of the non-verbal communication occurred in the ambulance with either the paramedic (45%) or the patient (51%) as the initiator. The neurologist and the nurse rarely communicated with the team using non-verbal means. The paramedic-patient pair accounted for 62% of all non-verbal interactions. The neurologist-patient pair accounted for 31%, and the paramedic-neurologist for 6% of all non-verbal interactions.
Telemedicine Stroke Care Tasks
Verbal Communication.
Verbal communication was frequently used while performing tasks related to updating patient history in the telemedicine system, constituting 25% of all verbal interaction among team members (Figure 4), with evaluation of patient sensation comprising 13%, completion of the NIH Stroke Scale cards 14% and visual field evaluation 8%. The remaining instances of verbal communication were distributed almost equally across other aspects of the stroke evaluation process including evaluation of facial palsy, arm movement, leg movement, and ataxia and for checking for tPA contraindications. There were relatively few instances of verbal communication updating demographics and vitals in the telemedicine system.
Non-verbal communication.
Non-verbal communication, primarily face-to-face interaction, was used for many of the interactions requiring evaluation of the patient’s condition including sensation (21%), ataxia (14%), arm movements (13%) and visual field and the completion of the NIH Stroke Scale cards (Figure 4).
Team member interactions during telemedicine stroke care delivery tasks.
Approximately half of the interactions recorded for the majority (7 of the 12) of the stroke care tasks occurred between the neurologist and the patient directly (Table 5). These tasks included updating patient information (51%); evaluating the level of consciousness (47%), facial palsy (45%), arm movement (56%), leg movement (52%), and ataxia (50%); and checking for tPA contraindications (58%). For these tasks, the paramedic was also actively involved in the interactions, communicating with both the patient and the neurologist to complete the stroke care evaluations. The majority of the interactions between the neurologist and the paramedic involved updating demographics (75%) and vitals (69%). For evaluating the visual field of the patient, 45% of the interactions recorded were between the paramedic and the neurologist and 39% between the paramedic and the patient. The paramedic and patient communicated frequently to complete evaluations related to sensation (53%) and the NIH Stroke Scale cards (50%). The nurse was not involved in completing any of the stroke care evaluation tasks, except for updating demographics where she was involved in 17% of the interactions.
Table 5.
Communication pair | |||||||
---|---|---|---|---|---|---|---|
Stroke care task | PM-MD | PM-Patient | PM-Nurse | MD- Patient |
MD-Nurse | Nurse- Patient |
MD-All |
Update demographics | 75% | 0% | 0% | 8% | 17% | 0% | 0% |
Update patient information | 33% | 6% | 2% | 51% | 5% | 0% | 4% |
Level of consciousness | 23% | 18% | 1% | 47% | 9% | 0% | 2% |
Eye movement | 38% | 28% | 0% | 26% | 3% | 0% | 5% |
Facial palsy | 30% | 20% | 0% | 45% | 1% | 0% | 4% |
Arm movement | 19% | 17% | 1% | 56% | 1% | 0% | 6% |
Leg movement | 17% | 23% | 1% | 52% | 1% | 0% | 6% |
Visual field | 45% | 39% | 0% | 13% | 1% | 0% | 2% |
Ataxia | 17% | 29% | 0% | 50% | 0% | 0% | 4% |
Sensation | 22% | 53% | 0% | 22% | 1% | 0% | 3% |
NIH Stroke Scale Cards | 28% | 50% | 0% | 17% | 2% | 0% | 2% |
Check for tPA contraindications | 14% | 9% | 5% | 58% | 8% | 0% | 6% |
Update vitals | 69% | 6% | 8% | 3% | 6% | 0% | 8% |
Waiting/other | 37% | 3% | 3% | 21% | 23% | 0% | 13% |
PM-Paramedic; MD-Neurologist
Disruptions during team interactions.
The majority of the flow disruptions (n=134, 87%) occurred during verbal interactions among team members, with communication-related disruptions being the most frequent constituting 44%, followed by interruptions at 19%, environmental hazards at 18%, usability issues at 12%, and layout issues at 8%. No instances of equipment failure were observed during team communications. Communication-related disruptions constituted approximately half (48%) of all disruptions during verbal communication. Fewer disruptions (n = 22) occurred during non-verbal interactions, with half of those observed being attributed to either layout or environmental hazards.
Team members interactions impacted by disruptions.
Approximately half of the disruptions were observed during interactions between the neurologist and the paramedic (48%), followed by interactions between the paramedic and the patient (23%) and the neurologist and the patient (19%) (Figure 5). Communication-related disruptions (51%) were the most common category during neurologist-paramedic interactions, while environment hazard related (36%), layout related (25%) and communication-related (22%) disruptions were those most frequently observed during paramedic-patient interactions, and communication-related disruptions (48%) were most frequent during neurologist-patient interactions. Few disruptions were observed while the nurse was interacting with the paramedic (3%) or neurologist (2%), with most being communication-related or related to the usability of the telemedicine system.
Communication related to acknowledgement/repetition between pairs and disruptions
Since communication related to giving and receiving acknowledgement/repetition may result because of disruptions and/or complexity in communication leading to adaptive strategies (Tiferes et al., 2019), communication events involving acknowledgements/repetitions occurring during disruptions were further analyzed. Twenty-nine percent (n=46) of communication events involving disruptions (46/156) were related to team members giving or requesting acknowledgement or repetition. Of these, the neurologist was involved in 98%, with the neurologist-paramedic pair accounting for 69% of these instances. Seventy-five percent of the disruptions observed during acknowledgement/repetition interactions involved communication issues due to confusion and a lack of response or understanding, 8% due to equipment usability issues, and the remaining 15% to layout related issues and environmental hazards.
Tasks disrupted during interactions.
Disruptions were most frequently experienced while conducting tasks related to updating patient information in the telemedicine system (20%), followed by tasks related to the evaluation of sensation (18%) and of the visual field (13%) (Table 6). In addition, team members experienced disruptions during the evaluation of leg movement (9%), the level of consciousness (8%), the NIH Stroke Scale cards (8%) and arm movement (7%). Few disruptions occurred during the remaining stroke care tasks.
Table 6.
Type of Disruptions | |||||||
---|---|---|---|---|---|---|---|
Stroke care task | Total Number of Disruptions |
Layout | Environmental hazards |
Interruptions | Communication | Equipment failure |
Usability |
Update demographics | 1 (1%) | 0% | 0% | 0% | 100% | 0% | 0% |
Update patient information | 31 (20%) | 0% | 0% | 3% | 29% | 68% | 0% |
Level of consciousness | 13 (8%) | 0% | 0% | 0% | 85% | 0% | 15% |
Eye movement | 2 (1%) | 0% | 50% | 0% | 0% | 0% | 50% |
Facial palsy | 6 (4%) | 0% | 33% | 50% | 0% | 0% | 17% |
Arm movement | 11 (7%) | 0% | 9% | 36% | 0% | 0% | 55% |
Leg movement | 14 (9%) | 71% | 0% | 0% | 14% | 0% | 14% |
Visual field | 21 (13%) | 10% | 52% | 0% | 19% | 0% | 19% |
Ataxia | 3 (2%) | 0% | 67% | 0% | 33% | 0% | 0% |
Sensation | 28 (18%) | 0% | 36% | 21% | 36% | 0% | 7% |
NIH Stroke Scale Cards | 12 (8%) | 0% | 0% | 25% | 75% | 0% | 0% |
Check for tPA contraindications | 1 (1%) | 0% | 0% | 0% | 100% | 0% | 0% |
Update vitals | 7 (4%) | 0% | 0% | 0% | 100% | 0% | 0% |
Waiting/other | 5 (4%) | 17% | 0% | 67% | 17% | 0% | 0% |
Types of disruptions during frequently disrupted tasks.
The type of disruption experienced most frequently by team members while interacting to update patient history were communication-related disruptions (68%) and interruptions (29%) (Table 6). While evaluating patient sensation, team members experienced environmental hazards (36%), communication-related disruptions (36%) and interruptions (21%) as well as usability issues (7%). Environmental hazards (52%) were the primary disruption experienced while evaluating the patient’s visual field, though team members also experienced disruptions due to usability of equipment (19%), communication issues (19%) and layout issues in the ambulance (10%). Layout issues were frequently experienced (71%) while evaluating tasks related to leg movement since the patient’s leg was often out of the field of view of the ambulance camera and, thus, not visible to the remotely located neurologist. Communication-related issues frequently occurred while interacting to perform tasks related to patient’s level of consciousness (85%) and NIHSS cards (75%).
Discussion
Effective communication among team members is critical to the successful implementation of telemedicine in ambulances. Using objective quantitative analysis of simulated telemedicine-enabled stroke care sessions, this study examined the dynamics of interactive team cognition during telemedicine stroke care in an ambulance setting.
The results found that regular communication among geographically distributed team members using verbal and non-verbal strategies was critical for developing a shared understanding of the patient’s condition and for conducting the stroke care evaluation. However, verbal interactions were more frequent, being observed in 82% of all communication events. The patient, neurologist and paramedic exhibited similar involvement in verbal interactions, though the neurologist directly addressed other members more frequently than anyone else on the team (Figure 6). Our results indicate significant back and forth verbal interactions between the neurologist and the paramedic and the neurologist and the patient, with the paramedic frequently serving as the intermediator between the other two. In many instances, the paramedic’s role was to be the ears and eyes of the neurologist in the ambulance to help complete patient care evaluation tasks that the neurologist could not easily do remotely due to a limited field of view. Thus, in this situation, where the patient information was not easily accessible to the neurologist, the verbal communication with the paramedic was critical in helping the neurologist develop a shared mental model about the patient’s condition.
The fewer instances of non-verbal interactions were primarily face-to-face interactions between the paramedic and the patient or were instances where a previous verbal communication was unsuccessful due to problems associated with the speaker/pager between the paramedic and the neurologist. In contrast to our study, previous research conducted in an operating room found that 59% of all observed interactions between team members were non-verbal (compared to 18% in this study), using face-to-face interactions as well as shared consoles (Tiferes et al., 2019). Similar to our study, Tiferes et al. (2019) found that team members in physical proximity to one another used face-to-face interactions more frequently, while those members who could not see one another used shared consoles and displays to communicate non-verbally. However, in this study, there were no instances of the telemedicine system being used to communicate between remotely located team members. Rather, verbal means were used extensively to confirm completion of tasks and coordinate activities among the neurologist, patient and paramedic. Over-reliance on verbal communication can potentially be problematic in an ambulance setting where loud background noise from sirens and traffic and poor audio signals may make communication difficult. This issue suggests a potential for redesigning the interface of the telemedicine system to support more non-verbal team communication. For example, the telemedicine system interface could be used to confirm completion of specific stroke care evaluation tasks so that all team members are aware of where they are in the process without having to verbally request information or acknowledgement from team members. This modification may help reduce the high reliance on verbal communication to support team cognition that was observed in this study. However, given the intrinsically distributed nature of telemedicine, verbal communication via audio and video will likely continue to be critical for teams in order to build shared mental models to support patient care delivery.
This study found that 29% of the communication events when team members were disrupted involved giving or requesting acknowledgement/repetition, with 98% of the interactions involving the neurologist. Communication challenges (lack of response, lack of understanding, confusion) contributed to 75% of the interactions where team members gave/requested acknowledgement. Improvements in the telemedicine setup, both the audio and the video systems, in the ambulance as well as the design of the interface have the potential to improve communication between the neurologist and the rest of the team and reduce communication-related disruptions.
This study shows that patients play a critical role in team interactions for telemedicine-enabled stroke care delivery as they are almost as frequently engaged in interactions as the neurologist and the paramedic. Due to the ambulance telemedicine setup, the neurologist, nurse and paramedic could see the patient either directly or virtually through video feeds. However, the patient could see only the paramedic in the ambulance and communicated verbally with the neurologist, often following prompts from the paramedic. The neurologist also had an incomplete view of the patients and could not easily see if they followed the prompts. This situation resulted in the neurologist asking the paramedic to perform tasks such as facial evaluation and to report the results. Given that the patient-neurologist pair contributed to 35% of the interactions observed and the provision of safe and effective care depends on good communication between the patient and neurologist, it is imperative that the telemedicine setup in the ambulance be designed to provide good visual and auditory connections between these two team members.
While verbal communication plays an important role in obtaining and updating patient information, non-verbal communication was used frequently during patient stroke care evaluation tasks such as the evaluation of sensation, arm and leg movement and visual field. This communication involved the patients responding to prompts from the paramedic or neurologist to move body parts or indicate how they were feeling. However, it was often difficult for the neurologist to evaluate the patient’s condition due to the incomplete field of view resulting from the positioning of the cameras, requiring the paramedic to serve as intermediator. Tasks related to updating patient information which involved frequent verbal exchanges were disrupted as often as those related to evaluating sensation where non-verbal means were also used.
Investigating the nature of communication and teamwork in telemedicine is challenging because the care team is geographically distributed and capturing team interactions simultaneously is difficult from a research perspective. This study was able to collect parallel video data at three locations and merge the video feeds such that they could be viewed and coded simultaneously to understand how team members dynamically interact with one another to coordinate information and make decisions in the care context. This coordination is particularly important in a constrained setting like an ambulance where in-person observation is difficult. This approach could be adapted and used in future studies of team behavior in telemedicine-enabled care.
Using interactive team cognition theory, this study demonstrates how different members of a geographically distributed care team with unique perspectives and physical locations communicate with one another to provide stroke care to patients. Studying team interactions in the context of critical stroke care evaluation tasks allowed us to understand how team members communicated to build the shared mental models needed to perform their tasks. In addition, the research reported here explored the role of the care delivery context in team cognition by studying different types of disruptions during communication events. Understanding the structure and nature of team interactions in this context is critical for identifying interventions that support successful implementation of telemedicine-enabled stroke care.
Limitations
While the interactions observed during the recorded sessions were realistic, involving practicing care providers, a moving ambulance with fully functional telemedicine apparatus and remotely located clinicians in their usual work locations, the sessions were simulations utilizing a standardized patient. There is a possibility that some of the patient and care team behaviors may be different during a real telemedicine stroke care event. For example, the simulated patient could not mimic certain aspects of a stroke such as facial droop, which may alter the assessment of symptoms. Another discrepancy was that there may have been a lack of urgency in treating the simulated patients as they were actors, not real patients. While the video recording enabled coders to observe most interactions, it is possible that some nuanced non-verbal interactions could not be captured. Additionally, the team had no control over the ambulance route taken by the ambulance driver/EMT though most of the time the same route was used. The specific challenges with internet connectivity and communication disruptions may be impacted by the coverage available on different routes.
Conclusion
Stroke care, when started early in the ambulance via telemedicine, can make a significant difference in patient outcomes. However, its successful implementation requires an understanding of the dynamic interactions among team members including patients and the care context. Successful telemedicine stroke care involves supporting both verbal and non-verbal communication among all team members using video and audio systems that provide effective coverage of the patient for the providers as well as vice versa. Successful implementation also involves understanding the specific tasks requiring frequent communication that could be simplified by system improvements such as interface design to support non-verbal communication regarding patient status.
Key points.
The majority (82%) of all team interactions in telemedicine-based stroke care involved verbal interactions among team members.
The neurologist, patient and paramedic were almost equally involved in team interactions during stroke care, though the neurologist initiated 48% of all verbal interactions.
Non-verbal interactions were primarily face-to-face interactions with the paramedic (45%) or the patient (51%) as the initiator.
Disruptions were observed in 8% of the interactions, and communication-related issues contributed to 44% of all observed disruptions, with interruptions and environmental hazards being other key causes.
Acknowledgments:
This study was supported by a grant from the Agency for Healthcare Research and Quality (RO3 HS026809), Human Factors Considerations in the Design and Implementation of Telemedicine-Integrated Ambulance-Based Environments for Stroke Care. The authors would like to extend a special acknowledgement to Suparna Qanungo and Jessica Hewitt and Dale Hewitt for their contributions with data collection. The authors would also like to acknowledge Seyedmohammad Ahmadshahi and Rutali Joshi for their assistance in developing the figures for this article.
Biographies
Anjali Joseph is the Endowed Chair in Architecture + Health Design and Research at Clemson University, the Director of the Center for Health Facilities Design & Testing, a Professor in the School of Architecture and the Department of Industrial Engineering. She received a Ph.D. in Architecture, Culture and Behavior from the Georgia Institute of Technology in 2006.
Kapil Chalil Madathil is the Wilfred P. and Helen S. Tiencken Assistant Professor of Civil & Industrial Engineering, the Director of the Center for Workforce Development, and the Director of the Human-Systems Integration Lab at Clemson University. He earned a Ph.D. in Industrial Engineering from Clemson University in 2013.
Roxana Jafarifiroozabadi is a Ph.D. Candidate in Planning, Design and the Built Environment at Clemson University. She is also a researcher in the Center for Health Facilities and Design and Testing at Clemson University. She received a Master’s Degree in Architecture from the University of Tehran in 2016.
Hunter Rogers is a Research General Engineer with the Collaborative Interfaces and Teaming branch of the 711th Human Performance Wing of the Air Force Research Lab at Wright Patterson Air Force base in Dayton, OH. She obtained her Ph.D. in Industrial Engineering from Clemson University in 2020.
Sahar Mihandoust is a Research Assistant Professor for the Center for Health Facilities Design and Testing at Clemson University. She obtained her Ph.D. in Environmental and Interior Design from Texas Tech University in 2019.
Amro Khasawneh is a Post-Doctoral Fellow at the Armstrong Institute for Patient Safety and Quality at Johns Hopkins School of Medicine. He received his Ph.D. in Industrial Engineering from Clemson University in 2019.
Nathan McNeese is an Assistant Professor and the Director of the Team Research Analytics in Computational Environments Research Group within the Division of Human-Centered Computing in the School of Computing at Clemson University. He received his PhD in Information Sciences & Technology with a focus on Team Decision Making, Cognition, and Computer Supported Collaborative Work from The Pennsylvania State University in 2014.
Christine Holmstedt is a Medical Director of Stroke Clinical Services, a Co-Director of the Comprehensive Stroke and Cerebrovascular Center, and a Professor of Neurology in the College of Medicine at the Medical University of South Carolina. She received a Doctorate of Osteopathic Medicine from the Lake Erie College of Osteopathic Medicine in 2005.
James T. McElligott is the Medical Director of the Center for TeleHealth and an Associate Professor of Pediatrics at the Medical University of South Carolina. He earned a Doctorate of Medicine from the Wake Forest School of Medicine, Wake Forest Baptist Medical Center.
References Cited
- Alsabri M, Boudi Z, Lauque D, Roger DD, Whelan JS, Östlundh L, … Bellou A (2020). Impact of Teamwork and Communication Training Interventions on Safety Culture and Patient Safety in Emergency Departments: A Systematic Review. Journal of Patient Safety, Publish Ahead of Print. doi: 10.1097/pts.0000000000000782 [DOI] [PubMed] [Google Scholar]
- Belt GH, Felberg RA, Rubin J, & Halperin JJ (2016). In-transit telemedicine speeds ischemic stroke treatment: preliminary results. Stroke, 47(9), 2413–2415. [DOI] [PubMed] [Google Scholar]
- Buljac-Samardzic M, Dekker-van Doorn CM, van Wijngaarden JD, & van Wijk KP (2010). Interventions to improve team effectiveness: a systematic review. Health Policy, 94(3), 183–195. [DOI] [PubMed] [Google Scholar]
- Butler L, Whitfill T, Wong AH, Gawel M, Crispino L, & Auerbach M (2019). The impact of telemedicine on teamwork and workload in pediatric resuscitation: a simulation-based, randomized controlled study. Telemedicine and e-Health, 25(3), 205–212. [DOI] [PubMed] [Google Scholar]
- Cooke NJ, Gorman JC, Myers CW, & Duran JL (2013). Interactive team cognition. Cognitive Science, 37(2), 255–285. [DOI] [PubMed] [Google Scholar]
- Demir M, McNeese NJ, & Cooke NJ (2017). Team situation awareness within the context of human-autonomy teaming. Cognitive Systems Research, 46, 3–12. [Google Scholar]
- Dietz AS, Pronovost PJ, Mendez-Tellez PA, Wyskiel R, Marsteller JA, Thompson DA, & Rosen MA (2014). A systematic review of teamwork in the intensive care unit: what do we know about teamwork, team tasks, and improvement strategies? Journal of Critical Care, 29(6), 908–914. [DOI] [PubMed] [Google Scholar]
- Gorman JC, Cooke NJ, & Amazeen PG (2010). Training adaptive teams. Human Factors, 52(2), 295–307. [DOI] [PubMed] [Google Scholar]
- Gorman JC, Cooke NJ, & Winner JL (2006). Measuring team situation awareness in decentralized command and control environments. Ergonomics, 49(12-13), 1312–1325. [DOI] [PubMed] [Google Scholar]
- Gorman JC, Grimm DA, Stevens RH, Galloway T, Willemsen-Dunlap AM, & Halpin DJ (2020). Measuring real-time team cognition during team training. Human Factors, 62(5), 825–860. [DOI] [PubMed] [Google Scholar]
- Gorman JC, Martin MJ, Dunbar TA, Stevens RH, & Galloway T (2013). Analysis of semantic content and its relation to team neurophysiology during submarine crew training. Paper presented at the International Conference on Augmented Cognition. [Google Scholar]
- Grimm DA, Gorman JC, Stevens RH, Galloway TL, Willemsen-Dunlap AM, & Halpin DJ (2017). Demonstration of a method for real-time detection of anomalies in team communication. Paper presented at the Proceedings of the Human Factors and Ergonomics Society Annual Meeting. [Google Scholar]
- Healey A, Sevdalis N, & Vincent C (2006). Measuring intra-operative interference from distraction and interruption observed in the operating theatre. Ergonomics, 49(5-6), 589–604. [DOI] [PubMed] [Google Scholar]
- Hughes AM, Gregory ME, Joseph DL, Sonesh SC, Marlow SL, Lacerenza CN, … & Salas E (2016). Saving lives: A meta-analysis of team training in healthcare. Journal of Applied Psychology, 101(9), 1266. [DOI] [PubMed] [Google Scholar]
- Kwah LK, & Diong J (2014). National institutes of health stroke scale (nihss). Journal of Physiotherapy. [DOI] [PubMed] [Google Scholar]
- Lazzara EH, Benishek LE, Patzer B, Gregory ME, Hughes AM, Heyne K, … Schulman C (2015). Utilizing telemedicine in the trauma intensive care unit: Does it impact teamwork? Telemedicine and e-Health, 21(8), 670–676. [DOI] [PubMed] [Google Scholar]
- Liman TG, Winter B, Waldschmidt C, Zerbe N, Hufnagl P, Audebert HJ, & Endres M (2012). Telestroke ambulances in prehospital stroke management: concept and pilot feasibility study. Stroke, 43(8), 2086–2090. [DOI] [PubMed] [Google Scholar]
- Lin F, Chaboyer W, & Wallis M (2009). A literature review of organisational, individual and teamwork factors contributing to the ICU discharge process. Australian Critical Care, 22(1), 29–43. [DOI] [PubMed] [Google Scholar]
- Marlow SL, Lacerenza CN, & Salas E (2017). Communication in virtual teams: A conceptual framework and research agenda. Human Resource Management Review, 27(4), 575–589. [Google Scholar]
- Marlow SL, Lacerenza CN, Paoletti J, Burke CS, & Salas E (2018). Does team communication represent a one-size-fits-all approach?: A meta-analysis of team communication and performance. Organizational Behavior and Human Decision Processes, 144, 145–170. [Google Scholar]
- McNeese MD (2016). The phenomenal basis of human factors: Situating and distributing cognition within real-world activities. Journal of Cognitive Engineering and Decision Making, 10(2), 116–119. [Google Scholar]
- Mills P, Neily J, & Dunn E (2008). Teamwork and communication in surgical teams: implications for patient safety. Journal of the American College of Surgeons, 206(1), 107–112. [DOI] [PubMed] [Google Scholar]
- Mohammed S, Ferzandi L, & Hamilton K (2010). Metaphor no more: A 15-year review of the team mental model construct. Journal of Management, 36(4), 876–910. [Google Scholar]
- Palmer G, Abernathy J, Swinton G, Allison D, Greenstein J, Shappell S, … Reeves ST (2013). Realizing Improved Patient Care through Human-centered Operating Room Design. Anesthesiology, 1–12. [DOI] [PubMed] [Google Scholar]
- Pappas Y, & Seale C (2009). The opening phase of telemedicine consultations: an analysis of interaction. Social Science & Medicine, 68(7), 1229–1237. [DOI] [PubMed] [Google Scholar]
- Pavlopoulos S, Kyriacou E, Berler A, Dembeyiotis S, & Koutsouris D (1998). A novel emergency telemedicine system based on wireless communication technology-AMBULANCE. IEEE Transactions on Information Technology in Biomedicine, 2(4), 261–267. [DOI] [PubMed] [Google Scholar]
- Rogers H, Chalil Madathil K, Joseph A, McNeese N, Holmstedt C, Holden RJ, & McElligott JT (2020). Task, Usability, and Error Analyses of Ambulance-based Telemedicine for Stroke Care. IISE Transactions on Healthcare Systems Engineering. [Google Scholar]
- Rogers H, Madathil KC, Agnisarman S, Narasimha S, Ashok A, Nair A, … McElligott J (2017). A systematic review of the implementation challenges of telemedicine systems in ambulances. Telemedicine and e-Health, 23(9), 707–717. [DOI] [PubMed] [Google Scholar]
- Rogers H, Ponathil A, Chalil Madathil K, Joseph A, McNeese N, Holmstedt C, & McElligott J (2020). Evaluation and Prediction of Human Error in Ambulance-Based Telemedicine Stroke Assessment. . Paper presented at the Human Factors and Ergonomics Society Annual Meeting, Virtual. [Google Scholar]
- Sacks GD, Shannon EM, Dawes AJ, Rollo JC, Nguyen DK, Russell MM, … & Maggard-Gibbons MA (2015). Teamwork, communication and safety climate: a systematic review of interventions to improve surgical culture. BMJ Quality & Safety, 24(7), 458–467. [DOI] [PubMed] [Google Scholar]
- Salas E, Cooke NJ, & Rosen MA (2008). On teams, teamwork, and team performance: Discoveries and developments. Human Factors, 50(3), 540–547. [DOI] [PubMed] [Google Scholar]
- Salas EE, & Fiore SM (2004). Team cognition: Understanding the factors that drive process and performance. American Psychological Association. [Google Scholar]
- Salas E, Reyes DL, & McDaniel SH (2018). The science of teamwork: Progress, reflections, and the road ahead. American Psychologist, 73(4), 593. [DOI] [PubMed] [Google Scholar]
- Saver JL (2006). Time is brain—quantified. Stroke, 37(1), 263–266. [DOI] [PubMed] [Google Scholar]
- Saver JL, Fonarow GC, Smith EE, Reeves MJ, Grau-Sepulveda MV, Pan W, … Schwamm LH (2013). Time to treatment with intravenous tissue plasminogen activator and outcome from acute ischemic stroke. JAMA, 309(23), 2480–2488. [DOI] [PubMed] [Google Scholar]
- Schmutz JB, & Eppich WJ (2019). When I say… team reflexivity. Medical Education, 53(6), 545–546. [DOI] [PubMed] [Google Scholar]
- Tiferes J, Hussein AA, Bisantz A, Higginbotham DJ, Sharif M, Kozlowski J, … Guru K (2019). Are gestures worth a thousand words? Verbal and nonverbal communication during robot-assisted surgery. Applied Ergonomics, 78, 251–262. [DOI] [PubMed] [Google Scholar]
- Wahr JA, Prager RL, Abernathy J III, Martinez EA, Salas E, Seifert PC, … Sundt TM III (2013). Patient safety in the cardiac operating room: human factors and teamwork: a scientific statement from the American Heart Association. Circulation, 128(10), 1139–1169. [DOI] [PubMed] [Google Scholar]
- Weaver SJ, Dy SM, & Rosen MA (2014). Team-training in healthcare: a narrative synthesis of the literature. BMJ Quality & Safety, 23(5), 359–372. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Whitten PS, & Allen A (1995). Analysis of telemedicine from an organizational perspective. Telemedicine Journal, 1(3), 203–213. [DOI] [PubMed] [Google Scholar]
- Yperzeele L, Van Hooff R-J, De Smedt A, Espinoza AV, Van Dyck R, Van de Casseye R, … De Keyser J (2014). Feasibility of AmbulanCe-Based Telemedicine (FACT) study: safety, feasibility and reliability of third generation in-ambulance telemedicine. PloS one, 9(10), e110043. [DOI] [PMC free article] [PubMed] [Google Scholar]