Feasibility and Acceptability of a Model Disaster Teleconsultation System for Regional Disaster Health Response

Abstract

Introduction:

The federally funded Region 1 Regional Disaster Health Response System (RDHRS) and the American Burn Association partnered to develop a model regional disaster teleconsultation system within a Medical Emergency Operations Center (MEOC) to support triage and specialty consultation during a no-notice mass casualty incident. Our objective was to test the acceptability and feasibility of a prototype model system in simulated disasters as proof of concept.

Methods:

We conducted a mixed-methods simulation study using the Technology Acceptance Model framework. Participating physicians completed the Telehealth Usability Questionnaire (TUQ) and semistructured interviews after simulations.

Results:

TUQ item scores rating the model system were highest for usefulness and satisfaction, and lowest for interaction quality and reliability.

Conclusions:

We found high model acceptance, but desire for a simpler, more reliable technology interface with better audiovisual quality for low-frequency, high-stakes use. Future work will emphasize technology interface quality and reliability, automate coordinator roles, and field test the model system.

Introduction

Disasters often lead to surging demand for medical care, overwhelming emergency systems with damage, loss of life, and deterioration of services. Local medical response may be impeded by limited access to specialist personnel, resource shortages, or other constraints. Mobile technology advances have enabled telehealth integration into disaster preparedness, response, and recovery.^1–4 Promising disaster applications include patient triage and stabilization,⁵ supporting bedside clinicians with expert teleconsultation,⁶ expanding patient access to care when local services are disrupted,^7–9 and just-in-time dissemination of education to frontline clinicians.¹⁰ Despite the potential to transform disaster medical response, telehealth has been underutilized partly due to administrative, legal, and regulatory barriers^11,12; lack of simple, rapidly deployable, interoperable technology; dependence on functional telecommunications infrastructure; and limited reimbursement.^13–15

The Pandemic and All Hazards Preparedness Reauthorization Act directed the Office of the Assistant Secretary of Preparedness and Response (ASPR) to launch the Regional Disaster Health Response System (RDHRS) program in 2018.¹⁶ One key RDHRS goal is to develop telehealth solutions for disasters, including rapid access to medical expertise. While many health systems launched or expanded telehealth services during the SARS-CoV-2 pandemic,¹⁷ our understanding of how to leverage a regional telehealth system within hours after a no-notice mass casualty incident (MCI) remains limited.^18,19

The Region 1 RDHRS program partnered with the American Burn Association to develop a model regional disaster teleconsultation system using a phased approach and a strategic development framework with stakeholder-led design.²⁰ In this model, the teleconsultation system operates within a regional Medical Emergency Operations Center (MEOC) to support new delivery of temporary, just-in-time telehealth services for triage and specialty consultation during an MCI. We envisioned the operational system needed in this circumstance as easy to use without prior training, flexible to support various scenarios, and functional within hours on existing devices and telecommunication networks. In the first system development phase, our objective was to test the acceptability and feasibility of using a prototype model system in simulated disasters.

Methods

STUDY DESIGN AND SETTING

We conducted a pilot simulation study using sequential explanatory mixed methods to capture provider ratings and perspectives after using the prototype model system to deliver care to high-fidelity patient manikins in MCI scenarios. We conducted three simulation sessions in July and August 2019. The final simulation session occurred during a larger multi-institutional functional exercise demonstrating overall RDHRS operations, including disaster teleconsultation service delivery. The simulations involved three academic medical centers—the regional MEOC was located at one site, while simulation centers at the other two sites served as community emergency departments (ED). This study was deemed exempt by Institutional Review Boards at each site.

PARTICIPANT ELIGIBILITY AND RECRUITMENT

We targeted physicians as end users to minimize confounding by heterogeneity of clinical experience and training. Board-certified emergency physicians practicing in community hospital settings in Massachusetts were eligible for inclusion as bedside physicians. Board-certified subspecialist physicians in trauma surgery, burn surgery, pediatric emergency medicine, and infectious disease practicing in Massachusetts RDHRS partner hospitals were eligible for inclusion as RDHRS teleconsultants. Participants were recruited as a convenience sample by e-mail to hospital physician distribution lists. Participants were protected from clinical responsibilities during simulation. Participation was voluntary and remunerated.

PROTOTYPE MODEL SYSTEM, PLATFORM, AND TRAINING

We wanted to know if we could rapidly deploy a telehealth platform that was in everyday clinical use in one health system to connect to outside providers in another health system(s) to support care delivery during a disaster. We reasoned that a no-notice MCI would not allow time to adapt or create new telehealth platform configurations or train new users. In the prototype model system, bedside physicians were connected to a pool of remote medical specialists (RDHRS teleconsultants) through a trained telehealth coordinator at the regional MEOC (Fig. 1). We modeled the telehealth coordinator role on similar roles in other emergency call centers (e.g., 9-1-1 systems and U.S. Poison Control Centers), where trained personnel triage callers, connect them to the appropriate expertise, and provide real-time instruction if needed. The telehealth coordinator connected the bedside physician with relevant RDHRS teleconsultants and guided all physicians with just-in-time platform use.

Fig. 1.

The prototype RDHRS disaster telehealth system workflow for connecting bedside clinicians with remote disaster specialists through a trained telehealth coordinator. (1) A bedside clinician caring for disaster victims contacts the regional MEOC to request disaster specialist consultation. (2) A telehealth coordinator receives the consult request, and guides the bedside clinician through telehealth application installation, platform access, and entry into a virtual room. (3) The telehealth coordinator then triages the requested consult to an available RDHRS teleconsultant from a regional or national pool of preregistered, licensed volunteer medical specialists with relevant expertise. (4) Finally, the telehealth coordinator directs the RDHRS teleconsultant into the virtual room with the bedside clinician to initiate the video consultation. MEOC, medical emergency operations center; RDHRS, Regional Disaster Health Response System.

To test if an existing telehealth system could feasibly be expanded in real-time to support out-of-network hospitals during a disaster, we chose a downloadable application (SBR Health, Inc.) that could function on any device with camera and microphone capability and was in clinical use at one of our institutions as the teleconsultation platform. All physicians received just-in-time instructions only without prior training.

SIMULATION PROTOCOL

The primary study objective was to examine user acceptance of a new technologic system to support disaster care delivery. We designed four simulated disaster patient cases (available on request). Three cases emphasized burn, trauma, or pediatric emergency care using a burn-blast MCI scenario. In this proof-of-concept stage, we also included a fourth case using a high-consequence infectious disease (HCID) scenario. Each case was designed to trigger the bedside physician to seek expert recommendations related to burn, trauma, pediatric emergency, or HCID care. All cases were standardized with preprogrammed patient vital signs, physical exam findings, clinical deterioration, and response to interventions.

The simulation team composition and protocol were identical at each site simulating a community ED setting. Each team included two technicians, a facilitator/observer, and a standardized “nurse” actor. Each simulation session started with team introductions, ground rules, and orientation to the simulation environment. The “nurse” actor provided scripted prompts and assisted the bedside physician with care delivery. The facilitator/observer recorded field observations and time intervals for the teleconsultation encounter.

Before each case, bedside physicians reviewed a description of the community ED setting and disaster event (MCI or HCID), and a simulated patient medical record. The bedside physician was then introduced to the simulated patient by the “nurse” actor who provided scripted details of prehospital care, nursing triage, and patient assessment. The bedside physician was informed that a surge of patients had overwhelmed local service capabilities, and disaster teleconsultation services assisting with specialty patient care, triage, and transfer decisions were available through a regional MEOC. In each case, the bedside physician performed initial assessment and management until specialty expertise was indicated (e.g., for emergent escharotomy). The bedside physician was directed to a tablet device with written just-in-time instructions for how to contact the MEOC, download, and use the teleconsultation platform application. Bedside physicians participated in all four simulation cases. RDHRS teleconsultants participated in the single simulation case related to their expertise.

DATA COLLECTION

After completing all assigned simulation case(s), each participant provided demographic information and rated the system on the Telehealth Usability Questionnaire (TUQ) before individual interviews. The TUQ assesses six system usability domains (usefulness, ease of use, interface quality, interaction quality, reliability, and satisfaction) with 21 items on 7-point Likert scales.²¹ The total usability score—the average rating for all TUQ items (maximum possible score = 7)—measured acceptability. Interface quality, interaction quality, and reliability domain items measured feasibility. As an additional feasibility measure, we recorded call time intervals, time-to-connection, and total call duration. Data were collected and stored in the REDCap (Research Electronic Data Capture) software platform.²²

The qualitative interview was designed to add context to survey findings. The semistructured interview guide focused on overall impressions of the prototype model system, the teleconsultation platform, just-in-time training tools, and the telehealth coordinator role, exploring successes and challenges for each. Brief interviews (15–30 min) conducted by one of two study team members (E.G., T.P.B.) were audiorecorded with consent.

ANALYSIS

We used standard qualitative methods to code interview transcripts. Interview audiorecordings were professionally transcribed verbatim for analysis and deidentified. Three investigators (M.-L.D., T.P.B., S.L.) reviewed two transcripts line by line to develop the initial coding scheme. Next, they independently coded two new transcripts incorporating and refining initial conceptual categories, adding new categories until consensus was reached. Two investigators (S.L., L.S.) then coded each remaining transcript independently, and met to review coding discrepancies and finalize coding decisions. Themes emerging from the inductive coding process were mapped to the Technology Acceptance Model framework²³ to understand factors influencing physician acceptance of teleconsultation in disaster settings. In the Technology Acceptance Model, a provider's perception of system usefulness (system use enhances/diminishes care delivery) and ease of use (technology is easy/difficult to use) influences their attitude (positive/negative feeling) toward the technology and behavioral intention (willingness) to use it. Behavioral intention measures acceptance and predicts technology use.

We used NVivo v12 software²⁴ for qualitative coding and analysis, and SAS v9.4 software²⁵ to calculate standard descriptive statistics for the TUQ.

Results

Table 1 details participant demographics. Four emergency physicians participated as bedside clinicians, and 11 specialists as RDHRS teleconsultants. Overall, 66% had ≥10 years of practice and 93% had prior simulation exposure. Half (53%) had used telemedicine in clinical practice, but none in a disaster. The mean total usability score was 5.2 (standard deviation [SD] 1.2). TUQ item scores were highest for usefulness and satisfaction, and lowest for interaction quality and reliability (Table 2). Mean time to connection was 5.1 min (SD 1.4) and total call duration was 17.6 min (SD 3.1). Aspects of care discussed by emergency physicians and RDHRS teleconsultants during disaster simulation are summarized in Supplementary Table S1.

Table 1.

Participant Demographics (n = 15)

DEMOGRAPHICS	n/N (%)
Male	12/15 (80)
Age in years, median (IQR)	46 (36, 62)
Physician type
Emergency medicine physician	4/15 (27)
Burn surgeon	3/15 (20)
Trauma surgeon	4/15 (27)
Pediatric emergency physician	2/15 (13)
Infectious disease physician	2/15 (13)
Years of clinical experience
<5 years	2/15 (13)
5–9 years	3/15 (20)
10–19 years	4/15 (27)
≥20 years	6/15 (40)
Primary practice setting
Academic tertiary care hospital	11/15 (73)
Community hospital	3/15 (20)
Critical access hospital	1/15 (7)
Prior simulation experience
Task trainer	12/15 (80)
Manikin based	14/15 (93)
Standardized patient	14/15 (93)
Virtual reality	5/15 (33)
None	1/15 (7)
Prior telemedicine experience
Routine care setting	8/15 (53)
Disaster setting	0/15 (0)

IQR, interquartile range.

Table 2.

Telehealth Usability Questionnaire Item Mean Scores (n = 15)

DOMAIN	ITEM	MEAN SCORE (SD)
Usefulness	Telemedicine improves my ability to give/receive consultation in a disaster	6.7 (0.6)
	Telemedicine saves me time when communicating with a clinician	4.5 (2.3)
	Telemedicine increases my access to patients/clinicians	5.9 (1.5)
Ease of use	It was simple to use the system	5.4 (1.6)
	It was easy to learn to use the system	5.9 (1.2)
	I could become productive quickly using the system	6.2 (0.9)
Interface quality	The way I interact with the system is pleasant	5.4 (2.0)
	I like using the system	5.5 (1.8)
	The system is simple and easy to understand	5.5 (1.7)
	The system does everything I would want it to be able to do	4.7 (1.9)
Interaction quality	I could talk to the other physician easily	4.5 (2.2)
	I could hear the other clinician easily	4.3 (1.9)
	I felt I was able to express myself effectively	5.1 (1.9)
	I could see the other clinician as well as in person	4.3 (1.8)
Reliability	I think teleconsultations are the same as in-person consultations	4.1 (1.6)
	When I made a mistake, I could recover easily and quicklya	5.2 (1.6)
	The user tip sheets told me how to fix problemsb	5.1 (2.1)
Satisfaction	I feel comfortable communicating with clinicians using this system	5.7 (1.4)
	Telemedicine is an acceptable way to give/receive virtual care consultation	5.9 (1.4)
	I would use this/similar platform again	6.2 (1.0)
	Overall, I am satisfied with the platform	5.6 (1.8)
	Total usability score	5.2 (1.2)

Participants rated agreement with each item on a 7-point Likert scale (1 = disagree, 7 = agree). The total usability score is the average for all items (maximum score = 7).

^a Cell represents n = 13 responses.

^b Cell represents n = 14 responses.

SD, standard deviation.

Two emergency physicians and eight specialists completed interviews. Table 3 details key themes with representative quotes. Overall, physicians who perceived disaster teleconsultation could enhance their ability to provide specialty care. Video enhanced peer-to-peer communication but had the potential to distract from the patient interaction. Some believed audio-only connections could suffice if bandwidth was limited. However, surgeons found video necessary for procedural guidance. Physicians perceived the prototype model system as simple and easy to learn. Most found mobile devices eased care delivery but recommended additional hands-free capability. Despite technical limitations, most expressed positive attitudes, and future intent to use the prototype model system. Recommendations for future use included clarifying the telecoordinator role, providing estimated wait times, and improving triage efficiency and video quality.

Table 3.

Key Themes and Representative Quotes from Physicians Regarding Perceived Usefulness and Ease of Use, Attitudes, Behavioral Intention to Use, and Future Recommendations for Using the Prototype Model System for Teleconsultation in Disaster Settings

THEME	REPRESENTATIVE QUOTES FROM PHYSICIANS
Usefulness
Access to teleconsultants can enhance bedside physician capabilities	“…the framework has enormous capability to add to my abilities, it's just great to have back up and expertise available” “…how quickly you connected with a consultant in something that is not your specialty, and you didn't have locally available was huge” “It would make me feel a little more comfortable that I wasn't grossly mis-stepping…in territory that I'm not familiar with.” “it's great to have that wider network available” “I found it most useful in the burn case out of all the cases because of the procedural piece of it.”
Video can enhance peer-to-peer interaction	“I was almost equally comfortable with the consultation being by telemedicine than in person.” “…the information was communicated very well back and forth” “it did seem really easy to communicate” “I'm not sure the best way of how to show the person on the other end of the line what's going on.” “Overall, it worked pretty well, and I felt like I was able to interact with the consultants and ask questions”
Video can interfere with bedside provider-to-patient interaction	“Technology treats my patient more as the museum objects to be displayed rather than interacting well with my patient” “I'm focusing more on the consultant than I am with what's going on with my patient” “…a little distracting from the clinical work” “Sometimes there's things that you want to say but don't want to panic the patient”
Video was necessary for remote procedural guidance	“I was instructing my colleague to actually go deeper with the escharotomy, and that sort of stuff can only be done with the video, moving images.” “…the images [are] key” “…quality was perfectly acceptable to do anything I need to do as a Senior Burn Surgeon shy of remotely operating instruments with telesurgery” “…the video was essential”
Ease of use
System was simple and easy to learn	“One of the challenges of these devices and interfaces is that you don't use it all the time, that you don't have the familiarity with it in a crisis situation” “It was very easy and intuitive to use” “…felt remarkably smooth given that complexity” “Reconnecting was easy” “there's a lot of things that, in a disaster situation, are not things that should be taking cognitive load” “It seemed like there was an enormous amount of complexity on the backend, which was not visible to me as frontline provider which is good. It's great that those aren't things I need to worry about so much”
Mobile, hands-free interface is preferred	“I like the mobility” “…having the ability to move around the different angles of the patient” “For extended conversations, it'd be nice to be able to do something else with my hands. Because if I'm holding it, I'm also focusing more on the consultant than I am with what's going on with my patient in case they start to decompensate” “there's a question about what you do for protective equipment and contamination, especially in a room [with] severe contact precautions” “The small size of the iPad itself, I don't know if that's considered as both a blessing and a curse”
Attitudes and behavioral intention to use
User attitudes and future intention to use	“I think it's critical and compulsory. I think that we have nowhere to turn, but to harness this.” “I think this is wonderful and I'd be an enthusiastic participant” “…I was actually pleasantly surprised.”
Recommendations for future use
Telehealth coordinator role needs clarification	“I'm unsure how much information to give to the coordinator” “More of an introduction, explain the roles, like I'm going to connect you with a consultant” “…a little bit more triage ability and knowing sort of the best way to use the resources would be helpful”
Estimating wait times could improve efficiency	“…if it's possible to have an estimate of how long you're going to be waiting for that. I could see that being a stressor in real time, in real life. Do you sit by the iPad or do you go see your next three patients?” “…some kind of estimate on time…”
Bandwidth sparing strategies are needed	“…bandwidth being a lot smaller and a lot more limiting. So again, having the option of just talking on a phone might be a way to free up some of that bandwidth when you don't need the video piece.” “…losing the video images and then going to still images and then losing the still images and still having the audio in the same platform, so to have some built-in graceful degradation would be very sensible” “I hope…under…austere conditions of a mass casualty with restriction in communications, that the bandwidth issue is not going to make this degree of quality not doable”
Interface audio/video quality should be improved	“It would be better to have higher resolution imaging capability” “I was only hearing about every, probably 50% of the words. The audio was in and out” “There were a couple times that I couldn't understand the physician” “It would be so great if there was some way to split the screen so that I see the monitor throughout the assessments.”

Discussion

This pilot simulation study demonstrates the potential utility of a regional disaster teleconsultation system to rapidly access specialists during the acute response to no-notice MCIs. We found high model acceptance, but need for a simpler, more reliable technology interface with better audiovisual quality to support high stakes use without prior training. The qualitative analysis provided insight about how disaster teleconsultation could affect interactions between bedside physicians and remote specialists. Overall user attitudes were positive, with most finding teleconsultation useful to access limited expert resources. However, teleconsultation also altered provider efficiency and the physician–patient interaction at times by emphasizing the teleconsultant.

Regional telehealth systems have demonstrated benefit in civilian and military settings,^6–8,19 but no-notice disaster application has lagged. In a computer simulation of an earthquake scenario, a regional telemedicine hub strategy had the potential to improve resource utilization efficiency, enhance surge capacity, and reduce time-dependent mortality.¹⁸ We identified key provider, technology, and system-level insights for future development by allowing participants to experience a prototype model system during simulated disaster care delivery.

First, physicians perceived video as enhancing communication between providers but noted the potential to distract from patient care, a finding observed in other emergency settings.²⁶ Training teleconsultants in best practice communication strategies and using simple, intuitive platforms with familiar workflows and devices could mitigate this effect. Second, some felt video was most beneficial for procedural guidance (e.g., a remote burn surgeon guiding a bedside clinician through an escharotomy) or when visualization of the patient, equipment, devices, or environment could impact or alter recommendations (e.g., assisting a bedside provider to adjust ventilator settings in real time). Selective application of store-and-forward technologies (e.g., email with a digital image) and dedicated two-way voice-only communication could be bandwidth-sparing strategies.

Third, although it was feasible to expand an existing telehealth system with a downloadable application, workflows configured for routine use in ambulatory settings did not adapt well to disaster use in emergency settings. Workflows adapted from teleconsultation systems providing acute episodic care, like telecritical care or telestroke, may be more suitable. Installation requirements may also limit adoption by stressed providers. Finally, the telehealth coordinator role supported accurate triage and just-in-time technology use but was rate limiting.

To increase throughput, a future model system could automate core telehealth coordinator functions (triage, expertise selection, instruction) and trial self-guided web-based applications without installation requirements. Logic rules that automatically sort incoming consults by time and acuity can expedite triage. However, secondary triage by skilled expert personnel may still be necessary to prevent over or under triage. Expert selection can also be automated by providing a menu of available teleconsultants to requesting clinicians. However, scenarios where patient care requires multiple experts or multidisciplinary input may be complex to automate reliably. Finally, a disaster telehealth platform that is simple, self-guided, and uses familiar clinical workflows will facilitate just-in-time use without instruction or training. For example, a platform that allows a requesting clinician to submit an unscheduled consult request and receive a “call-back” from an expert teleconsultant would follow anticipated workflows in emergency settings.

We anticipate real-world use of a regional disaster telehealth system with and without personnel in the telehealth coordinator role. When triage is primarily based on time and acuity, simple logic rules would be most efficient in high-volume settings. However, in some events, secondary triage by an individual or team with specific expertise may be required to ensure appropriate utilization of limited resources (e.g., transfer decisions based on regional burn center capacity). Thus, an optimal system would automate core telehealth coordinator functions but also adapt to more complex scenarios by layering in trained personnel when necessary.

This study has limitations. First, convenience sampling and small sample size are subject to selection bias. Second, we tested a single technology platform and recruited physicians only, which could limit generalizability. However, we found high model acceptance despite technical limitations, and our qualitative analysis focused on broad themes that could influence acceptance of a future system regardless of the specific technology used. Future research should examine user acceptance from diverse multidisciplinary groups, including physician extenders and nonphysicians. Third, this simulation study was designed to examine user acceptance and did not test other aspects of system function, such as load balancing capability or the impact of telecommunication network degradation. Such testing will be necessary to advance the prototype system design. Additionally, simulation cannot fully mimic complex real-world use so we may be overestimating user acceptance. Field tests of future model systems will be necessary to understand operational complexities, such as staffing and readiness to provide disaster teleconsultation services on-demand after hours. Last, we did not examine system efficacy or effectiveness in the development phase.

This study was completed before the SARS-CoV-2 pandemic began. Ultimately, legal and regulatory waivers were needed to expand telehealth for pandemic response.²⁷ Maintaining 24/7 readiness and the ability to operate a regional disaster teleconsultation system during a no-notice event will require preverified provider registries; integration with hospital, state, and federal emergency systems; and mechanisms to navigate legal and regulatory barriers.^1,14 Developing a regional disaster teleconsultation system with high provider acceptance that is reliable, intuitive to use without any training, and accessible across networks is a key first step. In future phases, we will improve technology interface quality and reliability, automate triage and coordinator roles, explore staffing strategies, and field test the next model system iteration.

Authors' Contributions

T.P.B.: conceptualization; methodology; investigation; qualitative coding and analysis; and writing—original draft. S.L.: qualitative coding and analysis; and writing—original draft. D.M.: conceptualization; methodology; investigation; and writing—review and editing. J.N.D.: data curation; formal analysis; and writing—review and editing. M.-L.D.: methodology; qualitative coding and analysis; and writing—review and editing.

ML.: project administration; resources; and writing—review and editing. R.T.B.: project administration; and writing—review and editing. L.S.: qualitative coding and analysis; and writing—review and editing. K.M.: resources; and writing—review and editing. P.D.B.: conceptualization; acquisition of funding; and writing—review and editing. E.G.: conceptualization; methodology; investigation; writing—original draft; and supervision.

Disclaimer

The contents herein are solely the authors' responsibility and do not necessarily represent the official views of ASPR or National Heart, Lung, and Blood Institute (NHLBI).

Disclosure Statement

No competing financial interests exist.

Funding Information

This work was funded by the ASPR through grant no. HITEP180042-01-00. T.P.B. is also funded by the NHLBI through grant no. 1K23HL145126-01A1.

Supplementary Material

Supplementary Table S1