Feasibility Assessment of a Pre-Hospital Automated Sensing Clinical Documentation System

Abstract

Clinical documentation in the pre-hospital setting is challenged by limited resources and fast-paced, high-acuity. Military and civilian medics are responsible for performing procedures and treatments to stabilize the patient, while transporting the injured to a trauma facility. Upon arrival, medics typically give a verbal report from memory or informal source of documentation such as a glove or piece of tape. The development of an automated documentation system would increase the accuracy and amount of information that is relayed to the receiving physicians. This paper discusses the 12-week deployment of an Automated Sensing Clinical Documentation (ASCD) system among the Nashville Fire Department EMS paramedics. The paper examines the data collection methods, operational challenges, and perceptions surrounding real-life deployment of the system. Our preliminary results suggest that the ASCD system is feasible for use in the pre-hospital setting, and it revealed several barriers and their solutions.

Introduction

Military and civilian medics are responsible for retrieving, stabilizing, and transporting the wounded and ill to a trauma facility¹. Accurately documenting medical care during transport is complicated for several reasons, including limited staff in the vehicle and ongoing care requirements for trauma and critically ill medical patients². Because it is not possible to manually document all activity during transport as it occurs, medics often first give a brief verbal report to the receiving facility staff, followed by documentation completed after the hand-off has occurred. The initial verbal report typically includes chief complaint, mechanism of injury, vital signs, and procedures performed³. This verbal report may be supported by brief notes written on the patient, a scrap piece of paper, the medic’s glove, or in some cases relying only on the medic’s memory. While accurate transmission of this information is essential for safe care transitions and plays a key role in the receiving team’s treatment plans, reporting may be incomplete^4,5 and include inaccuracies^3,6. As a result, communication, safety, and patient care may be impacted⁷.

Communicating some types of patient information can be done successfully with a verbal report alone (e.g., chief complaint, mechanism of injury, age, gender). However, specifics regarding the sequence of procedures performed, medication dosage and timing, and specific vital sign ranges are difficult to accurately recall from memory given the high-intensity setting of trauma care. This information is essential for optimal care management, resource allocation, and triage planning⁸.

An Automated Sensing for Clinical Documentation (ASCD) system leverages a combination of sensors to passively collect data, from which algorithms are used to create an abbreviated care record that describes the procedures performed. This record is designed to be generated in real time, or near real time, and transmitted to providers as a supplement to the verbal handoff. The goal of this system is to increase the accuracy and detail of clinical information transmitted to clinical providers and teams, particularly in high-acuity and trauma settings, without requiring the provider to actively produce the documentation. An ASCD system can be used in a range of environments from civilian patient transport, military patient transport, and operating rooms.

This paper reports on a feasibility study of an ASCD system for patient transport in a civilian metropolitan Emergency Medical Service (EMS). Specifically, this paper outlines the equipment used, the configuration of the equipment in a civilian ambulance, perception of medics wearing devices, data collection processes, and interfaces with the trauma facility. The system was deployed with the Nashville Fire Department (NFD) EMS in partnership with Vanderbilt University Medical Center, a level I trauma center in Nashville, Tennessee, which receives a high volume of acute trauma patients. Cameras were not deployed in the system during the pilot because of concerns regarding patient and provider privacy, although their use would be expected to increase system accuracy.

Background

High quality healthcare requires effective and accurate communication among providers, especially during transitions across care settings. The dynamic nature of care by medics in the pre-hospital setting can make it difficult to document procedures in real time and communicate vital clinical information to hospital providers. In their handoffs to emergency department (ED) staff, paramedics dedicate 75% of verbal reports to patient demographics and presenting signs/symptoms and ~7% to pre-hospital treatments⁹, even though pre-hospital treatments and clinical course largely drive resource allocation and treatments upon arrival¹⁰.

Various approaches have been considered to automatically document clinical care including speech-to-text and in-person/virtual scribes¹¹. However, given environmental noise, limited vehicle space, and the need for the medic to focus on patient care rather than documentation, alternative approaches to automated documentation systems are desired¹². Moreover, given the number of and fluctuation in transport vehicles and number of trips, ease of device acquisition and installation is important.

Off-the-shelf sensors offer an opportunity to gather the data necessary to produce documentation, while also being readily available and cost-effective. These sensors include electromyography (EMG) sensors, inertial measurement units (IMUs), such as accelerometers, and cameras. EMG sensors collect data that measure electrical currents during muscular contraction, which can be used to identify different neuromuscular activity. IMUs combine data from accelerometers, gyroscopes, and magnetometers to report the orientation and angular rate of a body. Cameras can track the medic and patient in the scene. These data feeds can be fed into machine learning algorithms for analysis and documentation generation.

For this pilot study, the NFD assisted in the evaluation of the system. NFD provides fire protection and emergency medical care for 533 square miles and transports patients to numerous hospitals within the metropolitan Nashville area. In 2018, NFD responded to approximately 130,000 calls. Of note, NFD EMS protocols for airway management do not include the use of medication for rapid sequence intubation.

Methods

An ASCD system for patient transport in a pre-hospital setting has many components. First, a series of sensors are deployed in a clinical environment, in this case EMG sensors and accelerometers measuring provider movement. Second, these data feeds are aggregated and analyzed by machine learning systems to detect the clinical activity that are performed. Third, from the detected activity, an abbreviated care record is produced that can be transmitted to upstream care providers. This paper describes the methods around the feasibility of deploying sensors on clinicians to collect data for analysis.

A range of sensors were considered for the feasibility test ranging from medic worn sensors, mounted cameras, to radio-frequency identification chips on each device and/or medication. Given the desire of a rapid, cost-effective, and simple deployment that protected patient privacy, sensors worn by medics were selected. Sensors worn by medics that were utilized in this study included Myo armbands, which collect EMG data and inertial measurement data (e.g., accelerometers), and Apple Watches, which collect yaw, pitch, roll, and acceleration data.

After obtaining institutional review board approval from the Vanderbilt University Human Subjects Protection Program, physicians, nurses, and paramedics were recruited to perform over 45 hours of procedures in Vanderbilt’s Center for Experimental Learning and Assessment (CELA), which is a high-fidelity simulation center. Each participant was consented and then asked to perform designated procedures multiple times over the duration of a three-hour period using a simulation patient. Data collected from the simulations were used to refine the sensors, data collection systems, and the algorithm prior to real-world deployment. These simulations included use of cameras.

The ASCD system was then deployed in partnership with NFD over a 12-week period in 2019, without cameras. Written informed consent was obtained from the participating paramedics. Paramedic shifts were selected based on the availability of paramedics and research staff. At the beginning of each shift, a trained researcher who is also a paramedic equipped one paramedic with sensors, specifically two Apple Watches and two Myo armbands. In order to collect and transmit the collected information, a laptop and two cell phones were also carried by the research observer (see Data Collection). The data were aggregated into a single server for future analysis.

During the NFD paramedic’s 12-hour shift, the trained researcher observed and recorded all clinical activity with a custom time-motion capture system the “start” and “stop” times for targeted procedures performed inside the ambulance (Table 1). These procedures were chosen based on focus groups with EMS personnel and common procedures performed in the ambulance¹³. Automated time-motion capture was not feasible given the subjectivity and complexity of the procedures. Procedures performed outside of the ambulance were excluded from the recorded observations due to the distance from the laptop and Bluetooth receivers to the Myo armbands and iPhones. The research observer did not participate in patient care.

Table 1:

Pre-hospital procedures assessed by the ASCD system and number of times observed

For the subset of patients transported to Vanderbilt University Medical Center (VUMC) adult emergency department (ED), observations also included the handoff between the team of paramedics and the ED team. Procedures and interventions performed during the ED visits were also documented. At the conclusion of each shift, the paramedic who wore the technology completed a debrief survey. These surveys featured a user-centered design approach, considered the context of use, specific requirements, and areas of design optimization¹⁴. Between each observation, the ASCD equipment was cleaned using SaniWipes^® and all components of the system were charged. No substantial damage was received to the equipment during use.

Data Collection

Transport of Equipment: The following equipment was housed in a Pelican case during transport: 2 Myo Armbands, 2 Apple Watches (Series 3), 2 Apple iPhone 7s, AT&T Unite Express 2 WiFi hotspot, 2013 Apple MacBook Air, and their accompanying chargers (Figure 1). The equipment was delivered at the beginning of each shift and kept with a member of the research team.

Figure 1:

Pelican case setup

Electromyography: Thalmic Labs Myo gesture control armbands were used as the source for EMG data collection. Myo armbands were connected to the MacBook Air using a Bluetooth connection. After the armbands were pulled onto the forearms, they were calibrated by having the paramedics position their hands as shown in Figure 2.

Figure 2:

Myo neutral position (A) to Myo sync position (B) to calibrate the armbands

Accelerometer: One Apple Watch Series 3 was worn on each wrist with the watch-face outward and was paired with an Apple iPhone 7. Both iPhones were stored in a secure cabinet in the rear of the ambulance. The iPhones were connected to a virtual private network (VPN), which was necessary to securely send data to a VUMC server. We originally used the PulseSecure application to access the VPN, but then the system migrated, causing the switch to F5 Access. To establish an internet connection, a AT&T Unite Express 2 WiFi hotspot was used.

Data Flow (Figure 3): Both the left and right Myo Armbands were connected to the laptop using a Bluetooth connection. The laptop was connected to the mobile hotspot over a VPN. The left and right Apple Watches were paired with a corresponding iPhone 7, which also used the hotspot WiFi.

Figure 3:

Flow of data inside the ambulance

Data Transfer: At the conclusion of each shift, the Myo data collection files were transferred from the laptop to a shared cloud account (VUMC Box) for review by the data analysts. Due to the large file sizes, the transfer was completed using VUMC WiFi instead of the hotspot.

Notification of VUMC research team: An application, Life360, was installed onto one of the iPhones. A geo-fence was set up around VUMC, which notified the clinical researcher at the VUMC ED when the phone was detected within 800 feet of the ambulance bay. In addition, the research observer would send a message via Life360 to the clinical researcher when enroute to VUMC. The clinical researcher met paramedics and the observer in the ED to document the handoff to hospital staff.

Documentation of Targeted Procedures: The research team thoroughly discussed the method for documenting the targeted procedures prior to deployment of the ASCD system. The research observer needed a quick and accurate system that allowed for the research observer to note any discrepancies in how the paramedics actually started and stopped each procedure. Variations in procedural equipment or protocols could cause the start and stop times to differ. Original discussions included methods such as (i) keeping field notes and manually keeping track of time, or (ii) creating an iPhone application that allowed the observer to press a start/stop button with pre-selected procedures. The most significant challenge with the former was accuracy of procedure times. For the latter, it was determined that an application with pre-selected options did not allow the observer much versatility to add other comments and raised concerns about accidental selection of a start or stop time. Ultimately, a simple Python logging application was used that allowed for free-text entries and recorded time stamps upon entry.

Research documentation began at the point in which the patient was loaded into the ambulance. During each transport, the observer typically sat in the captain’s or “airway” seat (rear facing), located behind the head of the patient, and documented from that position. Once the patient was loaded into the ambulance, the research observer opened the laptop, ensured proper connectivity of the system, and began recording procedures as they happened in real time. For example, if the paramedic was going to start an IV, the observer would type “IV start” into the log application when the paramedic touched the IV start kit. At the end of the procedure, indicated by taping down of the IV, the observer would type “IV end”. These were used as the gold standard for “start” and “stop” times for each procedure of interest (Table 1; Table 2 for representative data records).

Table 2:

Sample targeted procedures log

Procedure Occurred	Procedure Description
2019-01-09 12:42:13.030240	12 lead start
2019-01-09 12:42:38.984363	12 lead end
2019-01-09 12:43:07.886611	correction 3 lead
2019-01-09 12:44:53.242214	albuterol tx start
2019-01-09 12:45:11.890404	mask applied
2019-01-09 12:45:59.294573	tourniquet
2019-01-09 12:46:02.672012	IV start
2019-01-09 12:46:27.348737	IV in
2019-01-09 12:49:31.370297	IV procedure END
2019-01-09 12:49:54.562230	IV attempt fail
2019-01-09 12:57:29.503253	12 lead
2019-01-09 12:57:48.126144	12 lead end
2019-01-09 12:58:58.214071	albuterol tx end
2019-01-09 12:59:40.566805	check lung sounds

Paramedic Debriefs

After each observed shift, the paramedic completed a debrief survey. The research observer also provided feedback to the research team regarding lessons learned in the field, barriers encountered, and feedback obtained. Survey responses from the paramedic participants were entered into REDCap, a secure web application designed for creating and managing online surveys and databases¹⁵.

Establishing Rapport with the Paramedic Participants

The research observer was a paramedic, and this allowed him to build a productive rapport with the paramedic team. The observer arrived at the beginning of each shift, either 0530 or 1730 hours for a day or night shift, respectively. The observer typically stayed for 6 hours, depending on call volume. During busier shifts, the observer stayed with the crew for an extended period of time to collect more data.

Results

Over seven observations, two paramedics wore the system for a total of 49 hours. We observed the transport of 16 patients to 6 different hospitals. Information after handoff for six patients was obtained (Table 3). Using the first procedure logged as a start time and the end of the last procedure logged as an end time, the median time of active treatment during transport was 8 minutes and 15 seconds (standard deviation of 5 minutes and 24 seconds).

Table 3:

Breakdown of patients transported to VUMC (Emergency severity index, ESI)

Patient #	ESI Score	ED Disposition	Chief Complaint / Mechanism of Injury
1	2	Transfer to specialty care	Suspected ingestion
2	2	Transfer to specialty care	Overdose
3	2	Discharge	Auto vs. pedestrian
4	3	Unknown	Generalized weakness
5	2	Transfer to specialty care	Suicidal ideations
6	3	Discharge	Intoxication/Chest Pain

Table 4 describes the different barriers encountered during the data collection process and the approaches taken to mitigate the issues. The most common challenge was ensuring that all components of the system were properly functioning and collecting data simultaneously. Specific solutions are listed below. Additionally, the placement of the laptop in the ambulance and ASCD sensors on the medic are depicted in Figure 4.

Table 4:

Challenges encountered during data collection and their solutions

Barriers	Solutions
Intermittent interruption in Myo Armband connectivity to laptop	Laptop location moved to the head of the stretcher, under the patient’s head (Figure 4)
Script programs stopped recording when the laptop lid was shut	Installed disable lid sleep widget
Intermittent interruption in Apple Watch data collection	Implemented a live feedback system to visualize interruptions in data collection
Insufficient hotspot data for data collection	Data use was monitored proactively via web portal; ~1GB was needed per 6-hour observation
Confusing Apple Watch start/stop application	Start/stop feature changed from a tapping mechanism to a slide bar
Concerns of marrying pre-hospital observations to correct paramedic-to-ED handoff	Process developed to ensure consecutive subject data entry; relative time (since the start of paramedic shift) used to identify patients
Myo armbands intermittently vibrate if they are unsynced (caused by displacement of armband)	Paramedics were cautioned that this may occur, and they attempted to not desync the armbands; this vibrating functionality will be removed in future trials.
Systemwide VPN upgrade for VUMC users	We were forced to switch the VPN connection on the laptop; it had no apparent effect on data collection
Original hotspot data plan was canceled by the carrier for an unknown reason	Observations were delayed until we were able to obtain a new hotspot and data plan

Figure 4:

Laptop in ambulance (A) and a medic wearing two Myo armbands and two Apple Watches (B)

Current Documentation Techniques by Paramedic Participants

During field observations, the majority of documentation occurred in the ambulance. For “non-critical” patients (e.g., stable vital signs), paramedics typically used the charting software installed on their Toughbook^® laptops to document items such as past medical history, current medications, drug allergies, and demographic information. In addition, the cardiac monitors used by NFD had the ability to store vital signs such as blood pressure, heart rate, oxygen saturation, and respiration rate. This log could then be uploaded directly to the patient care report following the transport. During the care of patients who required more attention, or were more critical, documentation typically took place in the form of the paramedic writing on the glove of their non-dominant hand. In other situations, paramedics documented some procedures from memory, which is standard care practice for EMS providers. Patient care documentation into the EMS formal charting system typically occurred post-trip arrival. Paramedics indicated that charting for each patient took approximately 30 minutes in total, which is longer than the transport time but in line with other work assessing time that clinicians spend documenting¹⁶.

Operational Challenges

Although the study was designed around most patients being transported to VUMC, patients were transported to six different hospitals. This occurred for multiple reasons, including EMS diversion (hospital not accepting patients arriving by EMS), patient request to be transported to a specific hospital, proximity, and the clinical condition of the patient. Additionally, several scheduled observation periods were canceled due to mechanical malfunction of the ambulance, personnel illness, and work schedule changes.

Communication of Procedures to Receiving Facility Staff

Six handoffs between paramedics and ED clinical teams at VUMC were observed. Two handoffs occurred at ED triage, in which paramedics gave a verbal report to a triage nurse. Verbal reports from the paramedic to the ED staff largely consisted of the chief complaint and signs and symptoms. During the handoff of a level II trauma patient, the paramedic relayed all information regarding procedures (IV placement, medication administration, vital signs, and cervical collar application) to the trauma team, which was composed of physicians, nurses, and other ED staff.

General Feedback from Paramedics (Table 5)

Table 5:

Results from paramedic questionnaires

Factor	Response (N=7 Field Observations)
Ability to wear entire shift	Yes – 7/7
Perceived comfortableness	Neutral – 1/7 Slightly Uncomfortable – 6/7
Likeliness to wear entire shift	Unlikely – 1/5 Likely – 2/5 Extremely Likely – 2/5 Note: 5 responses due to addition of this question
Issues with devices interfering with uniform	“They do not interfere with clothing” “No issues” “There are no complications with uniform and armbands”
Feelings regarding devices tracking movements	“I do not have any concerns” “I do not see any real problems with the devices tracking my movements”
Overall experience	“They get more uncomfortable the longer they are on. Even with all of the links taken out, they are a bit tight” “It has been a good experience”
Perceived feasibility of automated documentation	“I think it’s completely feasible to have it automatically document time on action to improve documentation accuracy” “I feel like it would be very helpful in the pre-hospital setting with exact times and interventions”
Perceived usefulness of automated documentation	“It would be helpful on calls that require more hands on the patient where you don’t have time to document as you go” “It would be useful when we are dealing with a critical patient or have multiple patients on the same scene. It would also be helpful to have this information to help give a report to the ED”

The paramedic participants indicated that they could wear the armbands for the duration of a 12-hour shift with no anticipated difficulties. Participants reported that the armbands were tight but did not restrict their overall movement or interfere with patient care. An impression on the paramedic’s skin of the armbands was observed for ~30 minutes following the removal of the armbands. Participants suggested considering other devices that might be more comfortable or fit in clothing. Other comments included that this technology would be useful in situations where there are critical patients or multiple patients.

Discussion

To our knowledge, there have been no attempts to create and deploy an automated clinical documentation system using this range of sensors. We identified challenges surrounding logistics, connectivity, evaluation, and perception by the paramedics. The results of this pilot study support the feasibility of using an Automated Sensing for Clinical Documentation System in the pre-hospital setting. The paramedic debriefs suggest that the equipment used in this study can be deployed in a large study with more paramedics, allowing for more data to be captured.

The system’s aim is to support and supplement the documentation and upstream communication processes that occur during and after transport. For example, the ASCD system would be particularly useful in settings where verbal communication is limited, such as casualty evacuation from the battlefield or multiple simultaneous patient arrivals. Creation of an abbreviated care record with timestamps identifying which procedures were performed can enable upstream hospital providers to more effectively provide care. Specifically, the precision of such timestamps can aid in decisions regarding subsequent medication administration and other judgments surrounding patient viability.

The pilot highlighted several barriers to the system’s practical deployment. Most of the barriers encountered were logistical in nature. For example, ensuring that the data recorded continuously throughout the duration of the observation was of the most significant, which we addressed through various engineering enhancements. Modifications to the design of the system were made to make it more user-friendly, such as indicating when the system was actively recording data. These steps facilitated more accurate and complete data collections.

We anticipate that deployment of the system into the field would result in greater variation than the simulated procedures recorded in CELA, and therefore, introduce subsequent error into our model. Detection of such error will allow us to refine the current algorithm used for the identification of targeted procedures.

Methods for Future Data Analysis

This paper specifically does not discuss the algorithms used to convert the collected sensor data to an abbreviated care record. Briefly, the algorithms take the IMU and EMG data as input to generate summary statistics (e.g., entropy, power, etc.) and analyze temporal patterns. These data are then fed into classifiers to predict which procedure is being performed, if any. Some data cleaning are also performed such as removing the gravity vector and removing vehicle vibration.

Future processing will include employing a deep neural network using both convolutional and recurrent layers with memory trained classifiers in hopes for a greater efficacy¹⁷. We plan to move towards a real-time analysis using discrete, rolling windows for classification such that the start and stop of the event is not known by the classifier, which will be the case in real-world deployments.

Limitations

Our study intentionally limited the number of paramedic participants (2) in order to optimize the system and to identify and address system failures. Having a single observer and paramedic participant during the initial deployment allowed us to identify and address challenges before expanding the study. The second limitation can be attributed to the unknown nature of pre-hospital care. Because of the unpredictable nature of pre-hospital care, there were some procedures for which we were not able to collect any data.

Conclusion

This paper reported on the lessons learned from deploying an automated sensing clinical documentation system in a real-world environment. It discussed challenges of configuring equipment, collecting data, and dealing with failures. Many incremental steps were taken to reach the goal of a working system that could be safely deployed in the field and collect data, without interfering with care and patient privacy. Future work will analyze how well the algorithms are able to correctly identify which procedures were performed during transport.