Teletoxicology: Patient Assessment Using Wearable Audiovisual Streaming Technology

Abstract

Background

Audiovisual streaming technologies allow detailed remote patient assessment and have been suggested to change management and enhance triage. The advent of wearable, head-mounted devices (HMDs) permits advanced teletoxicology at a relatively low cost. A previously published pilot study supports the feasibility of using the HMD Google Glass® (Google Inc.; Mountain View, CA) for teletoxicology consultation. This study examines the reliability, accuracy, and precision of the poisoned patient assessment when performed remotely via Google Glass®.

Methods

A prospective observational cohort study was performed on 50 patients admitted to a tertiary care center inpatient toxicology service. Toxicology fellows wore Google Glass® and transmitted secure, real-time video and audio of the initial physical examination to a remote investigator not involved in the subject’s care. High-resolution still photos of electrocardiograms (ECGs) were transmitted to the remote investigator. On-site and remote investigators recorded physical examination findings and ECG interpretation. Both investigators completed a brief survey about the acceptability and reliability of the streaming technology for each encounter. Kappa scores and simple agreement were calculated for each examination finding and electrocardiogram parameter. Reliability scores and reliability difference were calculated and compared for each encounter.

Results

Data were available for analysis of 17 categories of examination and ECG findings. Simple agreement between on-site and remote investigators ranged from 68 to 100 % (median = 94 %, IQR = 10.5). Kappa scores could be calculated for 11/17 parameters and demonstrated slight to fair agreement for two parameters and moderate to almost perfect agreement for nine parameters (median = 0.653; substantial agreement). The lowest Kappa scores were for pupil size and response to light. On a 100-mm visual analog scale (VAS), mean comfort level was 93 and mean reliability rating was 89 for on-site investigators. For remote users, the mean comfort and reliability ratings were 99 and 86, respectively. The average difference in reliability scores between on-site and remote investigators was 2.6, with the difference increasing as reliability scores decreased.

Conclusion

Remote evaluation of poisoned patients via Google Glass® is possible with a high degree of agreement on examination findings and ECG interpretation. Evaluation of pupil size and response to light is limited, likely by the quality of streaming video. Users of Google Glass® for teletoxicology reported high levels of comfort with the technology and found it reliable, though as reported reliability decreased, remote users were most affected. Further study should compare patient-centered outcomes when using HMDs for consultation to those resulting from telephone consultation.

Electronic supplementary material

The online version of this article (doi:10.1007/s13181-016-0567-3) contains supplementary material, which is available to authorized users.

Introduction

Patients suffering from poisoning, drug overdose, adverse drug events, and envenomations have improved outcomes when treated by a medical toxicologist [1]. Toxicologists and Poison Control Centers (PCCs) serve large geographic areas, rendering direct bedside evaluation impractical in many cases. A variety of technological solutions have evolved to address these issues, beginning with a nationwide telephone-based infrastructure that currently supports most PCCs by allowing patients, caregivers, emergency medical personnel, and healthcare providers to seek the guidance of a medical toxicologist. As most poisoned patients initially present to healthcare facilities without bedside toxicologists, telephone consultation is widely utilized and has been shown to improve patient care and reduce resource utilization [2–6].

Bedside evaluation of the poisoned patient reveals important aspects of a physical examination that may be inaccurately conveyed through telephone consultation. Nuanced aspects of the physical examination (i.e., muscle tone, reflexes, or pattern of speech) strongly influence the toxicologic differential diagnosis but may be omitted in a focused physical examination performed in a busy emergency department or intensive care setting. These aspects of the examination are critical to establishing a toxidrome and causative agent. The initial interpretation of electrocardiograms (ECGs) reported by callers to PCCs has also been shown to be frequently inaccurate [7]. Real-time video telemedicine through head-mounted devices has been postulated to improve remote toxicology consultation by incorporating critical objective visual elements of a patient physical exam and remote ECG interpretation into the consultation process. Chai and colleagues, in a pilot study of 18 patients, recently demonstrated the feasibility of performing teletoxicology consultation via Google Glass®. In this small series, the ability to receive audiovisual feeds of patients resulted in changes in management in 56 % of cases and the recommendation of a specific antidote in one third of cases [8]. This study was limited by the lack of a gold standard for physical examination findings and lack of a control group. In addition, the supervisory consultants altered clinical management based on the assumption that audiovisual information as received through Google Glass® accurately reflected patient examination findings, though this was unproven.

In this study, we sought to further the nascent literature on teletoxicology by evaluating the quality of the poisoned patient assessment via wearable audiovisual streaming technology. We prospectively assessed the precision, comfort with use, and reliability of teletoxicology to evaluate poisoned and envenomated patients remotely.

Methods

This observational cohort study was reviewed and approved by our institutional review board and complied with our center’s best practices for human subject research. A waiver of consent was granted after demonstration of minimal potential harm to patients and strict compliance with information confidentiality practices.

We used a Google Glass® device (Google Inc.; Mountain View, CA) that had been cleared of its native software and prepared by Pristine®, a software company, to run Pristine Eyesight® (PES) (Pristine; Austin, TX). PES is a proprietary software solution that provides HIPAA-complaint video streaming through Google Glass®. When using Google Glass® with PES, video and audio are streamed securely to a remote user viewing video on a conventional web browser. Video transmission is one-way, from the wearer of the device to the remote user. High-definition photographs can be initiated by the wearer of the device, or taken remotely, with a countdown in the heads-up display to alert the wearer of an impending still photo. Photos can be annotated by the remote user and viewed in the heads-up display. Two-way audio communication is possible through the device’s built-in bone conduction audio and microphone. In addition, the remote user can send text messages to the heads-up display in Google Glass®. Medical toxicology fellows (serving as on-site investigators) and faculty (remote investigators) underwent training in the use of Google Glass® and PES. On-site investigators were outfitted with a fourth-generation (4G) wireless hotspot to connect Glass® to the internet.

All patients in this study were admitted to the medical toxicology service of a tertiary care center located in a major metropolitan area, between February 1, 2015 and July 31, 2015. The service, which admitted over 1100 patients in 2014, includes medical toxicology fellows in a rotating call schedule with 24-hour attending toxicologist supervision. A convenience sample of patients, based on immediate availability of a remote investigator, was used in the study. Study consultations were performed day or night, on any day of the week. Patients met study inclusion criteria if they were greater than 18 years of age and required admission to the ICU under the medical toxicology service.

On-site investigators then performed admission physical examinations while wearing Google Glass® running PES to stream first person live video of the patient to a remote investigator not directly involved in the patient’s care. The toxicology attending on call supervised all clinical care provided by the fellow in the usual manner and did not participate in the study. The remote investigator was able to hear audio from the patient examination but did not engage in two-way real-time communication with the on-site investigator regarding exam findings. ECGs were transmitted via high-resolution still images. No video or audio of patient interactions was recorded at any time. Both investigators then immediately and independently documented 18 key physical exam findings and ECG interpretations on a predetermined data collection form developed by investigator consensus (Appendix 1). Exam findings evaluated included pupils (size and response to light), nystagmus, diaphoresis, meningismus, presence of abdominal or muscle compartment tenderness, mental status, cranial nerve exam, motor exam, muscle tone, reflexes, coordination, and gait. After each patient interaction, both physicians independently evaluated the use of Google Glass® for comfort, reliability, and data transmission fidelity. Each investigator was queried on data interruptions, delays in transmission, and the duration of the longest delay. Overall, comfort and reliability were evaluated for each investigator using 100 mm visual analog scales (VAS). These data were recorded on a predetermined investigator survey form (Appendix 2). All data were de-identified and entered into a secure, encrypted database. Simple inter-rater agreement and Kappa statistic were calculated for physical examination and ECG findings. Comfort and reliability VAS scores were compared between on-site and remote investigators.

Results

During the study period, 50 non-consecutive patients were enrolled. The average patient age was 42 years; 24 (48 %) patients were male and 26 (52 %) were female. Twelve patients (24 %) were endotracheally intubated. The majority of patients (n = 36, 72 %) were admitted with acute drug toxicity or intentional self-poisoning. Seven patients had drug or alcohol withdrawal syndromes. Five patients had been envenomated by rattlesnakes; one suffered a massive honeybee envenomation. Medical and surgical comorbidity was common (Appendix 3, patient characteristics). Patient physical exam and ECG interpretations were analyzed for inter-rater reliability using simple inter-rater percent agreement and the Kappa statistic for each finding (IBM SPSS Statistics for Macintosh, Version 23.0). Reliability VAS scores were compared between on-site and remote investigators via a Bland-Altman plot (Difference plot) (Fig. 1).

Fig. 1.

Bland-Altman plot showing the comparison of reliabilities between investigators

Sufficient data were available to calculate simple agreement for 17/18 individual parameters and Kappa statistic for 11/18 parameters (Table 1). Simple agreement between on-site and remote investigators ranged from 68 to 100 % (median = 94 %, IQR = 10.5). Kappa scores demonstrated slight to fair agreement for two parameters and moderate to almost perfect agreement for nine parameters (median = 0.653, substantial agreement). The lowest Kappa scores were for pupil size and response to light (0.379 and 0.148, respectively). The lowest inter-rater agreement scores were for pupil size and ECG rhythm (68, 68 %). On a 100-mm VAS, mean comfort level was 93 and mean reliability rating was 89 for on-site investigators. For remote users, the mean comfort and reliability ratings were 99 and 86, respectively. Remote investigators reported interruptions in data transmission in 11 of 50 encounters. Delays or stuttering in data transmission was reported by remote investigators in 24 of 50 consults, lasting a mean of 6.3 s. The mean number of attempts prior to securing a streaming connection was 3.5, with a maximum of six attempts. Differences in reliability scores between on-site and remote investigators were compared via Bland-Altman plot (Fig. 1). The average difference in reliability scores was 2.6, with the difference increasing as reliability scores decreased.

Table 1.

Kappa statistic, strength of agreement, and percent agreement between on-site and remote investigators, by parameter

Measure	Kappa	Strength of agreement based on Kappaa	Percent agreement
Pupil size	0.379	Fair	68
Pupil response to light	0.148	Slight	76
Nystagmus	–	–	93
ECG rhythm (NSR vs. sinus tachycardia vs. other)	0.419	Moderate	68
ECG rhythm (SR vs. other)b	0.653	Substantial	97
Diaphoresis	–	–	96
Meningismus	–	–	100
Abdominal tenderness	0.692	Substantial	94
Muscle compartment tenderness	0.616	Substantial	91
Mental status (Grady coma scale)	0.788	Substantial	85
Cranial nerve examination	–	–	96
Motor examination	–	–	94
Reflexes	0.501	Moderate	88
Muscle tone	–	–	98
Coordination	1.000	Almost perfect	100
Gait	–	–	–
QRS duration	0.842	Almost perfect	97
QTc duration	0.842	Almost perfect	97

^aBased on Landis and Koch’s suggested nomenclature for interpretation of Kappa [9]: <0.00 = Poor; 0.00–0.20 = Slight; 0.21–0.40 = Fair; 0.41–0.60 = Moderate; 0.61–0.80 = Substantial; 0.81–1.00 = Almost perfect

^bPost hoc analysis, see Discussion

– = insufficient data to calculate

Discussion

This study shows Google Glass® can precisely and reliably transmit key physical exam and ECG findings in poisoned patients through real-time video and static images. Transmission of pertinent exam findings as well as interpretation of critical ECG findings in poisoned patients can be accomplished through Glass®.

Our data supports overall high precision with respect to physical examination and ECG findings transmitted through Google Glass®. Median Kappa scores demonstrated substantial agreement according to the interpretive framework suggested by Landis and Koch (Table 1, legend), and simple agreement between investigators for each data point was very high [9]. In the case of this study, simple agreement may better represent the intended study outcome because Kappa scores are adversely affected by high prevalence of the finding being observed (high prevalence of a single finding increasing the likelihood of agreement by chance).

Pupil size and response to light performed poorly by all measures and this was likely due to the overall quality of the video stream (see Fig. 2 for a comparison of streaming and still image quality). We anticipate iterative improvements in video streaming quality as well as improved camera resolution in later generations of Glass® and head-mounted devices (HMDs) that may improve the ability to perform the pupil exam.

Fig. 2.

a Screen capture of ECG as seen via video stream. b Screen capture of ECG as seen via remote, high-resolution snapshot

ECG rhythm interpretation demonstrated only moderate agreement by Kappa and simple agreement of only 68 % in the primary analysis. We noted disagreement on the reporting of “normal sinus rhythm” vs. “sinus tachycardia” between on-site and remote investigators and therefore performed a post hoc analysis examining agreement between sinus rhythms (including sinus tachy- or bradycardia) vs. other non-sinus rhythms (Table 1). This improved the agreement based on Kappa to substantial and simple percent agreement increase to 97 %, suggesting that investigators had little trouble distinguishing sinus from other rhythms but had different thresholds to report a sinus rhythm as tachycardic or bradycardic.

Previous studies have demonstrated the importance of connectivity in HMD deployment [8, 10]. We were unable to establish a reliable wireless connection between Glass® and our institution’s Wi-Fi network and used a cellular network hotspot instead. Though there was a high degree of comfort using Google Glass® among both on-site and remote users, reliability was overall perceived as worse by the remote investigators. Both site investigators and remote investigators perceived high reliability of the connection despite multiple attempts being frequently required to establish the initial video connection. Delays or stuttering in the video stream was common, though relatively brief. The Bland-Altman plot (Fig. 1) examines the reliability difference between investigators, which increased as reported reliability worsened; this affected the remote investigators disproportionately. The disproportionate effect on remote investigators was anticipated, since they relied completely on the performance of the audiovisual stream, while on-site users performed a patient examination unaffected by data quality. Video lag occurred during most sessions; however, this was not perceived to impact the overall quality of the teletoxicology videoconference.

The primary practical implication of our study is that HMDs, like Google Glass®, can precisely and reliably transmit examination findings that are likely to influence clinical management. Many poisoned patients are transferred to our institution through rural and critical access hospitals. Our study demonstrates the potential benefits of a virtual toxicology presence at these remote hospitals that may improve resource utilization through improved triage of prospective patient transfers. The cost of Google Glass® is far below that of traditional inpatient telemedicine systems and may be more appealing to small hospital administrators who can access telemedicine resources in a cost-effective manner.

Although the Google Glass® explorer program has now ended, we believe, as do other authors, that Glass® was initially marketed to individual consumers in spite of non-existent demand [11]. The potential applications in industry and medicine of HMDs, however, are plentiful and less subject to the lack of consumer appeal that may have plagued the initial version of Google Glass®. Google® has announced that it plans to revamp the Glass® product, previously suggesting it would be aimed at industry [12].

Limitations

This study had a number of limitations. This was a single-center study, and the reliability of Google Glass® was influenced by our institutional IT infrastructure. We ultimately utilized a Wi-Fi hotspot operating on a widely available cellular network. Other healthcare settings may have better or worse Wi-Fi or cellular network coverage, and the reliability of video streaming would be expected to vary accordingly.

Our patient population was a convenience sample of non-consecutive inpatients, though patients were evaluated remotely day and night. The sample size is small, despite the service volume, as enrollment was often limited by toxicology fellow availability while on call, as well as the immediate availability of a remote investigator. The intent of the study design was that each study subject served as its own “gold standard” as far as physical examination findings. That is, the physical examination findings as reported by the on-site investigator would be accurate, and the precision of the transmitted data measured by the agreement of the remote investigator’s findings. However, our on-site investigators were fellows in medical toxicology training, and it is possible that they reported some examination or ECG findings incorrectly based on their level of experience, though this seems unlikely. We did not have a blinded cardiologist or other gold standard interpretation for each ECG.

Investigators were often asked to place elements of the physical examination into nominal or bimodal categories. In reality, physical examination findings are, of course, nuanced. In addition, due to the clinical status of the included patients, there were insufficient data to calculate agreement and Kappa scores for some examination findings that were rarely tested because of patients’ clinical status (i.e., gait).

Our study reported user assessments of comfort and reliability of Google Glass® but assumed that the use of the device itself did not interfere with patient evaluation or the performance of patient care tasks. Early evidence from the laboratory setting and driving simulations now suggests that Google Glass® use interferes with concurrent tasks requiring selective attention and worsens driving performance compared to undistracted driving [13, 14]. Future research will need to determine what, if any, attention or performance cost is incurred by the use of HMDs for telemedicine, compared to a third-person perspective, passive audiovisual stream.

Conclusion

Google Glass® is able to transmit most patient examination findings important to medical toxicologists to a remote examiner with overall high precision and reliability. Current levels of streaming video quality limited the remote interpretation of some findings in this study, including pupil size and responsiveness to light. Google Glass® is a viable option for audiovisual teletoxicology consults at a cost lower than traditional telemedicine systems. The addition of audiovisual information to medical toxicology consultation may change management decisions and improve triage but this is not yet proven. Future studies should compare patient-centered outcomes resulting from teletoxicology consultation to those from traditional telephone consultation.

Compliance with Ethical Standards

This observational cohort study was reviewed and approved by our institutional review board and complied with our center’s best practices for human subject research. A waiver of consent was granted after demonstration of minimal potential harm to patients and strict compliance with information confidentiality practices.

Conflict of Interest

The authors declare that they have no competing interests.

Sources of Funding

Funding was provided by Banner Health and the Banner – University Medical Center Phoenix, Department of Graduate Medical Education.

Previous Presentation(s) of Data

None.