Designing and Validating a Low-Cost Real Time Locating System to Continuously Assess Patient Wait Times

Abstract

Objective

Outpatient clinics lack infrastructure to easily measure and understand patient wait times. Our objective was to design a low-cost, portable passive real time locating system within an outpatient clinic setting to measure patient wait times and patient-provider interactions.

Materials and Methods

Direct observation was used to determine workflow in an outpatient glaucoma clinic at the University of Michigan. We used off-the shelf, antenna-integrated ultra-high frequency (UHF) RFID readers (ThingMagic, Astra-Ex, Woburn, MA) and UHF re-useable passive RFID tags (Zebra Impinj Monza 4QT, Seattle, WA). We designed a custom RFID management application in the Java programming language that was equipped with ‘live’ device administration to collect time and location data from patients and providers. These hardware choices enabled low cost system installation. Hidden Markov Modeling (HMM) was used to smooth patient and provider location data. Location data were validated against direct observations and EHR evaluation.

Results

The HMM smoothed RFID system data accurately predicted patient location 80.6% of the time and provider location 79.1% of the time, compared to direct observation locations, an improvement over the raw RFID location data (65.0% and 77.9% accurate, respectively). Patient process time was on average 42.8 minutes (SD=27.5) and wait time was 47.9 minutes (SD=33.1). The installation and recurring capital costs of the system are approximately 10% of available commercially-supplied patient/provider tracking systems.

Discussion

Passive RFID time study systems can enable real-time localization of people in clinic, facilitating continuous capture of patient wait times and patient-provider interactions. The system must be tailored to the clinic to accurately reflect patient and provider movement.

Conclusions

Capturing wait time data continuously and passively can empower continuous clinical quality improvement initiatives to enhance the patient experience.

Graphical Abstract

1. INTRODUCTION

Long wait times are a key source of patient dissatisfaction with medical care [1,2]. The Institute of Medicine’s report on health care quality, “Crossing the Quality Chasm,” states that providing timely, efficient, and patient-centered care are critical for improving the quality of medical care in the United States [3], and yet long wait times are rampant in specialty care.[4] Specialty outpatient care workflow, such as that delivered in an academic glaucoma clinic, is characterized by inter-dependent activities that are not always linear. Care is provided by multiple members of the healthcare team, including clerks, medical assistants, ophthalmic technicians, residents/fellows and attending physicians. Patients need to undergo multiple tests with different providers before they are ready to be seen by the attending physician, and can need to be seen by the attending physician more than once (before and after a test or procedure). The complex nature of the work flow makes assessing overall patient wait time, or time when a patient is not co-located with a provider, difficult. The EHR can measure total visit time, but without accurate ways to measure patient wait time it is difficult to know whether changes in clinical operations are reducing time patients spend with providers or actual wait time, and time-motion studies carried out through direct observation are resource intensive [5].

2. OBJECTIVE

The objective of this work was twofold. The primary objective was to co-opt available off-the shelf technology to design a robust, passive real-time localization system (RTLS) in the outpatient ophthalmology glaucoma clinic that could a) determine both patient and health care provider presence and co-presence to assess patient location to a spatial position of 1 meter or less, at a frequency of 1 minute or less, and b) cost less than $50,000 in capital investment and less than $5,000 in yearly maintenance fees to enable the project to be scaled. The secondary objective was to assess the accuracy of the system in a working outpatient clinic. We show how our RTLS data compares to 1) direct observation of patient encounters; 2) direct observation of provider work-flow and 3) provider EHR timestamp audit logs from workstations located in clinic examination rooms. Presenting the design choices, algorithm development and validation provides a framework for others to extend this work to new domains of research and implementation.

3. MATERIALS AND METHODS

3.1. Choice of Technology

Real Time Locating Systems (RTLS) offer promise in providing an automated way to bridge the information gap between data provided by the EHR and the granular data needed to make informed clinical operations management decisions without employing the timeconsuming method of in-person observation. There are a variety of sensor modalities that can inform Real Time Locating Systems (RTLS), such as Bluetooth low energy beacons, camera vision, radio-frequency identification (RFID), infrared, visual recognition global positioning systems (GPS) and cellular signals. [6]. Passive RFID is one of the lower-cost technologies because the sensors that tag objects are very inexpensive ($0.04-$1.20), which is why it has been the technology of choice in industry [7,8]. In comparison, the Bluetooth iBeacon, camera vision and GPS systems still have relatively high cost tags. Thus, handing out RFID tags to all patients and providers, some of whom may inadvertently walk away with them, is not prohibitively expensive. Although the RFID readers may be more expensive upfront, it is mounted to a wall in a clinical environment so there is no concern for loss.

Some large-scale health networks have implemented RFID based RTLS systems to measure the impact of quality improvement efforts and have been able to demonstrate fewer medication errors [9], shorter wait times [10], and improved resource utilization [11]. An advantage of RFID is that each tag has a unique identifier so that each object, or in our case each health care provider and patient, can be tagged and tracked. Yet, healthcare utilization comprises only 7% of the total RFID market share [12]. Smaller health systems and outpatient clinics have yet to see the benefit RFID can provide due to the high entry cost from vendors [13] and previous research has noted that off-the-shelf technologies have had only modest accuracy [14]. We chose to use off-the shelf RFID technology due to its relatively low cost per tracked individual and ability to identify and track unique tags (individuals) over time with good spatial and temporal resolution. We will describe how we customized the RTLS system to meet our clinical needs to monitor all healthcare providers who interacted with each patient in a non-linear pattern to accurately assess additive wait time throughout a clinical encounter.

3.2. Clinical Setting

The University of Michigan (UM, Ann Arbor, MI) provides tertiary, multi-specialty care and medical student, residency, and fellowship training. The EHR system (Epic, Verona, WI) was implemented at the UM in 2012. The Kellogg Eye Center (KEC) at the UM includes approximately 80 faculty physicians who provided approximately 180,000 outpatient visits in 2017. The glaucoma clinic has 11 faculty physicians who provided approximately 13,000 outpatient visits in 2017. The RFID-based RTLS was implemented in the glaucoma clinic at UM. This study was exempt under Quality Improvement by the UM Institutional Review Board.

3.3. RFID-Based RTLS Design Choices

An RFID systems contain three main components: RFID tags, RFID readers with antennae, and middleware [15]. An RFID tag is a small transceiver with a microchip and an antenna that can be written with a unique identifier (Figure 1A, 1B). The microchip has enough memory to store the unique identifiers and the instructions for how to respond when excited by the signal from the RFID reader. The passive RFID antenna allows the tag to receive power and communicate with the reader. The tag is excited by the electromagnetic signal emitted by the RFID reader and responds by emitting its identifier back to the reader. The RFID reader contains an antennae and a processing module that interprets the electromagnetic signals received. (Figure 1C) This is referred to as transceiver/transponder interaction [16]. The role of the middleware is to collect and interpret the raw data (e.g. RFID Tag ID, signal strength) that is captured by the RFID reader. Many types of RFID components are commercially available allowing flexible design choices for each component that influence the system’s cost and scalability. For this study, we compared different types of RFID tags, readers and middleware to identify a system that would be accurate in a clinical setting and have modest up front cost.

Figure 1.

RFID System Components and Map of Clinical Placement of Readers A. Zebra MC3190-Z Handheld UHF RFID Reader/Writer. This device reads the Data Matrix Code from each patient label and writes a unique RFID tag. When tags are re-scanned, previous data is erased. B. UHF RFID tag on a clip for a patient or provider to wear. C. Picture of the UHF RFID Reader (Thing Magic Astra-Ex reader) installed in a clinic room. D. Clinic diagram denoting where each of the 23 UHF RFID readers (denoted by the wireless signals) are placed.

3.4. Reference Data Collection

After the RFID system was implemented, workflow observations were conducted in the glaucoma clinic. The staff in the glaucoma clinic, including 11 faculty physicians, 8 ophthalmic technicians, 4 medical assistants and all trainees (residents and fellows), wore RFID tags clipped to their shirts identified with their unique UM ID. Glaucoma patients who verbally assented to participation were included in the study and wore RFID tags identified with their unique Contact Serial Number (CSN), a unique number assigned to a patient visit that is linked to a patient’s medical record number. The CSN provided the link between the RFID signal data and the EHR data. The CSN was encrypted on the tag and the SQL database storing the data was encrypted and created to HIPAA standards to protect private health information (PHI).

A subset of patients who agreed to wear RFID tags were individually followed through their appointment. Their time spent in each clinic location was recorded. Audit log data for physicians and technicians providing care from glaucoma clinic workstations was abstracted from Clarity, the research data warehouse that contains the UM EHR data. Audit log data were abstracted between January 5 and July 3, 2018 from 19 providers (11 attending physicians and 8 ophthalmic technicians) over 12 examination rooms for comparison with RFID data.

3.5. Statistical Processing & Location Algorithm Development

Data selection and filtering

All RFID tag reads with time stamps occurring before 7:00 am or after 7:00 pm (outside of clinic hours) were excluded, as were RFID reads for patient visits where the matching medical record information could not be retrieved. Medical records and RFID reads were considered matching if both the date and CSN in the medical record matched the corresponding date and CSN from the RFID tag. RFID reads for patients were retained only when detected between the check-in and check-out times from the medical record. For both patients and providers, only the first two reads detected within each one-second time span were retained.

Preliminary location determination

RFID decibel-scale signal strength data were converted via exponentiation to signal strengths and summed within one-minute time blocks. These summations were obtained for each patient and provider, for every minute that the RFID tag was detected by an RFID reader. A preliminary assignment of a person in each one-minute interval was then made by assigning the person to the location with the greatest total signal strength during that minute. When no signals were obtained for a full minute, preliminary assignment to a “null” room was made.

Hidden Markov Model Parameters

Preliminary room assignments were further processed using Hidden Markov Models (HMMs) to eliminate most of the very short spells and frequent or impossible transitions that were unlikely to reflect true patient and provider locations [17]. The Viterbi linear programming algorithm was used to determine the most likely true sequence of locations for a patient or provider given their observed reads. The HMM used here has three types of parameters: a distribution of probabilities for the initial true state, a distribution of transition probabilities from one true state to the next, and a distribution of observed room assignments (corresponding to our “preliminary room assignment” above) given the true assignment. We set these parameters manually to eliminate unlikely or impossible location assignments, and to achieve concordance with our shadow observation data where available. We imposed a few “structural zeros” on the model, reflecting known constraints in the system: (i) patients are never in the lensometer room, (ii) patients cannot go directly from one exam room to another (i.e., some time must be spent in a hallway), and (iii) patients cannot go directly from one visual field room to another. Please see Appendix 1 for more detail.

3.6. Validation of RFID location data

Accuracy of the RFID location data (raw and smoothed by HMM) was measured by comparison to 1) direct observation of patient encounters; 2) direct observation of provider work-flow and 3) provider EHR timestamp audit logs from workstations located in clinic examination rooms. Accuracy is reported as the percentage of minutes where RFID location data matched these gold-standard direct observations or EHR timestamps.

3.7. Calculating wait times and process times from the HMM Data

Using location data generated by the HMM modeling, “wait time” was defined as minutes when the patient was not co-located with a provider. This could occur in the reception waiting room, the in-process waiting room inside the clinic, or in an exam room. Process times were defined as minutes’ different providers from the healthcare team (ophthalmic technicians, medical assistants, residents, fellows, attending physicians) spent with patients. Wait and process times are reported with means and standard deviations (SD).

4. RESULTS

4.1. RFID System Design Choices

RFID Tags

There are two main categories of RFID tags: active and passive. Active RFID tags are battery powered and have very long read ranges, whereas passive RFID tags are powered by the electromagnetic signal emitted from the RFID reader and have shorter read ranges. [9, 10, 11]

Evaluating our clinic’s floor plan (Figure 1D) we determined that 6 m would be the minimum read range suitable for patient/provider monitoring making either active tags or passive ultra high-frequency (UHF) tags reasonable choices.

We then performed a cost assessment. The clinic has 26 providers and sees up to 100 patients per day with 60% of patients seen in the morning, and provides 13,000 patient visits per year. Assuming a 10% loss rate for tags each month, we calculated the cost of the different tag types for use in our clinic. For active RFID tags ($75.00/tag), the clinic would need 86 re-useable tags monthly for an up-front cost of $6,450. Over 5 years, with a 10% loss rate, the active RFID tags would cost $38,700. Passive UHF RFID tags are available as either re-useable, re-writeable tags or as single-use tags. For re-useable passive UHF RFID tags ($1.20/tag, Zebra Impinj Monza 4QT, Seattle, WA), the clinic would need 86 re-useable tags monthly for an up-front cost of $103.20. Over 5 years, with a 10% loss rate, the re-useable passive UHF RFID tags would cost $1,680. For single-use disposable passive UHF RFID tags, ($0.14/tag, AD 383u7 UHF tag, Avery Dennison, Glendale CA), the clinic would need 1,083 tags per month for an up-front cost of $151.62 plus the cost of an RFID tag printer ($3,000, Zebra Technologies Corp, Lincolnshire, IL) for a total cost of $3,152. Over 5 years, though there is no loss rate, the single-use tags would cost $12,100. Therefore, we selected the re-useable passive UHF RFID tags due to the anticipated cost savings.

The re-useable passive UHF RFID tag was written with a custom-programmable hand-held RFID scanner (MC3190-Z, Zebra, Lincolnshire, IL) to include the participant’s Contact Serial Number (CSN) after obtaining the CSN from a 2-D Data Matrix Code in the EHR. The ID tag also included its own distinct tag ID number and the last time and date the tag was written (Figure 1A). The hand-held scanner cost $2700 and we purchased two scanners so that the two clerks at check-in could both hand out tags to patients.

RFID Reader and Antennae

Our team evaluated the two commercially available UHF RFID readers, the ThingMagic Astra-Ex (ThingMagic, Woburn, MA) and the Impinj Speedway R420 (Impinj, Seattle, WA) for possible use to monitor the examination rooms, testing rooms, and waiting rooms (23 clinical spaces, Figure 1D). The primary differences between the two devices were the antenna arrangement and networking abilities. The Astra-Ex has a self-contained (“integrated”) antenna and is capable of both Power Over Ethernet (POE) and Wi-Fi connectivity and cost $1,100 per unit. The Astra-Ex (4.5 lbs) can be mounted with 4 Velcro strips (3M Command Strips, Rated to 16 lbs each, $0.65/piece, $2.60/reader) and be plugged in and mounted by a non-professional. The Speedway can connect to 32 external antennas via coaxial cable, has Ethernet only network connectivity and costs $1,600. Though it may seem that the Speedway would have been the less expensive choice, as the single reader can connect to 32 different antennas ($150 per antenna + $270 per antenna hub that houses 4 antennas) that can read input from 32 different spaces, as opposed to the Astra-Ex which requires the full unit to be installed in each space in which tags will be read, the Speedway reader is a wired system. The Speedway reader needs coaxial cabling (approximately $75 per room) to connect it to the Ethernet network. The coaxial cabling must be installed in the ceiling in clinic to conform to healthcare standards at a cost of $1500 per clinic room. In total, the projected cost for the installed Impinj Speedway was $42,895 ($8,395 in parts and $34,500 in labor). Labor costs for the Astra-Ex were estimated at $14/hour plus 32% fringe for the 27.5 hours it took the research assistant to plug in and mount the readers, for total costs for the Astra-Ex of $25,868 ($25,360 in parts and $508.00 in labor). In addition, the cabled system would not be readily moveable and the Astra-Ex’s Wi-Fi ability provides future versatility for environments not equipped with POE. Therefore, we chose to install the Astra-Ex UHF RFID reader.

RFID readers have different read settings. Our RFID reader (Astra-EX UHF RFID) had three read settings: 1) maximum tag read rate (default); 2) two pings/second; 3) one ping/two minutes. The maximum tag read rate overwhelmed our database and would crash the application, and the slowest read rate could not generate location data frequently enough to capture movement through clinic. Thus, we chose to use the two pings/second setting. However, occasionally a reader would fail and the application would force a restart returning the reader to default settings. This would then allow pings to be read at the maximum read rate. Because our location algorithm relied on the sum of signal strength of all pings over 1 minute from a tag to a specific reader, those readers that recorded pings at greater frequencies would then predominately be chosen as the location. Therefore, to account for the times where RFID readers were collecting pings at maximum frequency, we used only the first 2 pings per second.

Middleware

To make this system as replicable as possible, we initially tried a commercial middleware software application from the Astra EX manufacturer, ThingMagic, called the Universal Reader Assistant (URA). The URA supported HTTP posting and TCP port streaming. However, it could only connect to a single RFID reader at a time. In order to connect to the 23 RFID readers in the clinic, we would need 23 instances of the URA windows program running simultaneously. In order to test the feasibility of this framework, we opened 2 instances of the URA and configured connections to two distinct readers but encountered numerous loss of connectivity errors. Reconnection could only be accomplished manually, making it clear that the URA middleware was not suitable to our goal of creating automated time-motion studies in clinic.

The next middleware solution we evaluated was a sample program from ThingMagic’s version 1.27.3.16 API called multireadasync.java. This program was capable of connecting to multiple RFID readers, but would not run continuously or store any of the raw data it received. We revamped this Java program to run without stopping and to connect to a backend database. Data were then processed and stored using the Spring JAVA infrastructure to insert raw data objects serially into an encrypted, secured, structured-query language database (MySQL). Although this modified multireadasync program addressed the challenges of the original program, it had its own shortcomings. The device-middleware connection had prevalent loss of connection errors with devices tested using both WiFi and POE. Therefore, the student team lead by the health information system research team designed a custom RFID management platform equipped with automatic reconnection subroutines, email and SMS status notifications, and a user interface for ‘live’ device administration. The custom RFID middleware records are reliably collected and stored in an encrypted, PHI-secure database.

4.2. RFID Location Validation

A total of 6,813 patient appointments occurred in the glaucoma clinic of the UM KEC from January 5 – July 3, 2017. Of these appointments, 1,972 (28.9%) corresponded to a patient (n=1589) that consented to wear an RFID tag for study. The majority of study patients (81.7%) were seen for a single appointment over the study period, while 13.7% were seen for 2 appointments, and 4.6% were seen for between 3–5 appointments. Patients were 56.2% female, 78.9% self-reported as White, and averaged 65.9 years old (SD=14.7) at the time of their first, or only, study appointment (Table 2). Most consented appointments were return visits (47%), followed by visual field checks (30.3%), new patient visits (9.5%), post-operative visits (8.5%), laser treatments (4.3%), and urgent visits (0.3%). 3,533 minutes of patient shadow time from 34 patients was captured in direct observations.

Table 2.

Sample descriptive statistics

	Study Sample (n=1589 patients, n=1972 appointments)				Patient Shadow Sample (n=34 patients, n=34 appointments)
Continuous Measures	N	Mean (SD)	Min, Max	Median	N	Mean (SD)	Min, Max	Median
Age (years)	1588	65.9 (14.7)	13.9, 102.5	67.8	34	67.6 (15.5)	13.9, 88.8	68.5
Appt Length* (mins)	1923	124.3 (61.1)	20.6, 626.5	113.1	32	112.6 (56.3)	36.5, 261.9	102.0
Categorical Measures	frequency (percent)				frequency (percent)
Gender
Female	892 (56.2)				24 (70.6)
Male	696 (43.8)				10 (29.4)
Race
American Indian/Alaskan Native	3 (0.2)				0 (0.0)
Asian	103 (6.6)				3 (8.8)
Black	196 (12.5)				5 (14.7)
Multiple Races	4 (0.3)				0 (0.0)
Other	24 (1.5)				0 (0.0)
White	1237 (78.9)				26 (76.5)
Ethnicity
Hispanic	35 (2.5)				0 (0.0)
Non-Hispanic	1380 (97.5)				28 (100.0)
Appt Type
Laser Treatment	85 (4.3)				0 (0.0)
New Patient	187 (9.5)				2 (5.9)
Return Visit	929 (47.1)				17 (50.0)
Post-op Visit	168 (8.5)				3 (8.8)
Urgent	5 (0.3)				0 (0.0)
Visual Field	598 (30.3)				12 (35.3)
Appt Time
AM	794 (40.3)				19 (55.9)
PM	1178 (59.7)				15 (44.1)
Appt Day
Monday	285 (14.5)				6 (17.7)
Tuesday	545 (27.6)				9 (26.5)
Wednesday	386 (19.6)				7 (20.6)
Thursday	360 (18.3)				7 (20.6)
Friday	396 (20.1)				5 (14.7)

^*excludes patient appointments where check-out occurred the following day due to appointment running later than check-out clerk work hours

Location Accuracy

For the 3,533 minutes of direct patient shadow time, our raw RFID data predicted the correct location for patients 65.0% (2,296/3,533) of the time while the HMM model data predicted the correct location for patients 80.6% (2,848/3,533) of the time (Table 3). Alternatively, the raw RFID locations were accurate for 77.9% (250/321) of provider shadow minutes and the HMM model location determinations were slightly more accurate at 79.1% (254/321). Using the audit log data from the EHR, there were 46,796 minutes when a healthcare provider opened a patient medical record while in an exam room. We assumed that the patient was co-located in the examination room when the provider had their record open. The raw RFID patient location matched the audit log location for 74.0% of minutes and the HMM patient location matched 77.2% of minutes. Compared to the 49,061 minutes of attending physician audit log data from the EHR, the raw RFID data accurately placed attending physician location 79.4% (38,954/49,061) of the time while the smoothed HMM model data accurately placed attending physician location 83.8% (41,113/49,061) of the time (Table 3). Compared to the 83,778 minutes of technician audit log data from the EHR, the raw RFID data accurately placed technician location 74% (61,996/83,778) of the time while the HMM model data had a 79.5% (66,604/83,778) location accuracy. Figure 2 displays location data for 2 example patients through the course of their appointment comparing direct patient shadow locations to raw RFID and HMM locations (top panels). Co-presence of a patient with a technician and attending are also displayed (bottom panels).

Table 3.

Accuracy of RFID location data

		Raw RFID	Smoothed HMM
		No. (%) Location	No. (%) Location
	N	Match	Match
Shadow Location
Patients (n=34)	3533 mins	2296 (65.0)	2848 (80.6)
Provider (n=2)	321 mins	250 (77.9)	254 (79.1)
EHR Timestamp Location
Patients	46796 min	34623 (74.0)	36131 (77.2)
Providers	49061 mins	38947 (79.4)	41096 (83.8)
Attending
Technician	83778 mins	61959 (74.0)	66603 (79.5)
		Raw RFID	Smoothed HMM	EHR
	N	Mean (SD)	Mean (SD)	Mean (SD)
Workflow Comparison
Provider
Daily # Patients (pts/day/attending)	370 attending days	20.3 (16.1)	14.4 (12.3)	14.6 (10.5)
Patients
Exam Room (pts/room/min)	114166 mins	1.02 (0.13)	1.02 (0.14)	N/A
VF Room (pts/room/min)	16682 mins	1.01 (0.09)	1.01 (0.09)	N/A

RFID, Radio Frequency Identification; HMM, Hidden Markov Model; No., Number; SD, Standard Deviation; EHR, Electronic Health Record

VF, Visual Field; pts, patients; min, minute; N/A, not applicable

Figure 2.

Line charts comparing patient location data between radio frequency identification (RFID), Hidden Markov Model (HMM) predictions, and shadow data (top panels); Line charts comparing HMM location data for patients and providers (bottom panels). Data are displayed for 2 example patients

Provider workflow (number of patients seen per day per attending physician) obtained from the EHR was also compared to calculations based on raw RFID data and HMM data as an additional accuracy measure. We included 370 attending clinical sessions (half-days of clinic) in this analysis. The EHR showed attending physicians saw an average of 14.6 patients per session (SD=10.5), whereas the raw RFID data estimated that physicians saw 20.3 patients per session (SD=16.1) and the HMM model data estimated that physicians saw 14.4 patients per session (SD=12.3). With respect to patient flow, for any given minute, both raw RFID and HMM locations showed an average of 1 subject per exam room or VF room, which is accurate.

4.3. Process Times and Wait Times

Using the HMM smoothed location data for patients and physicians, we calculated process times and wait times for patients in the glaucoma clinic. The total process time for patients was on average 42.8 minutes (SD=27.5) and the total wait time was on average 47.9 minutes (SD=33.1). The average process time for new patients (n=187) seen in clinic was 77.2 minutes (SD=36.2) and the average wait time was 55.1 minutes (SD=33.7). For return visit patients (n=929), the average process time was 31.7 minutes (SD=20.3) and the average wait time was 42.6 minutes (SD=31.6). There was on average 13.3 minutes (SD=24.3) of time during a patient visit where the RFID system did not receive a signal from the patient tag and thus did not record a location. This is due to patients receiving ancillary testing in photography and ultrasound where there were no RFID readers. Table 4 summarizes wait and process times for steps in the glaucoma patient visit (reception, visual field testing, in-process waiting, exam room), and shows that patients wait the longest in the exam rooms and the second longest in the in-process waiting room.

Table 4.

Patient (n=1972) process time and wait time (using HMM smoothed location data)

Time (minutes)	N	Mean	SD	Min	Max	Median
Process Time
Total	1894	42.8	27.5	1	217	37
Attending
Physician	1463	11.9	8.3	1	77	10
Technician	1635	22.4	13.4	1	93	20
Visual Field	748	22.3	14.5	2	98	20
Wait Time
Total	1972	47.9	33.1	0	321	42
Reception	1972	12.3	16.0	0	142	7
IPW	1972	15.8	19.2	0	159	9
Exam	1972	19.9	21.7	0	222	13
No Signal Time
Total	1972	13.3	24.3	0	432	2

SD, standard deviation; Min, minimum; Max, maximum

note: statistics exclude data from exam room 6 which has too much interference with the physician/technician break room

note: Patients showing no process time likely due to physicians not wearing their tag or interference with tag

note: total process time includes time spent with attending physician, resident physician, fellow physician, technician, or medical assistant

5.0. DISCUSSION

Our design decisions allowed us to implement a real-time tracking system in the glaucoma clinic that was relatively low cost, portable and 80% accurate. Our system hardware and installation costs were $25,868 for the readers, $265 for RFID tags, $5400 for the RFID scanners and $195 for data storage for a total of $31,728 in capital investment, and $318 in recurring fees (data storage and replacement RFID tags). Our labor costs were limited and not comparable to non-academic settings as student teams lead the implementation and customization effort. The quote we received from the University’s commercial vendor (Stanley Healthcare, AeroScout Enterprise Visibility Solutions) to implement a comparable system was $290,550.80 without including the costs of wiring the clinic (an additional $34,500), replacement tags, smoothing the data or the yearly maintenance fee quoted at $40,000. Using commercially available hardware, we were able to create an RTLS system for quality improvement and research purposes for 10% of the vendor cost. Using the hardware and software design choices outlined in this manuscript would enable other investigators involved in quality improvement and clinical operations in large organizations to implement a relatively low-cost real time tracking system in the outpatient setting.

Our analysis of the RFID data demonstrated two key findings: 1) raw RFID data were not sufficiently accurate to inform clinical operations, but 2) Hidden Markov Modeling overcame these inaccuracies to make it valuable for clinical operations. We used the RFID data to understand that patients spent the majority of their wait times in the exam room and the second longest amount of time waiting in an in-process waiting area location inside the clinic. In both of these locations, wait time is not captured by EHR audit logs so alternative data capture strategies to accurately assess wait time using sensor technologies such as RFID, must be employed.

In our assessment of 6 months of clinical care, the RTLS system found that patients spent almost the same amount of time waiting (mean wait time 47.9 minutes) as they did being seen by all the members of the health care team (mean process time 42.8 minutes). When we looked at wait time and visit time by patient type, we found that new patients spent more time with providers than waiting (77.2 minutes and 55.1 minutes, respectively) but that return visit patients had the opposite experience, spending less time with their provider than waiting (31.7 minutes and 42.6 minutes, respectively). This is likely a reason for the higher rates of complaints about wait time from return visit patients compared to new visit patients collected in comment cards in the clinic, and is a focus for improvement for clinical operations.

The few non-industry sponsored research studies that have evaluated the accuracy of RFID-based RTLS systems have focused either on monitoring patients[18] or on monitoring single types of health care providers [14,17,19,20] whereas we needed a system that could measure the time each member of the healthcare team spent interacting with the patient. There has been one prior study describing how an RFID-based RTLS system was used to assess patient-provider face time and patient wait time in an outpatient setting, and it was carried out in a primary care clinic [21]. To date, there have been no studies describing the deployment and accuracy of RFID based RTLS systems in monitoring all healthcare providers (ophthalmic technicians, medical assistants, trainees, faculty physicians) and patients in a specialty clinic with non-linear work flow to better assess patient wait time. We found that the RTLS system was 80.6% accurate in the percent location match per minutes for patients and 77.9% accurate in percent location match for providers after smoothing the data with the HMM compared to direct observation. Though the accuracy could be further improved with more nuanced HMMs, we felt that this level of accuracy was sufficient for our clinical operations needs to compare wait times and process times from one-time period to another after enacting clinical operations changes.

While it seems intuitive that one must measure an agreed upon outcome both before and after making operations changes in order to best quantify the impact of the change, this process is often difficult to implement. For example, prior to implementing the RFID based RTLS system in the clinic, the two ways to measure wait time were either using total clinic visit time abstracted from the EHR as a proxy for wait time, or having providers in clinic undertake the highly labor intensive process of measuring patient wait times with a stopwatch. Having an automated system frees physicians and staff from the estimated 3 second (s) time it takes to manually record start times and 3 s for end times of visits [22]. If ophthalmologists see 33 patients per day, this system would save 198 s/day over 240 work days/year, leading to a savings of 792 minutes per year. At 15 minutes per patient encounter, an automated RTLS system would allow an ophthalmologist to see an additional 52 patients per year while harnessing information that would engage the clinic in a process of continuous quality improvement [21] from operations policy changes to schedule optimization to reduce patient wait times [5]. For example, one of the next projects for our clinic using this RTLS data is to test the impact of a refraction (measurement for glasses) policy change on wait time and process time. Being able to accurately and passively measure the impact of clinical operations decisions on patient wait times and process times could greatly improve the ability of healthcare organizations to make evidence- based operations decisions.

There are a number of reasons why the raw RFID RTLS data was less accurate than the HMM processed data compared to direct observation. In the clinical space, RFID readers are located in relatively close proximity to one another leading to situations where tags are picked up by multiple RFID readers at the same time. The data processing is further complicated by the attenuating effects of both liquid and metal. For example, if a person (mostly liquid) is turned in such a way that they are between the reader and the tag they are wearing, the signal will be decreased in that room and may in fact be picked up in an adjacent room at a higher relative strength. Similarly, metal blocks RFID signals. In the ophthalmology office, the slit lamp biomicroscope used to examine patients is made of metal. When the provider brings the biomicroscope in front of the patient for the exam, the signal is blocked during that time (Appendix 2) so an adjacent room signal may then be stronger than the room the patient is actually in. Because of these issues, we needed to use HMMs to assign probability weights to location transitions in order to smooth the raw RFID data and improve our accuracy before using the data to inform clinical practice. This need for post-processing using HMMs [17] may be why previous studies have found that off-the-shelf RFID based RTLS systems were only moderately accurate and could not be used reliably to inform clinical practice [14]. The need for customized algorithms to smooth the RFID based RTLS data presents another barrier for scaling this technology, though our approach using HMM modeling could be used by others to overcome this barrier.

Limitations

Though RFID hardware can be used off the shelf, both the middleware implementation and the post-processing of the data using HMMs must be customized to the use case, making dissemination somewhat labor intensive. However, once those systems have been customized up-front, they require little additional resource to continuously produce data. Purchasing the RFID hardware off the shelf is still costly, but as RFID components become less and less expensive, the hardware is likely to decrease in price as a group that custom built their RFID readers did so for $142 each compared to the $1100 that we paid for each reader in this study [18]. As other sensors, such as Bluetooth beacons, [22] become less expensive than the RFID based technology, it will make more sense to use them in the future. Even after instituting HMM to smooth the RFID locations, the data were only 80% accurate compared to manual time-motion studies. More nuanced HMM models would likely improve this accuracy and will be a focus of future work. However, in quality improvement initiatives, we will be comparing our wait times from one week to another, so having 100% accurate wait times is not imperative as we are looking for relative improvement over time. Dilation time is currently included as wait time as there is no way to distinguish time in an exam room without a provider while dilating from wait time; this can be addressed in the future with simulation modeling that subtracts mean dilation time from wait time among patients receiving dilation. A final limitation is that the data from the RFID tag to the reader and from the reader to the SQL database were not encrypted during transit; further programming would be necessary to mitigate this potential data risk prior to widespread clinical implementation.

6.0. CONCLUSION

RFID-based real-time tracking of patient and provider locations provides large volumes of passively collected, granular data to facilitate data-based decision making regarding improving clinical operations.

Table 1.

Tag Type with Read Ranges and Prices

Type of RFID		Battery	Frequency	Max Read Range	Cost per Tag
Active		Present	433 – 915 MHz	1000 m	$50 – $100
Passive	Low Frequency	Absent	125–134 KHz	0.6 m	$0.75
	High Frequency	Absent	13.56 MHz	1.2 m	$0.50
	Ultra High Frequency (UHF)	Absent	860–960 MHz	20 m	$0.09 – $1.50

Highlights.

• RFID sensors were used in an ophthalmology clinic for passive real time localization

• Sensors had a sensitivity to identify location to a spatial position of <1meter

• Sensors had a sensitivity to identify spatial position at a frequency of <1 minute

• Customizing off-the-shelf passive RFID sensors enabled system cost to be 10% of vendor price

• Hidden Markov Modeling of the raw data gave ≥80% accuracy compared to direct observation