Measuring intraoperative surgical instrument use with radio-frequency identification

Abstract

Objective

Surgical instrument oversupply drives cost, confusion, and workload in the operating room. With an estimated 78%–87% of instruments being unused, many health systems have recognized the need for supply refinement. By manually recording instrument use and tasking surgeons to review instrument trays, previous quality improvement initiatives have achieved an average 52% reduction in supply. While demonstrating the degree of instrument oversupply, previous methods for identifying required instruments are qualitative, expensive, lack scalability and sustainability, and are prone to human error. In this work, we aim to develop and evaluate an automated system for measuring surgical instrument use.

Materials and Methods

We present the first system to our knowledge that automates the collection of real-time instrument use data with radio-frequency identification (RFID). Over 15 breast surgeries, 10 carpometacarpal (CMC) arthroplasties, and 4 craniotomies, instrument use was tracked by both a trained observer manually recording instrument use and the RFID system.

Results

The average Cohen’s Kappa agreement between the system and the observer was 0.81 (near perfect agreement), and the system enabled a supply reduction of 50.8% in breast and orthopedic surgery. Over 10 monitored breast surgeries and 1 CMC arthroplasty with reduced trays, no eliminated instruments were requested, and both trays continue to be used as the supplied standard. Setup time in breast surgery decreased from 23 min to 17 min with the reduced supply.

Conclusion

The RFID system presented herein achieves a novel data stream that enables accurate instrument supply optimization.

Lay Summary

Surgical instrument oversupply drives cost, confusion, and workload in the operating room. With an estimated 78%–87% of instruments being unused, many health systems have recognized the need for supply refinement. By manually recording instrument use and tasking surgeons to review instrument trays, previous quality improvement initiatives have achieved an average 52% reduction in supply. Despite these successes, methods for identifying required instruments are expensive, qualitative, lack scalability and sustainability, and are prone to human error. In this work, we develop and evaluate an automated radio-frequency identification (RFID) system for measuring surgical instrument use. Over 15 breast surgeries, 10 carpometacarpal (CMC) arthroplasties, and 4 craniotomies, instrument use was tracked by both a trained observer manually recording instrument use and the RFID system. The RFID system achieved near perfect agreement with the observer and enabled an instrument supply reduction of 50.8% in breast and orthopedic surgery. Over 10 monitored breast surgeries and 1 CMC arthroplasty with reduced trays, no eliminated instruments were requested, and both trays continue to be used as the supplied standard. Furthermore, setup time in breast surgery decreased from 23 min to 17 min with the reduced supply.

BACKGROUND AND SIGNIFICANCE

Surgical instruments are fundamental to surgical procedures, which are the leading revenue driver for health systems.¹ Despite their criticality, most health systems struggle to manage this essential asset. Instruments are typically stored in trays, some containing up to 188 instruments.² While hospitals monitor when a preconfigured tray is sterilized, where it is stored, and when it is sent to an operating room (OR), no information is gathered on the utilization of individual instruments. As such, instrument supply is predominantly determined historically. The surgical team is expected to refine instrument trays/supply (preference cards), yet while instruments are often added, few are removed. This trend toward excess incurs a large efficiency cost. An estimated 78%–87% of instruments in the OR go unused, introducing unnecessary costs in the form of cleaning and processing, delayed surgical operations due to missing, dirty, or broken instruments, increased workload of nursing assistants, and increased instrument wear.³

Many health systems are experiencing these challenges. Over the past 6 years, at least 15 quality improvement initiatives across 10 different hospitals and 14 service lines documented cost savings by identifying and eliminating unused instruments.^4–15 On average, a 52% reduction in supply was realized (35%–37% fixed effects and 41%–65% random effects (95% CI)) (Figure 1). Central to the success of all instrument supply optimization initiatives is the acquisition of data describing instrument usage. In the past, necessary instruments have been identified by enlisting observers to manually record use (ethnographers) and/or organizing committees to review instrument supplies. Despite the successes of these initiatives, cost savings have been diminished by the investment required to support personnel in monitoring operations, the opportunity cost of diverting surgeon effort, and their limited scalability. Optimizing instrument supply across an entire institution requires precise knowledge of the needs of each surgeon completing every procedure. For a general tray used in 4 service lines by 10 surgeons, organizing all stakeholders to review supply or sponsoring the observation of every relevant operation is logistically challenging and can be cost prohibitive.¹⁶ These methods are further hindered by subjectivity and lack quantitative data to guide reduction and drive adoption.

Figure 1.

Surgical instrument reduction studies.

RFID as a foundational technology is well positioned to automate intraoperative instrument tracking. RFID hardware consists of a reader, reader antennas, and tags. The reader adheres to a communication protocol that sends signals through antennas. Signals are focused by antenna gain and activate tags within range. The tags respond through backscatter, where an impedance is switched on and off, encoding identity. The reader receives the signal and decodes the tags’ identity. RFID does not require line of sight, is economical, and is already used in the OR, making it well suited for tracking surgical instrument use. Despite these advantages, adoption of RFID by health systems has been limited. Almost all RFID systems in healthcare target the tracking of large equipment and patients.^17–21 Few RFID systems for instruments have been commercialized, and all target either the identification of retained surgical instruments or the automation of instrument counts.^22,23 While these solutions can improve patient safety and surgical workflow, they do not identify which instruments are used in an operation.

While previous research has focused on improving the accuracy of the preference card (a list of instrument trays and supplies to be provided for a procedure), very little work has gone down to the instrument level.²⁴ Yoshikawa et al.²⁵ leveraged RFID-tagged surgical instruments to collect proximity data at 13.56 MHz. The system recorded instrument use but required the surgical team to scan each instrument. Similarly, Yamashita et al.²⁶ used a 13.56 MHz mat antenna on the Mayo stand for intraoperative tracking. Mayo stands act as a staging area for instruments that are anticipated as being necessary by the scrub nurse. However, instruments that are used by the surgeon may never touch the Mayo stand and instruments that are not used are often placed on the stand, making it a poor proxy for use. Moreover, a system monitoring the Mayo stand is only applicable to operations with Mayo stands.

OBJECTIVE

With this understanding, the goal of this study was to develop an automated RFID system that measures intraoperative surgical instrument use.

System design

Our team observed craniotomy for tumor operations, Chiari malformation decompressions, CMC arthroplasties, breast excisional biopsies, breast lumpectomies, axillary sentinel lymph node biopsy, sentinel lymph node biopsies, pelvic ring fracture repairs, kidney transplants, acetabulum fracture repairs, and radial fracture repairs. By documenting the surgical approach for each operation and interviewing stakeholders at all levels of the hospital ecosystem, we accrued a list of design criteria:

• The RFID system must not impede workflow;

• Line of sight is not guaranteed (instruments are often covered by a surgeon’s hand or biomaterial);

• The surgical site is varied, and Mayo stands are not ubiquitous;

• Duplicate instruments must be uniquely identified;

• The functionality of the instrument must not be impacted;

• All hardware must be sterile or outside the sterile field; and

• The tracking technology must be economical to apply to thousands of instruments.

These criteria informed design choices and were leveraged in evaluating system prototypes.

There are two types of RFID: passive and active. In active RFID, both the reader and the tag have onboard power supplies. Both components can amplify signals that enable communication over greater distances; however, active tags are larger than passive tags. As attachment to a surgical instrument necessitates size minimization, passive RFID was chosen for this work. Passive RFID communication range is bounded by the radiated power limit and the maximum size of the tag. The Federal Communications Commission (FCC) regulates multiple RFID bands: low frequency (130 kHz), high frequency (13.5 MHz), ultrahigh frequency (UHF) (915 MHz), and microwave frequency (2.4 and 5.6 GHz). Hardware is more accessible and affordable at lower frequencies, and at 2.4 and 5.6 GHz, passive tags are not commercially available. However, as antennas are designed around the wavelength of communication, and wavelength is inversely proportional to frequency, lower frequencies generally require larger antennas. To enable tags small enough to fit on surgical instruments while minimizing cost, UHF is currently best. Autoclavable tags purchased for this study cost $3.79/ea, 2 antennas cost $46.99/ea, and the reader cost $922.00. Thus, the burden of instrument mismanagement far outweighs the cost of an RFID system. The FCC limits the output power of UHF readers to 1 W, with a maximum reader antenna gain of 6 dBi.²⁷ Antennas in proximity to humans are further limited to have an effective isotropic radiated power of less than 0.61 mW/cm².²⁸ With a 2 × 3 × 10 mm tag, reliable communication range over FCC-compliant transmit power is limited to about 1 m.

With these principles in mind, the UHF RFID system was designed to be remote of the sterile field while viewing only the surgical site. Before surgery, RFID tags (Xerafy Ltd.²⁹) were attached to instruments and a decoder database pairing the tag ID to instrument information was developed. The tagged instruments were steam-sterilized, and the antenna system was installed in the OR before each operation. As instruments were used at the surgical site during the operation, the proximity between the tags and the reader antennas enabled reads. Throughout the operation, tag read data were written to an SD card by the reader. After monitoring multiple surgeries, software was written to generate a master list of all previously used instruments to be supplied in subsequent surgeries. The process flow for optimizing instrument supplies with the RFID system is shown in Figure 2.

Figure 2.

Workflow schematic of RFID-informed instrument supply optimization.

MATERIALS AND METHODS

Data collection

To characterize the accuracy of the system, 4 craniotomies, 10 CMC arthroplasties, and 25 breast surgeries were monitored. These operations were targeted because the equipment and surgical approaches are unique. By refining the system for ease-of-use in this varied subset of surgeries, we aimed to improve the generalizability of the system and results in application to operations outside the scope of this work.

For each targeted surgery, we obtained the preference card from the surgeon, identified the tray that contained the most instruments, and attached RFID tags with surgical instrument marking tape (Figure 3). Each tagged instrument was scanned into a database pairing tag ID numbers to instrument information. Tags and instrument tape are autoclave-compatible, and tagged instruments were processed through the conventional instrument sterilization cycle at Duke University Hospital. Before each surgery, the scrub nurse retrieved the tagged tray from the designated storage area, set up the instruments in standard fashion, and a team member placed the antennas to view the surgical site. After surgery, our team retrieved the antenna system, collected the SD card from the reader, and uploaded the information to a secure storage site.

Figure 3.

RFID-tagged surgical instrument.

Concurrent with each surgery, a team member manually recorded the onset-of-use for each instrument. Onset-of-use was defined as an instrument entering the operative field in the operator’s hand. A new use was recorded every time an instrument re-entered the field. Ethnographers were trained to identify instruments individually and leveraged color-coded instrument tape to improve the ease and accuracy of their recording. The surgeon’s name and procedure type were also recorded, and the operation proceeded per standard methods. The study was deemed exempt by Duke’s institution review board (Pro00100079), and no patient health information was collected.

Data processing

Four data types were collected: the decoder database linking tag IDs to instruments, the database of RFID data from each monitored surgery, the database of ethnography (onset-of-use) logs, and the database of surgical information (case times, operation type, observer notes, and surgical team names). Prior to analysis, RFID data logs were cropped to remove reads recorded before the start time and after the end time of each surgery. From the cropped RFID data and ethnography logs, we calculated the true positive rate (TPR) and false positive rate (FPR) of the system by considering the observer’s ethnography as ground truth and every RFID read as an instance of use. The Cohen’s Kappa agreement between the observer and the system was calculated from each positive or negative use recording over all 39 cases (3272 events).³⁰ To determine the number of surgeries necessary to monitor before eliminating excess supply, we also calculated the number of new instruments added to an inclusive list of instruments used in any prior surgery and plotted the number of new instruments by the chronological case number. All analysis was completed in MATLAB R2019b or Python 3.7.

Reduced supply evaluation study

In breast surgery, we configured a tray to contain only instruments the system recorded as being used over the first 15 cases and supplied the reduced tray during the subsequent 10 surgeries.³¹ Congruently, a reduced tray was also supplied in a 10th CMC arthroplasty operation. The instruments that were removed from each of the trays were made available in a separate, unopened tray. If the surgeon requested an eliminated instrument, the instrument could be retrieved from the excess tray. To gauge the effects of a reduced supply, we weighed instruments before and after supply reduction and compared the setup durations from operative timestamps in breast surgery. We also informally evaluated the experience of the surgical team: Did RFID tags impact the utility of a surgical instrument in any way? Did the RFID system get in your way?

RESULTS

Of the 3 surgeons and 6 nurses who interacted with the RFID system, none found the RFID tags or system to be obtrusive to surgical workflow. No unsterile equipment entered the sterile field, and there were no instances of postoperative infection. In breast surgery, the system identified 37 out of 62 (59.7%) supplied instruments as being used over the first 15 operations. Over 9 CMC arthroplasties, 53 out of 121 (43.8%) supplied instruments were recorded as used by the RFID system. Due to hospital-mandated restrictions put in place for COVID-19, only 4 neurosurgeries were monitored, making the used instrument list inconclusive. From breast and orthopedic surgery, 50.8% supply reduction was identified and implemented. In the orthopedic surgery and 10 breast surgeries with a reduced supply that were monitored, no eliminated instruments were required. Furthermore, the optimized trays in both breast and orthopedic surgery have been adopted as the standard supply and continue to be used in operations.

All reads from a single breast lumpectomy and sentinel lymph node biopsy operation are displayed in Figure 4. The vertical axis contains each surgical instrument that was logged throughout the operation. Use instances for each instrument are plotted on a timeline with surgical events depicted by vertical lines. Blue circles correspond to reads from the RFID system, while green ‘x’s represent the ethnographer’s observed onset of instrument use. Of the 62 tagged instruments supplied in this operation, only 27 are plotted here as the remaining 35 were neither recorded by the RFID system nor the observer. The security clamp was recorded as used by an observer but was missed by the RFID system. This is a false negative of the system. Similarly, false positives of the system correspond to RFID reads that were not recorded as used by the ethnographer. To gauge the sensitivity and specificity of the system, Figure 5 plots the TPR by the FPR calculated from each individual surgery (small circles) on a receiver operating characteristic plot. Large circles represent cumulative results from all surgeries accomplished by the same surgeon in a service line (breast, neuro, ortho). Over all 39 monitored cases, the system achieved a sensitivity of 93.8% and specificity of 80.8%. The agreement between the RFID system and human ethnography was near perfect, with a Cohen’s Kappa statistic of 0.81 (95% CI, 0.790–0.83). To gauge the precision of ethnography, 2 observers simultaneously monitored 2 CMC arthroplasties. Their average interrater agreement was 0.95 (near perfect agreement).

Figure 4.

Surgical instrument RFID and observer log from a breast lumpectomy and sentinel lymph node biopsy.

Figure 5.

Receiver operating characteristic of the RFID system.

While not statistically significant, median setup time in breast surgeries decreased from 23 min to 17 min after eliminating unused instruments (P = .23). Similarly, the weight of the instruments was reduced from 2.7 kg (62 instruments) to 1.9 kg (37 instruments) in breast surgery and from 5.5 kg (121 instruments) to 2.4 kg (53 instruments) in orthopedic surgery. This measurement does not include the weight reduction made possible by using a smaller tray.

To account for variation in surgical instrument use between congruent operations, multiple operations were monitored before optimized instrument trays were configured. To gauge how many surgeries are necessary to monitor before supplies can be reduced, we plotted the number of new instruments used in each chronological surgery (Figure 6). New instrument use follows an exponential decay in all surgery types with an average half-life occurring between surgeries 2 and 3.

Figure 6.

Number of monitored surgeries necessary to capture surgeon variation in instrument use.

DISCUSSION

Eliminating unused surgical instruments

The system achieved an average supply reduction of 50.8%, and no eliminated instruments were required in following cases. While this level of reduction is on par with the reduction achieved by similar quality improvement initiatives,^4–15 a previous quality improvement initiative in breast surgical oncology may have impacted the degree of reduction. The general breast tray had been previously paired down from 113 to 62 instruments based on the surgical teams’ anecdotal knowledge of instrument use. Despite applying the RFID system to an already-refined tray, the system realized a further reduction of 40.3% (62 to 37 instruments). This outcome suggests that current methods for determining instrument utility could be improved with quantitative data describing instrument use.

Eliminating unused surgical instruments from supply increases the efficiency of surgery by reducing workload and cost. While this study did not include a cost analysis, savings are anticipated to stem from reducing instrument processing burden (direct cost estimated at $0.77/per instrument per cycle), instrumentation errors (rust, contamination, missing/broken, etc), count discrepancies, the prevalence of retained surgical instruments, the frequency of instrument maintenance and acquisition, the number of instruments and corresponding storage infrastructure, and expediting OR setup and takedown.^{3,4,10,32–34} Rates of infection may decrease due to decreased traffic in and out of the OR from nurses retrieving missing instruments and an improved quality of sterilization due to reduced workload.^3,35 Staff turnover may also decrease as instrumentation error often catalyzes conflict between the sterile processing department and the OR. Eliminating instrumentation could help maintain tray weights below the 25 lbs. mandate,^31,36 and operative capacity may increase due to faster OR setup and takedown and fewer instrument-related delays. Ultimately, accurately eliminating unused instrumentation improves the value of healthcare by reducing cost and improving quality.

The accuracy of intraoperative RFID use tracking

As none of the surgeons requested eliminated instruments in follow-on surgeries with reduced supplies, the RFID system accurately identified unnecessary instruments. When measured against manual ethnography, the system achieved a Cohen’s Kappa statistic of 0.81 (95% CI, 0.790–0.83), or near perfect agreement. Congruently, the cumulative sensitivity of the system was 93.8%, and the specificity of the system was 80.8%. Therefore, almost all instruments that were used were recorded by the RFID system, while some that were not used were also recorded. We explored false positives of the system and linked many of them to operational abnormalities. For example, over the first 15 breast surgeries, 3 instruments were singularly recorded as used by the system. During the operation, a medical student had brought the instruments near the surgical site without applying them. The proximity was sufficient for the RFID system to record each instrument. False negatives of the system appeared to correlate with specific instrument geometries. As metal reflects RF, flat geometries that are oriented to shield the tag from the receiver effectively limit communication. Consequently, flat knife handles were missed more than any other instrument. To improve their readability, tags should be mounted on the side profile rather than the face, something that was not done in this study.

The relative importance of specificity in implementing supply change with RFID data is less than sensitivity. If an unnecessary instrument remains in supply, the cost is the opportunity cost of its removal. If the system misses an instrument that is used and it is removed, subsequent surgeries may require the missing instrument, potentially impacting patient safety and surgical workflow. However, as more surgeries are monitored and the body of data grows, the risk of eliminating necessary instruments decreases. To further reduce the possibility of eliminating necessary instruments, we provided surgeon stakeholders with a list of instruments that would be removed. In the orthopedic and breast surgical oncology pilots, the supply change was approved, and reduced trays were configured.

Implementing instrument supply change with RFID data

In this study, every RFID read was interpreted as an instance of use and every used instrument deemed necessary to future supply. Another way to interpret this data is by calculating a usage percentage for each instrument from the fraction of operations it was used in. For example, if an instrument was recorded in 1 of 15 operations, its usage percentage would be 6.7%. By eliminating instruments used in less than 10% of operations, instruments that were erroneously recorded could be eliminated from future supply. Furthermore, instruments with a usage percentage between 10% and 25% could be supplied in a separate tray or peel pack and would need to be reprocessed only if the tray or peel pack were opened for use. This methodology caters to institutional preferences and gains power with a growing body of data. A large dataset of monitored surgeries would improve the cumulative TPR. The FPR would also increase, but by leveraging a usage percentage cutoff, its effects could be negated.

The variation of used instrumentation is largely the same between surgery types. In Figure 6, the exponential decay of new instruments used by each surgeon for each service has approximately the same half-life. This allows for an approximate forecast of the number surgeries that are necessary to monitor before reducing supplies. In practice, the number of new instruments can be evaluated after each operation, and supply change can be implemented after several cases with no new instruments logged.

In this study, we engaged each surgeon to review the list of instruments to be removed from their trays before making the change. Their experience facilitated the review and presenting the data describing their history of use helped garner support for eliminating unused instruments. Engaging surgeons in this manner is critical when the number of monitored cases is low. If a seldomly used instrument was not applied in any of the monitored cases, it could be eliminated despite its indispensability to rare surgical occurrences. Surgeons’ approval of the supply change allows for a check before unused instruments are removed. As more cases are monitored and the body of collected data grows to encompass rare surgical occurrences, usage statistics begin to account for all use cases. In this way, RFID-captured usage data have the potential to become a critical tool for administrative stakeholders to engage their clinical teams as they perform supply optimizations.

The full value of the system is achieved when permanently applied to an OR and run in the background during all operations. A major limitation to quality improvement initiatives is their lack of sustainability. After the initiative, instruments seep back into supply as preferences change. Without recurring diligence, there will always be an imperfect supply. With an automated system collecting a history of use, instrument supply can be continuously updated to facilitate changing preferences while minimizing waste.

Limitations

The greatest limitation of this study is that only 3 surgeons were monitored. General trays are used by many surgeons for many types of operations, and all relevant procedures would need to be monitored before an optimized supply could be realized. Therefore, the main goal of this work is to demonstrate the utility of the RFID system in measuring intraoperative instrument use. With this system, all procedures relevant to an instrument tray can be monitored efficiently and supplies can be optimized across surgeon populations.

Limitations of the RFID system revolve around attaching RFID tags to surgical instruments. While tape provides an autoclavable method for fixation, concerns with sterilizability, degradation, and possibility for patient-retained tape have been voiced across the industry.³⁷ Applying tags to instruments with tape is also time consuming. A surgeon and medical student required 5 h to tag 124 instruments. Developing a more efficient and reliable method for affixing tags to instruments is a future direction of this work.

CONCLUSION

In this study, we developed an automated RFID system for recording surgical instrument use. We demonstrated the utility of the system in identifying unused instruments, characterized the accuracy of the system in 3 types of surgery, developed a framework for implementing supply change with intraoperative RFID data, and identified limitations and future directions of this work.

FUNDING

This work was supported in part by the Duke Institute for Health Innovation.

AUTHOR CONTRIBUTIONS

IH conceived and designed the RFID surgical instrument tracking system. Authors IH, LO, JH, WH, MJR, LHR, and PJC designed this study. Ethnography was performed by IH, LO, JH, EL, JW, PK, and JG. The original manuscript was drafted by IH, and all authors revised this work.