Abstract
Background
The Irish National Orthopaedic Register (INOR) provides a national mechanism for managing data on THA and TKA in Ireland, including a detailed implant record populated by intraoperative implant bar code scanning. It is critically important that implant details are recorded accurately for longitudinal outcome studies, implant recalls, and revision surgery planning. Before INOR’s 2014 launch, Irish hospitals maintained separate, local institutional arthroplasty databases. These individual databases typically took the form of hardcopy operating room (OR) logbooks with handwritten patient details alongside the descriptive stickers from the implant packaging and/or individual institution electronic records using manual electronic implant data input. With the introduction of the INOR, a single, unifying national database was established with the ability to instead collect implant data using bar code scanning at time of implant unpackaging in the OR. We observed that bar code data entry represented a novel and potentially substantial change to implant recording methods at our institution and so sought to examine the potential effect on implant data quality.
Questions/purposes
We compared the new bar code scanning method of implant data collection used by the INOR to the previously employed recording methods at our institution (in our case, the previous methods included both an electronic operation note database [Bluespier software] and a duplicate hardcopy OR logbook) and asked (1) Does bar code scanning improve the completeness of implant records? (2) Does bar code scanning improve the accuracy of implant records?
Methods
Although the INOR was launched in 2014, our institution went live with it in 2019. To avoid any potential recording issues that may have occurred during the 2019 introduction of the novel system, a clear period before the introduction of INOR was selected at our institution to represent an era of manual data input to Bluespier software: July 2018. Although we initially aimed for 2 months of data from July 1, 2018, to August 31, 2018 (n = 247), we decided to proceed to 250 consecutive, primary THAs or TKAs for clarity of results. No procedure meeting these criteria was excluded. A second recent period, January 2021, was identified to represent an era of bar code data input; 250 consecutive, primary THAs or TKAs were also included from this date (to February 15, 2021). No case meeting these criteria was excluded. A total of 4244 implant parameters from these 500 primary THAs or TKAs were manually cross-referenced for missing or incorrect data. Eleven THA and six TKA parameters were chosen for comparison, including implant names and component sizes. For each case, either the 2018 Bluespier electronic record or the 2021 INOR electronic record was manually interrogated, and implant details were recorded by two authors before they were compared against the duplicate record for every case (the reference-standard OR logbook containing the corresponding implant product stickers) for both completeness and accuracy. Completeness was defined binarily as the implant parameter being either present or absent; we did likewise for accuracy, either that parameter was correct or incorrect. The OR logbooks were chosen as the reference standard because we felt the risk of product stickers containing errors (inaccuracies) was negligible, and in our collective experience, missing stickers (incompleteness) has not been encountered. Logbook case completeness was also confirmed by comparison to our inpatient management system.
Results
With the introduction of the automated bar code data entry in the INOR, the proportion of missing data declined from 7% (135 of 2051) to 0% (0 of 2193), and the proportion of incorrectly recorded implant parameters declined from 2% (45 of 2051) to 0% (0 of 2193). The proportion of procedures with entirely accurate implant records rose from 53% (133 of 250) to 100% (250 of 250).
Conclusion
The completeness and accuracy of implant data capture was improved after the introduction of a contemporary electronic national arthroplasty registry that utilizes bar code data entry.
Clinical Relevance
Based on the results of this study, other local and national registers may consider bar code data entry in the OR to achieve excellent implant data quality. Future studies may examine implant data quality at a national level to validate the bar code–populated data of the INOR.
Introduction
Arthroplasty registers have been successfully implemented at local and national levels worldwide. Methods for implant data capture differ among registers, and the ideal method for data capture remains undetermined, with various registers using paper forms, manual electronic data input, or automated data input using bar code scanning [2]. The Irish National Orthopaedic Register (INOR) utilizes a customized, prospectively maintained implant library to enable bar code scanning of the implant unique device identifier (UDI) in the operating room (OR). In the event of revision arthroplasty or a device recall, it is of utmost importance that implant details are accurately recorded at the time of the index procedure [8, 14]. Before the introduction of the INOR, individual institutions used local logbooks and/or manually entered data to local electronic databases.
Previous studies have reported significant challenges in implementing bar code scanning systems for implant data entry. Paxton et al. [13] described an initially unsuccessful attempt to introduce bar code scanning at their institution due to software challenges and the need for dual documentation. Furthermore, technical issues with scanners, inability to scan devices unknown to that electronic library, and staff preferences for manual data input resulted in further difficulties [13]. Anecdotally, similar challenges were also encountered with the simultaneous introduction of the INOR and its bar code data collection method. However, the accuracy or completeness of this register’s implant data while using this system remains completely unknown.
We therefore compared the bar code scanning methods implemented as part of the INOR introduction with the legacy recording methods at our institution and asked: (1) Does bar code scanning improve the completeness of implant records? (2) Does bar code scanning improve the accuracy of implant records?
Patients and Methods
Study Design and Setting
We performed a retrospective, comparative study utilizing three separate, longitudinally maintained databases: (1) a local institutional electronic operation note database, Bluespier (Bluespier Software, Clanwilliam Group), which was used for implant data collection before 2019 using manual data entry methods; (2) the INOR database, which replaced Bluespier at our institution in 2019 and instead uses bar code data entry; and (3) hardcopy OR logbooks, which were used across both periods and which contain duplicate data to either of the electronic systems above.
Although launched in 2014, the rollout of the INOR across Ireland has been gradual, with the first annual report published in 2022. Our institution is the most recent to go live with the registry in 2019 due to the challenges and capacities of this rollout. The register is now active in eight public hospitals and one private hospital. The INOR currently captures primary and revision hip and knee arthroplasty procedures, including patient-reported outcome measures, with plans to include upper limb and ankle arthroplasty. Revision procedures are automatically linked to the previously captured primary procedure.
Patients
To avoid any potential recording issues in 2019 during the transition from Bluespier to INOR at our institution, we identified a clearly distinguished “before” period to represent an era where manual data input to Bluespier took place: July 2018. During this time, standard practice was for the surgical registrar (a resident) to type the operation note to a Bluespier template immediately after each case, in an area adjacent to the OR. This electronic entry required the recall and input of the implants used during the procedure, although the implant stickers were easily available to verify in cases of uncertainty. Although we initially aimed to obtain 2 months of data from July 1, 2018, to August 31, 2018 (n = 247), we decided to proceed to 250 consecutive, primary THAs or TKAs for clarity of results; no case meeting these criteria was excluded.
We identified a second “after” period to represent an era of bar code data input using the newer INOR system: January 2021. Two hundred fifty consecutive, primary THAs or TKAs were also included from this date (to February 15, 2021); no case meeting these criteria was excluded. During this time, standard practice was for the registrar to type the operating note to an INOR electronic template (using an online national portal) immediately after the procedure in an area adjacent to the OR. However, implant details were instead prepopulated (and not editable by the registrar) to this operation note by way of bar code data entry and so no implant details were typed into the note. Instead, an OR nurse scanned the implant stickers immediately at the time of unpacking in the OR.
During both periods, each of our institution’s seven operating rooms maintained a supplementary record in the form of a hardcopy OR logbook. This OR logbook is historical practice at our institution that continued after the introduction of electronic recording and is now a redundant practice for routine clinical purposes. The OR nurses record the implants used during each procedure by placing the accompanying implant information stickers in the logbooks. These logbooks were taken as the reference-standard for this study; we compared either the manually entered Bluespier implant record or the bar code–entered INOR implant record to these logbooks. We felt it was appropriate to use the OR logbook as the reference-standard for this study as in our collective experience, we have not encountered a case of missing implant stickers in this hardcopy database (record incompleteness), and we felt the risk of typographical errors on the implant stickers (record inaccuracy) was negligible.
Furthermore, all patients admitted for surgery at our tertiary referral arthroplasty unit were checked in using our local electronic in-patient management system (IPMS) on the day of surgery. We referenced the IPMS against our three databases (described previously) to ensure that all patients who were admitted to the hospital for primary THA or primary TKA during these two chosen periods were captured by those databases, thus further ensuring a consecutive series of patients for each period.
The Pinnacle® cup and Corail® stem (both DePuy, Johnson & Johnson) was the most common THA system seen, and the Triathlon® (Stryker Orthopaedics) was the most common TKA system (Supplementary Table 1; http://links.lww.com/CORR/A844). We did not include revision arthroplasties and partial knee arthroplasties in this study to maintain clarity of results.
Data Capture and Variables
Of these 500 records (250 manual Bluespier records and 250 bar code INOR records), 4244 implant parameters were cross-referenced by two physician authors (SPR, SDO) against the reference standard OR logbook. Implant parameters were included for cross-reference if we felt that information was of importance for future revision surgery, device monitoring, or device recall (Table 1). We selected 11 THA and six TKA critical implant parameters for inclusion. When a mismatch occurred between either of the electronic records (Bluespier or INOR) and the OR logbook, each record was rechecked again to rule out a recording error. Additionally, in three instances, we reviewed postoperative radiographs to further confirm the presence of acetabular screws when Bluespier records did not include acetabular screw placement when compared with the OR logbook.
Table 1.
Critical implant parameters selected for inclusion, with percentage of incomplete or inaccurate data before and after introduction of INOR automated data input
| Procedure | Implant parameter | Inaccurate/incomplete frequency | |
| Pre-INOR (n = 107) | Post-INOR (n = 137) | ||
| THA | Shell system name Shell diameter in mm Screws (yes or no) Liner material Head diameter in mm Head offset in mm Head material Stem system name Stem size Stem offset or neck angle Femoral plug (if cemented) in mm |
4 (4) 3 (3) 7 (8) 1 (1) 7 (7) 39 (42) 14 (15) 2 (2) 12 (13) 21 (22) 10 (11) |
0 0 0 0 0 0 0 0 0 0 0 |
| Pre-INOR (n = 143) | Post-INOR (n = 113) | ||
| TKA | TKA system name Femoral size Prosthesis design (CR/CS/PS/TS) Tibial insert thickness in mm Tibial size Patellar resurfacing (yes or no) |
1 (2) 2 (3) 2 (3) 24 (35) 3 (5) 3 (4) |
0 0 0 0 0 0 |
Data presented as % (n); CR = cruciate retaining; CS = cruciate substituting; PS = posterior stabilized; TS = total stabilizer.
The number of screws used, liner offset, whether the liner was neutral or lipped, tibial baseplate subtype, and patellar resurfacing implant sizes were not recorded as we felt these details were of secondary clinical significance to the critical parameters that were chosen (Table 1).
Primary and Secondary Study Outcomes
Our primary study goal was to ascertain the completeness of implant data entry of either the Bluespier record or the INOR record when compared with the reference standard. To achieve this, completeness was defined binarily as the implant parameter being either present or absent for that record.
Our secondary study goal was to evaluate the accuracy of the implant data of either the Bluespier record or the INOR record when compared with the reference-standard OR logbook, and likewise, accuracy was defined binarily as the implant parameter being correct or incorrect for that record. These objectives were achieved by manual documentation of the implant parameters from each of these three databases to an electronic spreadsheet for comparison.
Ethical Approval
Ethical approval was granted by our institutional research ethics committee.
Statistical Analyses
We used Microsoft Excel® (Microsoft) for database management and descriptive statistical analysis.
Results
Completeness of Records
After the introduction of the INOR with bar coding, the proportion of missing data declined from 7% (135 of 2051) to 0% (0 of 2193).
Accuracy of Records
The proportion of incorrectly recorded implant parameters declined from 2% (45 of 2051) to 0% (0 of 2193) after the implementation of the INOR with bar coding input.
The proportion of procedures with entirely accurate and entirely complete implant records rose from 53% (133 of 250) to 100% (250 of 250).
Regarding the before data, individual parameters that would be considered most important, such as implant system name, tended to have a lower error rate (1% to 4%) than parameters of less importance, such as femoral plug size (10%) (Table 1).
Two instances of misinformation were identified in the OR logbook, both occurring during the INOR comparison period and occurring when a wasted acetabular component sticker was also placed in the logbook alongside the actual implanted acetabular component sticker. In both instances, the INOR record clearly discriminated between the wasted shell, which was labeled as such, and the implanted shell, whereas the OR logbook did not. In addition, the three radiographs that were reviewed confirmed the reference-standard OR logbook had correctly recorded the implantation of acetabular screws, in contrast to an incorrect Bluespier record.
Discussion
In 2014, the INOR became the first national arthroplasty register to introduce automated implant data using UDI bar code scanning. Comparable to the 2013 FDA requirements [6], new European Union legislation (Medical Device Regulation [MDR]) now requires all medical implants to display a UDI [3, 5]. The MDR legislation mandates that a plain-text version and an Automatic Identification and Data Capture form (bar code) are displayed on device packaging. Therefore, UDI scanning, recognition of the UDI against a known implant library, and subsequent insertion of implant parameters in the electronic operating note and national register is possible for all implants, reducing the manual input burden and the risk of input error [1, 2, 3, 7, 11]. National and international implant catalogues are available for device recognition, with the INOR using the purpose-built and maintained Irish National Component Catalogue. The creation of a similar bar code–affiliated registry catalogue was achieved by the German Arthroplasty Register in 2015 [2, 7]. The Dutch Arthroplasty Registry also introduced bar code scanning upload of implant data in 2015 and has since quoted a 96% to 97% implant completeness rate [3]. The United Kingdom’s National Joint Registry has similarly eliminated manual implant data entry using bar code scanning. However, although the case capture rate is reportedly “as near to 100% as possible,” the implant data accuracy and completeness is not stated in the most recent annual report, perhaps because bar code scanning is reasonably assumed to be entirely accurate and complete [11].
To our knowledge, how the introduction of a novel bar code system affected data quality in orthopaedic implant databases compared with a legacy manual data entry system has not been reported. Significant challenges are expected with such a shift in data management, and studies have previously reported considerable difficulties, and indeed failure, with attempts to introduced comparable automated implant data entry systems [13]. This study demonstrates how implant data quality, both completeness and accuracy, can be improved with the introduction of automated implant data input.
Limitations
This is a retrospective, single institution study examining the effect of a national change project. The results of this study are therefore limited to this institution and do not affirm the same completeness and accuracy of data in the entire INOR. However, given that identical data capture methods are employed in each of the nine hospitals currently enrolled in the INOR, the authors predict similar findings at each location. For this comparison study, we used the hardcopy OR logbooks as the reference standard for comparison of either electronic system. The OR logbook record is not infallible; it is not possible to fully out rule incompleteness or inaccuracies of our reference standard comparison. However, the authors feel the possibility of missing procedures resulting in a nonconsecutive series of patients is negligible given that the institution’s IPMS confirmed no additional patients meeting the inclusion criteria were admitted during each series of 250 patients. Also, no additional case was identified in either electronic record, offering excellent reassurance that the OR logbook did not miss any cases. Secondly, incomplete individual records could potentially occur through the omission of an implant sticker from the OR logbook; however, no case was identified in the logbook that was obviously missing a sticker (for example, a THA case missing the femoral stem implant sticker), and no case existed in the electronic record with additional implants compared with the OR logbook. Regarding the accuracy of this reference-standard, the authors felt the risk of typographical errors on the implant stickers was trivial.
However, this study did identify two instances of misinformation in the reference-standard, when an OR logbook record contained both a wasted acetabular component and a used implant, with neither labeled accordingly. The acetabular liner sticker confirmed the diameter and therefore distinguished the wasted from the actual implant for both procedures. Additionally, the corresponding INOR record clearly labeled the wasted implant as such. Further research may enable the INOR database to be considered the reference-standard for future comparative purposes.
Completeness of Records
At our institution, this study found 100% completeness with regard to both case capture and individual case implant data capture after the introduction of bar code data input. Nationally, INOR enrollment is voluntary, and currently, the national case completeness rate is not known because of an information gap between the dual public and private systems in Ireland [12]. However, the proportion of public hospitals uploading to the INOR was known to be 70% to 75% in 2020, with individual hospital case completeness at 98% to 100% [12]. In addition to the THA and TKA implant parameters examined in this study, further parameters are automatically captured by UDI scanning; compared with manual data entry systems, bar code data entry also automatically captures screw length; cement brand; liner details such as lipped, neutral, or offset parameters; and importantly, batch and lot numbers for each implant.
Accuracy of Records
Before the introduction of bar code scanning, almost half of manually entered implant records contained errors; after barcoding was implemented, we found no implant data errors in the examined INOR database. The replacement of manual data entry by automated bar code data upload for healthcare information technology systems has been well described as an effective way to improve both data completeness and accuracy [5, 9, 10, 16]. In contrast to Barsoum et al. [1], who found that implant record accuracy was only 63% among individual hospitals’ registers in the United States using bar code implant data collection, this study demonstrated how a well-coordinated, national approach may achieve excellent results.
Conclusion
Excellent data quality is essential for unbiased, accurate registry-based study outcomes [4]. This study outlines the impact of automated bar code data input on data completeness and accuracy when compared against manual data input for component data in an arthroplasty registry. The continued rollout of the INOR with bar code data methodology to the remaining public and private hospitals in Ireland may be encouraged, and those hospitals can be reassured of the quality of data that may be expected. In addition, the results of this institutional study may both encourage other national registers already using bar code data input to continue doing so or indeed inspire registers still using manual implant data input to consider upgrading from that methodology. Further studies of data completeness and accuracy at a national level may provide further evidence to validate the implant data of the INOR [15].
Acknowledgment
We thank the staff at the National Orthopaedic Hospital Cappagh for their diligent work.
Footnotes
Each author certifies that there are no funding or commercial associations (consultancies, stock ownership, equity interest, patent/licensing arrangements, etc.) that might pose a conflict of interest in connection with the submitted article related to the author or any immediate family members.
All ICMJE Conflict of Interest Forms for authors and Clinical Orthopaedics and Related Research® editors and board members are on file with the publication and can be viewed on request.
Ethical approval for this study was obtained from the National Orthopaedic Hospital Cappagh, Dublin, Ireland (number NOHC-2021-ETH-DC-CEO-294).
This work was performed at the National Orthopaedic Hospital Cappagh, Dublin, Ireland.
Contributor Information
James M. Broderick, Email: jamesbrod@gmail.com.
Eoin Fahey, Email: fahey133@hotmail.com.
Paddy Kenny, Email: paddykenny@rcsi.ie.
James Cashman, Email: james.cashman@gmail.com.
References
- 1.Barsoum WK, Higuera CA, Tellez A, et al. Design, implementation, and comparison of methods for collecting implant registry data at different hospital types. J Arthroplasty. 2012;27:842-850.e1. [DOI] [PubMed] [Google Scholar]
- 2.Blömer W, Steinbrück A, Schröder C, et al. A new universal, standardized implant database for product identification: a unique tool for arthroplasty registries. Arch Orthop Trauma Surg. 2015;135:919-926. [DOI] [PubMed] [Google Scholar]
- 3.Denissen GAW, van Steenbergen LN, Lollinga WT, et al. Generic implant classification enables comparison across implant designs: the Dutch Arthroplasty Register implant library. EFORT Open Rev. 2019;4:344-350. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Espehaug B, Furnes O, Havelin LI, et al. Registration completeness in the Norwegian Arthroplasty Register. Acta Orthop. 2006;77:49-56. [DOI] [PubMed] [Google Scholar]
- 5.Eur-Lex. Regulation (EU) 2017/745 of the European Parliament and of The Council of 5 April 2017 on medical devices, amending directive 2001/83/ec, regulation (ec) no 178/2002 and regulation (ec) no 1223/2009 and repealing council directives 90/385/eec and 93/42/eec, European Parliament. EUR-Lex - 02017R074520200424 - EN. Available at: https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A02017R0745-20200424. Accessed May 13, 2022.
- 6.Food and Drug Administration. Unique Device Identification System. Available at: https://www.govinfo.gov/content/pkg/FR-2013-09-24/pdf/2013-23059.pdf. Accessed May 13, 2022.
- 7.Hassenpflug J, Liebs TR. Registries as a tool for optimizing safety of endoprostheses. Experiences from other countries and the setup of the German arthroplasty register [in German]. Bundesgesundheitsbl. 2014;57:1376-1383. [DOI] [PubMed] [Google Scholar]
- 8.Heinänen M, Brinck T, Lefering R, et al. How to validate data quality in a trauma registry? The Helsinki Trauma Registry internal audit. Scand J Surg. 2021;110:199-207. [DOI] [PubMed] [Google Scholar]
- 9.Laroche JA, Diniz AJ. Immunisation registers in Canada: progress made, current situation, and challenges for the future. Euro Surveill. 2012;17:20158. [DOI] [PubMed] [Google Scholar]
- 10.Lynch MF, Ghani KR, Frost I, et al. Preventing the forgotten ureteral stent: implementation of a web-based stent registry with automatic recall application. Urology. 2007;70:423-426. [DOI] [PubMed] [Google Scholar]
- 11.National Joint Registry. 18th Annual Report 2021. Available at: https://reports.njrcentre.org.uk. Accessed May 13, 2022.
- 12.National Office of Clinical Audit. INOR objectives. Available at: https://www.noca.ie/audits/inor-objectives. Accessed October 5, 2022.
- 13.Paxton EW, Inacio MCS, Khatod M, et al. Kaiser Permanente National Total Joint Replacement Registry: aligning operations with information technology. Clin Orthop Relat Res. 2010;468:2646-2663. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Pipino LL, Lee YW, Wang RY, et al. Data quality assessment. Communications of the ACM. 2002;45:211-218. [Google Scholar]
- 15.Varnum C, Pedersen AB, Gundtoft PH, et al. The what, when and how of orthopaedic registers: an introduction into register-based research. EFORT Open Rev. 2019;4:337-343. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Wilson K, Atkinson KM, Deeks SL, et al. Improving vaccine registries through mobile technologies: a vision for mobile enhanced Immunization information systems. J Am Med Inform Assoc. 2016;23:207-211. [DOI] [PMC free article] [PubMed] [Google Scholar]