Abstract
Background
The autoverification system for coagulation consists of a series of rules that allow normal data to be released without manual verification. With new advances in medical informatics, the laboratory information system (LIS) has growing potential for the autoverification, allowing rapid and accurate verification of clinical laboratory tests. The purpose of the study is to develop and evaluate a LIS-based autoverification system for validation and efficiency.
Methods
Autoverification decision rules, including quality control, analytical error flag, critical value, limited range check, delta check and logical check, as well as patient’s historical information, were integrated into the LIS. Autoverification limited range was constructed based on 5 and 95% percentiles. The four most commonly used coagulation assays, prothrombin time (PT), activated partial thromboplastin time (APTT), thrombin time (TT), and fibrinogen (FBG), were followed by the autoverification protocols. The validation was assessed by the autoverification passing rate, the true-positive cases, the true-negative cases, the false-positive cases, the false-negative cases, the sensitivity and the specificity; the efficiency was evaluated in the turnaround time (TAT).
Results
A total of 157,079 historical test results of coagulation profiles from January 2016 to December 2016 were collected to determine the distribution intervals. The autoverification passing rate was 77.11% (29,165/37,821) based on historical patient data. In the initial test of the autoverification version in June 2017, the overall autoverification passing rate for the whole sample was 78.75% (11,257/14,295), with 892 true-positive cases, 11,257 true-negative cases, 2146 false-positive cases, no false-negative cases, sensitivity of 100% and specificity of 83.99%. After formal implementation of the autoverification system for 6 months, 83,699 samples were assessed. The average overall autoverification passing rate for the whole sample was 78.86% and the 95% confidence interval (CI) of the passing rate was [78.25, 79.59%]. TAT was reduced from 126 min to 101 min, which was statistically significant (P < 0.001, Mann-Whitney U test).
Conclusions
The autoverification system for coagulation assays based on LIS can halt the samples with abnormal values for manual verification, guarantee medical safety, minimize the requirements for manual work, shorten TAT and raise working efficiency.
Keywords: Laboratory information systems, Medical safety, Autoverification, Coagulation, Turnaround time
Background
Following the analytical phase, a large number of manual verifications are performed in clinical laboratories to detect possible errors before the results are released to the electronic health records, which is time-consuming [1]. A significant answer for this issue may be autoverification, which is a set of well-designed rules with an aim to stop the samples with abnormal values for manual verification and at the same time permit those with normal values to be released without manual intervention [2]. Previous reports have demonstrated that autoverification can ensure medical safety [3], shorten turnaround time (TAT) [2–5], reduce labour requirements [2–4], improve operational efficiency [2, 4–6] and minimize error rate [2], as well as enable laboratory technologists to devote more attention to test results with greater potential error [2].
Until recently, those autoverification systems were commonly developed via third party commercial software or middleware, which are costly, and the autoverification decision rules were proprietary, such that no revision could be made according to the user’s requirements [7–10]. In addition, they cannot connect with the hospital information system (HIS) and obtain comprehensive clinical data, such as patient’s history and clinical diagnosis. With the progress of laboratory automation, it is challenging to achieve interconnection and intercommunication between analytical instruments and the laboratory information system (LIS) to design within-laboratory autoverification systems independent of any commercial software.
Coagulation assays are essential for assessment of patients requiring acute care [11], patients undergoing anticoagulant therapy [12], thrombolytic therapy [13] and pregnancy [14], as well as monitoring of disseminated intravascular coagulation [15]. Currently, in many laboratories the coagulation assays are still released by manual review, verification and release, and reports about autoverification in coagulation are scarce [16]. The four most routinely used coagulation assays in our laboratory, namely, prothrombin time (PT), activated partial thromboplastin time (APTT), thrombin time (TT) and fibrinogen (FBG), are commonly prescribed together. There is therefore an urgent need, but it is still a challenge to establish autoverification decision rules in coagulation, including quality control (QC) check, instrument error flags, critical values, limited range check, delta check, logical rules and patient’s historical data, with reference to the Autoverification of Clinical Laboratory Test Result; Approved Guideline (AUTO 10-A) [17]. This research extends our knowledge to establish a within-laboratory autoverification system for coagulation assays based on LIS to ensure medical safety and shorten the turnaround time (TAT). Moreover, it is the first study based on clinical large-scale data to evaluate the validation and efficiency of autoverification in terms of specificity and efficiency.
Methods
Setting
The study was conducted in the core clinical laboratory of the First Affiliated Hospital of China Medical University, which processes approximately 3,500,000 various laboratory samples per year. The HIS and the LIS were both provided by Donghua Co-creation software (Beijing, Donghua, China). The autoverification decision rules were converted into computer languages to integrate into the HIS. In the process of autoverification, the values that passed the autoverification protocols were marked in green, and those that failed were flagged in red. When abnormal results were held up, a quick attentive manual verification or re-analysis was needed. If the results were considered to be critical values, they were quickly released to the HIS, and the clinicians were alerted to the abnormalities by a call.
Data collection
Coagulation profiles from outpatients, inpatients and patients from the physical examination center of our hospital were collected for analysis. Historical values of 157,097 samples between January 2016 and December 2016 were used to determine the 5 and 95% percentiles of PT, APTT, TT and FBG. A total of 37,821 historical records released by manual verification from January 2017 to March 2017 were obtained to test the passing rate of autoverification protocols. In the test version phase, validation of the autoverification protocols was determined by 28,244 samples between May 2017 and June 2017. After the protocols were formally implemented online, 83,699 samples between July 2017 and December 2017 were taken to evaluate the single test autoverification passing rate and the overall autoverification passing rate.
Analytical instruments and reagents
All tests were measured using STA-R Evolution Analyzer (Stago Diagnostica, Brussels, Belgium). Reagents of STA-PTT (APTT), STA-Neoplastin R (PT), STA-TT (TT), STA-Fibrinogen (FBG) and controls were also provided by Stago.
Autoverification decision making
The autoverification system was designed with reference to Auto 10-A guideline [17]. A technologist-augmented decision-making tree was processed, and if any of the rules was violated, the result would be held for manual verification, as shown in Fig. 1. The autoverification steps consisted of quality control (QC) check, analytical error flags, critical values, limited range check, delta check and logical rules, as well as the patient’s clinical diagnosis. The autoverification protocols run the single test successively until the 4 items of the whole sample were finished. If all 4 items passed the autoverification, the logical rules were run for further verification; if any of the steps violated the rules, manual verification was required. Because the LIS is connected to the HIS in our hospital, any result that passed autoverification was directly released to the patients’ medical records without manual verification.
Fig. 1.
Autoverification decision-making tree. Autoverification procedures for single tests and whole samples in the order of quality control check, analytical error flags, critical values, limited range check, delta check, logical rules and patient’s clinical diagnosis
Quality control check
QC check is routinely performed in our laboratory, transmitting from the analyzer to the LIS according to Westgard quality control multi-rules [18]. The QC check ensured that each test was within the appropriate range before running the autoverification protocols. If there were no QC results within 24 h or if any QC rule was violated, the program halted all samples being autoverified.
Analytical error flags
If any instrument analytical flag was sent to the LIS along with testing problems, such as the two-dimensional barcode, reagents, and samples, as well as reagent crystallization or sample clots forming, the results were held for later manual verification.
Critical value
The critical values that may threaten the patient’s life and require urgent medical treatment were determined by laboratory technologists and clinicians. The critical values used for autoverification were as follows: PT ≤ 9.00 s or ≥ 70.00 s, APTT ≤ 15.00 s or ≥ 100.00 s, TT > 150.00 s and FBG < 1.00 g/L, as shown in Table 2. If critical values were detected, they were manually verified by a skilled laboratory specialist, repeated, and transferred to a critical value list for an immediate phone call to clinicians. The results within the range ran the following limited range check and delta check.
Table 2.
Clinical reference intervals, critical range, 5 and 95% percentiles, and the limited range
| Clinical reference intervals | Critical value | Percentiles (5 and 95%) | Limited Range | |
|---|---|---|---|---|
| PT | 11.00–14.30 s | ≤ 9.00 s or ≥ 70.00 s | 11.80–16.30 s | 11.00–16.30 s |
| APTT | 32.00–43.00 s | ≤ 15.00 s or ≥ 100.00 s | 30.40–46.40 s | 30.40–46.40 s |
| TT | 14.0 0–21.00 s | > 150.00 s | 15.00–19.40 s | 14.00–21.00 s |
| FBG | 2.00–4.00 g/L | < 1.00 g/L | 2.21–6.51 s | 2.00–6.51 g/L |
PT Prothrombin time, APTT Activated partial thromboplastin time, TT Thrombin time, FBG Fibrinogen
Limited range check
The limited range is a filter to verify if the result is within the extreme reference intervals specified for the assays in the LIS. The clinical reference intervals used in our laboratory were 11.00–14.30 s for PT, 32.00–43.00 s for APTT, 14.00–21.00 s for TT and 2.00–4.00 g/L for FBG. In clinical practice, PT and TT values greater than 3.00 s and APTT value greater than 100.00 s are considered pathological. However, due to biological variation, clinical reference intervals are not always suitable for autoverification; we therefore analyzed the distribution of 5 and 95% percentiles based on the historical large-scale data from January 2016 to December 2016. In the current study, after discussing with local clinicians, the limited range was determined by a combination of clinical reference intervals, critical value and 5 and 95% percentiles. The results that were within the limited range continued to the next step.
Delta check
It is widely accepted that test results with historical values should be confirmed by a delta check [19, 20]. In our study, delta check was used for the results with historical data and failing the limited check. The delta check was performed to compare present test results with previous results and to identify variations beyond patient’s expected values, such as changes in a patient’s clinical condition or specimens with presence of clots [20–24]. Delta check can be performed using various methods, including the delta difference, delta percent change, rate difference, and rate percent difference [22, 25, 26]. In the present study, delta differences were calculated as the difference value between the present results and the previous results. Meanwhile, different tests correspond to different delta differences and time intervals due to biological and pathological functions of coagulation assays. The delta check rules were as follows: PT < 10.00 s and the time interval < 10 days; APTT < 10.00 s and the time interval < 7 days; FBG < 1.00 g/L and the time interval < 3 days. However, if the patient was undergoing anticoagulant therapy, the time interval was < 30 days, and the international normalized ratio (INR) was between 2 and 3.
Logical rules
Based on clinical and practical diagnostic criteria, a few tests may have some necessary logical interrelationships between each testing item, e.g., prolonged PT and APTT associated with a decreased FBG. Logical rules were applied with the condition that all single tests passed the QC check, analytical error flags, critical value, limited range check and delta check. If any result violated the logical rules, the autoverification protocols were stopped, and the results were reviewed manually before being sent to the HIS. In logical check, even though the single results were within the limited range, the reports were held for manual verification when PT ≥ 14.00 s or APTT ≥ 43.00 s and FBG ≤ 2.00 g/L, PT ≥ 14.00 s and APTT ≥43.00 s, or APTT ≤43.00 s, PT ≤ 14.00 s and TT ≥ 21.00 s occurred. By logical rules, a presence of clot was avoided.
Patient’s clinical diagnosis and historical data
Because it is of great significance for us to design precise and customized rules that work for inpatients and outpatients, the patient’s specific information, such as gender, age, clinical presentation, medical treatment history, drug history, pregnancy and other conditions that may affect the results of coagulation assays should be comprehensively considered. Adjustment of the limited range was done according to clinical diagnosis and historical data. For instance, for patients taking warfarin, the limited range was PT > 40.00 s, and the allowable ranges for INR were as follows: 1.50–2.50 before non-hip surgery, 2.00–3.00 before hip surgery, 2.00–3.00 for deep vein thrombosis, 2.00–4.00 for treatment of pulmonary infarction, 3.00–4.00 for prevention of arterial thrombosis, and 3.00–4.00 for valve prosthesis surgery. For patients receiving heparin treatment, the limited range was APPT > 90.00 s. If the patient is undergoing anticoagulation therapy, even though the test result of PT was within the routine limited range rather than within the customized limited range, the report would be stopped for manual autoverification and would not be sent to the HIS. For late pregnancy, the upper limit intervals for PT and APTT were decreased by 10%, and the lower limited range for FBG was increased by 10% [14]. For patients undergoing thrombolytic therapy, the limited range for FBG was defined as 1.20 g/L – 4.00 g/L.
Assessment of the validation of the autoverification system
Assessment of the historical records test
According to CLSI Auto-10 A, all rules and settings must be validated and thoroughly tested before the autoverification protocols were implemented in a real-life clinical environment. First, we used historical clinical laboratory records stored in the LIS to verify the validation. To test if the program would screen the results outside the autoverification decision rules, special abnormal cases were chosen to test whether or not the autoverification program could be held up. Second, the autoverification passing rate of historical records in the LIS from January 2017 to March 2017 was determined to test if the autoverification protocols followed the rules as expected and to detect potential problems with the program.
Assessment of the initial online test version and autoverification system
Subsequently, the initial online test version program was run in a real-life environment to verify the actual validation in May 2017. In the test version phase, the autoverification decision rules were programmed in other computers and the autoverification reports were not sent to the HIS. Meanwhile, a laboratory technologist and a skilled laboratory specialist (professor in the core clinical laboratory) manually verified and revised all routine cases and then released reports to HIS as before. Comparisons were accomplished as follows: the number of samples, cases that passed the autoverification, cases that failed the autoverification, the true-positive cases (intercepted problematic reports), the true-negative cases (auto-released correct reports), the false-positive cases (intercepted correct problematic reports), the false-negative cases (auto-released problematic reports), the sensitivity, the specificity, and the autoverification passing rate. After initial online testing for 1 month, false-negative cases were found due to a tiny or partial clot that was not detected by the instrument. Thus, logical rules were modified to eliminate the false-negative rate in June 2017. Finally, after running and modifying the test version, the false-negative rate was zero, and the program was formally connected online with the LIS in July 2017. The results of all of the samples that passed all the autoverification decision rules would be directly released from the LIS to the HIS without manual verification.
Assessment of the efficiency of the autoverification system
In the current study, TAT was measured by two means, namely, TAT 1 and TAT 2. TAT 1 is defined as the interval from the sample receipt by the laboratory to the release of the results on the LIS; TAT 2 is defined as the interval from the complete analysis by the instruments to the report verification on LIS. Both TAT 1 and TAT 2 were compared with the corresponding previous months.
Statistical analysis
Data were analyzed using IBM SPSS statistics software, version 23.0 (SPSS Inc., Chicago, USA). Mann-Whitney U test was used for independent sample analysis. A P < 0.05 was considered significant.
Results
Definition of limited range
From January 2016 and December 2016, 157,079 historical test results of coagulation profiles were collected. Distribution intervals of clinical large-scale data of coagulation, such as the mean, standard deviation, median, and 5 and 95% percentiles, are shown in Table 1. In this study, we defined the limited range according the intervals of clinical reference intervals, critical range, and 5 and 95% percentiles, and the limited range was 11.00–16.30 s for PT, 30.40–46.40 s for APTT, 14.00–21.00 s for TT and 2.00–6.51 g/L for FBG, as shown in Table 2.
Table 1.
Distribution intervals of clinical large-scale data in coagulation
| n | Mean | SD | Median | Range | Minimum | Maximum | Percentiles | ||
|---|---|---|---|---|---|---|---|---|---|
| 5% | 95% | ||||||||
| PT | 157,097 | 13.60 | 2.51 | 13.10 | 109.20 | 10.00 | 119.20 | 11.80 | 16.30 |
| APTT | 157,097 | 36.97 | 5.76 | 36.10 | 153.30 | 20.80 | 174.10 | 30.40 | 46.40 |
| TT | 157,097 | 17.12 | 4.04 | 16.80 | 216.30 | 13.00 | 229.30 | 15.00 | 19.40 |
| FBG | 157,097 | 3.80 | 1.35 | 3.50 | 12.94 | 0.60 | 13.54 | 2.21 | 6.51 |
SD Standard deviation, PT Prothrombin time, APTT Activated partial thromboplastin time, TT Thrombin time, FBG Fibrinogen
Validation of the autoverification system
Validation of historical records
In historical data analysis, we evaluated 246 items of historical special abnormal data with high or low concentrations. After running through the autoverification protocols and comparing with the historical reports, all items ended in stopping autoverification and none ended in autoverification. Of the 37,821 historical reports of coagulation profiles from January 2017 to March 2017, 29,165 samples passed the autoverification protocols, indicating that the overall passing rate was 77.11%.
Validation of initial online test
The autoverification passing rate, number of samples, cases that passed the autoverification, cases that failed the autoverification, true-positive cases, true-negative cases, false-positive cases, false-negative cases, sensitivity and specificity in the real-life online test are shown in Table 3. Because coagulation requisition forms in our hospital usually contained PT, APTT, FBG and TT simultaneously, we evaluated the overall passing rate of the four tests as a whole sample. The overall autoverification passing rate for all of the samples was 77.63% (10,828/13,949) in May 2017. There were 5 cases that the two reviewers decided to stop for manual verification; however, the autoverification passed. These cases were considered a partial or tiny clot, characterized by PT ≥ 14.00 s or APTT ≥ 43.00 s and FBG ≤ 2.00 g/L, despite being within limited range. After modifying the protocols with logical rules, no false-negative cases occurred, and the passing rate was 78.75% (11,257/14,295) in June 2017.
Table 3.
Validation of the initial online test
| May 2017 | June 2017 | |
|---|---|---|
| n | 13,949 | 14,295 |
| Pass | 10,828 | 11,257 |
| Fail | 3121 | 3038 |
| True-positive | 858 | 892 |
| True-negative | 10,823 | 11,257 |
| False-positive | 2263 | 2146 |
| False-negative | 5 | 0 |
| Sensitivity | 99.42% | 100% |
| Specificity | 82.71% | 83.99% |
| Passing rate | 77.63% | 78.75% |
True-positive, intercepted problematic reports; True-negative, auto-released correct reports; False-positive, intercepted correct problematic reports; False-negative, auto-released problematic reports
Validation of the autoverification system
In July 2017, the autoverification was formally introduced, and we observed 83,699 samples from July 2017 to December 2017 to assess the validation. No cases failed the QC check, and approximately 120 items failed analytical error flags per month. Rules of critical value, limited range and delta check were determined by single tests, and the autoverification passing rate is shown in Table 4. After modifying the logical rules, the average overall passing rate was 78.86%, and the 95% confidence interval (CI) for the overall passing rate was [78.25, 79.59%]. To compare with other studies, the passing rates of different assays and systems (commercial tools, middleware or LIS) are shown in Table 5.
Table 4.
Passing rate of the autoverification system of single tests and whole samples
| n | Autoverification results | PT | APTT | TT | FBG | Whole sample | |
|---|---|---|---|---|---|---|---|
| July 2017 | 13,416 | Pass | 12,485 | 12,389 | 12,536 | 12,557 | 10,453 |
| Fail | 931 | 1027 | 880 | 859 | 2963 | ||
| Passing rate (%) | 93.06 | 92.34 | 93.44 | 93.60 | 77.91 | ||
| August 2017 | 14,477 | Pass | 13,349 | 13,413 | 13,723 | 13,573 | 11,354 |
| Fail | 1128 | 1064 | 754 | 904 | 3123 | ||
| Passing rate (%) | 92.21 | 92.65 | 94.79 | 93.76 | 78.43 | ||
| September 2017 | 13,673 | Pass | 12,784 | 12,456 | 12,585 | 12,932 | 10,740 |
| Fail | 889 | 1217 | 1088 | 741 | 2933 | ||
| Passing rate (%) | 93.50 | 91.10 | 92.04 | 94.58 | 78.55 | ||
| October 2017 | 13,004 | Pass | 12,235 | 12,059 | 11,989 | 12,304 | 10,289 |
| Fail | 769 | 945 | 1015 | 700 | 2715 | ||
| Passing rate (%) | 94.09 | 92.73 | 92.19 | 94.62 | 79.12 | ||
| November 2017 | 14,866 | Pass | 13,832 | 13,905 | 14,203 | 13,857 | 11,934 |
| Fail | 1034 | 961 | 663 | 1009 | 2932 | ||
| Passing rate (%) | 93.04 | 93.54 | 95.54 | 93.21 | 80.28 | ||
| December 2017 | 14,263 | Pass | 12,995 | 13,378 | 13,254 | 13,157 | 11,258 |
| Fail | 1268 | 885 | 1009 | 1106 | 3005 | ||
| Passing rate (%) | 91.11 | 93.80 | 92.93 | 92.25 | 78.93 |
PT Prothrombin time, APTT Activated partial thromboplastin time, TT Thrombin time, FBG Fibrinogen
Table 5.
Comparision of different autoverification systems in the studies
| Study | Year | Country | Assay | System Type | Sample Number | Passing Rate |
|---|---|---|---|---|---|---|
| Randell EW et al [6] | 2018 | Canada | Clinical Chemistry and immunoassay | Middleware | 80,867 | 91.1 to 94.7% |
| Xia LY et al [27] | 2017 | China | Clinical Chemistry and immunoassay | LIS | 31,349 | 74.0% |
| Krasowski MD et al [28] | 2014 | USA | Clinical Chemistry and immunoassay | Middleware | – | 99.5% |
| Palmieri R et al [29] | 2018 | Italy | Urinalysis | AutionMAX-SediMAX | 1002 | 52.4% |
| Sediq AM et al [5] | 2014 | Egypt | Thyroid Function | LIS | 563 | 63.8% |
| Zhao Y et al [16] | 2014 | China | Coagulation | Middleware | 2353 | 82.0% |
| Our Study | – | China | Coagulation | LIS | 83,699 | 78.86% |
LIS Laboratory information system
Efficiency of the autoverification system
Comparing the TAT from July to December 2017 with that from July to December 2016, for the routine coagulation assay samples, the median of TAT 1 was 126 min before using the autoverification system versus 101 min after processing the autoverification system. The median of TAT 2 was shortened from 41 min to 15 min. Statistically significant differences were observed for TAT 1 and TAT 2 (P < 0.001, P < 0.001, respectively, Mann-Whitney U test).
Discussion
Coagulation profiles were chosen for autoverification because coagulation provides essential and timely analytical information and is crucial for patient treatment and evaluation of physical condition. In our hospital, the numbers of routine coagulation profiles, including PT, APTT, TT and FBG, exceeded 12,000 routine samples per month, and autoverification could be the appropriate answer to process the huge workloads and save labour. Until recently, within-laboratory autoverification that integrates into LIS was still relatively rare.
Our study has taken all factors into consideration, including the instruments, standard serum samples, reagents, specimen and logical relationship of the test items. We designed an autoverification system based on a set of rules integrated with the LIS that runs in the following order: QC check, analytical error flags, critical value, limited range check, delta check, logical rules and clinical diagnosis, as shown in Fig. 1. If any of the rules are violated, the outcome will be held up in the LIS for manual verification. QC was the premise of the autoverification, and analytical error flags and critical value performed as the initial screening to avoid basic analytical error.
The key points of the autoverification protocols are building the limited range check, delta check and logical check. The limited range check is vital to samples without historical data, because for those patients delta check is not available. In our laboratory, because the LIS is connected with the HIS, the patient’s diagnosis would be considered first, e.g., if they were undergoing warfarin treatment, the limited range was PT > 40.00 s, and the allowable range for INR was changed. Thus, limited range check that corresponds with patient’s clinical diagnosis or health conditions facilitates the possibility of precise and customized autoverification. Delta check facilitated the technologists to quickly recognize changes in a patient’s status. Patient’s results with historical data failing the limited range but passing the delta check would pass the autoverification. In this circumstance, our LIS-based autoverification can potentially provide verification more precisely than commercial tools or manual verification. Logical rules were applied to enable the technologists to quickly detect the logical relationship between the items and analytical errors.
Our research extends the knowledge into LIS-based autoverification. We defined the limited range based on the large-scale data of 157,079 routine coagulation assays for 1 year, and observed the validation and efficiency of autoverification protocols based on 83,699 samples for half a year to report the validation and benefits of autoverification. The validation of the autoverification rules was assessed through both the historical data and online test version program. When using the historical data for analysis, the autoverification passing rate was 77.28%. During the online test version program, the passing rate was 77.63% in May 2017 and 78.75% in June 2017. We modified the logical rules to minimize the false-negative rate because we believe that the crucial issue of autoverification is to guarantee the utmost medical safety. The average passing rate was 78.86%, and the 95% CI for the overall passing rate was [78.25, 79.59%] when we formally implemented the program. Our results suggest that the decision rules permit the autoverification system to achieve better accuracy and guarantee medical safety with the availability of historical data and diagnosis.
To date, as shown in Table 5, the passing rate of the autoverification system is similar to the results based on other commercial tools or middleware [16, 29], and the passing rate for coagulation in our laboratory is to some extent higher than those of other studies that use autoverification integrated with LIS [5, 27]. Interestingly, Randell EW et al. [6, 8] and Krasowski MD et al. [29] reported higher autoverification passing rate of more than 90%. Well-designed limit range sets and the selective use of delta checks may contribute to the high autoverification passing rates in their researches [6, 8]. Both studies focused on in clinical chemistry and immunoassay tests, whereas our study focused on our coagulation. It is generally accepted that clinical chemistry and immunoassay are common and regular tests for outpatients, inpatients and patients from physical examination centres, while in contrast, coagulation is to some extent specifically prescribed for sick patients in emergency, undergoing anticoagulant therapy or thrombolytic therapy, or patients in pregnancy. Therefore, pre-analytical problems and problematic samples could be the reason for the lower passing rate of coagulation. In addition, our hospital is the perplexing and complicated illnesses treatment center in Northeast China, and many samples from severely ill patients were collected, which may enhance the proportion of problematic samples in coagulation assays.
Compared to other studies that assessed the interobserver degree of agreement between manual verification and autoverification [4, 5], we analyzed the true-positive rate, the true-negative rate, the false-positive cases rate, the false-negative rate, the sensitivity and the specificity. In our study, no false-negative case was observed prior to the high passing rate to ensure that every sample that passes the autoverification system and is auto-released is a correct report, and thus guaranteeing utmost medical safety. This is because higher autoverification passing rates and lower false-positive rates may inevitably bring higher false-negative rates, and thus some problematic samples would be released without attentive manual verification or re-analysis.
When determining the efficiency of autoverification, TAT is the indication most affected by the implementation of autoverification [2]. In the present study, the TAT 1 and TAT 2 were shortened, which is consistent with previous reports [5, 16]. Because the passing autoverification are released immediately to HIS, the TAT is significantly reduced compared with those processed by manual verification, which in turn could save time and decrease the manual workload in clinical laboratories [6]. Furthermore, another benefit is that the autoverification system follows consistent standardized processes, therefore minimizing the potential mistakes made by laboratory technologists due to lack of clinical experience or feelings of fatigue and stress.
This study has limitations to be considered. First, it is difficult to assess the error rate of the autoverification system. Because no obvious mistakes or errors have been reported by patients or clinicians at our hospital for nearly 2 years, we may infer that the autoverification protocols are at least as safe as the previous manual verification. Second, a higher passing rate of the autoverification system should be designed in the future research.
Conclusions
We developed and assessed an autoverification system for the most commonly used coagulation profile with a set of rules. The autoverification system for coagulation assays based on LIS can halt the samples with abnormal values for manual verification, guarantee medical safety, minimize the requirements for manual work, shorten TAT and raise working efficiency. Furthermore, we desire to design more precise and valuable rules to build a model in which autoverification may best be used on coagulation profiles.
Acknowledgements
Not applicable.
Abbreviations
- APTT
Activated partial thromboplastin time
- AUTO 10-A
Autoverification of Clinical Laboratory Test Result 10-A; Approved Guideline
- FBG
Fibrinogen
- HIS
Hospital information system
- INR
International normalized ratio
- LIS
Laboratory information system
- PT
Prothrombin time
- TAT
Turnaround time
- TT
Thrombin time
Authors’ contributions
ZW, HK and BQ designed the study. ZW, HK, XF, RM, LZ and MH undertook data collection. CP and BQ analyzed the data. ZW and CP wrote drafts of the manuscript. Then, BQ revised drafts of the paper. All authors contributed to the interpretation of findings and have read and approved the final manuscript.
Funding
This research was supported by Humanities Social Science Foundation of Ministry of Education (Program Grant 17YJCZH184) and Liaoning Education Science 13th Five-Year plan (Program Grant JG17DB557); Philosophy and Social Sciences promotion project of China Medical University (Program Grant 111–3110118083); and The National Natural Science Foundation (Program Grant 81700814). The funding bodies were not involved in the design of the study and collection, analysis, and interpretation of data and in writing the manuscript.
Availability of data and materials
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.
Ethics approval and consent to participate
Ethics approval was obtained from the Ethics Committee of the First Affiliated Hospital of China Medical University (2015 [23]). The patient consent was waived, as utilization of anonymized history data does not require patient consent.
Consent for publication
Not applicable.
Competing interests
The authors declare that they have no competing interests.
Footnotes
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- 1.Panteghini M. The future of laboratory medicine: understanding the new pressures. Clin Biochem Rev. 2004;25:207–215. [PMC free article] [PubMed] [Google Scholar]
- 2.Torke N, Boral L, Nguyen T, Perri A, Chakrin A. Process improvement and operational efficiency through test result autoverification. Clin Chem. 2005;51:2406–2408. doi: 10.1373/clinchem.2005.054395. [DOI] [PubMed] [Google Scholar]
- 3.Wu Jie, Pan Meichen, Ouyang Huizhen, Yang Zhili, Zhang Qiaoxin, Cai Yingmu. Establishing and Evaluating Autoverification Rules with Intelligent Guidelines for Arterial Blood Gas Analysis in a Clinical Laboratory. SLAS TECHNOLOGY: Translating Life Sciences Innovation. 2018;23(6):631–640. doi: 10.1177/2472630318775311. [DOI] [PubMed] [Google Scholar]
- 4.Li J, Cheng B, Ouyang H, Xiao T, Hu J, Cai Y. Designing and evaluating autoverification rules for thyroid function profiles and sex hormone tests. Ann Clin Biochem. 2018;55:254–263. doi: 10.1177/0004563217712291. [DOI] [PubMed] [Google Scholar]
- 5.Sediq AM, Abdel-Azeez AG. Designing an autoverification system in Zagazig University hospitals laboratories: preliminary evaluation on thyroid function profile. Ann Saudi Med. 2014;34:427–432. doi: 10.5144/0256-4947.2014.427. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Randell EW, Short G, Lee N, Beresford A, Spencer M, Kennell M, Moores Z, Parry D. Strategy for 90% autoverification of clinical chemistry and immunoassay test results using six sigma process improvement. Data Brief. 2018;18:1740–1749. doi: 10.1016/j.dib.2018.04.080. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Jones JB. A strategic informatics approach to autoverification. Clin Lab Med. 2013;33:161–181. doi: 10.1016/j.cll.2012.11.004. [DOI] [PubMed] [Google Scholar]
- 8.Randell EW, Short G, Lee N, Beresford A, Spencer M, Kennell M, Moores Z, Parry D. Autoverification process improvement by six sigma approach: clinical chemistry & immunoassay. Clin Biochem. 2018;55:42–48. doi: 10.1016/j.clinbiochem.2018.03.002. [DOI] [PubMed] [Google Scholar]
- 9.Gomez-Rioja R, Alvarez V, Ventura M, Alsina MJ, Barba N, Cortes M, Llopis MA, Martinez C, Ibarz M. Current status of verification practices in clinical biochemistry in Spain. Clin Chem Lab Med. 2013;51:1739–1746. doi: 10.1515/cclm-2012-0659. [DOI] [PubMed] [Google Scholar]
- 10.Li J, Cheng B, Yang L, Zhao Y, Pan M, Zheng G, Xu X, Hu J, Xiao T, Cai Y. Development and implementation of autoverification rules for ELISA results of HBV serological markers. J Lab Autom. 2016;21:642–651. doi: 10.1177/2211068215601612. [DOI] [PubMed] [Google Scholar]
- 11.Marsden NJ, Van M, Dean S, Azzopardi EA, Hemington-Gorse S, Evans PA, Whitaker IS. Measuring coagulation in burns: an evidence-based systematic review. Scars Burn Heal. 2017;3:2059513117728201. doi: 10.1177/2059513117728201. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Barr Drakeria, Epps Quovadis J. Direct oral anticoagulants: a review of common medication errors. Journal of Thrombosis and Thrombolysis. 2018;47(1):146–154. doi: 10.1007/s11239-018-1752-9. [DOI] [PubMed] [Google Scholar]
- 13.Favaloro EJ. Optimizing the verification of mean Normal prothrombin time (MNPT) and international sensitivity index (ISI) for accurate conversion of prothrombin time (PT) to international normalized ratio (INR) Methods Mol Biol. 2017;1646:59–74. doi: 10.1007/978-1-4939-7196-1_4. [DOI] [PubMed] [Google Scholar]
- 14.Gutiérrez García Irene, Pérez Cañadas Pablo, Martínez Uriarte Juan, García Izquierdo Olivia, Angeles Jódar Pérez María, García de Guadiana Romualdo Luis. D-dimer during pregnancy: establishing trimester-specific reference intervals. Scandinavian Journal of Clinical and Laboratory Investigation. 2018;78(6):439–442. doi: 10.1080/00365513.2018.1488177. [DOI] [PubMed] [Google Scholar]
- 15.Papageorgiou Chrysoula, Jourdi Georges, Adjambri Eusebe, Walborn Amanda, Patel Priya, Fareed Jawed, Elalamy Ismail, Hoppensteadt Debra, Gerotziafas Grigoris T. Disseminated Intravascular Coagulation: An Update on Pathogenesis, Diagnosis, and Therapeutic Strategies. Clinical and Applied Thrombosis/Hemostasis. 2018;24(9_suppl):8S–28S. doi: 10.1177/1076029618806424. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Zhao Y, Yang L, Zheng G, Cai Y. Building and evaluating the autoverification of coagulation items in the laboratory information system. Clin Lab. 2014;60:143–150. doi: 10.7754/clin.lab.2013.130109. [DOI] [PubMed] [Google Scholar]
- 17.CLSI. Autoverification of clinical laboratory test result; approved guideline (AUTO 10-A). Wayne: Clinical and Laboratory Standards Institute (CLSI); 2006.
- 18.Westgard JO. Internal quality control: planning and implementation strategies. Ann Clin Biochem. 2003;40:593–611. doi: 10.1258/000456303770367199. [DOI] [PubMed] [Google Scholar]
- 19.Ovens K, Naugler C. How useful are delta checks in the 21 century? A stochastic-dynamic model of specimen mix-up and detection. J Pathol Inform. 2012;3:5. doi: 10.4103/2153-3539.93402. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Randell Edward W, Yenice Sedef. Delta Checks in the clinical laboratory. Critical Reviews in Clinical Laboratory Sciences. 2019;56(2):75–97. doi: 10.1080/10408363.2018.1540536. [DOI] [PubMed] [Google Scholar]
- 21.Garner AE, Lewington AJ, Barth JH. Detection of patients with acute kidney injury by the clinical laboratory using rises in serum creatinine: comparison of proposed definitions and a laboratory delta check. Ann Clin Biochem. 2012;49:59–62. doi: 10.1258/acb.2011.011125. [DOI] [PubMed] [Google Scholar]
- 22.Park SH, Kim SY, Lee W, Chun S, Min WK. New decision criteria for selecting delta check methods based on the ratio of the delta difference to the width of the reference range can be generally applicable for each clinical chemistry test item. Ann Lab Med. 2012;32:345–354. doi: 10.3343/alm.2012.32.5.345. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Strathmann FG, Baird GS, Hoffman NG. Simulations of delta check rule performance to detect specimen mislabeling using historical laboratory data. Clin Chim Acta. 2011;412:1973–1977. doi: 10.1016/j.cca.2011.07.007. [DOI] [PubMed] [Google Scholar]
- 24.Yamashita T, Ichihara K, Miyamoto A. A novel weighted cumulative delta-check method for highly sensitive detection of specimen mix-up in the clinical laboratory. Clin Chem Lab Med. 2013;51:781–789. doi: 10.1515/cclm-2012-0752. [DOI] [PubMed] [Google Scholar]
- 25.Tran DV, Cembrowski GS, Lee T, Higgins TN. Application of 3-D Delta check graphs to HbA1c quality control and HbA1c utilization. Am J Clin Pathol. 2008;130:292–298. doi: 10.1309/VM6FVF6GGCYYJ9BV. [DOI] [PubMed] [Google Scholar]
- 26.Lacher DA, Connelly DP. Rate and delta checks compared for selected chemistry tests. Clin Chem. 1988;34:1966–1970. [PubMed] [Google Scholar]
- 27.Xia LY, Cheng XQ, Liu Q, Liu L, Qin XZ, Zhang L, Ding JW, Xu EM, Qiu L. Developing and application of an autoverification system for clinical chemistry and immunology test results. Zhonghua Yi Xue Za Zhi. 2017;97:616–621. doi: 10.3760/cma.j.issn.0376-2491.2017.08.012. [DOI] [PubMed] [Google Scholar]
- 28.Krasowski MD, Davis SR, Drees D, Morris C, Kulhavy J, Crone C, Bebber T, Clark I, Nelson DL, Teul S, et al. Autoverification in a core clinical chemistry laboratory at an academic medical center. J Pathol Inform. 2014;5:13. doi: 10.4103/2153-3539.129450. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Palmieri R, Falbo R, Cappellini F, Soldi C, Limonta G, Brambilla P. The development of autoverification rules applied to urinalysis performed on the AutionMAX-SediMAX platform. Clin Chim Acta. 2018;485:275–281. doi: 10.1016/j.cca.2018.07.001. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.