Application of sigma-based quality control rules for the efficiency of internal quality control

Abstract

Background

Ensuring stability in medical laboratories through quality control (QC) is crucial and requires fitted rules to prevent false alerts and identify errors. This study demonstrates how the introduction of new QC rules to align with individual total allowable error (TEa) affects laboratory efficiency and error detection.

Methods

Changes in the performance of 26 biochemical tests before and after applying new internal quality control (IQC) rules were studied. Pre-Phase utilized uniform QC rules (1–3s, 2-2s, 2/3-2s, R-4s, 4-1s, and 12-x) while Post-Phase adopted new QC rules selected using Westgard Adviser (Bio-Rad Inc., USA). Sigma metrics were calculated using TEa and precision and bias from IQC data, compared to the peer group. Efficiency was assessed by comparing QC-repeat rates, turnaround times (TAT), and proficiency test (PT) results.

Results

QC-repeats due to violations averaged 5.6 % in the Pre-Phase and decreased to 2.5 % in the Post-Phase. As a result, the rate of out-of-TAT in peak-time decreased from 29.4 % to 15.2 %. In Pre-Phase, 67 of 271 cases exceeded the 2 standard deviation index (SDI) in the PT, which was reduced to 24 cases in Post-Phase. Cases exceeding the 3 SDI significantly decreased from 27 to 4 in the Post-Phase.

Conclusion

The introduction of sigma-based rules in the internal quality control process improved laboratory efficiency by reducing QC-repeat, recalibration, and TAT while maintaining quality, demonstrating a valuable balance between efficiency and analytical performance.

Graphical abstract

1. Introduction

Quality control (QC) is a daily activity performed by a medical laboratories to ensure the stability and reliability of test results, as it directly impacts medical decisions, such as patient admission, discharge, and patient care, and laboratory errors can undermine physicians’ confidence in the laboratory and affect patient safety [[1], [2], [3]]. Most clinical laboratories have been trying to reduce errors in clinical results by continuously checking the condition of instruments through external quality control (proficiency test, PT) and internal quality control (IQC) [1,3]. The Westgard multirule is a method used for internal quality control, where the measured values of control materials are accumulated and assessed according to multiple control rules. These include rules such as 1_2s, 1_3s, 2_2s, 2 of 3_2s, 10_x, and R_4s. Appropriate rules are selected and applied depending on the specific context and requirements of the quality control process [4]. Since the broad application of the Westgard Multirules, there have been several discussions on setting QC rules for IQC [1,[5], [6], [7]].

The key to the existence of a QC rule is to detect significant analytical error conditions [7]. If the IQC rule is too stringent, it could unnecessarily detect random errors (false alerts) that might commonly occur, resulting in high rejection rates and overloading the laboratory with work (QC-repeat), calibration, time consumption, and delay in providing the results) [8]. Conversely, if the QC rules are too permissive, the probability of error detection (true alert) is reduced, and the analytical error condition may be missed [7,9]. Therefore, setting appropriate QC rules for each test is one of the most critical factors for laboratory efficiency and patient safety.

Decades ago, the concept of Six sigma, a manufacturing process methodology used in industry, was introduced into clinical laboratories [10]. A method for setting QC rules using Sigma has been proposed. Sigma can be calculated as follows [10],

$$\text{Sigma}\left(\mathrm{\sigma }\right)=\frac{TEa(\%)-Bias(\%)}{CV(\%)}$$

It can be calculated by combining the total allowable error (TEa), bias, and coefficient of variation (CV). The equations of bias and CV are as below and SD is standard deviation.

$$\text{Bias}(\%)=\frac{Mean\left(target\right)-Mean\left(measured\right)}{Mean\left(target\right)}\times 100$$

$$\text{CV}(\%)=\frac{SD}{Mean}\times 100$$

The higher the Sigma level, the fewer the test results that fall outside the TEa range. For example, at a 6 Sigma level, the failure rate (unreliable tests) is very low at 3.4 per million. In contrast, at 3 Sigma, the level required in clinical laboratories, the failure rate is about 67 per million [6,10].

Shoenmakers et al. suggested that every Westgard rule has a unique Sigma value; therefore, depending on the Sigma level, the rules available for management may vary [11]. However, calculating Sigma metrics and setting QC rules are challenging to implement in clinical laboratories. Determining CV and bias can be burdensome [6,11]. Even if Sigma is obtained using the Westgard operational process specification charts (OPSpecs charts), there may be difficulties in selecting practically acceptable QC rules. For example, using extremely permissive QC rules (e.g., a single 1–5s or 1–4s rule) for a test with high Sigma may raise concerns regarding the possibility of overlooking significant errors.

Bio-Rad's IQC program “Unity Real Time (URT)” (Hercules, CA, USA) has a feature called “Westgard Adviser” that calculates Sigma and provides QC rules in a simple way. In this study, we used this program to generate new Sigma-based QC rules and applied them in practice to evaluate whether they can help laboratories become more efficient and detect significant errors.

2. Materials and methods

2.1. Description of Unity Real Time (URT)

As previously mentioned, the components of Sigma are CV%, TEa, and Bias. In the “Westgard Adviser,” TEa is set to Clinical Laboratory Improvement Amendments of 1988 (CLIA88) by default, but the user can also alter TEa by inquiring CV% and bias of cumulative IQC results from our laboratory as well as from other laboratories worldwide employing the URT. Bias can be determined through multiple approaches, including manual input by the user or automatic retrieval via the URT by referencing peer group data from laboratories utilizing the same lot of the same QC material, stratified by instrument and analytical method. Sigma value is calculated based on the above three values (CV%, TEa, and bias), and the power function graph and OPsec charts are automatically generated by reflecting the number of QC materials currently in use. Based on this, QC rules that consider the false alert and error detection rates are presented for each analyte, allowing users to select appropriate QC rules.

2.2. Setting of new QC rules

The IQC data were collected from January 2020 to October 2020 in a large tertiary hospital, Pusan National University Yangsan Hospital, for 26 biochemical tests, as follows: Albumin, Alkaline Phosphatase (ALP), Alanine Aminotransferase (ALT), Aspartate Aminotransferase (AST), Blood Urea Nitrogen (BUN), Calcium, Chloride, Creatine Kinase (CK), total bicarbonate (CO₂), Creatinine, Direct bilirubin (D-bilirubin), Gamma-Glutamyl Transferase (GGT), Glucose, High-Density Lipoprotein Cholesterol (HDL-Cholesterol), Lactate Dehydrogenase (LDH), Low-Density Lipoprotein Cholesterol (LDL-Cholesterol), Lipase, Magnesium, Phosphorous, Potassium, Sodium, Total protein (T-protein), Total Bilirubin (T-Bilirubin), Total-Cholesterol, Triglyceride, and Uric acid.

January to May 2020 (5 months) was designated as “Pre-Phase,” and during that time, the existing QC rules were used. The new QC rules were induced at the end of May in aspects of highest total error (conservative setting) and applied during five months of “Post-Phase” from June to October. In the Pre-Phase, 1–3s, 2-2s, 2/3-2s, R-4s, 4-1s, and 12-x were used in batches for all analytes. To determine the new QC rule, we calculated two sets (A and B) of Sigma values based on the Pre-Phase QC data. To estimate bias, we established a peer group of laboratories that used the same equipment for testing. The difference between the A and B sets is in the TEa. In the first Set (A), TEa was calculated using CLIA 88 built into “Westgard Adviser.” In Set B, the bias and CV were the same as in Set A. However, TEa was set as a new criterion set of appropriate clinical differences (unpublished data) by our regional team of laboratory physicians - Busan Clinical Chemical Network (BCCN), considering the results of a survey of 102 board specialists in our area (66 internal medicine, 10 family medicine, 7 anesthesiology and pain medicine, 7 neurology). In that survey, we investigated the extent of change in laboratory test results that clinicians consider to be clinically significant. The Milan Consensus classified analytical performance specifications (APS) into three models based on the level of evidence [12]. Among them, surveys conducted among clinicians are considered an indirect outcome study, corresponding to Model 1b, and were used as the basis for allowable total error in this study. The CV% was derived from the IQC data of the URT's accumulated daily QC results. The bias is originally compared with a reference value, but in this study, bias was calculated by comparing the average value of the other laboratories (called the peer group) using same QC material (Liquid assayed multiqual, Bio-Rad) and analyzers (AU5800, Beckman Coulter Inc.), which “Westgard adviser” suggest.

2.3. Sigma levels and new QC rules for analytes

The Sigma values for each analyte during the Pre-Phase are listed in Table 1. Fig. 1 (A) illustrates a graphical representation of the Sigma distribution based on the CLIA 88 standards, whereas Fig. 1 (B) depicts the distribution using the BCCN TEa criteria. Based on CLIA88 standards, the Sigma levels for most analytes showed excellent performance (Table 2). Fourteen analytes had a Sigma value of six or higher, with a particularly high value of 30.8 T-bilirubin. There were only three analytes with values below 3: CO₂, LDH, and creatinine (0.2, 0.2, and 1.7, respectively). Overall, Sigma levels decreased when using BCCN TEa compared to CLIA88; 19 analytes had lower Sigma values, and 7 had higher Sigma values. The Sigma value of CO₂ and LDH, which was the lowest when using CLIA88, increased to 1.8 and 7.3, respectively. We decided that BCCN TEa would be more appropriate for clinical laboratories; therefore, we generated new QC rules based on BCCN TEa.

Table 1.

Sigma level of analytes according to the used total allowable error.

TEa	Sigma	Number of analytes	Analytes
CLIA88	6σ<	14	AST, Calcium, CK, D-Bilirubin, GGT, HDL-Cholesterol, LDL-Cholestrol, Lipase, Magnesium, Phosphorus, Potassium, T-Bilirubin, Triglyceride, Uric acid
	4σ-6σ	7	ALP, ALT, BUN, Chloride, Glucose, T.protein, Total-Cholesterol
	3σ-4σ	2	Albumin, Sodium
	<3σ	3	CO2, Creatinine, LDH
BCCN	6σ<	5	Chloride, D-Bilirubin, LDH, Potassium, Triglyceride
	4σ-6σ	11	Albumin, ALP, AST, Calcium, Glucose, HDL-Cholesterol, LDL-Cholestrol, Lipase, Magnesium, T.protein, Total-Cholesterol
	3σ-4σ	5	CK, GGT, Phosphorus, Sodium, T-Bilirubin
	<3σ	5	ALT, BUN, CO2, Creatinine, Uric acid

CLIA 88, Clinical Laboratory Improvement Amendments of 1988; BCCN, Busan Clinical Chemical Network; ALP, Alkaline Phosphatase; ALT, Alanine Aminotransferase; AST, Aspartate Aminotransferase; BUN, Blood Urea Nitrogen; CK, Creatine Kinase; CO2, total carbon dioxide; D-bilirubin, Direct bilirubin; GGT, Gamma-Glutamyl Transferase; HDL-Cholesterol, High-Density Lipoprotein Cholesterol; LDH, Lactate Dehydrogenase; LDL-Cholesterol, Low-Density Lipoprotein Cholesterol; T-protein, Total protein; T-Bilirubin, Total Bilirubin.

Fig. 1.

Standardized QC sigma charts for 26 items analyzed with AU5800. (A) Pre-Phase using the CLIA 88 standards, (B) Pre-Phase using the regional (“BCCN” team) requirement TEa criteria, and (C) Post-Phase, using BCCN TEa criteria.

Table 2.

The change of sigma level according to TEa reference and QC rules.

No	Analytes	TEa		Pre-phase		Post-phase		Sigma pre-phase		Sigma post-phase
		CLIA88	BCCN	Bias %	CV %	Bias %	CV %	CLIA88 TEa	BCCN TEa	BCCN TEa
1	Albumin	10	14.3	−2.34	2.01	−4.18	1.62	3.8	6.0	6.2
2	ALP	30	27.3	5.85	4.95	11	4.63	4.9	4.3	3.5
3	ALT	20	10	−2.04	3.29	−0.355	3.88	5.5	2.4	2.5
4	AST	20	10	−0.741	2.03	−0.524	1.77	9.5	4.6	5.4
5	BUN	13.2	8	2.49	2.13	2.27	1.18	5.0	2.6	4.9
6	Calcium	16	6.8	1.13	1.32	2.09	0.89	11.3	4.3	5.3
7	Chloride	5	6.5	0.495	0.816	0.262	0.732	5.5	7.4	8.5
8	CK	30	12	−0.851	3.15	−0.703	2.71	9.3	3.5	4.2
9	CO2	8.46	19.2	−7.12	6.76	−3.51	6.52	0.2	1.8	2.4
10	Creatinine	15	13	−0.69	2.92	−1.38	3.16	1.7	1.5	3.7
11	D-Bilirubin	66.8	20	−1.58	2.84	−1.22	3.25	23.0	6.5	5.8
12	GGT	33.2	15	−10.2	1.47	−9.81	1.07	15.6	3.3	4.9
13	Glucose	10.2	10.7	3.64	1.32	4.11	1.24	5.0	5.3	5.3
14	HDL-Cholesterol	30	12.5	−1.78	2.08	−1.48	2.37	13.6	5.2	4.6
15	LDH	5.68	20	5.18	2.03	8.09	2.28	0.2	7.3	5.2
16	LDL-Cholesterol	17.8	12	−2.12	1.75	−4.73	2.08	9.0	5.6	3.5
17	Lipase	56.8	33.3	2.09	6.28	−15.6	6.35	8.7	5.0	2.8
18	Magnesium	25	15	2.47	2.57	2.85	2.39	8.8	4.9	5.1
19	Phosphorus	15.2	10	3.47	1.79	2.65	1.8	6.6	3.6	4.1
20	Potassium	6.8	10.5	0.693	1	0.849	1.01	6.1	9.8	9.6
21	Sodium	3.54	4	0.292	0.918	0.168	0.869	3.5	4.0	4.4
22	T-protein	10	9.1	3	1.47	2.79	1.16	4.8	4.1	5.4
23	T-Bilirubin	64	13.3	7.65	1.83	8.25	1.25	30.8	3.1	4.0
24	Total-Cholesterol	10	8.9	1.39	1.57	1.31	1.42	5.5	4.8	5.3
25	Triglyceride	25	15	−1.82	1.27	−1.49	1.36	18.3	10.4	9.9
26	Uric acid	17	8	3.57	1.58	3.26	0.894	8.5	2.8	5.3

TEa, total allowable error; QC, quality control; CV, coefficient of variation; CLIA 88, Clinical Laboratory Improvement Amendments of 1988; BCCN, Busan Clinical Chemical Network; Total ALP, Alkaline Phosphatase; ALT, Alanine Aminotransferase; AST, Aspartate Aminotransferase; BUN, Blood Urea Nitrogen; CK, Creatine Kinase; CO2, total carbon dioxide; D-bilirubin, Direct bilirubin; GGT, Gamma-Glutamyl Transferase; HDL-Cholesterol, High-Density Lipoprotein Cholesterol; LDH, Lactate Dehydrogenase; LDL-Cholesterol, Low-Density Lipoprotein Cholesterol; T-protein, Total protein; T-Bilirubin, Total Bilirubin.

These rules are generally more straightforward than the previous six rules (1–3s, 2-2s, 2/3-2s, R-4s, 4-1s, and 12-x), and 11 (42.3 %) tests used only one rule for internal QC (Table 3).

Table 3.

The change of QC rules; post-phase QC rules were suggested by Westgard advisor.

Pre-phase QC rules	Post-phase QC rules	Analytes
1–3s, 2-2s, 2/3-2s, R-4s, 4-1s, 12-x	1–2s	ALT
	1–3s	HDL-Cholesterol, Lipase, Magnesium, Total-Cholesterol
	1–3.5s	LDL-Cholesterol
	1–4s	Albumin, D-Bilirubin
	1–5s	Chloride, Potassium, Triglyceride
	1–3s, 2/3-2s	ALP, AST
	1–3s, 2/3-2s, R-4s, 3-1s	Calcium, Sodium, T.protein
	1–3s, 2/3-2s, R-4s, 3-1s, 9x	Amylase, BUN, CK, GGT, Uric acid
	1–3s, 2/3-2s, R-4s, 3-1s, 8x	CO2a, Creatininea, LDHa
	1–3s, 2/3-2s, R-4s, 3-1s, 12-x	Glucose, Phosphorus, T-Bilirubin

QC, quality control; ALP, Alkaline Phosphatase; ALT, Alanine Aminotransferase; AST, Aspartate Aminotransferase; BUN, Blood Urea Nitrogen; CK, Creatine Kinase; CO2, total carbon dioxide; D-bilirubin, Direct bilirubin; GGT, Gamma-Glutamyl Transferase; HDL-Cholesterol, High-Density Lipoprotein Cholesterol; LDH, Lactate Dehydrogenase; LDL-Cholesterol, Low-Density Lipoprotein Cholesterol; T-protein, Total protein; T-Bilirubin, Total Bilirubin.

^aRequire 6 points (duplicate run 3 levels of material), other analytes require 3 points (one time run 3 levels).

2.4. Application of new QC rules and evaluation of its appropriateness

To check the appropriateness of the quality management, the results of the Post-Phase QC-repeat rate, turnaround time (TAT), proficiency test (PT) results, College of American Pathologists (CAP) PT, Korean Association of External Quality Assessment Service (KEQAS), and Sigma levels were compared with those of the Pre-Phase. The impact of the new QC rules on laboratory efficiency was measured by counting the rate of QC-repeat and the over-rate of TAT limitation for 1 h for stat tests. To ensure that introducing new QC rules did not lead to inappropriate IQC, changes in the PT program results and Sigma values between the two periods were checked. By checking the PT and Sigma values, we attempted to determine the quality of the test by adopting an integrated approach for imprecision and bias.

The Pre- and Post-Phase PTs were the first and second rounds of the KEQAS and CAP-A, and the third and fourth rounds of the KEQAS and CAP-B, respectively. In the case of the KEQAS, three different concentrations of each substance were tested for each item, and five different concentrations of each substance were tested for the CAP PT. The cases that exceeded three SDI and two SDI compared to the peer group for each analyte were compared between the Pre- and Post-Phases.

2.5. Statistics and Ethics

The Sigma distribution of each analyte was drawn using Microsoft Excel (Microsoft, USA), the comparison of CVs and bias of Pre- and Post-Phases was shown by a Bland-Altman plot, and the P-value was calculated using the Wilcoxon signed-rank test. All statistical analyses were performed using Analyze-it (version 5.90; Analyze-it Software Ltd., Leeds, UK), except for the Wilcoxon signed-rank test, which was performed using SPSS Statistics V.25 (IBM, Armonk, New York, USA). Additionally, a chi-squared homogeneity test was employed to compare the PT results between the Pre- and Post-Phases, with calculations conducted using Microsoft Excel. The study is not subject to institutional review board approval because no clinical information was required, and only laboratory data such as the value and distribution of QC material, calibration, and TAT were used.

3. Results

3.1. Application of new QC rules and the effect on the laboratory

The percentage of QC-repeat samples because of violations of QC rules averaged 5.6 % in the Pre-Phase and decreased to 2.5 % in the Post-Phase. The proportion of out-of-TAT samples because of recalibration decreased from 29.4 % to 15.2 % (Table 4). The difference between the QC-repeat and out-of-TAT cases was statistically significant between the Pre-Phase and Post-Phase. The number of analytes above four Sigma (excellent and good to world-class) increased from 17 items in the Pre-Phase to 20 in the Post-Phase. For the marginal-to-poor class (2–4 Sigma), the number of items decreased from 7 to 6 as seen in Figure (B). Two items (CO₂ and Creatinine) that were unacceptable in the Pre-Phase were moved to the poor and marginal classes, respectively, and the unacceptable items disappeared in the Post-Phase (Figure (C)).

Table 4.

The change of QC efficiency according to QC rule change.

Phase	Month	Total QC (test N)	QC-repeat (test N)	QC-repeat rate (%)	Total samples in peak time (N)	Total samples out-of-TAT (N)	Samples out-of-TAT due to QC failure (N)	Samples out-of-TAT due to QC failure (%)
Pre	Jan	1830	210	11.5 %	432	84	22	26.2 %
	Feb	1696	76	4.5 %	491	80	22	27.5 %
	Mar	1849	67	3.6 %	492	87	19	21.8 %
	Apr	1726	106	6.1 %	496	39	17	43.6 %
	May	1656	36	2.2 %	535	54	15	27.8 %
Pre phase mean		1751.4	99.0	5.6 %	489.2	68.8	19.0	29.4 %
Post	Jun	2021	41	2.0 %	533	105	13	12.4 %
	Jul	2146	76	3.5 %	572	61	15	24.6 %
	Aug	1931	41	2.1 %	513	46	4	8.7 %
	Sep	1929	39	2.0 %	424	58	9	15.6 %
	Oct	1761	51	2.9 %	441	60	9	15.0 %
Post phase mean		1957.6	49.6	2.5 %	496.6	66.0	10.0	15.2 %

N, number; QC, quality control; TAT, turnaround time.

To understand the cause of the decrease in Sigma, we analyzed bias and CV in the Pre- and Post-Phases. For each of the 26 analytes, we used three levels of QC materials; therefore, we analyzed the average CV between the Pre- and Post-Phases for a total of 78 QC materials, of which 61 materials showed a decrease in CV in the Post-Phase compared to the Pre-Phase, and 17 materials showed an increase. On average, the CV decreased by approximately 11.8 % (95 % CI, −42.0 to 18.39 %), which was statistically significant (P < 0.001).

Bias was analyzed as the value of the QC level with the largest total error for each analyte compared to the peer using the same QC material inherent in URT. Compared to the Pre-Phase, 15 out of 26 items showed a decrease in bias in the Post-Phase, and 11 items showed a decrease, with no statistically significant difference (P = 0.33).

3.2. Results of proficiency test

In the Pre-Phase, 67 of the 271 cases exceeded the 2 SDI, and 24 of the 271 cases exceeded the 2 SDI in the Post-Phase. Twenty-seven cases among the 271 cases exceeded 3 SDI in Pre-Phase, and 4 cases among the 271 cases exceeded the 3 in Post-Phase; the difference was significant (P < 0.05) (Table 5). As for PT results after the rule change, total bilirubin in the CAP-B trail and Calcium, Lipase, and Mg in the fourth KEQAS exceeded the 3 SDI or were unacceptable in each PT.

Table 5.

The change of result of proficiency tests; sum of CAP and KEQAS cases.

	Pre-phase		Post-phase
	over 3 SDI	over 2 SDI	over 3 SDI	over 2 SDI
Albumin	0/11 (0.0 %)	5/11 (0.5 %)	0/11 (0.0 %)	0/11 (0.0 %)
ALP	0/11 (0.0 %)	0/11 (0.0 %)	0/11 (0.0 %)	0/11 (0.0 %)
ALT	2/11 (0.2 %)	4/11 (0.4 %)	0/11 (0.0 %)	0/11 (0.0 %)
Amylase	0/11 (0.0 %)	1/11 (0.1 %)	0/60 (0.0 %)	0/60 (0.0 %)
AST	1/11 (0.1 %)	3/11 (0.3 %)	0/11 (0.0 %)	0/11 (0.0 %)
BUN	0/11 (0.0 %)	1/11 (0.1 %)	0/11 (0.0 %)	0/11 (0.0 %)
Calcium	0/11 (0.0 %)	3/11 (0.3 %)	1/11 (0.1 %)	3/11 (0.3 %)
CK	2/11 (0.2 %)	5/11 (0.5 %)	0/11 (0.0 %)	0/11 (0.0 %)
Chloride	5/11 (0.5 %)	5/11 (0.5 %)	0/11 (0.0 %)	1/11 (0.1 %)
Creatinine	0/50 (0.0 %)	2/50 (0.4 %)	0/50 (0.0 %)	0/50 (0.0 %)
D-bilirubin	0/11 (0.0 %)	0/11 (0.0 %)	0/11 (0.0 %)	2/11 (0.2 %)
GGT	0/60 (0.0 %)	2/60 (0.3 %)	0/60 (0.0 %)	0/60 (0.0 %)
Glucose	0/11 (0.0 %)	5/11 (0.5 %)	0/11 (0.0 %)	3/11 (0.3 %)
HDL-Cholesterol	1/11 (0.1 %)	2/11 (0.2 %)	0/11 (0.0 %)	1/11 (0.1 %)
Potassium	5/11 (0.5 %)	5/11 (0.5 %)	0/11 (0.0 %)	1/11 (0.1 %)
LDH	0/60 (0.0 %)	0/60 (0.0 %)	0/60 (0.0 %)	1/60 (0.2 %)
LDL-Cholesterol	0/11 (0.0 %)	0/11 (0.0 %)	0/11 (0.0 %)	0/11 (0.0 %)
Lipase	2/60 (0.3 %)	3/60 (0.5 %)	1/60 (0.2 %)	2/60 (0.3 %)
Magnesium	0/11 (0.0 %)	0/11 (0.0 %)	1/11 (0.1 %)	1/11 (0.1 %)
Sodium	0/11 (0.0 %)	0/11 (0.0 %)	0/11 (0.0 %)	0/11 (0.0 %)
Phosphorus	0/11 (0.0 %)	0/11 (0.0 %)	0/11 (0.0 %)	0/11 (0.0 %)
T-bilirubin	0/11 (0.0 %)	5/11 (0.5 %)	1/11 (0.1 %)	5/11 (0.5 %)
T-cholesterol	6/11 (0.5 %)	7/11 (0.6 %)	0/11 (0.0 %)	3/11 (0.3 %)
T-Protein	0/11 (0.0 %)	3/11 (0.3 %)	0/11 (0.0 %)	0/11 (0.0 %)
CO2	0/11 (0.0 %)	1/11 (0.1 %)	0/11 (0.0 %)	1/11 (0.1 %)
Triglyceride	0/11 (0.0 %)	0/11 (0.0 %)	0/11 (0.0 %)	0/11 (0.0 %)
Uric acid	3/11 (0.3 %)	5/11 (0.5 %)	0/11 (0.0 %)	0/11 (0.0 %)
Total	27/276 (0.1 %)	67/276 (0.2 %)	4/271 (0.0 %)	24/271 (0.1 %)

CAP, College of American Pathologists; KEQAS, Korean Association of External Quality Assessment Service; SDI, standard deviation index; ALP, Alkaline Phosphatase; ALT, Alanine Aminotransferase; AST, Aspartate Aminotransferase; BUN, Blood Urea Nitrogen; CK, Creatine Kinase; CO2, total carbon dioxide; D-bilirubin, Direct bilirubin; GGT, Gamma-Glutamyl Transferase; HDL-Cholesterol, High-Density Lipoprotein Cholesterol; LDH, Lactate Dehydrogenase; LDL-Cholesterol, Low-Density Lipoprotein Cholesterol; T-protein, Total protein; T-Bilirubin, Total Bilirubin.

4. Discussion

In terms of cost-effectiveness, increasing the error detection rate cannot be achieved without incurring significant costs or introducing the potential for an elevated false detection rate. This underscores the inherent challenge of finding a balance between cost considerations and the accuracy of error detection in a laboratory setting. This study aimed to evaluate whether new QC rules using Sigma metrics affect the efficiency of the laboratory or the quality of the tests in the clinical laboratory. Fortunately, a comparison between before and after the QC rule change reduced the burden on the laboratory due to Re-QC and recalibration tasks and reduced the time required to check the IQC results, which also reduced the time required for the first patient to receive the test results during the start of work. In addition, despite simplifying the QC rule, the Sigma level of each test did not deteriorate significantly, and no significant deterioration was observed, with fewer cases showing ±2 SDI or higher in external QC results.

Following the adjustment of the new rules, an examination of analytes in PT exceeding the 3 SDI or deemed unacceptable in comparison to the peer group revealed that, in most cases, the differences with the peer group were smaller than the Total Error Allowable (TEa), aligning with the laboratory's objectives. Specifically, for calcium, lipase, and magnesium, the relative differences between the mean values of the peer group and our laboratory results were consistently smaller than those of the target TEa (2.24 % vs. 6.8 %, 11.62 % vs. 14.3 %, and 4.55 % vs. 15.0 % for calcium, lipase, and magnesium, respectively).

In our study, the improved Sigma metrics in the Post-Phase was driven by a reduction in CV, the cause of which is uncertain. The possible explanation is the inclusion of QC results, which had been falsely rejected and deleted in the Pre-phase, and increased number of QC runs for three analytes, total CO2, creatinine, and LDH, and the enhanced proficiency of the designated QC practitioners over time. In Post-phase, the effect of repeated QC results on the QC mean disappeared. QC repeats should be conducted to rule out problems such as material deterioration or random errors and should be avoided repeatedly to obtain results within acceptable limits [13,14]. It is also possible that the QC practitioner who had been designated since January 2020 improved their understanding and skills during the study period. Actually, the monthly CV of most analytes for 5 months after the study period maintained the lower Post-phase values or showed only partial recovery to Pre-phase. The gap from the peer laboratory using the same reagent and QC material and a smaller model of same manufacturer was maintained, despite itemized differences (Supplement Tables). Although the introduction of automated instruments has reduced operator influence on the results [15], human variation still exists, and many performance evaluation guidelines require operators to be acclimated to the instrument or analyzer before testing [16]. Our findings underscore that operator skill can materially affect CV—and thus Sigma—and that a laboratory's performance level is not static. Regular reassessment of Sigma metrics and corresponding adaptation of IQC rules are therefore essential to sustain optimal performance. Westgard's multiple rules are widely known and have been used by many clinical laboratories as rules for judging QC results [4,7,17] and which rules to implement have been left to the responsible person's judgment based on testing needs. As a method for determining performance-specific QC rules for individual labs, the Westgard Six Sigma framework can underestimate analytical performance and give low-quality impressions [[18], [19], [20]]. Nevertheless, it is simple for laboratories to adopt, as it is based on familiar rules, embedded in common QC software, and does not require additional hardware. When laboratories want to increase efficiency by augmenting simple 2 SD/3 SD regimens, or simplifying complex multiple-rule protocols like ours, Sigma-based models offer clear and user-friendly benefits.

This study has several limitations, including inherent constraints of the Six Sigma methodology itself. First, determining an accurate bias is challenging, since true bias ideally derives from repeated measurements of reference materials —an impractical approach for every analyte in routine clinical laboratories. Although some external quality assessment (EQA) programs measure accuracy against reference methods, many rely on peer-group comparisons to estimate bias. In our work, we accepted URT's long-term peer-group QC data as a reliable surrogate for within-instrument bias. We also assessed accuracy via EQA outcomes to compare analytical performance between Pre- and Post-Phases; however, such interchangeability assessments may not fully reflect absolute accuracy. TEa is a critical [6,[20], [21], [22]] —and potentially hazardous—component of Sigma calculations, since over- or under-estimating TEa directly skews the resulting Sigma metric. Proper TEa selection is therefore paramount. TEa is often derived from within- and between-subject biological variation, but this approach has conceptual and metrological shortcomings for use as a performance specification [11,23]. In our study, we chose a Model 1b TEa based on regional clinicians' requirements—closer to a clinical outcome target—rather than a purely formula-driven Model 2 TEa, to better align with patient-centered goals.

Even the best guidelines face practical obstacles in the clinical setting. Managers and staff may resist change if it is perceived as burdensome or threatening to existing workflows [24]. Conversely, providing reassurance that new procedures are safe and beneficial enhances acceptance [25]. Although we initially encountered psychological resistance when simplifying our QC rules, the overall workload fell, turnaround times shortened, and analytical quality was maintained.

5. Conclusion

To the best of our knowledge, this is the first report of applying Sigma-based multirules in a clinical laboratory and comparing both laboratory efficiency and analytical quality before and after implementation. Although the Six Sigma approach may not be the most ideal or perfect method, a key strength of this study lies in its practical application of the proposed strategy within a clinical laboratory setting and the subsequent analysis of its impact. This study is a case of a single institution and cannot be considered representative of the whole; however, it can be used as a reference for other laboratories looking for an appropriate QC rule.

CRediT authorship contribution statement

Hyunji Choi: Writing – review & editing, Writing – original draft, Investigation, Data curation. Ina Jeong: Writing – original draft, Investigation, Data curation. Jeongeun Cheon: Writing – review & editing, Methodology, Investigation, Data curation. Chul-Min Park: Writing – review & editing, Supervision, Project administration, Conceptualization. Sun Min Lee: Writing – review & editing, Project administration, Funding acquisition, Data curation, Conceptualization.

Funding sources

This study was supported by a 2024 research grant from Pusan National University Yangsan Hospital.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have influenced the work reported in this paper.

Appendix A. Supplementary data

The following is the Supplementary data to this article.

Data availability

The authors do not have permission to share data.