Quantitative Application of Sigma Metrics in Medical Biochemistry

Abstract

Introduction: Laboratory errors are result of a poorly designed quality system in the laboratory. Six Sigma is an error reduction methodology that has been successfully applied at Motorola and General Electric. Sigma (σ) is the mathematical symbol for standard deviation (SD). Sigma methodology can be applied wherever an outcome of a process has to be measured. A poor outcome is counted as an error or defect. This is quantified as defects per million (DPM). A six sigma process is one in which 99.999666% of the products manufactured are statistically expected to be free of defects. Six sigma concentrates, on regulating a process to 6 SDs, represents 3.4 DPM (defects per million) opportunities. It can be inferred that as sigma increases, the consistency and steadiness of the test improves, thereby reducing the operating costs. We aimed to gauge performance of our laboratory parameters by sigma metrics.

Objectives: Evaluation of sigma metrics in interpretation of parameter performance in clinical biochemistry.

Material and Methods: The six month internal QC (October 2012 to march 2013) and EQAS (external quality assurance scheme) were extracted for the parameters-Glucose, Urea, Creatinine, Total Bilirubin, Total Protein, Albumin, Uric acid, Total Cholesterol, Triglycerides, Chloride, SGOT, SGPT and ALP. Coefficient of variance (CV) were calculated from internal QC for these parameters. Percentage bias for these parameters was calculated from the EQAS. Total allowable errors were followed as per Clinical Laboratory Improvement Amendments (CLIA) guidelines. Sigma metrics were calculated from CV, percentage bias and total allowable error for the above mentioned parameters.

Results: For parameters - Total bilirubin, uric acid, SGOT, SGPT and ALP, the sigma values were found to be more than 6. For parameters – glucose, Creatinine, triglycerides, urea, the sigma values were found to be between 3 to 6. For parameters – total protein, albumin, cholesterol and chloride, the sigma values were found to be less than 3.

Conclusion: ALP was the best performer when it was gauzed on the sigma scale, with a sigma metrics value of 8.4 and chloride had the least sigma metrics value of 1.4.

Introduction

The quality controls (QCs) in the biochemistry laboratory are internal QC and external QC. The internal quality control is run daily and it is interpreted by the standard Westgard rules. The external quality control is run once a month and it is interpreted by Z score, Standard deviation index (SDI). A Z score is a calculated value that tells us as to how many standard deviations a control result is from the mean value which is expected for that material. It is calculated by taking the difference between the control result and the expected mean and by, then dividing it by the standard deviation which is observed for that control material. The best Z score is zero. The further from zero the Z score is, the worse is the result. Generally, a Z score of less than 1.0 from zero is excellent and, that of up to 2.0 is acceptable. Z scores which are greater than 3.0 from zero are considered to be unacceptable and corrective action should be undertaken. SDI is calculated from the data of all the laboratories, which is analyzed to determine an overall average and standard deviation for the group. The program will generally report your performance which is relative to the group. The difference between laboratory’s test results and the overall average is often expressed by a Standard Deviation Index, or (SDI), which expresses the difference in terms of the number of standard deviations from the overall mean. An SDI of zero indicates perfect comparison with the peer group, an SDI of upto <2.0 is acceptable and an SDI of > 2.0 is unacceptable. The exact number of defects or errors done by the laboratory cannot be assessed by running internal and external QCs [1,2].

The exact number of defects or errors done by the laboratory can be quantified by employing sigma metrics in the laboratory. The correlation between the sigma metrics and defects are as follows: 1 sigma (σ) corresponds to 6,90,000 defects or errors per million reports, 2 sigma corresponds to 3,08,000 errors per million reports, 3 sigma corresponds to 66,800 errors per million reports, 4 sigma corresponds to 6,210 errors per million reports, 5 sigma corresponds to 230 errors per million reports and 6 sigma corresponds to 3.4 errors per million reports [3]. According to Nevalainen et al., “average products, regardless of their complexity, have a quality performance value of about 4σ. The best, or ‘world class quality,’ products have a level of performance of 6σ.” Thus, with the aid of Six Sigma principles and metrics, it is possible to assess the quality of laboratory testing processes and the QC that is needed to ensure that the desired quality is achieved. The present study was undertaken to evaluate the performance of biochemical parameters by calculating the sigma metrics for individual parameters and to determine the errors associated with each parameter [4].

Material and Methods

The six month internal QC (October 2012 to march 2013) and EQAS (external quality assurance scheme) were extracted for 13 parameters -Glucose, Urea, Creatinine, Total Bilirubin, Total Protein, Albumin, Uric acid, Total Cholesterol, Triglycerides, Chloride, SGOT, SGPT and ALP.

The parameters were measured in cobasintegraautoanalyzer.

These 13 parameters were divided into 3 groups – Group A, Group B and Group C.

Group A included glucose, urea, Creatinine, uric acid. (4 parameters).

Group B included Total bilirubin, total protein, albumin, SGOT, SGPT, ALP. (6 parameters).

Group C included Total cholesterol, triglycerides, chloride. (3 parameters).

Total allowable error: Laboratory quality specifications are often defined in terms of allowable total error limits (TEa). If the difference between the true concentration of an analyte and the reported concentration in a patient’s specimen exceeds TEa, the result is considered to be unreliable. Total allowable errors were followed as per Clinical laboratory Improvement Amendments (CLIA) guidelines.

Bias: Bias is the systematic difference between the expected results obtained by the laboratory’s test method and the results that would be obtained from an accepted reference method. The reference may be a consensus reference like a proficiency program or an inter-laboratory peer comparison program. Percentage bias for the parameters were calculated from the CMC, EQAS.

CV% is the analytical coefficient of variation of the test method. Coefficient of variance (CV) were calculated from Biorad internal QC for the parameters.

Sigma metrics were calculated from CV, percentage bias and total allowable error for the parameters by the following formula:

Sigma metrics = (%TEa - %Bias) / %CV.

Results

The sigma metrics for 6 months and overall sigma metrics for the parameters have been mentioned below [Table/Fig-1,2and3].

[Table/Fig-1]:

Group A

Sl. No.	Parameter	Oct 2012 Sigma metrics	Nov 2012 Sigma metrics	Dec 2012 Sigma metrics	Jan 2013 Sigma metrics	Feb 2013 Sigma metrics	March 2013 Sigma metrics	Overall 6 month sigma metrics
1	Glucose	3.2	2.8	2.75	2.6	4.5	3.6	3.2
2	Urea	1.7	6.4	3.3	5.3	7.0	7.4	5.2
3	Creatinine	0.8	2.2	2.2	3.8	5.6	4.0	3.1
4	Uric acid	4.3	7.4	6.7	4.6	15	5.6	6.1

[Table/Fig-2]:

Group B

Sl. No.	Parameter	Oct 2012 Sigma metrics	Nov 2012 Sigma metrics	Dec 2012 Sigma metrics	Jan 2013 Sigma metrics	Feb 2013 Sigma metrics	March 2013 Sigma metrics	Overall 6 month sigma metrics
1	Total bilirubin	4.3	7.4	6.7	4.6	6.5	10	6.5
2	Total protein	3.2	0.6	1.9	0	3.5	3.5	2.1
3	Albumin	0.4	3.4	2.3	0	4.3	4.5	2.5
4	SGOT	4.5	6.1	5.4	5.8	12	6	6.6
5	SGPT	6.4	4.6	6.1	5.4	8.3	5.7	6.1
6	ALP	9.3	7.2	8.1	6.4	11.3	8.6	8.4

[Table/Fig-3]:

Group C

Sl. No.	Parameter	Oct 2012 Sigma metrics	Nov 2012 Sigma metrics	Dec 2012 Sigma metrics	Jan 2013 Sigma metrics	Feb 2013 Sigma metrics	March 2013 Sigma metrics	Overall 6 month sigma metrics
1	Total cholesterol	1.8	2.6	2.6	1.1	1.2	3.6	2.2
2	Triglycerides	3.6	3.9	5.0	5.8	5.7	5.8	5.9
3	Chloride	0.24	0.8	0.8	0.8	5	1.0	1.4

Discussion

A good laboratory practice requires that laboratories design their quality control (QC) procedures to assure that reported patient results meet the quality required for their intended use [5]. The Sigma metrics is based on the statistical concept: laboratory errors can be reduced by maintaining 6 standard deviations between the parameter average and its upper and lower limits.

In our study, the sigma metrics value for glucose was 3.2. In a study which was done by James O Westgard et al., the sigma metrics value for glucose was found to be between 2.9 to 3.3 [6]. For cholesterol, the sigma metrics value was found to be 2.2. In a study done by James O Westgard et al., the sigma metrics value for total cholesterol was found to be between 2.9 to 3.0 [6].

In our study, the sigma value for urea was 5.2, which was not similar to the findings of the study done by Bhawna Singh et al., The sigma metrics value for triglycerides and SGOT were found to be more than 6, which were is similar to the findings of Bhawna Singh et al., For ALP, sigma metrics value in our study was more than 6, whereas the sigma metrics value obtained by the study done by Bhawna Singh et al., was between 3.1 to 5.9 [7]. In our study, the sigma metrics value for Creatinine was found to be 3.1. In a study done by Carl Garber, the sigma metrics value for Creatinine was found to be 6 [8].

Sigma values are useful for guiding QC strategy design. For a high sigma process, it is relatively easy for the laboratory to design a QC procedure, to detect any out-of-control condition that could pose a significant risk of producing unreliable results. A relatively large out-of-control condition would have to occur before there would be much chances of producing results that contained errors that exceeded the TEa specification and it is easy to design QC procedures that can detect large out-of-control conditions. The sigma metrics values are useful in setting the internal QC acceptability criteria. For a 6 sigma process (or higher), use 3.5 SD control limits with N (number of controls to be run per day)=2 have to be used; For a 5 sigma process, use 3.0 SD control limits with n=2 have to be used; For a 4 sigma process, use 2.5 SD control limits or a multirule procedure with n=4 have to be used; For a 3 sigma process, use a multirule procedure with n of 6 or 8 have to be used. For less than 3 sigma, method performance must be improved before the method can be used for routine production [9].

For parameters like glucose, Creatinine, T. Bilirubin, uric acid, SGOT, SGPT and ALP, sigma metrics value is above 6. So, for these parameters, the QC protocol does not need any change and patient results can be released. For parameters like glucose, Creatinine, triglycerides, the sigma metrics value is between 3 to 6. For these parameters, QC monitoring should be done, but still it is acceptable. For parameters like total protein, albumin, total cholesterol and chloride, the sigma metrics value was found to be less than 3. A very stringent internal QC has to be followed for these parameters, and the frequency of internal QC should be increased and corrective action should be taken for these parameters.

According to George G. Klee et al., “The application of six sigma principles and metrics is very valuable for all phases of the laboratory testing process. The core business of the laboratory is to produce accurate test results and, it makes sense to first apply six sigma to the analytical processes. This also is the easiest application, because there are tolerance limits in the form of acceptability criteria from peer comparison and proficiency testing programs, QC data available for estimating method precision, and peer data available for estimating method bias. Laboratories should next expand their efforts to the preanalytical and post–analytical processes, knowing that their core process is producing the necessary analytical quality” [1].

Conclusion

ALP was the best performer when it was gauzed on the sigma scale, with a sigma metrics value of 8.4 and chloride had the least sigma metrics value of 1.4.

Financial or Other Competing Interests

None.