LEGO Plot for Simultaneous Application of Multiple Quality Requirements During Trueness Verification of Quantitative Laboratory Tests

Abstract

Background

We developed a two‐dimensional plot for viewing trueness that takes into account potential shift and variable quality requirements to verify trueness using certified reference material (CRM).

Methods

Glucose, total cholesterol (TC), and creatinine levels were determined by two kinds of assay in two levels of a CRM. Available quality requirements were collected, codified, and sorted in an ascending order in the plot's header row. Centering on the mean of measured values from CRM, the “mean ± US CLIA '88 allowable total error” was located in the header of the leftmost and rightmost columns. Twenty points were created in intervening columns as potential shifts. Uncertainties were calculated according to regression between certified values and uncertainties of CRM, and positioned in the corresponding columns. Cells were assigned different colors where column and row intersected based on comparison of the 95% confidence interval of the percentage bias with each quality requirement.

Results

A glucose assay failed to meet the highest quality criteria, for which shift of +0.13–0.14 mmol/l was required. A TC assay met the quality requirement and a shift of ±0.03 mmol/l was tolerable. A creatinine assay also met the quality requirement but any shift was not tolerable.

Conclusion

The plot provides a systematic view of the trueness of quantitative laboratory tests.

INTRODUCTION

Quantitative laboratory tests for analytes such as glucose, total cholesterol (TC), and creatinine are used to diagnose diabetes, hyperlipidemia, and chronic kidney disease as well as monitor the response to treatment. Thus, it is important for clinical laboratories to verify the accuracy of their tests by subjecting them to external quality assessments (EQAs). Moreover, the verification of trueness is included to evaluate the performance of instruments and reagents 1, 2, 3. To evaluate bias statistically, commutable patient specimens are analyzed using both test and reference methods, or reference materials with certified or assigned values are analyzed using the test method and then the data are evaluated using a t‐test 4.

The larger the sample size is or the higher the precision is, the more likely to be significant the results of statistical test for the bias are, even though the bias is clinically small. It is because statistical significance depends on sample size and the dispersion of distribution of the data. Therefore, to evaluate the observed inaccuracy or bias, it is necessary to compare it with acceptable criteria for each analyte, taking into consideration clinical significance. This is a common approach in pharmaceutical clinical trials where a judgment on the effect size is made according to statistical and clinical significance.

When evaluating the analytical performance of quantitative laboratory tests, several quality requirements may be considered as criteria for clinical significance, and a hierarchy is used in the strategies for setting a quality specification 4, 5. However, judgment of the analytical performance, including accuracy or trueness, may differ depending on type and size of the quality requirements used. Although the observed bias should be compared with an allowable bias (Ba), allowable total error (TEa) can be considered as supportive when the number of replicate measurements is insufficient, because the mean of the measurements may include random errors. Further, the analytical performance characteristics observed at a single time reflect only the status of the analytical system at that particular time. For the quality control of an analytical system, it is important to determine the direction and the magnitude of shift from the system's current state that will be required to acquire trueness or tolerable to maintain already secured trueness.

To determine the laboratory significance of estimated bias during trueness verification of quantitative laboratory tests, we developed a two‐dimensional plot that applies various quality requirements simultaneously while simulating potential shifts to predict trueness. This plot is called the multi‐Level quality requirements trueness vErification for measurements simulated as Gradients at Once (LEGO) Plot and applied for trueness verification of assays for glucose, TC, and creatinine using certified reference materials (CRMs).

MATERIALS AND METHODS

Test Methods

Glucose, TC, and creatinine were analyzed using the Sekisui (Sekisui Medical (formerly Daiichi), Tokyo, Japan) and Wako (Wako Pure Chemical Industries, Osaka, Japan) assays with a Hitachi Labospect 008 (Hitachi High‐Tech Co., Tokyo, Japan).

Trueness Verification

CRM 111‐01‐01A and CRM 111‐01‐02A (Korea Research Institute of Standards and Science (KRISS), Daejeon, Republic of Korea) were analyzed to verify trueness, and the certified values with their expanded uncertainty for each analyte are shown in Table 3. Measurements and data analyses were performed according to the Clinical and Laboratory Standards Institute (CLSI) document EP15‐A2 6. CRM comprises two concentration levels. Glucose, TC, and creatinine were each measured five times. Measurements were performed for 5 days, once daily and once per run. The standard uncertainty (u_st) was calculated by dividing the expanded uncertainty described on the “KRISS Certificate of Analysis” with coverage factor 2. If the t‐statistic calculated using the certified value, mean, u_st, and the standard error of CRM measurements was greater than 2.776 (df = 4), the difference was considered statistically significant 4. Otherwise, trueness was considered verified.

Table 3.

Trueness Verification and Bias Estimation of Wako and Sekisui Assays for Glucose, TC, and Creatinine

CRM	Analyte	Specified value ± expanded uncertainty	ust	Source	Mean	SDa	CV	Bias	95% CI of bias		Percentage bias	95% CI of % bias		t‐Statistic
1b	Glucose (mmol/l)	7.27 ± 0.15	0.07	Wako	7.12	0.08	1.10%	−0.16	−0.22	−0.09	−2.10%	−3.10%	−1.20%	−1.94
				Sekisui	7.02	0.13	1.80%	−0.26	−0.37	−0.15	−3.50%	−5.00%	−2.00%	−2.80
	TC (mmol/l)	3.33 ± 0.07	0.03	Wako	3.39	0.02	0.60%	0.06	0.04	0.08	1.80%	1.20%	2.40%	1.73
				Sekisui	3.32	0.02	0.50%	−0.01	−0.02	0.01	−0.30%	−0.70%	0.20%	−0.28
	Creatinine (μmol/l)	53.98 ± 1.08	0.54	Wako	54.81	0.63	1.10%	0.83	0.28	1.37	1.50%	0.50%	2.50%	1.36
				Sekisui	53.57	0.48	0.90%	−0.41	−0.84	0.01	−0.80%	−1.50%	0.00%	−0.71
2c	Glucose (mmol/l)	4.56 ± 0.09	0.05	Wako	4.41	0.03	0.70%	−0.15	−0.18	−0.13	−3.40%	−4.00%	−2.80%	−3.24
				Sekisui	4.45	0.02	0.50%	−0.11	−0.13	−0.09	−2.40%	−2.80%	−2.00%	−2.33
	TC (mmol/l)	4.09 ± 0.08	0.04	Wako	4.14	0.03	0.70%	0.04	0.02	0.07	1.00%	0.40%	1.70%	0.98
				Sekisui	4.10	0.03	0.80%	0	−0.02	0.03	0.10%	−0.60%	0.80%	0.09
	Creatinine (μmol/l)	249.17 ± 4.98	2.49	Wako	245.40	1.01	0.40%	−3.77	−4.65	−2.89	−1.50%	−1.90%	−1.20%	−1.49
				Sekisui	247.34	1.70	0.70%	−1.83	−3.32	−0.34	−0.70%	−1.30%	−0.10%	−0.70

TC, total cholesterol; KRISS, Korea Research Institute of Standards and Science; CRM, certified reference material; u_st, standard uncertainty; CV, coefficient of variation.

^aNumber of replicates = 5.

^bCRM 111‐01‐01A.

^cCRM 111‐01‐02A.

Rows of the LEGO Plot: Simultaneous Application of Multiple Quality Requirements

The quality requirements were taken from the Westgard website (http://www.westgard.com/clia‐quality.htm) 7 and coded by combining Roman numerals and letters in square brackets extracted from the level and type within the hierarchy (Table 1) 4. The alphanumeric codes in brackets were concatenated with an abbreviation of the name of the quality requirement (Table 2). The coded quality requirements were sorted in ascending order according to size and placed in the first column of the plot.

Table 1.

Hierarchy of Procedures for Setting Analytical Quality Specifications for Laboratory Methods

Hierarchy of analytical quality specifications	Hierarchical code
I. Evaluation of the effect of analytical performance on clinical outcomes in specific clinical settings	[I]
II. Evaluation of the effect of analytical performance on clinical decisions in general
A. Data based on components of biological variation	[IIA]
B. Data based on analysis of clinicians' opinions	[IIB]
III. Published professional recommendations
A. From national and international expert bodies	[IIIA]
B. From expert local groups or individuals	[IIIB]
IV. Performance goals set by
A. Regulatory bodies	[IVA]
B. Organizers of EQA schemes	[IVB]
V. Goals based on the current state of the art
A. Data from EQA/proficiency testing scheme	[VA]
B. Data from current publications on methods	[VB]

EQA, external quality assessment.

Table 2.

Codes for Quality Requirements Indicate the Name and Level in the Hierarchy of Analytical Quality Specifications

Hierarchical code	Name code	Definition of the quality requirement
[IIA]	BBO	Optimum allowable Bias from “Biological Variation Database specifications”
[IIA]	EB	Allowable Bias from “European Biologic Goals”
[IIA]	BBD	Desirable allowable Bias from “Biological Variation Database specifications”
[IIA]	BBM	Minimum allowable Bias from “Biological Variation Database specifications”
[IIA]	BTO	Optimum allowable Total error from “Biological Variation Database specifications”
[IIA]	ET	Allowable Total error from “European Biologic Goals”
[IIA]	BTD	Desirable allowable Total error from “Biological Variation Database specifications”
[IIA]	RT	Allowable Total error from “RCPA (Australasian) Quality Requirements”
[IVA]	UT	Allowable Total error from “US CLIA Requirements for Analytical Quality”
[IIA]	BTM	Minimum allowable Total error from “Biological Variation Database specifications”
[IVA]	GT	Allowable Total error from “Rilibak–German Guidelines for Quality”
[IIB]	CI	Decision Intervals from “Clinical Quality Requirements”

RCPA, Royal College of Pathologists of Australasia.

Columns of the LEGO Plot: Simulation of Certified Values Up To ± US CLIA’88 TEa

The mean concentrations of analytes determined from the CRMs were entered in the center column (P₅). The value of the “mean + US CLIA’88 TEa (TEa^{US CLIA’88})” was defined as a simulated negative shift with size equal to TEa^{US CLIA’88} and entered in the leftmost column (P₁₀). The value of the “mean – TEa^{US CLIA’88}” was defined as a simulated positive shift with size equal to TEa^{US CLIA’88} and placed at the rightmost column (P₀). The columns from P₁₀ to P₀ were set to a range that would cover the material's virtual certified value. The concentrations that corresponded to the 10th, 20th,…, 90th percentiles of the range from columns P₁₀ to P₅ were arranged in the columns between P₁₀ and P₅ as P_9.5, P₉,…, P₆, and P_5.5. Similarly, the values corresponding to P_4.5, P₄,…, P₁, and P_0.5 were arranged between columns P₅ and P₀ and represent the spectrum of simulated certified values. The virtual uncertainty that corresponds to each of the simulated certified values was calculated by linear regression between uncertainties and certified values of the two CRM concentrations, and placed in columns P₁₀ to P₀.

Color of the t‐Statistic: Representation of the Statistical Significance of Bias

For each column representing the spectral shift, bias and percentage bias with 95% confidence interval (CI) were calculated from the measured concentrations, simulated certified values, and simulated uncertainty. t‐Statistics greater than 2.776—representing a statistically significant bias—were highlighted in red, and those lower were shown in green.

Color of the Cell in the LEGO Plot: Representation of Laboratory Significance of Bias in Each Simulated Shift

If the absolute value of the upper limit of the 95% CI of the percentage bias listed in each column was less than the quality requirements met in each row, the observed bias had no laboratory significance related to the particular quality requirement and trueness was verified. The cells in which the corresponding row and column intersect are green. If the absolute value of the lower limit of the 95% CI of the percentage bias in each column was greater than the quality requirements in each row, the bias had laboratory significance related to the particular quality requirement and trueness was not verified. The cells in which the corresponding row and column intersect are red. If the 95% CI of the percentage bias in each column included the quality requirements met in each row, trueness was judged borderline. The cells where the corresponding row and column intersect are yellow (Fig. 1).

Figure 1.

Development of the LEGO plot.

Data Analysis

Data were analyzed using Microsoft Excel 2007 (Microsoft, Redmond, WA). This research was carried out with the approval of the Catholic Medical Center Institutional Review Board (XC09FZZZ0084H).

RESULTS

Trueness Verification

Glucose: For the Wako assay, the t‐statistic for the measurement of CRM 111‐01‐01A was 1.94 and there was no statistically significant difference from the certified value. However, the t‐statistic of CRM 111‐01‐02A was −3.24. The average of the measured values was significantly lower than the certified value. Its percentage bias (95% CI) was −3.40% (−4.00, −2.80%). The t‐statistic of CRM 111‐01‐01A for the Sekisui assay was −2.80 and the mean of the measurements was statistically significantly and less than the certified value. The percentage bias was −3.50% (−5.00, −2.00%). The t‐statistic for CRM 111‐01‐02A was 2.33, and there was no statistically significant difference from the certified value (Table 3).

TC: The means of the TC concentration for the Wako and Sekisui assays measured from CRM 111‐01‐01A and CRM 111‐01‐02A were not statistically significantly different compared with the certified value. The biases were not statistically significant (Table 3).

Creatinine: The means of the creatinine concentrations for the Wako and Sekisui assays measured from CRM 111‐01‐01A and CRM 111‐01‐02A were not statistically significantly different compared with the certified value. The biases were not statistically significant (Table 3).

LEGO Plot

Glucose: The Wako assay met [IIA] BBM 3.4%, the fourth quality requirement, and subsequent requirements in hierarchy for the CRM 111‐01‐01A. With a positive shift of 0.14 mmol/l from the current state, all quality requirements including [IIA] BBO 1.1%, the highest quality criterion, were met. For CRM 111‐01‐02A, [IIA] ET 5.5%, the sixth quality requirement, and subsequent requirements were met for CRM 111‐01‐01A. With a positive shift of 0.13 mmol/l, all quality requirements including [IIA] BBO 1.1%, the highest quality criterion, were met.

The Sekisui assay met [IIA] ET 5.5%, the sixth quality requirement, and the subsequent requirements for CRM 111‐01‐01A. With a positive shift of 0.22–0.29 mmol/l from the current state, [IIA] BBD 2.2%, the third quality requirement, and the subsequent requirements in the hierarchy would be met. For CRM 111‐01‐02A, [IIA] BBM 3.4%, the fourth quality requirement, and subsequent requirements were met. With a positive shift of 0.09–0.13 mmol/l, all quality requirements, including [IIA] BBO 1.1%, the highest in the hierarchy, would be met (Fig. 2 and Table 4).

Figure 2.

LEGO plots for trueness verification of glucose assays. (A) CRM 111‐01‐01A, Wako assay. (B) CRM 111‐01‐02A, Wako assay. (C) CRM 111‐01‐01A, Sekisui assay. (D) CRM 111‐01‐02A, Sekisui assay. The first seven rows show the specified value and its uncertainty, bias, percentage bias, 95% CI, and t‐statistic. t‐Statistics (df = 4) greater than 2.776 are highlighted in red, indicating that the bias in the column is statistically significant. Those less than 2.776 are shown in green. The 12 quality requirements coded according to their name are sorted in ascending order by size in each row. Each column represents the virtual certified value possible for the CRM in a simulated shift. The column filled in blue box indicates the results of actual measurements of the CRM. Cell color is determined by comparing the 95% CI of percentage bias in each simulated shift, and the intersecting row represents various quality requirements, green for trueness verified, red for bias where 95% CI is greater than each quality requirement, or yellow for the bias where 95% CI includes each quality requirement.

Table 4.

Necessary and Tolerable Shifts for Trueness

				Current state						Desirable state
Analyte	Source	CRM	Specified value	Percentage bias	t‐Statistic	Statistical significance of bias	Highest level of fulfilled quality requirement		Shift necessary to fulfill quality requirement with level as high as possible	Level of quality requirement fulfilled after shift		Tolerable shift to maintain fulfillment of quality requirement
Glucose (mmol/l)	Wako	1a	7.27	−2.10%	−1.94	No	[IIA] BBM (4th/12)	3.40%	+0.14 mmol/l	[IIA] BBO (1st/12)	1.10%	0.00 mmol/l
		2b	4.56	−3.40%	−3.24	Yes	[IIA] ET (6th/12)	5.50%	+0.13 mmol/l	[IIA] BBO (1st/12)	1.10%	0.00 mmol/l
	Sekisui	1	7.27	−3.50%	−2.8	Yes	[IIA] ET (6th/12)	5.50%	+0.22 mmol/l	[IIA] BBD (3rd/12)	2.20%	+0.29 mmol/l
		2	4.56	−2.40%	−2.33	No	[IIA] BBM (4th/12)	3.40%	+0.09 mmol/l	[IIA] BBO (1st/12)	1.10%	+0.13 mmol/l
TC (mmol/l)	Wako	1	3.33	1.80%	1.73	No	[IIA] BBD (2nd/12)	4.00%	−0.03 mmol/l	[IIA] BBO (1st/12)	2.00%	−0.10 mmol/l
		2	4.09	1.00%	0.98	No	[IIA] BBO (1st/12)	2.00%	0.00 mmol/l	[IIA] BBO (1st/12)	2.00%	−0.08 mmol/l
	Sekisui	1	3.33	−0.30%	−0.28	No	[IIA] BBO (1st/12)	2.00%	0.00 mmol/l	[IIA] BBO (1st/12)	2.00%	±0.03 mmol/l
		2	4.09	0.10%	0.09	No	[IIA] BBO (1st/12)	2.00%	0.00 mmol/l	[IIA] BBO (1st/12)	2.00%	±0.04 mmol/l
Creatinine (μmol/l)	Wako	1	53.98	1.50%	1.36	No	[IIA] EB (2nd/12)	2.80%	0.00 μmol/l	[IIA] EB (2nd/12)	2.80%	0.00 μmol/l
		2	249.17	−1.50%	−1.49	No	[IIA] BBO (1st/12)	1.90%	0.00 μmol/l	[IIA] BBO (1st/12)	1.90%	+3.74 μmol/l
	Sekisui	1	53.98	−0.80%	−0.71	No	[IIA] BBO (1st/12)	1.90%	0.00 μmol/l	[IIA] BBO (1st/12)	1.90%	0.00 μmol/l
		2	249.17	−0.70%	−0.7	No	[IIA] BBO (1st/12)	1.90%	0.00 μmol/l	[IIA] BBO (1st/12)	1.90%	+3.74 μmol/l

TC, total cholesterol; KRISS, Korea Research Institute of Standards and Science; CRM, certified reference material.

^aCRM 111‐01‐01A.

^bCRM 111‐01‐02A.

TC: The Wako assay met the second quality requirement, [IIA] BBD 4.0%, and subsequent requirements in the hierarchy for CRM 111‐01‐01A. With a negative shift of 0.03–0.10 mmol/l, all quality requirements, including [IIA] BBO 2.0%, the highest, would be met. For CRM 111‐01‐02A, all quality requirements were met, including [IIA] BBO 2.0%, the highest. This level of trueness would be maintained even with a negative shift from the current state up to 0.08 mmol/l.

The measurements of both CRM 111‐01‐01A and CRM 111‐01‐02A using the Sekisui assay met all quality requirements in the hierarchy, including [IIA] BBO 2.0%, the highest quality criterion. This level of trueness for CRM 111‐01‐01A would be maintained up to a negative or positive shift of 0.03 mmol/l from the current state. For CRM 111‐01‐02A, the level of trueness would be maintained until a negative shift of 0.04 mmol/l or a positive shift of 0.04 mmol/l (Fig. 3 and Table 4).

Figure 3.

LEGO plots for trueness verification of TC assays. (A) CRM 111‐01‐01A, Wako assay. (B) CRM 111‐01‐02A, Wako assay. (C) CRM 111‐01‐01A, Sekisui assay. (D) CRM 111‐01‐02A, Sekisui assay.

Creatinine: The Wako assay met requirement [IIA] EB 2.8%, which is the second quality requirement in the measurement of CRM 111‐01‐01A. If there were any negative or positive shifts, the trueness secured at present would not be maintained. All quality requirements, including [IIA] BBO 1.9%, the highest, were met for the measurement of CRM 111‐01‐02A. This level of trueness would be maintained until a positive shift of up to 3.74 μmol/l from the current state.

All quality requirements, including [IIA] BBO 1.9%, the highest, were met in the measurement of CRM 111‐01‐01A by Sekisui assay. Trueness would be lost with any negative or positive shift from the current state. For the measurement of CRM 111‐01‐02A also, all quality requirements, including [IIA] BBO 1.9%, the highest, were met. This level of trueness would be maintained until the positive shift is 3.74 μmol/l from the current state (Fig. 4 and Table 4).

Figure 4.

LEGO plots for trueness verification of creatinine assays. (A) CRM 111‐01‐01A, Wako assay. (B) CRM 111‐01‐02A, Wako assay. (C) CRM 111‐01‐01A, Sekisui assay. (D) CRM 111‐01‐02A, Sekisui assay.

DISCUSSION

In the present study, we describe the development of the LEGO Plot and applied it to verify the trueness of assays for glucose, TC, and creatinine. It was possible to verify the trueness of quantitative laboratory tests while simultaneously considering multiple quality requirements when judging laboratory significance. Further, the results of trueness verification can be predicted for any shift. Wako and Sekisui assays on the Hitachi Labospect 008 were used as test methods in this study. Their trueness was verified according to the CLSI document EP15.

The trueness of the glucose measurement was statistically verified at only one of two concentrations for both assays. At the concentration where trueness was verified, the assays met the minimum Ba from biological variation database specifications of 3.4% for glucose, the fourth in the quality‐requirement hierarchy. At concentrations for which trueness was not statistically verified, the TEa from European biologic goals of 5.5% for glucose, the sixth requirement, was met. For an assay to meet all quality requirements, including the highest (the optimum Ba from the biological variation database specification of 1.1% for glucose), a positive shift of 0.13–0.14 mmol/l for the Wako assay and a positive shift of 0.09–0.13 mmol/l for the Sekisui assay were required. The statistically significant bias according to the t‐test was observed only when five highest quality requirements were not fulfilled at 4.56 mmol/l for the Wako assay and 7.27 mmol/l for the Sekisui assay. Although bias was not statistically significant at other concentrations in each assay, the top three quality requirements were not met. The trueness of the TC concentration measured using the Wako and Sekisui assays was statistically verified at two concentrations. For the Wako assay, a negative shift of 0.03–0.08 mmol/l would meet all quality criteria including the optimum Ba from biological variation database specifications of 2.0% for TC. For the Sekisui assay, all quality requirements were met at both concentrations and trueness would be maintained for a negative or positive shift of up to 0.03 mmol/l. The trueness of the creatinine assay was also statistically verified at both concentrations in both assays. For the Wako assay, the second requirement, Ba from the European biologic goals of 2.8% for creatinine and the subsequent ones were met at a lower concentration; all quality requirements, including the highest, optimum Ba from the biological variation database specifications of 1.9% for creatinine, were met at a higher concentration. For the Sekisui assay, all quality requirements were met at both concentrations. In the Wako and Sekisui assays, either a negative or a positive shift at a lower concentration would not maintain the current level of trueness. However, we expected that the secured trueness at a higher concentration would be maintained until a positive shift of 3.74 μmol/l.

Although there was no statistically significant bias observed in TC and creatinine assays from both manufacturers, the highest quality requirement was not fulfilled by an assay. Therefore, we can select the reagent that meets the highest quality level when choosing one among several candidate reagents for laboratory use. In the present study, TC and creatinine levels determined using the Sekisui assay fulfilled the highest quality criterion at two concentrations. In contrast, the Wako assay met only one.

The LEGO plot and its basal analysis apply to two situations encountered in clinical laboratories. First, when an assay does not fulfill a specific quality requirement, we can determine the shift required to secure it from the plot, and this information can be used to calibrate the analytical system. Here, the Wako glucose assay required a +0.13–0.14 mmol/l of shift to meet the highest criterion. When a certain quality requirement is already met, we can determine the magnitude of shift that will allow a certain quality criterion to be met, and this knowledge can be used as an alternative limit for internal quality control. The Sekisui TC assay represents this case, because it tolerated a ±0.03–0.04 mmol/l shift.

Commutable frozen human serum or reference materials consisting of an assigned value can be used for accuracy verification 4, 6. Here we used KRISS CRM 111‐01‐01A and 111‐01‐02A. According to CLSI EP15‐A2, if the verification interval does not contain the assigned value, bias and the total error should be judged as to whether they are acceptable for the laboratory's need or not 6. In the present study, the statistical significance of the bias was tested by calculating t‐statistics 4, and laboratory significance was considered by comparing the 95% CI values of the bias with the quality requirements. Therefore, the two methods are comparable. Although CLSI EP15‐A2 recommends repeating the measurement ten times for an accuracy verification 6, each analyte was measured five times per level because of the limited volume of a bottle of CRM. Considering available resources, this practice can occur frequently in a routine clinical laboratory. Therefore, we included TEa as well as Ba to the quality criteria for determining the laboratory significance of biases in consideration of random error components that can remain within the mean of measurements. The t‐test used here has a lower statistical power to detect a significant bias than when the measurement was repeated more than ten times. This may explain why the quality requirement placed on the top level of the hierarchy was not met for glucose assays (Wako assay for 111‐01‐01A and Sekisui assay for 111‐01‐02A) in which the t‐test indicated statistical insignificance of the bias.

To estimate the uncertainty corresponding to a virtual certified value in a simulation of shift, a simple linear regression from the certified values and their uncertainties of CRM was used, because KRISS CRMs comprise two concentrations. However, this was done solely because of the limited number of concentration levels of the CRM. If the number of concentration levels with a certified value and the uncertainty is more than 3, the relationship between them may not be presented by a simple linear regression.

The approach of applying two or more quality specifications to the data for the evaluation of laboratory tests can be found in published studies. For example, Dhatt et al. 8 sequentially compared imprecision obtained by measuring commercial quality control material according to two kinds of criteria. They first used a biological variation‐based model located at the upper section of the quality‐specifications hierarchy. Unless it could be met, an accredited EQA program‐based model located at the lower section was applied. Multiple quality requirements were also used in research on EQA where the number of the institute to meet each of the criteria was shown 9, 10, 11.

Although “Strategies to Set Global Quality Specifications in Laboratory Medicine” was published in 1999 12, laboratory tests are not easy to evaluate with regard to the effect on clinical outcomes in a specific clinical setting. For clinical laboratories where “The Stockholm Conference Hierarchy” has not been widely applied, other alternatives have been proposed 12, 13, 14. We consider the LEGO plot developed in the present study as an alternative. It has a vertical dimension showing the level of requirement from the hierarchical structure of the Stockholm consensus that can be achieved by the laboratory test. Further, the plot has a horizontal dimension simulating a variable degree of shift that may occur in the future. Accordingly, trueness verification can be accomplished objectively and systematically without controversy regarding the adopted criteria to be applied while simultaneously considering all available quality requirements for determining laboratory significance. If trueness is already verified, the plot can show how much shift is acceptable for the analytical system to maintain this condition. Otherwise, it can show how far the analytical system deviates from the requirement for bias or total error and how much shift is needed from the current state to secure the trueness that has not been reached. The LEGO plot can display such information under variable quality requirements. However, we believe that other independent studies applying the LEGO plot and the related analysis to various quantitative laboratory tests are necessary to verify its utility.

In conclusion, the results of trueness verification of quantitative laboratory tests can be presented graphically using the newly designed LEGO plot. It was applied for the first time to the trueness verification of two kinds of assays each for glucose, TC, and creatinine using CRMs. The LEGO plot can be used to present the results of trueness verification while simultaneously considering various quality requirements according to their size and position in the hierarchy even for a limited number of replicate measurements. Moreover, it can provide users with information regarding the shift required from the current state to obtain trueness or how large a shift is acceptable to maintain secured trueness.