Analytical performance evaluation of intelligent quality management of blood gas analyzer

Abstract

Objective

This study aimed to compare the application effectiveness and quality control (QC) performance of intelligent quality management for blood gas analysis (BGA) with those of traditional quality management.

Methods

We implemented intelligent quality management by employing the GEM Premier 5000 equipped with Intelligent Quality Management 2 (iQM 2). By collecting external quality assessment (EQA) and internal quality control (IQC) data, we compared the clinical application outcomes and quality control (QC) performance between the intelligent management and traditional management approaches.

Results

The average bias of EQA for pH, partial carbon dioxide pressure (pCO₂), partial oxygen pressure (pO₂), sodium (Na⁺) and calcium (Ca²⁺) decreased compared to pre-management levels; except for pO₂, the average coefficient of variation (CV%) of intelligent QC was lower. The average estimated total error (TE) in the intelligent QC met the specified acceptance criterion. According to the average sigma and the goal index ratio (QGI), both QC modes have issues with accuracy and precision; the probabilities of false rejection (Pfr) of traditional QC and intelligent QC are almost the same; except for pO₂ and Na⁺, the probability of error detection (Ped) of intelligent QC is greater, whereas the average detection time (ADT) of traditional QC is greater. In addition, intelligent QC identified errors in approximately 1.46 % of the samples.

Conclusions

The precision and accuracy of the BGA improved significantly compared to those before management, indicating significant advantages of intelligent quality management in quality management applications.

Highlights

• •The quality control performance of blood gas analyzer in intelligent quality control mode is better to traditional quality control mode.
• •GEM Premier 5000 should pay attention to the accuracy of pO₂ and the precision of NA⁺ in the intelligent quality control mode.
• •GEM Premier 5000 could detect errors in 1.46 % of patient samples during the analysis process.

1. Introduction

Blood gas analysis (BGA) is mainly used in clinical practice to determine the body's acid-base balance and degree of hypoxia, providing strong evidence for clinical diagnosis, treatment, and prognosis [1,2]. BGA is widely used in clinical practice due to its rapid detection, but the accuracy of its results is often affected by the neglect of quality management. Therefore, clinical laboratories should focus on the quality of BGA [[3], [4], [5], [6]]. In fact, most of the users of blood gas analyzer are not laboratory personnel, and their relevant knowledge and quality management awareness are quite different. Mario Plebani also proposed that the ability of operators is an important factor affecting the quality of BGA [7,8]. The quality of BGA in hospitals is affected by various brands and different operating standards of blood gas analyzers, so effective quality supervision is very important to ensure the accuracy of results [9,10].

The traditional quality control (QC) program is based on the Clinical and Laboratory Standards Institute (CLSI) guidelines [11], which include measuring and verifying the QC results. The traditional QC program can be performed every 24 h, which can detect errors during QC analysis but may miss errors between two QC tests [12]. Moreover, the traditional QC mode involves only process control at time points and could not achieve error monitoring throughout the entire analysis process of BGA [13,14]. To monitor errors in the analysis process, the intelligent QC system is optimized in the traditional QC mode. The Intelligent Quality Management 2 (iQM 2) of GEM Premier 5000 can monitor the instrument in real-time using five process control solutions (PCS): A, B, C, D, and E. It compares the differences between the detection results of PCS and the target values with the control limits. When a specific analytical parameter exceeds the established control limit, iQM2 uses specific quality checks including mechanical, electronic, and fluidic checks, temperature checks, and sensor/CO-Ox checks, along with pattern recognition to identify errors and interferences. Then iQM2 automatically applies the most appropriate corrective actions, such as performing flushing clot circulation, increasing PCS B frequency, performing wavelength and accuracy checks and so on [15].

In this study, intelligent quality management was implemented for blood gas analyzers in the hospital. According to the QC practice guidelines provided by CLSI and International Organization for Standardization (ISO) [16,17], the QC performance and error detection ability of traditional QC system and intelligent QC system were compared to evaluate the impact of intelligent quality management on BGA quality.

2. Materials and methods

2.1. Research object

This study included the external quality assessment (EQA) results of the BGA in various clinical departments of Southwest Hospital of Army Medical University from 2022–2023, as well as the QC performance data of ABL90 FLEX (Radiometer, Bronshoj, Denmark), i-STAT (Abbot Point of Care Inc., Princeton, NJ, USA), GEM Premier 3000 (Instrumentation Laboratory, Bedford, MA, USA), and GEM Premier 5000 (Instrumentation Laboratory, Bedford, MA, USA), which are commonly used in the clinic, including nonpatient-related information such as QC data and analysis package data. The evaluation parameters included pH, pCO₂, pO₂, K⁺, Na⁺, CL⁻ and Ca²⁺.

2.2. Quality management process for the BGA

The hospital uniformly uses a GEM Premier 5000 blood gas analyzer and sets up a remote monitoring system for centralized real-time monitoring of the instrument. The instrument automatically conducts quality verification, error detection and correction through Intelligent Quality Management 2 (iQM 2), and the real-time data are uploaded to the HIS and LIS system of the hospital for long-term preservation and quick checking of the results. Special personnel analyze and review the internal quality control (IQC) of BGA in the whole hospital, and the clinical department attends the EQA on time. At the same time, the operation process of the BGA is standardized, and regular theoretical and operational training and assessment are carried out for the blood gas analyzer operators and report audit personnel of the whole hospital. Only after passing the assessment can they work on the post, and the hospital grants the LIS account inspection and audit authority.

2.3. Statistics of bias in EQA and assessment scores

We collected the EQA results and corresponding target values of the BGA at the National Center for Clinical Laboratories for 2022 and 2023 in various clinical departments. The bias of each evaluation item was calculated according to the formula Bias=(Test value-Target value)/Target value × 100 %. The theoretical assessment scores of BGA operators in various clinical departments were collected for 2022 and 2023, and their average scores were calculated for comparison.

2.4. Calculation of CV% and TEa for the IQC

We collected the cumulative IQC of various evaluation items in clinical departments to obtain the average coefficient of variation (CV%) from the ratio of the standard deviation (SD) to the mean ($\bar{x}$). The estimated total error (TE) was calculated according to the formula TE = Bias + 2 × SD, and then the estimated TE was compared to ≤½ the allowable total error (TEa) [18]. The TEa values used in this study were derived from the Clinical Laboratory Improvement Amendments of 1988 (CLIA'88) in the United States, and the TEa standards for each parameter are shown in Table 1, which were calculated where appropriate both in absolute value (analyte measuring unit) and in percentage value, choosing the greater between these two results.

Table 1.

Allowable total error for each analyte of BGA.

Analyte	TEa
pH	0.04
pCO2	5 mmHg or 8 %whichever is greater
pO2	9 mmHg or 10 %whichever is greater
Na+	4 mmol/L
K+	0.5 mmol/L or 7 %whichever is greater
Ca2+	0.1 mmol/L or 10 %whichever is greater

2.5. Calculation of sigma and QGI

Sigma (σ) is an important parameter for evaluating the analysis performance of the analysis system. The sigma value is obtained according to the ratio of TEa to SD [19,20]. On the basis of the sigma level, the performance of the analytes is divided into the following six categories: world‐class (σ > 6), excellent (5 ≤ σ < 6), good (4 ≤ σ < 5), marginal (3 ≤ σ < 4), poor (2 ≤ σ < 3), and unacceptable (σ < 2) [21]. The goal index ratio (QGI) represents the relative extent to which both bias and precision meet their respective quality goals. The QGI ratios of the analytes with initial sigma values < 6 were calculated on the basis of the formula QGI = bias/(1.5 × CV%). A value of QGI <0.8 indicates that the precision needs to be improved, a value of QGI >1.2 indicates that the accuracy needs to be improved, and values of 0.8–1.2 indicate both imprecision and inaccuracy [22].

2.6. Calculation of Pfr, Ped and ADT

To compare the performance of the QC, these parameters were calculated via the following formula.

• (1)Control limit calculation (DL) = QC limit for each analyte/SD (or CV%).
• (2)The probability of false rejection (Pfr) = 1 - Cumulative normal standard distribution of DL.
• (3)The probability of error detection (Ped) = cumulative normal standard distribution (z = sigma - DL - 1.65).
• (4)Overall Pfr or Ped = (P_L1∗N_L1 + P_L2N_L2 + … P_LiN_Li)/(N_L1 + N_L2 + … N_Li), where P is the Pfr or Ped per PCS or QC level and N is the number of replicates.
• (5)The overall average run length for rejectable quality (ARL) = 1/Overall Ped.
• (6)Overall, the average detection time (ADT) = ARL × sampling time (or how often the QC material is run in the system).

Pfr, Ped, and ADT for each level of PCS or QC were calculated as described in Westgard et al. [19].

2.7. Evaluation of the ability to detect errors

This study collected and statistically analyzed all sample error types and corrective measures detected during the use of the GEM Premier 5000 analyzer reagent kit in all clinical departments of the hospital in 2023 and calculated the error detection rate.

2.8. Statistics

The data were analyzed via Microsoft EXCEL 2013 software (Microsoft, Redmond, DC, USA) and GraphPad Prism V.10.1.1 software (GraphPad Software, La Jolla, CA, USA). A t-test was used, and a p value < 0.05 was considered to indicate statistical significance.

3. Result

3.1. Research on the BGA before intelligent management

Before management, a survey was conducted on the implementation of blood gas analyzers and their participation in EQA in the hospital. In 2021, a total of 12 clinical departments in the hospital conducted BGA testing, with three different brands of instruments, including ABL90 FLEX, accounting for 38.5 %, GEM Premier 3000, accounting for 53.8 %, and i-STAT, accounting for 7.7 %. To standardize the management of BGA in the entire hospital and unify the brand of BGA instruments, by 2024, 95.5 % are GEM Premier 5000, and only 4.5 % are ABL90 FLEX (Fig. 1A). The number of clinical laboratories participating in EQA for BGA in our hospital has gradually increased from 2021 to 2024. However, only 5 laboratories participated in EQA CL⁻ in 2021, and only 6 laboratories participated in 2022, both of which did not exceed half of the number of laboratories participating in EQA testing that year (Fig. 1B).

Fig. 1.

Survey of blood gas analyzers in the hospital before intelligent management. (A), Percentages of different brands of blood gas analyzers in hospitals from 2021 to 2024. (B) The number of laboratories participating in the EQA for the BGA from 2021 to 2024.

3.2. Comparison of EQA bias and assessment scores

According to the deviation results between the EQA test results and target values in 2022 and 2023, there was no statistically significant difference in bias between pH, pCO₂, pO₂, K⁺, Na^+, Ca²⁺ and CL⁻ in 2022 and 2023 (P > 0.05). However, the average bias of pH, pCO₂, pO₂, Na⁺ and Ca²⁺ in 2023 was lower than that in 2022, and the average bias of CL⁻ in 2023 was higher than that in 2022. The average bias before and after K⁺ management was almost the same (Fig. 2A–G). We also compiled the assessment scores of all trainees in 2022 and 2023, and the results revealed that the average score in 2023 was significantly higher than that in 2022 (Fig. 2H).

Fig. 2.

Comparison of EQA bias and assessment scores before and after intelligent quality management. (A–G), Bias plot between the EQA results and target values for the BGA test items, including pH, pCO₂, pO₂, K⁺, Na+, Ca²⁺ and CL⁻, in all clinical departments from 2022 to 2023. (H), Comparison of blood gas analyzer operators' and reviewers' assessment scores in 2022 and 2023. ∗∗, p value < 0.01.

3.3. QC performance of traditional QC and intelligent QC

The IQC data of the intelligent QC mode and traditional QC mode were summarized to evaluate the QC performance of the two modes. The detailed analysis results were shown in Table 2, Table 3. CL⁻ was not included in the QC performance analysis because this item was not included in the traditional QC mode. Except for pO₂, the average CV% of the intelligent QC system was lower than that of the traditional QC mode. The average TE in both QC modes was within the TEa range. Except for the average TE values of pH and Ca²⁺, which were the same in both traditional and intelligent QC modes, the average TE values of the other four evaluation items were lower in intelligent QC than in traditional QC. Only the average estimated TE of the six evaluation items in the intelligent QC system passed the specified acceptance criterion (≤½ TEa).

Table 2.

Summary of the IQC data at low, medium and high levels in traditional QC mode.

		pH	pCO2 (mmHg)	pO2 (mmHg)	Na+ (mmol/L)	K+ (mmol/L)	Ca2+ (mmol/L)
Low	Mean	7.13	18.00	74.10	117.58	2.73	0.65
	SD	0.01	0.46	2.70	0.86	0.03	0.01
	CV%	0.12	2.40	3.64	1.06	1.30	1.82
	Estimated TE	0.02	3.42	5.41	3.37	0.17	0.06
Medium	Mean	7.44	36.73	105.75	139.60	3.87	1.05
	SD	0.01	0.83	2.82	0.79	0.03	0.02
	CV%	0.07	2.25	2.67	0.57	0.88	2.15
	Estimated TE	0.01	1.69	5.66	1.59	0.50	0.05
High	Mean	7.64	70.09	162.67	158.60	5.81	1.40
	SD	0.01	1.68	3.37	1.68	0.08	0.03
	CV%	0.11	2.53	2.07	0.73	1.00	2.08
	Estimated TE	0.02	0.96	6.76	1.73	0.06	0.04
Average CV (%)		0.10	2.39	2.79	0.79	1.06	2.02
Average Estimated TE		0.02	2.02	5.94	2.23	0.24	0.05
½ TEa		0.02	2.50	4.50	2.00	0.25	0.05
Average Sigma		5.80	6.70	3.10	4.00	13.20	5.20
QGI		0.69	–	0.55	0.46	–	0.32
Overall Pf'r		0.00	0.00	0.00	0.00	0.00	0.00
Overall Ped		0.74	0.90	0.98	1.00	0.88	0.80
Overall ADT (Hour)		34.00	27.00	24.00	24.00	29.00	26.00

Table 3.

Summary of the IQC data of the PCS in intelligent QC mode.

		pH	pCO2 (mmHg)	pO2 (mmHg)	Na+ (mmol/L)	K+ (mmol/L)	Ca2+ (mmol/L)
PCS A	Mean	6.90	64.21	110.54	106.07	7.14	1.84
	SD	0.01	0.91	1.36	0.67	0.05	0.01
	CV%	0.07	1.42	1.23	0.63	0.73	0.87
	Estimated TE	0.01	1.83	2.79	1.35	0.11	0.07
PCS B	Mean	7.41	33.30	179.56	155.05	1.96	0.80
	SD	0.01	0.62	2.48	1.09	0.02	0.01
	CV%	0.08	1.86	1.38	0.70	1.51	1.05
	Estimated TE	0.01	1.25	4.96	2.17	0.09	0.03
PCS C	Mean	8.04	33.50	2.64	–	–	–
	SD	0.01	0.78	1.03	–	–	–
	CV%	0.14	2.32	39.05	–	–	–
	Estimated TE	0.04	1.57	2.18	–	–	–
PCS D	Mean	7.35	24.31	59.15	164.54	7.30	1.21
	SD	0.01	0.34	1.98	0.76	0.03	0.01
	CV%	0.08	1.38	3.35	0.46	0.41	0.83
	Estimated TE	0.01	0.70	4.10	1.53	0.07	0.05
PCS E	Mean	7.21	68.95	97.95	128.10	4.60	0.58
	SD	0.01	1.01	2.78	1.08	0.04	0.01
	CV%	0.09	1.47	2.83	0.84	0.82	1.80
	Estimated TE	0.01	2.06	5.58	2.16	0.10	0.03
Average CV (%)		0.09	1.69	9.57	0.66	0.87	1.14
Average Estimated TE		0.02	1.48	3.92	1.80	0.09	0.05
½ TEa		0.02	2.50	4.50	2.00	0.25	0.05
Average Sigma		6.20	8.00	5.30	4.70	14.20	9.40
QGI		–	–	2.09	0.73	–	–
Overall Pf'r		0.00	0.00	0.00	0.00	0.00	0.00
Overall Ped		0.91	0.92	0.95	0.77	1.00	0.99
Overall ADT (Hour)		17.00	2.00	11.00	8.00	7.00	7.00

We calculated the sigma values at different levels of QC, and the results showed that the average sigma of pH, pCO₂, pO₂, Na⁺, K⁺ and Ca²⁺ in intelligent QC mode were greater than those in traditional QC mode. Moreover, the average sigma of pH, pCO₂, K⁺ and Ca²⁺ in the intelligent QC were all greater than 6, whereas the average sigma of pO₂ and Na⁺ were between 5-6 and 4–5, respectively. In traditional QC mode, only the average sigma values of pCO₂ and K⁺ were greater than 6, the average sigma values of pH and Ca²⁺ were between 5 and 6, and the average sigma values of pO₂ and Na⁺ were between 3 and 4.

To determine the main reason for the low sigma level in the QC performance of the evaluation project, sigma<6 was used as the benchmark for the QGI analysis. The results indicated that pH, pO₂, Na⁺ and Ca²⁺ were inaccurate in traditional QC mode, whereas pO₂ and Na ⁺ had issues with accuracy and precision respectively in intelligent QC mode.

In the QC system, Pfr, Ped and ADT were quantitative indicators that reflect the performance of the QC methods, and we also calculated these three indicators to evaluate the QC methods. The results showed that the Pfr values of traditional QC and intelligent QC were almost the same. The Ped of intelligent QC was lower only for pO₂ and Na ⁺ than that of traditional QC, whereas the ADT of traditional QC was much higher than that of intelligent QC.

3.4. Error detection rate throughout the entire process of automatic QC inspection

Statistics on the types of errors and corrective measures detected during the automatic QC testing of the GEM Premier 5000 in all clinical departments of the hospital in 2023. Table 4 showed that of the 173711 patient samples tested, 2527 erroneous samples were detected, with an error detection rate of 1.46 %. The error types with higher error detection rates were absorbance errors, small clots and interference substances.

Table 4.

Intelligent QC inspection error detection rates throughout the entire process.

Error type	Corrective measures	Number of specimens	Error detection rate (%)
Detect small clots	Perform clot rupture flushing to remove small clots from the sensor	563	0.32
Detect absorbance error	Perform wavelength and accuracy checks	1416	0.82
Detect interfering substances	Increase the frequency of PCS B and remove interfering substances	402	0.23
Analyte of QC out of control	Uninstall the analysis package and replace it with a new one	34	0.02
Detect high turbidity of the sample	–	106	0.06
Temperature exceeds the range	Shut down and restart	6	0.01
Total		2527	1.46

4. Discussion

This study takes a hospital as an example to explore intelligent quality management. Fig. 1 showed that the hospital had a diverse range of blood gas analyzers, which could lead to differences in test results and difficulties in comparison. The number of testing items involved in EQA varies among clinical laboratories, mainly due to differences in instrument brands and a lack of quality management awareness. There are also issues such as scattered testing points, lack of standardized operating procedures, untrained operators, and no unified management. In response to the quality risks faced by BGA, we have made a series of improvements to establish a more robust quality management system.

The calculated average bias of EQA revealed that there was no statistically significant difference in the average bias of the 7 evaluation items between 2022 and 2023, but the average bias of the other 5 evaluation items except K⁺ and CL⁻ in 2023 was lower than that in 2022. The result indicated that the quality management plan implemented may improve the accuracy of the BGA. While the average bias of CL⁻ increased after management, which may be due to fewer clinical departments conducting EQA of CL⁻, and the test data was few and not comparable, so CL⁻ was not included in the subsequent study. James O. Westgard analyzed the error rate of 101339 patient samples on GEM Premier 5000 and found that CL⁻ had the highest error rate of 0.39 %. However, it has not yet been determined whether this is caused by the physical properties of the instrument's electrode membrane or sample characteristics [23]. This may also be the reason for the increase in the average bias of CL^-after management. Regular training of operators was also a key part of the intelligent quality management system. Fig. 2H showed that the average score after management was significantly higher than that before management, indicating that theoretical training can help improve the theoretical knowledge of BGA for operators and may reduce pre-analytical errors.

The results showed that the accuracy of BGA was improved after management, possibly because of the change of QC mode from traditional to intelligent. Therefore, the performance of the two QC modes was evaluated. The average CV% of pH, pCO₂, K⁺, Na⁺ and Ca²⁺ in intelligent QC mode were lower than those in traditional QC. Only the pO₂ CV% of PCS C in intelligent QC was as high as 39.05 %, which is consistent with the research results of Nichols JH [9]. The average estimated TE of intelligent QC mode has passed the specified acceptance criterion (≤½TEa), which was consistent with the research results of Monica Maria Mion [15], and the above results indicated that the accuracy and precision of intelligent QC were better than those of traditional QC. Sigma has been widely used in clinical laboratories to evaluate the quality of the whole process. Higher sigma values indicate fewer analysis errors, fewer suspicious results, and better QC performance [[24], [25], [26]]. The average sigma of traditional QC mode for each detection item was lower than those of intelligent QC. In the traditional QC mode, only the average sigma of pO₂ was less than 4, indicating that its QC performance was very general. This may be due to the air pollution of manually mixed samples, while the intelligent QC mode of GEM Premier 5000 could avoid the problem. To improve the precision and accuracy of the evaluation parameters, the study using QGI showed that the accuracy of pO₂ and precision of Na⁺ were insufficient under intelligent QC mode, indicating that we need to pay attention to the quality of pO₂ and Na⁺ in the future.

In actual QC scheme design work, Ped>90 % and Pfr<5 % are often used as the goals. The results showed that the Ped of intelligent QC failed to meet the design goals for Na⁺ and was lower than that of traditional QC. However, owing to the automatic real-time detection and correction of errors in intelligent QC, increasing the QC detection frequency can balance the lower Ped, which can ensure the accuracy of its results to a certain extent. ADT represents the time when system errors are detected, and the calculated ADT proved the superiority of intelligent QC over traditional QC in terms of error detection time, which can quickly detect errors in the analysis process. Table 4showed that GEM Premier 5000 can detect errors in 1.46 % of patient samples, with the highest error detection rates for absorbance errors, small clots, and interference substances. D'Orazio's research has shown that the accuracy of intelligent QC in identifying small clots and interfering substances can reach 100 % [27], indicating that the intelligent QC of GEM Premier 5000 can accurately identify errors.

Our study highlights that the QC performance of blood gas analyzers under intelligent QC mode is better than traditional QC mode, but it has some limitations. We only evaluated pH, pCO₂, pO₂, K⁺, Na⁺and Ca²⁺ as primary indicators, without analyzing other tests such as hemoglobin, lactate, and glucose. Future research will further expand the evaluation items to fully compare the QC performance of blood gas analyzers under both different QC modes. Additionally, we did not analyze data from traditional manual QC performed on GEM Premier 5000, and more QC data needs to be collected for future validation.

In summary, the comprehensive QC performance of replacing intelligent QC mode was better than that of traditional QC mode and has significant advantages in quality management applications.

CRediT authorship contribution statement

Hongting Tang: Writing – original draft, Methodology, Investigation, Formal analysis, Data curation. Yawen Xiao: Methodology, Investigation, Formal analysis, Data curation. Hong Luo: Supervision, Methodology. Jian Jiang: Supervision, Methodology. Hanqing Xu: Supervision, Methodology. Jun Yang: Supervision, Methodology. Lihua Yang: Writing – review & editing, Supervision, Conceptualization. Xiang Yang: Writing – review & editing, Supervision, Funding acquisition, Conceptualization.

Research Funding

This study was supported by the Red Doctor Talent Training Project of Third Military Medical University (Army Medical University), China.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Data availability

Data will be made available on request.