Evaluation of a new hemoglobin A1c analyzer for point‐of‐care testing

Abstract

Background

There has been an increasing desire for the use of point‐of‐care testing (POCT) by both primary care clinicians and patients. This study aimed to evaluate the performance of a new POCT analyzer for hemoglobin A1c (HbA1c) testing.

Methods

We assessed the accuracy, precision, and linearity of the POCT HbA1c analyzer (A1C EZ 2.0) with the Tosoh G8 Analyzer as comparative instrument, following the Clinical and Laboratory Standards Institute (CLSI) protocols. We evaluated sensitivity and specificity of the A1C EZ 2.0 in the clinical diagnosis of diabetes among 842 subjects from 79 communities in Beijing, China.

Results

Using regression analysis, the slope of the A1C EZ 2.0 vs the Tosoh G8 Analyzer was 0.9938, with an intercept of 0.0964 and a concordance correlation coefficient of 0.978. For precision, the reproducibility of CV (CV_T) were 3.7% and 2.7% at a lower (36 mmol/mol (5.4%)) and higher (107 mmol/mol (11.9%)) level of HbA1c respectively. The area under the receiver operating characteristic (ROC) curve for clinical diagnosis of diabetes was 0.911 with the HbA1c cut‐off value of 44 mmol/mol (6.14%). At the HbA1c level of 48 mmol/mol (6.5%), the sensitivity and specificity were76.1% and 86.6%.

Conclusion

The A1C EZ 2.0 has a high accuracy and precision, with a wide range of linearity, compared to a comparative laboratory instrument. It met analytical quality specifications and could be suitable for the clinical management of diabetes mellitus.

Abbreviations:

ADA

American Diabetes Association;

AUC

Area under the curve;

CI

Confidence interval;

CLSI

Clinical and Laboratory Standards Institute;

CV

coefficients of variation;

CVr

Repeatability of CV;

CV_T

Reproducibility of CV;

HbA1c

Hemoglobin A1c;

HPLC

High‐performance liquid chromatography;

IFCC

International Federation of Clinical Chemistry;

NGSP

National Glycohemoglobin Standardization Program;

NICE

National Institute for Health and Care Excellence;

POCT

Point‐of‐care testing;

RCV

Reference change values;

ROC

Receiver operating characteristic;

WHO

World Health Organization

1. Introduction

Diabetes mellitus has emerged as a major chronic disease with a high prevalence and incidence worldwide.1, 2 The 2016 Global Report on Diabetes from the World Health Organization (WHO) reported that the number of people with diabetes has risen from 108 million in 1980 to 422 million in 2014, with the global prevalence of diabetes among adults over 18 years of age increasing from 4.7% in 1980 to 8.5% in 2014.3 In 2012, approximately 1.5 million people died of diabetes, and more than 80% of the deaths occurred in low‐income and middle‐income countries.3 In the United States, the American Diabetes Association (ADA) has estimated that the expenses for diabetes management were $245 billion in 2012, bringing a huge economic burden to society.4 In China, the overall prevalence of diabetes has been estimated to be 11.6% in adults, equaling to 113.9 million Chinese adults.5 Moreover, a large percentage of the diabetic communities occur in remote areas where medical facilities are limited, which brings great inconvenience and challenge to the management of diabetes.

Hemoglobin A1c (HbA1c) is used for the diagnosis of diabetes and the prediction of diabetic complications. Clinically, it can reflect the average blood glucose concentration over approximately 3 months, and it is not affected by the fasting status or short‐term changes in lifestyle.6 The ADA and the WHO suggest HbA1c ≥48 mmol/mol (≥6.5%) as the standard cut‐off point for the diagnosis of diabetes.7 HbA1c plays an important role in monitoring glycemic control in patients with diabetes.6 The clinical guidelines described by the ADA and the National Institute for Health and Care Excellence (NICE) show that a fluctuation of 6 mmol/mol (0.5%) of HbA1c is related to a significant difference6, 8 in the management of diabetes.

In recent years, there has been an increasing desire for the use of point‐of‐care testing (POCT) among both primary care clinicians and patients.9 POCT HbA1c analyzers have obvious advantages because they are small and portable, require a tiny blood volume, and have a convenient operation and easy storage of the reagent. Approximately 75% of medical institutions in Europe have adopted POCT HbA1c analyzers.10, 11 In China, community medical institutions serve as the healthcare providers for the majority of the population, and diabetic monitoring work is mainly completed by the community medical institutions. Therefore, POCT HbA1c analyzers are particularly desirable for use in community medical institutions in China. However, as noted in previous studies,12, 13, 14, 15, 16 there has been notable variability in the validity of some POCT HbA1c analyzers, which has limited their clinical application.

The purpose of this study was to evaluate the performance of a new POCT HbA1c analyzer (Botangping^® A1C EZ 2.0, hereinafter referred to as A1C EZ 2.0). We compared the HbA1c results measured by the A1C EZ 2.0 with the results measured by the Tosoh G8 HPLC Analyzer (a traceable to International Federation of Clinical Chemistry (IFCC) reference method instrument) using EDTA‐anticoagulated fresh venous whole‐blood samples. We evaluated the accuracy, precision, linearity, diagnostic sensitivity, and diagnostic specificity. In addition, this study also compared the HbA1c levels measured by the A1C EZ 2.0 using different types of samples (capillary blood vs venous blood) and different reagent lots.

2. Materials and Methods

This study protocol was approved by the medical ethics committee of our institution.

The operator was a technologist located in the clinical laboratory.

2.1. Evaluation of the POCT HbA1c analyzer against the comparative instrument traceable to the IFCC reference method

The POCT HbA1c analyzer A1C EZ 2.0 (BioHermes, Wuxi, China) is based on boric acid affinity chromatography. In this experiment, the reagent lot numbers were AMC15121501 (code501), AMC15121502 (code502), and AP16112803 (code652). The quality control products used were made by Canterbury Scientific Ltd. (lot No. 0105P; Christchurch, New Zealand). The manufacture's operating procedure was strictly followed.

The comparative instrument was the Tosoh G8 HPLC Analyzer (Tosoh Corporation, Tokyo, Japan), which uses ion‐exchange high‐performance liquid chromatography (HPLC) for automatic HbA1c analysis. Key supplies and reagents were used, including chromatographic column (batch No. YJ3908E), quality control materials (batch No. AB1040), the secondary commutable certified reference materials (CRM) with two levels. The target±uncertainty value for CRM level 1 (batch No. GBW (E) 090634) was 34.43±1.54 mmol/mol and that for level 2 (batch No. GBW (E) 090637) was 86.79±3.87 mmol/mol. The performance, methods, reagent, and quality control of the analyzer were in line with the requirements of the National Glycohemoglobin Standardization Program (NGSP).

The accuracy of A1C EZ 2.0 analyzer was evaluated following the guidelines described by the CLSI EP09‐A3.17 We used 40 EDTA‐anticoagulated venous whole‐blood specimens to cover the measurement range of the A1C EZ 2.0: 10 specimens with an HbA1c level between 20 and 42 mmol/mol (4.0%‐6.0%), 10 specimens between 42 and 63 mmol/mol (6.0%‐8.0%), 10 specimens between 64 and 86 mmol/mol (8.0%‐10.0%); and 10 specimens >86 mmol/mol (>10.0%). Samples were from subjects without the following diseases: anemia, hemolytic disease, red cell abnormalities, nephropathy, and jaundice.18 Each specimen was measured twice. If the absolute difference of the repeated measurements exceeded four times of the average absolute standard deviation, the set of data responsible for the outlier values was rejected. All specimens were tested in strict accordance with the operating procedures of the NGSP. The specimens were stored at 2‐8°C, and the testing was completed within 48 hours after blood collection.19, 20

Precision of the A1C EZ 2.0 analyzer was evaluated according to CLSI EP05‐A3.21 We used quality control samples (Canterbury Scientific Ltd.) for two levels of HbA1c, including a low concentration of 36 mmol/mol (5.4%) and a high concentration of 107 mmol/mol (11.9%). Each quality control product was divided into two batches, with each batch being tested once in the morning and once in the afternoon at an interval of 2 hours. We repeatedly ran this procedure each day for 10 consecutive days.14 Specimen retrieval, preservation, and processing were performed following the manufacturer's instructions.

We calculated the coefficients of variation (CV) and reference change values (RCV) for each of the 40 samples, which were tested twice, using EP Evaluator software (Data Innovations LLC., Burlington, VT, USA). The calculation was performed using the formulas in (1) and (2):

$${\text{CV}}_{\mathrm{a}}=\frac{\sqrt{\frac{\sum {(\mathrm{\Delta })}^2}{\mathrm{n}}}}{\overline{X}\sqrt{2}}\times 100,$$ (1)

where CV_a is analysis CV, Δ is the difference between duplicates, n is the number of duplicates, and $\overline{X}$ is the mean of the duplicates.

$$\text{RCV}(\%)=\sqrt{2}\times 1.96\times \sqrt{\left[{({\text{CV}}_{\mathrm{a}})}^2+{({\text{CV}}_{\mathrm{wp}})}^2\right]},$$ (2)

where CV_a is analysis CV, and CV_wp is individual or biological CV. In a recent study that assessed the biological variation of HbA1c in 21 presumably healthy hospital employees, the CV_wp was 0.8%.22

Linearity was assessed following the CLSI EP06‐A guidelines.23 By mixing a high‐concentration (124 mmol/mol, 13.5%) and a low‐concentration (19 mmol/mol, 3.9%) HbA1c sample, we generated 11 samples with different concentrations (19, 30, 40, 50, 61, 71, 82, 92, 103, 113, and 124 mmol/mol). We also evaluated the difference between two lots of reagent. We simultaneously measured 40 EDTA‐anticoagulated fresh blood specimens with reagents with two different lot numbers, and then the two‐tailed paired t‐test was used to analyze the results. Reagent preservation and operation were performed in accordance with the NGSP procedures.

2.2. Sensitivity and specificity of the POCT HbA1c analyzer for diabetes diagnosis

We assessed the sensitivity and specificity of the POCT HbA1c analyzer for diabetes diagnosis in a case–control study. The subjects, including 230 diabetic patients and 230 controls, were recruited from adults (≥18 years old) who underwent routine physical examination in the same hospital in Beijing, China. The male to female ratio was 1:1.

The sample size was calculated using the statistical software PASS 2008 (NCSS, Kaysville, UT, USA). Based on our preliminary data and literature findings, we estimated that at least 198 subjects were needed in each group, given that α was 0.05 and the one‐sided test 1‐β was 0.9. After considering a 15% nonresponse rate and other factors, the required sample size was 228 subjects in each group. Therefore, we had sufficient statistical power with 230 cases and 230 controls in this study.

The questionnaire information included gender, age, height, body weight, clinical diagnosis, medication use, and other laboratory testing analytes. EDTA‐anticoagulated venous blood specimens were collected, and the specimens were transported and stored at 2‐8°C. The testing was finished within 48 hours after blood collection.

Diabetes mellitus was defined according to the ADA diagnostic criteria for diabetes mellitus.24 The prediabetes group was defined as an HbA1c level of 39‐46 mmol/mol (5.7%‐6.4%) 25 as determined by the Tosoh G8 HPLC Analyzer in a central laboratory. The control group consisted of healthy subjects without diabetes or other major diseases as reported through a questionnaire survey or identified through other relevant laboratory results.

In order to compare the influence of different types of blood samples (ie, venous blood vs capillary blood) on the HbA1c results measured by the A1C EZ 2.0, we randomly selected 30 diabetes cases with HbA1c levels of 20‐42 mmol/mol (4.0%‐6.0%), 30 cases with HbA1c levels of 42‐64 mmol/mol (6.0%‐8.0%), and 12 cases with HbA1c levels of 64‐86 mmol/mol (8.0%‐10.0%) from the diabetes patients described above.

2.3. Statistical analysis

Correlation between the variables was visualized using a scatter diagram made with Microsoft^® Excel 2010 (Microsoft Corporation, Redmond, WA, USA) software. Data analysis, including the normality test and the two‐tailed t‐test for between‐group comparison, the receiver operating characteristic (ROC) curve for sensitivity and specificity, and the κ test for consistency analysis of methods, were performed using SPSS 21.0 Statistics (IBM, Armonk, NY, USA).

3. Results

We observed a high accuracy and precision as well as reasonable linearity comparing the POCT HbA1c analyzer with the comparative Tosoh G8 HPLC Analyzer. For the evaluation of accuracy, the slope of the A1C EZ 2.0 vs the Tosoh G8 HPLC Analyzer readings was 0.9938, with an intercept of 0.0964 and an R ² of 0.9827 by regression analysis. Pearson's correlation coefficient was 0.978. At 48 mmol/mol (6.5%) using the POCT HbA1c analyzer, the relative bias was 0.8% compared to the Tosoh G8 HPLC Analyzer (Figure 1). Using the CLSIEP5‐A3 to evaluate the precision of the results, we found that at a low concentration of 36 mmol/mol (5.4%), the CVr was 3.0%, CV_T was 3.7%, and the standard bias was 2 mmol/mol (0.2%). At a high concentration of 107 mmol/mol (11.9%), the CVr was 1.3%, CV_T was 2.7%, and the standard bias was 3 mmol/mol (0.3%). According to the CLSI EP9‐A3 of duplicate testing for precision evaluation, the analysis CV and RCV were 0.7% and 2.9%, respectively, at 40 mmol/mol (5.8%), and 0.8% and 3.0%, respectively, at 105 mmol/mol (11.8%). In terms of linearity, the A1C EZ 2.0 showed a wide linear range from 19 to 124 mmol/mol (3.9%‐13.5%), and the linear equation was Y=10.616X+8, R ²=0.9964 (Figure 2).

Figure 1.

Correlation (A) and bias (B) diagrams of the HbA1c results measured by the A1C EZ 2.0 and Tosoh G8 HPLC Analyzers by whole blood. The x‐axis indicates the results of the Tosoh G8 HPLC Analyzer in a central laboratory using an IFCC‐certified reference method. The y‐axis indicates the results of the A1C EZ 2.0. The linear equation in (A) is Y=0.9938X+0.0964, R ²=0.9827

Figure 2.

Linear range (A) and bias (B) of HbA1c measured by the A1C EZ 2.0 by whole blood. (A) The x‐axis indicates the dilution gradient 1‐11, and the y‐axis indicates the HbA1c results. The black solid line represents the theoretical value, and the red spot represents the actual measured value. (B) The x‐axis indicates the HbA1c concentration gradient 1‐11, the y‐axis indicates the relative deviation R, R=(theoretical value—testing value)/theoretical value. The dotted line represents the target value ±6% (NGSP and CAP standard) threshold. The red spot represents the relative deviation of the testing value

We also evaluated the diagnostic sensitivity and specificity of the analyzers in this study. We collected data and samples from 842 subjects undergoing physical examination in communities from six districts in Beijing: 154 subjects from Dongcheng District, 167 subjects from Xicheng District, 165 subjects from Chaoyang District, 144 subjects from Haidian District, 135 subjects from Daxing District, and 77 subjects from Tongzhou District. There were 449 men and 393 women, and the mean age was 55.1 years old in men and 51.5 years old in women. The A1C EZ 2.0 showed a high diagnostic potential. As shown in Figure 3, the area under the curve (AUC) was 0.911 (95% confidence interval (CI), 0.885‐0.937; P<.001). The Youden index was 0.758, and the corresponding cut‐off point was 44 mmol/mol (6.14%). When the HbA1c was at 44 mmol/mol (6.14%), the sensitivity of this method in the clinical diagnosis of diabetes was 94.7%, and the specificity was 81.1%. The sensitivity, specificity, positive predictive value, and negative predictive value of each clinical decision value are shown in Table 1.

Figure 3.

ROC curve of the A1C EZ 2.0 analyzer for the diagnosis of diabetes. The area value under the curve (AUC) was 0.911 (95% confidence interval, 0.885‐0.937, P<.001). The Youden index was 0.758, and the corresponding diagnosis cut‐off point was 43.61 mmol/mol (6.14%). At an HbA1c level of 43.61 mmol/mol (6.14%), the sensitivity was 94.7%, and the specificity was 81.1% for the clinical diagnosis of diabetes

Table 1.

Sensitivity, specificity, and positive and negative predictive values for the identification of individuals with varying HbA1c key points

Key points (mmol/mol, %)	Sensitivity (%)	Specificity (%)	PPV (%)	NPV (%)
38.80 (5.7%)	98.2	69.7	76.4	97.5
42.08 (6.0%)	96.5	77.4	80.2	95.7
44.26 (6.2%)	91.2	83.9	85.0	90.5
45.36 (6.3%)	87.6	84.9	85.3	87.3
47.54 (6.5%)	76.1	86.6	85.0	78.4
49.73 (6.7%)	70.8	89.3	86.9	77.3
53.01 (7%)	60.2	90.8	86.7	69.5
58.47 l (7.5%)	49.6	93.1	87.8	64.9
63.93 (8%)	33.6	94.5	86.0	58.7
69.40 (8.5%)	29.2	96	88.0	57.6

NPV, Negative predictive value; PPV, Positive predictive value.

We further divided the 842 subjects in this study into five groups (A‐E) according to the diabetes status and the testing results of the Tosoh G8 HPLC Analyzer by the central laboratory. We performed the κ test and ROC curve analyses to determine the consistency of the test results of the A1C EZ 2.0 in comparison with readings from the Tosoh G8 HPLC Analyzer. The AUCs and 95% CIs across the groups were 0.023 (0.014‐0.031) in group A consisting of healthy subjects, 0.682 (0.647‐0.716) in group B consisting of prediabetes subjects, 0.793 (0.764‐0.821) in group C comprising diabetes patients with an HbA1c level of 48‐53 mmol/mol (6.5%‐7.0%), 0.858 (95% CI 0.834‐0.882) in group D comprising diabetes subjects with an HbA1c level of 54‐58 mmol/mol (7.1%‐7.5%), and 0.996 (95% CI 0.990‐1.000) in group E comprising diabetes patients with an HbA1c level ≥60 mmol/mol (7.6%; Figure 4). The κ test value of the five groups was 0.766 (P<.005).

Figure 4.

Receiver operating characteristic curve of the A1C EZ 2.0 in subjects with varying diabetes status and HbA1c levels. (A) ROC curve for healthy person group, AUC=0.023 (95% CI=0.014‐0.031); (B) ROC curve for Pre‐diabetic populations group, AUC=0.682 (95% CI=0.647‐0.716); (C) ROC curve for diabetes population group which results between 47.54 and 53.01 mmol/mol, AUC=0.793 (95% CI=0.764‐0821); (D) ROC curve for diabetes patients with an HbA1c level of 54.10‐58.47 mmol/mol, AUC=0.858 (95% CI=0.834‐0.882); (E) diabetes patients with an HbA1c level ≥59.56 mmol/mol, AUC=0.996 (95% CI=0.990‐1.000)

In the analysis of batch effects with 40 samples repeatedly measured by reagents of two batches (AMC15121501 and AMC15121502), there was no significant difference in the HbA1c results using the two different lots (P=.066). In addition, among 72 diabetes cases with both EDTA‐anticoagulated venous blood specimens and capillary blood specimens simultaneously collected in this study, there was no significant difference (P=.075) in the HbA1c results between the two types of blood samples.

4. Discussion

In this study, we evaluated the performance of a POCT HbA1c analyzer (ie, A1C EZ 2.0) in comparison with the Tosoh G8 HPLC Analyzer traceable to the IFCC reference method, according to CLSI protocols. Our findings indicate that the A1C EZ 2.0 has a high accuracy and precision with a wide linear range for HbA1c testing, which met or exceeded the quality requirements according to the CLSI. It also has a reasonably high discriminative value for the diagnosis of diabetes.

HbA1c has been widely used as an indicator to reflect the average concentration of blood glucose and to monitor glycemic control in the clinical management of diabetes. In clinical application, the quality of the HbA1c instrument may directly affect diabetes diagnosis and treatment in patients.26 Previous studies have shown that the performance of some, but not all,12, 13, 14, 15, 16 POCT HbA1c analyzers is similar to the performances of central laboratory HbA1c instruments.10

In our study, the A1C EZ 2.0 analyzer used is a new POCT instrument that adopts boronic acid affinity chromatographic technology. In the evaluation of testing performance, the mean deviation of the A1C EZ 2.0 vs the Tosoh G8 HPLC Analyzer was 4.5%, which is in accordance with the NGSP standard (<6%) and the College of American Pathologists (CAP) standard.27 The precision of the A1C EZ 2.0 as evaluated according to EP 05‐A3 and EP 09‐A3 in our study was also in line with the NGSP and CAP standards at higher concentration (<3%). For lower concentration, the total precision surpassed the precision standard, the effect from operating staff could not be omitted. In this study, the linearity of the analyzer and reagents with different lot numbers was evaluated, and the quality requirements were met.

We also found that the A1C EZ 2.0 had an excellent performance on the diagnosis of diabetes, especially at 44 mmol/mol (6.14%). The sensitivity of the clinical diagnosis of diabetes was 94.7%, and the specificity was 81.1%; these results are close to the HbA1c‐related data reported in a recent large‐scale epidemiological survey in China.28 This finding is very important because there is evidence showing that a lower cut‐off point of HbA1c, for example, 45 mmol/mol (6.3%)28 or 43 mmol/mol HbA1c (6.1%),29 as compared to the standard cut‐off point of 48 mmol/mol (6.5%), is more suitable for the diagnosis of diabetes in the Chinese population. This study also evaluated the accuracy of important indices of clinical diagnosis, including sensitivity, specificity, positive predictive value, and negative predictive value. Our results indicated that the performance of the A1C EZ 2.0 analyzer met the quality requirements at the key cut‐off points of clinical diabetes diagnosis.

According to the ROC curve analysis of five groups ranging from healthy subjects to diabetes patients with high HbA1c levels, the AUC had a tendency to become more and more close to 1 as the measured HbA1c results increased from low to high. At HbA1c levels >39 mmol/mol (5.7%), the discriminative value of the A1C EZ 2.0 compared to the Tosoh G8 HPLC Analyzer was significantly higher. In addition, the κ test of the five groups of data also showed a high consistency between the two methods. These results indicated that the A1C EZ 2.0 analyzer could be suitable for routine diabetes management.

The A1C EZ 2.0 manufacturer recommends using capillary blood for testing nearby patients. For the convenience of regular comparison of the HbA1c results in the same medical institution, we compared the HbA1c levels measured by the A1C EZ 2.0 in venous blood with capillary blood. We found that the two blood collection methods had no significant differences. These results were consistent with previous findings that the HbA1c levels in capillary blood and venous blood samples are highly correlated.30, 31 It is also noteworthy that the A1C EZ 2.0 is equipped with a smartphone application, which provides convenience for physicians to save the results and give feedback to the patients.

There were two limitations in this study. Firstly, when the precision was evaluated based on EP 05‐A3, 10 days precision data were used instead of that of 20 days, mainly in the consideration of the stability of quality controls. Secondly, the cut‐off value established distinguishing non‐diabetics from diabetics was a little bit lower than that of large‐scale epidemiological studies, the possible reason was the relative small sample number.

In conclusion, the POCT HbA1c analyzer (A1C EZ 2.0) has a high accuracy and precision with a wide range of linearity, compared to a central laboratory instrument. It met analytical quality specifications and could be suitable in the clinical management of diabetes mellitus.

Zhou R, Wang W, Song Z‐X, Tong Q, Wang Q‐T. Evaluation of a new hemoglobin A1c analyzer for point‐of‐care testing. J Clin Lab Anal. 2018;32:e22172 10.1002/jcla.22172