Evaluation of PIMA Point-of-care CD4 Analyzer in Yunnan, China

Abstract

Background:

CD4 count is used to determine antiretroviral therapy (ART) eligibility. In China, flow cytometers are mostly located in urban areas with limited access by patients residing in remote areas. In an attempt to address this issue, we conducted a study to validate the performance of Alere PIMA point-of-care CD4 analyzer.

Methods:

Venous and finger-prick blood specimens were collected from HIV-positive participants from two voluntary counseling and testing sites in Yunnan Province. Both venous and finger-prick blood specimens were tested with the PIMA analyzer. Venous blood specimens tested with the Becton Dickinson FACSCalibur were used as a reference.

Results:

Venous specimens from 396 and finger-prick specimens from 387 persons were available for analysis. CD4 counts by PIMA correlated well with those from FACSCalibur with an R² of 0.91 for venous blood and 0.81 for finger-prick blood. Compared to FACSCalibur, the PIMA analyzer yielded lower counts with a mean bias of − 47.0 cells/μl (limit of agreement, [LOA]: −204–110 cells/μl) for venous blood and −71.0 cells/μl (LOA: −295–153 cells/μl) for finger-prick blood. For a CD4 threshold of 350 cells/μl, the positive predictive value (PPV) of PIMA was 84.2% and 75.7% and the negative predictive value (NPV) was 97.6% and 95.8% for venous and finger-prick blood, respectively. For an ART threshold of 500 cells/μl, the corresponding PPV was 90.3% and 84.0% and NPV was 94.3% and 93.4%, respectively.

Conclusions:

CD4 counting using venous blood with PIMA analyzers is a feasible alternative to a large flow cytometer to determine ART eligibility.

INTRODUCTION

CD4 count determination plays a critical role in HIV treatment and management decisions. In China, HIV-seropositive people receive semiannual CD4 tests and are eligible for free antiretroviral therapy (ART) if their CD4 count is 350/μl or lower.[1] In 2011, there were an estimated 780,000 HIV-infected people in China, and more than 126,000 of them were on ART.[2] Approximately, 156,000 CD4 tests were performed nationally in 2008,[3] and it was estimated more than 400,000 tests were performed in 2012 (unpublished results). Most conventional CD4 instruments are centrally located in provincial and prefecture-level facilities[3] with adequate infrastructure and skilled technicians. However, more than 70% of HIV-infected persons live in remote rural areas. Blood specimens have to be transported to centralized laboratories in a timely and secure manner in order to maintain CD4 cellular integrity and ensure the public's safety. These problems are more prominent in provinces such as Tibet, Xinjiang, Guangxi, and Yunnan with a large proportion of patients residing in hard-to-reach mountainous areas. A reliable, simple, low-cost, and robust point-of-care (POC) device would help lower these barriers. Recently, POC technologies including PIMA analyzers have improved significantly and have been shown to circumvent challenges in rural and resource-limited countries.[4,5,6,7,8,9,10,11] Theoretically, CD4 counting could be performed using the PIMA analyzer on finger-prick blood specimens with less invasive procedure and a very small blood specimen per test required, thus enabling it to be used where trained phlebotomists are not available or when difficulties are encountered with venous blood sampling. Here, we report the performance of PIMA analyzers at two voluntary counseling and testing (VCT) sites in Yunnan Province and compare the results with the routine CD4 analysis performed in well-equipped laboratories.

METHODS

Study participants and CD4 cell counting

This study used convenience sampling of blood from HIV-infected people aged 6–65 years attending VCT sites at the Yunnan Provincial Center for Disease Control and Prevention (CDC) in Kunming city and the Dehong prefecture CDC in Dehong city between May 2012 and September 2012 as part of routine CD4 monitoring. Participants 18 years or older provided written informed consent, those younger than 18 years had written informed consent provided by their legal guardians. Demographic information collected included gender and birth date.

Three PIMA CD4 analyzers were placed at each VCT site. Finger-prick blood specimens were collected using Sarstedt lancets provided by Alere and analyzed immediately on site using a PIMA analyzer. Venous blood specimens (2 ml) were drawn in a K₃-EDTA Vacutainer tube, and 25 μl blood tested with a PIMA analyzer on site within one hour of blood draw. The remaining venous blood was tested with FACSCalibur (Becton Dickinson, Franklin Lakes, NJ, USA) according to manufacturer's instructions in the adjacent laboratory within 2 h of blood draw. A study participant was assumed to yield 3 CD4 results from venous blood with FACSCalibur, venous blood with PIMA, and finger-prick blood with PIMA. Only the CD4 test result from venous blood tested by FACSCalibur was reported to the participant. To examine the reproducibility of the PIMA analyzer results, venous blood specimens from the first 66 and 51 participants in Kunming and Dehong, respectively, were tested at VCT sites in duplicate.

This study was approved by the Ethical Review Committees of the Chinese National Center for AIDS/STD Control and Prevention and the U.S. CDC.

Quality controls

Voluntary counseling and testing counselors and technicians who used the PIMA analyzers received half-day training from Alere technical experts on finger-prick sampling techniques and proper operation of the PIMA analyzer. A quality test was conducted with 2 PIMA internal quality control cartridges with predefined low and high CD4 counts prior to commencing testing each day, as required by the manufacturer. PIMA analyzers with quality control cartridge results within predefined ranges were used to test specimens from study participants. The cartridge with study participant's blood sample also has additional control features. The analyzer will report an invalid result if there is inappropriateness in cartridge expiry date, sample volume, reagent validation, and instrument function. The project officers and Alere technical experts in Beijing made supervisory site visits during the study. Study staffs in VCT sites were instructed to record mistakes or malfunctioning of PIMA analyzers. Daily calibration and internal quality controls were also performed on the FACSCalibur instrument. The two provincial and prefectural CD4 laboratories have fulfilled the CD4 laboratory establishment and management criteria required by the Chinese HIV laboratory management guidelines.[12]

Statistical analysis

Data were analyzed using SAS 9.1 (SAS Institute Inc., Cary, NC, USA) for correlation coefficients and linear regressions to estimate the correlation between CD4 counts obtained by the PIMA analyzer and FACSCalibur. Bias and limits of agreement (LOA) were analyzed using the Bland-Altman method[13] to determine whether the methods agree sufficiently. Relative bias was expressed as a percentage of the difference between CD4 counts obtained by the two methods divided by the average of the measurements. To determine accuracy of the PIMA analyzer, the mean of the two measurements, mean of bias, and relative bias were obtained for different CD4 ranges (≤200, 201–350, 351–500, and >500 cells/μl). Polynomial contrasts were used to test for trends of relative bias across the whole range. FACSCalibur CD4 results were used as the reference for the calculation of sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) for the PIMA analyzer. The precision of PIMA CD4 results was calculated as the mean percent coefficient of variation (CV) with the first 66 and 51 duplicates at VCT sites. All statistical tests were two-sided at alpha = 0.05.

RESULTS

Between May and September 2012, 462 HIV-1 infected persons receiving routine CD4 testing agreed to participate in this study. The mean age of the participants was 38.0 years (standard deviation 11.2, range: 7–65 years) and 61.9% were male. Four (1.0%) were under 18 years old. One incidence of FACSCalibur instrument failure resulted in the loss of data from 9 participants in Dehong. In addition, 57 participants’ finger-prick PIMA detection yielded invalid results, and laboratory staff did not test the corresponding venous blood using the PIMA analyzers in Dehong. These participants were excluded from the analyses for the venous blood measurement (n = 396). An additional nine participants in Kunming had invalid finger-prick blood results. These participants did have venous blood results by both PIMA and FACSCalibur. Specimens from 387 persons had values available for finger-prick blood analysis.

Quality assurance and reproducibility of PIMA

As part of routine quality assurance, PIMA analyzers underwent daily testing using manufacturer-supplied internal quality control cartridges. CD4 counts of the low and high internal quality control cartridges were 151–281 cells/μl and 623–1157 cells/μl, respectively. All PIMA analyzers produced cartridges results within the predefined ranges for the entire study period. The average of the low cartridges was 213 cells/μl (n = 333, range 151–257 cells/μl) with a CV of 12.3%. The average of the high cartridges was 904 cells/μl (n = 297, range 790–1014 cells/μl) with a CV of 5.1%.

In the beginning of the study, 117 venous blood specimens were tested in duplicate on the PIMA analyzers. These included 66 specimens from Kunming and 51 from Dehong VCT sites. The first CD4 result obtained for each participant correlated well with their corresponding second CD4 results. The R² values of the two measurements were 0.93 for Kunming and 0.96 for Dehong. No statistical difference was observed in bias of the duplicate measurements at the two VCT sites. Regression analysis on all specimens showed a correlation with a R² = 0.94 and y = 0.97x + 10.11 (data not shown). Bland-Altman analysis showed a small overall mean bias of 1.4 cells/μl (LOA: −96.7–97.0 cells/μl) and a CV of 10.4%.

Comparison of CD4 results of venous blood by PIMA and FACSCalibur

There were 396 specimens with venous blood specimens tested by FACSCalibur and the PIMA analyzer. Regression analysis showed results correlated well with a R² of 0.91 (y = 0.83x + 21.89 [Figure 1a]). However, CD4 counts derived from PIMA were lower than those from FACSCalibur (P < 0.001) as shown by the lower slope in Figure 1a and the Bland-Altman analysis plot [Figure 1b]. The overall mean relative bias was − 10.9% [Table 1]. To better examine bias over the entire CD4 cell count range, we divided cell counts into four groups (≤200, 201–350, 351–500, and >500 cells/μl) and found the respective relative bias expanded from −2.9%, −9.9%, −14.0%, to −15.3%. The relative bias was significantly higher in groups with CD4 counts higher than 200 cells/μl than that in groups with CD4 counts ≤200 cells/μl (P < 0.001) [Table 1].

Figure 1.

The relationship of CD4 counts using venous blood tested by FACSCalibur and PIMA analyzers as revealed by linear regression analysis (a) and Bland-Altman analysis (b) on 396 paired specimens. Horizontal lines indicate mean bias (solid line) and limits of agreement (dashed lines) representing ±1.96 standard deviation (95% confidence interval) of mean bias. Regression is plotted (broken line in b) and equation indicated.

Table 1.

Comparisons of CD4 counts with venous blood specimens determined by PIMA and BD FACSCalibur (cells/μl)

CD4 groups	All	≤200	201–350	351–500	>500
Number	396	83	113	88	112
FACSCalibur venous, mean ± SD	395 ± 258	106 ± 61.7	272 ± 42.6	418 ± 41.8	715 ± 220
Range	3–1954	3–199	200–348	352–499	501–1954
PIMA venous, mean ± SD	348 ± 223	101 ± 61.3	248 ± 53.3	369 ± 76.2	615 ± 196
Range	6–1621	6–312	141–464	137–673	216–1621
Mean bias ± SD	−47.0 ± 80.3	−5.2 ± 40.1	−23.3 ± 41.1	−49.2 ± 61.0	−100 ± 111
Mean relative bias (%) ± SD	−10.9 ± 22.6	−2.9 ± 37.2	−9.9 ± 15.9*	−14.0 ± 17.0*	−15.3 ± 15.4*

*Significantly higher as compared with the group of CD4 ≤200 cells/μl (P<0.001). SD: Standard deviation.

Comparison of CD4 results of finger-prick blood by PIMA and venous blood by FACSCalibur

We evaluated the correlation between CD4 results of finger-prick blood by PIMA analyzers and corresponding venous blood by the referent FACSCalibur. The R² was 0.81 (y = 0.74x + 30.52, Figure 2a), lower than that observed when using venous blood for both methods [Figure 1a]. The PIMA CD4 finger-prick results were lower than the corresponding FACSCalibur venous blood specimens (P < 0.001) with LOA between −295 and 153 cells/μl, an overall bias of −71.0 cells/μl [Figure 2b] and an overall relative bias of −18.6% [Table 2]. When cell counts were again divided into four groups, the relative bias was −9.7%, −14.2%, −20.1%, and −27.7%, respectively. The relative bias was significantly higher in groups with CD4 counts higher than 350 cells/μl compared with that in groups with CD4 counts lower than 350 cells/μl (P < 0.001) [Table 2]. Results obtained with finger-prick blood using the PIMA analyzer exhibited more bias from reference values than results from venous blood using PIMA.

Figure 2.

The relationship of CD4 counts using venous blood determined by FACSCalibur and finger-prick blood determined by PIMA analyzer as revealed by linear regression analysis (a) and Bland-Altman plots (b) on 387 specimens. Horizontal lines indicate mean bias (solid line) and limits of agreement (dash lines) representing ±1.96 standard deviation (95% confidence interval) of mean bias. Regression is plotted (dash line in b) and equation indicated.

Table 2.

Comparisons of absolute CD4 counts using venous blood on FACSCalibur and finger-prick blood on PIMA analyzer (cells/μl)

CD4 group	All	≤200	201–350	351–500	>500
Number	387	80	110	88	109
FACSCalibur venous, mean ± SD	396 ± 257	109 ± 61.3	272 ± 42.9	417 ± 41.3	715 ± 223
Range	3–1954	3–199	200–348	352–499	501–1954
PIMA finger-prick, mean ± SD	325 ± 213	97.0 ± 63.2	240 ± 62.4	354 ± 110	556 ± 213
Range	7–1446	7–311	112–493	20–987	4–1446
Mean bias ± SD	−70.8 ± 115	−11.8 ± 46.7	−32.0 ± 52.5	−63.7 ± 109	−159 ± 146
Mean relative bias (%) ± SD	−18.4 ± 31.3	−9.7 ± 44.6	−14.2 ± 20.0	−20.1 ± 30.0*	−27.7 ± 27.4*

*Significantly higher as compared with the groups of CD4 lower than 350 cells/μl (P<0.001). SD: Standard deviation.

Comparison of CD4 results of finger-prick and venous blood by PIMA

Lastly, we compared the performance of PIMA analyzers using venous and finger-prick blood specimens. This analysis used results obtained from the 387 participants with both types of the blood specimen. The correlation was R² = 0.82 (y = 0.87x + 22.14) (data not shown). Finger-prick blood yielded lower CD4 counts than when using venous blood. A Bland-Altman plot revealed that the overall bias was −22.9 cells/μl (LOA: 213–167 cells/μl) (P < 0.001) (data not shown). The mean total relative bias was −7.5 ± 29.8%, individual bias for the four CD4 groups increased from −2.6% to −13.6% (P < 0.05).

Positive and negative predictive values of CD4 results from PIMA

In China, the current CD4 threshold used for ART eligibility is 350 cells/μl. Recently, WHO recommended raising the threshold for eligibility to 500 cells/μl.[14] Thus, we examined the impact of using PIMA analyzers on ART initiation with both thresholds [Table 3]. With the threshold of 350 cells/μl, the PPVs using venous blood and finger-prick blood were 84.2% and 75.7%, respectively; and NPVs were 97.6% and 95.8%, respectively. With a threshold of 500 cells/μl, the PPVs were 90.3% and 84.0%, respectively, and the NPVs 94.3 and 93.4%, respectively.

Table 3.

Sensitivity, specificity, PPV, and NPV (95% CI) for venous blood and finger-prick blood using the PIMA CD4 analyzer based on CD4 thresholds of 350 and 500 cells/μl, with FACSCalibur as reference

Specimens	Sensitivity (%)	Specificity (%)	PPV (%)	NPV (%)
350 cells/μl
Venous	98.0 (94.9–99.9)	82.0 (76.0–87.1)	84.2 (78.8–88.7)	97.6 (94.0–99.4)
Finger-prick	96.8 (93.3–98.8)	70.0 (63.1–76.4)	75.7 (69.8–81.0)	95.8 (92.6–99.1)
500 cells/μl
Venous	98.2 (95.9–99.4)	73.2 (64.0–81.1)	90.3 (86.4–93.4)	94.3 (87.1–98.1)
Finger-prick	98.6 (96.4–99.6)	52.3 (42.5–62.0)	84.0 (79.6–87.9)	93.4 (84.1–98.2)

PPV: Positive predictive value; NPV: Negative predictive value; CI: Confidence interval.

DISCUSSION

Yunnan Province has the highest population of HIV-infected persons in China. Dehong Prefecture, in Southwest Yunnan, has a comparatively high HIV prevalence, with a large proportion of infected persons residing in mountainous areas. A POC analyzer, such as the PIMA analyzer evaluated in this study, requires 20 min/test, allowing VCT staff to provide testing results to 10–15 people daily. A simple POC analyzer is a good alternative to the widely used FACSCalibur to facilitate CD4 service in areas of China with low laboratory and human resource capacity.

Here, we demonstrated the PIMA analyzer can provide more reliable CD4 counts using venous blood than finger-prick blood, which is consistent with previous reports.[4,5] A potential reason for the lower CD4 counts could be excessive squeezing of a participant's finger to obtain sufficient blood. Another contributor to the PIMA analyzer's poor performance with finger-prick blood may be due to inadequate filling of the blood intake channel in the PIMA cartridge which would yield an invalid report. Our study yielded an average of 14.3% invalid reports with finger-prick blood, similar to studies previously conducted in South Africa (6.8%)[6] and Thailand (23%).[7] The lancet used for PIMA is designed to have a deeper hypodermic cut than lancets typically used in other programs, and thus the VCT staff might not perform finger-pricking correctly. However, in our study, we did not record these events. In order to improve the success of CD4 measurement using finger-prick blood, it is important to provide intensive and frequent training on the sampling technique to VCT site staff. Since phlebotomy is commonly practiced in China, even in rural VCT sites, the use of venous blood for PIMA CD4 counting was well accepted in this study and is not expected to constitute a barrier to implementation.

The PIMA analyzer consistently yielded CD4 counts lower than the referent FACSCalibur, with relative bias increasing with higher CD4 counts, as previously observed.[5,7,8,9] At higher cell numbers, CD4 cells might aggregate resulting in underestimation by the PIMA analyzer. However, recent reports from India and Zimbabwe did not observe this effect.[10,11] Our data also showed an elevated negative variation at the high end of CD4 [Figures 1 and 2]. Using a threshold of 350 cells/μl, our data on venous blood specimens showed a PPV and NPV of 84.2% and 97.6%, respectively. When the threshold was set at 500 cells/μl, the respective values were 90.3% and 94.3%. PIMA misclassified 10% and 9% of participants using the thresholds of 350 and 500 cells/μl, respectively. Because of the bias toward lower estimation, nearly 90% of PIMA-misclassified participants had lower counts than reference and thus would result in their earlier entry to ART treatment. Although there is a concern over the early ART initiation, which might increase the financial cost, the current international recognition of the benefit of early treatment for the patients and to reduce HIV transmission outweighs this concern. Given the encouraging results of the PIMA analyzer using venous blood, this POC method will be expanded to remote Chinese areas, especially those with ethnic minorities, to improve CD4 services and identify treatment-eligible patients in a timely fashion. However, when performing CD4 counting for patient treatment monitoring, it is recommended not to switch between PIMA device and current FACSCalibur.

This is the first study to evaluate the usefulness and feasibility of using a POC CD4 technology in Yunnan Province, China. The Alere PIMA analyzer provides satisfactory CD4 counting using venous blood, but is less reliable when finger-prick blood is used. However, other factors that may affect the performance of PIMA analyzer remained to be determined. Given the frequent use of phlebotomy in rural areas in China, the PIMA POC CD4 analyzer using venous blood could play an important role in improving HIV care and treatment in resource-limited settings in China.