Field Performance of PIMA Point-of-Care Machine for CD4 Enumeration Under a Mobile HIV Counseling and Testing Program in Remote Fishing Communities of Lake Victoria, Uganda

Abstract

Uganda is among the most HIV/AIDS-afflicted countries, and many HIV-infected persons live in remote areas with poor access to health care. The success of HIV care programs relies in part on patient monitoring using CD4 T cell counts. We conducted an evaluation of the point-of-care PIMA test using BD FACSCount as a gold standard. One hundred fifty-one participants were enrolled, provided venous blood and samples tested at the point of care with the Alere PIMA™ CD4 Analyzer and the BD FACSCount in the UVRI-IAVI main laboratory. Correlation between the methods was assessed, as was the ability of the Pima Analyzer to predict values <200, <350, and ≥500 CD4 cells/mm³ when compared with BD FACSCount as the gold standard. A near-perfect positive Pearson correlation coefficient (r = 0.948; p < .0001) between the two methods was observed. The Alere PIMA Analyzer had a mean bias of −32.5 cells/mm³. The sensitivity and specificity, for PIMA to predict CD4 lymphocyte count less than 200 cells/mm³, were 71.4% and 100%, respectively; less than 350 cells/mm³ were 84.6% and 94.6%, respectively; and at CD4 count less than 500 cells/mm³ were 94.4% and 100%. The Alere Pima Analyzer provides reliable CD4 cell count measurement and is suitable for monitoring and screening eligible HIV patients in hard-to-reach settings.

Introduction

Globally, it is estimated that 36.7 million people are living with HIV, the majority of whom live in sub-Saharan Africa.¹ Despite over three decades of research and deployment of multiple efficacious prevention interventions, the rate of adult new HIV infections registered in the years 2010 through 2016 has largely stagnated at 2.1 million infections per year. The UNAIDS has proposed that by the year 2020, 90% of all people living with HIV will know their HIV status, 90% of all people with diagnosed HIV infection will receive sustained antiretroviral therapy, and that 90% of all people receiving antiretroviral therapy will have suppressed their viral load.² To achieve this goal, much progress will need to be made to reach remote, marginalized, or underserved communities that might otherwise escape attention.

Uganda is among the countries most afflicted with HIV/AIDS, with the national prevalence estimated at 7.1%.³ Embedded in Uganda's mature and generalized HIV epidemic are hotspots of most-at-risk populations for HIV infection, which include fishing communities. The first case of HIV/AIDS in the country was reported in 1982 in Kansensero, a fishing village located on shores of Lake Victoria in the Southwestern part of the country.⁴ More than 30 years later, research still indicates HIV prevalence of 15%–40% and incidence of 6.04/100 pyo among the different Lake Victoria fishing communities being three to five times higher than the national average.^5–9 These unacceptably high HIV rates are partly due to the hard-to-reach nature of these populations, poor health care infrastructure, lack of electricity often needed to run medical equipment, and generally limited focus on the fishing communities by HIV/AIDS programs.

In countries and regions with subpar health care coverage, the importance of mobile, rapid and easy-to-use Point-of-Care (POC) diagnostic technology is critical in HIV/AIDS patient treatment and care services. Until recently CD4 testing as an integral part of HIV/AIDS treatment and care programs relied on flow cytometry machines located in reference laboratories, often away from many of the community-based clinics. POC platforms have been developed to aid remote CD4 testing at clinics where clinical decisions need to be made more rapidly. More so, rural HIV Clinics still experience operational challenges such as antiretroviral treatment (ART) stock outs that may require prioritizing ART to those in urgent need, particularly patients with a reactive Cryptococcal Antigen (CrAg) screening test. In this case, ART eligibility at CD4 T cell counts less than 350 and 500 cells/μL¹⁰ remains relevant. Also, the use of PIMA CD4 Analyzer is an alternative method for rapid measurement of CD4 count in patients with limited access to reference laboratories.¹¹ We, therefore, field tested the PIMA POC diagnostic test platform for CD4+ T cell enumeration and evaluated it against the laboratory-based BD FACSCount machine. The PIMA POC platform is a portable machine that we operated on a generator (HONDA10i) and utilized the manufacturer's reagent cartridges that can be kept at room temperature. All these factors make PIMA less expensive and a more feasible option for use in hard-to-reach settings such as the fishing communities.

Materials and Methods

Study design and settings

Home-based counseling and testing services were integrated into a larger study called Community HIV Epidemiology and Sociobehavioral Study (CHESS) designed to characterize factors associated with risk of HIV infection, measure HIV prevalence and incidence.^12–14 The CHESS was conducted between February 2014 and November 2015 among 2,400 residents of eight island (Kavenyanja, Jaana, Kiimi, Namisoke, Zinga, Kitobo, Makusa, and Myende) and two mainland (Kasenyi and Nakiwogo) fishing communities of Lake Victoria, Uganda. All the island communities were only accessible by combination of murram road vehicle drive to the lakeshore, followed by a boat ride (ranging from 45 min to >4 h). Weather conditions included humidity and some heavy precipitation, further underscoring the importance of evaluating the performance of the PIMA machine. All participants ≥13 years of age who fulfilled enrolment criteria were invited to enroll, provided assent/consent, tested for HIV, and received individual pre and posttest HIV counseling using a standardized protocol by a trained Clinical Research Nurse/Counselor.

HIV testing followed the standard Uganda National HIV rapid testing algorithm; All samples were collected in EDTA (ethylenediaminetetraacetic acid) lavender-top vacutainer tube by first selecting a suitable site for venipuncture on the arm, then the vein identified, cleaned with 70% ethanol, and allow the area to dry before collection of blood. The blood was initially tested using the Alere Determine™ HIV-1/2 (Alere, Inc., Waltham, MA), and if negative, results were reported as negative for HIV infection. Samples with a positive result were next tested using the HIV 1/2 STAT-PAK^® DIPSTICK test (CHEMBIO Diagnostic Systems, Inc., Medford, NY), and if positive too, results were reported as negative for HIV infection. If the results of HIV 1/2 STAT-PAK DIPSTICK test were discordant with those obtained on Alere Determine HIV-1/2 test, then the sample was tested on the Trinity Biotech Uni-Gold™ HIV test (Trinity Biotech, Plc., Ireland) as the tie-breaker test.

Field measurement of CD4 T cell counts on the Alere Pima™ CD4 Analyzer platform

Four milliliters of peripheral whole blood was drawn from each HIV-1-infected study participant into an EDTA lavender-top vacutainer tube. This blood was used for assessment of CD4 T cell counts. Field CD4 T cell count measurement was performed using the Alere Pima CD4 Analyzer (Alere, Inc.) according to the manufacturer's instructions by qualified, trained laboratory technicians with certified technical competence before carrying out the testing. The Alere Pima CD4 Analyzer passed initial instrument validation at the UVRI-IAVI HIV Vaccine program main laboratory in Entebbe, Uganda. The blood was measured using a calibrated single-channel volumetric pipette into the sample collector of the Alere Pima CD4 test cartridge and the test cartridge inserted in the PIMA CD4 Analyzer. After 20 min, a result printout was generated for laboratory review and authorization before being reported to the clinical team and the volunteers. In addition, CD4 results from the Alere Pima CD4 Analyzer were entered into an electronic database manually at the UVRI-IAVI Main Laboratory.

CD4 T cell count measurement by the gold standard (The BD FACSCount™ Flow Cytometer)

As part of quality control, on each work day the field laboratory technologist randomly (using the Microsoft Excel “RAND” function) sampled 20% of samples tested on the Alere Pima CD4 Analyzer in the field, packaged the leftover 4-mL EDTA vacutainer tubes, and shipped them by boat and/or vehicle to the UVRI-IAVI HIV Vaccine Program laboratory in Entebbe for retesting by the BD FACSCount Flow Cytometer (Becton Dickinson, Franklin Lakes, NJ). All testing was performed according to the manufacturer's instructions by a qualified trained laboratory technologist. Fifty microliters of EDTA whole blood was added to each of a pair of CD4/CD3 and CD8/CD3 reagent tubes, vortexed for 5 s and incubated in the dark at room temperature (20°C–25°C) for 60 min. After incubation, 50 μL of the fixative solution (cat. no. 339010) was added to each tube and vortexed. The No-Lyse stained samples were analyzed acquired on the BD FACSCount Flow Cytometer. Result printouts were reviewed and authorized following the laboratory's results reporting procedures before they were entered into the database manually at the UVRI-IAVI main laboratory. Technologists who performed the BD FACSCount CD4 measurements were blinded to the field CD4 count results generated using the Alere PIMA CD4 Analyzer. All blood was tested within the 48-h recommended limit, on average 26 h after collection.

Ethics approvals and considerations

The CHESS was reviewed and approved by the Uganda Virus Research Institute Research Ethics Committee (UVRI-REC), Federal Wide Assurance (FWA) number 00001354, and the Uganda National Council for Science and Technology (UNCST). All participants provided documented informed assent/consent to participate in the study, and to storage of samples for future research. All HIV-positive participants were counseled and linked to HIV care facilities of their choice.

Statistical methods

Descriptive statistics were used to summarize study participants' characteristics. We evaluated the utility of the Alere PIMA CD4 Analyzer method at three strata of immune function: to accurately classify immunological failures (<200 cells/mm³), ART eligible priority as per National Guidelines at the time (<350 CD4 cells/mm³), and nonpriority (≥500 CD4 cells/mm³) HIV patients.

The BD FACSCount CD4 test measurements were compared with the Alere PIMA Analyzer POC method results. Paired data were compared by Pearson correlation coefficients. Analysis of agreement between the two methods was done by the Bland–Altman method. The Bland–Altman method assesses bias as measured by the average difference and calculated 95% limits of agreement of all the paired measurements.¹⁵ Paired t-tests were done to compare differences in mean. Two by two tables were constructed to assess the diagnostic accuracy at CD4 thresholds of <200, <350, and <500 cells/mm³ previously recommended by WHO as thresholds for initiation of ART.¹⁶ We calculated sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). Receiver operating characteristic (ROCs) curves evaluated the sensitivity and specificity of PIMA POC method to predict CD4 cell count <200, <350, and <500 cells/mm³ by FACSCount. We calculated the area under the ROC curve of these two thresholds for PIMA POC method. Statistical analyses were performed using STATA 12.1 software (College Station, TX).

Results

Baseline characteristics of study participants

Participant characteristics are presented in Table 1. We measured 151 samples under field conditions using the Alere PIMA Analyzer and with the BD FACSCount analyzer under laboratory conditions. Participants had an average age of 32.6 years (standard deviation [SD] 7.4) and majority of participants (94 [62.3%]) were female. Mean CD4 T cell count was 570 (SD 308) and 537 (SD 273) by FACSCount and PIMA, respectively (Table 1).

Table 1.

Participant Characteristics

	Male	Female	Total
No. of study participants (%)	57 (37.7)	94 (62.3)	151
Age, mean (SD)	33.4 (7.2)	32.2 (7.5)	32.6 (7.4)
FACSCount, mean CD4 cells/μL (SD)	497 (262)	613 (326)	570 (308)
PIMA, mean CD4 cells/μL (SD)	468 (222)	578 (292)	537 (273)

SD, standard deviation.

PIMA versus FACSCount

In this study, we observed a near-perfect positive correlation between the FACSCount and Alere PIMA platforms in the measurement of CD4 T cell counts (r = 0.948; p < .0001) (Fig. 1). However, a perfect correlation coefficient does not necessarily imply perfect agreement and cannot inform if FACSCount CD4 T cell count measurements systematically underestimate those of the Alere PIMA platform and vice versa.¹⁵ We, therefore, assessed for the degree of agreement between CD4 T cell count measurements of the two methods using Bland–Altman analysis. Overall, we noted that the Alere PIMA Analyzer measurements were significantly lower than those of the BD FACSCount analyzer (p < .0001) and registered a mean bias of −32.5 (−48.5 to 16.4; p = .0001) cells/mm³ (Table 2 and Fig. 2). Because the utility CD4 T cell count measurements become clinically significant as patients' CD4 T cell numbers drop, we analyzed the impact of the observed negative bias on Alere PIMA CD4 T cell measurements at clinically relevant CD4 T cell cutoff strata of <200, <350, and <500 cells/mm³. We found that the significant negative bias observed was largely due to underestimation of CD4 T cell counts by the Alere PIMA analyzer at CD4 levels ≥500 cells/mm³ (p < .0001) when compared with the BD FACSCount analyzer (Table 2).

FIG. 1.

Scatter plots showing correlation analysis of PIMA versus FACSCount results. Pearson correlation coefficient (PIMA vs. FACSCount) = 0.948; p < .0001.

Table 2.

PIMA CD4 Count Results Versus FACSCount

Variable	Total	<200	<350	<500	≥500
	n = 151	n = 10 (6.6%)	n = 33 (21.9%)	n = 67 (44.4%)	n = 84 (55.6%)
CD4 T cell count (PIMA)
Mean (SD)	537 (273)	133 (40)	226 (73)	307 (104)	720 (222)
Median (IQR)		141 (103 to 164)		317 (234 to 384)	660 (553 to 841)
CD4 T cell count (FACSCount)
Mean (SD)	570 (308)	137 (52)	220 (78)	308 (117)	778 (247)
Median (IQR)		163 (76 to 183)		299 (211 to 413)	748 (596 to 887)
Absolute bias, cells/μL (SD)	−32.5 (99.9)	−4.2 (25.3)	5.9 (48.2)	−1 (66.7)	−57.6 (114.3)
LOA, mean (±1.96 SD)	−228.3 to 163.4	−53.7 to 45.3	−88.5 to 100.3	−131.7 to 129.7	−281.7 to 166.5
Paired t-test for difference in means (95% CI; p)	−32.5 (−48.5 to −16.4; p = .0001)	−4.2 (−22.3 to 13.9; p = .6119)	5.9 (−11.2 to 23.0; p = .4861)	−1 (−17.3 to 15.3; p = .9026)	−57.6 (−82.4 to −32.8; p < .0001)

LOA, limit of agreement; CI, confidence interval; IQR, interquartile range.

FIG. 2.

Bland–Altman plot comparing the difference between FACSCount and PIMA versus the mean of the two methods for CD4 cell count (cells/mm³).

We also analyzed the utility of the Alere PIMA analyzer to classify patients under the clinical CD4 T cell cutoff points of <200, <350, and <500 by determining the sensitivity, specificity, PPV, and NPV. We found that the sensitivity of the PIMA analyzer gradually dropped from 94.4% (95% confidence interval 86.2%–98.4%) to 71.4 (41.9%–91.6%) at the <500 and <200 CD4 T cell cutoffs, respectively. Except for NPV at <350 cutoff (84.6%), specificity, PPV, and NPV of the PIMA analyzer to predict CD4 T cell counts at the various cutoffs were above 94% (Table 3). Our findings were supported by ROC curve analysis showing area under the curves of 1.0, 0.9887 and 0.9922 for cutoffs at <200, <350, and <500 cells/mm³ (Fig. 3A–C).

Table 3.

Sensitivity, Specificity, Negative Predictive Value, and Positive Predictive Value at 200, 350, and 500 Cells/μL Threshold

	CD4 T cell cutoff points
	≤200 cells/μL	≤350 cells/μL	≤500 cells/μL
Sensitivity, (%) (95% CI)	71.4 (41.9–91.6)	84.6 (69.5–94.1)	94.4 (86.2–98.4)
Specificity, (%) (95% CI)	100 (97.3–100)	94.6 (88.7–98)	100 (94.8–100)
PPV, (%) (95% CI)	100 (69.2–100)	94.6 (88.7–98)	94.5 (86.6–98.5)
NPV, (%) (95% CI)	97.2 (92.9–99.2)	84.6 (69.5–94.1)	100 (94.6–100)

NPV, negative predictive value; PPV, positive predictive value.

FIG. 3.

ROC analysis for sensitivity and specificity of PIMA using FACSCount as a gold standard, in classifying patients with CD4 T cell count of (A) <200 cells/mm³, (B) <350 cells/mm³ and (C) <500 cells/mm³. ROC, receiver operating characteristic.

Discussion

In this study, we assessed the performance of the Alere PIMA point-of-care CD4 T cell analyzer for measuring CD4 T cell counts under our field conditions and compared it with the BD FACSCount analyzer. We also assessed its utility at predicting CD4 T cell counts at cutoff of <200, <350, and <500 cells/mm³. Currently, the WHO recommends ART is initiated as soon as the HIV diagnosis is confirmed¹⁷; however, in many resource-limited settings, the decision to prioritize who to start on treatment is still dependent upon a CD4 count result.¹⁸ Furthermore, CD4 T cell counts are still used to assess the degree of disease progression as well as monitor response to ART especially among immunological failures. Therefore, the utility of the CD4 count test in HIV treatment and care programs is likely to remain relevant for a while.

Our study findings add to, and are in agreement with already-existing data from earlier studies.^{10,11,19–23} Under our field settings the Alere PIMA analyzer significantly underestimated CD4 T cell measurements when compared with the FACSCount. The overall negative bias of −32.5 cells/mm³ is comparable to the −34.6 cells/mm³ bias obtained in a PIMA versus FACSCalibur study conducted under similar settings in Rakai,²⁰ Uganda. Moreover, several other studies have found that Alere PIMA measurements generally show a negative bias compared with the BD FACSCount or the BD FACSCalibur technologies.^{10,11,19–24} However, the finding that the bias in PIMA CD4 T cell count measurements was mostly due to underestimation at <500 cells/mm³ means that the utility of this POC technology in clinical practice under our field settings may be less affected by this bias. Indeed, we found very strong correlation between PIMA and FACSCount measurements generally, and strong PIMA performance characteristics of sensitivity, specificity, PPV, and NPV especially at clinical cutoffs of 350 and 500 cells/mm³. Thus, we find that the PIMA results compare quite favorably to those of FACSCount.

In resource-limited settings, where ART coverage may be limited, and/or stock outs of ART drugs persist, priority is given to those with CD4 <350.¹⁷ In addition, baseline monitoring of CD4 count is still done, as it is the key factor in determining the need to initiate opportunistic infection prophylaxis at CD4 <200, identify eligibility for CrAg testing²⁵ to inform decision on initiation of prophylactic treatment for cryptococcal meningitis, prioritization of ART initiation of patients with CD4 <350, and fast tracking those at CD4 <200. These measurements are important in monitoring patient immunological status at scheduled visits. The sensitivity of the PIMA to classify patients dropped as the cutoff threshold lowered, indicating that PIMA may, in these settings, leave out up to 29% and 15% of patients at thresholds of <200 and <350 cells/mm³, respectively. These patients may miss out on clinical benefits, for example, reduction of mortality in the hard-to-reach areas,²⁶ accorded to patients with CD4 counts below these thresholds, a situation that may affect their management.

In our study, we also evaluated the performance of PIMA compared with FACSCount at a cutoff of 200 cells/mm³, as this is the clinical threshold below which most opportunistic infections develop in HIV and AIDS patients. We observed a perfect PPV (100%) for PIMA to correctly identify participants with a CD4 T cell count less than 200. This implies that in resource-limited settings, a PIMA result of <200 cells/mm³ can be used effectively to determine which patients will need prophylactic treatment for opportunistic infections.

The modest number of participants in our study, particularly at low CD4 T Cell counts, is a limitation of this study. Additionally, the possibility of variability that may be introduced by the different operators and the variability in the blood collection from the study participants and its transportation to the laboratory was a limitation to our study. However, we minimized this bias as we used experienced technicians who had certified competence (Good Clinical Laboratory Practice training) for the blood collection and operating the PIMA and the FACSCount.

In resource-limited settings, the cost of monitoring patients for disease progress using viral loads is still prohibitive given the cost of machines and the reagents. Therefore, in such settings, POC technologies, such as the PIMA could facilitate monitoring patients' disease progression, screening for priority to initiation of ART and monitoring treatment response.

Author Disclosure Statement

No competing financial interests exist.