Laboratory-based performance evaluation of PIMA CD4+ T-lymphocyte count point-of-care by lay-counselors in Kenya

Abstract

Background:

CD4+ T-lymphocyte count testing at the point-of-care (POC) may improve linkage to care of persons diagnosed with HIV-1 infection, but the accuracy of POC devices when operated by lay-counselors in the era of task-shifting is unknown. We examined the accuracy of Alere’s Pima™ POC device on both capillary and venous blood when performed by lay-counselors and laboratory technicians.

Methods:

In Phase I, we compared the perfomance of POC against FACSCalibur™ for 280 venous specimens by laboratory technicians. In Phase II we compared POC performance by lay-counselors versus laboratory technicians using 147 paired capillary and venous specimens, and compared these to FACSCalibur™. Statistical analyses included Bland-Altman analyses, concordance correlation coefficient, sensitivity, and specificity at treatment eligibility thresholds of 200, 350, and 500 cells/μl.

Results:

Phase I: POC sensitivity and specificity were 93.0% and 84.1% at 500 cells/μl, respectively. Phase II: Good agreement was observed for venous POC results from both lay-counselors (concordance correlation coefficient (CCC) = 0.873, bias −86.4 cells/μl) and laboratory technicians (CCC = 0.920, bias −65.7 cells/μl). Capillary POC had good correlation: lay-counselors (CCC = 0.902, bias −71.2 cells/μl), laboratory technicians (CCC = 0.918, bias −63.0 cells/μl). Misclassification at the 500 cells/μl threshold for venous blood was 13.6% and 10.2% for lay-counselors and laboratory technicians and 12.2% for capillary blood in both groups. POC tended to under-classify the CD4 values with increasingly negative bias at higher CD4 values.

Conclusions:

Pima™ results were comparable to FACSCalibur™ for both venous and capillary specimens when operated by lay-counselors. POC CD4 testing has the potential to improve linkage to HIV care without burdening laboratory technicians in resource-limited settings.

1. Background

CD4 cell counts are important immune markers whose decline have been correlated with poor HIV disease prognosis and hence are used to assess antiretroviral treatment eligibility (Fahey et al., 1990; Hogg et al., 2001; Mellors et al., 1997; WHO, 2013). CD4 counts are also recommended to guide decisions for initiating and stopping prophylaxis for opportunistic infections and in clinical management of treated patients in areas with limited access to viral load tests (WHO, 2013).

The enumeration of CD4 cells has conventionally been done by use of the laboratory-based flow cytometry system (WHO, 2014). However such systems have not been able to fully cater to the testing needs of HIV infected patients in resource-limited settings (RLS) because of weak laboratory infrastructure and poor access to centralized facilities (WHO, 2014; Peter et al., 2008). Moreover lab-based flow-cytometers are sophisticated, and have high capital, operational and maintenance costs, and require highly trained laboratory technicians who are scarce in RLS (WHO, 2014; Peter et al., 2008; UNITAID, 2014). Logistic constraints in supply chain management, frequent equipment breakdowns, sample collection and transportation requirements further affect CD4 testing in these settings (UNITAID, 2014).

Due to these challenges the turnaround time between a CD4 test request and report of test result can be significantly long, which at times leads to late treatment initiation and loss to follow-up (LTFU) (Mugglin et al., 2012; Kranzer et al., 2012; Rosen and Fox, 2011). These limitations have necessitated the call for more affordable and accurate point-of-care (POC) tests, some of which are under evaluation, with one device – Alere’s Pima™ –currently on the market (UNITAID, 2014). Utilization of the POC devices is expected to address the challenges inherent in centralized laboratory systems. However prior to adoption there is a need for comprehensive evaluation to ensure their accuracy and effectiveness in meeting the gap caused by cost and logistic challenges of the weak laboratory infrastructure in RLS.

Most countries in RLS currently follow the World Health Organization Consolidated guidelines on the use of antiretroviral drugs for treating and preventing HIV infection, 2013, in which treatment initiation is based on a CD4 threshold below 350 cells per microliter (cells/μl) (WHO, 2013; Nelson et al., 2014). Based on this approach, misclassification by a test could either lead to a missed opportunity for early treatment initiation, or deny patients with more advanced HIV disease an opportunity for timely treatment placement.

Moreover the evaluation of these tests must also take into account the shortage of laboratory personnel in RLS. It is anticipated that expansion of laboratory services by using POC tests is only feasible if lay-counselors can use such devices effectively. Our purpose was to assess the accuracy of the Pima™ CD4 POC device when performed by lay-counselors versus laboratory technicians using capillary versus venous blood, as compared to the BD FACSCalibur™ system.

2. Material and methods

The Kenya Pima™ POC performance evaluation study was conducted in two phases. In Phase I remnant venous blood samples were obtained from HIV positive patients attending routine clinical visits at the Jaramogi Oginga Odinga Teaching and Referral Hospital (JOOTRH). Samples were initially tested on BD FACSCalibur™ and then the remaining specimens were re-tested on Pima™ POC in accordance with the manufacturer’s protocol. All testing was done at of the Kenya Medical Research Institute (KEMRI) ISO 15189-certified laboratory located at the Clinical Research Center at JOOTRH. Phase II involved evaluation of the Pima™ device on capillary blood samples as well as assessing the performance of lay-counselors compared to trained laboratory technicians in a laboratory setting. In Phase II, paired capillary and venous blood that were obtained from HIV-infected patients attending routine clinical care at JOOTRH. In both phases convenient sampling of remnant blood from routine testing was used.

Finger prick samples of one or two drops of blood were collected using Sarstedt safety lancets, while 3 ml of venous blood were collected in an evacuated tube containing K₂EDTA. Paired capillary and venous blood samples were collected by a phlebotomist and samples were transported to the laboratory where the analysis was conducted.

The Pima™ CD4 POC analyzer is a portable bench-top device that operates on either a built-in rechargeable battery or 100–240 V AC. It consists of a printer, analyzer, and disposable test cartridge. Up to 25 μl of blood is dispensed on the cartridge, which contains pre-loaded anti-CD3 and anti-CD4 antibodies conjugated with fluorescent-labeled dyes. Once capped, the cartridge is immediately loaded into the analyzer for the 20-min analysis that consists of dissolving of the fluorescent-labeled anti-CD3 and antiCD4 reagents in the sample, which is passed into an optical imaging chamber. The imaging chamber consists of multi-color fluorescence imaging optics, which analyzes the fluorescent-labeled cells after which data are collected and analyzed using embedded software in the system’s computer to derive absolute numbers of CD3 + CD4+ T-lymphocytes.

CD4 T-cell enumeration for venous and capillary blood by Pima™ was done within 12 h of sample collection. Due to the need for comparative testing with the gold standard FACSCalibur, and to minimize patient discomfort, capillary blood was first collected into micro-capillary tubes instead of the conventional direct loading into Pima™ cartridges. The micro-capillary tube introduced about 50 μl of blood but only 25 μl was used on the disposable cartridge pre-loaded with anti-CD3 and anti-CD4 fluorescent-labeled reagents. The cartridge was then immediately capped and inserted into the analyzer within 5 min after capping, as per the manufacturer’s instruction.

The CD4+ T-lymphocyte counting on FACSCalibur™ was performed using both venous and capillary blood by adding 50 μl of blood into TriTest tubes containing fluorescent labeled anti-CD3 and either anti-CD4 or anti-CD45 antibodies and fluorescent beads.

Quality was assured on CD4 testing using the Pima™ device by using the manufacturer-provided internal quality control cartridges (normal and low CD4) on each day of testing, which were required to pass prior to testing of study samples. Laboratory technicians and lay-counselors were trained on the device operation during the study. Internal controls and calibrators were run daily on the FACSCalibur™ and were required to have passed prior testing of study samples. The laboratory is also enrolled in the CD4 external quality assessment schemes through the United Kingdom National External Quality Assessment Service (UKNEQAS) and the College of America Pathologist USA (CAP) both assessments of which it had passed in the preceding 12 months.

2.1. Ethics review

The study was approved by the KEMRI Ethical Review Committee as well as the Institutional Review Board of the U.S. Centers for Disease Control and Prevention (CDC), Atlanta, GA, USA.

2.2. Statistical analysis

Our analysis consisted of three steps: assessment of Pima™ on venous blood by laboratory technicians (Phase I), comparisons between lay-counselors and laboratory technicians on both venous and capillary blood (Phase II), and elucidating the direction of bias by Pima™ compared to FACSCalibur™ (Combined Phase I & II). Lin’s concordance correlation coefficient (CCC: r_c) was used as a measure to simultaneously assess Pearson’s correlation coefficient for precision (r) and accuracy (c_a) between platforms and cadre of staff while Bland-Altman analyses were used to determine the mean biases and the limits of agreement (mean ± 95% LOA) between the same comparisons (Vander Heyden and Smeyers-Verbeke, 2007; Lin, 1989). Misclassification bias was assessed by determining the percentage of patients who were incorrectly classified as above or below the CD4 count treatment eligibility thresholds of 200, 350, and 500, while using the FACSCalibur™ as the gold standard. To further elucidate the direction of bias in the different comparisons with the gold standard we used scatter plots, plots of accuracy, boxplots, linear regression when Pima™ results were compared to FACSCalibur™, and errors-in-variables regression when Pima™ results were compared to other Pima™ results.

3. Results

A total of 427 patients were enrolled in the study; 280 in Phase I and 147 in Phase II. The median absolute CD4+ T-lymphocyte counts by FACSCalibur™ were 494 (1st and 3rd quartiles: 368.5, 693) for Phase I and 544 (1st and 3rd quartiles: 394, 718) in Phase II.

3.1. Phase I: comparison between Pima™ and FACSCalibur™ on venous blood

The Phase I Pima™-venous versus the reference FACSCalibur™-venous for CD4 count assessed by laboratory personnel showed excellent agreement with c_a = 0.990, r = 0.932, and r_c = 0.923 (95% CI 0.903–0.938) and a mean bias of −38.6 (−51.5, −25.8) cells/μl (Table 1, Fig. 1). Overall misclassification bias at 200 cells/μl threshold was 3.6% (10/280). The sensitivity and specificity was 95.2% and 97.7% respectively. Overall misclassification rate at the 350 cells/μl thresholds was 11.1% (31/280); 2.5% of the patients (7/280) were misclassified above the threshold while 8.6% (24/280) were under-classified. The sensitivity and specificity, at 350, was 88.7% and 89.0%, respectively. Overall misclassification rate at the 500 cells/μl was 11.4% (32/280); 3.6% (10/280) were misclassified above the threshold and 7.9% (22/280) were under-classified. The sensitivity and specificity at this threshold were 93.0% and 84.1%, respectively.

Table 1.

Performance of Pima™ point-of-care (POC) in measuring CD4+ T-lymphocyte counts as compared to the FACSCalibur™ reference from paired venous and capillary blood of HIV-infected persons by laboratory technicians and lay-counselors in Western Kenya.

	Statistic	Phase I (n = 280)	Phase II (n = 147)
		Pima Lab Venous CD4 Count	FACS Calibur Capillary CD4 Count	Pima Lab Venous CD4 Count	Pima Lab Capillary CD4 Count	Pima lay-counselors Venous CD4 Count	Pima lay-counselors Capillary CD4 Count
FACS Calibur Venous CD4 Count	Accuracy (Ca)	0.990	0.993	0.982	0.953	0.915	0.941
	Precision (r)	0.932	0.968	0.960	0.963	0.954	0.958
	CCC (rc)	0.923 0.903, 0.938	0.962 0.947, 0.972	0.920 0.904, 0.933	0.918 0.893, 0.937	0.873 0.838, 0.900	0.902 0.873, 0.925
	Bias (cells/μl, range)	−38.6 −51.5, −25.8	−26.9 −37.5, −16.4	−65.7 −78.4, −53.0	−63.0 −75.3, −50.6	−86.4 −100.4, −72.4	−71.2 −84.3, −58.2
	Bias by CD4 Class (cells/μl)
	<350 (n = 62)	−4.4 −16.8, 7.9	−6.0 −17.7, 5.8	−12.2 −25.6, 1.2	−6.5 −20.2, 7.1	−17.4 −29.7, −5.1	−12.7 −24.7, −0.6
	350–500 (n = 80)	−18.8 −34.6, −2.9	−14.2 −30.6, 2.2	−35.1 −54.1, −16.0	−35.2 −54.2, −16.2	−42.9 −61.6, −24.3	−45.2 −65.7, −24.7
	500–1000 (n = 115)	−53.4 −73.9, −32.8	−29.7 −46.5, −12.9	−87.9 −105.2, −70.5	−83.6 −99.8, −67.4	−118.0 −135.1, −101.0	−86.8 −102.7, −70.9
	>1000 (n = 23)	−126.4 −221.3, −31.6	−121.1 −160.8, −81.4	−174.2 −249.4, −99.1	−184.2 −245.9, −122.5	−215.3, −308.4, −122.3	−235.9 −303.4, −168.3
	Sensitivity Cutoff (%)
	<200	95.2 72.9, 99.3	100.0	88.89 50.01, 98.46	100.0	100.0	100.0
	<350	88.7 78.2, 94.5	93.6 77.6, 98.4	100.0	93.6 77.6, 98.4	93.6 77.6, 98.4	96.8 80.4, 99.6
	<500	93.0 87.4, 96.2	93.7 84.3, 97.6	96.8 88.2, 99.2	98.4 89.6, 99.8	100.0	98.4 89.6, 99.8
	Specificity (%)
	<200	97.7 94.9, 99.0	97.8 93.5, 99.3	97.8 93.5, 99.3	99.3 95.0, 99.9	97.1 92.5, 98.9	97.1 92.5, 98.9
	<350	89.0 84.1, 92.5	94.0 87.9, 97.1	92.2 85.8, 95.9	97.4 92.3, 99.2	94.8 89.0, 97.7	93.1 86.8, 96.5
	<500	84.1 77.0, 89.3	90.5 82.1, 95.2	83.3 73.8, 89.9	79.7 69.8, 87.0	76.2 66.0, 84.1	79.7 69.8, 87.0

Note for Table 1. ‘n’ refers to paired venous and capillary blood specimens. Bias is expressed in units of cells/μl. Phase I compared the Pima results with FACSCalibur when run by laboratory technicians (denoted ‘Lab’), while Phase II compared results of the Pima device by lay-counselors with those of laboratory technicians.

Fig. 1.

Validation Phase I and II data scatter plots of pairwise comparisons between FACSCalibur™ and the Pima™ point-of-care device for CD4+ T-lymphocyte counts of HIV-infected persons using venous and capillary blood samples. The solid and dashed lines are the 45 degree reference lines (perfect fit) and fitted lines from the linear regression model, respectively.

3.2. Phase II: FACSCalibur™ venous vs FACSCalibur™ capillary blood

Comparison between venous and capillary blood on FACSCalibur™ revealed good agreement with a c_a = 0.993, r = 0.968, and r_c = 0.962 (95%CI 0.947–0.972). The bias between the two sample types was −26.93 (LOA −37.47, to −16.4) (Table 1, Fig. 2). Overall misclassification by capillary sample type at the 200 cells/μl threshold was 2.0% (3/147) while the sensitivity and specificity were 100.0% and 97.8%, respectively. At the 350 cells/μl threshold, overall misclassification was 5.4% (8/147) while sensitivity and specificity was 93.6% and 94.0% respectively. Overall misclassification bias at the 500 cells/μl threshold was 8.2% (12/147) with sensitivity and specificity as 93.7% and 90.5%, respectively.

Fig. 2.

Validation Phase II Bland-Altman plots for all two-way comparisons of CD4+ T-lymphocyte counts by FACSCalibur™ and Pima™ point-of-care devices using venous and capillary operated by lay-counselors or laboratory technicians. Note for Fig. 2. The FACSCalibur venous (top row x-axis) is the reference standard. The solid red line is the mean bias and the 95% limits of agreement (LOA) are shown by the dashed red lines. FV, lay-counselor venous blood; FC, lay-counselor capillary blood; LV, laboratory technician venous blood; LC, laboratory technician capillary blood. All units in cells/μl.

3.3. Phase II: comparison between laboratory technicians and lay-counselors on venous blood

Pima™ POC results from the lay-counselors were comparable to those of the laboratory technicians when compared against FACSCalibur™- using the predicate method for venous blood with r_c = 0.873 (95% CI 0.838–0.900) for venous blood and 0.902 (95% CI 0.873–0.925) for capillary blood (Table 1, Fig. 1). The mean bias comparison between Pima™-POC from the lay-counselors and FACSCalibur™-venous results was −86.4 (LOA −100.4 and −72.4) (Table 1, Fig. 2). Similarly, high levels of agreements were observed for the laboratory technicians for venous with a negative mean-bias of −65.7 (LOA −78.4 and −53.0). Further Pima™ -Pima™ comparisons showed comparable results between the lay-counselors and the laboratory technicians for venous blood, with r_c = 0.950 (95% CI 0.932–0.964) and a mean bias of −20.7 (LOA −31.2 and −10.3) (Table 2, Fig. 2).

Table 2.

Comparative performance of capillary and venous CD4+ T-lymphocyte counts on Pima™ POC and FACSCalibur™ by laboratory technicians and lay-counselors in Western Kenya.

	Statistic	FACS Calibur Venous CD4 Count	FACS Calibur Capillary CD4 Count	PIMA Lab Venous CD4 Count	PIMA Lab Capillary CD4 Count	PIMA lay-counselors Venous CD4 Count	PIMA lay-counselors Capillary CD4 Count
FACS Calibur Venous CD4 Count	Accuracy (Ca)	1.0	0.993	0.950	0.953	0.915	0.941
	Precision (r)	1.0	0.968	0.960	0.963	0.954	0.958
	CCC (rc)	1.0	0.962 0.947, 0.972	0.912 0.885, 0.932	0.918 0.893, 0.937	0.873 0.838, 0.900	0.902 0.873, 0.925
	Bias	0.0	−26.9 −37.5, −16.4	−65.7 −78.4, −53.0	−63.0 −75.3, −50.6	−86.4 −100.4, −72.4	−71.2 −82.3, −58.2
FACS Calibur Capillary CD4 Count	Accuracy (Ca)		1.0	0.979	0.981	0.953	0.973
	Precision (r)		1.0	0.944	0.961	0.947	0.956
	CCC (rc)		1.0	0.924 0.898, 0.943	0.943 0.924, 0.957	0.902 0.872, 0.926	0.930 0.908, 0.948
	Bias		0.0	−38.8 −52.2, −25.3	−36.0 −47.5, −24.6	−59.5 −73.0, −45.9	−44.3 −56.5, −32.1
PIMA Lab Venous CD4 Count	Accuracy (Ca)			1.0	0.9999	0.994	0.9996
	Precision (r)			1.0	0.963	0.956	0.956
	CCC (rc)			1.0	0.963 0.949, 0.973	0.950 0.932, 0.964	0.955 0.939, 0.968
	Bias			0.0	2.7 −6.9, 12.4	−20.7 −31.2, −10.3	−5.5 −16.0, 5.0
PIMA Lab Capillary CD4 Count	Accuracy (Ca)				1.0	0.993	0.999
	Precision (r)				1.0	0.963	0.970
	CCC (rc)				1.0	0.956 0.939, 0.968	0.970 0.958, 0.978
	Bias				0.0	−23.5 −33.1, −13.9	−8.3 −16.9, 0.4
PIMA lay-counselors Venous CD4 Count	Accuracy (Ca)					1.0	0.997
	Precision (r)					1.0	0.957
	CCC (rc)					1.0	0.954 0.937, 0.967
	Bias					0.0	15.2 5.0, 25.4
PIMA lay-counselors Capillary CD4 Count	Accuracy (Ca)						1.0
	Precision (r)						1.0
	CCC (rc)						1.0
	Bias						0.0

Note to Table 2: CCC: concordance correlation coefficient. Bias (in units of cells/μl) was estimated using the Bland-Altman method. All comparisons to FACSCalibur™venous for CD4 counts are comparing to the reference. Comparisons for non-FACSCalibur™ venous have neither as a reference.

Using venous blood the overall misclassification bias at the 200 cells/μL was 2.7% (4/147) for both lay-counselors and laboratory technicians. The sensitivity was 100.0% and 88.9% while specificity was 97.1% and 97.8% for the lay-counselors and laboratory technicians, respectively. Overall rate of venous misclassification at the 350 cells/μl was 5.4% (8/147) for lay-counselors and 6.1% (9/147) for laboratory technicians. The sensitivity was 93.6% and 100.0% while specificity was 94.8% and 92.2% for the lay-counselors and laboratory technicians, respectively. At the 500 cells/μl threshold, the overall misclassification by the lay-counselors was 13.6% (20/147) while the laboratory technicians was 10.2% (15/147). The sensitivities at the 500 cells/μl threshold were 100.0% and 96.8 while specificity was 76.2% and 83.3% for the lay-counselors and laboratory technicians, respectively.

3.4. Phase II: comparison between lab-technicians and lay-counselors on capillary blood

Lay-counselors produced comparable results to the laboratory technicians using capillary blood (Table 1, Fig. 1). Compared to the gold standard – venous FACSCalibur™ – the lay-counselors accuracy, precision and concordance correlation coefficient were 0.941 (c_a), 0.958 (r), and 0.902 (95% CI 0.873–0.925, r_c) with a mean bias of −71.2 (LOA −84.3 to −58.2). In comparison, the laboratory technicians c_a was 0.953, r was 0.963, and r_c was 0.918 (95% CI 0.893–0.937) with a mean bias of −63.0 (LOA −75.3 to −50.6). There was good agreement between Pima™-Pima™ capillary CD4 results produced by laboratory technicians and the lay-counselors with a r_c = 0.970 (95% CI 0.958–0.978) and a mean bias of −8.3 (LOA −16.9 and 0.35) (Fig. 2). A good comparison was observed in Pima™-FACSCalibur™ comparisons on capillary blood with a r_c = 0.930 (95% CI 0.908–0.948) and mean bias of −44.3 (LOA −56.5 to −32.1) for lay-counselors and r_c = 0.943 (95% CI 0.924–0.957), mean bias of −36.0 (LOA −47.5 to −24.6) for the laboratory technicians (Table 2, Fig. 2).

Overall misclassification bias at 200 cells/μl threshold was 2.7% (4/147) for the lay-counselors and 0.7% (1/147) for the laboratory technicians. The sensitivities were 100.0% and specificities were 97.1% and 99.3% for the lay-counselors and laboratory technicians respectively. At the 350 cells/μl threshold, the overall misclassification bias was 6.1% (9/147) for the lay-counselors and 3.4% (5/147) for the laboratory technicians. The sensitivities were 96.8% and 93.6% while specificities were 93.1% and 97.4% for the lay-counselors and laboratory technicians respectively. The overall misclassification bias at the current eligibility threshold of 500 cells/μl was similar for the lay-counselors and the laboratory technicians at 12.2% (18/147). The sensitivities and specificities were also the same at 98.4% and 79.7%, respectively.

3.5. Ascertainment of the nature and direction of bias

All methods using the Pima™ platform compared to the reference FACSCalibur™ venous blood method underestimated the CD4 count, with on average negative bias: FACSCalibur™ capillary −26.9 (LOA −37.5, −16.4), Pima™ lab venous −65.7 (LOA −78.4, −53.0), Pima ™ lab capillary −63.0 (LOA −75.3, −50.6), Pima™ lay-counselors venous −86.4 (LOA −100.4, −72.4), and Pima™ lay-counselors capillary −71.2 (LOA (−84.3, −58.2) (Table 1, Fig. 2). Further assessment of the FACSCalibur™ capillary-venous comparison, showed that the under-estimations were mainly on larger values of CD4 counts (Figs. 1 & 2). Moreover, the negative bias by Pima™ was observed to increase with increasing venous CD4 counts, with the minimum bias seen at a CD4+ T-lymphocyte count of approximately 195 cells/μl (Fig. 3).

Fig. 3.

Bias of the validation Phase I and II (N = 427) FACSCalibur™ venous reference compared to the Pima™ laboratory venous CD4+ T-lymphocyte counts. Note to Fig. 3. The solid black line is the mean bias line estimated from the weighted least squares (WLS) model and the dashed black lines are the 95% CI for the mean bias line. Minimum bias occurs where the estimated WLS line crosses the x-axis (~195). All units are in cells/μL.

4. Discussion

The potential utility and benefits of CD4 POC have been documented in a number of studies demonstrating increased timely linkages to HIV care and treatment, timely clinical decision-making and reductions in loss-to-follow up (Jani et al., 2011; Larson et al., 2012; Wynberg et al., 2014). However few studies have assessed the practical concerns regarding implementation of these devices in RLS where there is a limited number of skilled health-care personnel. The effectiveness of POC diagnostics in RLS is dependent on their deployment with minimal additional burden to the existing infrastructure. This is further highlighted by the increasing task shifting of HIV care and treatment services from clinic-based health care providers to community based lay-counselors (Barnabas et al., 2014; van Rooyen et al., 2013; WHO, 2008). Our study indicates that lay-counselors with basic training can use the Pima™ CD4 POC device at a comparable performance level as laboratory-technicians. Therefore lay-counselors may be used in the expansion of CD4 testing using the Pima™ POC device without exacerbating the existing shortage of laboratory technicians. This validation was conducted in preparation for a randomized-controlled trial assessing the impact of the Pima™ POC in both facility and community set-up under routine home-based care and testing by lay-counselors. The use of home-based care and testing has proven to be an effective alternative for testing, improved linkage to care, adherence monitoring, and treatment delivery (Barnabas et al., 2014; van Rooyen et al., 2013; Genberg et al., 2015; Doherty et al., 2013). It is expected that POC devices could be more effective if they follow a similar decentralization and task-shifting model.

Our findings indicate that Pima™ POC has comparable performance to the reference standard FACSCalibur™ and this did not differ whether the POC device was in the hands of laboratory technicians or lay-counselors. Our findings also indicate that CD4 tests from finger-prick have comparable performance to that of conventional venous blood. However we observed a tendency towards modest under-estimation of CD4 results by Pima™, which increased at higher counts. The agreement of Pima™ was better at lower CD4 counts, with minimal misclassifications at the initial eligibility threshold of 200 cells/μl. However the rates of misclassification tended to be more pronounced at the 350 or 500 cells/-μl eligibility thresholds with higher rates on capillary compared to venous blood. For both lay-counselors and laboratory technicians, upward misclassification was below 5% while downward misclassifications were below 15%. Similar findings were observed with use of finger-stick to venous blood comparisons on FACSCalibur™ where upwards misclassifications were below 2% and downward misclassification below 10%. Our findings are consistent with previous reports of Pima™ performance (Scott et al., 2015; Mwau et al., 2013). Data from a recent meta-analysis on Pima™ from 22 independent studies comprising 11,803-paired observations reported a downward misclassification of 10.9% and 17.9% for venous-blood tests, 18.7% and 26.3% for capillary-blood tests at thresholds of 350 and 500 cells/μl respectively (Scott et al., 2015). From the clinical perspective, the observed downward misclassification is likely to be more beneficial to patients who would be initiated on treatment earlier than expected. While this would increase programmatic costs, as more patients would be treated due to the negative bias associated with Pima™ POC, this could be desirable at the population-level due to reduction in the risk of HIV transmission. The misclassifications would lead to a somewhat more conservative clinical management approach to individual patient care, potentially a positive outcome. These repeated observations of misclassifications at higher CD4 counts by Pima™ highlight the need for continuous evaluation and improvement of HIV diagnostic tests under a wide range of practical applications to ensure reliable adaptability to the evolving guidelines. Moreover as reported from a recent meta-analysis by Scott et al., it’s imperative that clear criteria be established to determine the acceptable performance characteristics of CD4 diagnostic tests (Scott et al., 2015).

Our study demonstrates that lay-counselors with minimal training are able to conduct CD4 testing with similar performance outcomes as laboratory technicians. This can be attributed to the ease of operation of the Pima™ POC device. The device is a portable, battery-powered self-contained device that uses heat-stable disposable cartridges. The cartridge contains all the required reagents and also serves as the sample carrier thus simplifying the sample preparation and reagent preparation process. The analyzer also has an integrated keypad and LCD display, which further simplifies the operations. Despite these observations it will still be necessary for programs to implement continuous quality assurance mechanisms and training to ensure provision of reliable and accurate results (Drain et al., 2014).

Some study limitations exist. The use of phlebotomists to draw capillary blood that was used for performance evaluation of the lay-counselors is likely to have masked performance errors inherent in blood-draw practices. However in this study it was not feasible to have multiple draws of blood from the patient to allow for independent evaluation in both the FACSCalibur and the test-device Pima™ CD4 POCT. Henceforth it’s likely that the field performance of the lay-counselors may differ from those observed in the study.

Second, the future role of CD4 in the management of HIV is likely to become more restricted following the revised WHO strategies for ‘test and treat’ and use of viral-load for treatment monitoring. However as resources to implement these strategies are still limited it’s likely that CD4 testing may in the short-term continue to play an important role in both treatment initiation and monitoring. More important CD4 tests may also remain useful in determination of patients with low-immunity (CD4 ≤ 200 cells/μl) who are at risk of developing deadly opportunistic infections such as cryptococcal meningitis (WHO, 2016) and tuberculosis, especially as this population forms more than half of the patients starting or re-initiating treatment (WHO, 2015). Furthermore, WHO also recommends cotrimoxazole prophylaxis for patients with CD4 ≤ 350 cells/μl until when stable on ART (WHO, 2015).

In conclusion, our study demonstrates comparable performance of lay-counselors to trained laboratory technician in the conducting of CD4 tests by use of Pima™ POC. This suggests the possible rollout of Pima™ CD4 testing in the community-based health-care without additional burden to the limited laboratory workforce in resource-limited settings.