Evaluation of a High-Throughput Diagnostic System for Detection of HIV-1 in Dried Blood Spot Samples from Infants in Mozambique

Abstract

We performed a comparative analysis between Roche Amplicor HIV-1 DNA test and CAPTAQ assay for the detection of HIV in 830 dried blood spot (DBS) pediatric samples collected in Mozambique. Our results demonstrated no statistical difference between these assays. The CAPTAQ assay approached nearly 100% repeatability/accuracy. The increased throughput of testing with minimal operator interference in performing the CAPTAQ assay clearly demonstrated that this method is an improvement over the Roche Amplicor HIV-1 DNA test, version 1.5.

TEXT

Early infant diagnosis (EID) and immediate initiation of antiretroviral therapy are paramount to prevent AIDS-related deaths and to evaluate the effectiveness of prevention of mother-to-child transmission (PMTCT) programs (6, 8, 16, 17). Currently, in most resource-poor settings with high burdens of HIV infections, a significant proportion of exposed infants do not have access to timely diagnosis of HIV. While it is necessary to seek simpler diagnostic assays for decentralization of testing, it is also urgent to increase the throughput of centralized testing to meet demand.

The Roche Amplicor HIV-1 DNA test, version 1.5 (Roche Molecular Diagnostics, Branchburg, NJ), has been used by many diagnostic laboratories (13) for detection of HIV in dried blood spot (DBS) (1, 10) samples. This assay has demonstrated high sensitivity and specificity (9). However, because this assay is performed manually, it has a low throughput in comparison with automated tests (3, 4, 5, 14, 15). In order to increase the testing capacity of Instituto Nacional de Saúde in Mozambique, where approximately 2,900 children were tested each month for HIV infections in 2010, we evaluated the performance of the Roche COBAS AmpliPrep/COBAS TaqMan (CAPTAQ) HIV-1 Qual test (Roche Molecular Diagnostics, Branchburg, NJ), which is an automated qualitative test for HIV detection in blood or DBSs (12). Eight hundred thirty country-wide pediatric (6 weeks to 9 months old) DBS samples that were consecutively received and tested for HIV by Amplicor assay in 2010 were subsequently tested using the CAPTAQ assay for comparison of results, following the manufacturer's recommendations (10, 12). Results of both assays are summarized in Table 1. A total of eight samples (0.96%) provided discordant results between the methods. All eight discordant samples were retested using the Amplicor assay, and results were identical to the initial testing. Unfortunately, insufficient DBS material was available to repeat the testing with the CAPTAQ assay.

Table 1.

Summary of comparative results of CAPTAQ and Amplicor assays

CAPTAQ result	No. of samples with Amplicor result			Sensitivity (95% CI)	Specificity (95% CI)	McNemar's test (P value)	Cohen's kappa (95% CI)a
	HIV+b	HIV−c	Total
HIV+	98	2	100	94.2% (93%, 99.8%)	99.7% (98.2%, 99.7%)	1.125 (0.29)	0.88 (0.84, 0.92)
HIV−	6	724	730
Total	104	726	830

^a95% CI derived from 10,000 ordinary bootstrap replicates.

^bHIV+, HIV detected.

^cHIV−, HIV not detected.

McNemar's test and Cohen's kappa statistic test were used to determine agreement between the two assays (7). Additionally, we calculated sensitivity and specificity of the CAPTAQ assay, as well as overall, positive, and negative indices of agreement, using the Amplicor assay as the gold standard method (2, 11). There was no evidence of statistically significant discordance between the two assays as shown by McNemar's test (Table 1). Additionally, the kappa statistic was 0.88 with a lower 95% confidence limit of 0.84. These numbers are above 0.8, which indicates good agreement between the CAPTAQ and the Amplicor assays.

Lastly, compared to the Amplicor assay, the sensitivity and specificity of the CAPTAQ assay was 94.2% and 99.2% with lower 95% confidence bounds of 93% and 98.2%, respectively. The proportion of overall agreement was 99% with a lower bound of 98.4%. However, due to the high number of undetected HIV-1 samples there was a tendency or bias for the overall agreement to be high. Therefore, we also computed the proportions of agreements for HIV-1 detected and HIV-1 undetected results separately, estimating the conditional probability of both tests producing the same results (Table 2). Our data demonstrated that the proportion of positive agreement is also high at 96.1% (95% confidence interval [CI], 93.4%, 98.8%).

Table 2.

Proportions of overall and specific agreement for the CAPTAQ and Amplicor assays in detecting HIV-1 for 200 DBS

Type of agreement	Proportion	SE	Confidence limits
Overall agreement (po)a	99%	0.03%	98.4%, 99.7%
Positive agreement (ppa)b	96.10%	0.14%	93.4%, 98.8%
Negative agreement (pna)c	99.50%	0.02%	99.1%, 99.8%

^ap_o, proportion of cases for which Amplicor and CAPTAQ tests agree.

^bp_pa, estimated probability that one test will detect HIV given that the other test will also produce the same result.

^cp_na, estimated probability that one test will not detect HIV given that the other test will also produce the same result.

Blood samples (5 ml) from 20 HIV-1 infected individuals, with known viral loads, and 50 HIV-1 noninfected individuals were acquired from commercial sources (ZeptoMetrix, Buffalo, NY, and Tennessee Blood Services, Memphis, TN, respectively) and used to determine the degree of repeatability of the CAPTAQ assay. Five 100-μl whole DBS samples were prepared from each donor sample. We examined the repeatability of the CAPTAQ assay along with its sensitivity and specificity by testing DBS specimens from all individuals on three separate runs following the manufacturer's recommendations. To assess repeatability of the CAPTAQ assay, first we used a generalized McNemar's test to determine whether the proportion of detected HIV in each run was different. Second, we examined repeatability through the use of Fleiss' kappa statistic as described above. Results demonstrated a 99% degree of agreement in all runs (see Table 4). The only false-negative result was from an HIV-infected individual with a viral load below 50 copies/ml, which is approximately the limit of detection of most molecular assays. The repeatability of the instrument across samples was high, with a Fleiss kappa of 0.98 in each category as well as the overall rating. The lower 95% confidence bound of this statistic is 0.83, which is above the 0.81 threshold generally held to indicate strong repeatability (Table 3). Additionally, the proportion of positive agreement was 98.3% with a lower 95% CI boundary of 94.7%, indicating with reasonable confidence that the true proportion of replicates agreeing on a result demonstrating presence of HIV is between 94.7% and 100% (Table 4).

Table 4.

Proportions of overall and specific agreement for the CAPTAQ assay for three replicates of DBS samples prepared from 20 HIV-1 infected and 50 HIV uninfected individuals

Type of agreement	Proportion	Bootstrap confidence limits
Specific agreement (ps) (HIV+)a	98.30%	94.70%, 100%
Specific agreement (ps) (HIV−)a	99.30%	98.00%, 100%
Overall agreement (pob)	99.10%	97.10% 100%

^ap_s(j), the proportion of agreement specific to category j is equal to the total number of agreements on category j divided by the total number of opportunities for agreement on category j, where j is either HIV + or HIV−. HIV+, HIV detected; HIV−, HIV not detected.

^bp_o, proportion of cases for which all three replicates of the HIV-1 Amplicor version 1.5 DNA-PCR test agree.

Table 3.

Summary of repeatability of results by the CAPTAQ assay on DBS samples prepared from 20 HIV-1 infected and 50 HIV uninfected individuals

Diagnosis	Test 1	Test 2	Test 3	Fleiss' kappa (SE)	P value
HIV detected	19	20	20	0.98 (0.069)	P < 0.001
HIV not detected	51	50	50	0.98 (0.069)	P < 0.001
Overall (total count)	70	70	70	0.98 (0.069)	P < 0.001

A prospective study of a cohort of HIV-exposed children would be necessary to determine the clinical sensitivity and specificity of the CAPTAQ assay. However, we can infer that these parameters are similar to those of the Amplicor assay based on our results and previously published studies (9, 12). The discordant results observed between the two methods, which constituted 1% of samples, could be attributed to the combined rates of sensitivity and specificity of both assays.

The CAPTAQ configuration used in this study was one COBAS AmpliPrep instrument coupled to one COBAS TaqMan 48 instrument. Method comparison, sensitivity, specificity, and repeatability parameters of the CAPTAQ assay verified by the Instituto Nacional de Saúde in Mozambique during routine laboratory operations were satisfactory. Minor modifications in the preparation of DBS specimens were introduced in the laboratory workflow to adapt for the CAPTAQ instrument. However, barcode scanning capability, prevention of operator-generation errors by the CAPTAQ system, and increased throughput of testing from 200 samples/technician/week to 450 samples/technician/week were advantages over the Amplicor assay. In the absence of virological point-of-care assays, automated laboratory diagnostic systems such as the CAPTAQ assay that are capable of processing a large number of DBS specimens in a cost-efficient manner seem to be an appropriate solution to increase access to EID in high-demand settings.