Comparative Evaluation of the Roche Cobas AmpliPrep/Cobas TaqMan HIV-1 Version 2 Test Using the TaqMan 48 Analyzer and the Abbott RealTime HIV-1 Assay

Abstract

Acceptable precision was achieved in a comparison study of the Abbott RealTime (RT) and Roche CAP/CTM-48 V2 HIV-1 assays, but viral load quantification was under- and overestimated, respectively, compared to the 2nd HIV-1 WHO International Standard. The same quantification patterns were observed for patient cohorts from Africa and the United States.

The aim of this study was to evaluate and compare the Abbott RealTime HIV-1 assay (Abbott RT), which uses the automated m2000sp/m2000rt platform (Abbott Molecular Inc., Des Plaines, IL) and targets the pol/IN region of HIV-1 (1, 4), with the newly released Roche Diagnostics Cobas AmpliPrep/Cobas TaqMan HIV version 2 assay (Roche CAP/CTM-48 V2) on the TaqMan 48 analyzer (Roche Molecular Systems, Branchburg, NJ), which targets the gag and long terminal repeat regions of the HIV-1 genome (5) for the detection and quantification of HIV RNA in plasma samples. Both assays were performed according to the manufacturers' instructions, with RNA extracted using the Abbott m2000sp and Cobas AmpliPrep platforms, respectively.

For assay standardization, the Abbott RT assay used a viral standard from the Virology Quality Assurance (VQA) Laboratory of the AIDS Clinical Trial Group, while Roche used the 2nd HIV-1 WHO International Standard, which slightly complicated the comparison. For the purposes of this study, we used the 2nd HIV-1 WHO International Standard (National Institute for Biological Standards and Control [NIBSC] code 97/650), reconstituted it in 1 ml of sterile water, and prepared serial dilutions. To determine the number of copies/ml, a conversion factor of 0.6 for the Roche assay or 0.58 for the Abbott assay was used. For intra- and interassay precision, multiple replicates of two different WHO International Standard dilutions were analyzed in a single run and then repeated over five separate runs.

Although the precision of both methods was acceptable and within the specifications stated by each manufacturer (Table 1), both assays failed to accurately recover the known WHO International Standard at the two concentrations tested. The Abbott RT assay appeared to underestimate the viral load by around 0.26 to 0.30 log₁₀ copies/ml (40 to 50%), whereas the Roche CAP/CTM-48 V2 assay overestimated by 0.17 to 0.31 log₁₀ copies/ml (60 to 100%). However, the differences were generally within the accepted range of ±0.3 log₁₀.

TABLE 1.

Intra- and interassay precision based on comparison with the 2nd HIV-1 WHO International Standard

Assay	WHO International Standard		Viral load detected (log10 copies/ml)		Over-/underquantification with regard to diluted WHO International Standard [log10 copies/ml (95% CI)]
	Dilution level	Dilution (log10 copies/ml)a	Meanb (95% CI)	SD, totalc
Abbott RT	Low	2.29	1.99 (1.85; 2.13)	0.14	−0.30 (−0.44; −0.16)
	High	4.38	4.12 (4.07; 4.17)	0.04	−0.26 (−0.31; −0.21)
Roche CAP/CTM-48 V2	Low	2.30	2.61 (2.52; 2.69)	0.13	0.31 (0.22; 0.39)
	High	4.40	4.57 (4.50; 4.63)	0.13	0.17 (0.10; 0.23)

^aConversion factor used for Abbott RT was 0.58 copies/ml per IU/ml and for Roche CAP/CTM-48 V2, 0.6 copies/ml per IU/ml.

^bMean value over 5 separate runs with four replicates each.

^cLevel acceptable according to manufacturer's specifications: ≤0.25 for Abbott RT and ≤0.3 for Roche CAP/CTM-48 V2.

To verify the limit of detection (LOD), seven replicates of each WHO International Standard dilution were analyzed during three separate experiments, giving a total of 21 replicates. The LOD was defined as the lowest concentration of HIV RNA (in copies/ml) at which either a quantitative measurement was obtained or a result was reported as “detected” for least 95% of samples. For the Roche assay, this was achieved at the 20-copies/ml dilution, confirming the manufacturer's LOD claim. For the Abbott assay, viral RNA was detected in 90.5% of replicates at the 39-copies/ml dilution and in all replicates at the 77-copies/ml dilution, suggesting that the manufacturer's LOD claim was met. A high percentage coefficient of variation (%CV) was observed for both assays at the low dilution levels, ranging from 19.4% to 326.9% for the Roche CAP/CTM-48 V2 assay and from 35.1% to 223.3% for the Abbott RT assay. The %CV for Roche CAP/CTM-48 V2 showed a gradual decline from the highest to the lowest dilution, while the Abbott RT showed an unexpected blip at the 39-copies/ml dilution (223.3%), with much lower %CVs on either side of this dilution (64.8% and 92.5%) (Table 2). These results are not statistically significant since a limited number of repeats were tested (3).

TABLE 2.

LOD determination for Abbott RT and Roche CAP/CTM-48 V2

WHO standard dilution for Abbott/Roche (copies/ml)a	% detectedb		%CVc
	Abbott	Roche	Abbott	Roche
10/10	47.6	33.3	107.5	326.9
19/20	71.4	95.2	64.8	115.5
39/40	90.5	95.2	223.3	59.7
77/80	100	100	92.5	41.4
193/200	100	100	35.1	19.4

^aConversion factor used for Abbott RT was 0.58 copies/ml per IU/ml; for Roche CAP/CTM-48 V2, it was 0.6 copies/ml per IU/ml.

^bPercent detected with numeric value or result reported as “Detected” over 21 replicates.

^c%CV, 100·(standard deviation/mean number of copies).

Method comparison with more than 100 clinical samples (sample set 1, supplied by Roche, consisted of 69 samples from a geographically diverse HIV-1 population in the United States, and sample set 2, supplied by Abbott, consisted of 48 group M and O samples from Africa) showed consistently higher quantification levels (11.2%/0.4 log₁₀ difference for sample set 1 and 4.5%/0.2 log₁₀ difference for sample set 2) with the Roche assay than with the Abbott assay. Bland- Altman analyses for both cohorts are shown in Fig. 1 A and B. This observation is consistent with results reported by Scott et al. (6), who found a positive bias of 0.33 log₁₀ for the Roche CAP/CTM-48 V2 method relative to the Abbott RT method, with 23% of samples showing a difference of >log 0.5 copies/ml. In our study, a difference of >log 0.5 copies/ml was observed for 30% and 10.4% of samples in the two groups of clinical samples, respectively.

FIG. 1.

(A) Bland-Altman plot of Abbott RT versus Roche CAP/CTM-48 V2 assays of sample set 1 (log scale; horizontal lines represent approximate 95% confidence interval for individual differences). (B) Bland-Altman plot of Abbott RT and Roche CAP/CTM-48 V2 assays of HIV-1 group M and O samples (sample set 2) (log scale; horizontal lines represent approximate 95% confidence interval for individual differences).

Assay specificity was assessed using 88 samples from patients receiving highly active antiretroviral therapy (HAART) who had previous assay results of <50 copies/ml in each sample tested. Qualitative agreement between the two methods (nondetectable viral load) was observed for 69 samples (78.4%), of which 27 were positive and 42 negative. For 16 (18.2%) of the remaining samples, the Roche CAP/CTM-48 V2 assay reported a positive result but the Abbott RT result was negative, and for 3 (3.5%) samples the converse was true. The kappa coefficient, a measure of interassay agreement based on paired binary data (2), was calculated as κ = 0.56 (95% confidence interval (CI): 0.40; 0.73) for this data, which suggests only moderate agreement between the two assay methods (data not shown).

Both assays met the required acceptance criteria as specified by each manufacturer. A difference in quantification was observed between the two methods for both clinical samples as well as the WHO International Standard. The Roche CAP/CTM-48 V2 method consistently provided higher viral load levels than the Abbott assay. It can be concluded that it is not advisable to switch methods during longitudinal viral load monitoring of patients.