Clinical Comparison of an Enhanced-Sensitivity Branched-DNA Assay and Reverse Transcription-PCR for Quantitation of Human Immunodeficiency Virus Type 1 RNA in Plasma

Abstract

The performance characteristics of an enhanced-sensitivity branched-DNA assay (bDNA) (Quantiplex HIV-1 version 2.0; Chiron Corp., Emeryville, Calif.) and a reverse transcription (RT)-PCR assay (AMPLICOR HIV-1 Monitor; Roche Diagnostic Systems, Inc., Branchburg, N.J.) were compared in a molecular diagnostic laboratory. Samples used in this evaluation included linearity and reproducibility panels made by dilution of a human immunodeficiency virus type 1 (HIV-1) stock culture of known virus particle count in HIV-1-negative plasma, a subtype panel consisting of HIV-1 subtypes A through F at a standardized level, and 64 baseline plasma specimens from HIV-1-infected individuals. Plots of log₁₀ HIV RNA copies per milliliter versus log₁₀ nominal virus particles per milliliter demonstrated that both assays were linear over the stated dynamic ranges (bDNA, r = 0.98; RT-PCR, r = 0.99), but comparison of the slopes of the regression lines (bDNA, m = 0.96; RT-PCR, m = 0.83) suggested that RT-PCR had greater proportional systematic error. The between-run coefficients of variation for bDNA and RT-PCR were 24.3 and 34.3%, respectively, for a sample containing 1,650 nominal virus particles/ml and 44.0 and 42.7%, respectively, for a sample containing 165 nominal virus particles/ml. Subtypes B, C, and D were quantitated with similar efficiencies by bDNA and RT-PCR; however, RT-PCR was less efficient in quantitating subtypes A, E, and F. One non-B subtype was recognized in our clinical specimens based on the ratio of values obtained with the two methods. HIV-1 RNA was quantitated in 53 (83%) baseline plasma specimens by bDNA and in 55 (86%) specimens by RT-PCR. RT-PCR values were consistently greater than bDNA values, with population means of 142,419 and 67,580 copies/ml, respectively (P < 0.01). The results were highly correlated (r = 0.91), but the agreement was poor (mean difference in log₁₀ copies per milliliter ± 2 standard deviations, 0.45 ± 0.61) for the 50 clinical specimens that gave discrete values with both methods.

Plasma human immunodeficiency virus type 1 (HIV-1) RNA levels have recently been shown to provide important prognostic information about the progression to AIDS and useful information about the short-term efficacy of antiretroviral drugs (7, 12, 16, 18). However, it remains unclear whether long-term outcomes of therapy will be substantially improved when guided by plasma HIV-1 RNA levels, and the precise role of these levels in the clinical management of individual patients has yet to be defined (6). Despite these uncertainties, the measurement of plasma HIV-1 RNA levels has quickly become an accepted clinical practice.

Currently, there are four commercially available assays for the quantitation of HIV-1 RNA: the AMPLICOR HIV-1 Monitor test (Roche Diagnostic Systems, Inc., Branchburg, N.J.), a reverse transcription (RT)-PCR; the Digene Sharp Signal system (Digene Diagnostics, Inc., Silver Spring, Md.), another RT-PCR; the Quantiplex HIV-1 version 2.0 (Chiron Corp., Emeryville, Calif.), a branched-DNA assay (bDNA); and the NASBA-QT (Organon-Teknika Corp., Durham, N.C.), a nucleic acid sequence-based amplification assay (8, 9, 13, 20). Only the AMPLICOR HIV-1 Monitor test has been approved by the U.S. Food and Drug Administration for in vitro diagnostic use, but a variety of assays have been used in multicenter clinical trials of antiviral drugs and to determine the prognostic significance of HIV-1 RNA levels.

The AIDS Clinical Trials Group virology laboratories quality assurance program for quantitation of HIV-1 RNA in plasma demonstrated that 65% of participant laboratories, using different commercial and in-house assays, could attain a level of intrassay precision to reliably detect a fivefold difference in RNA copy number (22). However, few comparative clinical evaluations of these methods have been published, and as a result, the comparability of HIV-1 viral load data generated with the different methods remains an important question (4, 14, 17).

In this study, the performance characteristics of the AMPLICOR HIV-1 Monitor test and the Quantiplex HIV-1 version 2.0 assay were determined in a hospital-based molecular diagnostic laboratory by using baseline plasma specimens from HIV-1-infected patients as well as linearity, reproducibility, and HIV-1 subtype plasma panels.

MATERIALS AND METHODS

HIV-1 stock suspension.

A sucrose-density-gradient-purified and concentrated suspension of HIV-1 SF-2 (subtype B) was obtained from Advanced Biotechnologies, Inc., Columbia, Md. The suspension (lot no. 98-156) contained 2.48 × 10⁹ virus particles/ml, as determined by electron microscopy; 10^6.33 50% tissue culture infective doses (TCID₅₀s)/ml, as determined in HuT78 cells over a 4-week period; and 181,100 ng of HIV-1 p24 antigen/ml, as determined by a p24 antigen capture assay.

Linearity panel.

Serial threefold dilutions of the concentrated viral stock suspension were made in fresh frozen HIV-1-seronegative plasma obtained from the blood bank, and four aliquots of each dilution were frozen at −70°C. A single unit of plasma was used to make all dilutions, each dilution was tested in two separate runs of each assay, and the results were averaged.

Reproducibility panel.

Two dilutions, 1.5 × 10⁶- and 1.5 × 10⁷-fold, of the concentrated viral stock suspension were made in fresh frozen plasma and frozen at −70°C in single-use aliquots. The nominal virus particles per milliliter in the two dilutions were 1,650 and 165. Each sample was tested in triplicate in six different assay runs by two technologists for a total of 18 determinations per sample.

HIV-1 subtype panel.

A panel of HIV-1 isolates, representing subtypes A through F, was prepared as previously described (19). Each of the 11 samples was adjusted to contain a standardized level of virus (approximately 10 pg of p24 antigen). The subtype classification of the panel members was determined by sequence homology analysis of the env region of the genome. Each panel member was tested in duplicate in two different assay runs, and the results were averaged.

Clinical specimens.

Blood from 50 HIV-1-infected patients seen at a large urban AIDS clinic was collected in sterile tubes with EDTA as the anticoagulant. Forty-one (82%) patients were receiving antiretroviral therapy with a variety of drugs at the time of specimen collection. The plasma was removed from the cells within 6 h of collection, frozen at −20°C on site, and transported to the laboratory on ice. In the laboratory, each sample was thawed and aliquoted into four tubes and refrozen at −70°C until tested.

bDNA.

The Quantiplex HIV-1 version 2.0, an enhanced-sensitivity bDNA assay, was performed in accordance with the manufacturer’s instructions (8). The stated linear dynamic range is 500 to 800,000 copies/ml.

RT-PCR.

The AMPLICOR HIV-1 Monitor test, an RT-PCR with an internal quantitation standard, was performed in accordance with the manufacturer’s instructions (13). The stated linear dynamic range is 400 to 750,000 copies/ml.

Statistical analysis.

For the linearity panel, separate linear regression models were used to describe the relationship between the nominal HIV-1 particles per milliliter and each of the assay measurements. The regression analyses were performed on log₁₀ transformed data because of the skewed nature of the raw data. Analysis-of-covariance methods were also used to compare the two assays relative to the nominal level, and post hoc contrasts were used to test the equality of slopes and intercepts.

For the reproducibility panel, means, standard deviations, and coefficients of variation (CV) were used to describe the replicate testing. Separate descriptions were made for the two concentration levels.

For the subtype panel, only the copies per milliliter observed for each sample is reported. No statistical comparisons were made between or within subtypes because of the limited sample size.

For the clinical specimens, means were compared on the log₁₀ transformed data. Similarly, linear regression was used to explore the relationship between the RT-PCR and bDNA measurements. Agreement between the two measurements was assessed by the methods of Bland and Altman (1).

Where statistical comparisons were made, significance was defined by using a type I error rate of 0.05. All reported P values were unadjusted for the number of comparisons made, but conclusions regarding associations were based on adjustment for this multiplicity.

RESULTS

Linearity panel.

Plots of log₁₀ HIV-1 RNA copies versus log₁₀ nominal virus particles per milliliter for the linearity panel are shown in Fig. 1. Separate regression lines were fitted to each assay measurement. The regression analysis results are given in Table 1. The correlations between the assay measurements and the nominal virus particle counts were significant and very close to 1, indicating that both assays gave linear responses over the stated dynamic ranges. The agreements between the assay measurements and the nominal virus particle counts were poor. The range of nominal virus particle counts tested was 2.5 to 5,500/ml. The ranges of RNA copies per milliliter with bDNA and RT-PCR were 900 to 768,000 and 3,360 to 1,880,000, respectively.

FIG. 1.

Line plots of log₁₀ HIV-1 RNA copies per milliliter versus log₁₀ nominal virus particles per milliliter for serial threefold dilutions of an independently quantitated suspension of HIV-1 SF2.

TABLE 1.

Linear relationships between assay measurements and nominal HIV-1 virus particle counts shown in Fig. 1 (log₁₀ transformed data, n = 8).

Assay	Correlation	Intercept	Slope	P (H0: slope = 1)a
bDNA	0.984	2.322	0.965	0.633
RT-PCR	0.995	3.263	0.833	0.003

^aNull hypothesis of slope equal to 1. 

The slope for the bDNA line, 0.965, was closer to an ideal slope of 1 than was the slope for the RT-PCR line, 0.833. This latter slope was significantly different from 1, whereas the slope for the bDNA line was not significantly different from 1. By modeling both assay measurements relative to the nominal level and conservatively assuming independence of observations, the two slopes were not significantly different from each other (P = 0.118). However, the simultaneous test of the lines being the same (same slope and intercept) was rejected (P = 0.001). A plot of the predicted measurements with confidence intervals versus the transformed virus particle counts of the two regression lines showed that at higher values of virus particle count, the errors became smaller.

Reproducibility panel.

To test the reproducibility of each assay, two specimens were tested three times in six separate runs for a total of 18 determinations. The means, standard deviations, and CV for the replicate testing are given in Table 2. The between-run CV for bDNA and RT-PCR were 24.3 and 34.3%, respectively, for the higher-concentration specimen and 44.0 and 42.7%, respectively, for the lower-concentration specimen. The average within-run CV for bDNA and RT-PCR assays were 16.0 and 16.1%, respectively, for the higher-concentration specimen and 29.2 and 23.9%, respectively, for the lower-concentration specimen. We also calculated a standard deviation for the log₁₀ copy number for each set of 18 replicates. The standard deviations for bDNA and RT-PCR were 0.105 and 0.143 log₁₀, respectively, for the higher-concentration specimen, and 0.177 and 0.159 log₁₀, respectively, for the lower-concentration specimen.

TABLE 2.

Between-run reproducibility of bDNA and RT-PCR for quantitation of HIV-1 RNA copies per milliliter at two concentrations^a

Sampleb	Method	Mean	SD	CV (%)
1	bDNA	297,066	72,205	24.3
	RT-PCR	780,826	267,937	34.3
2	bDNA	24,011	10,571	44.0
	RT-PCR	110,843	47,322	42.7

^aEach sample was tested in triplicate in six separate runs for a total of 18 replicates. 

^bSamples 1 and 2 were 1.5 × 10⁶- and 1.5 × 10⁷-fold dilutions, respectively, of an HIV-1 SF2 stock suspension quantitated at 2.48 × 10⁹ virus particles/ml. 

Subtype panel.

The results of quantitating HIV-1 RNA by both bDNA and RT-PCR assays for each subtype panel member are shown in Table 3. HIV-1 subtypes B, C, and D gave higher RNA values by RT-PCR than by bDNA. The ratios of the RT-PCR values to the bDNA values for these subtypes were similar and ranged from 2.19 to 3.63. However, subtypes A, E, and F gave lower RNA values by RT-PCR. The RT-PCR/bDNA value ratios were 0.01 and 0.02 for subtype A panel members, 0.62 and 0.78 for subtype E panel members, and 0.89 and 0.10 for subtype F panel members. In addition, the values across the subtypes for bDNA and RT-PCR differed by as much as 3-fold and 450-fold, respectively. The data suggest that RT-PCR does not detect subtypes A, E, and F with the same efficiency as it detects subtypes B, C, and D since each sample was constructed to contain a standardized level of virus.

TABLE 3.

Comparison of bDNA and RT-PCR for quantitating six different subtypes of HIV-1 in plasma

Sample no.	Subtype	bDNA copies/ml	RT-PCR copies/ml	RT-PCR/bDNA assay value ratio
1	A	71,850	1,446	0.02
2	A	60,105	818	0.01
3	B	49,115	112,398	2.29
4	C	102,660	225,093	2.19
5	C	98,610	358,264	3.63
6	D	36,530	107,559	2.94
7	D	35,575	121,371	3.41
8	E	56,645	34,963	0.62
9	E	59,120	46,381	0.78
10	F	104,425	92,546	0.89
11	F	90,680	9,198	0.10

Clinical specimens.

The number of HIV RNA copies per milliliter was quantitated in 53 of 64 (83%) clinical specimens tested by bDNA and in 55 of 64 (86%) by RT-PCR. The mean values for the 50 specimens that gave discrete values in both assays were 67,580 and 142,419 copies/ml for bDNA and RT-PCR, respectively (P < 0.01). The values ranged from 555 to 410,300 copies/ml with bDNA and from 2,506 to 635,904 copies/ml with RT-PCR. The RT-PCR values were greater than the bDNA values in all but two plasma specimens. Both specimens were retested by RT-PCR; the value for one specimen went from 162,197 to 583,592 copies/ml and the value for the second specimen remained essentially unchanged, 25,402 versus 27,034 copies/ml. The bDNA values for these specimens were 224,500 and 27,080 copies/ml, respectively. There was insufficient volume to retest by bDNA.

Figure 2 shows a scatter plot of log₁₀ HIV-1 RNA copies per milliliter determined by RT-PCR and bDNA for the 50 clinical specimens. The fitted regression line was described by the equation y = 0.733x + 1.627 and r = 0.905. Although the results were highly correlated, both the slope and the y intercept of the regression line differed significantly from those of the line of equality (slope = 1; y intercept = 0), indicating a lack of agreement between the two methods (P < 0.001).

FIG. 2.

Scatter plot of log₁₀ HIV-1 RNA copies per milliliter determined by RT-PCR and bDNA for 50 baseline plasma specimens. The equation for the fitted regression line (solid line) and the correlation coefficient (r) are given in the upper left corner of the figure. The dashed line is the line of equality.

The agreement between the methods was also assessed by plotting the difference between the values of both methods, RT-PCR value minus bDNA value, against their mean by using log₁₀ transformed data (Fig. 3). The mean difference in log₁₀ HIV-1 copies per milliliter was 0.45 with a standard deviation of 0.306. Thus, the limits of agreement, defined as the mean difference ± 2 standard deviations were −0.16 to 1.07. Only 29 (58%) of the results were concordant (difference in log₁₀ copies per milliliter, <0.50) between RT-PCR and bDNA. The results between methods differed by 0.50 to 0.99 log₁₀ for 19 (38%) of the specimens and by ≥1.00 log₁₀ for 2 (4%) of the specimens.

FIG. 3.

Scatter plot of the difference between RT-PCR and bDNA values of log₁₀ HIV-1 RNA copies per milliliter against their means. The mean difference (solid line) ± 2 standard deviations (dashed lines) are shown.

DISCUSSION

The ability to accurately and precisely determine viral load is essential in understanding the natural history of HIV-1 infection, predicting disease progression, and assessing response to antiviral drugs and vaccines. Until recently, viral load determinations were available only in a limited number of research and reference laboratories. The proven clinical utility of viral load testing and the development of several commercially available assays have led to a dramatic increase in the use of these determinations to manage patients. In this study, we compared the performance characteristics of two commercially available assays, bDNA and RT-PCR.

The ability of bDNA and RT-PCR to assign an absolute HIV-1 RNA level cannot be assessed without a common set of standards. We assessed the linear response and proportional systematic error of each assay by plotting the estimated HIV-1 RNA copies per milliliter against the nominal virus particles per milliliter in a dilution series prepared from an independently quantitated viral suspension as surrogate markers of accuracy. The fitted regression lines for each method had r values very close to 1, indicating that both assays gave a linear response over the stated dynamic ranges. If an assay is free of proportional systematic error, then the slope of the regression line should also be 1. The regression lines for the two assays were different, with the slope of the bDNA values essentially being equal to 1. Although the slope of the RT-PCR line over the entire dynamic range did not approximate the ideal slope, RT-PCR appeared to have a better response than bDNA over the lower concentration range (log₁₀ nominal virus particles per milliliter, <1.5). The apparent superiority of RT-PCR for distinguishing small increments in HIV-1 RNA is an important consideration as the manufacturers continue to drive the detection and quantitation limits lower. The errors for both assays became smaller and the regression lines got closer together as the virus particle counts increased.

The agreement between the viral particle counts and RNA copy measurements was poor, with the RNA copies per milliliter always 2 to 3 logs greater than the virus particles per milliliter determined for the same samples. The lack of agreement may be due to the presence of a substantial amount of free viral RNA in the stock virus suspension or to inherent differences in the accuracies of virus particle counting and RNA quantitation methods.

The within-run and between-run precision levels of the two assays were similar to each other and to values previously reported in the literature (22). Our data suggest that bDNA may be more precise than RT-PCR with higher-concentration specimens. The precision levels of the two assays were almost identical at the lower concentration used in the reproducibility panel. It is remarkable that the CV are so similar, considering that each determination by RT-PCR was made with a single test and each determination by bDNA was made by averaging duplicate tests.

HIV-1 evolves by rapid mutation and by recombination, with both processes actively contributing to its genetic diversity (11). The majority of isolates that have been characterized genetically belong to subtypes A through J in the main (M) group, but a less prevalent, highly distinct outlier (O) group has also been described. The subtypes differ in their geographic distribution, with subtype B predominating in Europe and North America. However, in other parts of the world, greater subtype diversity exists.

Several groups have reported that RT-PCR may fail to detect or may detect inefficiently HIV-1 subtypes A, E, F, G, and O (3–5, 10). We used the same subtype panel that was used by Dunne and Crowe (5) in their comparison of bDNA version 1.0 and RT-PCR for quantitation of different subtypes, with essentially the same results. Our results differed from theirs only in that they were unable to detect subtype A by RT-PCR, whereas we found that subtype A was detected by RT-PCR but with 50- to 100-fold less efficiency than that of bDNA.

The differences in the abilities of the two assays to detect the different subtypes of HIV-1 are explained by the oligonucleotide primers and probes used in each assay. RT-PCR uses a pair of primers (SK462/SK431) and a probe (SK102) to produce cDNA from the target RNA, amplify the cDNA, and capture the amplified product DNA. The primers and probe bind to a conserved region of the HIV-1 gag gene based on the sequence of a subtype B isolate, ARV-2 (13). The bDNA assay employs 45 different specific target probes which hybridize to a conserved region of the HIV-1 pol gene and mediate capture of the target RNA and binding of the preamplifier molecules. Each probe includes 33 bases that are complementary to HIV-1 sequences and that were chosen by analyzing the pol sequences from 18 isolates of HIV-1 of subtypes A, B, and D (8).

The abilities of the various commercially available assays to determine the same HIV-1 level in patient samples has been addressed for only a small number of patients (4, 14, 17). The data suggest that there may be variations among the assays in assessing the values for an individual patient but that these variations are not consistent among patients. We determined the HIV RNA levels in 64 plasma specimens from HIV-1-infected individuals by using both RT-PCR and bDNA. We found essentially no difference in the clinical sensitivity of the two assays in our patient population, with 14 to 17% of the samples with RNA copy numbers below the quantitation limit of each assay. All of these samples were obtained from patients who were receiving antiretroviral therapy. Our study was not designed to assess specificity, since these assays should be used only for patients known to be infected with HIV-1. However, as the need to assess ever-lower RNA concentrations increases, the ability of these assays to discriminate low-copy-number specimens from negative specimens needs to be examined in greater detail.

The population mean value with RT-PCR was approximately twofold greater than that with bDNA for the 50 clinical specimens that gave discrete values in both assays, and the RT-PCR value was greater than the bDNA value in all but one specimen. The individual from whom this specimen was obtained acquired his HIV-1 infection in Colombia, South America. Synthetic peptide enzyme immunoassay and nucleic acid sequence analysis performed on this specimen at the Centers for Disease Control and Prevention, Atlanta, Ga., were unable to assign a specific HIV-1 subtype but indicated that it was not subtype B (15a).

The results of the two methods for quantitating HIV-1 RNA in the clinical specimens were highly correlated (r = 0.90), but the agreement was poor. High correlation and poor agreement have been previously reported between bDNA version 1.0 and RT-PCR (4). Although these conclusions may seem contradictory, data which are in poor agreement can be highly correlated (1). The correlation coefficient measures the strength of a relation between two variables, not the agreement between them. In fact, it would be surprising if two methods designed to measure the same quantity were not related.

The agreement between the methods was assessed by plotting the difference between the values of the methods against their means by using log₁₀ transformed data. The mean difference in log₁₀ HIV-1 copies per milliliter was 0.45, but the differences were not consistent among patients. The limits of agreement were defined as the mean difference ± 2 standard deviations and were found to be −0.16 to 1.07 log. Changes in plasma HIV-1 RNA levels of >0.5 log₁₀ are thought to reflect biologically relevant changes in viral replication. Since the limits of agreement between the methods exceeded what is considered a biologically relevant change, we conclude that the assays cannot be used interchangeably. The frequency of concordant results (difference in log₁₀ RNA copies per milliliter, <0.50) for clinical specimens in our study was only 58%. Costa et al. (4) reported a higher concordance of results, 77.5%, in their comparison of bDNA version 1.0 with RT-PCR; however, the previous version of bDNA was not informative for values below 10,000 copies/ml, so comparisons at lower RNA levels were not possible. These findings have significant implications for establishing viral RNA levels that predict the risk of progression to AIDS or that might be used as thresholds to start antiviral therapy (12, 15). The data also provide support for the recommendations that clinicians should use one method to monitor individual patients over time or reestablish a baseline value if the assay method is changed (15, 21).

The assays also differ in their operational efficiencies. bDNA was less labor-intensive than RT-PCR due primarily to a simpler sample preparation method. bDNA also has a higher throughput than RT-PCR. According to the manufacturer, one technologist may generate as many as 42 patient results in an 8-h shift with RT-PCR. In our experience, one technologist can perform as many as 84 patient tests over a 2-day period, devoting approximately 4 h each day to the assay. bDNA is not amenable to testing small batches, owing to the large number of controls and standards that must be run with each batch. The manufacturers of both assays have developed instruments designed to automate the amplification and detection steps. Although these attempts at automation are welcomed, the major labor components for both assays are in the specimen-processing steps.

A number of factors should be considered when selecting an assay for quantitation of HIV-1 RNA. These include accuracy, precision, sensitivity, sample volume, risk of contamination, work space requirements, ease of use, volume of tests, turnaround time, and cost (2, 14). RT-PCR requires 200 μl of plasma, can be completed in a single day, and currently is the only test cleared by the Food and Drug Administration. However, it must be performed in three separate laboratory areas and does not quantitate all HIV-1 subtypes equally. bDNA has essentially the same reportable range as RT-PCR, is slightly more precise, can be performed in a single work area, is less labor-intensive, is amenable to large-batch testing, and quantitates different HIV-1 subtypes with similar efficiencies. However, it is performed in duplicate, requires 2 ml of plasma, and takes 2 days to complete. Both assays are technically demanding and extremely costly. A thorough understanding of each assay’s strengths and limitations will ensure that the laboratories will select the assay that best suits their needs.