Intra- and Interlaboratory Variabilities of Results Obtained with the Quantiplex Human Immunodeficiency Virus Type 1 RNA bDNA Assay, Version 3.0

Abstract

Normal assay variation associated with bDNA tests for human immunodeficiency virus type 1 (HIV-1) RNA performed at two laboratories with different levels of test experience was investigated. Two 5-ml aliquots of blood in EDTA tubes were collected from each patient for whom the HIV-1 bDNA test was ordered. Blood was stored for no more than 4 h at room temperature prior to plasma separation. Plasma was stored at −70°C until transported to the Central Pennsylvania Alliance Laboratory (CPAL; York, Pa.) and to the Hershey Medical Center (Hershey, Pa.) on dry ice. Samples were stored at ≤−70°C at both laboratories prior to testing. Pools of negative (donor), low-HIV-1-RNA-positive, and high-HIV-1-RNA-positive plasma samples were also repeatedly tested at CPAL to determine both intra- and interrun variation. From 11 August 1999 until 14 September 2000, 448 patient specimens were analyzed in parallel at CPAL and Hershey. From 206 samples with results of ≥1,000 copies/ml at CPAL, 148 (72%) of the results varied by ≤0.20 log₁₀ when tested at Hershey and none varied by >0.50 log₁₀. However, of 242 specimens with results of <1,000 copies/ml at CPAL, 11 (5%) of the results varied by >0.50 log₁₀ when tested at Hershey. Of 38 aliquots of HIV-1 RNA pool negative samples included in 13 CPAL bDNA runs, 37 (97%) gave results of <50 copies/ml and 1 (3%) gave a result of 114 copies/ml. Low-positive HIV-1 RNA pool intrarun variation ranged from 0.06 to 0.26 log₁₀ while the maximum interrun variation was 0.52 log₁₀. High-positive HIV-1 RNA pool intrarun variation ranged from 0.04 to 0.32 log₁₀, while the maximum interrun variation was 0.55 log₁₀. In our patient population, a change in bDNA HIV-1 RNA results of ≤0.50 log₁₀ over time most likely represents normal laboratory test variation. However, a change of >0.50 log₁₀, especially if the results are >1,000 copies/ml, is likely to be significant.

Quantitation of human immunodeficiency virus type 1 (HIV-1) loads in infected patients is routinely being used to determine the risk of disease progression and to assess the short- and long-term response to therapy with antiretroviral drugs (1, 7, 8). Accurate, reliable clinical interpretations of changes in the HIV-1 viral load over time cannot be made, however, without an understanding of the normal biological and laboratory variation associated with the assay selected for use (3, 9). The availability of standardized HIV-1 RNA assays should help to decrease both the inter- and intralaboratory variability of results, an improvement in test performance which is essential for both patient management and clinical trials (3). It has been reported previously that the minimal decrease in HIV-1 RNA copies/ml over time, indicating a favorable response to therapy, is a decrease of greater than 0.5 log₁₀ (threefold) and that changes of less than or equal to 0.5 log₁₀ are more likely related to normal biological and laboratory test variation (1, 7, 18). Normal variation in HIV-1 RNA levels in infants under 1 year of age has been reported to be as high as 0.7 log₁₀ (9).

Accurate and reproducible results of viral load assays may be compromised when suboptimal specimen collection, transporting, and processing procedures are used (3, 7, 9). In addition, selection of a particular assay method may influence the accuracy and reproducibility of results. Results from the Quantiplex HIV-1 RNA bDNA signal amplification assay, version 3.0 (Bayer Corp., Norwood, Mass.), and the Amplicor HIV-1 Monitor reverse transcription (RT)-PCR target amplification assay (Roche Molecular Systems, Branchburg, N.J.) have been found to be highly correlated and in good agreement (4, 6, 10). The bDNA assay, however, has been reported to provide more reproducible results than target amplification assays (7, 10, 11, 15). In addition, between-run reproducibility was better when bDNA (and PCR) testing was performed on samples with high rather than low viral loads (11). For bDNA, PCR, and NASBA (Organon Teknika Corp., Durham, N.C.), reproducibility of duplicate analyses performed in the same assay run were found to be similar to that of duplicates tested in different assay runs (15). Furthermore, the reproducibilities of results within a run were reported for each of these methods to be independent of the HIV RNA concentration (15).

Comparison of RNA copy numbers in plasma samples from local and distant sites in one study showed no significant differences (with the Amplicor Monitor PCR target amplification assay) using samples in EDTA shipped overnight either on dry ice or at ambient temperatures (9). The current study was undertaken to determine the inter- and intrarun variation in one laboratory, as well as the interlaboratory variation of results using the Quantiplex HIV-1 RNA assay. The objective was to provide physicians in our own and other patient populations with data to assist in determining the significance of viral load changes when this assay is used for testing their patients over time.

MATERIALS AND METHODS

Specimen collection and processing.

When the bDNA HIV-1 RNA test was ordered for HIV-1-infected patients, two 5-ml aliquots of blood were collected either at York Hospital or at its local satellite specimen collection centers into lavender-topped K3-EDTA tubes (Becton Dickinson, Cockeysville, Md.). The blood was stored for up to 4 h before separation by centrifugation at 1,000 × g for 15 min. The plasma was immediately frozen at −70°C until aliquots from each patient were sent to both the Central Pennsylvania Alliance Laboratory (CPAL) and the laboratory of the Hershey Medical Center for testing. Transportation to CPAL (2 miles from York Hospital) and to Hershey (approximately 40 miles away) occurred within 4 days of specimen collection, with the specimens on dry ice. Plasma samples at both laboratories were stored at ≤−70°C for up to 7 to 10 days prior to testing. Personnel at each of the laboratories were unaware of the other laboratory's results for each patient until the tests were completed. In addition to testing samples from individual patients, pools of negative plasma from donors and low- and high-HIV-1 RNA-positive plasma from infected patients were created. The pools were stored in aliquots at ≤−70°C and repeatedly tested at CPAL over 6 to 16 weeks.

Specimen testing and data analysis.

The Quantiplex HIV-1 RNA bDNA assay, version 3.0, was performed on individual patient specimens (at both CPAL and Hershey) and on pool samples (at CPAL) using the semiautomated Quantiplex 340 system exactly as specified by the manufacturer and as previously described (4). The manufacturer's threshold of detection of this version of the bDNA assay for HIV-1 RNA is 50 copies/ml, although one study reported that the limit of detection (the lowest HIV-1 RNA concentration that gave positive results in at least 95% of replicate reactions) of the test was 100 copies/ml (5). The upper limit of detection is 500,000 copies of HIV-1 RNA per ml (4). Log₁₀ values were calculated for each result. The log₁₀ value for a result of <50 copies/ml is <1.70. The interlaboratory variation of log₁₀ results from patients' samples and the inter- and intrarun variation of results from the pooled samples tested at CPAL over time were calculated. The z test for differences in proportions for independent samples was used for statistical analysis of results (17). A P value of <0.05 was selected as the minimum level of significance. Prior to the start of the current study, the technologists performing the bDNA assay had only a few months and over 6 years of experience with the method at CPAL and Hershey, respectively.

RESULTS

Interlaboratory variation.

From 11 August 1999 until 14 September 2000, 448 plasma samples from HIV-1-infected patients were tested in 28 bDNA test runs at CPAL. Of the samples tested at CPAL, 206 (46%) and 242 (54%) gave results of ≥1,000 and <1,000 copies of HIV-1 RNA per ml, respectively. Aliquots of each of these samples were also tested using the same technique at Hershey. Of the 448 specimens tested at CPAL, results for 358 (80%) varied by ≤0.20 log₁₀, another 79 (18%) varied by 0.21 to 0.50 log₁₀, and 11 (2%) varied by >0.50 log₁₀ when tested at Hershey (Table 1). However, all 11 of the samples that had results which varied by >0.5 log₁₀ when tested at the two laboratories had results at both facilities of <1,000 copies/ml. The result of one of these patient samples was 513 copies/ml at CPAL and <50 copies/ml at Hershey, for a variation of ≥1.02 log₁₀. In all, results for 9 (82%) of the 11 samples that had a variation of >0.5 log₁₀ when tested at the two facilities were <50 copies/ml at one of the two laboratories and ranged from 169 to 513 copies/ml at the other laboratory. There was no significant shift in the interlaboratory variation during the 13-month duration of the study.

TABLE 1.

Interlaboratory variability of HIV-1 bDNA results from patient plasma samples tested at both the CPAL and Hershey laboratories

Log10 variation of CPAL and Hershey results	No. (%) of samples	HIV-1 RNA (copies/ml) from samples tested at CPAL
		<50	50–999	1,000–9,999	10,000–99,999	100,000–500,000	>500,000
0–0.10	272 (61)	149	29	35	35	13	11
0.11–0.20	86 (19)	3	29	25	25	4
0.21–0.30	42 (9)		11	14	14	2	1
0.31–0.40	26 (6)	1	6	3	11	5
0.41–0.50	11 (2)		3	4	3	1
0.51–0.60	4 (1)	1	3
0.61–0.70	4 (1)	1	3
0.71–0.80	2 (0.4)	1	1
>0.80a	1 (0.2)		1
Total no. (%) of samples	448 (100)	156 (35)	86 (19)	81 (18)	88 (20)	25 (6)	12 (3)

^aThe log₁₀ variation of results from this sample was ≥1.02. 

Of 206 specimens with CPAL results of ≥1,000 copies/ml, 148 (72%) varied by ≤0.20 log₁₀ when aliquots of the same specimens were tested at Hershey, only 8 (4%) varied by >0.40 log₁₀, and none varied by >0.50 log₁₀. However, of the 242 specimens with CPAL bDNA results of <1,000 copies/ml, while 210 (87%) varied by only ≤0.20 log₁₀ when the same specimens were tested at Hershey, 11 (5%) varied by >0.50 log₁₀. This difference in the incidence of the variation of results by >0.50 log₁₀ (0 of 206 samples with CPAL results of ≥1,000 copies/ml versus 11 of 242 samples with CPAL results of <1,000 copies/ml) is significant (P<0.01). The incidence of >0.50 log₁₀ variation was greatest when the CPAL results were in the range of 50 to 999 copies/ml (8 [9%] of 86 specimens) compared to the samples with CPAL results of <50 copies/ml (3 [2%] of 156 specimens) and that difference is also significant (P < 0.05).

Of the 448 patient samples-tested, 156 (35%) had results of <50 copies/ml at CPAL. Of these 156 specimens, 142 (91%) also had Hershey results of <50 copies/ml and 14 (9%) had Hershey results ranging from 53 to 252 copies/ml (maximum variation was ≥0.71 log₁₀). Fourteen additional samples had Hershey results of <50 copies/ml and CPAL results ranging from 51 to 513 copies/ml (maximum variation was ≥1.02 log₁₀). Only 12 samples were associated with CPAL results of >500,000 copies/ml. Ten of these 12 specimens had identical results at Hershey, while the remaining two Hershey results were 276,458 (variation of ≥0.26 log₁₀) and 400,294 (variation of ≥0.10 log₁₀) copies/ml. One additional specimen tested at Hershey had a result of >500,000 copies/ml. The CPAL result for that specimen was 339,342 copies/ml (variation of ≥0.17 log₁₀). Of the 448 patient samples tested, the results from both laboratories were in complete agreement for 152 (142 were <50 copies/ml and 10 were >500,000 copies/ml in both laboratories). Of the remaining 296 samples, results from CPAL were higher for 182 (61.5%) and those from Hershey were higher for 114 (38.5%) (P < 0.001).

Intralaboratory variation. (i) Negative pool.

Of 38 aliquots of an HIV-1-negative plasma pool tested at CPAL in 13 consecutive bDNA assay runs over a 16-week interval, 37 (97%) gave results of <50 copies/ml and one (3%) gave a result of 114 copies/ml. The variation in the latter case was ≥0.37 log₁₀. There were no Quantiplex bDNA assay run failures due to detectable HIV-1 RNA in the kit's negative control.

(ii) Low-positive pool.

The low-positive pool gave results ranging from 1,530 to 4,995 copies/ml when 21 aliquots of the pool were tested in seven assay runs. The within-run variation in results ranged from 0.06 to 0.26 log₁₀ while the maximum between-run variation was 0.52 log₁₀. There was no significant increase or decrease in the low-positive pool results over the 7-week testing interval.

(iii) High-positive pool.

The high-positive pool gave results ranging from 33,572 to 119,490 copies/ml when 23 aliquots of the pool were tested at CPAL in six test runs. The intrarun variation of results ranged from 0.04 to 0.32 log₁₀, while the maximum interrun variation was 0.55 log₁₀. Again, there was no significant increase or decrease in the high-positive pool results over the 6-week interval of testing.

DISCUSSION

In order to maximize the ability to use quantitative HIV-1 RNA assays for patient management over time, it is essential to understand the expected level of normal laboratory and biological variation associated with the tests and the factors that contribute to test result variation. Because of the possibility of low-level false-positive results, these plasma viral load assays should not be used for the routine diagnosis of HIV-1 infections without additional laboratory and clinical information (5, 10, 12, 13, 16). The enzyme-linked immunosorbent assay (ELISA) screen for HIV antibodies followed by Western blot confirmation provides a greater than 99% accuracy for detection of HIV infections but may give negative or indeterminate results during the primary infection for 3 to 4 weeks prior to seroconversion (12, 13). During this window period following infection, initial viremia usually occurs within 4 to 11 days, and the viral load reaches very high levels immediately prior to seroconversion (13). During the first 30 days after infection, the median HIV-1 RNA level was found in one study using bDNA technology to be 235,000 copies/ml, ranging from a low of 27,200 to 1,600,000 copies/ml (14). In the current study, 1 aliquot of 38 tested from an HIV-1-negative plasma pool gave a low false-positive bDNA result of 114 copies/ml. Others have reported that 2 of 32 HIV-1-negative plasma replicates were positive (51 and 71 copies/ml, respectively) (5), and 2 of 100 anti-HIV-negative volunteers had results of 189 and 1,033 copies/ml (10) using bDNA, version 3.0. Another report found false-positive patient results of 1,254 and 1,574 copies/ml with the bDNA assay and 1,300 copies/ml using a RT-PCR assay (13). These false-positive results from uninfected patients are therefore considerably less than 10% of the lowest reported plasma viral load during the interval of seroconversion in newly infected patients (14). False-positive HIV-1 RNA levels are typically at the low end of the range of detection, while during acute infection, the levels of virus are usually quite high (12). There is one report, however, of a false-positive RT-PCR result of 10⁴ to 10⁵ copies/ml from the serum of an HIV-1 vaccine recipient (16). With the bDNA assay, false-positive results may occur at least 2 to 6% of the time due to factors including the non specificity of the assay chemistry (5, 12) as well as specimen misidentifications and laboratory errors (13). If a viral load test has been used for any reason to help establish a diagnosis of HIV infection, low concentrations of the virus, when detected, may result only from laboratory and test variation. The final diagnosis must subsequently be confirmed with clinical data and traditional laboratory assays, including the HIV-1/2 ELISA and Western blot tests, to determine HIV seroconversion (5, 10, 12, 13).

Variation in the viral load of a patient, as determined in the current and previous (5, 9, 15, 18) studies, could be due to variations in the conditions of specimen collection, transport, and storage prior to testing (3, 7–9) as well as the type of assay used and the experience and training of the technologists performing the assays (9, 15, 18). Serum HIV-1 RNA levels are consistently lower than those in plasma (8). Levels in plasma have been 30 to 80% higher than those in serum (1, 2, 9). In the current study, only plasma samples were used. The selection of an anticoagulant is also important. Higher RNA levels have been reported using EDTA (as was used in the current study) with both the RT-PCR assay (3) and the bDNA assay (8), compared to heparin or acid citrate dextrose. A Vacutainer-EDTA tube, as used in the current study, is the preferred blood collection tube for the bDNA assay (3, 8). PPT tubes (Becton-Dickinson, Franklin Lakes, N.J.) efficiently remove platelets, accounting for about a 5% reduction in RNA levels (8).

The time and temperature of storage may also influence the accuracy of an HIV-1 RNA result. The worst condition for stability of the viral load was storage of uncentrifuged blood at room temperature (8). The average rate of HIV-1 RNA loss in whole blood stored for 24 h at room temperature with EDTA was reported to be 0.8%/h with the RT-PCR assay (3). However, the rate of RNA loss in that study was greatest during the first 6 h postcollection (1.8% loss/h in EDTA). Separated plasma was found to have stable RNA titers even after storage at room temperature for at least 24 to 48 h and after repeated freeze-thaw cycles (2, 9). Ideally, blood should be collected in EDTA tubes, plasma separated within 2 to 6 h and then stored at −70 or −80°C until tested (1–3, 8, 9). These conditions of specimen collection, processing, and storage must be the same, as much as possible, from one testing time to another to ensure the accuracy and interpretability of results from the same patients. Collection, transportation, and processing of the plasma samples should be rigidly controlled by the collection and testing sites to minimize temperature and time variations that may aggravate normal testing variation. When possible, patients should be instructed to come to collection sites that have appropriate centrifugation and freezer capabilities onsite. As in the current study, plasma should be shipped to the testing site on dry ice (9).

In the current study, the observed variation in patients' HIV-1 RNA results could not be attributed to the different levels of experience with the Quantiplex bDNA assay at our two laboratories at the start of the study. No significant shift in the interlaboratory variation of results was observed between the first and last months of the 13-month study as the CPAL testing site gained additional experience with the assay. However, higher HIV-1 RNA results were found significantly more frequently at CPAL than at Hershey when the Quantiplex bDNA assay was performed at both sites on aliquots of the same specimens. Despite our attempts to tightly control the time and temperature conditions of specimen transportation, a “lesion of transportation” appears to be the most likely explanation for the frequently higher CPAL results. This interlaboratory variation, however, was rarely greater than 0.5 log₁₀. In previous reports, plasma RNA levels were not significantly influenced by pregnancy status, patient sex, age, use or nonuse of antiretroviral or antimicrobial therapy, multiple freeze-thaw cycles, hemolysis, lipidemia, or elevated bilirubin (2, 3, 9).

In one study using bDNA technology, interlaboratory differences across runs were ≤0.10 log₁₀ at all concentrations of RNA tested (5). The within-run standard deviation of the log₁₀ estimated HIV-1 RNA concentration using the bDNA assay (version 3.0) varied inversely with log₁₀ concentrations below 1,000 copies/ml but did not vary systematically at higher RNA concentrations (5). In another study, the highest interlaboratory difference in viral load result was 0.18 log₁₀, which was not considered clinically relevant (15). Interlaboratory variation has been reported to be greater than interkit variation with both spiked (18) and clinical (15) samples. In the current study, the within-run and between-run variations of the bDNA results obtained from both the low-positive and high-positive pools were similar. From these results as well as the interlaboratory comparison of bDNA results from patients' specimens, we conclude that, in our population of HIV-1-infected patients, a change in bDNA results of ≤0.50 log₁₀ from one sampling time to another very likely represents normal laboratory test variation. Over time, a change in a patient's bDNA results of >0.50 log₁₀, especially if the results are >1,000 copies/ml, is most likely significant. When the change in bDNA results is between 0.51 and 1.00 log₁₀, caution should exercised in interpreting the significance if the results were between <50 and 1,000 copies/ml. An additional plasma sample collected and tested a few weeks or more later may clarify the status of the patient.