Longitudinal Variability of Human Immunodeficiency Virus Type 1 RNA Viral Load Measurements by Nucleic Acid Sequence-Based Amplification and NucliSens Assays in a Large Multicenter Study

Abstract

Human immunodeficiency virus type 1 (HIV-1) RNA measurements were evaluated within an externally controlled multilaboratory program. Three external standards (1.5 × 10³ to 1.5 × 10⁶ copies/ml) were included in 814 assay runs by four laboratories. Results indicate that HIV-1 RNA levels can be measured with a precision equal to that of the pre-highly active antiretroviral therapy era (standard deviations, ±0.16 to 0.25 log₁₀ units).

The Women's Interagency HIV-1 Study (WIHS), a prospective multicenter study established in August 1993 to carry out comprehensive investigations of the impact of human immunodeficiency virus (HIV) infection on women, utilizes nucleic acid sequence-based amplification technology (NASBA) and NucliSens assays (Organon Teknika Corporation [OTC], Durham, N.C.) to quantify HIV type 1 (HIV-1) RNA (1). Decreasing viral loads resulting from the introduction of highly active antiretroviral therapy (HAART) have necessitated the use of more sensitive HIV-1 RNA assays (17; NucliSens: Synergy of Strengths, OTC pamphlet, OTC, 1997; Amplicor HIV-1 Monitor Assay, Roche pamphlet, Roche Diagnostic Corporation, Durham, N.C., 1999).

Conflicting estimates of the contribution of biological and assay variability to the total variance of consecutive HIV-1 RNA measurements have been reported (2, 9, 11). Findings indicate that approximately two-thirds of total variance is due to biological fluctuation and that one third is due to assay variance (2, 5). The increasing number of multilaboratory-based studies has made differences between laboratories another important issue. A study by Skidmore et al. found high interlaboratory variability, indicating a need for an external quality assurance-quality control system for HIV-1 RNA measurements (15).

The WIHS HIV-1 RNA viral load data are unique because of the long observation period and the homogeneity of material used for the external standards. This report clarifies the key issues of laboratory differences and systematic fluctuations over time that affect the nonbiological variability component of HIV-1 RNA viral load measurements.

In the WIHS, four laboratories (all certified by the Virology Quality Assurance Laboratory [VQA], Rush Presbyterian/St. Luke's Medical Center, Chicago, Ill.) performed HIV-1 RNA testing using three versions of OTC assays and a common set of HIV-1 RNA external standards. The standard consisted of HIV-1-negative plasma spiked with HIV-1 virus (4, 5). NASBA assays (0.1-ml input [NASBA_{0.1 ml}]; lower detection limit [LDL], >4,000 copies/ml) were run with sets of 1.5 × 10⁶, 1.5 × 10⁵, and 1.5 × 10⁴ (∼10^6.18, 10^5.18, and 10^4.18, respectively) HIV-1 RNA copies/ml. The corresponding values for standards for both versions of NucliSens (0.2-ml input; LDL, >400 copies/ml; and 1.0-ml input; LDL, >80 copies/ml) were 1 log unit lower. The manufacturer retrained laboratory technicians prior to the distribution of NucliSens with reformulated enzymes (October 1998).

We used the mean square error from a one-way analysis of variance to obtain estimates for the intra-assay variance of the log₁₀ unit of the absolute copy number for each external standard. We used Bartlett's test to compare variances between labs (14). To evaluate systematic fluctuations over time, we fit smooth curves to scatter plots of all log-transformed results for each external standard using the S-plus lowess function (6, 16). P values indicate the statistical significance of the observed nonlinear trends.

Assay format and input volume.

Table 1 shows overall means and standard deviations (SD) of the log-transformed results of the external standards for each assay-input volume combination. Observed variability was lowest for NucliSens_{1.0 ml} and was similar for NASBA_{0.1 ml}.

TABLE 1.

HIV-1 RNA results from each VQA control-OTC assay combination

Test (no. of runs)	Mean ± SD (coefficient of variation) of log10-transformed results for indicated VQA control targetsa
	104.18 copies/ml for NASBA and 103.18 copies/ml for NucliSens (lowest range)	105.18 copies/ml for NASBA and 104.18 copies/ml for NucliSens (middle range)	106.18 copies/ml for NASBA and 105.18 copies/ml for NucliSens (highest range)
NASBA0.1 ml (474)	4.16 ± 0.24 (5.8%)	5.20 ± 0.17b (3.3%)	6.16 ± 0.17b (2.8%)
NucliSens0.2 ml (178)	3.11 ± 0.34 (10.9%)	4.17 ± 0.18 (4.3%)	5.16 ± 0.18 (3.5%)
NucliSens1.0 ml (162)	3.20 ± 0.16 (5.0%)	4.18 ± 0.11 (2.6%)	5.15 ± 0.11 (2.1%)

^aSD were obtained separately for low-range and combined middle- and high-range results from each assay as the mean square errors from one-way analysis of variance. 

^bAdjusted for the laboratory. 

In a retrospective analysis of assay run acceptability, we defined run rejection criteria as (i) two results exceeding the target value by more than 2 SD, (ii) one result exceeding the target value by more than 3 SD, or (iii) an R square of <0.95 (8). Overall rejection rates were 6% for NASBA_{0.1 ml}, 3% for NucliSens_{0.2 ml}, and <1% for NucliSens_{1.0 ml} (data not shown).

Fluctuations in HIV concentrations greater than 0.5 log₁₀ unit have been described as clinically relevant (7, 10, 12, 13). The NASBA_{0.1 ml} result was more than ± 0.5 log unit from the target of 10^4.18 copies/ml for 23 of 474 runs (5%) compared to 0 of 178 runs for NucliSens_{0.2 ml} and 0 of 162 runs for NucliSens_{1.0 ml} (P < 0.0001). NucliSens_{0.2 ml} missed the target of 10^3.18 copies/ml 26 times (15%) compared to 0 times for NucliSens_{1.0 ml} (P < 0.0001).

Laboratory differences.

There were statistically significant differences in levels of variance between labs for the lowest-range standard of all three assay-input volume combinations and for the highest-range standard for NucliSens_{1.0 ml} (data not shown).

Variability over time.

Table 2 summarizes SD for 6-month time intervals corresponding to WIHS visit windows for each assay-input volume combination and VQA external standard. Within each combination and for all standards, the SD was stable and there were no statistically significant trends except with the lowest-range control value for NucliSens_{0.2 ml}.

TABLE 2.

Longitudinal trends in SD of log₁₀ HIV-1 RNA results from each VQA control-OTC assay combination^a

Time interval	NASBA0.1 ml SD for VQA control target:			NucliSens0.2 ml SD for VQA control target:			NucliSens1.0 ml SD for VQA control target:
	104.18	105.18	106.18	103.18	104.18	105.18	103.18	104.18	105.18
April–October 1996	0.29	0.19	0.19		NDb			ND
October 1996–April 1997	0.23	0.15	0.18		ND			ND
April–October 1997	0.22	0.17	0.15		ND			ND
October 1997–April 1998	0.24	0.18	0.15	0.31	0.20	0.15		ND
April–October 1998	0.18	0.17	0.10	0.35	0.17	0.17		ND
October 1998–April 1999		ND		0.36	0.19	0.18	0.19	0.10	0.12
April–October 1999		ND		ND			0.15	0.12	0.09
October 1999–January 2000		ND		ND			0.17	0.10	0.11
Overall	0.24	0.18	0.18	0.34	0.18	0.17	0.16	0.11	0.10

^aControl target values are numbers of copies per milliliter. 

^bND, not determined. This assay was not used during the given time period. 

For NASBA_{0.1 ml} (April 1996 to October 1998), the observed nonlinear trends were statistically significant for all standards (lowest range, P = 0.003 [Fig. 1A]; middle range, P < 0.0001 [Fig. 1B]; highest range, P < 0.0001 [Fig. 1C]). NucliSens_{0.2 ml} (October 1997 to January 1999) demonstrated a marginally significant trend in the highest-range standard (Fig. 1F; P = 0.03). NucliSens_{1.0 ml} (February 1999 to January 2000) demonstrated significant trends in the middle and high-range standards (Fig. 1H, P = 0.003, and I, P = 0.001, respectively).

FIG. 1.

Scatter plots with nonlinear smooth curves of all log-transformed results over the observation period of each assay for each of the three VQA controls (solid line). Each dot represents the result from one run. Nominal target values (dashed lines) and values 2 SD above and below the nominal value (dotted lines) give context to the measurement fluctuations over time. Nonlinear P values indicate the strength (fit) of the association between time and the control result. (C) 1.5 × 10⁶ copies/ml; (B, F, and I) 1.5 × 10⁵ copies/ml; (A, E, and H) 1.5 × 10⁴ copies/ml; (D and G) 1.5 × 10³ copies/ml.

While we observed no gain in precision in the transition from NASBA_{0.1 ml} to Nuclisens_{0.2 ml}, there was a gain in precision and reproducibility for all standards in the transition from Nuclisens_{0.2 ml} to Nuclisens_{1.0 ml}. This was most likely the cumulative result of kit reformulation and increased input volume (M. Cronin, OTC, personal communication, 2000; Introducing a New Family of Assays Based on NASBA, OTC pamphlet, 1997; NucliSens: Synergy of Strengths, OTC pamphlet, 1997).

Theoretically, the observed systematic decrease in results over the 2.5-year observation period for NASBA_{0.1 ml} may be an indication of the deterioration of the virus in the viral stock used for VQA standards over time. However, an internal VQA study to address this possibility found no indication of deterioration of the virus in quality assurance panels (3). The observed increase in the level of HIV-1 RNA over the 1-year observation period for NucliSens_{1.0 ml} also argues against deterioration of the virus in the viral stock. All statistically significant longitudinal trends were well within 2 SD, indicating no biologically relevant bias. We conclude that it is not necessary to make an intra-assay adjustment for time.

We have described the performance over a period spanning the introduction of HAART of three versions of OTC assays within an externally controlled multilaboratory program. NASBA technology is invariant to specimen type, an advantage that makes this assay valuable to clinicians and researchers testing nonplasma body fluids for HIV-1 RNA. Our data demonstrate that the precision of the OTC assays is comparable to that of other assays not invariant to specimen type. We also demonstrated that patient viral load levels in the HAART era can be measured with the same precision as or with better precision than in the pre-HAART era, despite an average HIV-1 RNA decrease of 2 log₁₀ units. We have shown that HIV-1 RNA measurements across multiple WIHS sites over a period of 4 years were comparable despite the introduction of HAART, different versions of the assay, and differences between labs.