Comparison of Three Roche Hepatitis B Virus Viral Load Assay Formats

Abstract

Two FDA-approved (in vitro diagnostic [IVD]) hepatitis B virus (HBV) viral load assays, the manual Cobas TaqMan HBV Test for use with the High Pure System (HP) and the automated Cobas AmpliPrep/Cobas TaqMan HBV Test v2.0 (CAP/CTM), were compared to a modified (not FDA-approved) version of the HP assay by automating the DNA extraction using the Total Nucleic Acid Isolation (TNAI) kit on the Cobas AmpliPrep. On average, CAP/CTM measurements were 0.08 log IU/ml higher than HP results (n = 206), and TNAI results were 0.17 log IU/ml higher than HP results (n = 166). The limit of detection (LOD), as determined by probit analysis using dilutions of the 2nd HBV international standard, was 10.2 IU/ml for CAP/CTM. The data sets for HP and TNAI were insufficient for probit analysis; however, there was 100% detection at ≥5 or ≥10 IU/ml for TNAI and HP, respectively. Linearity was demonstrated between 60 and 2,000,000 IU/ml, with slopes between 0.95 and 0.99 and R² values of >0.99 for all assays. Total precision (log percent coefficient of variance [CV]) was between 0.8% and 2.1% at 4.3 log IU/ml and between 1.4% and 4.9% at 2.3 log IU/ml. Correlation of samples, reproducibility, linearity, and LOD were acceptable and similar in all assays. The CAP/CTM assay and, to a lesser extent, the TNAI assay reduced hands-on time due to automation. There were no instances of contamination detected in negative samples during the course of the study, despite testing several samples up to 9.6 log IU/ml. The incidence of false-positive negative controls in HP and CAP/CTM clinical testing was <0.5% over 6 to 7 months of testing.

INTRODUCTION

Hepatitis B virus (HBV) is a DNA virus that infects up to 400 million people worldwide and causes up to 5,500 deaths annually in the United States from the resulting liver failure, cirrhosis, and hepatocellular carcinoma (5). In the last decade, significant progress has been made in HBV treatment, with the development of new therapeutics (2, 5, 8, 12) with improved genetic barriers and potency. In addition, the techniques for measuring HBV DNA levels in serum and plasma have improved. HBV DNA levels are used to predict response to therapy, to determine therapy initiation, to monitor resistance to therapy, and to establish treatment success (2, 5, 8, 12). Since HBV DNA levels can vary from very low levels to more than 9 log IU/ml, the most recently approved assays that use real-time PCR to generate results over a large dynamic range are preferred. Other available quantitative methods use signal amplification or conventional PCR. The real-time PCR assays typically incorporate robotic automation, making them attractive for high-throughput laboratories.

Roche has released two FDA-approved (in vitro diagnostic [IVD]) HBV real-time PCR viral load assays. The first assay approved was the Cobas TaqMan HBV Test for use with the High Pure System (HP) (11). HP uses a manual 12-well silica-based plate format for DNA extraction, followed by manual inoculation into PCR tubes, which are thermocycled and detected using the TaqMan 48 instrument. Most recently, the Cobas AmpliPrep/Cobas TaqMan HBV Test v2.0 (CAP/CTM) (6) integrates automated sample extraction and PCR assembly using the AmpliPrep instrument with the TaqMan instrument for thermocycling and detection. The transfer of assembled reaction mixtures to the TaqMan can be automated using the optional docking station. For off-label (not approved by the FDA) automation of the HP assay, a general purpose nucleic acid extraction using the Total Nucleic Acid Isolation kit (TNAI) can be performed on the AmpliPrep (10). The TNAI process requires the preparation of a diluted quantitation standard (QS) and its insertion into one of the reagent cassettes. The extracted nucleic acids are placed by the AmpliPrep in screw-top output tubes for manual PCR inoculation.

At the time of the study, ARUP Laboratories, Salt Lake City, Utah (ARUP), used the HP assay for routine testing, so it was used as the reference assay. The HP, CAP/CTM, and TNAI assays were compared by evaluating correlation, reproducibility, limit of detection (LOD), and linearity. Version 2 of the CAP/CTM assay adds serum as an approved sample type (in addition to plasma in version 1) (6, 7). Therefore, the correlation of matrices (serum versus plasma) was evaluated in the CAP/CTM assay. HBV viral loads can be extremely high (>9 log IU/ml); therefore, the possibility of cross-contamination both while preparing samples prior to loading the CAP/CTM system and/or onboard the system leading to false-positive results is of particular concern. To assess the chances of cross-contamination, we performed “checkerboard” testing using high-titer specimens among aliquots of HBV DNA-negative plasma in the HP, TNAI, and CAP/CTM assays. In addition, we compared the rates at which the negative controls were positive during 6 to 7 months of clinical testing involving thousands of samples for both the HP and CAP/CTM assays.

MATERIALS AND METHODS

Correlation samples.

The remaining plasma or serum (and the corresponding viral load results) from 208 clinical specimens submitted to ARUP for HBV viral load testing by HP were deidentified and saved at −20°C.

Limit of detection.

To verify the LOD, 14 replicates of the 2nd WHO International Standard for Hepatitis B Virus DNA Nucleic Acid Amplification Techniques (National Institute for Biological Standards and Control [NIBSC] code 97/750; National Institute for Biological Standards and Control, United Kingdom) at 0, 2.5, 5, 10, 15, 20, and 25 IU/ml were tested in each assay.

Linearity and precision.

Serial dilutions of a high-titer sample were used to determine linearity and precision. The expected values were based on the average of prior duplicate measurements of the sample using the CAP/CTM assay and corrected for the dilution factor. A series of dilutions were prepared with nominal concentrations from 60 to 2,000,000 IU/ml for evaluating linearity and intra-assay precision on day 1, with triplicate measurements. In addition, triplicate measurements were made in each assay at nominal inputs of 2.3 and 4.3 log IU/ml on days 2 to 5 to establish total precision.

International standard.

To test agreement with the 2nd WHO International Standard for Hepatitis B Virus DNA Nucleic Acid Amplification Techniques (NIBSC code 97/750; National Institute for Biological Standards and Control, United Kingdom), an aliquot was resuspended as directed, diluted 1:20 with HBV DNA-negative plasma, and tested in triplicate in each assay. The results were compared to the expected value after correcting for the dilution factor.

Cross-contamination.

Cross-contamination was evaluated for all three assays by testing high-titer (7.7 to 9.6 log IU/ml based on diluted samples tested in HP or CAP/CTM) samples interspersed among aliquots of HBV DNA-negative plasma. The high-titer samples were placed at least every 6th position in the batches of 24 tests (21 samples and 3 controls).

Matrix equivalency.

Five HBV-negative donors provided paired serum and plasma samples, which were spiked at three different levels with a high-titer HBV DNA-positive sample. Each sample was tested in duplicate in the CAP/CTM assay. The mean of the duplicates for serum was compared to the mean of the duplicates from plasma for each sample.

CAP/CTM assay.

The CAP/CTM assay was performed according to the manufacturer's product insert (Roche Molecular Systems, Inc., Branchburg, NJ) using the docked configuration of the Cobas AmpliPrep/Cobas TaqMan system.

HP assay.

The HP assay was performed according to the manufacturer's product insert (Roche Molecular Systems, Inc., Branchburg, NJ).

TNAI assay.

The TNAI assay was performed with the Total Nucleic Acid Isolation kit according to the manufacturer's product insert (Roche Molecular Systems, Inc., Branchburg, NJ). To be consistent with the CAP/CTM and HP assays, the Ampliprep processed 500 μl (650-μl input) of sample. The QS from the TaqMan HBV Test kit was diluted 1:18 in quantitation standard dilution buffer and added to the general purpose vial (GPV) in cassette 3 of the TNAI kit to provide the appropriate amount of QS for each sample. The extracted nucleic acids were inoculated manually into the reaction mixtures using the TaqMan HBV Test kit directions.

Statistical analysis.

Sample results were log transformed for analysis. The correlation of assays was compared by Deming regression and Bland-Altman analysis. Limit of detection probit analysis (95% detection rate) was performed using SAS (version 9.2). An unplanned exploratory analysis of the differences in variance among the assays at nominal inputs of 200 and 20,000 IU/ml was performed using Levene's test. Comparison of the assays at each nominal input was performed as an overall test of the 3 assays at the 0.05 significance level. If the P value was significant for a nominal input, pairwise Levene's tests were performed for the three pairings of assays, using 0.01667 as the significance level to control for multiple comparisons.

Clinical sample data.

ARUP databases were queried for the total and number of contaminated negative controls for HP and CAP/CTM from May 2010 through January 2011 and February 2011 through September 2011, respectively. The distribution of HBV DNA results for samples tested by ARUP between November 2009 and October 2011 was produced for 54,892 clinical samples by querying the ARUP databases.

RESULTS

Two hundred eight archived samples that had quantitative results by the HP assay were used to evaluate assay correlation. In the CAP/CTM assay, 206 samples produced quantitative results; one sample had a clot detected, and another was detected below the limit of quantification (LLOQ) for the CAP/CTM assay (<29 IU/ml) with a result of 58 IU/ml in HP. The data were analyzed by Deming regression (CAP/CTM = 0.978 [HP] + 0.156: R² = 0.981) and Bland and Altman analyses as shown in Fig. 1A. The average difference between the HP and CAP/CTM assays was −0.08 log IU/ml, with an average ±2 standard deviation range of −0.48 to 0.32 log IU/ml. Eighty-four percent of the samples (174/206) were within 0.3 log IU/ml of the HP results, and 99% of the samples (203/206) were within 0.5 log IU/ml of the HP results. After testing with CAP/CTM, 37 samples had insufficient volume for testing in the TNAI assay, leaving 171 available. One sample with results of 36 IU/ml in HP and 72 IU/ml in CAP/CTM was detected for TNAI (<29 IU/ml). Four samples with historic HP results of 51E6, 101E6, 29E6, and 4.5E6 and CAP/CTM results of 60E6, 61E6, 24E6, and 50E6 were detected above the upper limit of quantification (>110,000,000 IU/ml), leaving 166 with results for Deming regression (TNAI = 1.012 [HP] + 0.126; R² = 0.987) and Bland and Altman analysis (Fig. 1B). The average difference between the HP and TNAI assays was −0.17 log IU/ml, with an average ±2 standard deviation range of −0.46 to 0.13 log IU/ml. Eighty-one percent of the samples (134/166) were within 0.3 log IU/ml of the HP results, and 98% of the samples (163/166) were within 0.5 log IU/ml of the HP results.

Fig 1.

Bland-Altman analysis performed for the correlation of HP and CAP/CTM (A) and HP and TNAI (B) results. The difference between the two assays is shown as a function of the average result of the assays for each sample. The average difference and average ±2 standard deviations are shown by the heavy solid lines.

The limits of detection of the assays were evaluated using dilutions of the WHO standard material as shown in Table 1. Probit analysis predicts a limit of detection of 10.2 IU/ml (95% confidence interval = 6.8 to 27.8 IU/ml) for CAP/CTM. The same concentrations of diluted standard had inadequate resolution for probit analysis for HP and TNAI. However, 100% of the samples were detected at ≥10 IU//ml in the HP assay, and 100% of the samples were detected at ≥5 IU/ml in the TNAI assay.

Table 1.

Limits of detection^a

IU/ml	CAP/CTM			HP			TNAI
	No. detected	No. not detected	% Detected	No. detected	No. not detected	% Detected	No. detected	No. not detected	% Detected
0	0	14	0	0	14	0	0	14	0
2.5	8	6	57	10	3	77	11	3	79
5	10	4	71	13	1	93	14	0	100
10	13	1	93	14	0	100	14	0	100
15	14	0	100	14	0	100	14	0	100
20	14	0	100	14	0	100	14	0	100
25	14	0	100	13	0	100	14	0	100

^aProbit results (95% confidence interval) were as follows: CAP/CTM, 10.2 IU/ml (6.8 to 27.8); HP and TNAI, not available.

Linearity was demonstrated between 60 and 2,000,000 IU/ml in each assay, using triplicate measurements of diluted high-titer specimen. The regression statistics for each assay were as follows: y = 0.95x + 0.20, R² = 0.992; y = 0.96x + 0.19, R² = 0.998; and y = 0.99x + 0.23, R² = 0.995 for the CAP/CTM, HP, and TNAI assays, respectively.

The precision of the assays is shown in Table 2. Intra-assay precision (%CV) based on log-transformed results ranged from 0.7% to 9.1% for CAP/CTM, 0.2% to 3.1% for HP, and 0.6% to 7.3% for TNAI. Total precision ranged from 0.8% to 4.9% at nominal inputs of 2.30 and 4.30 log IU/ml. The overall analysis of the differences in variance among the assays was significant at both nominal inputs (P = 0.0149 and 0.0105). Pairwise analyses indicated that CAP/CTM did not differ from HP at either nominal level (P = 0.0987 and 0.1555, respectively), nor did TNAI differ from HP (P = 0.0455 and 0.0399, respectively). Comparisons of CAP/CTM and TNAI did reach significance (P = 0.0155 and 0.0049, respectively).

Table 2.

Intra-assay and total precision

Expected log IU/ml	Mean ± SD (%CV)
	CAP/CTM		HP		TNAI
	Intra-assaya	Totalb	Intra-assaya	Totalb	Intra-assaya	Totalb
6.30	6.16 ± 0.10 (1.7)		6.35 ± 0.04 (0.6)		6.59 ± 0.06 (1.0)
5.30	5.38 ± 0.04 (0.7)		5.26 ± 0.02 (0.4)		5.42 ± 0.03 (0.6)
4.30	4.50 ± 0.15 (3.3)	4.49 ± 0.09 (2.1)c	4.44 ± 0.01 (0.2)	4.40 ± 0.07 (1.6)	4.61 ± 0.05 (1.0)	4.62 ± 0.04 (0.8)
3.30	3.41 ± 0.10 (2.8)		3.57 ± 0.03 (0.9)		3.71 ± 0.12 (3.2)
2.30	2.49 ± 0.22 (9.0)	2.50 ± 0.12 (4.9)	2.51 ± 0.01 (0.3)	2.41 ± 0.07 (3.1)	2.58 ± 0.04 (1.7)	2.60 ± 0.04 (1.4)
2.00	2.16 ± 0.20 (9.1)		2.12 ± 0.03 (1.4)		2.17 ± 0.08 (3.9)
1.78	1.90 ± 0.10 (5.1)		1.94 ± 0.06 (3.1)		2.11 ± 0.16 (7.3)

^an = 3.

^bn = 15.

^cn = 14; one sample was invalid.

Triplicate measurements of a 1:20 dilution of the international standard averaged 49,359 (4.69), 34,880 (4.54), and 53,410 (4.73) IU/ml (log IU/ml) in the CAP/CTM, HP, and TNAI assays, respectively, compared to an expected value of 50,000 IU/ml (4.70 log IU/ml) (Table 3).

Table 3.

WHO international standard (1:20 dilution)

Parameter	Value
	CAP/CTM		HP		TNAI
	IU/ml	Log IU/ml	IU/ml	Log IU/ml	IU/ml	Log IU/ml
Avg (range)	49,359 (45,655–52,673)	4.69 (4.66–4.72)	34,880 (31,975–38,554)	4.54 (4.50–4.59)	53,410 (48,224–58,021)	4.73 (4.68–4.76)
SD	3,525	0.03	3,356	0.04	4,924	0.04
%CV	7.1	0.6	9.6	0.9	9.2	0.8
Difference from expecteda	641	0.01	15,120	0.16	−3,410	−0.03

^aExpected result, 50,000 IU/ml (4.70 log IU/ml).

The difference between paired plasma and serum samples in CAP/CTM was between −0.05 and 0.17 log IU/ml, 0.00 and 0.06 log IU/ml, and 0.03 and 0.28 log IU/ml at inputs of approximately 6, 4, and 2 log IU/ml, respectively (Table 4).

Table 4.

Serum versus plasma in CAP/CTM

Donor	Plasma mean − serum mean (log IU/ml) for input:
	∼6 log IU/ml	∼4 log IU/ml	∼2 log IU/ml
1	−0.01	0.05	0.06
2	0.04	0.05	0.11a
3	−0.05	0.02	0.28
4	0.04	0.05	0.03
5	0.17	0.00	0.26

^an = 1 for serum (one sample failed due to a detected clot).

Cross-contamination was tested by placing high-titer samples at least every 6th position among HBV DNA-negative plasmas on each of three runs of 24 tests (21 samples and 3 controls) for each assay. Despite testing several samples between 7.7 and 9.6 log IU/ml among negative samples, none of the negative plasma samples on the cross-contamination runs had detectable HBV DNA in any assay. There were no negative controls with any assay run that had detectable HBV DNA.

Between May 2010 and January 2011, approximately 21,200 samples were tested using the HP method. There were 6 contaminated negative controls in 1,013 batches (0.59%). During the period between February 2011 and September 2011, approximately 17,600 samples were tested using the CAP/CTM method. There were 4 contaminated negative controls in 840 batches (0.48%).

The distribution of HBV DNA viral loads for 54,892 samples tested at ARUP between November 2009 and October 2011 is shown in Fig. 2. The majority of samples were “not detected” (44.4%) or “detected but not quantifiable” (DNQ) (15.5%). A small proportion of samples were higher than 8 log IU/ml (4.8%), while 35.3% were between 1.3 and 8.0 log IU/ml. The distribution of these samples had a mean of 3.6 log IU/ml and a median of 3.2 log IU/ml. Overall, 8.6% of samples had a viral load of more than 6 log IU/ml.

Fig 2.

Distribution of HBV viral load results for 54,892 clinical samples at ARUP. The pie chart shows the proportions of samples that were not detected, DNQ, and quantitated between 1.3 and 8.0 log IU/ml and at >8 log IU/ml. For the samples between 1.3 and 8.0 log IU/ml, the distribution is shown by the bar chart.

DISCUSSION

The use of real-time PCR techniques has greatly improved the ability to quantify the wide range of HBV DNA concentrations that occur in patients and is the method recommended in the current American Association for the Study of Liver Diseases (AASLD) guidelines (9). These techniques allow greater sensitivity and dynamic range and decrease the potential for cross-contamination by assay-generated amplicons due to the closed-tube nature of the assays. The three Roche assays examined in the study also use dUTP/uracil-N-glycosylase in the master mixture, further decreasing the theoretical risk of contamination due to compromise of the amplification tube.

This study is the first to simultaneously compare the performance of these three assays and corroborates prior reports of the performance of CAP/CTM (3, 4, 6), HP (3, 11), and TNAI (10). Sensitivities were similar, with HP and TNAI slightly more sensitive than CAP/CTM in the study, and the sensitivity of the CAP/CTM assay closely matched previously reported results (6). All assays were well correlated. The average difference between HP and CAP/CTM was −0.08 log₁₀ IU/ml and that between HP and TNAI was −0.17 log₁₀ IU/ml. Ninety-nine percent and 98% of samples were within 0.5 log IU/ml of each other when comparing HP and CAP/CTM or HP and TNAI, respectively. The observed correlations and low bias were expected, since the amplification chemistry is similar for all three assays. Despite being fully automated, CAP/CTM had significantly higher variance than the semiautomated TNAI assay at both tested input levels (2.3 and 4.3 log IU/ml); no significant differences were observed between CAP/CTM and HP or between TNAI and HP assays. The suitability of serum and plasma samples in the CAP/CTM assay was confirmed (1); there was no more than 0.28 log IU/ml difference between the matrices for paired serum/plasma samples spiked with virus between approximately 2 and 6 log IU/ml. All assays produced linear results through their measureable ranges; R² values were >0.99 with slopes of ≥0.95 for all assays. The CAP/CTM assay, and to a lesser extent TNAI, significantly improved throughput and reduced hands-on time due to automated sample preparation (TNAI) and both automated sample and master mixture preparation (CAP/CTM). The HP assay may fill a niche for laboratories where the addition of automated extraction instrumentation is an issue and testing volumes are relatively low. Its requirements for multiple well and tube opening/closing cycles, plate rotations, incubations, reagent additions, centrifugations, and rearraying of samples from two 2- by 6-well extraction plates to one 5- by 5-tube PCR carrier also present significant additional challenges in training and operation, especially with higher test volumes.

Of particular concern for analytes such as HBV DNA that can reach high levels is the chance of cross-contamination during sample handling, nucleic acid extraction, or master mixture assembly. In a recent multilaboratory study comparing four HBV real-time PCR tests, including the HP and CAP/CTM assays tested in our study, Caliendo et al. (3) reported 2/28 (7%) and 4/28 (14%) false-positive results for HBV DNA-negative panel members in the CAP/CTM and HP assays, respectively, indicating a significant rate of contamination. In the current study, despite testing samples with viral loads as high as 3.7E9 IU/ml, there was no evidence of cross-contamination in any assay; no negative controls or aliquots of HBV DNA-negative plasma had detectable results.

During any limited assay validation and study, it is difficult to rigorously test a system's or assay's performance with regard to cross-contamination potential. Additionally, the viral loads of samples used in the study, the study design, personnel training and experience, laboratory workflows and procedures, and other factors can impact the results of such studies. A more realistic measure of cross-contamination for an assay that better accounts for actual laboratory test conditions is its performance over long periods of routine clinical laboratory testing. This study characterized the distribution of HBV viral loads from a broad distribution of clinical samples from throughout the United States. Based on the distribution of viral loads at ARUP shown in Fig. 2, we estimate that an average of 1.8 samples with a viral load of >6 log IU/ml occurs in a batch of 24 tests (21 patients and 3 controls) for the HP and CAP/CTM assays. Examination of 1,853 runs for potential contamination of the negative control therefore provides an important measure of contamination under actual conditions of testing. Over a period of 6 to 7 months of testing with the HP and CAP/CTM assays at ARUP, a very low (∼0.5%) rate of contaminated negative controls was observed for both assays, in which thousands of samples were tested. Although the source of the contamination in these negative controls and in the samples in the study by Caliendo et al. is unknown, possibilities include cross-contamination with high-titer specimens during sample preparation, handling, or extraction or contamination with PCR amplicon. We have no evidence that amplification tubes are ever compromised on these platforms. However, potential amplicon contamination is mitigated to some extent by the inclusion of dUTP/uracil-N-glycosylase in the Roche assays.

While no cross-contamination was evident in this study and the chance appears to be low based on historic clinical data, false-positive results do occur and could have adverse results for patient care. Although the risk of contamination for any molecular infectious disease test has long been recognized, most work has taken the form of analytic challenges, which often do not adequately simulate the conditions of actual laboratory testing either in the titer of the test material or in the duration and conditions of routine clinical testing. Current guidelines for high-titer pathogens such as HBV do not address the potential risk of contamination, much less provide guidance for interpretation of low-titer results that may be confused with false positives. Continued work is warranted to more clearly identify the risks and causes of contamination for this class of testing and to convey to clinicians the small but real risk of contaminants as they consider clinical action based especially on low positive results.