Performance Evaluation of the VERSANT HCV RNA Qualitative Assay by Using Transcription-Mediated Amplification

Abstract

A preclinical evaluation of a qualitative assay for the detection of hepatitis C virus (HCV) RNA by transcription-mediated amplification (TMA) was conducted according to the guidelines of the National Committee for Clinical Laboratory Standards and the U.S. Food and Drug Administration. Our results showed that this assay, HCV TMA, detected 95% of samples with HCV RNA concentrations of 5.3 IU/ml and 29 copies/ml. HCV TMA showed an overall specificity of 99.6% and was highly reproducible, detecting 99.3% of samples with HCV RNA concentrations of 50 copies/ml across seven different lots of reagents. Experiments with clinical samples showed that HCV TMA detected all HCV genotypes with similar efficiencies, detecting ≥95% of samples at 50 HCV RNA copies/ml from patients infected with HCV genotypes 1a, 2b, 3a, 4a, 5a, and 6a. In experiments with RNA transcripts, HCV TMA detected ≥96.6% of transcripts derived from HCV genotypes 1a, 1b, 2a, 2c, 3a, 4a, 5a, and 6a at 50 HCV RNA copies/ml. Detection of transcripts derived from HCV genotype 2b was slightly lower (88.4%) at 50 copies/ml but was 97.0% at 75 copies/ml. In addition, HCV TMA exhibited robust performance in detecting HCV RNA in samples subjected to various conditions commonly encountered in a clinical laboratory, including long-term storage, multiple freeze-thaw cycles, different collection tubes, and the presence of endogenous substances, commonly prescribed drugs, or other microorganisms and viruses. With its high sensitivity, specificity, reproducibility, and equivalent genotype reactivity, HCV TMA may provide an attractive alternative for routine qualitative HCV RNA testing in clinical laboratories.

As global population estimates reach 170 million infected with the hepatitis C virus (HCV) (23), there has never been a more pressing need for sensitive, precise tests for active infections. Although enzyme immunoassays (EIA) followed by confirmatory immunoblot assays have been traditionally used for screening and testing of blood, neither assay can differentiate between active and resolved infection. Qualitative and quantitative HCV RNA testing as well as HCV antigen detection methods can identify active infection, but with quantitative tests usually being 1 to 2 logs less sensitive than qualitative tests and with the limited availability of antigen methods, qualitative HCV RNA testing is the method of choice for confirming active infection and assessing viral clearance in response to therapy (8).

Qualitative HCV RNA assays currently used are based on PCR technology and include the AMPLICOR HCV 2.0, COBAS AMPLICOR HCV 2.0, and SuperQual assays. Both the semiautomated AMPLICOR HCV 2.0 and the automated COBAS AMPLICOR HCV 2.0 are commercially available through Roche Molecular Systems, Inc. (Pleasanton, Calif.). The claimed sensitivity at 95% detection for AMPLICOR HCV 2.0 is 100 IU/ml in serum and 50 IU/ml in EDTA plasma (4), and that for COBAS AMPLICOR HCV 2.0 is 100 IU/ml in serum and 60 IU/ml in EDTA plasma (5). The SuperQual assay, developed by the National Genetics Institute (Los Angeles, Calif.) and performed only on location, has a stated limit of detection (LoD) of 100 copies/ml, which is equivalent to approximately 29 IU/ml based on the conversion factor of 3.4 given in their product license application (National Genetics Institute, 1999).

Recently, a new HCV RNA qualitative assay with transcription-mediated amplification (TMA) has been introduced. This assay, designated HCV TMA, is available from Bayer Corporation, Diagnostics Division (Tarrytown, N.Y.) and from its affiliates outside the United States as the VERSANT HCV RNA qualitative assay. Developed and manufactured by Gen-Probe Incorporated (San Diego, Calif.) and marketed by Bayer Corporation, HCV TMA has shown results that may increase the potential benefit for both diagnosis and monitoring of therapy and follow-up (2, 21, 22).

Introduction of a new qualitative nucleic acid assay involves a thorough preclinical evaluation performed by the manufacturer as part of the U.S. Food and Drug Administration (FDA) approval process. In an effort to provide guidance for premarket approval applications for HCV assays, the FDA has released a draft document that outlines current FDA recommendations for assay evaluation (6). The National Committee for Clinical Laboratory Standards (NCCLS) also has developed several guidelines that are relevant to evaluation of qualitative HCV assays, some of which are referenced in the FDA draft document. These documents not only provide guidance for evaluating performance features of the assay such as sensitivity, specificity, and reproducibility but also address features of practical concern to the clinical laboratory such as potentially interfering substances, cross contamination, and sample handling specifications. In this study, we conducted a preclinical evaluation of HCV TMA referring to the FDA draft document (6) and the NCCLS guidelines (13-16).

MATERIALS AND METHODS

Dilution panels for LoD.

Stock solutions of HCV genotype 1a were diluted into HCV-negative serum or EDTA plasma to concentrations of 25, 50, and 75 HCV RNA copies/ml. The HCV stock solutions were value assigned by using the quantitative VERSANT HCV RNA 3.0 assay (bDNA) (Bayer Corporation, Diagnostics Division), which is based on branched DNA (bDNA) technology. Multiple replicates of each panel member were tested with seven different lots of reagents. The LoD with the value-assigned viral panel was determined by logistic regression.

The World Health Organization (WHO) International Standard for HCV RNA code 96/790 obtained from the National Institute for Biological Standards and Control (NIBSC; Potters Bar, Hertfordshire, United Kingdom) was reconstituted in 0.5 ml of distilled water per the package insert to yield a solution containing 10⁵ IU/ml (20). This solution was diluted into HCV-negative serum to concentrations of 1, 2.5, 5, 7.5, 10, 18.5, and 50 IU/ml. Multiple replicates of each dilution were tested with three different reagent lots. The LoD determined with the WHO standard dilution panel was calculated by logistic regression.

To facilitate comparison between results reported in copies per milliliter and results reported in international units per milliliter, we used a conversion factor of 5.2 copies/IU. This conversion factor is used for converting VERSANT HCV RNA 3.0 assay (bDNA) values from copies per milliliter to international units per milliliter and was derived from testing the WHO International Standard for HCV RNA (NIBSC code 96/790) with an RNA transcript reference standard in place of the kit standards. In internal studies performed at Bayer Corporation, the 5.2 conversion factor was first estimated by testing a six-member dilution panel of the WHO standard with four manufacturing lots in a total of 36 runs and 144 replicates per panel member tested and then confirmed by testing the six-member panel with three manufacturing lots in a total of 27 runs and 108 replicates per panel member tested (Bayer Corporation, internal communication).

Specimens for specificity studies.

Specificity was determined with 1,000 serum samples and 1,504 EDTA plasma samples from HCV-negative volunteer blood donors. The HCV antibody-negative serum samples were obtained from ProMedDx (Plainville, Mass.). The EDTA plasma specimens were collected from anonymous donors with informed consent and shown to be HCV antibody negative by the HCV 2.0 assay (Abbott Laboratories, Chicago, Ill.). A total of 4 of 1,000 serum samples and 18 of 1,504 EDTA plasma samples that initially gave invalid results were retested. All specimens with invalid results yielded nonreactive results on retest. In addition, all samples that gave reactive results were retested. Only valid results from valid runs were included in the specificity analysis. Any specimen giving a reactive result upon initial testing was considered to be a false positive. Assay specificity was calculated with the equation specificity = true negatives/(true negatives + false positives) × 100. The sample sizes used in this study provided 95% confidence that the true specificity was above 99%.

HCV genotype reactivity.

RNA transcripts representing HCV genotypes 1a, 1b, 2a, 2b, 2c, 3a, 4a, 5a, and 6a were used to assess the ability of HCV TMA to detect all major HCV genotypes. RNA transcripts were prepared by standard methods as described by Collins et al. (1) and Detmer et al. (3) and characterized with regard to size and integrity by agarose gel electrophoresis. Preparation includes a DNase step that effectively removes template DNA. RNA transcript preparations were independently quantified by phosphate analysis with the National Institute of Standards and Technology phosphate standard and confirmed by optical density readings at 260 nm and hyperchromicity. After value assignment, transcripts were diluted gravimetrically to 50 or 75 copies/ml into target capture reagent. HCV-negative serum was added to simulate a patient sample in approximately half of the replicates. A minimum of 60 and a maximum of 710 replicates from each HCV subtype were tested with four different lots of reagents.

The HCV genotype reactivity of HCV TMA was further analyzed by testing panels prepared from serum or plasma samples of patients infected with HCV genotypes 1 to 6. Samples from patients infected with HCV genotypes 1a, 2b, 3a, and 6a were obtained from ProMedDx, and samples from patients infected with HCV genotypes 4/4a and 5a were obtained from Millennium Biotech (Fort Lauderdale, Fla.). HCV genotypes were confirmed with the VERSANT HCV LiPA assay (Bayer Corporation, Diagnostics Division) and also by sequencing at Sequetech (Mountain View, Calif.). Viral load in patient samples was measured with VERSANT HCV RNA 3.0 assay (bDNA), and samples were diluted to 50 or 75 copies/ml in normal serum.

Reproducibility.

The reproducibility of HCV TMA was assessed by testing a six-member viral panel that included HCV RNA concentrations at the design goal cutoff as well as above and below the cutoff over the course of 6 days. The six-member viral panel was comprised of four serum members with viral concentrations of 0, 25, 50, and 75 copies/ml and two EDTA plasma members with viral concentrations of 0 and 50 copies/ml. The panel was created by adding HCV genotype 1a-positive donor serum to HCV antibody-negative serum or EDTA plasma specimen pools from healthy blood donors. The amount of HCV RNA in the HCV genotype 1a-positive donor serum was quantified with VERSANT HCV RNA 3.0 assay (bDNA). The reproducibility study was conducted by three operators using three different lots of reagents and three luminometers. Each operator performed two runs per day on each of two separate days using a different luminometer each day with each lot of reagent, and each panel was assayed six times per run. Reproducibility was assessed by estimating the variance in signal-to-cutoff (S/CO) ratios due to operator, lot, luminometer, and between-run and within-run effects by nominal concentration of HCV RNA.

Specimen stability studies.

The stability of HCV RNA in clinical specimens from the Sacramento Blood Bank as detected with HCV TMA was evaluated under several sample processing and storage conditions, including storage of whole blood at room temperature, storage of decanted serum and plasma at 2 to 8°C, long-term storage at −20 and −80°C, and exposure to multiple freeze-thaw cycles.

To assess HCV RNA stability in whole blood at room temperature, blood was collected from four anonymous healthy donors with informed consent into each of the following Vacutainer tubes from Becton Dickinson (Franklin Lakes, N.J.): serum separator tube (SST) (PLUS, plastic), K₂-EDTA (PLUS, plastic), K₂-EDTA (plasma preparation tube), sodium citrate (glass, 4%), Acid citrate dextrose (ACD) solution A (glass), and sodium heparin (PLUS, plastic, 60 USP units). Immediately after blood collection, half of the tubes from each donor were spiked with an HCV genotype 1a sample to a final concentration of 50 copies/ml. To ensure a consistent final HCV concentration, all HCV-spiked tubes were filled to the specified draw volume. After tubes were allowed to stand at room temperature for 4, 6, or 24 h, they were centrifuged according to the manufacturer's instructions and the serum or plasma was removed. All samples were then stored at 2 to 8°C for 48 h, after which they were frozen at −20°C until they were tested. For each time point, three replicates of HCV-spiked samples and three replicates of unspiked samples from each of the four donors were tested.

To assess HCV RNA stability in decanted serum and plasma at 2 to 8°C, blood was collected from 30 healthy donors into the six Vacutainer tube types described above. Immediately after blood collection, tubes from each donor were spiked with an HCV genotype 1a sample to a final concentration of 50 copies/ml. Each tube was centrifuged within 4 h of collection, and the decanted serum or plasma was stored at 2 to 8°C for 0, 8, 24, or 48 h. All samples were frozen prior to testing. The number of replicates of specimens tested for each tube type at each time point ranged from 40 to 60.

The stability of HCV RNA in clinical specimens subjected to long-term storage at −20 and −80°C was tested with serum samples from 11 anonymous HCV RNA-positive donors (8 of genotype 1a and 3 of genotype 2b) from the Sacramento Blood Bank. Samples were diluted to 50 copies/ml in normal serum and stored at −20 or −80°C for 4, 6, 7, 8, and 13 months. Samples for each time point were tested by HCV TMA in 56 to 86 replicates.

The stability of HCV RNA in serum samples subjected to multiple freeze-thaw cycles was tested with serum samples from 11 HCV RNA-positive donors (8 of genotype 1a and 3 of genotype 2b). Samples were first tested for viral load by the quantitative VERSANT HCV RNA 3.0 assay (bDNA), then diluted to 50 copies/ml in normal serum, and frozen in aliquots at −60 to −80°C. For each of the three freeze-thaw cycles performed, samples were thawed on ice for at least 1 h and then refrozen.

Specimens for interference studies.

For interference studies, serum and plasma specimens were obtained from subjects with defined medical conditions, including autoimmune hepatitis, alcohol- or drug-related liver disease, primary biliary cirrhosis, anti-nuclear antibody (ANA)-positive status, myeloma, anti-double-stranded DNA (anti-dsDNA)-positive status, rheumatoid factor-positive status, and systemic lupus erythematosus. Specimens were collected from various sites under institutional review board approval or from sites with exempted protocols. All specimens were shown to be HCV antibody negative by two FDA-approved assays: HCV EIA 3.0 assay (Ortho Diagnostics, Raritan, N.J.) and HCV SIA 3.0 RIBA assay (Chiron Corporation, Emeryville, Calif.). Samples were stored and shipped frozen prior to HCV RNA testing.

To test for potential interference by endogenous substances, 10 HCV-negative serum specimens from healthy blood donors from the Sacramento Blood Bank that had triglyceride levels less than 200 mg/dl and had no visible signs of hemolysis or icterus were used. Individual specimens were thawed and filtered with sterile cheesecloth, and antimicrobial agents were added (final concentrations, 0.05% sodium azide and 0.05% gentamicin). Each specimen was divided into two aliquots. One aliquot served as the HCV RNA-negative control, and HCV genotype 1a-positive serum was added to the other aliquot to a final concentration of 50 copies/ml. The resulting HCV RNA-negative and HCV RNA-positive samples were further divided, and aliquots were spiked with potentially interfering endogenous substances at levels recommended in the NCCLS guidelines (13). Final added concentrations were 500 mg of hemoglobin/dl, 60 mg of conjugated bilirubin/dl, 60 mg of unconjugated bilirubin/dl, 3,000 mg of triglyceride standard (Liposyn, 20%; Abbott Laboratories)/dl, and 8 g of human immunoglobulin/dl. All specimens were frozen within 4 h to minimize differences due to specimen processing. All aliquots of each HCV RNA-negative or -positive specimen with and without added endogenous substances were tested in the same run.

Potential interference by antiviral or immunosuppressant drugs commonly prescribed to treat HCV or other viral infections or to prevent rejection of liver transplants also was evaluated. Aliquots of HCV RNA-positive and HCV RNA-negative specimens were prepared as described above and spiked with combinations of drugs at five times the mean steady-state level in serum or plasma. Drug pool 1 contained 1,500 IU of alfa interferon 2b (Intron A; Schering-Plough, Kenilworth, N.J.)/ml, 11 μg of ribavirin (Schering-Plough)/ml, and 5 μg of azathioprine (Sigma, St. Louis, Mo.)/ml. Drug pool 2 contained 0.625 μg of cyclosporine (Sigma)/ml, 2 μg of spironolactone (Aldactone; Sigma)/ml, and 85 μg of prednisone (Sigma)/ml. Drug pool 3 contained 101 ng of alfa interferon 2a (Roferon-A; Hoffman-La Roche Inc., Nutley, N.J.)/ml, 0.15 μg of tacrolimus (Fujisawa, Deerfield, Ill.)/ml, and 1.1 μg of amantadine HCl (Sigma)/ml. Drug pool 4 contained 1,510 ng of fluoxetine HCl (Sigma)/ml, 1,600 pg of pegulated interferon (Peginterferon Alfa-2b; Sigma)/ml, and 5.3 μg of azidothymidine (Sigma)/ml. Drug pool 5 contained 5.9 μg of ganciclovir (Sigma)/ml and 126 ng of dideoxycytidine (Sigma)/ml. Drug pool 6 contained 11.6 μg of didanosine (Sigma)/ml and 20.75 μg of didehydrodeoxythymidine (Sigma)/ml. All aliquots of each HCV RNA-negative or -positive specimen with and without added drug combinations were tested in the same run.

The effect of microorganisms and coinfecting viruses that may be found in the blood of individuals with HCV infection also was tested. Aliquots of HCV RNA-positive and HCV RNA-negative specimens were prepared as described above and spiked with combinations of microorganisms and viruses in five pools as described in the footnotes to Table 7. Each bacterium was tested at a final concentration of 5 × 10⁴ CFU/ml. Human immunodeficiency virus type 1 (HIV-1) was tested at a concentration of 5 × 10⁴ RNA copies/ml. Hepatitis B virus and cytomegalovirus (Towne) were each tested at a final concentration of 5 × 10⁴ DNA copies/ml.

TABLE 7.

Potential interference by endogenous substances, microorganisms and other viruses, and commonly prescribed drugs

Potentially interfering substance, microorganism pool, or drug poola	Specificityb (%)	Sensitivityc
Endogenous substances (level tested)
Hemoglobin (500 mg/dl)	98.0	100
Conjugated bilirubin (60 mg/dl)	98.0	98.0
Unconjugated bilirubin (60 mg/dl)	100	96.0
Triglycerides (3,000 mg/dl)	100	98.0
Human immunoglobulin (8 g/dl)d	100	100e
Microorganisms and viruses
Pool 1	98.0	96.0
Pool 2	100	100
Pool 3	100	100
Pool 4	100	100
Pool 5	100	100
Commonly prescribed drugs
Pool 1	100	98.0
Pool 2	100	100
Pool 3	100	98.0
Pool 4	100	100
Pool 5	100	100
Pool 6	100	100

^aFor the microorganisms pools, the testing was done at 50,000 CFU/ml or 50,000 copies/ml. The microorganism pools included the following: pool 1, Escherichia coli, Pseudomonas aeruginosa, Klebsiella pneumoniae, Haemophilus influenzae, and cytomegolovirus; pool 2, Enterobacter cloaceae, Pseudomonas fluorescens, Staphylococcus aureus, Serratia marcescens, and Streptococcus pneumoniae; pool 3, Staphylococcus epidermis, Streptococcus group B, Candida albicans, HIV-1 subtype B, and hepatitis B virus; pool 4, HIV-1 subtypes A, C, and D; and pool 5, HIV-1 subtypes E and F and group O, and Propionibacterium acnes. The drug pools included the following: pool 1, alpha interferon 2b, ribavarin, and azathioprine; pool 2, cyclosporine, spironolactone, and prednisone; pool 3, alpha interferon 2b, tacrolimus, and amantidine HCl; pool 4, fluoxetine HCl, Peginterferon Alfa-2b, and azidothymidine; pool 5, ganciclovir and dideoxycytidine; and pool 6, didanosine and dedehydreoxythymidine.

^bSpecificity defined as the percentage of nonreactive samples of total samples tested. A total of 50 samples were tested for each endogenous substance and disease-causing organism pool. A total of 50 samples were tested for each of drug pools 1 and 3, 60 samples were tested for drug pool 2, and 40 samples were tested for drug pools 4 to 6.

^cSensitivity defined as the percentage of reactive samples of a total of 50 samples tested.

^dNormal human serum contains 4 to 8 g of immunoglobulin per dl. With an additional 8 g/dl added, this would bring the total concentration to 12 to 16 g of immunoglobulin per dl.

^eOnly 49 samples tested.

Cross-contamination study.

HCV TMA was evaluated for cross-contamination potential by testing replicates of a high-titered HCV-positive specimen (10⁶ HCV RNA copies/ml) alternated with replicates of an HCV-negative specimen in a checkerboard pattern. Forty-five replicates each of HCV-negative and HCV-positive samples were tested per run. Five runs were performed, totaling 225 replicates of the HCV-positive specimen and 225 replicates of the HCV-negative specimen for the entire study.

HCV TMA.

HCV TMA was performed with kits provided by Bayer Corporation according to the instructions of the manufacturer. The assay consists of three steps that are performed in a single tube: target capture, target amplification by isothermal TMA, and detection of target amplicons by the hybridization protection assay and the dual kinetic assay (7, 12, 17, 22). Each assay run includes three replicates each of a negative calibrator (defibrinated normal human plasma with gentamicin and sodium azide) and a positive calibrator (inactivated HCV-positive plasma in defibrinated normal human plasma with gentamicin and sodium azide). Also, an internal control is added to each specimen and calibrator with the target capture reagent to monitor assay performance. The internal control is an RNA transcript used to monitor all assay steps, including target capture, amplification, and detection (11). A positive internal control signal in samples or calibrators that do not contain HCV indicates that all assay steps were performed correctly and that inhibition did not occur.

To begin, 1.0 ml of internal control reagent (RNA transcript in HEPES buffer with detergent) was added to the 50-ml container of target capture reagent (capture oligonucleotides and magnetic microparticles in HEPES buffer with detergent) and mixed thoroughly. Then, 400 μl of target capture reagent with internal control was added to each polypropylene tube (10-tube units) followed by 500 μl of sample or calibrator, and the mixture was vortexed and incubated at 60°C for 20 min and then incubated at room temperature for 15 min. After the magnetic microparticles were washed twice with wash buffer (HEPES buffer with detergent), 75 μl of amplification reagent (primers, deoxynucleoside triphosphates, nucleoside triphosphates, and cofactors in Tris buffer) and 200 μl of oil reagent (100% silicone oil) were added to each tube, and the mixture was incubated at 60°C for 10 min and then at 41.5°C for 10 min. To produce RNA amplicons, 25 μl of enzyme reagent (Moloney murine leukemia virus reverse transcriptase and T7 RNA polymerase in HEPES-Tris buffer) was added to each tube, and the mixture was incubated at 41.5°C for 1 h. For detection by hybridization protection assay, 100 μl of HCV probe reagent (chemiluminescent oligonucleotide probe in succinate buffer with detergent) was added to each tube, and the mixture was incubated at 60°C for 15 min. Then, 250 μl of selection reagent (borate buffer with surfactant) was added, and the mixture was incubated at 60°C for 10 min. After the tubes were allowed to cool at 19 to 27°C for at least 10 min, the chemiluminescent signal for each sample and calibrator was read in a Leader HC or Leader HC+ luminometer. The data reduction software automatically measures the analyte and internal control signal for each sample and reports them as both relative light units (RLU) and S/CO ratios. When the S/CO ratio was ≥1, the specimen was considered reactive or having detectable HCV RNA.

To be considered valid, positive calibrators were required to have an analyte signal between 400,000 and 2,700,000 RLU, inclusive, and an internal control signal of ≤475,000 RLU. Also, negative calibrators were required to have an analyte signal between 0 and 40,000 RLU, inclusive, and an internal control signal between 75,000 and 300,000 RLU, inclusive. Assay runs were considered valid when two or more of the positive calibrators and two or more of the negative calibrators were valid. A sample result was considered to be valid and nonreactive when the sample generated an analyte signal less than the analyte cutoff and an internal control signal greater than or equal to the internal control cutoff. A sample result was considered to be valid and reactive when the sample generated an analyte signal greater than or equal to the analyte cutoff and an internal control signal of ≤475,000 RLU. The cutoff value for the internal control is positioned to minimize the number of invalid results and to prevent the occurrence of false-negative results and is calculated as follows: internal control cutoff is equal to 50% of the mean of the negative calibrator values for the internal control RLU. The analyte cutoff values are determined as follows: analyte cutoff is equal to the mean of the negative calibrator values for the analyte RLU plus 4% of the mean of the positive calibrator values for the analyte RLU.

Reference methods.

Specimens were tested for HCV antibodies by the Chiron HCV SIA 3.0 RIBA assay, the Abbott HCV 2.0 assay, and the Ortho HCV EIA 3.0 assay. Unless otherwise noted, HCV RNA was quantified by VERSANT HCV RNA 3.0 assay (bDNA). HCV RNA was measured in selected samples with AMPLICOR HCV 2.0. HCV genotype was determined in selected samples with VERSANT HCV LiPA. All assays were performed according to manufacturers' instructions.

RESULTS

Sensitivity.

The sensitivity of HCV TMA was evaluated with panels prepared from HCV genotype 1a stock solutions diluted into HCV-negative serum or EDTA plasma. This panel included multiple replicates at HCV RNA concentrations of 25, 50, and 75 copies/ml. Figure 1A shows a logistic regression analysis of the percentage of samples in which HCV RNA was detected as a function of HCV RNA concentration in copies per milliliter. The dotted lines represent the upper and lower 95% confidence limits (CL). The LoD was defined as the concentration at which HCV RNA was detected 95% of the time and was calculated to be 29.1 HCV RNA copies/ml (95% CL = <25 to 32.8 HCV RNA copies/ml). With the conversion factor of 5.2 copies/IU (described in Materials and Methods), the LoD of 29.1 HCV RNA copies/ml is equivalent to 5.6 HCV RNA IU/ml. To further evaluate the sensitivity of HCV TMA, a panel was prepared by diluting the WHO International Standard for HCV RNA (NIBSC code 96/790) into normal serum. This panel included multiple replicates at HCV RNA concentrations of 0.1, 1, 2.5, 5, 7.5, 10, 18.5, and 50 IU/ml. A logistic regression analysis of the percentage of samples in which HCV RNA was detected as a function of HCV RNA concentration in IU/ml is shown in Fig. 1B. In this analysis the LoD was calculated to be 5.3 HCV RNA IU/ml (CL = 4.9 to 6.2 HCV RNA IU/ml).

FIG. 1.

(A) Sensitivity of HCV TMA as a function of HCV RNA concentration in copies per milliliter. The dashed lines represent the upper and lower 95% CL. The LoD was calculated to be 29.1 HCV RNA copies/ml and is shown as the x coordinate where the calculated curve intercepts the 95% detection line. (B) Sensitivity of HCV TMA as a function of HCV RNA concentration in international units per milliliter. The dashed lines represent the upper and lower 95% CL. The LoD was calculated to be 5.3 IU/ml and is shown as the x coordinate where the calculated curve intercepts the 95% detection line.

Specificity.

The specificity of HCV TMA was determined by testing HCV-negative serum and EDTA plasma samples, and the results are shown in Table 1. The data were combined following Fisher's exact test for equality of proportions, and no difference in specificity was observed between serum and EDTA plasma (P = 0.5). The overall specificity was 99.6% (lower CL = 99.4%). Upon retesting, four of the five serum samples that initially gave reactive results were nonreactive and one of five remained reactive. Initial S/CO values for these serum samples were 2.14, 2.56, 1.01, 1.05, and 7.74; retest S/CO values for these samples were 0.27, 0.06, 0.06, 0.17, and 9.66, respectively. Of the four EDTA plasma samples that initially gave reactive results, three were nonreactive and one was reactive upon retesting. Initial S/CO values for these samples were 1.44, 1.57, 1.33, and 4.25, while retest S/CO values were 0.27, 0.08, 0.17, and 4.17.

TABLE 1.

Specificity of HCV TMA

Specimen type	No. tested	No. nonreactive	No. reactive	Specificity (%)
Serum	1,000	995	5a	99.5
EDTA plasma	1,504	1,500	4b	99.7
Combined	2,504	2,495	9	99.6

^aUpon retest, four of five initially reactive serum specimens were nonreactive.

^bUpon retest, three of four initially reactive EDTA plasma specimens were nonreactive.

HCV genotype reactivity.

To verify the sensitivity of HCV TMA in detecting all major HCV genotypes, the assay was used to test RNA transcripts representing HCV genotypes 1a, 1b, 2a, 2b, 2c, 3a, 4a, 5a, and 6a. As shown in Table 2, HCV TMA detected greater than 95% of transcript replicates representing all six HCV genotypes. For HCV genotype 2 at an input of 50 copies/ml, HCV TMA detected 98.8% of genotype 2a and 96.6% of genotype 2c transcripts but only 88.4% of genotype 2b transcripts (P < 0.001). Additional analysis of genotype 2b transcripts at 75 copies/ml showed 97.0% detection. Using interpolation analysis of these data, we estimate that the LoD for genotype 2b is approximately 69 copies/ml.

TABLE 2.

HCV genotype reactivity of HCV TMA

Specimen tested and HCV subtype	HCV RNA (copies/ml)	No. of valid specimens	% Reactive
RNA transcripts
1a	50	502	99.8
1b	50	449	99.1
2a	50	415	98.8
2b	50	138	88.4
2b	75	656	97.0
2c	50	59	96.6
3a	50	448	98.7
4a	50	448	99.6
5a	50	443	98.2
6a	50	708	97.2
Viral panels
1a	50	504	99.2
2b	50	210	95.2
2b	75	176	98.3
3a	50	120	100
4/4a	50	120	95.0
5a	50	118	100
6a	50	117	97.4

The HCV genotype reactivity of HCV TMA was further analyzed by testing panels prepared from plasma samples of patients infected with HCV genotypes 1 through 6 (Table 2). HCV TMA detected ≥95% of samples of HCV subtypes 1a, 2b, 3a, 4/4a, 5a, and 6a at 50 copies/ml and 98.3% of HCV subtype 2b samples at 75 copies/ml.

Reproducibility.

The reproducibility of HCV TMA was assessed by testing of a six-member viral panel of HCV genotype 1a by several operators using different lots of reagents and luminometers. The objective of our reproducibility study was to confirm that the variance of the S/CO ratios was reasonable and that the assay could reliably detect samples at the target level of 50 HCV RNA copies/ml at least 95% of the time across experimental conditions. Table 3 shows the variance in S/CO ratios determined for each of the components expressed as a standard deviation (SD). Very little variance was associated with reagent lots (SD, 0.00 to 1.03), operators (SD, 0.00 to 0.36), and luminometers (SD, 0.00 to 0.82). The variance between runs was also relatively low (SD, 0.00 to 2.09), whereas most of the variance was observed within runs (SD, 0.30 to 6.58). Reproducibility of the assay was also assessed with a viral panel of three concentrations of HCV genotype 2b with similar results (data not shown). As with the HCV genotype 1a panel, the highest variance was within run.

TABLE 3.

Reproducibility of HCV TMA as assessed by variance components

HCV RNA (copies/ml)	SD estimates for variance components					Overall SD	Mean S/CO ratio
	Lot	Operator	Luminometer	Between run	Within run
0	0.00	0.05	0.08	0.11	0.30	0.34	0.20
25	0.00	0.00	0.00	0.00	6.58	6.58	10.84
50	1.03	0.29	0.38	2.09	5.53	6.02	14.69
75	0.00	0.36	0.82	1.61	4.25	4.63	16.13

A further evaluation of reproducibility was performed by comparing the detection rate at 50 copies/ml for multiple lots of reagents. As shown in Table 4, HCV TMA reproducibly detected HCV RNA at 50 copies/ml across seven different lots of reagents (mean, 99.3%; range, 97.2 to 99.7%).

TABLE 4.

Reproducibility of HCV TMA as assessed by percent detection across kit lots

Kit lot	No. tested	No. reactive	% Detected
A	142	139	97.9
B	144	140	97.2
C	961	947	98.5
D	959	957	99.8
E	1,148	1,145	99.7
F	585	583	99.7
G	363	360	99.2
Total	4,302	4,271	99.3

Cross-contamination study.

The potential for cross-contamination in HCV TMA was evaluated by testing replicates of a high-titered HCV RNA-positive specimen alternated with replicates of an HCV-negative specimen in a checkerboard pattern. No false-negative or false-positive results occurred during the study, indicating a cross-contamination frequency of 0%.

HCV RNA stability in clinical samples.

The stability of HCV RNA as detected by HCV TMA was assessed in clinical samples subjected to various sample processing and storage conditions. Results of testing HCV-spiked whole blood samples stored at room temperature in six collection tube types are shown in Table 5. HCV TMA detected HCV RNA in 100% of replicates of each of the six tube types tested at each time point, indicating a sensitivity of 100%. A total of 12 valid results were obtained for 12 replicates, yielding a 95% CL of 75.8 to 100% in all but two cases. Only 11 of 12 replicates gave valid results in sodium heparin tubes at 6 h and in SST at 24 h, yielding a 95% CL of 74.1 to 100%. By contrast, no HCV RNA was detected by HCV TMA in unspiked whole blood stored at room temperature in any of the six tube types at any time point tested, indicating a specificity of 100% (data not shown). In these experiments, a total of 12 valid results were obtained for 12 replicates, yielding a 95% CL of 75.8 to 100% in all but one case. In ACD tubes at 4 h only 11 of 12 replicates gave valid results, yielding a 95% CL of 74.1 to 100%.

TABLE 5.

Percent detection of HCV RNA in specimens subjected to various processing conditions in six collection tube types

Specimen and time (h)	% Detected
	SST	K2-EDTA	K2-EDTA (PPTc)	Sodium citrate	ACD	Sodium heparin
Whole blooda
4	100	100	100	100	100	100
6	100	100	100	100	100	100
24	100	100	100	100	100	100
Decanted sampleb
0	100	100	98.3	100	100	100
8	100	100	100	98.3	100	96.0
24	100	100	100	94.9	97.4	100
48	100	100	98.3	94.9	100	100

^aWhole blood with 50 HCV RNA copies/ml was allowed to stand at room temperature for the times indicated prior to processing by centrifugation.

^bDecanted serum and plasma samples with 50 HCV RNA copies/ml were centrifuged within 4 h of collection and stored at 2 to 8°C for the times indicated prior to freezing.

^cPPT, plasma preparation tube.

The stability of HCV RNA in decanted serum and plasma samples stored at 2 to 8°C was also assessed by HCV TMA (Table 5). Results showed that HCV TMA detected HCV RNA in ≥94.9% of replicates of each of the six tube types tested at each time point. The 95% CL varied from a low of 86.1 to 98.3% to a high of 94.0 to 100%, according to the number of valid results obtained for the number of replicates tested.

HCV TMA also was used to test HCV RNA stability in specimens subjected to multiple freeze-thaw cycles and in specimens stored frozen for up to 13 months (Table 6). Results showed that HCV TMA detected HCV RNA in over 98% of specimens frozen and thawed up to three times. Moreover, HCV TMA results showed no significant reduction in percent detected over the 13-month period among samples stored at −20 and −80°C.

TABLE 6.

Percent detection of HCV RNA in specimens stored under various conditions

Storage condition	No. valid	No. reactive	% Detected
Freeze-thaw cycles
0	56	56	100
1	55	54	98.2
2	54	53	98.1
3	55	54	98.2
Long-term storage at −20°C
0 mo	56	56	100
4 mo	70	68	97.1
6 mo	82	80	97.6
7 mo	75	73	97.3
8 mo	85	82	96.5
13 mo	86	85	98.8
Long-term storage at −80°C
0 mo	56	56	100
4 mo	78	73	93.6
6 mo	81	81	100
7 mo	75	75	100
8 mo	81	79	97.5
13 mo	69	69	100

Potentially interfering substances.

Potential interference of endogenous substances, nontarget microorganisms and viruses, and commonly prescribed drugs with HCV TMA performance was evaluated by testing HCV-negative and HCV-positive specimens (50 copies/ml) spiked with the substances, microorganisms, and viruses shown in Table 7. No statistically significant reduction in sensitivity or specificity was observed in samples with elevated levels of hemoglobin, bilirubin, triglycerides, and protein. Also, no reduction in sensitivity or specificity was observed in samples containing pools of nontarget microorganisms and viruses or antiviral or immunosuppressant drugs.

Potential interference by specimens from patients with non-HCV liver disease also was tested in HCV TMA. Specimens from anti-HCV antibody-negative patients with autoimmune hepatitis (n = 9), primary biliary cirrhosis (n = 10), alcohol- or drug-related liver disease (n = 10), myeloma (n = 14), anti-dsDNA-positive status (n = 6), rheumatoid factor-positive status (n = 19), or systemic lupus erythematosus (n = 10) showed 100% specificity, and ANA-positive specimens (n = 60) showed 97.9% specificity. In initial experiments, two myeloma patient specimens were tested, of which one was nonreactive and one was invalid. Subsequently, 12 additional myeloma patient specimens were tested, all of which gave valid results and were nonreactive, yielding a specificity of 100%. Also in initial experiments, 10 ANA-positive specimens were tested in HCV TMA, of which one was reactive. Since there was not sufficient volume for retesting of this specimen, a larger experiment was performed in which 50 ANA-positive samples were tested, of which 3 were reactive in HCV TMA. The three specimens with reactive results were retested in duplicate and also were tested with the AMPLICOR HCV Monitor 2.0 assay. Two specimens that were repeatedly reactive in HCV TMA also were positive for HCV RNA in the AMPLICOR HCV Monitor 2.0 assay and were therefore removed from the analysis, whereas the other specimen was HCV RNA negative in both assays and was therefore considered to be an initial false positive. In this experiment, of the 48 ANA-positive samples from patients who met the acceptance criteria, 47 were nonreactive in HCV TMA, yielding a specificity of 97.9%.

Sufficient sample volume was available for sensitivity testing of specimens from ANA-positive (n = 10), anti-dsDNA-positive (n = 6), rheumatoid factor-positive (n = 19), systemic lupus erythematosus (n = 10), and myeloma (n = 14) patients. For these samples, sensitivity was tested by adding HCV-positive donor serum to a final concentration of 50 HCV RNA copies/ml. HCV TMA detected HCV RNA in 100% of ANA-positive, anti-dsDNA-positive, rheumatoid factor-positive, and systemic lupus erythematosus patient samples, yielding a sensitivity of 100%. Of the 14 myeloma patient samples, 13 gave valid results, of which 11 were HCV RNA positive by HCV TMA, yielding a sensitivity of 84.6%. During testing of myeloma patient specimens it was noted that the pellet was somewhat diffused during the target capture step in some specimens, indicating a possible cause for the two false negatives and one invalid specimen in this group.

DISCUSSION

Qualitative HCV RNA assays are the only tests currently in widespread use that can confirm active infection. In order to accurately diagnose and monitor HCV infection, these qualitative assays must be highly sensitive, specific, and reproducible and have the ability to detect all HCV genotypes equally. These assays also should be able to detect RNA from samples collected and stored in different manners to allow for flexibility of sample handling and testing in today's high-volume laboratories. In addition, assays should be free from interference by endogenous substances or commonly encountered microorganisms, viruses, or drugs. In this study, we evaluated the performance of HCV TMA for qualitative detection of HCV RNA, referring to the FDA draft document (6) and the NCCLS guidelines (13-16).

One of the strengths of our study is the large number of samples evaluated for each performance attribute. We tested over 18,000 samples, of which 5,800 were tested for sensitivity alone. This number of samples is substantially more than that tested in earlier studies that evaluated the performance of HCV TMA, and our findings both confirm and extend the findings of these earlier studies. The analytical sensitivity, defined as ≥95% detection, that we report here of 5.3 IU/ml and 29.1 copies/ml (equivalent to 5.6 IU/ml) is not significantly different from the 6.0 IU/ml reported by Krajden et al. (9) or the 5 IU/ml reported by Ross et al. (19) (none of the P values were significant at the 0.05 level). Similarly, the overall specificity of 99.6% (99.5% specificity for serum, 99.7% specificity for EDTA plasma) that we report for HCV TMA is consistent with the >99% specificity reported by Ross et al. (19). Finally, our finding that HCV TMA exhibits >95% detection of all HCV genotypes tested at 50 HCV RNA copies/ml in viral panels is consistent with that of Ross et al. (19), who found 100% detection at 50 copies/ml in a panel of clinical samples of HCV genotypes 1a, 1b, 2a, 2b, 2c, 3a, 4, and 5. Although we found 95.2% detection of HCV genotype 2b at 50 copies/ml in viral panels, we noted lower detection of HCV genotype 2b RNA transcripts at 50 copies/ml. The reduced sensitivity of some genotype 2b specimens is not completely understood. Unfavorable secondary structures in the viral target region and/or in the TMA amplicons of some HCV genotype 2b specimens might contribute to the slightly reduced sensitivity.

Similar performance evaluations also have been published for the qualitative HCV RNA assays AMPLICOR HCV 2.0 and COBAS AMPLICOR HCV 2.0. A study by Lee et al. (10) found both of these assay formats to be less sensitive than HCV TMA, with 100% detection at 50 IU/ml and ≥90% detection at 25 IU/ml. Consistent with these findings, Krajden et al. (9) reported an analytical sensitivity for the COBAS AMPLICOR HCV 2.0 assay of 45 IU/ml without resolution of gray zone results and 32 IU/ml with resolution of gray zone results. In their premarket approval applications, Roche Molecular Systems claimed a LoD based on ≥95% detection of 100 IU/ml in serum and 60 IU/ml in EDTA plasma for AMPLICOR HCV 2.0 (4) and 60 IU/ml in serum and 50 IU/ml in EDTA plasma for COBAS AMPLICOR HCV 2.0 (5) without resolution of gray zone results. Results of earlier studies also indicate that the specificity of HCV TMA is at least comparable to that of AMPLICOR HCV 2.0 and COBAS AMPLICOR HCV 2.0. Although the study by Lee et al. (10) reported a specificity of 100% for the AMPLICOR HCV 2.0 and COBAS AMPLICOR HCV 2.0, a larger study by Nolte et al. (18) reported lower specificities of 97.0% for serum and 93.1% for plasma by AMPLICOR HCV 2.0 and 97.9% for serum and 96.6% for plasma by COBAS AMPLICOR HCV 2.0.

The clinical relevance of the high sensitivity of HCV TMA has been demonstrated in several earlier studies. In three of these studies (2, 21, 22), HCV TMA detected residual HCV RNA in end-of-treatment samples that tested HCV RNA negative by other less sensitive assays from patients who experienced virologic relapse after the end of therapy. Of 22 patients who relapsed after standard interferon treatment with and without ribavirin, 8 (36%) were HCV RNA positive by HCV TMA but HCV RNA negative by AMPLICOR HCV 2.0 at the end of treatment (22). Similarly, HCV TMA detected residual HCV RNA in 5 of 18 (28%) samples that were negative by AMPLICOR HCV 2.0 and in 9 of 31 (29%) samples that were negative by SuperQuant from relapsing patients treated with multiple therapies (2). A recent study by Sarrazin et al. (21) showed that HCV TMA detected residual HCV RNA in 6 of 18 (33%) interferon-treated patients and 4 of 60 (7%) patients treated with the long-acting Peginterferon who had tested HCV RNA negative by COBAS AMPLICOR HCV 2.0. These results demonstrate the greater sensitivity of HCV TMA in clinical samples.

The clinical relevance of HCV TMA's greater sensitivity also has been demonstrated in comparative studies evaluating the concordance between results of HCV TMA and the COBAS AMPLICOR HCV 2.0 assay. Although the study by Krajden et al. (9) found an overall concordance of 99.1%, results showed that HCV TMA detected HCV RNA in 3 of 79 clinical samples that were positive by anti-HCV EIA and negative by COBAS AMPLICOR HCV 2.0 and in 1 of 112 samples that were indeterminate by EIA and negative by COBAS AMPLICOR HCV 2.0. It is likely that these four samples were true HCV positives, since two were from patients who reported intravenous drug use and two were from patients on interferon plus ribavarin (Rebetron) therapy for HCV infection. By comparison, all 105 samples that were positive by EIA and COBAS AMPLICOR HCV 2.0 were also positive by HCV TMA (overall concordance, 100%). Similarly, Ross et al. (19) found that 146 of 150 clinical samples gave identical results by HCV TMA and the COBAS AMPLICOR HCV 2.0 assay (overall concordance, 97.3%). In that study all four of the discordant samples were HCV RNA positive by HCV TMA but negative by COBAS AMPLICOR HCV 2.0, and the presence of HCV RNA in these four samples was confirmed by an in-house reverse transcription-PCR assay. These results reflect the greater sensitivity of HCV TMA.

As therapeutic regimens for chronic hepatitis C continue to improve, ever more sensitive qualitative HCV RNA assays may be needed to effectively monitor response and determine optimal treatment duration. Indeed, studies are currently under way to determine whether patients who have residual HCV RNA detectable by HCV TMA at the end of therapy or during the early follow-up period would benefit from an extended treatment period. These types of clinical studies should help to shed light on the true clinical value of the enhanced sensitivity of HCV TMA.