Human Immunodeficiency Virus Type 1 Drug Resistance Testing: a Comparison of Three Sequence-Based Methods

Abstract

The use of genotypic assays for determining drug resistance in human immunodeficiency virus (HIV) type 1 (HIV-1)-infected patients is increasing. These tests lack standardization and validation. The aim of this study was to evaluate several tests used for the determination of HIV-1 drug resistance. Two genotypic tests, the Visible Genetics TruGene HIV-1 Genotyping Kit and the Applied Biosystems HIV Genotyping System, were compared using 22 clinical samples. Genotyping results were also obtained from an independent reference laboratory. The Visible Genetics and Applied Biosystems genotyping tests identified similar mutations when differences in the drug databases and reference strains were taken into account, and 19 of 21 samples were equivalent. The concordance between the two assays was 99% (249 of 252 mutation sites). Mutations identified by the reference laboratory varied the most among those identified by the three genotypic tests, possibly because of differences in the databases. The concordance of the reference laboratory results with the results of the other two assays was 80% (201 of 252). Samples with 500 to 750 HIV RNA copies/ml could be sequenced by the Visible Genetics and Applied Biosystems assays using 1 ml of input. The Visible Genetics and Applied Biosystems assays both generated an accurate sequence. However, the throughput of the Visible Genetics assay is more limited and may require additional instruments. The two assays differ technically but are similar in overall complexity. Data analysis in the two assays is straightforward, but only the reports provided by Visible Genetics contain information relating mutations to drug resistance. HIV drug resistance genotyping by sequencing is a complex technology which presents a challenge for analysis, interpretation, and reporting.

The rapid rate of replication and high frequency of mutation of human immunodeficiency virus (HIV) type 1 (HIV-1) favor the development of drug-resistant virus populations. As a consequence, HIV-1-infected individuals are routinely treated with combination therapy in an attempt to avoid the emergence of resistance. While combination antiretroviral drug therapies have provided significant advances in HIV-1 treatment, clinical failures are still seen. Preexisting or selected drug-resistant viral mutants constitute one of the important causes of antiretroviral drug failure. Other factors include poor patient compliance, lack of access to adequate care, suboptimal dosing, and drug pharmacology issues, such as absorption, elimination, and drug interactions.

The limited drug options available for treatment following failure and increased awareness of the prevalence of resistant strains of virus have led to the development of tests to determine the presence of drug-resistant HIV-1. Two types of testing are available: genotypic tests that identify resistance-associated mutations in the HIV-1 genome and phenotypic tests that directly measure the susceptibility of a viral isolate to drugs. The rapid development of these tests, their technical complexity, the large amount of information generated, and the lack of definitive clinical outcome studies have made the interpretation of results an extremely challenging process which may be confusing for clinicians.

Numerous retrospective studies have evaluated the clinical utility of both genotypic and phenotypic drug resistance testing (3, 4, 9, 13, 14, 16, 17, 22), while prospective studies on clinical utility are more limited (1, 6; C. Cohen, S. Hunt, M. Sension, C. Farthing, M. Conant, S. Jacobson, J. Madler, W. Verbiest, K. Hertogs, M. Ames, A. Rinehart, and N. Graham, Abstr. 7th Conf. Retro. Opport. Infect., abstr. 237, 2000). Prospective studies generally support an improved response to therapy in groups treated with access to antiretroviral drug resistance information. In these studies, response is measured as a reduction in HIV RNA levels or as the percentage of patients in whom undetectable HIV RNA levels are achieved. These initial data have been used to justify resistance testing in certain clinical settings. In contrast to studies that support the use of drug resistance testing, a French study of 541 patients randomized by genotype, phenotype, or standard care failed to show a significant benefit from resistance testing (J. L. Meynard, M. Vray, L. Morand-Joubert, S. Matheron, G. Peytavin, F. Clavel, F. Brun-Vezinet, and P. M. Girard, Abstr. 4th Int. Workshop HIV Drug Resist. Treat. Strategies, abstr. 85, 2000). Possible factors contributing to the different interpretations from various studies include the technical expertise of the laboratories, the subjective interpretation of electropherograms, lack of careful assay validation, lack of general standardization of methods, and variations in the patient populations studied.

Despite clinical and technical uncertainties regarding drug resistance testing, the use of such assays in patient management is increasing. The recent consensus statement developed by the International AIDS Society-USA panel (12) recommends resistance testing as a guide for the treatment of patients for whom therapy has failed and for HIV-1-infected pregnant women. The panel also suggests considering resistance testing for the treatment of naive patients with established infection and for patients with primary infection.

Two major molecular phenotypic assays are commercially available and have been used in clinical trials and outcome studies. Antivirogram (Virco, Mechelen, Belgium) and PhenoSense (ViroLogic, South San Francisco, Calif.) are assays that require specialized facilities and are not routinely performed in clinical laboratories. Other reference laboratories have introduced additional molecular and nonmolecular phenotypic tests. Sequence-based genotypic assays are under development by Applied Biosystems (AB) (Foster City, Calif.) and Visible Genetics (VG) (Toronto, Ontario, Canada), and reference laboratories are offering assays developed in-house. Genotypic tests are complex assays, requiring careful technical performance and data interpretation. Despite the complexities, there are currently no standards for either phenotypic or genotypic testing for HIV-1 drug resistance. Both methodologies demand adherence to consistent test procedures and thorough quality control review to ensure reliable results. An understanding of the characteristics and limitations of the assays and the complexity of resistance patterns is required for the consistent and reliable interpretation of drug resistance test results. In this study, we specifically address the performance characteristics of two genotypic tests and compare results from those tests to results from a reference laboratory.

MATERIALS AND METHODS

Clinical samples.

Plasma specimens were obtained from 22 HIV-1-infected individuals at the University of Utah Medical Center Infectious Diseases Clinic. Whole blood was drawn in tubes containing EDTA as an anticoagulant and processed within 30 min. Plasma was separated from the cells by centrifugation at 1,500 × g for 10 min at room temperature, divided into aliquots, and frozen for storage at −20°C or lower. All patients had a history of receiving treatment with antiretroviral drugs.

TruGene HIV-1 Genotyping Kit.

The TruGene HIV-1 Genotyping Kit(VG) was run according to the manufacturer's instructions and using reagents provided with the kit for all steps except nucleic acid extraction. The QIAamp Viral RNA Mini Kit (Qiagen Inc., Valencia, Calif.) was used for the extraction of samples run in the VG assay. For samples 1 through 9, 140 μl of plasma specimen was processed directly without centrifugation in the Qiagen kit as recommended by VG. For samples 10 through 22, in order to optimize the recovery of virus in samples with a low viral load, 1 ml of plasma specimen was processed. The 1-ml sample was centrifuged at 23,500 × g for 1 h at 4°C to pellet the virus, and 860 μl of supernatant fluid was removed and discarded. The remaining 140 μl of concentrated sample was processed in the Qiagen kit according to the manufacturer's instructions. Briefly, the sample was lysed under denaturing conditions and then applied to a silica gel-based membrane in a spin column. Viral RNA bound to the membrane was washed, eluted, and stored at −70°C.

A 1.3-kb sequence of the HIV-1 polymerase gene containing the protease and most of the reverse transcriptase (RT) coding regions was amplified by RT-PCR with a GeneAmp PCR System 9700 (AB) thermal cycler. The RT-PCR was a one-step, two-enzyme reaction using Moloney murine leukemia virus (M-MuLV) RT and thermostable DNA polymerases. Extracted RNA samples were added to a buffered solution containing primers, deoxynucleotides (dNTPs), dithiothreitol, and RNase inhibitor but no enzymes in PCR tubes. The RNA template was denatured for 2 min at 90°C, after which the temperature was lowered to 50°C. The primers were annealed to the template at 50°C for 6 min, the tubes were opened in the thermal cycler, and the RT-PCR enzyme mixture was added. RT-PCR was continued without further intervention. The amplicon generated was sequenced directly after amplification or stored at −20°C until sequenced.

Sequencing was done using the VG CLIP sequencing technique, which allowed forward and reverse directions of the sample amplicon to be sequenced in one tube using two different fluorescent dye-labeled primers. The sample amplicon was incubated with the fluorescent dye-labeled primers, dNTPs, AmpliTaq FS DNA polymerase, and one of four dideoxynucleotides. In this primer chemistry method, one primer set required four tubes, one for each of the dideoxynucleotides. In the VG assay, the protease gene of HIV-1 was sequenced using two sets of primers. Two different forward primers were paired with the same reverse primer. Two separate regions of the RT gene were sequenced using two different sets of primers. Sequencing of the protease and RT regions required 16 tubes for each sample (four primer sets times four dideoxynucleotides). All sequencing reactions were set up in 96-tube trays, allowing six samples per tray to be sequenced with relative ease. There were 16 kit-provided solutions containing primers, dNTPs, and either A, C, G, or T dideoxynucleotides, which were added to appropriate tubes of the 96-tube trays. CLIP master mix containing buffered AmpliTaq FS enzyme was divided into aliquots and placed in separate 0.5-ml tubes for each sample. The RT-PCR product for each sample was added to CLIP master mix, and then that solution was added to appropriate tubes of the 96-tube trays. Cycle sequencing was done with the GeneAmp PCR System 9700. Following cycle sequencing, a stop solution containing loading dye and formamide was added to each reaction tube. The sequenced samples could be electrophoresed directly or stored at 4°C until needed.

Electrophoresis was done with an OpenGene Automated DNA Sequencing System (VG) with GeneObjects software and a Long Read Tower sequencer. An acrylamide gel mixture was injected into 50-μm-thick gel plates and polymerized by a 3-min exposure to UV light. Two sizes of plates were used, a 14- by 14-cm plate used early in the study and a 14- by 21-cm plate which became available later in the study and which was found to provide better sequence resolution. Each plate size contained 16 lanes, allowing electrophoresis of all sequencing reactions from one sample to be run on one plate. Prior to loading, samples were denatured for 3 min at 85°C and then placed on ice. Electrophoresis was completed in 30 min for the 14-cm-long plates and in 50 min for the 21-cm-long plates.

The VG GeneLibrarian HIV Module, version 3.1, was used for sequence analysis. The software automatically trimmed and aligned the sequences generated for the two protease and two RT regions. Forward and reverse sequence data were generated for codons 1 through 99 for the protease gene, codons 39 through 145 for the beginning of the RT gene, and codons 138 through 244 for the middle of the RT gene. The software presented the electropherograms for the individual aligned sequences in a format that compared the sample sequences with the consensus of the sample sequences, the reference sequence for the B-LAV-1 strain of HIV-1, and a sequence containing mutations identified as related to drug resistance. Manual examination of the electropherograms was done to confirm the sequence calls made by the software. Forward and reverse sequences were available for all areas evaluated. No calls were routinely made for sequence data from only one strand. After editing was completed, the software generated a report that listed the primary and secondary resistance mutations associated with 18 drugs or drug combinations, based on a database provided in the GeneLibrarian HIV Module, version 3.1. This database was prepared using the resistance effects reported previously (11). Additional information in this version of the report included all silent mutations, polymorphisms defined as coding changes not at resistance sites, and unexpected mutations at resistance sites.

HIV Genotyping System.

The HIV Genotyping System, version 2 (AB), was run according to the manufacturer's instructions and using reagents provided with the kit for all steps. Nucleic acid preparation for the AB assay was done using the method provided with the genotyping assay kit. For all 22 specimens, 500 μl of sample was centrifuged at 23,500 × g for 1 h at 4°C to pellet the virus. Supernatant fluid was removed and discarded, and the pellet was resuspended in a lysis solution containing guanidine thiocyanate. The nucleic acid was purified by isopropanol precipitation followed by an ethanol wash. The RNA pellet was resuspended in buffer supplied by the manufacturer in a volume based on the level of HIV RNA in the original sample.

A 1.8-kb sequence of the HIV-1 polymerase gene containing the protease and most of the RT coding regions was amplified by RT-PCR with the GeneAmp PCR System 9700 thermal cycler. The RT-PCR was a two-step, two-enzyme reaction using M-MuLV RT and AmpliTaq Gold. Extracted RNA samples were added to PCR tubes containing a single primer for RT, dNTPs, buffer, dithiothreitol, RNase inhibitor, and M-MuLV. The RT reaction was performed with the thermal cycler, and the RT product was kept at 4°C until the PCR was performed or stored long term at −20°C. PCR was done by direct addition of a PCR master mix consisting of sequence-specific primers, dNTPs, buffer, AmpliTaq Gold, and uracil N-glycosylase to the tubes containing the RT product. The PCR product was purified immediately following amplification or stored at −20°C until purification. The PCR product was purified using Microcon-100 centrifugal filter devices (Millipore Corp., Bedford, Mass.), and the purified product was stored at −20°C. Prior to sequencing, the yield and purity of the PCR product were evaluated by electrophoresis on an agarose gel and comparison to a DNA mass ladder with known DNA sizes and amounts. A visual comparison of the band intensity of the PCR product and the band intensity of the DNA mass ladder was made, and dilutions of the PCR product were prepared according to recommendations in the kit package insert.

Sequencing was done using BigDye Terminator chemistry, in which the four dideoxynucleotides were labeled with four different fluorescent dyes. Four forward and three reverse primers were used to sequence the protease and RT genes. Since BigDye Terminator chemistry was used, all four dideoxynucleotides could be included in one tube with each primer, and therefore seven reaction tubes were required per sample. The PCR amplicon for each sample was added to PCR tubes containing sequencing reaction mixtures for each of the seven primers. The reaction mixtures consisted of buffered sequence-specific primers, dNTPs, and AmpliTaq FS DNA polymerase. Cycle sequencing was done with the GeneAmp PCR System 9700. Following cycle sequencing, the sequencing products were purified by sodium acetate-ethanol precipitation. The purified sequencing products could be electrophoresed directly or stored at −20°C until needed. An alternative method suggested by AB for sequence product purification was to use spin columns. Column purification was not used in this study.

Electrophoresis was done with an ABI Prism 377 DNA Sequencer (AB) in a slab gel format and a Long Ranger polyacrylamide gel (FMC BioProducts, Rockland, Maine). The purified sequencing products were denatured in a loading buffer-formamide solution at 95°C for 2 min prior to loading. Electrophoresis was completed in 7 h.

Sequence analysis was done using DNA Sequencing Analysis software, version 3.4, and HIV Genotyping System software, version 2.1. The sequencing analysis software was used to trim the sequences, and the HIV Genotyping System software was used to further trim and align the seven sequences generated. This analysis provided forward and reverse sequence data for codons 1 to 99 of the protease gene and codons 1 to 320 of the RT gene. The software presented the electropherograms for the individual aligned sequences with the consensus sequence for the samples and the reference sequence for the pNL4-3 strain of HIV-1. Manual examination of the electropherograms was done to confirm the sequence calls made by the software. Forward and reverse sequences were available for all areas evaluated. No calls were routinely made for sequence data from only one strand. After editing was completed, the software generated a report that listed the reported nucleotide variants associated with known resistance mutations as presented in the Los Alamos HIV database (http://hiv-web.lanl.gov/). The report also included novel nucleotide variants that were changes in the sequence but that were not noted as resistance mutations in the Los Alamos HIV database as well as insertions detected in the samples.

Reference testing.

Twenty-two samples were sent to a reference laboratory for HIV-1 drug resistance testing by the VircoGEN (Virco, Mechelen, Belgium) genotype procedure.

Dilution study.

A clinical specimen with an HIV-1 RNA level determined with the COBAS Amplicor HIV Monitor Test (Roche Diagnostics, Indianapolis, Ind.) to be 97,000 HIV RNA copies/ml was diluted in HIV-seronegative, delipidated, defibrinated normal human serum. Dilutions were prepared to obtain HIV-1 RNA levels of 5,000, 2,500, 1,000, 750, 500, and 250 HIV RNA copies/ml. The HIV-1 RNA levels in the diluted samples at 750 and 500 HIV RNA copies/ml were confirmed by analysis with the ultrasensitive version of the COBAS Amplicor HIV Monitor Test. Two extraction methods were evaluated. For evaluation of the VG assay, 1 ml of sample was spun at 23,500 × g and 4°C for 1 h and processed in the QIAamp Viral RNA Mini Kit as described above. For analysis of the AB assay, 1 ml of sample was spun at 23,500 × g and 4°C for 1 h and processed in the AB extraction protocol as described above. A sample size of 1 ml was chosen as a convenient sample size that would provide optimal virus recovery.

RESULTS

Technical issues.

The major technical differences in the AB and VG assays are presented in Table 1. The specific protocols for the AB and VG assays differed, but the overall complexities of the two assays were comparable. Reliable performance of both assays required well-trained technologists, but sequencing from both assays was consistent once training was completed. Since the gels for the AB sequencer have up to 96 lanes and the gels for the VG sequencer have 16 lanes, throughput was higher in the AB assay. However, the use of more than one VG sequencer can extend the capacity of that assay.

TABLE 1.

Comparison of technical differences in the AB and VG assays

Step or parameter	Details for the following assay:
	AB	VG
Sample preparation	Spin to concentrate	Spin to concentrate
	500-μl input	1,000-μl input
	Guanidine thiocyanate lysis	QIAamp Viral RNA Mini Kit
	Isopropanol precipitation, ethanol wash	Silica-based extraction
	2 h/24 samples	2 h/24 samples
RT-PCR	Two-enzyme, two-step RT-PCR	Two-enzyme, one-step RT-PCR
	Add RT enzyme, incubate; add PCR enzyme, incubate	Add two enzymes at the same time, incubate
	1.8-kb RT-PCR product	1.3-kb RT-PCR product
	60-min setup, 60-min RT reaction, 3-h PCR	1.5-h setup, 4-h RT-PCR
PCR purification	Column cleanup	None required
	Run PCR product on gel
	Dilute product
	1-h purification, 1-h gel run
Sequencing reaction	Big Dye Terminator chemistry	CLIP dye primer chemistry
	Seven primers (forward and reverse in separate tubes)	Four primer sets (forward and reverse in one tube)
	Dideoxynucleotides labeled with four dyes	Separate tubes for each dideoxynucleotide
	Seven reactions	Primers labeled
	45-min setup, 3-h sequence run	Sixteen reactions
		1-h setup, 3-h sequence run
Length of sequence (kb)	1.2	0.94
Sequence product purification	Precipitation (10 min) or column cleanup (45 min) for 13 samples	None required
Electrophoresis	96 lanes/gel	16 lanes/gel
	13 patients/gel	1 patient/gel
	7 h/gel	50 min/gel
Analysis	HIV Genotyping System software	GeneObjects software
	Macintosh operating system	OpenStep operating system
	HIV-1 reference strain pNL4-3	HIV-1 reference strain B-LAV-1
	10 min/sample	10 min/sample

Sequence analysis software for each assay was straightforward, and very little data manipulation was needed to assemble and analyze the sequences. The report generated in the VG assay was more detailed than the AB assay report and provided an association of mutations with specific drugs. In the VG software used in this study, the primary and secondary mutations were provided for the 14 drugs approved by the U.S. Food and Drug Administration (FDA) for HIV; for 3 additional drugs, adefovir, emtricitabine, and foscarnet; and for multidrug resistance to zidovudine, didanosine, and zalcitacine. The AB report provided nucleotide variants without associating the mutations with any drug resistance phenotypes.

Sample results.

Twenty-two samples were genotyped for HIV-1 drug resistance by three methods, AB, VG, and a reference method (VircoGEN procedure). Results were obtained by all three methods for 21 of the 22 samples tested. Sample 20 had a plasma HIV-1 RNA level of <50 copies/ml and could not be sequenced by any of the methods. The mutations identified at common resistance sites by the three methods are presented in Table 2. A summary of the major mutations identified in this study is presented in Table 3.

TABLE 2.

Common mutation sites identified in 21 samples by three methods for determining the HIV-1 drug resistance genotype^a

^aRL, reference laboratory; X, mutation reported as implicated in drug resistance; P, mutation identified as polymorphism in VG assay; U, mutation reported as unexpected in VG assay; V, mutation reported as novel variant in AB assay; I, insertion; −, no mutation reported. Boxes indicate sites of discrepancy. Sample 20 contained <50 HIV RNA copies/ml and could not be sequenced by any assay. 

TABLE 3.

Mutations identified in this study by three sequence-based methods for determining the HIV-1 drug resistance genotype^a

Protease mutation	Identification by the following assay:
	VG	AB	RL
R8Q	+	+	−
L10F	+	+	−
L10I	+	+	+
K20I	+	+	+
K20R	+	+	+
L24I	+	+	+
D30N	+	+	+
V32I	+	+	+
L33F	+	+	+
M36I	+	+	+
M46L	+	+	+
I54V	+	+	+
D60E	+	+	−
L63P	+	+	−
A71V	+	+	+
G73S	+	+	+
V77I	+	+	+
V82A	+	+	+
I84V	+	+	+
N88D	+	+	+
N88S	+	+	+
L90M	+	+	+

RT mutation	Identification by the following assay:
	VG	AB	RL
M41L	+	+	+
D67N	+	+	+
T69D	+	+	+
T69N	+	+	+
K70R	+	+	+
L74I	+	+	−
L74V	+	+	+
L75I	+	+	+
L74M	+	+	+
L100I	+	+	+
K101E	+	+	+
K103N	+	+	+
K103S	+	+	+
V108I	+	+	+
Y115F	+	+	+
V179D	+	+	+
Y181C	+	+	+
M184V	+	+	+
Y188L	+	+	+
G190A	+	+	+
G190S	+	+	+
H208Y	+	+	−
L210W	+	+	+
R211K	+	+	−
T215D	+	+	−
T215Y	+	+	+
K219H	+	+	+
K219N	+	+	−
G333D	−	−	+
G333E	−	−	+

^aRL, reference laboratory; +, mutation identified; −, no mutation identified. 

Identification of mutations by AB and VG assays.

Differences in mutations reported in the AB and VG assays were generally the result of mutation classification. Each assay used a unique algorithm for categorizing mutations. The report generated by the VG version 3.1 software identified mutations in four categories: (i) resistance mutations, (ii) silent mutations (at all positions), (iii) polymorphisms (coding changes not at resistance sites), and (iv) unexpected mutations at resistance sites. The report generated by the AB version 2.0 software identified mutations in three categories: (i) reported nucleotide variants, (ii) novel nucleotide variants, and (iii) insertions.

When the results from the VG and AB assays are compared solely on the basis of identification of individual mutations and not in relation to category, there was agreement for 19 of 21 samples sequenced. For sample 15, the protease mutation L10F was identified in the AB assay but was difficult to interpret in the VG assay due to a false stop in the VG run. Repeat testing, which was not done, may have resolved the call. Based on examination of the electropherograms, samples 13 and 15 contained highly mixed populations of virus, making identification of certain mutations challenging. For sample 13, the AB assay did not identify the protease I54I/V mutation that was identified in the VG assay. Similarly, for sample 15, the AB assay did not identify the RT Y115Y/F mutation that was identified in the VG assay. In both cases, although the AB software did not note the mutations due to programming limitations, the double peaks indicating a mixed population were identified by manual inspection of the AB electropherograms.

Mutations in the RT that were identified as novel nucleotide variants by the AB assay were outside the area amplified by the VG assay. These mutations included V35I, V245E, E248D, A272P, R277K, Q278H, T286A, V292I, V293I, P294H, L295Y, E297K, K311R, E312S, P313T, V317A, Y318I/F, Y319L, and D320T. None of these mutations is included in the Los Alamos HIV database. The reference laboratory also did not identify these mutations.

The AB assay identified mutations N37S and G57R as novel nucleotide variants in the protease region and mutations Q102K, C162S and F214L as novel nucleotide variants in the RT region in several samples. These mutations were not identified by the VG assay. The sequence generated was identical in the two assays but, due to the use of different reference strains, B-LAV-1 in the VG assay and pNL4-3 in the AB assay, the VG assay did not report the sequence as containing mutations.

There were 117 mutations in the protease region and 138 mutations in the RT region that were identified either by the VG or AB assay or by both assays. Of the 252 total mutations, there were discrepant calls for only 3 sites, for a concordance between the two assays of 99% (249 of 252 mutations).

Identification of mutations by the reference laboratory.

More discrepancies were noted between the results from the reference laboratory and those of either the AB or the VG assay. In several cases, the discrepancies could be accounted for by database differences. The protease mutation L63P was not identified in 18 of 18 samples that were identified by the AB and VG assays as containing the mutation. This mutation was not included in the reference laboratory database and has been removed from the list of common mutations conferring drug resistance which was prepared by the International AIDS Society-USA panel (11, 12). A number of mutations not noted by the reference laboratory were uncommon mutations associated with non-FDA-approved drugs or with multiple nucleoside resistance and also may not have been in the database. In the protease region, these included R8Q, L10I, and D60E, and in the RT region, they included L74I, K101E, H208Y, R211K, T215D, and K219N. Only the reference laboratory noted the G333E mutation. This mutation is outside the area of the genome amplified by the AB and VG assays but is included in the Los Alamos HIV database. The protease mutations M46I in sample 11 and V77I in sample 13 were identified only by the reference laboratory and not by the AB or VG assay. Other mutations in the protease region not noted by the reference laboratory included M36I and V77I in samples 13 and 15, respectively, as well as the RT mutations V108I and Y115F in sample 15. As noted above, these two samples contained mixed populations of virus, and the sequences could have been difficult to analyze depending on sequence quality and technical expertise.

Of the 252 mutations identified either by the VG or AB assay or by both assays, the reference laboratory identified 201, for 80% concordance. Standardization of methodology, quality control, and interpretation among laboratories might improve concordance between assays and laboratories.

Insertion mutations.

An insertion at codon 69 in the RT region was seen in sample 16. This sample contained both wild-type virus and a mutant virus with a six-nucleotide insertion. The electropherograms in both the AB and the VG assays exhibited multiple sequencing signals following the insertion point. Since there were multiple signals, the software was unable to align the sample with the reference. Neither the AB nor the VG software used in this study was able to identify the insertion, and both types of software failed to generate base assignments beyond the insertion. Calls made for this sample were based on the sequence from only one strand over this section of the RT gene. The insertion was identified and characterized based on visual inspection of the electropherograms by the laboratory technologist.

Mixed populations.

The samples with highly mixed populations of virus increased the complexity of interpretation. Mixed populations are seen in electropherograms as double peaks at the same nucleotide. Each peak represents a specific population and is discriminated from background and recognized as a separate peak when the fluorescence signal is above the threshold set for the assay. For this study, the AB software was set at a threshold of 30% and the VG software was set at a threshold of 20% for the discrimination of double peaks. As an example of a mixed population, the electropherograms for the forward and reverse sequences of sample 13 at protease codon 54 are presented in Fig. 1. The AB software did not identify the mixture at codon 54 in either direction, and the VG software identified the mixture only in the forward direction. Visual inspection of the electropherograms provided additional information to verify the mixed population. Reliance solely on the software identification from the AB assay missed the mixture in this sample. Similarly, the AB software did not identify a mixture at RT codon 115 that was evident upon manual inspection. All 21 samples for which a sequence was obtained in this study showed some evidence of a mixed population of virus, as indicated by double peaks in the electropherograms. In most cases, the software identifications from both assays were adequate to resolve the mixtures. The greatest difficulty in identification occurred when one virus population was present at a much lower level than the other or when the sequencing reaction was not optimal. Visual inspection of the electropherograms by the technologist could generally resolve the identification.

FIG. 1.

Electropherogram for sample 13 at protease codon 54. (A) VG electropherogram set at 20% threshold; (B) AB electropherogram set at 30% threshold. Arrows indicate a mixture of wild-type codon ATC and mutant codon GTC.

Dilution study.

Samples with viral load levels of 5,000, 2,500, 1,000, 750, 500, and 250 HIV RNA copies/ml and extracted using 1 ml of sample in the standard Qiagen extraction were run in the VG assay. The minimum viral load that was needed for sequence analysis in the VG assay was between 500 and 750 HIV RNA copies/ml.

Samples containing 5,000, 2,500, 1,000, 750, 500, and 250 HIV RNA copies/ml were extracted using 1 ml of sample in the standard AB extraction. The minimum viral load that was needed for sequence analysis in the AB assay was between 500 and 750 HIV RNA copies/ml.

DISCUSSION

The measurement of plasma HIV-1 RNA viral load levels is standard care for the management of HIV-1-infected individuals and provides prognostic information and an indication of the efficacy of therapeutic regimens (15, 18, 19). Treatment failure, even with the availability of combination drug therapy, continues to occur and is generally first noted as a rebound in HIV-1 load. While a number of factors, such as patient compliance and drug pharmacology, must be considered, the development of resistant virus is a contributing factor in treatment failure. With a limited number of alternative drugs available for making adjustments in therapy, the ability to determine the resistance characteristics of the infecting virus appears important. Resistance testing has been advanced as a valuable tool in the management of HIV-1-infected individuals (12); however, clinical data supporting the use of resistance testing is still accumulating (7). Information concerning technical issues and the performance characteristics of tests is lacking.

The comparison done here between the VG and AB HIV genotyping assays indicates that the two test methods are alike with respect to technical complexity but are quite different with respect to protocol. The assays use different chemistries and procedures for RT-PCR and sequencing, but both are labor-intensive and highly complex, with many steps.

The RT-PCR and sequencing chemistry in the VG assay provided reliable and reproducible results. The technical complexity of running the assay necessitated thorough and detailed training of personnel. Repeat testing in the VG assay due to failed amplification or sequencing was noted only during the training of new personnel. Once personnel were adequately trained and competent in following the procedures, repeat testing was not an issue. The quality of the sequence obtained in the VG assay was excellent, and the software identifications were reliable. Reviewing the electropherograms was straightforward, and the need for edits was minimal. The instrumentation for sequencing in the VG assay was simple to use and dependable. The major limitation of the VG instrumentation is the gel size of 16 lanes, which limits throughput.

Like the VG assay, the AB assay required thorough and complete training of personnel. Repeat testing in the AB assay due to failed amplification or sequencing was minimized once adequate training of personnel was done. The need for repeat testing due to reagent problems was occasionally noted in the AB assay. These problems were resolved with the support of the manufacturer and were generally due to the use of nonstandardized, nonoptimized reagents obtained on a research basis. In this study, electrophoresis for the AB assay was performed with an ABI Prism 377 slab gel-based instrument. The quality of the sequence obtained with this instrument was very good, and the software identifications were reliable; however, reviewing the electropherograms was more challenging than for the VG assay, and more time was spent editing. Additional sequencing instruments are available from AB; these include the capillary electrophoresis instruments Prism 310, Prism 3100, and Prism 3700. The samples used in this study were later analyzed with a Prism 3100 instrument, and a sequence of exceptional quality was obtained (data not shown). Throughput with the Prism 377 instrument is much higher than in the VG assay, and the use of a capillary instrument further increases throughput. Additional benefits of using a capillary instrument include labor and reagent savings.

Interlaboratory differences in the identification of mutations in blinded samples with a number of different genotyping technologies have been reported (5, 20). In these studies, there were large variations in the results obtained among the laboratories. The different technologies, variations in the protocols performed, and the technical expertise of personnel in the laboratories all contributed to the variable results. The procedural complexity and technical challenge of appropriately performing either the VG or the AB HIV genotyping assay demand a laboratory with well-trained personnel. It is therefore essential that technical guidelines be established to standardize the procedures as performed in all laboratories. Standardization of procedures will help to ensure consistency of results and reporting between runs in one laboratory and ideally among all laboratories.

The minimum level of plasma HIV-1 RNA needed to obtain an adequate sequence depends on the amount of sample processed. The VG protocol recommended a sample volume of 140 μl, and the AB protocol recommended 500 μl. A sample size of 1 ml was chosen for this study as a sample size that would be reasonable to obtain and would provide optimal viral recovery. For a 1-ml sample size, a level of 1,000 HIV-1 RNA copies/ml was found to be adequate for both the AB and the VG assays. The minimum viral load needed to obtain an adequate sequence in the two assays was not determined for any sample size other than 1 ml. In this study, using one sample, adequate sequences were obtained with 500 HIV RNA copies/ml in the AB assay and with 750 HIV RNA copies/ml in the VG assay. To determine the resistance mutations in samples with very low viral loads, volumes of greater than 1 ml will need to be processed. Reliable genotyping of patients who have recently gone from undetectable to low levels of HIV-1 RNA could be problematic, although the need for this type of testing has not been established.

The software provided for sequence analysis in both the VG and the AB assays is straightforward and user-friendly. Both companies continued to work on software upgrades during this study to further improve analysis and reporting. The most significant problem in analyzing electropherograms is the interpretation of mixed populations of virus that result in double peaks (Fig. 1). In this study, the samples with the most discrepant results were those that had mixed populations of virus. The software in the different assays did not always identify the minor sequence, and visual examination and editing were sometimes necessary to override the software identifications. It is assumed that low levels of mutated virus contribute significantly to therapeutic outcomes. The ability to accurately and reproducibly detect all mutations in an individual's viral population will be necessary to optimize clinical utility. Additional concerns about the reproducibility of sequencing of mixed populations were presented in a recent study (8), which suggested that variations during PCR amplification may affect which dominant population is sequenced.

Manual performance of the assays, interpretation of the electropherograms, and data analysis all require careful and skilled evaluation. An added problem in reporting is the nonstandardization of the databases used to associate mutations with particular drugs. Many of the discrepancies among the mutations reported by the VG, AB, and reference laboratory assays appeared to be due to differences in databases. The Los Alamos HIV database (http://hiv-web.lanl.gov/) and the Stanford database (http://hivdb.stanford.edu/hiv/) are often referenced, and there are periodic consensus statements concerning notable mutations and patterns (11, 12); however, there is no standardized interpretation of significant mutations. Even more complexity is added to interpretation when reporting goes beyond just providing mutation lists to specifying resistance or susceptibility to a given drug. Interpretation of individual mutations and mutation patterns to provide information about drug resistance currently depends on the interpretation rules used in individual laboratories. As a consequence, commercially available assays are using diverse databases and interpretation rules, and these are continually being updated as new information becomes available. Clinical interpretation of drug resistance reports should take into account that variations in the databases can affect the reported mutations and that variations in interpretation rules can affect the identification of resistance.

The three methods studied here also differed in the strains of HIV that were used as references for comparisons. The VG assay used B-LAV-1, the AB assay used pNL4-3, and the reference laboratory used HXB2-D. Due to the extensive polymorphism of HIV, it is possible that some sequences could be identified as containing mutations depending on the reference strain used. Again, consensus on the appropriate reference strain is lacking, and clinicians should be aware of the limitations that this situation places on the interpretation of a drug resistance report.

Several studies have compared results obtained using both genotypic and phenotypic assays (2; N. T. Parkin, C. Chappey, and ViroLogic Clinical Reference Laboratory, Abstr. 4th Int. Workshop HIV Drug Resist. Treat. Strategies, abstr. 64, 2000; S. H. Qari, R. Respess, H. Weinstock, E. Beltrami, K. Hertogs, B. A. Larder, C. J. Petropoulos, N. Hellman, and W. Heneine, Abstr. 4th Int. Workshop HIV Drug Resist. Treat. Strategies, abstr. 62, 2000; W. Verbiest, M. Peeters, K. Hertogs, P. Schel, S. Bloor, A. Reinhart, N. Graham, C. Cohen, and B. A. Larder, Abstr. 4th Int. Workshop HIV Drug Resist. Treat. Strategies, abstr. 81, 2000) as well as different genotyping methods (21; G. Collin, D. Descamps, F. Telles, S. Matheron, V. Obry, D. Costagliola, and F. Brun-Vezinet, Abstr. 4th Int. Workshop HIV Drug Resist. Treat. Strategies, abstr. 68, 2000; M. L. Hoover, D. N. Wentworth, J. D. Neaton, D. L. Mayers, T. C. Merigan, M. A. Winters, J. D. Baxter, and the CPCRA 046 Study Team for the Terry Beirn Community Programs for Clinical Research on AIDS, Abstr. 4th Int. Workshop HIV Drug Resist. Treat. Strategies, abstr. 78, 2000). A thorough understanding of the relationship between genotype and phenotype could allow for phenotypic interpretation of genotypic information, as proposed for the VirtualPhenotype available from Virco (B. A. Larder, S. D. Kemp, and K. Hertogs, Abstr. 4th Int. Workshop HIV Drug Resist. Treat. Strategies, abstr. 63, 2000; Verbiest et al., Abstr. 4th Int. Workshop HIV Drug Resist. Treat. Strategies). A recent study (10) using both phenotypic and genotypic analyses of samples provides some support for the use of databases relating genotype to phenotype. Expanded clinical studies are essential to assess the benefit of genotypic and phenotypic testing in the management of HIV-infected patients, and further studies to evaluate the standardization of reporting among laboratories are necessary to ensure consistent and accurate data. The need for careful evaluation of the facility performing HIV drug resistance testing should be evident, as should the need for a thorough understanding of the complexities of testing and analysis and the need for method validation and standardization.

High-quality sequences were obtained in this study using both the VG and the AB assays. The VG assay has been submitted to the FDA, and the AB system is being prepared for submission to the FDA. Concordance of mutations reported by the three assays evaluated was fairly good, but the interpretation of the mutations with respect to resistance characteristics was difficult. It will be challenging to provide reliable and meaningful interpretations until additional standardization of mutations and mutation patterns is provided. In addition, adequate quality control measures and assay validation are needed to ensure accurate and reproducible reporting among laboratories. Physicians should be aware of the limitations and issues involved in genotypic assays for HIV-1 drug resistance and are cautioned to use care in the interpretation of results.