HIV-1 Drug Resistance Assay Using Ion Torrent Next Generation Sequencing and On-Instrument End-to-End Analysis Software

ABSTRACT

HIV-1 antiretroviral therapy management requires sequencing the protease, reverse transcriptase, and integrase portions of the HIV-1 pol gene. Most resistance testing is performed with Sanger sequencing, which has limited ability to detect minor variants. Next generation sequencing (NGS) platforms enable variant detection at frequencies as low as 1% allowing for earlier detection of resistance and modification of therapy. Implementation of NGS assays in the clinical laboratory is hindered by complicated assay design, cumbersome wet bench procedures, and the complexity of data analysis and bioinformatics. We developed a complete NGS protocol and companion analysis and reporting pipeline using AmpliSeq multiplex PCR, Ion Torrent S5 XL sequencing, and Stanford’s HIVdb resistance algorithm. Implemented as a Torrent Suite software plugin, the pipeline runs automatically after sequencing. An optimum variant frequency threshold of 10% was determined by comparing Sanger sequences of archived samples from ViroSeq testing, resulting in a sensitivity of 98.2% and specificity of 99.0%. The majority (91%) of drug resistance mutations were detected by both Sanger and NGS, with 1.7% only by Sanger and 7.3% only by NGS. Variant calls were highly reproducible and there was no cross-reactivity to VZV, HBV, CMV, EBV, and HCV. The limit of detection was 500 copies/mL. The NGS assay performance was comparable to ViroSeq Sanger sequencing and has several advantages, including a publicly available end-to-end analysis and reporting plugin. The assay provides a straightforward path for implementation of NGS for HIV drug resistance testing in the laboratory setting without additional investment in bioinformatics infrastructure and resources.

INTRODUCTION

Guidance of antiretroviral therapy (1) for the most commonly used drug classes to treat HIV-1 requires knowledge of preexisting or emerging mutations in protease (PR), reverse transcriptase (RT), and integrase (IN) portions of the HIV-1 pol gene (2–4). Most testing is performed using population Sanger sequencing, including the widely used ViroSeq Genotyping (5, 6) and Integrase kits (Abbott Molecular) (the former having FDA-IVD approval), which has a sensitivity for detecting resistance variants at ~20% of the total population (6–11). The ViroSeq assays were designed primarily to sequence subtype B (6), the predominant subtype in North America (12–14) and performance of some ViroSeq primers has been problematic for non-B subtype samples (15, 16). Although the ViroSeq assays have been used extensively for several years, they were discontinued by the manufacturer at the end of 2021, necessitating a replacement assay.

Next generation sequencing (NGS) platforms enable detection of variants at frequencies as low as 1% of the viral population allowing for earlier detection and modification of therapy (17–24). The presence of HIV-1 mutations below 20% frequency has been associated with treatment failure (25–28), although establishing clinically relevant frequency thresholds is problematic (29–32). NGS assay design for HIV-1 is challenging, considering the highly mutated genome and wide range of viral loads in patient samples that require drug resistance testing. Robust NGS assays require careful selection of priming sites, optimization of thermocycling conditions and reagent component concentrations, and establishment of the best conditions for sequencing.

Ion AmpliSeq technology (https://ampliseq.com/; Thermo Fisher) (33) is a method allowing highly multiplexed PCR and sequencing of more than 20,000 amplicons. Separate amplicon pools can be combined to create redundant coverage of regions of interest. The resulting products are ligated to barcoded adapters to uniquely identify samples during sequencing. The barcoded product libraries are clonally amplified on capture beads in emulsion PCR to generate single-template substrates for sequencing on a semiconductor chip. This clonal amplification and chip loading can be automated with the Ion Chef instrument (Thermo Fisher) (33). The method can be used in many applications, and has been used recently for SARS-CoV-2 sequencing (34, 35), cancer genotyping of FFPE tissue (36), forensics (37, 38), exome sequencing (39), gene fusion analysis (40–43), detection of variants in cell-free DNA (44), gene expression profiling (45, 46), analysis of solid tumors (47), and identification of variants in inherited disease (48).

Another barrier to the implementation of NGS assays in the clinical laboratory is the complexity of the data analysis and bioinformatics. The additional software, hardware, expertise to develop the analysis, and regulatory requirements can be daunting. We developed the assay for the Ion Torrent platform, which allows for user-developed custom analysis via the Torrent Suite software that runs on the sequencing system. Torrent Suite is web browser-based, network accessible, and provides secure user-level access control. The software is installed as a plugin on any Ion S5 or GeneStudio series sequencer and provides a complete analysis solution, including a user-friendly web-based results and reporting portal, thus eliminating some of the barriers associated with NGS testing.

Interpretation of HIV-1 drug resistance testing is a complicated and evolving process. There are several resources for resistance test interpretation (49, 50). Stanford University’s HIV Drug Resistance Database (HIVdb) (https://hivdb.stanford.edu/) (4, 51, 52) is widely used and considered the gold standard among many experts in the field. It provides several advantages, including regular updates, well-documented and transparent algorithms, and a web service that allows users to interact programmatically with the interpretation algorithm through an application programming interface (API).

Currently there is only one HIV-1 NGS assay with FDA-IVD approval, the Sentosa SQ HIV-1 Genotyping Assay (Vela Diagnostics) (53–56). The Sentosa assay consists of reagents and instruments to perform semi-automated sample extraction, amplification, library preparation, sequencing, and report generation using Ion Torrent sequencing.

We developed a complete NGS assay system using AmpliSeq multiplex PCR and sequencing on the browser-based Ion Torrent S5 XL platform (Thermo Fisher), built a companion analysis and reporting pipeline in-house that incorporates the Stanford HIVdb algorithm and web service, and implemented it as a freely available, configurable Torrent Suite plugin.

MATERIALS AND METHODS

The assay was validated by comparing NGS results to Sanger sequencing results produced by the ViroSeq assays using archived plasma samples. Direct comparison of drug resistance results was not done due to a number of factors; (i) the ViroSeq assay for PR/RT was FDA-approved several years ago and uses a proprietary interpretation algorithm, (ii) the ViroSeq assay does not cover amino acid 348 in RT (which is included in Stanford’s algorithm), and (iii) many of the plasma samples used have been archived over a number of years, so the corresponding archived ViroSeq interpretations may have changed over time. Therefore, rather than compare drug resistance results, the nucleotide sequences that were generated by the ViroSeq assays were used for comparison. Sanger and NGS sequences were both analyzed with the Stanford HIVdb algorithm so a direct comparison could be made.

Study approval.

This study was performed under IRB protocol 24431 from the University of Utah.

Sanger sequencing.

Sanger sequencing was performed according to the manufacturer directions (ViroSeq HIV-1 Genotyping System v2.0 and ViroSeq HIV-1 Integrase Genotyping Kit; Abbott Molecular) with the following modifications: the input volume for nucleic acid extraction was increased from 500 μL to 1 mL to improve sensitivity; samples were extracted on a QIAsymphony SP system using the QIAsymphony Virus/Bacteria Midi Kit (Qiagen); and cycle sequencing products were electrophoresed on a 3730XL DNA Analyzer (Thermo Fisher). The ViroSeq Genotyping kit amplifies a 1.8 kb amplicon that is used as a sequencing template for 7 primers to generate a consensus sequence of 1302 bp (codons 1–99 of PR and 1–335 of RT). The ViroSeq Integrase kit amplifies the entire integrase gene (codons 1–288) in a 1.1 kb amplicon that is used as a sequencing template for four primers to generate a consensus sequence of 864 bp. PR/RT sequences were aligned, trimmed, and edited using the FDA-approved ViroSeq HIV-1 Genotyping Software version 3.0 (Abbott Molecular). Integrase sequences were aligned and trimmed using SeqScape v2.6 (Thermo Fisher) (49). The resulting consensus nucleotide sequences for PR/RT and IN were compared to NGS results. Drug resistance mutations (DRM) and interpretations were determined using the Stanford HIV Resistance Database (4) (version 8.8; 2/13/2019).

NGS assay design.

Two AmpliSeq primer pools generating a total of 17 overlapping amplicons were designed in collaboration with Thermo Fisher’s AmpliSeq Custom Services “White Glove” team for redundant coverage of all 87 resistance-associated sites (Table S1 in the supplemental material) from Stanford’s HIVdb algorithm (version 8.8; 2/13/2019), except two with single amplicon coverage (Fig. 1) (RT:E44 and RT:Q151). This version of Stanford’s HIVdb reports resistance levels for 24 drugs [8 Protease Inhibitors (PI), 7 Nucleos(t)ide Reverse Transcriptase Inhibitors (NRTI), 5 Non-nucleoside Reverse Transcriptase Inhibitors (NNRTI), and 4 Integrase Strand Transfer Inhibitors (INSTI)]. The assay was designed to detect all subtypes of HIV-1 group M, based on an alignment of ~75,000 HIV sequences mapped to the reference HXB2 genome (GenBank K03455), including patient sequences generated at ARUP Laboratories using the ViroSeq PR/RT (n = ~52,500) and IN Sanger systems (n = ~5,200). Publicly available sequences containing the pol region (n =~3,800 from the Web Alignments at https://www.hiv.lanl.gov/content/sequence/NEWALIGN/align.html) and the vif region (n =~12,800 from GenBank) were also included. Primers were designed with mixed nucleotide bases to account for the most common single nucleotide variants. No more than two mixed nucleotide positions were allowed per primer, nor were they allowed at the three 3′ terminal positions.

FIG 1.

NGS assay diagram. The regions of the HIV genome covered by the assay are shown (PR: Protease, RT: Reverse Transcriptase, RN: RNase, IN: Integrase). Pool 1 amplicons (first row) and Pool 2 amplicons (second row) are shown with 87 drug resistance mutation sites based on Stanford’s HIVdb version 8.8 indicated by vertical bars.

NGS library preparation.

Library preparation is outlined in Fig. S1 in the supplemental material. RNA was extracted from 1 mL of patient plasma in batches of 24 (including negative and positive controls) using the Chemagic Viral NA/gDNA kit on the Chemagic MSM I instrument (PerkinElmer) and eluted in 100 μL. RNA (7 μL) was reverse transcribed using the SuperScript VILO cDNA Synthesis Kit (Thermo Fisher) in a 10 μL reaction. Two amplicon pools were generated with 3 μL of cDNA in a final volume of 10 μL per pool using the Ion AmpliSeq Library Kit 2.0 (Thermo Fisher). The two amplicon libraries were combined and digested with 2 μL FuPa reagent provided in the kit. Unique IonCode Barcode Adapters (Thermo Fisher) for each sample on a run were ligated to the amplicon libraries. Libraries were purified using 40 μL Ampure XP magnetic beads (Beckman), washed twice in 70% ethanol, and eluted in Low TE buffer. The resulting libraries were quantitated using the KAPA Library Quantification Kit (Roche), concentrations were normalized to 50 pM, and up to 48 samples were pooled at equimolar amounts. The automated Ion Chef (Thermo Fisher) instrument automatically performed template preparation of 25 μL sample pool using Ion 510/520/530 Chef Kits and Ion 530 Chips that were sequenced using the Ion S5 XL sequencer (Thermo Fisher). A detailed procedure can be accessed online (see Data Availability section).

NGS assay error analysis.

A commercially produced plasmid (IDT) containing the HIV reference sequence (HXB2, GenBank: K03455) spanning the NGS assay was used to assess errors specific for this NGS assay. The plasmid insert was confirmed by the vendor and by ARUP via Sanger sequencing. NGS libraries prepared from two replicates of the plasmid and one replicate of an RNA transcript generated from the plasmid were sequenced on two separate runs (six replicates total).

NGS analysis plugin.

All analysis and reporting functions are contained in a single Torrent Suite plugin developed by the authors, summarized here and available online (see Data Availability section below). A flowchart of the analysis and reporting process is shown in Fig. S1 in the supplemental material. Figure S2 in the supplemental material shows the interface for user configurable settings (HIVdb web address; variant calling and reporting thresholds). The plugin is specifically designed to work with the AmpliSeq primers, library preparation, and sequencing process described above. To summarize, the BAM file produced by Ion Torrent sequencing is processed to identify and filter human reads using Kraken2 (57), the remaining reads are aligned to the HXB2 reference sequence (GenBank K03455) with the Burrows-Wheeler Aligner (BWA) (58). Aligned reads undergo quality screening, primer trimming, amplicon tagging, and down-sampling using internally developed scripts to create a processed BAM file. Coverage depth at each position is calculated using Samtools (59) and single nucleotide variants and indels at 1% frequency or greater are called using LoFreq (60). A consensus sequence including positions with a minimum of 100 reads is generated for each sample. The processed BAM file is then realigned using the sample consensus instead of the HXB2 reference to refine the results. Variant calls that are produced predominantly from one strand direction (Fig. 2) are filtered. The resulting variant calls are evaluated in codon space, indels and the surrounding regions are corrected, and the final variants (at or above the user-selected reporting frequency threshold; Fig. S2) are submitted to the Sierra Web Service using the “Patterns” option (https://hivdb.stanford.edu/page/webservice/). The plugin automatically considers any DRM sites with insufficient coverage (<100×) and submits an additional query to the Stanford tool with all possible amino acid substitutions at the missing DRM sites (described further in Supplementary Information and Table S2). The plugin compares the drug resistance profiles of both submissions. If the supplemented result changes the drug resistance level of any drug, that drug is reported as “not determined.” If the resistance profile is unchanged, a complete report is released. Missing DRM sites are noted on the report.

FIG 2.

Assay errors and effect of strand bias filtering at drug resistance mutation (DRM) sites. Yellow circles (False) = NGS errors relative to known sequence not filtered by strand bias; Gray diamonds (True) = NGS errors filtered out by strand bias implementation in plugin. Blue line shows the variant calling threshold at 10%. Inset: Strand bias ratio (R_sb) calculation. Measurement of variant (alternative; alt) detection biased to either the Forward (F) or Reverse (R) strand. For example, a variant detected in 622 of 1026 Forward reads and 77 of 342 Reverse reads has a ratio of 2.7 [(622/1026)/(77/342) = 0.606/0.225 = 2.7].

A graphical user interface in the Torrent Suite software contains visual details on the sample results, including the locations and frequencies of variants, drug resistance interpretations, a variant table, a JSON output suitable for use by a LIS, and a PDF report. Further details and examples can be found in the Supplementary Information (Fig. S1–S6 in the supplemental material).

Validation samples.

Thirty unique HIV-1 seronegative plasma samples were tested by the NGS assay. One hundred sixty (160) unique HIV-positive plasma samples with successful ViroSeq Sanger sequencing results (PR/RT and/or IN) were tested by NGS (Table S3 in the supplemental material). These archived samples were selected to cover a range of viral loads (previously determined) and to include as many DRMs as possible. They include transitions, transversions, and indels. Approximately 5–8% of United States HIV-1 infections are caused by non-B subtypes (12–14), so 30 of 160 (18.8%) samples were non-B subtypes [A (n = 5), B+F (n = 1), C (n = 4), CRF01_AE (n = 3), CRF02_AG (n = 9), CRF06_cpx (n = 1), CRF24_BG (n = 2), D (n = 1), F (n = 2), G (n = 2)]. To determine the status of codon 348 in RT, which is not covered by the ViroSeq assay, this region was sequenced for 19 samples (Table S3 in the supplemental material) as described in the Supplemental Information.

Nine patient samples were tested for reproducibility. Three were tested in triplicate (all processes for each sample, including extraction, were performed on the same run) to evaluate intra-assay reproducibility; three were tested on three independent runs (all processes including extraction) to evaluate inter-assay reproducibility; and three samples were tested for both inter- (three replicates) and intra-assay (two additional replicates) reproducibility.

Three samples were tested to determine the analytical sensitivity (limit of detection [LOD]). The WHO International Standard (HIV-1 RNA, 2nd International Standard, NIBSC code: 97/650) was tested in triplicate and two patient samples were tested without replicates. Two-fold dilutions from 4000 to 31 copies/mL were tested for each sample. The lowest concentration where all DRM sites in all replicates had adequate (≥100×) coverage was defined as the LOD.

Receiver operating characteristic (ROC) analysis.

ROC analysis (61) shows the trade-off relationship between sensitivity and specificity as the variant frequency threshold is varied by comparing the results of a new test to those of an established “gold standard” test. The analysis is used to find the threshold where sensitivity and specificity are maximized. This threshold represents the optimum setting for the new test to match results from the gold standard test. This analysis was performed for variants at 1, 5, 10, 15, and 20% variant frequency thresholds. Sanger sequences were used as the gold standard for characterizing variants. True positives were defined as variants that matched in Sanger and NGS. False positives were defined as variants that were detected in NGS at or above the given frequency threshold but were not detected by Sanger. True negatives were defined as variants that matched the reference sequence (i.e., not mutated) in both Sanger and NGS. False negatives were defined as variants present in Sanger but not detected by NGS at the given frequency threshold or higher. False negative calls were adjusted to count unique amino acid sites; multiple variants in a sample at the same amino acid site were counted as a single result.

Data availability.

The Torrent Suite plugin including source code, a User Guide, and a library preparation procedure can be accessed at https://github.com/kes1smmn/ARUPHIVGenotyper/releases/.

RESULTS

NGS assay error.

Variants that did not match the expected sequence in the plasmid and/or transcript were observed scattered at 64 unique positions throughout all genes covered by the assay, with a mean variant frequency of 6.7% (data not shown). At drug resistance mutation (DRM) sites specifically (Fig. 2), seven sites had errors (RT:E44G, RT:A98T, IN:T66A, IN:L74L, IN:Q146K, IN:S147V, IN:S153P), with a mean variant frequency of 6.8%. A Strand Bias Ratio filter strategy (Fig. 2 inset) was implemented to remove calls that had ≥5x the proportion of reads from one direction compared to the other direction (Strand Bias Ratio ≥5). After applying the strand bias filter, two DRM sites had errors that were not filtered: five replicates between 2 and 6% (IN:L74L) and one replicate at 1% (IN:S153P). This filtering strategy was integrated into the plugin as part of the standard analysis of all samples.

Overall sequencing success.

One hundred twenty-four of 160 (77.5%) accuracy samples had adequate coverage (≥100×) of all 87 DRM sites, 29 (18.1%) were missing a single DRM site, and seven (4.4%) were missing two or more DRM sites (Table S4 in the supplemental material). For samples missing a single DRM site, 23/29 (79.3%) were missing RT:E44 (one of two sites with only single amplicon coverage), none of which had the mutation combination required for RT:E44 to affect drug resistance (Supplementary Information and Supplementary Table S2). Therefore, one hundred forty-seven of the 160 (91.9%) accuracy samples produced drug resistance reports for all 24 drugs. One hundred twenty-six of 130 (96.9%) subtype B samples and 27/30 (90.0%) non-B subtype samples were missing one or fewer DRM sites (P = 0.095).

Establishing optimum variant frequency threshold with ROC analysis.

At the 87 DRM sites among the 160 samples, a total of 12,578 amino acid codons were successfully sequenced by both NGS and Sanger. Of those, 11,094 (88.2%) amino acids matched the reference sequence in both Sanger and NGS (Table 1) using a variant frequency threshold of 1%. There were 887 (7.1%) amino acid variants detected in both Sanger and NGS, 583 (4.6%) in NGS only, and 14 (0.1%) in Sanger only. This resulted in a true positive rate (TPR; sensitivity) of 0.989 and false positive rate (FPR; 1-specificity) of 0.05 at a variant frequency threshold of 1%. As the variant frequency threshold is increased, the sensitivity decreases and the specificity increases. Compared to the 1% threshold, three true positives are not detected at the 5% threshold, but 372 false positives are eliminated. These results are not unexpected due to Sanger’s poorer ability to detect minor frequency variants compared to NGS; most of these additional false positives are likely present in the sample (see discordant analysis below). As shown in Fig. 3, as the TPR approaches 1.0, more variants detected by Sanger are also detected by NGS. As the FPR approaches 1.0, more variants are detected by NGS that were not detected in Sanger sequence. The best compromise between TPR and FPR (TPR approaching 1; FPR approaching zero) occurs at a variant frequency threshold of 10%, where sensitivity was 98.2% and specificity was 99.0%. Similar results were found when comparing nucleotides (missense and synonymous amino acids) (Table S5, Fig. S7 in the supplemental material).

TABLE 1.

Summary of amino acid results (missense) at drug resistance mutation sites used for receiver operating characteristic (ROC) analysis with variant frequency thresholds between 1 and 20%^a

Threshold (%)	True positive rate (sensitivity)	False positive rate (1−specificity)	Specificity	True positive	False positive	True negative	False negative	Adjusted false negative
1	0.989	0.050	0.950	887	583	11094	14	10
5	0.987	0.019	0.981	884	211	11093	16	12
10	0.982	0.010	0.990	879	117b	11092	20c	16
15	0.977	0.007	0.993	875	77	11087	27	21
20	0.970	0.004	0.996	869	50	11083	43	27

^aTrue positive: Sanger and NGS variants match; False positive: NGS positive, Sanger negative; True negative: Sanger and NGS match reference; False negative: NGS negative, Sanger positive; Adjusted false negative: Unique amino acid sites.

^bSee Discordant analysis and Table 3.

^cSee Discordant analysis and Table 4.

FIG 3.

Receiver Operating Characteristic (ROC) curve. The relationship between the True Positive rate (sensitivity) and False Positive rate (1-specificity) for missense amino acids at drug resistance mutation sites as the variant frequency threshold is varied between 1 and 20%. The ideal variant frequency (where sensitivity and specificity are maximized) is 10%.

Variant agreement with Sanger for drug resistance mutations.

At the 10% variant frequency threshold, there were 532 DRMs among the 160 samples (Table 2). As expected, the majority (91%) were detected in both Sanger and NGS. A small proportion (1.7%) of variants were present in Sanger but not detected at that threshold in NGS. A larger proportion (7.3%) were not detected in Sanger but were present above the threshold in NGS. Similar proportions were observed in all three genes (PR, RT, and IN).

TABLE 2.

Summary of drug resistance mutation agreement between Sanger and NGS at 10% variant frequency threshold

Gene	Sanger and NGS variants agree	Sanger negative & NGS positive	Sanger positive & NGS negative or below threshold	Total
PR	99 (88.4%)	12 (10.7%)	1 (0.9%)	112 (100%)
RT	312 (91.5%)	23 (6.7%)	6 (1.8%)	341 (100%)
IN	73 (92.4%)	4 (5.1%)	2 (2.5%)	79 (100%)
Total	484 (91.0%)	39 (7.3%)	9 (1.7%)	532 (100%)

Most discordant variants could be confirmed by manually evaluating the Sanger electropherograms. Discordant analysis of the 117 false positives (NGS variants identified at or above 10% but not detected by Sanger) (Table 1) was performed by reviewing the Sanger electropherograms. The majority (85/117; 72.6%) of false positive discordant results had evidence in Sanger electropherograms supporting the NGS results (Table 3). Support was categorized as “yes” (clear evidence in Sanger), “no” (No evidence in Sanger), and “inconclusive” (variable, inconsistent, or low-quality evidence in Sanger). Further testing of these samples by another method (e.g., Sentosa sequencing) was not possible due to inadequate remaining plasma volume.

TABLE 3.

Discordant analysis details for 117 Sanger negative/NGS positive variant results (false positives) at 10% variant frequency threshold

	Sanger electropherogram review supports NGS result
Variant frequency	Yes	No	Inconclusive	Total
10–19%	51 (43.6%)	13 (11.1%)	2 (1.7%)	66 (56.4%)
20–29%	21 (17.9%)	3 (2.6%)		24 (20.5%)
30–39%	8 (6.8%)	6 (5.1%)	1 (0.9%)	15 (12.8%)
40–49%	4 (3.4%)	1 (0.9%)	2 (1.7%)	7 (6.0%)
50–100%	1 (0.9%)		4 (3.4%)	5 (4.3%)
Total	85 (72.6%)	23 (19.7%)	9 (7.7%)	117 (100.0%)

The HIV genome position and Sample ID were evaluated in the false positive discordant samples to determine if there were any specific genome areas (Fig. 4) or samples (Fig. S8 in the supplemental material) that were more prone to errors or sequencing problems. Discordant results were found at most DRM sites and in most samples, indicating that the results were random and not likely caused by specific nucleotide sequences or problematic samples.

FIG 4.

False Positive results. Discordant results at drug resistance mutation sites by HIV genome position and variant frequency for 117 Sanger negative/NGS positive results at 10% variant frequency threshold.

For 20 false negatives (Sanger variants not identified by NGS at or above 10%) (Table 1), four were not evaluated since they were at the same amino acid site as other false negatives and six were detected by NGS but at <10% frequency. The remaining 10 were included in the review of Sanger electropherograms for evidence supporting the NGS results (Table 4). Six of 10 variants were present in Sanger sequence, three were Inconclusive, and one was an artifact caused by an insertion that was removed from the Sanger sequence (therefore, not discordant).

TABLE 4.

Discordant analysis details for 10 unique drug resistance mutation (DRM) position Sanger positive/NGS negative variant results (false negatives) at 10% variant frequency threshold

DRM site	DRM missed causes drug resistance	Sanger electropherogram review supports Sanger result
IN:143	True	Inconclusive
IN:157	False	True
RT:108	True	True
PR:90	False	Inconclusive
RT:68	Artifact	Artifact
RT:219	True	Inconclusive
IN:149	False	True
RT:215	True	True
IN:95a	False	True
RT:74	True	True

^aPresent in NGS; filtered for strand bias (Strand Bias Ratio = 5.8).

Specificity, reproducibility, and limit of detection.

Nucleic acid extracts from samples containing high titers of other viruses (VZV, HBV, CMV, EBV, and HCV) were tested in the NGS assay (Table S6 in the supplemental material). None of these samples generated ≥100x coverage depth at any amino acid position in the assay.

Thirty unique HIV-seronegative plasma samples were tested (data not shown). Two samples had ≥100× coverage at two DRM sites [470× coverage (RT:E40 and RT:M41); 339× coverage (RT:E40) and 362× coverage (RT:M41)]. All these samples would be interpreted as “indeterminate” (no drug resistance results reported) by the plugin, as expected.

Intra- and inter-assay variant frequencies (Table S7 in the supplemental material) were highly reproducible; the maximum difference in frequencies between replicates for any variant was 13% in one inter-assay sample. The maximum percent coefficient of variation (% CV) was 12.4 for intra-assay and 16.9 for inter-assay replicates. One inter-assay sample had one replicate for which 100x coverage was not obtained for 4 of 11 DRMs detected, but the other 2 replicates had only 4% differences between the variant frequencies. This sample had a lower viral load (3.2 log copies/mL).

The limit of detection (analytical sensitivity) was defined as the lowest concentration for which all 87 DRM sites had adequate (≥100×) coverage. For the WHO International Standard (triplicate) and two patient samples, the LOD was defined as 500 copies/mL (Table S8 in the supplemental material). Nevertheless, several accuracy samples with lower viral loads were successfully sequenced (Table S4 in the supplemental material).

Proportion of samples with variants at drug resistance mutation sites.

For each of the 87 DRM sites in the accuracy samples, the percentages of variants that contribute to drug resistance (DRMs) and other mutations (variants that do not contribute to drug resistance) were determined (Table S1 in the supplemental material). DRMs were detected in 75/87 (86.2%) DRM sites. Ten DRM sites only had variants not associated with drug resistance detected; only two sites had no variants detected (amino acids 121 and 145 in the IN gene).

DISCUSSION

We developed and validated an HIV drug resistance assay based on AmpliSeq PCR and Ion Torrent sequencing and implemented a customized companion analysis and reporting pipeline as a Torrent Suite software plugin. The assay was optimized with a single set of conditions for a wide range of HIV viral loads and the validation included 130 subtype B samples and 30 samples consisting of 10 non-B subtypes. A variant frequency threshold of 10% was selected based on ROC analysis showing 98.2% sensitivity and 99.0% specificity for amino acid calls at 87 DRM sites compared to Sanger sequencing. For 532 DRMs detected at this threshold, 91% agreed for NGS and Sanger, 7.3% were detected only by NGS, and 1.7% were detected only by Sanger. The LOD was determined to be 500 copies/mL, although complete results were obtained for some samples as low as 20 copies/mL.

Sequencing error types and frequencies vary between NGS platforms, which are summarized by Casadellà and Paredes (18), but sequencing errors are also dependent on factors specific to the assay, such as library preparation methods, the target sequence, and bioinformatics pipelines (62). Implementation of a strand bias filtering strategy removed all errors with variant frequencies above the 10% frequency threshold. This strategy compensates for some of the limitations of Ion Torrent sequencing regarding sequencing polynucleotide repeats and other anomalies that lead to strand bias.

Failure to adequately sequence a DRM site due to insufficient viral load or polymorphisms in primer binding regions can cause incomplete results. The NGS assay was designed with redundant amplicon coverage of all but two of the DRM sites (RT:E44 and RT:Q151) to mitigate the effects of amplicon drop-out due to unexpected polymorphisms in primer binding regions. Since Stanford’s HIVdb scoring algorithm to determine drug resistance includes evaluating single mutations and mutation combinations, the missing site must be evaluated in context to produce as complete a report as possible. The plugin accounts for missing DRM sites to produce a report for all unaffected drugs (described in the Supplementary Information, Table S2). This strategy provides as much information as possible to the treating physician instead of invalidating an entire drug class or reporting no drug resistance results at all and can produce useful results for low viral load samples. This analysis is cumbersome with Sanger sequencing without a bioinformatic solution.

The NGS assay has several advantages over Sanger sequencing. Adjacent nucleotide variants within a codon that can confound Sanger sequencing can be resolved using NGS (63), eliminating ambiguous amino acid calls that can affect drug resistance interpretation. The proportion of variants in the HIV population circulating in a patient can be quantitated and reporting thresholds can be set. Although no specific threshold has been established for clinical significance, there is mounting evidence that minority variants that Sanger sequencing is unable to detect play an important role in therapeutic outcomes (22, 23, 27, 31, 64–70). Setting a variant threshold that matches the performance of the Sanger assay as closely as possible, as was done in the present study, could be the near-term practice. As more laboratories adopt NGS methods and clinical studies are conducted, the variant frequency that provides optimum treatment outcomes can be established. One large, carefully controlled study demonstrated the importance of balancing sensitivity and specificity by selecting the variant frequency threshold (32), but concluded that further studies are required. Additionally, ideal thresholds may be variable for different drug classes, specific variants, and patient populations.

Although there are several algorithms available for analysis of NGS data (50, 71, 72), some only generate amino acid variant tables and require several further steps to generate a drug resistance interpretation report. Noguera-Julian et al. (73) outlined seven characteristics for the ideal NGS analysis software including ease of use, scalability, data management, quality control, and limited infrastructure. The analysis pipeline we developed meets these criteria for users of the Ion Torrent platform. The implementation of the Torrent Suite plugin for analysis and reporting simplifies and streamlines the difficult task of reviewing, interpreting, and reporting HIV drug resistance results. The plugin processes data automatically after the completion of a sequencing run and produces files that can be consumed by laboratory information systems (LIS) to minimize the amount of technical hands-on time and reduce errors.

The ViroSeq PR/RT assay was the only FDA-IVD approved Sanger assay available for several years. However, it was discontinued at the end of 2021. Therefore, there is a need for replacement assays. Vela Diagnostics has received FDA-IVD approval for their Sentosa SQ assay based on Ion Torrent sequencing. The Sentosa SQ assay uses a previous generation of Ion Torrent sequencers (Ion PGM; Thermo Fisher) with a lower sequencing capacity per run and shorter read length than the more current Ion S5 and GeneStudio series of sequencers and Ion Chips used in the current assay. The Sentosa SQ assay is semi-automated and includes a liquid handler for sample extraction and library preparation. The assay described here used a Chemagic liquid handler for extraction of 24 samples. The library preparation steps were performed manually, but could be automated by common liquid handling systems, and the template and chip preparation processes are completely automated by the Ion Chef instrument. In this validation, we combined 48 samples on a single Ion 530 Chip, but similar results were obtained with 96 samples on an Ion 530 Chip or with 24 samples on an Ion 520 Chip (data not shown). The Sentosa SQ assay has been evaluated compared to other NGS platforms (54), other Sanger sequencing protocols (including one for proviral DNA) (53, 55, 73), but only recently to the PR/RT portion of the ViroSeq assay on a small number of samples (74).

In conclusion, the NGS assay performed comparably to ViroSeq Sanger sequencing, but has several advantages, including others not discussed here such as decreased cost and increased scalability. The analysis and drug resistance interpretation are completely automated, thereby eliminating the subjectivity that can be involved in interpreting low signal peaks produced by variants in Sanger sequencing. The plugin is integrated in the sequencer’s Torrent Suite software and runs on the same hardware as the sequencer itself, so no additional hardware investment is required. As a self-contained plugin, it also requires no sophisticated bioinformatics experience, but the source code is available for those that wish to customize the analysis or output. The Sierra Server used for HIV drug interpretations is decoupled from the plugin allowing for use of the current publicly hosted HIVdb (v9.0) or a locally installed server, thus controlling the scoring algorithm, software version, and update processes. As expected, NGS detected low frequency mutations that could not be detected by Sanger (Table 1), which could improve treatment selection and patient outcomes. In addition, there is evidence that NGS will allow successful resistance mutation detection for lower titer samples than with ViroSeq Sanger sequencing and is better able to detect indels that sometimes confound analysis of Sanger sequencing.