Comparative clinical evaluation of the IsoAmp® HSV assay with ELVIS® HSV culture/ID/typing test system for the detection of herpes simplex virus in genital and oral lesions

Abstract

Background

The novel IsoAmp® HSV Assay employs isothermal helicase-dependent nucleic acid amplification and a user-friendly disposable test device to achieve rapid (<1.5 hours), on-demand qualitative detection of herpes simplex virus (HSV) types 1 and 2 in oral and genital lesions.

Objectives

To compare performance of the IsoAmp® HSV Assay with the ELVIS® HSV ID/typing (shell-vial culture and DFA) test system for clinical specimens collected from oral and genital lesions in symptomatic patients.

Study design

A total of 994 specimens from male and female genital and oral lesions were obtained and evaluated at five study sites in the United States. Results from the IsoAmp® HSV Assay were compared to those from the ELVIS® system. Separate reproducibility studies were performed at 3 sites using a blinded and randomized study panel. Discrepant specimens were resolved by bidirectional sequencing analysis.

Results

After discrepant analysis, overall agreement of IsoAmp® with ELVIS® was 98.8% with 37.0% overall prevalence (all study sites). Reproducibility rates were well within expectations.

Conclusion

The IsoAmp® HSV assay showed excellent performance for clinical use for detection of HSV in genital and oral specimens. In contrast to ELVIS®, IsoAmp® HSV offers excellent sensitivity plus rapid on-demand testing and simpler specimen preparation.

1. Background

Herpes simplex viruses (HSV), serotype 1 (HSV-1) and 2 (HSV-2), are the etiologic agents responsible for a spectrum of human diseases with local, or severe or fatal disseminated presentations including: skin and genital infections, viral meningitis, meningo-encephalitis, and neonatal herpes.^1,2 Currently there is no therapeutic cure for HSV latent infection; treatment aims to reduce symptoms, viral shedding, frequency of recurrence, and transmission during antiviral administration. Diagnosis of genital herpes solely by clinical presentation is insensitive and nonspecific.⁵ Timely and accurate diagnosis is necessary to assist antiviral therapeutic management and counseling for primary infection, intrapartum delivery, and suppressive therapy.

Optimized culture methods are widely available and often preferred for in vitro detection of HSV in mucocutaneous, genital and ocular lesions. The enzyme-linked virus inducible system (ELVIS®, Diagnostics Hybrids, Inc., Athens, OH) detects HSV using transgenic shell vial culture followed by typing of HSV-1 or HSV-2 by fluorescein-labeled monoclonal antisera.^6–8 The detection limit for ELVIS® culture is estimated to be between 0.65- and 8.5-TCID50 for HSV-1 and 0.1- and 8.0-TCID50 for HSV-2 depending on the strain.⁹ Compared to conventional roll tube culture, ELVIS® reduces the maximum time to detection from days to 24 hours while maintaining adequate sensitivity and eliminating subjective detection of cytopathic effect, but both methods require a cell culture facility and associated technical expertise. Polymerase chain reaction (PCR) -based molecular assays detect and subtype HSV with greater sensitivity than shell vial and ELVIS® methods.^7,10,11 Currently, only a few PCR-based assays are FDA-cleared as in-vitro diagnostic tests for HSV: The MultiCode® HSV-1&2 assay (EraGen Biosciences, Inc. Madison, WI) uses novel probe-free PCR-based technology for qualitative detection and differentiation of HSV-1 and HSV-2 in vaginal lesions;¹² and, the BD ProbeTec™ HSV Q^X Amplified DNA Assays (BD Diagnostics, Sparks, MD) employ automated strand-displacement amplification technology for detection and typing of HSV in external anogenital specimens. However, these assays require specialized and relatively expensive instruments. In comparison, the IsoAmp® HSV assay (BioHelix Corporation, Beverly, MA) offers a facile, user-friendly approach for rapid instrument-free detection of HSV that maintains the performance benefits of PCR.

2. Objectives

To compare performance of the IsoAmp® HSV Assay (IsoAmp®) with the ELVIS® HSV ID/typing (shell-vial and DFA) test system (ELVIS®) for clinical specimens collected from oral and genital lesions in symptomatic patients suspected of having herpes infection.

3. Study Design

3.1. Comparative clinical evaluation

A total of 994 clinical specimens (962 prospective, 32 retrospective) from male and female genital and oral lesions were obtained and evaluated at five study sites in the United States: Boston Medical Center, Boston, MA; Cleveland Clinic, Cleveland, OH; University of Virginia Health System, Charlottesville, Virginia; Vanderbilt University Medical Center, Nashville, TN; Laboratory Alliance of Central New York, Liverpool, New York. Institutional Review Board approval was obtained at each site as needed prior to the study. Clinical lesional specimens were collected at each site using Dacron swabs, transported in viral transport medium (VTM) to the respective microbiology laboratory, and processed for HSV detection using ELVIS®. An aliquot of left over eluate from each sample was tested by IsoAmp®. Specimens were de-identified and linked only by a unique study number. Results from IsoAmp® were compared to ELVIS® at the end of the study.

3.2. Reproducibility

Reproducibility studies were performed at 2 clinical sites and at BioHelix using a blinded and randomized 7-member study panel.¹³ HSV viral stock was diluted in HSV Negative Matrix (pooled clinical HSV negative samples) to make the different panel concentration levels measured as TCID50/mL with a negativity (neg %) or positivity rate (pos %) as follows: HSV-1 High Negative =1.75E+04 (neg 30%); HSV-2 High Negative = 2.2E+03 (neg 30%); HSV-1 and HSV-2 Low Positives = 1.1E+05 and 1.1E+04, respectively, (pos 95% each). HSV-1 and HSV-2 Moderate Positives = 3.3E+05 and 3.3E+04, respectively (pos 100%). HSV Negative Matrix was used as the HSV Negative (pos 0%). Panels and controls from one lot were tested twice daily for five days by two operators at each site. Lot-to-lot reproducibility was performed by one site using three lots of the panel.

3.3. IsoAmp® HSV assay procedure

Specimens were tested on IsoAmp® according to the manufacturer’s Research Use Only kit (BioHelix Corp.) as detailed previously.¹³ Briefly, specimen in VTM was diluted with buffer and transferred to an amplification tube to which was added master mix, then mineral oil. Tubes were placed in a 64°C heat block for 60 minutes to amplify target DNA, then placed in a Type II BESt^TM Cassette (BioHelix Corp.) for amplicon detection.^14,15 IsoAmp® uses helicase-dependent amplification (HDA) to achieve isothermal PCR-based amplification of the target HSVglycoproteinB (gB) gene.^16,17 Fluorescien- and digoxigenin-labeled target amplicon and an internal control amplicon are captured and visualized as colored lines on a vertical flow strip within the disposable cassette. Results were read visually after 15 minutes and scored as HSV-present or HSV-absent based on the presence or absence of a test line plus control. This version of the BESt™ Cassette is not designed to discriminate between HSV-1 and HSV-2. The specifics of HDA technology and preliminary evaluation of IsoAmp® HSV were previously reported.^13,16,17 Analytical sensitivity of the assay was estimated at 5.5 and 34.1 copies/reaction for HSV-1 and HSV-2 respectively with excellent specificity.¹³

3.3. ELVIS viral cultures

Viral cultures for detecting HSV were performed using the Enzyme-Linked Virus Inducible System (ELVIS®, Diagnostics Hybrids, Inc., Athens, OH) shell vial assay according to the package insert.⁹

3.4. Discrepant analysis

Residual aliquots of discrepant specimens were analyzed by bidirectional sequencing performed at the GMP sequencing facility of Beckman Coulter Genomics (Morrisville, NC) after sample-processing and purification of PCR products at BioHelix. Primers were designed to have M13 tails at the 5’ ends for bidirectional sequencing; the PCR reaction amplified the 399-base pair target sequence of HSV that encompasses the target sequence of the IsoAmp® HSV Assay. Raw sequence files were imported using DNASTAR Lasergene 8, SeqMan Pro Version 8.0.2 (16), 402. Consensus sequences were analyzed using DNASTAR Lasergene 8, MegAlign Version 8.0.2 (13), 402; Jotun Hein method was used for alignment.

4. Results

4.1. Comparative clinical evaluation

A total of 962 prospective specimens (803 genital, 159 oral) and 32 retrospective specimens (15 genital, 17 oral) were comparatively evaluated by ELVIS® and IsoAmp®. Retrospective specimens (all specimens, all sites), showed 100% agreement between IsoAmp® and ELVIS® (data not shown). Agreement was obtained for 902 prospective specimens (93.8%) with 309 specimens positive and 593 specimens negative by both methods (Table 1). Using ELVIS® as the reference, there were 60 discrepant results for IsoAmp® (49 false-positive and 11 false-negative). After bidirectional sequencing of these discrepants determined that 42 of 49 alleged false-positive IsoAmp® specimens were true positives and 6 of 11 of alleged false-negative IsoAmp® specimens were true negatives, there was 98.8% total agreement with ELVIS® (Table 1). The false negative and false positive samples that were HSV-1 or HSV-2 positive by sequencing were compared to see if discrepancies could be explained by sequence differences but there were no polymorphisms between these two groups.

Table 1.

Overall agreement between IsoAmp® HSV Assay and ELVIS®: Before and after discrepant analysis by bidirectional sequencing, all prospective specimens

Before discrepant analysis					After discrepant analysis
Overall		ELVIS			Overall		ELVIS
		POS	NEG	Total			POS	NEG	Total
IsoAmp	POS	309	49 i	358	IsoAmp	POS	351	7	358
	NEG	11 ii	593	604		NEG	5	599	604
	Total	320	642	962		Total	356	606	962
	Value (%)		95% CI			Value (%)		95% CI
Positive agreement	96.6		94.0 – 98.1		Positive agreement	98.6		96.8 – 99.4
Negative agreement	92.4		90.1 – 94.2		Negative agreement	98.8		97.6 – 99.4
PPV	86.3		82.4 – 89.5		PPV	98.0		96.0 – 99.0
NPV	98.2		96.8 – 99.0		NPV	99.2		98.1 – 99.6
Prevalence	33.3		30.4 – 36.3		Prevalence	37.0		34.0 – 40.1
Total Agreement	93.8				Total Agreement	98.8

POS = positive (HSV detected); NEG = negative (HSV not detected); PPV = positive predictive value; NPV = negative predictive value; CI = confidence interval

ⁱ49 samples were tested using bidirectional sequencing analysis. Sequence analysis detected HSV target in 42 of the 49 discordant samples [18 HSV-1 and 24 HSV-2] identified as HSV Positive by the IsoAmp® HSV Assay. Sequence analysis did not detect HSV in seven (7) of the discordant samples.

ⁱⁱ Eleven (11) samples were tested using bidirectional sequencing analysis. Sequence analysis did not detect HSV target in six (6) of the 11 samples identified as HSV Negative by the IsoAmp® HSV Assay. Sequence analysis did detect HSV in five (5) samples [3 HSV-1 and 2 HSV-2]

After discrepant analysis the individual agreements between IsoAmp® and ELVIS® for the 803 genital and 159 oral specimens were 98.8% and 98.7%, respectively. At each of the five study sites the overall agreement (percentage, total specimen number) between IsoAmp® and ELVIS® was 89.2 (n=74), 90.9 (n=99), 94.1 (n=353), 94.4 (n=350), and 95.3 (n=86), respectively.

4.2 Reproducibility

Reproducibility rates were within expectations with no significant lot-to-lot or site-to-site variation (data not shown). HSV-1 and HSV-2 High Negative samples each showed a 32% negativity rate (expected rate 20 to 80%). HSV Low Positive samples showed an overall 97.5% positivity rate; HSV-1, 99.0%; HSV-2, 96.0%, (expected rate ≥95%). HSV Moderate Positive samples showed a positivity rate of 100% for both viral types. HSV Negative sample performance was 99.5%. One false positive result was observed.

5. Discussion

Compared to ELVIS®, IsoAmp® demonstrated comparable clinical sensitivity and specificity for detection of HSV in specimens from genital and oral lesions from patients suspected of having herpes infection. All positive and negative controls performed as expected. Good inter-laboratory reproducibility was observed for five study sites in the United States. As used with TypeII BESt^™ detection cassettes, IsoAmp® cannot distinguish between HSV-1 and HSV-2. However, in many clinical settings viral typing is secondary to the need for rapid, accurate detection of HSV for patient management.

Discrepant analysis suggests that IsoAmp® assay has greater sensitivity than ELVIS®; this likely accounted for detection of HSV in 42 of 49 discrepant cultures that may have represented low-viral load and/or non-viable virus. Specimen collection devices and transport-to-specimen preparation intervals were not standardized between study sites; this may have affected results for culture-based ELVIS® more so than for the molecular IsoAmp® assay. The 42 discrepants (sequencing +/IsoAmp® +/ELVIS® -) were reviewed with this in mind (data not shown). The rate of these “alleged IsoAmp® false positive” results ranged from 2.1% to 9.1% for the five study sites. At each site the majority of ELVIS® testing was performed between 0 and 3 days from specimen collection (range 0 to 5 days). For each daily test interval there was no significant difference between the percentage of these discrepants compared to IsoAmp® +/ELVIS® + results. The majority of HSV-positive samples were associated with either M4® viral transport medium (Thermo Fisher Scientific) or BD™ Universal Viral Transport System (Becton, Dickinson and Company) -- 49% and 46%, respectively. Of interest, 71% of “alleged IsoAmp® false positives” (now considered probable ELVIS® false-negatives) were associated with M4 compared to 24% associated with BD. Additional pre-analytical variables that may have contributed to the overall results cannot be excluded. The effect of viral load on the accuracy of sequencing analysis could help explain the remaining discordant IsoAmp® false positives not resolved by discrepant analysis.

The IsoAmp® HSV assay consists of simple, dilution-only sample-preparation and isothermal HDA amplification followed by instrument-free detection. It requires an inexpensive heat block and can be performed on-demand without batching -- features that can bring accurate and rapid HSV diagnostics to laboratories lacking culture expertise and/or resources to support expensive molecular instruments. The TypeII BESt^TM disposable cassette virtually eliminates the risk of cross-specimen amplicon contamination. The IsoAmp® format enables stat testing across all work shifts regardless of staff expertise. The IsoAmp® HSV Assay is now FDA-cleared as an in vitro diagnostic. Future modifications of the assay and TypeII BESt^™ Cassette to provide HSV viral typing could further increase the clinical utility of this novel device.

Abbreviations used

HSV

herpes simplex virus

FDA

Food and Drug Administration

HIV

human immunodeficiency virus

ELVIS®

enzyme linked virus inducible system

ID

identification

DFA

direct fluorescent antibody

PCR

polymerase chain reaction

TCID50

50% tissue culture infective dose

DNA

deoxyribose nucleic acid

HDA

helicase-dependent amplification

VTM

viral transport medium