Evaluation of the Luminex ARIES HSV 1&2 Assay and Comparison with the FTD Neuro 9 and In-house Real-Time PCR Assays for Detecting Herpes Simplex Viruses

Abstract

Background

Human herpes simplex virus type 1 (HSV-1) and type 2 (HSV-2) are responsible for a plethora of human diseases, of which cutaneous and mucocutaneous infections are the most prevalent. In its most severe form, HSV infection can cause meningitis/encephalitis. We compared the Luminex ARIES HSV 1&2 assay (Luminex Corp., Austin, TX, USA), an automated sample-to-result molecular solution, to two non-automated HSV DNA assays.

Methods

A total of 116 artificial controls were used to determine the analytical performance of the ARIES assay. Controls were prepared by spiking universal transport medium (UTM) and cerebrospinal fluid (CSF) samples from patients who tested negative for HSV by an in-house HSV-1 and -2 DNA assay with reference materials (SeraCare Life Sciences, MA, USA; ZeptoMetrix Corp., MA, USA). Another 117 clinical samples were then used to compare the clinical performance of the ARIES assay with those of an in-house assay and the FTD Neuro 9 assay (Fast Track Diagnostics, Junglinster, Luxembourg).

Results

The analytical sensitivity (95% limit of detection) of the ARIES assay was 318 copies/mL (UTM samples) and 935 copies/mL (CSF samples) for HSV-1 strain 96 and 253 copies/mL (UTM samples) and 821 copies/mL (CSF samples) for HSV-2 strain 09. No cross-reactivity was observed in samples spiked with 14 non-HSV microorganisms. Compared with the reference result (agreement between the in-house and FTD Neuro 9 results), the ARIES assay had overall concordance rates of 98.2% (111/113) and 100% (113/113) for HSV-1 and HSV-2, respectively.

Conclusions

The ARIES assay appears to be an excellent alternative for rapid detection and differentiation of HSV in skin and genital infections, meningitis, and encephalitis.

INTRODUCTION

Herpes simplex virus types 1 and 2 (HSV-1 and -2; Family Herpesviridae, genus Simplexvirus) are double-stranded DNA viruses; they are a common cause of herpetic infection, which is characterized by sores around the mouth or face and the genitals, buttocks, or anal area, respectively [1,2]. The WHO estimated in 2012 that several billion people were infected with HSV, making it a public concern [3]. HSV is transmitted easily through direct contact with cutaneous and mucocutaneous lesions, as well as by exposure to infected biofluids. HSV infection can range from relatively benign asymptomatic infection to chronic infection with recurrent symptomatic flares. Moreover, viral shedding episodes can occur in the absence of active lesions or symptoms and is largely responsible for genital HSV transmission [3,4,5]. Genital herpes during pregnancy is another major concern. At delivery, HSV can cause severe neonatal morbidity and mortality [6]. In its most severe form, HSV infection can cause meningitis/encephalitis, which can prove fatal [7,8]; in these cases, rapid diagnosis is crucial. Thus, rapid and accurate diagnosis of HSV infection is essential for competent clinical decision making and effective real-time patient care.

HSV infection can be diagnosed using a variety of laboratory methods, including viral culture, serology (immunofluorescence and direct immunofluorescent antibodies), and PCR assays [9,10]. Of these methods, viral culture is labor-intensive and has a long turnaround time, whereas serological testing may be affected adversely by cross-reactivity between the two virus subtypes. Thus, PCR is the preferred method for rapid identification of HSV infections. More recently, a new 24-hour culture test, the enzyme-linked virus inducible system (ELVIS; Quidel, San Diego, CA, USA), has been applied instead of the conventional time-consuming viral culture system in diagnostic testing [11]. Nonetheless, PCR assays still outperform ELVIS in terms of sensitivity and turnaround time [12,13]. A previous study found that 9% of asymptomatic pregnant women in labor are positive for HSV infection by PCR despite testing negative for HSV by viral culture [14]. However, an inherent limitation of PCR assays is that they require highly trained technicians.

The Luminex ARIES HSV 1&2 assay (Luminex Corp., Austin, TX, USA) is a fully automated sample-to-result commercial PCR assay for rapid detection and differentiation of HSV DNA using the ARIES system. This assay has recently been approved by the US Food and Drug Administration (FDA) for exclusively testing cutaneous or mucocutaneous lesion samples [15]. The ARIES system can process up to 12 samples per run with a turnaround time of two hours. The entire process is fully automated, which helps prevent human error. In a recent study, Binnicker et al [16] found the performance of the ARIES assay to be comparable with that of two other automated sample-to-result assays: the Aptima HSV-1 and -2 on the Panther system (Hologic, Marlborough, MA, USA) and the Roche HSV-1 and -2 on the cobas 4800 system (Roche Diagnostics, Indianapolis, IN, USA).

We evaluated the analytical performance of the ARIES assay using 116 artificial controls that were prepared using reference materials (SeraCare Life Sciences, Milford, MA, USA; ZeptoMetrix Corp., Franklin, MA, USA). Universal transport medium (UTM) and cerebrospinal fluid (CSF) samples from patients who tested negative for HSV by a previously described in-house HSV-1 and -2 DNA assay were spiked with the reference materials [17]. Given the importance of HSV testing in the differential diagnosis of meningitis/encephalitis, we further evaluated the analytical sensitivity and specificity of the ARIES assay on CSF samples, although the assay is yet to receive FDA approval for this purpose. We then compared the clinical performance of the ARIES assay with those of an in-house assay and the FTD Neuro 9 assay (Fast Track Diagnostics, Junglinster, Luxembourg), using 117 lesion swab samples [17].

METHODS

1. Ethics

This evaluation study involved solely the use of residual lesion swab samples that were submitted for routine clinical testing at the National University Hospital, Singapore. The study design was approved by the National Healthcare Group Domain-Specific Review Board (reference: DSRB/2016/00044).

2. Luminex ARIES HSV 1&2 Assay

The ARIES assay uses probe-free MultiCode-RTx technology (Luminex Corp.), which is dependent on isoguanine and 5-methylisocytosine base-pairing chemistry for PCR and melt curve analysis [15]. All reagents needed for detecting and differentiating HSV-1 and HSV-2 are lyophilized and stored in an individually packaged, self-contained disposable cassette. The cassette also contains a non-competitive sample process control, which is processed along with the sample to rule out the possibility of a false-negative run due to either reagent/device failure or PCR inhibitors. The design of the primers used in the assay is proprietary. Testing was performed according to the kit insert. Briefly, 200 µL of a UTM sample was transferred into the sample chamber of the cassette. The cassette was then manually sealed and loaded onto the ARIES system, wherein the sample was subjected to automated nucleic acid extraction and amplification, and melt curve analysis. Following melt curve analysis, the onboard instrument software automatically interpreted the analytical data for each sample tested. A run was considered invalid when the sample processing control and the HSV targets were not detected. For troubleshooting, the sample was re-tested and if a second invalid result was obtained, the sample was reported as “invalid.” According to the kit insert, the 95% limit of detection (LoD) of the ARIES assay is 7,110 median tissue culture infective dose (TCID₅₀)/mL for the HSV-1 MacIntyre strain, 16.5 TCID₅₀/mL for the HSV-1 F strain, 2.7 TCID₅₀/mL for the HSV-2 MS strain, and 2.8 TCID₅₀/mL for the HSV-2 G strain.

3. Analytical Performance Evaluation

We first determined the analytical sensitivity of the ARIES assay by using artificially prepared positive controls that represented the matrix of patient samples. To prepare the controls, multiple UTM and CSF samples from patients who tested negative for HSV as per an in-house assay were retrieved and pooled according to sample type [17]. The pooled samples were re-tested in duplicate using the ARIES assay to ensure that they were truly negative for HSV. Next, 5 mL aliquots of UTM and CSF were individually spiked with 100 µL of either HSV-1 strain 96 or HSV-2 strain 09 reference materials (ACCURUN 440 HSV-1 and ACCURUN 445 HSV-2 Reference Materials; SeraCare Life Sciences) at 100,000 copies/mL to obtain 2,000 copies/mL UTM and CSF-positive stock samples. The positive stock samples were then serially diluted using negative pooled samples to obtain a six-point serial dilution series for each HSV strain (50 to 2,000 copies/mL for UTM and 100 to 2,000 copies/mL for CSF). Each dilution was then tested using the ARIES assay with eight and six analytical replicates for UTM (N=48) and CSF (N=36), respectively. The number of CSF analytical replicates was comparatively low because of the difficulty in obtaining CSF samples. The LoD was defined as the lowest concentration of HSV DNA that could be detected by the assay with a positivity rate of 95%.

Next, we assessed the analytical specificity of the ARIES assay by using spiked artificial UTM (N=16) and CSF (N=16) controls that were prepared using commercially available NATtrol ME controls (ZeptoMetrix Corp., Franklin, MA, USA), ACCURUN 430 Epstein-Barr Virus Reference Materials, and ACCURUN 450 JC Virus Reference Materials. The controls comprised cytomegalovirus, echovirus type 11, human herpesvirus 6, herpes simplex virus 1 and 2, varicella zoster virus, human parechovirus, Epstein Barr virus, John Cunningham virus, Cryptococcus gattii, Escherichia coli K1, Streptococcus pneumoniae, Streptococcus agalactiae, Neisseria meningitidis, Haemophilus influenzae, and Listeria monocytogenes. To prepare the spiked controls, 20 µL of each target was separately added to the 16 UTM sample aliquots (200 µL each) and 16 CSF sample aliquots (200 µL each) that previously tested negative as per the in-house and ARIES assays. Each control was then tested with the ARIES assay.

4. Clinical Performance Evaluation

The clinical performance of the ARIES assay was evaluated using 117 retrospective, de-identified lesion swab samples. These samples were initially submitted for routine clinical testing at the National University Hospital, Singapore, from July 2016 to January 2017 and had been confirmed as positive or negative for HSV-1 or -2 by a validated in-house molecular assay [17]. Following initial testing, the UTM samples were re-suspended in 3 mL of Copan UTM and stored at 4℃ prior to testing with the ARIES assay (within one week). Viral nucleic acid extracts were stored at −20℃ prior to being tested with an alternative molecular assay, the FTD Neuro 9 assay, within four weeks. For this evaluation, an agreement between the in-house and FTD Neuro 9 results was considered real (reference result).

5. In-house HSV-1 and -2 Assay

The clinical samples were first tested for HSV-1 and -2 using a Förster resonance energy transfer (FRET)-based PCR assay, as previously described [17]. Briefly, total nucleic acid was extracted using 400 µL of the UTM sample, and the purified nucleic acid extract was recovered in 60 µL elution buffer using the Qiagen EZ1 Virus Mini Kit v2.0 with the BioRobot EZ1 instrument (Qiagen, Hilden, Germany). Rapid-cycling real-time PCR and melt curve analysis were performed using 10 µL of the nucleic acid extract on a LightCycler 2.0 instrument (Roche Diagnostics). The specific melting temperature of HSV-1 and -2 was 58±1℃ and 71±1℃, respectively. The 95% LoD of the assay for both subtypes was 100 copies/mL.

6. FTD Neuro 9 Assay

The FTD Neuro 9 assay comprises four separate TaqMan-based multiplex PCR assays, which detect the 11 most common viral pathogens associated with meningitis/encephalitis. We performed testing using one of the four multiplex assays, the FTD Neuro 9 - Vesic assay, which detects HSV-1, HSV-2, and varicella zoster virus. Murine cytomegalovirus was included as an internal control. Real-time PCR was performed using 10 µL of the nucleic acid extract on the Applied Biosystems 7500 Real-Time PCR System (Thermo Fisher Scientific, Waltham, MA, USA). According to the kit insert, the 95% LoD of the assay is 10,000 copies/mL for HSV-1 and 5,000 copies/mL for HSV-2.

7. Statistical Analysis

Probit analysis was performed for the ARIES assay results using R version 3.3.1 (R Core Team, Vienna, Austria) to calculate the 95% positivity rate LoD. Following completion of testing, overall agreement, positive percent agreement (PPA), negative percent agreement (NPA), unweighted Cohen's kappa coefficient (κ), and 95% confidence intervals (CI) were determined by comparing the results of the ARIES assay with the reference results. An agreement between the in-house and FTD Neuro 9 results was considered as the reference result.

RESULTS

1. Analytical Performance of the ARIES Assay

The LoD for HSV-1 strain 96 was 318 copies/mL (95% CI, 232 to 437 copies/mL) and 935 copies/mL (95% CI, 586 to 1,497 copies/mL) for UTM and CSF samples, respectively (Fig. 1A and 1B). The LoD for HSV-2 strain 09 was 253 copies/mL (95% CI, 198 to 321 copies/mL) and 821 copies/mL (95% CI, 500 to 1,376 copies/mL) for UTM and CSF samples, respectively (Fig. 1C and 1D).

Fig. 1. Luminex ARIES HSV 1&2 assay limit of detection (LoD). (A) The LoD of the ARIES assay for detecting herpes simplex virus type 1 (HSV-1) strain type 96 in artificial universal transport medium (UTM) controls. (B) The LoD of the ARIES assay for detecting HSV-1 strain type 96 in artificial cerebrospinal fluid (CSF) controls. (C) The LoD of the ARIES assay for detecting herpes simplex virus type 2 (HSV-2) strain type 09 in artificial UTM controls. (D) The LoD of the ARIES assay for detecting HSV-2 strain type 09 in artificial CSF controls.

In the analytical specificity study, the ARIES assay did not cross-react with the 14 non-HSV microorganisms present in the spiked UTM and CSF controls.

2. Clinical Performance of the ARIES Assay

A total of 117 retrospective, de-identified lesion swab samples were evaluated. However, two samples (one HSV-1 positive and one HSV-2 positive) were swapped during the viral nucleic acid extraction step. As a result, incorrect HSV subtypes were inadvertently reported by the in-house and FTD Neuro 9 assays, whereas the ARIES assay reported the correct HSV subtypes. Two other samples exhibited discrepant results as per the in-house and FTD Neuro 9 assays (hence no reference results). These four samples were excluded, leaving 113 samples for comparative analyses. Overall, 34 samples were positive for HSV-1, and 31 were positive for HSV-2.

The HSV-1 detection performance of the ARIES assay was compared with the reference result (Table 1); the overall concordance rate of the ARIES assay was 98.2% (111/113) with a Cohen's unweighted κ of 0.96 (95% CI, 0.90 to 1.00), indicating a high degree of agreement. The ARIES assay had a PPA of 94.1% (95% CI: 80.3–99.3%) and a NPA of 100% (95% CI: 95.4–100%). Table 1 also shows the HSV-2 detection performance of the ARIES assay compared with the reference result; the ARIES assay was 100% (113/113) concordant with the reference result.

Table 1. Clinical performance of the Luminex ARIES HSV 1&2 assay compared with the reference result, using 113 clinical lesion swab samples.

	Reference result*			Unweighted κ (95% CI)	Positive percent agreement, % (95% CI)	Negative percent agreement, % (95% CI)
	Positive (%)	Negative (%)	Total (%)
Herpes simplex virus type 1
Positive	32 (94.1)	0 (0)	32 (28.3)
Negative	2 (5.9)	79 (100)	81 (71.7)	0.96 (0.90–1.00)	94.1 (80.3–99.3)	100 (95.4–100)
Total	34 (100)	79 (100)	113 (100)
Herpes simplex virus type 2
Positive	31 (100)	0 (0)	31 (27.4)
Negative	0 (0)	82 (100)	82 (72.6)	1.00 (1.00)	100 (88.8–100)	100 (95.6–100)
Total	31 (100)	82 (100)	113 (100)

^*For this evaluation, in-house and FTD Neuro 9 results that were in agreement were considered the reference result.

Abbreviation: CI, confidence interval.

DISCUSSION

We have evaluated the analytical performance of the Luminex ARIES HSV 1&2 assay, a moderately complex and fully automated sample-to-result molecular solution that detects and differentiates HSV-1 and HSV-2, using artificial controls that were prepared using reference materials. Our results suggest that the ARIES assay has lower sensitivity for CSF samples. This observation can be attributed to intrinsic matrix differences; previous studies have documented PCR inhibitory effects in CSF samples due to the presence of red blood cells or proteins [18,19]. However, further investigations are required to confirm this finding. No cross-reactivity was observed for the 14 non-HSV microorganisms added to the UTM or CSF samples. We did not assess the reproducibility of the ARIES assay; however, a previous large multi-center study has reported it to be 95% for low positive samples (at approximately the LoD of the ARIES assay) and 100% for negative samples [15]. We also compared the performance of the ARIES assay with those of the in-house and FTD Neuro 9 assays by using 117 lesion swab samples and found that the ARIES assay had comparable clinical performance for the detection of HSV-1 (κ=0.96) and HSV-2 (κ=1.00) with the reference result. Of the three assays (in-house, FTD Neuro 9, and ARIES), the ARIES assay had the lowest hands-on-time and fastest assay time (two hours for up to 12 samples). Moreover, non-automated assays, i.e., the in-house and FTD Neuro 9 assays, require highly skilled technologists and the workflows are labor-intensive, increasing the risk of human error such as sample swapping. Finally, the ARIES system has a small footprint and requires minimal maintenance and no calibration. These qualities make ARIES an appealing option for diagnostic laboratories with limited floor space and technical expertise. Our findings show that the Luminex ARIES HSV 1&2 is a practical and suitable sample-to-result solution for detecting and differentiating HSV-1 and -2 in skin and genital infections, meningitis, and encephalitis.