Comparison of the Panther Fusion and a laboratory-developed test targeting the envelope gene for detection of SARS-CoV-2

Highlights

• •The Stanford EUA rRT-PCR and Panther Fusion SARS-CoV-2 assay demonstrated similar performance using clinical specimens.
• •High agreement allowed increased testing capacity and limited the impact of supply disruption and instrument down-time.
• •These assays are suitable for the routine diagnosis of active SARS-CoV-2 infection.

Abstract

Background

Numerous nucleic acid amplification assays have recently received emergency use authorization (EUA) for the diagnosis of SARS-CoV-2 infection, and there is a need to assess their test performance relative to one another.

Objectives

The aim of this study was to compare the test performance of the Hologic Panther Fusion SARS-CoV-2 assay targeting two regions of open reading frame 1ab (ORF1ab) to a high complexity molecular-based, laboratory-developed EUA from Stanford Health Care (SHC) targeting the SARS-CoV-2 envelope (E) gene.

Study design

We performed a diagnostic comparison study by testing nasopharyngeal samples on the two assays. Assay agreement was assessed by overall percent agreement and Cohen’s kappa coefficient.

Results

A total of 184 nasopharyngeal samples were tested using the two assays, of which 180 showed valid results and were included for the comparative analysis. Overall percent agreement between the assays was 98.3 % (95 % confidence interval (CI) 95.2–99.7) and kappa coefficient was 0.97 (95 % CI 0.93–1.0). One sample was detected on the SHC laboratory developed test (LDT) and not on the Panther Fusion, and had a Ct of 35.9. Conversely, 2 samples were detected on the Panther Fusion and not on the LDT, and had Ct values of 37.2 and 36.6.

Conclusion

The Panther Fusion SARS-CoV-2 assay and the SHC LDT perform similarly on clinical nasopharyngeal swab specimens. Other considerations, including reagent availability, turnaround time, labor requirements, cost and instrument throughput should guide the decision of which assay to perform.

1. Background

The severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic has highlighted the importance of rapid and accurate diagnostic testing to identify infections promptly and enable appropriate therapeutic management and mitigation of virus spread. Real-time reverse transcription polymerase chain reaction (rRT-PCR) is currently the standard of care for the diagnosis of acute coronavirus disease (COVID-19). Multiple test kit manufacturers, commercial laboratories, and other clinical laboratories certified to perform high-complexity testing have developed nucleic acid amplification tests for the diagnosis of SARS-CoV-2, of which an increasing number have received Emergency Use Authorization (EUA) from the U.S Food and Drug Administration. Two such assays include the Hologic Panther Fusion SARS-CoV-2, a sample-to-answer platform that utilizes rRT-PCR targeting two conserved regions of open reading frame 1ab (ORF1ab), and the Stanford Health Care (SHC) Clinical Virology Laboratory rRT-PCR. This laboratory-developed test (LDT) is based on the work from Corman et al. and targets the envelope (E) gene [[1], [2], [3]]. There is a need to understand the relative test performance of these assays to guide laboratories’ selection of assays and understand advantages and disadvantages of methods that may be used in parallel.

1.1. Objectives

The aim of this study was to assess the test performance of the Panther Fusion SARS-CoV-2 assay compared to the SHC LDT for the qualitative detection of SARS-CoV-2.

1.2. Study design

We performed a comparative diagnostic accuracy study at the SHC Clinical Virology Laboratory from samples collected from adults and children at 2 academic tertiary care hospitals. Nasopharyngeal (NP) samples collected in viral transport medium (VTM) between March 17, 2020 and April 1, 2020 were included. Testing was performed initially on either of the 2 assays. Subsequently, a new aliquot of the original sample was tested by the second assay for comparison.

1.3. Real-time RT-PCR assays

The SHC Laboratory Developed Test was performed as described in the EUA documentation [3]. Briefly, total nucleic acids were extracted from 500 μL VTM on the QIAsymphony SP using the QIAsymphony DSP Virus/Pathogen Midi Kit (both from Qiagen, Germantown, MD) according to the manufacturer’s recommendations, and eluted in 60 μL buffer AVE. SARS-CoV-2 RNA was detected using previously described primer and probe sequences targeting the E gene [2]. These were combined in multiplex with RNase P primers and probe. Real-time RT-PCR was performed using the SuperScript III One-Step RT-PCR System with Platinum Taq DNA Polymerase Kit (Invitrogen, Carlsbad, CA) on the Rotor-Gene Q instrument (Qiagen).

The Panther Fusion SARS-CoV-2 Assay (Hologic, Inc., San Diego, CA) is a high throughput, sample-to-answer nucleic acid amplification test that is comprised of the following steps: sample lysis, nucleic acid capture, elution transfer, and internally-controlled, multiplex rRT-PCR. The assay targets two conserved regions of ORF1ab in the same fluorescence channel. Briefly, 500 μL VTM is transferred to a specimen lysis tube containing specimen transport media, and this tube is loaded directly on the Panther Fusion system. The remaining steps in the process are automated.

1.4. Statistics

Overall percent agreement, positive percent agreement (PPA), negative percent agreement (NPA) and associated 95 % confidence intervals (CI) were performed with the E gene LDT serving as the reference method. Cohen’s kappa coefficient (κ) of qualitative results (detected/non-detected) between the two assays with 95 % CI was also calculated. Cohen’s kappa values greater than 0.81 were interpreted to indicate excellent agreement [4]. All analyses were performed using Stata v15.1.

2. Results

A total of 184 samples were tested by the SHC LDT and Panther Fusion assays. Four samples were excluded due to reproducibly invalid internal control values (RNase P) on the SHC assay, and 180 samples were included for the comparative analysis. There were no indeterminate results on the Panther Fusion assay. The LDT interpreted 42.8 % (77/180) specimens as detected for SARS-CoV-2, and the Panther Fusion interpreted 43.3 % (78/180) as detected (Table 1 ). Detected samples included a wide range of cycle threshold (Ct) values, with a median of 31.5 (IQR 23.9–35.3) for the SHC assay, and 31.4 (IQR 25.9–36.1) for the Panther Fusion. Overall percent agreement between the two assays was 98.3 % (95 % CI 95.2–99.7). The PPA was 98.7 % (95 % CI 93.0–100) and the NPA was 98.1 % (95 % CI 93.1–99.8). The kappa coefficient was 0.97 (95 % CI 0.93–1.0), indicating excellent agreement. One sample was detected on the LDT and not on the Panther Fusion, and had a Ct of 35.9. Repeat testing from the original sample on the Panther Fusion remained negative, and repeat testing from the eluate on the LDT was positive. Conversely, 2 samples were detected on the Panther Fusion and not on the LDT, and had Ct values of 37.2 and 36.6. Repeat testing on the Panther Fusion was not possible due to lack of remaining sample, and repeat testing on the LDT remained negative.

Table 1.

Comparison of the Stanford Health Care Clinical Virology Laboratory SARS-CoV-2 Laboratory-Developed Test and the Panther Fusion SARS-CoV-2 Assay.

		Panther Fusion		Total
		Detected	Not Detected
SHC LDT	Detected	76	1	77
	Not Detected	2	101	103
	Total	78	102	180

LDT: Laboratory-developed test; SARS-CoV-2: severe acute respiratory syndrome coronavirus 2; SHC: Stanford Health Care.

3. Discussion

The COVID-19 pandemic placed significant pressure on clinical and public health laboratories to rapidly expand diagnostic test capacity and to provide timely turnaround of results to inform clinical management. Furthermore, the global shortage of critical reagents and consumables necessitated that laboratories diversify their arsenal of platforms and methods in order to attempt to meet testing demand. This unique situation created an urgent need to understand the relative test performance of each SARS-CoV-2 assay in use, which is particularly important given the clinical and public health implications of both false-positive and false-negative results.

In this study, we showed that the Panther Fusion SARS-CoV-2 assay performed similarly to the SHC SARS-CoV-2 LDT on clinical nasopharyngeal swab specimens, with an overall agreement of 98.3 %. There were a limited number of discrepant results, and these were observed at relatively late cycle thresholds. The similar performance of these assays allowed our laboratory to increase testing capacity when needed, and also strengthened our ability to withstand supply chain disruption and instrument down-time. The Panther Fusion has a higher throughput with less hands-on time that the SHC LDT, though the SHC LDT has EUA for bronchoalveolar lavage fluids, a specimen type that is not authorized on the Panther Fusion. The E gene primer-probe set, which underlies the SHC LDT, was previously adapted to the Panther Fusion open access protocol to automate SARS-CoV-2 testing [5]. The Panther Fusion SARS-CoV-2 EUA assay used in this study targets two distinct, conserved regions of ORF1ab in one channel. Compared to a single-target approach, this dual-target design helps to ensure virus detection should mutation impact one of the target regions.

Limitations of this study include the use of specimens from a single center in northern California and the evaluation of only two SARS-CoV-2 molecular diagnostics. The numerous FDA EUA assays include diverse target genes, amplification methods, turnaround times, and instruments. While EUA requires analytical evaluation, many of these assays lack performance data on clinical specimens from individuals presenting with suspicion for COVID-19. Further studies are therefore warranted to compare the performance of other assays using patient specimens that span the range of clinically observed viral loads.

In summary, the Panther Fusion SARS-CoV-2 assay and the SHC SARS-CoV-2 LDT demonstrated comparable performance for the detection of SARS-CoV-2 RNA in clinical nasopharyngeal swab specimens from patients under investigation for COVID-19. Our findings support the interchangeable use of these assays for the routine diagnosis of active SARS-CoV-2 infection.

Funding

None.

CRediT authorship contribution statement

Catherine A. Hogan: Conceptualization, Methodology, Writing - original draft, Writing - review & editing. Malaya K. Sahoo: Investigation. ChunHong Huang: Investigation. Natasha Garamani: Investigation. Bryan Stevens: Investigation. James Zehnder: Writing - review & editing. Benjamin A. Pinsky: Conceptualization, Methodology, Supervision, Writing - review & editing.

Declaration of Competing Interest

The authors declare no conflicts of interest.