Evaluation of the BioFire® COVID-19 test and Respiratory Panel 2.1 for rapid identification of SARS-CoV-2 in nasopharyngeal swab samples

Abstract

The BioFire® COVID-19 Test and Respiratory Panel 2.1 (RP2.1) are rapid, fully automated assays for the detection of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) in nasopharyngeal swabs. In the case of the RP2.1, an additional 21 viral and bacterial pathogens can be detected. Both tests have received emergency use authorization from the U.S. Food & Drug Administration and Interim Order authorization from Health Canada for use in clinical laboratories. We evaluated the performance characteristics of these tests in comparison to a laboratory-developed real-time PCR assay targeting the viral RNA-dependent RNA polymerase and E genes. A total of 78 tests were performed using the BioFire COVID-19 Test, including 30 clinical specimens and 48 tests in a limit of detection study; 57 tests were performed using the RP2.1 for evaluation of SARS-CoV-2 detection, including 30 clinical specimens and 27 tests for limit of detection. Results showed 100% concordance between the BioFire assays and the laboratory-developed test for all clinical samples tested, and acceptable performance of both BioFire assays at their stated limits of detection. Conclusively, the BioFire COVID-19 Test and RP2.1 are highly sensitive assays that can be effectively used in the clinical laboratory for rapid SARS-CoV-2 testing.

1. Introduction

Accurate and timely detection of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) in upper respiratory tract specimens is the cornerstone of the global effort to minimize transmission of the virus (World Health Organization, 2020). At the time of writing, there are 175 molecular diagnostic COVID-19 tests approved for Emergency Use Authorization by the U.S. Food & Drug Administration and 27 nucleic acid technology tests authorized under Interim Order by Health Canada (Health Canada, 2020, U.S. Food and Drug Administration, 2020). The Infectious Diseases Society of America guidelines on the diagnosis of COVID-19, released in May 2020, advocate for nucleic acid amplification testing of all symptomatic individuals even when clinical suspicion is low (Infectious Diseases Society of America, 2020). The guidelines highlight the fact that a knowledge gap currently exists regarding the use of rapid tests (defined as turnaround time ≤1 hour) versus standard RNA tests in terms of performance characteristics, and therefore makes no recommendations for or against them.

The BioFire tests are fully automated multiplexed PCR assays that detect multiple SARS-CoV-2 regions. The tests offer a rapid turn-around time while requiring very little hands-on time. The assays have both received Emergency Use Authorization in the U.S. and Interim Order authorization in Canada for use with nasopharyngeal (NP) swabs. The COVID-19 Test was developed with funding from the U.S. Department of Defense, and detects three targets within the open reading frame (ORF) region of the viral genome (CAP Today, 2020). As a result of the contractual agreement with the Department of Defense, the RP2.1 panel was developed with 2 novel genome targets – the spike (S) and membrane (M) proteins. The RP2.1 builds upon the RP2 panel, which includes 21 viral and bacterial targets. This panel was extensively evaluated in a prospective study analyzing 1612 NP swabs, with 33,843 results generated; the evaluation demonstrated excellent analytical sensitivity and specificity of the multiplex panel (Leber et al., 2018).

This report describes the results of an independent evaluation of the performance characteristics of the BioFire COVID-19 Test and the RP2.1 for detection of SARS-CoV-2. The assays were compared to our laboratory-developed test (LDT) targeting the SARS-CoV-2 RNA-dependent RNA polymerase (RdRP) and envelope protein (E) genes, for which the average turnaround time, from receipt to result, is 5 to 9 hours. In the first part of the evaluation, a retrospective analysis of 25 SARS-CoV-2 positive and 5 negative patient specimens was performed using both BioFire assays on the FilmArray Torch system. In the second part of the evaluation, viral transport media from NP swab collection devices was spiked with reference material containing inactivated whole virus of known concentrations, and tested in parallel using BioFire and our LDT.

2. Materials and methods

Clinical specimens: Thirty archived, SARS-CoV-2 positive and negative NP swabs collected in BD™ Universal Viral Transport System (BD; Franklin Heights, NJ) or Copan UTM® Universal Transport Medium (Copan; Murrieta, CA) between February and May 2020 from adult patients were selected. Twenty-five positive specimens were chosen based on the cycle threshold (Ct) value of the E gene result from the initial LDT: 5 specimens with a Ct <20, 10 specimens 31< Ct ≤35, and 10 specimens 35< Ct ≤40. Five SARS-CoV-2 negative NP swabs positive for alternate respiratory viruses (influenza A, influenza B, human metapneumovirus, coronavirus HKU1, and human rhinovirus and/or enterovirus) were included for analysis by both BioFire assays.

Laboratory-developed reverse transcriptase real time PCR assay: Nucleic acids were extracted from 300 µL of NP swab medium on the MagnaPure Compact instrument (Roche Diagnostics; Laval, Canada) using the MagNA Pure Compact Nucleic Acid Isolation Kit I (Roche Diagnostics; Laval, Canada) according to manufacturer instructions. Five microliters of nucleic acid was added to a 20 µL reverse transcriptase real-time polymerase chain reaction (PCR) reaction using TaqMan Fast Virus 1-Step Master Mix (Thermo Fisher Scientific; Waltham, MA). The reaction was subjected to the following protocol on an ABI 7500 Fast thermocycler (Thermo Fisher Scientific; Waltham, MA): 1 cycle for 5 minutes at 50°C, 1 cycle for 20 sec at 95°C, followed by 45 two-step cycles of 3 sec at 95°C and 30 sec at 60°C. Primer and probe sequences targeting the envelope (E) gene of SARS-CoV-2 were based on the World Health Organization protocol (Corman et al., 2020) and the primers and probes targeting the RdRP gene were developed by the British Columbia Centre for Disease Control. A primer and probe set targeting the human ribonuclease P gene serves as an internal control (World Health Organization, 2009). Results are reported as positive, indeterminate, or negative; see Supplementary Materials.

BioFire COVID-19 test and RP2.1: Testing was performed according to the manufacturer's (BioFire Defense LLC, and BioFire Diagnostics LLC; Salt Lake City, UT) instructions. Briefly, 300 µL of sample was mixed with sample buffer and injected into a test pouch containing all necessary reagents for nucleic extraction, PCR amplification and detection of the respective targets. The test pouch was inserted into the BioFire FilmArray Torch instrument and run using the provided software. The COVID-19 Test is a combination of 3 independent and non-overlapping assays, 2 of which have genomic targets in the ORF1ab region and 1 with a target in the ORF8 region. SARS-CoV-2 was interpreted by the software as “detected” when at least 2 out of the 3 assays were positive based on melt curve analysis. In the event of a single positive target, interpretation will be deemed “equivocal” and the user will be prompted to retest the original sample. The tentative limit of detection (LoD) of the test is 330 genomic copies per milliliter, as published by the manufacturer (BioFire Defense, LLC 2020). The RP2.1 contains 2 independent assays for SARS-CoV-2 with targets in the S and M protein genomic regions. SARS-CoV-2 is reported as “detected” if 1 or both targets are detected. The LoD is reported as 500 genomic copies per milliliter for heat inactivated virus, and 160 genomic copies per milliliter for infectious virus (BioFire Diagnostics, LLC 2020). Quality control was performed using the BioFire External Positive Control as per the manufacturer protocol prior to study testing.

Limit of detection study: A stock of purified, intact SARS-CoV-2 viral particles (Zeptometrix; Buffalo, NY) was diluted with BD Universal Viral Transport medium to various concentrations. For the COVID-19 Test, concentrations of 16.5, 165, 330, and 660 viral particles/mL, representing 0.05, 0.5, 1.0, and 2.0 times the documented LoD respectively were tested; an additional 2 samples consisting of transport medium alone were included as negative controls. Dilutions were tested in parallel with the BioFire COVID-19 Test and SARS-CoV-2 LDT in triplicate over the course of 3 consecutive days. For the RP2.1, concentrations of 25, 100, 500, and 1000 viral particles/mL, representing 0.05, 0.2, 1.0, and 2.0 times the documented LoD respectively were tested. A single additional sample consisting of transport medium alone was included as a negative control. Dilutions were tested with the BioFire RP2.1 in triplicate over the course of 3 consecutive days, and twice each with the SARS-CoV-2 LDT.

3. Results

Archived Clinical Specimens: The 25 previously positive specimens all tested concordantly positive by the BioFire COVID-19 Test, representing a positive percent agreement of 100% between the 2 assays (Table 1 ). Three samples had an initial LDT E gene Ct value between 38 and 40; of those, 2 were positive for all 3 targets included in the BioFire COVID-19 Test, whereas 1 sample was positive for 2 targets (ORF1ab) and negative for the third (ORF8). This result was interpreted as a positive result by the BioFire software. Similarly, the RP2.1 reported all 25 SARS-CoV-2 positive specimens as detectable. Five previously confirmed SARS-CoV-2 negative NP swabs were concordantly negative for the virus by both BioFire assays, and the RP2.1 accurately detected the alternate viral pathogens in all 5 specimens. There were no discordant results observed, for an overall agreement of 100% for both assays.

Table 1.

Performance of the BioFire COVID-19 test and RP2.1 in comparison to SARS-CoV-2 LDT for archived clinical specimens.

E gene Ct valuea	Number of results for BioFire COVID-19/LDT				Agreement	Number of results for BioFire RP2.1/LDT				Agreement
	+/+	+/-	-/+	-/-	%	+/+	+/-	-/+	-/-	%
Ct <20	5				100	5				100
31< Ct ≤35	10				100	10				100
35 < Ct ≤40	10				100	10				100
Negative				5	100				5	100

^aBased on initial SARS-COV-2 LDT result.

3.1. Limit of detection

All replicates with concentrations at or slightly above the LoD were positive by the BioFire COVID-19 Test, and both gene targets were detected in the LDT (Table 2 ). Likewise, at 0.5LoD, all replicates tested positive by both methods. The mean Ct values of the LDT ranged from 33.3 to 34.8 for the E gene, and 33.5 to 35.6 for the RdRP gene for samples at these concentrations Of the 12 replicates at a concentration of 0.05LoD, 7 were positive, 2 were equivocal, and 3 were negative by the BioFire COVID-19 Test. Similarly, 7 of the 12 replicates tested by LDT were detectable for both gene targets, 3 were detectable for 1 target, and 2 were negative. For the RP2.1, 11 replicates with concentrations at 1LoD and 2LoD were positive and 1 was negative (Table 3 ). When tested in duplicate on the SARS-CoV-2 LDT, the E gene for the 1 LoD dilution was not detected in the first test, and had a Ct value of 36.4 in the second. The mean E gene Ct value at 2LoD was 35.6.

Table 2.

Results of the BioFire COVID-19 test compared to the LDT using serial dilutions of inactivated SARS-CoV-2 viral particles.

			BioFire COVID-19			SARS-CoV-2 LDT
			Test resulta			Number of targets detected
BioFire LoD	Genomic copies/mL	Number of replicates	Positive	Equivocal	Negative	2/2	½	0/2
0	0	6	0	0	6	0	0	6
0.05	16.5	12	7	2	3	7	3	2
0.5	165	12	12	0	0	12	0	0
1	330	12	12	0	0	12	0	0
2	660	6	6	0	0	6	0	0

^aPositive result ≥2/3 targets detected; equivocal result 1/3 targets detected.

Table 3.

Results of the BioFire RP2.1 compared to the LDT using serial dilutions of inactivated SARS-CoV-2 viral particles.

			BioFire RP2.1		SARS-CoV-2 LDT
			Test result		Number of targets detected
BioFire LoD	Genomic copies/mL	Number of replicates	Positive	Negative	2/2	1/2	0/2
0	0	3	0	3	0	0	2
0.05	25	6	2	4	0	1	1
0.2	100	6	3	3	1	1	0
1	500	6	5	1	1	1	0
2	1000	6	6	0	2	0	0

4. Discussion

Results of this evaluation demonstrate 100% agreement between the BioFire COVID-19 Test, the BioFire RP2.1 (SARS-CoV-2 target), and our LDT in NP swabs collected from both COVID-19 positive and negative patients. The SARS-CoV-2 positive clinical specimens were chosen to represent a wide range of Ct values, including several at the upper LoD of the LDT. The negative specimens, which included 1 Betacoronavirus (CoV HKU1) positive sample, did not produce any false positive results for SARS-CoV-2 in either BioFire assay.

The BioFire assays were evaluated at various concentrations of inactivated viral particles to determine their performance at, above, and below their documented limits of detection. At 0.5LoD (165 genomic copies/mL) and above, 100% of replicates were positive in the BioFire COVID-19 Test, which was fully concordant with the LDT. At a further 10-fold dilution, the BioFire COVID-19 Test was positive in 67% of tests, equivocal in 17% of tests, and negative in 17% of tests. Similarly, the LDT demonstrated variable results at this low concentration. Overall, the BioFire COVID-19 Test exhibited good sensitivity that was comparable to the LDT, even below the manufacturer's stated LoD. The RP2.1 demonstrated slightly diminished sensitivity in comparison to the COVID-19 Test. At 1LoD (500 genomic copies/mL), 5 out of 6 replicates were positive; at 2LoD, all replicates were positive. The difference in LoD results between the COVID-19 Test and the RP2.1 may be due to the highly multiplexed nature of the latter assay, the difference in SARS-CoV-2 genomic targets, or simply the technical limitations associated with reproducing consistent concentrations of viral particles at such high dilution factors. The evaluation of clinical samples, however, demonstrated equivalent analytical sensitivity between the assays.

The Ct value for SARS-CoV-2 genomic targets obtained during testing is dependent on the initial concentration of nucleic acid in the specimen; consequently, this value has been used as semi-quantitative means of tracking viral burden over time (Chen et al., 2020; He et al., 2020; Wölfel et al., 2020) and has been shown to be correlated with disease severity (Liu et al., 2020). Notably, the BioFire platform does not report Ct values for test interpretation, and users will therefore have no indication of viral burden from the test result. Based on the LoD results of this evaluation, there may be instances where the BioFire COVID-19 Test is positive, whereas our LDT would be interpreted as negative or indeterminate. The high sensitivity of the assays in detecting viral nucleic acid, which does not necessarily represent infectious particles, could potentially result in positive test results without clear evidence of transmissibility (Wölfel et al., 2020; Lan et al., 2020). Both BioFire assays displayed 100% sensitivity for clinical specimens with E gene Ct values >35.

As the next influenza season approaches, it will become increasingly important for laboratories to rapidly detect and differentiate co-circulating respiratory pathogens. The flexibility to redesign and adapt existing multiplex assays to expediently match changes in epidemiology is a highly innovative feature, and could offer extensive benefits to infection control and public health domains. The BioFire COVID-19 Test and Respiratory Panel 2.1 are easy-to-use, highly sensitive, and rapid assays for the detection of SARS-CoV-2 in NP swab specimens. This evaluation demonstrates that the assays perform comparably to our laboratory developed real-time PCR assay, with 100% agreement in testing results for clinical specimens and acceptable performance at their stated limits of detection. The BioFire FilmArray platform is designed for use in a clinical laboratory and is not approved as a point-of-care test in Canada; it does, however, still maintain the benefits of rapid turnaround time and random access (no batching required). Additionally, it does not require any technical expertise on the part of the user, which is a significant benefit when highly skilled laboratory staff are limited. The BioFire assays are viable alternatives to traditional real-time SARS-CoV-2 PCR assays for rapid results in the laboratory, including in remote areas where higher complexity assays are not feasible. Although the future of the BioFire COVID-19 Test remains unclear with the subsequent introduction of the RP2.1, this single-pathogen test may prove useful for pooling of patient specimens as resources become more limited during the pandemic.

Funding

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Declaration of competing interest

The authors report no conflicts of interest relevant to this article.