Multicenter Evaluation of the BioFire Respiratory Panel 2.1 (RP2.1) for Detection of SARS-CoV-2 in Nasopharyngeal Swab Samples

ABSTRACT

As the incidence of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) begins to overlap with the traditional respiratory season in the Northern Hemisphere, simultaneous testing for SARS-CoV-2 and the other common causes of respiratory infections is imperative. This has led to the development of multiplex respiratory assays that include SARS-CoV-2 as a target. One such assay is the BioFire respiratory panel 2.1 (RP2.1), which is an expansion of the original BioFire FilmArray respiratory panel 2 (RP2) to include SARS-CoV-2. In this multicenter evaluation, we assessed the performance characteristics of the BioFire RP2.1 for the detection of SARS-CoV-2. One or more targets on the panel were detected in 19.3% (101/524) of specimens tested, with SARS-CoV-2 detected in 12.6% (66/524) of specimens. Human rhinovirus/enterovirus was also detected in 32.7% (33/101) and adenovirus in 3.0% (3/101) of positive specimens, with one dual positive for both SARS-CoV-2 and adenovirus being detected. A further breakdown of pathogens by age revealed a 4-fold predominance of human rhinovirus/enterovirus in subjects 0 to 18 years of age, whereas in all other age groups, SARS-CoV-2 was clearly the predominant pathogen. Overall, SARS-CoV-2 results obtained from the BioFire RP2.1 were highly concordant with the composite result, exhibiting 98.4% (61/62) positive percent agreement (95% confidence interval [CI], 91.4 to 99.7%) and 98.9% (457/462) negative percent agreement (95% CI, 97.5 to 99.5%) with further analysis of discordant results suggesting that the concentration of SARS-CoV-2 in the specimens was near the limit of detection (LoD) for both the BioFire RP2.1 and the comparator assays. Overall, the BioFire RP2.1 exhibited excellent performance in the detection of SARS-CoV-2.

INTRODUCTION

As the COVID-19 pandemic has progressed, concurrent circulation of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) with prepandemic respiratory pathogens has become a major diagnostic challenge. This is especially evident with the onset of the 2021 respiratory season (1). Due to heavily overlapping symptomology, it is not possible to reliably distinguish between SARS-CoV-2 and the other common respiratory infections, such as influenza and respiratory syncytial virus (RSV) based on clinical presentation (2). In addition, coinfections of SARS-CoV-2 with other respiratory pathogens have been reported (3–8) and will likely become more common as the trend of cocirculation continues. Symptomology alone is also no longer efficient nor effective to drive infection control practices or determine therapeutic choices. As such, laboratory results that can differentiate between SARS-CoV-2 and other respiratory pathogens are an essential component of clinical care and infection control practices, especially in the hospital setting. While this is the case, testing for individual respiratory pathogens is time-consuming as well as labor and resource intensive.

One way to address this need has been the development and implementation of multiplex respiratory panels to simultaneously detect SARS-CoV-2 and the other common pathogens, saving both time and resources. The BioFire RP2 (BioFire Diagnostics, LLC) was FDA cleared in May 2017 for the simultaneous qualitative detection and identification of nucleic acids from multiple common viral and bacterial respiratory pathogens in nasopharyngeal swabs (NPS) (9). The BioFire FilmArray system utilizes an automated sample purification and multiplex-nested PCR and melting analysis approach. In response to the outbreak of COVID-19, which began in December 2019, the existing BioFire RP2 and BioFire FilmArray respiratory panel 2 plus (RP2plus) products were modified to add assays for detection of SARS-CoV-2. This new product, named the BioFire respiratory panel 2.1 and BioFire respiratory panel 2.1 plus (RP2.1plus), is an expansion of the BioFire RP2 and BioFire RP2plus to include assays for the detection of this new analyte. The BioFire RP2.1 and BioFire RP2.1plus are identical test reagents with results for Middle East respiratory syndrome coronavirus (MERS-CoV) masked by the software for the BioFire RP2.1 version; for simplicity, both versions are referred to simply as the “BioFire RP2.1” throughout this manuscript except where a distinction is required.

The purpose of this study was to assess the positive percent agreement (PPA) and negative percent agreement (NPA) of the BioFire RP2.1 SARS-CoV-2 assay when testing prospectively collected NPS specimens from the intended use population. This study was used in support of the BioFire RP2.1 FDA de novo authorization, which was granted in March 2021.

MATERIALS AND METHODS

Clinical specimens.

The study was conducted at three geographically distinct U.S. sites (Site 1, Tampa General Hospital, Tampa, FL; Site 2, Northwell Health Laboratories, Lake Success, NY; and Site 3, Loyola University, Maywood, IL) over a period of approximately 4 months (July 2020 through October 2020). Residual specimens from subjects of all ages meeting the following inclusion criteria were enrolled: specimen was residual nasopharyngeal swab (NPS) in transport medium left over from standard-of-care (SOC) testing for SARS-CoV-2 by an assay that included an extraction step and had received an emergency use authorization (EUA) designation, specimen was held at room temperature for less than or equal to 4 h or 4°C for less than or equal to 3 days before enrollment, and at least 1.3 mL of specimen was remaining after SOC testing and was available for use in the study. A waiver of the requirement for informed consent was obtained from the institutional review boards (IRBs) at each study site for the use of residual NPS specimens to collect subject information from medical records. Demographic and limited clinical data were collected, including date of specimen collection, age range at time of collection, sex, the results of SARS-CoV-2 test performed in laboratory for SOC testing, and additional respiratory pathogen testing performed on the specimen enrolled in the study.

BioFire RP2.1 testing.

The BioFire RP2.1 obtained EUA status from FDA on 1 May 2020. This study was conducted with a CE-marked version of the BioFire RP2.1plus that is the identical reagent pouch to the BioFire RP2.1, but uses a software module that displays results for MERS-CoV. All specimen handling occurred in a biosafety cabinet with operators wearing appropriate personal protective equipment, preparing one specimen at a time, and cleaning between specimens, all according to the manufacturer’s instructions (https://www.online-ifu.com/ITI0105). Approximately 300 μL of specimen was subject to BioFire RP2.1 testing. The BioFire RP2.1 test consists of automated nucleic acid extraction, reverse transcription, nucleic acid amplification, and automated results analysis in about 45 min per run (i.e., per specimen). If either internal control fails, the software automatically provides a result of “Invalid” for all panel analytes. Analytes are reported qualitatively as “Detected,” “Not Detected,” or “Equivocal” (only applicable to influenza A). The BioFire RP2.1 contains two independent assays for the identification of SARS-CoV-2 (one assay targeting the M gene, another assay targeting the S gene). A positive result for either assay provides a Detected result for SARS-CoV-2.

Comparator testing.

A composite comparator consisting of three tests with U.S. FDA EUA designation was used as the reference method. Concordance for two out of three of the EUA tests was considered the final result for the reference method. Specimens without two valid concordant EUA results were excluded from analysis. Study personnel performing the comparator assays were blinded to the BioFire RP2.1 result and the final composite comparator result.

(i) SOC EUA test.

All NPS specimens enrolled in the study were tested with a SARS-CoV-2 test having EUA designation that was performed for SOC testing. The EUA tests used for specimens enrolled at each site are listed (see Table 5).

TABLE 5.

Investigation of FN and FP SARS-CoV-2 results

Discrepancy	BioFire RP2.1 result	Comparator EAU tests				Cepheid Xpert Xpress SARS-CoV-2 (31) result/CT (if applicable)
		Composite EUA result	SOC (NeuMoDx) result/CT (if applicable)	COVID-19 test result	New York RT-PCR result/CT (if applicable)
FNa	NDb	Positive	Positive/(31.98, 31.25)	D	Positive/(34.1/36.0)	NTc
FPd	De	Negative	Negative	Ef	Negative	NT
FP	D	Negative	Negative	D	Negative	NT
FP	D	Negative	Negative	D	Negative	NT
FP	D	Negative	Negative	E	Negative	NT
FP	D	Negative	Negative	ND	Negative	Positive/(38.2, 36.0)

^aFN, false negative.

^bND, not detected.

^cNT, not tested.

^dFP, false positive.

^eD, detected.

^fE, equivocal.

(ii) BioFire COVID-19 test.

A second specimen aliquot was also tested on site with the BioFire COVID-19 test (BioFire Defense, LLC), which had EUA designation at the time of the study. This is a single analyte SARS-CoV-2 test that contains 3 assays for SARS-CoV-2 (two targeting nonoverlapping regions of the ORF1ab gene and one targeting the ORF8 gene), all of which are different than the BioFire RP2.1 SARS-CoV-2 assays. Testing was performed according to the manufacturer’s instructions.

(iii) Study-specific EUA test.

A third frozen specimen aliquot was initially sent to BioFire. These aliquots were shipped in batches to Northwell Health Laboratories (NHL, Lake Success, NY; site 2) on dry ice and transferred directly to a <−70°C freezer until they were tested with the New York SARS-CoV-2 real-time reverse transcriptase (RT)-PCR diagnostic panel (Wadsworth Center, New York State Department of Public Health). The New York SARS-CoV-2 real-time reverse transcriptase (RT)-PCR diagnostic panel had EUA designation at the time of the study and is identical to the CDC 2019-novel coronavirus (2019-nCoV) real-time RT-PCR diagnostic panel except that the control materials were manufactured by Wadsworth Center located in Albany, New York and Bio-Synthesis located in Lewisville, Texas. Results were interpreted according to the test’s instructions for use (IFU).

Results and discrepant analysis.

A BioFire RP2.1 result was considered a true positive (TP) or true negative (TN) only when it agreed with the composite comparator result. A result was considered a false positive (FP) or false negative (FN) when it disagreed with the composite comparator result. Positive percent agreement (PPA) was calculated as 100 × TP/(TP + FN), while negative percent agreement (NPA) was calculated as 100 × TN/(TN + FP). Additional testing could not be performed by BioFire for discrepant specimens due to insufficient volume. Note that the performance data for PPA and NPA presented in this manuscript consist of unresolved data as presented in the IFU for the de novo test; any discrepancy investigation, which included review of the subjects’ charts for additional SARS-CoV-2 testing, is provided but was not used to recalculate performance data.

Statistical analysis.

The exact binomial two-sided 95% confidence intervals (95% CI) were calculated for performance measures according to the Wilson score method (10).

RESULTS

Demographics.

A total of 524 prospectively collected specimens from geographically and demographically diverse U.S. populations were included in this study. Overall, slightly more male (52%; n = 270) than female (48%; n = 251) subjects were enrolled. In addition, 10% (n = 55) of subjects were 18 years of age or younger, with 19 to 40 years of age representing the largest demographic at 32% (n = 170), 41 to 60 years of age at 28% (n = 146), and 61+ years of age as the second largest demographic at 29% (n = 153). A summary of subject demographics for all specimens enrolled is presented in Table 1.

TABLE 1.

Demographic data

Demographic	Overall no. (%)
Sex
Male	270 (52)
Female	251 (48)
Unknown	3 (<1)
Age
0–18 yrs	55 (10)
19–40 yrs	170 (32)
41–60 yrs	146 (28)
61+ yrs	153 (29)
Total no.	524

System performance.

Initially, 527 specimens met the inclusion criteria and are included in the analysis of system performance. Of these, 311 (59.0%) were run on BioFire 2.0 systems, while the remaining 216 (41.0%) were run on BioFire Torch systems. The overall success rate on the initial test of these specimens was 99.6% (525/527); two tests were unsuccessful (both due to an instrument error). One of these specimens was able to be retested, and valid results were produced after a single retest. Additionally, two specimens were excluded because the composite EUA comparator interpretation could not be determined (e.g., a minimum of two valid, concordant EUA test results were not obtained), leaving a total of 524 specimens in the final analysis.

Study findings.

In this evaluation, the BioFire RP2.1 had at least one Detected result in 19.3% (101/524) of specimens tested. During the testing time frame (July to October 2020), only three analytes were detected, including adenovirus detected in 0.6% (3/524) of specimens, human rhinovirus/enterovirus detected in 6.3% (33/524) of specimens, and SARS-CoV-2 detected in 12.6% (66/524) of specimens. The BioFire RP2.1 reported a single specimen with dual detection of both adenovirus and SARS-CoV-2 representing 0.2% (1/524) of all specimens and 1.0% (1/101) of positive specimens.

A further analysis of detected results by age range revealed that while human rhinovirus/enterovirus was the predominant infection in the 0 to 18 year age range group (n = 19) at a nearly a 4-fold predominance over SARS-CoV-2 (n = 5), this trend was reversed in all other age groups with SARS-CoV-2 infections being clearly predominant. Overall, the 0 to 18 year age range group accounted for 7.6% of SARS-CoV-2 infections, followed by 19 to 40 years at 36.4%, 41 to 60 years at 33.3%, and 61+ years at 22.7% (Table 2).

TABLE 2.

Number of detected analytes stratified by age

Virus	BioFire RP2.1 result stratified by age
	Overall (n = 524)	0–18 yrs (n = 55)	19–40 yrs (n = 170)	41–60 yrs (n = 146)	61+ yrs (n = 153)
Adenovirus	3	1	2	0	0
Severe acute respiratory syndrome coronavirus 2	66	5	24	22	15
Human rhinovirus/enterovirus	33	19	5	7	2

The method used to assess accuracy of the BioFire RP2.1 for the detection of SARS-CoV-2 was a composite interpretation of three SARS-CoV-2 EUA test results representing the SOC assay at each individual site (Table 3), the BioFire COVID-19 test, and the New York SARS-CoV-2 real-time reverse transcriptase (RT)-PCR diagnostic panel. Overall, results obtained from the BioFire RP2.1 were highly concordant with the composite result, exhibiting 98.4% (61/62) positive percent agreement (95% CI, 91.4 to 99.7%) and 98.9% (457/462) negative percent agreement (95% CI, 97.5 to 99.5%) (Table 4).

TABLE 3.

Comparator methods for the BioFire RP2.1 clinical evaluation

Study site	EUA test 1a	EUA test 2b	EUA test 3c
Site 1 Tampa General Hospital	NeuMoDx SARS-CoV-2 Assay (NeuMoDx) (26)	BioFire COVID-19 test (BioFire Defense, LLC) (27)	New York SARS-CoV-2 real-time reverse transcriptase (RT)-PCR diagnostic panel (Wadsworth Center, NYSDOH) (28)
Site 2 Northwell Health Laboratories	Hologic Aptima SARS-CoV-2 (Hologic, Inc.) (29)
Site 3 Loyola University Medical Center	Abbott RealTime SARS-CoV-2 (Abbott, Inc.) (30)

^aPerformed at the source laboratory as part of patient care.

^bTested fresh at the source laboratory as study-specific testing.

^cSpecimen aliquots from source laboratory were frozen and shipped to study site 2 for study-specific testing.

TABLE 4.

Analytical performance of the BioFire RP 2.1 as compared to composite EUA results

RP2.1	Composite EUA		BioFire RP2.1 performance no. (%)	95% CI (%)
	No. positive	No. negative
No. positive	61	5	61/62 (98.4)	91.4–99.7
No. negative	1	457	457/462 (98.9)	97.5–99.5
Total	62	462

The single false negative (FN) and five false positive (FP) results were further analyzed. In the case of the FN, both the SOC NeuMoDx SARS-CoV-2 assay and the New York SARS-CoV-2 real-time RT-PCR diagnostic panel showed late threshold cycle (C_T) values for both assay targets (see Table 5), which was also consistent with the results of the BioFire COVID-19 test, in which 2 of 3 assay targets were detected. These findings are indicative of analyte concentration near limit of detection (LoD) of each of these assays. In the case of the five FP specimens, both the SOC NeuMoDx SARS-CoV-2 assay and the New York SARS-CoV-2 real-time RT-PCR diagnostic panel were negative for all five specimens, whereas four specimens had a Detected (2 or 3 targets detected) or Equivocal (1 of 3 targets detected) result on the BioFire COVID-19 test. For one of the FP specimens, the study site initiated additional SOC testing based on their internal review of the NeuMoDx SARS-CoV-2 amplification data. This specimen was additionally tested with the Cepheid Xpert Xpress SARS-CoV-2 test (Cepheid, Sunnyvale, CA) and was positive with mean C_T values for both assays indicative of analyte concentration near LoD, according to the manufacturer’s IFU (see Table 5). Due to insufficient sample volume, these specimens could not be investigated further. Taken together, these data suggest that the concentration of SARS-CoV-2 in each of these specimens was near or below the LoD for each of these molecular assays (Table 5).

DISCUSSION

In this prospective, multisite study, we evaluated the performance of the BioFire RP2.1 for the detection of SARS-CoV-2 in NPS specimens. This study is unique in the sense that while the BioFire RP2.1 was issued FDA EUA status for the detection of SARS-CoV-2, there were no products available that had been granted de novo authorization by the FDA at the time that this work was planned and completed. Therefore, this evaluation was performed to generate the data for a de novo submission, which is the process by which a novel medical device is submitted for FDA approval when there is no legally marketed predicate FDA authorized device available by which to compare.

This panel is identical to the BioFire RP2 (9) except for the addition of two independent assays for the detection of SARS-CoV-2. A previous study evaluating the BioFire RP2.1 was published during the first 6 months of the pandemic but was performed with a limited number of stored frozen specimens (11).

This current evaluation adds to the previous evaluations done for both the BioFire RP2 and BioFire RP2.1 by performing a robust prospective analysis of the BioFire RP2.1 performance for the detection of SARS-CoV-2, comparing the panel to several previously well-defined EUA assays (NeuMoDx SARS-CoV-2 assay, Hologic Aptima SARS-CoV-2, Abbott RealTime SARS-CoV-2, and New York SARS-CoV-2 real-time reverse transcriptase (RT)-PCR diagnostic panel).

This evaluation of 524 prospectively collected NPS specimens included in the final analysis encompassed a diverse testing population, both from a geographic and demographic perspective with all testing age groups being represented. Of the three pathogens detected, SARS-CoV-2 represented the majority at 65% (66/101) of positive specimens, followed by human rhinovirus/enterovirus at 33% (33/101), and adenovirus at 3% (3/101). A further breakdown of pathogens by age revealed a 4-fold predominance of human rhinovirus/enterovirus in the 0 to 18 years of age group, whereas in all other age groups, SARS-CoV-2 was clearly the predominant pathogen. It is currently thought that most infected children appear to have a milder course of infection (12), which could theoretically have an impact on whether diagnostic testing is performed. Future studies will need to be done to further investigate this trend.

Overall, the BioFire RP2.1 exhibited excellent performance with a PPA of 98.4% (95% CI, 91.4 to 99.7%) and an NPA of 98.9% (95% CI, 97.5 to 99.5%) in the detection of SARS-CoV-2 in NPS specimens. This performance for the detection of SARS-CoV-2 is comparable to some of the most sensitive commercial assays available (13–17). In addition, in the single FN and five FP cases where discordant results were obtained, further analysis of the results suggested that in each of these cases, the concentration of SARS-CoV-2 in the specimens was near the LoD for both the BioFire RP2.1 and the comparator assays.

A limitation of this study is that it is an evaluation of the performance of SARS-CoV-2 detection only. While this is the case, the BioFire RP2.1 is identical to the BioFire RP2 except for the addition of the two assays for the detection of SARS-CoV-2, and the performance of the BioFire RP2 has previously been published (9). Another limitation is that this study only evaluated NPS specimens, which are traditionally considered the optimal specimen type for upper respiratory viral testing and the only specimen type claimed by the BioFire RP2.1 IFU. An additional limitation of this study was that it was a prospective study done over a relatively short period of time, so enrollment of specimens positive for SARS-CoV-2 was dependent on the positivity rate, which was lower than at some other time points during the pandemic. In addition, the most recent SARS-CoV-2 variants (such as omicron) were not circulating during the time of this study (July 2020 to October 2020) and were not assessed.

It is clear that diagnosing SARS-CoV-2 versus other respiratory viruses, especially influenza viruses and respiratory syncytial virus (RSV), based on symptoms alone is not possible (18, 19). In addition, as we consider the possibility of seasonality for SARS-CoV-2 as we see in the other common circulating coronaviruses (20), the need for simultaneous detection of SARS-CoV-2 and other respiratory viruses during respiratory season is clear. In 2021, outbreaks of RSV were seen across the southern United States (21), and additional similar events are likely to continue. In addition, the CDC has issued guidance recommending SARS-CoV-2 and influenza testing for any symptomatic patient being admitted to a hospital (22). The incorporation of SARS-CoV-2 into a multiplex panel allows for the simultaneous detection of the most common respiratory pathogens, leading to a more rapid result for clinical decision-making. This multiplex testing approach also saves valuable resources and is potentially time-saving for the laboratory due to being able to test for multiple respiratory pathogens at once. Moving forward, it is clear that both smaller (e.g., panels testing for SARS-CoV-2, influenza, and RSV) and larger multiplex respiratory panels will need to include SARS-CoV-2 for the foreseeable future. In addition, panel designs, such as the BioFire RP2.1, that include more than one target for SARS-CoV-2 will be a good strategy in planning for potential detection issues posed by emerging variants (23, 24).

In conclusion, this prospective evaluation of the BioFire RP2.1 for the detection of SARS-CoV-2 in NPS specimens demonstrates that the BioFire RP2.1 has an excellent PPA and NPA of greater than 98% when compared to several other available molecular assays. Moving forward, SARS-CoV-2 will need to be incorporated into any testing algorithm during respiratory season in symptomatic patients to guide patient management and treatment decisions, making respiratory panels containing SARS-CoV-2 an efficient and rapid option for syndromic respiratory testing.

The BioFire RP2.1 received FDA de novo authorization in March 2021 (25) based in part on the findings of this prospective evaluation. The BioFire RP2.1plus, which also reports out MERS-CoV, was CE-marked in August 2020 and is commercially available outside the United States only. Note that product availability of the BioFire RP2.1plus varies by country.