Clinical performance evaluation of BD SARS-CoV-2 reagents for BD MAX^TM System in asymptomatic individuals

Abstract

Transmission by asymptomatic individuals is a persistent hurdle in the effort to control the spread of SARS-CoV-2. Therefore, it is essential to continue developing assays and evaluate their performance for detection of SARS-CoV-2 in individuals without COVID-19 symptoms. In this study, 223 nasopharyngeal swab specimens collected from COVID-19 asymptomatic individuals were tested using the BD SARS-CoV-2 (RT-PCR-based) reagents for the BD MAX™ System and compared with results obtained with the Biomerieux BioFire® Respiratory RT-PCR Panel. Positive and negative percent agreements of 100% (95% CI, 84.5%–100%) and 99.0% (95% CI, 96.5%–99.7%), respectively, were observed for the BD SARS-CoV-2 assay. These results demonstrate the effectiveness of the BD SARS-CoV-2 assay for detecting SARS-CoV-2 in asymptomatic individuals and suggest that this assay can facilitate optimized case surveillance and infection control efforts. Investigations using larger sample sizes of asymptomatic individuals would be beneficial to support the findings in this study.

1. Introduction

As the reactive strategies implemented during the SARS-CoV-2 pandemic to curb the spread of the virus are delivering their expected outcomes and COVID-19 becomes an endemic disease in many parts of the world, testing will continue to play a major role in surveillance and infection control strategies [1]. Since those activities require the availability of rapid and accurate diagnostic methods [2], it is expected that molecular testing for SARS-CoV-2, in tandem with other mitigation measures such as the isolation of infected individuals and continued vaccination efforts, will continue to play a crucial role in preventing COVID-19 outbreaks in communities and workplaces in the foreseeable future [3]. Given that asymptomatic transmission frequently occurs [4], [5], [6], [7], [8], [9], and that as many as 40% of confirmed cases (those receiving a screening or confirmatory polymerase chain reaction [PCR] test) involve asymptomatic individuals [10,11], it is crucial to continue developing and validating highly sensitive SARS-CoV-2 assays to ensure that asymptomatic cases are identified and properly managed.

Since the beginning of the pandemic in 2019, the World Health Organization (WHO) and the U.S. Food and Drug Administration (FDA) have issued emergency use (Emergency Use Listing [EUL] and Emergency Use Authorization [EUA], respectively) for the development of in vitro diagnostic assays to detect the SARS-CoV-2 virus, including molecular and antigen testing systems [12], [13], [14]. At the onset of the pandemic, however, limited resources (in terms of health care providers’ (HCP) effort, laboratory supplies, and assay reagents) and overwhelming demand for SARS-CoV-2 testing resulted in tests being primarily performed with symptomatic individuals. As the understanding of SARS-CoV-2 transmission grew and the role of asymptomatic individuals in viral transmission became established, however, the validation and availability of molecular testing for asymptomatic individuals gained importance. In June 2020, a guidance document was released by the FDA to inform tests developers on the validation of COVID-19 assays intended for the screening of individuals without symptoms or other reasons to suspect infection with the virus, and for pooled sample testing [15]. In March 2021, the FDA also released supplemental guidance for the development of assays involving a serial testing indication for individuals without symptoms or other reasons to suspect COVID-19 [16].

The BD SARS-CoV-2 Reagents for BD MAX™ System assay (“BD SARS-CoV-2 assay;” Becton, Dickinson and Company; BD Life Sciences – Integrated Diagnostic Solutions, Sparks, MD) is a molecular testing system using reverse transcriptase-polymerase chain reaction (RT-PCR) that was initially authorized under EUA for the detection of SARS-CoV-2 RNA from individuals suspected of COVID-19 [17]. This study evaluated the performance of this assay in a population without symptoms or other reasons to suspect COVID-19 infection by comparing its results to those obtained with the Biomerieux BioFire® Respiratory Panel 2.1 (“BioFire SARS-CoV-2 assay;” Biomerieux, BioFire Diagnostics, Salt Lake City, UT).

2. Methods and materials

2.1. Specimen collection and storage

Clinical nasopharyngeal (NP) specimens were consecutively collected from 3 clinical sites and one specimen vendor in the US from October 2020 to October 2021 to meet the required sample size, based on the FDA EUA guidance documentation for developers of molecular diagnostic tests [18] (July 2020 & October 2021 versions). These specimens were obtained from asymptomatic male and female individuals 1 year old and over who elected to undergo SARS-CoV-2 testing for any reasons other than (1) confirmed exposure to the virus or (2) suspected active infection. The NP specimens were stored in universal viral transport media (BD Universal Viral Transport, Becton, Dickinson and Company; BD Life Sciences – Integrated Diagnostic Solutions, Sparks, MD) or Viral Transport Media (VTM; COPAN Diagnostics Inc., Murrieta, CA) and frozen at -70°C.

2.2. Participant consent statement

For those specimens collected prospectively, no study procedures were performed without an informed consent process and signature of a consent form. Institutional review board approval of the protocol was received from Advarra Institutional Review Board prior to study initiation and de-identified specimens were used for testing. De-identified remnant specimens were obtained with Ethical & Independent Review Services or Western Institutional Review Board approval with waiver of informed consent. The study was also conducted according to the principles set forth by the Declaration of Helsinki and Good Clinical Practices.

2.3. Testing procedure

Deidentified clinical specimens were allowed to thaw at room temperature before aliquoting. For each clinical specimen, 3 aliquots were prepared according to manufacturers’ instructions. One aliquot (750uL of pipetted sample) was prepared for testing with the BD MAX SARS-CoV-2 assay, which utilizes multiplexed primers and probes that are designed to amplify 2 unique regions of the SARS-CoV-2 nucleocapsid (N) gene, N1 and N2, and the human ribonuclease P (RNase P) gene. Another aliquot (300uL of pipetted sample) was prepared for testing with an FDA De Novo granted RT-PCR assay, the Biomerieux BioFire® Respiratory Panel 2.1 (“BioFire SARS-CoV-2 assay;” Biomerieux, BioFire Diagnostics, Salt Lake City, UT), which utilizes multiplexed primers and probes that are designed to amplify unique regions of the SARS-CoV-2 spike and membrane proteins. The third aliquot (400uL of pipetted sample) was stored frozen for future discordant testing with the EUA Cepheid Xpert® Xpress SARS-CoV-2/Flu/RSV assay (“Xpert SARS-CoV-2 assay,” Cepheid, Sunnyvale, CA), if necessary. All initial testing was performed internally at BD Life Sciences – Integrated Diagnostic Solutions, however, discordant samples were shipped on dry ice to an external laboratory for testing with the Xpert SARS-CoV-2 assay. All testing, inclusive of input sample volume, was conducted as described in each test's respective Instructions for Use document. The study team was blind to the original reported standard-of-care result when performing the tests. The standard-of-care result was only utilized to ensure the sample cohort would have sufficient numbers of positive and negative samples and was not used in the assessment of agreement between the test under evaluation and the comparator methods. All environmental monitoring swabs generated negative results for all SARS-CoV-2 targets before any testing occurred and external processing controls yielded the expected results on each day of validation testing.

2.4. Data analysis

Test results from the BioFire SARS-CoV-2 assay were used as the comparator. Specimens are considered positive for SARS-CoV-2 by the BD MAX SARS-CoV-2 assay (index test) if either the N1 or N2 gene target are detected. The positive percent agreement (PPA) and the negative percent agreement (NPA) between the BD MAX SARS-CoV-2 assay results and comparator were calculated as [(index and comparator positive) / total comparator positive] and [(index and comparator negative) / total comparator negative], respectively [19]. The 95% confidence intervals (CI) were computed based on the Wilson Score method [20]. Acceptance criteria for the BD MAX SARS-CoV-2 assay for FDA's Emergency Use Authorization for asymptomatic SARS-CoV-2 testing were ≥95% for PPA (lower bound of the two-sided 95% confidence internal >76%) and ≥98% NPA (lower bound of the two-sided 95% confidence internal >95%) [12]. This manuscript was prepared according to STARD guidelines for diagnostic accuracy studies reporting [21].

3. Results

A total of 224 specimens meeting the eligibility criteria were collected; one specimen was excluded due to insufficient volume to test, bringing the number of evaluable specimens to 223. Of those evaluable specimens, 36.8% (82/223) were from male and 63.2% (141/223) from female participants, and the following age distribution was observed: 3.1% (7/223) from participants <18 years old, 39.0% (87/223) for participants 18 to 29 years old, 26.9% (60/223) for those 30 to 39, 27.8% (62/223) for participants 40 to 65 years of age, and 3.1% (7/223) for those over the age of 65. Twenty-one (21) specimens were positive and 200 were negative by both BD MAX SARS-CoV-2 and the comparator method (positivity rate of 9.4% for both assays). There were 2 BD MAX SARS-CoV-2-positive but comparator negative specimens. The PPA was 100% (95% CI, 84.5%–100%) and the NPA was 99.0% (95% CI, 96.5%–99.7%) (Table 1 ). Upon testing of the 2 discordant specimens by a third method, the Xpert SARS-CoV-2 assay, it was found that one BD MAX SARS-CoV-2 positive specimen was also positive by Xpert SARS-CoV-2 and 1 specimen positive by BD MAX SARS-CoV-2 was negative by Xpert SARS-CoV-2 (Table 2 ). Three (3) invalid test results were observed when executing the BioFire RP 2.1 test. Per BioFire IFU, these invalid samples were subjected to a repeat run and all were resolved, that is, upon repeat testing, 2 results were negative for SARS-CoV-2 and one was positive, There were no invalid results observed on the BD MAX assay.

Table 1.

Performance of the BD MAX SARS-CoV-2 assay compared to BioFire SARS-CoV-2 assay.

		BioFire
		Positive	Negative	Total
MAX	Positive	21	2	23
	Negative	0	200	200
	Total	21	202	223
PPA 100% (CI, 84.5%, 100%)
NPA 99.0% (CI, 96.5%, 99.7%)

Abbreviations: PPA = positive percent agreement; NPA = negative percent agreement; CI = confidence interval.

Table 2.

Discordant specimens tested by Xpert SARS-CoV-2 assay.

Specimen ID	MAX N1 Ct	MAX N2 Ct	BioFire	Xpert (Ct)
1	Pos (35.3)	Neg (-1)	Neg	Pos (42.4)
2	Neg (-1)	Pos (34.2)	Neg	Neg (0.0)

Abbreviations: Ct = threshold cycle.

4. Discussion

The BD MAX SARS-CoV-2 assay showed a 100% (21/21) (95% confidence internal (84.5%–100%) PPA and 99.0% (200/202) (95% confidence interval (96.5%–99.7%) NPA, compared to the BioFire RP2.1 SARS-CoV-2 assay. Of the 2 BD MAX SARS-CoV-2 assay-positive/ BioFire RP2.1 SARS-CoV-2 assay-negative specimens, one was positive and one was negative by the discordant test.

Although the precise percentage of the COVID-19 population that remains asymptomatic through the course of the infection has not been determined [[6], [7], [8],22,23], it is clear that this segment of the population contributes to the spread of SARS-CoV-2 and may exhibit viral loads comparable to those found in individuals with symptoms [6,[23], [24], [25]]. The breakthrough infections that occur among vaccinated individuals are also, in part, likely due to transmission from the asymptomatic population [26]. In order to effectively mitigate SARS-CoV-2 transmission, it may be beneficial to test asymptomatic individuals – especially those with contact or within close proximity to positive individuals, or those engaging in social activities (travel, sport venues, school, etc.) with large crowds or where physical proximity to others is required.

In response to the COVID-19 pandemic, substantial effort has been dedicated to developing sensitive diagnostic assays to detect the SARS-CoV-2 virus. With emerging SARS-CoV-2 variants, molecular testing platforms, particularly those detecting multiple genome targets, exhibit greater sensitivity and offer more robustness than antibody tests [27]. While it is already known that molecular testing is a highly sensitive SARS-CoV-2 detection method when used with symptomatic individuals, our study confirms that such molecular testing also performs well to detect the virus in asymptomatic individuals. However, the results obtained should be considered in relation to the limited number of positive samples evaluated, and as such, conducting further studies involving larger sample sizes would be useful.

5. Conclusion

The BD MAX SARS-CoV-2 assay met FDA's acceptance criteria for EUA SARS-CoV-2 detection method in asymptomatic individuals and displayed a high clinical sensitivity and specificity for the detection of SARS-CoV-2, even in individuals without symptoms or other reasons to suspect a COVID-19 infection. Investigations using larger sample sizes of asymptomatic individuals would be beneficial to support the findings in this study.

Authors’ Contributions

Karen Yanson: Conceptualization, resources, writing review and editing, project administration; William Laviers: Methodology, investigation, validation, formal analysis, data curation; Faten Suhaidi: Investigation; Zachary Greeley: Investigation; Courtney Merryman: Investigation; Reda Proctor: Investigation; Dominique Hall: Investigation; and Lori Neely: Supervision, methodology, writing review and editing, resources.

Declaration of competing interests

KY, WL, FS, ZG, CM, RP, DH, and LN are employees of the study sponsor, Becton, Dickinson and Company; these authors have no other potential conflicts of interest to declare.