Clinical Comparison of Three Sample-to-Answer Systems for Detecting SARS-CoV-2 in B.1.1.7 Lineage Emergence

Abstract

Purpose

Accurate molecular diagnostic assays for detecting severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), the causative agent of COVID-19, are needed for epidemiology studies and to support infection-control measures. We evaluated the analytical sensitivity and clinical performance of three sample-to-answer molecular-diagnostics systems for detecting SARS-CoV-2 using 325 nasopharyngeal swab clinical samples from symptomatic patients.

Methods

The BioFire Respiratory Panel 2.1 (RP2.1), cobas Liat SARS-CoV-2 and Influenza A/B, and Cepheid Xpert Xpress SARS-CoV-2/Flu/RSV platforms, which have been granted emergency-use authorization by the US FDA, were tested and compared.

Results

The positive percent agreement, negative percent agreement, and overall percent agreement among the three point of care testing systems were 98–100%, including for the wild-type SARS-CoV-2 (non-B.1.1.7) and a variant of concern (B.1.1.7). Notably, the BioFire RP2.1 may fail to detect the SARS-CoV-2 S gene in the B.1.1.7 lineage because of the spike protein mutation.

Conclusion

All three point of care testing platforms provided highly sensitive, robust, and almost accurate results for rapidly detecting SARS-CoV-2. These automated molecular diagnostic assays can increase the effectiveness of control and prevention measures for infectious diseases.

Introduction

A cluster of pneumonia cases of unknown etiology was reported in December of 2019 and was confirmed to be the novel coronavirus 2019 (2019-nCoV).^1–3 By July 2021, more than 200 million reported cases of coronavirus disease (COVID-19), caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), were reported and were associated with over four million deaths globally (https://covid19.who.int/). Accurate and reliable molecular-diagnostic assays for detecting SARS-CoV-2 may help clinicians better understand the etiology of suspected SARS-CoV-2 infection.⁴ Nucleic acid-amplification tests are highly sensitive and specific and are considered as the gold standard for SARS-CoV-2 diagnosis.^5–7 A correct SARS-CoV-2 diagnosis contributes to disease treatment and regimens. Although many molecular diagnostic platforms have become available to meet the enormous demands during the COVID-19 pandemic,^8–10 the clinical performance of these diagnostic methods has not been thoroughly evaluated. Affordable point of care testing (POCT) kits serve as alternative diagnostic methods but are limited by their availability, although they are widely used to test the general public.^11,12 Recently, a SARS-CoV-2 test was added to the Liat SARS-CoV-2 and Influenza A/B (Roche Molecular Systems, Inc., Pleasanton, CA, USA), Cepheid Xpert Xpress SARS-CoV-2/Flu/RSV (Cepheid, Sunnyvale, CA, USA), and BioFire Respiratory Panel 2.1 (RP2.1; BioFire Diagnostics, LLC; Salt Lake City, UT, USA) tests, which are commonly used multiplex PCR panels for diagnosing upper respiratory tract infections by multiple co-existing respiratory viruses. Importantly, new SARS-CoV-2 variants of concern (VOCs) may enhance virus transmissibility and/or disease severity, as well as diagnostic and/or treatment failure.¹³ SARS-CoV-2 lineages carrying the amino acid substitution N501Y spread rapidly in the United Kingdom in late autumn 2020.¹⁴ No relevant research exploring the performance and accuracy of POCT platforms for detecting SARS-CoV-2 VOCs has been performed.

Here, we evaluated the analytical and clinical performance of three sample-to-answer POCTs in terms of their abilities to qualitatively detect SARS-CoV-2 RNA from wild-type (non-B.1.1.7) or VOC (B.1.1.7) strains. These POCTs include the BioFire RP2.1 test, Liat SARS-CoV-2 and Influenza A/B, and Cepheid Xpert Xpress SARS-CoV-2/Flu/RSV tests. We independently evaluated the performance characteristics of the three platforms for detecting SARS-CoV-2, particularly with respect to the VOC B.1.1.7 lineage.

Materials and Methods

Study Design and Clinical Specimens

This study was registered on February 8, 2021 and approved by the Tri-Service General Hospital Institutional Review Board (approval number C202005041). We tested 325 de-identified nasopharyngeal swab specimens collected from patients suspected of having COVID-19 using LIBO Specimen Collection and Transport Swab Kits with Universal Transport Medium (New Taipei City, Taiwan). Residual viral transport medium was collected and stored at −80°C. The sample collection periods were between June and September 2020 and May and June 2021. Figure 1 depicts the study design. All SARS-CoV-2 testing results were confirmed by RT-PCR developed by us as described previously.¹⁵ Briefly, the SARS-CoV-2 assay simultaneously detected SARS-CoV-2 E and ORF1ab gene along with the human RP gene to monitor the quality of nucleic acid. Results were interpreted as positive or negative based on the detection of E and ORF1ab or lack of detection of those genes, respectively.

Figure 1.

Flow chart of study design.

BioFire Respiratory Panel 2.1

BioFire Respiratory Panel 2.1 (RP2.1; BioFire Diagnostics, LLC; Salt Lake City, UT) tests are commonly used multiplex PCR panels for diagnosing upper respiratory tract infections, including 22 viral and bacterial respiratory pathogens such as influenza A/B, respiratory syncytial virus, and SARS-CoV-2 etc. The BioFire RP2.1 test was performed using the BioFire FilmArray EZ Configuration system according to the manufacturer’s instructions. Briefly, 300 μL of sample was mixed with sample buffer and injected into a test pouch containing all necessary reagents for nucleic extraction, PCR amplification, and detection of the respective targets. The RP2.1 test contains two independent assays targeting the spike (S) and membrane (M) genes in the SARS-CoV-2 genome. The results were interpreted using the BioFire system software, which interprets each assay independently. If either one or both of the assays are positive, the test will show that SARS-CoV-2 was detected. If both assays are negative, the test report result will show that SARS-CoV-2 was not detected.

Liat SARS-CoV-2 and Influenza A/B Test

The Liat SARS-CoV-2 and Influenza A/B test is a multiplex reverse transcriptase (RT)-PCR assay for rapid in vitro detection and discrimination of RNA targets for three viruses, namely, SARS-CoV-2, influenza A virus, and influenza B virus, in nasopharyngeal swabs. The test was performed using the Liat system according to the manufacturer’s instructions. Briefly, 200 μL of sample was added to a Liat tube inserted into the tube slot of the cobas Liat System. An internal control was used for procedural checks. The Liat assay was used to detect two target regions (ORF1a/b and nucleocapsid [N]) of the SARS-CoV-2 genome. The results of Liat SARS-CoV-2 and Influenza A/B are interpreted as follows: SARS-CoV-2 not detected (no SARS-CoV-2 RNA detected), SARS-CoV-2 detected (SARS-CoV-2 RNA present).

Cepheid Xpert Xpress SARS-CoV-2/Flu/RSV Test

The GeneXpert Dx system is an integrated diagnostic device that performs automated specimen processing and real-time RT-PCR analysis. The Xpert Xpress SARS-CoV-2/Flu/RSV test consists of two main components, the Xpert plastic cartridge and GeneXpert instrument. The Cepheid Xpert Xpress SARS-CoV-2/Flu/RSV Test was designed to amplify three viral pathogens: SARS-CoV-2, influenza, and respiratory syncytial virus (RSV). The assay detects the gene sequences of the envelope (E) and nucleocapsid (N2) proteins of SARS-CoV-2. Each cartridge includes a sample processing control and a probe check control to ensure that the sample was processed correctly and the GeneXpert instrument measures the fluorescence signal from the probes to monitor bead rehydration, reaction tube filling, probe integrity, and dye stability. Briefly, 300 µL of each nasal pharyngeal swab specimen was added to a separate Xpert Xpress SARS-CoV-2/Flu/RSV cartridge and loaded into an Xpert instrument. The result was interpreted according to the manufacturer’s instructions. Positive results indicated the presence of RNA of influenza A and B, RSV, or SARS-CoV-2 viruses. The results were interpreted automatically by the GeneXpert System.

Detection of SARS-CoV-2 VOCs

To screen a SARS-CoV-2 VOC (B.1.1.7 lineage), we used VirSNiP SARS-CoV-2 Spike N501Y and Spike del H69/V70 (TIB Molbiol, Berlin, Germany), which use real-time RT-PCR post-melting curve analysis to detect the mutations N501Y and del 69–70 in SARS-CoV-2-positive specimens. Here, we detected the spike gene mutations del 69–70 and N501Y on a LightCycler 480 (Roche Molecular Systems, Inc.) according to the manufacturer’s instructions. To confirm that the results were consistent with those of rapid detection of the SARS-CoV-2 VOC, presumptive cases (denoted as TSGH42 and TSGH43) caused by the VOC were also assessed by whole-genome sequencing as described previously, and the lineages were confirmed as B.1.1.7 (UK) using GISAID software.¹⁶

Analytical Tests

The limit of detection (LoD) values for SARS-CoV-2 were determined using AMPLIRUN SARS-CoV-2 RNA controls (Vircell, Granada, Spain), which contained purified genomic RNA from the indicated viruses. These controls were used for absolute quantification. The controls were used to prepare a serial-dilution panel with 1–10 replicates. The analytical sensitivities of the BioFire RP2.1, Liat SARS-CoV-2 and Influenza A/B, and Cepheid Xpert Xpress SARS-CoV-2/Flu/RSV tests were defined as the lowest dilution at which all replicates were identified as positive for SARS-CoV-2.

Comparison of Clinical Test Performance Using Clinical Samples

We included 325 retrospective nasopharyngeal swab specimens from patients hospitalized at the Tri-Service General Hospital (Taipei City, Taiwan). Clinical testing was performed for all 325 clinical specimens, and the results were compared to those of our developed RT-PCR, which was used as the reference method.¹⁵

Results

Analytic Sensitivities of Three Sample-to-Answer Platforms for Detecting SARS-CoV-2

We assessed the empirical sensitivity of the BioFire RP2.1, Liat SARS-CoV-2 and Influenza A/B, and Cepheid Xpert Xpress SARS-CoV-2/Flu/RSV for detecting SARS-CoV-2. For consistency, we used the unit copies/mL to compare the results obtained from all three platforms when determining the LoD of SARS-CoV-2. Accordingly, we defined the LoD as the minimum concentration at which a detection rate of 100% could be achieved for 3–10 replicates. The LoD established as per this criterion ranged from 2000 to 25 copies/mL in the three platforms (Table 1).

Table 1.

Detection Limit of Three Molecular Diagnostic Assays for SARS-CoV-2 Nucleic Acid

Molecular Diagnostic Assay	SARS-CoV-2 Target Region	% (No.) of Detected Replicates at the Indicated Dilution (Copies/mL)
		2000	1000	500	250	100	50	25
BioFire RP2.1	S, M	100 (1/1)	100 (1/1)	100 (3/3)	100 (10/10)	20 (1/5)	0 (0/1)	NA
Cobas Liat System	N, ORF1a/b	100 (3/3)	100 (6/6)	100 (8/8)	100 (10/10)	70 (7/10)	10 (1/10)	0 (0/3)
Cephedi Xpert Xpress	N2, E	100 (3/3)	100 (6/6)	100 (8/8)	100 (10/10)	50 (5/10)	0 (0/10)	0 (0/3)

Abbreviations: S, spike; M, membrane proteins; N, nucleocapsid; E, envelope; ORF1 a/b, open reading frame 1 a/b.

Clinical Performance of Three Sample-to-Answer Platforms

We analyzed 325 clinical samples in this study; 125 samples tested as positive, which was confirmed by the Taiwan CDC central laboratory, and 200 samples were negative for SARS-CoV-2. The 125 positive samples were further identified as wild-type (non-B.1.1.7) or VOC (B.1.1.7). Concordant and discordant positive results were obtained for all three assays when detecting SARS-CoV-2, when detecting SARS-CoV-2, and when the detection results of any given pair of assays were compared, they showed 98–100% agreement (Table 2 and Supplementary Table 1). The positive percent agreement, negative percent agreement, and overall percent agreement between the Cepheid Xpert Xpress system and cobas Liat System were 100% for wild-type (non B.1.1.7) SARS-CoV-2 specimens, and those between these two assays and BioFire RP2.1 were 98%, 100%, and 99.6% respectively. Among the 50 positive wild-type (non B.1.1.7) SARS-CoV-2 specimens, one sample (sample 7) had cycle threshold (Ct) values of 36.8 and 35.3 for the Cepheid Xpert Xpress system and Cobas Liat System, respectively. However, it was falsely detected as negative with BioFire RP2.1, as BioFire tests do not report Ct values. The Cepheid Xpert Xpress system, BioFire RP2.1, and cobas Liat System showed 100% positive agreement for the other 75 positive VOC (B.1.1.7) SARS-CoV-2 specimens. Unexpectedly, based on further analysis of the results of the BioFire RP2.1 platform, we found one positive result with only M detected, but the test failed to detect S (Supplementary Figure 1). This sample had a Ct value of 26.5 on the cobas Liat System. This result indicates that a novel mutation caused mismatches with the primer for S-gene assays in the VOC of SARS-CoV-2.

Table 2.

Comparison of Clinical Performance Among Three Sample-to-Result Molecular Assays for SARS-CoV-2

Molecular Assay Platform	Number of Results Obtained versus the Reference RT-PCR Method
	SARS-CoV-2 Wild Type (Non-B.1.1.7)		SARS-CoV-2 VOC (B.1.1.7)
	Positive	Negative	Positive	Negative
BioFire RP2.1 panel
Positive	49	0	75	0
Negative	1	200	0	200
Cobas Liat System
Positive	50	0	75	0
Negative	0	200	0	200
Cepheid Xpert Xpress
Positive	50	0	75	0
Negative	0	200	0	200
Positive percent agreement (95% CI)	98.0–100% (89.5–100%)		100% (95.1–100%)
Negative percent agreement (95% CI)	100% (98.1–100%)		100% (98.1–100%)
Overall percent agreement (95% CI)	99.6–100% (97.8–100%)		100% (98.6–100%)

Workflow Evaluation

We next evaluated the total turnaround time (TAT) for each specimen, including the sample preparation, hand-on time, assay detection time, and data interpretation steps. Table 3 shows the overall workflow assessment and a comparison of the three POCT platforms.

Table 3.

Characteristics of Three Sample-to-Answer Molecular SARS-CoV-2 Assays

Characteristics	BioFire Diagnostics Panel	Cobas Liat System	Cepheid Xpert Xpress
Manufacturer	BioMérieux	Roche	Cepheid
Sample type(s)	NP, NSa	NP, NS	NP, NS
Sample volume required (μL)	300	200	300
Detection assay	BioFire Respiratory Panel 2.1 (RP2.1)	Cobas Influenza A/B & SARS-CoV-2	Xpert Xpress SARS-CoV-2/Flu/RSV
Limit of detection (copies/mL)	250	250	250
Assay Detection time	~45 min/sample	~20 min/sample	~42 min/sample
Maximum throughout	1 per instrument	1 per instrument	2 per instrumentb
Hands-on time per sample	2 min	1 min	2 min
Result interpretation	Automated	Automated	Automated
RNA extraction required	No	No	No
Running cost (per assay)	135–150 U.S.D	50–75 U.S.D	50–75 U.S.D
Instrument dimensions (H,W,L)	16.5cm, 25.4cm, 39.3cm	19cm, 11.5cm, 24.1cm	30.5cm, 16.1cm, 29.7cmb

Note:^bGX-II-2 module instrument.

^aAbbreviations: NP, nasopharyngeal swab; NS, nasal swab; H, height; W, width; L, length.

Analysis of the overall TAT, from sample collection to results, showed that the cobas Liat System had the lowest TAT (approximately 20 min), followed by the Cepheid Xpert Xpress system (42 min) and BioFire RP2.1 (45 min). All three platforms evaluated in this study decreased the hands-on time and reduced the risk of exposure compared to conventional RT-PCR methods, which is particularly important when handling samples from patients suspected of having COVID-19. These POCT platforms are valuable additions to diagnostic laboratories for rapid and accurate diagnostic results in detecting SARS-CoV-2.

Discussion

In vitro diagnostic nucleic acid-amplification tests assays for SARS-CoV-2 employ RT-PCR, which takes less than 1 h or a few hours depending on the number of samples processed or if a specific facility is well-equipped, which facilitates reliable and easy-to-implement in vitro diagnostic assays for the molecular diagnosis of COVID-19.^17,18 In this study, we determined the analytic sensitivity of three POCT platforms. Similar LoD values were obtained for the BioFire RP2.1, cobas Liat SARS-CoV-2 and Influenza A/B, and Cepheid Xpert Xpress SARS-CoV-2/Flu/RSV tests. We also examined the clinical performance of the three POCTs. Our findings suggest that the lower diagnostic sensitivity of the BioFire RP2.1 for clinical samples led to false-negative results in samples with very low viral loads, Moreover, this test failed to detect S gene CoV-2 and Influenza A/B and Cepheid Xpert Xpress SARS-CoV-2/Flu/RSV, BioFire RP2.1 showed limitations at lower viral load concentrations in wild-type (non-B.1.1.7) SARS-CoV-2 (eg, sample 7 in our study). Additionally, detection by BioFire RP2.1 was impaired for VOCs (B.1.1.7) of SARS-CoV-2 because of the spike protein mutation.

When relying on fully automated platforms from samples to answers, all of the three evaluated platforms have shortened the time required to obtain results and expanded the number of laboratories capable of testing for SARS-CoV-2, while also testing for co-infecting pathogens (such as other upper respiratory pathogens) as alternative diagnoses, which is key for determining the most effective treatment regimen. Various co-factors influence SARS-CoV-2-detection results, including the input volume, extraction methods, and workflow requirements.¹⁹ Our study provides a perspective for deciding which molecular diagnostic test should be implemented in clinical laboratories. To date, several reports of concurrent infections with other pathogens, such as influenza virus and other seasonal coronaviruses, have suggested that coinfection influences the morbidity and mortality of patients with COVID-19.^20–22 Therefore, it is crucial for clinicians to rule out SARS-CoV-2 or other upper respiratory viral infections. Notably, BioFire RP2.1 provided more than 20 detection results, including SARS-CoV-2, providing important data for laboratories to rapidly detect and differentiate co-circulating respiratory pathogens and make these tests accessible in remote areas where higher-complexity assays are not feasible.²³

Conclusions

The three molecular-diagnostics platforms examined simultaneously test for SARS-CoV-2, influenza A/B, and/or RSV, which can greatly benefit hospitals by enabling the management and control of infectious diseases.

Funding Statement

This study was supported by the Tri-Service General Hospital, Taipei, Taiwan, ROC, (grant number TSGH-D-110100). The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Abbreviations

COVID-19, coronavirus disease 2019; LOD, limit of detection; RT-PCR, reverse transcriptase polymerase chain reaction; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2; POCT, point of care testing; VOC, variant of concern; WHO, World Health Organization.

Institutional Review Board Statement

This study was approved by the Institutional Review Board of Tri-Service General Hospital (TSGHIRB No.: C202005041), registered on Feb 8, 2021.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study. This study was conducted in accordance with the Declaration of Helsinki.

Author Contributions

Conceptualization, Jung-Chung Lin, Kuo-Ming Ye, Chien-Wen Chen, and Feng-Yee Chang; Data curation, Ming-Jr Jian and Hsing-Yi Chung; Formal analysis, Shih-Yi Li and Ming-Jr Jian; Investigation, Hsing-Yi Chung, Chih-Kai Chang, Ming-Tsan Liu, Ji-Rong Yang, and Cherng-Lih Perng; Methodology, Chih-Kai Chang, Shan-Shan Hsieh, and Cherng-Lih Perng; Supervision, Hung-Sheng Shang, Jung-Chung Lin, Kuo-Ming Ye, Chien-Wen Chen, Feng-Yee Chang, Ji-Rong Yang, Sheng-Hui Tang, and Cherng-Lih Perng; Writing – original draft, Ming-Jr Jian; Writing – review & editing, Ming-Jr Jian, Feng-Yee Chang and Hung-Sheng Shang. All authors made substantial contributions to conception and design, acquisition of data, or analysis and interpretation of data; took part in drafting the article or revising it critically for important intellectual content; agreed to submit to the current journal; gave final approval of the version to be published; and agree to be accountable for all aspects of the work.

Disclosure

The authors declare no conflicts of interest for this work.