Comparison of commercial realtime reverse transcription PCR assays for the detection of SARS-CoV-2

Highlights

• •Commercial SARS-CoV-2 RT-PCR analytical performance comparison.
• •Analytical sensitivity of 13 SARS-CoV-2 RT-PCRs varied between 3.3–330 RNA copies.
• •All but 1 assay had sensitivity within one order of magnitude of the reference assay.
• •Only 1 assay cross reacted with another coronavirus (MERS).

Abstract

The emergence of a new coronavirus in Wuhan China has triggered a global need for accurate diagnostic assays. Initially, mostly laboratory developed molecular tests were available but shortly thereafter different commercial assays started to appear and are still increasing in number. Although independent performance evaluations are ongoing, available data is still scarce. Here we provide a direct comparison of key performance characteristics of 13 commercial RT-PCR assays. Thirteen RT-PCR assays were selected based on the criteria that they can be used following generic RNA extraction protocols, on common PCR platforms and availability. Using a 10-fold and 2-fold dilution series of a quantified SARS-CoV-2 cell-cultured virus stock, performance was assessed compared to our in house validated assay. Specificity was tested by using RNA extracted from cultured common human coronaviruses. All RT-PCR kits included in this study exhibited PCR efficiencies > 90%, except for the Sentinel Diagnostics B E-gene RUO assay (80%). Analytical sensitivity varied between 3.3 RNA copies to 330 RNA copies. Only one assay cross reacted with another human coronavirus (MERS). This study provides a technical baseline of 13 different commercial PCR assays for SARS-CoV-2 detection that can be used by laboratories interested in purchasing any of these for further full clinical validation.

1. Introduction

The emergence of a new coronavirus, SARS-CoV-2, following a spillover event in Wuhan, China [1] has triggered a global need for accurate diagnostic assays. Detection of viral RNA in clinical specimens is the hallmark of diagnosis. Initial in-house PCR protocols [2] have been developed and validated rapidly following the public release of the full genome of virus isolated from a patient on January 10^th [3]. Since then, the spread of the virus has increased the global need for accurate and validated diagnostic PCR assays. While the initial response was mainly based on the use of laboratory developed tests (LDT) in specialized reference centers familiar with rapid deployment of assays in response to outbreaks, commercial assays may have an advantage for more rapid implementation in routine hospital laboratories [4]. Currently there are close to 300 test commercialized/in development [5] however performance data is still only scarcely available [6]. Here, we provide a direct comparison of key performance characteristics of 13 RT-PCR assays using a standard RNA reference panel for assessment of analytical sensitivity (limit of detection, LoD) and specificity.

2. Methods

2.1. Selection of commercial RT-PCR assays

Commercial RT-PCR assays (Table 1 ) were selected on basis of the following criteria: i) The assay is a generic realtime reverse-transcriptase PCR that can be performed with isolated RNA from generic manual or automated nucleic acid extraction methods, ii) the assay can be performed on standard realtime PCR thermocyclers available globally, ii) the assay is available on the market and can be obtained within the timeframe of this study (1–2 weeks) or the assay could be made available by the manufacturer in a pre-release version. As the COVID-19 confinement measures affected the global market, equity in access could not be assessed.

Table 1.

Details of the compared PCR kits.

Company	Product	Country	Regulatory status	Target gene(s)
Altona Diagnostics	RealStar SARS-CoV2 1.0	Germany	CE-IVD	E, S
Tibmolbiol	LightMix Sarbeco-E/SARS-CoV-2 RdRp	Germany	RUO	E, RdRp
ThermoFisher	Taqman 2019-nCoV Assay kit v1	USA	RUO	ORF1ab, S, N
DAAN Gene	Detection Kit for 2019-nCoV	China	RUO	ORF1ab, N
Kogene Biotech	Powercheck 2019-nCoV	Korea	RUO	ORF1ab, E
Liferiver	2019-nCoV realtime multiplex RT-PCR	China	CE-IVD	ORF1ab, N
Maccura Biotechnology	SARS-CoV2 fluorescent PCR	China	NMPA	ORF1ab, E, N
R-Biopharm	Ridagene SARS-CoV2	Germany	RUO	E
Sansure Biotech	2019-nCoV nucleic acid diagnostic kit	China	CE-IVD, NMPA	ORF1ab, N
Sentinel Diagnostics B (RUO)	STAT-NAT COVID19 B	Italy	RUO	E, RdRp, N
Sentinel Diagnostics B (CE-IVD)	STAT-NAT COVID19 B CE-IVD	Italy	CE-IVD	E, RdRp, N
Sentinel Diagnostics HK (RUO)	STAT-NAT COVID19 HK	Italy	RUO	ORF1ab, N
Sentinel Diagnostics HK (CE-IVD)	STAT-NAT COVID19 HK CE-IVD	Italy	CE-IVD	ORF1ab, N
XABT	RT PCR kit for detection of 2019-nCoV	China	CE-IVD	ORF1ab, E, N
Hecin Scientific	SARS-CoV-2 Realtime PCR assay kit	China	RUO	RdRp, N
Reference assay	Corman et al.			E, RdRp

CE-IVD: Conformité Européenne-In Vitro Diagnostic ; RUO: Research Use Only; NMPA: National Medical Products Administration; CDC- Center for Disease Control and Prevention.

2.2. Assessment of analytical sensitivity and specificity

To assess analytical sensitivity of the different assays extracted RNA (MagNApureLC, Roche Diagnostics) from a SARS-CoV-2 cell-cultured virus stock (hCoV-19/Netherlands/Diemen_1363454/2020, GISAID: EPI_ISL_413570) was quantified by determining the amount of RNA copies based on the E-gene RT-PCR and quantified E-gene in vitro RNA transcripts as described by Corman et al., [2]. For each assay equal amounts of 10-fold and 2-fold dilution series were tested in triplicate on either LightCycler480 (Roche Diagnostics) or Quantstudio5 (Thermofisher Scientific) thermocycler depending on manufacturer’s recommendation and the LoD of each assay was expressed as the lowest amount of RNA copies that could be detected in 3 out of 3 replicates.

The reference assay was based on the E-gene and RdRP-gene assays described by Corman et al., [2]. using Fast Virus Master Mix (Thermofisher Scientific) and Lightcycler480 (Roche Diagnostics).

Analytical specificity was assessed by triplicate testing of a standardized RNA panel available from the European Virus Archive (EVAg; https://www.european-virus-archive.com/nucleic-acid/coronavirus-rna-specificity-panel) which contained RNA from cell-cultured coronaviruses hCoV-NL63, hCoV−OC43, hCoV-229E, MERS-CoV, and SARS-CoV-1.

All commercial kits were used according to the instructions of the manufacturer with the alteration of using the same volume of template RNA across the assays (5ul). In most kits the polymerase enzymes were included and the kits were consequently provided on dry-ice however the kit from DAAN Gene arrived at room temperature at the laboratory. The TIBmolbiol kit did not include a PCR enzyme and in this study Fast Virus Master enzyme (Thermofisher Scientific) was used. The Sentinel kits (both B and HK) were supplied with lyophilized enzyme and consequently were provided at room temperature.

3. Results

All RT-PCR kits included in this study exhibited PCR efficiencies > 90%, as determined from the slopes of the standard curve from the 10-fold dilution series, except for the Sentinel Diagnostics B RUO E-gene assay (80%). Analytical sensitivity of assays in the different kits varied between 3.3 RNA copies to 330 RNA copies (Table 2 ). One kit was designed to include only one PCR target (R-Biopharm), all other assays were designed to include multiple (2 or 3) PCR targets. These PCR targets were located in conserved regions of the ORF1ab, S, E or N genes. In most cases at least one of these PCR assays showed a sensitivity within one order of magnitude of the reference E gene assay, which was considered to be the most sensitive. Only for the R-Biopharm and Sentinel Diagnostics B RUO kits the included assays showed a LoD that was higher than one order of magnitude compared to the reference E gene PCR. R-Biopharm one target PCR had an LoD 15-fold lower than the reference assay (E gene, 50 copies).

Table 2.

Sensitivity and specificity of the PCR kits.

Company	Target gene	Sensitivity1	Specificity
Altona Diagnostics	E	3.3	SARS-1
	S	3.3	ND
Tibmolbiol	E	25	SARS-1
	RdRp	100	ND
ThermoFisher	ORF1ab	33	ND
	S	33	ND
	N	3.3	ND
DAAN Gene	ORF1ab	3.3	ND
	N	NA2, 3	NA2, 3
Kogene Biotech	ORF1ab	100	ND
	E	33	MERS, SARS-1
Liferiver	ORF1ab	33	ND
	N	3.3	SARS-1
Maccura Biotechnology	ORF1ab	3.3	ND
	E	3.3	SARS-1
	N	3.3	ND
R-Biopharm	E	50	ND
Sansure Biotech	ORF1ab	3.3	ND
	N	3.3	SARS-1
Sentinel Diagnostics B (RUO)	E	33	SARS-1
	RdRp	330	SARS-1
	N	330	ND
Sentinel Diagnostics B (CEIVD)	E	3.3	SARS-1
	RdRp	330	SARS-1
	N	330	SARS-1
Sentinel Diagnostics HK (RUO)	ORF1ab	330	SARS-1
	N	33	SARS-1
Sentinel Diagnostics HK (CE-IVD)	ORF1ab	33	SARS-1
	N	33	SARS-1
XABT	ORF1ab	3.3	ND
	E	3.3	ND
	N	3.3	ND
Hecin Scientific	RdRp	25	ND
	N	25	ND
Reference assay	E	3.3	SARS-1
	RdRp	25	SARS-1

ND: Not detected; NA: not available.

¹Expressed as lowest amount of RNA copies detectable in 3/3 replicates.

²For N-gene assay only background signal was observed.

³Kit was defrosted on arrival.

None but the Kogene Biotech E gene PCR detected other coronaviruses, which detected also MERS. Detection of SARS-CoV-1 reflected the design of the RT-PCR and was not considered as cross-reaction.

4. Discussion

Here we provide a direct comparison of key performance characteristics (LoD and analytical specificity) of commercial RT-PCR kits for the detection of SARS-CoV-2. While most assays showed good sensitivity for at least one of the included targets, others were significantly less sensitive. As all but one kits contain multiple PCR targets with at least one PCR target approaching similar sensitivity as the reference E gene, it can be concluded that the majority of commercial PCR kits can be used for the diagnosis of SARS-CoV-2 infection if single target detection is considered sufficient. However, confirmation of positive RT-PCR results is advised, especially when a low-incidence in the population is expected. Four kits (Altona Diagnostics, Maccura Biotechnology, Sansure Biotech and XABT) showed comparable sensitivity with the reference E gene assays for multiple PCR targets and would be excellent candidates for further clinical evaluation by laboratories interested in implementing RT-PCR diagnostics capacity.

Most commercial RT-PCR kits did not cross-react with other circulating coronaviruses, except the Kogene Biotech kit, which cross-reacted with MERS. This kit can be used in regions where MERS coronavirus is not circulating. However, in Middle Eastern countries this kit should be used with care, for instance by confirming positives by a different assay or by sequencing. Most RT-PCR kits included the DNA polymerase, as a consequence they have to be shipped and stored below -20⁰C. It is interesting to note that the kits from Sentinel Diagnostics contain a lyophilized DNA polymerase and can be an attractive alternative when cold chain cannot be achieved. While the initial RUO kits from Sentinel were not among the most sensitive assays, the updated STAT-NAT COVID19 B CE-IVD kit that will be released on the market showed improved sensitivities for the E gene target, comparable to the reference assay.

Although our study provides a comparative analysis of performance of the different commercial RT-PCR kits, several limitations apply. First, this is not a comprehensive study, simply because not all RT-PCR kits available on the market could be included. Second, although analytical sensitivity and specificity can be compared based on the quantified virus stock that we used, a true estimation of the LoD for all the different gene targets requires quantification to all PCR targets. With this consideration the LoDs that we report can be used for comparison but should be taken with care as a definite LoD. Third, we did not perform an evaluation using clinical samples. It is difficult to establish criteria for a minimal clinical performance level. PCR positivity in clinical samples is influenced by a number of factors including the sample type and timing. Furthermore, presence of RNA does not necessarily correlate with infectivity or capability for transmission. Nevertheless, our data provides a technical baseline for laboratories interested in purchasing any of these assays for further full clinical validation.

Author contributions

ZI supervised the project and contributed to the writing, reviewing and editing of manuscript; ZA, BW carried out the experiments; MI, VM, JC and MK contributed to the critical reviewing of the manuscript; RM supervised the project and wrote the original draft. All authors approved the final submitted version.

Funding

This study was partly funded by H2020 projects RECoVer [grant number 101003589] and EVAg [grant number 653316].

All commercial RT-PCR kits were provided free-of-charge by the different companies.

Declaration of Competing Interest

None declared.