Abstract
Nucleic acid tests to detect the SARS-CoV-2 virus have been performed worldwide since the beginning of the COVID-19 pandemic. For the quality assessment of testing laboratories and the performance evaluation of molecular diagnosis products, reference materials (RMs) are required. In this work, we report the production of a lentiviral SARS-CoV-2 RM containing approximately 12 kilobases of its genome including common diagnostics targets such as RdRp, N, E, and S genes. The RM was measured with multiple assays using two different digital PCR platforms. To measure the homogeneity and stability of the lentiviral SARS-CoV-2 RM, reverse transcription droplet digital PCR (RT-ddPCR) was used with in-house duplex assays. The copy number concentration of each target gene in the extracted RNA solution was then converted to that of the RM solution. Their copy number values are measured to be from 1.5 × 105 to 2.0 × 105 copies/mL. The RM has a between-bottle homogeneity of 4.80–8.23% and is stable at 4 °C for 1 week and at −70 °C for 6 months. The lentiviral SARS-CoV-2 RM closely mimics real samples that undergo identical pre-analytical processes for SARS-CoV-2 molecular testing. By offering accurate reference values for the absolute copy number of viral target genes, the developed RM can be used to improve the reliability of SARS-CoV-2 molecular testing.
Supplementary Information
The online version contains supplementary material available at 10.1007/s00216-021-03846-y.
Keywords: SARS-CoV-2, Reference material (RM), Digital PCR, Lentiviral particle
Introduction
The severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) that originated in Wuhan, China, in 2019 causes COVID-19 infections [1] and can lead to severe lung damage and mortality [2]. While antigen and antibody testing kits have been developed for the rapid diagnosis of COVID-19, the World Health Organization (WHO) recommends the use of nucleic acid tests (NATs) as a standard means of confirmation of SARS-CoV-2 infection due to their detection capabilities for the unique viral sequences of SARS-CoV-2 [3]. So far, most molecular diagnostic kits use the reverse transcription quantitative polymerase chain reaction (RT-qPCR) method for its high sensitivity and specificity [4, 5].
The importance of reference materials (RMs), meaning homogenous and stable materials with specified properties [6], has been highlighted as molecular diagnostics are widely used in diagnosing infectious diseases worldwide [7]. RMs for specific pathogens are needed for the accurate quantification of their genetic materials by setting reliable values to compare and evaluate the performance of diagnostic kits and to compensate human errors during the process. Current SARS-CoV-2 RMs for molecular diagnosis take various forms, including plasmid DNA, in vitro transcribed RNA [8–12], virus particles containing SARS-CoV-2 RNA [13, 14], and the SARS-CoV-2 virus itself [15]. Comparisons of the established NAT for SARS-CoV-2 have shown that the Cq values from different assays can vary greatly with the same amount of template, indicating the necessity of RMs with specified target concentrations [16, 17]. Various RMs for SARS-CoV-2 are in fact currently available; however, most are limited as users prefer RMs with reference values in absolute quantification of specific genes rather than total nucleic acid measurements or relative quantifications.
Developed by the Korea Research Institute of Standards and Science (KRISS), RM 111–10-507 (batch 2) consists of partial Orf1ab, RdRp, N, E, and S genes of SARS-CoV-2 enclosed in a lentiviral packaging system. The reference values, or more specifically the copy number concentration of the selected genes, are obtained using one-step reverse transcription droplet digital PCR (RT-ddPCR), which can quantify sequence-specific RNA without extra calibrators [18–21]. The between-bottle homogeneity as well as short-term and long-term stability was tested, and measurement uncertainty was evaluated according to ISO Guide 35 [22].
Materials and methods
Cell cultures and preparation of RNA
Jurkat (Korean Cell Line Bank, Seoul, Korea) cells were grown in RPMI 1640 media (Cat. 11,875,093, Gibco, Carlsbad, CA) supplemented with 10% FBS (Cat. 26,140,079, Gibco) at 37 °C, 5% CO2. 5 × 106 of these cells were prepared as pellets to extract total RNA using an RNeasy Mini Kit (Cat. 74,104, QIAGEN, Hilden, Germany) according to the manufacturer’s instructions. The extracted RNA was eluted with nuclease-free water (Cat. W4502, Ambion, Austin, TX) and the concentration was measured using a NanoDrop (XP205, METTLER TOLEDO, Columbus, OH). Carrier RNA (Cat. 1,068,337, QIAGEN) and Jurkat cell RNA were used to stabilize the RM.
Preparation of lentiviral particles
A first-strand cDNA was synthesized from SARS-CoV-2 RNA (NCCP43326) provided by the NCCP (National Culture Collection of Pathogen) using a SuperScript III first-strand synthesis system for RT-PCR (Invitrogen, Waltham, MA) according to the manufacturer’s instructions. Eight target regions were amplified with specific primers (see Electronic Supplementary Material Table S1) and nPfu Forte polymerase (Enzynomics, Daejeon, Korea) or Herculase II Fusion Enzyme with dNTPs Combo (Agilent Technologies, Santa Clara, CA). Through multiple overlap extension PCR [23], insert A was obtained from six PCR products and insert B was obtained from two PCR products. Primers were designed to include sequences with overlapping fragments for PCR stitching as well as extra base pairs including restriction enzyme sites or stop codons. SnaB1, BamH1, and Sal1 restriction enzymes (NEB, Ipswich, MA) were used to insert the final PCR products into the lentiviral vectors (Lugen, Seoul, Korea). Completed pCDH-A and pCDH-B vectors were verified through Sanger sequencing (Cosmogenetech, Seoul, Korea). Each lentivirus sample (1 × 108 IFU/mL) was obtained through polyethylene glycol (PEG) precipitation using PEG-it Virus precipitation solution (SBI, Palo Alto, CA).
Production of reference material
Each lentivirus (Lenti-A and Lenti-B) was heat-inactivated for 30 min at 65 °C and diluted together with human RNA into a viral transport medium (Liofilchem, Abruzzo, Italy), which is called a positive RM. For negative RM, only the total RNA from Jurkat cells was spiked into the viral transport medium. The positive and negative RMs were dispensed in 1-mL aliquots into 1.8-mL Cryotubes (Thermo Fisher, Waltham, MA) and stored at − 70 °C.
RNA extraction, PCR assays, and diagnostic kits
A QIAamp Viral RNA Mini Kit (QIAGEN) was used to extract 60 μL RNA from 140 μL RM according to the manufacturer’s instructions. RNA copy number concentration was measured by one-step RT-ddPCR and one-step RT-qPCR methods using in-house designed assays and published assays by the WHO (Table 1) [3]. In addition, the following commercial diagnostic kits were used: Kaira® 2019-nCoV Detection Kit (Optolane, Seongnam, Korea), PowerChek™ 2019-nCoV Real-time PCR Kit (Kogene, Seoul, Korea), DiaPlexQ™ Novel Coronavirus (2019-nCoV) Detection Kit (Solgent, Daejeon, Korea), and careGENE™ COVID-19 RT-PCR kit (WELLSBIO, Seoul, Korea).
Table 1.
Target | Probe sequences | Forward/reverse primers |
---|---|---|
RdRp | 5′-[6-FAM]-CCGTAGCTGGTGTCTCTATCTGT-[SFCQ]-3′ | 5′-TGCAAAGAATAGAGCTCGCA-3′ |
5′-CTCCTCTAGTGGCGGCTATT-3′ | ||
RdRp* | 5′-[6-FAM]-CAGGTGGAACCTCATCAGGAGATGC-[SFCQ]-3′ | 5′-GTGARATGGTCATGTGTGGCGG-3′ |
(Germany Charité) | 5′-CARATGTTAAASACACTATTAGCATA-3′ | |
E | 5′-[6-FAM]-TCTTGCTTTCGTGGTATTCTTGCT-[SFCQ]-3′ | 5′-CGGAAGAGACAGGTACGTTAA-3′ |
5′-[HEX]-TCTTGCTTTCGTGGTATTCTTGCT-[SFCQ]-3′ | 5′-GCAGTAAGGATGGCTAGTGT-3′ | |
E* | 5′-[6-FAM]-ACACTAGCCATCCTTACTGCGCTTCG-[SFCQ]-3′ | 5′-ACAGGTACGTTAATAGTTAATAGCGT-3′ |
(Germany Charité) | 5′-ATATTGCAGCAGTACGCACACA-3′ | |
N | 5′-[6-FAM]-CACCAATAGCAGTCCAGATGACC-[SFCQ]-3′ | 5′-ACTCAACATGGCAAGGAAGA-3′ |
5′-GCTCTTCGGTAGTAGCCAAT-3′ | ||
N* | 5′-[6-FAM]-TTGCTGCTGCTTGACAGATT-[SFCQ]-3′ | 5′-GGGGAACTTCTCCTGCTAGAAT-3′ |
(China CDC) | 5′-CAGACATTTTGCTCTCAAGCTG-3′ | |
N* | 5′-[6-FAM]-AGAT/ZEN™/ | 5′-GACCCCAAAATCAGCGAAAT-3′ |
(US CDC N1) | ACCCCGCATTACGTTTGGTGGACC-[IBFQ]-3′ | 5′-TCTGGTTACTGCCAGTTGAATCTG-3′ |
S | 5′-[SFC-V]-TCAGACAAATCGCTCCAGGGCA-[SFCQ]-3′ | 5′-TCTGCTTTACTAATGTCTATGC-3′ |
5′-GCTATAACGCAGCCTGTAAA-3′ |
FAM, fluorescein; HEX, hexachloro-fluorescein. *Assay published by the WHO [15]
Reverse transcription droplet digital PCR (RT-ddPCR)
Experiments were conducted with a QX200 Droplet Digital PCR system (Bio-Rad, Hercules, CA). For all comparisons, 5 μL of supermix, 2 μL of reverse transcriptase, 1 μL of 300 mM dithiothreitol (DTT) from a One-Step RT-ddPCR Advanced Kit for Probes (Bio-Rad) with 5 μL of RNA, 2 μL nuclease-free water (Ambion), and standardized primer and probe concentrations of 1 μM forward and reverse primers and 250 nM probe were used. For each assay, a no template control (NTC) reaction was included. An Automated Droplet Generator (Bio-Rad) was used to generate the droplets. PCR was performed in a Veriti 96-well Thermal Cycler (Applied Biosystems, Waltham, MA). The reactions were conducted under the conditions of 60 min at 42 °C, 10 min at 95 °C, 70 cycles of 30 s at 95 °C, 1 min at 59 °C, and 10 min at 98 °C. For the homogeneity test, a reverse transcription temperature of 46 °C was used. After amplification, the plate was loaded onto a QX200 Droplet Reader (Bio-Rad) and analyzed using QuantaSoft software version 1.7.4 (Bio-Rad). All of the thresholds were set up manually to allow the distinction between positive and negative droplets. Only the reactions with more than 10,000 accepted droplets were used for analysis. The final concentration of 1 mL RM was calculated by the following formula, including RNA extraction from the total copy of the PCR results: copy number concentration of RM = total copies in a reaction / 5 μL × 60 μL / 140 μL × 1000 (copies/1 mL RM). The droplet volume used to calculate copy concentration is 0.85 nL (the manufacturer’s value).
Reverse transcription digital real-time PCR (RT-drPCR)
Total 30 μL reaction mixtures containing 10 μL 3X Dr. PCR Master mix (Optolane), 10 μL nuclease-free water (Ambion), 5 μL assays with 1 uM of each primer and 0.25 μM FAM probe per reaction, and 5 μL of RNA were used. The reaction mixture was loaded into the cartridge and spread evenly within the chip using a LOAA POSTMAN sample loader (Optolane). The PCR reactions were performed with the LOAA (Optolane) [24]. The reactions were conducted under the conditions of 10 min at 50 °C, 15 min at 95 °C, and 40 cycles of 10 s at 95 °C and 10 s at 57 °C. After amplification, results were analyzed using a Dr. PCR Analyzer version 1.3.24 (Optolane) [24].
Reverse transcription quantitative PCR (RT-qPCR)
A One Step PrimeScript™ RT-PCR Kit (Takara) with 5 μL of RNA and standardized primers and probe concentrations of 1 μM forward and reverse primers and 0.25 μM probe were used for all comparisons. For each assay, a NTC reaction was included. The PCR cycler conditions were as follows: reverse transcription for 30 min at 42 °C and an initial denaturation for 5 min at 95 °C, followed by 45 cycles of 10 s at 95 °C and 30 s at 59 °C in a StepOne or StepOnePlus Real-Time PCR System (Applied Biosystems). For the commercial SARS-CoV-2 diagnostic kits, RT-qPCR was performed according to the manufacturer’s instructions.
Homogeneity and stability tests
To determine between-bottle homogeneity, 12 positive tubes of SARS-CoV-2 RM were randomly selected for RT-ddPCR measurements using assays targeting four different genes: RdRp, E, N, and S. Between-bottle homogeneity was estimated by subtracting the method repeatability from the between-bottle relative standard deviation (RSD) for each target. Method repeatability for each target was obtained from the RSD between repeated measurements using the same template within an experiment. Short-term stability for shipping, long-term stability, and freeze and thaw tests were performed using up to three positive tubes per experiment with triplicate repeats. In the case of short-term stability, three sets of RMs stored at − 70 °C were randomly selected and transferred to 4 °C and 22 °C. The number of copies was then measured in samples stored for 0, 1, 2, 5, and 7 days. For long-term stability, one or three samples stored at − 70 °C were randomly selected and thawed, and the number of copies was measured at the point of storage at 2, 3, 5, and 6 months. Results were then compared with the results of the homogeneity test. In the freeze and thaw tests, three sets of RMs stored at − 70 °C were melted at 4 °C and then frozen again at − 70 °C 3 or 5 times. Assessment was performed by the following formula [22]:
Uncertainty and statistical analyses
Each source of uncertainty considered was individually evaluated by conducting type A and type B evaluations separately for each target gene [6]. For type A evaluations, standard deviations from independent experiments were calculated. For type B evaluations, the relative standard uncertainty during manual thresholding was calculated as the RSD of three different threshold settings from more than ten independent measurements. In addition, the standard uncertainty from partition volume variations was calculated assuming a uniform rectangular distribution within the range of droplet volumes reported [18, 21, 25–27]. The RSDs of type A and type B were combined by taking the positive square root of the summed squared RSDs to generate a combined relative standard uncertainty. The combined standard uncertainty for each target was combined to generate the expanded uncertainty with a coverage factor of k = 2.2 (95% level of confidence, degree of freedom = 11). Experiments were repeated at least in triplicate, or otherwise as indicated in the corresponding figures, and were analyzed with Welch’s t-test (two-tailed) using Microsoft Excel 2016 (Microsoft, Redmond, WA). Error bars in the graphical data represent the mean ± standard deviation. Statistical significance was assumed when the p-value was lower than 0.05. The plots were drawn using R (4.1.1) with reshape2 (1.4.4), ggplot2 (3.3.5), ggh4x (0.2.0), and readxl (1.3.1) libraries (R Core Team, Vienna, Austria) [28–32].
Results
Design and preparation of SARS-CoV-2 reference material
In this section, the overall scheme for the production of the KRISS 111–10-507 RM (batch 2) is summarized (Fig. 1a). First, cDNA was synthesized from the SARS-CoV-2 total RNA. The selected regions within the viral genome including ORF1ab, E, N, and S that are commonly used for COVID-19 diagnosis were individually amplified through PCR (Fig. 1b and Table 2). To improve the biosafety of the lentivirus particles carrying SARS-CoV-2 RNA, a replication-defective lentiviral vector with deleted cytomegalovirus (CMV) promoter sequences was used. In addition, four stop codons were inserted between PCR products (see Electronic Supplementary Material Table S1 and Fig. S1). The summed length of the intended targets was approximately 12 kilobases (kb), which might cause a reduction in the lentivirus packaging efficiency [33]. To ensure a high yield, total six individual PCR products (A1–A6) of ORF1ab and E were made into fragment A, and two PCR products (B1, B2) of S and N were made into fragment B (Table 2) using overlap extension PCR [23]. The resulting two fragments (A1–A6 and B1, B2) of approximately 6 kb each were inserted into the lentivirus packaging vector (pCDH) to create pCDH-A and pCDH-B, respectively (see Electronic Supplementary Material Table S1). Some minor variants were found in the RM compared to the reference (GenBank: MW466791) by Sanger sequencing (see Electronic Supplementary Material Fig. S1). The lentiviral particles harvested from pCDH-A and pCDH-B vector transfection are named Lenti-A and Lenti-B, respectively (Table 2).
Table 2.
Gene | Protein | Start (bp) | End (bp) | Location inserted in RM (bp) | Fragments | Virus particle | |
---|---|---|---|---|---|---|---|
ORF1ab | Leader protein | 266 | 805 | 417–1899 | A1 | Lenti-A | |
Nsp2 | 806 | 2719 | |||||
Nsp3 | 2720 | 8554 | 3094–3360 | A2 | |||
RNA-dependent RNA polymerase (RdRp) | 13,442 | 16,236 | 13,291–13,560 | A3 | |||
14,700–15,950 | A4 | ||||||
3′-to-5′ exonuclease | 18,040 | 19,620 | 18,577–19,051 | A5 | |||
S | Spike glycoprotein | 21,563 | 25,384 | 21,363–26,001 | B1 | Lenti-B | |
ORF3a | ORF3a protein | 25,393 | 26,220 | ||||
E | Envelope protein | 26,245 | 26,472 | 25,801–28,200 | A6 | Lenti-A | |
M | Membrane glycoprotein | 26,523 | 27,191 | ||||
ORF8a | ORF8 protein | 27,894 | 28,259 | 27,952–29,873 | B2 | Lenti-B | |
N | Nucleocapsid phosphoprotein | 28,274 | 29,533 | ||||
ORF10 | ORF10 protein | 29,558 | 29,674 |
For the positive material, Lenti-A and Lenti-B were filtered after heat inactivation. A small fraction of the concentrated positive materials were used to measure the original concentration. Based on this, we diluted the positive materials down to the value of about 2 × 105 copies/mL for each target in universal transport media supplemented with Jurkat total RNA (2.5 μg/mL) and carrier RNA (2.5 μg/mL). In the case of the negative material, a similar process was applied except for the addition of Lenti-A and Lenti-B particles (Fig. 1a). The KRISS 111–10-507 RM was assembled as a set of positive and negative materials of 1 mL per vial and immediately stored at − 70 °C.
Determination of the SARS-CoV-2 RM measurement method
To determine the method to measure the copy number concentration of the RM, the RNA extracted from either negative or positive material was used as a template for one-step RT-ddPCR. The analytical performance of in-house designed primer/probe sets (RdRp, E, N, and S) was compared with the widely used WHO assays (Table 1 and Fig. 2a). A series of experiments confirmed that the in-house performance of the assays was acceptable. First, these assays yielded no false-positive results when using RNA from the negative RM or in NTC reactions, showing their high specificity (Fig. 2a). Second, for E and N, the resulting copy number concentrations were comparable to those from the WHO assays (Fig. 2a, c). In the case of RdRp, the in-house assay showed a significantly higher copy number than the Germany Charité RdRp under the same experimental condition (Fig. 2a, c), suggesting a higher analytical sensitivity. The relatively low analytical sensitivity of the Germany Charité RdRp has previously been noticed by RT-qPCR [17, 34]. In addition, the copy numbers using two duplex combinations (RdRp-FAM and S-VIC; N-FAM and E-HEX) were comparable to the respective singleplex results (Fig. 2a). Based on the combined results, the duplex in-house assays using RT-ddPCR were confirmed to be competent for the designed SARS-CoV-2 RM measurements.
To further test the suitability of the RT-ddPCR method, cross-validation was carried out using reverse transcription digital real-time PCR (RT-drPCR), a distinct type of digital PCR (Fig. 2b). The copy number values from the N assays using RT-ddPCR and RT-drPCR were comparable, showing statistically non-significant differences (Fig. 2c). On the other hand, a slightly higher copy number of the E gene was obtained using RT-drPCR than using RT-ddPCR (Fig. 2c). In previous and current studies, it is consistently shown that RNA measurement values are similar between RT-drPCR and RT-ddPCR, with some limited exceptions in certain assays [24]. In summary, as the RT-ddPCR method using duplex in-house assays was confirmed to be valid for NAT-based RNA measurements, it was chosen to measure the produced SARS-CoV-2 RM.
Homogeneity of the SARS-CoV-2 RM
To test between-bottle homogeneity and determine the reference values of the developed RM, RT-ddPCR was performed using extracted RNA from 12 randomly chosen positive RM vials with three technical replicates. This process ensures that the reference values are valid with any positive vial regardless of the filling order and that the measurement uncertainty covers potential between-bottle variations. The copy number concentrations from the 12 vials were averaged to assign the reference values in 1 mL of positive material. The values for RdRp, E, N, and S genes were approximately 1.7 × 105, 1.5 × 105, 2.0 × 105, and 2.2 × 105 copies per mL, respectively (Fig. 3a, b). It is worth noting that these four targets are encapsulated in two lentiviral particles (Lenti-A: RdRp and E; Lenti-B: N and S). Consequently, the reference values of the two targets in the same particle are relatively similar. However, these values are not identical probably due to assay-dependent variations [16, 17]. The calculated between-bottle homogeneity of each target was about 4.80–8.23% (Fig. 3c).
Stability of SARS-CoV-2 RM
A short-term stability study was conducted to test the stability of the reference values over typical transport periods. After positive materials were stored at 4 °C and 22 °C for 0, 1, 2, 5, and 7 days, the number of copies of the RdRp, E, N, and S genes was measured. When stored at 4 °C, the RM was found to be stable for 7 days (Fig. 4a). However, the stability of the RM at 22 °C was not as well maintained; storage at 22 °C for 2 days was acceptable, but the copy number values dropped after 5 days, failing the stability assessment tests for all four targets (Fig. 4a). Although the values at 7 days seemed to sufficiently recover to pass the stability tests, it is evident that the overall values had been compromised after storage at 22 °C for 7 days.
Long-term stability tests were also performed to evaluate the RM stability during storage in a specified condition (− 70 °C). Even after 6 months at − 70 °C, the copy numbers did not change significantly (Fig. 4b), passing the stability assessment tests. In addition, as viral particles and RNA are known to be unstable during freeze and thaw cycles, stability assessments during freeze and thaw cycles were conducted. Results confirmed that the RM stability was not affected by up to 5 freeze and thaw cycles (Fig. 4c). These results suggest that the packaged RNA RMs are highly stable at the storage condition and also adequate for transport.
Effect of variables on measurement uncertainty
To estimate the measurement uncertainty, a number of uncertainty sources were considered. Essentially, the reference values are measured with extracted RNA from twelve different RM vials. This practice was thought to contain multiple uncertainty sources: between- and within-bottle homogeneity, variations in RNA extraction efficiency, variations in reverse transcription efficiency, day-to-day (plate-to-plate) and person-to-person variations, and repeatability of PCR runs.
For type B evaluation, two major components that affect the measurement values during data analysis were considered: uncertainty in assigning the partition volume for the specific dPCR instrument, and uncertainty in manually setting the threshold, especially when there was a group of droplets scattered between the positive and negative droplet population. As the partition volume directly affects the copy number values during dPCR data analysis, the probability distribution was assumed to be rectangular from minimum and maximum reported values [18, 21, 25–27]. This relative standard uncertainty for the partition volume is likely overestimated because these measurements were performed for different PCR supermixes and droplet generators. The relative contributions of the aforementioned sources for each target are compared (Fig. 5). Ultimately, the combined standard uncertainty for each target was expanded with a coverage factor of k = 2.20 (95% level of confidence, degree of freedom = 11). The resulting expanded uncertainty and the reference values are summarized in Table 3. By dividing the expanded uncertainty by the reference value for each target, the relative expanded uncertainties are calculated to be 24.7 to 29.4%.
Table 3.
Target | Reference value (copies/mL) | Homogeneity (%) | Expanded uncertainty (copies/mL) | k (95% level of confidence) |
---|---|---|---|---|
RdRp | 1.7 × 105 | 5.4 | 4.1 × 104 | 2.2 |
E | 1.5 × 105 | 6.3 | 4.3 × 104 | 2.2 |
N | 2.0 × 105 | 8.2 | 5.9 × 104 | 2.2 |
S | 2.2 × 105 | 4.8 | 5.5 × 104 | 2.2 |
Measurement of the SARS-CoV-2 RM by RT-qPCR
To provide additional informative values of the KRISS 111–10-507 RM, it was tested via RT-qPCR using in-house and WHO assays as well as commercial diagnostic kits. Similarly as with RT-ddPCR (Fig. 2a), the RT-qPCR results from singleplex and duplex in-house assays were comparable. The Cq values of the four genes (RdRp, E, N, and S) ranged from 28 to 30 (Fig. 6). As expected, the in-house assays with the RT-qPCR system performed comparably with Germany Charité RdRp, Germany Charité E, China CDC N, and USA CDC N1 under the same thermocycling and enzyme buffer conditions. Unlike the in-house assay tests, the commercial diagnostic kits, used as instructed, had different assays as well as different enzyme buffer and thermocycling conditions, but their Cq values were comparable (Fig. 6). It is worth noting that the Cq value by Germany Charité RdRp was higher than those by the in-house assay and commercial diagnostic kits, consistent with the previous RT-ddPCR results (Fig. 2a). As a result, the KRISS 111–10-507 RM was found to be compatible with multiple commercial diagnostic kits and WHO assays as well as in-house assays using the RT-qPCR method.
Discussions
The KRISS 111–10-507 positive RM includes selected regions of SARS-CoV-2 in a lentiviral packaging system. As a reference material, the lentiviral particle has a number of advantages over alternative forms. First, the lentiviral particle form closely mimics real samples from nasal swabs, providing a reference throughout the entire process from RNA extraction to diagnosis. Second, the encapsulated RNAs in lentiviral particles are uniform in length and sequence. The inserted full-length RNAs are transcribed up to the long terminal repeat (LTR) [35, 36]. In contrast, in vitro transcribed RNA, another form of RNA RM, may be heterogeneous in length due to incomplete transcription [37]. In addition, the lentiviral membrane and capsid structure may protect RNA from RNase or hydrolysis [38, 39]. Despite such advantages though, lentiviral particles are considered less safe than naked RNAs or DNA [40, 41]. To circumvent these concerns and improve the biosafety of lentiviral particles carrying SARS-CoV-2 RNA, in addition to heat inactivation of the virus [42], multiple steps were included during the RM production process in this work. A replication-defective lentiviral vector was used, and the CMV promoter in the vector was deleted. Additionally, four extra stop codons in different frames were inserted between PCR products to prevent gene expression. With such measures, the developed lentiviral SARS-CoV-2 RM is a safe and effective material to calibrate and correct bias throughout the entire process of SARS-CoV-2 RNA quantification from RNA extraction to thermocycling, similar to the inactivated SARS-CoV-2 virus RM that does not completely eliminate the possibility of infection.
The lentiviral SARS-CoV-2 RM provides a specific copy number concentration of the representative targets: 1.5 × 105 to 2 × 105 copies/mL. These RM values in absolute copy number confer a concrete reference across diverse molecular testing applications including RT-qPCR, next-generation sequencing, loop-mediated isothermal amplification, and CRISPR nuclease-based detection [4, 5]. Analyses of the Emergency Use Authorization (EUA) SARS-CoV-2 virus tests in the USA show an unexpectedly broad limit of detection (LoD) of approximately 0.01 to 100 copies/μL [5]. This brings up a critical issue: for every tenfold increase in LoD, the false-negative rate is expected to increase by 13% [42]. This can be interpreted to derive from the absence of reliable RMs that are internationally available, currently unsatisfactory method validation, or simple inter-laboratory and inter-personnel variations. In this light, the KRISS 111–10-507 RM can be broadly applied to increase reliability in molecular testing, such as in comparisons of diverse NAT methods or EUA diagnostic kits, the determination of a valid LoD for a given method, the validation of a new analytical process, and evaluations of an instrument or personnel. In particular, the reference values in copy number units can serve as a firm standard to compare different RT-qPCR based methods, represented in Cq values.
The major measurement uncertainty of the developed RM stems from pre-analytical processes such as RNA extraction and the handling of viscous solutions. After combining the considered uncertainty values, the expanded uncertainty is summarized in Table 3. These values were obtained using RNA extracted from a subsample of the RM with a specific commercial viral RNA extraction kit; it has been shown that RNA extraction efficiency varies widely depending on the method applied and the personnel [43, 44]. Another source of variation in the reference values is the viscosity of the viral transport medium (VTM), which is used as the matrix of this RM for its protective and stabilizing properties. However, its high viscosity can interfere with an even dispersion of the virus and reduce the precision of pipetting during subsampling. It is supposed that the VTM matrix contributed to the lower homogeneity of the lentivirus-based SARS-CoV-2 RM compared to the in vitro transcribed RNA RM [12].
The KRISS 111–10-507 RM as a set of positive and negative samples is widely applicable. The positive RM can substitute for real patient samples of confirmed cases, especially in the beginning of a pandemic or if the biosafety level of the related facilities is not suitable. Together with the positive samples, the negative RM containing only human RNA can be used as a practice material to improve proficiency or as a test material in external and internal quality assessments in testing laboratories. In summary, the KRISS 111–10-507 RM provides valuable measurement standards for SARS-CoV-2 molecular testing.
Supplementary Information
Below is the link to the electronic supplementary material.
Biographies
Sang-Soo Lee
is a postdoctoral researcher in the Bio-Metrology Group at Korea Research Institute of Standards and Science (KRISS). He has been working on developing the measurement standards in molecular diagnosis and understanding the molecular mechanisms of extracellular vesicles.
Seil Kim
is a member of the microbiological analysis team at the Korea Research Institute of Standard and Science, and he is also a member of the national committee for risk assessment of industrial living modified organisms. He has been working for several years on the genome sequencing and measurement of virus based on nucleic acid amplification.
Hee Min Yoo
is Team Leader of the Microbiological analysis team at the Korea Research Institute of Standards and Science (KRISS), and he is also Associate Professor of the Department of Bio-Analysis Science of the University of Science & Technology (UST). He is developing reference materials and measurement technology in the field of viral diagnosis.
Da-Hye Lee
is Senior Research Scientist in the Bio-metrology group at Korea Research Institute of Standards and Science (KRISS), the National Metrology Institute in the Republic of Korea. She has been working for several years on the measurement of biological materials based and the development of biologic standard materials.
Young-Kyung Bae
is Leader of the Bio-metrology group at Korea Research Institute of Standards and Science (KRISS). Trained as molecular geneticist, she has been working on establishing the measurement standards in biological substances especially nucleic acids. She is also Adjunct Professor in GRAST at Chungnam National University.
Author contribution
Sang-Soo Lee: conceptualization, methodology, formal analysis, investigation, data curation, writing—original draft, writing—review and editing, visualization, funding acquisition. Seil Kim: conceptualization, formal analysis, resources, visualization, funding acquisition. Hee Min Yoo: conceptualization, resources, funding acquisition. Da-Hye Lee: conceptualization, methodology, validation, formal analysis, investigation, resources, writing—review and editing, project administration, funding acquisition. Young-Kyung Bae: conceptualization, methodology, formal analysis, resources, data curation, writing—original draft, writing—review and editing, supervision, project administration, funding acquisition.
Funding
This study is supported by a National Research Council of Science and Technology grant given to the Korea Research Institute of Standards and Science (GP2020-0003/20011003) and a Young Scientists Fellowship to Sang-Soo Lee (IP2021-0019/21601001).
Declarations
Conflict of interest
The authors declare no competing interests.
Footnotes
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Contributor Information
Da-Hye Lee, Email: dahye.lee04@kriss.re.kr.
Young-Kyung Bae, Email: ybae@kriss.re.kr.
References
- 1.Phelan AL, Katz R, Gostin LO. The novel coronavirus originating in Wuhan, China: challenges for global health governance. JAMA. 2020;323(8):709–710. doi: 10.1001/jama.2020.1097. [DOI] [PubMed] [Google Scholar]
- 2.Wang D, Hu B, Hu C, Zhu F, Liu X, Zhang J, Wang B, Xiang H, Cheng Z, Xiong Y, Zhao Y, Li Y, Wang X, Peng Z. Clinical characteristics of 138 hospitalized patients with 2019 novel coronavirus-infected pneumonia in Wuhan, China. JAMA. 2020;323(11):1061–1069. doi: 10.1001/jama.2020.1585. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.World Health Organisation. Diagnostic testing for SARS-CoV-2. 2020.
- 4.U.S. Food & Drug administration, SARS-CoV-2 Reference Panel Comparative Data. 2020.
- 5.MacKay MJ, Hooker AC, Afshinnekoo E, Salit M, Kelly J, Feldstein JV, Haft N, Schenkel D, Nambi S, Cai Y, Zhang F, Church G, Dai J, Wang CL, Levy S, Huber J, Ji HP, Kriegel A, Wyllie AL, Mason CE. The COVID-19 XPRIZE and the need for scalable, fast, and widespread testing. Nat Biotechnol. 2020;38(9):1021–1024. doi: 10.1038/s41587-020-0655-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Joint Committee for Guides in Metrology: International Vocabulary of Metrology—Basic and GeneralConcepts and Associated Terms (VIM 3rd edition): Sèvres, France; 2012.
- 7.Madej RM, Davis J, Holden MJ, Kwang S, Labourier E, Schneider GJ. International standards and reference materials for quantitative molecular infectious disease testing. J Mol Diagn. 2010;12(2):133–143. doi: 10.2353/jmoldx.2010.090067. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.SARS-Cov-2 Synthetic RNA Fragments Guidance Sheet for Research Grade Test Material ID 10169 NIST. https://www.nist. gov/image/sars-cov-2-research-grade-test-material. Accessed 1 Sept 2021.
- 9.NIM to provide urgently needed 2019-nCoV RNA certified reference materials | National Institute of Metrology of China | NIM. https://en.nim.ac.cn/node/657. Accessed 12 Jun 2020.
- 10.AccuPlex™ SARS-CoV-2 Molecular Controls Kit - Full Genome 0505–0159 Seracare. https://www.seracare.com/AccuPlex-SARSCoV2-Molecular-Controls-Kit--Full-Genome-0505-0159. Accessed 1 July 2020.
- 11.Akyurek S, Demirci SNS, Bayrak Z, Isleyen A, Akgoz M. The production and characterization of SARS-CoV-2 RNA reference material. Anal Bioanal Chem. 2021;413(13):3411–3419. doi: 10.1007/s00216-021-03284-w. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.SARS-CoV-2 RNA reference material 111–10–506 Korea Institute of Standards and Science (KRISS). https://eshop.kriss.re.kr/crm/crmList.do. Accessed 1 July 2020.
- 13.National Institute for Biological Standards and Control: CE Marked Material: 2019 novel coronavirus (SARS-CoV-2) Working Reagent for Nucleic Acid Amplification Testing (NAT) NIBSC code: 20/110-XXX: London, United Kingdom; 2021.
- 14.AccuPlex™ SARS-CoV-2 Reference Material Kit 0505–0126 Seracare. https://www.seracare.com/AccuPlex-SARSCoV2-Reference-Material-Kit-0505-0126. Accessed 1 July 2020.
- 15.National Institute for Biological Standards and Control: WHO International Standard: First WHO International Standard for SARS-CoV-2 RNA NIBSC code: 20/146: London, United Kingdom; 2020.
- 16.Park C, Lee J, Hassan Zu, Ku KB, Kim S-J, Kim HG, Park EC, Park G-S, Park D, Baek S-H, Park D, Lee J, Jeon S, Kim S, Lee C-S, Yoo HM, Kim S. Comparison of digital PCR and quantitative PCR with various SARS-CoV-2 primer-probe sets. J Microbiol Biotechnol. 2021;31(3):358–367. doi: 10.4014/jmb.2009.09006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Jung Y, Park G-S, Moon JH, Ku K, Beak S-H, Lee C-S, Kim S, Park EC, Park D, Lee J-H, Byeon CW, Lee JJ, Maeng J-S, Kim S-J, Kim SI, Kim B-T, Lee MJ, Kim HG. Comparative analysis of primer–probe sets for RT-qPCR of COVID-19 causative virus (SARS-CoV-2) ACS Infect Dis. 2020;6(9):2513–2523. doi: 10.1021/acsinfecdis.0c00464. [DOI] [PubMed] [Google Scholar]
- 18.Dong L, Meng Y, Sui Z, Wang J, Wu L, Fu B. Comparison of four digital PCR platforms for accurate quantification of DNA copy number of a certified plasmid DNA reference material. Sci Rep. 2015;5:13174. doi: 10.1038/srep13174. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Sanders R, Mason DJ, Foy CA, Huggett JF. Evaluation of digital PCR for absolute RNA quantification. PLoS ONE. 2013;8(9):e75296. doi: 10.1371/journal.pone.0075296. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Devonshire AS, Sanders R, Whale AS, Nixon GJ, Cowen S, Ellison SL, Parkes H, Pine PS, Salit M, McDaniel J, Munro S, Lund S, Matsukura S, Sekiguchi Y, Kawaharasaki M, Granjeiro JM, Falagan-Lotsch P, Saraiva AM, Couto P, Yang I, Kwon H, Park SR, Demsar T, Zel J, Blejec A, Milavec M, Dong L, Zhang L, Sui Z, Wang J, Viroonudomphol D, Prawettongsopon C, Partis L, Baoutina A, Emslie K, Takatsu A, Akyurek S, Akgoz M, Vonsky M, Konopelko LA, Cundapi EM, Urquiza MP, Huggett JF, Foy CA. An international comparability study on quantification of mRNA gene expression ratios: CCQM-P103.1. Biomol Detect Quantif. 2016;8:15–28. doi: 10.1016/j.bdq.2016.05.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Corbisier P, Pinheiro L, Mazoua S, Kortekaas AM, Chung PY, Gerganova T, Roebben G, Emons H, Emslie K. DNA copy number concentration measured by digital and droplet digital quantitative PCR using certified reference materials. Anal Bioanal Chem. 2015;407(7):1831–1840. doi: 10.1007/s00216-015-8458-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.ISO Guide 35:2017 - Reference materials -Guidance for characterization and assessment of homogeneity and stability. https://www. iso.org/standard/60281.html. Accessed 1 Dec 2020.
- 23.Kadkhodaei S, Memari HR, Abbasiliasi S, Rezaei MA, Movahedi A, Shun TJ, Ariff AB. Multiple overlap extension PCR (MOE-PCR): an effective technical shortcut to high throughput synthetic biology. RSC Adv. 2016;6(71):66682–66694. doi: 10.1039/C6RA13172G. [DOI] [Google Scholar]
- 24.Lee SS, Park JH, Bae YK. Comparison of two digital PCR methods for EGFR DNA and SARS-CoV-2 RNA quantification. Clin Chim Acta. 2021;521:9–18. doi: 10.1016/j.cca.2021.06.016. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Pinheiro LB, Coleman VA, Hindson CM, Herrmann J, Hindson BJ, Bhat S, Emslie KR. Evaluation of a droplet digital polymerase chain reaction format for DNA copy number quantification. Anal Chem. 2012;84(2):1003–1011. doi: 10.1021/ac202578x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Kosir AB, Divieto C, Pavsic J, Pavarelli S, Dobnik D, Dreo T, Bellotti R, Sassi MP, Zel J. Droplet volume variability as a critical factor for accuracy of absolute quantification using droplet digital PCR. Anal Bioanal Chem. 2017;409(28):6689–6697. doi: 10.1007/s00216-017-0625-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Emslie KR, JL HM, Griffiths K, Forbes-Smith M, Pinheiro LB, Burke DG. Droplet volume variability and impact on digital PCR copy number concentration measurements. Anal Chem. 2019;91(6):4124–4131. doi: 10.1021/acs.analchem.8b05828. [DOI] [PubMed] [Google Scholar]
- 28.R Core Team (2020). http://www.r-project.org/index.html. Accessed 1 Sept 2021.
- 29.Bryan HWaJ (2019) readxl: Read Excel Files. R package version 1.3.1. . https://CRAN.R-project.org/package=readxl. Accessed 1 Sept 2021.
- 30.Wickham H. Reshaping Data with the reshape Package. J Stat Softw. 2007;21(12):1–20. doi: 10.18637/jss.v021.i12. [DOI] [Google Scholar]
- 31.Wickham H (2009) ggplot2 : Elegant Graphics for Data Analysis. Use R! Springer-Verlag New York. 10.1007/978-0-387-98141-3
- 32.Brand Tvd (2021) ggh4x: Hacks for ‘ggplot2’. R package version 0.2.0. . https://CRAN.R-project.org/package=ggh4x. Accessed 1 Sept 2021.
- 33.Canté-Barrett K, Mendes RD, Smits WK, van Helsdingen-van Wijk YM, Pieters R, Meijerink JPP. Lentiviral gene transfer into human and murine hematopoietic stem cells: size matters. BMC Res Notes. 2016;9(1):312. doi: 10.1186/s13104-016-2118-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Vogels CBF, Brito AF, Wyllie AL, Fauver JR, Ott IM, Kalinich CC, Petrone ME, Casanovas-Massana A, Catherine Muenker M, Moore AJ, Klein J, Lu P, Lu-Culligan A, Jiang X, Kim DJ, Kudo E, Mao T, Moriyama M, Oh JE, Park A, Silva J, Song E, Takahashi T, Taura M, Tokuyama M, Venkataraman A, Weizman OE, Wong P, Yang Y, Cheemarla NR, White EB, Lapidus S, Earnest R, Geng B, Vijayakumar P, Odio C, Fournier J, Bermejo S, Farhadian S, Dela Cruz CS, Iwasaki A, Ko AI, Landry ML, Foxman EF, Grubaugh ND. Analytical sensitivity and efficiency comparisons of SARS-CoV-2 RT-qPCR primer-probe sets. Nat Microbiol. 2020;5(10):1299–1305. doi: 10.1038/s41564-020-0761-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Pachulska-Wieczorek K, Le Grice SFJ, Purzycka KJ. Determinants of genomic RNA encapsidation in the Saccharomyces cerevisiae long terminal repeat retrotransposons Ty1 and Ty3. Viruses. 2016;8(7):193. doi: 10.3390/v8070193. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Dull T, Zufferey R, Kelly M, Mandel RJ, Nguyen M, Trono D, Naldini L. A third-generation lentivirus vector with a conditional packaging system. J Virol. 1998;72(11):8463–8471. doi: 10.1128/JVI.72.11.8463-8471.1998. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Kirov N, Tsaneva I, Einbinder E, Tsanev R. In vitro transcription through nucleosomes by T7 RNA polymerase. EMBO J. 1992;11(5):1941–1947. doi: 10.1002/j.1460-2075.1992.tb05247.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Fang P-Y, Bowman JC, Gómez Ramos Lizzette M, Hsiao C, Williams LD. RNA: packaged and protected by VLPs. RSC Adv. 2018;8(38):21399–21406. doi: 10.1039/c8ra02084a. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Shehrazade J, Rob H, Omar PSF, Alexander HA, Marco EF, Armando HG, Roland B, Renko V. Virus-like particles of mRNA with artificial minimal coat proteins: particle formation, stability, and transfection efficiency. Nucleic Acid Therapeutics. 2017; 27 (3):159–167. 10.1089/nat.2016.0660 [DOI] [PubMed]
- 40.Vink CA, Counsell JR, Perocheau DP, Karda R, Buckley SMK, Brugman MH, Galla M, Schambach A, McKay TR, Waddington SN, Howe SJ. Eliminating HIV-1 packaging sequences from lentiviral vector proviruses enhances safety and expedites gene transfer for gene therapy. Mol Ther. 2017;25(8):1790–1804. doi: 10.1016/j.ymthe.2017.04.028. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Bukovsky AA, Song J-P, Naldini L. Interaction of human immunodeficiency virus-derived vectors with wild-type virus in transduced cells. J Virol. 1999;73(8):7087–7092. doi: 10.1128/JVI.73.8.7087-7092.1999. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Arnaout R, Lee RA, Lee GR, Callahan C, Yen CF, Smith KP, Arora R, Kirby JE (2020) SARS-CoV2 testing: the limit of detection matters. bioRxiv. 10.1101/2020.06.02.131144
- 43.Ambrosi C, Prezioso C, Checconi P, Scribano D, Sarshar M, Capannari M, Tomino C, Fini M, Garaci E, Palamara AT, De Chiara G, Limongi D. SARS-CoV-2: comparative analysis of different RNA extraction methods. J Virol Methods. 2021;287:114008. doi: 10.1016/j.jviromet.2020.114008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Sathiamoorthy S, Malott RJ, Gisonni-Lex L, Ng SHS. Selection and evaluation of an efficient method for the recovery of viral nucleic acids from complex biologicals. npj Vaccines. 2018;3(1):31. doi: 10.1038/s41541-018-0067-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.