Converting to an international unit system improves harmonization of results for SARS-CoV-2 quantification: Results from multiple external quality assessments

Abstract

Background

The detection of SARS-CoV-2 vRNA in clinical samples has relied almost exclusively on RT-qPCR as the gold standard test. Published results from various external quality assessments (“ring trials”) worldwide have shown that there is still a large variability in results reported for the same samples. As reference standards of SARS-CoV-2 RNA are available, we tested whether using standard curves to convert Ct values into copies/mL (cp/mL) improved harmonization.

Methods

Nine laboratories using 23 test systems (15 of which were unique) prepared standard dilution curves to convert C_t values of 13 SARS-CoV-2 positive samples to cp/mL (hereafter IU/mL). The samples were provided in three rounds of a virus genome detection external quality assessment (EQA) scheme. We tested the precision and accuracy of results reported in IU/mL, and attempted to identify the sources of variability.

Results

Reporting results as IU/mL improved the precision of the estimated concentrations of all samples compared to reporting Ct values, although some inaccuracy remained. Variance analysis showed that nearly all variability in data was explained by individual test systems within individual laboratories. When controlling for this effect, there was no significant difference between all other factors tested (test systems, EQA rounds, sample material).

Conclusions

Converting results to copies/mL improved precision across laboratory test systems. However, it seems the results are still very specific to test systems within laboratories. Further efforts could be made to improve accuracy and achieve full harmonization across diagnostic laboratories.

1. Introduction

The rapid establishment and dissemination of reliable RT-qPCR assays for the diagnosis of SARS-CoV-2 infections had an immeasurable benefit in the early pandemic phase. This diagnostic assay provides the gold standard assessment of whether a person is vRNA-negative or vRNA-positive, and its success is surely measured by the large number of commercially available test kits and test systems. As a highly sensitive test, the C_t value can also be used to estimate the viral concentration in a sample by converting to copy number based on a standard dilution curve. Given the absence of a proper standard at the beginning of the SARS-CoV-2 pandemic, it was understandable that the C_t value was reported and used as an approximate indicator for disease status [1,2]. However, C_t values have become firmly anchored in SARS-CoV-2-associated laboratory diagnostics.

Relatively early in the pandemic, several scientific organizations discouraged using C_t values in management of patients with COVID-19, and the dangers of using C_t values as a quantitative measure for SARS-CoV-2 RNA burden have been reported [3], [4], [5]. Among them, the most important are that C_t values are assay-specific, highly variable, and are mistaken for quantitative and harmonized values [6], [7], [8], [9], [10]. It has been well documented that there exists large variability in C_t values reported by different test systems from the same samples [1,2,[11], [12], [13]].

Thus, to date, it is advisable to compare results only from assays of the same type (from same test kit and/or for the same target), if not only from the same individual instrument [8,12]. This complicates recommendations for patients based on specific viral load cut-offs to make clinical decisions and determine infectivity [5,6,9,14,15], given the variability not only between test systems and between laboratories, but also given the demonstrated variability between sample type and sampling method [15], [16], [17], [18], [19]. There been only tentative efforts to harmonize SARS-CoV-2 RT-qPCR assays and the reporting of quantitative results [12,20].

The goal of the study was to test whether the use of standard curves to convert C_t values to copies/mL (cp/mL, hereafter IU/mL) improved harmonization across laboratories. We used clinical samples and reference standards over a range of concentrations distributed to blinded laboratories in three rounds of a dedicated external quality assessment (EQA) scheme. In such schemes, samples meeting requirements for homogeneity and stability are used and the same samples are analyzed by all participant laboratories almost simultaneously with their routinely used test systems [21]. This makes EQA schemes particularly useful for evaluating and comparing the performance of a wide range of assays, in addition to assessing the performance of individual laboratories and test systems.

2. Materials and methods

2.1. Test material

The data for this study came from three recent rounds of the SARS-CoV-2 virus genome detection EQA scheme: November 2021 (three positive and one negative samples); February 2022 (six positive and one negative samples); and May 2022 (four positive and one negative samples). Each round had one positive clinical sample, one negative sample, and the rest were dilutions of a reference standard based on the WHO reference sequence (SeraCare AccuPlex SARS-CoV-2 Molecular Controls Kit Full Genome). The three positive clinical samples were determined to be a Delta variant (lineage B.1.617.2) and two Omicron variants (lineages BA.2 and BA.5) by whole genome sequencing. The reference standard was prepared from two lots from the manufacturer as follows: 1000 copies/mL supplied five times in two lots; 5000 copies per mL supplied twice from one lot; and 50, 100 and 500 copies/mL supplied once each from the same lot (Table 4).

2.2. Data collection and organization of the EQAs

Preparation and characterization of samples followed a procedure described earlier [11]. Namely, sample stability was verified by replicating shipping conditions (shipping at room temperature and storage for up to one week at 4°C). Characterization also included determination of copy number by digital PCR (dPCR). Details of sample preparation for the February 2022 EQA round were included in another study [22].

2.3. Dilution curve and conversion of Ct to IU/mL

Participant laboratories were asked to establish a standard dilution curve with the first WHO International Standard for SARS-CoV-2 RNA for each of their test systems [1,2]. As five laboratories reported only single values for each dilution (Lab3, Lab5, Lab7, Lab8, Lab9), we therefore used the average of duplicates (Lab1, Lab2, Lab4, Lab6) reported by some laboratories to calculate the conversion equation (Table 1 ).

Table 1.

Reaction efficiency and conversion of Ct-value to IU/mL based on a standard dilution series of a SARS-CoV-2 reference

			All values			Optimized
IDa	Deviceb	Reagentb	Slope	Y-int	Efficiency	Slope	Y-int	Efficiency
Lab1	Abbott Alinity (1)	Alinity m SARS-CoV-2 AMP Kit (1)	-3.71	43.4	86%	-3.69	43.5	87%
Lab2	Abbott Alinity (2)	Alinity m SARS-CoV-2 AMP Kit (2)	-3.79	44.5	83%	-3.84	44.9	82%
Lab3	Bio Molecular Systems, Magnetic Induction Cycler	New England Biolabs Luna Probe One-Step RT-qPCR Kit (No ROX)	-3.27	45.7	102%	-3.34	45.9	99%
Lab4	BioRad CFX96 (1)	Anchor SARS-CoV-2 PCR Kit 2.0	-3.65	49.0	88%	-3.65	49.0	88%
Lab4	BioRad CFX96 (2)	Shimadzu 2019 Novel Coronavirus Detection Kit	-3.39	45.2	97%	-3.39	45.2	97%
Lab5	Cobas 6800 (1)	cobas SARS-CoV-2 Test (1)	-2.85	40.9	124%	-3.16	42.6	107%
Lab6	Cobas 6800 (2)	cobas SARS-CoV-2 Test (2)	-3.07	42.0	112%	-3.21	42.8	105%
Lab7	Cobas 6800 (3)	cobas SARS-CoV-2 Test (3)	-3.14	41.9	108%	-3.40	43.1	97%
Lab2	Cobas 6800 (4)	cobas SARS-CoV-2 Test (4)	-3.18	43.2	106%	-3.36	44.2	98%
Lab6	cobas Liat	cobas Liat Assay Tube	-3.50	42.6	93%	-3.50	42.6	93%
Lab8	GeneXpert (1)	Xpert Xpress SARS-CoV-2 (1)	-3.45	45.0	95%	-3.55	45.6	91%
Lab5	GeneXpert (2)	Xpert Xpress SARS-CoV-2 (2)	-3.24	43.1	104%	-3.24	43.1	104%
Lab2	GeneXpert (3)	Xpert Xpress SARS-CoV-2 (3)	-3.45	46.2	95%	-3.82	48.1	83%
Lab8	Liaison MDX (1)	Simplexa COVID-19 Detection Kit (1)	-3.41	42.0	97%	-3.84	44.3	82%
Lab7	Liaison MDX (2)	Simplexa COVID-19 Detection Kit (2)	-3.28	41.5	102%	-4.22	45.9	73%
Lab9	Liaison MDX (3)	Simplexa COVID-19 Detection Kit (3)	-3.25	40.7	103%	-3.25	40.7	103%
Lab2	Light Cycler	TIB MOLBIOL SARS-Cov-2 N gene	-3.06	42.3	112%	-3.33	43.6	100%
Lab5	LightCycler 480 II	RIDA GENE SARS-CoV-2	-3.72	47.2	86%	-3.72	47.2	86%
Lab4	LightCycler 480	TIB MOLBIOL Sarbeco E gene	-3.31	46.0	101%	-3.31	46.0	101%
Lab9	LineGene 9600 (1)	artus SARS-CoV-2 Prep&Amp UM Kit	-3.42	44.3	96%	-3.42	44.3	96%
Lab9	LineGene 9600 (2)	PerkinElmer SARS-CoV-2 Real-time RT-PCR Assay	-3.31	44.5	100%	-3.31	44.5	100%
Lab2	NeuMoDx 288	NeuMoDx SARS-CoV-2 Assay	-3.08	41.1	111%	-3.28	42.7	102%
Lab8	Panther Fusion	Aptima SARS-CoV-2-Assay	-2.86	42.9	124%	-3.17	44.5	107%

^anine participant laboratories were given unique anonymized identifiers, and some laboratories reported multiple test systems (e.g., Lab2 used five test systems)

^beach row contains an individual test system where numbers in brackets indicate the same Device/Reagent was used multiple times.

2.4. Statistical analysis

For each standard dilution curve (i.e., each within-laboratory test system), the efficiency was estimated by the following equation: $Efficiency=100\%\times \left(-1+{10}^{-\frac{1}{slope}}\right)$. The equation $c={10}^{\frac{Ct-a}{b}}$ was used for conversion of C_t values into log₁₀ IU/ml, where c is the concentration, a is the intercept, and b is the slope.

Precision was analyzed by subtracting the sample mean value from all measurements (“mean centered” differences). Accuracy was analyzed by subtracting the target value from all measurements (“target centered” differences). For values converted to IU/mL (log₁₀ copy number / mL), the concentration measured by dPCR was used as the target value; for C_t values, the mean of the validated sample material from the reference laboratory was used as a target C_t value. Validation of the prepared reference samples showed a 1:1 linear correspondence between the target dilutions of the standard and results measured by dPCR, however the actual dilutions contained approximately 10^{−0 . 26} (=1.8-fold) fewer copies of viral RNA than expected (Figure S1). In order to compare between the two units (difference in C_t value and difference in log₁₀ IU/mL) we transformed difference in C_t values (i.e., difference from the mean or target) into the log₁₀ scale, assuming perfect efficiency of a RT-qPCR reaction.

For precision, variances of mean-centered differences were compared with the variance test. To determine accuracy, a one-way repeated measures ANOVA of target-centered differences was used, followed by one sample T-tests. Type I error was controlled at alpha = 0.05, adjusting of p-values to account for multiple comparisons using Bonferoni's method. Multiple factor repeated measures ANOVAs were used to identify sources of variance.

3. Results

The standard curves, expressing C_t as a function of log₁₀ cp/mL, were mostly linear (Figure S2), as seen by the variation in efficiencies (Table 1). In some cases, the individual standard curves could be optimized by excluding some values, based on identifying departures from a least squares line (typically by excluding the lowest and/or highest values; Figure S2). However, this did not result in a significant improvement of the results (Figure S3)

3.1. Clinical samples

In comparing the accuracy based on difference from target, there was a significant difference due to units (F _1,112=64.86, p < 0.001), but not due to sample (F _2,112 = 0.052, p = 0.950), controlling for repeated measures (Fig. 1 , Figure S4). Using post-hoc one sample T-tests, C_t values were significantly different from the target for one of three samples. After converting the measurements IU/mL, the means were different from the target value for two of the three samples.

Fig. 1.

The accuracy of measuring the concentration of SARS-CoV-2 RNA in three clinical samples (S1, S11, and S5) expressed as C_t values and converted into IU/mL based on an international standard. The values are displayed as differences in observed C_t values from target C_t values, adjusted to log₁₀ copies (assuming perfect efficiency, left) and the difference in observed IU/mL from target log₁₀ copies/mL, determined by digital droplet PCR (right). The individual observations from 15 unique test systems from nine laboratories are shown as black dots, with boxes indicating the inter-quartile range and a median-value horizontal bar. Asterisks indicate sample means that were significantly different from the target (one sample T-test, adjusted p-values < 0.05).

Using mean-centered differences to test precision, the variance of the C_t values was 2.68 times higher than the variance of the log₁₀ IU/mL values (variance test, F _68,68 = 2.686, p < 0.001, Table 2 ), when both were expressed on the log₁₀ scale. The C_t values were different from the mean by a range spanning 5.8-6.3 C_t values over all three samples (1.74-1.90 log₁₀ copies, Table 2, Fig. 1). The concentrations converted to IU/mL were different from the target value by a range spanning 1.19-1.39 log₁₀ copies (Fig. 1). This corresponds to C_t values varying by up to ∼10-fold above and ∼10-fold below a given target value, whereas converting to IU/mL reduced this variability to ∼2-fold differences above/below a given target value.

Table 2.

Variance in measuring SARS-CoV-2 RNA in three clinical samples (S1, S11, S5) using two measurement systems – IU/mL and C_t values – where values were set on the same scale by mean-centering (Δ) the IU/mL (expressed as log₁₀ copies/mL) and converting ΔC_t values to log₁₀ scale (middle columns). The range of C_t values is provided in the far right column.

Sample	Var(ΔIU)	Var(ΔCt)	Range of Ct (Min∼Max)
S1	0.086	0.261	5.8 (20.2∼26.0)
S11	0.103	0.266	6.3 (23.0∼29.3)
S5	0.124	0.314	5.8 (20.0∼25.8)
All	0.101	0.272*

*ratio of Var(ΔC_t):Var(ΔIU) = 2.69; variance test, F_68,68 = 2.686, p < 0.001.

3.2. Measurement of reference samples

The variance of scaled mean-centered C_t values was 1.8 times higher than the variance of scaled mean-centered IU values (variance test, F _220,220=1.851, p < 0.001) (Table 3 ); however the largest variances were due to the sample with 50 copies/mL (target C_t value = 38.2, with a range spanning 9.7 C_t units from C_t 32.3∼42.0). Ignoring that sample, the variance in C_t values ranged from 0.25 to 0.30 per sample (log₁₀ scale), while the variance of the mean-centered differences in log₁₀ IU/mL were significantly lower (0.10 to 0.19, variance test, F _203,203=1.81, p > 0.001). The corresponding C_t values had differences in ranges from 5.5 to 6.9 units, corresponding to a range of 45- to 119-fold differences between measurements of the same sample, or approximately 5- to 15-fold differences above and below the sample mean. The variability in log₁₀ IU/mL values (range of 10^{1 . 22} to 10^{1 . 84} copies) corresponds to approximately 16- to 70-fold differences between measurements of the same sample, or approximately 3.7- to 13.5-fold differences from the sample mean.

Table 3.

Ten test samples made from dilutions of SARS-CoV-2 reference material (Accuplex) were supplied to labs in a series of ring tests diluted to various concentrations and then reported in C_t units and converted to IU/mL based on a standard curve (WHO standard). The target concentrations of test samples were checked with digital drop PCR. The differences from the measured target concentration were used to compare the accuracy of each reporting method.

Sample	Lot	Target (copies/mL)	dPCRa (copies/mL)	Ct Range (Min.∼Max.)	ΔCt (95% C.I.)b	ΔIU/mL (95% C.I.)c
S8d	10593976	50	57	9.7 (32.3∼42.0)	-2.16 (-3.42 ∼ -0.90)*	0.71 (0.45 ∼ 0.97)*
S4e	10593976	100	230	5.5 (32.8∼38.3)	-1.97 (-2.77 ∼ -1.17)*	0.30 (0.10 ∼ 0.49)*
S6e	10593976	500	1400	6.0 (29.3∼35.3)	-1.99 (-2.81 ∼ -1.18)*	0.18 (0.002 ∼ 0.37)
S3e	10576151	1000	1200	6.9 (30.1∼37.0)	-0.51 (-1.26 ∼ 0.25)	0.14 (-0.03 ∼ 0.31)
S7	10593976	1000	2400	5.6 (29.1∼34.7)	-1.68 (-2.46 ∼ -0.91)*	0.15 (-0.02 ∼ 0.33)
S9	10593976	1000	2700	6.5 (28.2∼34.7)	-1.78 (-2.55 ∼ -0.10)*	0.13 (-0.04 ∼ 0.30)
S10	10576151	1000	1400	5.6 (31.0∼36.6)	-0.44 (-1.17 ∼ 0.29)	0.04 (-0.15 ∼ 0.22)
S13	10593976	1000	1700	6.3 (29.6∼35.9)	-1.18 (-1.97 ∼ -0.40)*	0.16 (-0.002 ∼ 0.32)
S2	10576151	5000	3300	6.3 (28.0∼34.3)	-0.16 (-0.98 ∼ 0.56)	0.25 (0.11 ∼ 0.39)*
S12	10576151	5000	5200	6.5 (28.0∼34.5)	0.03 (-0.68 ∼ 0.75)	0.004 (-0.15 ∼ 0.16)

^adPCR = digital PCR

^bmean difference in C_t value (ΔC_t) from target value measured and validated by the national reference laboratory, converted to log₁₀ copies, reported with 95% confidence intervals

^cmean difference in log₁₀ IU/mL (ΔIU/mL) from target value measured by ddPCR, reported with 95% confidence intervals

^dthis sample was reported negative from six of 15 unique test systems

^ethese samples were reported negative from one test system each

^⁎indicates P_adj < 0.05 by a one-sample T-test with H₀ true mean = 0, adjusting for multiple comparisons by Bonferroni method

In assessing the accuracy, there was a clear influence of sample (Fig. 2 ) and there was a statistically significant difference in target-centered differences due to reporting unit (F(unit)_1,409 = 155.79, p < 0.001; F(sample)_9,409 = 2.31, p = 0.015). Laboratories significantly overestimated the target value when reporting IU/mL by 5.2-fold and 2.0-fold copies for the two samples with the lowest concentration (Table 4). Of the eight remaining samples whose measured concentration was between 1200 and 5200 cp/mL, only one of the tests samples (S2) was measured incorrectly by IU/mL (Fig. 2, Table 4). Six of 15 unique test systems could not detect the sample with the lowest concentration (“S8”, 57 copies/mL), and one test system each could not detect S4 (230 copies/mL), S6 (1400 copeis/mL), and S3 (1200 copies/mL)

Fig. 2.

Accuracy of measuring SARS-CoV-2 vRNA in various dilutions of a standard test sample, and reporting results by either C_t value or by converting C_t values to IU/mL based on laboratory-specific standard curves. The values are displayed as differences in observed C_t values from target C_t values (red-filled boxplots), adjusted to log₁₀ copies (assuming perfect efficiency) and the difference in observed log₁₀ IU/mL from target log₁₀ copies/mL (blue), determined by digital droplet PCR. The individual observations from 15 unique test systems from nine laboratories are shown as black dots, with filled boxes indicating the inter-quartile range and a median-value horizontal bar. The sample target values are ordered left-to-right from lowest target concentrations (S8 = 50 copies/mL, S4 = 100 copies/mL, S6 = 100 copies/mL) to highest concentrations (S3, S7, S9, S10, S13 = 1000 copies/mL; S2, S12 = 5000 copies/mL).

Overall, the C_t values and converted IU/mL clustered by lab independently of sample (Figure S5). Focusing on a subset of six laboratories, wherein three combinations of assays on specific platforms were used (cobas SARS-CoV-2 test/Cobas 6800; Xpert Xpress SARS-CoV-2 test/GeneXpert, and Simplexa COVID-19 Detection kit/Liaison MDX), with five of these labs using more than one of the three systems (Table 1, Fig. 3 ), we tested the factors explaining the variance in the data. While the test system was not significant (F _2,3=0.462, p=0.67), the within-laboratory effect of test system was significant (F _2,81=8.425, p < 0.001) (Fig. 3). This was true when also adding the lot number used to prepare the standard and EQA round as independent predictors, as well as controlling for variance due to specific samples: the within-laboratory effects were significant, but not the overall main effects.

Fig. 3.

Points and boxplots of the difference in measured SARS-CoV-2 vRNA in nine standard test samples of various concentrations (from 230 to 5200 copies/mL) from the target concentration in log₁₀ copies/mL (expressed in IU/mL) over three test systems (unique combinations of platform and specific assay) for six laboratories (colors) in three rounds of an external quality assessment scheme. The boxes show the interquartile range and an internal horizontal median line with statistical outliers not connected by a vertical line. The within-laboratory variance was statistically significant while the test system did not explain a significant amount of the variance in mean difference from target. A gray background was used to aid in visualization of the three test systems, used by four, three, and three laboratories (respectively, left to right).

4. Discussion

The goal of this study was to improve the harmonization of measuring the concentration of SARS-CoV-2 vRNA by RT-qPCR. To this end, we tested whether using laboratory-specific standard dilution curves to convert C_t values to IU/mL would improve the accuracy and precision of results across laboratories and test systems compared to the now-traditional practice of reporting C_t values, alone.

Comparison of standard curves clearly supported the ability of all laboratories to implement the reporting of IU/mL rather than C_t values. The efficiencies were within acceptable ranges (90-110%), exhibited an appropriate dynamic range (detections from ∼10⁶-10¹ cp/mL), and seemed to be highly reproducible. While we noted that we could optimize some standard curves by removing visual “outliers,” we confirmed that this did not bias the outcomes of our analysis (Figure S3). However, there are important caveats and considerations to using standard curves to report IU/mL rather than C_t values. For example, here we based the conversion equation on single values from each dilution series, as some laboratories did not provide replicates. We assume that, in practice, laboratories would base their conversion on replicated dilution series within test systems, and that laboratories would include within- and between-run replicates to ensure a confident conversion of C_t to IU/mL. At the very least, once a conversion equation is established, laboratories should include known standard(s) that give expected results (target +/- s.d.) in order to validate each test run.

Laboratories should remain vigilant in monitoring the emergence of variants, which may contain mutations that affect the efficiency of their chosen RT-qPCR assay(s). The use of multiple targets is recommended by many regulatory agencies, and is less likely to fail to detect novel variants. In the event that a new test system is needed, laboratories would need to reestablish conversion equations as part of their validation procedure. With a standard curve, laboratories can also determine the dynamic range of their test system, paying particular attention to limits of detection. Fortunately, there now exist commercially available reference materials of both the “wildtype” and variant strains for the purposes of establishing standard curves and validating test systems.

We and others have observed through EQAs that there exists relatively high variability when reporting C_t values, and we presume this is also true outside of ring trials, for example, when measuring virus concentration in patient samples [5,8,9,11,12,20,22]. Our data suggested that converting C_t values to copies per mL (and reporting them as IU/mL) statistically improved precision for all samples. Moreover, reporting IU/mL provides comparability between laboratories, as the units are adjusted to a practical and biologically relevant unit scale (cp/mL) rather than “C_t value units.” Our data showed that converting to IU/mL based on within-laboratory standard curves reduced the variability of C_t values (which vary in a range of approximately 6 C_t units ∼ 100-fold differences) by an order of magnitude, with laboratories reporting approximately 10-fold differences for a given sample.

However, it is clear that the accuracy of the tests, and the ability of estimating a “true” concentration, is still limited. As we noted, our estimation of accuracy for C_t values was based on a target value from the reference laboratory. This may have biased the analysis of the accuracy of C_t values results, given that we also determined that Ct values are specific to each laboratory test-system (Fig. 3). We used dPCR to validate the “true” concentration when evaluating the accuracy, and although we noted that the measured concentration was different than the target for some samples, we did not evaluate the accuracy of dPCR, but note that it was different than we expected (Figure S1). We therefore stress that, for the purposes of our analysis, it was not critical to conclude whether one test system consistently over- or under-estimates the true value. Nonetheless, our results demonstrated that the average value of all laboratories was different from the true concentration in 2 of 3 clinical samples, and 3 of 10 reference samples (Fig. 1 and Fig. 2). Despite this demonstration of inaccuracy, we found that the overall accuracy – in a practical sense – was actually relatively high (no more than 5-fold difference from the true cp/mL on average), and accuracy mostly improved when reporting IU/mL versus C_t (Fig. 1 and Fig. 2).

Fundamentally, the issue of accuracy – and the degree to which accurate results are required – depends on the desired goal of knowing the specific concentration. Certainly, reporting results in a biologically meaningful way makes it possible to compare these to in vitro results, for example, on the limit of isolation from patient samples (reported in terms of concentration of virus particles, e.g., TCID₅₀/mL or pfu/mL). As this was the basis of some health regulations to set target viral loads, below which patients are considered noncontagious [14, 15, 23, 24], perhaps there is some demand in achieving high accuracy. However, the use of “viral loads” to make clinical decisions – particularly when concentrations are estimated based on C_t values – has received a fair amount of criticism in the scientific literature [[3], [4], [5], [6],9, 10].

We were able to investigate the sources of variability underlying the reporting of results. Our data suggest that the overall variability – and thus the limitation on accuracy – does not depend on the test system, per se, but is intrinsic to each within-laboratory test system. This accounted for both within-laboratory variance when two or more test systems were used, as well as between-laboratory variance for a given test system (Fig. 3). This agrees with the results of others, who have attempted to calibrate the C_t values across multiple laboratories using reference samples with a specific target value, and concluded that C_t values are essentially unique to a given lab [12]. The fact that the variability was not necessarily attributed to a given test system is fortunate, given the large number of commercially-available kits to identify SARS-CoV-2 by RT-qPCR. We hypothesize that variations in sample preparation (e.g., efficiency of nucleic acid extraction, total volume used as template) contribute to variations within and between laboratories and/or within and between test systems (Fig. 3), however this remains to be tested. If these are truly the underlying reasons for the strong within-laboratory within-test system effect, improving the accuracy across all laboratories could come from implementing strict “universal” standard operating procedures or improving external quality assessments to providing laboratories the opportunity to adjustment their protocols to achieve higher accuracy [7,12,20,21].

While we assessed analytical variability within well-characterized samples, variability also depends on pre-examination processes, e.g., quality of sampling [[16], [17], [18], [19],25]. Although there have been promising approaches to normalize virus quantitation results from swab specimens by relating them to quantitation results of host cell housekeeping genes, similar to the delta-delta C_t method used in quantifying relative gene expression, these have not been adequately pursued nor have they received wide acceptance [26].

5. Conclusions

We demonstrated that results reported by RT-qPCR can be delivered as quantitative results in a biologically meaningful unit system, as is performed with many other viruses in clinical diagnostics (e.g., HIV, hCMV). The high precision of results across samples and across laboratories when utilizing standard curves to report results contributes to harmonization and thus allows a more reliable comparison of SARS-CoV-2 RT-qPCR results obtained by different assays. We note that accuracy could be improved, but the reported results mostly reached target values. Importantly, our results demonstrate the value of EQAs as tools for within- and between-laboratory comparisons; and we believe this is the key to improve the accuracy of measurements for all laboratories.

Author contributions

CB: Conceptualized, conducted and analysed this EQA study, wrote and edited the manuscript draft; DK: Analysed data and provided technical EQA support, visualized data, critically reviewed manuscript; JJ: Analysed data and provided technical EQA support, critically reviewed manuscript; WH: supervised statistical analysis, critically reviewed manuscript; VD: Provided metrological advice, critically reviewed the manuscript; EPS: Provided scientific advice, critically reviewed the manuscript; MM: Performed and analysed dPCR of samples, critically reviewed manuscript; MMM: Provided scientific advice to the overall study, reviewed and edited the manuscript; SS: critically reviewed the manuscript; AG: Provided scientific advice to the overall study, reviewed and edited the manuscript; SWA: Provided sample material, conceptualized, conducted and supervised this EQA study, provided scientific advice to the study, reviewed and edited the manuscript; IG: Conceptualized, conducted and analysed this EQA study, critically reviewed the manuscript; JVC: Conducted and analysed this EQA study, wrote and edited the manuscript.

Funding

The authors declare no funding sources.

Ethical approval

Ethical approval was not applicable for this study.

Consent for publication

Not applicable

Availability of data and materials

The datasets generated and/or analysed during the current study are not publicly available as they may be used to identify specific diagnostic laboratories. However, de-identified data are available from the corresponding author on reasonable request.

Declaration of Competing Interest

The authors declare no competing financial nor non-financial interests.