Abstract
Objective
Analytical validation of newly released SARS-CoV-2 antibody assays in the clinical laboratory is crucial to ensure sufficient performance in respect to its intended use. We aimed to assess analytical and diagnostic performance of 8 (semi-)quantitative assays detecting anti-nucleocapsid IgG (Euroimmun, Id-Vet) or total Ig (Roche), anti-spike protein IgG (Euroimmun, Theradiag, DiaSorin, Thermo Fisher) or both (Theradiag) and 2 rapid lateral flow assays (LFA) (AAZ-LMB and Theradiag).
Methods
Specificity was evaluated using a cross-reactivity panel of 85 pre-pandemic serum samples. Sensitivity was determined at both the manufacturer's and a 95% specificity cut-off level, using 81 serum samples of patients with a positive rRT-PCR. Sensitivity was determined in function of time post symptoms onset.
Results
Specificity for all assays ranged from 92.9% to 100% (Roche and Thermo Fisher) with the exception of the Theradiag IgM LFA (82.4%). Sensitivity in asymptomatic patients ranged between 41.7% and 58.3%. Sensitivity on samples taken <10 days since symptom onset was low (23.3%–66.7%) and increased on samples taken between 10 and 20 days and > 20 days since symptom onset (80%–96% and 92.9%–100%, respectively). From 20 days after symptom onset, the Roche, Id-vet and Thermo Fisher assays all met the sensitivity (>95%) and specificity (>97%) targets determined by the WHO. Antibody signal response was significantly higher in the critically ill patient group.
Conclusion
Antibody detection can complement rRT-PCR for the diagnosis of COVID-19, especially in the later stage, or in asymptomatic patients for epidemiological purposes. Addition of IgM in LFAs did not improve sensitivity.
Keywords: COVID-19, SARS-CoV-2, Antibody, Serology, Humoral response
Abbreviations: AARD, anti-nuclear antibody associated rheumatic disease; CDC, Centers for Disease Control and Prevention; CLIA, chemiluminescence immunoassays; COVID-19, Coronavirus Disease 2019; CV, coefficient of variation; DS-S, LIAISON SARS-CoV-2 S1/S2 IgG (DiaSorin S.P.A., Italy) targeting IgG anti-S antibodies; EI-N, Anti-SARS-CoV-2-NCP (Euroimmun, Germany) targeting IgG anti-N antibodies; EI-S, Anti-SARS-CoV-2 (Euroimmun, Germany) targeting IgG anti-S antibodies; ELISA, enzyme-linked immunosorbent assays; FAMHP, Belgian Federal Agency for Medicines and Health Products; FEIA, fluoro-enzyme-immunoassay; iQC, internal quality control materials; Id-N, ID Screen® SARS-CoV-2-N IgG Indirect ELISA (Id-vet, France) targeting IgG anti-N antibodies; LFA, lateral flow assay; LFA PR-SN, COVID-PRESTO® (AAZ-LMB, France) targeting IgG + IgM anti-S + anti-N antibodies; LFA TD-S, TDR Covid-19 IgG + IgM Thera (Theradiag, France) targeting IgG + IgM anti-S antibodies; N, nucleocapsid protein; R-N, Elecsys Anti-SARS-CoV-2 (Roche, Germany) targeting total Ig anti-N antibodies; rRT-PCR, real-time reverse transcriptase-polymerase chain reaction; S, spike protein; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2; TD-S, Covid-19 ELISA THERA02 IgG (Theradiag, France) targeting IgG anti-S antibodies; TD-SN, COVID19-ELISA IgG (Theradiag, France) targeting IgG anti-S + anti-N antibodies; TF-S, EliA SARS-CoV-2 Sp1 IgG (Thermo Fisher Scientific, Sweden) targeting IgG anti-S1 antibodies
1. Introduction
In December 2019 several cases of pneumonia of unknown cause occurred in Wuhan, Hubei Province, China. On January 72,020, a novel betacoronavirus, severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) was isolated from the patients in Wuhan (Wang et al., 2020a). This virus is responsible for a viral pneumonia called coronavirus disease 2019 (COVID-19) (Zhu et al., 2020; Lu et al., 2020). Although most people with COVID-19 disease have mild to moderate symptoms, the disease can cause severe medical complications such as acute respiratory distress syndrome, septic shock, bleeding and coagulation disorders, and can lead to death in pre-disposed people (Chen et al., 2020). Due to a combination of high human-to-human transmissibility, absence of natural immunity in the population and a lot of international traffic, the virus has quickly spread around the world and evolved to a global pandemic (World Health Organization, 2020a; Worobey et al., 2020). At the time of writing, the virus is globally disrupting society. Therefore, a rapid and correct identification of the virus is crucial, not only for the diagnosis of COVID-19 disease and subsequent correct treatment, but also to take necessary isolation precautions and thereby avoid further spreading. Also, vaccination will probably soon be a possible solution for reducing the spread of the virus by evoking humoral immunity in the vaccinated people.
The current gold standard for the diagnosis of COVID-19 is the detection of viral RNA in respiratory tract samples with real-time reverse transcriptase-polymerase chain reaction (rRT-PCR) targeting SARS-Cov-2 specific sequences coding for spike (S), envelope (E), or nucleocapsid proteins (Corman et al., 2020; Wang et al., 2020b; Chu et al., 2020; World Health Organization, 2020b; Bohn et al., 2020). rRT-PCR is highly sensitive and specific, especially in the acute phase of the infection (Infantino et al., 2020; Tang et al., 2020). The sensitivity of the PCR test depends on the time of sample collection in relation to the diagnostic testing window. Negative PCR results may be caused by extremely low viral load when tested shortly after exposure or at late stages of infection. Maximum viral load in throat swabs was observed 2 days before until 5 days after symptom onset (Wölfel et al., 2020; Kampf et al., 2020). The median [interquartile range] period between symptom onset and a negative rRT-PCR result has been reported to be 20 [17–24] days (Xiao et al., 2020). Furthermore, higher viral load and a longer mean duration of viral detection in respiratory samples correlate with disease severity (Clementi et al., n.d.). rRT-PCR can be false negative due to pre-analytical issues such as the sample collection technique. As to be expected, bronchoalveolar lavage (BAL) and sputum samples have shown to contain a higher viral load, and thus to remain positive for a longer time, compared to nasopharyngeal, nose or throat samples (Wang et al., 2020b). However, the Centers for Disease Control and Prevention (CDC) recommends using upper respiratory specimens for initial diagnostic testing, for logistical purposes and to limit invasive sampling procedures (Centers for Disease Control and Prevention, 2020). The nasopharyngeal swab (NP) is currently proposed as the gold standard sample for detection of SARS-CoV-2 due to the higher sensitivity for the detection of SARS-CoV-2 compared to oropharyngeal swabs and saliva samples (Wang et al., 2020b), restricting the latter sample types to specific screening strategies (Williams et al., 2020).
Serological assays have the potential to play a complementary role in the diagnosis of rRT-PCR-negative COVID-19 cases (Xiang et al., 2020). Seroconversion for SARS-CoV-2 is typically detected between 7 and 14 days post symptom onset (Guo et al., 2020; Burbelo et al., 2020; Long et al., 2020a). Among the four SARS-CoV-2 structural proteins, the spike (S) and nucleocapsid (N) proteins are the most immunogenic (Meyer et al., 2014; Qiu et al., 2005). Different types of tests are available to detect anti-SARS-CoV-2 antibodies: rapid lateral flow assays (LFA) as point of care tests, enzyme-linked immunosorbent assays (ELISA) and automated immunoassays. The contribution of serological assays to seroprevalence studies and evaluation of the results of vaccine trials is currently under debate (Okba et al., 2020a).
The aim of this study is to compare the diagnostic performance of ten commercial SARS-CoV-2 antibody test assays: five ELISA's, one fluoro-enzyme-immunoassay (FEIA), two rapid LFA's, and two chemiluminescence immunoassays (CLIA) (Supplementary Material 1). The study was performed in co-operation with the Belgian Federal Agency for Medicines and Health Products (FAMHP) that had set up a validation scheme for serological SARS-CoV-2 assays, whereby positively evaluated laboratory assays are reimbursed by the national health insurance.
2. Materials and methods
2.1. Patient selection
This retrospective study was performed using 166 patient samples collected at the OLV Hospital Aalst, Belgium.
Specificity was assessed on a selection of 85 serum samples from unique patients, collected before the COVID-19 pandemic, from March 2017 to March 2020. This cross-reactivity panel consisted of a) 35 samples of patients with a rRT-PCR confirmed non-coronavirus respiratory pathogen infection b) 19 samples of patients with a rRT-PCR confirmed non-SARS-CoV-2 coronavirus infection c) 10 samples of patients with a confirmed systemic auto-immune rheumatic disease and d) 21 samples of patients with antibodies against other viral/bacterial/parasitic pathogens. A detailed description of this specificity cohort is listed together with the results description in Table 3.
Table 3.
Overview of the cross reactivity of every SARS-CoV-2 antibody assay for the total specificity cohort.
| Sample group |
micro-organism or diagnosis |
number of samples tested |
EI-S |
EI-N |
Id-N |
TD-S |
R-N |
DS-S |
TD-SN |
TF-S |
LFA PR-SN |
LFA TD-S |
||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| IgG | IgM | IgG + IgM | IgG | IgM | IgG + IgM | |||||||||||
| Non-coronavirus Respiratory infection (n = 35) | adenovirus | 1 | – | – | – | – | – | – | – | – | – | – | – | – | – | – |
| bocavirus | 2 | – | – | – | – | – | – | – | – | – | – | – | – | – | – | |
| Chlamydophila pneumoniae | 2 | – | – | – | 1 | – | – | – | – | – | 1 | 1 | – | 1 | 1 | |
| enterovirus | 3 | – | – | – | – | – | – | – | – | – | – | – | – | – | – | |
| human metapneumovirus | 2 | – | – | – | – | – | – | – | – | – | – | – | – | – | – | |
| influenza A virus | 7 | 1 | – | – | – | – | 1 | 1 | – | – | – | – | – | 1 | 1 | |
| influenza B virus | 1 | – | – | – | – | – | – | – | – | – | – | – | – | – | – | |
| Legionella pneumophilae | 1 | – | – | – | – | – | – | – | – | – | – | – | – | 1 | 1 | |
| Mycoplasma pneumoniae | 3 | 1 | – | – | – | – | – | – | – | – | – | – | – | – | – | |
| parainfluenza virus 1 | 3 | – | – | – | – | – | – | – | – | – | – | – | – | – | – | |
| parainfluenza virus 2 | 1 | – | – | – | – | – | – | – | – | – | – | – | – | – | – | |
| parainfluenza virus 3 | 1 | – | – | – | – | – | – | – | – | – | – | – | – | – | – | |
| parainfluenza virus 4 | 1 | – | – | – | – | – | – | – | – | – | – | – | 1 | 1 | 1 | |
| rhinovirus | 4 | – | – | – | – | – | – | – | – | – | – | – | – | – | – | |
| respiratory syncitial virus | 3 | 2 | 1 | – | – | – | – | – | – | – | – | – | – | – | – | |
| Other human coronavirus infections (n = 19) | human coronavirus 229E | 4 | – | – | – | – | – | – | – | – | – | – | – | – | 1 | 1 |
| human coronavirus HKU | 6 | – | 1 | – | – | – | – | – | – | – | – | – | – | 1 | 1 | |
| human coronavirus NL63 | 5 | – | – | – | – | – | – | – | – | – | – | – | – | 1 | 1 | |
| human coronavirus OC43 | 4 | – | 1 | – | – | – | 1 | 1 | – | – | – | – | 1 | 1 | 1 | |
| Systemic rheumatic disease (n = 10) | mixed connective tissue disease | 2 | – | – | – | – | – | – | – | – | – | – | – | – | 1 | 1 |
| rheumatoid arthritis | 3 | – | – | 1 | – | – | – | – | – | – | – | – | 1 | 2 | 2 | |
| systemic lupus erythematosus | 3 | – | – | – | – | – | – | – | – | – | – | – | 1 | 2 | 2 | |
| sjögren syndrome | 2 | – | – | – | – | – | – | – | – | – | – | – | – | – | – | |
| Other pathogens (n = 21) | Borrelia burgdorferi (IgG) | 1 | – | 1 | – | – | – | – | – | – | – | – | – | – | – | – |
| cytomegalovirus | 3 | – | – | – | – | – | – | – | – | 1 | – | 1 | – | – | – | |
| epstein-barr virus | 3 | – | – | – | – | – | – | – | – | – | – | – | – | – | – | |
| hepatitis A virus | 1 | – | – | – | – | – | – | – | – | – | – | – | 1 | 1 | 1 | |
| hepatitis B virus (HBsAg) | 2 | – | 1 | – | – | – | – | – | – | – | – | – | – | – | – | |
| hepatitis C virus | 2 | – | – | – | – | – | – | – | – | – | – | – | – | – | – | |
| hepatitis E virus | 3 | – | – | – | – | – | – | 1 | – | 1 | – | 1 | – | – | – | |
| parvovirus | 2 | – | – | – | – | – | 1 | – | – | – | – | – | – | – | – | |
| Toxoplasma gondii | 2 | – | – | – | – | – | – | – | – | – | – | – | – | – | – | |
| Treponema pallidum | 2 | – | 1 | – | – | – | 1 | – | – | 1 | – | 1 | – | 1 | 1 | |
| Total | 85 | 4 | 6 | 1 | 1 | 0 | 4 | 3 | 0 | 3 | 1 | 4 | 5 | 15 | 15 | |
Abbreviations: DS-S, Liaison SARS-CoV-2 S1/S2 IgG (DiaSorin S.P.A., Italy); EI-N (Anti-SARS-CoV-2-NCP (Euroimmun, Germany); EI-S, Anti-SARS-CoV-2 (Euroimmun, Germany); Id-N, ID screen SARS-CoV-2-N (Id-vet, France); LFA PR-SN, COVID-PRESTO® (AAZ-LMB, France); LFA TD-S = TDR Covid-19 IgG + IgM Thera (Theradiag, France); R-N, Elecsys Anti-SARS-CoV-2 (Roche, Germany); TD-S, COVID-19 THERA02 (Theradiag, France); TD-SN, COVID19-LISA IgG (Theradiag, France); TF-S, EliA SARS-CoV-2 Sp1 IgG (Thermo Fisher Scientific, Sweden).
Sensitivity was assessed on a selection of 81 serum samples from 77 patients with a rRT-PCR confirmed SARS-CoV-2 infection on nasopharyngeal swab. rRT-PCR was performed using an in-house method complying with the WHO guidelines (Corman et al., 2020). The time between symptom onset and sampling date was a) less than ten days (n = 30) b) between 10 and 20 days (n = 25) and c) more than 20 days (n = 14). Also 12 samples of asymptomatic patients were included. The median time between symptom onset and serum sampling was 11 days (range 1–51). The group consisted of 53 male and 28 female patients with a median age of 66 years (range 17–97). Of note, in case of multiple samples per patient, only the first sample per time-category was used to assess sensitivity.
All samples were stored at −20 °C until analysis.
2.2. Data collection
The protocol was approved by the local Ethics Committee OLV Hospital Aalst with Belgian registration number B126202000015. For all COVID-19 patients, disease severity status was collected. Patients were classified as a) “mild” if no hospital admission was required b) “moderate” in case of admission to a non-ICU ward, c) “critical” in case of admission to the ICU-ward or death and d) “asymptomatic”. Serum samples of immunosuppressed patients (hematological malignancies, solid organ transplant) and patients younger than one year were excluded from the data set (specificity and sensitivity).
2.3. Assays
Four new ELISA's, one FEIA and two new rapid LFA's were evaluated and compared to one established ELISA and two established CLIA's.
The new ELISA's and FEIA included were respectively Anti-SARS-CoV-2-NCP (Euroimmun, Germany) targeting IgG anti-N antibodies (used abbreviation throughout the manuscript: EI-N), ID Screen® SARS-CoV-2-N IgG Indirect ELISA (Id-vet, France) targeting IgG anti-N antibodies (abbreviation: Id-N), Covid-19 ELISA THERA02 IgG (Theradiag, France) targeting IgG anti-S antibodies (abbreviation: TD-S), COVID19-LISA IgG (Theradiag, France) targeting IgG anti-S + anti-N antibodies (abbreviation: TD-SN) and EliA SARS-CoV-2 Sp1 IgG (Thermo Fisher Scientific, Sweden) targeting IgG anti-S1 antibodies (abbreviation: TF-S). The new rapid LFA's included TDR Covid-19 IgG + IgM Thera (Theradiag, France) targeting IgG + IgM anti-S antibodies (abbreviation: LFA TD-S) and COVID-PRESTO® (AAZ-LMB, France) targeting IgG + IgM anti-S1 and anti-N antibodies (abbreviation: LFA PR-SN).
The established ELISA and CLIA's included concerned respectively Anti-SARS-CoV-2 (Euroimmun, Germany) targeting IgG anti-S antibodies (abbreviation: EI-S), LIAISON SARS-CoV-2 S1/S2 IgG (DiaSorin S.P.A., Italy) targeting IgG anti-S antibodies (abbreviation: DS-S) and Elecsys Anti-SARS-CoV-2 (Roche, Germany) targeting total Ig anti-N antibodies (abbreviation: R-N). A detailed description of the different assays, including type of analyzer used, is provided in Supplementary Material 1. The samples were analyzed within controlled pre-analytical sample conditions in batch by the laboratory of OLV Hospital Aalst according to the instructions of the different collaborating companies.
2.4. Performance measures and statistical analysis
Analytical performance of each assay was assessed by calculating imprecision (coefficient of variation (CV), %) using the manufacturer's internal quality control materials (iQC) and three patient serum samples with a low, medium and high SARS-CoV-2 Ab concentration. All iQC samples were measured before and after every run during 10 runs (CLSI EP5-A2) (Clinical and Laboratory Standard Institute (CLSI), 2004). Linearity was assessed by diluting a high level serum SARS-CoV-2 Ab sample with increasing amounts of a serum sample with very low levels of SARS-CoV-2 Ab (CLSI EP06-A) (Clinical and Laboratory Standard Institute (CLSI), 2003).
Diagnostic performance characteristics (sensitivity, specificity) were calculated for every SARS-CoV-2 antibody assay based on the manufacturer's cut-off and compared with the McNemar test. For calculation of performance characteristics, borderline results were considered as positive. For (semi-)quantitative assays, receiver operating characteristic (ROC) curve analyses were performed to verify company cut-off values. Cut-off values at the 95% specificity level were determined and corresponding sensitivity for diagnosing COVID-19 was compared between the different assays. Finally, a Box and Whisker analysis was performed between the different disease severity patient cohorts. Quantitative variables are presented as median and range and categorical variables with number and percentage or frequency. Data analysis was performed in MEDCALC® Statistical Software version 17.1 (MedCalc Software Ltd., Ostend, Belgium), except for ROC curves, which are performed in Microsoft Excel + Analyse-it® Software version 5.65.3 (Leeds, UK). A p-value <0.05 was considered statistically significant.
3. Results
3.1. Patient demographics
An overview of the demographic features of the different patient cohorts is shown in Supplementary Material 2a-d. In general, we retained no significant difference in gender distribution between the sensitivity and specificity patient cohorts (p = 0.1174), but regarding age, the sensitivity patient group was significantly older (p < 0.0001).
3.2. Analytical performance
3.2.1. Imprecision
Results of the imprecision study are presented in Supplementary Material 3. For the ELISA's of Euroimmun (EI-S & EI-N), the imprecision obtained for the patient sample iQC was higher than for the kit iQC which can be explained by the fact that the kit iQC's are prediluted and their imprecision results didn't include a predilution step. The latter is not true for the ELISA's of Theradiag (TD-S & TD-SN) and Id-vet (Id-N), with comparable imprecision results for the kit and patient sample iQC. Assays based on ELISA format obtained the highest CV% results.
3.2.2. Linearity
No deviation from linearity was revealed for any of the assays, which is illustrated in Supplementary Material 4a-h. The lower results for TD-S are related to imprecision rather than to non-linearity.
3.3. Diagnostic performance
3.3.1. Specificity cohort
Based on the 85 pre-pandemic serum samples of patients with non-coronavirus respiratory infections (n = 35), other human coronavirus infections (n = 19), systemic rheumatic diseases (n = 10) and other pathogens (n = 21), a specificity of 100% [95.8–100] was obtained for the R-N and TF-S assay (Tables 1 , 2 ). The lowest specificity was obtained for the LFA TD-S IgM assay (82% [72.6–89.8]), with the highest cross-reactivity in the cohort of anti-nuclear antibody associated disease (AARD) and non-SARS-CoV-2 coronaviruses (15/85 false positive results). All other specificities ranged from 93% to 99% (EI-N 92.9% [83.3–97.4], LFA TD-S IgG 94.1% [86.8–98.1], EI-S 95.3% [88.4–98.7], DS-S 95.3% [88.4–98.7], TD-SN 96.5%[90.0–99.3], LFA PR-SN IgG 96.5% [90.0–99.3], LFA PR-SN IgM 98.8% [93.6–100], Id-N 98.8% [93.6–100], TD-S 98.8% [93.6–100]). An overview of the aspecific reactivities is given for every SARS-CoV-2 antibody assay in Table 3 .
Table 1.
Diagnostic performance characteristics of all (semi-)quantitative assays according to different sensitivity cohorts.
Sensitivity is calculated at both the manufacturer's cut-off (SN) and at the cut-off corresponding to a specificity level of 95% (CU 95% SP SN).
Abbreviations: anti-N, anti-nucleocapsid protein; anti-S, anti-spike protein; AU, arbitrary units; AUC, area under the ROC curve, CI, confidence interval; CU: cut-off; DS-S, Liaison SARS-CoV-2 S1/S2 IgG (DiaSorin S.P.A., Italy); EI-N (Anti-SARS-CoV-2-NCP (Euroimmun, Germany); EI-S, Anti-SARS-CoV-2 (Euroimmun, Germany); Id-N, ID screen SARS-CoV-2-N (Id-vet, France); NS, not stated; ROC, receiver operating curve analysis; R-N, Elecsys Anti-SARS-CoV-2 (Roche, Germany); SN, sensitivity; SP, specificity; TD-S, COVID-19 THERA02 (Theradiag, France); TD-SN, COVID19-LISA IgG (Theradiag, France); TF-S, EliA SARS-CoV-2 Sp1 IgG (Thermo Fisher Scientific, Sweden). * in concordance to reference (World Health Organization, 2020c).
Table 2.
Diagnostic performance characteristics of the rapid LFA according to different sensitivity cohorts.
Abbreviations: CI, confidence interval; LFA PR-SN, COVID-PRESTO® (AAZ-LMB, France); LFA TD-S = TDR Covid-19 IgG + IgM Thera (Theradiag, France); SN, sensitivity; SP, specificity. *in concordance to reference (World Health Organization, 2020c).
3.3.2. Sensitivity cohort
The evaluated SARS-CoV-2 antibody assays showed a significant difference in diagnostic performance with the 81 serum samples selected from patients with a rRT-PCR confirmed SARS-CoV-2 infection. Data of all (semi-)quantitative assays are shown in Table 1 and corresponding receiver operating characteristic (ROC) curves in Fig. 1 . The observed differences are mainly related to the diagnostic performance in the early phase of antibody detection. The diagnostic performance of the antibody tests was directly proportional to the time period after onset of symptoms: the longer this time period, the higher the diagnostic performance of all antibody tests and consequently, the lower the difference in diagnostic performance between tests. In addition, assays using a recombinant N-antigen revealed generally higher sensitivities compared to those targeting the S-antigen, although specificity results were generally comparable. Based on these results, the R-N assay showed the best diagnostic performance characteristics.
Fig. 1.
Receiver operating characteristic (ROC) curves of all (semi-)quantitative assays according to different sensitivity cohorts. A. Concerning the ‘all sensitivity cohort’ (n = 81), R-N showed significantly higher AUC than any other assay (p < 0.05); DS-S showed significantly lower AUC than any other assay (all p < 0.05) Id-N showed significantly higher AUC than TF-S (p = 0.0465). B. In the sensitivity cohort ‘asymptomatic’ (n = 12), DS-S showed significantly lower AUC than Id-N (p = 0.0357), TD-S (p = 0.0383), R-N (p = 0.0056), TF-S (p = 0.0062) and TD-SN (p = 0.0475); R-N showed significantly higher AUC than EI S (p = 0.0101) and TF-S (p = 0.0286) and Id-N showed significantly higher AUC than TF-S (p = 0.0465). C. In sensitivity cohort ‘<10 days after symptom onset’, DS-S showed significantly lower AUC than any other assay (all p < 0.05); additionally, R-N showed significantly higher AUC than EI S (p = 0.0147), EI N (p = 0.0174), TF-S (p = 0.0161) and TD-SN (p = 0.0067). D. In sensitivity cohort ‘10–20 days after symptom onset’, DS-S showed significantly lower AUC than Id-N (p = 0.0195) and R-N (p = 0.0197). E. Concerning sensitivity cohort ‘≥20 days after symptom onset’ no significantly differences in AUC were revealed (all p > 0.05).
Equally, all data on diagnostic performance characteristics of the rapid LFA are shown in Table 2. Overall, LFA PR-SN showed the most favorable results, with the highest sensitivity in all cohorts (<10 days after symptom onset: 66.7% [41.2–82.7] to ≥20 days after symptom onset: 100.0% [76.8–100]) and overall specificity of 96.5% [90.0–99.3]).
For all assays, Box and Whisker analysis revealed significantly higher antibody results in the ‘critically ill’ patient cohort (n = 33) compared to the ‘moderately ill’ cohort (n = 33) (Supplementary Material 5; for all assays p < 0.05). However, the proportion of samples collected ≥20 days after symptom onset was significantly higher in the ‘critically ill’ patient cohort (36% versus 3%; p = 0.0008), which could attribute to the higher antibody levels. No significant difference was observed when comparing antibody results between patient cohort ‘asymptomatic’ (n = 12) and cohort ‘mildly ill’ (n = 3) or ‘mildly ill’ and cohort ‘moderately ill’ patients for all assays (p > 0.05).
4. Discussion
Since the start of the COVID-19 pandemic, an increasing number of serological SARS-CoV-2 assays have been introduced to the diagnostic market (Deeks et al., 2020). The expertise of laboratory professionals is critical in the validation of these diagnostic assays to ensure sufficient analytical performance in respect to the intended use (Vermeersch et al., 2020).
This study evaluated the diagnostic performance of eight (semi-)quantitative (IgG/total Ig) and two rapid LFA (IgM and IgG) serological assays for the detection of SARS-CoV-2 (N/S protein). Taking into account the manufacturer's threshold, the overall sensitivity in our cohort (n = 81) ranged from 56.8% [45.3–67.8] (TD-S) to 84.0% [74.1–91.2] (R-N). The lower overall sensitivity is not surprising given the median time of 11 days between symptom onset and sample collection in our cohort. Our sensitivity findings (range 23.3–50.0%) in the subgroup of ‘patients < 10 days of symptoms’ don't support the use of serological SARS-CoV-2 assays in patients presenting at the emergency ward, as confirmed by previous studies (Zhao et al., 2020). Furthermore, the addition of IgM analysis for the LFA didn't significantly improve sensitivity. Not taking into account the quicker seroconversion of total antibodies (R-N) possibly attributed to the double-antigen sandwich, previous antibody kinetic studies have shown a similar median time of seroconversion for IgM and IgG (Long et al., 2020a; Zhao et al., 2020).
Antibodies against N protein are reported to appear earlier in infection than those against S protein (Grzelak et al., 2020). Within the subgroup of ‘patients < 10 days of symptoms’ and in the asymptomatic patient cohort we also revealed higher sensitivities for the N-based assays (range of respectively 46.7–50.0%, 50.0–58.3%) versus S-based assays (range of respectively 23.3–36.7%, 41.7–50%), with significantly different areas under the diagnostic (AUC) receiver operating curve (ROC) between some of the N and S-protein based assays. However, the Ag-source clearly appeared not to be the only factor attributing to diagnostic sensitivity (Table 1). Our data are in concordance with other head to head SARS-CoV-2 antibody comparison studies (Lassaunière et al., 2020; Van Elslande et al., 2020; National SARS-CoV-2 Serology Assay Evaluation Group, 2020; Pieri et al., 2020; Charpentier et al., 2020; Herroelen et al., 2020; Perkmann et al., 2020) and, if compared on the same level of specificity (95%), the R-N revealed the best overall sensitivity (84.0% [74.1–91.2]) versus DS-S the lowest (61.7% [50.3–72.3]). For symptomatic patients, all tests, except for TD-S, revealed a sensitivity of 100% ≥ 20 days after symptom onset (Table 1). At this time-point, there are no significant differences in area under the diagnostic curve (AUC) between the serological tests (Fig. 1).
The sensitivity in the asymptomatic cohort was significantly lower than the overall sensitivity. Importantly, 9 of the 12 samples were taken <10 days after positive rRT-PCR and 4 of those 9 serum samples tested negative in all antibody assays. Most likely, the lower sensitivity can be attributed to early infection or to a difference in Ab kinetics as described earlier in this patient category (Jiang et al., 2020). In the study of Jiang and colleagues, IgG/IgM titers and plasma neutralisation capacity were, at the time of virus clearance, significantly lower in recovered asymptomatic than in recovered symptomatic patients. Reinforced by the fact that a major part of asymptomatic and pauci-symptomatic patients is not even tested for viral RNA, serology ultimately offers the greatest potential to understand the true scale of SARS-CoV-2 infections. The persistence of the SARS-CoV-2 specific antibody response is still under review. It is observed that IgG levels and neutralizing antibodies in a high proportion of individuals who recovered from SARS-CoV-2 infection start to decrease within 2–3 months after infection, especially for the asymptomatic patients. 40.0% (120) of asymptomatic individuals, but only 12.9% (4/31) of symptomatic individuals, became seronegative for IgG eight weeks post hospital discharge (Long et al., 2020b).
The added value of SARS-CoV-2 antibody detection for the diagnosis of COVID-19 in rRT-PCR negative patients presenting in the late stage of the disease is well known (Long et al., 2020a). As we have shown in Table 1, antibody detection shows a high sensitivity from 10 days post symptom onset onwards. In this regard, serology can potentially offer added value in patients with a single respiratory sample with low viral load (cycle threshold ≥32), to distinguish acute from past infection. At this stage of the pandemic, quite some people have already gone through a COVID-19 infection some without knowing. As SARS-CoV-2 RNA remains detectable for several weeks to months after an infection, this can result in unexpected positive rRT-PCR results (with low viral load) at routine (pre-)admission screening of patients. In the absence of respiratory symptoms, the presence of antibodies in combination with a low viral load on rRT-PCR is highly suggestive for late stage of the disease or past infection. The patient can be considered noncontagious and there is no need for specific isolation precautions. When the antibody test is negative, the high cycle threshold most likely indicates a very recent infection, and thus an infectious patient. A new respiratory sample for the detection of SARS-CoV-2 RNA is warranted to confirm a recent onset of infection. The potential role of serology in monitoring the immune response after vaccination is a next topic of research.
Regarding specificity, N protein-based serological assays were more often associated with cross-reactivity than the S-based assays (Infantino et al., 2020). For the latter, recombinant and more standardized S1-based assays have shown to be more specific compared to assays to full viral antigens (Okba et al., 2020b; Amanat et al., 2020) (Supplementary Material 1). Overall specificity in the samples collected prior to the pandemic (n = 85) ranged from 82.4% (LFA TD-S) to 100% (R-N & TF-S) (Table 1). Cross-reactivity is mainly attributed to antigens well-conserved among different coronaviruses and to cross-reaction with antibodies of autoimmune diseases (Wang et al., 2004). When using antibody assays on a population level, a high specificity is of utmost importance, as every small drop in specificity will seriously reduce the positive predictive value (Galli and Plebani, 2020). In the future, if antibodies prove to be protective, false positive results can potentially also have an important impact on the individual patient level if these results are used to decide whether or not to administer (re)vaccination or to use personal protective equipment.
In accordance with preceding studies (Long et al., 2020a; Okba et al., 2020a), we found that all (semi-)quantitative assays result in significantly (p < 0.05) higher antibody levels in the ‘critically ill’ patient cohort compared to the ‘moderately ill’ cohort. However, the proportion of samples collected ≥20 days after symptom onset was also significantly higher in the ‘critically ill’ patient cohort (36% versus 3%; p = 0.0008), which could attribute to the higher antibody levels. Nevertheless, our observations are completely in line with earlier findings that antibody levels are associated with disease severity (Gudbjartsson et al., 2020).
A strength of our study is the parallel evaluation of the diagnostic performance of several new serologic SARS-CoV-2 assays and assays with established diagnostic performance. Furthermore, we've performed a separate diagnostic performance analysis in asymptomatic people. In this subgroup, overall sensitivity revealed to be lower than the overall sensitivity obtained for the several assays, as mentioned above. This is not surprising taking into account the earlier mentioned difference in Ab kinetics in the asymptomatic population.
A limitation of our study is that the samples used to evaluate specificity were all challenging. We thus expect a higher specificity in a routine laboratory setting. Another limitation is the limited sample size, which results in a small number of cases in the subgroup analyses concerning timing post symptom onset and severity of symptoms. Finally, the categorization of the patient cohorts “mild”, “moderate”, “critical” was only based on whether or not the patient was admitted to the hospital/intensive care unit. Information on duration and severity of symptoms of individual cases is lacking, due to the retrospective design of this study.
We can conclude that, in this study, the R-N serological assay revealed the best overall performance. However, for the intended use of antibody detection (>20 days after symptom onset), the R-N, Id-N and TF-S assays all met the sensitivity (95–98%) and specificity (97–99%) targets determined by the WHO (World Health Organization, 2020c).
Funding
No funding was received for conducting this study.
Data availability
Data will be available from the author upon request.
Ethics approval
The protocol was approved by the local Ethics Committee OLV Hospital Aalst with Belgian registration number B126202000015. The procedures used in this study adhere to the tenets of the Declaration of Helsinki.
Authors' contributions
MM, EVH, LC, AB and LVH contributed to the study conceptualization. Data curation and project administration were performed by MM, EVH, LN, SVDB and LH; Formal data analysis was performed by LVH. MM, EVH, LN and LVH wrote-original draft. Writing-review & editing was performed by MM, EVH, LN, LC, AB and LVH. The final manuscript was read and approved by all authors.
Declaration of Competing Interest
AB and LVH have been consultants for Thermo Fisher Scientific.
Acknowledgments
We thank Euroimmun, Id-vet, AAZ-LMB, Theradiag and Thermo Fisher Scientific for the donation of the assays. We are very grateful to the laboratory technicians for their most appreciated efforts.
Footnotes
Supplementary data to this article can be found online at https://doi.org/10.1016/j.jim.2021.113043.
Appendix A. Supplementary data
Supplementary material
References
- Amanat F., Stadlbauer D., Strohmeier S., Nguyen T.H.O., Chromikova V., McMahon M., et al. A serological assay to detect SARS-CoV-2 seroconversion in humans. Nat. Med. 2020 Jul;26(7):1033–1036. doi: 10.1038/s41591-020-0913-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bohn M.K., Lippi G., Horvath A., Sethi S., Koch D., Ferrari M., et al. Molecular, serological, and biochemical diagnosis and monitoring of COVID-19: IFCC taskforce evaluation of the latest evidence. Clin. Chem. Lab. Med. 2020 Jun;58(7):1037–1052. doi: 10.1515/cclm-2020-0722. [DOI] [PubMed] [Google Scholar]
- Burbelo P.D., Riedo F.X., Morishima C., Rawlings S., Smith D., Das S., et al. Sensitivity in detection of antibodies to Nucleocapsid and spike proteins of severe acute respiratory syndrome coronavirus 2 in patients with coronavirus disease 2019. J. Infect. Dis. 2020 Jun;222(2):206–213. doi: 10.1093/infdis/jiaa273. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Centers for Disease Control and Prevention . 2020. Interim Guidelines for Collecting, Handling, and Testing Clinical Specimens from Persons for Coronavirus Disease 2019 (COVID-19) [Google Scholar]
- Charpentier C., Ichou H., Damond F., Bouvet E., Chaix M.-L., Ferré V., et al. Performance evaluation of two SARS-CoV-2 IgG/IgM rapid tests (Covid-Presto and NG-test) and one IgG automated immunoassay (Abbott) J. Clin. Virol. Off. Publ. Pan. Am. Soc. Clin. Virol. 2020 Sep;132:104618. doi: 10.1016/j.jcv.2020.104618. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Chen N., Zhou M., Dong X., Qu J., Gong F., Han Y., et al. Epidemiological and clinical characteristics of 99 cases of 2019 novel coronavirus pneumonia in Wuhan, China: a descriptive study. Lancet. 2020 Feb;395(10223):507–513. doi: 10.1016/S0140-6736(20)30211-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Chu D.K.W., Pan Y., Cheng S.M.S., Hui K.P.Y., Krishnan P., Liu Y., et al. Molecular diagnosis of a novel coronavirus (2019-nCoV) causing an outbreak of pneumonia. Clin. Chem. 2020 Apr;66(4):549–555. doi: 10.1093/clinchem/hvaa029. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Clementi N, Ferrarese R, Tonelli M, Amato V, Racca S, Locatelli M, et al. Lower nasopharyngeal viral load during the latest phase of COVID-19 pandemic in a Northern Italy University Hospital. Clin. Chem. Lab. Med. 58(9):1573–7. [DOI] [PubMed]
- Clinical and Laboratory Standard Institute (CLSI) 2003. Evaluation of the Linearity of Quantitative Measurement Procedures: A Statistical Approach; Approved Guideline. Wayne, PA. [Google Scholar]
- Clinical and Laboratory Standard Institute (CLSI) In: Evaluation of Precision Performance of Quantitative Measurement Methods, Approved Guideline. Second ed, editor. 2004. Wayne, PA. (CLSI document). Report No.: EP5-A2. [Google Scholar]
- Corman V.M., Landt O., Kaiser M., Molenkamp R., Meijer A., Chu D.K., et al. Detection of 2019 novel coronavirus (2019-nCoV) by real-time RT-PCR. Euro Surveill. 2020 Jan;25(3) doi: 10.2807/1560-7917.ES.2020.25.3.2000045. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Deeks J.J., Dinnes J., Takwoingi Y., Davenport C., Spijker R., Taylor-Phillips S., et al. Antibody tests for identification of current and past infection with SARS-CoV-2. Cochrane Database Syst. Rev. 2020 Jun;6(6) doi: 10.1002/14651858.CD013652. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Galli C., Plebani M. SARS-CoV-2 antibody performances: we need better criteria. Clin. Chem. Lab. Med. 2020 Sep;58(12):e303–e305. doi: 10.1515/cclm-2020-1358. [DOI] [PubMed] [Google Scholar]
- Grzelak L., Temmam S., Planchais C., Demeret C., Huon C., Guivel F., et al. SARS-CoV-2 serological analysis of COVID-19 hospitalized patients, pauci-symptomatic individuals and blood donors. Sci. Transl. Med. 2020;12(559) doi: 10.1126/scitranslmed.abc3103. [DOI] [Google Scholar]
- Gudbjartsson D.F., Norddahl G.L., Melsted P., Gunnarsdottir K., Holm H., Eythorsson E., et al. Humoral immune response to SARS-CoV-2 in Iceland. N. Engl. J. Med. 2020 Oct;383(18):1724–1734. doi: 10.1056/NEJMoa2026116. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Guo L., Ren L., Yang S., Xiao M., Chang D., Yang F., et al. Profiling early Humoral response to diagnose novel coronavirus disease (COVID-19) Clin. Infect. Dis. 2020 Mar;71(15):778–785. doi: 10.1093/cid/ciaa310. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Herroelen P.H., Martens G.A., De Smet D., Swaerts K., Decavele A.-S. Humoral Immune Response to SARS-CoV-2. Am. J. Clin. Pathol. 2020 Oct;154(5):610–619. doi: 10.1093/ajcp/aqaa140. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Infantino M., Damiani A., Gobbi F.L., Grossi V., Lari B., Macchia D., et al. Serological assays for SARS-CoV-2 infectious disease: benefits, limitations and perspectives. Isr. Med. Assoc. J. 2020 Apr;22(4):203–210. [PubMed] [Google Scholar]
- Jiang C., Wang Y., Hu M., Wen L., Wen C., Wang Y., et al. Antibody seroconversion in asymptomatic and symptomatic patients infected with severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) Clin. Transl. Immunol. 2020;9(9) doi: 10.1002/cti2.1182. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kampf G., Brüggemann Y., Kaba H.E.J., Steinmann J., Pfaender S., Scheithauer S., et al. Potential sources, modes of transmission and effectiveness of prevention measures against SARS-CoV-2. J. Hosp. Infect. 2020 Dec;106(4):678–697. doi: 10.1016/j.jhin.2020.09.022. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lassaunière R., Frische A., Harboe Z.B., Nielsen A.C.Y., Fomsgaard A., Krogfelt K.A., et al. Evaluation of nine commercial SARS-CoV-2 immunoassays. medRxiv. 2020 Jan 2020.04.09.20056325. [Google Scholar]
- Long Q.-X., Liu B.-Z., Deng H.-J., Wu G.-C., Deng K., Chen Y.-K., et al. Antibody responses to SARS-CoV-2 in patients with COVID-19. Nat. Med. 2020 Jun;26(6):845–848. doi: 10.1038/s41591-020-0897-1. [DOI] [PubMed] [Google Scholar]
- Long Q.-X., Tang X.-J., Shi Q.-L., Li Q., Deng H.-J., Yuan J., et al. Clinical and immunological assessment of asymptomatic SARS-CoV-2 infections. Nat. Med. 2020 Aug;26(8):1200–1204. doi: 10.1038/s41591-020-0965-6. [DOI] [PubMed] [Google Scholar]
- Lu R., Zhao X., Li J., Niu P., Yang B., Wu H., et al. Genomic characterisation and epidemiology of 2019 novel coronavirus: implications for virus origins and receptor binding. Lancet. 2020 Feb;395(10224):565–574. doi: 10.1016/S0140-6736(20)30251-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Meyer B., Drosten C., Müller M.A. Serological assays for emerging coronaviruses: challenges and pitfalls. Virus Res. 2014 Dec;194:175–183. doi: 10.1016/j.virusres.2014.03.018. [DOI] [PMC free article] [PubMed] [Google Scholar]
- National SARS-CoV-2 Serology Assay Evaluation Group Performance characteristics of five immunoassays for SARS-CoV-2: a head-to-head benchmark comparison. Lancet Infect. Dis. 2020 Sep;20(12):1390–1400. doi: 10.1016/S1473-3099(20)30634-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Okba N.M.A., Müller M.A., Li W., Wang C., GeurtsvanKessel C.H., Corman V.M., et al. Severe acute respiratory syndrome coronavirus 2-specific antibody responses in coronavirus disease patients. Emerg. Infect. Dis. 2020 Jul;26(7):1478–1488. doi: 10.3201/eid2607.200841. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Okba N.M.A., Müller M.A., Li W., Wang C., GeurtsvanKessel C.H., Corman V.M., et al. SARS-CoV-2 specific antibody responses in COVID-19 patients. Emerg. Infect. Dis. 2020 Jul doi: 10.3201/eid2607.200841. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Perkmann T., Perkmann-Nagele N., Breyer M.-K., Breyer-Kohansal R., Burghuber O.C., Hartl S., et al. Side-by-side comparison of three fully automated SARS-CoV-2 antibody assays with a focus on specificity. Clin. Chem. 2020 Nov;66(11):1405–1413. doi: 10.1093/clinchem/hvaa198. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pieri M., Ciotti M., Carlozzi N., Frassanito M.L., Meloni A., Cistera A., et al. SARS-CoV-2 infection serology validation of different methods: usefulness of IgA in the early phase of infection. Clin. Chim. Acta. 2020 Sep;511:28–32. doi: 10.1016/j.cca.2020.09.033. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Qiu M., Shi Y., Guo Z., Chen Z., He R., Chen R., et al. Antibody responses to individual proteins of SARS coronavirus and their neutralization activities. Microbes Infect. 2005 May;7(5–6):882–889. doi: 10.1016/j.micinf.2005.02.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Tang Y.-W., Schmitz J.E., Persing D.H., Stratton C.W. Laboratory DIAGNOSIS of COVID-19: current issues and challenges. J. Clin. Microbiol. 2020 May;58(6) doi: 10.1128/JCM.00512-20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Van Elslande J., Decru B., Jonckheere S., Van Wijngaerden E., Houben E., Vandecandelaere P., et al. Antibody response against SARS-CoV-2 spike protein and nucleoprotein evaluated by four automated immunoassays and three ELISAs. Clin. Microbiol. Infect. 2020 Jul;26(11):1557.e1–1557.e7. doi: 10.1016/j.cmi.2020.07.038. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Vermeersch P., Cotton F., De Smet D., Martens G., Oyaert M., Cavalier E. Lessons from the Belgian experience with regulatory control during the COVID-19 pandemic for the implementation of the European IVD regulation 2017/746. Acta Clin. Belg. 2020 Jul:1–3. doi: 10.1080/17843286.2020.1787659. [DOI] [PubMed] [Google Scholar]
- Wang C., Horby P.W., Hayden F.G., Gao G.F. A novel coronavirus outbreak of global health concern. Lancet. 2020 Feb;395(10223):470–473. doi: 10.1016/S0140-6736(20)30185-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wang W., Xu Y., Gao R., Lu R., Han K., Wu G., et al. Detection of SARS-CoV-2 in different types of clinical specimens. JAMA. 2020 May;323(18):1843–1844. doi: 10.1001/jama.2020.3786. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wang Y., Sun S., Shen H., Jiang L., Zhang M., Xiao D., et al. Cross-reaction of SARS-CoV antigen with autoantibodies in autoimmune diseases. Cell. Mol. Immunol. 2004 Aug;1(4):304–307. [PubMed] [Google Scholar]
- Williams E., Bond K., Zhang B., Putland M., Williamson D.A. In: Saliva as a Noninvasive Specimen for Detection of SARS-CoV-2. AJ McAdam., editor. 58(8) 2020 Apr. (J Clin Microbiol.) [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wölfel R., Corman V.M., Guggemos W., Seilmaier M., Zange S., Müller M.A., et al. Virological assessment of hospitalized patients with COVID-2019. Nature. 2020 May;581(7809):465–469. doi: 10.1038/s41586-020-2196-x. [DOI] [PubMed] [Google Scholar]
- World Health Organization . 2020. Timeline of WHO’s Response to COVID-19. [Google Scholar]
- World Health Organization . 2020. Laboratory Testing of 2019 Novel Coronavirus (2019-nCoV) in Suspected Human Cases: Interim Guidance, 17 January 2020. [Google Scholar]
- World Health Organization . 2020. COVID-19 Target Product Profiles for Priority Diagnostics to Support Response to the COVID-19 Pandemic v.1.0. [Google Scholar]
- Worobey M., Pekar J., Larsen B.B., Nelson M.I., Hill V., Joy J.B., et al. The emergence of SARS-CoV-2 in Europe and North America. Science. 2020 Oct;370(6516):564–570. doi: 10.1126/science.abc8169. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Xiang F., Wang X., He X., Peng Z., Yang B., Zhang J., et al. Antibody detection and dynamic characteristics in patients with coronavirus disease 2019. Clin. Infect. Dis. Off. Publ. Infect. Dis. Soc. Am. 2020 Nov;71(8):1930–1934. doi: 10.1093/cid/ciaa461. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Xiao A.T., Tong Y.X., Gao C., Zhu L., Zhang Y.J., Zhang S. Dynamic profile of RT-PCR findings from 301 COVID-19 patients in Wuhan, China: A descriptive study. J. Clin. Virol. Off. Publ. Pan. Am. Soc. Clin. Virol. 2020 Jun;127 doi: 10.1016/j.jcv.2020.104346. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Zhao J., Yuan Q., Wang H., Liu W., Liao X., Su Y., et al. Antibody responses to SARS-CoV-2 in patients with novel coronavirus disease 2019. Clin. Infect. Dis. Off. Publ. Infect. Dis. Soc. Am. 2020 Nov;71(16):2027–2034. doi: 10.1093/cid/ciaa344. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Zhu N., Zhang D., Wang W., Li X., Yang B., Song J., et al. A novel coronavirus from patients with pneumonia in China, 2019. N. Engl. J. Med. 2020;382(8):727–733. doi: 10.1056/NEJMoa2001017. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Supplementary material
Data Availability Statement
Data will be available from the author upon request.