Abstract
Background
Seroprevalence studies of coronavirus disease 2019 (COVID-19) cases, including asymptomatic and past infections, are important to estimate the scale of the disease outbreak and to establish quarantine measures. We evaluated the clinical performance of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) antibody assays available in Korea for use in seroprevalence studies.
Methods
The sensitivity, specificity, cross-reactivity, and interference of five SARS-CoV-2 antibody assays were evaluated using the following 398 serum samples from confirmed COVID-19 patients, 510 negative control samples from before 2018 (pre-pandemic), 163 serum samples from patients with SARS, Middle East respiratory syndrome (MERS), and other viral infections, and five samples for the interference study.
Results
The sensitivities of the five assays ranged from 92.2% to 98%, and their specificities, including cross-reactivity and interference, ranged from 97.5% to 100%. The agreement rates were excellent (kappa >0.9). Adjustment of the cutoff values could be considered through ROC curve analysis. The positive predictive values of the individual assays varied from 3.5% to 100% at a 0.1% prevalence but were as high as ≥95% when two assays were combined.
Conclusions
The prevalence of COVID-19 in Korea is considered to be exceptionally low at present; thus, we recommend using a combination of two or more SARS-CoV-2 antibody assays rather than a single assay. These results could help select SARS-CoV-2 antibody assays for COVID-19 seroprevalence studies in Korea.
Keywords: COVID-19, SARS-CoV-2, Antibody, Seroprevalence
INTRODUCTION
Coronavirus disease 2019 (COVID-19), which originated in Wuhan, China in December 2019, is caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) [1]. More than 100 million people have been infected with SARS-CoV-2 and more than two million deaths due to COVID-19 have been reported worldwide in approximately one year [2]. The number of patients with confirmed disease includes only those who have been tested positive for SARS-CoV-2 following a hospital visit [3]. Therefore, the actual number of COVID-19 positive cases has been underestimated. To determine the size of the infected population and to establish quarantine measures, accurate serological testing is required.
Seroprevalence studies have been conducted in many countries, including the United States, the United Kingdom, Spain, and Korea [4-8]. In less than a year, several types of antibody assays have been developed worldwide. However, comparative studies on the performance of assays available in Korea to determine seroprevalence have not yet been conducted. The available antibody assays mainly use recombinant spike (S) proteins, nucleocapsid (N) proteins, receptor-binding domains, S1 antigens, and combinations of these antigens to detect IgG, IgM, and total antibody levels [9-16].
We evaluated the clinical performance of COVID-19 antibody assays available in Korea for seroprevalence studies. We further estimated the positive predictive values (PPVs) of individual and two combined assays using the sensitivities and specificities obtained from this study and the expected prevalence in Korea. We also investigated cross-reactivity using serum samples from patients with antibodies to various viruses and bacteria, autoimmune disease, or monoclonal gammopathy.
MATERIALS AND METHODS
Clinical samples
Serum samples, leftover from laboratory tests and designated to be discarded, from 398 patients diagnosed as having COVID-19 at two hospitals (Seoul Medical Center, Seoul, Korea and Hallym University Dongtan Sacred Heart Hospital, Hwaseong, Korea) and the Korea Disease Control and Prevention Agency (KDCA) were collected between March and September 2020 and stored at –70°C until analysis. The dates of symptom onset and hospital admission were obtained retrospectively from the medical records at the two hospitals. Serum samples of 510 negative controls, collected before 2018 (pre-pandemic period), were obtained from the National Biobank of Korea, the KDCA, and the High-Risk Human Serum Bank of Chung-Ang University (Seoul, Korea). A total of 168 samples were tested for cross-reactivity, including 136 residual serum samples of patients with antibodies to other viruses (human (h)CoV-229E, -NL63, -OC43, and -HKU1; adenovirus; influenza A virus; influenza B virus; human metapneumovirus; parainfluenza virus type 1/2/3/4; respiratory syncytial virus; rhinovirus; Mycoplasma pneumoniae; Chlamydia pneumoniae; hepatitis A/B/C virus; Epstein-Barr virus; cytomegalovirus; herpes simplex virus; mumps virus; and varicella zoster virus) collected from two hospitals (Severance Hospital, Seoul, Korea and Hallym University Dongtan Sacred Heart Hospital). Serum samples positive for fluorescence anti-nuclear antibody, rheumatoid factor, and monoclonal gammopathy were collected from the same two hospitals.
Twenty-three serum samples positive for antibodies to SARS-CoV-1, Middle East respiratory syndrome (MERS)-CoV, hCoV-229E, -NL63, -OC43, and -HKU1, influenza B virus, respiratory syncytial virus, and adenovirus were purchased from Trina Bioreactives (Nänikon, Switzerland). Five serum samples from patients with monoclonal gammopathy were collected from the Korea Institute of Radiological and Medical Sciences Radiation Biobank (Seoul, Korea), and four MERS-CoV convalescent serum samples were provided by the International Vaccine Institute (Seoul, Korea) (Fig. 1). The Public Institutional Review Board designated by the Ministry of Health and Welfare (http://public.irb.or.kr/) approved the study protocol and waived the need for informed consent (P01-202008-31-002).
Fig. 1.
Samples tested in this study.
Abbreviations: CAU, Chung-Ang University; C. pneumoniae, Chlamydia pneumoniae; CMV, cytomegalovirus; EBV, Epstein-Barr virus; FANA, fluorescence anti-nuclear antibody; HAV, hepatitis A virus; HCV, hepatitis C virus; hCoV, human coronavirus; HSV, herpes simplex virus; HUDSHH, Hallym University Dongtan Sacred Heart Hospital; IVI, International Vaccine Institute; KDCA, Korea Disease Control and Prevention Agency; KIRMS, Korea Institute of Radiological and Medical Sciences; MERS-CoV, Middle East respiratory syndrome coronavirus; M. pneumoniae, Mycoplasma pneumoniae; PIV, parainfluenza virus; RSV, respiratory syncytial virus; SARS-CoV-1, severe respiratory syndrome coronavirus 1; SH, Severance Hospital; SMC, Seoul Medical Center; VZV, varicella zoster virus.
SARS-CoV-2 antibody assays
In August 2020, the KDCA received applications from 18 manufacturers to evaluate the performance of 21 SARS-CoV-2 antibody assays. It was not feasible to evaluate all 21 assays given the limited amount of serum samples; hence, a screening evaluation was required. After the Evaluation Committee did a preliminary assessment of the data submitted by the manufacturers, five assays were selected for performance evaluation: Elecsys Anti-SARS-CoV-2 assay on the Cobas e801 platform (Roche Diagnostics, Mannheim, Germany), Abbott Architect SARS-CoV-2 IgG assay on the Architect i2000SR platform (Abbott Laboratories, Abbott Park, IL, USA), Atellica IM SARS-CoV-2 Total assay on the Atellica platform (Siemens, Munich, Germany), SD Biosensor Standard E COVID-19 Total Ab assay (SD Biosensor, Suwon, Korea), and LG Chem AdvanSure SARS-CoV-2 IgG assay (LG Chem, Seoul, Korea). SD Biosensor and LG Chem ELISAs were performed using the Epoch Microplate Spectrophotometer and ELx50 Filter Microplate Washer (both from BioTek Instruments, Winooski, VT, USA). The principle, instrument, antibody detected, reagents used, cutoff value, sample volumes, and time to first results of all assays are listed in Table 1. All assays were carried out according to the manufacturers’ instructions. Most assays were performed at Hallym University Dongtan Sacred Heart Hospital by one laboratory technician and one scientific researcher, and the Atellica IM SARS-CoV-2 total assay was performed at Severance Hospital by one laboratory technician.
Table 1.
Characteristics and sensitivity and specificity of the five SARS-CoV-2 antibody assays evaluated in this study
| Roche (Roche Diagnostics, Mannheim, Germany) | Abbott (Abbot Laboratories, Abbott Park, IL, USA) | Siemens (Siemens, Munich, Germany) | SD Biosensor (SD Biosensor, Suwon, Korea) | LG Chem (LG Chem, Seoul, Korea) | |
|---|---|---|---|---|---|
| Characteristics of the assays | |||||
| Product name | Elecsys anti-SARS-CoV-2 | SARS-CoV-2 IgG | SARS-CoV-2 total (COV2T) | STANDARD E COVID-19 Total Ab | AdvanSure SARS-CoV-2 IgG (S1) |
| Analyzer | Elecsys Cobas e801 | Architect i2000SR | Atellica IM | ELISA | ELISA |
| Principle | ECLIA | CMIA | CLIA | ELISA | ELISA |
| Target antibody | Total (IgG+IgM) | IgG | Total (IgG+IgM) | Total (IgG+IgM) | IgG |
| Used reagent antigen | Nucleoprotein | Nucleoprotein | RBD | Spike+Nucleoprotein | S1 |
| Sample type | Serum, plasma | Serum, plasma | Serum, plasma | Serum, plasma | Serum, plasma |
| Sample volume | 20 µL | 25 µL | 50 µL | 50 µL | 10 µL |
| Cut-off value (unit) | 1.0 (COI) | 1.4 (index) | 1.0 (index) | NC+0.3 (OD) | 1.0 (S/CO) |
| Time to first result (min) | 18 | 29 | 15 | 150 | 150 |
| Sensitivity and specificity of the assays according to days after symptom onset (or admission) | |||||
| 1–7 (N = 13) | 15.4% (2/13) | 15.4% (2/13) | 15.4% (2/13) | 53.8% (7/13) | 46.2% (6/13) |
| 8–14 (N = 65) | 86.2% (56/65) | 86.2% (56/65) | 95.4% (62/65) | 98.5% (64/65) | 92.3% (61/65) |
| 15–21 (N = 84) | 98.8% (83/84) | 98.8% (83/84) | 98.8% (83/84) | 100% (84/84) | 100% (84/84) |
| 22–28 (N = 43) | 100% (43/43) | 100% (43/43) | 100% (43/43) | 100% (43/43) | 100% (43/43) |
| > 28 (N = 97) | 96.9% (93/96) | 94.8% (92/97) | 98.9% (95/96) | 98.9% (95/96) | 98.9% (95/96) |
| Unknown (N = 96) | 97.9% (94/96) | 93.8% (90/96) | 97.9% (94/96) | 100% (96/96) | 100% (96/96) |
| Total sensitivity (N = 398) | 93.5% (371/397, 95% CI 90.6–95.7) | 92.2% (367/398, 95% CI 90.0–95.3) | 95.7% (380/397, 95% CI 93.2–97.5) | 98.0% (389/397, 95% CI 96.1–99.1) | 97.0% (385/397, 95% CI 94.5–98.2) |
| Total specificity | |||||
| Pre-pandemic controls (N = 510) | 99.6% (508/510) | 99.2% (506/510) | 100% (510/510) | 99.6% (508/510) | 97.2% (496/510) |
| Including cross-reactivity and interference (N = 683) | 99.7% (681/683, 95% CI 98.9–100) | 99.4% (679/683, 95% CI 98.5–99.8) | 100% (683/683, 95% CI 99.5–100) | 99.3% (678/683, 95% CI 98.3–99.8) | 97.5% (666/683, 95% CI 95.9–98.4) |
Abbreviations: CMIA, chemiluminescence microparticle immunoassay; CLIA, chemiluminescence immunoassay; CI, confidence interval; COI, cutoff index; ECLIA, electrochemiluminescence immunoassay; ELISA, enzyme-linked immunosorbent assay; NC, negative control; OD, optical density; RBD, receptor-binding domain; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2.
Performance evaluation of the five SARS-CoV-2 antibody assays
Sensitivity and specificity
The diagnostic sensitivity and specificity (and the 95% confidence intervals) of the five assays were calculated according to days after symptom onset using MedCalc software version 19.8 (MedCalc Software, Ostend, Belgium). The date of symptom onset was not available for some patients.
Cross-reactivity and interference
The potential cross-reactivity with other coronaviruses, such as SARS-CoV-1, MERS, respiratory viruses, and other viruses, bacteria, autoimmune diseases, and monoclonal gammopathy was evaluated using patient serum samples that were positive for antibodies against the relevant pathogen. For interference testing, Hb (10 mg/mL), bilirubin (0.4 ng/mL), a triglyceride mix (20 mg/mL), or zanamivir (10 mg/mL; all from Sigma-Aldrich) were spiked into serum samples that were negative for these antibodies. The results of spiked samples were compared with those of non-spiked samples and considered acceptable if the difference was <15%.
Agreement and correlation
The agreement and correlations between the results of the five SARS-CoV-2 antibody assays were calculated using MedCalc software and were reported as correlation graphs. For the Siemens Atellica IM SARS-CoV-2 Total assay, antibody titers <0.05 were considered 0, and those >10 were considered 10. The upper limits of detection values were determined to be 4.0 for the SD Biosensor assay and 4.5 for the LG Chem assay. Positive, negative, and total agreement between assays were evaluated using Cohen’s kappa statistics and categorized as poor (<0.00), slight (0.00–0.20), fair (0.21–0.40), moderate (0.41–0.60), substantial (0.61–0.80), or nearly perfect (0.81–1.00).
ROC curve analysis
ROC curve analyses and areas under the ROC curve (AUC) calculations for the five assays were performed using the MedCalc software. P≤0.05 was considered statistically significant. For each ROC curve, we recalculated the cutoff values so that the Youden index (=sensitivity+specificity–1) was maximized, and the sensitivity and specificity were determined from the calculated cutoff values.
PPVs and negative predictive values (NPVs) for individual and two combined assays using varying seroprevalence settings
PPVs and NPVs of individual and two combined assays using the calculated sensitivity and specificity values and the predictive seroprevalence of 10%, 5%, 2%, 1%, and 0.1% were calculated using a US Food and Drug administration online calculator [17].
RESULTS
Sensitivity and specificity
The diagnostic sensitivities were 93.5% for the Roche assay, 92.2% for the Abbott assay, 95.7% for the Siemens assay, 98.0% for the SD Biosensor assay, and 97.0% for the LG Chem assay. The diagnostic specificities were 99.7% for the Roche assay, 99.4% for the Abbott assay, 100% for the Siemens assay, 99.3% for the SD Biosensor assay, and 97.5% for the LG Chem assay (Table 1). The positivity rate was low in the first week after symptom onset but was nearly 100% three to four weeks after symptom onset. The SD Biosensor assay showed the highest sensitivity and the Siemens assay showed the highest specificity.
Cross-reactivity and interference
The Abbott, Roche, and Siemens assays showed negative cross-reactivity for 33 species of viruses and bacteria in 168 positive serum samples (Supplemental Data Table S1). The SD Biosensor and LG Chem assays each showed three false-positive results in the cross-reactivity study. No samples showed interference for Hb, a triglyceride mix, bilirubin, albumin, or zanamivir.
Agreement and correlation
The agreement among all assays was nearly perfect (kappa>0.90) (Table 2); however, correlations among the assays were non-linear (Supplemental Data Fig. S1), and several samples exceeded the upper limit of detection in the Siemens, SD Biosensor, and LG Chem assays. The Siemens and SD Biosensor assays showed the highest agreement rate at 98.7% (95% CI, 97.8–99.3; kappa, 0.966). Discordant results among the assays are shown in Supplemental Data Table S2.
Table 2.
Agreement rates between the five SARS-CoV-2 antibody assays
| A/B | Roche/ Abbott | Roche/ Siemens | Roche/SD Biosensor | Roche/ LG Chem | Abbott/ Siemens | Abbott/SD Biosensor | Abbott/LG | Siemens/ SD Biosensor | Siemens/LG Chem | SD Biosensor/ LG Chem |
|---|---|---|---|---|---|---|---|---|---|---|
| Positive/Positive (total number) | 364 | 368 | 371 | 371 | 363 | 366 | 366 | 380 | 378 | 385 |
| Positive/Negative (total number) | 9 | 5 | 2 | 2 | 7 | 4 | 4 | 0 | 2 | 17 |
| Negative/Positive (total number) | 6 | 12 | 23 | 31 | 17 | 28 | 36 | 14 | 24 | 9 |
| Negative/Negative (total number) | 700 | 694 | 683 | 675 | 692 | 682 | 674 | 685 | 675 | 669 |
| Positive agreement of A to B (%) (95% CI) | 98.4 (96.5–99.4) | 96.8 (94.5–98.4) | 94.2 (91.4–96.3) | 92. 3 (78.2–94.7) | 95.5 (92.9–97.4) | 92.9 (89.9–95.2) | 91.0 (87.8–93.6) | 96.5 (94.1–98.0) | 94.0 (91.2–96.1) | 97.8 (95.7-99.0) |
| Negative agreement of A to B (%) (95% CI) | 98.7 (97.6–99.4) | 99.3 (98.3–99.8) | 99.7 (98.9–100) | 99.7 (98.9–100) | 99.0 (97.9–99.6) | 99.4 (98.5–99.8) | 99.4 (98.5–99.8) | 100 (99.5–100) | 99.7 (98.9–100) | 97.5 (96.1-98.6) |
| Positive agreement of B to A (%) (95% CI) | 97.6 (95.5–98.9) | 98.7 (96.9–99.6) | 99.5 (98.1–99.9) | 99.5 (98.1–99.9) | 98.1 (96.1–99.2) | 98.9 (97.3–99.7) | 98.9 (97.3–99.7) | 100 (99.0-100) | 99.5 (98.1–99.9) | 95.8 (93.3-97.5) |
| Negative agreement of B to A (%) (95% CI) | 99.2 (98.2–99.7) | 98.3 (97.1–99.1) | 96.7 (95.2–97.9) | 95.6 (93.8–97.0) | 97.6 (96.2–98.6) | 96.1 (94.4–97.4) | 94.9 (93.0–96.4) | 98.0 (96.7–98.9) | 96.6 (94.9–97.8) | 98.7 (97.5-99.4) |
| Total agreement (%) (95% CI) | 98.6 (97.7–99.2) | 98.4 (97.5–99.1) | 97.7 (96.6–98.5) | 96.9 (95.7–97.9) | 97.8 (96.7–98.6) | 97.0 (95.8–98.0) | 96.3 (95.0–97.3) | 98.7 (97.8–99.3) | 97.6 (96.5–98.4) | 97.6 (96.5-98.4) |
| Kappa | 0.961 | 0.962 | 0.946 | 0.930 | 0.943 | 0.934 | 0.918 | 0.966 | 0.942 | 0.949 |
Abbreviations: CI, confidence interval; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2.
ROC curve and cutoff analysis
The AUCs calculated from ROC curve analysis were 0.976, 0.987, 0.984, 0.994, and 0.987 for the Roche, Abbott, Siemens, SD Biosensor, and LG Chem assays, respectively (Table 3). The optimal cutoff values, calculated using the Youden index method, were 0.188 cutoff index (COI) for the Roche assay, 0.44 index for the Abbott assay, 0.57 index for the Siemens assay, 0.404 OD for the SD Biosensor assay, and 1.163 S/CO (signal/cutoff) for the LG Chem assay, which differed from the original manufacturers’ cutoff values. Using the calculated cutoff values, the sensitivities of the Roche, Abbot, and Siemens assays increased (Roche, from 93.5% to 96.5%; Abbott, from 92.2% to 96.2%; Siemens, from 95.7% to 96.7%), whereas the specificities decreased (Roche, from 99.7% to 98.1%; Abbott, from 99.4% to 99.0%; Siemens, from 100% to 99.6%). By contrast, the sensitivities and specificities of the SD Biosensor and LG Chem assays did not significantly differ when using the manufacturers’ or calculated optimal cutoff values (Table 3).
Table 3.
ROC curve analysis and calculated cutoff values for the five SARS-CoV-2 antibody assays
| Roche | Abbott | Siemens | SD Biosensor | LG Chem | |
|---|---|---|---|---|---|
| AUC (P) | 0.976 (P < 0.001) | 0.987 (P < 0.001) | 0.984 (P < 0.001) | 0.994 (P < 0.001) | 0.987 (P < 0.001) |
| Manufacturer’s cutoff | 1.0 COI | 1.4 index | 1.0 index | (NC+0.3) OD | 1.0 S/CO |
| Sensitivity % (95% CI) according to the manufacturer’s cutoff | 93.5 (90.6–95.7) | 92.2 (90.0–95.3) | 95.7 (93.2–97.5) | 98.0 (96.1–99.1) | 97.0 (94.5–98.2) |
| Specificity % (95% CI) according to the manufacturer’s cutoff | 99.7 (98.9–100) | 99.4 (98.5–99.8) | 100 (99.5–100) | 99.3 (98.3–99.8) | 97.5 (95.9–98.4) |
| Cutoff calculated based on the Youden index | 0.19 COI | 0.44 index | 0.57 index | 0.40 OD | 1.16 S/CO |
| Sensitivity % (95% CI) according to the calculated cutoff | 96.5 (94.2–98.1) | 96.2 (93.9–97.9) | 96.7 (94.5–98.2) | 97.7 (95.7–99.0) | 96.7 (94.5–98.2) |
| Specificity % (95% CI) according to the calculated cutoff | 98.1 (96.8–99.0) | 99.0 (97.9–99.6) | 99.6 (98.7–99.9) | 99.4 (98.5–99.8) | 98.0 (96.6–98.9) |
Abbreviations: AUC, area under the curve; COI, cutoff index; NC, negative control; OD, optical density; S/CO, signal/cutoff; CI, confidence interval; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2.
PPVs and NPVs for individual and two combined assays using determined sensitivity, specificity, and seroprevalence
The lower the prevalence rate (from 10% to 0.1%), the lower is the PPV. The Siemens assay showed the highest specificity of 100% (95.2%; PPVs calculated using the lowest value of the 95% CI of the calculated specificity are shown in parentheses because the specificity was calculated as 100%, even at the lowest prevalence rate) among the five assays at a 10% prevalence and the highest specificity of 100% (15.1%) at a 0.1% prevalence (Table 4). When the predicted prevalence rate of 0.1% in Korea was considered, the PPV was as low as 24.2% for the Roche assay and 13.7% for the Abbott assay. However, the PPVs increased when two assays were both positive or when the orthogonal test algorithm (i.e., employing two assays in sequence when the first assay yields a positive result) [17] was used (Table 4).
Table 4.
PPVs when one assay or two combined assays were positive for five exemplary populations with 10%, 5%, 2%, 1%, and 0.1% SARS-CoV-2 prevalence
| PPV (%) when one assay or two combined assays are both positive* | |||||
|---|---|---|---|---|---|
|
| |||||
| SARS-CoV-2 10% prevalence | SARS-CoV-2 5% prevalence | SARS-CoV-2 2% prevalence | SARS-CoV-2 1% prevalence | SARS-CoV-2 0.1% prevalence | |
| Roche | 97.3 (†90.8) | 94.4 (†82.4) | 86.7 (†64.4) | 76.3 (†47.2) | 24.2 (†8.2) |
| Abbott | 94.6 (†87.4) | 89.3 (†76.6) | 76.0 (†56.0) | 61.6 (†38.6) | 13.7 (†5.9) |
| Siemens | 100 (†95.2) | 100 (†90.3) | 100 (†78.4) | 100 (†64.2) | 100 (†15.1) |
| SD Biosensor | 93.7 (†86.5) | 87.6 (†75.2) | 73.2 (†54.1) | 57.5 (†36.8) | 10.8 (†5.5) |
| LG Chem | 80.3 (†72.2) | 65.9 (†55.2) | 42.8 (†32.3) | 27.0 (†19.1) | 3.5 (†2.3) |
| Roche+Abbott | 100 (†99.8) | 100 (†99.7) | 99.9 (†99.1) | 99.8 (†98.2) | 98.1 (†84.7) |
| Roche+Siemens | 100 (†99.9) | 100 (†99.9) | 100 (†99.7) | 100 (†99.4) | 100 (†94.0) |
| Roche+SD Biosensor | 100 (†99.8) | 100 (†99.6) | 99.9 (†99.1) | 99.8 (†98.1) | 97.7 (†83.6) |
| Abbott+Siemens | 100 (†99.9) | 100 (†99.8) | 100 (†99.6) | 100 (†99.1) | 100 (†91.7) |
| Abbott+SD Biosensor | 100 (†99.7) | 99.9 (†99.5) | 99.8 (†98.7) | 99.5 (†97.3) | 95.5 (†78.2) |
| Siemens+SD Biosensor | 100 (†99.9) | 100 (†99.8) | 100 (†99.5) | 100 (†99.0) | 100 (†91.1) |
*Calculated using online calculators on the US Food and Drug administration [13]; †PPVs calculated using the lowest value of the 95% CI of the calculated specificity are shown in parentheses.
Abbreviations: CI, confidence interval; PPV, positive predictive particle; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2.
DISCUSSION
The sensitivities of the five assays were all >93%, and the sensitivities were low in the first week after symptom onset but were nearly 100% three to four weeks after symptom onset. These results are similar to or better than those in previous studies [10, 12-20]. In a previous study, some SARS-CoV-2 antibody assays showed a sensitivity of 100%; however, that study used only samples collected two weeks after symptom onset [17]. The assay sensitivity may vary across studies depending on how many of the samples that are collected in the first week after symptom onset (especially on days 0–3) are included. In this study, the SD Biosensor and LG Chem assays had higher sensitivity but lower specificity than the Roche, Abbott, and Siemens assays. The Roche, Abbott, and Siemens assays use high cutoff values to increase specificity, according to the CDC guidelines [21]. The sensitivity and specificity and the cutoff may vary depending on the purpose of the assay. If the assay is used for diagnostic screening, a high sensitivity is preferred. However, the specificity should be high when the assay is used to investigate seroprevalence or the effect of vaccination. Laboratories can adjust assay cutoff values according to the intended purpose.
We extensively investigated cross-reactivity using serum samples of patients with antibodies to various viruses and bacteria, autoimmune disease, or monoclonal gammopathy. The Roche, Abbott, and Siemens assays showed negative results for 163 samples tested for cross-reactivity, whereas the SD Biosensor and LG Chem assays showed false-positive results in three out of the 163 samples. These false-positive results seemed to be due to nonspecific reactions as only one sample yielded a positive result in each species. We found no cross-reactivity in serum with anti-SARS-CoV-1 antibody, which is considered the most similar to the anti-SARS-CoV-2 antibody, or in hypergammaglobulinemia serum.
This study was the first to estimate the PPVs of individual and two combined assays based on the orthogonal test algorithm, using the sensitivities and specificities calculated in this study and the expected prevalence in Korea. As the prevalence in Korea is expected to be <0.1% at present, if only one assay was used to determine the seroprevalence, the PPV was as low as 24.2% (8.2%) for the Roche assay, 13.7% (5.9%) for the Abbott assay, 100% (15.1%) for the Siemens assay, 57.5% (10.8%) for the SD Biosensor assay, and 3.5% (2.3%) for the LG Chem assay (Table 4). When two assays were combined, the PPV increased to >95% for all combinations.
We evaluated the performances of SARS-CoV-2 antibody assays available in Korea during September 2020, and the assay reagents were still in development during this period; hence, the sensitivity and specificity of all assays could be improved, and the cutoff values could be adjusted. Presently, other SARS-CoV-2 antibody assays have been developed and are available in Korea.
One limitation of this study is that neutralizing antibody assays that may represent personal immunity were not included [16]. Because most vaccines use the S protein as an immunogen, SARS-CoV-2 antibody assays targeting the N protein, such as the Abbott and Roche assays evaluated in this study, would not be useful to evaluate the vaccine response. However, the use of combinations of assays targeting the N and S proteins may allow discriminating natural infections from vaccinations.
In summary, this study was performed to select appropriate SARS-CoV-2 antibody assays for implementation in a large-scale seroprevalence study in Korea. We estimated PPVs of individual and two combined assays based on the orthogonal test algorithm, using calculated sensitivity and specificity and expected prevalence of COVID-19 in Korea. Because the prevalence of COVID-19 in Korea is considered exceptionally low at present, we recommend using a combination of two or more assays rather than a single assay.
Supplemental Materials
ACKNOWLEDGEMENTS
We appreciate Soyeon Yun for excellent technical support and data collection. The negative control samples used in this study were provided by the National Biobank of Korea, KDCA, and High-Risk Human Serum Bank (NRF-2017M3A9B8069581) at Chung-Ang University. The monoclonal gammopathy samples used in this study were provided by the Korea Institute of Radiological and Medical Sciences Radiation Biobank. MERS-CoV convalescent serum samples were kindly provided by Dr. Manki Song (International Vaccine Institute, Korea). Serum collection was funded by the Coalition for Epidemic Preparedness Innovations.
Footnotes
AUTHOR CONTRIBUTIONS
All authors have accepted responsibility for the entire content of this submitted manuscript and approved the submission. Kim HS designed the study, analyzed the data, and wrote the manuscript. Park Y and Hong KH analyzed the data and wrote the manuscript. Lee SK, Kim SH, Do HN, and Kim AR performed the assays and helped analyze the data. Hyun J, Oh EJ, Lee J, Lee H, Song SH, Kee SJ, Kwon GC, Lee JW, and Kim SS helped analyze the data and review the manuscript.
CONFLICTS OF INTEREST
No potential conflicts of interest relevant to this article were reported.
RESEARCH FUNDING
This research was supported by a fund (2020ER550900) from the KDCA.
REFERENCES
- 1.Zhu N, Zhang D, Wang W, Li X, Yang B, Song J, et al. A novel coronavirus from patients with pneumonia in China, 2019. N Engl J Med. 2020;382:727–33. doi: 10.1056/NEJMoa2001017. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.World Health Organization, author. [Updated on Jan 2021];Coronavirus disease (COVID-19) pandemic. https://www.who.int/emergencies/diseases/novel-coronavirus-2019 .
- 3.Gao Z, Xu Y, Sun C, Wang X, Guo Y, Ma K. A systemic review of asymptomatic infections with COVID-19. J Microbiol Immunol Infect. 2020;54:12–6. doi: 10.1016/j.jmii.2020.05.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Anand S, Montez-Rath M, Han J, Bozeman J, Kerschmann R, Beyer P, et al. Prevalence of SARS-CoV-2 antibodies in a large nationwide sample of patients on dialysis in the USA: a cross-sectional study. Lancet. 2020;396:1335–44. doi: 10.1016/S0140-6736(20)32009-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Clarke C, Prendecki M, Dhutia A, Ali MA, Sajjad H, Shivakumar O, et al. High prevalence of asymptomatic COVID-19 infection in hemodialysis patients detected using serologic screening. J Am Soc Nephrol. 2020;31:1969–75. doi: 10.1681/ASN.2020060827. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Pollán M, Pérez-Gómez B, Pastor-Barriuso R, Oteo J, Hernán MA, Pérez-Olmeda M, et al. Prevalence of SARS-CoV-2 in Spain (ENE-COVID): a nationwide, population-based seroepidemiological study. Lancet. 2020;396:535–44. doi: 10.1016/S0140-6736(20)31483-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Noh JY, Seo YB, Yoon JG, Seong H, Hyun H, Lee J, et al. Seroprevalence of anti-SARS-CoV-2 antibodies among outpatients in Southwestern Seoul, Korea. J Korean Med Sci. 2020;35:e311. doi: 10.3346/jkms.2020.35.e311. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Song SK, Lee DH, Nam JH, Kim KT, Do JS, Kang DW, et al. IgG Seroprevalence of COVID-19 among individuals without a history of the coronavirus disease infection in Daegu, Korea. J Korean Med Sci. 2020;35:e269. doi: 10.3346/jkms.2020.35.e269. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Cheng MP, Yansouni CP, Basta NE, Desjardins M, Kanjilal S, Paquette K, et al. Serodiagnostics for severe acute respiratory syndrome-related coronavirus 2: a narrative review. Ann Intern Med. 2020;173:450–60. doi: 10.7326/M20-2854. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Deeks JJ, Dinnes J, Takwoingi Y, Davenport C, Spijker R, Taylor-Phillips S, et al. Antibody tests for identification of current and past infection with SARS-CoV-2. Cochrane Database Syst Rev. 2020;6:CD013652. doi: 10.1002/14651858.CD013652. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Kontou PI, Braliou GG, Dimou NL, Nikolopoulos G, Bagos PG. Antibody tests in detecting SARS-CoV-2 infection: a meta-analysis. Diagnostics (Basel) 2020;10:319. doi: 10.3390/diagnostics10050319. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Mekonnen D, Mengist HM, Derbie A, Nibret E, Munshea A, He H, et al. Diagnostic accuracy of serological tests and kinetics of severe acute respiratory syndrome coronavirus 2 antibody: a systematic review and meta-analysis. Rev Med Virol. 2021;31:e2181. doi: 10.1002/rmv.2181. [DOI] [PubMed] [Google Scholar]
- 13.Hong KH, Lee SW, Kim TS, Huh HJ, Lee J, Kim SY, et al. Guidelines for Laboratory Diagnosis of Coronavirus Disease 2019 (COVID-19) in Korea. Ann Lab Med. 2020;40:351–60. doi: 10.3343/alm.2020.40.5.351. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Aoki K, Takai K, Nagasawa T, Kashiwagi K, Mori N, Matsubayashi K, et al. Combination of a SARS-CoV-2 IgG Assay and RT-PCR for Improved COVID-19 Diagnosis. Ann Lab Med. 2021;41:568–76. doi: 10.3343/alm.2021.41.6.568. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Kim IH, Kang BH, Seo SH, Park YE, Kim GJ, Lee SW, et al. Early Laboratory Preparedness of the Korea Disease Control and Prevention Agency and Response to Unknown Pneumonia Outbreak from Wuhan, China, in January 2020. Ann Lab Med. 2021;41:532–9. doi: 10.3343/alm.2021.41.6.532. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Yun S, Ryu JH, Jang JH, Bae H, Yoo SH, Choi AR, et al. Comparison of SARS-CoV-2 Antibody Responses and Seroconversion in COVID-19 Patients Using Twelve Commercial Immunoassays. Ann Lab Med. 2021;41:577–87. doi: 10.3343/alm.2021.41.6.577. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.US Food and Drug Administration, author. [Updated on Jan 2021];EUA authorized serology test performance. https://www.fda.gov/medical-devices/coronavirus-disease-2019-covid-19-emergency-use-authorizations-medical-devices/eua-authorized-serology-test-performance .
- 18.Irsara C, Egger AE, Prokop W, Nairz M, Loacker L, Sahanic S, et al. Evaluation of four commercial, fully automated SARS-CoV-2 antibody tests suggests a revision of the Siemens SARS-CoV-2 IgG assay. Clin Chem Lab Med. 2021;59:1143–54. doi: 10.1515/cclm-2020-1758. [DOI] [PubMed] [Google Scholar]
- 19.Schnurra C, Reiners N, Biemann R, Kaiser T, Trawinski H, Jassoy C. Comparison of the diagnostic sensitivity of SARS-CoV-2 nucleoprotein and glycoprotein-based antibody tests. J Clin Virol. 2020;129:104544. doi: 10.1016/j.jcv.2020.104544. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.National SARS-CoV-2 Serology Assay Evaluation Group, author. Performance characteristics of five immunoassays for SARS-CoV-2: a head-to-head benchmark comparison. Lancet Infect Dis. 2020;20:1390–400. doi: 10.1016/S1473-3099(20)30634-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Centers for Disease Control and Prevention, author. [Updated on Mar 2021];Interim guidelines for COVID-19 antibody testing. https://www.cdc.gov/coronavirus/2019-ncov/lab/resources/antibody-tests-guidelines.html .
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.