Abstract
Objectives
Serologic testing for SARS-CoV-2 is an important element in the fight to slow the COVID-19 pandemic. This study aimed to validate two serologic tests for total (IgM, IgG, IgA) SARS-CoV-2 antibodies, (i) the Ortho-Clinical Diagnostics Anti-SARS-CoV-2 Total Antibody assay for the Vitros 5600 analyzers and (ii) a manual laboratory developed ELISA (FDA EUA pending), for use in parallel orthogonal testing of asymptomatic healthcare workers and affiliates of the University of Maryland Medical System.
Design and Methods
Validation and verification of the two tests was performed using samples from hospitalized patients that were found to be PCR positive for SARS-CoV-2, samples pre-COVID-19, and samples from individuals with current/previous infections with other viruses. Healthcare workers and affiliates from across the University of Maryland Health System were provided testing free of charge and their results were reported as reactive or non-reactive if the two tests were concordance, or indeterminate if the results were discordant.
Results
Validation testing found the Ortho Vitros test to be 100% (73/73) sensitive, and 99.3% (152/153) specific, while the UMMC ELISA was found to be 97.6% (204/209) sensitive and 100% (288/288) specific. Real world testing among 8399 healthcare workers found that 2.9% (247/8399) of healthcare workers were positive for anti- SARS-CoV-2 antibodies by both tests. An indeterminate rate of 1.1% (91/8399), in which one test reported reactive results, and one as non-reactive was also seen.
Conclusions
Parallel orthogonal testing improves the positive and negative predictive value of serologic testing in populations with low prevalence. The use of an indeterminate result from parallel orthogonal testing allows for the follow-up and re-testing, which helps resolve discrepancies between assays.
Keywords: COVID-19, SARS-CoV-2, Serology
1. Introduction
As the COVID-19 pandemic continues across the United States and the globe, serologic testing will play an important role in understanding the scope of the pandemic. While SARS-CoV-2 serologic testing is not intended to diagnose or rule-out infection, the results from sero-surveys are key to understanding the rate of exposure to SARS-CoV-2 in various populations including healthcare workers, first responders, elderly adults, and children[1].
Early on, the U.S. Food and Drug Administration (FDA) recognized the need for serological testing and allowed for rapid deployment of these tests to clinical labs. However, without regulations in place the market was flooded with serologic tests. It was quickly realized that not all tests were equal, and that many had significant limitations[2], [3]. This lead to a tightening of requirements, through Emergency Use Authorizations, which many manufacturers could not meet[2], [3]. Further, the presumed low prevalence in the overall US population limits the utility of these tests, as tests with specificities as high as 99% are needed to provide meaningful information at an individual level[4], [5], [6]. The U.S. Centers for Disease Control and Prevention (CDC) recently recommended an orthogonal testing approach to improve the positive predictive value of the tests, in an effort to minimize false positive results in populations with low prevalence[7].
The University of Maryland Medical System (UMMS) offered free serologic testing to all its employees and other healthcare workers affiliated with UMMS. UMMS decided to implement two different assays performed in tandem (parallel orthogonal approach) for testing of asymptomatic healthcare workers as recommended by the CDC. One test uses a commercial platform and the other a manual Enzyme Linked Immunosorbant Assay (ELISA), using a modified version of the test utilized by the CDC (Comm. Natalie Thornburg). The goal of the UMMS serologic testing initiative is to determine the overall prevalence of Anti-SARS-CoV-2 antibodies among UMMS healthcare workers, and to provide a high quality and reliable result to the individual healthcare worker.
The aims of this study are to describe the validation of two serological tests and findings when the parallel orthogonal testing was used in a large population of healthcare workers. To our knowledge, this is the first study to compare the utilization of two parallel tests, versus one test for classification of individuals as either reactive or non-reactive for anti-SARS-CoV-2 antibodies, in a large population of healthcare workers.
2. Methods:
2.1. Serologic testing methods
Samples were tested following the manufacturers’ instructions for the Ortho-Clinical Diagnostics VITROS Immunodiagnostic Products Anti-SARS-CoV-2 Total Reagent Pack using the Vitros 5600 platform (Ortho Vitros test). The Ortho Vitros test provides a result of reactive if the signal from the test sample divide by the signal at the cut-off is ≥ 1.00, called the (S/C).
Samples were also tested using the manual University of Maryland Medical Center Enzyme Linked Immunosorbant Assay (UMMC ELISA) developed using full length spike ectodomain protein provided by the CDC (comm. Natalie Thornburg; [8]. The coating concentration for the ELISA plates (Nucu Thermofisher) was 500 ng/ml. 100 ul of the 500 ng/ml spike protein was added to each well of the ELISA plate. ELISA plates were incubated at 4 °C overnight (and up to 3 days). Plates were washed with 300ul/well PBS-T (1xPBS + 0.05% tween 20) (Corning) three times. After washing, plates were blocked for two to four hours using 5% nonfat milk (Omniblok). Plates were washed three times with PBS-T between each subsequent step. 1) 100 ul of diluted patient samples were added to each well. Patient samples were diluted 1:100 in 5% non-fat milk. 2) Goat Anti-human IgG, IgA, IgM –HRP (Invitrogen) was diluted 1:10,000 in 5% milk and 100 ul was added to each well. 3) 100 ul of TMB substrate (KPL) was added to each well and incubated at RT for 10 min at which time 100 ul of 1 N sulfuric acid was added to each well to stop the reaction. Plates were read at 450 nm. Positive, Cut-off, and Negative controls were run in quadruplicate on each plate. Samples were considered reactive if the sample OD divided by the cut-off OD is ≥ 1.00, called the index value.
2.2. Validation testing
All validation testing was performed using residual samples that were de-identidied and unlikent to individual information. This activity was determined to be Not Human Subject Research by the Insitutional Review Board, University of Maryland Baltimore protocol #52988. The sensitivity of the assays was determined using remnant EDTA and lithium heparin plasma samples from patients that were determined to be SARS-CoV-2 positive by Polymerase Chain Reaction (PCR) testing at University of Maryland Medical System hospitals, the Maryland Department of Health, LabCorp, Quest, ARUP reference labs. PCR methodology included the CDC RT-PCR test method, lab developed tests at the reference labs, and automated commercial platforms including the Cepheid, BD, Abbott 2000 m, and GenMark. Samples for validation testing were collected between March and June of 2020. Specificity was determined using historical lithium heparin and EDTA plasma samples from normal healthy individuals collected in 2012, samples from SARS-CoV-2 PCR negative individuals, and samples from patients with other confirmed viral infections.
2.2.1. Ortho Vitros and UMMC ELISA validation samples
Seventy-three (73) remnant EDTA samples were collected from SARS-CoV-2 PCR positive patients at > 6 days since PCR positivity to evaluate the Ortho Vitros test and the UMMC ELISA. Additionally, 100 remnant EDTA plasma samples from normal individuals collected in 2012 were used to evaluate specificity on both the Ortho Vitros test and the UMMC ELISA tested for specificity. Finally, remnant lithium heparin samples were also used determine the specificity of these assays. 10 samples from patients with Anti-HIV antibodies, 9 samples from patients with Anti-HBV antibodies, 9 samples from patients with Anti-HCV antibodies, 10 samples from patients with Anti-CMV antibodies, 8 samples from patients with Anti-EBV antibodies, and 7 samples from patients that had recently had Coronavirus NL63 and 229E were tested for reactivity.
2.2.2. UMMC ELISA validation samples
The UMMC ELISA underwent further evaluation using remnant lithium heparin samples and EDTA samples (FDA Emergency Use Authorization pending). An additional 115 lithium heparin and 21 EDTA samples collected from patients with PCR confirmed SARS-CoV-2. The remnant samples were collected > 6 days from the date of the first PCR positive test to further investigate sensitivity. An additional, 100 remnant lithium heparin samples from normal individuals collected in 2020 were tested for specificity, along with 35 samples from confirmed SARS-CoV-2 PCR negative individuals.
2.3. Sero-survey, asymptomatic healthcare worker testing
Free antibody testing was offered to all employees and affiliates of the University of Maryland Health System (UMMS) starting June 3rd 2020and all testing was completely voluntary. Samples were collected in EDTA plasma separator tubes (BD, Sparks MD) and sent to the UMMC main laboratory. Samples were run on both the UMMC ELISA and the Vitros 5600 analyzers. Results that agreed on the UMMC ELISA and the Vitros were reported as either reactive or non-reactive. Discordant results were reported as indeterminate and were asked to return in 14–21 days for follow-up testing.
2.4. Statistics
Confidence intervals (95%) for sensitivity and specificity were calculated using the Wilson score interval. Apparent and estimated true prevalence calculations, positive predictive values, negative predictive values were calculated using Epitools. Estimated true prevalence was calculated based on the sensitivity, specificity, and concordance/ discordance of the two tests. Wilson score interval was used for calculating 95% confidence intervals and Bayesian estimate of true prevalence was used for the parallel testing. [9]
3. Results
3.1. Validation studies
The Ortho-Clinical Diagnostics VITROS Immunodiagnostic Products Anti-SARS-CoV-2 Total Reagent Pack and Calibrator (Ortho Vitros test) run on the Vitro 5600 was reactive for 73/73 EDTA plasma samples collected from SARS-CoV-2 PCR positive patients, >6 six days from PCR positivity (Table 1 A). Specificity studies were conducted using lithium heparin samples (Table 2 ). Results were non-reactive for 10/ 10 samples from patients with Anti-HIV antibodies, 9/9 samples from patients with Anti-HBV antibodies, 9/9 samples from patients with Anti-HCV antibodies, 10/10 samples from patients with Anti-CMV antibodies, 8/8 samples from patients with Anti-EBV antibodies, and 7/7 samples from patients that had recently had Coronavirus NL63 (Table 2). Of pre COVID-19 samples from healthy individuals (2012), the results were non-reactive for 99/100 samples (Table 1A). The sensitivity and specificity are 100% (CI:95–100%) (>6 days since PCR positivity) and 99.3% (CI:96.2–99.9%), respectively (Table 1A).
Table 1.
Validation Studies.
| SARS-CoV-2 PCR Positive | SARS-CoV-2 PCR Negative | |
|---|---|---|
| ||
| Ortho Vitros Test Positive | 73 | 1 |
| Ortho Vitros Test Negative | 0 | 152 |
| Total | 73 | 153 |
| Sensitivity | 73/73; 100% (CI: 93.8–100) | |
| Specificity | 152/153; 99.3 (CI: 95.9–100) | |
| SARS-CoV-2 Positive | SARS-CoV-2 Negative | |
|---|---|---|
| ||
| UMMC ELISA Positive | 204 | 0 |
| UMMC ELISA Negative | 5 | 288 |
| Total Samples | 209 | 288 |
| Sensitivity | 204/209; 97.6% (CI: 94.2–99.1) | |
| Specificity | 288/288; 100% (CI: 98.4–100) | |
Table 2.
Cross-Reactivity Testing.
| Positive Test | Number of Samples |
Results |
|
|---|---|---|---|
| Ortho Vitros Test | UMMC ELISA | ||
| Anti-HCV IgG, Immunoassay | 9 | 0/9 | 0/9 |
| Anti-HBV IgG, Immunoassay | 9 | 0/9 | 0/9 |
| Anti-CMV IgG, Immunoassay | 10 | 0/10 | 0/10 |
| Anti-229E (alpha coronavirus), PCR | 1 | 0/1 | 0/1 |
| Anti-NL63 (alpha coronavirus), PCR | 7 | 0/7 | 0/7 |
| Anti-EBV IgG, Immunoassay | 8 | 0/8 | 0/8 |
| Anti-HIV, Immunoassay | 10 | 0/10 | 0/10 |
| PUIa Ruled-Out | 35 | Not Performed | 0/35 |
PUI = Person Under Investigation
The UMMC ELISA had 92/94 EDTA samples and 112/115 lithium heparin plasma samples from PCR positive patients that were reactive (Table 1B). The UMMC ELISA was non-reactive for 10/ 10 samples from patients with Anti-HIV antibodies, 9/9 samples from patients with Anti-HBV antibodies, 9/9 samples from patients with Anti-HCV antibodies, 10/10 samples from patients with Anti-CMV antibodies, 8/8 samples from patients with Anti-EBV antibodies, and 7/7 samples from patients that had recently had Coronavirus NL63 (Table 2). The results were non-reactive for 100/100 EDTA and 100/100 Lithium Heparin samples collected in 2012 and for 35/35 confirmed PCR- samples from 2020 (Table 1B; Table 2). The UMMC ELISA was found to be 97.6% (CI: 94.5–99.0) sensitive for samples greater than > 6 days since first PCR positive sample and 100% (CI: 98.4–100%) specific overall. (Table 1B)
The five patients with negative UMMC ELISA results (2/94 EDTA and 3/115 lithium heparin), but PCR confirmed SARS-Cov-2 infection were investigated further by testing additional samples from later time points during their hospitalization. Two of the five patients became reactive when later samples were tested. Two patients remained non-reactive, one patient was on immunosuppressive therapy and one patient never seroconverted on repeat sampling up to day 13-post initial PCR positive sample, at which point the patient was discharged. The remaining patient died before additional samples could be collected.
Concordance between the UMMC ELISA and the Ortho Vitros Test was investigated by testing 73 EDTA plasma samples from PCR positive individuals, 100 EDTA plasma samples from healthy individuals (collected in 2012), and 53 lithium heparin plasma samples from individuals with other viral infections or antibodies targeting other infectious diseases using both platforms. The UMMC ELISA and the Ortho Vitro test had an overall agreement of 98.8% (224/226), 99% (152/153) for the negative samples, and 98.6% (72/73) for the PCR positive samples. When taking into account later seroconversion the UMMC ELISA and the Ortho Vitros test had 100% agreement for the PCR positive samples. (Table 3 )
Table 3.
Concordance Between Ortho Vitros Test and UMMC ELISA.
| Ortho Vitros Test Positive | Ortho Vitros Test Negative | |
|---|---|---|
| UMMC ELISA Positive | 72 | 0 |
| UMMC ELISA Negative | 2a | 152 |
| Concordance | 224/226; 98.8% (CI: 96.8–99.8) | |
The Ortho Viros test had one false positive (sample was pre-2019), while the UMMC ELISA had one false negative (Sample was from a PCR + individual). A later sample from this patient was positive on the UMMC ELISA
3.2. Sero-survey, asymptomatic healthcare workers
8399 healthcare workers from across UMMS facilities were tested for antibodies directed at SARS-CoV-2 using both the UMMC ELISA test and the Ortho Vitros test between June 3, 2020 and July 24, 2020.
The testing resulted in an apparent prevalence (positive results on both tests) of 2.9% [95% CI: 2.6–3.3] for Anti-SARS-CoV-2 antibodies. Separately, the apparent prevalence of Anti-SARS-CoV-2 antibodies was 3.8% [95% CI: 3.4–4.2] and 3.2% [CI:2.8–3.5] for the Ortho Vitro test and the UMMC ELISA, respectively. Estimated true prevalence (prevalence calculated from the sensitivity and specificity of the tests) for the orthogonal testing was 3.5% [95% CI: 3.0–4.0]. The estimated true prevalence for the Ortho Vitros, and UMMC ELISA tests individually were found to be 3.1 [95% CI: 2.7–3.6] and 3.2 [95% CI: 2.9–3.6], respectively. Prevalence was calculated using results after follow-up testing for indeterminate individuals. (Table 4 )
Table 4.
Prevalence of Anti-SARS-CoV-2 antibodies in Healthcare workers.
| Combineda | Ortho Vitros Test | UMMC ELISA | |
|---|---|---|---|
| Sample Size | 8399 | 8399 | 8399 |
| Number Positive | 247 | 320 | 265 |
| Apparent Prevalence | 2.94% (CI: 2.6–3.332) | 3.81 (CI: 3.42–4.24) | 3.16 (CI: 2.8–3.55) |
| Estimated True Prevalence | 3.52% (CI: 3.01–3.98) | 3.13 (CI: 2.73–3.57) | 3.23 (CI: 2.86–3.64) |
| Positive Predictive Value | 100.0% | 82.2% | 100.0% |
| Negative Predictive Value | 100.0% | 100.0% | 99.9% |
Combined: Only samples with positive results from the Ortho Vitros Test and UMMC ELISA are used for prevalence calculations and Positive Predictive value. To be considered negative results from both the Ortho Vitros Test and UMMC ELISA must be negative. These results are used for negative predictive value calculations.
Orthogonal testing method resulted in an initial indeterminate rate of 1.3% (107/8399). Eighty-three (83) samples were reactive with the Ortho Vitros test but non-reactive by the UMMC ELISA, while 24 were reactive on the UMMC ELISA but non-reactive by the Ortho Vitros test. The one hundred and seven (1 0 7) individuals with discordant results were asked to return for follow-up testing in 2–3 weeks and thirty-six (36) individuals returned for follow-up testing. Twenty (20) individuals were repeated as indeterminate; results of the remaining 16 were concordant on repeat testing, either positive by both tests or negative by both tests. (Table 5 ) The overall, negative, and positive concordance for the two tests (after repeat testing), in the healthcare workers, was 98.9%, 98.8%, and 73.1%, respectively. (Table 6 ).
Table 5.
Indeterminate Results: If results from the UMMC ELISA test and Ortho Vitros tests were discordant a result of indeterminate was given. Individuals with indeterminate results were asked to return for follow-up testing 14–21 days later.
| Indeterminate at First Test | Repeat Testing |
||
|---|---|---|---|
| Initial Results | Repeat Results | ||
| Number of Indeterminate Samples | 107/8399 (1.27%) | 107/8399 (1.27%) | 91/8399 (1.08%) |
| Number of Samples Re-Tested | N/A | 36/107 (33.6%) | 36/107 (33.6%) |
| Ortho Vitros Test Positive & UMMC ELISA negative | 82/107 (76.6%) | 27/36 (75%) | 18/36 (50%)a |
| Ortho Vitros Test negative & UMMC ELISA Positive | 25/107 (23.4%) | 9/36 (25%) | 2/36 (5.5%)a |
| Ortho Vitros Test Negative & UMMC ELISA Negative | N/A | 10/36 (27.8%)b | |
| Ortho Vitros Test Positive & UMMC ELISA Positive | 6/36 (16.7%)c | ||
There was no change in results from initial and repeat testing
3 individuals were initially positive by the Ortho Vitros Test and 7 were initially positive by the UMMC ELISA.
All were initially negative by UMMC ELISA and positive by Ortho Vitros test
Table 6.
Concordance between UMMC ELISA and Ortho Vitros test for healthcare workers.
| UMMC ELISA |
|||
|---|---|---|---|
| Reactive | Non-reactive | ||
| Ortho Vitros Test | Reactive | 247 | 18 |
| Non– reactive | 73 | 8061 | |
| Positive Agreement | 73.1% (247/ 338) | ||
| Negative Agreement | 98.9% (8061/8152) | ||
| Overall Concordance | 98.9% (8308/8399) | ||
Individuals with initial indeterminate testing but repeat concordant results were counted as concordant.
A comparison of the Ortho Vitros test raw data (S/C values > 1.00 are reactive) indicates that there is 100% positive agreement between the UMMC ELISA and the Ortho Vitros test at S/C values above 260 and minimal agreement between the UMMC ELISA and the Ortho Vitros test at S/C values below 12; 1/53 agreed.
The negative and positive predicative values for orthogonal testing are 100%, while for the Ortho test alone the positive predictive value is only 82.2%, with a 100% negative predictive value. The UMMC ELISA alone has positive and negative predictive value of 100% and 99.9%, respectively. The predicative values are based on the estimated true prevalence.
4. Discussion
The testing design employed for the Ortho Vitros test and the UMMC ELISA indicated high sensitivity and specificity for both tests as well as significant agreement between the two tests in patients hospitalized with SARS-CoV-2 infections. Given the significant agreement and high sensitivity and specificity seen during the validation and verification of these two tests it was expected that these tests would both provide sensitive and accurate results when utilized in testing a large population of asymptomatic healthcare workers. The orthogonal testing is recommended for use in low prevalence population and it was thought that this testing approach would result in a high agreement rate between the two tests for the healthcare workers, as was seen with the validation testing using patient samples. It was important to provide highly accurate results to the healthcare workers since the presence of antibodies to SARS-CoV-2 indicate exposure to individuals with COVID-19.
At first look, an initial indeterminate rate of 1.3% for this population, seemed like a significant disagreement between the tests. Twenty- six percent (26%; 83/320) of all the reactive results from the Ortho Vitros test were reported as non-reactive on the UMMC ELISA, and only 9% (24/265) of the reactive UMMC ELISA results reported as negative on the Ortho Vitros test. Although, both tests target the Spike protein of SARS-CoV-2, however methodology differs significantly, as the Ortho test uses chemiluminescent tagged antigen and the UMMC ELISA test is a classic indirect ELISA. A determination was made that the tests would be independent, therefore some amount of discordance was expected, and would help reduce both the false positive and false negative rate. Investigation into the indeterminate results and the return of 34% of these individuals provides evidence that parallel orthogonal testing, with follow-up, minimizes the number of false positive results, and allows false negative results to be corrected thereby, providing meaningful information at the individual level.
Validation studies are designed and conducted with the intention of identifying and quantifying testing limitations. All tests have limitations and when sampling a large population discordant results are expected. Taking the 247 concordantly reactive results for the UMMC ELISA and Ortho Vitros as true positive outcomes and the 8061 non-reactive results on both tests as true negative results, the 91 indeterminate results represent the sum of falsely positive and falsely negative results for these tests. Given that the 95% confidence intervals for sensitivity and specificity with the Ortho Vitros test are 93.8–100% and 95.9–100%, respectively, an estimated 0–16 false negatives and 0–345 false positive results would be expected for this cohort. Further, the 95% confidence intervals for the UMMC ELISA sensitivity and specificity are 94.2–99.1% and 98.4–100%, respectively, resulting in and expectation of 2–15 false negative and 0–131 false positive results. These data indicate that 91 or 1.1% (91/8399) indeterminate results would not be unexpected when sampling a large population as was tested here.
The larger number of reactive results for the Ortho Vitros test could be due to lower specificity as compared to the UMMC ELISA, but may also be due to the lower sensitivity of the UMMC ELISA. Evidence of the latter is seen by the seroconversion of six individuals with indeterminate results for which the UMMC ELISA was initially negative (Table 5). Additionally, the sensitivity of the UMMC ELISA may be even lower for asymptomatic individuals. A rapid decrease in antibody titers has been seen in individuals with mild COVID-19[10]. Further, there is evidence that asymptomatic individuals lack a robust immune response to SARS-CoV-2 infection[11]. Both studies suggest that validation with samples from hospitalized patients may have resulted in an overestimation of the sensitivity of many serologic tests. Further, the raw S/C values of the Ortho Vitros test show 100% agreement with the UMMC ELISA at high values and almost no agreement at values below 12. It is likely at levels below an S/C of 12 there are true positive results that the UMMC ELISA is missing due to a lack of sensitivity, but it is also likely that many of these are false positive results for the Ortho Vitros test.
While it is difficult to make a determination as to which individual result is a false positive or a false negative, the indeterminate classification and the subsequent follow up helps resolve this issue. Follow up testing for 36 individuals allowed for the indeterminate status to be resolved for 44.5% of these cases.
The data from the validation study suggests that false positive results are likely minimized by the classification of samples as indeterminate, as the reporting of indeterminate is skewed heavily in the direction of non-reactive on the UMMC ELISA and reactive on the Ortho Vitros. A limitation to this study is that both the Ortho Vitros test and the UMMC ELISA were designed using samples from hospitalized SARS-CoV-2 PCR positive patients and samples from healthy individuals. The sensitivity and specificity of the tests are representative of those groups. It is possible that the results would be different had the tests been designed and validated using samples from patients with mild or asymptomatic disease. The cut-off optical density for the UMMC ELISA was determined based on a representative set of negative samples to maximize specificity, which likely resulted in a decrease in the sensitivity especially for asymptomatic individuals. However adjustment of the cut-off for the UMMC ELISA would result in additional false positives with minimal increase in true positives.
Orthogonal testing improved the positive and negative predictive values and ensured that the reactive and non-reactive results being reported are true results. The indeterminate classification and the repeat testing allowed for the correction of results when the sensitivity threshold of the UMMC ELISA wasn’t met initially, thereby correcting the limitation of decreased sensitivity due to use of the UMMC ELISA, while minimizing the reporting of false positives due to the Ortho Vitros test. This indicates that parallel orthogonal testing and additional follow-up of discordant results is beneficial and provides highly accurate results for the individual as well as for overall prevalence studies.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
References
- 1.Lerner A.M., Eisinger R.W., Lowy D.R., Petersen L.R., Humes R., Hepburn M., Cassetti M.C. The COVID-19 Serology Studies Workshop: Recommendations and Challenges. Immunity. 2020;53(1):1–5. doi: 10.1016/j.immuni.2020.06.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.The Food and Drug Administration. “EUA Authorized Serology Test Performance”. (2020) https://www.fda.gov/medical-devices/emergency-situations-medical-devices/eua-authorized-serology-test-performance.
- 3.Abbasi J. The Promise and Peril of Antibody Testing for COVID-19. JAMA. 2020;323(19):1881. doi: 10.1001/jama.2020.6170. [DOI] [PubMed] [Google Scholar]
- 4.Rosenberg, Eli S., et al. “Cumulative incidence and diagnosis of SARS-CoV-2 infection in New York.” medRxiv (2020). [DOI] [PMC free article] [PubMed]
- 5.Bendavid E. “COVID-19 Antibody Seroprevalence in Santa Clara County. California.“ MedRxiv. 2020 doi: 10.1093/ije/dyab010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Sood N., Simon P., Ebner P., Eichner D., Reynolds J., Bendavid E., Bhattacharya J. Seroprevalence of SARS-CoV-2–Specific Antibodies Among Adults in Los Angeles County, California, on April 10-11, 2020. JAMA. 2020;323(23):2425. doi: 10.1001/jama.2020.8279. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.The Centers for Disease Control and Prevention. “Interim Guidelines for COVID-19 Antibody Testing” (2020) https://www.cdc.gov/coronavirus/2019-ncov/lab/resources/antibody-tests-guidelines.html.
- 8.Wrapp D., Wang N., Corbett K.S., Goldsmith J.A., Hsieh C.-L., Abiona O., Graham B.S., McLellan J.S. Cryo-EM structure of the 2019-nCoV spike in the prefusion conformation. Science. 2020;367(6483):1260–1263. doi: 10.1126/science.abb2507. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Sergeant E.S.G. Epitools Epidemiological Calculators. Ausvet. 2018 [Google Scholar]
- 10.Ibarrondo F.J., Fulcher J.A., Goodman-Meza D., Elliott J., Hofmann C., Hausner M.A., Ferbas K.G., Tobin N.H., Aldrovandi G.M., Yang O.O. Rapid Decay of Anti–SARS-CoV-2 Antibodies in Persons with Mild Covid-19. N Engl J Med. 2020;383(11):1085–1087. doi: 10.1056/NEJMc2025179. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Ko, Jae-Hoon, et al. “Neutralizing Antibody Production in Asymptomatic and Mild COVID-19 Patients, in Comparison with Pneumonic COVID-19 Patients.” Journal of Clinical Medicine 9.7 (2020): 2268. [DOI] [PMC free article] [PubMed]