Herroelen 2020 [A].

Study characteristics
Patient Sampling	Purpose: A comparative analysis of analytical sensitivity was performed of seven commercial SARS‐CoV‐2 serology assays on 171 sera from 135 subjects with PCR‐confirmed SARS‐CoV‐2 infection, composed of 71 patients hospitalised for COVID‐19 pneumonia and 64 healthcare workers with paucisymptomatic infections. Specificity was verified on 57 pre‐pandemic samples. 2‐group study to estimate sensitivity and specificity for diagnosis of active disease/identification of previous disease Design: [1] subjects with PCR‐confirmed SARS‐CoV‐2 infection, composed of 71 patients hospitalised for COVID‐19 pneumonia and 64 healthcare workers with paucisymptomatic infections (n = 135 patients, 171 samples) [2] pre‐pandemic serum samples obtained from patients with PCR‐confirmed infection by other HCoV respiratory viruses (n = 7), other pathogens and viruses (n = 42) or presence of auto‐immune antibodies (n = 8) (n = 57 samples) [3] healthcare workers who presented WHO‐listed COVID‐19 symptoms but were not tested by PCR (n = 84, 84 samples) (this group not used in sensitivity/specificity analyses and not extracted. This group was also not mentioned in the published version) Recruitment: Unclear Prospective or retrospective: Unclear, but probably mixed. No informed consent from the hospitalised COVID‐19 patients (so likely serum samples already available = retrospective), but with written informed consent from participants with paucisymptomatic and suspected SARS‐CoV‐2 infections (so prospective). Pre‐pandemic samples = retrospective Sample size: 276 (135) patients with 312 (171) samples of which 228 (171) samples were included in this review (the excluded group [3] was not mentioned in the published version). Further detail: Not stated
Patient characteristics and setting	Setting: Hospital inpatient and home‐quarantined Location: Inpatients at AZ Delta General Hospital in Roeselare, Belgium Country: Belgium Dates: Inpatients = March 1 to April 27, 2020 ; healthcare workers unclear Symptoms and severity: 71/135 = inpatients admitted for severe COVID‐19 pneumonia ; PCR‐confirmed SARS‐CoV‐2 infections; very high level of suspicion of COVID‐19 pneumonia on chest CT (CO‐RADS score = 5) 64/135 = healthcare workers with PCR‐confirmed SARS‐CoV‐2 infection with mild (n = 61) or no (n = 3) WHO‐listed COVID‐19 symptoms: myalgia (present in 62.5%), fever (60.9%), dry cough (56.2%), dyspnoea (40.6%), severe fatigue (35.9%), headaches (30.0%), loss of smell or taste (26.6%) or diarrhoea (18.8%). These patients were home‐quarantined without the need for hospitalisation. Demographics: Inpatients = 48 males (median age 65 years, IQR 53‐80) and 23 females (median age 79 years, IQR 67‐86) Health care workers = Not reported Exposure history: Not stated Non‐Covid group 1: Pre‐pandemic cross‐reactivity Source: Pre‐pandemic Characteristics: PCR‐confirmed infection by other HCoV respiratory viruses (n = 7; HCoV 229E, n = 1; HCoV HKU1, n = 3; HCoV OC43, n = 2; HCoV OC43 + adenovirus, n = 1); other pathogens and viruses (n = 42); presence of auto‐immune antibodies (n = 8)
Index tests	Test name: [A] COVID‐19 IgG/IgM Rapid Test [B] Innovita 2019‐nCoV Ab Test [C] Wantai SARS‐COV‐2 Ab ELISA [D] Anti‐SARS‐CoV‐2 IgG and IgA assays [E] Anti‐SARS‐ CoV‐2‐NCP (IgG) assay [F] Elecsys Anti‐SARS‐CoV‐2 assay for Cobas e601 module [G] LIAISON SARS‐CoV‐2 S1/S2 IgG Manufacturer: [A] Zhejiang Orient Gene Biotech Co., Ltd., Zhejiang, China [B] Innovita Biological Technology Co., Ltd., Beijing, China [C] Beijing Wantai Biological Pharmacy Enterprise, Beijing, China [D] EUROIMMUN AG (a PerkinElmer Company, Luebeck, Germany) [E] EUROIMMUN AG (a PerkinElmer Company, Luebeck, Germany) [F] Roche Diagnostics, Basel, Switzerland [G] DiaSorin, Saluggia, Italy Antibody: [A] IgM and IgG antibodies to recombinant N‐ and S‐proteins [B] IgM and IgG antibodies to undisclosed SARS‐CoV‐2 epitopes [C] all antibody isotypes (IgM, IgA, IgG) against the RBD domain of the S1‐protein [D] IgA and IgG antibodies against the S1‐protein [E] IgG to the N‐protein [F] all antibody isotypes (IgM, IgA, IgG) against the N‐protein [G] IgG antibodies against S1/S2 proteins Antigen target: [A] recombinant N‐ and S‐proteins [B] undisclosed SARS‐CoV‐2 epitopes [C] RBD domain of the S1‐protein [D] S1‐protein [E] N‐protein [F] N‐protein [G] S1/S2 proteins Evaluation setting: [A] POC, assessed in laboratory [B] POC, assessed in laboratory [C] Laboratory test, assessed in laboratory [D] Laboratory test, assessed in laboratory [E] Laboratory test, assessed in laboratory [F] Laboratory test, assessed in laboratory [G] Laboratory test, assessed in laboratory Test method: [A] solid phase immunochromatographic assay [B] colloidal gold lateral flow assay [C] ELISA double‐antigen sandwich immunoassay [D] indirect ELISA [E] indirect ELISA [F] Electrochemiluminescence immunoassay (CLIA) [G] Electrochemiluminescence immunoassay (CLIA) Timing of samples: Inpatients = Serum samples ranged from 0 to 39 days after patient‐reported symptom onset. Healthcare workers = Serum samples ranged from 11 to 54 days after patient‐reported symptom onset < 10 days pso: 53/171 10‐20 days pso: 42/171 > 20 days pso: 76/171 Samples used: [A]‐[G] Serum Test operator: [A]‐[G] Laboratory personnel Definition of test positivity: [A] considered positive if a line was observed for either IgM, IgG or both [B] considered positive if a line was observed for either IgM, IgG or both [C] Samples with a cut‐off ratio (absorbance of the sample at 459 nm divided by 0.19 higher than 0.9 were considered positive, classifying gray zone results 0.9‐1.1 as positive. [D] cut‐off = 0.8 units, classifying gray zone results 0.8‐1.1 units as positive [E] cut‐off = 0.8 units, classifying gray zone results 0.8‐1.1 units as positive [F] cut‐off = 1 Cut‐off Index [G] cut‐off = 12 AU/mL, classifying gray zone results between 12 and 15 AU/mL as positive Blinding reported: Unclear Threshold predefined: All serology assays were used according to the manufacturers’ protocol using the cut‐offs specified.
Target condition and reference standard(s)	Reference standard: PCR: Allplex 2019‐nCoV assay (Seegene, Seoul, Korea) for E/N/RdRP genes on nasopharyngeal swab Threshold not reported Samples used: nasopharyngeal swab Timing of reference standard: Not stated Blinded to index test: Done prior index test Incorporated index test: No Definition of non‐COVID cases: Pre‐pandemic Samples used: Pre‐pandemic Timing of reference standard: Pre‐pandemic Blinded to index test: Yes Incorporated index test: No
Flow and timing	Time interval between index and reference tests: Not stated All patients received same reference standard: Yes Missing data: yes (specificity results for most tests for only 56 of 57 samples, also missing samples for sensitivity) Uninterpretable results: Not stated Indeterminate results: Not stated Unit of analysis: Samples
Comparative
Notes	Funding: This work was supported by a private donation by board members of Fagron (Nazareth, Belgium), a healthcare company, to RADar, the teaching and education initiative of AZ Delta General Hospital, to be used as unconditional research grant for data collection, collaborative collaboration and open access publication. The sponsor had no influence on the study design, data interpretation and drafting of the manuscript. Publication status: Pre‐print (not peer reviewed); now published Source: medRxiv preprint doi: https://doi.org/10.1101/2020.06.09.20124719 Journal (American Journal of Clinical Pathology) Author COI: The authors declared no conflict of interest. Not stated in published version
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Unclear
Was a case‐control design avoided?	No
Did the study avoid inappropriate exclusions?	Unclear
Did the study avoid inappropriate inclusions?	No
Could the selection of patients have introduced bias?		High risk
Are there concerns that the included patients and setting do not match the review question?			High
DOMAIN 2: Index Test (All tests)
DOMAIN 2: Index Test (Antibody tests)
Were the index test results interpreted without knowledge of the results of the reference standard?	Unclear
If a threshold was used, was it pre‐specified?	Yes
Could the conduct or interpretation of the index test have introduced bias?		Unclear risk
Are there concerns that the index test, its conduct, or interpretation differ from the review question?			Unclear
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	Yes
The reference standard does not incorporate the index test	Yes
Could the reference standard, its conduct, or its interpretation have introduced bias?		Low risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			High
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Unclear
Did all patients receive the same reference standard?	No
Were all patients included in the analysis?	Yes
Did all participants receive a reference standard?	No
Were results presented per patient?	No
Could the patient flow have introduced bias?		High risk