Coste 2021 [A].

Study characteristics
Patient Sampling	Purpose: Assessment of clinical performance of multiple tests for COVID‐19 diagnosis Design: Two‐group study estimating both sensitivity and specificity Group [1]: PCR‐confirmed COVID‐19 cases (n = 178); Group [2]: Pre‐pandemic controls (n = 404) [Some assays only had preliminary evaluation results for subgroup of 113 COVID‐19 samples and 69 non‐COVID samples] Recruitment: Unclear Prospective or retrospective: Retrospective Sample size: 582 (178); 182 (113) in preliminary evaluation Further detail: No more details available
Patient characteristics and setting	Setting: Unclear Location: Lausanne University Hospital Country: Switzerland Dates: Not stated Symptoms and severity: Not available Demographics: Not available Exposure history: Not available Non‐Covid group 1: Pre‐pandemic controls Source: Lab stocked samples obtained before Nov 1st, 2019 from patients with multiple infectious or autoimmune diseases. Characteristics: Other infections/conditions documented in supplementary table, including 129 nonspecific, 17 herpes simplex virus 1 and 2, 18 respiratory syncytial virus, 22 Epstein‐Barr virus, 33 cytomegalovirus, 27 mumps and/or measles virus, 14 parvovirus B19, 17 rubella virus, 45 influenza A, B or RSV, plus others (varicella‐zoster virus, human immunodeficiency virus, hepatitis virus A, B, C, D, and E, and some rheumatoid factors, or auto‐antibodies (anti‐PR3, ‐PR4, SCL70, SCL71)). Preliminary evaluation controls included 18 HCov, 12 lupus, and 39 nonspecific
Index tests	Test name: [A] EDI™ Novel Coronavirus COVID‐19 IgG ELISA Kit [B] EDI™ Novel Coronavirus COVID‐19 IgM ELISA Kit [C] Anti‐SARS‐CoV‐2 ELISA (IgG) [D] Anti‐SARS‐CoV‐2 ELISA (IgA) [E] SARS‐CoV‐2 NP IgG ELISA Kit [F] SARS‐CoV‐2 NP IgM ELISA Kit [G] COVID‐19 ELISA IgG [H] COVID‐19 ELISA IgM+IgA [I] SARS‐CoV‐2 IgG ELISA Kit [J] SARS‐CoV‐2 IgM ELISA Kit [K] 2019‐nCOV IgG/IgM Rapid Test [L] NADAL® COVID‐19 IgG/IgM Test [M] One Step Test for Novel Coronavirus (2019‐nCoV) IgM/IgG Antibody [N] ISON® SARS‐CoV‐2 IgG kit [O] MAGLUMITM 2019‐nCoV IgG + IgM [P] Elecsys anti‐SARS‐CoV‐2 Manufacturer: [A] Epitope Diagnostics, USA [B] Epitope Diagnostics, USA [C] EUROIMMUN AG, Germany [D] EUROIMMUN AG, Germany [E] Immunodiagnostics limited, Hong Kong [F] Immunodiagnostics limited, Hong Kong [G] Vircell, Spain [H] Vircell, Spain [I] Creative Diagnostics, USA [J] Creative Diagnostics, USA [K] Dynamiker, China [L] Nal Von Minden, Germany [M] Augurix Diagnostics, Switzerland/China [N] Diasorin, Italy [O] Snibe, China [P] Snibe, China [Q] Roche, Germany Antibody: [A] IgG [B] IgM [C] IgG [D] IgA [E] IgG [F] IgM [G] IgG [H] IgM, IgA [I] IgG [J] IgM [K] IgG, IgM [L] IgG, IgM [M] IgG, IgM [N] IgG [O] IgG, IgM [P] Total antibody Antigen target: [A] N‐protein [B] N‐protein [C] S1 domain of the spike‐protein [D] S1 domain of the spike‐protein [E] N‐protein [F] N‐protein [G] N and S‐proteins [H] N and S‐proteins [I] Whole virus lysate [J] N and S‐proteins [K] N‐protein [L] S‐protein [M] N and S‐proteins [N] S1 and S2 domains of the spike‐protein [O] N and S‐proteins [P] N‐protein Evaluation setting: [A] ‐ [J] and [N] ‐ [Q]: Lab tests; likely done in lab but not explicitly stated. [K] ‐ [M]: POC tests, unclear where they were performed. Test method: [A] ‐ [J]: Enzyme‐linked immunosorbent assay (ELISA) [K] ‐ [M]: Lateral flow assay [N] ‐ [O]: Chemiluminescence immunoassay (CLIA) [P]: Electrochemiluminescence immunoassay (ECLIA) Timing of samples: Obtained during the first 2 months post‐symptom onset. No more details available Samples used: Serum Test operator: Not stated Definition of test positivity: As per manufacturer (no more details available) Blinding reported: Not stated Threshold predefined: Yes, as per manufacturer
Target condition and reference standard(s)	Reference standard: RT‐PCR test (no more details available) Samples used: Not stated Timing of reference standard: Not stated Blinded to index test: Yes (done earlier) Incorporated index test: No Definition of non‐COVID cases: No testing (pre‐pandemic samples) Samples used: NA
Flow and timing	Time interval between index and reference tests: Not stated All patients received same reference standard: No (type of RT‐PCR unknown for cases; controls were pre‐pandemic samples) Missing data: Apparently no. However, only some tests were done on all samples. Selection for full evaluation was based on: i) sensitivity and specificity performance of the preliminary evaluation, ii) protein detected (anti‐N: ED IgG ELISA and Dynamiker IgG/IgM; anti‐S: Diasorin IgG CLIA, anti N+S: Snibe IgG/IgM CLIA) iii) availability of the kits on 15th April 2020 in Switzerland, iv) specific detection of IgG and/or IgM or IgA, and v) compatibility of the kits to most laboratory needs (e.g. median to low samples volumes per day and extended expiration days upon kits opening). "Despite its good performance, the ECLIA from Roche was selected as it detects pan‐180 Ig, which is not the most appropriate for infectious serology diagnostic". Uninterpretable results: None Indeterminate results: Yes, varied by test but the number of indeterminate results for each test was unclear due to contradictory numbers in supplementary tables [A] Epitope Diagnostics IgG: 0/178, 13/404 (full evaluation); 3/113, 4/69 (preliminary evaluation); more missing from preliminary evaluation compared to full [B] Epitope Diagnostics IgM: 12/178, 5/404 (full evaluation) [C] EUROIMMUN IgG: 8/113; 1/69 [D] EUROIMMUN IgA: 8/113; 5/69 [E] Immunodiagnostics limited, IgG: 1 extra sample reported for D+; 3/69 missing for [D]‐[F] Immunodiagnostics limited, IgM: 2 extra samples reported for D+; 2/69 missing for [D]‐ [G] Vircell, IgG: 3/113, 5/69 missing [H] Vircell, IgM+IgA: 7/113, 13/69 missing [I] and [J] Creative Diagnostics: 0 indeterminate [K] Dynamiker: preliminary evaluation 2/113 for IgG and 5/113 for IgM missing, but for full evaluation there was 1 extra sample reported for IgG (179 instead of 178), and only 2/178 missing for IgM (for disease‐negative there was 1/404 missing for IgG and 3/404 missing for IgM); more missing from preliminary evaluation compared to full [L] Nal Von Minden: IgG 6/113, 1/69 missing; IgM 7/113, 2/69 missing [M] Augurix Diagnostics IgM 8/113; 2/69; IgM 18/113; 1/69 [N] Diasorin, Italy: preliminary evaluation dataset showed 6/113 and 2/69 indeterminate; full evaluation showed 3/178 D+ missing but 5 extra results for D‐ (409 instead of 404); more missing from preliminary evaluation compared to full [O] Snibe, IgG: full evaluation 2/178, 1/404 missing; preliminary evaluation showed 6/113, 1/69 missing; more missing from preliminary evaluation compared to full Snibe, IgM: full evaluation 1/178, 2/404 missing; preliminary evaluation showed 5/113, 3/69 missing; more missing from preliminary evaluation compared to full [P] Roche pan‐IgG: 6/113, 2/69 missing Unit of analysis: Unclear ‐ referred to 'patients' but did not describe if 1 sample per patient
Comparative
Notes	Funding: None reported Publication status: Pre‐print article Source: Pre‐print server (medXriv) Author COI: None reported
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Unclear
Was a case‐control design avoided?	No
Did the study avoid inappropriate exclusions?	Unclear
Did the study avoid inappropriate inclusions?	Unclear
Could the selection of patients have introduced bias?		High risk
Are there concerns that the included patients and setting do not match the review question?			High
DOMAIN 2: Index Test (All tests)
DOMAIN 2: Index Test (Antibody tests)
Were the index test results interpreted without knowledge of the results of the reference standard?	Unclear
If a threshold was used, was it pre‐specified?	Yes
Could the conduct or interpretation of the index test have introduced bias?		Unclear risk
Are there concerns that the index test, its conduct, or interpretation differ from the review question?			Unclear
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	Yes
The reference standard does not incorporate the index test	Yes
Could the reference standard, its conduct, or its interpretation have introduced bias?		Low risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			High
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Unclear
Did all patients receive the same reference standard?	No
Were all patients included in the analysis?	No
Did all participants receive a reference standard?	Yes
Were results presented per patient?	Unclear
Could the patient flow have introduced bias?		High risk