PHE 2020 [A].

Study characteristics
Patient Sampling	Design: Multi‐group study to assess sensitivity and specificity [1] Convalescent Covid patients (the total N samples used across all assay evaluations was not fully clear, however the overlap in samples between assays was provided. For each assay, numbers were made up to near 100 from the following sources: the Royal Free Hospital (RFH, n = 14), Basingstoke Hospital (n = 26) and the Porton Down laboratory (n = 4). [2] Non‐Covid patients (n = 499) [2a] historic negative samples (n = 399 per assay) [2b] cross‐reactive samples (n = 100 per assay unless otherwise stated) Recruitment: Not described; appeared to be convenience Prospective or retrospective: Retrospective Sample size: Total number unclear: [A] 592 (93) samples [B]‐[E] 599 (100) samples Further detail: [1] PCR‐confirmed Covid cases with sufficient volume of serum to cover multiple assays [2a] Historical serum samples collected before December 2019 [2b] Confounder serum samples collected before December 2019 Exclusion: [A] 7 sample results were removed post‐testing/post‐analysis as these were found to be PCR‐negative. No other exclusions stated
Patient characteristics and setting	Setting: Mainly community cases, very few admitted to hospital and those that were may have only been admitted for isolation during the containment phase Location: GPs in the community (FF 100 study, n = 82), Royal Free Hospital (RFH, n = 14), Basingstoke Hospital (n = 26) and the Porton Down laboratory (n = 4) Country: UK Dates: Date of evaluations: 5 April ‐ 14 July 2020 Samples collected before late April 2020 Symptoms and severity: Mostly mild disease (representative of the general population) [B] Samples were taken from patients with a range of disease severities Demographics: Not available for 14 positive samples from RFH Age range 10‐ > 64 years. 10‐24 years: 6‐8 samples 25‐34 years: 3‐6 samples 35‐44 years: 15‐17 samples 45‐54 years: 20‐22 samples 55‐64 years: 22‐27 samples > 64 years: 7‐28 samples Exposure history: Not stated. Non‐Covid group 1: [2a] Pre‐pandemic controls Source: Before December 2019, from existing reference panels at SEU, Manchester or Porton and Colindale. Characteristics: No confounder disease Non‐Covid group 2: [2b] Cross‐reactivity controls Source: Before December 2019, from SEU, Manchester or RIPL 2015 Lyme disease‐negative sample collection from Porton. Detail per assay: [A] 50 from SEU (12 RF, 6 CMV, 19 EBV, 13 VZV); 50 from RIPL 2015 Lyme disease‐negative sample collection; 399 pre‐pandemic [B] 351 samples CMV, EBV or VZV positive (no further details); 11 seasonal hCoV positive; 395 pre‐pandemic [C] 50 from SEU (12 RF, 6 CMV, 19 EBV, 13 VZV); 35 from RIPL 2015 Lyme disease‐negative sample collection; 387 pre‐pandemic [D] 49 from SEU (12 RF, 6 CMV, 19 EBV, 12 VZV); 50 from RIPL 2015 Lyme disease‐negative sample collection; 391 pre‐pandemic [E] 50 from SEU (12 RF, 6 CMV, 19 EBV, 13 VZV); 50 from RIPL 2015 Lyme disease‐negative sample collection; 399 pre‐pandemic (114 from PHE Immunoassay Group reference panel; 285 from SEU)* [F] to [H] appear to have used the same samples (all reported numbers were the same): 50 from SEU (12 RF, 6 CMV, 19 EBV, 13 VZV); 50 from RIPL 2015 Lyme disease negative sample collection; 399 pre‐pandemic (313 from PHE Immunoassay Group reference panel; 86 from SEU)
Index tests	[A] Euroimmun anti‐SARS‐CoV‐2 ELISA (IgG) serology assay (EI 2606‐9601 G) [B] Abbott SARS‐CoV‐2 IgG kit (Architect i2000SR system) (reagent batch number 16253FN00, exp date 16/07/2020) [C] Elecsys Anti‐SARS‐CoV‐2 assay (Lot 49025901, exp 31/05/20) [D] VITROS Immunodiagnostic Products anti‐SARS‐CoV‐2 IgG assay [E] Siemens Atellica‐IM SARS‐CoV‐2 Total (COV2T) serology assay (batch no. 11206711, exp 2021‐05‐12) [F] Ortho Clinical VITROS Anti‐SARS‐Cov‐2 Total Ab [G] LIAISON SARS‐CoV‐2 S1/S2 IgG serology assay [H] Beckman Coulter Access SARS‐CoV‐2 IgG Assay Manufacturer: [A] Euroimmun Medizinische Labordiagnostika AG [B] Abbott [C] Roche [D] Ortho Clinical Diagnostics [E] Siemens Healthcare GmbH [F] Ortho Clinical Diagnostics [G] DiaSorin S.p.A [H] Beckman Coulter Antibody: [A], [B], [D], [G], [H] IgG [C], [E], [F] Total antibody Antigen target: [A] S1‐protein [B] N‐protein [C] N‐protein [D] S‐based [E] Recombinant antigen [F] S1‐protein [G] S1 and S2‐protein [H] RBD of S1‐protein Evaluation setting: Laboratory used in laboratory Test method: [A] ELISA [B] CMIA [C] ECLIA [D] CLIA [E TO H] CLIA Timing of samples: variable; e.g. for the EUROIMMUN assay the interval pso was known for 79/93 samples; for 14/93 the interval was measured from when the patient was admitted to hospital to sample collection date (making the interval artificially low compared to actual time pso). Vast majority of samples across all evaluations was > 21 d pso, e.g. for EUROIMMUN, 75/93 were > 21 d [Data for <= 10 d was not included in the review because of lack of accurate sample timing] Samples used: Serum Test operator: Skilled research scientists in PHE Porton Down laboratory Definition of test positivity: [A] Ratio < 0.8 negative, >= 0.8 to < 1.1 borderline, >= 1.1 positive (borderline considered negative) [B] S/C < 1.4 negative, ≥ 1.4 positive [C] COI; signal sample/cut‐off < 1.0 negative, ≥ 1.0 positive [D] Signal for test sample/signal at cut‐off (cut‐off value) < 1.0 negative, ≥ 1.0 positive [E] < 1.0 index negative, >= 1.0 index positive [F] S/C < 1.0 negative; S/C >= 1.0 positive [G] < 12.0 AU/mL negative, 12.0 <= x < 15.0 AU/mL equivocal, >= 15.0 AU/mL positive. [H] <= 0.80 S/CO negative, > 0.80 to < 1.00 equivocal, >= 1.0 S/CO positive Blinding reported: Yes Threshold predefined: Yes, according to manufacturer
Target condition and reference standard(s)	Reference standard: RT‐PCR, threshold not stated Samples used: swab sample Timing of reference standard: Not stated Blinded to index test: Yes, prior Incorporated index test: No Definition of non‐COVID cases: Pre‐pandemic Samples used: NA, pre‐pandemic Timing of reference standard: pre‐pandemic Blinded to index test: yes, prior Incorporated index test: no
Flow and timing	Time interval between index and reference tests: Not stated All patients received same reference standard: no Missing data: yes, not all samples used for all test evaluations [E] 8 samples that did not yield results were excluded from the analysis. [A] [B] [C] [D] no exclusions Uninterpretable results: No results were excluded as uninterpretable [A] [B] [C] [D] no exclusions. [E] 8 samples that did not yield results were excluded from the analysis. Indeterminate results: [B] 3 equivocal results classed as negative for sensitivity [C] 6 equivocal results classed as negative for sensitivity [A] [D] [E] No equivocal range Unit of analysis: Samples
Comparative
Notes	Funding: Asked to perform evaluation by Department of Health and Social Care. Funding not stated Publication status: Published report Source: Public Health England Author COI: Not stated
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Unclear
Was a case‐control design avoided?	No
Did the study avoid inappropriate exclusions?	Unclear
Did the study avoid inappropriate inclusions?	No
Could the selection of patients have introduced bias?		High risk
Are there concerns that the included patients and setting do not match the review question?			High
DOMAIN 2: Index Test (All tests)
DOMAIN 2: Index Test (Antibody tests)
Were the index test results interpreted without knowledge of the results of the reference standard?	Yes
If a threshold was used, was it pre‐specified?	Yes
Could the conduct or interpretation of the index test have introduced bias?		Low risk
Are there concerns that the index test, its conduct, or interpretation differ from the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	Yes
The reference standard does not incorporate the index test	Yes
Could the reference standard, its conduct, or its interpretation have introduced bias?		Low risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			High
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Unclear
Did all patients receive the same reference standard?	No
Were all patients included in the analysis?	Yes
Did all participants receive a reference standard?	No
Were results presented per patient?	No
Could the patient flow have introduced bias?		High risk