Nilles 2020 [A].

Study characteristics
Patient Sampling	Purpose: The study evaluated two commercial (Roche Diagnostics and Epitope Diagnostics IgM/IgG) and two non‐commercial (Simoa and Ragon/MGH IgG) immunoassays against 68 confirmed positive and 232 pre‐pandemic negative controls. 2‐group study to estimate sensitivity and specificity for diagnosis of active disease and identification of previous disease Design: [1] patients that had been hospitalised at the Brigham and Women’s Hospital testing positive by SARS‐CoV‐2 RT‐PCR (n = 28 patients, 68 samples) [2] Pre‐pandemic controls with and without recent respiratory infections (n = 232 patients/samples) Recruitment: Samples from Mass General Brigham Biobank (a biorepository that contains biological samples and linked demographic and clinical data from > 117,000 patients enrolled through the MGB network) Prospective or retrospective: Retrospective Sample size: Patients: 260 (28) Samples: 300 (68) Further detail: Cases = RT‐PCR‐positive To determine if recent respiratory infections may be associated with increased cross‐reactivity and false positives, we selected negative controls with and without recent respiratory infections. We only selected controls with both serum and plasma available from the same individual and time point.
Patient characteristics and setting	Setting: Hospital Inpatients Location: Brigham and Women’s Hospital (BWH). Samples from Mass General Brigham Biobank Country: USA Dates: March 30 to May 4, 2020 Symptoms and severity: 40/68 samples were from patients who required ICU. Symptoms included cough, fever, dyspnoea, myalgias, new loss of taste or smell, or sore throat. Demographics: Median age of patients was 57 years (range 32‐79) and 35/68 (51%) were female. Race: White 22/68 (32%); black 22/68 (32%); Asian or Pacific Islander 6/68 (9%); American Indian or Alaskan native 3/68 (4%); Other or not recorded 15/68 (22%) Ethnicity: Non‐Hispanic 53/68 (78%); Hispanic 9/68 (13%); other or not recorded 6/68 (9%) Exposure history: Not stated Non‐Covid group 1: Pre‐pandemic controls Source: Samples from Mass General Brigham Biobank. August 28, 2017 to September 26, 2019 Characteristics: The median age was 55 years (range 20‐89) and 90/232 (39%) were female. Of the total 232 negative control samples, 100 were from individuals without recent respiratory illness; 31 from individuals with prior laboratory‐confirmed viral respiratory infections; 101 from individuals with a recent clinical diagnosis of respiratory infections including upper respiratory tract infection (n = 50) or viral (n = 11), bacterial (n = 20) or unspecified (n = 20) pneumonia based on diagnoses recorded in the electronic health record between 1 and 31 days prior to sample collection. Non‐Covid group 2: NA Source: NA Characteristics: NA
Index tests	Test name: [A] Elecsys Anti‐SARS‐CoV‐2 immunoassay [B] EDI New Coronavirus COVID‐19 IgG ELISAs [C] EDI Novel Coronavirus COVID‐19 IgM ELISA Manufacturer: [A] Roche Diagnostics, Indianapolis, USA [B] Epitope Diagnostics, USA [C] Epitope Diagnostics, USA Antibody: [A] IgG [B] IgG [C] IgM Antigen target: [A] nucleocapsid (NC) antigen (thought to include IgG, IgM, and IgA, although IgM and IgA were not specified in product information) [B] IgG against the NC antigen [C] IgM against an unspecified antigen Evaluation setting: [A] Laboratory, used in laboratory [B] Laboratory, used in laboratory [C] Laboratory, used in laboratory Test method: [A] Electro‐chemiluminescence immunoassay (ECLIA) [B] ELISA [C] ELISA Timing of samples: Samples were collected a mean of 10.5 days (standard deviation 6.0 days) post‐RT‐PCR confirmation and 16.1 days (standard deviation 5.4 days) post‐symptom onset (pso). 8‐14 days pso: 30/68 15‐21 days pso: 29/68 > 21 days pso: 9/68 Samples used: Serum or plasma (different requirements for different tests, but not specified) To ensure valid comparison between assays and given differences in plasma/sera requirements according to manufacturer/assay specifications, we only selected controls with both serum and plasma available from the same individual and time point. All samples were stored at ‐80°C following sample processing and none underwent thaw‐refreezing cycles prior to analysis. Test operator: [A] Brigham and Women’s Hospital laboratories [B] Brigham and Women’s Hospital laboratories [C] Brigham and Women's Hospital laboratories Definition of test positivity: [A], [B] and [C] Threshold cut‐offs for defining positive, negative or indeterminate/borderline test results were defined according to manufacturer specifications for commercial assays. Blinding reported: Yes ‐ "laboratories were blinded to sample group" Threshold predefined: [A] Threshold cut‐offs for defining positive, negative or indeterminate/borderline test results were defined according to manufacturer specifications for commercial assays. [B] and [C] Threshold cut‐offs for defining positive, negative or indeterminate/borderline test results were defined according to manufacturer specifications for commercial assays.
Target condition and reference standard(s)	Reference standard: RT‐PCR Samples used: Not stated Timing of reference standard: Mean of 5.6 days after onset of symptoms Blinded to index test: Yes, prior Incorporated index test: No Definition of non‐COVID cases: Pre‐pandemic Samples used: Pre‐pandemic Timing of reference standard: Pre‐pandemic Blinded to index test: Yes, prior Incorporated index test: No
Flow and timing	Time interval between index and reference tests: Not stated for pre‐pandemic samples Samples for index test were collected a mean of 10.5 days (standard deviation 6.0 days) post‐RT‐PCR confirmation. All patients received same reference standard: No Missing data: Given limited negative control aliquots, the Epitope assays were tested against 230 samples, versus 232 for the remaining assays. Uninterpretable results: Not stated Indeterminate results: Indeterminate or borderline results were considered negative for all analyses. Unit of analysis: Samples The median number of samples per individual was two (range 1‐5) and the median interval between sample collection was three days (range 2‐6 days).
Comparative
Notes	Funding: This work was largely funded by Brigham Health. EN is supported by a CDC U01 GH002238. LB is supported by NIH UM1AI069412 and UL1TR001102. D.S. is supported by NIH K08 AR075850. Publication status: Pre‐print (not peer reviewed) Source: medRxiv preprint Author COI: Not stated
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Unclear
Was a case‐control design avoided?	No
Did the study avoid inappropriate exclusions?	Unclear
Did the study avoid inappropriate inclusions?	No
Could the selection of patients have introduced bias?		High risk
Are there concerns that the included patients and setting do not match the review question?			High
DOMAIN 2: Index Test (All tests)
DOMAIN 2: Index Test (Antibody tests)
Were the index test results interpreted without knowledge of the results of the reference standard?	Yes
If a threshold was used, was it pre‐specified?	Yes
Could the conduct or interpretation of the index test have introduced bias?		Low risk
Are there concerns that the index test, its conduct, or interpretation differ from the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	Yes
The reference standard does not incorporate the index test	Yes
Could the reference standard, its conduct, or its interpretation have introduced bias?		Low risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			High
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Unclear
Did all patients receive the same reference standard?	No
Were all patients included in the analysis?	Yes
Did all participants receive a reference standard?	No
Were results presented per patient?	No
Could the patient flow have introduced bias?		High risk