TABLE 1.

Key features for eligible systematic data syntheses

Features	Meyerowitz‐Katz	Rostami	Bobrovitz	Imperial college COVID‐19 response team	Ioannidis	O’Driscoll
Types of information included	SP, non‐serological and modelling studies	SP studies	SP studies	SP studies	SP studies	SP studies
Last search	16 June	14 August	28 August	Unclear	9 September	Unclear (1 September?)
Search sources	PubMed, preprints (medRxiv, SSRN), Google, Twitter searches, government agency reports eligible	PubMed, Scopus, EMBASE, medRxiv, bioRxiv, research reports eligible	MEDLINE, EMBASE, Web of Science, and Europe PMC, Google, communication with experts	SeroTracker searches (see Bobrovitz)	PubMed (LitCOVID), medRxiv, bioRxiv, Research Square, national reports, communication with experts for additional studies	Unclear
Types of SP studies included	Excluded targeted populations with selection bias, also four other studies a	Excluded at‐risk populations (eg HCW), known diseases (eg dialysis, cancer)	All studies included if they reported on sample, date, region and SP estimate	Studies with defined sampling framework, defined geographic area, with availability of test performance, preferentially validation done as part of the study (not just by manufacturers), >100 deaths at SP study mid‐point b ; excluded healthcare workers, symptoms of COVID‐19, self‐referral or self‐selection, narrow age range, confined settings, clinical samples	General population or approximations (including blood donors, excluding high risk, eg HCW, communities), sample size >500, area with population >5000	Unclear, but eventually it includes some general population studies, some blood donors and some hospital samples
Number of studies, countries, locations	24‐27 studies c , of which 16 serological from 14 countries	107 data sets from 47 studies from 23 countries	338 studies (184 from general population) from 50 countries (36 from general population) d	10 studies (six national, four subnational), nine countries e	82 estimates, 69 studies, 51 locations, 36 countries (main analysis at the location level)	25 studies from 20 countries (only 22 national representing 16 countries used in the ensemble model)
Studies published in peer‐review journals at the time of the evaluation	1/16	61/107	4/40 included in final analysis of under‐ascertainment ratio	5/10	35/82	6/20 countries

Abbreviations: HCW, healthcare workers; IFR, infection fatality rate; SP, seroprevalence.

^aOne study (LA County) ¹² with very low IFR was excluded with the justification that it ‘explicitly warned against using its data to obtain an IFR’; as a co‐investigator of the study, both myself and my colleagues are intrigued at the rationale for exclusion; in the publication of the study in JAMA, ¹² we did list limitations and caveats, as it is appropriate for any seroprevalence study to do; excluding studies that are honest to discuss limitations would keep only the worst studies that discuss no limitations. Two other studies with low IFR were excluded as well. One was done in Rio Grande do Sul ¹³ where its authors even report IFR estimates in their paper (0.29%, 0.23%, 0.38% in the three rounds of the serosurvey); the other was done in Boise, ⁸⁵ where its authors properly discuss limitations but an approximation of IFR is possible; even if not perfectly accurate, it is certainly lower than the IFR estimates included in the Meyerowitz‐Katz meta‐analysis. For the fourth excluded study, ¹¹ the justification offered for its exclusion is that it ‘calculated an IFR, but did not allow for an estimate of confidence bounds’. ¹ However, this study presents results of a New York study that Meyerowitz‐Katz did include in their meta‐analysis. Of note, that fourth study ¹¹ also presents a cursory review of seroprevalence studies arriving at a median IFR = 0.31%, half of the summary estimate of Meyerowitz‐Katz.

^bClear bias introduced since number of deaths is the numerator itself in the calculation of IFR, and exclusion of studies with low numerator is thus excluding studies likely to have low IFR

^cDifferent numbers provided by the authors for total studies in abstract (n = 24), text of the paper (n = 25), tabulated studies (n = 27) and forest plot studies (n = 26)

^d39 estimates from 17 countries used in main calculation of median under‐ascertainment ratio (N. Bobrovitz, personal communication)

^eOne of the 10 included studies violates the eligibility criterion of the investigators having validated themselves the antibody test used; the ICCRT included this study invoking validation data for the same antibody kit done by a different team in a study in a completely different setting and continent (San Francisco); based on this rationale, perhaps many other studies could have been included, if the same violation of the eligibility criteria was tolerated. The included study was an Italian survey ³⁰ which had only been released in the press with a preliminary report at the time of the ICCRT evaluation and which included crude results on only 64 660 of the intended 150 000 participants (missingness 57%). Its inferred IFR estimate (2.5%) is an extreme outlier, as it is 2‐ to 20‐fold larger than other typical estimates reported from numerous European countries. Moreover, that IFR estimate even matches/exceeds case fatality rates, and thus, it is simply impossible. It is widely accepted that IFR must be several times smaller than case fatality rate, even in locations with substantial testing. Italy had very limited testing in the first wave and modest testing in the second wave. One estimate suggests that the number of infections in Italy at the peak of the first wave was 12 times more than the number of documented cases; that is, the IFR would be more than an order of magnitude lower than the case fatality rate. ³¹

This article is being made freely available through PubMed Central as part of the COVID-19 public health emergency response. It can be used for unrestricted research re-use and analysis in any form or by any means with acknowledgement of the original source, for the duration of the public health emergency.