Table 1.

Sampling Frame, Sample, and Reference Standards of Studies

Study (year)	Country	Sampling frame			Sample			Reference standard				Benchimol score (max. 38)†
		Scope	Time period	Inclusion criteria	Selection	N	Pneumonia prevalence*	Explicit criteria	Application	Blinded to code	Multiple reviewers
Wiese (2018)18	USA	State-wide health care network	2008–2012	Age > 50 years, MedicAid eligible and hospitalization coded with pneumonia	Random	340	96%	Yes	Retrospective	No	No	20
Ahmed (2014)19	USA	County-wide health care network	2010	Admission to ICU and high risk of ARDS	All	1443‡	8%	No	Prospective	Yes	Yes	26
Holland-Bill (2014)20	Denmark	Single center	2006–2010	Coded with cancer and hospitalization coded with pneumonia	Random	93	89%	Yes	Retrospective	Yes	No	22
Meropol (2012)21	UK	Nation-wide database	1985–2006	Hospitalization within 30 days of an outpatient visit coded with pneumonia	Random	59	86%	No	Retrospective	No	No	17
Yu (2011)22	USA	State-wide health care network	1997–2005	Hospitalization coded with pneumonia	All	3991	60%	No	Retrospective	No	No	27
Grijalva (2008)23	USA	State-wide health care network	1995–2004	MedicAid eligible, coded with rheumatoid arthritis and hospitalization coded with pneumonia	Random	161	84%	No	Retrospective	Yes	Yes	20
Skull (2008)24	Australia	Two hospitals	2000–2002	Age ≥ 65 years, hospitalization coded with pneumonia (cases) or not (controls)	All	3343	46%	No	Retrospective	Yes	No	29
Gedeborg (2007)25	Sweden	Single center	1994–1999	Admission to ICU	All	7615	5%	No	Prospective	Yes	No	24
van de Garde (2007)10	Netherlands	Seven hospitals	2000–2004	Enrolled in pneumonia clinical trial	All	293	100%	Yes	Prospective	Yes	No	21
Schneeweiss (2007)26	USA	State-wide health care network	2001–2004	Veteran, hospitalization for ≥3 days, hospitalization coded with pneumonia	All	23	70%	Yes	Retrospective	No	No	25
Aronsky (2005)8	USA	Single center	1999–2000	Admission to ED	All	10,748	2%	No	Retrospective	Yes	Yes	25
Whittle (1997)27	USA	Single center	1989–1990	Hospitalization coded with pneumonia	Random	144	65%	No	Retrospective	Yes	Yes	23
Marrie (1987)28	Canada	Single center	1984	Hospitalization with clinical diagnosis of pneumonia or coded with pneumonia	All	159	66%	No	Prospective	Yes	No	20

USA refers to United States. ICU refers to intensive care unit. ARDS refers to acute respiratory distress syndrome. *As per the application of the reference standard. ^†Number of positive items out of the 38 quality items proposed by Benchimol et al.¹⁷ (for details of the performance of each study in individual quality items, please see Appendix 2). ^‡Validation sample for this study