Table 2.

Evaluation of CIMBAL along with complete-only meta-analysis approach and gold standard (oracle) approach using multiple metrics across different simulated data scenarios. The metrics MSE (mean squared error), rel. MSE (relative MSE compared to oracle meta-analysis approach), mean width of 95% CI, and type I error inflation factor at 5% significance level (ratio of type I error estimate to 0.05) are estimated using 2,500 independent replicate datasets for each scenario: (1) fewer cohorts or (2) equal number of cohorts or (3) more cohorts with no confounder information than with complete confounder information. Ideal rel. MSE value is 1 × and larger values indicate departure from oracle. Ideal type I error inflation value is 1; larger than 1 indicates inflation, smaller than 1 indicates conservativeness. The underlying data generative model assumes there is no exposure-outcome association (true β=0)

		Scenario 1 (fewer)			Scenario 2 (equal)			Scenario 3 (more)
	Method	MSE	mean	type I	MSE	mean	type I	MSE	mean	type I
		(rel. MSE)	width	error IF	(rel. MSE)	width	error IF	(rel. MSE)	width	error IF
Setting I	M: complete only	0.006 (1.5 ×)	0.30	0.98	0.007 (1.8 ×)	0.35	0.86	0.011 (2.8 ×)	0.43	0.91
	M: CIMBAL (v0.7)	0.004 (1.0 ×)	0.27	1.00	0.005 (1.3 ×)	0.29	0.92	0.006 (1.5 ×)	0.32	1.00
	M: fully adjusted (oracle)	0.004 (1 ×)	0.25	1.02	0.004 (1 ×)	0.25	1.02	0.004 (1 ×)	0.25	1.02
Setting II	M: complete only	0.008 (1.6 ×)	0.34	0.96	0.010 (2.0 ×)	0.40	0.94	0.015 (3.0 ×)	0.49	0.91
	M: CIMBAL (v0.7)	0.006 (1.2 ×)	0.30	1.02	0.007 (1.4 ×)	0.32	1.06	0.008 (1.6 ×)	0.35	1.02
	M: fully adjusted (oracle)	0.005 (1 ×)	0.28	0.96	0.005 (1 ×)	0.28	0.96	0.005 (1 ×)	0.28	0.96
Setting III	M: complete only	0.008 (1.6 ×)	0.35	0.96	0.010 (2.0 ×)	0.41	0.82	0.016 (3.2 ×)	0.50	0.89
	M: CIMBAL (v0.7)	0.006 (1.2 ×)	0.30	1.02	0.006 (1.2 ×)	0.31	1.07	0.008 (1.6 ×)	0.34	0.98
	M: fully adjusted (oracle)	0.005 (1 ×)	0.29	1.00	0.005 (1 ×)	0.29	1.00	0.005 (1 ×)	0.29	1.00

Abbreviations: IF, inflation factor; M, meta-analysis of 60 cohorts