. Author manuscript; available in PMC: 2022 Apr 6.

Published in final edited form as: Res Synth Methods. 2021 Aug 26;12(6):731–749. doi: 10.1002/jrsm.1504

TABLE 5.

Main simulation results for ${\hat{P}}_{> q} (z) - {\hat{P}}_{> q} (z_{0})$ in all scenarios, in scenarios with normally distributed population effects, when excluding scenarios whose covariate contrast used an extreme quantile of the continuous covariate (“not BC-rare”), and in scenarios fulfilling the reporting guidelines. Values are medians across scenarios with 10^th and 90^th percentiles given in parentheses. No. scenarios: the number of scenarios analyzed. Coverage: coverage of 95% confidence intervals. Cov. <85%: proportion of scenarios with coverage <85%. CI width: width of 95% confidence intervals. Width >0.90: proportion of scenarios with very large average CI width (>0.90). These results are discussed in Section 5.2.3

Scenarios	No. scenarios	${\hat{P}}_{z} - {\hat{P}}_{z_{0}}$ bias	${\hat{P}}_{z} - {\hat{P}}_{z_{0}}$ abs. error	${\hat{P}}_{z} - {\hat{P}}_{z_{0}}$ rel. bias	$\hat{E} [θ ∣ Z = z]$ rel. bias	${\hat{τ}}_{ϵ}^{2}$ rel. bias	${\hat{P}}_{z} - {\hat{P}}_{z_{0}}$ coverage	${\hat{P}}_{z} - {\hat{P}}_{z_{0}}$ cov. < 85%	${\hat{P}}_{z} - {\hat{P}}_{z_{0}}$ CI width	${\hat{P}}_{z}$ width >0.90
All	4679	0 (−0.02, 0.08)	0.08 (0.03, 0.24)	−0.02 (−0.26, 0.33)	0.05 (0.01, 0.17)	0.35 (0.14, 1.41)	0.92 (0.67, 0.99)	0.24	0.39 (0.12, 0.90)	0.10
Normal effects	2350	0 (−0.03, 0.01)	0.07 (0.03, 0.23)	−0.03 (−0.18, 0.16)	0.05 (0.01, 0.17)	0.32 (0.13, 1.41)	0.92 (0.72, 0.99)	0.18	0.39 (0.13, 0.90)	0.10
Not BC-rare	3098	0 (−0.02, 0.09)	0.07 (0.03, 0.23)	−0.01 (−0.17, 0.33)	0.05 (0.01, 0.17)	0.35 (0.14, 1.39)	0.93 (0.84, 0.99)	0.11	0.37 (0.12, 0.89)	0.09
Not BC-rare nor clustered exponential effects; k ≥ 20 (i.e., scenarios fulfilling guidelines)	1908	0 (−0.02, 0.03)	0.06 (0.02, 0.17)	−0.02 (−0.14, 0.15)	0.04 (0.01, 0.13)	0.29 (0.13, 1.25)	0.94 (0.9, 0.99)	0.02	0.29 (0.11, 0.81)	0.05