Skip to main content
. Author manuscript; available in PMC: 2022 Jan 30.
Published in final edited form as: Stat Med. 2020 Nov 12;40(2):403–426. doi: 10.1002/sim.8781

TABLE 2.

Performance of different methods for synthesizing standardized mean differences based on 10,000 replicates of simulated meta-analyses with N = 5 studies under various settings.

Method τ = 0.1 τ = 0.5
ni = 6–14 ni = 30–70 ni = 6–14 ni = 30–70
Bias RMSE CP, % Bias RMSE CP, % Bias RMSE CP, % Bias RMSE CP, %
θ = 0:
Cohen’s d, Eq. (4) 0.000 0.299 97.6 (95.4) −0.001 0.137 95.2 (94.6) 0.001 0.367 95.4 (95.8) 0.002 0.260 89.6 (95.4)
Cohen’s d, Eq. (5) 0.000 0.303 96.3 (96.0) −0.001 0.137 95.2 (94.6) 0.001 0.371 94.0 (96.1) 0.002 0.260 89.6 (95.4)
Cohen’s d, Eq. (6) 0.000 0.312 96.0 (95.9) −0.001 0.138 95.2 (94.6) 0.000 0.383 93.6 (96.0) 0.002 0.260 89.6 (95.4)
Hedges’ g, Eq. (10) 0.000 0.267 97.6 (95.6) −0.001 0.135 95.3 (94.6) 0.001 0.327 95.5 (96.0) 0.002 0.255 89.6 (95.4)
Hedges’ g, Eq. (11) 0.000 0.271 96.3 (95.9) −0.001 0.135 95.2 (94.7) 0.001 0.330 94.1 (96.2) 0.002 0.255 89.6 (95.4)
Hedges’ g, Eq. (12) 0.000 0.279 96.0 (95.8) −0.001 0.135 95.2 (94.7) 0.000 0.341 93.7 (96.0) 0.002 0.256 89.6 (95.4)
Hedges’ g, Eq. (13) 0.000 0.281 97.1 (95.5) −0.001 0.135 95.3 (94.6) 0.000 0.345 94.7 (95.8) 0.002 0.256 89.6 (95.4)
Hedges’ g, Eq. (14) 0.000 0.266 97.6 (95.6) −0.001 0.135 95.3 (94.6) 0.001 0.326 95.5 (96.0) 0.002 0.255 89.7 (95.4)
Hedges’ g, Eq. (20) −0.001 0.313 96.2 (95.2) [96.9] −0.001 0.137 95.2 (94.5) [95.3] −0.001 0.382 93.7 (95.4) [94.8] 0.002 0.260 89.3 (95.2) [89.5]
Hedges’ g, Eq. (21) 0.000 0.309 96.9 (94.9) [96.6] −0.001 0.137 95.2 (94.5) [95.2] 0.000 0.381 94.4 (95.1) [94.0] 0.002 0.260 89.4 (95.2) [89.4]
Hedges’ g, Eq. (22) −0.001 0.315 95.1 (95.5) [97.4] −0.001 0.137 95.0 (94.6) [95.3] −0.001 0.382 92.6 (95.6) [95.5] 0.002 0.260 89.3 (95.2) [89.6]
θ = 0.3:
Cohen’s d, Eq. (4) −0.001 0.300 97.7 (95.6) 0.000 0.138 95.6 (95.0) −0.007 0.372 95.2 (95.3) 0.000 0.263 88.5 (94.8)
Cohen’s d, Eq. (5) 0.002 0.303 96.4 (95.9) 0.000 0.138 95.4 (95.0) −0.001 0.378 93.8 (95.8) 0.000 0.263 88.4 (94.8)
Cohen’s d, Eq. (6) 0.010 0.312 96.1 (95.8) 0.000 0.138 95.3 (95.0) 0.007 0.391 93.5 (95.8) 0.000 0.264 88.4 (94.8)
Hedges’ g, Eq. (10) −0.032 0.270 97.5 (95.7) −0.005 0.135 95.6 (95.0) −0.037 0.334 95.1 (95.5) −0.006 0.259 88.5 (94.8)
Hedges’ g, Eq. (11) −0.030 0.272 96.4 (96.0) −0.005 0.136 95.4 (95.0) −0.033 0.338 93.9 (95.7) −0.005 0.259 88.5 (94.8)
Hedges’ g, Eq. (12) −0.023 0.280 96.0 (95.9) −0.005 0.136 95.4 (95.0) −0.026 0.349 93.5 (95.7) −0.005 0.259 88.5 (94.8)
Hedges’ g, Eq. (13) −0.020 0.281 97.0 (95.6) −0.005 0.136 95.6 (95.0) −0.023 0.352 94.6 (95.4) −0.005 0.259 88.6 (94.8)
Hedges’ g, Eq. (14) −0.033 0.269 97.6 (95.6) −0.005 0.135 95.6 (95.0) −0.039 0.333 95.2 (95.5) −0.006 0.259 88.5 (94.8)
Hedges’ g, Eq. (20) 0.003 0.311 96.3 (95.4) [97.1] −0.001 0.137 95.3 (94.9) [95.4] 0.002 0.393 93.6 (95.0) [94.6] 0.000 0.264 88.3 (94.6) [88.4]
Hedges’ g, Eq. (21) 0.002 0.307 97.0 (95.2) [96.6] −0.001 0.137 95.4 (94.9) [95.4] 0.001 0.391 94.2 (94.8) [93.9] 0.000 0.264 88.3 (94.6) [88.3]
Hedges’ g, Eq. (22) 0.002 0.313 95.3 (95.6) [97.5] −0.001 0.138 95.1 (94.9) [95.5] 0.001 0.393 92.7 (95.3) [95.4] 0.000 0.264 88.2 (94.6) [88.5]
θ = 0.8:
Cohen’s d, Eq. (4) −0.006 0.308 98.0 (95.8) 0.000 0.141 95.8 (95.0) −0.023 0.386 94.6 (95.0) 0.000 0.261 89.2 (95.2)
Cohen’s d, Eq. (5) 0.004 0.313 96.4 (95.9) 0.000 0.141 95.6 (95.0) −0.009 0.390 93.5 (95.4) 0.001 0.261 89.2 (95.2)
Cohen’s d, Eq. (6) 0.023 0.324 96.1 (95.9) 0.000 0.141 95.6 (95.0) 0.013 0.402 93.4 (95.4) 0.002 0.261 89.2 (95.2)
Hedges’ g, Eq. (10) −0.087 0.289 96.7 (94.9) −0.013 0.139 95.6 (94.9) −0.103 0.359 93.5 (94.3) −0.013 0.257 89.2 (95.1)
Hedges’ g, Eq. (11) −0.082 0.292 95.1 (95.2) −0.013 0.139 95.3 (94.9) −0.093 0.360 92.0 (94.7) −0.012 0.257 89.1 (95.1)
Hedges’ g, Eq. (12) −0.065 0.296 95.0 (95.2) −0.013 0.139 95.3 (94.9) −0.075 0.365 92.2 (94.8) −0.012 0.257 89.1 (95.1)
Hedges’ g, Eq. (13) −0.056 0.294 96.5 (95.1) −0.012 0.139 95.5 (94.9) −0.068 0.366 93.5 (94.6) −0.012 0.257 89.1 (95.0)
Hedges’ g, Eq. (14) −0.090 0.289 96.7 (94.8) −0.013 0.139 95.6 (94.9) −0.107 0.359 93.4 (94.2) −0.013 0.257 89.2 (95.1)
Hedges’ g, Eq. (20) 0.005 0.334 96.3 (95.0) [97.0] −0.002 0.141 95.5 (94.9) [95.7] −0.001 0.405 93.3 (94.4) [94.3] 0.002 0.261 89.0 (94.9) [89.2]
Hedges’ g, Eq. (21) 0.002 0.327 97.1 (94.8) [96.8] −0.002 0.141 95.7 (94.9) [95.6] −0.004 0.403 93.9 (94.1) [93.7] 0.002 0.261 89.1 (94.9) [89.1]
Hedges’ g, Eq. (22) 0.003 0.337 95.0 (95.3) [97.4] −0.002 0.141 95.3 (94.9) [95.7] −0.004 0.405 92.1 (94.7) [95.0] 0.002 0.261 89.1 (95.0) [89.3]

Abbreviation: CP, coverage probability of 95% confidence interval; RMSE, root mean squared error.

Notation: N, no. of studies; τ, between-study standard deviation; ni, study-specific sample size; θ, true overall standardized mean difference.

Note: coverage probabilities include those based on conventional meta-analysis method in Equation (23) (outside parentheses and square brackets), based on the Hartung–Knapp–Sidik–Jonkman method in Equation (24) (inside parentheses), and based on the method in Equation (26) (inside square brackets).