Skip to main content
. Author manuscript; available in PMC: 2015 Sep 14.
Published in final edited form as: J Stat Comput Simul. 2014 Nov 25;85(17):3498–3511. doi: 10.1080/00949655.2014.983111

Table 1.

Performance of AUC estimation using MI methods as well as case deletion under MAR, MCAR and MNAR.

% miss. Case deletion Joint imputation Sequential imputation Re-sampling (bootstrap)
MM TV CD(SE) Y* Y CR AW RMSE AUC (SE) CR AW RMSE AUC (SE) CR AW RMSE AUC (SE) CR AW RMSE
Scenario 1: {n = 100 πY* = 0.5 σY = 10}
MCAR 0.64 0.64 (0.04) 0.30 0.30 0.84 0.10 0.03 0.64 (0.03) 0.91 0.13 0.03 0.64 (0.04) 0.93 0.15 0.03 0.64 (0.04) 0.95 0.15 0.03
MAR 0.64 0.61 (0.03) 0.27 0.90 0.10 0.03 0.64 (0.03) 0.90 0.12 0.02 0.63 (0.04) 0.88 0.16 0.04 0.63 (0.04) 0.95 0.14 0.02
MNAR 0.64 0.60 (0.04) 0.31 0.30 0.76 0.12 0.06 0.61 (0.03) 0.71 0.13 0.06 0.62 (0.08) 0.99 0.36 0.05 0.62 (0.05) 0.92 0.20 0.05
Scenario 2:{n = 100 πY* = 0.5 σY = 30}
MCAR 0.60 0.61 (0.08) 0.30 0.30 0.78 0.18 0.003 0.62 (0.07) 0.91 0.28 0.004 0.61 (0.08) 0.96 0.32 0.003 0.61 (0.08) 0.96 0.31 0.003
MAR 0.60 0.58 (0.07) 0.32 0.88 0.18 0.003 0.58 (0.07) 0.97 0.27 0.006 0.62 (0.08) 0.96 0.33 0.006 0.61 (0.08) 0.98 0.33 0.003
MNAR 0.64 0.62 (0.09) 0.36 0.30 0.83 0.18 0.007 0.63 (0.09) 0.94 0.34 0.008 0.68 (0.13) 0.97 0.43 0.008 0.64 (0.11) 0.99 0.43 0.006
Scenario 3:{n = 100 πY* = 0.5 σY = 40}
MCAR 0.58 0.59 (0.08) 0.30 0.30 0.80 0.19 0.003 0.61 (0.07) 0.91 0.28 0.004 0.59 (0.08) 0.97 0.32 0.003 0.60 (0.08) 0.97 0.31 0.003
MAR 0.58 0.57 (0.07) 0.31 0.91 0.19 0.002 0.57 (0.07) 0.98 0.27 0.004 0.61 (0.09) 0.97 0.33 0.005 0.60 (0.08) 0.98 0.32 0.003
MNAR 0.58 0.58 (0.08) 0.33 0.30 0.86 0.21 0.004 0.61 (0.07) 0.92 0.29 0.007 0.64 (0.11) 0.99 0.48 0.009 0.61 (0.09) 0.98 0.37 0.005
Scenario 4:{n = 100 πY* =0.5 σY = 20}
MCAR 0.64 0.64 (0.06) 0.10 0.10 0.74 0.13 0.001 0.64 (0.06) 0.95 0.239 0.001 0.64 (0.06) 0.96 0.25 0.001 0.64 (0.06) 0.96 0.25 0.001
MAR 0.64 0.63 (0.06) 0.09 0.75 0.13 0.001 0.63 (0.06) 0.94 0.237 0.001 0.64 (0.06) 0.95 0.25 0.001 0.64 (0.06) 0.96 0.24 0.001
MNAR 0.64 0.62 (0.06) 0.11 0.10 0.80 0.15 0.002 0.65 (0.06) 0.93 0.291 0.002 0.63 (0.07) 0.97 0.29 0.003 0.64 (0.07) 0.97 0.27 0.002
Scenario 5:{n = 100 πY* = 0.5 σY = 20}
MCAR 0.64 0.64 (0.09) 0.40 0.40 0.73 0.16 0.005 0.6 (0.07) 0.92 0.30 0.006 0.64 (0.09) 0.96 0.37 0.004 0.65 (0.09) 0.97 0.36 0.004
MAR 0.64 0.62 (0.08) 0.42 0.76 0.16 0.004 0.61 (0.07) 0.96 0.28 0.007 0.64 (0.09) 0.98 0.35 0.006 0.64 (0.09) 0.98 0.35 0.003
MNAR 0.64 0.61 (0.10) 0.39 0.40 0.85 0.19 0.006 0.64 (0.07) 0.91 0.29 0.008 0.66 (0.12) 1.00 0.52 0.007 0.64 (0.10) 0.99 0.41 0.006
Scenario 6:{n = 100 πY* = 0.25 σY = 20}
MCAR 0.64 0.65 (0.09) 0.30 0.30 0.72 0.15 0.004 0.64 (0.09) 0.96 0.35 0.004 0.64 (0.10) 0.96 0.385 0.004 0.65 (0.10) 0.98 0.38 0.003
MAR 0.64 0.62 (0.07) 0.30 0.81 0.16 0.003 0.63 (0.08) 0.94 0.32 0.006 0.65 (0.09) 0.95 0.373 0.005 0.63 (0.09) 0.98 0.36 0.003
MNAR 0.64 0.61 (0.09) 0.36 0.30 0.85 0.18 0.006 0.64 (0.09) 0.94 0.35 0.009 0.68 (0.13) 0.97 0.525 0.008 0.64 (0.11) 0.99 0.44 0.005
Scenario 7:{n = 100 πY* = 0.35 σY = 20}
MCAR 0.64 0.64 (0.08) 0.30 0.30 0.74 0.15 0.004 0.64 (0.08) 0.94 0.31 0.004 0.64 (0.09) 0.96 0.34 0.004 0.64 (0.09) 0.97 0.34 0.003
MAR 0.64 0.62 (0.07) 0.29 - 0.79 0.15 0.003 0.63 (0.07) 0.97 0.29 0.004 0.64 (0.08) 0.95 0.33 0.005 0.63 (0.08) 0.97 0.32 0.003
MNAR 0.64 0.61 (0.09) 0.36 0.30 0.85 0.18 0.007 0.63 (0.09) 0.94 0.34 0.009 0.68 (0.13) 0.97 0.53 0.008 0.64 (0.11) 0.98 0.43 0.006
Scenario 8:{n = 100 πY* = 0.50 σY = 20}
MAR 0.64 0.62 (0.06) 0.27 0.82 0.15 0.05 0.60 (0.06) 0.93 0.27 0.08 0.63 (0.07) 0.96 0.31 0.06 0.64 (0.08) 0.97 0.31 0.05
MCAR 0.64 0.64 (0.08) 0.30 0.30 0.72 0.15 0.06 0.65 (0.07) 0.92 0.29 0.06 0.63 (0.08) 0.96 0.34 0.06 0.64 (0.08) 0.97 0.33 0.06
MNAR 0.64 0.61 (0.08) 0.32 0.30 0.86 0.18 0.07 0.64 (0.07) 0.94 0.30 0.08 0.63 (0.10) 0.98 0.47 0.08 0.64 (0.09) 0.98 0.41 0.07
Scenario 9:{n = 100 πY* = 0.25 σY = 20}
MAR 0.75 0.71 (0.07) 0.42 0.34 0.09 0.07 0.73 (0.08) 0.86 0.32 0.09 0.73 (0.09) 0.84 0.36 0.09 0.73 (0.10) 0.96 0.40 0.07
MCAR 0.76 0.77 (0.07) 0.30 0.30 0.30 0.07 0.06 0.75 (0.08) 0.92 0.33 0.06 0.75 (0.09) 0.93 0.37 0.07 0.75 (0.09) 0.97 0.38 0.06
MNAR 0.76 0.72 (0.07) 0.22 0.30 0.28 0.09 0.07 0.72 (0.08) 0.85 0.31 0.09 0.73 (0.10) 0.92 0.46 0.08 0.74 (0.09) 0.94 0.39 0.07
Scenario 10:{n = 100 πY* = 0.50 σY = 20}
MAR 0.76 0.71 (0.07) 0.35 0.33 0.09 0.07 0.69 (0.07) 0.78 0.26 0.10 0.72 (0.07) 0.85 0.30 0.08 0.74 (0.08) 0.95 0.32 0.05
MCAR 0.76 0.76 (0.07) 0.30 0.30 0.36 0.07 0.05 0.76 (0.06) 0.89 0.26 0.06 0.75 (0.07) 0.92 0.30 0.06 0.75 (0.07) 0.94 0.31 0.05
MNAR 0.76 0.72 (0.07) 0.17 0.30 0.32 0.09 0.07 0.74 (0.06) 0.85 0.26 0.07 0.72 (0.09) 0.90 0.39 0.08 0.74 (0.07) 0.92 0.32 0.06
Scenario 11:{n = 100 πY* = 0.25 σY = 20}
MAR 0.85 0.80 (0.07) 0.54 0.08 0.05 0.08 0.81 (0.07) 0.85 0.28 0.10 0.82 (0.07) 0.74 0.31 0.11 0.81 (0.09) 0.10 0.40 0.08
MCAR 0.85 0.85 (0.06) 0.30 0.30 0.13 0.03 0.05 0.83 (0.07) 0.93 0.27 0.05 0.85 (0.07) 0.93 0.30 0.05 0.84 (0.08) 0.97 0.32 0.05
MNAR 0.85 0.81 (0.06) 0.24 0.30 0.09 0.04 0.06 0.83 (0.06) 0.88 0.26 0.08 0.81 (0.10) 0.95 0.44 0.09 0.83 (0.08) 0.96 0.32 0.06
Scenario 12:{n = 100 πY* = 0.50 σY = 20}
MAR 0.86 0.80 (0.07) 0.42 0.11 0.05 0.08 0.79 (0.06) 0.77 0.23 0.10 0.79 (0.06) 0.74 0.27 0.10 0.82 (0.07) 0.96 0.32 0.06
MCAR 0.86 0.86 (0.06) 0.30 0.30 0.13 0.02 0.04 0.85 (0.05) 0.88 0.21 0.04 0.85 (0.06) 0.91 0.23 0.04 0.85 (0.06) 0.94 0.25 0.04
MNAR 0.86 0.81 (0.06) 0.17 0.30 0.13 0.04 0.06 0.84 (0.05) 0.86 0.20 0.05 0.81 (0.07) 0.90 0.34 0.08 0.84 (0.06) 0.94 0.26 0.05

Notes: MM (missingness mechanism), TV (true value), CD (case deletion), SE (standard error), CR (coverage rate), AW (average width of confidence intervals), RMSE (root-mean-square error). CR and AW are based on 95% nominal confidence rate. CD(SE) column contains AUC estimate and estimate of its standard error under case deletion.