Table 3. Simulation results for comparing the differences in predictive performance of two prediction models under quota sampling NCC (QNCC) designs.
Model 1 | Model 2 | Model 1 − Model 2 | |||||||
---|---|---|---|---|---|---|---|---|---|
Est. | SE | Est. | SE | True | Est. | SE | SD | Cov. | |
TPRt(0.25) | 0.686 | 0.022 | 0.644 | 0.025 | 0.041 | 0.042 | 0.017 | 0.017 | 0.944 |
FPRt(0.25) | 0.148 | 0.014 | 0.161 | 0.015 | −0.011 | −0.013 | 0.009 | 0.009 | 0.947 |
PCF(0.2) | 0.614 | 0.018 | 0.569 | 0.019 | 0.045 | 0.045 | 0.011 | 0.011 | 0.928 |
PNFt(0.85) | 0.58 | 0.019 | 0.527 | 0.019 | 0.05 | 0.053 | 0.013 | 0.012 | 0.933 |
AUCt | 0.859 | 0.009 | 0.83 | 0.011 | 0.027 | 0.029 | 0.007 | 0.007 | 0.920 |
Mean(
![]() |
0.767 | 0.012 | 0.739 | 0.013 | 0.024 | 0.028 | 0.012 | 0.012 | 0.933 |
Mean (
![]() |
0.215 | 0.012 | 0.24 | 0.013 | −0.024 | −0.025 | 0.011 | 0.011 | 0.948 |
MRDt | 0.552 | 0.018 | 0.499 | 0.019 | 0.047 | 0.053 | 0.013 | 0.013 | 0.912 |
TPRt(ρ) | 0.46 | 0.02 | 0.411 | 0.021 | 0.051 | 0.05 | 0.012 | 0.011 | 0.938 |
FPRt(ρ) | 0.124 | 0.007 | 0.136 | 0.008 | −0.011 | −0.012 | 0.003 | 0.003 | 0.945 |
AARDt | 0.336 | 0.022 | 0.275 | 0.022 | 0.063 | 0.061 | 0.014 | 0.014 | 0.937 |
NBt(p) | 0.116 | 0.009 | 0.108 | 0.009 | 0.008 | 0.008 | 0.003 | 0.003 | 0.946 |
Est. mean estimates, SD empirical standard deviation, SE average of the estimated standard error, CP empirical coverage level of the 95 % confidence intervals for proposed estimators