Table 1. Performance evaluation based on simulated gene expression profiles with m = 2 conditions/groups.
Results for the small-sample case (n1 = n2 = 3) | ||||||||||||||||
Methods | TPR | FPR | TNR | FNR | FDR | MER | AUC | pAUC | TPR | FPR | TNR | FNR | FDR | MER | AUC | pAUC |
Without outlying expressions | For 1 outlier with each of 5% genes | |||||||||||||||
ANOVA | 0.939 | 0.001 | 0.998 | 0.072 | 0.072 | 0.004 | 0.915 | 0.184 | 0.475 | 0.011 | 0.989 | 0.525 | 0.525 | 0.021 | 0.474 | 0.095 |
SAM | 0.955 | 0.001 | 0.999 | 0.045 | 0.045 | 0.002 | 0.954 | 0.191 | 0.490 | 0.010 | 0.990 | 0.510 | 0.510 | 0.020 | 0.491 | 0.098 |
LIMMA | 0.958 | 0.001 | 0.999 | 0.042 | 0.042 | 0.002 | 0.959 | 0.192 | 0.485 | 0.011 | 0.989 | 0.515 | 0.515 | 0.021 | 0.484 | 0.097 |
eLNN | 0.932 | 0.001 | 0.999 | 0.068 | 0.068 | 0.003 | 0.931 | 0.186 | 0.372 | 0.013 | 0.987 | 0.627 | 0.627 | 0.025 | 0.371 | 0.074 |
EBarrays | 0.938 | 0.001 | 0.999 | 0.062 | 0.062 | 0.002 | 0.939 | 0.188 | 0.307 | 0.014 | 0.986 | 0.692 | 0.692 | 0.028 | 0.306 | 0.061 |
BetaEB | 0.938 | 0.001 | 0.999 | 0.062 | 0.062 | 0.002 | 0.937 | 0.188 | 0.940 | 0.001 | 0.999 | 0.060 | 0.060 | 0.002 | 0.941 | 0.188 |
KW | 0.958 | 0.000 | 1.000 | 0.042 | 0.042 | 0.001 | 0.977 | 0.196 | 0.497 | 0.010 | 0.990 | 0.502 | 0.502 | 0.020 | 0.496 | 0.100 |
Proposed | 0.939 | 0.001 | 0.998 | 0.072 | 0.072 | 0.004 | 0.915 | 0.184 | 0.936 | 0.001 | 0.998 | 0.064 | 0.064 | 0.004 | 0.914 | 0.182 |
For 1 outlier with each of 10% genes | For 1 outlier with each of 75% genes | |||||||||||||||
ANOVA | 0.318 | 0.014 | 0.986 | 0.682 | 0.682 | 0.027 | 0.318 | 0.063 | 0.087 | 0.019 | 0.981 | 0.912 | 0.912 | 0.036 | 0.086 | 0.018 |
SAM | 0.323 | 0.014 | 0.986 | 0.677 | 0.677 | 0.027 | 0.324 | 0.064 | 0.087 | 0.019 | 0.981 | 0.912 | 0.912 | 0.036 | 0.086 | 0.018 |
LIMMA | 0.318 | 0.014 | 0.986 | 0.682 | 0.682 | 0.027 | 0.317 | 0.064 | 0.087 | 0.019 | 0.981 | 0.912 | 0.912 | 0.036 | 0.088 | 0.018 |
eLNN | 0.250 | 0.015 | 0.985 | 0.750 | 0.750 | 0.030 | 0.251 | 0.050 | 0.025 | 0.020 | 0.980 | 0.975 | 0.975 | 0.039 | 0.026 | 0.005 |
EBarrays | 0.210 | 0.016 | 0.984 | 0.790 | 0.790 | 0.032 | 0.211 | 0.042 | 0.025 | 0.020 | 0.980 | 0.975 | 0.975 | 0.039 | 0.024 | 0.005 |
BetaEB | 0.940 | 0.001 | 0.999 | 0.060 | 0.060 | 0.002 | 0.941 | 0.188 | 0.025 | 0.020 | 0.980 | 0.975 | 0.975 | 0.039 | 0.024 | 0.005 |
KW | 0.325 | 0.014 | 0.986 | 0.675 | 0.675 | 0.027 | 0.326 | 0.065 | 0.087 | 0.019 | 0.981 | 0.912 | 0.912 | 0.036 | 0.088 | 0.018 |
Proposed | 0.932 | 0.001 | 0.998 | 0.098 | 0.098 | 0.004 | 0.907 | 0.181 | 0.920 | 0.001 | 0.998 | 0.080 | 0.080 | 0.004 | 0.887 | 0.177 |
Results for the large-sample case (n1 = n2 = 15) | ||||||||||||||||
Methods | TPR | FPR | TNR | FNR | FDR | MER | AUC | pAUC | TPR | FPR | TNR | FNR | FDR | MER | AUC | pAUC |
Without outlying expressions | For 1 or 2 outliers with each of 5% genes | |||||||||||||||
ANOVA | 0.971 | 0.002 | 0.998 | 0.026 | 0.026 | 0.001 | 0.972 | 0.195 | 0.560 | 0.009 | 0.991 | 0.440 | 0.440 | 0.018 | 0.561 | 0.112 |
SAM | 0.978 | 0.000 | 1.000 | 0.022 | 0.022 | 0.001 | 0.979 | 0.196 | 0.613 | 0.008 | 0.992 | 0.388 | 0.388 | 0.015 | 0.612 | 0.122 |
LIMMA | 0.978 | 0.000 | 1.000 | 0.022 | 0.022 | 0.001 | 0.977 | 0.196 | 0.562 | 0.009 | 0.991 | 0.438 | 0.438 | 0.018 | 0.563 | 0.112 |
eLNN | 0.975 | 0.001 | 0.999 | 0.025 | 0.025 | 0.001 | 0.974 | 0.195 | 0.850 | 0.003 | 0.997 | 0.150 | 0.150 | 0.006 | 0.849 | 0.170 |
EBarrays | 0.973 | 0.001 | 0.999 | 0.028 | 0.028 | 0.001 | 0.974 | 0.195 | 0.562 | 0.009 | 0.991 | 0.438 | 0.438 | 0.018 | 0.561 | 0.112 |
BetaEB | 0.973 | 0.001 | 0.999 | 0.028 | 0.028 | 0.001 | 0.975 | 0.195 | 0.973 | 0.001 | 0.999 | 0.028 | 0.028 | 0.001 | 0.972 | 0.195 |
KW | 0.973 | 0.001 | 0.999 | 0.028 | 0.028 | 0.001 | 0.973 | 0.194 | 0.911 | 0.002 | 0.998 | 0.089 | 0.089 | 0.008 | 0.801 | 0.160 |
Proposed | 0.971 | 0.002 | 0.998 | 0.026 | 0.026 | 0.001 | 0.973 | 0.195 | 0.975 | 0.001 | 0.999 | 0.025 | 0.025 | 0.001 | 0.976 | 0.195 |
For 1 or 2 outliers with each of 10% genes | For 1 or 2 outliers with each of 75% genes | |||||||||||||||
ANOVA | 0.420 | 0.012 | 0.988 | 0.580 | 0.580 | 0.023 | 0.420 | 0.084 | 0.312 | 0.014 | 0.986 | 0.688 | 0.688 | 0.028 | 0.311 | 0.062 |
SAM | 0.497 | 0.010 | 0.990 | 0.502 | 0.502 | 0.020 | 0.497 | 0.099 | 0.347 | 0.013 | 0.987 | 0.652 | 0.652 | 0.026 | 0.347 | 0.069 |
LIMMA | 0.425 | 0.012 | 0.988 | 0.575 | 0.575 | 0.023 | 0.425 | 0.085 | 0.347 | 0.013 | 0.987 | 0.652 | 0.652 | 0.026 | 0.348 | 0.069 |
eLNN | 0.885 | 0.002 | 0.998 | 0.115 | 0.115 | 0.005 | 0.885 | 0.177 | 0.855 | 0.003 | 0.997 | 0.145 | 0.145 | 0.006 | 0.856 | 0.171 |
EBarrays | 0.427 | 0.012 | 0.988 | 0.573 | 0.573 | 0.023 | 0.427 | 0.085 | 0.228 | 0.016 | 0.984 | 0.772 | 0.772 | 0.031 | 0.227 | 0.045 |
BetaEB | 0.973 | 0.001 | 0.999 | 0.028 | 0.028 | 0.001 | 0.973 | 0.195 | 0.225 | 0.016 | 0.984 | 0.775 | 0.775 | 0.031 | 0.224 | 0.045 |
KW | 0.851 | 0.004 | 0.996 | 0.149 | 0.149 | 0.008 | 0.807 | 0.161 | 0.808 | 0.009 | 0.991 | 0.192 | 0.192 | 0.002 | 0.936 | 0.187 |
Proposed | 0.973 | 0.001 | 0.999 | 0.028 | 0.028 | 0.001 | 0.973 | 0.195 | 0.978 | 0.000 | 1.000 | 0.022 | 0.022 | 0.001 | 0.979 | 0.196 |
Average performance results of eight methods (ANOVA, SAM, LIMMA, eLNN, EBarrays, BetaEB, KW and Proposed) based on 100 datasets generated using a one-way ANOVA model with m = 2 groups/conditions and σ 2 = 0.05 for both sample sizes n1 = n2 = 3 and n1 = n2 = 15. Each dataset for each case contained 300 true DE genes, and the remainder were 19700 true EE genes. The performance indices/measures TPR, FPR, TNR, FNR, FDR, MER and AUC were calculated for each method based on the top 300 estimated DE genes, under the assumption that the other estimated genes in each dataset for each case were EE genes for each method. The performance measure ‘pAUC’ was calculated at FPR = 0.2 for each method and for each dataset.