Table 3.
Performance evaluation based on PPARG benchmark data
| Methods | Missing | PPV (%) | NPV (%) | Specificity (%) | FPR (%) | Sensitivity (%) | FNR (%) | Accuracy (%) | MCC | AUC | hser-AUC | hspr-AUC |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Class one: function prediction methods | ||||||||||||
| FATHMM | 0 | 7.43 | 95.89 | 49.87 | 50.13 | 65.31 | 34.69 | 50.77 | 0.071 | 0.700 | 0.626 | 0.584 |
| fitCons | 0 | NA | 94.20 | 100.00 | 0.00 | 0.00 | 100.00 | 94.20 | NA | 0.539 | 0.550 | 0.512 |
| LRT | 0 | 7.84 | 99.44 | 29.59 | 70.41 | 97.28 | 2.72 | 33.52 | 0.140 | 0.642 | 0.505 | 0.623 |
| MutationAssessor | 7 | 13.63 | 98.63 | 66.71 | 33.29 | 85.03 | 14.97 | 67.78 | 0.252 | 0.852 | 0.625 | 0.599 |
| MutationTaster | 0 | 6.37 | 100.00 | 9.47 | 90.53 | 100.00 | 0.00 | 14.73 | 0.078 | 0.580 | 0.580 | 0.502 |
| PolyPhen2-HDIV | 0 | 7.78 | 99.56 | 28.46 | 71.54 | 97.96 | 2.04 | 32.49 | 0.139 | 0.755 | 0.630 | 0.523 |
| PolyPhen2-HVAR | 0 | 8.16 | 98.83 | 35.41 | 64.59 | 93.20 | 6.80 | 38.77 | 0.141 | 0.809 | 0.617 | 0.577 |
| PROVEAN | 0 | 13.47 | 98.92 | 65.00 | 35.00 | 88.44 | 11.56 | 66.36 | 0.257 | 0.855 | 0.654 | 0.593 |
| SIFT | 0 | 8.95 | 99.57 | 39.02 | 60.98 | 97.28 | 2.72 | 42.40 | 0.176 | 0.827 | 0.658 | 0.538 |
| VEST3 | 0 | 8.56 | 100.00 | 34.16 | 65.84 | 100.00 | 0.00 | 37.98 | 0.171 | 0.880 | 0.719 | 0.646 |
| Class two: conservation methods | ||||||||||||
| GERP++ | 0 | 6.16 | 97.81 | 9.35 | 90.65 | 96.60 | 3.40 | 14.41 | 0.049 | 0.596 | 0.533 | 0.533 |
| phastCons | 0 | 6.85 | 98.92 | 19.11 | 80.89 | 96.60 | 3.40 | 23.61 | 0.095 | 0.580 | 0.565 | 0.502 |
| phyloP | 0 | 6.69 | 98.09 | 19.36 | 80.64 | 93.88 | 6.12 | 23.69 | 0.080 | 0.744 | 0.525 | 0.538 |
| SiPhy | 0 | 6.95 | 98.09 | 23.64 | 76.36 | 92.52 | 7.48 | 27.64 | 0.090 | 0.618 | 0.568 | 0.568 |
| Class three: ensemble methods | ||||||||||||
| CADD | 0 | 8.02 | 100.00 | 29.38 | 70.62 | 100.00 | 0.00 | 33.48 | 0.154 | 0.773 | 0.692 | 0.531 |
| DANN | 0 | 7.30 | 97.65 | 31.31 | 68.69 | 87.76 | 12.24 | 34.58 | 0.097 | 0.666 | 0.609 | 0.519 |
| Eigen | 0 | 7.02 | 99.38 | 20.03 | 79.97 | 97.96 | 2.04 | 24.56 | 0.107 | 0.821 | 0.610 | 0.543 |
| FATHMM-MKL | 0 | 6.42 | 97.83 | 15.13 | 84.87 | 94.56 | 5.44 | 19.74 | 0.064 | 0.647 | 0.533 | 0.508 |
| GenoCanyon | 0 | 6.71 | 99.00 | 16.64 | 83.36 | 97.28 | 2.72 | 21.32 | 0.089 | 0.569 | 0.546 | 0.502 |
| M-CAP | 20 | 6.46 | 100.00 | 9.97 | 90.03 | 100.00 | 0.00 | 15.24 | 0.080 | 0.817 | 0.650 | 0.582 |
| MetaLR | 0 | 8.07 | 98.60 | 35.54 | 64.46 | 91.84 | 8.16 | 38.81 | 0.135 | 0.731 | 0.605 | 0.594 |
| MetaSVM | 0 | 9.12 | 98.86 | 43.63 | 56.37 | 91.84 | 8.16 | 46.43 | 0.168 | 0.719 | 0.613 | 0.522 |
| REVEL | 0 | 9.54 | 99.80 | 42.37 | 57.63 | 98.64 | 1.36 | 45.64 | 0.196 | 0.849 | 0.722 | 0.585 |
AUC, area under the curve; hser-AUC, high-sensitivity regional area under the curve; hspr-AUC, high-specificity regional area under the curve; FNR, false negative rate; FPR, false positive rate; MCC, Mathew correlation coefficient; NA, not available; NPV, negative predictive value; PPV, positive predictive value. We obtained experimentally validated 2533 missense mutations including 147 pathogenic mutations and 2386 benign mutations of PPARG gene from MITER database.