Table 4.
Accuracy, F1 score, Sensitivity, Precision, Extraction time, Training time (TrT), and Test time (TsT) obtained by classifying features extracted by different combinations of CNN architectures and features classifiers
| CNN | Classifier | Accuracy (%) | F1 score (%) | Sensitivity (%) | Precision(%) | Extraction time (ms) | Training time (s) | Test time (ms) |
|---|---|---|---|---|---|---|---|---|
| NASNetMobile | Bayes | 80.392 ± 1.471 | 81.142±1.377 | 80.392±1.471 | 83.221±1.190 | 47.332 ± 1.928 | 0.5479±0.345 | 0.050±0.034 |
| MLP | 93.259±0.328 | 93.279±0.317 | 93.259±0.328 | 93.307±0.302 | 1283.897±158.429 | 0.026±0.009 | ||
| Nearest Neighbors | 89.044±1.203 | 89.169±1.092 | 89.044±1.203 | 89.535±0.847 | 0.383±0.020 | 14.627±0.402 | ||
| RF | 87.150±2.894 | 87.599±2.602 | 87.150±2.894 | 89.496±1.298 | 153.024±4.645 | 4.862±2.414 | ||
| SVM Linear | 83.038±5.165 | 83.900±4.831 | 83.038±5.165 | 89.341±1.828 | 74.485±4.740 | 10.390±0.257 | ||
| SVM Polynomial | 63.771±18.362 | 51.538±20.011 | 63.771±18.362 | 44.039±18.362 | 64.385±5.740 | 37.749±0.293 | ||
| SVM RBF | 87.628±4.863 | 86.544±7.129 | 87.628±4.863 | 88.903±3.641 | 225.685±37.106 | 15.320±2.733 | ||
| Xception | Bayes | 87.986±0.522 | 88.345±0.495 | 87.986±0.522 | 89.477±0.440 | 108.338 ± 1.728 | 1.196±0.717 | 0.024±0.004 |
| MLP | 95.188±0.335 | 95.166±0.332 | 95.188±0.335 | 95.166±0.333 | 3592.964±259.828 | 0.204±0.111 | ||
| Nearest Neighbors | 93.618±0.954 | 93.651±0.936 | 93.618±0.954 | 93.709±0.909 | 2.7631±1.055 | 28.666±0.401 | ||
| RF | 92.321±0.833 | 92.061±0.967 | 92.321±0.833 | 92.466±0.660 | 1206.235±31.802 | 6.891±1.732 | ||
| SVM Linear | 95.666±0.379 | 95.661±0.377 | 95.666±0.379 | 95.662±0.375 | 148.377±2.667 | 14.083±0.498 | ||
| SVM Polynomial | 76.331±6.758 | 67.134±11.181 | 76.331±6.758 | 60.526±14.612 | 127.277±3.767 | 72.632±0.698 | ||
| SVM RBF | 94.983±0.932 | 94.881±0.979 | 94.983±0.932 | 95.030±0.907 | 178.985±33.990 | 26.651±0.142 | ||
| MobileNet | Bayes | 91.553±0.648 | 91.732±0.617 | 91.553±0.648 | 92.276±0.517 | 19.447 ± 0.651 | 0.5849±0.504 | 0.047±0.061 |
| MLP | 95.358±0.475 | 95.336±0.464 | 95.358±0.475 | 95.342±0.478 | 584.750±67.986 | 0.157±0.149 | ||
| Nearest Neighbors | 95.137±0.291 | 95.162±0.283 | 95.137±0.291 | 95.214±0.267 | 2.291±2.210 | 12.140±0.364 | ||
| RF | 94.317±0.533 | 94.278±0.534 | 94.317±0.533 | 94.284±0.547 | 469.650±12.416 | 6.070±2.112 | ||
| SVM Linear | 95.290±0.595 | 95.311±0.586 | 95.290±0.595 | 95.348±0.570 | 75.606±15.417 | 5.587±0.111 | ||
| SVM polynomial | 96.177±0.723 | 96.161±0.727 | 96.177±0.723 | 96.172±0.734 | 62.206±18.617 | 8.524±0.915 | ||
| SVM RBF | 95.853±0.751 | 95.844±0.726 | 95.853±0.751 | 95.866±0.694 | 139.965±12.778 | 9.206±2.161 | ||
| DenseNet121 | Bayes | 86.826±0.783 | 86.631±0.769 | 86.826±0.783 | 86.584±0.774 | 72.683 ± 0.749 | 0.526±0.661 | 0.016±0.351 |
| MLP | 95.717±0.302 | 95.717±0.311 | 95.717±0.302 | 95.722±0.320 | 2019.860±67.986 | 0.039±0.010 | ||
| Nearest neighbors | 94.198±0.378 | 94.251±0.364 | 94.198±0.378 | 94.378±0.328 | 1.996±2.720 | 14.402±0.355 | ||
| RF | 94.556±0.360 | 94.528±0.336 | 94.556±0.360 | 94.574±0.332 | 507.653±11.816 | 5.871±1.613 | ||
| SVM linear | 94.881±0.591 | 94.948±0.572 | 94.881±0.591 | 95.155±0.509 | 134.274±47.817 | 6.073±0.305 | ||
| SVM polynomial | 66.126±20.064 | 56.317±24.237 | 66.126±20.064 | 51.149±25.882 | 101.674±49.817 | 36.615±0.194 | ||
| SVM RBF | 95.768±0.220 | 95.772±0.197 | 95.768±0.220 | 95.808±0.142 | 222.004±88.078 | 9.322±2.201 | ||
| DenseNet169 | Bayes | 90.324±0.555 | 89.985±0.614 | 90.324±0.555 | 90.283±0.565 | 90.651 ± 0.830 | 0.918±0.541 | 0.019± 0.092 |
| MLP | 96.007±0.394 | 95.997±0.375 | 96.007±0.394 | 96.022±0.359 | 1797.904±109.458 | 0.169±0.078 | ||
| Nearest neighbors | 94.522±0.419 | 94.559±0.405 | 94.522±0.419 | 94.633±0.374 | 2.355±2.210 | 23.841±0.495 | ||
| RF | 94.590±0.417 | 94.550±0.407 | 94.590±0.417 | 94.580±0.420 | 1533.250±61.574 | 5.874±1.953 | ||
| SVM linear | 95.802±0.662 | 95.840±0.630 | 95.802±0.662 | 95.957±0.498 | 223.547±27.513 | 8.92±0.512 | ||
| SVM polynomial | 45.410±22.488 | 31.527±24.508 | 45.410±22.488 | 25.678±22.488 | 183.123±31.613 | 59.115±0.406 | ||
| SVM RBF | 96.212±0.444 | 96.197±0.433 | 96.212±0.444 | 96.228±0.434 | 191.653±40.839 | 14.366±2.462 | ||
| DenseNet201 | Bayes | 90.904±0.352 | 90.639±0.401 | 90.904±0.352 | 90.844±0.350 | 114.376 ± 0.898 | 3.107±1.722 | 0.021±0.092 |
| MLP | 96.331±0.683 | 96.329±0.677 | 96.331±0.683 | 96.338±0.682 | 596.263±88.912 | 0.149±0.129 | ||
| Nearest neighbors | 94.898±0.698 | 94.926±0.687 | 94.898±0.698 | 94.983±0.670 | 1.025±0.415 | 27.472±0.093 | ||
| RF | 94.949±0.453 | 94.950±0.461 | 94.949±0.453 | 94.970±0.461 | 508.184±13.306 | 4.064±2.110 | ||
| SVM linear | 95.717±0.238 | 95.768±0.238 | 95.717±0.238 | 95.943±0.268 | 138.142±33.954 | 10.613±0.759 | ||
| SVM polynomial | 63.771±18.362 | 51.538±20.011 | 63.771±18.362 | 44.039±18.362 | 108.342±38.154 | 32.620±0.362 | ||
| SVM RBF | 96.416±0.492 | 96.415±0.489 | 96.416±0.492 | 96.427±0.484 | 284.151±23.886 | 17.863±4.141 | ||
| VGG16 | Bayes | 88.840±0.837 | 88.066±0.913 | 88.840±0.837 | 89.360±0.946 | 96.921 ± 1.470 | 0.154±0.0526 | 0.015±0.01 |
| MLP | 95.444±0.284 | 95.427±0.284 | 95.444±0.284 | 95.425±0.284 | 1850.59±107.380 | 0.037±0.026 | ||
| Nearest neighbors | 94.966±0.498 | 94.926±0.495 | 94.966±0.498 | 94.933±0.502 | 0.3745±0.155 | 7.947±0.504 | ||
| RF | 94.078±0.605 | 94.070±0.593 | 94.078±0.605 | 94.073±0.588 | 939.220±35.012 | 5.864±2.330 | ||
| SVM linear | 94.863±0.799 | 94.921±0.777 | 94.863±0.799 | 95.077±0.711 | 50.683±15.664 | 3.585±0.209 | ||
| SVM polynomial | 67.491±21.467 | 57.576±25.827 | 67.491±21.467 | 51.849±26.917 | 47.132±18.164 | 17.610±1.386 | ||
| SVM RBF | 96.007±0.288 | 96.001±0.289 | 96.007±0.288 | 96.000±0.292 | 69.159±2.775 | 5.302±0.027 | ||
| VGG19 | Bayes | 88.089±0.861 | 87.246±0.978 | 88.089±0.861 | 88.513±0.938 | 121.694 ± 1.517 | 0.2276±0.075 | 0.03±0.03 |
| MLP | 95.461±0.668 | 95.455±0.648 | 95.461±0.668 | 95.471±0.624 | 1850.59±107.380 | 0.133±0.094 | ||
| Nearest neighbors | 94.608±0.907 | 94.563±0.921 | 94.608±0.907 | 94.566±0.923 | 0.288±0.148 | 8.173±0.734 | ||
| RF | 94.164±0.653 | 94.113±0.653 | 94.164±0.653 | 94.114±0.665 | 118.690±3.456 | 7.271±1.175 | ||
| SVM linear | 95.529±0.299 | 95.573±0.291 | 95.529±0.299 | 95.698±0.271 | 28.096±4.723 | 3.373±0.276 | ||
| SVM polynomial | 60.887±28.226 | 52.255±34.683 | 60.887±28.226 | 49.182±36.603 | 22.136±5.123 | 18.145±0.440 | ||
| SVM RBF | 96.468±0.644 | 96.461±0.644 | 96.468±0.644 | 96.463±0.647 | 43.972±14.761 | 5.570±2.032 | ||
| InceptionV3 | Bayes | 86.826±0.783 | 86.631±0.769 | 86.826±0.783 | 86.584±0.774 | 66.000 ± 0.908 | 0.989±0.322 | 0.039±0.029 |
| MLP | 93.925±0.549 | 93.897±0.555 | 93.925±0.549 | 93.893±0.558 | 3270.333±294.801 | 0.158±0.128 | ||
| Nearest neighbors | 92.167±0.739 | 92.171±0.743 | 92.167±0.739 | 92.190±0.744 | 2.854±2.347 | 28.876±0.498 | ||
| RF | 92.372±0.631 | 92.227±0.711 | 92.372±0.631 | 92.331±0.607 | 1805±69.116 | 5.075±2.292 | ||
| SVM linear | 93.669±0.469 | 93.734±0.463 | 93.669±0.469 | 93.880±0.454 | 188.825±36.993 | 20.035±1.538 | ||
| SVM polynomial | 54.590±22.488 | 41.533±24.508 | 54.590±22.488 | 34.858±22.488 | 110.250±39.490 | 72.814±0.460 | ||
| SVM RBF | 93.072±2.697 | 92.840±3.106 | 93.072±2.697 | 93.211±2.355 | 378.215±57.102 | 30.017±4.826 | ||
| InceptionResNetV2 | Bayes | 81.280±0.759 | 81.791±0.698 | 81.280±0.759 | 82.893±0.652 | 158.771 ± 1.248 | 1.36211±0.467 | 0.017±0.064 |
| MLP | 93.754±0.469 | 93.709±0.459 | 93.754±0.469 | 93.774±0.393 | 724.022±97.302 | 0.250±0.117 | ||
| Nearest neighbors | 88.959±1.053 | 89.282±0.969 | 88.959±1.053 | 90.360±0.629 | 1.615±1.445 | 20.464±0.019 | ||
| RF | 91.092±1.158 | 91.003±1.109 | 91.092±1.158 | 91.055±1.088 | 800.504±30.570 | 3.056±1.800 | ||
| SVM linear | 92.935±0.450 | 93.025±0.453 | 92.935±0.450 | 93.259±0.492 | 113.940±22.140 | 14.001±0.738 | ||
| SVM polynomial | 74.863±2.974 | 68.497±8.693 | 74.863±2.974 | 65.793±15.524 | 101.340±19.140 | 51.355±6.868 | ||
| SVM RBF | 93.908±0.844 | 93.868±0.854 | 93.908±0.844 | 93.883±0.871 | 128.050±18.991 | 18.942±2.977 | ||
| ResNet50 | Bayes | 86.433±1.384 | 86.768±1.269 | 86.433±1.384 | 87.614±0.936 | 57.701 ± 1.134 | 1.362±0.461 | 0.046± 0.027 |
| MLP | 94.539±0.280 | 94.531±0.286 | 94.539±0.280 | 94.538±0.297 | 724.0229±97.079 | 0.159±0.083 | ||
| Nearest neighbors | 93.618±0.671 | 93.670±0.653 | 93.618±0.671 | 93.784±0.622 | 1.616±1.447 | 28.474±0.012 | ||
| RF | 93.584±0.587 | 93.537±0.570 | 93.584±0.587 | 93.545±0.582 | 800.504±30.570 | 3.464±1.508 | ||
| SVM linear | 94.539±0.831 | 94.587±0.804 | 94.539±0.831 | 94.717±0.739 | 113.940±22.140 | 14.714±0.577 | ||
| SVM polynomial | 27.048±0.000 | 11.517±0.000 | 27.048±0.000 | 7.316±0.000 | 99.340±25.340 | 72.765±0.299 | ||
| SVM RBF | 94.983±0.513 | 94.935±0.516 | 94.983±0.513 | 94.949±0.526 | 128.052±18.853 | 25.461± 5.514 | ||
| NASNetLarge | Bayes | 82.611±1.138 | 83.107±1.032 | 82.611±1.138 | 84.261±0.774 | 313.715 ± 2.359 | 1.492±0.522 | 0.042±0.113 |
| MLP | 94.437±1.004 | 94.411±1.009 | 94.437±1.004 | 94.430±1.011 | 2486.365±282.362 | 0.215±0.102 | ||
| Nearest neighbors | 89.863±1.141 | 90.085±1.060 | 89.863±1.141 | 90.733±0.823 | 2.436±0.428 | 56.197±0.496 | ||
| RF | 91.809±1.220 | 91.722±1.190 | 91.809±1.220 | 91.815±1.110 | 862.818±27.096 | 5.496±2.070 | ||
| SVM linear | 94.488±0.585 | 94.527±0.563 | 94.488±0.585 | 94.611±0.516 | 304.663±31.350 | 36.452±0.933 | ||
| SVM polynomial | 36.229±18.362 | 21.522±20.011 | 36.229±18.362 | 16.497±18.362 | 283.125±34.120 | 143.020±0.448 | ||
| SVM RBF | 93.942±1.473 | 93.818±1.615 | 93.942±1.473 | 93.975±1.365 | 441.346±6.816 | 52.329±7.522 |
Bold values highlight the performance of the proposed algorithm