Table 5.
Overall metrics, per-class F1-score, calibration, and ACS hypnodensity graph similarity measures of the SSN models obtained from 10-fold cross-validation on IS-RC dataset, from 25-fold cross-validation on DOD-H dataset, and from 10-fold cross-validation on DOD-O dataset. Best shown in bold.
| Overall metrics | Per-class F1-score | Calibration | Hypn. | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Models | α | Acc. | MF1 | k | F1 | W | N1 | N2 | N3 | R | ECE | conf. | ACS | |
| IS-RC | base | — | 81.8 | 60.8 | 0.72 | 80.8 | 86.3 | 29.9 | 85.3 | 24.3 | 78.1 | 0.174 | 99.4 | 0.806 ± 0.052 |
| base+LS U | 0.3 | 82.5 | 59.8 | 0.72 | 81.1 | 86.5 | 28.8 | 86.5 | 18.7 | 78.7 | 0.169 | 99.3 | 0.811 ± 0.058 | |
| base+LS SC | 0.7 | 83.1 | 60.2 | 0.73 | 81.6 | 86.7 | 27.6 | 86.8 | 20.1 | 79.8 | 0.162 | 99.2 | 0.817 ± 0.047 | |
| DOD-H | base | — | 87.1 | 80.2 | 0.81 | 87.1 | 83.6 | 55.5 | 90.0 | 83.3 | 89.0 | 0.126 | 99.7 | 0.890 ± 0.047 |
| base+LS U | 0.4 | 87.6 | 81.0 | 0.81 | 87.5 | 85.5 | 57.3 | 90.2 | 82.1 | 90.3 | 0.120 | 99.5 | 0.899 ± 0.034 | |
| base+LS SC | 0.5 | 88.8 | 82.3 | 0.83 | 88.7 | 86.4 | 58.8 | 90.9 | 83.2 | 92.1 | 0.108 | 99.6 | 0.907 ± 0.039 | |
| DOD-O | base | — | 85.3 | 75.9 | 0.77 | 85.2 | 88.2 | 50.4 | 87.1 | 65.9 | 88.0 | 0.145 | 99.7 | 0.889 ± 0.056 |
| base+LS U | 0.1 | 85.6 | 75.8 | 0.78 | 85.2 | 88.2 | 51.2 | 87.3 | 64.3 | 88.4 | 0.141 | 99.6 | 0.893 ± 0.052 | |
| base+LS SC | 1 | 86.8 | 77.7 | 0.79 | 86.7 | 89.0 | 51.0 | 88.3 | 69.3 | 91.1 | 0.125 | 99.2 | 0.906 ± 0.043 | |