Table 10.
Ratio of model performance between standard training and cross-dataset training using a leave-one-dataset-out strategy.
Model | Time | Frequency | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
KH | MS | RW-T | RW-W | UCI | WDM | Mean | KH | MS | RW-T | RW-W | UCI | WDM | Mean | |
KNN | 1.18x | 0.75x | 0.87x | 0.69x | 0.52x | 0.72x | 0.77x | 0.70x | 0.90x | 1.02x | 0.79x | 0.73x | 0.80x | 0.82x |
Random Forest | 0.67x | 0.68x | 0.76x | 0.63x | 0.75x | 0.69x | 0.70x | 0.76x | 0.89x | 0.83x | 0.93x | 0.86x | 0.80x | 0.84x |
SVM | 0.85x | 0.81x | 0.77x | 0.76x | 0.85x | 0.75x | 0.80x | 0.74x | 0.98x | 0.98x | 0.93x | 0.85x | 0.88x | 0.90x |
CNN (1D)12 | 0.81x | 0.86x | 1.03x | 0.93x | 0.85x | 0.78x | 0.87x | 0.90x | 0.93x | 0.96x | 0.85x | 0.82x | 0.82x | 0.88x |
CNN (2D)12 | 0.76x | 0.75x | 0.84x | 0.89x | 0.75x | 0.69x | 0.78x | 0.86x | 0.91x | 0.99x | 0.86x | 0.83x | 0.82x | 0.88x |
CNN PF34 | 0.78x | 0.71x | 0.93x | 0.83x | 0.77x | 0.64x | 0.77x | 0.89x | 0.91x | 1.08x | 0.84x | 0.82x | 0.81x | 0.88x |
CNN PFF34 | 0.79x | 0.71x | 0.96x | 0.84x | 0.77x | 0.64x | 0.77x | 0.90x | 0.92x | 1.08x | 0.85x | 0.81x | 0.82x | 0.89x |
ConvNet13 | 0.81x | 0.68x | 0.74x | 0.79x | 0.72x | 0.62x | 0.72x | 0.86x | 0.93x | 0.92x | 0.84x | 0.87x | 0.87x | 0.88x |
IMU CNN14 | 0.69x | 0.71x | 0.72x | 0.70x | 0.71x | 0.71x | 0.71x | 0.87x | 0.92x | 1.05x | 0.88x | 0.83x | 0.82x | 0.89x |
IMU Transf.14 | 0.86x | 0.91x | 0.57x | 0.79x | 1.00x | 1.30x | 0.88x | 0.95x | 1.08x | 1.07x | 0.88x | 0.97x | 1.27x | 1.03x |
MLP (2 Layers) | 0.73x | 0.85x | 0.96x | 0.88x | 0.86x | 0.75x | 0.83x | 0.85x | 0.92x | 0.95x | 0.80x | 0.81x | 0.81x | 0.85x |
MLP (3 layers) | 0.67x | 0.88x | 0.94x | 0.88x | 0.82x | 0.73x | 0.81x | 0.91x | 0.94x | 0.98x | 0.84x | 0.81x | 0.83x | 0.88x |
ResNet15 | 0.72x | 0.85x | 0.61x | 0.90x | 0.84x | 0.72x | 0.78x | 0.87x | 0.93x | 0.99x | 0.81x | 0.82x | 0.81x | 0.87x |
ResNetSE67 | 0.75x | 0.83x | 0.68x | 0.91x | 0.81x | 0.70x | 0.78x | 0.86x | 0.92x | 0.98x | 0.90x | 0.88x | 0.79x | 0.89x |
ResNetSE-567 | 0.59x | 0.79x | 0.67x | 0.96x | 0.79x | 0.63x | 0.73x | 0.92x | 0.92x | 0.98x | 0.84x | 0.82x | 0.87x | 0.89x |
Max (Ratio) | 0.77x | 0.82x | 0.97x | 0.85x | 0.84x | 0.78x | 0.86x | 0.90x | 0.92x | 0.92x | 0.84x | 0.86x | 0.86x | 0.88x |
Values close to one indicate no significant difference between the two scenarios, values below one indicate better performance in the standard training scenario, and values above one indicate better performance in the cross-dataset scenario. The Max (Ratio) line is the ratio between both maximum values of the two scenarios.