Table 3.

Overview of the test results on the nested cross-validation using the full dataset

Prediction model	Accuracy	F1-score	Precision	Recall	Specificity	BACC	MCC
Feature-based models
Elastic Net	0.900 ± 0.002	0.842 ± 0.006	0.881 ± 0.015	0.808 ± 0.022	0.946 ± 0.009	0.877 ± 0.007	0.771 ± 0.006
SVM	0.900 ± 0.001	0.843 ± 0.003	0.880 ± 0.012	0.809 ± 0.014	0.945 ± 0.007	0.877 ± 0.004	0.771 ± 0.002
Random Forest	0.867 ± 0.005	0.768 ± 0.010	0.904 ± 0.004	0.667 ± 0.013	0.965 ± 0.001	0.816 ± 0.007	0.692 ± 0.012
XGBoost	0.908 ± 0.005	0.855 ± 0.008	0.896 ± 0.013	0.818 ± 0.019	0.953 ± 0.007	0.885 ± 0.007	0.790 ± 0.011
MLP	0.902 ± 0.003	0.843 ± 0.007	0.896 ± 0.006	0.796 ± 0.018	0.954 ± 0.004	0.875 ± 0.007	0.775 ± 0.008
Hybrid sequence-based models
LSTM_BasicDesc	0.877 ± 0.006	0.795 ± 0.022	0.891 ± 0.040	0.723 ± 0.064	0.954 ± 0.023	0.838 ± 0.021	0.719 ± 0.014
Bi-LSTM_BasicDesc	0.891 ± 0.001	0.829 ± 0.004	0.860 ± 0.009	0.802 ± 0.014	0.935 ± 0.006	0.868 ± 0.004	0.751 ± 0.004
Purely sequence-based models
MLP_Embedding	0.871 ± 0.019	0.803 ± 0.013	0.824 ± 0.078	0.795 ± 0.056	0.908 ± 0.055	0.852 ± 0.006	0.713 ± 0.030
LSTM	0.866 ± 0.021	0.768 ± 0.064	0.890 ± 0.073	0.697 ± 0.132	0.949 ± 0.040	0.823 ± 0.048	0.697 ± 0.045
Bi-LSTM	0.871 ± 0.014	0.798 ± 0.024	0.842 ± 0.079	0.775 ± 0.094	0.919 ± 0.056	0.847 ± 0.023	0.713 ± 0.023
vanilla-Transformer	0.883 ± 0.009	0.802 ± 0.020	0.913 ± 0.020	0.716 ± 0.039	0.966 ± 0.009	0.841 ± 0.016	0.732 ± 0.021
BigBird	0.884 ± 0.004	0.811 ± 0.004	0.876 ± 0.023	0.756 ± 0.017	0.947 ± 0.013	0.851 ± 0.003	0.732 ± 0.008
ProLaTherm	0.970 ± 0.004	0.955 ± 0.005	0.963 ± 0.015	0.947 ± 0.005	0.982 ± 0.008	0.964 ± 0.002	0.933 ± 0.008

Each cell shows the mean and standard deviation across the three outer folds for the given evaluation metric and prediction model. The prediction models are grouped as feature-based as well as hybrid and purely sequence-based. The best result for each evaluation metric is highlighted in bold. ProLaTherm outperforms all comparison partners.