Table 1.
Performance of model on Training set.
| Datasets (Model) | Feature Encoding | Tenfold CV test (before WD) | Tenfold CV test (after WD) | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Acc | Sen | Spe | MCC | AUC | Acc | Sen | Spe | MCC | AUC | ||
| ACPmain (SVM) | CPSR | 0.761 | 0.714 | 0.808 | 0.526 | 0.749 | 0.736 | 0.711 | 0.761 | 0.473 | 0.801 |
| HOG-PSSM | 0.554 | 0.328 | 0.783 | 0.142 | 0.607 | 0.732 | 0.768 | 0.696 | 0.465 | 0.774 | |
| ProtT5 | 0.700 | 0.689 | 0.710 | 0.401 | 0.786 | 0.737 | 0.667 | 0.809 | 0.480 | 0.791 | |
| ESM-2 | 0.652 | 0.676 | 0.627 | 0.304 | 0.712 | 0.877 | 0.872 | 0.881 | 0.756 | 0.945 | |
| F1 + F2 | 0.668 | 0.607 | 0.728 | 0.339 | 0.738 | 0.906 | 0.913 | 0.899 | 0.812 | 0.986 | |
| F1 + F2 + F3 | 0.784 | 0.768 | 0.801 | 0.570 | 0.869 | 0.768 | 0.720 | 0.816 | 0.539 | 0.827 | |
| F1 + F2 + F4 | 0.700 | 0.689 | 0.710 | 0.401 | 0.786 | 0.778 | 0.732 | 0.824 | 0.559 | 0.854 | |
| ACPmain (CDF) | CPSR | 0.720 | 0.627 | 0.813 | 0.448 | 0.789 | 0.730 | 0.691 | 0.769 | 0.463 | 0.811 |
| HOG-PSSM | 0.739 | 0.721 | 0.756 | 0.480 | 0.802 | 0.663 | 0.634 | 0.692 | 0.328 | 0.711 | |
| ProtT5 | 0.737 | 0.716 | 0.759 | 0.477 | 0.823 | 0.692 | 0.660 | 0.724 | 0.387 | 0.764 | |
| ESM-2 | 0.753 | 0.708 | 0.797 | 0.508 | 0.832 | 0.914 | 0.926 | 0.901 | 0.833 | 0.963 | |
| F1 + F2 | 0.734 | 0.671 | 0.797 | 0.471 | 0.817 | 0.964 | 0.955 | 0.972 | 0.930 | 0.991 | |
| F1 + F2 + F3 | 0.919 | 0.916 | 0.922 | 0.845 | 0.974 | 0.918 | 0.904 | 0.932 | 0.843 | 0.972 | |
| F1 + F2 + F4 | 0.972 | 0.965 | 0.980 | 0.947 | 0.992 | 0.975 | 0.961 | 0.990 | 0.953 | 0.994 | |
| ACPmain (NB) | CPSR | 0.628 | 0.749 | 0.507 | 0.264 | 0.714 | 0.681 | 0.609 | 0.754 | 0.366 | 0.742 |
| HOG-PSSM | 0.604 | 0.498 | 0.710 | 0.212 | 0.617 | 0.676 | 0.580 | 0.772 | 0.362 | 0.756 | |
| ProtT5 | 0.658 | 0.730 | 0.586 | 0.321 | 0.719 | 0.812 | 0.787 | 0.768 | 0.855 | 0.865 | |
| ESM-2 | 0.653 | 0.707 | 0.599 | 0.308 | 0.687 | 0.792 | 0.817 | 0.768 | 0.587 | 0.870 | |
| F1 + F2 | 0.652 | 0.601 | 0.702 | 0.306 | 0.695 | 0.830 | 0.805 | 0.855 | 0.663 | 0.871 | |
| F1 + F2 + F3 | 0.807 | 0.839 | 0.775 | 0.617 | 0.881 | 0.818 | 0.848 | 0.788 | 0.639 | 0.889 | |
| F1 + F2 + F4 | 0.839 | 0.811 | 0.866 | 0.681 | 0.879 | 0.853 | 0.829 | 0.878 | 0.710 | 0.893 | |
| ACPAlter (SVM) | CPSR | 0.901 | 0.867 | 0.936 | 0.805 | 0.953 | 0.804 | 0.608 | 1.000 | 0.661 | 0.944 |
| HOG-PSSM | 0.837 | 0.813 | 0.861 | 0.681 | 0.909 | 0.876 | 0.876 | 0.876 | 0.753 | 0.921 | |
| ProtT5 | 0.896 | 0.882 | 0.910 | 0.794 | 0.951 | 0.903 | 0.909 | 0.897 | 0.807 | 0.957 | |
| ESM-2 | 0.916 | 0.928 | 0.905 | 0.834 | 0.973 | 0.923 | 0.910 | 0.936 | 0.846 | 0.973 | |
| F1 + F2 | 0.886 | 0.894 | 0.878 | 0.773 | 0.952 | 0.903 | 0.885 | 0.922 | 0.807 | 0.972 | |
| F1 + F2 + F3 | 0.834 | 0.698 | 0.970 | 0.696 | 0.935 | 0.871 | 0.753 | 0.987 | 0.762 | 0.954 | |
| F1 + F2 + F4 | 0.812 | 0.640 | 0.984 | 0.666 | 0.942 | 0.865 | 0.727 | 1.000 | 0.757 | 0.977 | |
| ACPAlter (CDF) | CPSR | 0.883 | 0.848 | 0.919 | 0.770 | 0.955 | 0.878 | 0.840 | 0.915 | 0.761 | 0.958 |
| HOG-PSSM | 0.839 | 0.805 | 0.872 | 0.683 | 0.912 | 0.832 | 0.784 | 0.880 | 0.669 | 0.913 | |
| ProtT5 | 0.907 | 0.889 | 0.925 | 0.816 | 0.964 | 0.909 | 0.889 | 0.928 | 0.819 | 0.967 | |
| ESM-2 | 0.928 | 0.954 | 0.902 | 0.857 | 0.979 | 0.998 | 0.997 | 0.999 | 0.996 | 1.000 | |
| F1 + F2 | 0.920 | 0.897 | 0.943 | 0.842 | 0.974 | 0.997 | 0.997 | 0.996 | 0.994 | 1.000 | |
| F1 + F2 + F3 | 0.968 | 0.977 | 0.959 | 0.936 | 0.994 | 0.997 | 0.996 | 0.999 | 0.995 | 1.000 | |
| F1 + F2 + F4 | 0.954 | 0.946 | 0.963 | 0.909 | 0.988 | 0.998 | 0.997 | 0.999 | 0.996 | 1.000 | |
| ACPAlter (NB) | CPSR | 0.800 | 0.809 | 0.791 | 0.602 | 0.882 | 0.717 | 0.533 | 0.900 | 0.466 | 0.760 |
| HOG-PSSM | 0.806 | 0.758 | 0.855 | 0.619 | 0.886 | 0.718 | 0.730 | 0.705 | 0.444 | 0.812 | |
| ProtT5 | 0.854 | 0.814 | 0.893 | 0.712 | 0.923 | 0.736 | 0.603 | 0.873 | 0.497 | 0.835 | |
| ESM-2 | 0.862 | 0.914 | 0.811 | 0.729 | 0.929 | 0.831 | 0.833 | 0.828 | 0.661 | 0.917 | |
| F1 + F2 | 0.853 | 0.805 | 0.901 | 0.711 | 0.882 | 0.932 | 0.867 | 1.000 | 0.873 | 0.933 | |
| F1 + F2 + F3 | 0.890 | 0.938 | 0.842 | 0.784 | 0.951 | 0.898 | 0.800 | 1.000 | 0.814 | 0.900 | |
| F1 + F2 + F4 | 0.872 | 0.832 | 0.911 | 0.748 | 0.904 | 0.932 | 0.867 | 1.000 | 0.873 | 0.967 | |
| ACP740 (SVM) | CPSR | 0.692 | 0.470 | 0.921 | 0.435 | 0.847 | 0.811 | 0.810 | 0.811 | 0.624 | 0.899 |
| HOG-PSSM | 0.753 | 0.797 | 0.708 | 0.509 | 0.832 | 0.709 | 0.750 | 0.667 | 0.424 | 0.811 | |
| ProtT5 | 0.690 | 0.470 | 0.917 | 0.431 | 0.850 | 0.864 | 0.863 | 0.866 | 0.732 | 0.917 | |
| ESM-2 | 0.888 | 0.877 | 0.900 | 0.779 | 0.947 | 0.946 | 0.895 | 1.000 | 0.898 | 0.999 | |
| F1 + F2 | 0.865 | 0.867 | 0.863 | 0.730 | 0.925 | 0.941 | 0.937 | 0.945 | 0.888 | 0.992 | |
| F1 + F2 + F3 | 0.848 | 0.823 | 0.873 | 0.699 | 0.892 | 0.871 | 0.867 | 0.876 | 0.745 | 0.921 | |
| F1 + F2 + F4 | 0.868 | 0.830 | 0.907 | 0.743 | 0.922 | 0.895 | 0.893 | 0.897 | 0.792 | 0.948 | |
| ACP740 (CDF) | CPSR | 0.814 | 0.793 | 0.835 | 0.635 | 0.898 | 0.836 | 0.807 | 0.866 | 0.675 | 0.889 |
| HOG-PSSM | 0.748 | 0.740 | 0.756 | 0.500 | 0.847 | 0.757 | 0.763 | 0.750 | 0.519 | 0.849 | |
| ProtT5 | 0.968 | 0.977 | 0.959 | 0.936 | 0.994 | 0.789 | 0.793 | 0.783 | 0.582 | 0.882 | |
| ESM-2 | 0.873 | 0.870 | 0.876 | 0.749 | 0.939 | 0.980 | 0.970 | 0.990 | 0.962 | 0.996 | |
| F1 + F2 | 0.858 | 0.860 | 0.856 | 0.717 | 0.930 | 0.988 | 0.987 | 0.990 | 0.977 | 1.000 | |
| F1 + F2 + F3 | 0.875 | 0.863 | 0.887 | 0.753 | 0.933 | 0.985 | 0.977 | 0.993 | 0.971 | 0.999 | |
| F1 + F2 + F4 | 0.968 | 0.977 | 0.959 | 0.936 | 0.994 | 0.992 | 0.990 | 0.993 | 0.983 | 0.999 | |
| ACP740 (NB) | CPSR | 0.695 | 0.520 | 0.876 | 0.422 | 0.753 | 0.717 | 0.533 | 0.900 | 0.466 | 0.760 |
| HOG-PSSM | 0.718 | 0.730 | 0.705 | 0.444 | 0.812 | 0.718 | 0.730 | 0.705 | 0.444 | 0.812 | |
| ProtT5 | 0.736 | 0.603 | 0.873 | 0.497 | 0.835 | 0.736 | 0.603 | 0.873 | 0.497 | 0.835 | |
| ESM-2 | 0.821 | 0.760 | 0.883 | 0.652 | 0.852 | 0.831 | 0.833 | 0.828 | 0.661 | 0.917 | |
| F1 + F2 | 0.787 | 0.713 | 0.862 | 0.586 | 0.821 | 0.932 | 0.867 | 1.000 | 0.873 | 0.933 | |
| F1 + F2 + F3 | 0.836 | 0.777 | 0.897 | 0.681 | 0.864 | 0.898 | 0.800 | 1.000 | 0.814 | 0.900 | |
| F1 + F2 + F4 | 0.829 | 0.777 | 0.883 | 0.665 | 0.875 | 0.932 | 0.867 | 1.000 | 0.873 | 0.967 | |
Best results highlighted in bold.