Skip to main content
. 2024 Mar 1;5(3):100945. doi: 10.1016/j.patter.2024.100945

Table 2.

Performance of several models trained using classical machine learning methods and federated learning methods, where the number of participating clients in the federation is n = 2, tested on the PDBP dataset

Algorithm name ROC-AUC AUC-PR Balanced accuracy Precision Recall F 0.5 F 1 F 2 Log loss Matthews correlation coefficient
AdaBoost classifier 0.834 ± 0.021 0.891 ± 0.015 0.697 ± 0.026 0.757 ± 0.023 0.917 ± 0.033 0.835 ± 0.016 0.834 ± 0.009 0.905 ± 0.004 0.639 ± 0.005 0.456 ± 0.029
Bagging 0.812 ± 0.01 0.871 ± 0.01 0.696 ± 0.019 0.753 ± 0.015 0.932 ± 0.015 0.828 ± 0.007 0.84 ± 0.004 0.903 ± 0.003 1.291 ± 0.226 0.463 ± 0.027
GradientBoosting classifier 0.856 ± 0.013 0.9 ± 0.016 0.716 ± 0.018 0.766 ± 0.014 0.938 ± 0.013 0.856 ± 0.007 0.857 ± 0.003∗ 0.908 ± 0.003∗ 0.572 ± 0.042∗ 0.502 ± 0.024
KNeighbors classifier 0.586 ± 0.024 0.735 ± 0.019 0.551 ± 0.019 0.664 ± 0.011 0.946 ± 0.024 0.707 ± 0.008 0.783 ± 0.001 0.899 ± 0.0 3.322 ± 0.641 0.169 ± 0.042
LinearDiscriminantAnalysis classifier 0.702 ± 0.012 0.794 ± 0.008 0.64 ± 0.013 0.734 ± 0.01 0.776 ± 0.01 0.622 ± 0.305 0.661 ± 0.324 0.751 ± 0.368 2.104 ± 0.19 0.288 ± 0.024
LogisticRegression classifier 0.771 ± 0.011 0.842 ± 0.009 0.657 ± 0.008 0.73 ± 0.005 0.901 ± 0.01 0.791 ± 0.004 0.81 ± 0.005 0.901 ± 0.001 0.996 ± 0.039 0.368 ± 0.02
MLP classifier 0.671 ± 0.013 0.826 ± 0.007 0.619 ± 0.012 0.711 ± 0.008 0.839 ± 0.01 0.749 ± 0.009 0.789 ± 0.007 0.899 ± 0.001 8.313 ± 0.708 0.265 ± 0.026
QuadraticDiscriminantAnalysis classifier 0.525 ± 0.022 0.721 ± 0.022 0.525 ± 0.022 0.671 ± 0.024 0.366 ± 0.097 0.688 ± 0.0 0.779 ± 0.0 0.898 ± 0.0 18.716 ± 1.33 0.05 ± 0.042
RandomForest 0.736 ± 0.006 0.825 ± 0.005 0.524 ± 0.005 0.649 ± 0.003 0.985 ± 0.005∗ 0.764 ± 0.007 0.792 ± 0.005 0.899 ± 0.0 0.596 ± 0.004 0.132 ± 0.025
SGD classifier 0.662 ± 0.017 0.845 ± 0.007 0.65 ± 0.016 0.728 ± 0.011 0.878 ± 0.024 0.758 ± 0.01 0.803 ± 0.007 0.898 ± 0.0 10.11 ± 0.525 0.343 ± 0.034
SVC classifier 0.701 ± 0.007 0.808 ± 0.004 0.593 ± 0.011 0.693 ± 0.006 0.844 ± 0.02 0.742 ± 0.004 0.793 ± 0.002 0.901 ± 0.001 0.65 ± 0.019 0.214 ± 0.029
XGBoost classifier 0.862 ± 0.008∗ 0.905 ± 0.007∗ 0.719 ± 0.021 0.77 ± 0.016 0.932 ± 0.013 0.864 ± 0.006∗ 0.857 ± 0.003∗ 0.906 ± 0.003 0.691 ± 0.031 0.504 ± 0.03
XGBoost Random Forest classifier 0.829 ± 0.007 0.89 ± 0.006 0.732 ± 0.02∗ 0.781 ± 0.016∗ 0.918 ± 0.01 0.849 ± 0.003 0.855 ± 0.002 0.905 ± 0.003 2.715 ± 0.254 0.515 ± 0.031∗
FedAvg LR 0.665 ± 0.128 0.826 ± 0.011 0.565 ± 0.052 0.673 ± 0.028 0.96 ± 0.032∗ 0.745 ± 0.045 0.794 ± 0.012 0.899 ± 0.002 0.829 ± 0.108 0.187 ± 0.147
FedAvg MLP 0.69 ± 0.018 0.78 ± 0.012 0.629 ± 0.01 0.719 ± 0.007 0.828 ± 0.022 0.744 ± 0.008 0.791 ± 0.007 0.899 ± 0.001 1.038 ± 0.239 0.282 ± 0.024
FedAvg SGD 0.775 ± 0.011 0.847 ± 0.008 0.689 ± 0.011 0.77 ± 0.008∗ 0.8 ± 0.01 0.794 ± 0.005 0.809 ± 0.004 0.902 ± 0.002 0.559 ± 0.013∗ 0.385 ± 0.023
FedAvg XGBRF 0.794 ± 0.007∗ 0.876 ± 0.009∗ 0.695 ± 0.023∗ 0.754 ± 0.017 0.919 ± 0.012 0.825 ± 0.007∗ 0.838 ± 0.008∗ 0.902 ± 0.003∗ 0.691 ± 0.0 0.451 ± 0.035∗
FedProx μ = 0.5 LR 0.704 ± 0.101 0.823 ± 0.015 0.584 ± 0.042 0.683 ± 0.022 0.943 ± 0.03 0.762 ± 0.038 0.795 ± 0.009 0.9 ± 0.002 0.866 ± 0.092 0.232 ± 0.115
FedProx μ = 0.5 MLP 0.7 ± 0.008 0.791 ± 0.006 0.63 ± 0.011 0.719 ± 0.007 0.833 ± 0.016 0.748 ± 0.007 0.794 ± 0.004 0.899 ± 0.001 1.312 ± 0.124 0.284 ± 0.023
FedProx μ = 2 LR 0.761 ± 0.008 0.835 ± 0.007 0.601 ± 0.005 0.691 ± 0.003 0.947 ± 0.013 0.787 ± 0.008 0.804 ± 0.003 0.9 ± 0.001 0.875 ± 0.014 0.293 ± 0.01
FedProx μ = 2 MLP 0.695 ± 0.022 0.785 ± 0.015 0.631 ± 0.02 0.722 ± 0.013 0.818 ± 0.02 0.747 ± 0.014 0.791 ± 0.005 0.899 ± 0.001 1.231 ± 0.285 0.282 ± 0.044

Data reported are mean and standard deviation across K = 6-fold cross-validation. Best performing algorithms for each metric are indicated by an asterisk.