Table 1.
Test accuracy of different methods
| Strong non-IID | MNIST | FashionMNIST | CIFAR-10 | |||
|---|---|---|---|---|---|---|
| Hard label | Soft label | Hard label | Soft label | Hard label | Soft label | |
| IndepLearn | 10.00 ± 0.00 | 10.00 ± 0.00 | 10.00 ± 0.00 | |||
| FedMD | 18.89 ± 0.30 | 88.71 ± 0.28 | 16.54 ± 0.25 | 64.63 ± 0.37 | 10.71 ± 0.38 | 15.78 ± 1.39 |
| FedED | 11.49 ± 0.25 | 11.92 ± 0.41 | 12.45 ± 0.44 | 12.52 ± 0.38 | 11.83 ± 0.26 | 12.04 ± 0.30 |
| DS-FL | 19.72 ± 0.32 | 35.25 ± 0.36 | 17.54 ± 0.11 | 35.98 ± 0.43 | 10.87 ± 0.25 | 12.07 ± 0.32 |
| FKD | 10.00 ± 0.00 | 10.00 ± 0.00 | 10.00 ± 0.00 | 10.00 ± 0.00 | 10.00 ± 0.00 | 10.00 ± 0.00 |
| PLS | 10.00 ± 0.00 | 10.00 ± 0.00 | 10.00 ± 0.00 | 10.00 ± 0.00 | 10.00 ± 0.00 | 10.00 ± 0.00 |
| Selective-FD | 85.92 ± 0.37 | 94.68 ± 0.52 | 73.41 ± 0.98 | 75.31 ± 0.29 | 80.22 ± 0.74 | 80.98 ± 0.39 |
| Weak Non-IID | MNIST | FashionMNIST | CIFAR-10 | |||
|---|---|---|---|---|---|---|
| Hard label | Soft label | Hard label | Soft label | Hard label | Soft label | |
| IndepLearn | 19.96 ± 0.00 | 19.82 ± 0.01 | 19.52 ± 0.02 | |||
| FedMD | 26.77 ± 0.57 | 95.16 ± 0.52 | 41.92 ± 0.30 | 74.83 ± 0.41 | 62.14 ± 0.22 | 84.31 ± 0.53 |
| FedED | 59.95 ± 1.11 | 60.26 ± 1.80 | 32.62 ± 1.09 | 37.12 ± 0.85 | 53.11 ± 0.34 | 56.13 ± 0.14 |
| DS-FL | 25.53 ± 1.43 | 47.87 ± 0.31 | 23.08 ± 0.23 | 39.22 ± 0.26 | 33.22 ± 0.54 | 52.51 ± 0.70 |
| FKD | 19.97 ± 0.01 | 19.98 ± 0.00 | 19.54 ± 0.13 | 19.71 ± 0.07 | 19.50 ± 0.02 | 19.51 ± 0.01 |
| PLS | 19.96 ± 0.01 | 19.97 ± 0.00 | 19.64 ± 0.09 | 19.70 ± 0.03 | 19.51 ± 0.01 | 19.52 ± 0.01 |
| Selective-FD | 86.82 ± 0.26 | 96.30 ± 0.25 | 75.57 ± 0.61 | 77.27 ± 0.31 | 81.06 ± 0.67 | 85.38 ± 0.35 |
| IID | MNIST | FashionMNIST | CIFAR-10 | |||
|---|---|---|---|---|---|---|
| Hard label | Soft label | Hard label | Soft label | Hard label | Soft label | |
| IndepLearn | 98.18 ± 0.04 | 86.07 ± 0.07 | 84.06 ± 0.03 | |||
| FedMD | 98.59 ± 0.04 | 98.63 ± 0.01 | 86.88 ± 0.02 | 87.25 ± 0.02 | 86.02 ± 0.09 | 86.31 ± 0.06 |
| FedED | 98.20 ± 0.11 | 98.26 ± 0.06 | 86.83 ± 0.14 | 86.88 ± 0.04 | 86.54 ± 0.07 | 86.87 ± 0.12 |
| DS-FL | 98.22 ± 0.14 | 98.56 ± 0.02 | 86.15 ± 0.07 | 86.62 ± 0.09 | 85.75 ± 0.08 | 85.82 ± 0.08 |
| FKD | 98.40 ± 0.05 | 98.44 ± 0.01 | 86.10 ± 0.15 | 86.14 ± 0.06 | 84.03 ± 0.02 | 84.10 ± 0.08 |
| PLS | 98.45 ± 0.02 | 98.48 ± 0.03 | 86.27 ± 0.08 | 86.52 ± 0.05 | 84.60 ± 0.13 | 84.77 ± 0.06 |
| Selective-FD | 98.55 ± 0.01 | 98.60 ± 0.04 | 86.92 ± 0.08 | 87.16 ± 0.06 | 85.94 ± 0.07 | 86.06 ± 0.16 |
Each experiment is repeated five times. The results in bold indicate the best performance, while the results underlined represent the second-best performance. In the non-IID settings, our Selective-FD method performs better than the baseline methods, and the accuracy gain is more significant when using hard labels in knowledge distillation than soft labels. In the IID scenario, all the methods achieve satisfactory accuracy.