Fig. 6. Test accuracy and percentage p_proxyunder (top) different values ofτ_clientand (bottom) different values ofτ_server.

We denote the percentage of proxy samples selected for knowledge distillation as p_proxy. When τ_client is too large or τ_server is too small, the selectors filter out most of the proxy samples, leading to a small batch size and increased training variance. Conversely, when τ_client is too small, the local outputs may contain an excessive number of incorrect predictions, leading to a decrease in the effectiveness of knowledge distillation. Besides, when τ_server is too large, the ensemble predictions may exhibit high entropy, indicating ambiguous knowledge that could degrade local model training. These empirical results align with the analysis in Remark 1.