Algorithm 1 Scaffold [18]

Global inputs: Initial ${\mathsf{\Theta }}_G^0$, $c_g^0$, and global step size ${\eta }_g$.

Client inputs: Local client datasets $D=\{D^1,\dots ,D^K\}$, initial $c_k^0$, and local step size ${\eta }_l$.

Output: Global model weights ${\mathsf{\Theta }}_G^R$.

1:for round $i=1,\dots ,R$ do   2:   download ${\mathsf{\Theta }}_G^i$ and $c_g^i$ for all K clients   3:   for client k in K do   4:         for optimization step $s=1,\dots ,S$ do   5:          Compute gradient $g_k\left({\mathsf{\Theta }}_k^i\left(x_s\right)\right)$   6:          ${\mathsf{\Theta }}_k^i\stackrel{}{\leftarrow }{\mathsf{\Theta }}_k^i-{\eta }_l(g_k\left({\mathsf{\Theta }}_k^i\left(x_s\right)\right)-c_k^i+c_g^i)\left)\right)$   7:         $c_k^{+}\stackrel{}{\leftarrow }$ (i) $g_k\left({\mathsf{\Theta }}_k^i\right)$, or (ii) $c_k^i-c_g^i+\frac{1}{S{\eta }_l}({\mathsf{\Theta }}_G^i-{\mathsf{\Theta }}_k^i)$   8:         upload $(\Delta {\mathsf{\Theta }}_k^i,\Delta c_k^i,c_k^i)\stackrel{}{\leftarrow }({\mathsf{\Theta }}_k^i-{\mathsf{\Theta }}_G^i,c_k^{+}-c_k^i,c_k^{+})$   9:   $(\Delta {\mathsf{\Theta }}_G^i,\Delta c^i)\stackrel{}{\leftarrow }\frac{1}{K}\sum (\Delta {\mathsf{\Theta }}_k^i,\Delta c_k^i)$ 10:   ${\mathsf{\Theta }}_G^{i+1}\stackrel{}{\leftarrow }{\mathsf{\Theta }}_G^i+{\eta }_g\Delta {\mathsf{\Theta }}_G^i$ and $c_g^{i+1}\stackrel{}{\leftarrow }{c}_g^i+\Delta c^i$