Algorithm 3 Communication-Efficient Accurate Statistical Estimators (CEASE)
Input: Initial value θ0, regularizer α ≥ 0, number of iterations T.
For t = 0,1,2,⋯,T–1:
• Each machine evaluates ∇fk(θt) and sends to the 1st machine;
• The 1st machine computes $\nabla f({\theta }_t)=\frac{1}{m}\sum _{k=1}^m\nabla f_k({\theta }_t)$ and
${\theta }_{t+1}={\text{argmin}}_{\theta }\left\{f_1(\theta )+g(\theta )-\langle \nabla f_k({\theta }_t)-\nabla f({\theta }_t),\theta \rangle +\frac{\alpha }{2}\Vert \theta -{\theta }_t{\Vert }_2^2\right\}$,
and broadcasts to other machines.
Output: θT.