Algorithm 4 CEASE with averaging
Input: Initial value θ0, regularizer α ≥ 0, number of iterations T.
For t = 0,1,2,⋯,T–1:
• Each machine evaluates ∇fk(θt) and sends to the central processor;
• The central processor computes $\nabla f({\theta }_t)=\frac{1}{m}\sum _{k=1}^m\nabla f_k({\theta }_t)$ and broadcasts to machines;
• Each machine computes
${\theta }_{t,k}={\text{argmin}}_{\theta }\left\{f_k(\theta )+g(\theta )-\langle \nabla f_k({\theta }_t)-\nabla f({\theta }_t),\theta \rangle +\frac{\alpha }{2}\Vert \theta -{\theta }_t{\Vert }_2^2\right\}$
and sends to the central processor;
• The central processor computes ${\theta }_{t+1}=\frac{1}{m}\sum _{k=1}^m{\theta }_{t,k}$ and broadcasts to machines.
Output: θT.