Algorithm 1.

The cluster Primal Dual Splitting Algorithm (cPDS).

Cluster PDS Method
INPUT: ∀j,
Prepare data/objectives fj and gj. Set parameters Γj and Θ j.
INITIALIZE: ∀j,
${\mathbf{x}}_j^0\in {\mathrm{ℝ}}^{d+1}$, ${\mathbf{y}}_j^0\in {\mathrm{ℝ}}^{n_j}$, ${\mathbf{q}}_j^{-1}=\mathit{0}$, ${\mathbf{q}}_j^0={\mathbf{\Gamma }}_j({\mathbf{A}}_j^{\top }{\mathbf{x}}_j^0-{\mathbf{y}}_j^0)$, ${\mathit{\lambda }}_j^{-1}=\mathit{0}$ and ${\mathbf{x}}_j^0={\mathbf{y}}_j^0-{\displaystyle {\sum }_{i\in {\mathcal{N}}_j\cup \left\{j\right\}}{w}_{ji}}{\mathbf{x}}_i^0$.
REPEAT
x – update (locally): ∀j
${\mathbf{x}}_j^{k+1}=\arg \underset{{\mathbf{x}}_j}{\min }\left\{\langle 2{\mathbf{q}}_j^k-{\mathbf{q}}_j^{k-1},{\mathbf{\Gamma }}_j{\mathbf{A}}_j{\mathbf{x}}_j\rangle +{\mathbf{g}}_j({\mathbf{x}}_j)+\langle 2{\mathit{\lambda }}_j^k-{\mathit{\lambda }}_j^{k-1},{\mathbf{x}}_j\rangle +0.5{\Vert {\mathbf{x}}_j-{\mathbf{x}}_j^k\Vert }_{{\mathbf{\Theta }}_j}^2\right\}$
y- update (locally): ∀j
${\mathbf{y}}_j^{k+1}=\arg \underset{{\mathbf{y}}_j}{\mathit{min}}\left\{{\mathbf{f}}_j({\mathbf{y}}_j)+\langle {\mathbf{q}}_j^k,-{\mathbf{\Gamma }}_j{\mathbf{y}}_j\rangle +0.5{\Vert {\mathbf{y}}_j-{\mathbf{A}}_j{\mathbf{x}}_j^{k+1}\Vert }_{{\mathbf{\Gamma }}_j^{\top }{\mathbf{\Gamma }}_j}^2\right\}$
q-update (locally): ∀j
${\mathbf{q}}_j^{k+1}={\mathbf{q}}_j^k+{\mathbf{\Gamma }}_j\left({\mathbf{A}}_j{\mathbf{x}}_j^{k+1}-{\mathbf{y}}_j^{k+1}\right)$
λ-update (requires information exchange): ∀j
${\mathit{\lambda }}_j^{k+1}={\mathit{\lambda }}_j^k+{\mathbf{x}}_j^{k+1}-{\displaystyle {\sum }_{i\in {\mathcal{N}}_j\cup \left\{j\right\}}{w}_{ji}}{\mathbf{x}}_i^{k+1}$.
UNTIL specific criteria are met.