Algorithm 1.

decentralized joint ICA (djICA)

Require: data $\{{\mathbf{X}}_{i,\text{red}}\in {ℝ}^{r\times N_i}:i=1,2,\dots ,s\},$ where r is the same across sites, tolerance level t = 10−6, j = 0, maximum iterations $J,\Vert {\mathrm{\Delta }}_{\mathbf{W}}(0){\Vert }_2^2=t,$ initial learning rate ρ = 0.015/ln(r)
1:	Initialize $\mathbf{W}\in {ℝ}^{r\times r}$	⊳ for example, W = I
2:	while $j<J,\Vert {\mathrm{\Delta }}_{\mathbf{W}}(j){\Vert }_2^2$ ≥ t do
3:	for all sites i = 1, 2, . . . , s do
4:	${\mathbf{Z}}_i(j)=\mathbf{W}(j-1){\mathbf{X}}_i+\mathbf{b}(j-1){\mathbf{1}}^{\top }$
5:	${\mathbf{Y}}_i(j)=g\left({\mathbf{Z}}_i(j)\right)$
6:	${\mathbf{G}}_i(j)=\rho (\mathbf{I}+(\mathbf{1}-2{\mathbf{Y}}_i(j)){\mathbf{Z}}_i{(j)}^{\top })\mathbf{W}(j-1)$
7:	${\mathbf{h}}_i(j)=\rho {\sum }_{m=1}^{N_i}(\mathbf{1}-2{\mathbf{y}}_{m,i}(j))$
8:	Send Gi(j) and hi(j) to the aggregator site.
9:	end for
10:	At the aggregator site, update global variables
11:	${\mathrm{\Delta }}_{\mathbf{W}}(j)={\sum }_{i=1}^s{\mathbf{G}}_i(j)$
12:	$\mathbf{W}(j)={\mathbf{W}}_i(j-1)+{\mathrm{\Delta }}_{\mathbf{W}}(j)$
13:	$\mathbf{b}(j)=\mathbf{b}(j-1)+{\sum }_{i=1}^s{\mathbf{h}}_i(j)$
14:	Check upper bound and learning rate adjustment.
15:	Send global W(j) and b(j) back to each site
16:	end while