Algorithm 4.

Private DOLA Using Mini-Batches

1:	Input: $f_t^i(w):=\mathcal{l}\left(w,x_t^i,y_t^i\right),i\in [1,m]$ and $t\in [1,T]$; doubly stochastic matrix At = (aij(t)) ∈ Rm×m.
2:	for t = 0,...,T do
3:	for each node i = 1,...,m do
4:	$b_t^i=\sum _{j=1}^m{a}_{ij}(t+1)\left(w_t^j+{\sigma }_t^j\right)$, where ${\sigma }_t^j$ is a Laplace noise vector in ${ℝ}^n$;
5:	$g_k^i\leftarrow \nabla f_k^i\left(b_t^i\right)$, which is computed based on examples $\left(x_k,y_k\right)\in H_t$;
6:	$w_{t+1}^i=\mathrm{Pro}\left[b_t^i-{\alpha }_{t+1}\left(\phi w_t^i+\frac{1}{h}{\sum }_{\left(x_k,y_k\right)\in H_t}{g}_k^i\right)\right]$;  (Projection onto W)
7:	broadcast the output $\left(w_{t+1}^i+{\sigma }_{t+1}^i\right)\text{to}\mathcal{G}{(t)}_i$;
8:	end for
9:	end for