Algorithm 4 Coded sI-ADMM (csI-ADMM)

1:initialize: $\{z^1=x_i^1=y_i^1=\mathbf{0}|i\in \mathcal{N}\}$, batch size $\overline{M}$; 2:Local Data Allocation: 3:for$\mathrm{agent}i\in \mathcal{N}$do 4:    divide ${\mathcal{D}}_i$ labeled data based on repetition schemes in [34] and denote each partition as ${\mathbf{\xi }}_{i,j},j\in {\mathcal{K}}_i$; 5:    for $\mathrm{ECN}j\in {\mathcal{K}}_i$ do 6:        allocate ${\mathbf{\xi }}_{i,j}$ to ECN j; 7:        partition ${\mathbf{\xi }}_{i,j}$ examples into multiple batches with each size $(S_i+1)\overline{M}/K_i$; 8:    end for 9:end for 10:Updating Process: 11:for$k=1,2,\dots$do 12:    Steps of Active Agent $i=i_k=(k-1)modN+1$: 13:    run steps 13–14 of Algorithm 3 14:    ECN $j\in {\mathcal{K}}_i$ computes gradient in parallel: 15:      run step 16 of Algorithm 3 16:       select batch $$I_{i,j}^k=mmod\lfloor |{\mathbf{\xi }}_{i,j}|·K_i/(S_i+1)\overline{M}\rfloor ;$$ (18) 17:      update $g_{i,j}$ via encoding function $p_{enc}^j(·)$; 18:      transmit $g_{i,j}$ to the connected agent; 19:    until the $R_i$-th fast responded message is received; 20:    update gradient via decoding function $q_{dec}^i(·)$; 21:    run steps 22–26 of Algorithm 3; 22:end for