A Cluster-Driven Adaptive Training Approach for Federated Learning

. 2022 Sep 18;22(18):7061. doi: 10.3390/s22187061

Algorithm 2 Algorithm of CATA-Fed

Input: Set of connected clients C , the number of global iteration E , deadline T, the number of cluster P, learning rate

η

, the number of participating clients N, lower limit size of sub-dataset

π

Output: Global model W

1:
procedure Server( $E, k$ )
2:
Initialize $W_{0}, A_{i} [0] (1 \leq i \leq | C |)$
3:
$X_{m} (1 \leq m \leq P) \leftarrow$ client clustering according to Equation (20)
4:
for global iteration $0 \leq t \leq E - 1$ do
5:
if $0 < t$ , $S_{t - 1}^{D} \neq \emptyset$ and detect report of client $c_{i} \in S_{t - 1}^{S}$ then
6:
Reclustering $X_{m} (1 \leq m \leq P)$ according to Equation (20)
7:
end if
8:
Update $A_{i} [t]$ of client $c_{i} \in C$ according to Equation (21)
9:
Grouping clients $G_{x}$ according to Equation (23)
10:
Calculate priority $p_{x}$ according to Equation (24)
11:
$S_{t} \leftarrow G_{x}^{*}$ according to Equation (25)
12:
for each client $c_{i} \in S_{t}$ in parallel do
13:
Broadcast $W_{t}, T$ to client $c_{i}$
14:
$W_{t}^{i}, I_{i} \leftarrow$ ClientUpdate $(c_{i}, W_{t}, T)$
15:
end for
16:
Update global model $W_{t + 1} \leftarrow \sum_{i = 1}^{| S_{t} |} ψ_{i} I_{i} W_{t}^{i}$
17:
$C \leftarrow C - S_{t}^{D}$
18:
end for
19:
end procedure
20:
21:
procedureclientupdate( $c_{i}, W_{t}, T$ )
22:
Initialize $I_{i} = 1$
23:
$n_{i} \leftarrow n_{i} + 1$
24:
Select training data $D_{i, j}$ , $j = n_{i} m o d (2^{u_{i, t}}) + 1$
25:
Replace local model $w_{t}^{i}$ ← $W_{t}$
26:
Check start training time $τ_{s t a r t}$
27:
Working BreakProcess( $T, τ_{s t a r t}, I_{i}$ ) in parallel
28:
for local epoch $0 \leq e$ do
29:
if $(T - ε_{i}) - \sum_{u = 0}^{e - 1} τ_{u}^{i} < τ_{m e a n}^{i}$ then
30:
Break training
31:
else
32:
$w_{t, e + 1}^{i} = w_{t, e}^{i} - η ▿ F_{i} (w_{t, e}^{i}, D_{i, j}) \times I_{i}$
33:
Get training time $τ_{e}^{i}$
34:
Update average of training time $τ_{m e a n}^{i}$
35:
end if
36:
end for
37:
Upload $W_{t}^{i}, I_{x}$ to server
38:
if $I_{x}$ is 0 then
39:
Update $u_{i, t}$ according to Equation (10)
40:
Partitioning sub-dataset $D_{i, j}, j \in [1, 2^{u_{i, t}}]$ according to Equation (11)
41:
Report $| D_{i, j} |, j \in [1, 2^{u_{i, t}}]$ to server
42:
end if
43:
end procedure
44:
45:
procedurebreakprocess( $T, τ_{s t a r t}, I_{i}$ )
46:
if $(T - ε_{i}) \leq τ_{n o w} - τ_{s t a r t}$ then
47:
Break training & $I_{i} = 0$
48:
end if
49:
end procedure