Algorithm 3. DynamicBL. dynamically allocate batch size and learning rate

• Input ${\mathit{s}}_{k,r}$, ${\eta }_{\max } \mathrm{maximum} \mathrm{learning} \mathrm{rate}, \beta \mathrm{number} \mathrm{of} \mathrm{SGD} \mathrm{update}$ • Output $\mathrm{Batch} \mathrm{size} b_{k,r}, \mathrm{learning} \mathrm{rate} {\eta }_{k,r}$ 1: $b_{k,r}\leftarrow \lfloor \frac{{{\displaystyle \sum }}_{𝓁}^L{s}_{k,r}^{𝓁}}{\beta }\rfloor$ 2: ${\eta }_{k,r}\leftarrow {\eta }_{\max }\times \arctan \left(b_{k,r}\right)$ 3: return $b_{k,r}, {\eta }_{k,r}$