Algorithm 2 Procedure of BOHB algorithm.

1 : Input the number of maximum budget R, setting $\eta$

2 : Initialization the number of setting S${}_{m}ax$ = ceil(log$\eta$ R)

3 : for s = S${}_{m}ax$ to 0 do

4 :     set current Configuration A

5 :           for i = 0 to s do

6 :                  Select hyperparameter Configuration A${}_i$

7 :                  Get loss L using Configuration A${}_i$

8 :                  A = min(L(A), L(A${}_i$))

9 :           for t = 1 to T do

10 :                  Calculate a probability function p(g|D) using Gaussian process

11 :                  Select observation where x${}_{s}elect$ = $argma{x}_x\in$x a(x)

12 :                  Evaluate the objective function y${}_{s}elect$ = g(x${}_{s}elect$) + $ϵ$

13 :                  Add dataset D = D ∪ ( x${}_{s}elect$, y${}_{s}elect$ )

14 :                  Update best observation x${}_{b}est$ = $argmin$${}_x\in Dg\left(x\right)$

15 :     end for

16 : end for