Figure 6. Hyperparameter tuning for machine learning algorithms by grid search.

The area under the receiver operating characteristic curve (AU-ROC) was used as the performance metric, which was estimated by 5-fold cross validation method. (A) Hyperparameter tuning for Adaboost which included three hyperparameters: the number of iterations for which boosting is run or the number of trees to use, learning coefficient and maximum depth of the trees. In Breiman method, $\alpha ={\displaystyle \frac{1}{2}\ln \left({\displaystyle \frac{1\mathrm{–}\mathrm{e}\mathrm{r}\mathrm{r}\mathrm{o}\mathrm{r}}{\mathrm{e}\mathrm{r}\mathrm{r}\mathrm{o}\mathrm{r}}}\right)}$ is used. The Freund uses $\alpha =\ln \left({\displaystyle \frac{1\mathrm{–}\mathrm{e}\mathrm{r}\mathrm{r}\mathrm{o}\mathrm{r}}{\mathrm{e}\mathrm{r}\mathrm{r}\mathrm{o}\mathrm{r}}}\right)$. In both cases the AdaBoost.M1 algorithm is used and α is the weight updating coefficient. On the other hand, if the Zhu method is used, the SAMME algorithm is implemented with $\alpha =\ln \left({\displaystyle \frac{1\mathrm{–}\mathrm{e}\mathrm{r}\mathrm{r}\mathrm{o}\mathrm{r}}{\mathrm{e}\mathrm{r}\mathrm{r}\mathrm{o}\mathrm{r}}}\right)+\ln (\mathrm{n}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r}\mathrm{o}\mathrm{f}\mathrm{c}\mathrm{l}\mathrm{a}\mathrm{s}\mathrm{s}-1)$. (B) Hyperparameter tuning for SVM. The cost and class weight were tuned in a grid. (C) Hyperparameter tuning for generalized linear model with regularization. Hyperparameters included alpha (Mixing Percentage) and lambda (Regularization Parameter). (D) Hyperparameter tuning for neural networks. Abbreviations: Adaboost: adaptive boosting with classification trees; SVM: support vector machines.