Algorithm 1. Nested cross-validation (nCV)

Input: Dataset D = ($x_i$, $y_i$), …,($x_m$, $y_m$) Set of hyperparameters ϴ Classifier C Integer k Outer fold: for each partition D into D1, D2,…,Dk   Inner fold:   Inner-fold data iD = concatenate iD1,…,iDk−1   partition iD into iD1, iD2,…,iDk   for θ in ϴ     for i = 1….k       $Ac{c}_{i,\theta }$= C(iDi, θ;$y_i$)      Acc(θ) = $\frac{1}{k}{\displaystyle \sum }_{i=1}^{k}Ac{c}_{i,\theta }$ // mean accuracy for HP set     totalAcc(θ) = ${\displaystyle \sum }_{i=1}^{k}Acc\left(\theta \right)$ // mean HP accuracy                   for all inner folds θ* = argmaxθ[totalAcc(θ)] // optimal set of hyper               parameters across all folds for i = 1….k $Ac{c}_{i,{\theta }^{*}}$ = C(Di, θ*) Acc(θ*) = $\frac{1}{k}{\displaystyle \sum }_{i=1}^{k}Ac{c}_{i,{\theta }^{*}}$, $y_i$ // mean test accuracy                 with a single set of HPs