Skip to main content
. 2023 Jan 17;23(3):1065. doi: 10.3390/s23031065
Algorithm 1: Model training and hyperparameter optimization for the standard AI models.
Require: Sets of model hyperparameter values to evaluate      ▹ Cartesian product
Ensure: Average model performance across all folds
  • 1:

    for each external fold iteration do                ▹ External 5-fold CV

  • 2:

        Hold-out the samples of this fold (i.e., the test set)

  • 3:

        Use the rest of samples as calibration set

  • 4:

        for each hyperparameter set do

  • 5:

            for each internal fold iteration in the calibration set do    ▹ Internal 5-fold CV

  • 6:

               Hold-out the samples of this fold (i.e., the validation set)

  • 7:

               Fit the model on the rest (i.e., the training set)

  • 8:

               Use the model to predict the held-out samples

  • 9:

               Calculate and store an accuracy metric for this fold

  • 10:

            end for

  • 11:

            Calculate the average performance across all folds

  • 12:

        end for

  • 13:

        Determine the optimal hyperparameter set

  • 14:

        Fit the final model using the calibration set with the optimal hyperparameters

  • 15:

        Use the model to predict the held-out samples

  • 16:

        Calculate and store accuracy metrics for this fold

  • 17:

    end for

  • 18:

    Calculate the average performance across all folds