Figure 5.

A visual overview of the nested leave-one-experiment-out cross-validation used to optimize and evaluate the artificial neural network. The nested procedure consisted out of an outer loop and inner loop. The hyperparameters were adjusted in the inner loop to optimize a model selection criterion. The weights were adjusted in the outer loop to optimize a model fitting criterion. The visualization is limited to five experiments (E1–E5). The dashed line denotes that the visualization is given for a single iteration of the outer loop, visualizing the tuning procedure for left-out test experiment E1. For this single iteration of the outer loop, experiment 1 (E1) was left out as a true holdout test set. The remaining experiments (E2–E5) were iteratively used as a holdout validation set to optimize the network hyperparameters in the inner loop. The hyperparameter set that resulted in the lowest mean squared error (MSE) was used to fit the model on all experiments of the outer loop (E2–E5). Lastly, this trained model was used to evaluate the model predictions on the left-out test experiment (E1). This conservative approach ensures separation among model selection, model fitting, and model evaluation.