Skip to main content
. 2026 Sep 22;14(10):2139. doi: 10.3390/biomedicines14102139
Algorithm 1. APDHE-Pareto ensemble-selection procedure within each training partition
Input: Training partition Dtrain candidate classifier set M; allowable ensemble sizes K=2,3,4,5,6
Output: Selected ensemble E* and decision threshold t*
  1. For each candidate classifier, generate calibrated training-OOF probability predictions using the nested model-development procedure.

  2. Using the OOF predictions, calculate ROC-AUC, balanced accuracy, F1-score, and MCC for each candidate classifier at a common probability threshold of 0.50.

  3. Transform MCC to the [0, 1] interval and assign Pareto layers by sequential non-dominated sorting across ROC-AUC, balanced accuracy, F1-score, and transformed MCC.

  4. Calculate model-level prediction diversity from pairwise disagreement and transformed Q-statistic measures.

  5. Calculate model stability as 1/[1+SD(AUC)], where SD(AUC) is obtained from the fold-level OOF AUC values.

  6. Rank candidate classifiers lexicographically by: lower Pareto-layer index, higher diversity, higher stability, higher ROC-AUC, higher balanced accuracy, higher F1-score, higher transformed MCC, and alphabetical model name as the final deterministic tie-breaker.

  7. For each allowable ensemble size K∈{2,3,4,5,6}, define the candidate ensemble as the first K classifiers in the ranked list. Non-prefix subsets are not evaluated.

  8. For each candidate ensemble, uniformly average the calibrated OOF probabilities of its constituent classifiers.

  9. Determine an OOF decision threshold for each candidate ensemble by maximizing Youden’s J; when equivalent thresholds occur, prioritize higher balanced accuracy and then the threshold closest to 0.50.

  10. Compare candidate ensemble sizes lexicographically by higher OOF ROC-AUC, lower OOF Brier score, higher OOF balanced accuracy, and smaller ensemble size.

  11. Select the highest-ranked candidate ensemble as E*.

  12. Recalculate the fused OOF probabilities for E* and determine the final training-derived threshold t* using the same Youden’s J rule.

  13. Fit the corresponding calibrated candidate models on the complete current training partition.

  14. Apply the fixed ensemble composition E*,  uniform probability averaging, and fixed threshold t*  to the unseen validation or test observations.