Table 2.
Metrics of prediction performance.
Prediction model metrics | ||||
---|---|---|---|---|
2A: Models trained with PGRN-AMPS escitalopram, PGRN-AMPS citalopram, and CO-MED escitalopram + placebo patients. | ||||
Model Set 1 (Metabolomic) | Model Set 2 (Multi-omics) | |||
XGBoost | Penalized regression | XGBoost | Penalized regression | |
Testing-set metrics | ||||
AUC | 0.76 | 0.85 | 0.83 | 0.86 |
Accuracy | 0.727 | 0.766 | 0.761 | 0.775 |
NIR | 0.62 | 0.62 | 0.63 | 0.63 |
p-Value | 0.053 | 0.0045 | 0.017 | 0.0067 |
Sensitivity | 0.75 | 0.69 | 0.69 | 0.71 |
Specificity | 0.69 | 0.90 | 0.88 | 0.88 |
Training-set metrics (in cross-validation) | ||||
AUC | 0.69 | 0.69 | 0.68 | 0.72 |
Accuracy | 0.64 | 0.66 | 0.67 | 0.65 |
NIR | 0.68 | 0.68 | 0.69 | 0.69 |
2B: Models trained with PGRN-AMPS escitalopram and PGRN-AMPS citalopram patients. | ||||
Testing-set metrics | ||||
AUC | 0.75 | 0.84 | 0.74 | 0.86 |
Accuracy | 0.753 | 0.753 | 0.732 | 0.775 |
p-Value | 0.026 | 0.026 | 0.085 | 0.0067 |
Sensitivity | 0.65 | 0.73 | 0.80 | 0.71 |
Specificity | 0.93 | 0.79 | 0.62 | 0.88 |
Training-set metrics (in cross-validation) | ||||
AUC | 0.68 | 0.68 | 0.72 | 0.72 |
Accuracy | 0.64 | 0.65 | 0.67 | 0.68 |
NIR | 0.69 | 0.69 | 0.70 | 0.70 |
For the penalized regression models, cartesian grid search was used to tune penalty and mixture hyperparameters, with 20 evenly spaced penalty values ranging from 1e − 6 to 10 and mixture values of 0, 0.05, 0.2, 0.4, 0.6, 0.8, and 1. For the XGBoost models, we tuned the number of trees, tree depth, minimum number of data points to split a node, learning rate, loss function reduction, and sample size using Bayesian optimization and a stopping criterion of no improvement over 30 iterations.