Table 5.
Hyperparameter optimization strategy and optimal configurations.
| Model | Hyperparameters Tuned | Search Space | Optimization Method | Optimal Hyperparameters |
|---|---|---|---|---|
| Gradient Boosting (GBR) | n_estimators, learning_rate, max_depth | n_estimators ∈ {100, 200, 300}; learning_rate ∈ {0.01, 0.05, 0.1}; max_depth ∈ {3, 5, 7} | Grid Search (5-fold CV) | n_estimators = 200, learning_rate = 0.1, max_depth = 5 |
| XGBoost | n_estimators, learning_rate, max_depth, subsample | n_estimators ∈ {100, 200, 500}; learning_rate ∈ {0.01, 0.1, 0.2}; max_depth ∈ {3, 5, 7}; subsample ∈ {0.7, 0.8, 1.0} | Randomized Search (5-fold CV, 10 iterations) | n_estimators = 200, learning_rate = 0.1, max_depth = 5, subsample = 0.8 |
| LightGBM | n_estimators, learning_rate, max_depth, num_leaves | n_estimators ∈ {100, 200, 500}; learning_rate ∈ {0.01, 0.05, 0.1}; max_depth ∈ {3, 5, 7}; num_leaves ∈ {31, 50, 100} | Grid Search (5-fold CV) | n_estimators = 500, learning_rate = 0.05, max_depth = 5, num_leaves = 50 |
| CatBoost | iterations, learning_rate, depth, l2_leaf_reg | iterations ∈ {200, 500}; learning_rate ∈ {0.01, 0.05, 0.1}; depth ∈ {6, 8, 10}; l2_leaf_reg ∈ {1, 3, 5} | Grid Search (5-fold CV) | iterations = 500, learning_rate = 0.05, depth = 8, l2_leaf_reg = 3 |
| Neural Network (MLP) | Number of hidden units | units ∈ {32, 64, 96, 128} | Keras Tuner (Random Search) | Hidden units = 64, 32; epochs = 50; Adam optimizer |
| LSTM | Number of LSTM units | Fixed (empirical selection) | Manual tuning | LSTM units = 50; Dense = 25; epochs = 50 |
| Proposed Stacked Regressor | Meta-learner α (Ridge) | α ∈ {0.1, 1.0, 10.0} | Grid Search (5-fold CV) | Ridge α = 1.0 |