Table 2.
Comparing the effect of adaptation on the performance of models designed to predict exercise-related changes in glucose
| Population model |
Personalized model, coefficient adaptation |
Comparison between population model and personalized model |
||||
|---|---|---|---|---|---|---|
| RMSE (MAE) [mg/dL] | [Sensitivity, specificity] (Accuracy) [%] | RMSE (MAE) [mg/dL] | [Sensitivity, specificity] (Accuracy) [%] | Δ MAE [%] | Δ Accuracy [%] | |
| Predicting minimum glucose at the end of exercise | ||||||
| MARS model | ||||||
| Training, 16-fold CV | 24.1 (19.2) | [73, 67] (69) | -- | – | -- | -- |
| Validation, Holdout Set | 26.5 (23.4) | [50, 86] (75) | 23.1 (19.6) | [70, 86] (81) | −16.2 | +8.3 |
| Validation, 20-fold CV | 24.6 (20.0) | [63, 63] (67) | 23.0 (18.1) | [61, 78] (78) | - 9.5 | +16.1a |
| MARS model + exercise history features | ||||||
| Training, 16-fold CV | 23.1 (18.2) | [75, 65] (68) | -- | – | -- | -- |
| Validation, HoldouaSet | 18.7 (14.3) | [73, 86] (81) | 19.7 (15.8) | [73, 95] (88) | +10.1 | +7.7 |
| Validation, 20-fold CV | 22.6 (17.6) | [66, 69] (70) | 22.1 (17.5) | [51, 83] (77) | - 0.6 | +10.1a |
| AR model: Population modelb | ||||||
| Training, 16-fold CV | 28.8 (22.7) | [71, 94] (83) | -- | – | -- | -- |
| Validation, Holdout Set | 32.8(28.6) | [59, 87] (72) | 27.6 (233) | [59, 87] (72) | −18.7 | +0 |
| Validation, 20-fold CV | 29.6 (23.8) | [71, 91] (81) | 27.7 (22.0) | [76, 90] (83) | - 7.4 | +3.1 |
| Logistic regression | ||||||
| Training, 16-fold CV | -- | [66, 67] (66) | -- | – | – | – |
| Validation, Holdout Set | -- | [73, 76] (75) | -- | [73, 90] (84) | – | +12.5 |
| Validation, 20-fold CV | -- | [64, 56] (61) | -- | [68, 61] (70) | – | +15.5a |
| Predicting minimum glucose 4 h after exercise | ||||||
| MARS modelb | ||||||
| Training, 16-fold CV | 25.8 (19.7) | [67, 68] (68) | -- | – | -- | -- |
| Validation, Holdout Set | 25.7 (21.6) | [18, 76] (56) | 21.5 (163) | [33, 96] (78) | - 24.8 | +38.9 |
| Validation, 20-fold CV | 25.1 (20.1) | [62, 51] (56) | 23.3 (18.3) | [56, 70] (68) | - 9.0 | +21.4a |
| MARS model + exercise history featuresb | ||||||
| Training, 16-fold CV | 24.8 (18.6) | [79, 61] (69) | -- | – | -- | -- |
| Validation, Holdout Set | 30.7 (26.1) | [29, 61] (47) | 23.0 (16.0) | [56, 96] (84) | −38.8 | +80.0 |
| Validation, 20-fold CV | 26.3 (21.1) | [74, 52] (57) | 23.9 (18.2) | [57, 70] (69) | - 13.8 | +20.0 |
| Logistic regressionb | ||||||
| Training, 16-fold CV | – | [57, 72] (65) | – | – | – | -- |
| Validation, Holdout Set | – | [32, 77] (50) | – | [53, 92] (69) | – | + 37.5 |
| Validation, 20-fold CV | – | [63, 50] (58) | – | [64, 74] (70) | – | +20.4a |
Values represent the mean performance across participants.
Training is performed with data from n = 16 participants, whereas the holdout set includes data from n = 4 participants.
The 20-fold validation includes data from all n = 20 participants.
indicates that the significance p < 0.05 determined Wilcoxon signed-rank test for paired, nonparametric data comparing the change in error or accuracy on a per-participant basis.
These models return predicted CGM, not SMBG values. The AR model is only designed to predict glucose approximately 43.2 min after the start of exercise, and the results for a 4 h prediction horizon are not shown.