Table 2.
Comparison of models for translating transdermal data into estimates of BrAC for Skyn (Models 1–3) and SCRAM (Model 4)
| Model SM. (SCRAM Model) Machine Learning applied to SCRAM readings | Model 1. (Comparison Model) Linear Regression without Time Series Features | Model 2. (Comparison Model) Machine Learning without Time Series Features | Model 3. (Full Model) Machine Learning with Time Series Features | |
|---|---|---|---|---|
| MAE [95% CI] | .018 [.016, .020] | .022 [.020, .024] | .016 [.014, .018] | .010 [.008, .012] |
| RMSE[95%CI] | .021 [.019, .024] | .025 [.022, .028] | .018 [.016, .021] | .013 [.011, .015] |
| r [95% CI] | .764 [.736, .790] | .637 [.601, .671] | .776 [.751, .799] | .907 [.896, .917] |
| All Conditions | ||||
| % within .01 of BrAC | 62.1% | 12.7% | 61.5% | 70.8% |
| % within .02 of BrAC | 76.2% | 73.6% | 74.9% | 86.4% |
| % within .03 of BrAC | 87.2% | 83.6% | 87.3% | 94.5% |
| Alcohol Condition | ||||
| % within .01 of BrAC | 32.1% | 24.4% | 30.7% | 44.1% |
| % within .02 of BrAC | 57.3% | 48.9% | 57.0% | 73.3% |
| % within .03 of BrAC | 77.1% | 65.6% | 80.0% | 89.6% |
| No-Alcohol Condition | ||||
| % within .01 of BrAC | 99.8% | 2.1% | 89.7% | 95.1% |
| % within .02 of BrAC | 99.8% | 96.1% | 91.2% | 98.2% |
| % within .03 of BrAC | 99.8% | 100.0% | 93.9% | 98.9% |
MAE, RMSE, and r values are presented for data aggregated across all conditions (i.e., alcohol and no-alcohol). 95% confidence intervals are presented within brackets for MAE, RMSE, and r values above.
Model SM employed Extra-Trees machine learning to SCRAM readings, incorporating the closest SCRAM reading preceding a BrAC reading as a predictor (due to the sparse sampling interval of SCRAM, calculating TAC time series features was not an option). Model 1 employed linear regression including a single Skyn TAC value (TAC-reading taken immediately preceding BrAC reading) as a predictor. Model 2 employed Extra-Trees machine learning incorporating only the immediately preceding Skyn TAC value as a predictor. Model 3—the “full” (final) Skyn model—incorporated Extra-Trees machine learning with Skyn TAC time series features as predictors. All models were trained and tested using 4-fold participant-level cross-validation. N=67 SCRAM model, N=73 Skyn models.
MAE=Average absolute distance between measured BrAC and eBrAC, calculated per-participant and then averaged across participants; RMSE=Root mean squared error, also calculated per-participant and averaged across participants. “r” refers to the Pearson correlation between eBrAC and BrAC. Mixed models accounted for participant-level clustering of BrAC values. “% within XX of BrAC”=percentage of eBrAC values that fall within XX of measured BrAC (or, put differently, % MAE values <XX)