Table 3.
Instrumental activities of daily living classifier performance. Italic values represent the optimal performance in each data set.
| Model | AUROCa (95% CI) | AUPRCb (95% CI) | ||
| Filtered cohort | ||||
| Training set | ||||
| Deep learning | 0.948 (0.931-0.964) | 0.677 (0.617-0.736) | ||
| Bio+Clinical BERTc | 0.860 (0.797-0.918) | 0.730 (0.625-0.826) | ||
| Logistic regression | 0.970 (0.958-0.980) | 0.714 (0.656-0.766) | ||
| LASSOd | 0.961 (0.945-0.975) | 0.704 (0.644-0.758) | ||
| Random forest | 0.966 (0.951-0.979) | 0.722 (0.668-0.774) | ||
| SVMe | 0.968 (0.955-0.980) | 0.735 (0.679-0.786) | ||
| XGBoost | 0.970 (0.956-0.981) | 0.703 (0.644-0.765) | ||
| Validation set | ||||
| Deep learning | 0.806 (0.243-1.00) | 0.551 (0.003-1.00) | ||
| Bio+Clinical BERT | 0.830 (0.777-0.876) | 0.758 (0.679-0.818) | ||
| Logistic regression | 0.952 (0.901-0.998) | 0.396 (0.067-0.803) | ||
| LASSO | 0.978 (0.935-0.999) | 0.414 (0.155-0.869) | ||
| Random forest | 0.941 (0.863-0.998) | 0.309 (0.062-0.744) | ||
| SVM | 0.934 (0.792-0.998) | 0.430 (0.125-0.831) | ||
| XGBoost | 0.995 (0.988-0.999) | 0.528 (0.255-0.925) | ||
| Unfiltered validation cohort | ||||
| Deep learning | 0.794 (0.191-1.00) | 0.568 (0.002-1.00) | ||
| Bio+Clinical BERT | 0.750 (0.499-1.00) | 0.584 (0.001-1.00) | ||
| Logistic regression | 0.960 (0.869-1.00) | 0.538 (0.014-1.00) | ||
| LASSO | 0.986 (0.972-0.999) | 0.271 (0.042-0.833) | ||
| Random forest | 0.945 (0.828-1.00) | 0.521 (0.011-1.00) | ||
| SVM | 0.959 (0.867-1.00) | 0.456 (0.022-1.00) | ||
| XGBoost | 0.991 (0.972-1.00) | 0.552 (0.050-1.00) | ||
aAUROC: area under the receiver operating characteristic curve.
bAUPRC: area under the precision-recall curve.
cBERT: bidirectional encoder representations from transformers.
dLASSO: least absolute shrinkage and selection operator.
eSVM: support vector machine.