Figure 2.

Model performance using the combined MGH and BIDMC test data. (A) Receiver operating characteristic (ROC) curve using logistic regression with notes, ICD codes, and medications as input. The area under the ROC curve (AUROC) was 0.968 (95% CI 0.940-0.982). The optimal cutoff (according to the Youden index) achieved a false positive rate of 11.9% (230/1903; ie, specificity at 88.1%) and a true positive rate of 97.7% (852/870; ie, sensitivity at 97.7%). (B) Precision-recall (PR) curve using logistic regression with notes, ICD codes, and medications as input. The area under the PR curve (AUPRC) was 0.921 (95% CI 0.835-0.969). The optimal cutoff achieved a recall of 96.78% (842/870; ie, sensitivity at 96.78%) and a precision of 80.88% (842/1081). BIDMC: Beth Israel Deaconess Medical Center; ICD: International Classification of Diseases; MGH: Mass General Hospital.