Table 2:

Average performance and [95% confidence intervals] for logistic regression models using ICD Only, Med Only, Note Only, and ICD+MED+Note in the MGH+BIDMC training sets and MGH+BIDMC testing sets

Model Input	Accuracy	Specificity	F1-score	Recall	Precision	AUROC	AUPRC
ICD Only	0.54 [0.52–0.55]	0.37 [0.35–0.38]	0.60 [0.59–0.62]	0.7 [0.69–0.72]	0.53 [0.51–0.55]	0.54 [0.52–0.55]	0.69 [0.68–0.70]
Med Only	0.56 [0.55–0.58]	0.43 [0.41–0.45]	0.61 [0.60–0.63]	0.69 [0.68–0.71]	0.55 [0.53–0.57]	0.56 [0.55–0.58]	0.70 [0.69–0.71]
Note Only	0.88 [0.87–0.91]	0.90 [0.89–0.93]	0.88 [0.86–0.90]	0.87 [0.85–0.88]	0.88 [0.87–0.91]	0.94 [0.93–0.95]	0.94 [0.93–0.95]
ICD+MED+Note	0.89 [0.88–0.90]	0.90 [0.89–0.92]	0.88 [0.87–0.90]	0.88 [0.86–0.90]	0.88 [0.88–0.91]	0.95 [0.94–0.96]	0.95 [0.94–0.96]

AUROC: Area Under the Receiver Operating Characteristic curve, shows model’s ability to distinguish between classes.

AUPRC: Area Under the Precision-Recall Curve, summarizes the precision and recall across different thresholds.

Inputs: ICD Only: Models using only International Classification of Diseases codes. Med Only: Models using only medication data.

Note Only: Models using only textual note data. ICD+MED+Note: Models combining ICD codes, medication data, and textual note data.

Data Sets: MGH+ BIDMC: Data derived from Massachusetts General Hospital and Beth Israel Deaconess Medical Center.