Table 2.

A comparison of the risk prediction ability among the risk models. (1) Risk Model 1 utilizing the structured dataset (i.e., OASIS and EHR); (2) Risk Model 2 utilizing both the structured dataset and clinical notes processed using machine learning-based NLP approaches (concerning/not concerning note); (3) Risk Model 3 utilizing both the structured dataset and clinical notes processed with the Omaha System; (4) Risk Model 4 utilizing both the structured dataset and clinical notes processed using both machine learning-based NLP approaches and with the Omaha System.

	Sensitivity (Precision)	PPV (Recall)	F-score	PRC Area
Logistic regression
Risk Model 1	0.794	0.64	0.709	0.736
Risk Model 2	0.812	0.652	0.723	0.756
Risk Model 3	0.833	0.683	0.751	0.774
Risk Model 4	0.837	0.694	0.759	0.812
Random Forest
Risk Model 1	0.896	0.692	0.781	0.818
Risk Model 2	0.909	0.693	0.786	0.84
Risk Model 3	0.918	0.707	0.799	0.845
Risk Model 4	0.927	0.721	0.811	0.864
Bayes Network
Risk Model 1	0.721	0.643	0.680	0.71
Risk Model 2	0.749	0.708	0.728	0.757
Risk Model 3	0.815	0.72	0.765	0.795
Risk Model 4	0.827	0.762	0.793	0.836
SVM
Risk Model 1	0.801	0.675	0.733	0.765
Risk Model 2	0.82	0.687	0.748	0.784
Risk Model 3	0.902	0.697	0.786	0.807
Risk Model 4	0.922	0.731	0.815	0.821
Naïve Bayes
Risk Model 1	0.702	0.65	0.675	0.688
Risk Model 2	0.721	0.677	0.698	0.701
Risk Model 3	0.692	0.661	0.676	0.682
Risk Model 4	0.702	0.682	0.692	0.684

Note: PPV: positive predictive value; PRC: precision-recall curve; SVM: support vector machine