Table 3.

Summary of results according to categories, data granularities, and approaches for the testing subset

Category (data granularity)	Measurement	Rule-based classifier [mean (95% CI)]	Machine learning classifiers [mean (95% CI)]
Documented Family or Friends (macro)	Sensitivity	0.954 (0.882–1.000)	0.875 (0.758–0.992)
	Specificity	0.990 (0.980–1.000)	0.971 (0.954–0.988)
	AUROC	0.972 (0.934–1.000)	0.923 (0.862–0.984)
Visits (micro)	Sensitivity	0.761 (0.644–0.878)	0.801 (0.740–0.861)
	Specificity	0.958 (0.936–0.980)	0.940 (0.916–0.964)
	AUROC	0.860 (0.800–0.919)	0.871 (0.839–0.902)
Visits (macro)	Sensitivity	0.856 (0.745–0.967)	0.674 (0.572–0.776)
	Specificity	0.908 (0.873–0.942)	0.871 (0.831–0.910)
	AUROC	0.882 (0.820–0.943)	0.772 (0.726–0.819)
Phone Calls (micro)	Sensitivity	0.915 (0.861–0.970)	0.800 (0.702–0.899)
	Specificity	0.970 (0.948–0.993)	0.780 (0.674–0.886)
	AUROC	0.943 (0.916–0.970)	0.790 (0.711–0.869)
Phone Calls (macro)	Sensitivity	0.980 (0.939–1.000)	0.689 (0.588–0.791)
	Specificity	0.969 (0.952–0.987)	0.776 (0.699–0.853)
	AUROC	0.975 (0.952–0.998)	0.733 (0.669–0.796)

Note: bold text refers to the best mean AUROC among the tested NLP approaches.