Table 2.

Comparison of outcomes for structured and structured-plus-unstructured surveillance against the clinician review (gold standard) for identifying Hypertension, Abnormal liver/renal function, Stroke history, Bleeding history or predisposition, Labile INR, Elderly, Drug/alcohol usage (HAS-BLED) and congestive heart failure, hypertension, age ≥75 years, diabetes mellitus, stroke or transient ischemic attack, vascular disease, age 65 to 74 years, sex category (CHA2DS2-VASc) components.

Method			HAS-BLED											CHA2DS2-VASc
			Structured surveillance		Structured and NLPa surveillance		Difference		T test		P value		Structured surveillance			Structured and NLP surveillance		Difference		Test statistic		P value
Sensitivity
	McNemar method	0.382		0.806		0.424		72		<.001		—b			—		—		—		—
	Exact binomial method	—		—		—		—				0.942			0.983		0.0413		—		.002
Specificity
	McNemar method	0.947		0.777		–0.17		16		<.001
	Exact binomial method	—		—		—		—		—		0.955			0.909		–0.0455				>.99c
PPVd
	Generalized score method	0.929		0.867		.061		4.487		.03		0.996			0.992		0.004		0.915		.34
NPVe
	Generalized score method	0.459		0.689		0.23		47.757		<.001		0.6			0.833		0.233		11.662		<.001

^aNLP: natural language processing.

^bThere is a small number of discordant cells, such that for the gold standard’s CHA₂DS₂-VASc <2, there is 1 case that was identified as CHA₂DS₂-VASc ≥2 in the structured and NLP method but not in the structured method. The exact binomial P value is calculated as

^cThere is a small number of discordant cells, such that for the gold standard’s CHA₂DS₂-VASc <2, there is 1 case that was identified as CHA₂DS₂-VASc >2 in the Structured and NLP method but not in the structured method. The exact binomial P value is calculated as

^dPPV: positive predictive value.

^eNPV: negative predictive value.