Table 2.

Performance characteristics of natural language processing classifier to identify patient-reported herpes zoster events associated with arthritis medications of interest

	Manual review of all posts mentioning shingles
Automated Treato NLP algorithm	Post contained shingles report	Post did not contain shingles report	Total
Shingles case detected in post	213	20	233	PPV (precision) = 91% (87–95%)
Shingles case was not detected in post	89	103	192	NPV = 54% (46–61%)
Total	302	123	425
	Sensitivity (recall) = 71% (65–76%)	Specificity = 84% (76–90%)

Performance characteristics are reported as proportion (95% confidence interval). NLP natural language processing, PPV positive predictive value, NPV negative predictive value