Table 2.

Performance of algorithms to identify definite or probable pseudogout in an electronic health record (EHR) dataset

	Performance among gold-standard labels (N=900)					Cases identified in EHR dataset (N=30,089)
Algorithm	Sensitivity	Specificity	PPV	AUC	F-score
≥1 billing codea	0.65	0.63	0.22	0.64	0.32	12,035
≥3 billing codesa	0.46	0.79	0.26	0.63	0.32	7,213
Presence of CPP crystalsb	0.29	1.00	0.92	0.64	0.44	1,630
Topic modeling approachc	0.29	0.98	0.79	0.86	0.42	1,870
Combined algorithm: topic modeling approach and/or presence of CPP crystals	0.42	0.98	0.81	0.70	0.55	2,490

^aAmong ICD-9 or 10 billing codes for chondrocalcinosis or calcium metabolism disorder: ICD-9 712.1*, 712.2*, 712.3*, 275.49; ICD-10 M11.1*, M11.2*, M11.8*, E83.59. Adapted from Bartels CM, et al. J Clin Rheumatol 2015;21(4):189–92, which only included ICD-9 codes, by also including ICD-10 codes

^bPresence of synovial fluid CPP crystals was ascertained via manual review of laboratory results recorded as free text in the EHR

^cTopic modeling approach includes: score for propensity of pseudogout from a topic modeling method (sureLDA) including all relevant features, counts of the NLP concept “pseudogout”, and whether synovial fluid crystal analysis was performed (regardless of result)