Table 1.

Measurement properties of the final iteration of the NLP tool

Accuracy*
Instances reviewed	Instances annotator agreed with NLP classification		BED diagnosis
			‘Yes’			‘No’			‘Possible’†
			NLP-identified	Annotator agreed		NLP-identified	Annotator agreed		NLP-identified	Annotator agreed
N	N	%	N	N	%	N	N	%	N	N	%
1000	918	91.8	731	663	90.7	177	171	96.6	92	84	91.3

Sensitivity‡
	Patient records reviewed	Missed BED diagnoses	Sensitivity
Sample selection category	N	N	%
Initial keyword list§	200	1	99.5
EDNOS¶	530	27	94.7
Total	730	28	96.2

*Used 1000 randomly selected instances of variants of the phrase ‘BED’.

†Includes differential diagnoses.

‡Used 200 patient records containing any of the initial terms and phrases and 530 patient records of patients with an EDNOS diagnosis that were not identified and classified by NLP as having BED.

§Patient records were selected that contained one of the initial keywords or phrases (‘addictive eating’, ‘BED’, ‘binge eat’, ‘binge eater’, ‘binge eating’, ‘eating disorder’, ‘EDNOS’, ‘eating episode’, ‘over eat’, ‘over eater’, ‘over eating’).

¶Patient records were selected that contained a diagnosis code for EDNOS (ICD-9 307.50).

BED, binge eating disorder; EDNOS, eating disorder not otherwise specified; ICD-9, International Classification of Diseases, 9th edition; NLP, natural language processing.