Table 3.

The per item and per letter accuracy of ExECT when extracting epilepsy information from a validation set of 200 clinic letters, where an information item is defined as a single item in any category identified by the human annotator (see the Materials and methods section for more details)

Variable	200 letters—results per item				200 letters—results per letter
	Number of information items identified by clinician	Precision %	Recall %	F1 score %	Number of letters containing information items identified by clinician	Precision %	Recall %	F1 score %
Clinic date	191	98.9	97.4	98.2	186	100.0	97.3	98.6
Date of birth	201	100.0	98.0	99.0	199	100.0	98.0	99.0
Epilepsy diagnosis	383	88.1	89.0	88.6	150	94.1	95.3	94.7
Epilepsy type	89	89.9	79.8	84.5	70	91.0	87.1	89.1
Focal seizures	145	96.2	69.7	80.8	71	96.7	83.1	89.4
Generalised seizures	151	88.8	52.3	65.8	76	89.7	68.4	77.6
Seizure frequency	153	86.3	53.6	66.1	119	92.2	59.7	72.4
Medication	316	96.1	94.0	95.0	157	98.6	91.1	94.7
CT scan	17	55.6	58.8	57.1	16	76.9	62.5	69.0
MRI scan	109	82.4	68.8	75.0	66	86.7	78.8	82.5
EEG	170	81.5	75.3	78.3	79	86.6	89.9	88.2
All	1925	91.4	81.4	86.1	1189	95.1	87.3	91.1

EEG, electroencephalogram; ExECT, extraction of epilepsy clinical text.