Table 1.

Clinician validation of episodes and outcome categories

	Number identified by algorithm	Number identified by clinician	Percent completeness of algorithm compared to clinician reviewa	Percent agreement between algorithm and clinicianb	Clinicians’ average confidence score (−1, 0, 1)	Standard deviation of clinicians’ confidence scores
Number of non-episodes (among 40 females)	40	42	92.9	97.5	N/A	N/A
Number of episodes (among 166 females)	240	245	88.6	90.4	N/A	N/A
Outcome
Live birth	80	89	89.9	100.0	0.96	0.19
Stillbirth	40	38	100.0	95.0	0.97	0.16
Abortion	40	47	78.7	92.5	1	0
Ectopic pregnancy	40	33	100.0	82.5	0.85	0.44
No outcome	40	38	76.3	72.5	N/A	N/A
Gestational age at outcome (±1 week)
Live birth	N/A	N/A		98.8	0.71	0.46
Stillbirth	N/A	N/A		89.5	0.57	0.55
Abortion	N/A	N/A		67.6	−0.34	0.61
Ectopic pregnancy	N/A	N/A		63.6	−0.58	0.50

As these cases were randomly selected for clinician validation, they do not reveal information about the subpopulation and are not subject to the same policy regarding masking small numbers of patients.

^aPercent completeness of algorithm compared to clinician review is the number of matching algorithmically defined episodes out of all possible clinician-defined episodes for these categories.

^bPercent agreement between algorithm and clinician is the number of matching clinician-defined episodes out of all possible algorithmically defined episodes for these categories.