Table 3.
Performance measures of the rule-based natural language processing algorithm in the ED and inpatient settings.
| Information searched | Performance measures for ED test group | Performance measures for inpatient validation group | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Sensitivity | Specificity | PPV * | NPV * | F1 score | Sensitivity | Specificity | PPV * | NPV * | F1 score | |
| Sexual history | 97/97 (100.0%) | 82/82 (100.0%) | 97/97 (100.0%) | 82/82 (100.0%) | 100.0% | 97/97 (100.0%) | 220/224 (98.2%) | 97/101 (96.0%) | 220/220 (100.0%) | 98.0% |
| Protection used | 33/33 (100.0%) | 143/146 (97.9%) | 33/36 (91.7%) | 143/143 (100.0%) | 95.7% | 23/23 (100.0%) | 290/298 (97.3%) | 23/31 (74.2%) | 290/290 (100.0%) | 85.2% |
| STI testing offered/previously performed | 55/56 (98.2%) | 121/123 (98.4%) | 55/57 (96.5%) | 121/122 (99.2%) | 97.3% | 23/37 (85.2%) | 268/294 (91.2%) | 23/49 (46.9%) | 268/282 (95.0%) | 60.5% |
| Not sexually active | 35/39 (89.7%) | 126/126 (100.0%) | 35/35 (100.0%) | 126/130 (96.9%) | 94.6% | 41/52 (78.8%) | 268/269 (99.6%) | 41/42 (97.6%) | 268/279 (96.1%) | 87.2% |
| Partner gender | 26/26 (100.0%) | 118/152 (77.6%) | 26/60 (43.3%) | 118/118 (100.0%) | 60.4% | 33/36 (91.7%) | 220/285 (77.2%) | 33/98 (33.7%) | 220/223 (98.7%) | 49.3% |
| Type of sexual practice | 9/9 (100.0%) | 170/170 (100.0%) | 9/9 (100.0%) | 170/170 (100.0%) | 100.0% | 7/8 (87.5%) | 313/313 (100.0%) | 7/7 (100.0%) | 313/314 (99.7%) | 93.3% |
| Contraception Used | 32/32 (100.0%) | 146/146 (100.0%) | 32/32 (100.0%) | 146/146 (100.0%) | 100.0% | 21/21 (100.0%) | 295/300 (98.3%) | 21/26 (80.8%) | 295/295 (100.0%) | 89.4% |
*
PPV, positive predictive value; NPV, negative predictive value.