Table 4.
Assessment of performance of the ED and admissions models, calibrated to 70%, 80%, and 90% sensitivities during training, in identifying COVID-19 in patients presenting to or admitted to hospital in the held-out test set
|
Calibrated threshold during training |
|||
|---|---|---|---|
| Sensitivity 0·70 | Sensitivity 0·80 | Sensitivity 0·90 | |
| ED model performance on test set | |||
| Sensitivity | 0·697 (0·009) | 0·774 (0·019) | 0·847 (0·014) |
| Specificity | 0·986 (0·005) | 0·957 (0·009) | 0·917 (0·018) |
| Precision (PPV) | 0·979 (0·007) | 0·944 (0·012) | 0·905 (0·018) |
| NPV | 0·777 (0·005) | 0·820 (0·013) | 0·866 (0·011) |
| AUROC | 0·939 (0·003) | 0·939 (0·003) | 0·939 (0·003) |
| Admissions model performance on test set | |||
| Sensitivity | 0·663 (0·029) | 0·774 (0·013) | 0·854 (0·007) |
| Specificity | 0·973 (0·000) | 0·948 (0·005) | 0·891 (0·009) |
| Precision (PPV) | 0·950 (0·002) | 0·922 (0·006) | 0·861 (0·010) |
| NPV | 0·785 (0·014) | 0·841 (0·007) | 0·886 (0·005) |
| AUROC | 0·940 (0·001) | 0·940 (0·001) | 0·940 (0·001) |
Data are performance (SD). The test set was generated from an 80:20 stratified train-test split of the dataset and balanced equally with controls (50% assumed prevalence). AUROC=area under the receiver operating characteristic curve. ED=emergency department. NPV=negative predictive values. PPV=positive predictive values.