Table 3.
Manual validation in unlabeled data (Full Model)
| Unlabeled Data |
||||||
|---|---|---|---|---|---|---|
| Bottom 50% |
Top 50% |
Total |
||||
| Instances (N = 130) | Tests (N = 944 156) | Instances (N = 130) | Tests (N = 30 776 801) | Instances (N = 260) | Tests (N = 31 720 957) | |
| Total Correct | 87 (66.9%) | 798 268 (84.5%) | 108 (83.1%) | 26 054 265 (84.7%) | 195 (75.0%) | 26 852 533 (84.7%) |
| Predicted Correct | 70 (53.9%) | 599 043 (63.4%) | 106 (81.5%) | 25 910 603 (84.2%) | 176 (67.7%) | 26 509 646 (83.6%) |
| No LOINC Coverage, Code Synonymous | 17 (13.1%) | 199 225 (21.1%) | 2 (1.5%) | 143 662 (0.5%) | 19 (7.3%) | 342 887 (1.1%) |
| Total Incorrect | 26 (20%) | 114 632 (12.1%) | 19 (14.6%) | 4 285 372 (13.9%) | 45 (17.3%) | 4 400 004 (13.9%) |
| Predicted Incorrect | 22 (16.9%) | 114 622 (12.1%) | 19 (14.6%) | 4 285 372 (13.9%) | 41 (15.8%) | 4 399 994 (13.9%) |
| No LOINC Coverage, Code Incorrect | 4 (3.1%) | 10 (<0.1%) | 0 (0%) | 0 (0%) | 4 (1.5%) | 10 (<0.1%) |
| Insufficient or Conflicting Information | 17 (13.1%) | 31 256 (3.3%) | 3 (2.3%) | 437 164 (1.4%) | 20 (7.7%) | 468 420 (1.5%) |
Full Model refers to the 1-versus-rest classifier fit to the full labeled dataset.
Label definitions: Predicted Correct: model-predicted label is correct; No LOINC Coverage, Code Synonymous: LOINC code does not exist for the combination of test and specimen type in the source data, but the predicted LOINC code is the most reasonable alternative; Predicted Incorrect: model-predicted label is incorrect; No LOINC Coverage, Code Incorrect: LOINC code does not exist for the combination of test and specimen type in the source data, and the predicted LOINC code is not a reasonable alternative; Insufficient or Conflicting Information: either not enough source data to infer code (ie, units missing and would be necessary to assign code), or source data conflict (ie, test name includes the word “blood” and specimen type is “urine”).