Table 7.
Expert review of terms for each use case, Recommended vs. Accepted terms. Qualitative terms are reported separately because they were not considered when the ontologies were first constructed
| Use Case | Reco-mmended | Matched | Qualitative | Accepted (2 + revs) | Total Accepted |
Accepted
(<2 revs) |
Misses | Fleiss’ Kappa |
|---|---|---|---|---|---|---|---|---|
| ACMS | 173 | 25 (14%) | 31 (18%) | 48 (28%) | 103 (60%) | 23 (13%) | 47 (26%) | 0.38 (Fair) |
| MUS | 271 | 61 (23%) | 35 (13%) | 67 (25%) | 163 (60%) | 26 (9%) | 83 (31%) | 0.29 (Fair) |
| ECHO | 363 | 289 (80%) | N/A | 37 (10%) | 326 (90%) | 14 (4%) | 23 (6%) | 0.66 (Substantial) |