Table 3. Mini-expert system performance evaluation results.
Combined + weighted (mini-expert system) | Combined + unweighted | Cosine + weighted | Cosine + unweighted | |
---|---|---|---|---|
MRR | 0.72 | 0.70 | 0.72 | 0.68 |
% success at 1 | 62 | 59 | 63 | 57 |
% success at 5 | 86 | 85 | 85 | 83 |
% success at 10 | 90 | 91 | 90 | 89 |
% success at 20 | 93 | 92 | 92 | 91 |
Mean reciprocal rank (MRR) measures how close the correct match is to the top rank on average. It ranges from 0 to 1, and values close to 1 indicate that correct matches appear closer to the top on average. % success at N = % of cases with correct diagnoses within top N ranks. Combined = combined cosine and semantic similarity. Cosine = cosine similarity only.