Table 3.
Comparison of performance characteristics for RA classification algorithm
| General RA mart (n = 200) | ||||||
|---|---|---|---|---|---|---|
| AUC | PPV | NPV | Specificity | Sensitivity | # classified w/ RA by algorithm (n)c | |
| NLP-based algorithms | ||||||
| 2010 RA algorithma | 0.932 | 0.905 | 0.869 | 0.952 | 0.760 | 15 312 |
| Updated RA algorithma | 0.937 | 0.906 | 0.875 | 0.952 | 0.773 | 16 358 |
| Rule-based algorithms | ||||||
| ≥3 ICD9 RA codesb | __ | 0.536 | 0.822 | 0.592 | 0.787 | 25 707 |
| ≥3 ICD9 or ICD10 RA codesb | __ | 0.558 | 0.900 | 0.576 | 0.893 | 28 445 |
| RA subjects with ICD10 codes and no RA ICD9 codes (n = 100) | ||||||
|---|---|---|---|---|---|---|
| AUC | PPV | NPV | Specificity | Sensitivity | ||
| NLP-based algorithm | ||||||
| Updated RA algorithma | 0.784 | 0.926 | 0.600 | 0.954 | 0.472 | |
| Rule-based algorithms | ||||||
| ≥3 ICD10 RA codesb | __ | 0.585 | 0.500 | 0.500 | 0.585 | |
a
Specificity set at 0.95. bBinary classification, no AUC shown. cNumber computed by applying the algorithm on RA Mart. AUC: area under the receiver operating characteristic curve; ICD9/10: International Classification of Diseases; NLP: natural language processing; NPV: negative predictive value; PPV: positive predictive value.