Table 4. Statistical performance of models to discriminate DAT cases in an independent test set.
| Input Data | Model Type | Threshold | Sensitivity Test Set | Specificity Test Set | trAuROC | trSens | trSpec |
|---|---|---|---|---|---|---|---|
| Spot 2159 | ROC curve | minMC | 100% | 38.5% | 0.917 | 91.7% | 85.0% |
| Spot 3486 | ROC curve | minMC | 85.7% | 69.2% | 0.842 | 75.0% | 90.0% |
| Spot 3486 | ROC curve | opT | 100% | 61.5% | 0.842 | 83.3% | 80.0% |
| Eosinophil Count | ROC curve | minMC | 57.1% | 92.3% | 0.759 | 72.7% | 76.5% |
| 49 spots | CANN101 | minMC | 100% | 23.1% | 1.000 | 100% | 100% |
| 2159 + 3486 | CANN101 | minMC | 100% | 61.5% | 1.000 | 100% | 100% |
| 2159 + Eos | CANN101 | minMC | 85.7% | 84.6% | 1.000 | 100% | 100% |
| 3486 + Eos | CANN101 | minMC | 85.7% | 84.6% | 1.000 | 100% | 100% |
| 2159 + 3486 + Eos | CANN101 | minMC | 85.7% | 76.9% | 1.000 | 100% | 100% |
| 2159 + 3486 + Eos | CT | NA | 85.7% | 84.6% | NA | 90.9% | 100% |
| 2159 + 3486 + Eos | k-NN | NA | 85.7% | 69.2% | NA | 72.7% | 89.5% |
| 2159 + 3486 + Eos | RF | NA | 85.7% | 69.2% | NA | 100% | 100% |
| 2159 + 3486 + Eos | SVM | NA | 85.7% | 92.3% | NA | 81.8% | 94.7% |
An AuROC for the test set was not determined since this set was only used to qualify models derived from the training set. The top performing model, SVM using spots 2159, 3486 and eosinophil counts, is bolded. NA, Not Applicable. Abbreviations: ROC curve, receiver operator characteristic curve; CANN, combined artificial neural networks; CT, classification tree; k-NN, k-nearest neighbor; RF, random forest; SVM, support vector machine; minMC, minimum mis-classified; opT, optimum threshold; AuROC, area under the ROC curve; tr prefix, values are from training set.