Table 1.
Evaluation metrics shown for each method, averaged over 25 datasets × 20 random train/test splits
| Runtime (s) | Active inputs (%) | Accuracy (%) | AUC (%) | F1 (%) | |
|---|---|---|---|---|---|
| CoDaCoRe—Balances (ours) | 4.5 ±0.4 | 1.9 ±0.3 | 75.2 ± 2.4 | 79.5 ± 2.6 | 73.7 ± 2.6 |
| CoDaCoRe—Amalgamations (ours) | 4.4 ± 0.4 | 1.9 ± 0.3 | 71.8 ± 2.4 | 74.5 ± 2.8 | 69.8 ± 2.9 |
| selbal (Rivera-Pinto et al., 2018) | 79 033.7 ± 2094.1 | 2.4 ± 0.2 | 61.2 ± 1.9 | 80.0 ± 2.4 | 70.9 ± 1.1 |
| Pairwise log-ratios (Greenacre, 2019b) | 14 207.0 ± 1038.4 | 2.5 ± 0.4 | 73.3 ± 1.7 | 75.2 ± 2.4 | 67.8 ± 3.0 |
| Lasso | 1.6 ± 0.1 | 4.4 ± 0.6 | 72.4 ± 1.7 | 75.2 ± 2.3 | 65.2 ± 3.7 |
| CoDaCoRe—balances with λ = 0 (ours) | 9.8 ± 2.2 | 6.1 ± 0.7 | 77.6 ± 2.2 | 82.0 ± 2.3 | 76.0 ± 2.5 |
| Coda-lasso (Lu et al., 2019) | 1043.0 ± 55.4 | 19.7 ± 2.7 | 72.5 ± 2.3 | 78.0 ± 2.4 | 64.2 ± 4.4 |
| amalgam (Quinn and Erb, 2020) | 7360.5 ± 209.8 | 87.6 ± 2.1 | 74.4 ± 2.5 | 78.2 ± 2.7 | 73.9 ± 2.8 |
| DeepCoDA (Quinn et al., 2020) | 296.5 ± 21.4 | 89.3 ± 0.6 | 70.6 ± 2.9 | 77.6 ± 2.9 | 64.7 ± 7.4 |
| CLR-lasso (Susin et al., 2020) | 2.0 ± 0.2 | 100.0 ± 0.0 | 77.5 ± 1.8 | 81.6 ± 2.2 | 75.8 ± 2.7 |
| Random Forest | 10.6 ± 0.4 | – | 78.0 ± 2.2 | 82.2 ± 2.2 | 77.3 ± 2.5 |
| Log-ratio lasso (Bates and Tibshirani, 2019)* | 135.0 ± 11.1 | 0.7 ± 0.0 | 72.0 ± 2.4 | 76.4 ± 2.3 | 69.2 ± 2.7 |
Note: Standard errors are computed independently on each dataset, and then averaged over the 25 datasets. The models are ordered by sparsity, i.e. percentage of active input variables. CoDaCoRe (with balances) is the only learning algorithm that is simultaneously fast, sparse and accurate. The penultimate row shows the performance of Random Forest, a powerful black-box classifier which can be thought of as providing an approximate upper bound on the predictive accuracy of any interpretable model. The bottom row is shown separately and marked with an asterisk because the corresponding algorithm failed to converge on 432 out our 500 runs (averages were taken after imputing these missing values with the corresponding values obtained with pairwise log-ratios, which is the most similar method). We highlight in bold the CoDa models that are fast to run, as well as the CoDa models that are most sparse and accurate.