Table 3.
Overall results of the search process
| Data set | Time spent (h) | # of distinct combinations tested | Error rate on the test data (%) | |||
|---|---|---|---|---|---|---|
| Auto-WEKA | Our method | Auto-WEKA | Our method | Auto-WEKA | Our method | |
| Mammographic Mass | 30 | 0.3 ± 0.3 | 4085 ± 1798 | 1615 ± 25 | 17.36 ± 0.49 | 16.94 ± 0.38 |
| Car | 30 | 2.0 ± 0.4 | 1954 ± 1050 | 1524 ± 35 | 0.00 ± 0.00 | 0.00 ± 0.00 |
| Yeast | 30 | 1.7 ± 0.4 | 6675 ± 2721 | 1602 ± 23 | 38.29 ± 1.51 | 38.88 ± 1.20 |
| German credit | 30 | 1.0 ± 0.3 | 3364 ± 952 | 1602 ± 29 | 28.40 ± 1.26 | 27.00 ± 1.11 |
| Diabetic retinopathy debrecen | 30 | 1.4 ± 0.3 | 4621 ± 1434 | 1745 ± 39 | 25.57 ± 0.66 | 24.58 ± 1.58 |
| Parkinson speech | 30 | 0.09 ± 0.01 | 3570 ± 250 | 1669 ± 35 | 0.00 ± 0.00 | 0.00 ± 0.00 |
| Abalone | 30 | 4.3 ± 0.5 | 675 ± 181 | 1501 ± 20 | 72.99 ± 0.30 | 73.25 ± 0.47 |
| Cardiotocography | 30 | 0.5 ± 0.2 | 3578 ± 1962 | 1600 ± 34 | 1.63 ± 0.29 | 1.54 ± 0.13 |
| Wine quality | 30 | 4.2 ± 0.8 | 1429 ± 823 | 1633 ± 33 | 33.97 ± 0.75 | 34.12 ± 0.70 |
| KR-vs-KP | 30 | 1.5 ± 0.6 | 2192 ± 1522 | 1574 ± 23 | 0.42 ± 0.24 | 0.40 ± 0.11 |
| Arrhythmia | 30 | 3.83 ± 0.59 | 259 ± 72 | 1464 ± 39 | 33.24 ± 1.83 | 30.59 ± 1.12 |
| Waveform | 30 | 3.5 ± 0.6 | 801 ± 248 | 1596 ± 21 | 14.33 ± 0.15 | 14.29 ± 0.12 |
| Semeion | 30 | 4.7 ± 0.7 | 408 ± 103 | 1594 ± 9 | 5.58 ± 0.82 | 5.03 ± 0.21 |
| Shuttle | 30 | 1.7 ± 0.5 | 275 ± 92 | 1610 ± 22 | 0.010 ± 0.004 | 0.008 ± 0.003 |
| Secom | 30 | 1.1 ± 0.1 | 119 ± 90 | 1699 ± 47 | 8.13 ± 0.57 | 7.83 ± 0.09 |
| Madelon | 30 | 2.6 ± 0.3 | 529 ± 261 | 1763 ± 49 | 22.08 ± 2.52 | 18.31 ± 1.47 |
| Arcene | 30 | 3.37 ± 0.68 | 108 ± 37 | 1635 ± 10 | 18.00 ± 3.16 | 16.40 ± 1.52 |
| Convex | 30 | 5.5 ± 0.4 | 72 ± 49 | 1766 ± 73 | 26.24 ± 4.93 | 23.37 ± 0.87 |
| KDD09-appentency | 30 | 6.0 ± 0.5 | 155 ± 127 | 1783 ± 72 | 1.74 ± 0.00 | 1.74 ± 0.00 |
| Dexter | 30 | 4.4 ± 0.6 | 98 ± 14 | 1569 ± 28 | 8.33 ± 1.24 | 5.11 ± 1.20 |
| MNIST basic | 30 | 11.1 ± 0.7 | 53 ± 16 | 1674 ± 23 | 10.14 ± 4.40 | 3.70 ± 0.46 |
| ROT. MNIST + BI | 30 | 7.0 ± 0.3 | 13 ± 13 | 1637 ± 17 | 63.31 ± 5.76 | 55.81 ± 0.16 |
| Amazon | 30 | 8.9 ± 0.8 | 53 ± 19 | 1561 ± 41 | 38.31 ± 14.73 | 24.31 ± 1.91 |
| Gisette | 30 | 9.4 ± 0.3 | 43 ± 25 | 1586 ± 31 | 2.31 ± 0.55 | 1.89 ± 0.29 |
| CIFAR-10-small | 30 | 10.0 ± 0.5 | 46 ± 21 | 1546 ± 34 | 67.87 ± 5.36 | 57.42 ± 0.52 |
| Dorothea | 30 | 7.3 ± 0.9 | 52 ± 45 | 1625 ± 25 | 6.29 ± 2.09 | 5.74 ± 0.48 |
| CIFAR-10 | 30 | 10.0 ± 0.9 | 29 ± 24 | 1529 ± 72 | 62.02 ± 7.68 | 52.83 ± 0.78 |