Table 10. Randomness and variability of the results.
| Dataset | Performance metrics with different random seeds | |||||
|---|---|---|---|---|---|---|
| Accuracy | F1-score | |||||
| Mean | Std | Conf. interval (Conf. Level 95%) |
Mean | Std | Conf. interval (Conf. Level 95%) |
|
| ceylon-ide-eclipse | 68.18 | 6.47 | [64.17–72.19] | 76.94 | 7.01 | [72.60–81.29] |
| BroadleafCommerce | 75.57 | 1.58 | [74.60–76.55] | 84.45 | 1.32 | [83.63–85.27] |
| hazelcast | 61.36 | 1.32 | [60.54–62.18] | 65.17 | 4.57 | [62.34–68.00] |
| elasticsearch | 65.24 | 1.51 | [64.31–66.18] | 74.02 | 3.56 | [71.82–76.23] |
| MapDB | 69.21 | 4.86 | [66.20–72.23] | 74.44 | 5.27 | [71.17–77.70] |
| netty | 65.03 | 5.39 | [61.69–68.37] | 73.38 | 5.98 | [69.67–77.08] |
| orientdb | 66.05 | 3.59 | [63.83–68.28] | 73.86 | 4.45 | [71.10–76.62] |
| neo4j | 66.56 | 4.44 | [63.81–69.32] | 74.67 | 5.02 | [71.55–77.78] |
| titan | 70.19 | 2.99 | [68.34–72.04] | 79.78 | 2.64 | [78.14–81.42] |
| mcMMO | 56.68 | 4.54 | [53.87–59.50] | 64.09 | 6.21 | [60.24–67.94] |
| Android-Universal-Image-Loader | 64.70 | 3.59 | [62.47–66.92] | 74.23 | 2.28 | [72.81–75.65] |
| antlr4 | 71.25 | 1.25 | [70.48–72.02] | 81.68 | 0.97 | [81.08–82.28] |
| junit | 70.96 | 2.90 | [69.16–72.75] | 80.64 | 2.35 | [79.19–82.10] |
| mct | 93.18 | 5.47 | [89.79–96.57] | 94.39 | 5.79 | [90.80–97.98] |
| oryx | 88.77 | 1.72 | [87.70–89.83] | 93.80 | 1.10 | [93.12–94.48] |