Table 3. Reproducibility levels for paper corpus; ‘-’ is category not available.
For full references of the corpus papers see Supplemental Material.
| Author | Short paper | Input data | Preprocessing | Method/analysis/ processing | Computational environment | Results |
|---|---|---|---|---|---|---|
| Zhu et al. (2017) | 0 | 1 | 1 | 1 | 1 | |
| Knoth et al. (2017) | 0 | – | 0 | 1 | 1 | |
| Konkol et al. (2017) | 2 | 2 | 1 | 1 | 1 | |
| Haumann et al. (2017) | X | 0 | 1 | 1 | 0 | 1 |
| Brinkhoff (2017) | X | 0 | – | 1 | 0 | 0 |
| Almer et al. (2016) | 0 | – | 1 | 1 | 1 | |
| Wiemann (2016) | 2 | – | 1 | 1 | 1 | |
| Juhász & Hochmair (2016) | 0 | 1 | 1 | 0 | 0 | |
| Josselin et al. (2016) | X | 1 | – | 0 | 0 | 1 |
| Rosser et al. (2016) | X | 0 | – | 1 | 0 | 0 |
| Kuhn & Ballatore (2015) | – | – | – | – | – | |
| Mazimpaka & Timpf (2015) | 2 | 1 | 1 | 1 | 1 | |
| Steuer et al. (2015) | 2 | 0 | 1 | 1 | 1 | |
| Fogliaroni & Hobel (2015) | X | – | – | – | – | – |
| Heinz & Schlieder (2015) | X | 0 | 0 | 1 | 1 | 1 |
| Scheider et al. (2014) | 1 | 1 | 2 | 1 | 1 | |
| Gröchening et al. (2014) | 2 | 0 | 1 | 0 | 1 | |
| Fan et al. (2014) | 0 | 1 | 1 | 0 | 1 | |
| Soleymani et al. (2014) | X | 0 | 0 | 1 | 0 | 0 |
| Wiemann & Bernard (2014) | X | 0 | 0 | 1 | 0 | 0 |
| Osaragi & Tsuda (2013) | 0 | 1 | 1 | 0 | 1 | |
| Baglatzi & Kuhn (2013) | – | – | – | – | – | |
| Li et al. (2013) | X | 0 | 0 | 1 | – | 1 |
| Stein & Schlieder (2013) | X | 0 | – | 1 | 0 | 1 |
| Osaragi & Hoshino (2012) | 0 | 0 | 1 | 0 | 1 | |
| Magalhães et al. (2012) | 0 | 0 | 1 | 0 | 0 | |
| Foerster et al. (2012) | 1 | – | 1 | 1 | 1 | |
| Merki & Laube (2012) | X | 0 | – | 1 | 1 | 1 |
| Kiefer et al. (2012) | X | 0 | 1 | 1 | 0 | 1 |
| Raubal & Winter (2010) | – | – | – | – | – | |
| Schäffer et al. (2010) | 0 | 0 | 1 | 1 | 1 | |
| Körner et al. (2010) | – | – | – | – | – |