Surname/year of first author of main paper | 1: Mechanism of assignment: was the allocation or identification mechanism able to control for selection bias? | 2. Group equivalence: was the method of analysis executed adequately to ensure comparability of groups throughout the study and prevent confounding? | 3: Performance bias: was the process of being observed free from motivation bias? | 4: Spill‐overs, cross‐overs and contamination: was the study adequately protected against spill‐overs, cross‐overs and contamination? | 5: Selective outcome reporting: was the study free from selective outcome reporting? | 6: Selective analysis reporting: was the study free from selective analysis reporting? | 7: Other risks of bias: Is the study free from other sources of bias? | Are there any unit of analysis errors? | Final RoB Assessment |
---|---|---|---|---|---|---|---|---|---|
Hedge & Bull, 2011 | No | No | Yes | No | Yes | No | Yes | No | Critical |
Jindal, 2012 | No | No | Yes | Yes | Yes | No | Yes | No | Critical |
Garbach | Unclear | No | No | Unclear | Unclear | Unclear | Yes | No | Critical |
Honey‐Roses (2011) | Unclear | Yes | Yes | Yes | Yes | Yes | Yes | No | Medium |
Beauchamp, 2018 | Unclear | Unclear | Yes | No | Yes | Yes | Unclear | No | Critical |
Sharma, 2015 | Unclear | Unclear | Yes | No | Yes | Yes | Yes | No | High |
Arriagada, 2011 | Unclear | No | Yes | Unclear | Yes | Unclear | Yes | No | High |
Arriagada, 2012 | Yes | Yes | Yes | Yes | Yes | Yes | Yes | No | Low |
Arriagada, 2015 | Yes | Yes | Yes | Unclear | Yes | Unclear | Yes | No | Medium |
Robalino, 2013 | No | Unclear | Yes | No | Yes | Yes | Yes | Unclear | High |
Robalino, 2014 | Unclear | No | Yes | Unclear | Yes | Unclear | Yes | Unclear | High |
Robalino, 2015—also checked Robalino 2008 | No | Unclear | Yes | No | Yes | Unclear | Unclear | Unclear | Critical |
Sierra, 2006 | No | No | Yes | No | Yes | Unclear | Unclear | No | Critical |
Alix‐Garcia, 2015a | Yes | Yes | Yes | Unclear | Yes | Yes | Yes | No | Medium |
Arriagada, 2018 | Unclear | Unclear | Yes | No | Yes | Yes | Unclear | Unclear | High |
Le Velley 2017—Le Velley 2015 also checked | Yes | Yes | Yes | Yes | Yes | Yes | Yes | No | Low |
Scullion 2011 | No | No | Yes | No | Yes | No | Yes | Unclear | Critical |
Sims 2017 | Unclear | Unclear | Yes | Yes | Yes | Yes | Yes | No | High |
Liu, 2015 | No | No | Yes | No | Yes | Unclear | Unclear | No | Critical |
Liu, 2014 | No | Unclear | Yes | No | Yes | Unclear | Unclear | No | Critical |
Lin, 2014 | No | No | Yes | No | Yes | Unclear | Unclear | No | Critical |
Liu, 2018 | Unclear | Unclear | Yes | No | Yes | Unclear | Unclear | No | Critical |
Demurger, 2012 | Unclear | No | Yes | Unclear | Yes | Unclear | Unclear | No | High |
Duan, 2015 | No | No | Yes | No | No | No | Unclear | No | Critical |
Groom, 2010 | Unclear | No | Yes | No | Yes | Yes | No | No | Critical |
Liang, 2012 | No | No | Yes | No | Yes | Unclear | Unclear | No | Critical |
Uchida, 2009 | No | No | Yes | No | No | Yes | Unclear | No | Critical |
Xu, 2010 | Unclear | Unclear | Yes | No | Yes | Unclear | Unclear | No | Critical |
Yao, 2010 | Unclear | No | Yes | No | Yes | Unclear | Unclear | No | Critical |
Kwayu, 2017 | No | No | Yes | No | Yes | Unclear | Unclear | Unclear | Critical |
Lokina, 2016 | No | No | Yes | No | Yes | Yes | Yes | Unclear | Critical |
Hayes, 2017 | Unclear | No | Yes | No | Yes | Yes | Yes | No | Critical |
Jones, 2016 | Unclear | Unclear | Yes | No | Yes | Yes | Yes | No | High |
Mohebalian, 2016 | Unclear | Unclear | Yes | No | Yes | Yes | Yes | No | High |
Mohebalian, 2018 | Unclear | Unclear | Yes | No | Yes | Yes | Yes | No | High |
Jayachandran et al., 2017 | Yes | Yes | Yes | Yes | Yes | Yes | Yes | No | Low |
Pagiola, 2013 (Pagiola, 2016 also checked) | Unclear | Unclear | Yes | Unclear | Yes | Yes | Yes | No | High |
Chervier, 2017a | Unclear | Unclear | Yes | Yes | Yes | Yes | Yes | Unclear | High |
Chervier, 2017b | No | Unclear | Yes | Yes | No | Yes | Yes | Unclear | High |
Zhang, 2013 | Unclear | No | Yes | No | Yes | Unclear | Yes | No | Critical |
Jack, 2017 | Yes | Yes | Yes | Yes | Yes | Yes | Yes | No | Low |
Simonet, 2017 | No | Yes | Yes | Yes | Yes | Yes | Yes | No | High |
Zhang, 2005 | Unclear | No | Yes | No | Yes | Unclear | Unclear | No | Critical |
Liu, 2014 | No | Unclear | Yes | No | Yes | Unclear | Yes | No | Critical |
Costedoat, 2015 | Unclear | Yes | Yes | Yes | Yes | Yes | Yes | Unclear | Medium |