Table 2.
Comparison of risk of bias domains in RCTs and NRSs, and example of the application of the ROBINS-I tool in a recent meta-analysis investigating the association between adherence to a Mediterranean diet and risk of stroke [51], and the corresponding quality rating by applying the Newcastle–Ottawa Scale
| Bias in RCTs | Action that protect against biases | Bias in NRSs | ROBINS-I | Newcastle–Ottawa Scale (max. 9 points) | |||||
|---|---|---|---|---|---|---|---|---|---|
| (Cochrane RoB tool) | Domains | Rating (n = studies, %) | Domains | Rating (n = studies, %) | |||||
| Bias arising from randomization process | Random sequence generation | Confounding | Bias due to Confounding | Moderate RoB | 20/20 (100%) | Comparability of cohorts on the basis of the design or analysis | High quality | 20/20 (100%) | |
| Allocation concealment | Low quality | 0/20 (0%) | |||||||
| Selection | Bias in selection of participants into the study | Low RoB | 16/20 (80%) | Representativeness of the exposed cohort | High quality | 13/20 (65%) | |||
| Low quality | 7/20 (35%) | ||||||||
| Serious RoB | 4/20 (20%) | Selection of the non-exposed cohort | High quality | 20/20 (100%) | |||||
| Low quality | 0/20 (0%) | ||||||||
| Demonstration that outcome of interest was not present at start of study | High quality | 20/20 (100%) | |||||||
| Low quality | 0/20 (0%) | ||||||||
| Misclassification | Bias in classification of exposure | Low RoB | 16/20 (80%) | Ascertainment of exposure | High quality | 18/20 (90%) | |||
| Low quality | 2/20 (10%) | ||||||||
| Moderate RoB | 2/20 (10%) | ||||||||
| Serious RoB | 2/20 (10%) | ||||||||
| Bias due to deviations from intended intervention | Blinding of participants and personnel | Performance | Bias due to deviations from intended exposure | Low RoB | 1/20 (5%) | N/A | |||
| Moderate RoB | 19/20 (95%) | ||||||||
| Bias due to missing outcome data | Complete outcome data | Attrition | Bias due to missing outcome data | Low RoB | 15/20 (75%) | Adequacy of follow-up of cohorts | High quality | 19/20 (95%) | |
| Moderate RoB | 5/20 (25%) | Low quality | 1/20 (5%) | ||||||
| Bias in measurement of the outcome | Blinding of outcome assessment | Detection | Bias in measurement of the outcome | Low RoB | 19/20 (95%) | Assessment of outcome | High quality | 20/20 (100%) | |
| Serious RoB | 1/20 (5%) | ||||||||
| Low quality | 0/20 (0%) | ||||||||
| Was follow-up long enough for outcomes to occur? | High quality | 20/20 (100%) | |||||||
| Low quality | 0/20 (0%) | ||||||||
| Bias in the selection of the reported results | Avoid selective reporting | Reporting | Bias in the selection of the reported results | Low RoB | 14/20 (70%) | N/A | |||
| Moderate RoB | 3/20 (15%) | ||||||||
| Serious RoB | 3/20 (15%) | ||||||||
| Overall RoB judgementa | Low RoB | 0/20 (0%) | Overall study qualityb | High quality (7–9) | 20/20 (100%) | ||||
| Moderate RoB | 13/20 (65%) | Lower quality (0–6) | 0/20 (0%) | ||||||
| Serious RoB | 7/20 (35%) | ||||||||
N/A not applicable, RoB risk of bias, ROBINS-I risk of bias in non-randomized studies of interventions
aFor ROBINS-I overall RoB judgements across studies were based to the most severe of the RoB item-level judgments. Since no single study was judged as low RoB for the domain “confounding”, also in the overall judgement no study was judged with a low RoB
bFor the Newcastle–Ottawa Scale, overall study quality judgements across studies were based on points (0–9). If a single study reached ≥ 7 points, it was defined as high-quality study