Table 4. Risk of bias assessment (ACROBAT-NRSI).
| a Authors- Article- Y.o.P. | Bias due to confounding | Bias in selection of participants into the study | Bias in measurement of interventions | Bias due to departures from intended interventions | Bias due to missing data | Bias in measurement of outcomes | Bias in selection of the reported result | Overall Judgement |
|---|---|---|---|---|---|---|---|---|
| Ritucci-Nanda[8], 1986 |
Outcomes: O1, O2. Risk: Moderate SfJ: confounding expected (pre-treatment overjet), but probably eliminated since as it was stated “the occlusion changed from one with an underjet to one with an overjet”. |
Outcomes: O1, O2. Risk: Serious SfJ: Selection into the study was related to intervention and probably the outcome. |
Outcomes: O1, O2. Risk: Moderate SfJ: Intervention status is well defined, but data were determined retrospectively in a way that could have been affected by knowledge of the outcome. |
Outcomes: O1, O2. Risk: Low SfJ: No bias due to departures from intended intervention is expected. |
Outcomes: O1, O2. Risk: Low SfJ: Data were reasonably complete. |
Outcomes: O1, O2. Risk: Serious SfJ: Outcome measure was subjective, the assessor was aware of the received intervention and any systematic error in measurements was related to intervention status. |
Outcomes: O1, O2. Risk: Moderate SfJ: Outcome measurements and analysis were clearly defined, internally and externally consistent, there was no indication of selection of the reported analysis from among multiple analyses and no indication of selection of the cohort or subgroups for analysis and reporting on the basis of the results. |
Risk: Serious SfJ: The study has some important problems, e.g: serious risk of selection bias (selection of the participants), detection bias, measurement bias and observer bias. |
| Alacrόn et al.[25], 2011 |
Outcomes: O1, O2. Risk: Low SfJ: No confounding expected. |
Outcomes: O1, O2. Risk: Serious SfJ: Selection into the study was related to intervention and probably the outcome. |
Outcomes: O1, O2. Risk: Moderate SfJ: Intervention status is well defined, but data were obtained retrospectively. |
Outcomes: O1, O2. Risk: Low SfJ: No bias due to departures from intended intervention is expected. |
Outcomes: O1, O2. Risk: Low SfJ: Data were reasonably complete. |
Outcomes: O1, O2. Risk: Low SfJ: Comparable outcome assessment methods between the 2 groups and blinded assessor to intervention status and any systematic error in measurements unrelated to intervention status. |
Outcomes: O1, O2. Risk: Moderate SfJ: Outcome measurements and analysis were clearly defined, internally and externally consistent, there was no indication of selection of the reported analysis from among multiple analyses and no indication of selection of the cohort or subgroups for analysis and reporting on the basis of the results. |
Risk: Serious SfJ: The study has an important problem, e.g: serious risk of selection bias (selection of the participants). |
| Arman et al.[17], 2004 |
Outcomes: O1, O2,O3. Risk: Serious SfJ: Critically important confounders (ethnicity, Skeletal Class of malocclusion and soft-tissue profile variation) not measured and not adjusted for in the analysis. |
Outcomes: O1, O2, O3. Risk: Serious SfJ: Selection into the study was related to intervention and probably the outcome. |
Outcomes: O1, O2, O3. Risk: Serious SfJ: Intervention status may be well defined, but data were obtained retrospectively and determined in a way that could have been affected by knowledge of the outcome. |
Outcomes: O1, O2, O3. Risk: Serious SfJ: Co-interventions are apparent (OBP, RHG) and not adjusted for in the analysis. |
Outcomes: O1, O2, O3. Risk: Moderate SfJ: Reasons for missingness (for baseline characteristics and confounders) differ minimally across interventions and missing data were not addressed in the analysis. |
Outcomes: O1, O2, O3. Risk:Moderate SfJ:Comparable outcome assessment methods between the groups and outcome measure only minimally influenced by knowledge of the intervention (blinding not reported) and any systematic error in measurements only minimally related to intervention status. |
Outcomes: O1, O2, O3. Risk: Moderate SfJ: Relative low risk of selective reporting, because all results were declared for each group in comparison with each other. |
Risk: Serious SfJ: The study has some important problems, e.g: serious risk of selection bias (residual confounding, selection of the participants) and performance bias. |
| Wendell et al.[27], 1985. |
Outcomes: O1, O2. Risk: Serious SfJ: Serious residual confounding, because of variation in the onset, duration and amount of the growth peaks of the patients, as well as in the onset and duration of the treatment time. Also the 2 treatment periods of patient 13 indicate time-varying confounding. |
Outcomes: O1, O2. . Risk: Serious SfJ: Selection into the study was related to intervention and probably the outcome. |
Outcomes: O1, O2. . Risk: Serious SfJ: Intervention status is not well defined (not explicit timing of intervention), retrospectively obtained and likely to have been influenced by knowledge of the received intervention. |
Outcomes: O1, O2. . Risk: Moderate SfJ: No co-interventions, no switches and probably minor implementation fidelity. |
Outcomes: O1, O2. . Risk: Serious SfJ: The nature of the missing data (on baseline confounders) means that the risk of bias cannot be removed through appropriate analysis. |
Outcomes: O1, O2. . Risk: Serious SfJ: Any systematic error in measurements was related to intervention status and the assessor was aware of the received intervention. |
Outcomes: O1, O2. . Risk: Moderate SfJ: Outcome measurements and analysis were clearly defined, internally and externally consistent, there was no indication of selection of the reported analysis from among multiple analyses and no indication of selection of the cohort or subgroups for analysis and reporting on the basis of the results. |
Risk: Serious SfJ: The study has some important problems, e.g: serious risk of selection bias (residual confounding, selection of the participants), information bias, performance bias, detection bias, measurement bias and observer bias. |
| Alacrόn et al.[28], 2015 |
Outcomes: O1, O2,O3. Risk: Serious SfJ: Critically important confounders (pretreatment overjet, soft tissue variation) not measured and not adjusted for in the analysis. |
Outcomes: O1, O2,O3. Risk: Serious SfJ: Selection into the study was related to intervention and probably the outcome. |
Outcomes: O1, O2,O3. Risk: Moderate SfJ: Intervention status is well defined but data were obtained retrospectively. |
Outcomes: O1, O2,O3. Risk: Low SfJ: No bias due to departures from intended intervention is expected. |
Outcomes: O1, O2,O3. Risk: Moderate SfJ: Reasons for missingness (for baseline confounders) differ minimally across interventions and missing data were not addressed in the analysis. |
Outcomes: O1, O2,O3. Risk: Low SfJ: Comparable methods of outcome assessment between the 2 groups, and blinded assessor to intervention status and any error in measuring the outcome unrelated to intervention status. |
Outcomes: O1, O2,O3. Risk: Moderate SfJ: Outcome measurements and analysis were clearly defined, internally and externally consistent, there was no indication of selection of the reported analysis from among multiple analyses and no indication of selection of the cohort or subgroups for analysis and reporting on the basis of the results. |
Risk: Serious SfJ: The study has some important problems, e.g: serious risk of selection bias (residual confounding and in the selection of the participants). |
| Tuncer et al.[18], 2009 |
Outcomes: O1. Risk: Serious SfJ: one critically important confounder (ethnicity) not measured and not adjusted for in the analysis. |
Outcomes: O1. Risk: Serious SfJ: Selection into the study was related to intervention and probably the outcome. |
Outcomes: O1. Risk: Moderate SfJ: Intervention status is well defined but data were obtained retrospectively. |
Outcomes: O1. Risk: Serious SfJ: Co-intervention is apparent (OBP) and not adjusted for in the analysis. |
Outcomes: O1. Risk: Moderate SfJ: Reasons for missingness (for baseline confounder) differ minimally across interventions and missing data were not addressed in the analysis. |
Outcomes: O1. Risk: Moderate SfJ: Comparable method of outcome assessment between the 2 groups, outcome measure is only minimally influenced by knowledge of intervention status and any systematic error in measurements only minimally related to intervention status. |
Outcomes: O1. Risk: Moderate SfJ: Outcome measurements and analysis were clearly defined, internally and externally consistent, there was no indication of selection of the reported analysis from among multiple analyses and no indication of selection of the cohort or subgroups for analysis and reporting on the basis of the results. |
Risk: Serious SfJ: The study has some important problems, e.g: serious risk of selection bias (residual confounding, selection of the participants), information bias and performance bias. |
| Gökalp and Kurt [29], 2005. |
Outcomes: O1,O2. Risk: Serious SfJ: critically important domains (ethnicity, age, skeletal Class of malocclusion) not measured and not adjusted for in the analysis. |
Outcomes: O1,O2. Risk: Serious SfJ: Selection into the study was related to intervention and probably the outcome. |
Outcomes: O1,O2. Risk: Moderate SfJ: Intervention status is well defined but data were obtained retrospectively. |
Outcomes: O1,O2. Risk: Low SfJ: No bias due to departures from intended intervention is expected. |
Outcomes: O1,O2. Risk: Moderate SfJ: Reasons for missingness (missing data on baseline confounders) differ minimally across interventions and missing data were not addressed in the analysis. |
Outcomes: O1,O2. Risk: Moderate SfJ: Comparable method of outcome assessment between the 2 groups, outcome measure only minimally influenced by knowledge of intervention status and any systematic error in measurements only minimally related to intervention status. |
Outcomes: O1. Risk: Moderate SfJ: Outcome measurements and analysis were clearly defined, internally and externally consistent, there was no indication of selection of the reported analysis from among multiple analyses and no indication of selection of the cohort or subgroups for analysis and reporting on the basis of the results. |
Risk: Serious SfJ: The study has some important problems, e.g: serious risk of selection bias (residual confounding and in the selection of the participants). |
|
Sakamoto et al. [23], 1984. |
Outcomes: O1. Risk: Serious SfJ: one critically important domain (age) not adjusted for in the analysis (skeletal maturity stage not addressed). |
Outcomes: O1. Risk: Serious SfJ: Selection into the study was related to intervention and probably the outcome. |
Outcomes: O1. Risk: Serious SfJ: Intervention status is not well defined (skeletal maturity stage, number of patients with additional lingual arch) and major aspects of the assignments of intervention were determined in a way that could have been affected by knowledge of the outcome. |
Outcomes: O1. Risk: Serious SfJ:Co-intervention (lingual arch) is apparent and not adjusted for in the analysis. |
Outcomes: O1. Risk: Serious SfJ: Because the reported results correspond to the 26 treated patients, while the untreated control group of 423 patients is used supplementarily to illustrate the aforementioned results, judgement is basically based on the missing data regarding the treated patients (data on baseline characteristics and confounders). |
Outcomes: O1. Risk: Serious SfJ: Outcome measure likely to be influenced by knowledge of the intervention status and assessed by outcome assessors aware of the received intervention. Any error in measuring the outcomes was also related to intervention. |
Outcomes: O1. Risk: Serious SfJ: Because there is moderate risk of bias in selective reporting of the results of the study, but critical risk only for the supplementary comparison to untreated controls (1 patient compared to 423 controls). |
Risk: Serious SfJ: The study has some important problems, e.g: serious risk of selection bias (residual confounding, selection of the participants),information bias, performance bias, detection bias, measurement bias and observer bias, outcome reporting bias. |
| Deguchi and McNamara [26], 1999. |
Outcomes: O1,O2 Risk: Low SfJ: No bias due to confounding is expected. |
Outcomes: O1,O2. Risk: Serious SfJ: Selection into the study was related to intervention and probably the outcome. |
Outcomes: O1,O2. Risk: Moderate SfJ: Intervention status is well defined, but data were retrospectively obtained. |
Outcomes: O1, O2. Risk: Serious SfJ: Co-intervention is apparent (upper lingual arch) and not adjusted for in the analysis. |
Outcomes: O1,O2. Risk: Low SfJ: Data were reasonably complete. |
Outcomes: O1,O2. Risk: Moderate SfJ: Comparable method of outcome assessment between the 2 groups, outcome measure only minimally influenced by knowledge of intervention status (no blinding reported) and any systematic error in measurements only minimally related to intervention status. |
Outcomes: O1,O2. Risk: Moderate SfJ: Outcome measurements and analysis were clearly defined, internally and externally consistent, there was no indication of selection of the reported analysis from among multiple analyses and no indication of selection of the cohort or subgroups for analysis and reporting on the basis of the results. |
Risk: Serious SfJ: The study has some important problems, e.g: serious risk of selection bias (selection of the participants),information bias and performance bias. |
| Akin et al.[19], 2015. |
Outcomes: O1,O2,O3. Risk: Serious SfJ: Critically important domains (ethnicity, individual soft-tissue variation, pre-treatment overjet) not measured and not adjusted for in the analysis. |
Outcomes: O1,O2,O3. Risk: Serious SfJ: Selection into the study was related to intervention (intervention was known and blinding was only performed for the selection of 15 cases for each group) and probably the outcome. |
Outcomes: O1,O2,O3. Risk: Serious SfJ:Intervention status is not well defined (not declared whether CC therapy was continued or ended by the time fixed appliances were placed and for what reason the flat Essix plate was used). |
Outcomes: O1,O2,O3. Risk: Serious SfJ: Co-interventions are apparent (flat Essix plate, possibly simultaneous use of fixed appliances) and not adjusted for in the analysis. |
Outcomes: O1,O2,O3. Risk: Moderate SfJ: Missing data (on baseline confounders) were not addressed in the analysis and reasons for missingness differ minimally across interventions. |
Outcomes: O1,O2,O3. Risk: Moderate SfJ: Comparable method of outcome assessment between the groups, outcome measure only minimally influenced by knowledge of intervention status (no blinding reported) and any systematic error in measurements only minimally related to intervention status. |
Outcomes: O1,O2,O3. Risk: Moderate SfJ: Outcome measurements and analysis were clearly defined, internally and externally consistent, there was no indication of selection of the reported analysis from among multiple analyses and no indication of selection of the cohort or subgroups for analysis and reporting on the basis of the results. |
Risk: Serious SfJ: The study has some important problems, e.g: serious risk of selection bias (residual confounding, selection of the participants), information bias and performance bias. |
| Lin et al.[20], 2007. |
Outcomes: O1,O2 Risk: Serious SfJ: Critically important domain (ethnicity) not measured and not adjusted for in the analysis. |
Outcomes: O1,O2. Risk: Serious SfJ: Selection into the study was related to intervention and probably the outcome. |
Outcomes: O1,O2 Risk: Moderate SfJ: Intervention status is well defined, but data were obtained retrospectively. |
Outcomes: O1,O2 Risk: Serious SfJ: Co-intervention (OMA) is apparent and not adjusted for in the analysis.. |
Outcomes: O1,O2 Risk: Moderate SfJ: Reasons for missingness differ minimally across the 2 groups and missing data were not addressed in the analysis. |
Outcomes: O1,O2 Risk: Moderate SfJ: Comparable method of outcome assessment between the groups, outcome measure only minimally influenced by knowledge of intervention status (no blinding reported) and any systematic error in measurements only minimally related to intervention status. |
Outcomes: O1,O2. Risk: Moderate SfJ: Outcome measurements and analysis were clearly defined, internally and externally consistent, there was no indication of selection of the reported analysis from among multiple analyses and no indication of selection of the cohort or subgroups for analysis and reporting on the basis of the results. |
Risk: Serious SfJ: The study has some important problems, e.g: serious risk of selection bias (residual confounding, selection of the participants), performance bias. |
| Barrett et al.[21], 2010. |
Outcomes: O1, O2,O3. Risk: Serious SfJ: Critically important confounder (Skeletal Class of malocclusion) not measured and not adjusted for in the analysis. |
Outcomes: O1,O2,O3. Risk: Serious SfJ: Selection into the study was related to intervention and probably the outcome. |
Outcomes: O1, O2,O3. Risk: Moderate SfJ: Intervention status is well defined, but data were retrospectively obtained. |
Outcomes: O1, O2,O3. Risk: Serious SfJ: Co-intervention is apparent (QH) and not adjusted for in the analysis. |
Outcomes: O1, O2,O3. Risk: Moderate SfJ: Reasons for missingness differ minimally across interventions and missing data (on baseline confounder) were not addressed in the analysis. |
Outcomes: O1, O2,O3. Risk: Moderate SfJ: Comparable method of outcome assessment between the groups, outcome measure only minimally influenced by knowledge of intervention status (no blinding reported) and any systematic error in measurements only minimally related to intervention status. |
Outcomes: O1,O2,O3. Risk: Moderate SfJ: Outcome measurements and analysis were clearly defined, internally and externally consistent, there was no indication of selection of the reported analysis from among multiple analyses and no indication of selection of the cohort or subgroups for analysis and reporting on the basis of the results. |
Risk: Serious SfJ: The study has some important problems, e.g: serious risk of selection bias (residual confounding, selection of the participants) and performance bias. |
| Abu Alhaija and Richardson [24], 1999. |
Outcomes: O1, O2,O3. Risk: Serious SfJ: Critically important confounders (ethnicity, age, Skeletal Class of malocclusion and soft-tissue variation) not measured and not adjusted for in the analysis. |
Outcomes: O1,O2,O3. Risk: Serious SfJ: Selection into the study was related to intervention and probably the outcome. |
Outcomes: O1, O2,O3. Risk: Moderate SfJ: Intervention status is well defined, but data were obtained retrospectively. |
Outcomes: O1, O2,O3. Risk: Serious SfJ: Co-intervention is apparent (URA) and not adjusted for in the analysis. |
Outcomes: O1, O2,O3. Risk: Moderate SfJ: Reasons for missingness differ minimally across the 2 groups and missing data (on baseline confounders) were not addressed in the analysis. |
Outcomes: O1, O2,O3. Risk: Moderate SfJ: Comparable method of outcome assessment between the groups, outcome measure only minimally influenced by knowledge of intervention status (no blinding reported) and any systematic error in measurements only minimally related to intervention status. |
Outcomes: O1, O2,O3. Risk: Moderate SfJ: Outcome measurements and analysis were clearly defined, internally and externally consistent, there was no indication of selection of the reported analysis from among multiple analyses and no indication of selection of the cohort or subgroups for analysis and reporting on the basis of the results. |
Risk: Serious SfJ: The study has some important problems, e.g: serious risk of selection bias (residual confounding, selection of the participants) and performance bias. |
| Abdelanby and Nassar [22], 2010. |
Outcomes: O1,O2 Risk: Serious SfJ: Critically important domain (ethnicity) not measured and not adjusted for in the analysis. |
Outcomes: O1,O2. Risk: Serious SfJ: Selection into the study was related to intervention and probably the outcome. |
Outcomes: O1,O2 Risk: Moderate SfJ: Intervention status is well defined but data were retrospectively obtained. |
Outcomes: O1,O2 Risk: Serious SfJ: Co-intervention is apparent (utilization of force magnitude) and not adjusted for in the analysis. |
Outcomes: O1,O2 Risk: Moderate SfJ: Reasons for missingness differ minimally across interventions and missing data (on baseline confounder) were not addressed in the analysis. |
Outcomes: O1,O2 Risk: Moderate SfJ: Comparable method of outcome assessment between the groups, outcome measure only minimally influenced by knowledge of intervention status (no blinding reported) and any systematic error in measurements only minimally related to intervention status. |
Outcomes: O1, O2 Risk: Moderate SfJ: Outcome measurements and analysis were clearly defined, internally and externally consistent, there was no indication of selection of the reported analysis from among multiple analyses and no indication of selection of the cohort or subgroups for analysis and reporting on the basis of the results. |
Risk: Serious SfJ: The study has some important problems, e.g: serious risk of selection bias (residual confounding, selection of the participants) and performance bias. |
a The cited references of the articles in the first column of the table correspond to the numbered references in the text.Y.o.P: indicates year of publication, O1: skeletal effects, O2: dentoalveolar effects, O3: soft-tissue effects, SfJ: support for judgement, OBP: occlusal bite plate, RHG: reversed headgear, CC: chin-cup, OMA: occipitomental anchorage appliance of maxillary protraction combined with chin-cup traction, QH: quad-helix, URA: upper removable appliance.