Skip to main content
. 2018 Apr 5;2018(4):CD008509. doi: 10.1002/14651858.CD008509.pub3
Potential source of bias Assessment criteria
Random sequence generation
Selection bias (biased allocation to interventions) due to inadequate generation of a randomised sequence
Low risk of bias: Random number table; computer random number generator; coin tossing; shuffling cards or envelopes; throwing dice; drawing of lots; minimization (minimization may be implemented without a random element, and this is considered to be equivalent to being random).
High risk of bias: Sequence generated by odd or even date of birth; date (or day) of admission; sequence generated by hospital or clinic record number; allocation by judgement of the clinician; by preference of the participant; based on the results of a laboratory test or a series of tests; by availability of the intervention.
Unclear: Insufficient information about the sequence generation process to permit judgement.
Allocation concealment
Selection bias (biased allocation to interventions) due to inadequate concealment of allocations prior to assignment
Low risk of bias: Randomization method described that would not allow investigator/participant to know or influence intervention group before eligible participant entered in the study (e.g. central allocation, including telephone, web‐based, and pharmacy‐controlled, randomization; sequentially numbered drug containers of identical appearance; sequentially numbered, opaque, sealed envelopes).
High risk of bias: Using an open random allocation schedule (e.g. a list of random numbers); assignment envelopes were used without appropriate safeguards (e.g. if envelopes were unsealed or non‐opaque or not sequentially numbered); alternation or rotation; date of birth; case record number; any other explicitly unconcealed procedure.
Unclear: Randomization stated but no information on method used is available.
Blinding of participants and personnel
Performance bias due to knowledge of the allocated interventions by participants and personnel during the study
Low risk of bias: No blinding or incomplete blinding, but the review authors judge that the outcome is not likely to be influenced by lack of blinding; blinding of participants and key study personnel ensured, and unlikely that the blinding could have been broken.
High risk of bias: No blinding or incomplete blinding, and the outcome is likely to be influenced by lack of blinding; blinding of key study participants and personnel attempted, but likely that the blinding could have been broken, and the outcome is likely to be influenced by lack of blinding.
Unclear: Insufficient information to permit judgement
Blinding of outcome assessment
Detection bias due to knowledge of the allocated interventions by outcome assessors.
Low risk of bias: No blinding of outcome assessment, but the review authors judge that the outcome measurement is not likely to be influenced by lack of blinding; blinding of outcome assessment ensured, and unlikely that the blinding could have been broken.
High risk of bias: No blinding of outcome assessment, and the outcome measurement is likely to be influenced by lack of blinding; blinding of outcome assessment, but likely that the blinding could have been broken, and the outcome measurement is likely to be influenced by lack of blinding.
Unclear: Insufficient information to permit judgement
Incomplete outcome data
Attrition bias due to amount, nature or handling of incomplete outcome data.
Low risk of bias: No missing outcome data; reasons for missing outcome data unlikely to be related to true outcome (for survival data, censoring unlikely to be introducing bias); missing outcome data balanced in numbers across intervention groups, with similar reasons for missing data across groups; for dichotomous outcome data, the proportion of missing outcomes compared with observed event risk not enough to have a clinically relevant impact on the intervention effect estimate; for continuous outcome data, plausible effect size (difference in means or standardized difference in means) among missing outcomes not enough to have a clinically relevant impact on observed effect size; missing data have been imputed using appropriate methods.
High risk of bias: Reason for missing outcome data likely to be related to true outcome, with either imbalance in numbers or reasons for missing data across intervention groups; for dichotomous outcome data, the proportion of missing outcomes compared with observed event risk enough to induce clinically relevant bias in intervention effect estimate; for continuous outcome data, plausible effect size (difference in means or standardized difference in means) among missing outcomes enough to induce clinically relevant bias in observed effect size; ‘as‐treated’ analysis done with substantial departure of the intervention received from that assigned at randomization; potentially inappropriate application of simple imputation.
Unclear: Insufficient information to permit judgement
Selective reporting
Reporting bias due to selective outcome reporting
Low risk of bias: The study protocol is available and all of the study’s pre‐specified (primary and secondary) outcomes that are of interest in the review have been reported in the pre‐specified way; the study protocol is not available but it is clear that the published reports include all expected outcomes, including those that were pre‐specified (convincing text of this nature may be uncommon).
High risk of bias: Not all of the study’s pre‐specified primary outcomes have been reported; one or more primary outcomes is reported using measurements, analysis methods or subsets of the data (e.g. subscales) that were not pre‐specified; one or more reported primary outcomes were not pre‐specified (unless clear justification for their reporting is provided, such as an unexpected adverse event); one or more outcomes of interest in the review are reported incompletely so that they cannot be entered in a meta‐analysis; the study report fails to include results for a key outcome that would be expected to have been reported for such a study.
Unclear: Insufficient information to permit judgement
Other bias
Bias due to problems not covered elsewhere in the table
Low risk of bias: The study appears to be free of other sources of bias.
High risk of bias: Had a potential source of bias related to the specific study design used; stopped early due to some data‐dependent process (including a formal‐stopping rule); had extreme baseline imbalance; has been claimed to have been fraudulent; had some other problem.
Unclear: Insufficient information to assess whether an important risk of bias exists; insufficient rationale or evidence that an identified problem will introduce bias.