Random sequence generation (selection bias due to inadequate generation of a randomised sequence)
For each included trial, we described the method used to generate the allocation sequence in sufficient detail to allow an assessment of whether it should produce comparable groups.
Low risk of bias: trial authors achieved sequence generation using computer‐generated random numbers or a random numbers table. Drawing lots, tossing a coin, shuffling cards or envelopes, and throwing dice are adequate if an independent person performed this who was not otherwise involved in the trial. We will consider use of the minimisation technique as equivalent to being random Unclear risk of bias: insufficient information about the sequence generation process High risk of bias: the sequence generation method was non‐random or quasi‐random (e.g. sequence generated by odd or even date of birth; sequence generated by some rule based on date (or day) of admission; sequence generated by some rule based on hospital or clinic record number; allocation by judgment of the clinician; allocation by preference of the participant; allocation based on the results of a laboratory test or a series of tests; or allocation by availability of the intervention)
Allocation concealment (selection bias due to inadequate concealment of allocation prior to assignment)
We described for each included trial the method used to conceal allocation to interventions prior to assignment, and we assessed whether intervention allocation could have been foreseen in advance of or during recruitment or changed after assignment.
Low risk of bias: central allocation (including telephone, interactive voice‐recorder, Internet‐based and pharmacy‐controlled randomisation); sequentially numbered drug containers of identical appearance; sequentially numbered, opaque, sealed envelopes Unclear risk of bias: insufficient information about allocation concealment High risk of bias: open random allocation schedule (e.g. a list of random numbers) used; assignment envelopes used without appropriate safeguards; alternation or rotation; date of birth; case record number; any other explicitly unconcealed procedure
We also evaluated trial baseline data to incorporate assessment of baseline imbalance into the 'Risk of bias' judgement for selection bias (Corbett 2014). Chance imbalances may also affect judgements on risk of attrition bias. In the case of unadjusted analyses, we distinguished between trials that we rated as being at low risk of bias on the basis of both randomisation methods and baseline similarity, and trials that we judged as being at low risk of bias on the basis of baseline similarity alone (Corbett 2014). We re‐classified judgements of unclear, low, or high risk of selection bias as specified in Appendix 4.
Blinding of participants and study personnel (performance bias due to knowledge of allocated interventions by participants and personnel during the trial)
Low risk of bias: blinding of participants and key study personnel was ensured, and it was unlikely that the blinding could have been broken; no blinding or incomplete blinding, but we judge that the outcome is unlikely to have been influenced by lack of blinding Unclear risk of bias: insufficient information about blinding of participants and study personnel High risk of bias: no blinding or incomplete blinding; outcome is likely to have been influenced by lack of blinding; blinding of trial participants and key personnel attempted, but likely that blinding could have been broken, and the outcome is likely to be influenced by lack of blinding
Blinding of outcome assessment (detection bias due to knowledge of allocated interventions by outcome assessment)
Low risk of bias: blinding of outcome assessment is ensured, and it was unlikely that blinding could have been broken; no blinding of outcome assessment, but we judge that the outcome measurement was unlikely to have been influenced by lack of blinding Unclear risk of bias: insufficient information about blinding of outcome assessors High risk of bias: no blinding of outcome assessment, and outcome measurement was likely to have been influenced by lack of blinding; blinding of outcome assessment, but likely that blinding could have been broken, and the outcome measurement was likely to be influenced by lack of blinding
Incomplete outcome data (attrition bias due to quantity, nature, or handling of incomplete outcome data)
For each included trial or each outcome, or both, we described the completeness of data, including attrition and exclusions from analyses. We stated whether the trial reported attrition and exclusions, and we reported the number of participants included in the analysis at each stage (compared with the number of randomised participants per intervention/comparator groups). We also noted if the trial reported reasons for attrition or exclusion, and whether missing data were balanced across groups or were related to outcomes. We considered the implications of missing outcome data per outcome such as high dropout rates (e.g. above 15%) or disparate attrition rates (e.g. difference of 10% or more between trial arms).
Low risk of bias: no missing outcome data; reasons for missing outcome data unlikely to be related to true outcomes (for survival data, censoring unlikely to introduce bias); missing outcome data balanced in numbers across intervention groups, with similar reasons for missing data across groups; for dichotomous outcome data, the proportion of missing outcomes compared with observed event risk was not enough to have a clinically relevant impact on the intervention effect estimate; for continuous outcome data, plausible effect size (mean difference or standardised mean difference) among missing outcomes was not enough to have a clinically relevant impact on observed effect size; appropriate methods, such as multiple imputations, were used to handle missing data Unclear risk of bias: insufficient information to assess whether missing data in combination with the method used to handle missing data were likely to induce bias High risk of bias: reason for missing outcome data was likely to be related to true outcome, with either imbalance in numbers or reasons for missing data across intervention groups; for dichotomous outcome data, the proportion of missing outcomes compared with observed event risk enough to induce clinically relevant bias in the intervention effect estimate; for continuous outcome data, plausible effect size (mean difference or standardised mean difference) among missing outcomes enough to induce clinically relevant bias in observed effect size; 'as‐treated' or similar analysis done with substantial departure of the intervention received from that assigned at randomisation; potentially inappropriate application of simple imputation
Selective reporting (reporting bias due to selective outcome reporting)
We assessed outcome reporting bias by integrating results of the appendix 'Matrix of trial endpoints (publications and trial documents)' (Boutron 2014; Jones 2015a; Mathieu 2009), with those of the appendix 'High risk of outcome reporting bias according to the Outcome Reporting Bias In Trials (ORBIT) classification' (Kirkham 2010). This analysis formed the basis for the judgement of selective reporting.
Low risk of bias: the trial protocol was available and all the trial's pre‐specified (primary and secondary) outcomes that were of interest to this review were reported in the pre‐specified way; the study protocol was unavailable, but it is clear that published reports included all expected outcomes (ORBIT classification) Unclear risk of bias: insufficient information about selective reporting High risk of bias: not all the trial's pre‐specified primary outcomes were reported; 1 or more primary outcomes were reported using measurements, analysis methods, or subsets of the data (e.g. sub‐scales) that were not pre‐specified; 1 or more reported primary outcomes were not pre‐specified (unless clear justification for their reporting was provided, such as an unexpected adverse effect); 1 or more outcomes of interest in the Cochrane Review were reported incompletely, so that we could not enter them into a meta‐analysis; the trial report failed to include results for a key outcome that we would expect to have been reported for such a trial (ORBIT classification)
Specific to cluster‐RCTs: recruitment bias
For all cluster‐RCTs, we assessed recruitment bias by assessing whether individual participants were recruited to the trial, and individual‐level data were collected before or after clusters were randomised to an intervention or control group. This served as the basis for judgement of recruitment bias. Although the unit of randomisation is the cluster, bias could be introduced if individual participants knew whether the school or the classroom would receive the intervention or control condition prior to deciding whether or not to join the study.
Low risk of bias: investigators describe a procedure in which all participants were recruited and data were collected before randomising clusters to intervention or control groups, or if individual participants were not recruited at all but were identified prior to randomisation Unclear risk of bias: insufficient information about the process to permit judgement High risk of bias: clusters were randomised before recruitment to the trial was complete, and knowledge of whether each cluster was an ‘intervention’ or ‘control’ cluster could affect types of participants recruited
Specific to cluster‐RCTs: baseline imbalance
Often in cluster‐randomised trials, individuals within a cluster are more similar than participants across clusters. Particularly when the number of clusters randomised is small, there may be differences in baseline characteristics across study groups even if randomisation was successful.
Low risk of bias: investigators explored baseline imbalances and report that no imbalance was found or properly adjusted for any baseline balance in the analysis Unclear risk of bias: investigators did not explore baseline imbalances, or it Is unclear whether any baseline imbalances exist High risk of bias: baseline imbalances were observed between study groups and were not accounted for in the analysis
Specific to cluster‐RCTs: loss of clusters
Low risk of bias: no individual outcomes or clusters of missing data; reasons for clusters or outcomes missing unlikely to be related to intervention (for survival data, censoring unlikely to be introducing bias); missing outcome data balanced in numbers across clusters, with similar reasons for missing data across clusters; for dichotomous outcome data, the proportion of missing outcomes compared with observed event risk was not enough to have a clinically relevant impact on the intervention effect estimate; for continuous outcome data, plausible effect size (difference in means or standardised difference in means) among missing outcomes not enough to have a clinically relevant impact on observed effect size; missing data have been imputed using appropriate methods Unclear risk of bias: insufficient reporting of attrition/exclusions to permit judgement (e.g. number randomised not stated, no reason for missing data provided) High risk of bias: reason for missing outcome data likely to be related to true outcome, with either imbalance in numbers or reasons for missing data across intervention groups; for dichotomous outcome data, the proportion of missing outcomes compared with observed event risk enough to induce clinically relevant bias in intervention effect estimate; for continuous outcome data, plausible effect size (difference in means or standardised difference in means) among missing outcomes enough to induce clinically relevant bias in observed effect size; 'as‐treated' analysis done with substantial departure of interventions received from that assigned at randomisation; potentially inappropriate application of simple imputation
Specific to cluster‐RCTs: incorrect analysis
Because participants in any cluster are often more similar than participants across clusters and tend to respond to an intervention in a similar manner, their data cannot be assumed to be independent. Failing to account for this is often referred to as 'unit‐of‐analysis error' because the unit of analysis is different from the unit of allocation (Whiting‐O'Keefe 1984), and many cluster‐randomised trials have been incorrectly analysed in this manner (Eldridge 2008).
Low risk of bias: investigators clearly describe consideration of the clustered nature of the data in their statistical analysis Unclear risk of bias: insufficient reporting of statistical analysis procedures to permit judgement High risk of bias: data are analysed with the unit of analysis at the individual level, and investigators do not consider the clustered nature of data in the analysis
|
