Skip to main content
. 2020 Jul 20;2020(7):CD013684. doi: 10.1002/14651858.CD013684
Item Judgment Description
1. Random sequence generation (selection bias). We described the method used to generate the allocation sequence in sufficient detail for each included trial to allow an assessment of whether it should have produced comparable groups Low risk The investigators described a random component in the sequence generation process such as:
  • random‐number table;

  • computer random‐number generator;

  • coin tossing;

  • shuffling cards or envelopes;

  • throwing dice;

  • drawing of lots; or

  • minimisation.*


*Minimisation may be implemented without a random element (treatment sums are equal), and this is considered to be equivalent to being random
High risk The researchers described a (systematic or non‐systematic) non‐random component in the sequence generation process such as:
  • systematic, non‐random approach

    • generating the sequence by, for example:

      • odd or even date of birth;

      • date (or day) of admission;

      • hospital or clinic record number; or

      • alternation.

  • non‐systematic, non‐random approach

    • allocating the participant by, for example:

      • judgement of the clinician;

      • preference of the participant;

      • results of a laboratory test or a series of tests; or

      • availability of the intervention.

Unclear risk Insufficient information provided to permit a judgement of ‘Low risk’ or ‘High risk’.
2. Allocation concealment (selection bias). For each RCT we described the method used to conceal the allocation sequence in sufficient detail to determine whether intervention allocations could have been foreseen in advance of, or during, enrolment Low risk Participants and investigators enrolling participants could not have foreseen assignment because one of the following, or an equivalent method, was used to conceal allocation:
  • central allocation (including telephone, web‐based and pharmacy‐controlled randomisation);

  • sequentially‐numbered drug containers of identical appearance; or

  • sequentially‐numbered, opaque, sealed envelopes

High risk Participants or investigators enrolling participants could possibly have foreseen assignment and thus introduced selection bias because one of the following methods was used:
  • open random allocation schedule (e.g. a list of random numbers);

  • assignment envelopes without appropriate safeguards (e.g. if envelopes were unsealed or non‐opaque or not sequentially numbered);

  • alternation or rotation;

  • date of birth;

  • case record number; or

  • any other explicitly unconcealed procedure.

Unclear risk Insufficient information was provided to permit a judgement of ‘Low risk’ or ‘High risk’. This was usually the case if the method of concealment was not described or not described in sufficient detail to allow a definite judgement (e.g. if the use of assignment envelopes was described, but it remained unclear whether envelopes were sequentially numbered, opaque and sealed)
3. Blinding of participants and personnel (performance bias): objective outcomes. For each included trial we described all methods used to blind trial participants and personnel from knowledge of which intervention a participant received. We provided any information relating to whether the intended blinding was effective. We assessed blinding separately for different classes of outcomes. Outcomes were divided into objective (e.g. cortisol) and subjective (e.g. self‐reported resilience and other psychological outcomes). We considered the same outcomes at different time points Low risk Any one of the following:
  • no blinding or incomplete blinding, but the review authors judged that the outcome was not likely to have been influenced by lack of blinding; or

  • blinding of participants and key study personnel ensured, and unlikely that the blinding could have been broken

4. Blinding of participants and personnel (performance bias): subjective outcomes. For each included trial we described all methods used to blind trial participants and personnel from knowledge of which intervention a participant received. We provided any information relating to whether the intended blinding was effective. We assessed blinding separately for different classes of outcomes. Outcomes were divided into objective (e.g. cortisol) and subjective (e.g. self‐reported resilience and other psychological outcomes). We considered the same outcomes at different time points Low risk Blinding of participants and intervention providers, and unlikely that the blinding could have been broken
High risk Any one of the following:
  • no blinding or incomplete blinding, and the outcome was likely to have been influenced by lack of blinding; or

  • blinding of key study participants and personnel attempted, but likely that the blinding could have been broken; and the outcome was likely to have been influenced by the lack of blinding

Unclear risk Insufficient information provided to permit a judgement of ‘Low risk’ or ‘High risk’
5. Blinding of outcome assessors (detection bias): objective outcomes. For each included trial we described all methods used to blind outcome assessors from knowledge of which intervention a participant received. We provided any information relating to whether the intended blinding was effective. We assessed blinding separately for different classes of outcomes. Outcomes were divided into objective (e.g. cortisol) and subjective (e.g. self‐reported resilience and other psychological outcomes). We considered the same outcomes at different time points Low risk Any one of the following:
  • no blinding of outcome assessment, but the review authors judged that the outcome measurement was not likely to have been influenced by lack of blinding; or

  • blinding of outcome assessment ensured, and unlikely that the blinding could have been broken

6. Blinding of outcome assessors (detection bias): subjective outcomes. For each included trial we described all methods used to blind outcome assessors from knowledge of which intervention a participant received. We provided any information relating to whether the intended blinding was effective. We assessed blinding separately for different classes of outcomes. Outcomes were divided into objective (e.g. cortisol) and subjective (e.g. self‐reported resilience and other psychological outcomes). We considered the same outcomes at different time points Low risk Any one of the following:
  • no blinding of outcome assessment, but the review authors judged that the outcome measurement was not likely to have been influenced by lack of blinding; or

  • blinding of outcome assessment ensured, and unlikely that the blinding could have been broken

High risk Any one of the following:
  • no blinding of outcome assessment, and the outcome measurement was likely to have been influenced by lack of blinding; or

  • blinding of outcome assessment, but likely that the blinding could have been broken, and the outcome measurement is likely to have been influenced by lack of blinding

Unclear risk Insufficient information provided to permit a judgment of ‘Low risk’ or ‘High risk’
7. Incomplete outcome data (attrition bias). For each RCT we described the completeness of outcome data for each main outcome, including attrition and exclusions from the analysis. We stated whether attrition and exclusions were reported, the numbers included at each stage (compared with the total number of participants randomised), reasons for attrition or exclusions (where reported), and whether missing data were balanced across groups or were related to outcomes. Where sufficient data were reported, or could be provided by the study authors, we re‐included the missing data in the analyses Low risk Any one of the following:
  • no missing outcome data;

  • reasons for missing outcome data were unlikely to be related to true outcome (for survival data, censoring unlikely to be introducing bias);

  • missing outcome data balanced in numbers across intervention groups, with similar reasons for missing data across groups;

  • for dichotomous outcome data, the proportion of missing outcomes compared with observed event risk was not enough to have a clinically‐relevant impact on the intervention effect estimate;

  • for continuous outcome data, plausible effect size (difference in means or standardised difference in means) among missing outcomes was not enough to have a clinically‐relevant impact on the observed effect size;

  • missing data were imputed using appropriate methods; or

  • intention‐to‐treat; all randomised participants were analysed in the group to which they were allocated by randomisation, irrespective of noncompliance and co‐interventions

High risk Any one of the following:
  • reasons for missing outcome data were likely to be related to true outcome, with either imbalance in numbers or reasons for missing data across intervention groups;

  • for dichotomous outcome data, the proportion of missing outcomes compared with observed event risk was enough to induce clinically‐relevant bias in the intervention effect estimate;

  • for continuous outcome data, plausible effect size (difference in means or standardised difference in means) among missing outcomes was enough to induce clinically‐relevant bias in observed effect size;

  • potentially inappropriate application of simple imputation; or

  • ‘as‐treated’ analysis done with substantial departure of the intervention received from that assigned at randomisation

Unclear risk Insufficient reporting of attrition or exclusions to permit a judgement of ‘Low risk’ or ‘High risk’ (e.g. number randomised not stated, no reasons for missing data provided, number of dropouts not reported for each group)
8. Selective outcome reporting (reporting bias). For each included trial we described how the possibility of selective outcome reporting was examined, and what was found Low risk Any one of the following:
  • the study protocol was available and all of the study’s prespecified (primary and secondary) outcomes that were of interest in the review have been reported in the prespecified way; or

  • the study protocol was not available, but it was clear that the published reports included all expected outcomes, including those that were prespecified (convincing text of this nature may be uncommon)

High risk Any one of the following:
  • not all of the study’s prespecified primary outcomes have been reported;

  • one or more primary outcomes were reported using measurements, analysis methods or subsets of the data (e.g. subscales) that were not prespecified;

  • one or more reported primary outcomes were not prespecified (unless clear justification for their reporting was provided such as an unexpected adverse effect);

  • one or more outcomes of interest in the review were reported incompletely so that they could not be entered in a meta‐analysis; or

  • the study report failed to include results for a key outcome that was expected to have been reported for such a study

Unclear risk Insufficient information provided to permit a judgement of ‘Low risk’ or ‘High risk.
RCT(s): randomised controlled trial(s)