Table 1.
Quality assessment criteria of trials
| Selection bias | ||
|---|---|---|
| Score | Criteria | Impact on bias risk |
| Randomisation and concealment | ||
| A | (i) Randomisation: Details of any adequate type of allocation method that generates random sequences with the patient as unit of randomisation are reported.1 | Doubts may still exist whether the trial results are influenced by selection bias but no indication can be found from the trial report to support such doubt. |
| (ii) Concealment: Trial provides evidence2 that concealment was indeed effective and that the random sequence could not have been observed or predicted throughout the duration of the trial. | ||
| B | (i) Randomisation: Details of any adequate type of allocation method that generates random sequences with the patient as unit of randomisation are reported.1 | Despite the implementation of method considered to be able to prevent unmasking of the concealed allocation sequence through direct observation and prediction, there are reasons to expect that the concealed allocation sequence may have been unmasked during the cause of the trial. |
| (ii) Concealment: Trial reports on any adequate method to prevent direct observation3 and prediction4 of the allocation sequence and sequence generation rules. | ||
| C | (i) Randomisation: Details of any adequate type of allocation method that generates random sequences with the patient as unit of randomisation are reported.1 | Despite the implementation of method considered to be able to prevent unmasking of the concealed allocation sequence through direct observation, there are reasons to expect that operators could have predicted the concealed allocation sequence. |
| (ii) Concealment: Trial reports on any adequate method to prevent direct operator observation of allocation sequence and sequence generation rules3. However, the allocation sequence and sequence generation may have been sufficiently predicted. | ||
| D | (i) Randomisation: Details of any adequate type of allocation method that generates random sequences with the patient as unit of randomisation are reported.1 | Despite the theoretical chance for each patient to be allocated to either treatment group, operator knowledge of the allocation sequence may have lead to patient allocation that favoured the outcome of one type of treatment above the other. |
| (ii) Concealment: The trial report does not include information on how the allocation of random sequence was concealed. The allocation could have been directly observed and/or predicted. | ||
| 0 | Trial does not comply with criteria A - D. | No guaranty of equal chance for patients to be allocated to either treatment group, thus allocation may have favoured the outcome of one type of treatment above the other. |
| Baseline data for randomised trials | ||
| A | Baseline data collected before randomisation and reported for both treatment groups. Data shows no significant differences between both groups. | Evidence is given that randomisation has lead to equal groups suggesting little risk of selection bias. |
| B | Baseline data collected before randomisation and reported for both treatment groups. Data shows significant differences between both groups but has been statistically adjusted appropriately. | Differences have been adjusted, thus the influence of possible selection bias appears to be reduced. |
| C | Baseline data collected before randomisation and reported for both treatment groups. Data shows significant differences between both groups without being statistically adjusted. | Reported differences may be due to ineffective randomisation, thus indicate risk of selection bias. |
| 0 | Trial does not comply with criteria A - C. | No evidence is given whether randomisation has indeed lead to equal groups with differences beyond chance, thus differences may exists indicating selection bias. |
| Detection/Performance bias | ||
| Blinding/Masking | ||
| Score | Criteria | Impact on bias risk |
| A | (i) Trial reports on any type of method that is known to prevent patient AND operator AND evaluator to discern whether patients are allocated to the test- or the control group (Blinding/Masking). | Evidence is given that the trial results may not have been influenced by detection/performance bias that may have favored the outcome of one type of treatment above the other. |
| (ii) Trial reports a process with which the effect of Blinding/Masking was evaluated, as well as the results of such evaluation. | ||
| B | (i) Trial reports on any type of method that is known to prevent patient AND operator AND evaluator to discern whether patients are allocated to the test- or the control group (Blinding/Masking). | Doubts may still exist whether the trial results are influenced by detection/performance bias but no indication can be found from the trial report to support such doubt. However, no evaluation of the Blinding/Masking effect has been included in the trial, thus no evidence for lack of bias is given. |
| (ii) Trial report does not give reason for doubt that the patient allocation to either the test- or the control group has been unmasked throughout the duration of the trial. | ||
| C | (i) Trial reports on any type of method that is known to prevent patient AND operator AND evaluator to discern whether patients are allocated to the test- or the control group (Blinding/Masking). | Despite the implementation of method considered to be able to prevent unmasking, there are reasons to expect that operators/patients could have discovered the allocation. |
| (ii) Trial report gives reason for doubt that the patient allocation to either the test- or the control group has been unmasked throughout the duration of the trial. | ||
| 0 | No process reported or implemented able to blind/mask patients AND operators whether patients where allocated to either the test- or the control group (It is insufficient to report that blinding/masking was done without reporting the details of the process). | Knowledge about the patient allocation may have caused patients/operator to act in a way that may have favoured the outcome of one type of treatment above the other, |
| Attrition bias | ||
| Loss - to follow up | ||
| Score | Criteria | Impact on bias risk |
| A | Available case analysis, loss-to-follow up reported per treatment group. Subsequent sensitivity analysis does not indicate a possible risk of bias. | The trial allows extracting evidence that attrition may not have favoured the outcome of one type of treatment above the other. |
| B | Available case analysis, loss-to-follow up reported per treatment group. Subsequent sensitivity analysis indicates a possible risk of bias. | The trial allows assessing the risk that attrition may have favoured the outcome of one type of treatment above the other. |
| 0 | Trial does not report number of included participants per treatment group at baseline or gives any indication that would allow ascertaining the loss-to-follow up rate per treatment group. | The trial carries an unknown risk that attrition may have favoured the outcome of one type of treatment above the other. |
| Run-in phase | ||
| A | No run-in phase reported or discernable during which patients were given the active treatment or the placebo/control. | The trial may not carry the risk of bias due to exclusion of patients who would not respond well to e.g. the active treatment. |
| 0 | Run-in phase reported or discernable during which patients were given the active treatment or the placebo/control. | During a run-in phase only patients were selected for randomisation that have responded/not responded to the active treatment of the placebo/control. This may favour the outcome of one type of treatment above the other as patients who did not respond well to either are excluded. |
| Trial endpoints | ||
| 0 | The trial reports on secondary or surrogate outcomes as endpoints. | Even if the surrogate results would highly correlate with primary (i.e. clinical) outcomes, they cannot serve as valid replacements and need to be regarded for hypothesis development, only. |
| A | The trial reports on primary outcomes as endpoints. | Primary outcomes may provide evidence for hypothesis testing. |
1 Excluded are types of allocation methods that are considered as inadequate: cluster randomisation, fixed block randomisation with block size 2, minimization, alternation, randomisation of teeth, use of date of birth or patient record number, "quasi"-randomisation, split-mouth
2 E.g. by reporting results of the Berger-Exner Test or any other statistical tests that show that covariates of compared groups were similar at baseline
3 E.g. by opening of opaque envelope, obtaining allocation from tables, computer generated or from other sources
4 E.g. central randomisation, sequence allocation by other than operator; excluding varied block randomisation