Risk of bias: why measure it, and how?

Selection bias

High quality RCTs will randomize patients, and as important, conceal that randomization. Why? To limit selection bias. Selection bias is best described as a fundamental difference between the patients included in the treatment arms of a study due to the way in which patients were allocated to the treatment groups [1, 4]. To assess selection bias, one must consider both the random sequence generation and allocation concealment methods of the RCT [3].

Sequence generation refers to the method in which patients were randomized to the treatment groups. A truly random sequence for treatment allocation means that the baseline characteristics of both groups will be inherently balanced, but a bias in this allocation can result in systematic differences between the comparison groups [1, 3]. Some possible risks of bias due to sequence generation may exist due to non-, quasi-randomized methods of allocation. These methods may allow clinicians to choose which treatment patients’ will get within the study based on their expertize and prior experiences (i.e., A non-random factor). In addition, allocation concealment refers to the methods used to prevent anyone from being able to predict or deduce patient allocations [3]. Proper allocation concealment can prevent anyone in the research team from determining or predicting which patient’s have gotten which treatment within the trial. In summary, sequence generation refers to how patients are allocated to the comparison groups, and allocation concealment refers to how that allocation is kept secret from all relevant parties.

Performance bias

What about factors that can influence how a patient of clinician performs during the course of an RCT? Performance bias may be present if there are differences between the study groups as a result of systematic differences in performance outside of the study treatment received [1]. Risks of performance bias can result due to the masking (or blinding) methods of participants and personnel [1]. If masking is appropriately implemented, one can be assured that there was no additional and undue influence on the outcome of patients that occurred other than the assigned intervention [3, 5]. Many vital outcomes in ophthalmology, such as assessments of visual acuity, could be skewed if the patient or assessor are aware of the treatment allocation. When assessing performance bias, it is important to consider if the lack of masking could reasonably impact the outcomes being assessed [1].

Detection bias

The prior biases have focussed on the methods for randomizing and masking patients and clinicians, but what about biases in the way outcomes are measured? Detection bias can be described as the possibility for differences between the comparison groups with regard to how the outcomes are measured or assessed [1]. Detection bias also focuses on the concept of masking; however, it is the outcome assessor that should be masked in order to mitigate detection bias [3]. Masking of outcome assessors ensures that the methods in which an outcome is measured does not differ between patients allocated to the comparison groups—meaning that the outcome measurement is consistent for all participants in the study [1, 3].

Attrition bias

After patients have been included in an RCT, there is always a potential for them to withdrawal from the study before completing their follow-ups. Attrition bias can occur as a result of systematic cause of patient withdrawals in a study that disproportionately affect a certain subset of patients [1]. If a cause for withdrawal is present—or more predominant—in the comparison groups, the withdrawal imbalance could impact the results and conclusions drawn from the study [1, 3]. If a specific group of patients were more likely to withdrawal from the study within one of the comparison groups, the imbalance would have clear implications on results [1, 6].

Reporting bias

The final form of bias that any clinician should consider when reading an RCT is reporting bias. Reporting bias may occur when there are concerns with regard to the outcomes reported within the results of a study [1]. Selective outcome reporting is the primary concern in this form of bias, which refers to the reporting of some, but not all, measured outcomes within a study’s results [1, 3]. This commonly manifests as a study reporting on significant outcome findings, while omitting outcome findings that are not significant [1, 7]. Although this can be difficult to detect, it highlights the importance of a pre-defined study protocol that identifies all outcomes that will be assessed. You, as a reader, should actively seek the confirmation of that important step.

How should we interpret risk of bias assessments?

The next time you read an RCT, consider these risks of bias before making changes to your clinical practice. RCTs that are deemed to have a high risk of bias should be interpreted cautiously, as biases directly impacts the validity of the findings [1]. Empirical investigations have shown studies with high risk of bias may lead to an exaggeration of treatment effects within trials when compared to studies with a low risk of bias [8, 9]. It is common to assess the risks of bias in a study based solely on the reporting in the study manuscript, but poor reporting is not the same as biased conduct [3]. This is an important distinction to make with regard to risk of bias assessment that requires thoughtful consideration of the potential validity implications of study design decisions. You can refer to the Cochrane Handbook for Systematic Reviews of Interventions, “Chapter 8: Risk of Bias in Randomized Trials” for a comprehensive guide to risk of bias assessment for RCTs [1].

Summary

What is risk of bias

• Clinicians read and interpret randomized controlled trials (RCTs) on a regular basis to inform their practice—but how can they be certain that the RCT is accurate and reliable? Not all RCTs are the same, and thus careful consideration needs to be taken when determining if RCT results are worthy of changing the way you manage future patients. The validity of an RCT can best be evaluated by understanding the possible risks of bias for that particular study.

• Bias exists when a component of the design or execution of a study has systematic impacts on the results of the study that deviate from the truth. When such a bias exists, a study could result in over- or underestimation of the truth, compromising the validity of the study findings or results—even if all other facets of the study were appropriate [1–3].

• Imagine, for example, providing navigation using a compass that was not accurately pointing “North”, but instead had a bias of pointing toward “North East”. Even if you provided thorough navigation steps to a fellow traveller, the end result will not be accurate due to the bias that existed from the inaccuracy of the compass. Similarly, an otherwise robust study that has some form of bias may provide results to clinicians and patients that are not accurate, despite the comprehensiveness of the investigation. With this in mind, it is important to understand the types of bias that may exist within RCTs, how to detect these potential biases, and how to interpret the results of a study in the context of such possible biases.

What types of bias exist, and how can we assess them

• There are five main forms of bias that are important to consider for clinical trials: Selection Bias, Performance Bias, Detection Bias, Attrition Bias, and Reporting Bias (Table 1) [1, 3]. The Cochrane Risk of Bias assessment tool is the gold standard bias assessment tool for RCTs, as it assesses the risk of each of these forms of bias [3]. Below, we provide a summary of each form of bias and discuss how to minimize the risk of each bias during the design, conduct, analysis and reporting of trials.

Table 1.

Types of Bias Summary.

Bias	Summary	Example
Selection bias	Bias due to the methods used to assign patients to study treatment groups.	A surgeon in a glaucoma laser versus topical medicine RCT can accurately guess the allocation of future patients. They may then preferentially wait to identify the “ideal” patient for each treatment arm, opposed to having them assigned at random.
Performance bias	Bias that occurs when patients or clinicians are aware of the assigned treatment, and perform differently as a result.	A patient learns that they received the placebo treatment in a study. When they are performing a visual acuity test they, consciously or subconsciously, do not perform their best due to knowing they received a null treatment.
Detection bias	Bias in the measurement of study outcomes when outcome assessors are aware of the assigned treatment.	A surgeon grading post operative inflammation in an ophthalmology RCT is not masked to the patient’s treatment, and this knowledge influences their assessments based on prior knowledge and experiences.
Attrition bias	Bias due to an influencing factor that causes non-random withdrawals from the study groups.	A study assessing visual acuity after retinal detachment has a large number of withdrawals that occurred primarily in patients of lower socioeconomic status.
Reporting bias	Bias in the outcomes reported by a study, mainly when non-significant findings are ignored.	A published RCT on cataract surgery stated that they would assess visual acuity, adverse events, and quality of life within their protocol; however, only visual acuity and adverse event outcomes are reported in the manuscript.

Author contributions

MP was responsible for conception of idea, writing of paper and review of paper. VC was responsible for conception of idea, writing of paper and review of paper. MB was responsible for conception of idea, writing of paper and review of paper. PK was responsible for critical review and feedback on paper. LT was responsible for critical review and feedback on paper.

Competing interests

MP: Nothing to disclose. LT: Nothing to disclose. VC: Advisory Board Member: Alcon, Roche, Bayer, Novartis; Grants: Bayer, Novartis—unrelated to this study. PK: Consultant; Novartis, Bayer, Regeneron, Kanghong, Allergan, RegenxBio—unrelated to this study. MB: Research funds; Pendopharm, Bioventus, Acumed—unrelated to this study.