Skip to main content
. 2016 Jan 18;2016(1):CD003067. doi: 10.1002/14651858.CD003067.pub4

2. Criteria for 'Risk of bias' assessment.

Random sequence generation (selection bias) Was the method used to generate the allocation sequence appropriate to produce comparable groups? We graded this domain to 'low' risk of bias if study authors described a random component in the sequence generation process (e.g. random number table, coin tossing, drawing of lots). If information about the random sequence generation process was not provided or was insufficient, we graded this domain to 'unclear' risk
Allocation concealment (selection bias) Was the method used to conceal the allocation sequence appropriate to prevent the allocation from being known in advance of, or during, enrolment? We graded this domain to ‘low’ risk of bias if study authors described adequate concealment (e.g. by means of central randomisation, or sequentially numbered, opaque and sealed envelopes) and to 'high' risk of bias if inadequate concealment was documented (e.g. alternation, use of case record numbers, date of birth or day of the week) or if allocation concealment was not used. If insufficient or no information on allocation concealment was provided, the judgement was graded ‘unclear’ risk
Blinding (performance bias)   We did not carry out assessment of blinding of participants and personnel in this review because we think that personnel and participants potentially knowing which of the 2 active preventive treatments a child is given is unlikely to cause bias (e.g. affect dental behaviour of a child during the trial, especially when follow‐up is long (≥ 1 year in this review)).
Blinding of outcome assessment (detection bias) Were outcome assessors blinded to the intervention a participant had received? As sealant materials are visible, blinding of the outcome assessor is possible only if a sealant has been lost. Thus outcome measurement is related to sealant retention and blinding of outcome assessor is usually impossible. On the other hand, it is difficult to assess how likely (or not likely) it is that the outcome measurement is influenced by lack of blinding of outcome assessors in preventive sealant studies.
 
 We decided to grade this domain as having 'low' risk of bias if study authors stated that the outcome assessor was not involved in the study design, and as having 'unclear' risk of bias if the study simply reported blinded outcome assessment or if blinding was indicated (e.g. examinations performed independently of previous records, outcome assessors not involved in applying treatments). If a trial reported nothing about blinding of outcome measurement, our judgement was 'high' risk of bias in this domain.
Incomplete outcome data (attrition bias) How complete were the outcome data for primary caries outcomes? Were drop‐out rates and reasons for withdrawals reported? Were missing data imputed appropriately? In caries prevention studies, follow‐up times can last several years. Studies with long follow‐up have the problem of high drop‐out rates causing uncertainty about data. We decided to base the judgement of this domain on caries efficacy outcome at 24 or 36 months (commonly used follow‐up times in sealant studies). When both follow‐up times were reported, we based our judgement on 24 months. If either of these 2 follow‐up times was not reported, we based our judgement on the first caries efficacy outcome reported in the study (which in this review should be ≥ 1 year). However, the risk of bias was assessed separately and was reported in the 'Risk of bias' table for caries outcomes despite follow‐up times, and the assessments were taken into account in the overall risk of bias assessment for caries outcomes within a study
We decided to grade this domain as having 'low' risk of bias if the total proportion of missing outcome data was marginal (< 5%); or if the proportion of missing outcome data was < 25% regardless of the follow‐up time and groups (in parallel‐group studies) were balanced in numbers for missing data; or if missing data have been imputed using appropriate methods. If no information on reasons for drop‐out across intervention groups was provided, or if the proportion of missing data was documented as total proportion (5% to 25%), not by group in parallel‐group studies, our judgement was 'unclear' risk. Classifying missing data > 25% as having 'high' risk of bias in all study designs was a pragmatic approach to this domain to make the judgement uniform and transparent. If several teeth were sealed in a child’s mouth (a child is a cluster), missing outcome data had to be stated (or counted) at child level (not at tooth level).
Selective reporting (reporting bias) Were appropriate outcomes reported and were key outcomes missing? To be included in this review, caries outcomes had to be reported. However, studies could report the outcome in different ways, for example, incidence of dentinal carious lesion on treated occlusal surfaces of molars or premolars (yes or no); changes in mean figures of decayed, missing and filled surfaces (DMFS); or progression of caries lesion into enamel or dentine. In this review, selective outcome reporting was graded as 'low' risk of bias if the study's pre‐specified caries outcomes had been reported in the pre‐specified way
Other sources of bias This domain included information on comparability of intervention and control groups, and possible use of co‐interventions by group Comparability of groups 
 We decided to base our judgement of comparability of groups on baseline information given to groups available at follow‐up times because if only information provided at the start of the study is available, it is impossible to assess whether groups are balanced with each other after follow‐up time as well. The comparability of groups after follow‐up is especially problematic when small studies include children with several teeth and the drop‐out rate is high, even if drop‐outs are balanced in numbers and reasons between groups. If no information on the groups was available at follow‐up time, we decided that if the drop‐out rate (regardless of follow‐up time) was < 25% and drop‐outs were balanced in numbers and reasons by group, our judgement would be based on information given for groups at the start of the study
We decided to grade this domain as having ‘low’ risk of bias if groups were balanced in demographic characteristics (such as sex, age and social class) and in baseline caries risk level, or if possible imbalance of groups at baseline and/or after follow‐up had been taken adequately into account in the analyses. If baseline characteristics in parallel‐group studies were not given to groups available at follow‐up and the drop‐out rate was > 25%, we graded the study as having ‘unclear’ risk
Co‐interventions 
 We decided to grade this domain as having ‘low’ risk of bias if groups were balanced in number and quality of co‐interventions, or if no co‐interventions were included in the protocol, and as having ‘high’ risk of bias if groups received different numbers or quality of co‐interventions during the trial. If no information was provided on co‐interventions, our judgement was 'unclear' risk