Table 2.
Risk of bias.
| Selection bias |
Performance bias |
Detection bias |
Attrition bias |
Reporting bias |
|||
|---|---|---|---|---|---|---|---|
| Reference | Random sequence generation | Allocation concealment | Blinding of participants and personnel | Blinding of outcome assessment | Incomplete outcome data | Selective reporting | Other bias |
| Solomon et al. (2004) | Low – ‘ … were randomized into to … ’ | Unclear – Insufficient information | Unclear – Insufficient information | Unclear – Insufficient information | Low – All participants assessed | Unclear – Insufficient information | Unclear – Insufficient information |
| Phadtare et al. (2009) | Low – ‘Random numbers were generated with … based on program of origin’ | Low – ‘Group assignments were placed in sealed envelopes and revealed after participants had signed informed consent’ | Low – ‘To ensure blinding, assignments were disclosed to analysts only after the results had been delivered’ | Low – ‘To ensure unbiased findings, statistical analysis was blinded, with analysts being unaware of which group participants were assigned to until the study analysis was complete’ | Low – All participants assessed | Unclear – Insufficient information | Unclear – Insufficient information |
| Raupach et al. (2009) | Low – ‘Students who had signed up together were randomized as a group to either the control or the intervention setting’ | Unclear – Insufficient information | Unclear – Insufficient information | Low – ‘On the last day, all students took a summative examination made up of 68 multiple choice questions mainly assessing factual knowledge’ | Low – ‘5 out of 148 participants dropped out’ | Unclear – Insufficient information | Unclear – Insufficient information |
| Bhatti et al. (2011) | Low – ‘The students were randomly assigned to either group A or group B using QUICKCALCS online software’ | Unclear – Insufficient information | Low – ‘Students were made aware that they had taken part in a study to compare educational methods, but they were not told until after the information had been delivered’ | Low – ‘The papers were marked by an individual blinded to the teaching method given, using a pre-agreed marking schedule’ | High – ‘121 out of 146 participants completed the study’ | Unclear – Insufficient information | Unclear – Insufficient information |
| Heiman et al. (2012) | Low – ‘Upon matriculation, students were assigned randomly to one of four colleges’ | Unclear – Insufficient information | Low – ‘All second-year students were randomly assigned a case from the bank of six assessment cases’ | Low – ‘Raters were paid per case completed. They were blinded to the training status of students but not to the timing of the evaluation’ | Low – all participants assessed | Unclear – Insufficient information | Unclear – Insufficient information |
| Serena et al. (2012) | High – ‘Participants are enrolled in different semester year’ | Unclear – Insufficient information | Low – ‘Participants in the two groups were in different academic years’ | Low – ‘An independent scorer applied the rubric to all pre- and post-tests’ | Low – All participants assessed | Unclear – Insufficient information | Unclear – Insufficient information |
| Subramanian et al. (2012) | Low – ‘Medical students were consented and randomly assigned to two groups’ | Unclear – Insufficient information | Unclear – Insufficient information | Low – ‘A bank of multiple-choice questions was created. The questions were randomly selected for a preintervention test and postintervention test’ | Low – All participants assessed | Unclear – Insufficient information | Unclear – Insufficient information |
| Yeung et al. (2012) | Low – ‘Randomization and allocation concealment were achieved through an automatic randomization process’ | Low – ‘Randomization and allocation concealment were achieved through an automatic randomization process’ | Low – ‘Access to each module was restricted to the individuals randomized to each respective study group’ | Low – ‘The primary outcome measure was a multiple-choice quiz’ | Low – All participants assessed | Unclear – Insufficient information | Unclear – Insufficient information |
| Jordan et al. (2013) | High – This is a single group cross-sessional study | Unclear – Insufficient information | Unclear – Insufficient information | Low – “A multiple choice post-test was used to assess their knowledge gain” |
Low – ‘4 out of 48 participants dropped out’ | Unclear – Insufficient information | Unclear – Insufficient information |
| Sendra et al. (2013) | High – “The project was accepted by 89 students out of 191 (46.6%), who integrated the group P, attending only virtual lectures. The remaining 102 students (53.4%) did not participate in the project, being the control group NP. “ | Unclear – Insufficient information | Unclear – Insufficient information | Low – ‘The final oral examination and an anonymous evaluation on image interpretation, where the name of the students remained unknown’ | High – ‘74 out of 89 in group P and 56 out of 102 in group NP’ | Unclear – Insufficient information | Unclear – Insufficient information |
| Porter et al. (2014) | Low – ‘All students who enrolled in the course through the preregistration process were randomly assigned to either the classroom or online section using block randomization’ | Unclear – Insufficient information | Unclear – Insufficient information | Low – ‘The lecturing faculty member was blinded to the participation status of the students.’ | High – ‘140 students participated in the study, which is a participation rate of 83.3%.’ | Unclear – Insufficient information | Unclear – Insufficient information |
| Assadi et al. (2015) | High – ‘Divided into two groups by odd and even month’ | Unclear – Insufficient information | Unclear – Insufficient information | Low – ‘Evaluation of participants was assessed by an EM attending (M.M.) who was blinded to the training methods.’ | Low – ‘9 out of 90 interns were not available.’ | Unclear – Insufficient information | Unclear – Insufficient information |
| Pusponegoro et al. (2015) | Low – ‘Subjects were randomized into two groups using a computer-generated random number table’ | Unclear – Insufficient information | Unclear – Insufficient information | Low – ‘Complete a 20-item multiple-choice test’ | Low – ‘4 out of 75 participants dropped out’ | Unclear – Insufficient information | Unclear – Insufficient information |
| Arne et al. (2016) | Low – ‘The allocation to the various branches of the study was carried out by randomization’ | Low – ‘All students were anonymously assigned in advance, with a number (“token”) that was used for identification purposes throughout the study’ | Unclear – Insufficient information | Low – ‘These tests were based on a 24-item multiple-choice questionnaire. Each question included five possible answers, of which only one was correct’ | Low – ‘21 out of 244 participants dropped out’ | Unclear – Insufficient information | Unclear – Insufficient information |
| Farahmand et al. (2016) | High – ‘This was a blinded quasi-experimental study’ | Low – ‘To conceal the allocation, the first group, who started their emergency medicine rotation in September to October 2013, entered the control group and the nature of the future intervention was not revealed to them. We did not inform them about the existence of the educational DVD’ | Unclear – Insufficient information | Low – ‘Both groups and raters who scored the students during the OSCE were blinded to the content of the educational package and the intervention of each group’ | Low – All participants were assessed | Unclear – Insufficient information | Unclear – Insufficient information |
| Shenoy et al. (2016) | Low – ‘Students were randomly divided into two groups’ | Unclear – Insufficient information | Unclear – Insufficient information | Unclear – Insufficient information | Low – All participants were assessed | Unclear – Insufficient information | Unclear – Insufficient information |