Table 2.
Psychometric Properties of Study Tool Compared With Other Available EBM Competence Tests
Test property | Measure to be used | Acceptable results | RESET tool | ACE tool | Berlin (two different sets) | Fresno |
---|---|---|---|---|---|---|
Content validity | Expert opinion | Test covers EBM topics | All items were scored at least “important” by > 50% of experts | Acceptable | Not reported | “revisions based on experts’ suggestions” |
Inter-rater reliability | Inter-rater correlation | Expected to be high (> 0.6) | 0.9 | N/A | N/A | 0.76–0.98 |
Internal reliability |
Cronbach’s alpha [23] Item-total correlation |
0.6–0.7 = questionable 0.7–0.8 = acceptable 0.8–0.9 = good > 0.9 = excellent |
0.6 | 0.69 | 0.75, 0.82 | 0.88 |
> 0.30 (per Fresno) | 0.37 | 0.14 to 0.20, apart from three items (0.03, 0.04, and 0.06) | 0.47 to 0.75 | |||
Item difficulty | Percentage of candidates who correctly answer each item | Wide range of results allows test to be used in expert and novice groups | 7–86% (trainees), 30–100% (experts) | 36–84% | Not reported | 24 to 73% |
Item discrimination | Item discrimination index (ranges from − 1.0 to 1.0) | All items should be positively indexed, > = 0.2 is considered acceptable | 0.17–0.65 | 0.37–0.84 | Not reported | 0.41–0.86 |
Construct validity | Mean scores of experts and novices compared by t test % passing for expert and novice groups compared by χ2 test | 48-point test: novice 29.4, expert 35.4 (p < 0.001) | 15-point test: novice 8.6, intermediate 9.5, advanced 10.4 (p < 0.001) | ANOVA: 4.2 control, 6.3 course participants, 11.9 expert (p < 0.001)_ | 212-point test: novice 95.6, expert 147.5 (p < 0.001) |
N/A, not assessed