Skip to main content
. 2019 Dec 17;35(5):1530–1536. doi: 10.1007/s11606-019-05595-2

Table 2.

Psychometric Properties of Study Tool Compared With Other Available EBM Competence Tests

Test property Measure to be used Acceptable results  RESET tool ACE tool Berlin (two different sets) Fresno
Content validity Expert opinion Test covers EBM topics All items were scored at least “important” by > 50% of experts Acceptable Not reported “revisions based on experts’ suggestions”
Inter-rater reliability Inter-rater correlation Expected to be high (> 0.6) 0.9 N/A N/A 0.76–0.98
Internal reliability

Cronbach’s alpha [23]

Item-total correlation

0.6–0.7 = questionable

0.7–0.8 = acceptable

0.8–0.9 = good

> 0.9 = excellent

0.6 0.69 0.75, 0.82 0.88
> 0.30 (per Fresno) 0.37 0.14 to 0.20, apart from three items (0.03, 0.04, and 0.06) 0.47 to 0.75
Item difficulty Percentage of candidates who correctly answer each item Wide range of results allows test to be used in expert and novice groups 7–86% (trainees), 30–100% (experts) 36–84% Not reported 24 to 73%
Item discrimination Item discrimination index (ranges from − 1.0 to 1.0) All items should be positively indexed, > = 0.2 is considered acceptable 0.17–0.65 0.37–0.84 Not reported 0.41–0.86
Construct validity Mean scores of experts and novices compared by t test % passing for expert and novice groups compared by χ2 test 48-point test: novice 29.4, expert 35.4 (p < 0.001) 15-point test: novice 8.6, intermediate 9.5, advanced 10.4 (p < 0.001) ANOVA: 4.2 control, 6.3 course participants, 11.9 expert (p < 0.001)_ 212-point test: novice 95.6, expert 147.5 (p < 0.001)

N/A, not assessed