Table 3.

Summary of overall error rates on the scenarios of the calibration set, the randomized set, and the set of all scenarios

	N (scenarios in set)	Minimum number of participants making an error on a scenario (%)	Maximum number of participants making an error on a scenario (%)	Mean number of participants making an error per scenario (%)	SD
Calibration set	9	2 (8.3%)	19 (79.2%)	7.44 (31%)	6.65
Randomized set	21	1 (4.2%)	14 (58.3%)	7.14 (29.8%)	3.77
All scenarios	30	1 (4.2%)	19 (79.2%)	7.23 (30.1%)	4.70

The range, mean and SD of the number of candidates who made an error in the scenarios of the three sets is shown. For example, the most difficult scenario in the randomization set had 14 of the 24 participants (58.3%) make an error. Percentages (error rates) were calculated by dividing the total number of errors made on scenarios assigned to the alert type by 168d that is, the total number of times scenarios assigned to each alert type were performed.