Table 3.
Number and Concordance of Usability Problems Per Criterion According to End-Users and Usability Experts. The Diagonal (Boldface) Indicates Agreement between Raters. This Table Does Not Include Emergent Issues and Categories Requiring Clinical Expertise to Identify (see Appendix E for This Data)
| End-users | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Guidance | Workload | Compatibility | Signif. of codes | Error manag. | Consistency | Adaptability | Explicit control | No criterion | Sum | ||
| Usability experts | Guidance | 45 | 7 | 4 | 3 | 2 | 1 | 3 | 1 | 16 | 82 |
| Workload | 8 | 32 | 5 | 1 | 2 | 0 | 4 | 0 | 12 | 64 | |
| Compatibility | 1 | 4 | 25 | 1 | 0 | 1 | 5 | 0 | 6 | 43 | |
| Significance of codes | 0 | 1 | 1 | 23 | 0 | 0 | 0 | 0 | 2 | 27 | |
| Error manag. | 0 | 1 | 1 | 0 | 15 | 0 | 0 | 0 | 3 | 20 | |
| Consistency | 0 | 0 | 0 | 0 | 0 | 11 | 0 | 0 | 0 | 11 | |
| Adaptability | 1 | 0 | 0 | 1 | 0 | 0 | 3 | 0 | 1 | 6 | |
| Explicit control | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 3 | 1 | 5 | |
| Sum | 55 | 45 | 37 | 29 | 19 | 13 | 15 | 4 | 41 | 258 | |