Table 1.
List of testing instruments used in our usability assessment protocol. We adapted usability dimensions and definitions from the ISO [16], Quesenbery [50], and Davis [51]. All instruments are available in Multimedia Appendices 1-9.
| Instrument | Data collected | Measure type | Usability dimension | Usability dimension definition |
| Demographic screener (Multimedia Appendix 1) | Tester demographics, technology literacy, and attitudes and perceptions | Qualitative, binary (yes/no), and ordinal (Likert-type) | N/Aa | N/A |
| ATAb (Multimedia Appendix 2) | Simulation step completion rates | Ordinal (trichotomous) | Effective (learner) | How completely and accurately learners can complete a task to reach their goals [50] |
| ATA (Multimedia Appendices 2, 7, and 8) | Usability issues | Qualitative | Error tolerance | How well a design prevents errors or helps users recover from mistakes [50] |
| SEQc (Multimedia Appendix 3) | Task difficulty | Ordinal (Likert-type) | Easy to learn | The degree to which a person believes using a system would be free of effort [50] |
| Telemedicine competency rubric (Multimedia Appendix 9) | Clinical task completion rates, and competency scores | Task count and competency score | Effective (faculty) | How completely and accurately faculty could score the telemedicine rubric |
| Adapted SUSd (Multimedia Appendix 4) | User perceptions | Ordinal (Likert-type) | Ease of use | How intuitive the product is, ensuring learners can understand expectations with little to no additional instruction [51,52] |
| SSESe (Multimedia Appendix 4) | Learner perceptions | Ordinal (Likert-type) | Satisfaction | How relevant, valuable, and supportive the simulation was for acquiring skills and knowledge [53] |
| NASA-TLXf (Multimedia Appendix 5) | Cognitive load | Ordinal | Efficiency | The resources expended, including time, cognitive load, and effort in relation to the accuracy and completeness of goals [54] |
| Debrief interview guide (Multimedia Appendix 6) | Usability issues | Qualitative | Satisfaction | How pleasant, satisfying, and aesthetically pleasing the product is [16] |
| Debrief interview guide (Multimedia Appendix 6) | Usability issues | Qualitative | Fidelity | How completely and accurately the tasks align with real-world scenarios [11,55] |
aN/A: not applicable.
bATA: Agile Task Analysis.
cSEQ: Single Ease Question.
dSUS: System Usability Scale.
eSSES: Satisfaction With Simulation Experience Scale.
fNASA-TLX: NASA (National Aeronautics and Space Administration) Task Load Index.