Table 3.
Summary table of quantitative results across instruments.
| Usability dimension, instrument, and measured item | Frequency, n/N (%) | Median (IQR) | Maximum possible value | |||||
| Effectiveness | ||||||||
|
|
Agile Task Analysis | |||||||
|
|
|
Total number steps completed successfully | 165/180 (92) | —a | — | |||
|
|
|
Total number steps partially completed | 11/180 (6) | — | — | |||
|
|
|
Total number of failed steps | 4/180 (2) | — | — | |||
| Error tolerance | ||||||||
|
|
Agile Task Analysis | |||||||
|
|
|
Number of usability issues | 52 | — | — | |||
|
|
|
Number of critical usability issues | 15/52 (29) | — | — | |||
| Easy to learn | ||||||||
|
|
Single Ease Question | |||||||
|
|
|
Reviewing materials | — | 6.00 (6.00-6.25) | 7.00 | |||
|
|
|
Starting encounter | — | 6.00 (4.75-7.00) | 7.00 | |||
|
|
|
Standardized telemedicine tasks | — | 5.50 (5.00-6.25) | 7.00 | |||
|
|
|
History collection | — | 6.50 (5.00-7.00) | 7.00 | |||
|
|
|
Virtual physical examination | — | 4.50 (2.75-6.00) | 7.00 | |||
|
|
|
Diagnosis and plan | — | 6.00 (5.00-6.50) | 7.00 | |||
|
|
|
Overall encounter | — | 5.00 (5.00-6.00) | 7.00 | |||
| Ease of use | ||||||||
|
|
Adapted System Usability Scale | |||||||
|
|
|
Total score round 1 | — | 73.80 (72.63-80.50) | 100.00 | |||
|
|
|
Total score round 2 | — | 77.50 (73.13-80.00) | 100.00 | |||
|
|
|
Total score overall | — | 75.00 (71.88-81.25) | 100.00 | |||
| Satisfaction | ||||||||
|
|
Satisfaction With Simulation Experience Scale | |||||||
|
|
|
Total score round 1 | — | 44.50 (39.5-45.0) | 50.00 | |||
|
|
|
Total score round 2 | — | 42.00 (41.00-46.75) | 50.00 | |||
|
|
|
Total score overall | — | 43.50 (40.5-45.25) | 50.00 | |||
| Workload | ||||||||
|
|
NASAb Task Load Index | |||||||
|
|
|
Physical demand round 1 | — | 7.50 (5.00-17.50) | 100.00 | |||
|
|
|
Temporal demand round 1 | — | 12.50 (6.25-22.50) | 100.00 | |||
|
|
|
Frustration level round 1 | — | 22.50 (10.00-42.50) | 100.00 | |||
|
|
|
Perceived performance round 1 | — | 32.50 (26.25-35.00) | 100.00 | |||
|
|
|
Mental demand round 1 | — | 37.50 (21.25-57.50) | 100.00 | |||
|
|
|
Effort (mental and physical) round 1 | — | 42.50 (27.50-57.50) | 100.00 | |||
|
|
|
Physical demand round 2 | — | 15.00 (11.30-18.80) | 100.00 | |||
|
|
|
Temporal demand round 2 | — | 20.00 (11.30-43.80) | 100.00 | |||
|
|
|
Frustration level round 2 | — | 25.00 (10.00-43.80) | 100.00 | |||
|
|
|
Perceived performance round 2 | — | 32.50 (21.30-43.80) | 100.00 | |||
|
|
|
Mental demand round 2 | — | 60.00 (56.30-63.80) | 100.00 | |||
|
|
|
Effort (mental and physical) round 2 | — | 55.00 (46.30-63.80) | 100.00 | |||
|
|
|
Mental demand overall | — | 57.50 (23.75-61,25) | 100.00 | |||
|
|
|
Effort (mental and physical) overall | — | 50.00 (32.50-60.00) | 100.00 | |||
| Effectiveness | ||||||||
|
|
Competency rubricc | |||||||
|
|
|
Incorporates telehealth | 12/12 (100) | — | — | |||
|
|
|
Attends to environment | 11/12 (92) | — | — | |||
|
|
|
Identifies and uses equipment | 12/12 (100) | — | — | |||
|
|
|
Develops rapport | 12/12 (100) | — | — | |||
|
|
|
Obtains and uses history | 12/12 (100) | — | — | |||
|
|
|
Troubleshoots technology | 12/12 (100) | — | — | |||
|
|
|
Prepares for escalating care | 12/12 (100) | — | — | |||
|
|
|
Adheres to professional requirements | 10/12 (83) | — | — | |||
aNot applicable.
bNASA: National Aeronautics and Space Administration.
cIndicates the number of learners successfully assessed.