Reply to: Eye-tracking as a tool to evaluate defibrillator handling

We thank Buehler et al. for their encouraging comments¹ on our study.² First, we would like to clarify that the different defibrillators were indeed presented to the subjects in randomised order.

Under the premise that usability per se can only be investigated in a defined user context, we focused on comparing the usability of the tested devices in terms of intuitive handling, choosing an artificial setup with the intention of eliminating rather than creating distractions that would cause additional stress for our inexperienced subjects. It should be noted that in another study from our group, usability was perceived comparably by experienced and inexperienced users.³ We did not focus on investigating stress levels in our study as we anticipated that this would be of limited value, as the reason for stress in our subjects would be their lack of user experience rather than the pressure caused by a clinically demanding situation. Indeed, realistic scenarios seem to be more important when it comes to optimising clinical procedures or arrangements. With such intention, it would be worthwhile to investigate the confounding factors of clinical reality, such as poor positioning of the device, poor visibility of the device screen and performing additional distracting tasks, such as preparing medication, summoning assistance, etc., to ensure a safe and rapid response to a patient emergency. This would require a trial with trained and experienced users who have little or no experience with the device.

Area of interest (re-)visits and fixation duration appear to be more meaningful when investigating realistic scenarios while focusing on complete procedures and large scale areas of interest (e.g. patient/monitor/ventilator), and particularly when investigating the impact of distraction on user performance. In the context of our study investigating individual tasks with higher resolution areas of interest, i.e. display items, and with explicitly low distraction, these metrics become difficult to interpret, as they depend strongly on time to first fixation and task completion time. However, as additional metrics we provide the numbers for visits and total interest duration in Table 1, excluding tasks that required multiple operating steps. The number of task-dependent visits was generally small. Higher visit counts and longer interest duration corresponded with long completion times and emphasize the user’s confusion.

Table 1.

Number of visits and total interest duration of the area of interest for each task.

Task	Number of visits				Total interest duration [s]
	Lifepak 20e	Defigard Touch 7	Corpuls 3	X-Series	Lifepak 20e	Defigard Touch 7	Corpuls 3	X-Series
Switch the device on.	1.1 ± 0.3	1.7 ± 0.9	1.5 ± 0.7	1.7 ± 0.8	2.5 ± 1.2	2.9 ± 1.4	2.3 ± 0.8	5.1 ± 2.9
Connect the therapy cable.	2.7 ± 1.4	2.8 ± 1.6	1.4 ± 0.5	2.1 ± 1.3	26.2 ± 15.1	31.0 ± 25.4	42.4 ± 27.7	70.7 ± 29.8
Connect the ECG cable.	1.8 ± 0.8	1.7 ± 0.9	1.6 ± 0.8	2.6 ± 2.4	6.2 ± 2.7	37.6 ± 31.3	65.7 ± 53.8	46.0 ± 29.5
Set defibrillation energy to 150 J.*	3.1 ± 1.9	n.a.	3.4 ± 2.3	2.3 ± 1.3	16.7 ± 12.8	n.a.	25.9 ± 16.2	7.2 ± 3.7
Deliver shock.	2.1 ± 1.0	2.3 ± 1.1	2.8 ± 1.6	2.4 ± 0.9	14.2 ± 10.6	20.1 ± 14.0	16.7 ± 12.7	13.0 ± 10.4
Set defibrillation energy to 200 J.*	2.0 ± 0.9	1.3 ± 0.6	1.2 ± 0.4	1.6 ± 0.8	4.5 ± 1.2	2.8 ± 1.0	4.7 ± 2.2	3.7 ± 1.1
Discharge defibrillation energy.	4.6 ± 3.6	3.3 ± 1.6	1.7 ± 1.0	2.6 ± 1.3	38.2 ± 23.5	23.5 ± 10.8	18.5 ± 14.1	18.4 ± 16.8
Activate synchroni-zed defibrillation.	1.6 ± 0.7	1.8 ± 1.3	3.6 ± 2.0	1.6 ± 0.7	4.5 ± 2.0	15.9 ± 24.3	18.6 ± 17.2	6.6 ± 4.6
Activate pacemaker mode.	3.5 ± 2.9	n.a.	3.0 ± 2.2	1.5 ± 0.8	25.0 ± 34.5	n.a.	18.2 ± 21.2	6.9 ± 9.7
Set pacemaker rate to 80 beats/min.§	1.6 ± 0.7	n.a.	2.3 ± 1.4	7.8 ± 4.5	4.4 ± 2.6	n.a.	8.3 ± 4.9	31.8 ± 17.7
Set pacemaker current to 40 mA.	1.8 ± 1.0	n.a.	2.0 ± 0.8	5.1 ± 3.0	6.6 ± 4.6	n.a.	5.7 ± 1.5	24.2 ± 25.2
Activate AED mode.	2.2 ± 1.2	3.7 ± 2.3	n.a.	n.a.	5.1 ± 3.1	39.6 ± 31.7	n.a.	n.a.
Print an ECG strip.	1.7 ± 0.8	n.a.	1.5 ± 0.6	2.0 ± 2.1	4.5 ± 3.6	n.a.	7.5 ± 4.2	5.0 ± 4.1
Show state of charge.	1.2 ± 0.5	1.2 ± 0.4	1.6 ± 0.9	1.3 ± 0.6	3.3 ± 4.2	3.7 ± 2.4	4.8 ± 2.9	3.1 ± 2.2
Switch the device off.	1.2 ± 0.4	n.a.	n.a.	1.5 ± 0.9	2.5 ± 1.3	n.a.	n.a.	7.4 ± 11.3

^*= Numbers are given exemplarily. Defibrillation energy to set ranged from 100 to 360 Joule, depending on the defibrillator; ^§ pacemaker rate to set ranged from 80 to 110 beats/min and current to set ranged from 40 to 100 mA, respectively. n.a. = not applicable, because of multi-step procedure or function not available.

In our study the eye tracking results corresponded well with the subjective usability scoring and with individual free comments. Of course, we agree that it would be interesting to find devices or conditions where the objective variables of gaze analysis and task completion contradict the subjective assessment and to identify reasons for such a contradiction. With this in mind, the sole focus on time may fall short. We would argue that usability is about more than completing a task as quickly as possible; it is about design, transparency, visual clarity, feedback and structure. Last but not least, patient safety is also a matter of confidence of the decision making user.

Funding

None.

CRediT authorship contribution statement

Stefan Schumann: Writing – original draft. Axel Schmutz: Writing – review & editing. Kim Feger: Writing – review & editing. Johannes Spaeth: Writing – review & editing.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.