Evaluating Eyegaze Targeting to Improve Mouse Pointing for Radiology Tasks

Abstract

In current radiologists’ workstations, a scroll mouse is typically used as the primary input device for navigating image slices and conducting operations on an image. Radiological analysis and diagnosis rely on careful observation and annotation of medical images. During analysis of 3D MRI and CT volumes, thousands of mouse clicks are performed everyday, which can cause wrist fatigue. This paper presents a dynamic control-to-display (C-D) gain mouse movement method, controlled by an eyegaze tracker as the target predictor. By adjusting the C-D gain according to the distance to the target, the mouse click targeting time is reduced. Our theoretical and experimental studies show that the mouse movement time to a known target can be reduced by up to 15%. We also present an experiment with 12 participants to evaluate the role of eyegaze targeting in the realistic situation of unknown target positions. These results indicate that using eyegaze to predict the target position, the dynamic C-D gain method can improve pointing performance by 8% and reduce the error rate over traditional mouse movement.

Introduction

In order for image viewing software and user interaction hardware to be valuable, it must be easy for the user to interact with the displayed images. The typical display mode seen in radiology shows an axial and coronal view spanning multiple monitors. Precise targeting is frequently needed in radiology tasks. For example, it is often necessary in MRI and CT scans to cross-reference small lesions (<10 mm) between anatomic planes (axial, saggital, and coronal). This is done with a mouse click on the lesion in one plane, which, by nature of its small size, must be precise. Furthermore, for MRI, it is often necessary to cross-reference small lesions between different pulse sequences to better understand the tissue characteristics. In the case where multiple images are shown simultaneously, it becomes necessary to routinely move from one monitor to another for cross-referencing.¹ It is easy to become fatigued and stressed after hundreds of such procedures, particularly when the target is small, and the distance to move is large, such as across two display monitors.²

Fitts’ Law³ states that the movement time (MT) in pointing to a target depends on the distance D of the target and the width W of the target:

1

where a and b are constants characterizing the system and user. The quantity is named the index of difficulty (ID). Hence, Fitts’ law links the movement time to acquire a target to the task’s ID.

The goal of this research is to accelerate target selection by reducing the index of difficulty. In an interactive system, movement of the mouse for a specified distance causes the cursor to move a corresponding distance on the screen. The gain of the resulting cursor distance to the control (mouse) distance is called the control-to-display gain (C-D gain)¹⁴. Setting this gain high implies that small mouse movements cause larger cursor movements. The user can cross long screen distances with less effort, but precise pointing can be difficult. Setting the C-D gain low has the inverse effect: Long movements require more effort, but precise pointing is easier. Thus, to reduce the index of difficulty and improve performance for pointing tasks, Blanch et al.⁴ suggested dynamically varying the C-D gain according to the distance to the target.

The difficulty in implementing this is in determining where the target is. Several methods for predicting target location have been proposed,^5–7 outlined in the following section. Many are based on information gathered from the mouse movement, predicting the target from a predetermined target list. For radiology tasks, the target is unpredictable, since suspicious features can lie anywhere in the image. We proposed that predicting the target position using eyegaze would work well because previous work showed that people look at their pointing target before moving the pointing device.^8,9 Given a predicted target, we could dynamically adjust the C-D gain according to the distance to the target, reducing the gain as the cursor nears the target. Thus, we should get a bigger effective target, allowing the user to acquire the target using a coarser, faster arm motion, improving pointing performance.

Dynamic Control-to-Display Gain

Equation 2 shows the change of ID when the cursor is slowed by a constant R:

2

Previous work has shown that a dynamic C-D gain can improve the performance of the mouse pointing tasks in 1D space.^4,10 To employ this method in 2D space, some issues must be addressed. The first issue is path variation. There is only one path to the target in 1D space (Fig. 1a). In 2D space, the cursor no longer strictly follows a straight line to the target. Thus, only slowing the cursor down over the target is not effective. We need to expand the slowing down width (S) to more effectively capture the cursor (Fig. 1b).

Fig 1.

Dynamic C-D gain (a) in 1D space and b in 2D space.

The second issue is how to define the target width. In 1D space, the target width is fixed since there is only one entry angle. In 2D space, width potentially varies with the angle of entry. To simplify the analysis, we used circular targets for which the target width and region of increased C-D gain remain fixed for every entry angle.

The new equation for calculating the ID becomes

3

Target Prediction

The dynamic C-D gain relies on the prediction of the target. There are a number of works addressing this problem. Some of them rely on the graphical user interface (GUI) context in which targets are pre-set (known), such as icons on the Windows desktop.^5,6 For those methods independent from the GUI context, the prediction is not quite accurate.⁷ For example, the Delphian Desktop predicts the target location by the maximum velocity during the mouse movement. It enables the user to warp sparse areas. But still, it requires the user to correct the cursor position after the warping and sometimes has directional errors for the prediction.⁷

We propose that we can use an eyegaze tracker to predict the target location. Previous work has shown that people gaze at intended targets before pointing to them.^8,9 Although the eyegaze tracker is only accurate within approximately 1 cm viewed at a distance of 50 cm, we expect that pointing performance can still be improved using the dynamic C-D gain method with an expanded slowing down area.

Method

We ran two experiments to evaluate the performance improvement of the dynamic C-D gain technique. The two experiments were separated by a period of several months and featured different participants.

Before the first experiment, we conducted a pilot to determine the best parameter combination for the slowing down width S and slowing down gain R (Table 1). Previous work has shown that the user perceives no change in mouse speed if cursor slows down over the target only.¹¹ However, there might be a different story if the slowing down area is expanded beyond the visual boundary of the target. As stated earlier, precise targeting is frequently needed in radiology tasks. For example, a radiologist may have to cross-reference between anatomic planes the centre of a small lesion in the foot, such as a Morton’s neuroma in the first web space of the foot. These lesions are typically only 5–10 mm in size, and the centre must be accurately identified to allow accurate surgical excision. Therefore, we only used ten pixels for the target width, which is considered small.

Table 1.

Experiment Parameters

Parameter	Value
D (pixel)	200, 400, 800
W (pixel)	10
S (pixel)	40, 60, 80
R	2, 3.3, 5 10

Parameter includes the variables we took for the experiment. D stands for the distance, W is the width, S is the slowing down area size and R is the slowing down ratio. Value refers to the value for each parameter

In the pilot, we found that the best value for R was 2, and the best value for S was 80 pixels.¹² In addition, R = 2 also made the change in C-D gain less perceptible to participants, and with S = 80, the region of decreased C-D gain captured the actual target despite the imprecision in eyegaze targeting. Both of the experiments used these settings for S and R.

Experimental Design

The two experiments used similar designs, only differing in how the dynamic C-D gain was applied. In experiment 1, we adjusted the C-D gain as the cursor approached the known target. This experiment estimated the optimal improvement possible with a dynamic C-D gain in a radiology context.

In experiment 2, a Tobii 1750 eye tracker¹³ was used to record subjects’ saccades and gaze points. This experiment estimated performance of the dynamic gain in the more realistic condition, where the pointing target was unknown to the system. The location of each sampled gaze point was used to adjust the C-D gain. Note that, unusually, we did not use fixations of durations >150 ms, as taking the average of several gaze points would not leave enough time to slow the mouse down before it reached the target.

Setup Subjects sat at a desk with a standard personal computer and a mouse. A Tobii 1750 17″ monitor with integrated eye tracker displayed sets of four MR images (Fig. 2). In experiment 1, the Tobii only functioned as a display, whereas in experiment 2, the eye-tracking features were also enabled.

Fig 2.

The experiment platform layout.

The study was run under Microsoft Windows. Windows scales the C-D gain on an abstract scale from 1 (slowest) to 20 (fastest). We set the gain to the Windows default value, 10.

Task Subjects were required to use the mouse to point to targets on the MR images. Four sets of images were used. To begin each image set, subjects first right-clicked anywhere on the screen to activate a popup menu and selected “linking” from a menu. The software displayed a red target dot. After clicking on each target, a new dot appeared; there were eight targets for each set of four images. When all eight dots had been clicked, subjects again right-clicked and selected “linking” to begin the next set. Image sets were blocked, with each set recurring four times per block. The total number of trials was eight dots × four sets × four blocks = 128 trials per subject. All mouse clicks were logged and time-stamped by the experimental software.

The order of target dots must be carefully arranged. If dots appear on the screen in a fixed order, subjects might learn to predict the location of the next target, confounding the results. Appropriate dot order can also provide a balanced variety of movement distances and angles. In our experiment, there were four image sets in each block. Target dots appeared in one image at a time in pseudo-random positions at different distances and angles. Dot sequences for each set are shown in Figure 3, and distances for each direction are shown in Table 2. Table 3 shows how many targets were at each distance D. The expected ID for each distance is shown in Table 4. We applied this sequence to all four blocks.

Fig 3.

The order in which targets appear on the screen. The big circle indicates the starting point; however, subjects may click other place to activate “linking”.

Table 2.

The Distances (Pixels) Between Targets Labeled in Figure 3

	a	b	c	d
1	450	550	600	1,025
2	400	500	650	1,025
3	450	750	884	700
4	600	500	650	1,025
5	600	750	600	700
6	450	750	650	750
7	600	500	884	750
8	450	750	650	700

Letters a–d are the corresponding subpanels in Figure 3. Numbers 1–8 are the corresponding labeled lines in Figure 3

Table 3.

Number of Tasks N for Each Distance D

D	N
400	48
450	144
500	144
600	192
650	192
700	144
750	288
884	96
1,025	96

D distance, N number of tasks

Table 4.

Original ID vs New ID′ (R = 2, S = 80)

D (pixel)	W (pixel)	Original ID (bit)	New ID (bit)	Improvement (%)
400	10	6.32	5.46	13.6
450	10	6.49	5.61	13.6
500	10	6.64	5.75	13.4
600	10	6.91	6.00	13.2
650	10	7.02	6.11	13.0
700	10	7.13	6.21	12.9
750	10	7.23	6.30	12.9
884	10	7.47	6.53	12.6
1,025	10	7.68	6.73	12.4

D distance, W width, Orginal ID index of difficulty calculated using constant C-D gain, New ID′ index of difficulty calculated using dynamic C-D gain, Improvement improvement of the index of difficulty

Subjects We recruited 12 subjects for each experiment. Subjects were all students at Simon Fraser University with an average age of 24.8 for experiment 1 and 25.6 for experiment 2. None of them participated in the pilot study, and no one participated in both experiments.

Procedure Both experiments were conducted following a strict written procedure and were managed by the experiment administrator. Subjects were first briefed on the procedure and gave their written consent. They were then introduced to the task and completed a general questionnaire to gain information regarding their age, education, and experience using a mouse.

For experiment 2 (only), the gaze tracker was calibrated to the individual subject, even for the condition where the eyegaze was not used to alter the C-D gain. Subjects then began to perform trials. In order to minimize the impact of fatigue, subjects were required to have a short break every 5 min. Order of each condition was counterbalanced, with half of the subjects starting with the constant C-D gain before the dynamic C-D and vice-versa for the remaining subjects. After each condition, subjects were requested to complete a questionnaire about the preceding condition. After completion of both conditions, each subject was given a questionnaire for comparing perceived differences between the conditions. Each session was about 30 min.

Results and Discussion

The results from two experiments were analyzed separately. The analysis for both experiments consisted of controlling for learning, times, accuracy, and user preference.

Experiment 1—Specified Target

The result of experiment 1 provides the optimal improvement. Figure 4 and Table 5 show the improvement and the p value (t test) over different distances of tasks. This figure shows that there was an improvement of about 14% for the movement time with statistically significant difference (t = 3.87, p < 0.01, df = 11). In addition, in this stage, subjects all thought they performed better with the dynamic C-D gain than with the constant C-D gain. In terms of the perceptibility, eight subjects claimed that they did not notice any difference. All four subjects who noticed a difference claimed they prefer the dynamic C-D gain. One interesting thing we found was that two subjects claimed they did not notice differences but still prefer the condition with dynamic C-D gain. Details are given in Tan et al.¹².

Fig 4.

Experiment 1: Mean performance comparison for both conditions at each distance D.

Table 5.

Experiment 1: The Mean, Standard Deviation and Number of Tasks Under Both Conditions for Each Distance D, and the p Value from the t Test

D	Constant C-D	Dynamic C-D	Improvement (%)	N	p value
400	1.73 ± 0.44	1.47 ± 0.21	15.1	36	0.04
450	1.81 ± 0.50	1.46 ± 0.35	19.2	108	0.02
500	1.82 ± 0.54	1.53 ± 0.29	16.1	108	0.02
600	1.90 ± 0.48	1.58 ± 0.29	16.7	144	0.01
650	1.81 ± 0.61	1.53 ± 0.31	15.0	144	0.01
700	1.96 ± 0.52	1.73 ± 0.40	11.7	108	0.01
750	1.92 ± 0.61	1.64 ± 0.34	14.5	216	0.01
884	1.88 ± 0.40	1.65 ± 0.35	12.0	72	0.01
1,025	1.97 ± 0.50	1.64 ± 0.28	16.7	72	0.00

D distance, Constant C-D mean performance under constant C-D gain, Dynamic C-D mean performance under dynamic C-D gain, Improvement percentage improvement of the dynamic C-D gain method, N number of tasks, p value p value from the t test

Experiment 2—Eyegaze Prediction of Likely Target

The result of experiment 2 indicates the practical performance of the dynamic C-D gain method with an eyegaze tracker. Due to the imprecise prediction, we expected the result to be less promising than the first stage, which is the optimal placement of the slowing down area.

Controlling for Learning

Since we did not provide any practice for subjects before starting the experiment, the learning effects may influence the result. Then, we should determine how many blocks that the subjects required for learning. Because we counterbalanced the order of the two conditions during the experiment, six subjects (subject 1, 3, 5, 7, 9, and 11) firstly did the experiment under the constant C-D gain and vice-versa for the other.

Figure 5a shows the completion time by block for subjects who began with a constant C-D gain. Figure 5b shows the completion time by block for subjects who began with a dynamic C-D gain. Figure 5a shows a strong reduction in time from blocks 1 to 2 for four of the six subjects. The time for the remaining blocks is generally less than for the first block. In Figure 5b, however, all subjects have approximately the same completion time for each block. We have no explanation for the apparent absence of learning in this group. To reduce the confounding effect of learning, block 1 was excluded from the analysis for all subjects.

Fig 5.

Experiment 2: The completion time of each block for each subject. a Subjects who performed under the constant C-D gain first. b Subjects who performed under the dynamic C-D gain first.

Times

Figure 6 shows the average movement time with constant C-D gain and dynamic C-D gain. For targets with D > 600 pixels, the average movement time with dynamic C-D gain is always shorter than that with constant C-D gain. However, when we look into the p value through the paired t test as shown in Table 6, it does not indicate any statistically significant difference. The possible reason is that, due to the inaccurate prediction from the eyegaze tracker, some tasks with dynamic C-D gain were misled to an incorrect location. The inaccurate prediction might have been caused by inaccurate calibration, a substantial change of head position, or other reasons. Then, the cursor might have slowed down within an improper area or it did not slow down at all. In order to get a more analytic result, we categorized the data by the precision of the target prediction.

Fig 6.

Experiment 2: Mean performance comparison for both conditions at each distance D.

Table 6.

Experiment 2: The Mean, Standard Deviation, and the Number of Tasks Under Both Conditions for each Distance D and the p Value from the t Test

D	Constant C-D	Dynamic C-D	Improvement (%)	N	p value
400	1.66 ± 0.24	1.74 ± 0.18	−4.8	36	0.23
450	1.66 ± 0.27	1.67 ± 0.25	−0.2	108	0.48
500	1.72 ± 0.16	1.70 ± 0.16	1.0	108	0.39
600	1.74 ± 0.16	1.74 ± 0.20	0.2	144	0.48
650	1.80 ± 0.22	1.74 ± 0.16	3.5	144	0.14
700	1.87 ± 0.18	1.77 ± 0.19	5.4	108	0.04
750	1.81 ± 0.20	1.77 ± 0.19	2.6	216	0.16
884	2.01 ± 0.21	1.86 ± 0.16	7.8	72	0.04
1,025	1.97 ± 0.20	1.88 ± 0.25	4.2	72	0.21

D distance, Constant C-D mean performance under constant C-D gain, Dynamic C-D mean performance under dynamic C-D gain, Improvement percentage improvement of the dynamic C-D gain method, N number of tasks, p value p value from the t test

For each trail, we traced back the previous three gaze points and use the average as the target fixation point. Each target fixation point was analyzed to see how far it was from the target. Trials were split into three categories. Trials with target prediction errors of less than 20 pixels were categorized as “precise” (43% of the trials). For these trials, no matter where the area of reduced cursor speed was placed relative to the target, the cursor would nonetheless be slowed for some distance. Trials with prediction errors between 20 and 40 pixels were categorized as “medium” (34% of the trials). For such predictions, the cursor might not be slowed down in the worst case. Note that the worst case only occurs when the actual target lies between the predicted target location and the starting point. The rest of the trials were categorized as “imprecise” (23% of the trials). For prediction errors over 40 pixels, the actual target location will always be outside of the area of the reduced cursor speed. All subjects had some trials in every category.

The mean movement time for the “precise” trials is shown in Table 7. The mean time with dynamic C-D gain for trials with 650 pixels ≤D < 1,025 pixels is significantly lower than for constant C-D gain. Overall, there is an average of 7.8% improvement for those trials. Furthermore, more statistically significant difference for the improvement is found in this group. We cannot explain why dynamic gain had no effect for trials with D = 1,025. Perhaps, this distance was too large to cross with a single-hand motion under the active mouse speed setting, requiring the user to physically re-position the mouse.

Table 7.

Experiment 2 Group Precise: The Mean, Standard Deviation, and the Number of Subjects Involved Under Both Conditions for each Distance D and the p Value from the t Test

D	Constant C-D	Dynamic C-D	Improvement (%)	Degree of freedom	p value
400	1.68 ± 0.17	1.75 ± 0.25	−3.9	6	0.29
450	1.74 ± 0.25	1.68 ± 0.32	3.4	11	0.26
500	1.75 ± 0.26	1.63 ± 0.17	6.9	11	0.06
600	1.78 ± 0.21	1.72 ± 0.18	3.2	11	0.22
650	1.82 ± 0.34	1.65 ± 0.23	9.2	11	0.04
700	1.89 ± 0.21	1.78 ± 0.12	5.7	11	0.05
750	1.82 ± 0.25	1.74 ± 0.24	3.9	11	0.06
884	2.02 ± 0.31	1.77 ± 0.18	12.4	9	0.02
1,025	1.95 ± 0.23	1.93 ± 0.25	0.6	10	0.45

D distance, Constant C-D mean performance under constant C-D gain, Dynamic C-D mean performance under dynamic C-D gain, Improvement percentage improvement of the dynamic C-D gain method, Degree of freedom degree of freedom of the t test, p value p value from the t test

The mean movement time for the “medium” trials is shown in Table 8. The performance under dynamic gain for all trials is faster than for constant gain. However, the improvements are small and none is close to statistically significant. The lack of strong effect for these trials is expected because dynamic gain is unreliable in this case, where the region of reduced cursor speed can be any direction from the actual target. If the slower region is on the line connecting the starting point and the target, the cursor slows as it approaches the target. If instead, the predicted location is offset from the line, the dynamic gain has little to no effect on the cursor.

Table 8.

Experiment 2 Group Medium: The Mean, Standard Deviation, and the Number of Subjects Involved Under Both Conditions for each Distance D and the p Value from the t Test

D	Constant C-D	Dynamic C-D	Improvement (%)	Degree of freedom	p value
400	1.74 ± 0.39	1.64 ± 0.19	6.0	6	0.28
450	1.68 ± 0.24	1.64 ± 0.20	2.4	8	0.29
500	1.72 ± 0.29	1.67 ± 0.24	3.2	10	0.31
600	1.74 ± 0.28	1.72 ± 0.20	1.4	11	0.41
650	1.71 ± 0.21	1.70 ± 0.25	0.2	11	0.48
700	1.83 ± 0.19	1.77 ± 0.23	3.3	11	0.19
750	1.81 ± 0.23	1.77 ± 0.18	2.3	11	0.30
884	1.83 ± 0.17	1.78 ± 0.27	2.9	9	0.29
1,025	1.92 ± 0.33	1.81 ± 0.27	5.9	9	0.22

D distance, Constant C-D mean performance under constant C-D gain, Dynamic C-D mean performance under dynamic C-D gain, Improvement percentage improvement of the dynamic C-D gain method, Degree of freedom degree of freedom of the t test, p value p value from the t tests

Since the target prediction for the “imprecise” trials was completely incorrect, there should be no difference between tasks with constant C-D gain and dynamic C-D gain in this group. The result in Table 9 shows that the average movement time in both conditions is almost identical and no significant difference was found, except for D = 400, in which the dynamic C-D gain was significantly slower. This result likely arose because we had only three observations for this distance.

Table 9.

Experiment 2 Group Imprecise: The Mean, Standard Deviation, and the Number of Subjects Involved Under Both Conditions for Each Distance D and the p Value from the t Test

D	Constant C-D	Dynamic C-D	Improvement (%)	Degree of freedom	p value
400	1.53 ± 0.12	1.78 ± 0.22	−15.8	3	0.05
450	1.70 ± 0.57	1.69 ± 0.36	0.3	10	0.49
500	1.78 ± 0.41	1.81 ± 0.28	−1.5	9	0.44
600	1.70 ± 0.20	1.77 ± 0.28	−3.8	10	0.21
650	1.75 ± 0.26	1.82 ± 0.12	−4.1	10	0.21
700	1.76 ± 0.24	1.74 ± 0.25	1.2	9	0.41
750	1.78 ± 0.29	1.73 ± 0.16	2.9	9	0.31
884	1.97 ± 0.46	2.02 ± 0.26	−2.2	6	0.39
1,025	1.96 ± 0.29	1.76 ± 0.29	10.6	7	0.10

D distance, Constant C-D mean performance under constant C-D gain, Dynamic C-D mean performance under dynamic C-D gain, Improvement percentage improvement of the dynamic C-D gain method, Degree of freedom degree of freedom of the t test, p value p value from the t test

Accuracy

The cursor was displayed as a cross 20-pixel wide with an active area of a single pixel. We computed the percentage of error clicks (clicks outside the ten-pixel wide circular target) over all trials. Figure 7 shows the error rates of each subject under each condition. The error under dynamic gain is mostly lower than that under constant gain. The average rate with constant gain was 8.20% versus 6.25% with dynamic gain (t = 2.20, p < 0.03, df = 11).

Fig 7.

Experiment 2: The percentage error for every subject.

User Preference

Most subjects (10/12) claimed that they did not perceive any differences between the two conditions. For the remaining two, one preferred the dynamic gain, and the other preferred the constant gain. This differs strongly from the preferences expressed by subjects in experiment 1. One possible reason is that many trials in the dynamic gain condition were performed with medium or imprecise target prediction. The dynamic gain condition effectively mixed dynamic- and constant-gain trials, reducing perceptibility of the dynamic gain.

Summary

A dynamic CD gain can reduce mouse pointing times. In the ideal condition, where the target was precisely known in advance, times were reduced 14%. In a more realistic condition, in which eyegaze was used to predict the likely target location, improvement varied with prediction accuracy. For trials in which the prediction was within 20 pixels of the target, pointing time was improved 7.8% for distances larger than 600 pixels. However, no effect was found for the longest distance, 1,025 pixels, perhaps because this distance required the user to reposition the mouse, overwhelming the effect of the dynamic gain.

For trials with moderately accurate prediction (error within 20 and 40 pixels), dynamic gain had essentially no effect. For trials with the most inaccurate prediction (error greater than 40 pixels), the effect of dynamic gain varied widely, in the worst case actually slowing the user’s performance.

Dynamic gain also reduced the error rate slightly and was typically imperceptible to users.

Our experimental setup produced precise target predictions (defined as an error of less than 20 pixels) for 43% of the trials. This prediction accuracy was necessary for substantially improved performance. We suggest that this level of accuracy could be obtained on a higher proportion of trials by developing calibration methods for the gaze tracker and better processing of the tracking data. More accurate predictions might be obtained by combining gaze data with methods that predict targets from samples of the initial cursor movement. If these methods bear fruit, future radiology workstation could feature an integrated eyegaze tracker. Then, the potential target will be recognized and cursor can be slowed down to make the selection faster and also reduce the error rate.