Modeling Two Dimensional Touch Pointing

Abstract

Modeling touch pointing is essential to touchscreen interface development and research, as pointing is one of the most basic and common touch actions users perform on touchscreen devices. Finger-Fitts Law [4] revised the conventional Fitts’ law into a 1D (one-dimensional) pointing model for finger touch by explicitly accounting for the fat finger ambiguity (absolute error) problem which was unaccounted for in the original Fitts’ law. We generalize Finger-Fitts law to 2D touch pointing by solving two critical problems. First, we extend two of the most successful 2D Fitts law forms to accommodate finger ambiguity. Second, we discovered that using nominal target width and height is a conceptually simple yet effective approach for defining amplitude and directional constraints for 2D touch pointing across different movement directions. The evaluation shows our derived 2D Finger-Fitts law models can be both principled and powerful. Specifically, they outperformed the existing 2D Fitts’ laws, as measured by the regression coefficient and model selection information criteria (e.g., Akaike Information Criterion) considering the number of parameters. Finally, 2D Finger-Fitts laws also advance our understanding of touch pointing and thereby serve as the basis for touch interface designs.

INTRODUCTION

Despite the widely recognized “fat finger” problem, finger-touch based interaction has played a central role in the mobile revolution of everyday computing and communication. Among a number of touch operations, pointing is one of the most basic and common actions. Because of its prevalence, modeling touch pointing is critical to touch interface design, development, and evaluation. The most widely known pointing model is Fitts’ law [7, 11, 25], which relates the pointing movement time (MT) to the relative precision of the tasks $(\frac{A}{W})$. Although Fitts’ law has been very successful for modeling mouse or stylus based pointing actions, it does not take the finger touch ambiguity problem into account and hence cannot accurately model touch-based pointing as is.

Finger-Fitts law [4] revised Fitts’ law into a one dimensional pointing model by accounting for the fat finger ambiguity (absolute error caused by finger input) problem that arises in touch interaction. Recent research [2, 4, 24] has shown it is an useful extension and has been adopted for modeling and interface development. However, Finger-Fitts law is only dimensional, whereas most of the interface elements are two dimensional such as buttons and icons. There are both theoretical and practical needs for expanding Finger-Fitts law for modeling two dimensional pointing (2D).

There are challenges that need to be resolved for modeling 2D touch pointing. First, we need to develop and select the right model forms. Although the Fitts’ law literature [1, 26] has produced multiple 2D pointing models, none of them accommodate the absolute ambiguity of finger touch. Second, 2D pointing is governed by both amplitude and directional constraints; the former is the constraint along the movement direction while the latter is the constraint perpendicular to the movement direction. How can the model define these two constraints from the screen coordinates of the target, especially when the movement direction does not align with either of the screen’s x or y direction?

In this paper we generalize Finger-Fitts law to 2D touch pointing by resolving these two critical challenges. We first revise the two common 2D Fitts’ models – the Euclidean [1] and Smaller-Of models [26] – to accommodate the absolute ambiguity of finger touch. The Finger-Fitts Euclidean model, which generalizes Fitts Euclidean model for touch pointing, is expressed as:

$$MT=a+b{\log }_2(\sqrt{{(\frac{A}{\sqrt{W^2-c^2}})}^2+\eta {(\frac{A}{\sqrt{H^2-c^2}})}^2}+1).$$ (1)

where a,b,c, and η are empirically determined parameters.

The Finger-Fitts Smaller-Of model, which generalizes Fitts Smaller-Of Model for touch pointing, is stated as:

$$MT=a+b{\log }_2(\frac{A}{\sqrt{min{(W,H)}^2-c^2}}+1).$$ (2)

where a,b, and c are empirically determined parameters.

Compared with the existing 2D Fitts’ models [1, 26], these two 2D Finger-Fitts models ( Equations 1 and 2) introduce a parameter (denoted by c²) to account for the variance caused by the absolute error of finger touch.

Our evaluation showed that these 2D Finger-Fitts models significantly improved the prediction accuracy for movement time compared to the existing 2D Fitts’ models [1, 26], as measured by regression coefficient and model selection information criteria (AIC, BIC, and WAIC) metrics considering the number of parameters. Among all the models, Finger-Fitts Euclidean (Equation 1) performed the best. Adding more free parameters to it brings no further performance gain. In addition to creating new model forms, we investigate how to define the amplitude and directional constraints. We discover that it is effective to approximate amplitude and directional constraints with nominal target width and height defined in screen coordinates.

RELATED WORK

We review related work on (1) modeling touch pointing using Fitts’ law, and (2) modeling 2D pointing.

Modeling touch pointing

As finger touch has become the dominant input modality in mobile computing, a sizable amount of research has been carried out to understand and model the uncertainty in touch interaction. On a capacitive touchscreen, a touch point is converted from the contact region of the finger. This is an ambiguous and “noisy” procedure, which inevitably introduces errors. Factors such as finger angle [17, 18] and pressure [13] may affect the size and shape of the contact region, unintentionally altering the touch position. The lack of visual feedback on where the finger has landed due to occlusion (the “fat finger” problem) further exacerbates the issue [17, 18, 28, 29, 30]. As a result, it is hard to precisely control the touch position even with fine motor control ability.

This “fat finger” problem, or the lack of absolute precision in finger touch, presented a challenge to use Fitts’ law as a model of finger touch-based pointing, because the only variable in Fitts’ law, namely Fitts’ index of difficulty, log₂(A/W +1), is solely determined by the relative movement precision, or the distance to target size ratio.

Bi, Li and Zhai [4, 5, 6] identified this challenge, and proposed the Finger Fitts law [4] to address it. They derived their model by separating two sources of end point variance - those due to the absolute imprecision of finger touch (denoted by σ_a²) and those due to the speed-accuracy trade-off demonstrated in a pointing process (denoted by σ_r²). The end point variance caused by the imprecision of finger touch (σ_a²) is irrelevant to the speed-accuracy tradeoff that the traditional Fitts’ law models. They accounted for it by subtracting σ_a² from the observed variance σ², which led to Finger-Fitts law (Equation 3). Following the notation of effective width $W_e=\sqrt{2\pi e}\sigma$ (or 4.133σ) [9, 27, 31], Finger-Fitts law (Equation 3) can be re-expressed as Equation 4:

$$MT=a+b{\log }_2\left(\frac{A}{\sqrt{2\pi e\left({\sigma }^2-{\sigma }_a^2\right)}}+1\right)$$ (3)

$$=a+b{\log }_2\left(\frac{A}{\sqrt{W_e^2-2\pi e{\sigma }_a^2}}+1\right),$$ (4)

Later research [2, 4, 24] showed that Finger-Fitts law was useful in modeling various touch interactions. For example, research [2] showed it was more accurate than the typical Fitts’ law in estimating the upper bound of typing speed on a virtual keyboard. Researchers [24] extended the Finger-Fitts law to the crossing action with finger touch, which improved the model fitness (R²) from 0.75 to 0.84 over the original Fitts’ law.

The underlying assumption behind Finger-Fitts law is that the absolute ambiguity caused by the input device (e.g., finger touch) should be separated when modeling pointing tasks. This assumption has later been generalized to model moving target selection [19, 20, 21], and to model target acquisition in VR/AR [32]. The generalized assumptions all led to improved modeling performance. Despite the success, Finger-Fitts law is limited as it is a 1D model. In this paper, we generalize it for 2D pointing.

Modeling 2D pointing

As one of the best known theoretical foundations of HCI, Fitts’ law [11, 25] has served as a cornerstone for interface and input device evaluation [7, 25], interface optimization [22, 23], and interaction behavior modeling [8].

As the majority of the graphical interfaces are two dimensional, a considerable amount of research has been conducted to extend Fitts’ law to 2D pointing. The very first study was carried out by Crossman [10]. He proposed that target height had a similar logarithmic effect on the pointing time (Equation 5). Although it was the first proposed model for 2D pointing, the additive nature of two indexes of difficulty, log₂(A/W +1) and log₂(A/H +1), failed to reflect the interaction effect between target height and width:

$$MT=a+b{\log }_2(\frac{A}{W}+1)+c\cdot lo{g}_2(\frac{A}{H}+1).$$ (5)

MacKenzie and Buxton [26] studied bivariate pointing in the spirit of keeping the form of Fitts’ law, i.e., predicting the movement time using the ratio of distance to be covered and the target “extent”. Among the five candidates of target “extent”, they found the ID_min(W,H) model (referred to as the smaller-of model, Equation 6), which used the minimum width and height as the target “extent”, had the best correlation with the experimental data:

$$MT=a+b{\log }_2(\frac{A}{min(W,H)}+1).$$ (6)

Accot and Zhai [1] used the notion of ℓ_p norm to model bivariate pointing with explicit amplitude and directional constraints (a.k.a, a Euclidean model). Their investigation led to their model with one more free weight :

$$MT=a+b{\log }_2(\sqrt{{(\frac{A}{W})}^2+\eta {(\frac{A}{H})}^2}+1).$$ (7)

Unlike the previous research which viewed ID of a pointing task as a function of task variables such as A, W, and H, Grossman and Balakrishnan [15] proposed a different approach of predicting MT: firstly modeling the probability of landing end points within a target and then calculating the ID from such a probability. The MT is then predicted based on the estimated ID.

In the present research, we adopted the approach of modeling ID as a function of task variables. To select the best model form we based the current 2D Finger-Fitts model development on both the smaller-of and Euclidean models: we revised them to account for the ambiguity caused by the finger touch.

Previous research has also shown that it is not straightforward to properly define spatial constraints for 2D pointing. For example, previous work [1] identified that there were both amplitude (W) and directional constraints (H) in 2D pointing. However, their research considered only horizontal and vertical movement directions. We generalized Accot and Zhai’s model [1] to angled movement directions in this research. One option we investigated in this research is using apparent width and apparent height to define amplitude and direction constraint. Such an option was inspired by the apparent width idea in Fitts’ law research, though this option slightly under-performed another option – using nominal width/height to approximate amplitude/direction constraint.

Next, we describe how we generalize Finger-Fitts law to 2D pointing and how we tested it.

TWO DIMENSIONAL FINGER-FITTS LAW

There are two critical challenges for successfully modeling 2D touch pointing: (1) model form selection and (2) amplitude and directional constraint definition. We address them as follows.

Model Form Selection

We propose models for 2D touch pointing by extending the dual Gaussian distribution hypothesis [4] to the two most successful 2D Fitts’ model for cursor and stylus pointing: Euclidean model (Equation 7) [1] and Smaller-Of model (Equation 6) [26].

The dual Gaussian distribution hypothesis is the assumption under which Finger-Fitts law was derived. It hypothesized that the observed variability in the touch point distribution $(W_e^2)$ is the sum of variables from dual independent sources: (a) the relative precision governed by the speed-accuracy tradeoff of human motor systems, and (b) the absolute precision governed by the uncertainty of finger touch (denoted by $2\pi e{\sigma }_a^2$). Under this assumption, the endpoint variability caused by source (a), which is related to the relative precision the movement follows, should be calculated as $(W_e^2-2\pi e\cdot {\sigma }_a^2)$, because $W_e^2$ is the sum of variabilities from sources (a) and (b), and the variance from (b) is caused by the ambiguity of finger input and is independent from the movement. Therefore, it would be logically more sound to replace W_e in Fitts’ law with effective width with $\sqrt{W_e^2-2\pi e\cdot {\sigma }_a^2}$, because the latter more e a truly reflect the speed-accuracy tradeoff human motor system actually follows in the task. It means, that the variance caused by the finger touch ambiguity is taken away when modeling the movement.

Here we extend the dual Gaussian distribution hypothesis to 2D pointing. In 2D pointing, the endpoints have variability in two directions: (1) the direction parallel to the movement direction of the finger (i.e., W), and (2) the direction perpendicular to the movement direction of the finger (i.e., H). We hypothesize that the absolute error of finger input has contributed variance to both directions, and these variances should be subtracted when modeling the speed-accuracy tradeoff of human motor system.

Finger-Fitts Euclidean Model

Applying this hypothesis to the Fitts’ Euclidean model, we obtain the follows:

$$MT=a+b{\log }_2(\sqrt{{(\frac{A}{\sqrt{W^2-c^2}})}^2+\eta {(\frac{A}{\sqrt{H^2-c^2}})}^2}+1).$$ (8)

where c is an empirically determined parameter. We refer to Equation 8 as Finger-Fitts Euclidean model.

Compared with the Fitts’ Euclidean model, Finger-Fitts Euclidean model replaces W with $\sqrt{W^2-c^2}$, where c² represents the absolute variance caused by finger touch along the direction of W (the direction parallel to the movement of the finger). The physical meaning of $\sqrt{W^2-c^2}$ is that by taking away the absolute variance caused by finger touch, the remaining value would more truly reflect the speed-accuracy tradeoff human motor system actually follows. In other words, it means the absolute variance caused by the ambiguity of finger touch is independent from the movement and should be subtracted. It also means that, to attain a given level of accuracy, the subject would have to aim at a higher level, because touch input introduces a certain amount of noise (represented by c²). We also applied the same principle to the direction perpendicular to the movement of the finger (i.e., H), and replace H with $\sqrt{H^2-c^2}$.

Equation 9 shows a variant of Finger-Fitts Euclidean model, which assumes the absolute variances in W and H directions are different:

$$MT=a+b{\log }_2(\sqrt{{(\frac{A}{\sqrt{W^2-c^2}})}^2+\eta {(\frac{A}{\sqrt{H^2-d^2}})}^2}+1).$$ (9)

where c, and d are empirically determined parameters. Equation 9 is the more complete version of 2D Finger-Fitts Euclidean model, and Equation 8 is a simplified one. The complete model increases the fitting flexibility, but adds one extra parameter.

We also made the following two further simplifications over the original Finger-Fitts law. First, we use the nominal target width W, rather than the effective width W_e. This assumes that participants respect the spatial constraint set by the task parameters. Second, instead of assuming the absolute finger touch variance is a pre-defined constant across tasks (denoted by ${\sigma }_a^2$ in Finger-Fitts law), we let it be an empirically determined parameter from the task (denoted by c² in Equation 8). The assumption here is that the absolute error caused by finger varies in different task contexts. Thus it would be logically more sound to determine it from the empirical data.

The Finger-Fitts Euclidean models are expansions of the Fitts’ Euclidean model which can be viewed as a special case where c² ≈ 0 and d² ≈ 0. It indicates that the input device is accurate enough and the variance caused by it is negligible (near 0). This could happen in mouse or stylus input.

Finger-Fitts Smaller-Of Model

We also follow a similar approach to extend another widely used 2D pointing model, the smaller-of model [26] for touch pointing. We assume that the absolute ambiguity of finger touch cause variance in endpoint distribution, and such amount of variance should be accounted for by subsracting it from min(W,H)², which is the counterpart of W_e in the smaller-of model. This assumption leads to the Finger-Fitts Smaller-Of model:

$$MT=a+b\cdot {\log }_2(\frac{A}{\sqrt{min{(W,H)}^2-c^2}}+1).$$ (10)

where a, b, and c are empirically determined parameters, and c² accounts for the noise or input uncertainty caused by finger touch.

In sum, Equations 8, 9, and 10 are three model candidates for 2D touch pointing. We have derived them by building the absolute error of finger touch into the most successful 2D pointing models for desktop interaction [1, 26].

Defining amplitude (W) and directional (H) constraints

Another key challenge of modeling 2D pointing is to appropriately determine the amplitude (W) and directional constraints (H) [1], which are defined as:

• Amplitude Constraint (W): it is the constraint along the movement direction of the finger, specifying how wide the end point spread can be along the movement direction.

• Directional Constraint (H): it is the constraint perpendicular to the movement direction of the finger, specifying to what degree the end point can deviate from the travel direction. In other words, it specifies how wide the end point spread can be perpendicular to the finger travel direction.

To avoid confusion, we refer to the nominal width of a rectangular target as x-length, and the nominal height as y-length. The x-length of a rectangle as the length of the side parallel to the x direction on the device screen which is normally parallel to the human body (i.e., left/right direction), and y-length as the length of the side parallel to the y direction on tablet (i.e., up/down direction). The x– and y– lengths are defined in the screen coordinate system, while the amplitude (W) and directional H constraints are defined in the movement coordinate system.

W and H for vertical and horizontal movements

Both amplitude and directional constraints are well defined for horizontal and vertical movements. For example, if a user moves the finger horizontally (along the x-axis) to select a 20mm (x-length) × 10mm (y-length) rectangle, the 20mm x-length is the amplitude constraint and the 10 mm y-length is the directional constraint. In contrast, if the user moves the finger vertically (along the y-axis) to select the target, amplitude and directional constraints are swapped for the identical target: the amplitude constraint is 10mm and the directional constraint is 20mm.

W and H for angled movements

However, it is a challenge for defining W and H for an angled movement, which referred to movement directions that are not horizontal or vertical. Because both x-length and y-length may contribute to the amplitude and directional constraints, as shown in Figure 1, the question is how they should be defined.

Figure 1:

A illustration of option 1: using nominal width (x-length) and height (y-length) to define amplitude (W) and directional (H) constraints. (a): in vertical movement direction, y-length is W and x-length is H. (b): in horizontal movement direction, x-length is W and y-length is H. (c): in angled movement direction, if the direction falls within the grey area, x-length is W and y-length is H; if the direction is within the white area, y-length is W and x-length is H.

We explored 2 options for defining amplitude and direction constraints for angled movement directions:

• Option 1: Nominal Width and Height. As shown in Figure 1, this method uses nominal width (x-length) and height (y-length) of a target to approximate W and H. If θ fell within the ranges of [0°, 45°], [135°, 225°], [315°, 360°] x-length served as amplitude constraint and y-length served as the directional constraint. If θ was with [45°, 135°] and [225°, 315°], y-length was the amplitude while x-length was the directional constraint. This method essentially simplifies angled conditions to their nearest vertical or horizontal conditions, using either nominal width (x-length) or nominal height (y-length) as amplitude or directional constraints according to the movement angles.

• Option 2: Apparent Width and Height. As shown in Figure 2, this method uses the apparent widths and heights as amplitude and directional constraints. We first draw a straight line connecting the center of starting rectangle to the target rectangle. The apparent width is the length of the line segment intersected within the target. We then draw another straight line crossing the center of the target and perpendicular to the movement direction. The length of the line segment intersected within the target is the apparent height.

Figure 2:

A illustration of option 2: using apparent width (in blue) and height (in green) to define amplitude (W) and directional (H) constraints.

After creating model forms and methods for defining W and H, we carried out a user study to evaluate them.

EVALUATION IN 2D POINTING TASKS

We conducted a rectangular target acquisition experiment to evaluate the three proposed 2D Finger-Fitts pointing models, and the two options of deciding W and H in a 2D rectangular targets pointing task.

Participants and Apparatus

We recruited 18 participants (8 females) aged from 21 to 29 (mean: 24.89, std: 2). All were right-handed. The participants performed tasks on a Android Pixel C Tablet with Android API 27. The tablet was 211 mm wide and 148 mm high. Throughout the entire experiment, the device was fixed in the landscape orientation and placed on a table.

Design

The study followed a within-subject factorial design. The independent variables were distance (A) between target centers, x-length and y-length of a rectangular target, and pointing direction(θ). There were 3 different A: 36, 54 and 80mm. Similar to the previous 2D pointing study [1], our experiment had 3 basic x-length and y-length values (4, 8 and 10 mm) along with 4 x-length and y-length ratios (1, 1.5, 2, 2.5). Sixteen different pointing direction were also considered (0°, 22.5°, 45°, 67.5°, 90°, 112.5°, 135°, 157.5°, 180°, 202.5°, 225°, 247.5°, 270°, 292.5°, 315°, and 337.5° ). There were in total 1008 x-length × y-length × distance × angle combinations as shown in Table 1.

Table 1:

Distance (A), target sizes (x-length and y-length), and movement directions (θ) in experiment.

Distance (A mm)	36, 54, 80
x-length = y-length (mm)	(4, 4), (8, 8), (10, 10)
x-length < y-length (mm)	(4, 6), (4, 8), (4, 10), (8, 12), (8, 16), (8, 20), (10, 15), (10, 20), (10, 25)
x-length > y-length (mm)	(6, 4), (8, 4), (10, 4), (12, 8), (16, 8), (20, 8), (15, 10), (20, 10), (25, 10)
Angle (θ°)	0, 22.5, 45, 67.5, 90, 112.5, 135, 157.5, 180, 202.5, 225, 247.5, 270, 292.5, 315, 337.5

Procedure

We designed a reciprocal target acquisition task. Each participant placed the tablet on the table and selected the targets using the index finger of their dominant hands. In each trial (Figure 4), two rectangles were displayed on the screen with a randomly chosen x-length × y-length × distance × angle combination from the experimental conditions. The combinations showed up in random order for each participant. We defined a trial as a target acquisition action. At the beginning of each trial, the starting rectangle was in light blue and the target rectangle was in light red. A participant was instructed to select the starting rectangle to start the trial. Upon successfully selecting the starting rectangle, the target rectangle changed to light blue, and the participant was instructed to select the target rectangle as quickly and accurately as possible. If the participant failed to select the target (the touch point fell outside the target), a failure sound was played and she/he would repeat the trial until the selection was successful. If the selection was successful, a success sound was played, and the previous starting rectangle became the target. The participant then moved the finger to select the target, which was considered the next trial.

Figure 4:

Left: a participant in the study. Right: a screenshot of the task.

Because a reciprocal target selection task included two movement angles (e.g., a horizontal reciprocal target selection task included both 0° and 180° angles), the 1008 x-length × y-length × distance × angle combinations formed 504 reciprocal target selection tasks. After the participant successfully performed a reciprocal target selection a task 6 times, a new reciprocal target selection task was randomly selected and displayed and the participant moved on to the next trials.

The experiment collected 59,427 target selection trials in total. We considered the trails in which the touch points fell beyond 3 std as outliers. After removing the outliers (1.08% of total trials), we had 58,785 trials. There were 54,432 success trials: 504 (reciprocal tasks) × 6 (success trials in a task) × 18 (participants) = 54,432 (success trials). The average error rate was 7.4%, which was the percentage of the failed trials.

Results

Although our main focus was to model movement time MT, we first examined the pointing performance measured by movement time and error rates in different conditions (by x-length, y-length, and A). The movement time MT is defined as the duration between the moment the finger lifted from the starting rectangle and the moment the finger landed on the target rectangle. The error rate was the percentage of trials where touch points landed outside the target. MT (s) and error rates by x-length, y-length and A are shown in Table 2. As x-length (or y-length) decreased, or A increased, MT increased.

Table 2:

Movement time (s) and error rate by x-length, y-length, and A (mm)

x-length	MT Mean (SD)	Error	y-length	MT Mean (SD)	Error
4	0.59(0.16)	17.0%	4	0.58(0.17)	15.0%
6	0.58(0.15)	13.8%	6	0.59(0.16)	16.2%
8	0.43(0.12)	6.4%	8	0.42(0.13)	6.1%
10	0.40(0.13)	4.9%	10	0.39(0.11)	4.9%
12	0.38(0.12)	3.0%	12	0.40(0.11)	3.6%
15	0.33(0.10)	1.3%	15	0.36(0.10)	3.1%
16	0.36(0.15)	2.3%	16	0.39(0.11)	4.5%
20	0.33(0.13)	2.3%	20	0.38(0.13)	4.2%
25	0.31(0.11)	1.4%	25	0.35(0.11)	2.0%
Distance A	MT Mean (SD)	Error Rate
36	0.38(0.12)	8.2%
54	0.43(0.13)	7.1%
80	0.52(0.14)	7.3%

Model candidates

We evaluated the following five candidates for modeling MT:

• Fitts’ Euclidean Model (Equation 7). It is an Euclidean model with one free weight, a successful 2D pointing model for mouse and stylus pointing.

• Finger-Fitts Euclidean Simplied Model (Equation 8). As previously described, it built the absolute error of finger touch (denoted by c²) into the Fitts’ Euclidean model.

• Finger-Fitts Euclidean Complete Model (Equation 9). As previously described, it introduced two extra parameters (i.e., c² and d²) into the Fitts’ Euclidean model.

• Fitts’ Smaller-Of Model (Equation 6). It was another widely used 2D pointing model for mouse and stylus pointing, which uses min(W,H) in lieu of W in Fitts’ law.

• Finger-Fitts Smaller-Of Model (Equation 10). It was an extension of Fitts’ Smaller-Of model, which built the absolute error of finger touch (denoted by c²) into the Fitts Smaller-Of model.

Besides evaluating different models, we also compared two approaches to defining amplitude (W) and directional (H) constraints, as described in the previous section:

• Nominal Width and Height. As previously described, it uses nominal target width (x-length) and height (y-length) to approximate W and H (Figure 1).

• Apparent Width and Height. As previously described, it defines W along the movement direction and H perpendicular to the movement direction (Figure 2)

In sum, we evaluated 5 model candidates × 2 W and H definition options.

Model Evaluation

We grouped trials by width × height × distance × angle, resulting in 504 groups. We then obtained the mean movement time (MT) of each group. These 504 MT means were data in our model evaluation. By a least-squares fit method, we estimated the parameters of the 5 model candidates, using two W and H definition methods separately.

Nominal Width(Height) vs. Apparent Width(Height)

We first compared the two W and H definition methods by examining the coefficient of determination (R²) across all the five model candidates.

As shown in Table 3, using nominal width and height outperforms using apparent width and height in all the model candidates. The former led to a higher R² for every model. Using nominal width and height is also simple, straightforward, and easy for implementation. The rest of the data analysis was based on this W and H definition approach.

Table 3:

Coefficient of determination (R²) of 5 model candidates with 2 W and H definition methods.

Model		R2 of Using Nominal Width & Height	R2 of Using Apparent Width & Height
Smaller-Of	Fitts’	0.886	0.834
	Finger-Fitts	0.915	0.848
Euclidean	Fitts’	0.914	0.898
	Finger-Fitts Simplified	0.943	0.914
	Finger-Fitts Complete	0.944	0.915

Model Comparision

Figure 5 and Table 4 show the regression results by ID for all of the 5 models. The two Finger-Fitts Euclidean models had the best fitting results, with R² = 0.943 and R² = 0.944. The Finger-Fitts Smaller-Of model also outperformed the Fitts’ Smaller-Of model, with R² = 0.915.

Figure 5:

MT vs. ID regressions for 5 model candidates. As shown, the Finger-Fitts Euclidean Simplified model (d) performed the best, and all the 2D Finger-Fitts models (b, d, and e) outperform their counterparts 2D Fitts’ models (a [26] and c [1]).

Table 4:

Model parameters and evaluation results of 5 model cadidates. Information criteria (AIC, BIC, and WAIC) measure prediction accuracy after taking into account model complexity. The smaller the value, the better a model.

Model		R2	RMSE	AIC	WAIC	BIC	Parameters
Smaller-Of	a) Fitts	0.886	0.041	−2697.94	−2716	−2689.49	a=−0.126, b=0.177
	b) Finger-Fitts	0.915	0.036	−2842.04	−2860.49	−2829.37	a=−0.035,b=0.138,c2=9.213
Euclidean	c) Fitts	0.914	0.036	−2839.5	−2857.76	−2826.83	a=−0.150,b=0.175,η =0.960
	d) Finger-Fitts Simplified	0.943	0.029	−3044.95	−3063.56	−3028.06	a=−0.060,b=0.138,c2=9.634,η=0.967
	e) Finger-Fitts Complete	0.944	0.029	−3045.08	−3064.17	−3023.95	a=−0.058,b=0.138,c2=9.164,d2=9.999, η =0.897

Cross validation.

To maintain external validity of the analysis, we also examine the leave-one-out Root Mean Square Error (RMSE) for each model on the mean movement time. As shown in Table 4, the results were consistent with R² values. For both Euclidean and Smaller-Of models, the Finger-Fitts versions outperformed their standard Fitts’ law model counterpart.

Information Criteria.

Additionally, we also examined the Information criteria, which are commonly used metrics for model selection because (1) they reflect the relative quality of a model, and (2) they take into account the complexity of the model (i.e., the number of parameters). Some information criteria such as Akaike information criterion (AIC) approximates the amount of information lost with a model. Other commonly used information criteria include Watanabe–Akaike information criterion (WAIC), and Bayesian Information Criterion (BIC). Each of these three criteria penalizes the complexity of a model to various degrees: AIC has the least while BIC has the most severe penalties. Quantitatively, the smaller the information criterion value, the better the model is.

The information criteria results are displayed in Table 4. The results were consistent with R² and RMSE. The Finger-Fitts models have smaller values than their Fitts’ counterpart in each of the three information criteria, confirming that the Finger-Fitts 2D models outperformed the Fitts’ version.

Likelihood Ration Test (LRT).

We carried out LRT to examine whether the prediction improvements of Finger-Fitts 2D models are significant compared with their Fitts’ counterparts. LRT is often used to compare multiple models if one of them is a constrained version (or nested) of another. Our comparision fell within this category. The Fitts’ Smaller-Of model can be considered as a constrained version of Finger-Fitts Smaller-of in which c = 0; the Fitts’ Euclidean model is a constrained version of Finger-Fitts Euclidean where c = 0 and d = 0. The LRT tests showed p < 0.001 for Finger-Fitts Smaller-Of vs. Fitts’ Smaller-of(χ²(1) = 145.82), for Finger-Fitts Euclidean Simplified vs. Fitts’ Euclidean (χ²(1) = 207.8), and for Finger-Fitts Euclidean Complete vs. Fitts’ Euclidean (χ²(2) = 208.46). However, no significant difference was observed for Finger-Fitts Euclidean Simplified vs. Finger-Fitts Euclidean Complete (χ²(1) = 1.3816, p = 0.24).

DISCUSSION

Model Selection

Finger-Fitts Euclidean Simplified model performed the best among all the candidates. Its R² is 0.943, almost the same with the Finger-Fitts Euclidean Complete model which introduced one more extra parameter. Its prediction accuracy measured by information criteria is also stronger than other models. Comparing the three Euclidean models, the results showed introducing one parameter c² leads to a great fitness gain, increasing R² from 0.914 to 0.943, while further introducing another parameter d² does not improve the model fitness. The results suggested Finger-Fitts Euclidean Simplified model (Equation 8) as the extension of Euclidean model for 2D touch pointing.

Finger-Fitts Smaller-Of model also improved the fitness performance over the original Smaller-Of model. It improved R² from 0.886 to 0.915; information criteria also showed prediction accuracy improvement and LRT showed the improvement was statistically significant. It could serve as a simplified version of 2D touch pointing model.

Overall, the results showed that subtracting the absolute variance caused by finger touch (denoted by c²) is effective in improving 2D pointing model fitness. It has led to improved model for both Euclidean model and Smaller-Of model.

Defining amplitude W and directional H constraints.

The results showed using nominal width and height outperforms using apparent width and height, across all the model candidates. Additionally, using nominal width and height is a conceptually simpler approach. We recommend using this approach to define W and H in modeling 2D touch pointing.

Explaining Violation of Scale-Independence

The Fitts’ law predicts that the pointing performance is scale-independent: the MT is determined by the ratio of $\frac{A}{W}$ and is independent from the absolute value of A or W. However, previous research [16] has shown that this scale-independence claim is inconsistent with empirical data for mouse-based pointing tasks. The touchscreen users’ experience has shown that touch pointing performance is also no longer scale-independent. For example, empirical evidence shows that typing on a smaller keyboard is harder even if the ratio $\frac{A}{W}$ remains the same as that on a large keyboard. Studies [3, 14] showed that the input speed decreased from 40 Word Per Minute (WPM) on a phone-sized keyboard to 22 WPM on a watch-sized keyboard, even with the help of modern predictive and statistical decoding primed on modeling language regularities.

The 2D Finger-Fitts models provide a mathematical explanation for the violation of scale-independence in touch pointing. The Finger-Fitts Euclidean Simplified model shows that given a constant $\frac{A}{W}$, and $\frac{A}{H}$ ratio, a greater W and H lead to better performance, which contradicts the Fitts’ law scale-independence prediction that the same $\frac{A}{W}$ and $\frac{A}{H}$ lead to the same performance. Below is a detailed explanation.

Fitts’ law predicts that when A, W, and H decrease or increase proportionally, ID will remain unchanged because the decrease/increase in A and W (or H) will not cancel each other out. However, introducing c² to the denominator of ID alters this scale-independent relationship. The Finger-Fitts Euclidean Simplified model predicts that proportionally increasing A, W, and H will reduce ID. Here is a simple mathematical demonstration.

Finger-Fitts Euclidean model has defined the index of difficulty of task (ID) as:

$$ID={\log }_2(\sqrt{{(\frac{A}{\sqrt{W^2-c^2}})}^2+\eta {(\frac{A}{\sqrt{H^2-c^2}})}^2}+1).$$ (11)

Assuming the movement distance A, target width W, and target height H proportionally increase to A′ = αA, W′ = αW, and H′ = αH with α > 1, the index of difficulty (ID′) becomes:

$$I{D}^{\prime }={\log }_2(\sqrt{{(\frac{A^{\prime }}{\sqrt{W^{\prime 2}-c^2}})}^2+\eta {(\frac{A^{\prime }}{\sqrt{H^{\prime 2}-c^2}})}^2}+1)$$ (12)

We obtain the following relationship with a simple math derivation:

$$\frac{A^{\prime }}{\sqrt{W^{\prime 2}-c^2}}=\frac{\alpha A}{\alpha \sqrt{W^2-{\left(\frac{c}{\alpha }\right)}^2}}<\frac{\alpha A}{\alpha \sqrt{W^2-c^2}}=\frac{A}{\sqrt{W^2-c^2}}$$ (13)

Similarly, we have:

$$\frac{A^{\prime }}{\sqrt{H^{\prime 2}-c^2}}<\frac{A}{\sqrt{H^2-c^2}}$$ (14)

Plugging Inequations 13 and 14 into Equation 12 and assuming η is positive, we have:

$$I{D}^{\prime }={\log }_2(\sqrt{{(\frac{A^{\prime }}{\sqrt{W^{\prime 2}-c^2}})}^2+\eta {(\frac{A^{\prime }}{\sqrt{H^{\prime 2}-c^2}})}^2}+1)$$ (15)

$$<lo{g}_2(\sqrt{{(\frac{A}{\sqrt{W^2-c^2}})}^2+\eta {(\frac{A}{\sqrt{H^2-c^2}})}^2}+1)=ID$$ (16)

Inequation 16 shows that proportionally increasing A, W, and H will reduce ID, leading to shorter movement time. Such a finding is consistent with the touchscreen interface design guideline that target sizes should be above a pre-defined lower-bound (e.g., 48 dp in Android [12]) to ensure satisfying pointing performance.

CONCLUSION

We proposed 2D Finger-Fitts models, which extend Finger-Fitts law to bivariate touch pointing:

$$MT=a+b{\log }_2(\sqrt{{(\frac{A}{\sqrt{W^2-c^2}})}^2+\eta {(\frac{A}{\sqrt{H^2-c^2}})}^2}+1).$$ (17)

where a,b, c, and η are empirically determined parameters. A simplified version is:

$$MT=a+b\cdot {\log }_2(\frac{A}{\sqrt{min{(W,H)}^2-c^2}}+1).$$ (18)

where a, b, and c are empirically determined parameters. The 2D Finger-Fitts models account for the ambiguity of finger touch by subtracting components representing the variance caused by touch ambiguity (represented by c²).

Our evaluation shows 2D Finger-Fitts law can predict the movement time in bivariate pointing tasks well, and outperform the existing 2D pointing models including the Fitts’ Euclidean [1] and Smaller-Of models [26] originally proposed for desktop pointing by two measures together. The first measure is the coefficient of determination (R²) and second is information criteria which factors in model complexity. The 2D Finger-Fitts Euclidean model (Equation 17)’s performance is nearly saturated as adding more free parameters bring no performance gain. Our investigation also shows using nominal target width and height is a convenient yet effective approach for defining amplitude and directional constraints of 2D finger touch pointing.

Figure 3:

An illustration of experimental setting. The dotted circles show 3 possible movement distances (A). θ is the angle between movement direction and the x-axis of the screen coordinate system. The blue rectangle is the starting rectangle and the red rectangle is the target.

CCS Concepts.

• Human-centered computing → HCI theory, concepts and models; Pointing