Skip to main content
PLOS ONE logoLink to PLOS ONE
. 2009 Oct 22;4(10):e7481. doi: 10.1371/journal.pone.0007481

Fast and Flexible Selection with a Single Switch

Tamara Broderick 1,*, David J C MacKay 2
Editor: Nick Monk3
PMCID: PMC2760780  PMID: 19847300

Abstract

Selection methods that require only a single-switch input, such as a button click or blink, are potentially useful for individuals with motor impairments, mobile technology users, and individuals wishing to transmit information securely. We present a single-switch selection method, “Nomon,” that is general and efficient. Existing single-switch selection methods require selectable options to be arranged in ways that limit potential applications. By contrast, traditional operating systems, web browsers, and free-form applications (such as drawing) place options at arbitrary points on the screen. Nomon, however, has the flexibility to select any point on a screen. Nomon adapts automatically to an individual's clicking ability; it allows a person who clicks precisely to make a selection quickly and allows a person who clicks imprecisely more time to make a selection without error. Nomon reaps gains in information rate by allowing the specification of beliefs (priors) about option selection probabilities and by avoiding tree-based selection schemes in favor of direct (posterior) inference. We have developed both a Nomon-based writing application and a drawing application. To evaluate Nomon's performance, we compared the writing application with a popular existing method for single-switch writing (row-column scanning). Novice users wrote 35% faster with the Nomon interface than with the scanning interface. An experienced user (author TB, with Inline graphic 10 hours practice) wrote at speeds of 9.3 words per minute with Nomon, using 1.2 clicks per character and making no errors in the final text.

Introduction

In single-switch communication, user input consists of repeated clicks, distinguished only by timing information; these clicks might be generated by pressing a button or blinking. For instance, the range of movement of individuals with severe motor impairments may be limited to a single muscle. Alternatively, a crowded or jostled mobile technology user may be able to click precisely while other actions are difficult or sloppy. A single switch may also be useful when information conveyed, such as a PIN, is sensitive and hand location on a normal keyboard might betray this content. Our method, Nomon (Figures 1, 2), expands the application scope of existing methods and facilitates faster writing than the most common single-switch writing interface.

Figure 1. An example Nomon application for selecting between 16 points on screen (screenshot).

Figure 1

The horizontal and vertical positions of the option points were chosen uniformly at random in the box shown to illustrate the flexibility of the method.

Figure 2. The Nomon Keyboard, a writing application (screenshot).

Figure 2

Words that are prefixed by the concatenation of the current context and the letter X appear next to the letter X. Underscore represents a space. Options for period, a character-deletion function, and an undo function are also available.

Existing single-switch communication methods include scanning [1][9] and One-Button Dasher [10][15]. (Morse Code does not fall under the strict definition of a single switch interface since it requires either click duration information or multiple switches.) Scanning is the most popular single-switch selection method. In a scanning interface, options such as letters are arranged in a grid (Figure 3). For standard row-column scanning, each row of the grid is highlighted in turn, with the highlight moving to the next row at fixed time intervals, a.k.a. scanning delays. When a click is made, the columns of the selected row are then highlighted in turn, typically iterating at the same fixed time intervals. To select a column, and thereby make a final selection, the user clicks when the highlight is on that column. A variety of customizable commercial scanning software exists for writing and computer navigation [16][19] although customization is often not single-switch accessible. The Gnome Onscreen Keyboard [19], by contrast, can generate a grid for new applications “on the fly.”

Figure 3. The scanning grid from The Grid 2 used in this comparison study (screenshot).

Figure 3

The six long rectangles on the left hold word completions. The remaining options are fixed and include letters, an underscore for space, a period, a character-deletion function, and a word-deletion function.

While the scanning method can be used to select anything that can be arranged in a grid, One-Button Dasher is limited to writing with alphabetic character sets. Dasher works by arranging all possible character strings in alphabetic order and having the user zoom in on the desired string. More likely strings, according to the language model, are given relatively more space and are thus easier to select.

Scanning and One-Button Dasher require options to be arranged in a particular configuration. By contrast, traditional operating systems, web browsers, and free-form applications such as drawing place options at arbitrary points on the screen. Scanning, the most popular single-switch communication method, is limited in further ways by its grid structure. For instance, the grid options may theoretically be reordered after any selection to allow the most likely options to be selected the most quickly. However, in practice this reordering requires that users either learn many grid arrangements or search the grid for their desired option upon each reordering.

Even scanning a grid that maintains a fixed layout at all times has drawbacks. Previous studies suggest that, at least among children, scanning a fixed grid demands a higher cognitive load than direct selection [20][23]—though an earlier study found no difference [24]. One implicated factor is the need for a user to divide her attention between the scanning highlight and the desired option [20], [23]. Another issue in scanning is the possibility of distraction, and loss of the target from working memory, while highlighting progresses [23], [25].

Therefore, we seek a single-switch selection method that is not limited to certain forms of option placement. We want our method to work for any number of options; to be able to effectively reorder the set of selections without imposing additional cognitive load; and to allow the user to attend only to the desired target.

Below, we begin by describing such a method, which we call “Nomon.” We also describe how our method can adapt to individuals' clicking abilities and how it can incorporate prior beliefs about option selection frequency. In order to evaluate our method's performance, we note that much single-switch research has focused on optimizing writing speed [1][6] and the number of clicks per output symbol [7][9] in scanning interfaces. In light of these studies, we developed a writing application, the Nomon Keyboard (Figure 2), using our method and compared its performance with a popular commercial scanning interface, The Grid 2 [16] (Figure 3). We examined the study participants' writing speeds, error rates, and number of clicks made per character as well as the subjective ratings of their experiences.

The full technical report describing Nomon is available online at http://www.inference.phy.cam.ac.uk/nomon/files/nomon_tech_report.pdf. The Nomon Keyboard, as well as a drawing application (Nomon Draw) and instructions for the use of both applications, is available for download at http://www.inference.phy.cam.ac.uk/nomon/ under the GNU General Public License 3.0.

A New Method

Nomon, a new single-switch communication method, does not limit the user to selecting options that can be arranged in a grid or alphabetically. Rather, it can be used to select among any points of interest on a screen. The trademark of a Nomon application is a set of small clocks, one clock associated with each selectable option. Each clock appears alongside its corresponding option on the screen. For instance, Figure 1 illustrates clocks corresponding to 16 arbitrary option locations. Another example might be a drawing application where a clock appears at every “pixel” on the canvas and also next to each menu option. In a writing application (the Nomon Keyboard), a clock appears next to each character, word completion, or text editing function (Figure 2).

Just as menu options and drawing tools in a point-and-click interface are accessed in the same way by the mouse, all Nomon clocks are selected in the same way by a single switch. Each Nomon clock features a moving hand and a fixed line at noon. All moving hands rotate at the same, fixed speed but, at any time, are located at a variety of angles relative to noon. The user tries to click precisely when the moving hand on her desired clock is at noon. She repeats this action until the clock is selected. Selection is signalled by the desired clock being highlighted with a darker color and the entire application flashing a lighter color; there may also be audio feedback. Between clicks (if more than one click is required to select a clock), the clock angular offsets are adjusted by a heuristic to maximize the expected information content of the user's next click.

Row-column scanning can be viewed as a special case of the Nomon selection method where clocks are arranged in a grid, moving-hand angular offsets are aligned alternately across rows and columns of clocks, and each selection is based only on the times of the last two clicks. But this synchrony does not take full advantage of the continuous, periodic representation of the clock and imposes an order on the set of options relative to their positions onscreen. Rather, by allowing more general clock hand positions, we can, effectively, completely reorder the set of selections after each click without demanding any extra cognitive load from the user.

Similarly, the independent movement of the clock hands frees the user to attend only to the desired target, in contrast to the need, in scanning, for the user to attend both to the desired target and the moving highlight. Further, the scanning user may forget her target as highlighting progresses. But in Nomon, once the target is located visually, the user is free (without suffering a performance penalty) to focus on selecting a single, fixed clock. Since the clock periods are usually much shorter than a full scanning rotation, there is also no significant penalty for missing a potential click time.

In Nomon, by contrast with scanning, we assume that the user will not always click perfectly at the desired time. The details of Nomon operation are described more fully in the Nomon Operation section below and outlined here. Nomon can learn a user's probability of clicking at different (typically small) offsets relative to noon. This learning is accomplished via an approximate Parzen window estimator, with contributions from more recent clicks weighted more strongly to allow adaptation to a user whose skill changes over time. We can also specify a prior probability distribution over clocks according to a predictive model of user choices. For instance, in the writing application tested below, our language model assigned prior probabilities to letters and word completions based on the British National Corpus word-frequency list [26]. These prior probabilities could also be adaptive and context dependent.

During a particular selection process, the posterior probability of any clock given the clicks thus far can be calculated from Bayes' theorem. When the probability of a single clock is sufficiently high, we declare it the winner. The probability threshold for winning is an adjustable parameter of the model; it can vary according to context or from clock to clock. A higher threshold can ensure greater safety for critical actions.

Results

We developed a writing program using the Nomon method, the Nomon Keyboard (Figure 2), and conducted a study to compare writing with Nomon to writing with a popular commercial scanning interface, The Grid 2 [16] (Figure 3). To that end, sixteen study participants with no previous experience of either interface wrote with Nomon and The Grid 2. In each of two sessions, a participant used one of the interfaces to write short phrases appearing on screen. A session was divided into four blocks, each lasting approximately Inline graphic minutes. During the first three (of four) writing blocks, each participant was allowed to adjust the rotation-period or scanning-delay parameter, as appropriate to the current interface, at the end of each written phrase. No changes were allowed during the final block. For each interface, cash prizes were won by the faster half of participants in the final block.

In total, we collected 34 hours of data from 16 novice participants and one experienced single-switch user (TB, with Inline graphic hours experience in each interface). We compared three objective measures of the novice participants performance between the two interfaces: text-entry rate, error rate, and click load (clicks per character). We also examined subjective ratings of the two interfaces given by the novice participants.

Text-entry Rate

We calculated text-entry rate in words per minute, where a word is defined as five consecutive characters in the output text. At the beginning of each fourteen-minute block, the participants were asked to write two periods “..” using the interface for that session. This action signalled that they were ready to begin and initiated the display of the first target phrase. Timing started once the two periods were written. After every phrase, participants wrote two periods to signal that they were ready for a new phrase. Timing stopped after the final two periods following the last phrase were written. All periods except the first two in a block were counted as characters in what follows, and the time spent writing them was counted as well.

The left panel of Figure 4 shows the novice participants' mean entry rates across the four blocks for each interface. Also shown, for comparison, is the performance of the experienced user. Participants wrote faster with Nomon than with The Grid 2 during the first block (Inline graphic, Inline graphic). The total session time was short for both interfaces, but participants' writing speed with each interface improved with practice. Participants became faster at writing using the Nomon Keyboard during the Nomon session (Inline graphic, Inline graphic) and became faster at writing using The Grid 2 during the scanning session (Inline graphic, Inline graphic). In the final block we see that participants remained faster at writing with Nomon than with The Grid 2 (Inline graphic, Inline graphic). In this fourth block, participants wrote Inline graphic faster with Nomon than with the scanning interface; participants were writing at Inline graphic words per minute on average with The Grid 2 and Inline graphic words per minute with the Nomon Keyboard. The experienced user wrote, on average, at Inline graphic words per minute with the Nomon Keyboard and Inline graphic words per minute with The Grid 2.

Figure 4. Mean entry rate (left) and click load (right) across interface blocks.

Figure 4

Mean entry rate is measured in words per minute, and click load is measured in clicks per output character. In both panels, error bars represent 95% confidence intervals for the novice user means, and the average experienced user (TB) performance is illustrated by horizontal lines for comparison.

While the alphabetic layout was easy for novices to use, a computer simulating writing from a conversational corpus with no errors has been shown to achieve a Inline graphic faster writing speed with a frequency-ordered layout than with an alphabetic layout [6]. Even if we artificially inflate the novice writing speeds using The Grid 2 by Inline graphic, novices remain faster at writing with Nomon (Inline graphic, Inline graphic).

Error Rate

To find the error rate during a block, we begin by computing the character-level Levenshtein distance [27] Inline graphic between the Inline graphic target phrase in the block and the text written by the participant; Inline graphic is also known as the edit distance. We define the error rate for the block to be Inline graphic, where Inline graphic is the number of characters in the Inline graphic target phrase.

The average novice character-level error rate (over all blocks) for the Nomon Keyboard was Inline graphic, and the average novice error rate for The Grid 2 was Inline graphic. There was no significant difference in novice error rate between the two interfaces (Inline graphic, Inline graphic). The experienced user made no errors while using Nomon (Inline graphic) and made one error while using The Grid 2, for a mean scanning block error rate of Inline graphic.

We believe that the participants' output errors were mostly caused by poor recall of the target sentence. For instance, one participant pluralized “head” in “head_shoulders_knees_and_toes” and wrote “reading_week_is_almost_here” instead of “reading_week_is_just_about_here”.

Click Load

The click load is the number of clicks per output-text character. Other names for this measure include “keystrokes per character” [28] and “gestures per character” [14]. The click load is calculated as the number of button presses in a block divided by the number of characters in the output. Clicking often can be tiring for any user and especially so for some users with specific motor impairments.

While the inclusion of word-completion options in a scanning grid has been shown to have no positive effect on writing speed with a scanning interface [7], other studies confirm that word completion options yield substantial click-rate savings over the baseline (mistake-free) row-column click load of two clicks per character [8], [9]. Therefore, we included six word-completion options in the leftmost row of our scanning grid (consistent with the default layouts in The Grid 2 [16]). These were ordered from top to bottom and filled in automatically by the software.

Click loads are illustrated in the right panel of Figure 4. The average novice rate (over all blocks) for the Nomon Keyboard was Inline graphic clicks per character, and the average novice rate for The Grid 2 was Inline graphic clicks per character. There was no significant difference in novice click load between the two interfaces (Inline graphic, Inline graphic).

While the experienced user required, on average, Inline graphic clicks per character in The Grid 2, she required only Inline graphic clicks per character using the Nomon Keyboard. For comparison, writing with the same character set on a normal keyboard requires at least one key press for each character and thus at least 1 click per character (possibly more due to error correction). To compare to Morse code, we find letter, space, and period frequencies directly from our phrase set. We assume the Morse encoding of [17], [29]. In this case, an error-free Morse code click load estimate is Inline graphic clicks per character. This load is over twice as high as the click load of the experienced user on the Nomon Keyboard.

Subjective Ratings

We assessed novice participants' opinions with a questionnaire immediately after writing with an interface was completed. The questionnaires for each interface were identical (except for the name of the interface). Participants were asked to rate how much they agreed with a series of statements on a scale from 1 (strongly disagree) to 7 (strongly agree). These statements were largely the same as those in [30]. Participants were encouraged to write any thoughts about the interfaces in an “Open Comments” box.

Participants' responses to selected statements are summarized in Table 1. Not only did participants like using the Nomon Keyboard in aggregate, but every participant individually liked using Nomon at least as much as The Grid 2. Contributing factors for why the Nomon Keyboard was preferred became apparent in the remaining responses. Participants found it easier to select word completions and easier to correct errors with the Nomon Keyboard. These responses corroborate our objective findings above.

Table 1. Subjective ratings of the two interfaces by novice participants.

Statement Nomon The Grid 2
mean (sd) mean (sd)
I liked writing using X. 5.6 (1.4) 3.9 (1.5)
It was easy to select word completions (the, and, cat, …). 6.1 (0.7) 4.8 (1.3)
It was easy to correct errors. 4.5 (1.8) 3.9 (1.7)

Each response to the lefthand statements was on a scale from 1 (strongly disagree) to 7 (strongly agree). In the questionnaires, the interface name was substituted for X. Mean responses are shown with standard deviations in parentheses. Boldface is used to highlight the means corresponding to a more positive user experience.

While many written comments agreed with participants' numerical ratings, unique to the open comments section was the sentiment that Nomon looks unusual at first but is worth getting to know. One participant remarked, “Surprisingly, I found this more user-friendly.” Another noted, “The writing system looks intimidating when it first comes up on screen but is actually very easy to use.”

Discussion

Nomon benefits in this comparison from its nice scaling properties and clock-position flexibility. Our posterior-based selection method implies that the time taken to make a selection in Nomon scales logarithmically with the number of clocks if the prior over clocks is uniform. The entropy of the discrete uniform distribution, which happens to be the highest-entropy (finite) discrete distribution, scales logarithmically with respect to the number of points in the support. Figure 5 shows that, generally, Inline graphic clicks are required by an experienced Nomon user (TB, with Inline graphic hours experience) to make a selection in a 30-clock application. In a Nomon application with uniform prior and Inline graphic clocks, Inline graphic clicks are generally required for this user to make a selection. The difference in entropy between the prior for the Inline graphic-clock application and the highest-entropy prior for the Inline graphic-clock application is about Inline graphic bits, in agreement with Inline graphic.

Figure 5. Entropy of the estimated probability distribution over clocks for two Nomon applications.

Figure 5

Entropy is shown as a function of clicks remaining to selection. Each solid line represents a single selection process. Dotted lines decreasing to zero at respective rates of Inline graphic (lower) and Inline graphic (upper) bits per click are illustrated for reference. Left: 25 selections on the Nomon Keyboard: 30 clocks, non-trivial prior Inline graphic, clock period Inline graphic seconds, switch input from joystick button. Right: 25 selections on another Nomon application: 401 clocks, uniform prior Inline graphic, clock period Inline graphic seconds, switch input from space bar. Data was generated by the experienced user (TB).

Not only does the number of clicks to selection in Nomon scale well, but including additional options with small prior probabilities has little effect on clicks-to-selection for more-likely clocks. Therefore, we could place many more word completions on screen than would be feasible for a scanning interface. We limited ourselves to three per character so as to allow fast reading of the three relevant options. Placing word completions next to letters in Nomon was feasible since clock position onscreen does not affect Nomon operation. Interspersing word completions with letters in row-column scanning would increase the number of scanning steps required to reach many options.

While a Nomon writing application allows a straightforward comparison of Nomon with existing single-switch communication methods, the Nomon selection method is not limited to writing. For example, Nomon can be used for internet browsing by placing a Nomon clock next to each link. Or Nomon can be used for drawing by placing a dense grid of, say, hundreds of clocks on a canvas. (The Nomon Draw application works in this way.) A user can draw a line by selecting points directly from the canvas. Options for colors, shape drawing, saving, and printing can likewise be accessed with clocks. A general graphical user interface can be navigated with Nomon by placing clocks at the points where a user might traditionally point and click.

It is worth pointing out that the flexibility of Nomon is not specific to our clock display choice. Other local periodic representations of the global set of options would also allow the arbitrary placement of options onscreen. For instance, the clocks could be replaced by bouncing balls at different points in their trajectories; instead of clicking at noon, the user would click when the desired ball hits the ground. It remains to be studied whether such alternative display choices might facilitate even faster or easier use of this system.

Materials and Methods

We begin by detailing the experimental method used in the study above and follow with a description of how Nomon functions.

Experiment

Participants

We recruited sixteen participants from the university community across a wide range of academic disciplines. All participants gave written informed consent. In accordance with the University of Cambridge ethical review procedure as defined in the Cambridge Psychology Research Ethics Committee Handbook (http://www.bio.cam.ac.uk/sbs/psyres/), the experimental design received an internal peer review within the department, where it was decided that ethics approval from the committee was not necessary.

The participants' ages ranged from 22 to 39 (mean = 26, sd = 4). Eight were women, and eight were men. Participants were screened for motor or cognitive difficulties; in particular, no participant had dyslexia or RSI. None of the participants had used a scanning or Nomon interface before. No participant had regularly used any single-switch interface before. Twelve of the participants had used word completion (e.g. on cell phones).

In addition to the sixteen novice participants, an experienced user of Nomon and The Grid 2 (Inline graphic hours writing with each interface) was run through the same experimental procedure for comparison.

Apparatus and Software

All sessions were run on a Dell Latitude XT Tablet PC with a partitioned hard drive. The 12.1 inch color screen had a physical screen size of 261Inline graphic163 mm. The single-switch hardware device in all cases was the trigger button of a Logic3 Tornado USB joystick. Participants operated the trigger button with the first finger of their left or right hand. None of the other joystick inputs was used. For both writing interfaces, automated spoken feedback was provided as the user wrote.

Nomon Keyboard

We ran the Nomon Keyboard (Figure 2) on an Ubuntu 8.10 operating system running the Linux kernel. The screen resolution was 1280Inline graphic800 pixels, and the physical size of the keyboard display was 224Inline graphic85 mm (1125Inline graphic416 pixels). The interface was docked in the upper part of the screen. A text box and phrase box were located below the keyboard in the same window. The keys of the keyboard were arranged in six rows and five columns. Each key contained a principal character, with letters in alphabetical order (across and then down) first, followed by four special characters: an underscore (representing space), a period, a character-deletion function, and an undo function. Each letter key also contained up to three word completions. The undo function undid the previous selection if it was a character selection, word-completion selection, or deletion.

The clock rotation period Inline graphic could be set to Inline graphic seconds for Inline graphic. Higher Inline graphic corresponded to faster rotation. The initial setting of the period for novices was Inline graphic (Inline graphic). The experienced user initially chose Inline graphic (Inline graphic).

The Grid 2

We ran The Grid 2 (Figure 3) on a Windows Vista Service Pack 1 operating system. The screen resolution was again 1280Inline graphic800 pixels. The physical size of The Grid 2 display, using the scanning grid we designed for this experiment, was 261Inline graphic102 mm (1280Inline graphic500 pixels). The interface was docked in the upper part of the screen, and the text box and phrase box were docked immediately below. Six word-completion boxes appeared on the left side of the main interface. The remaining space was divided into six rows and five columns of keys. Each key contained a single character. First were letters arranged in alphabetical order (across and then down), followed by an underscore, a period, a character-deletion function, and a word-deletion function.

The Grid 2 allowed scanning delay values Inline graphic at Inline graphic for Inline graphic. Higher Inline graphic corresponded to faster scanning. The initial setting of the delay for novices was Inline graphic (Inline graphic). The experienced user initially chose Inline graphic (Inline graphic).

Procedure

The experiment consisted of two identical sessions, one for each interface. The starting interface was balanced across participants, and sessions were spaced at least four hours apart.

Each session proceeded according to the same schedule. The first ten minutes were introductory. First, the supervisor either explained or reviewed the experimental procedure according to the session number. Then the participant was shown how to use one of the interfaces. The demonstration included basic writing, word completion, and error correction.

The next hour was divided into four 14-minute blocks, separated by short breaks. During the blocks, participants were asked to write phrases drawn from a modified version of the phrase set provided by [31], with British spellings and words substituted for their American counterparts. For each participant, a different random ordering of the initial phrase set was generated. Phrases appeared one at a time in the phrase box at the bottom of the screen. Once a participant finished a phrase, writing the period character twice would cause a new target phrase to appear and the text box to empty. Participants were instructed that no changes relevant to a particular phrase could be made after the two periods were written.

During the first three (of four) writing blocks, each participant was allowed to adjust the rotation-period or scanning-delay parameter at the end of each written phrase. In particular, immediately after writing two periods and receiving the new target phrase, the participant could increment or decrement Inline graphic (defined above) by one. The experienced user incremented to Inline graphic (Inline graphic) after two blocks using the Nomon Keyboard and incremented to Inline graphic (Inline graphic) after two blocks using The Grid 2. No other changes were made by this user.

Novice participants were paid £10 for each of the two sessions; the experienced participant was not paid. Novice participants were informed at the beginning of the study that they could receive a £5 bonus for achieving a writing speed among the top half of novice participants for each interface. They were further informed that, for the purposes of the bonus, writing speed would be measured only during the final writing block. They were told that they would not be allowed to change the rotation-period or scanning-delay parameter during this block and thus would have to calibrate it as they saw fit during the previous blocks. Information about their own writing speeds across full blocks and also phrase-by-phrase was made available to participants during the break after each block.

We performed seven significance tests with a family-level significance of Inline graphic. Observing the Bonferroni correction, we performed each individual test at a significance level of Inline graphic. Wherever Inline graphic values are quoted, an analysis of variance (ANOVA) test for repeated measures was performed.

Nomon Operation

We here describe the prior over clocks, click likelihood (given a clock), and the resulting posterior over clocks in turn. While we focus on a prior for a specific application (the Nomon Keyboard), the likelihood and posterior discussions are germane to a general Nomon application.

Prior

In the absence of information about clock probabilities, we use a uniform prior Inline graphic over clocks Inline graphic. We can choose a more informative prior for our Nomon writing application, the Nomon Keyboard (Figure 2). This interface features four special characters (underscore representing space; period; Delete; and Undo), 26 letters, and up to three word completions per letter. We assign fixed prior probabilities to the special characters and assign the remaining priors according to Laplace smoothing out of the leftover probability mass Inline graphic. Let Inline graphic (Inline graphic) be the context (all letters from the end of the current output text) before the user begins to make another selection. Let Inline graphic be the set of word completions appearing on screen, and set Inline graphic. To form our corpus, we begin with the British National Corpus word list [26], then we remove single-letter words besides “I” and “a” and keep only words appearing with some small minimum frequency (Inline graphic appearances in the corpus).

When an appropriate word completion is offered, the user may nevertheless choose the next single letter; the following model assumes that the user is equally likely to choose either of these options. If Inline graphic is the number of occurrences of word Inline graphic in our corpus, we define a context frequency Inline graphic and a screen word-completion summed frequency Inline graphic. If Inline graphic is the clock corresponding to letter Inline graphic and Inline graphic the clock corresponding to word Inline graphic,

graphic file with name pone.0007481.e102.jpg (1)
graphic file with name pone.0007481.e103.jpg (2)

To model an ideal user, we would subtract the count of words onscreen prefixed by Inline graphic from the numerator of Inline graphic, and both denominators would equal Inline graphic. Finally, while the number of letters is fixed at 26, Inline graphic is variable since, for any letter, we include only those word completions among the three most probable above a certain threshold. It was judged that requiring Inline graphic yielded a reasonable balance between displaying common words and not cluttering the screen.

Click Distribution

Any particular clock Inline graphic defines a desired click time at noon. We wish to estimate a user's click time distribution relative to noon Inline graphic, where we distinguish Inline graphic only up to the clock period Inline graphic and set Inline graphic to zero at noon. To that end, we begin with a broad, and slightly offset, initial setting of our estimate for Inline graphic: Inline graphic. The Inline graphic-dependence ensures the estimate will be nontrivial at any user-chosen period. We update the Inline graphic distribution with a (modified) Parzen window estimator—with width given below—and a damping factor Inline graphic that allows learning to continue over time. After any selection is made, we modify the distribution estimate with the data from the Inline graphic selection before the latest one (here Inline graphic). This delay allows the user to choose Undo after a selection, in which case we do not use the clicks toward that selection for learning. Once a selection occurred Inline graphic rounds in the past, we assume that it was correctly chosen. With the clock choice Inline graphic known for the Inline graphic selection, we are able to calculate click times around noon Inline graphic for each click that was made toward this selection. We treat these as data from the distribution Inline graphic we are estimating. To calculate our estimate Inline graphic for Inline graphic after the Inline graphic selection, we make use of the unnormalized distributions Inline graphic.

graphic file with name pone.0007481.e130.jpg (3)

The update equation specifies that, after each selection, Inline graphic is damped by the factor Inline graphic. The next term is a sum over clicks Inline graphic leading to the Inline graphic selection. Within the summation is a normal density centered at the click time Inline graphic, as in Parzen window estimation. The width for this Parzen-window term is given by Inline graphic, which is derived from the normal scale rule estimate [32], [33] for the Parzen window. That is,

graphic file with name pone.0007481.e137.jpg (4)

where Inline graphic is the standard (Gaussian maximum likelihood) variance estimator obtained from the last Inline graphic clicks before the Inline graphic round. The factor Inline graphic in the initial Inline graphic definition is an effective number of samples derived from the damping factor. Using this factor and the unnormalized update, we ensure that the initial estimate Inline graphic dominates Inline graphic even after the first few selections. Without the Inline graphic factor, the Parzen window term for the first click, Inline graphic, would have nearly equal weight with the initial estimate.

This estimate for Inline graphic allows us to save the estimated distribution and update it quickly and easily during operation of the application. As a result, users can start the Nomon application immediately, without a waiting or calibration period, but they can also enjoy an experience tailored to their abilities. For instance, a user need not click at noon (or any offset) exactly. Their personal offset, reflecting reaction time, is learned by this method rather than hard-coded and, as long as it is not too close to 6 o'clock, will make no difference to program operation. The precision around this personal offset determines the number of clicks necessary to make a selection.

Posterior

With a prior and likelihood, we may calculate the posterior probability of each clock Inline graphic given the Inline graphic clicks thus far using Bayes' theorem: Inline graphic. In practice, we store the unnormalized log probabilities for each Inline graphic. Checking that the highest clock probability Inline graphic exceeds some threshold would require exponentiating every stored value and summing over the results. Noting that Inline graphic is equivalent to Inline graphic for some Inline graphic, we instead declare a winner when Inline graphic. The choice of Inline graphic represents a desired upper bound on error fraction, per selection, of Inline graphic. In a sample of 1,714 consecutive selections made by an experienced Nomon user (TB) on the Nomon Keyboard under this setting, the average value of Inline graphic over all selections after the deciding click was Inline graphic, and the average value of Inline graphic was Inline graphic, suggesting our heuristic stopping criterion is a reasonable approximation to the desired one. In the 1,714 selections, 3 (non-consecutive) selections were Undo, indicating mistakes and giving an empirical error rate of about Inline graphic, in line with the calculated rate.

Acknowledgments

We are grateful to Per Ola Kristensson, Geoffrey Hinton, Keith Vertanen, Philipp Hennig, Carl Scheffler, and Philip Sterne for helpful discussions.

Footnotes

Competing Interests: This research was supported in part by an unrestricted donation from Nokia (http://www.nokia.co.uk/). Sensory Software (http://www.sensorysoftware.com/) provided the 60-day free trial of The Grid 2 used in our performance comparison. All donations to this study, financial and of software, were unrestricted. There are no patents related to this study.

Funding: This research was supported by the Nine Tuna Foundation. (http://www.ninetuna.org/) and Nokia (http://www.nokia.co.uk/). TB's research was supported by a Marshall Scholarship (http://www.marshallscholarship.org/). Sensory Software (http://www.sensorysoftware.com/) generously provided the 60-day free trial of The Grid 2 used in our performance comparison. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

References

  • 1.Damper R. Text composition by the physically disabled: A rate prediction model for scanning input. Applied Ergonomics. 1984;15:289–296. doi: 10.1016/0003-6870(84)90201-1. [DOI] [PubMed] [Google Scholar]
  • 2.Simpson R, Koester H. Adaptive one-switch row-column scanning. IEEE Transactions on Rehabilitation Engineering. 1999;7:464–473. doi: 10.1109/86.808950. [DOI] [PubMed] [Google Scholar]
  • 3.Evreinov G, Raisamo R. Paris, France: Springer-Verlag Berlin/Heidelberg; 2004. Optimizing menu selection process for single-switch manipulation. pp. 836–844. In: Proceedings of the 9th International Conference on Computers Helping People with Special Needs (ICCHP 2004) [Google Scholar]
  • 4.Szeto AYJ, Allen EJ, Littrell MC. Comparison of speed and accuracy for selected electronic communication devices and input methods. Augmentative and Alternative Communication. 1993;9:229–242. [Google Scholar]
  • 5.Baljko M, Tam A. New York, NY, USA: ACM; 2006. Indirect text entry using one or two keys. pp. 18–25. In: Assets '06: Proceedings of the 8th International ACM SIGACCESS Conference on Computers and Accessibility. doi: http://doi.acm.org/10.1145/1168987.1168992. [Google Scholar]
  • 6.Venkatagiri HS. Efficient keyboard layouts for sequential access in augmentative and alternative communication. Augmentative and Alternative Communication. 1999;15:126–134. [Google Scholar]
  • 7.Koester H, Levine S. Learning and performance of able-bodied individuals using scanning systems with and without word prediction. Assistive Technology. 1994;6:42–53. doi: 10.1080/10400435.1994.10132226. [DOI] [PubMed] [Google Scholar]
  • 8.Koester H, Levine S. Effect of a word prediction feature on user performance. Augmentative and Alternative Communication. 1996;12:155–168. [Google Scholar]
  • 9.Lesher G, Moulton B, Higginbotham D. Optimal character arrangements for ambiguous keyboards. IEEE Transactions on Rehabilitation Engineering. 1998;6:415–423. doi: 10.1109/86.736156. [DOI] [PubMed] [Google Scholar]
  • 10.MacKay DJC, Ball CJ, Donegan M. Efficient communication with one or two buttons. In: Fischer R, Preuss R, Toussaint vonU, editors. Maximum Entropy and Bayesian Methods. Melville, NY, USA: American Institute of Physics, volume 735 of AIP Conference Proceedings; 2004. pp. 207–218. [Google Scholar]
  • 11.MacKay DJ, Ball CJ. Dasher's one-button dynamic mode—theory and preliminary results. Technical report, Cavendish Laboratory, University of Cambridge. 2006. Available at http://www.inference.phy.cam.ac.uk/mackay/abstracts/OneButton.html.
  • 12.MacKay DJ. Another one-button dynamic mode for Dasher: ‘two-click mode’. Technical report, Cavendish Laboratory, University of Cambridge. 2007. Available at http://www.inference.phy.cam.ac.uk/mackay/abstracts/OneButton2.html.
  • 13.Mead JMG, Cowans PJ, MacKay DJC. Efficient communication through the timings of one or two buttons. Technical report, Cavendish Laboratory, University of Cambridge. 2009. Available at http://www.inference.phy.cam.ac.uk/mackay/buttondasher2b.pdf.
  • 14.Ward DJ, Blackwell AF, MacKay DJC. New York, NY, USA: ACM; 2000. Dasher—a data entry interface using continuous gestures and language models. pp. 129–137. In: UIST '00: Proceedings of the 13th Annual ACM Symposium on User Interface Software and Technology. doi: http://doi.acm.org/10.1145/354401.354427. [Google Scholar]
  • 15.Ward DJ, MacKay DJC. Fast hands-free writing by gaze direction. Nature. 2002;418:838. doi: 10.1038/418838a. [DOI] [PubMed] [Google Scholar]
  • 16.Sensory Software International Ltd. The Grid 2: Version 2.4. 2008. Sensory Software International Ltd. URL http://www.sensorysoftware.com/
  • 17.Words+, Inc. EZ Keys™ User Manual. 2000. Words+, Inc. URL http://www.words-plus.com/website/products/manuals/manual.htm.
  • 18.Words+, Inc. EZ Keys™ XP Product Specification Sheet. 2004. Words+, Inc. URL http://www.words-plus.com/website/pdf_files/ezkeysxp.pdf.
  • 19.Sun Java Desktop System Documentation Team. GOK Manual V2.3. 2004. Sun Microsystems, Inc. URL http://www.gok.ca/usermanual.shtml.
  • 20.Horn EM, Jones HA. Comparison of two selection techniques used in augmentative and alternative communication. Augmentative and Alternative Communication. 1996;12:23–31. [Google Scholar]
  • 21.Mizuko M, Reichle J, Ratcliff A, Esser J. Effects of selection techniques and array sizes on short-term visual memory. Augmentative and Alternative Communication. 1994;10:237–244. [Google Scholar]
  • 22.Ratcliff A. Comparison of relative demands implicated in direct selection and scanning: Considerations from normal children. Augmentative and Alternative Communication. 1994;10:67–74. [Google Scholar]
  • 23.Wagner BT, Jackson HM. Developmental memory capacity resources of typical children retrieving picture communication symbols using direct selection and visual linear scanning with fixed communication displays. Journal of Speech, Language, and Hearing Research. 2006;49:113–126. doi: 10.1044/1092-4388(2006/009). [DOI] [PubMed] [Google Scholar]
  • 24.Mizuko M, Esser J. The effect of direct selection and circular scanning on visual sequential recall. Journal of Speech and Hearing Research. 1991;34:43–48. doi: 10.1044/jshr.3401.43. [DOI] [PubMed] [Google Scholar]
  • 25.Petersen K, Reichle J, Johnston SS. Examining preschoolers' performance in linear and row-column scanning techniques. Augmentative and Alternative Communication. 2000;16:27–36. [Google Scholar]
  • 26.Kilgarriff A. BNC database and word frequency lists. 1998. Available at http://www.kilgarriff.co.uk/bnc-readme.html. Accessed on 18 March, 2009.
  • 27.Wagner RA, Fischer MJ. The string-to-string correction problem. Journal of the Association for Computing Machinery. 1974;21:168–173. [Google Scholar]
  • 28.MacKenzie IS. Heidelberg, Germany: Springer-Verlag; 2002. KSPC (keystrokes per character) as a characteristic of text entry techniques. pp. 195–210. In: Proceedings of the Fourth International Symposium on Human Computer Interaction with Mobile Devices. [Google Scholar]
  • 29.ITU Radiocommunication Assembly. 2004. International Morse code, Recommendation ITU-R M.1677.
  • 30.Kristensson PO, Denby LC. New York, NY, USA: ACM; 2009. Text entry performance of state of the art unconstrained handwriting recognition: a longitudinal user study. pp. 567–570. In: CHI '09: Proceedings of the 27th International Conference on Human Factors in Computing Systems. doi: http://doi.acm.org/10.1145/1518701.1518788. [Google Scholar]
  • 31.MacKenzie IS, Soukoreff RW. New York, NY, USA: ACM; 2003. Phrase sets for evaluating text entry techniques. pp. 754–755. In: CHI '03 Extended Abstracts on Human Factors in Computing Systems. doi: http://doi.acm.org/10.1145/765891.765971. [Google Scholar]
  • 32.Wand MP, Jones MC. London, UK: Chapman & Hall/CRC; 1995. Kernel smoothing. [Google Scholar]
  • 33.Silverman BW. London, UK: Chapman & Hall/CRC; 1986. Density Estimation for Statistics and Data Analysis. [Google Scholar]

Articles from PLoS ONE are provided here courtesy of PLOS

RESOURCES