Effectiveness and usability of Scanning Wizard software: a tool for enhancing switch scanning

Abstract

Scanning Wizard software helps scanning users improve the setup of their switch and scanning system. This study evaluated Scanning Wizard’s effectiveness and usability. Ten people who use switch scanning and ten practitioners used Scanning Wizard in the initial session. Usability was high, based on survey responses averaging over 4.5 out of 5, and qualitative feedback was very positive. Five switch users were able to complete the multi-week protocol, using settings on their own scanning system that were recommended from the Scanning Wizard session. Using these revised settings, text entry rates improved by an average of 71%, ranging from 29% to 172% improvement. Results suggest that Scanning Wizard is a useful tool for improving the configuration of scanning systems for people who use switch scanning to communicate.

Background

Switch scanning allows people with severe physical impairments, who may also be unable to speak, to independently use a computer or augmentative and alternative communication (AAC) device. The impairments may stem from a variety of medical conditions, such as cerebral palsy, traumatic brain injury, muscular dystrophy, and neuromuscular diseases such as multiple sclerosis and amyotrophic lateral sclerosis, affecting hundreds of thousands of people in the United States. Physicist Stephen Hawking is perhaps the most well-known user of switch scanning.

While switch scanning affords spoken and written communication using only one controlled input movement, it is a slow method of text entry. A very fast user may achieve six words per minute (wpm) [1–4], while rates of 1 wpm and lower are common [3–6]. Despite its limitations, scanning is often the only alternative for individuals who cannot use other interfaces. Technologies such as eye gaze and speech recognition may not be usable for individuals with severe spasticity, poor head control, or limited verbal abilities. Direct brain interfaces, while promising, are still in the development stage [4,7].

The most common implementation of scanning is row-column scanning, which can be used with as little as one switch for input. Typical operation with one switch, often called automatic scanning, requires two switch hits to select an item from a two-dimensional matrix of letters, numbers, symbols, words, or phrases. Each row of the matrix, beginning with the first, is highlighted in turn until the first switch hit is made to select a row. Each column of the row is then highlighted in turn until the target is highlighted, when the second switch hit is made to select the target. In a two-switch alternative, called step scanning, the user hits one switch to move the highlight at their own pace, and a second switch to make a selection.

Variations on this theme include group-row-column scanning, which adds another level in which a group is first selected, followed by the rows then columns in that group, as well as manual initiation, which requires an additional switch hit to resume scanning after a selection is made. Figure 1 shows an example of a group–row–column layout.

Figure 1.

Example of group-row-column scanning display. It includes eight groups, scanned in following order: (1) titlebar, (2) message window, (3) pronouns, (4) helping verbs, (5) letters + prediction, (6) verbs, (7) chat, (8) prepositions. Once a group is selected, scanning continues through rows, and then columns, of that group.

Given the challenges of efficient text entry with switch scanning, product developers have implemented numerous features and configuration settings to allow for customization of scanning software with the goal of increasing text entry rate (TER). Some of the settings available in current scanning systems are shown in Table 1.

Table 1.

Some configuration options found in commercial scanning interfaces [3,5].

Category	Configuration option	Definition
Language features	Character prediction	One or more items in the matrix are dynamically updated based on which letters are most likely to be selected next
	Word prediction	One or more items are dynamically updated based on what word the user is most likely entering. Additional settings control the number of words in the prediction list, when the list is displayed, etc.
	Fixed words	One or more items contain fixed words or phrases
Item positions	Group layout	Positions of groups of items relative to each other
	Item layout	Positions letters or items relative to each other
Scan pattern	Number of levels	Nesting of levels in the scan pattern (e.g., group-row-column or row-column).
	Scan initiation (auto/manual)	After a selection, Manual requires a switch hit to initiate scanning, while Auto resumes scanning automatically
	Loop count	Number of passes through the columns in a row before returning to the row scan
Dead time	Scan message window	Include the message display window as an item in the scan pattern
	Scan title bar	Include the title bar as an item in the scan pattern
	Post-selection delay	Time delay after each selection before scanning resumes
Timing	Scan time	How long an item is highlighted for selection
	Extra delay	Delay added to the scan time for the first row or column
	Acceptance time	Length of time a switch must be activated before the activation is registered. Can reduce effect of bouncing

In addition to these settings, another major aspect of configuration is the layout and position of the items in the selection matrix. Each item has a specific scan distance, defined by the number of scan periods required to reach it. In a row–column matrix, items in the upper left corner have the shortest scan distance, while those in the lower right have the longest. Placing the most frequently used items in the locations with the shortest scan distance is a common strategy for enhancing TER [8–10].

Proper configuration of the features available within scanning systems can make a major difference in communication rate [3–6,11,12]. The timing parameters are a key factor in TER with one-switch automatic scanning, particularly the scan time, which defines the amount of time an item is available for user selection. As a simple example, if a person is capable of using a scan time of 1.0 s, but their system is set to 2.0 s, their TER will be roughly half of what it could be. One case study demonstrated how modifications to both item layout and scan time yielded an increase in TER of 321% for one individual [10], and the five individuals in another study showed differences of 20 to 25% when using different configurations [6].

The overall goal of this work is to establish an effective and efficient process for tailoring a scanning interface to a particular user. Our previous work detailed a manual method for enhancing performance and demonstrated its effectiveness, as the nine switch users in the study improved their TER by an average of more than 120% [3]. The next step was to make this method readily and efficiently usable by practitioners and switch users, which led to the development of the Scanning Wizard software.

Scanning Wizard

Scanning Wizard software guides switch users and practitioners through a series of tasks and generates individually tailored recommendations based on the data collected and principles of scanning optimization. There are four main parts to Scanning Wizard:

1. Switch configuration: verifies the user’s switch(es) is/are connected to the computer

2. Switch test: checks basic switch use by measuring response time to an onscreen prompt.

3. Scan test: checks basic scanning skill by presenting items for selection using a simplified scanning layout (with either letters or symbols).

4. Final recommendations: presents a set of recommendations for configuring the user’s scanning system to best meet their needs.

Generally, a practitioner and a switch user run through the wizard together. The switch and scan tests are each run at least once in a full run-through, although any test can be repeated as many times as desired. At the conclusion of the session, the team decides which recommendations to implement and then makes those adjustments to the user’s scanning system. Scanning Wizard runs as a web application within any browser, and is freely available at scanningwizard.com.

Switch configuration

The Scanning Wizard can make recommendations based on either one-switch or two-switch scanning. The configuration step (shown in Figure 2) verifies that the user’s switch is working and the switch interface is compatible with Scanning Wizard. For one-switch scanning, any switch interface that can emulate a left mouse click or space key can be used. For two-switch scanning, any switch interface that can emulate both a right and left mouse click or a space key and enter key can be used.

Figure 2.

Screenshot before and after successful switch configuration.

Switch test

The switch test is designed to determine whether the switch setup itself is meeting the user’s needs, independent of the more complicated task of making scanning selections. The switch test consists of three phases, testing the user’s ability to press a switch once (Phase 1), twice in succession (Phase 2), and three times in succession (Phase 3). Each phase consists of a three-trial practice session (which can be repeated as often as needed) followed by a test with eight trials, although users are free to exist a phase early or skip a phase entirely. Figure 3 shows the progression of a single trial in Phase 1.

Figure 3.

Prompt and response screenshots for a phase 1 (i.e., 1-hit) switch trial. The green response appears on switch release.

In each trial, the total time from initial prompt to final switch release is recorded, along with the duration of each switch press (i.e., the time elapsed between the switch being pressed and the switch being released). The number of switch hits is also recorded.

These data are analysed and presented in a report (see Figure 4). The most prominent aspect of the report is the Difficulty Score meter, which is used to recommend whether the current switch setup appears to be generally appropriate for effective use of scanning or whether the current switch setup may need to be re-evaluated and adjusted to provide better performance for the user. A Difficulty Score is calculated for each phase as the sum of the average trial time, standard deviation of trial time, and the average number of extra hits per trial, and then summed across all phases to get one overall score. A higher score suggests greater difficulty with using the switch, while a lower score indicates lower difficulty. The score is categorized into low, medium, and high difficulty zones (shown as green, yellow, and red, respectively, on the report), using an internal algorithm. The report also presents tables for summary data (Figure 5) and detailed trial data, as well as graphical timelines showing each switch press and release made during the test.

Figure 4.

Screenshot showing top portion of switch test report.

Figure 5.

Summary data and difficulty score, presented within a switch test report.

Scan test

The scan test (see Figure 6) evaluates the user’s ability to select items from a simplified scanning interface. The user is given a four-item practice session (which can be repeated as often as needed) followed by a test requiring the user to select about 25 items. Test sentences were written to be similar with respect to their length, reading level, average scan distance, and distribution of first-row and first-item selections. For any of the three built-in layouts (see Figure 7), there is no more than 10% difference in average scan distance between any two sentences. This similarity is crucially important to ensure that differences in performance are not strongly influenced by differences in the sentences.

Figure 6.

Screenshot of the scan test.

Figure 7.

Predefined keyboard layouts included with Scanning Wizard. Users can also define custom layouts.

The scan test supports the following settings found in commercial scanning systems: scan time, extra delay, scan initiation, loop count, and keyboard layout (see Table 1 for definitions). For users who already have a scanning system, the practitioner can choose settings that match the user’s existing system. Alternatively, the practitioner and user can choose to use one or more values suggested by the Scanning Wizard. For example, the scan time and extra delay values will be populated with recommended values based on the user’s switch test data.

An additional scan test setting, the item display setting, helps accommodate the test for individuals who have limited literacy and for those who are less familiar with a given letter layout. The default is all letters, which causes all keyboard items to be displayed as shown in Figure 6. The target letters only setting removes all key labels except for the target key, making it much easier to find the desired item in the layout. The target picture only setting removes all key labels and displays a smiley-face picture on the target key. This setting also removes the sentence from the top of screen, so the task becomes one of selecting a series of smiley-face items from the keyboard.

For each item selected during the scan test, speed and accuracy data are recorded, including the following:

• Trial time: how long it took to select the item, in seconds.

• # of errors: the number of scanning errors that occurred in this trial. See Table 2 for definitions of the types of errors that are counted.

• Switch press and release events are also recorded as timestamps, although these data are not currently presented in the report.

Table 2.

Scanning errors tracked during Scanning Wizard’s scan test.

Scanning error	Definition
No row	Occurs when the scanner moves through all rows without making a selection
Just before correct row	Occurs when the incorrect row immediately before the correct row is selected
Just after correct row	Occurs when the incorrect row immediately after the correct row is selected
Other incorrect row	Occurs when an incorrect row that is not immediately before or after the correct row is selected
Right row, no item	Occurs when the correct row is selected, but the scanner is allowed to move through all items in the row without making a selection
Right row, just before item	Occurs when the item immediately before the correct item is selected
Right row, just after item	Occurs when the item immediately after the correct item is selected
Right row, other item	Occurs when an incorrect item in the correct row is selected, but it is not immediately before or after the correct item

These data are analysed and presented in a report (see Figure 8). Like the switch test, the report gives a difficulty score, categorized into low, medium, and high zones. The difficulty score is the same as the overall error rate for the test, i.e., the total number of scanning errors as a percentage of correct item selections. Note that this counts all errors that do not result in incorrect item selections, such as selecting the wrong row and allowing the scanner to loop. The report also includes summary statistics across all items in the test, detailed item-by-item data, as well as a measure of TER. TER is the number of correct items entered over the total test time. The words per minute calculation assumes five items per word.

Figure 8.

Screenshot showing top portion of scan test report.

In order to visualize the impact of errors on performance, the scan test report provides a comparison of the actual time to complete the test, compared to the estimated optimal time for the same settings. Optimal time approximates the fastest time possible to select the entire text, based on the current scan settings. This assumes that no errors are made and that each switch hit requires 1/2 of the scan time. An example is shown in Figure 9.

Figure 9.

Actual total test time vs optimal test time.

Final recommendations

The final recommendations report summarizes the overall results for the current switch and scan tests and makes suggestions for configuration settings that may benefit the user. The overall approach is to present recommendations to improve both accuracy and efficiency, but the emphasis depends on the user’s data. For example, if the user’s scan difficulty score is above 35, Scanning Wizard suggests focusing on accuracy in the short term, in order to enhance the user’s ability to select the correct item at the very first opportunity. Accuracy-focused recommendations include slowing down the Scan Time, using a familiar layout, and/or using manual scan initiation. If the user’s scan difficulty score is below 20, Scanning Wizard’s recommendations will emphasize efficiency improvements i.e., helping the user select items as quickly as possible, without sacrificing accuracy. For example, a frequency-based layout is 65% more efficient than an alphabetic layout. So, if the user can select from a frequency layout just as accurately as an alphabetic one, TER will improve dramatically.

Scanning Wizard makes recommendations for six settings based on the user’s data. These settings and a basic overview of how they are recommended are provided below:

• Switch setup: Switch difficulty scores in the medium or high zone produce a recommendation to revisit the current switch setup, with links provided for further information about that process.

• Scan time: the long-term scan time recommendation is based on the user’s measured switch press time, using the 0.65 rule [13]. The short-term recommendation may differ from that depending on the error rate measured in the scan test.

• Extra delay: a non-zero delay is recommended based on the recovery times measured during the switch test. Recovery time is the time from one switch release to the next switch press, when making two or three switch hits in a row.

• Scan initiation: Auto initiation is recommended, unless scan test error data are in the high difficulty zone. In that case, Manual initiation is recommended, as a strategy for reducing errors.

• Loop count: the default is to recommend a loop count of 1. If a high number of right row, No item errors are observed, a loop count of 2 may be recommended.

• Keyboard layout: the long-term recommendation is to use a frequency-based layout. In the short term, if scan test error data are in the high difficulty zone, we recommend using the layout that is most familiar to the user, as a strategy for reducing errors.

Scanning Wizard also makes recommendations that are not directly based on the user’s data but do incorporate basic principles that apply to many scanning situations:

• Reduce dead time: typical sources of unnecessary time in the scanning process are identified, with suggestions for removing them. If scan test error data are in the high difficulty zone, this is presented as a longer-term goal, once errors have been reduced.

• Prediction features: research-based guidelines for successful word prediction setups are presented, including list length, list location, and ensuring adequate list search time.

• Scan pattern: general guidelines for when to use row–column and block-row–column patterns are presented.

• Core words: the use of groups of pre-defined core words is left to the user’s discretion. If combined with spelling, we recommend putting the letters and any word prediction lists in the first block, followed by blocks of core words, for greater efficiency.

Research objectives and hypotheses

One purpose of this study was to evaluate the effectiveness of Scanning Wizard at enhancing performance for people who use switch scanning. Specifically, we hypothesized that the revised settings resulting Scanning Wizard’s recommendations would significantly improve users’ TER compared to baseline. We did not have a specific hypothesis regarding the effect on errors across all subjects, as this was expected to vary depending on baseline performance.

A second purpose was to determine the usability and utility of Scanning Wizard, based on user feedback as well as the time required for use. Our target for required usage time was 30 min for a single pass through all activities in the wizard.

Methods

Study design

This study used a longitudinal ABA design to compare the performance of switch users with the scanning settings that they use every day (condition A, baseline settings) to their performance with the settings recommended by Scanning Wizard (condition B, revised settings). The protocol was also designed to provide meaningful data from the initial session alone (see baseline measurement and Scanning Wizard session). Screen-capture video was used in all activities to allow for thorough review of all sessions.

Subjects

The protocol was approved for human subjects, and all participants provided informed consent. Subjects included users who either already used switch scanning or were being evaluated for the use of switch scanning, as well as assistive technology practitioners with experience working with switch users. Practitioners were recruited primarily from disability-related organizations in the New York City and Detroit metropolitan areas. Interested practitioners then assisted with recruiting switch users from their client base. Ten practitioners and ten switch users participated. Practitioners included three men and seven women, working in a variety of practice settings.

As shown in Table 3, nine of the switch users regularly use switch scanning to access their AAC system (eight using single switch, and one using two switches), while one was being evaluated for provision of his first switch scanning system. Of the nine regular scanning users, eight had used scanning for at least five years, and the ninth had used it for two years. All but one subject used letter-by-letter spelling; most subjects also had word prediction and core word items available. Subjects’ devices were six different models from four different manufacturers. Five subjects activated their switch(es) with head motion; four used hand activation, and one used a cheek twitch. Self-reported satisfaction with their AAC systems averaged 3.8 on a scale of 1 (very dissatisfied) to 5 (very satisfied). For all subjects, a practitioner had assisted in establishing the setup of their current AAC system.

Table 3.

Key characteristics of switch user participants and their baseline systems.

ID	Sex	Age	Dx	Scanning system	Letter layout	Prediction	Scan time (s)
001	M	14	CP	Dynavox Series 5	Alphabetic	None	2.35
002	M	38	CP	Tobii I15 Communicator 5	QWERTY	11-word	1.00
003	M	45	DMD	Dynavox Vmax+	QWERTY	5-word	0.83
004	M	50	SMA	EZKeys	Freq-based	6-word	0.12
005	F	32	CP	EZKeys	Freq-based	6-word	0.75
006	F	56	CP	iPad, TouchChat	QWERTY	5-word	1.90
007	F	6	CP	Dynavox Compass	Symbols (12)	None	2.00
008	M	53	CP	None	–	–	–
009	F	30	Enc	Tobii I15	Word Power Freq-based	3-word	1.50
010	F	20	CP	PRC Accent 1000	Unity	11-word	Step scan

CP: cerebral palsy; DMD: Duchenne’s muscular dystrophy; SMA: spinal muscular atrophy; Enc: encephalopathy.

Baseline measurement (condition A1)

In the first session of the study, each subject’s baseline performance was measured on the subject’s original (pre-intervention) scanning system and configuration. They completed one two-sentence transcription task. Subjects were free to select any prediction items, single-word items, Unity symbols, etc. that were present on their system, at their own discretion. For the subject who used single-meaning symbols only (007), the task was modified to ask her to select a sequence of 10 symbols from her system, rather than creating sentences. All subjects were allowed to make errors and decide whether or not to fix them.

Scanning Wizard run-through

Also in the first session, the practitioner and the switch user ran through the Scanning Wizard software together. None of the practitioners or switch users had used Scanning Wizard prior to this study. Practitioners were asked to do a brief run-through with it on their own prior to the initial session, to gain some basic familiarity; seven of the nine practitioners did this, while the other two (for subjects 007 and 009) had not used it at all prior to the initial session.

Both the switch user and the practitioner completed a short survey after the run-through. The questions for switch users asked whether they understood how to do the switch and scan tasks, and whether using Scanning Wizard took a reasonable amount of time and effort. The practitioner survey had questions on whether Scanning Wizard was pleasant, understandable, worth the effort, took an acceptable amount of time, and whether they were likely to use it in the future. All questions were Likert-type, with a scale anchored by 1 = strongly disagree and 5 = strongly agree.

Defining and implementing revised settings

Based on the recommendations provided by Scanning Wizard, the practitioner and switch user decided what changes to make to the switch user’s scanning system. These changes were listed on a worksheet, then the practitioner or a researcher actually made the changes to the user’s own scanning system prior to the next session.

Intervention phase (condition B)

In four weekly sessions, the switch user completed a two-sentence transcription task. The text used in each test was unique, but all sentences were equivalent in terms of letter frequency and reading level. All subjects used their new setup during daily life for the duration of the Intervention Phase.

Reversal phase (condition A2)

After four weeks of using the revised settings, the baseline settings were restored to each subject’s system. They then completed a two-sentence transcription task.

Post-study survey questions

At the end of the study, switch users were asked to complete the following questionnaire, using the same 1–5 Likert scale:

1. Overall, I now prefer the new settings to my original ones.

2. I did not like the new settings at first.

3. I think I typed faster with the new settings.

4. I would like to keep some of the new settings to use permanently.

Dependent variables

Text entry rate

Text entry rate was measured for each transcription test as the number of correct characters present at the end of the test, divided by the total time for the test. The total time included all time from start to end of transcription, including any time for fixing incorrect selections. This character/seconds measure was converted to words/minute (wpm) by assuming five characters/word.

Scanning errors

A scanning error was counted any time the user selected an incorrect item and for any timing error during scanning. We recorded the total number of scanning errors in each transcription test. Total scanning errors included the following error types:

1. Failing to press the switch when the target group, row, or column was highlighted;

2. Pressing the switch too early (before the intended target);

3. Pressing the switch too late (after the intended target).

Selection errors are a subset of the total scanning errors that resulted in the wrong item (i.e., column) being selected. Total selection errors included all selection errors in the session, while Net selection errors were the number of selection errors that were not corrected by the end of the task. All errors were reported as a percentage of the correct selections made in the test.

Scanning Wizard usage

During the Scanning Wizard run-through, we recorded the path users took through the wizard as well as the time required for each task in the software and for a single pass through all activities (which includes the time required for switch configuration, one switch test, one scan test, and reviews of the switch, scan, and final recommendation reports). We also validated the accuracy of Scanning Wizard’s workflow and recommendations, to ensure that this newly-developed software was performing correctly.

Data analysis

Paired t-tests (with an alpha of 0.05) were conducted to examine the main effect of baseline vs. revised settings for the following variables:

1. TER (correct words per minute)

2. Total scanning errors (%)

3. Total selection errors (%)

4. Net selection errors (%)

In each statistical test, baseline (condition A) was calculated from the average of session A1 and A2 results for each subject and revised (condition B) was the results from each subject’s fourth session with the revised settings (B4). Additionally, data from each individual subject was examined for a clear increase in TER (at least 20%) with revised settings relative to baseline, and a full reversal back to baseline performance when settings were reverted to their original values. Full reversals add confidence that any significance in the group statistics was truly due to enhancement provided by the revised settings rather than extraneous or random factors.

For the Scanning Wizard usage time, the times for a single run-through were averaged and compared against the target of 30 min. A t-test (with an alpha of 0.05) was used to determine if the observed times were significantly less than 30 min.

Responses to the questionnaire were analysed using Nielsen’s guidelines for 5-point Likert scales [14], to accommodate for subjects’ tendency to be polite when responding to these types of questions. Responses to positive questions were considered significant if the mean response was greater than 3.6 (or lower than 2.4 for a negative question), based on a one-sample t-test (with an alpha of 0.05).

Results

Trajectory of subjects through the protocol

Of the ten switch users, five completed the full multi-week protocol. Five completed only the initial session (baseline measurement and Scanning Wizard run-through): two because Scanning Wizard did not recommend changes to their scanning setup (004 and 005), one who did not have his own scanning system yet (008), and two who were unable to attend additional sessions (007, 010). The sections below first present the results for all ten switch users in the initial session, followed by the results for the five switch users who completed the full protocol.

Baseline text entry performance

Figure 10 shows the baseline TERs from the initial session. TER averaged 3.11 wpm with a standard deviation of 3.95, a maximum of 12.94, and minimum of 0.15. Six of nine subjects had TER below 3 wpm. Subject 008 did not have a baseline TER measure, because he did not have his own scanning system yet. Total scanning errors averaged 52%, with a standard deviation of 99, and range of 0–312%. Subject 007 had the maximum scanning error rate of 312%; with that score removed, the group average was 19.3%. Total selection errors averaged 12.7%, with a standard deviation of 23.6, and range of 0 to 69.2%. Subject 007 again had the maximum value, and the group averaged 5.6% selection errors with her score removed.

Figure 10.

Baseline text entry rates for participants, using their original scanning setup.

Scanning Wizard usage

In the Scanning Wizard run-through, all ten switch users completed a single switch test; there were no repeated switch tests. For the scan test, four users completed only one; five users completed two scan tests; and one user (007) did a set of five scan tests across multiple sessions within her school setting. All users except 007 completed all of their Scanning Wizard activities within a single session. Scanning Wizard performed correctly in all sessions; all reports were checked for accuracy, and no errors were found.

The time required for a single pass through all wizard activities averaged 1923s across all nine subjects, with a 95% confidence interval of [1487, 2358], ranging from a low of 1432s to a high of 3501 s. This suggests that a typical run-through lasts a bit longer than 30 min, and may last as long as an hour. One run through the switch test took an average of 358 s (about 6 min), while each scan test required 578 s (almost 10 min) on average.

Initial session survey responses

For the initial session questionnaire, answered by 10 switch users and 10 practitioners, responses to all 8 questions were significantly greater than the target of 3.6. The average for the switch users’ responses across their three questions was 4.67 out of 5 (SD = 0.57), indicating that they understood how to do the switch and scan tasks, and that they felt that using Scanning Wizard took a reasonable amount of time and effort. Similarly, the average for practitioners’ responses across their five questions was 4.54 out of 5 (SD = 0.61), indicating high ratings for ease of use, amount of time required, and value of using Scanning Wizard.

Qualitative comments from practitioners

Qualitative comments from practitioners indicated that they liked the concept underlying Scanning Wizard, with comments including:

This quantifies things and justifies things that you might want to do with respect to settings. The alternative is to just eyeball it with my experience.

Definitely helped give objective information to help decision-making in this case.

Additional “likes” included the simple and clear interface, plain language that “anybody could understand,” the recommendation of appropriate scan time matched to the user, and the ability to collect baseline performance and track performance over time.

For feature requests, several practitioners asked for a two-switch step scanning mode for the scan test, and that feature has been added (Subject 010 uses two-switch step scanning, and used the new two-switch mode during her participation in the study.) Other major feature requests included: more keyboard layouts, to make it easier to match the test layout to the user’s own layout; additional audio feedback; choice of colours and contrast; and help for inexperienced practitioners in applying the recommendations. Development of those additional features is ongoing.

There were no major usability problems or points of confusion reported by practitioners. Several noted that they would like a deeper understanding of all the information presented by the wizard, in order to interpret and discuss with clients confidently; these practitioners felt that this clarity would develop with more experience, given that they had only used the wizard once or twice in this study. One practitioner had difficulty with the switch interface they used to connect the switch to the computer running Scanning Wizard; this generated extra switch hits initially, but they fixed the problem by reconnecting the switch interface snugly.

Characteristics of modified configurations

Table 4 summarizes the types of changes made to the scanning setups for the five subjects who continued through the full protocol. Letter layouts were modified for all five subjects; three were originally using a QWERTY layout, one an alphabetic layout, and one a quasi-frequency layout that was not optimally efficient. The other change made for all five subjects was to reduce dead time; this typically took the form of moving the message window or other infrequently-used item from the top of the scan pattern to a location further down, thus allowing the scan to reach the letters and words sooner.

Table 4.

Types of changes made for subjects who received adjustments.

Settings category	001	002	003	006	009
Language features			✓
Letter layout	✓	✓	✓	✓	✓
Scan pattern		✓		✓	✓
Less dead time	✓	✓	✓	✓	✓
Faster timing				✓	✓

Subject 006 provides a more in-depth example of the changes made for one particular individual. Her scores for both the switch and scan test were both in the low difficulty zone, so the challenge was to enhance her already highly-skilled performance. Based on her switch press time, her scan time was reduced from 1.9 to 1.4 s per scan period. As shown in Figure 11, her familiar-but-inefficient QWERTY keyboard layout was changed to a frequency-based arrangement, saving almost two scan periods per letter. Notable dead time was also identified in her system, as scanning cycled through the message window, the delete key, and a “Back” button before reaching any actual text items. At 1.9 s per scan, this applied almost 6 s of extra time before any word or letter selection could be made. The delete button was moved to a lower row location, thus saving 1 scan period per selection. The practitioner wanted to move the “Back” button as well, but was unable to do so within 006’s AAC app. Another source of dead time was the 0.95 s pause after each item selection, due to the auto tap setting within iOS switch control. It was not possible to remove this entirely, however, as it provided her only way to access the iOS scanner menu, but it was reduced to 0.75 s. (While the iOS Long Press setting might have been a more efficient way to access the scanner menu, this would have required an iOS upgrade to a version that her iPad was not designed to handle.) Despite the fact that her system provided some limitations to implementing all of the recommendations, these changes yielded a 55% increase in TER for 006.

Figure 11.

Original (left) and revised (right) layouts for subject 006 (Note: revised layout includes word prediction cells in the top row, but they are empty in this screenshot.).

Effect of revised settings for text entry

Overall results of our statistical analyses are shown in Table 5. All five subjects who completed the full protocol had higher TER with the new settings as compared to original. The average improvement was 70.9% and ranged from 28.6% (002) to 171.8% (003). The 95% confidence interval was [−0.36, 142.2], so the group TER improvement was not quite statistically significant at the p = .05 level (p = .051). The means for the error-related measures were quite similar for baseline as compared to revised settings.

Table 5.

Statistical analyses for the five subjects who used revised settings.

Variable	Mean		p value
	Baseline	Revised
Text Entry Rate (wpm)	1.42	2.35	.051
Total Scanning Errors (%)	25.85%	25.79%	.994
Total Selection Errors (%)	6.98%	4.23%	.313
Net Selection Errors (%)	3.89%	3.76%	.254

Figure 12 shows the individual ABA results for each subject’s TER. The reversal patterns were fairly symmetrical, and all subjects except 002 met the criterion for at least a 20% change in TER for each change in scanning setup.

Figure 12.

Baseline (A1), intervention (B4), and reversal (A2) TER for each subject who received adjustments.

Post-study survey responses

For the post-study questionnaire, all three positive questions had responses significantly above 3.6, and the negative question (“I did not like the new settings at first”) received an average of 2.0, showing slight disagreement. Notably, all five subjects gave the highest rating of 5 to the question of “Overall, I now prefer the new settings to my old ones,” and all subjects kept the new settings to use for their daily life at the end of the study.

Discussion

Effectiveness of Scanning Wizard

These results suggest that use of Scanning Wizard can increase a user’s TER, because it helps determine appropriate adjustments to the scanning setup. The suggestions made by Scanning Wizard are fairly straightforward, such as suggesting the use of a frequency-based letter layout. However, these known principles are not always put into practice, as none of the five experienced subjects were using an appropriate layout at the start of the study. Scanning Wizard’s recommendations also address areas that may not be as familiar to practitioners or switch users, so they act as a checklist for making sure that all settings are considered and adjusted appropriately.

For the other five subjects who did not complete the full study and did not change their scanning settings, the use of Scanning Wizard in the initial session still appeared to provide some value. For 004 and 005, Scanning Wizard confirmed that their setups were already providing solid performance that was unlikely to improve with settings adjustments. For 008, the Scanning Wizard session allowed this individual to compare the use of two different switches as part of his initial exploration of single-switch scanning. The team learned that one of the switches provided significantly faster and more accurate performance, providing a good foundation for moving forward with that switch. And for 007, a young emergent switch user, the Scanning Wizard session demonstrated her ability to understand and use row–column scanning, whereas before her experience had only been with linear scanning.

Text entry rates

It is worth noting that the baseline TERs for this group had a very large range from 0.15 to 12.94 wpm. The 12.94 wpm for 004 may well be a world record, as it is twice as fast as the maximum we have seen in previous studies [3,4,15]. This individual has been working to share his techniques with other switch users; many of these follow the same principles as Scanning Wizard’s recommendations, but 004 somehow can use an incredibly fast scan time that may not be possible for the vast majority of people.

Usability of Scanning Wizard

The high usability ratings observed in this study are encouraging, particularly since participants had never used Scanning Wizard prior to the study and were given no particular training in its use. The amount of time required for a Scanning Wizard session was somewhat longer than our target of 30 min. However, both switch users and practitioners showed strong agreement with the statement that Scanning Wizard takes “a reasonable amount of time and effort” (avg 4.4 [3.9, 5.0] CI for switch users; avg 4.6 [4.1, 5.1] CI for practitioners). Overall, participants understood and appreciated the purpose of Scanning Wizard and used it appropriately.

Limitations

A primary limitation of this study is the fairly small number of switch users who completed the full multi-week protocol. The 70% average TER improvement is encouraging, and every one of the five subjects enjoyed an increase in TER, but a larger number of subjects would allow for stronger conclusions. Results from our previous study, which involved nine subjects, mitigate this limitation somewhat; using the same basic “recipe” for enhancing TER as was used in this study, those individuals averaged a significant 120% improvement in TER. The two studies combined provide confidence in the merit of this approach.

This work is most directly applicable to individuals who require fairly complex scanning layouts and who have significant prior experience with purposeful switch use. For example, recommendations related to word prediction and letter-based layouts are most relevant to people who spell. However, use of Scanning Wizard can still be of value to people who are in the early stages of language development and switch use. Those characteristics describe subject 007, and, as noted above, her Scanning Wizard information revealed useful abilities that had not been noticed before. It also provided a baseline performance score to document her current switch use, which gives the team a way of measuring and demonstrating improvements over time.

A limitation in the current Scanning Wizard approach is that there is no direct connection to the user’s actual AAC or computer access device. While users can and do connect their own switch for use with Scanning Wizard, the wizard has no direct knowledge of how the existing scanning system is configured, and it cannot directly modify the existing scanning system with recommended changes. This means that the practitioner or other team member is responsible for the tasks of setting up Scanning Wizard to match any existing system and of implementing any recommendations made by the wizard. In our study, practitioners generally set up the scan test appropriately to be a valid representation of the user’s own system, but in 005’s case, there was a definite mismatch in her scan test which made it harder to interpret the meaning of her data. The step of implementing the recommendations appeared to be a manageable process for the practitioners in this study, and we are also available as resources to help with that. Nonetheless, a more streamlined integration would be a desirable improvement.

Future work

We plan to continue to develop new features for the Scanning Wizard software, particularly to support more emerging switch users. Informally, the practitioners involved in the study indicated to us that many of the switch users they work with are young and pre-literate. Accommodating this population more thoroughly within Scanning Wizard is therefore an important goal for us. We also plan to help practitioners and switch users build awareness of how to use the tool most effectively for their particular situation.

Conclusions

These results suggest that Scanning Wizard can be a useful and usable tool for improving the configuration of scanning systems for people who use switch scanning to communicate. These results are consistent with our previous work [3] demonstrating that the performance of scanning users can be improved substantially by systematic adjustments to the scanning system itself.

IMPLICATIONS FOR REHABILITATION.

• Some individuals with severe physical impairments use switch scanning for spoken and written communication.

• Scanning Wizard software helps scanning users improve the setup of their switch and scanning system.

• This study demonstrated high usability of Scanning Wizard (with 10 switch userpractitioner teams) and increased text entry rate by an average of 71% (for five switch users).

• Results suggest that Scanning Wizard is a useful tool for improving the configuration of scanning systems for people who use switch scanning to communicate.