A model to estimate the optimal layout for assistive communication touchscreen devices in children with dyskinetic cerebral palsy

Abstract

Excess involuntary movements and slowness of movement in children with dyskinetic cerebral palsy often result in the inability to properly interact with Augmentative and Alternative Communication (AAC) devices. This significantly limits communication. It is, therefore, essential to know how to adjust the device layout in order to maximize each child’s rate of communication. The aim of this study was to develop a mathematical model to estimate the information rate in children with dyskinetic cerebral palsy and to determine the optimal AAC layout for a touchscreen tablet that results in enhanced speed of communication. The model predicts information rate based on button size, number, spacing between buttons and the probability of making an error or missing target buttons. Estimation of the information rate confirmed our hypothesis of lower channel capacity in children with dyskinetic cerebral palsy compared to age-matched healthy children. Information rate increased when the AAC layout was customized based on the optimal parameters predicted by the model. In conclusion, this study quantifies the effect of motor impairments on communication with assistive communication devices and shows that communication performance can be improved by optimally matching the parameters of the AAC touchscreen device to the abilities of the child.

I. Introduction

Approximately 500,000 children in the U.S. with complex communication needs require assistive technology in order to communicate or participate in school [1]. 20–60 % of these children may require computer-based augmentative and alternative communication (AAC) devices such as adapted keyboard or programmable touch screens [1], [2]. Children with dyskinetic cerebral palsy (CP) or other movement disorders affecting the arm are even further impaired in their ability to use AAC devices to communicate with parents or friends and participate in school. Children with dyskinetic CP often exhibit a combination of excess involuntary movements and slowed voluntary movements that together interfere with many aspects of function [3]-[5].

These motor deficits can be particularly detrimental to the use of AAC devices. Dyskinetic CP describes a subtype of CP where dystonia represents the dominant impairment, though other deficits may occur such as chorea, choreoathetosis, and bradykinesia [6], [7]. The abnormal movements in dyskinetic CP are particularly frustrating due to the occurrence of normal or near-normal intelligence in most cases [8], [9], so that a child may become isolated or locked-in without the ability to express thoughts, emotions or needs. Information rate (i.e. communication rate) is acknowledged to be a primary determinant of success in AAC [10]. Normal speech produces 150–250 words per minute, while AAC produce only between 2–15 words per minute [11], [12], which can considerably decrease in children with dyskinetic CP. The consequences of low information rate are striking, resulting in a very limited amount of words produced per day, such that the expression of cognitive and social skills for these children can be profoundly limited by their motor impairments. Therefore, the design of AAC device layout becomes essential to maximize the information rate.

There is still a significant lack of systematic studies in AAC that investigate the motor ability to activate buttons in children with movement disorders [13], [14]. Current methods for programming the layouts consist primarily of trial-and-error. The smallest button the child can press with reasonable accuracy generally determines the physical characteristics of the AAC layouts, without regarding to measurement of speed of movement and rate of successful buttons contacts. This approach may unnecessarily limit the information rate. For many children, any advantage of an increased number of buttons would be offset by an even greater increase in the time required to accurately contact the desired button. Conversely, use of a small number of larger buttons could mean that any advantage due to increased speed of movement would be offset by an even greater decrease in information rate due to the need to type longer sequences of buttons to express different words.

Sanger and Henderson proposed a quantitative measurement to estimate the optimal AAC layout that can result in enhanced speed of communication in children with motor impairments [15]. They proposed a mathematical model to estimate information rate and channel capacity for each child with motor impairments while using an AAC device. The model exploited the foundations of information theory [16], the trade-off between speed and accuracy in human movements [17], and the cognitive processes during multiple choice tasks [18]. Inspired by information theory [16], Fitts’ Law states that there is a trade-off between speed and accuracy that represents a limiting factor in the ability to interact with AAC devices such that smaller buttons lead to slower movement when accuracy must be maintained. Fitts’ mathe matical relation has been replicated in many different tasks and confirmed both in healthy subjects and in participants with neurological disorders including childhood dystonia and children with dyskinetic CP [19]-[24]. Moreover, there have been many previous studies examining the influence of speed-accuracy trade-off in human-computer interaction [19], [25]. Hick’s Law states that the reaction time prior to the initiation of movement increases with the number of button choices [18], [26]. The increased reaction time in Hick’s Law is due in part to the need for visual search in order to identify the correct button [27], or the need for short-term memory storage of the location of multiple buttons. Based on the model proposed by Sanger and Henderson [15], reprogramming AAC devices with each child’s predicted optimal layout parameters significantly improved information rate in a subset of children when compared with ad hoc programming by professionals. No child worsened performance with the reprogrammed layout. The study showed that measurement of movement and reaction time as a function of the AAC layout parameters can predict information rate and channel capacity with possible improvement of rate of communication in a subset of children with motor impairments. Unfortunately, 5 of the 10 children tested did not benefit from the optimization, highlighting the need to explore other AAC layout parameters that may result in broader applicability.

In recent years, planning and control of goal-oriented aiming movements have been described within the framework of decision-making under risk. In typical decision-making, the subject must take into account not only the uncertainty of motor outcome after selecting the plan, but also the rewards or costs of any outcomes that may occur [28]-[31]. Studies have shown that when healthy young adults perform pointing movements to a target with a penalty region around it, they shift their mean endpoints in response to changes in penalties and location of the penalty region relative to the target region [32], [33]. Particularly, a recent study showed that movement time in Fitts’ Law emerges not only from the accuracy constraints of the task, but also depends on the probability and perceived cost of error for missing the targets [24]. Subjects were asked to touch targets on a screen with different costs for missed targets. Authors manipulated the probability of error by comparing children with dystonia, who are characterized by increased intrinsic motor variability [22], to typically developing children. The results showed a strong effect of the probability and cost of error on the Fitts’ Law relationship characterized by an increase in movement time as cost and probability of error increased. These studies suggest that the cost of activating the unwanted button would affect the AAC user’s performance, and this would be exacerbated with increased probability of making errors due to motor impairments in children with dyskinetic CP. Activating an unwanted button requires two additional movements: one to delete or otherwise reverse the incorrect button press, and another to achieve the correct button press. Missing the target button only requires a single additional movement to achieve the correct button press. Consequently, it is suggested that spacing between buttons and the effects of errors and corrective movements may be important parameters for AAC layout programming that need to be incorporated in the model to estimate information rate and channel capacity.

This study extends our previous work to address several issues that emerged [15]. Previous work did not account for the probability or cost of errors, nor did it estimate the effect of button spacing on probability of error. Previous work did not separate the effects of sensory from motor impairments, and this was particularly true of tasks requiring visual search to identify the correct target among a field of possible targets. Previous work did not separately assess the effect of button size and number of buttons, since the number of buttons was chosen to be the largest number that could fit on the screen for each chosen size. These limitations may explain the poor prediction and optimization of communication rate in half of the tested children.

The aim of this study is to test the hypothesis that including the button spacing, probability of making an error or missing target buttons, and interactions between different parameters in the information rate model will result in greater prediction accuracy and broader applicability of this method. We anticipate that the new model will differentiate performance between children with CP and healthy controls and determine the most relevant features of the AAC touchscreen devices that affect communication rate for children with dyskinetic CP. We hypothesize that the estimated optimal layout parameters for each child will result in enhanced speed of communication.

The availability of the latest generation of touchscreen tablets has changed the concept of using augmented and assistive communication devices in children with speech and motor impairments [34]. These devices are extremely portable, easy to manage, and the screen layouts are relatively simple to program. Therefore, in this study we aimed to use the Apple iPad^® which provides a flexible general-purpose AAC interface at much lower cost than previously available devices.

II. Methods

A. Subjects

Ten children with a clinical diagnosis of dyskinetic CP and ten age-matched children with typical development (TD) participated in the current study (14.5 years old ± 3.5 SD). Dystonia was the primary impairment of the upper extremities in one or both hands for children with CP, but other less significant motor impairments included spasticity, weakness, ataxia, or dyspraxia. Children with CP were not all dependent upon AAC devices to communicate, although two used touchscreen devices to write and read. One child with CP used an AAC touchscreen device for his school homework. All children with CP were recruited from the movement disorders clinic at Children’s Hospital Los Angeles. Children were rated on the Barry-Albright Dystonia scale (BAD) and the dissociated movements subscale (section A) of the Quality of Upper Extremity Skills Test (QUEST) for the arm used for the task [35], [36]. Cognitive function was assessed with the Test of Nonverbal Intelligence TONI-4 and children were excluded if the age-equivalent score was less than 5 years old [37]. Subject characteristics are outlined in Table I. Children’s parents gave written informed consent for participation, and all children gave written assent. Authorization for analysis, storage, and publication of protected health information was obtained from parents according to the Health Information Portability and Accountability Act (HIPAA). The study protocol was performed in accordance with the Declaration of Helsinki.

TABLE I.

Characteristics of Participants & Channel Capacity (CCp)

	Age	Sex	Dominant Arm	Arm used for the task	CCp HL0 (bits/sec)	CCp HL1 (bits/sec)	Left Arm BAD score	Right Arm BAD score	QUEST	TONI-4 Index Score
CPI	11	M	Right	Right	3.28	2.56	2	2	75.0	95
CP2	11	F	Left	Right	0.32	0.79	1	3	22.2	61
CP3	10	M	Left	Right	4.01	5.47	1	3	25.0	112
CP4	19	M	Right	Left	2.62	3.71	3	3	53.1	97
CP5	15	M	Right	Left	3.07	6.24	1	1	90.0	89
CP6	15	M	Left	Right	4.45	4.74	3	3	56.3	106
CP7	17	M	Right	Left	4.40	5.87	1	1	87.5	82
CP8	11	M	Left	Right	1.93	2.65	2	3	48.4	90
CP9	19	F	Left	Right	3.39	4.25	3	3	62.5	77
CP10	17	M	Left	Right	0.46	1.71	3	3	18.8	62
TD1	11	F	Right	Right	5.32	7.32
TD2	11	M	Right	Right	2.80	5.61
TD3	10	M	Right	Right	7.76	9.13
TD4	19	M	Right	Right	5.62	9.49
TD5	15	M	Right	Right	5.86	6.25
TD6	15	M	Right	Right	4.99	9.56
TD7	17	M	Right	Right	6.16	9.38
TD8	11	M	Right	Right	5.19	8.48
TD9	19	F	Left	Left	5.68	8.19
TD10	17	M	Right	Right	5.20	8.50

B. Apparatus

Testing consisted of two tasks on separate visits. Task 1 was used to find parameters for each individual child that determined a mathematical model of the relationship between button number, size, spacing and information rate for that child while using a touchscreen device. Task 2 was used to test whether the predicted optimal touchscreen layout led to improvement of communication rate.

Participants sat in a chair or their own wheelchair in front of a table whose surface height was adjusted at the midpoint between the hip and the xiphoid process. They placed the hand that was not used for the task on their lap. An iPad^® (9.7-inch, 1024-by-768-pixel resolution, Apple Inc, Cupertino, CA, USA) was located on the table in landscape mode in front of the participants at a distance that ranged between 40 to 55 cm. The size of the screen was 19.5 × 14.6 cm. Custom software was developed for the experimental task 1 and 2 (XCode 3.2 development environment, iOS 4.2 operating system; Apple Inc, Cupertino, CA, USA; Cocos2D-ObjC, MIT License, Boston, MA, USA). The subjects were required to touch targets on the iPad^® screen with the index or middle finger of their most affected (children with CP) or preferred arm (TD children). Subjects were asked to maintain their trunk posture upright without touching the table with their pointing arm.

C. Procedure

Task 1

Each trial was initiated by touching a vertically centered rounded blue “start” button (diameter 3 cm) with the center 1.8 cm from the right or left edge of the screen for right and left handed participants respectively. Subjects were required to maintain contact with the start button until a square grid of square target buttons was displayed after a random interval between 500 and 1500 ms. The grid was vertically centered at a point 9.5 cm from the right or left side of the center of the start button for right and left handed participants respectively. The square grid was randomly chosen with a varying number of target buttons (b): 9 or 16, button size (w): 1 or 3 cm, and space between button (s): 1 or 1.5. The space between buttons (s) was defined as the ratio of the distance between target centers and the button size (w). This resulted with eight different display conditions (see Figure 1). Each of the target buttons contained a cartoon graphic image of a common object or action randomly chosen from a set of 31 objects or actions. During the hold time at the start button and before the target grid was displayed, one of the objects was randomly selected, its 2.5 cm square image, representing the “desired” target, was displayed above the start button with the center 3.8 cm from the upper edge, and its name was spoken by the iPad^®. The spoken name was generated from sampled human speech. As soon as the square grid appeared, participants were instructed to touch the button in the target grid that was the same as the desired target image, as soon and as accurately as possible. The image of the desired target remained displayed until the release of the start button, and the square grid disappeared after the participant touched the target button. Reaction time (RT) and movement time (MT) were recorded for all successful trials, and participants continued the task until they achieved 16 successful trials for each of the eight square grid conditions. RT was defined as the time between the square grid appearance and the release of the start button. MT was defined as the time between the release of the start button and contact with the target button. Displayed target button location, grid condition, and the desired target were selected and arranged randomly on each trial, and the same random order was used for each participant. Participants were encouraged to maintain motion close to the plane of the iPad^® without an excessive displacement in the perpendicular direction. However, sliding the finger along the iPad^® to the target button from the start button was not permitted, and success required that the first contact with the finger on the screen be within the desired target. To control for differences between subjects in the ability to memorize target locations, and to reduce trial-to-trial practice effects during testing, the cartoon graphic image and target button location were rearranged randomly between trials. Testing in this manner simulated novice performance on a completely unfamiliar touchscreen layout. The limited time available for practice could not completely predict each subject’s potential performance after prolonged practice and familiarization with the location of targets on each screen. In order to estimate the potential performance if all target locations were memorized, we tested a novel highlighting condition, in which the correct target was identified visually. Thus, task 1 consisted of two target highlighting (HL) conditions. The above described procedure was called “NotHighlighting” (HL0) condition, and participants did not receive any indication about the number of target buttons (b), target size (w), spacing (s) between buttons and location of the button within the grid. In the “Highlighting” (HL1) condition, the correct target button location was displayed on the screen shortly before the full square grid of target buttons appeared. Moreover, to facilitate visibility within the grid of buttons, the correct target button was surrounded by a bright green border (thickness of 3 mm) when the square grid appeared (see Figure 1 right panel). The complete set of grid conditions for all values of b, w, and s was tested as a block in each HL condition, with the order of HL0 and HL1 testing randomized between participants. Therefore, each participant performed a total of 256 trials: 8 square grid conditions (b: 9 and 16; w: 1 and 3 cm; s: 1 and 1.5) × 16 successful trials × 2 target highlighting conditions (HL0 and HL1). The custom software was designed such that the test carried on until 16 successful trials were achieved for each grid condition. This could result in more than 256 moves at the end of task 1. Participants were required to rest at least 1 min every 25 trials, and 3 min between the HL conditions. To encourage speed, a score was rewarded by showing 1 to 5 stars after each touch based on MT, RT, b, and index of difficulty (ID) of the trial. ID was computed according to Fitts’ Law as ID = log₂(2A/w) [17], where A was the distance between the center of the start button and the center of the target button, and w was the width of the target button. The score was computed as follows: Score (S) = (K × ID/MT) + log₂(b + 1)/RT. K was a constant term selected among 1.4, 1.6, 2.0, 3.0, 4.0 values to return a tangible and reasonable score of performance proportional to the severity of motor impairment in children with CP, which was chosen after the practice before starting the test. A constant term of 1.4 was always chosen for TD children. Eventually, S was converted to a reward scaled in number of stars from 1 to 5 proportionally to S. At the end of each highlighting condition (128 successful trials) a total score (TS) was displayed on the screen in term of stars as the sum of S for each trial. To ensure attempts at accuracy, a penalty of 5 stars was given for touching an erroneous target button, while a penalty of 1 star was given in the event of missing the target button and touching between buttons. The penalties were subtracted from the TS at the end of each highlighting condition. Three different tones were played to indicate when the participants hit the target button, an incorrect button, or missed the target button. Task 1 required less than an hour for each participant. Participants were given an initial 48 trials of practice (three trials in each grid condition and highlighting) to familiarize themselves with the device, task, and square grids, and to reduce practice effects.

Fig. 1.

Touchscreen grid conditions for task 1. The left panel represents three representative grid conditions with non-highlighting condition. The right panel represents three representative grid conditions with highlighting condition in which the correct target button was surrounded by a green border, thickness of 3 mm (in the figure the thickness was enlarged for display purposes only).

Task 2

Six out of ten children with CP (CP1 to CP6) were recruited for task 2 within five months after task 1. Task 2 aimed to test whether the predicted optimal touchscreen layout led to improvement of communication rate in children with CP. The performance with the optimal layout (Opt) estimated with our mathematical model was compared to a “standard” layout (Stn) for each child with CP. Children with CP recruited for task 2 were not dependent upon AAC devices. TONI-4 for all six children resulted with an age-equivalent cognitive function older than 6 years old. Therefore, we chose as the “standard” dictation layout a full English alphabet board plus the “space” and “delete” keys, which resulted in 28 buttons (b), 2 cm size (w) and spacing 1 (s), as shown in figure 2. The Opt layout (b, w and s) was estimated by the mathematical model for each child as explained below in the methods. To compare the two layouts, in both optimal (Opt) and standard (Stn) the letters, space and delete keys were substituted with cartoon graphic images of common objects or actions as used in task 1.

Fig. 2.

Representative touchscreen layouts for task 2. The layout on the left represents the Standard layout, 28 buttons, 2 cm size and spacing 1. The layout on the right represents the Optimal layout for CP3 non-highlighting condition, 6 buttons, 4.2 cm size and spacing 1.2.

Each trial was initiated by touching a rectangular (4 × 2 cm) start button in the center of the screen labeled with the word “START’. As the participant activated the start button a square grid with either the Opt or Stn layout appeared centered on the screen enclosing the target button. Buttons contained the same cartoon graphic images, as for task 1, randomly chosen from a set of 31. Simultaneously with the square grid appearance the name of the target button was spoken by the iPad^® generated by the same sampled human speech used for task 1. Subsequent square grids appeared in a sequential manner on the screen, without time interval between square grids appearance, and disappeared at the instant when the participant touched the screen. Participants were asked to touch as many as possible target buttons they could in 60 seconds with the same arm used for task 1. Participants were encouraged to continue the task in case of missing or hitting the incorrect button. Two different tones were played to indicate when the participants hit the target button or missed and hit incorrect target button. Ten sets of 60 seconds trials were performed for each layout (Otp and Stn). The total number of correct target button responses (i.e. correct hits) was recorded for each set. Displayed target button locations in the square grid were selected and arranged randomly on each trial, and the same random order was used for each participant. Each participant performed a total number of 40 sets: 2 square grid layouts (Opt and Stn) × 10 sets of 60 seconds trials × 2 two target highlighting conditions (HL0 and HL1). Participants were required to rest at least 1 min every set, and 3 min between the HL conditions. They were given an initial 60 seconds of practice for each square grid layout and highlighting condition. Task 2 required less than an hour for each participant.

D. Mathematical model to estimate the channel capacity

The combination of time to contact the button, accuracy demand and symbol probability determines information rate (in bits per second), which is our primary outcome measure. Therefore, the elements time, error rate and symbol probability must be measured. Since information rate depends on the total amount of time required to contact the target button, we used the total time TT = RT + MT as the dependent variable in the mathematical model. Several touchscreen layout factors influence TT for each child through influence on either RT, MT or both: 1) number of target buttons (b), 2) button width (w), 3) relative spacing between buttons (s) and 4) presence or absence of target highlighting (HL); so that total time is a function of these factors TT(b,w,s,HL). From Fitts’ Law we expected that MT depends linearly on the index of difficulty, ID = log₂(2A/w), where A is the movement amplitude and w the button size [17]. From Hick’s Law [18], we expected that RT depends linearly on log₂(b), but the nature of dependence was affected by the presence or absence of highlighting since the presence of highlighting would reduce the need for a visual search for the correct target [27], [38], [39]. The effect of target spacing is an important modifiable element of AAC devices. Since the spacing between buttons affects the margin of errors and the perceived cost of missing buttons [24], it could contribute to an increase or decrease in reaching speed using AAC devices, and therefore could influence the performance in some children. We defined s as the ratio of the distance between button centers to the button size. Thus, the effective button width was 2(s-1)w + w = (2s-1)w. The effective button width represents the area in which the attempt to hit the button does not result in unwanted target button activation. Increasing the spacing factor s increases the effective button width by a factor of (2s-1). From Fitts’ Law, we expected the dependence of TT on s to have the form log₂(2A/(2s-1)w) = log₂(2A/w) - log₂(2s-1), so in order to estimate the effect of s we introduced the additional term log₂(2s-1) in the model. We thus constructed the following candidate linear model for the predicted total time TTp:

$$\text{TTp}\left(\text{w},\text{b},\text{s},\text{HL}\right)=c0+c1•{\log }_2\left(2\text{A}/\text{w}\right)+c2•{\log }_2\left(\text{b}\right)+c3•{\log }_2\left(2\text{s}-1\right)$$

Where c0, c1, c2, c3 are the regression coefficients. c0 is the minimum movement time that would occur if only a single button occupied the entire screen. c1 is the increase in TT due to decreasing button with w and movement amplitude A (i.e. Fitts’ Law). c2 is the increase in TT due to increasing number of buttons b (i.e. Hick’s Law). c3 (which may be negative) is the decrease in TT with increasing spacing s between buttons. The presence or absence of highlighting condition (HL) was expected to affect the value of c2, but could also affect c0, c1, c2. Therefore, we constructed separate models for the highlighting off (HL0) and on (HL1) conditions. Thus, there was a four-dimensional space of parameters as a function of w, b, s and HL. To estimate the c0, c1, c2, c3 regression coefficients we constructed a nested hierarchy of generalized linear models (RStudio Inc., Version 0.98.109, Boston, MA; glm R Stats Package 3.1.2) of TT on all combinations and interactions of log₂(2A/w), log₂(b), log₂(2s-1) for each subject and each HL condition. The model was selected that minimized the Akaike Information Criterion (AIC) and maximize the variance explained r² [40]. Separate models were created for HL0 and HL1. We used the same four-dimensional space of parameters linear model and constructed a nested hierarchy of generalized linear models for the probability of making an error (Pe) or missing the target (Pm), namely activating the unwanted button or missing the target button respectively (glm, family = binomial, R Stats Package 3.1.2):

$$\begin{array}{ll}\text{Pe}\left(\text{w},\text{b},\text{s},\text{HL}\right)\hfill & =c0+c1•{\log }_2\left(2\text{A}/\text{w}\right)+c2•{\log }_2\left(\text{b}\right)+c3•{\log }_2\left(2\text{s}-1\right)\hfill \\ \text{Pm}\left(\text{w},\text{b},\text{s},\text{HL}\right)\hfill & =c0+c1•{\log }_2\left(2\text{A}/\text{w}\right)+c2•{\log }_2\left(\text{b}\right)+c3•{\log }_2\left(2\text{s}-1\right)\hfill \end{array}$$

We assumed that in actual use of AAC devices, an error requires two additional movements: one to delete or otherwise reverse the incorrect button press, and another to achieve the correct button press. We assumed that a miss requires only a single additional movement to achieve the correct button press. However, there is a chance of error on each subsequent attempt at correct movement. Therefore, if Pm is the probability of missing all targets and Pe is the probability of making an error, then the expected number of movements (Np) to achieve a successful result is:

$$\begin{array}{ll}\text{Np}\left(\text{w},\text{b},\text{s},\text{HL}\right)\hfill & =1+\text{Pm}+2{\text{pm}}^2+3{\text{Pm}}^3+\dots +2\text{Pe}+4{\text{Pe}}^2+6{\text{Pe}}^3+\dots \dots +3\text{PmPe}+\dots \hfill \\ \hfill & ={\displaystyle \sum }_{\text{ij}}\left(\text{i}+2\text{j}\right){\text{Pm}}^{\text{i}}{\text{Pe}}^{\text{j}}\hfill \end{array}$$

In general, the probability of misses or errors depend on all task 1 factors w,b,s, and HL. We created linear models for Pm(w,b,s) and Pe(w,b,s) in highlighting conditions HL0 and HL1. We then substituted these models into equation 4 (and truncated small values in the series expansion) to create an estimate of Np for each condition. We assumed that the TTp per successful button press (including all failed attempts) is Np (w,b,s) × TTp(w,b,s), and the predicted information rate IRp in bits per second is:

$$\text{IRp}\left(\text{w},\text{b},\text{s}\right)={\log }_2\left(\text{b}\right)/\left(\text{Np}\left(\text{w},\text{b},\text{s}\right)•\text{TTp}\left(\text{w},\text{b},\text{s}\right)\right)$$

This is a smooth function of w, b, and s, and it depends on the participant’s child error rate, movement time, and symbol probability. For any particular set of communication device parameters, information rate depends on both the child and the device layout parameters. Maximum information rate is called channel capacity, and the channel capacity is the measure of a child’s best ability to communicate. Thus, to obtain the channel capacity (CCp) for each child, we found the maximum value of information over all possible parameters for each HL condition:

$$\text{CCp}\left(\text{HL}\right)=\underset{\text{w},\text{b},\text{s}}{\max }\text{IRp}\left(\text{w},\text{b},\text{s}\right)$$

All possible layout parameters were chosen over a seed of 833 different combinations of w, b and s fitted within a square grid of 16 × 11 cm (176 cm²). Parameter w ranged from 1 to 4.6 cm (0.2 cm interval), b ranged from 2 to 36 buttons and s from 1 to 2 (0.2 interval). This area was chosen to maximize the square grid within the iPad^® touchscreen leaving at least 1 cm distance from the edges. TTp, Pe and Pm were estimated using predict function (R Stats Package 3.1.2, type = c(“link”, “response”, “terms”), se.fit = true; RStudio Inc., Version 0.98.109, Boston, MA). Since Fitts’ Law factor is a function of either w and A (log₂(2A/w)), TTp, Pe and Pm were predicted by considering all the movement distances performed during task 1 (128 successful trials) for each w of the parameters range, and averaged to estimate IRp and CCp.

E. Testing the communication rate between optimal and standard layouts

The mathematical model predicted the optimal touchscreen layout (e.g. w, b, s) for each participant and HL condition. The w and s parameters of the 28 buttons dictation standard layout were chosen to maximize the size of the buttons within the square grid of 16 × 11 cm as used for the optimal layout. This resulted with 2 cm buttons width and spacing factor s of 1. To compare the two layouts, we estimated the empirical information rate (IR) (in bits/sec) for task 2 as follows:

$$\text{IR}={\log }_2\left(\text{b}\right)•\text{R}/\text{T}$$

Where T is the time duration (60 seconds) and R is the number of correct responses (i.e. correct hits) for each trial. Separate IR values were calculated for each HL condition. We also calculated IR by considering R as the correct responses subtracted from the number of incorrect hits. We assumed that an error would require a subsequent attempt to correct the movement and touch the correct target button with a consequent decreased of information transmitted per unit of time. However, the results remained unchanged with the two methods, thus only IRp calculated with correct responses without errors subtracted will be presented in the manuscript.

F. Statistical analysis

Statistical analysis was performed using RStudio (RStudio Inc., Version 0.98.109, Boston, MA). To test the difference on CCp between groups and HL conditions we performed a linear mixed effects analysis (lmer, R lme4 package, version 1.1–7). As fixed effects, we entered HL (2 levels: HL0 and HL1) and Group (2 levels: CP and TD) into the model. As random effects, we had intercepts for subjects, as well as by-subject random slopes for the effect of HL. Once we had created the models, in order to test if the fixed effects significantly affected the dependent variable, we compared the model including all the factors (Full) against a reduced model without the effect in question (Null), for each dependent variable and for each factor. Similarly, to test interaction, that is, interdependence between 2 fixed effects, we compared the model that takes into account the interaction between fixed effects (Full) against the model without the interaction (Null), for each dependent variable. For all comparisons, P values and Akaike’s information criterion values (AIC) were obtained by likelihood ratio tests of the Full model with the Null model [40]. If the factor in question significantly affects the dependent variable, then the comparison will report a significant P value (<.05) and an AIC value lower for the Full model (AIC_Full). Similarly, a significant interaction between factors will result in a significant difference between the Full and the Null models (P value < .05), with the Full model characterized by a lower AIC. Linear regressions were performed by the method of least squares, and the Pearson correlation coefficient was used to indicate the goodness of the fit of CCp as a function of QUEST score. Spearman Rank-Order Correlation Coefficient was used to test the relationship between CCp and BAD score. To test the hypothesis that the model predicts the difference in communication rate between TD and CP, we calculated the difference between each matched pair TDi vs CPi (i = 1,2,3, …10) in empirical information rate IRe (ΔIRe) and predicted information rate IRp (ΔIRe) for each set of conditions (w, b, s, and HL) of task 1. The hypothesis was tested using linear regression of ΔIRe on ΔIRp using the method of least squares, and significance was tested using correlation coefficient and the F statistic for regression [41]. Linear regressions were also performed the goodness of fit between IRe and IRp for each participant. To test whether the Opt layout resulted with a higher information rate IR respect to the Stn layout we performed a linear mixed effects analysis (lmer, R lme4 package, version 1.1–7). As fixed effects, we entered HL (2 levels: HL0 and HL1) and layout (2 levels: Opt and Stn) into the model. As random effects, we had intercepts for subjects, as well as by-subject random slopes for the effect of HL and layout. The models were compared and significance was tested in the same manner as for CCp (see above). Moreover, a within-subject paired t-test was performed for IR to test the improvement in individual participants to their change from Opt to Stn conditions separately for each HL condition. We used a criterion of P < 0.05 to signify a significant difference.

III. Results

To predict the IRp and CCp for each child and HL condition we selected the linear model that minimized the AIC using the data collected during task 1. Figure 3 shows the predicted information rate IRp over a seed of 833 different combinations of w, b and s for participants CP1 and TD1 (solid circles) for HL0 condition. The numbered grey solid circles represent IRp predicted by the model for the eight experimental conditions of task 1. The maximum value of IRp over all possible parameters combinations represents the channel capacity CCp (grey open circle) of the participant, in which we selected the optimal layout of parameters b, w, and s. The parameters b, w and s were distributed over the seed such that the number of b increased from left to right with a consequent decrease of w and s to fit the square grid into the iPad^® touchscreen. Thus, it can be seen that IRp increased with an increase of b for TD1, while for CP1 the peak of IRp corresponded to a parameters combination that did not maximize the number of buttons (b) but resulted with larger buttons size (see optimal layouts in Table II).

Fig. 3.

Plot showing the predicted information rate over a seed of 833 different combinations of size, buttons and spacing (solid circles) for participants CP1 (on the top) and TD1 (in the bottom) for non-highlighting condition. The numbered grey solid circles from 1 to 8 represent IRp predicted by the model for the eight experimental conditions of task 1. The maximum value of predicted information rate over all possible parameters combinations represents the channel capacity (grey open circle) of the participant, in which we selected the optimal layout of parameters size, buttons and spacing. The parameters size, buttons and spacing were distributed over the seed such that the number of buttons increased from left to right with a consequent decrease of size and spacing to fit the square grid into the touchscreen. It can be seen that predicted information rate increased with an increase of b for TD1, while for CP1 the peak of predicted information rate corresponded to a parameters combination that did not maximize the number of buttons but resulted with larger buttons size.

TABLE II.

Optimal layouts & t.test IR opt vs. stn

	Layout Opt HL0			Layout Opt HL1			Opt vs. Stn HL0		Opt vs. Stn HL1
	b	w (cm)	s	b	w (cm)	s	t (d.f = 9)	p	t (d.f = 9)	p
CP1	12	3.6	1.0	24	2.0	1.0	5.172	< 0.001	4.209	< 0.01
CP2	12	2.0	1.6	12	3.6	1.0	4.107	< 0.01	7.021	< 0.001
CP3	6	4.2	1.2	24	2.6	1.0	4.257	< 0.01	3.002	< 0.05
CP4	4	4.2	1.0	6	4.6	1.0	2.833	< 0.05	2.751	< 0.05
CP5	24	2.2	1.0	18	2.6	1.0	2.867	< 0.05	3.046	< 0.05
CP6	24	2.6	1.0	24	2.6	1.0	2.659	< 0.05	5.256	< 0.001

The likelihood ratio test reported a significant effect on CCp for Group (AIC_Full = 147.18; AIC_Null = 165.70; P < 0.001) and HL (AIC_Full = 147.18; AIC_Null = 166.21; P < 0.001). On average CP resulted with lower channel capacity (M = 3.30 bits/s ± 1.71) respect to TD (M = 6.82 bits/s ± 1.89). Also, the model predicted higher channel capacity with HL1 (M = 5.99 bits/s ± 2.75) respect to HL0 (M = 4.13 bits/s ± 1.90), meaning that the speed of communication decidedly increased when the target locations were known by the participants (see Fig.4 and Table I).

Fig. 4.

Plot showing the means and standard errors of predicted channel capacity (CCp) for both groups and highlighting conditions (non-highlighting on the left and highlighting on the right). Grey bars: children with cerebral palsy; white bars: typically developing children. Asterisk mark (∗∗) indicates a statistical difference P < .01.

The averaged CCp (CCp-HL0 & CCp-HL1) showed a linear correlation with the QUEST in children with CP: r = 0.642, F(1, 8) = 5.607, P < 0.05. BAD score showed an inverse linear relationship with CCp despite a significant correlation was not found (p=0.15). To test whether the model predicted the difference in information rate between TD and CP, we performed a linear regression between ΔIRe (difference on empirical information rate, see Methods) and ΔIRp (difference on predicted information rate) for each participant matched pair and for each of eight conditions of task 1. We found a significant linear correlation between ΔIRe and ΔIRp for either HL0 (R² = 0.724, F(1, 78) = 204.1, p < .001) and HL1 (R² = 0.779, F(1, 78) = 274.3, p < 0.001) conditions (Fig.5). When we performed the linear correlation between IRe and IRp for each participant and each HL condition we found significant linear regressions (F(1, 6) > 5.98, p < 0.05) for all the subjects and HL condition, except for CP2 with HL1 (p=0.16), CP6 with HL0 (p=0.48) and TD9 for HL1 (p = 0.06).

Fig. 5.

Difference on empirical information rate as a function of difference on empirical information rate for non-highlighting (left plot) and highlighting (right plot) conditions. The straight lines show the best fits by the least squares method. Each participant matched pair is represented by different color of unfilled circles.

To test whether the predicted optimal touchscreen layout led to improvement of communication rate in children with CP we compared a “standard” dictation layout simulating a full English alphabet board with the layout predicted by the mathematical model. Six out of ten children with CP recruited for task 1 participated to the second session (task 2) to test the optimal layout. Table II shows the optimal parameters predicted by the model for each the six CP participants and for both HL conditions. The likelihood ratio test reported a significant effect on IR for Layout (AIC_Full = 647.29; AIC_Null = 712.63; P < 0.001) and HL (AIC_Full = 647.29; AIC_Null = 800.11; P < 0.001) effects. On average the performance was greater with the Opt layout respect to Stn layout for both the highlighting conditions during task 2 (HL0-Stn: M = 1.01 bits/s ± 0.40; HL0-Opt: M = 1.50 bits/s ± 0.40; HL1-Stn: M = 2.15 bits/s ± 1.05; HL1-Opt: M = 3.65 bits/s ± 1.50). A within-subject analysis showed a significant effect for Layout and HL with higher IR for Opt than Stn with both the highlighting conditions in all six CP subjects (df = 9, t = range: 2.66 – 7.02, P<0.05) (see Fig.6 and Table II).

Fig. 6.

Plot showing the means of Information Rate during task 2 considering each participant separately (the bars represent the standard deviations) for the layout (Optimal and Standard) and highlighting conditions. Grey bars: NotHighlighting condition, HL0; white bars: Highlighting condition, HL1. Asterisk mark (∗) indicates a statistical difference P < .05.

IV. Discussion

In this study, we propose a mathematical model to estimate information rate and channel capacity that considers spacing between buttons, probability of making an error or missing target buttons in combination with Fitts’ and Hick’s Laws [15]. We hypothesized that this multilevel model, as well as its interactions between different parameters, would lead to an improved prediction of information rate and channel capacity and enhanced speed of communication in children with dyskinetic CP interacting with AAC touchscreen devices.

The results show that the model for predicted information rate agrees well with measurements. Indeed, Fig. 5 shows that the differences in values for patients and controls between empirical information rate (measured from data collected during task 1) and predicted information rate have a good correlation over the age-matched pairs. Results also demon strates a significant relationship within each participant for both groups between empirical and predicted information rate.

The value of the mathematical model is that it allows prediction of the optimal button size, button number and spacing between buttons without the requirement of testing all possible combination of parameters. Explicit testing of all possible combination of size, number and spacing would require an excessive amount of time, which would be even more prohibitive for children with motor impairments. The distribution of IRp values allows as to find the maxima that occurs for a specific combination of parameters while the empirical testing only tested two button sizes (1 and 2 cm), button numbers (9 and 16 cm) and buttons spacing (1 and 1.5). The Fig.3 demonstrates the curves of a representative child with CP and an age-matched control participant. As expected, for the control participant the optimal combination of parameters (grey open circle) occurs for the largest number of buttons and small button size which results in the highest information rate (see methods). Interestingly, for the child with dyskinetic CP, there exists an optimal set of parameters that maximize the information but that do not result in the greatest possible number of buttons on the screen. As one can note in Fig.3, when the combination of parameters is not optimal (i.e., number, size and spacing), then the information rate drops precipitously. Different children show different values for the optimal combination of parameters, demonstrating the need to perform the evaluation in individual children.

We conjectured that one of the consequence of brain injury in dyskinetic CP may be a reduction in the channel capacity even with an optimized device layout. The results support our hypothesis by showing that channel capacity is decreased in children with dyskinetic CP by half with respect to age-matched unaffected control subjects. This suggests that channel capacity provides an objective assessment of motor performance that combines several elements of motor skill. Indeed, channel capacity correlated with the Quality of Upper Extremity Skills Test scoring showing its relationship with clinical motor skills evaluations. Although the Barry-Albright Dystonia scale did not show a significant correlation, the tendency of decreased channel capacity with an increase of the scoring is in accordance with a previous study that showed a strong correlation of the scale with movement time during Fitts’ task in children with dystonia [7]. The lack of significance may reflect an insufficient number of subjects recruited and a heterogonous channel distribution of the Barry-Albright Dystonia scale scoring. Essentially, our results suggest that capacity is a particularly interesting measurement since it is based on the optimal performance of each child for a specific AAC device. This is equivalent to adjusting the layout parameters (button size, number and spacing) so that each child’s best possible performance is measured. Channel capacity could therefore be an excellent measure of potential performance on an individually-adapted AAC device, and thus information theory may be exploited to understand how injuries of the sensorimotor system limit the ability to deliver information and interact with ACC devices.

The best fitting model including spacing buttons and parameters interaction resulted for 70% and 35% of all participants respectively, and the factor of the probability of making an error or missing target buttons was particularly critical in children with dyskinetic CP to estimate the channel capacity. To test the hypothesis that the estimated optimal layout leads to an increased information rate we compared the performance on a AAC touchscreen device between the optimal layout and a full English alphabet dictation layout (standard). Figure 6 shows that information rate with the optimal layout increases respect to the standard layout in all six children with dyskinetic CP with, on average, an increment of 50% and 70% for HL0 and HL1 highlighting conditions respectively, and a range of increment between 10% and 150%. Our results, supported by previous theoretical and clinical findings [24], [29], [30], [42], suggest that perceived cost of error and the probability to activate the unwanted button are highly affected by spacing between buttons. Also, it may be possible that perceived cost of error influences reaction time (Hick’s Law parameters), since increased number of choices increases the number of incorrect possibilities, which supports the supplement of parameters interaction in the proposed model.

In our study, all participants’ best fitting model included the Fitts’ Law factor with either the highest or one of the highest regression coefficient. Fitts’ Law has been explained as a strategy to compensate for signal-dependent noise of the sensorimotor system [43]. This theory is of particular relevance to dyskinetic CP, since increasing speed of movement worsens the apparently random and uncontrollable hyperkinetic movements. A predicted consequence that it has been confirmed in previous studies is that children with dyskinetic CP decrease speed to a greater extent than control subjects when accurate movement is required [3], [22], [23]. Therefore, increasing the number of buttons (which forces a decrease in their size) will disproportionately worsen performance in children with CP.

As expected the results show higher performance, e.g. channel capacity, with highlighting condition than no-highlighting condition for both children with dyskinetic CP and age-matched controls. And it explained the differences in the optimal layout arrangements between the two highlighting conditions, which would further result with different information rate curves. This suggests that the optimal AAC layout is practice-dependent and its configuration requires updates when the child becomes more familiar with the buttons layout, since the performance is likely to be very different following extended practice [27], [39].

Our study has limitations that will need to be addressed in future research. The children with dyskinetic CP recruited in the study were not dependent upon AAC devices to communicate. However, their motor impairments considerably hampered the ability to interact with AAC touchscreen devices although their cognitive level resulted with age equivalent for school-age or not lower than 5 years, and it allowed them to understand the task instructions during testing. For future testing, it would be worthwhile to test the mathematical model to children who are dependent upon AAC devices to quantify the improvement of speed of words per unit time after their own AAC device has been customized with the optimal layout. Future research would also be required to test the model with a larger cohort of children with dyskinetic CP, and to explore supplementary model parameters that could lead to more accurate estimates of information rate in patients that exhibit a combination of excess involuntary movements and slowed voluntary movements.

In conclusion, this study quantifies the effect of motor impairments on communication with assistive communication devices and shows that communication performance can be improved by maximizing the transmission between the child and an AAC touchscreen device. An important product of this study will be software tools usable with inexpensive touchscreen interfaces for selection of optimal communication device parameters. The results will have immediate and significant benefit to children with dyskinetic CP who depend on assistive communication devices to communicate by providing a rational basis for the design of device interfaces based upon specific individual abilities. Further research will be needed to determine whether the results are applicable to adults or to children with other movement disorders.

Biography

Matteo Bertucco studied human movement science (B.S. 2003, M.S. 2006) at the University of Verona, Italy, and he obtained the Ph.D. (2010) in physical exercise and human movement sciences at the same University. He performed postdoctoral training at the University of Southern California (2010–2016) with Dr. T.D. Sanger. He is currently an assistant professor at the Department of Neuroscience, Biomedicine and Movement Sciences, University of Verona. Dr. Bertucco is primarily interested to conduct research in the areas of motor control, motor learning and neuromechanics. His overall approach is to combine neurological, engineering and computational principles to understand the mechanisms that underlie the control and learning of human movement in developmental motor disorders.

Terence D. Sanger holds an SM in Applied mathematics (Harvard), PhD in Electrical Engineering and Computer Science (MIT), and MD (Harvard), with medical specialization in Child Neurology and Movement Disorders. He is currently Provost Professor of Biomedical Engineering, Neurology, and Biokinesiology at the University of Southern California (USC), Director of the Pediatric Movement Disorders Clinic and Deep Brain Stimulation Program at Childrens Hospital of Los Angeles (CHLA), and the founding Academic Director of the Health Technology and Engineering program at USC (HTE@USC). His research on disorders of developmental motor control is driven by his interest in finding new treatments for children with movement disorders including dystonia, chorea, spasticity, and dyspraxia. He has a particular interest in computational motor learning, and the role of motor learning in recovery from childhood brain injury. Major focus areas of laboratory research include wearable devices to promote motor learning, EMG-driven communication devices and assistive prosthetics, and modeling of the electrophysiology of deep-brain stimulation. Personal involvement in motor control and motor learning includes snowboarding, jazz and classical piano, bluegrass banjo, blues harmonica, and ballroom dance with particular focus on Argentine Tango.