Evaluation of the Tongue Drive System by Individuals with High-Level Spinal Cord Injury

Abstract

Tongue Drive System (TDS) is a tongue operated, unobtrusive, minimally invasive, wireless assistive technology (AT), which can enable people with severe disabilities to control different devices using their tongue motion. TDS can translate specific tongue movements into user-defined commands by detecting the position of a small permanent magnetic tracer attached to the users’ tongue. We have built an external TDS (eTDS) prototype on a wireless headphone and interfaced it to a laptop and a commercial powered wheelchair (PWC). eTDS performance was evaluated by eight subjects with high level (C3~C5) spinal cord injury (SCI) at the Shepherd Center in Atlanta, GA. Preliminary results show that all the subjects can successfully perform common tasks related to computer access, such as controlling a mouse cursor or playing a computer game, as well as complex wheelchair navigation tasks, such as driving through an obstacle course.

I. Introduction

Persons with severe disabilities as a result of causes ranging from traumatic brain and spinal cord injuries (TBI/SCI) to amyotrophic lateral sclerosis (ALS) and stroke rely on assistive technologies (AT) for daily tasks, and gain greater independence to lead a self-supportive life. Among ATs those providing alternative control for computer access and wheeled mobility are considered the most important for today’s lifestyle since they can potentially improve users’ quality of life by easing two major limitations: effective communication and independent mobility [1], [2].

Although a few ATs are available for either computer access or powered wheelchair (PWC) control, none can effectively and safely address both applications. Therefore, users are burdened with learning how to use multiple ATs for various tasks, and switching among them often with the help of a caregiver. Sip-n-puff, for example, is a simple, low-cost, and easy to use AT, which allows users to control their PWC by blowing or sucking through a straw. However, its limited number of direct choices, slow command input, and appearance, which is often associated with severe disability, are unattractive to the end users. It also needs frequent cleaning and cannot be used by those without diaphragm pressure. A group of ATs, based on eye movement tracking [3], [4], have been successful for computer access. However, they are not suited for controlling PWCs because they require extra eye movements that may interfere with the users’ normal vision. Another group of ATs known as head pointers require a certain level of head movement ability that may not exist. They are also susceptible to inertial forces applied to the head during motion. There are ATs, utilizing bioelectric signal such as Electromyogram (EMG) signal from muscle twitches or electroencephalograms (EEG) signal from the brain [5]. These devices are relatively slow and offer limited degrees of freedom (DoF). Invasive brain-computer interfaces are also under development, which will be costly and impose considerable risks and hardship associated with neurosurgery [6]. A few controllers use voice commands as input signals [7]. These systems are suitable for computer access in quiet places, but unreliable for PWC control in noisy and outdoors environments.

Tongue occupies a considerable area of the motor cortex in humans and as a result, is inherently capable of sophisticated manipulation tasks [8]. The tongue is innervated via hypoglossal cranial nerve, which generally escapes damage even in severe SCIs. The tongue muscle does not fatigue easily, particularly when it moves freely in the mouth [9]. The tongue is noninvasively accessible, and is not influenced by the position of the rest of the body, which can be adjusted for maximum user comfort. The above reasons have resulted in development of a few tongue-operated ATs, such as the Tongue-Touch-Keypad (TTK), in the past. However, they have not been widely adopted because of requiring bulky objects inside the mouth [10], [11].

Tongue Drive System (TDS) is an unobtrusive, minimally invasive, wireless, tongue-operated AT that can offer multiple control functions over a wide variety of devices in the users’ environments. At EMBC’07 and EMBC’08 we reported on the TDS architecture, TDS-PWC interface, and performance evaluation by able-bodied subjects [12], [13]. Here, we are presenting the results of TDS performance evaluations by eight subjects with high level SCI and a few improvements on the TDS hardware and user interface.

II. Design and Development

A. External Tongue Drive System (eTDS) Prototype

TDS infers its users’ intentions, represented through their voluntary tongue motion, by detecting the position of a small permanent magnetic tracer, the size of a lentil, attached to their tongues by tissue adhesives, piercing, or implantation. It uses an array of magnetic sensors to measure the changes in the magnetic field around the mouth due to tongue motion. This information is wirelessly sent to a portable computing device which translates specific tongue movements to user-defined commands [14], [15].

In the latest eTDS prototype, shown in Fig. 1, small disk-shaped (∅4.8 mm × 1.5 mm) rare earth permanent magnets (K&J Magnetics, Jamison, PA) were used as the tracers on the users’ tongues. A pair of 3-axial magnetic sensor modules (PNI, Santa Rosa, CA) was extended towards the users’ cheeks by a pair of goosenecks, mounted bilaterally on a commercial wireless headphone (Sennheiser, Old Lyme, CT) to facilitate sensor positioning, while maintaining an acceptable appearance. A miniaturized control unit was placed inside the left earpiece, including an ultra low-power MSP430 microcontroller (Texas Instruments, Dallas, TX), which activates only one sensor at a time to save power. In the active mode, all six sensors (3 per module) are sampled at 13 Hz, packed in a data frame, and wirelessly transmitted to a laptop across a 2.4 GHz link established between two nRF24L01 transceivers (Nordic Semiconductor, Norway). In the standby mode, only the right module is sampled at 1 Hz locally and the transceiver stays off to save power.

Fig. 1.

Block diagram of the external Tongue Drive System (eTDS) prototype, built on a wireless headphone.

Sensor signal processing algorithm (SSP), running on a 2.0 GHz laptop, uses the K-Nearest-Neighbors (KNN) classifier to identify the incoming samples based on their features, which are extracted by Principal Components Analysis (PCA). SSP then associates them to particular commands, which are defined by the users in a training session [12].

B. Tongue Drive-Powered Wheelchair (PWC) Interface

We have developed a dedicated graphical user interface (GUI) and an adapter circuitry, shown in Fig. 2, to operate commercial PWCs with eTDS [13]. In the GUI, a universal PWC control protocol has been implemented based on two state vectors (left column in Fig. 2a), one for linear motions, and one for rotations [16]. The PWC speed of movement/rotation is proportional to the absolute values of these state vectors, and the direction of movement/rotation is determined by the vectors’ polarity. Five commands are defined in eTDS GUI to modify the state vectors, resulting in the PWC moving forward-FD and backward-BD, turning right-TR and left-TL, and stopping-N (middle column in Fig. 2a). Each command increments/decrements its associated state vector by a certain amount until a predefined maximum/minimum level is reached. The neutral “N” command, which is issued automatically when the tongue returns back to its resting position, always returns the state vectors back to zero. Therefore, by simply returning the tongue to its resting position, the user can bring the PWC to a standstill.

Fig. 2.

(a) TDS GUI provides the user with visual feedback while operating the PWC. (b) Adapter circuitry connecting a laptop to the PWC controller via USB and standard DB-9 connectors.

Based on the above rules, we have implemented two different control strategies: discrete and continuous. In discrete control, state vectors are mutually exclusive, i.e. only one state vector can be nonzero at any time. If a command changes the current state, for example from FD to TR, the old state vector (linear) has to be gradually reduced/increased to zero before the new vector (rotation) can be changed. Hence, the user is not allowed to change the PWC’s moving direction before stopping. This is a safety feature particularly for novice users at the cost of reducing the PWC agility. In continuous control strategy, the state vectors are no longer mutually exclusive, and the user is allowed to steer the PWC to left or right as it is moving forward or backward. Thus, the PWC movements are continuous and much smoother, making it possible to follow a curve, for example.

As long as the users can remember and correctly issue commands, seeing the GUI screen is not necessary while driving the PWC. After the PWC state vector is modified, they are sent to the adapter circuitry through a USB port. The adapter converts the vectors into voltage levels (3 ~ 9 V) and applies them through a DB-9 connector to the PWC controller (see Fig. 2b), which is compatible with most commercial PWCs, including C500 (Permobil Inc., Lebanon, TN) and Q6000 (Pride Mobility, Exeter, PA).

III. Performance Evaluation

The system performance was evaluated by eight subjects (two female and six male) aged 20 to 55 years old with high level SCI (C3~C5) from the Shepherd Center (Atlanta, GA) inpatient (5) and outpatient (3) population. Informed consent was obtained from all subjects. The experiment was carried out in the SCI unit of the Shepherd Center, with approvals from Georgia Institute of Technology and Shepherd Center institutional review boards (IRB).

Each subject participated in two individual sessions; computer access (CA) followed by PWC control (PWCC). In the CA session, the subjects were either sitting in their own PWC or lying on bed with a 22” LCD monitor placed ~1.5 m in front of them. In the PWCC session, subjects were transferred to a Q6000 PWC and a 12” laptop was placed on a wheelchair tray in front of them, as shown in Fig. 3.

Fig. 3.

A SCI subject wearing the eTDS prototype to drive a Q6000 PWC.

A magnetic tracer was sterilized and attached to a 20 cm string of dental floss using superglue. The other end of the string was tied to the eTDS headset during the trials. This was a safety measure to avoid the tracer from being accidentally swallowed or aspired if it was detached from the subject’s tongue. The top surface of the tracer was smoothed with a layer of silicone rubber to prevent possible harm to the subjects’ teeth or gums. The magnet was then adhered to the subjects’ tongues using Cyanodent tissue adhesive (Ellman International, Oceanside, NY) [17]. eTDS headset was placed on the subjects’ heads and the magnetic sensor positions were adjusted near their cheeks by bending the goosenecks.

The tasks in each session were arranged from easy to difficult to facilitate learning as the trial went on. At the beginning of each level, the subjects were asked to train the SSP algorithm by defining tongue positions corresponding to the commands in that level, and repeating them for 10 times in a sequence [15]. In the CA session, the number of eTDS commands was increased from 2 to 4 and then to 6 in three levels. In the PWCC session, the subjects were asked to define 4 commands and use them to operate the PWC in discrete mode first, and then continuous mode. Finally, they were asked to drive the PWC without seeing the GUI.

A. CA-1: Playing Computer Game with 2 Commands

The purpose of this test was to familiarize the subjects with TDS commands, and train them on manipulating an object on a PC screen using their tongue. In this test, the subjects were first asked to define two commands; Left and Right, and use them to play a “breakout” game by moving a paddle horizontally with their tongue, preventing a bouncing ball from hitting the bottom of the screen. Subjects were then instructed to define another two commands; Up and Down, and use them to play a “scuba diving” game by moving a scuba driver vertically to catch treasures while avoiding fish and rocks. Subjects repeated each game for 3 times and their scores were registered manually.

B. CA-2: Maze Navigation with 4 Commands

Subjects were asked to define four commands; Left, Right, Up and Down, and then complete a navigation task by using these commands to move the mouse cursor through an on-screen maze as quickly and accurately as possible from start to stop points, while the cursor path and elapsed time were being recorded [15]. The maze was wider at the beginning so that subjects can start easily, and then gradually became narrower towards the end of the track. Subjects were required to repeat this task three times. Average completion time across eight subjects was 54.2 s with a standard deviation of 16.6 s. This experiment was an emulation of PWC control task on a PC. It allowed subject to practice with navigation using tongue commands without potential risks associated with PWCs.

C. CA-3: Response Time Measurement with 6 Commands

Subjects were asked to add two more commands to the directional commands in CA-2 for single and double mouse clicks. Then they were instructed to issue a randomly selected command within a specified time period, T, on an audio-visual cue [14]. T was changed from 2 s to 1.5, 1.2, and 1 s, and 40 commands were issued each time. Fig 4a shows the percentage of correctly completed commands (CCC%) for each T averaged across 8 subjects. The information transfer rate (ITR) calculated for each T using Wolpaw’s definition in [5], are shown in Fig. 4b. On average, a CCC% = ~80% was achieved with T = 1 s, yielding an ITR of ~85 bits/min.

Fig. 4.

Results of the response time measurement experiment based on 8 SCI subjects: (a) average values and 95% confidence interval of CCC%. (b) Information transfer rates and their 95% confidence interval.

D. PWC Control Session

Subjects started this session by defining four commands (FD, BD, TR, TL) in addition to the tongue resting position (N) for stopping, and practiced for ~5 min on the PC. Then they drove the PWC, using TDS, through an obstacle course, which required using all TDS commands to perform various navigation tasks such as making a U-turn, backing up, and fine tuning direction in a limited space. Subjects were asked to navigate the PWC as fast as possible, while avoiding obstacles. Since trials were conducted in different places, three slightly different courses were utilized. However, they were all close to the layout shown in Fig. 5. The average track length was 40.1 ± 5.3 m with 10.9 ± 1.5 turns.

Fig. 5.

Plan of the PWC navigation track showing dimensions, obstacles, and approximate PWC trajectory.

During the experiment, the laptop was placed on a tray in front of the subjects, initially with the lid opened to provide them with visual feedback (VF) and later with the lid closed. Subjects were required to repeat each experiment at least twice for discrete and continuous control strategies. The navigation time, number of collisions, and number of issued commands were recorded for each trial.

All subjects successfully completed the PWC navigation task without difficulty. Fig. 6 shows the average navigation speed along with its 95% confidence intervals, and number of collisions during each experiment. In general, the continuous control was much more efficient than the discrete control. Subjects consistently performed better by navigating faster with fewer collisions without VF. These results demonstrated that using TDS is quite simple and intuitive such that subjects can easily remember and correctly issued tongue commands without requiring much training or a computer screen in front of them, which may distract their attention or block their front view. Improved performance without VF can also be associated with the learning effect.

Fig. 6.

Average navigation speed and number of collisions for discrete and continuous control strategies with and without visual feedback (VF).

IV. Conclusion

Tongue Drive System is a new tongue operated wireless assistive technology, which can detect its users’ intentions by tracking their tongue motion with a magnetic tracer attached on the tongue and an array of magnetic sensors near their cheeks. TDS can offer its users multiple control functions over a wide variety of devices. The latest eTDS prototype is built on a wireless headphone and linked to computers and PWCs via custom designed hardware. Preliminary human trials on eight subjects with high level SCI demonstrated that the current eTDS prototype can potentially provide its end users with effective control over both computers and PWCs. We intend to improve TDS response time using new sensors, and add proportional control capability to the system.