CCi-MOBILE: Design and Evaluation of a Cochlear Implant and Hearing Aid Research Platform for Speech Scientists and Engineers

Abstract

Hearing loss is an increasingly prevalent condition resulting from damage to the inner ear which causes a reduction in speech intelligibility. The societal need for assistive hearing devices has increased exponentially over the past two decades; however, actual human performance with such devices has only seen modest gains relative to advancements in digital signal processing (DSP) technology. A major challenge with clinical hearing technologies is the limited ability to run complex signal processing algorithms requiring high computation power. The CCi-MOBILE platform, developed at UT-Dallas, provides the research community with an open-source, flexible, easy-to-use, software-mediated, powerful computing research interface to conduct a wide variety of listening experiments. The platform supports cochlear implants (CIs) and hearing aids (HAs) independently, as well as bimodal hearing (i.e., a CI in one ear and HA in the contralateral ear). The platform is ideally suited to address hearing research for: both quiet and naturalistic noisy conditions, sound localization, and lateralization. The platform uses commercially available smartphone/tablet devices as portable sound processors and can provide bilateral electric and acoustic stimulation. The hardware components, firmware, and software suite are presented to demonstrate safety to the speech scientist and CI/HA user, highlight user-specificity, and outline various applications of the platform for research.

I. Introduction

A cochlear implant (CI) is a medical device consisting of an implanted intracochlear electrode array and an external sound processor used as a solution for severe to profound hearing loss. CIs simulate hearing by stimulating the auditory nerve through series of pulses bypassing all inner-ear mechanics. According to the World Health Organization, 5% of the world’s population has disabling hearing loss. Approximately 35 out of 10,000 children suffer from sever to permanent hearing loss, which makes the phenomenon the most common birth defect in developed countries [1]. In the United States alone, the percentage of individuals impacted by hearing loss ranges from 15% in school-age children (ages 6–19) to about 33% at 65 years of age [2]. Assistive hearing devices, such as hearing aids (HAs), bone-conduction devices, and CIs, are some examples of the prevalent technology that can help to provide/improve hearing sensation. Success of this technology, to a vast extent, depends on the effectiveness of the sound processing and presentation of the stimuli to the human auditory system. Improvements in sound processing technology have played a critical role in the advancement of assistive technology. Hearing scientists generally rely on research tools/interfaces to conduct perceptual studies with assistive devices. However, research interfaces commonly provided by manufacturers either have limited functionalities or are not suitable for conducting a broad range of experiments. Computing limitations, portability, and ease of programming limit the use of existing research interfaces. Furthermore, the complexity of sound processing solutions for assistive hearing devices have also increased as medical interventions for hearing loss have evolved. This has created unique opportunities for researchers to design and perform complex experiments that involve combinations of electric and acoustic stimulation modalities. However, there is a lack of research tools and systems capable of supporting the growing needs of this research community.

For over a decade, the Center for Robust Speech Systems - Cochlear Implant Processing Lab (CRSS-CILab) has focused on developing speech systems, signal processing solutions for the normal hearing/hearing impaired, and a research platform capable of evaluating these proposed solutions [3]. The CCi-MOBILE research platform was developed at the CI Lab (UT-Dallas) as a flexible, open-source platform to provide researchers the freedom to control and define signal processing parameters for in-lab and field testing either in desktop or mobile applications. This tool was designed to help researchers to test the efficacy of signal processing solutions and to assist in conducting short and long-term listening experiments. In this study, a high-level overview of the hardware and software are provided in addition to results from validation and verification experiements to demonstrate safety and performance.

II. Research Platform

The CCi-MOBILE research platform is a custom-built interface board for communication with behind-the-ear (BTE) microphone units, HA transducers, and radio-frequency (RF) coils to provide electric stimulation to CIs manufactured by Cochlear Corporation. The functional diagram of the platform is shown in Fig. 1 and is described as follows: (1) the hardware interface board continuously samples incoming analog signals from the BTE microphone unit; (2) the board responds by transmitting 512 byte blocks of audio collected previously in digitized form; (3) the PC/smartphone processes the digitized audio data with the signal processing strategy programmed within the software suite (Matlab/Java); (4) electrical stimuli is passed from the PC/smartphone and an electrical signal is generated for stimulation (for bimodal, the acoustic stimuli is sent to the board also); (5) the stimuli is specifically encoded by the FPGA for CI24 cochlear implant system (Cochlear Corp.) proprietary format and streams data on a frame-by-frame basis as biphasic pulses to the RF-coil; (6) in bimodal applications, acoustic and electric signals are sent in a synchronous manner. This section describes the functionality of both hardware and software components needed for data transmission.

Fig. 1.

High-level functionality of CCi-MOBILE research platform.

A. Hardware Design

The computing platform, which can be a PC or mobile device, sends implant characteristics data (such as electrode number, amplitude, and simulation mode) on a frame-by-frame basis to the CCi-MOBILE interface board. The size of data in each frame is constant (1,032 bytes) for both bimodal and variable rate electric stimulation. The various modules communicating with the FPGA (see Fig. 2), are internal state machines which run concurrently. The four state machines are: WM8983 codec, UART-TX, UART-RX, and RF-coil. These run at different clock speeds while the data synchronization is managed via RAM and handshake design-techniques. The communication link between the computing platform and the CCi-MOBILE platform is interfaced at 5 Mbps. Sampling of audio data and stereo playback is achieved at 16 kHz, and CI24 implants require a data burst at 5 MHz. Hence, the platform operates at a higher clock, which in this design is 80 MHz.

Fig. 2.

Hardware Design architecture for Electric and Acoustic Stimulation (EAS) and Variable rate Stimulation.

The hardware implementation of CCi-MOBILE is shown in Fig. 3. To achieve real-time performance with minimal processing delay, the hardware-interface board buffers the incoming and outgoing data stream on a frame-by-frame basis. In this design, a ping-pong strategy [7] is used for data-storage and processing. Pong buffers (BUF 1 of electric and acoustic) will process data when ping buffers (BUF 0 of electric and acoustic) are storing data. For variable rate electric stimulation, this design uses 16-bit wide RAM buffers, which store both electrode numbers and current levels.

Fig. 3.

Hardware implementation of CCi-MOBILE platform

The FPGA design is programmed in Verilog using Xilinx ISE software. The accuracy of the output received at the RF-coil is achieved by parameter-matching the electric stimuli generated, such as the inter pulse gap, pulse width and inter phase gap. These parameters are measured and verified using an externally connected oscilloscope. Audacity is used to analyze the acoustic signals.

B. Software Implementation

For cochlear implant signal processing, an ‘n-of-m’ strategy selects a subset of ‘n’ electrodes out of ‘m’ total electrodes based on the frame energy of the corresponding frequency channels. Each electrode is mapped one-to-one with a band pass filter where apical electrodes represent low frequency and basal electrodes represent higher frequencies. Frequency allocation (default set by Cochlear Corp.) of these filters are pre-programmed into the software routine. Intracochlear electrodes are stimulated in a continuous interleaved manner, or CIS strategy. Default is set to the clinical ‘n-of-m’ processing strategy, Advanced Combination Encoder (ACE) depicted in Fig. 4. After pre-emphasis and frame-by-frame decomposition, the signal is passed through the 22-channel filter bank and energy and envelope information calculated from the output of each channel. Selection of ‘n’ highest channels according to spectral energy are selected for stimulation, (i.e. if all channels are selected (‘n’ = ‘m’), the CIS strategy stimulates each electrode in numerical order). The amplitudes of each selected channel are logarithmically compressed and clinical levels are assigned according to the CI-user-specific MAP (electric to acoustic mapping of individual’s hearing loss). Finally, biphasic pulses are generated within the software suite and information sent to the CCi-MOBILE interface board for stimulation to the CI user. All processing is computed using MATLAB using either a host PC or Android phone for computational power.

Fig. 4.

Basic block diagram of ACE processing strategy to simulate the CI-users signal used within CCi-MOBILE research platform.

C. Android Application

The CCi-MOBILE Android application demonstrates both power and flexibility of the research platform through a simple interface that can be easily customized for each user’s needs. In the base app, user MAPs can be selected and saved via the home page which also provides easy access to common parameters such as sensitivity and gain. Streaming to the CCi-MOBILE board is controlled through a start/stop button on the app, and board connection status is conveyed through text and icons. The settings page provides users with full access to all changeable parameters, and a demonstrational environments page allows preset acoustic environments to be quickly and easily selected (e.g., classroom, driving, conversation). The Android application was written in Java using Android Studio, and the CI user MAPs are formatted in JSON.

III. PERFORMANCE EVALUATIONS AND SAFETY

A three tier testing paradigm was developed to assess the CCi-MOBILE research platform for efficacy, performance, safety, and integrity. Functionality of the device was first compared against a standard clinical processor for sentence intelligibility. A comprehensive evaluation of acoustic inputs were verified to produce electric output within safety constraints. Lastly, the integrity and performance was quantified dependent on user-defined input parameters.

A. Functional Validation

This section describes the one-to-one comparison of human CI speech recognition evaluated for CCi-MOBILE against a standard clinical processor [3].

(1). Experimental protocol:

Eight post-lingually deafened adult CI users participated in this evaluation and were scored for words correct on a sentence recognition task. The speech corpus battery included sentences from AzBio, IEEE, CNC, and BKB-SIN databases (at various SNR levels and noise types). Subjects were presented speech through a loud-speaker setup within a sound booth using their clinical processor and CCi-MOBILE with equivalent processing parameters.

(2). Results:

Mean sentence recognition across the entire speech battery for CI users resulted in 59.86% and 56.38% with CCi-MOBILE and clinical processor respectively. Statistical evaluation revealed no significant mismatch effect for the processing platform (F_7,49=4.882, p=0.069).

B. Performance Under Diverse Acoustical Conditions

In this section, electric stimuli produced from CCi-MOBILE research platform were assessed across an extensive audio/speech data set to ensure current levels were maintained within safety limits independent of acoustic input [4].

(1). Testing protocol:

A diverse set of acoustic conditions spanning 11 databases included 260, 46, and 76 hours of speech, music, and noise respectively. Only clinical levels (corresponding to the current levels sent to the intracochlear electrodes) are recorded from CCi-MOBILE to examine the effect on loudness and safety.

(2). Results:

Electric stimuli produced from CCi-MOBILE did not exceed safety limits for the entire 380+ hours of audio data tested. Software and firmware routines used to develop electric stimuli remained with the safety limits of cochlear implant stimulation to ensure that CCi-MOBILE is a safe, reliable research platform.

C. User Defined Parameter Testing

CCi-MOBILE gives freedom to researcher by allowing users to define a subset of signal processing parameters. This section describes our testing paradigm to ensure user-defined settings remain within operational limits of the platform. Total possible user-defined parameters were first quantified based on three input parameters: (a) stimulation rate, (b) pulse width, and (c) number of electrodes. Functional implementation of user-defined configurations were determined for feasibility. Feasible configurations were then checked for integrity (i.e. parameters set at input were recorded at output).

(1). Testing protocol:

Each of the three input parameters were tested iteratively with integer step sizes within respective operational ranges: (a) stimulation rate (125–14400 pulses per sec), (b) pulse width (25–400µs), (c) number of electrodes (1–22). This resulted in a total of 242,880 possible user-defined configurations. Each combination was classified as realizable or non-realizable based on operating specifications. Non-realizable configurations are combinations of user-defined parameters that cannot produce a corresponding electrical stimuli within operational constraints of CCi-MOBILE. Realizable configurations were further tested for integrity by recording the electrical stimuli produced by CCi-MOBILE using the DIET-BOX (Cochlear Corp.). Integrity was quantified by sample loss and error (measured on a sample by sample basis).

(2). Results:

A total of 73,255 configurations of user-defined parameters were classified as non-realizable, while the remaining 169,625 configurations were classified as realizable. Approximately 70% of user-defined combinations can produce a feasible electrical output. Further evaluation was done on the realizable configurations to quantify if built-in software and firmware self-checking algorithms made adjustments to the user-defined parameters (denoted as ‘adjusted valid’). A large number of adjusted valid realizable configurations, 162,818 were observed compared to 6,807 combinations that were accepted without changes made from self-checking algorithms (denoted as ‘true valid’). Based on the true valid configurations, only 2,848 configurations (or approximately 2% of the total reliable configurations) are unique. Adjustments to user-defined parameters resulted in commonly repeated configurations.

Only unique and true valid configurations of user-defined parameters were evaluated for integrity (i.e., 9,656 configurations). Electric stimuli consists of current levels at the corresponding electrodes produced from the aforementioned valid configurations. Current levels and electrodes were recorded from output of the RF coils based on a subset of the acoustic test battery described in Sec. III.B for each configuration. Integrity of the CCi-MOBILE research platform was confirmed for all electrical stimuli produced from valid configurations. This ensures that the software and hardware communication routines are transmitting all information without error. In depth analysis of data transmission was performed to quantify any amount of samples lost between the software suite and electrical output. Sample loss was quantified based on percentage of total samples in the acoustic input. Fig. 5 illustrates that nearly 99% of user-defined stimulation configurations incur a sample loss of less than 1%. Only 20 of 9,656 configurations resulted in sample loss greater than 85%.

Fig. 5.

Histogram of sample loss (as percentage of total number of valid configurations) across user-defined stimulation configurations.

IV. Applications

The multi-disciplinary nature of CCi-MOBILE enables a range of investigations within the field of hearing science and communications. Applications and templates within the software suite have been developed for speech scientists to conduct experiments for CIs, HAs, and bimodal users. Saba et al. [6] implemented a proposed channel selection algorithm, one of the CI signal processing steps, within the software routine of CCi-MOBILE to investigate the effect of a new selection criteria on speech intelligibility. Efficacy of the proposed signal processing strategy was determined from eight CI subjects in a direct-connect listener evaluation [6]. Unlike traditional listening experiments where the participant is presented speech in a sound booth through loud speakers, CCi-MOBILE can directly communicate with the intra-cochlear array, bypassing the proprietary sound processing schemes programmed within the user’s clinical processor. Additionally, a vocoder was developed within the CCi-MOBILE software suite to yield an acoustic approximation for NH listeners of the electric input heard by CI users [7]. For scientists exploring acoustics in various environments and noise types (e.g., the multi-talker phenomenon known as the cocktail party effect), the platform can record naturalistic audio without the user being constrained to a sound booth. Integration of pre-processing strategies with the software suite, such as speech enhancement and noise suppression, can enable engineers to collect subjective evaluations independent of real-time optimization in an iterative manner without qualitative and quantitative metrics. Other uses for the CCi-MOBILE research platform can be demonstrated for lateralization, localization tasks, and the exploration of specific HA/CI parameters (volume, frequency-allocations, dB gain, etc.).

V. CONCLUSION

This study has presented an overview of the CCi-MOBILE research platform, an open-source and flexible tool developed at UT-Dallas to stimulate the field of speech science. Several domains were considered, including hardware, software/firmware, signal processing, safety, and applications. To ensure the safety and reliability of the platform, hardware stimulation and software verification tests were simulated for a wide range of acoustic signals. A successful testing paradigm of selecting valid user configurations was explained and quantified (i.e. the number of possible user requested configurations versus valid implemented setting configurations). Results showed that the CCi-MOBILE platform is a safe and reliable device for any valid set of user configurations. This system can be used to enable parameter customization while facilitating similar performance to clinical processor.