Stable long-term BCI-enabled communication in ALS and locked-in syndrome using LFP signals

Abstract

Restoring communication for people with locked-in syndrome remains a challenging clinical problem without a reliable solution. Recent studies have shown that people with paralysis can use brain-computer interfaces (BCIs) based on intracortical spiking activity to efficiently type messages. However, due to neuronal signal instability, most intracortical BCIs have required frequent calibration and continuous assistance of skilled engineers to maintain performance. Here, an individual with locked-in syndrome due to brain stem stroke and an individual with tetraplegia secondary to amyotrophic lateral sclerosis (ALS) used a simple communication BCI based on intracortical local field potentials (LFPs) for 76 and 138 days, respectively, without recalibration and without significant loss of performance. BCI spelling rates of 3.07 and 6.88 correct characters/minute allowed the participants to type messages and write emails. Our results indicate that people with locked-in syndrome could soon use a slow but reliable LFP-based BCI for everyday communication without ongoing intervention from a technician or caregiver.

NEW & NOTEWORTHY This study demonstrates, for the first time, stable repeated use of an intracortical brain-computer interface by people with tetraplegia over up to four and a half months. The approach uses local field potentials (LFPs), signals that may be more stable than neuronal action potentials, to decode participants’ commands. Throughout the several months of evaluation, the decoder remained unchanged; thus no technical interventions were required to maintain consistent brain-computer interface operation.

INTRODUCTION

Communication is a major goal for people with locked-in syndrome (LIS) (Plum and Posner 1972), who due to their motor impairment cannot move their limbs or speak but remain conscious and mentally engaged (Bauer et al. 1979). Enabling people with LIS to communicate has a positive effect on their quality of life (Bach 1993; Hecht et al. 2002) and can promote reintegration into society, improving the ability to lead a fulfilling, productive life (Doble et al. 2003; Laureys et al. 2005; Young and McNicoll 1998).

Brain-computer interfaces (BCIs)—devices that translate a user’s neural activity into computer commands—have emerged as promising tools for helping people with LIS communicate. Previous studies have used BCIs for communication with people with LIS (Bacher et al. 2015; Birbaumer et al. 1999; Chaudhary et al. 2017; Gallegos-Ayala et al. 2014; Jarosiewicz et al. 2015; Kennedy et al. 2000; Kübler et al. 1999, 2001, 2005; McCane et al. 2014; Nijboer et al. 2008; Sellers et al. 2014; Vansteensel et al. 2016). Some of these studies relied on shaping of neural responses using biofeedback to achieve accuracy permitting communication, a lengthy process lasting for weeks and, in some cases, requiring EEG preparation by technicians before each use (Birbaumer et al. 1999; Chaudhary et al. 2017; Kübler et al. 1999, 2005; Vansteensel et al. 2016). Other studies relied on daily calibration of BCIs, which required intermittent assistance of highly skilled engineers (Bacher et al. 2015; Gallegos-Ayala et al. 2014; Nijboer et al. 2008; Sellers et al. 2014). A BCI system capable of providing independent communication to people with LIS, where technical support or interaction with a caretaker is not required frequently, still needs to be developed.

Among these approaches, BCIs based on intracortically recorded action potentials have been used to achieve communication by a person with incomplete LIS (Bacher et al. 2015; Jarosiewicz et al. 2015) or amyotrophic lateral sclerosis (ALS) (Gilja et al. 2015; Jarosiewicz et al. 2015; Pandarinath et al. 2017); and enable people with tetraplegia to control robotic arms (Aflalo et al. 2015; Collinger et al. 2013; Hochberg et al. 2012, 2006; Wodlinger et al. 2015) or their own muscles using functional electrical stimulation (Ajiboye et al. 2017; Bouton et al. 2016). In addition to spiking signals, intracortically implanted microelectrodes also record local field potentials (LFPs), neuronal signals thought to represent the summed activity of neuronal populations in the vicinity of the microelectrode (Buzsáki et al. 2012). LFPs recorded in the motor cortex of people with paralysis modulate with attempted movements in multiple frequency bands (Ajiboye et al. 2012; Hochberg et al. 2006) and have enabled a person with LIS to spell using a BCI (Kennedy et al. 2000). Information about arm actions has been found in nonhuman primates in both signal modalities (Ajiboye et al. 2012; Bansal et al. 2012, 2011; Flint et al. 2012a, 2012b, 2013; Heldman et al. 2006; Mehring et al. 2003; Rickert et al. 2005; Stark and Abeles 2007).

Sorted spike recordings from intracortical implants can be unstable across periods of days to months (Dickey et al. 2009; Perge et al. 2013, 2014; Simeral et al. 2011). Signals from individual neurons can be lost due to array movements as small as a few tens of micrometers in relation to the brain. The number of recorded well-discriminated neurons can also diminish over time, due to degradation of the electrode materials, biological responses to the sensor, or other factors (Barrese et al. 2013; Bjornsson et al. 2006; Shain et al. 2003; Simeral et al. 2011). Partly in response, previous studies have predominantly used frequent recalibration of decoders, often by highly trained engineers and researchers on site, to maintain a high level of performance despite signal instability. While recent studies showed that able-bodied nonhuman primates can use BCIs with an unchanged decoder for over six months without a significant drop in performance (Flint et al. 2013; Nuyujukian et al. 2014), other recent studies pointed toward comparatively greater intracortical signal variation in people (Jarosiewicz et al. 2015; Perge et al. 2014). One approach to this problem is to build an adaptive decoder (Jarosiewicz et al. 2015). In addition, multiunit threshold crossings have been proposed as a more stable alternative to sorted spikes (Flint et al. 2016; Fraser et al. 2009; Gilja et al. 2015). Alternatively, LFP signals could be employed to achieve long-term reliable BCI control.

Here we demonstrate stable, long-term LFP-based communication in two people with tetraplegia. Our study included an individual with LIS, unable to move his limbs or speak, and an individual with progressive tetraplegia secondary to ALS. They used an LFP-based BCI with a decoder unchanged for 76 and 138 days to communicate with family members, type messages, and send emails, without a loss of performance (Fig. 1). Our study demonstrates that an intracortical LFP-based BCI can be used for independent communication without the need for recalibration, thereby reducing the need for caregiver and/or family intervention during communication.

Fig. 1.

LFP-based communication brain-computer interface enables a person with a locked-in syndrome (T2) and a person with tetraplegia and ALS (T6) to write messages over long time periods without decoder recalibration and with minimal technical intervention. Top: the BCI used a historical decoder that was calibrated to data recorded over a period of 23 and 42 days (blue) and then remained unchanged during the use of the BCI for communication for 76 and 138 days afterward (red) for T2 and T6, respectively. Middle: the participants sat in front of a computer monitor that displayed a BCI-controlled speller FlashSpeller. The participants selected the options by attempting or executing selected movements, which were accompanied by LFP responses decoded as selections. The gray arrow traces the processing of the intracortical neural signals. Raw signals from each of 96 electrodes (first box) were processed into low-pass filtered, intermediate frequency and high-frequency component features (second box). Out of these 288 features, we selected 50 (third box). These features were analyzed by the decoder that outputted a “click” probability (fourth box). If the probability crossed the threshold, the option was selected (fifth box). If no selections were detected during the option presentation time (1.5 s-2 s), the FlashSpeller presented the next option in the cue. Bottom: the FlashSpeller is a two-tier speller. The first tier is composed of groups of characters, space (SP>>), and backspace (<<BK) options. Once a group of characters is selected, FlashSpeller enters into the second tier composed of individual characters. Selecting a character adds it to the entered text and restarts the first tier cycle. If all second tier options have been passed, the FlashSpeller continues the first tier cycle. If all first tier options have been passed, the first tier cycle restarts from the backspace option. Additional “word completion” option was added during free-spelling, which allowed completion of a partially spelled word (middle; purple panel). ALS, amyotrophic lateral sclerosis; BCI, brain-computer interface; LFP, local field potential.

Our results provide evidence that intracortical BCIs could be used in-home, for months and with minimal technical oversight, all of which are necessary for a practical communication system. With this new benchmark for stable, LFP-based neural control, future work can continue to address more complex control modalities and higher-resolution neural signals to achieve stable control of more complex neuroprosthetic systems. This study is a step toward a reliable and robust BCI that will allow people with LIS to communicate independently and, therefore, provide greater and more extensive interactions with their friends, family, and caregivers.

MATERIALS AND METHODS

Participants.

Permission for these studies was granted by the US Food and Drug Administration (Investigational Device Exemption) and Institutional Review Boards of Stanford University (protocol no. 20804), Partners Healthcare/Massachusetts General Hospital (2011P001036), Providence VA Medical Center (2011-009), and Brown University (0809992560). The participants in this study were T2, a man with classic LIS secondary to a brain stem stroke, and T6, a woman with tetraplegia resultant from ALS. They were enrolled in a pilot clinical trial of the BrainGate Neural Interface System (http://www.clinicaltrials.gov/ct2/show/NCT00912041). As part of the trial, the participants received one 96-channel intracortical multielectrode array (Blackrock Microsystems) in the arm area of the dominant precentral gyrus (Yousry et al. 1997) using a previously described procedure (Hochberg et al. 2006). Both participants were right handed. At the time of the study, T2 was fed through a percutaneous gastrostomy tube and had nighttime-only volume-control lung ventilation to support his breathing. His respiratory drive remained intact throughout the study. T6 had a tracheostomy and on-demand ventilation, though she did not require constant ventilatory support (mostly at night, similar to T2). She was able to eat. Her ALSFRS-R score was 16. T6 could speak and use accessibility devices to control a computer with limited hand movement. Informed consent was obtain from both participants. The participant with LIS (T2) had sufficient eye movements to ask questions by choosing letters from an audibly recited alphabet. Consent was further observed and attested to by a family member and third party witness. Neural recordings were collected, stored, and processed for online BCI control as described previously (Simeral et al. 2011).

Task.

Participants took part in research sessions in their homes during which they interacted with the FlashSpeller text-entry application through selections: discrete events identified their attempts to perform a particular movement based on their neuronal activity. The FlashSpeller scanned through character-entry options and other options such as backspace, space, word completion, text-to-speech, or email. An option was presented for 1.5 s for T6 and 2 s for T2. Presentations of two options were separated by a period of 0.1 s for T6 and 0.3 s for T2. If selected, the option button turned green for 0.9 s. As selected movements, referred to as “click actions” in the remainder of the text, T2 attempted to squeeze his right hand and T6 flexed her right index finger.

Each individual session was divided into blocks lasting up to 12 min each. We asked the participant to perform one of four tasks, depending on the block type: 1) normalization (OLC)—observe the FlashSpeller automatically typing a phrase while not interacting in any way; 2) open-loop calibration (OLC)—observe the FlashSpeller automatically typing a phrase while trying to perform click actions as if to spell the phrase; 3) copy-phrase—control the FlashSpeller using a BCI with either a same-day decoder or the historical decoder to type an assigned phrase; and 4) free-spelling—control the FlashSpeller using a BCI with a historical decoder to type a message of choice.

Three types of sessions were run in this study: collection, comparison, and communication (Fig. 2). All sessions started with a normalization block. In collection sessions, we collected open-loop data through open-loop calibration blocks that were used afterward to calibrate a historical decoder. In comparison sessions, after three open-loop calibration blocks were used to calibrate a same-day decoder, the historical and same-day decoder were compared in a double-blind fashion using copy-phrase blocks. Occasionally, comparison sessions would end with one or more free-spelling blocks. In communication sessions, the historical decoder was used for free typing.

Fig. 2.

Schemes of the sessions used in our study. Top left: scheme of the data collection sessions. These started with a normalization block and proceeded with two to four open-loop calibration blocks. Recordings from five consecutive data collection sessions were used to calibrate the historical decoder, which remained unchanged throughout all subsequent sessions. Bottom: scheme for the comparison sessions. These started with a normalization block and three open-loop calibration blocks. Same-day decoder was calibrated to the three open-loop calibration blocks and the historical decoder was loaded from file. The two decoders were then blind randomized and used in the four pairs of copy phrase blocks, each decoder once in each pair. The historical decoder was used for typing in the free-spelling blocks that followed. Top right: scheme of the Communication session in which a normalization block was followed by free-spelling blocks in which the historical decoder was used for typing. Comparison and communication sessions ended when the participant did not express his intention to start a new block.

Preprocessing.

Neuronal recordings were first down sampled to 15 kHz. We then computed power spectral densities for each individual channel to exclude channels that exhibited large peaks at a line noise frequency (60 Hz) or its harmonics. We then referenced the recordings to the common average calculated over the remaining low-noise channels (80 channels for T2 and 33 channels for T6).

Spectral amplitude normalization.

At the beginning of every session, the participant completed a normalization block during which the participant was instructed to observe the monitor during the presentation of an open-loop calibration block but not to respond to any cues. Recordings from this block were first preprocessed, then low-pass filtered using a 30th order windowed linear-phase finite impulse response filter (WLPFIR) and then downsampled to 1 kHz. A short-time Fourier transform (STFT; 256-ms window; using a Hamming window) was then used to estimate spectral amplitudes in 3.91-Hz-wide frequency bins in 20-ms steps. For each frequency bin, we calculated the average amplitude recorded by a single channel over the normalization block. This produced a normalization matrix of size 96 × 129, where 96 was the number of channels and 129 was the number of frequency bins.

Determining frequency bands of the intermediate- and high-frequency components.

To estimate the appropriate frequency bands that would provide selection-related neuronal responses, we processed the recordings from all open-loop calibration blocks by preprocessing the data, low-pass filtering it using a 30th order WLPFIR, down sampling it to 1 kHz and then using a STFT (256-ms window; using a Hamming window) to estimate amplitudes for each of the frequency bins in 50-ms steps. Amplitudes were then normalized using the spectral amplitude normalization calculated from the normalization block. Selection-related epochs were then defined as the time from the selection cue until 1.8 s after the cue for T2 and until 1.5 s after the cue for T6. Amplitudes calculated during the remaining time (no cue or cue that should not be responded to) were taken as baseline. We then calculated the signal-to-noise ratio (SNR) between cue-related neuronal responses and the baseline activity for each electrode el and for each of the frequency bands, defined by the starting and ending bin of a given frequency band, b_S and b_E respectively (Milekovic et al. 2012).

$${\widehat{X}}_i^{el}\left(t;b_S,b_E\right)=\frac{1}{b_E-b_S}{\displaystyle \sum _{b=b_S}^{b_E}{X}^{el}\left(t-t{c}_i;b\right)}$$

$$\begin{array}{l}{\mu }_C^{el}\left(t;b_S,b_E\right)=\frac{1}{K}{\displaystyle \sum _{i=1}^K{\widehat{X}}_i^{el}\left(t;b_S,b_E\right)}\\ {\sigma }_C^{el}\left(t;b_S,b_E\right)=\frac{1}{K}\sqrt{{\displaystyle \sum _{i=1}^K{\left[{\widehat{X}}_i^{el}\left(t;b_S,b_E\right)-{\mu }_C\left(t;b_S,b_E\right)\right]}^2}}\\ {\mu }_B^{el}\left(b_S,b_E\right)=\frac{1}{T}{\displaystyle \sum _{t\in T_B}{\widehat{X}}^{el}\left(t;b_S,b_E\right)}\\ {\sigma }_B^{el}\left(b_S,b_E\right)=\frac{1}{T}\sqrt{{\displaystyle \sum _{t\in T_B}{\left[{\widehat{X}}^{el}\left(t;b_S,b_E\right)-{\mu }_B\left(b_S,b_E\right)\right]}^2}}\end{array}$$

$$SN{R}^{el}\left(t;b_S,b_E\right)=\frac{\left|{\mu }_C^{el}\left(t;b_S,b_E\right)-{\mu }_B^{el}\left(b_S,b_E\right)\right|}{{\sigma }_C^{el}\left(t;b_S,b_E\right)+{\sigma }_B^{el}\left(b_S,b_E\right)}$$

where X^el(t;b) is the amplitude in the frequency bin b and estimated from a window that ends at time t on an electrode el, K is the number of all selection-related epochs in a session, T_B is the set of all time points that belong to the baseline, and T is the cardinal size of T_B. To reach a single value representing the signal to noise ratio as a function of a frequency band, we evaluated the frequency band SNR by calculating the maximum SNR over time and averaging it over all electrodes:

$$\begin{array}{l}SNR\left(b_S,b_E\right)=\frac{1}{N_{el}}{\displaystyle \sum _{el}SN{R}^{el}\left(t_{MAX};b_S,b_E\right)}\\ t_{MAX}=\underset{t}{\max \hspace{0.17em}\arg }\left[SN{R}^{el}\left(t;b_S,b_E\right)\right]\end{array}$$

This procedure was performed for collection sessions from session days 452, 453, 460, 467, and 474 for T2; and session days 313, 322, 336, 348, and 355 for T6.

These same sessions were used to build the historical decoders for each of the participants. All of these sessions showed three well-isolated local maxima of the SNR(b_S,b_E) (Fig. 3). One maximum was present in the very low frequencies, typically 0–6 Hz. Selection-related neuronal responses in this band were captured by the low-frequency component (LFC) obtained by low-pass filtering the common-average referenced signals using the filter with a −3-dB cutoff frequency of 4.63 Hz. We determined the start and end bins for the other two frequency bands for each of the five sessions and used the average as the start and end bins for the intermediate- and high-frequency components (IFC and HFC, respectively; Table 1).

Fig. 3.

Signal-to-noise ratio (SNR) for five collection sessions, identified by number, for both participants (T2 and T6), averaged over all channels, which have been used to determine intermediate- and high-frequency component (IFC and HFC, respectively) frequency bands. SNR has been normalized to the maximum value for each session. The black circles mark the frequency band that gave the highest SNR in the intermediate- and high-frequency ranges.

Table 1.

SNR local maxima for five collection sessions used to determine the IFC and HFC frequency bands (average) for each participant

Sessions	Intermediate-Frequency SNR Local Maximum		High-Frequency SNR Local Maximum
	First bin	Last bin	First bin	Last bin
T2
452	5 (14 Hz)	9 (33 Hz)	14 (49 Hz)	42 (162 Hz)
453	5 (14 Hz)	9 (33 Hz)	16 (57 Hz)	60 (232 Hz)
460	5 (14 Hz)	8 (29 Hz)	14 (49 Hz)	45 (174 Hz)
467	5 (14 Hz)	9 (33 Hz)	15 (53 Hz)	41 (158 Hz)
474	6 (18 Hz)	8 (29 Hz)	14 (49 Hz)	61 (236 Hz)
Average	5 (14 Hz)	9 (33 Hz)	15 (53 Hz)	50 (193 Hz)
T6
313	6 (18 Hz)	9 (33 Hz)	16 (57 Hz)	55 (213 Hz)
322	7 (21 Hz)	9 (33 Hz)	18 (64 Hz)	90 (350 Hz)
336	7 (21 Hz)	14 (53 Hz)	18 (64 Hz)	81 (314 Hz)
348	7 (21 Hz)	9 (33 Hz)	17 (61 Hz)	86 (334 Hz)
355	7 (21 Hz)	13 (49 Hz)	17 (61 Hz)	53 (205 Hz)
Average	7 (21 Hz)	11 (41 Hz)	17 (61 Hz)	73 (283 Hz)

Numbers in parentheses show the rounded lower edge frequency of the first frequency bin or the rounded top edge frequency of the last frequency bin. HFC, high-frequency component; IFC, intermediate-frequency component; SNR, signal-to-noise ratio.

Extraction of neural features.

We then extracted three different components of the LFPs: 1) LFC, 2) IFC, and 3) HFC. LFC was extracted by low-pass filtering the neural recordings (−3-dB cutoff frequency of 4.63 Hz). To extract IFC and HFC, we first calculated the spectral amplitudes of neural recordings using an STFT. We then divided the amplitude in each frequency bin by its mean in the normalization block and averaged these normalized amplitudes across the IFC and HFC bands. We then defined a neuronal feature, Φ_f, at time t as one of LFC, IFC, or HFC derived from the electrode el:

$${\Phi }_f\left(t\right)=\{\begin{array}{l}LF{C}^{el}\left(t\right)\\ IF{C}^{el}\left(t\right)\\ HF{C}^{el}\left(t\right)\end{array}$$

To account for the drifts of the neuronal features, we z-scored the features using recursive estimates of the mean and variance (see Adaptive feature normalization in the appendix).

Decoder calibration and feature selection.

Data for historical decoder calibration were collected over several collection sessions spanning up to 42 days. This data was then used to rank all 288 available features according to the SNR between peri-click and baseline feature values (see SNR used for selection of neural features in the appendix). We then selected the top 5 LFC, top 5 IFC, and top 5 HFC features, followed by the top 35 remaining features for a total of 50. This procedure selected 16 LFC, 5 IFC, and 29 HFC features for T2, and 5 LFC, 5 IFC, and 40 HFC features for T6.

These features were then used to calibrate a regularized linear discriminant analysis (rLDA) classifier that calculated the selection probability for a given set of feature values (see Calibration of decoding algorithms in the appendix). Parameters of this historical decoder were optimized using the normalized mutual information (C_YX) between the decoded selection and selection cues provided during the open-loop calibration blocks of the collection sessions. A same-day decoder was calibrated to data from three open-loop calibration blocks recorded on that day following the same procedure.

Decoding algorithm.

An rLDA classifier, i.e., either a historical or a same-day decoder, was used to calculate the probability of a selection every 20 ms. When the probability crossed a threshold of 95%, a selection command was sent to the text-entry application. After the selection was detected, a refractory period of 0.5 s was imposed, during which another selection would not be detected. In addition, the probability had to drop to 75% or below before another selection could be detected.

Cross-session stability of LFP responses.

To investigate the stability of LFP signals over time, we calculated the correlation coefficient between mean single-feature LFP responses during click actions recorded on two sessions s1 and s2, r_f(s1,s2) (see Measuring cross-session stability of LFP responses in the appendix). To extract an overall estimate of stability of LFP signals between two sessions, we then calculated the median of r_f(s1,s2) over all features, M_r(s1,s2). To estimate the trends of LFP signal stability, we examined the relationship between the M_r(s1,s2) and the number of days between two sessions. A slope b of the line fitted to M_r(s1,s2) as a function of time using ordinary least squares gave us the estimate on the rate of instability of the LFP signals.

Performance measures.

Decoding accuracy was measured using normalized mutual information C_YX (Coombs et al. 1970), a metric based on information theory, between the decoded selections and either the selection cues provided to the participants, in open-loop blocks, or the assumed selections a participant wanted to perform based on the final spelled phrase in copy-phrase and free-spelling blocks. C_YX was used to optimize the parameters of a decoding algorithm during calibration and to obtain a measure of spelling performance that was independent of the spelled phrase. The spelling performance of our communication interface was measured by counting the number of correctly typed characters per minute. For a more elaborate derivation of these measures, see Measuring spelling and decoding performance in the appendix.

Retrospective comparison to a hypothetical single electrode.

It has been shown that people can use BCIs based on neural signals recorded from the surface of the cortex (electrocorticography) for rapid spelling (Brunner et al. 2011) and control of computer cursors and robotic arms (Wang et al. 2013). For a simple, switch-based communication demonstrated in this study, a BCI based on single electrocorticographic electrode might perform comparably to a BCI based on LFPs recorded from the whole implanted intracortical array, while covering a similar cortical surface. We explored this secondary question by simulating the case where only one broadband signal—the average of LFP signals recorded across the intracortical array—was used for neural decoding. In a retrospective analysis, this average LFP signal was used to define intermediate- and high-frequency bands and to calibrate an average signal decoder based on open-loop calibration blocks of the data collection sessions. We then used this decoder to decode participant’s actions during the open-loop blocks of the comparison sessions. Since participant did not receive any decoding feedback during these blocks, this allowed for a fair comparison between this average signal decoder and the historical decoder that used LFPs from multiple electrodes. Furthermore, while the closed-loop blocks could vary in size, the open-loop calibration sequence was standardized for each participant over all sessions. Therefore, the same amount of data was used to calibrate (15 OLC blocks) and test (3 OLC blocks) both historical and average signal decoders on each session. We compared the decoders by calculating C_YX from offline decoding. To prevent an undue penalty on the average signal decoder from particularly noisy signals originating from one or more electrodes, the average LFP was calculated using only the LFPs from the channels that contributed to the common average in the multiple-electrode case.

RESULTS

Interaction of participants with a FlashSpeller communication BCI.

Our study shows that two people with tetraplegia, one of whom was locked-in, can spell text at a rate of about a word per minute using a FlashSpeller BCI with a decoder that remained unchanged, without a drop in performance, for time periods of several months. This BCI decoder was calibrated on data collected during five 1-day sessions (spanning a period of up to 42 days) and was then used unchanged for communication for up to 138 days afterward. To spell messages on a computer monitor, two participants—an individual with LIS secondary to brain stem stroke (T2) and an individual with tetraplegia secondary to ALS (T6)—used a custom text-entry application, called FlashSpeller, with an LFP-based BCI (Figs. 1 and 4; Table 2). FlashSpeller was designed for individuals with limited or no eye movement, which occurs commonly in people with LIS. In the middle of the screen, a large gray button displayed options for 1.5–2 s. To select the currently displayed option, i.e., to “click,” the participant attempted or executed a chosen movement. LFP responses elicited during these click actions were recorded from the motor cortex and decoded as clicks by the BCI.

Fig. 4.

Example of selection probabilities and selected options throughout the first 316 s of the first free-spelling block, session T6 day 493. During this time, participant T6 spelled the phrase, “Free speech is one of the most important rights.” Top to bottom: successive panels show increasing time resolution of a region outlined in red. Three top rows in each panel display the periods during which the FlashSpeller button was green after selecting an option (Feedback), presenting an option (Option), and empty during transition between options (Transition). Fourth row shows the click probability with a probability threshold. Rows five through seven show three neural features selected for decoding, one for each component. The last row shows all 50 neural features selected for decoding. HFC, high-frequency component; IFC, intermediate-frequency component; LFC, low-pass filtered component.

Table 2.

Examples of sentences typed by participants T2 and T6 using the historical decoder during the free-spelling blocks

Session	Phrase	Time, min	Number of Correct Characters
T2 510	Thankyou for your time friday. We had a g ood time. ****.	21.8	56
T2 524	A tale told by an idiot, full of sound and fury, signifying nothing	15.0	67
T2 529	Would it help if i came saturday in smaller chair?	13.3	50
T6 403	Embracing als is an important step toward living with als.	8.5	59
T6 441	I want to thank all my caregivers who made the trip to hawaii possible	11.0	70
T6 493	Free speech is one of the most important rights we have whether you are able or disabled, and w	11.9	95
	Where technolgy is a	3.8	20
	Available, there should be a constitutional mandate to make available such techn	11.9	80
	Technology to people in need free of charge.	6.2	43

In the phrase from session T2 510, four characters that contained personal information have been redacted (*). In session T6 493, the participant typed a sentence over four free-spelling blocks. Typing was interrupted either due to the block maximum length of 12 min or by the participant.

The study lasted for 99 and 181 days in total for T2 and T6, respectively. At the beginning of the study, we ran “data collection” sessions to accumulate a large number of neuronal responses during participants’ click actions, which were used to calibrate a historical decoder. Data collection sessions were followed by “comparison” sessions, in which the historical decoder was compared with a decoder calibrated on data recorded on the same day (“same-day” decoder) by measuring the participants’ performance in spelling assigned phrases during “copy-spelling” blocks. Often, at the end of comparison sessions, participants used the historical decoder to freely spell messages during “free-spelling” blocks. We also ran separate “communication” sessions, in which the participants used the historical decoder to spell messages. The historical decoder was calibrated only once for each participant and was used in all comparison sessions and all communication sessions.

To create BCI decoders, we identified three different features of LFPs that modulated in relation to participants’ attempts to perform click actions (Figs. 5 and 6A). The low-pass filtered component (continuous signal filtered using a low-pass filter below 4.63 Hz) reflected a slow, large-amplitude deflection related to the selection actions. Activity in the IFC (normalized spectral amplitude in 14–33 Hz for T2 and 21–41 Hz for T6) showed a decrease following the cue, while the HFC (normalized spectral amplitude in 53–193 Hz for T2 61–283 Hz for T6) showed an increase in activity. Modulations of these features could be distinguished from LFPs recorded on individual trials and were used to detect participants’ intentions to select the presented options (Fig. 4, Supplemental Movies S1–S8; Supplemental Material for this article is available online at the Journal website).

Fig. 5.

Examples of participant T2’s neuronal responses to attempts to squeeze his right fist recorded on a single electrode (el. 95) during three open-loop calibration blocks for three sessions 48 or more days apart. Panels show average spectrogram (top row) and single trials of low-pass filtered (second row), intermediate-frequency (third row), and high-frequency (bottom row) component LFP features. Single trials are shown in units of standard deviation (s.d.). For better visualization, the colorbar scale spans the values from 0.5th to 99.5th percentile.

Fig. 6.

Long-term stability of participants’ local field potential (LFP) responses during click actions. A: mean (thick lines) and 95% confidence intervals (thin lines) of single electrode low-pass filtered component (LFC; top), intermediate-frequency component (IFC; middle), and high-frequency component (HFC; bottom) LFP features calculated over all trials recorded in open-loop calibration blocks are shown for three selected sessions at least 1 mo apart (blue, red, and black traces in temporal order). Left and right: signals for participant T2, electrode 95; and participant T6, electrode 80, respectively. Average spectrograms and single-trial LFP responses for these sessions are shown in Fig. 5. s.d., Standard deviation; el., electrode. B: black dots show the median of single-feature cross-session LFP response correlation coefficients [rf(s1,s2)] over all channels for all compared session pairs s1 and s2 against the intersession period. Gray dots show the mean Mr(s1,s2) when the LFP responses are triggered on random events. Gray lines show the linear ordinary least squares fit to Mr(s1,s2) with the slope b shown at the bottom of the panel.

In addition to quantifying the performance and stability of our BCI system, we investigated different causes that led to stable long-term spelling performance, formulated in the following questions: 1) Was the stable performance a direct consequence of the stability of LFP responses that occurred during click actions? 2) Does the resolution of intracortical recordings substantially alter the decoding performance as compared with a hypothetical electrocortigraphic electrode? 3) Was the design of our decoder responsible for stable decoding performance despite the instability of LFP responses during click actions?

Stability of LFP responses to click actions over the whole microelectrode array.

LFP responses during click actions in features used for decoding were similar over the duration of the study (Figs. 5 and 6A). We also measured the stability of LFP signals recorded over the whole microelectrode array over time as follows: For each session, we calculated the mean LFP response time course for each feature, ranging from the click cue to 1.5 s after the click cue, across all click actions recorded during that session’s open-loop calibration blocks (34 and 68 cue responses for sessions T2 466 and T2 467, respectively; 51 cue responses for all other sessions). For each feature’s mean, we then calculated the correlation coefficient between any pair of sessions, hereafter referred to as “single-feature cross-session correlation coefficient.” Finally, we calculated the median cross-session correlation coefficient across all 288 features (96 electrodes times three feature types), thus obtaining a single value for each session pair.

This median single-feature cross-session correlation coefficient did not change significantly with intersession period for T2 (b = −0.02 yr⁻¹; P = 0.78; Spearman’s rank correlation test) while it significantly diminished with the intersession period for T6 (T6: b = −0.21 yr⁻¹; P < 0.05; testing nonzero correlation between the correlation coefficient and intersession period, Spearman’s rank correlation test; Fig. 6B).

Stability of detection based on LFP neuronal responses.

We tested the stability of the LFP-based detection of clicks by using a decoder, calibrated on all open-loop calibration blocks from one collection or comparison session, to detect clicks on open-loop calibration blocks from another session in the future or in the past. The standardized calibration sequence for each participant over all sessions ensured that the same amount of data was used to calibrate (three OLC blocks) and test (three OLC blocks) each decoder. Since participants did not control the spelling during open-loop blocks and, therefore, did not receive any feedback on their performance, they could not adapt their strategy to modify the decoding performance. Decoding performance was measured using normalized mutual information C_YX between the decoded clicks and the clicks that would lead to errorless spelling (Coombs et al. 1970; Milekovic et al. 2013).

For both participants, offline detection performance, as measured by C_YX, was variable, but did not degrade as a function of the number of days since decoder calibration, ΔT (Fig. 7). Correlation between C_YX and ΔT was not significant (T2: P = 0.82; T6: P = 0.46, Spearman’s rank test). Average C_YX calculated this way was 0.68 and 0.72 for T2 and T6, respectively.

Fig. 7.

Demonstration of the detection stability based on stability of local field potentials (LFPs) for both participants T2 and T6. Top: normalized mutual information (C_YX) calculated by calibrating a decoder on all open-loop calibration blocks from one session and validating it on all open-loop calibration blocks of another sessions. The same amount of data was used to calibrate and validate each decoder. Bottom: same C_YX values as a function of the decoder age ΔT (time between the session used to calibrate the decoder and the session used to validate it). Black lines show the linear ordinary least squares fit to the C_YX values. P value of testing for significant correlation between ΔT and C_YX is given.

Spelling performance and other performance measures.

Both study participants controlled the FlashSpeller using LFP signals and a historical decoder to communicate by spelling messages (Fig. 8). The average spelling performance, as measured by the number of correct characters per minute (CCPM), during free-spelling blocks using word completion was 3.07 ± 0.20 CCPM and 6.88 ± 0.33 CCPM for T2 and T6, respectively. These rates enabled participants to spell full sentences within several minutes. The period over which the unchanged historical decoder was used by the participants for communication spanned 76 and 138 days for T2 and T6, respectively. During this period the participants’ spelling performance did not change significantly (P value for testing nonzero correlation between spelling performances during free-spelling blocks and historical decoder age: T2: P = 0.40; T6: P = 0.73; Spearman’s rank correlation test).

Fig. 8.

Spelling performance of participants T2 and T6 when using local field potential (LFP)-based FlashSpeller communication brain-computer interface (BCI). Dots on the left show participants’ spelling performance, as measured by correct characters per minute, as a function of decoder age when using same-day and historical decoders during copy-phrase blocks (yellow and blue dots, respectively) and when using historical decoder during free spelling with the word completion option (purple dots). Broken lines show the linear ordinary least squares fit over all sessions with P values for testing significant correlation between the spelling performance and the age of the historical decoder shown below the lines. Bars on the right side show the mean spelling performance over all sessions. Error bars show 95% confidence intervals. ns, Nonsignificant difference (P ≥ 0.05); *P < 0.05. CCPM, correct characters per minute.

Spelling performance takes into account both the accuracy of the selection detections and the difficulty of the typed phrases. It may be possible that participants chose to type easier phrases during the sessions in which the decoders were not as accurate, thus masking the instability of LFP responses used for detection of selections. We used C_YX to compare the performance of decoders irrespective of the typed phrases (Fig. 9). Over all sessions with free-spelling blocks in which word completion was used, C_YX in those blocks did not change significantly (P value for testing nonzero correlation between C_YX during free-spelling blocks and historical decoder age: T2: P = 0.33; T6: P = 0.13; Spearman’s rank correlation test). The historical decoder spelling performance was not significantly different from the same-day decoder performance (mean spelling performance difference during copy-phrase blocks over all comparison session: T2: 0.19 ± 0.18 CCPM, P = 0.10; T6: 0.03 ± 0.22 CCPM, P = 0.85; t-test). In addition, spelling performance of both the historical and same-day decoders during copy-phrase blocks did not change significantly over the time we conducted the comparison sessions (P value for testing nonzero correlation between spelling performances during copy-phrase blocks and historical decoder age: same-day decoder: T2: P = 0.68; T6: P = 0.95; historical decoder: T2: P = 0.18; T6: P = 0.48; Spearman’s rank correlation test). Furthermore, C_YX did not change significantly over the same time (P value for testing nonzero correlation between C_YX during copy-phrase blocks and historical decoder age: same-day decoder: T2: P = 0.78; T6: P = 0.67; historical decoder: T2: P = 0.42; T6: P = 0.76; Spearman’s rank correlation test), thus further confirming that LFP responses remained stable over the tested period.

Fig. 9.

Performance of participants T2 and T6’s interaction with the FlashSpeller application as measured by the normalized mutual information (C_YX). Left: C_YX when using same-day and historical decoders during copy-phrase blocks (yellow and blue bars, respectively) and when using historical decoder during free spelling with and without the word completion option (full and empty purple bars). Maximum theoretical performance, described as participant always selecting the intended spelling option or character, is 1. Right: the C_YX as a function of time (colored dots) and its linear ordinary least squares fit (broken lines). Top right corner displays P values for testing significant correlation between the C_YX and time. Performance is shown as mean C_YX. Error bars show 95% confidence intervals. Error bars show 95% confidence intervals. ns, Nonsignificant difference (P ≥ 0.05); *P < 0.05.

The participants self-assessed the performance of both historical and same-day decoders on a scale from 1 to 10 (best) after each copy-phrase block (Fig. 10). Scores were the same for both decoders for all participants. On average, T2 scored the historical decoder with 9.50 ± 0.15 and the same-day decoder with 9.34 ± 0.37, T6 scored the historical decoder with 9.23 ± 0.43 and the same-day decoder with 8.94 ± 0.47. The score differences in all participants were not significant (T2: P = 1; Wilcoxon signed rank test; T6: P = 0.47; Wilcoxon rank sum test). The score in one of the blocks in session T6 473 was not recorded.

Fig. 10.

Participants T2 and T6’s self-assessed quantitative scores of historical and same-day decoders during copy-phrase blocks. Each colored dot shows the participant’s score for a single copy-phrase block. Score was given on a scale from 1 to 10 (best). Bars on the right show the mean score over all scored blocks and error bars show the 95% confidence intervals. ns, Nonsignificant difference (P ≥ 0.05).

Contribution of intracortical resolution.

We compared decoding performance of the average signal decoder, which used the average LFP across the array and represented a simulated single electrocorticographic electrode signal, to the historical decoder. For both participants, the historical decoder was more effective (P < 0.05, bootstrap test; Fig. 11).

Fig. 11.

Hypothetical performance of an average signal decoder compared with the performance of the historical decoder. We simulated the case where the intracortical microelectrode array was replaced with a single electrocorticographic electrode by averaging the recorded signal over all electrodes. A: as with the historical decoder, we identified IFC and HFC frequency bands for each of the participants from the peri-click SNR in five collection sessions. Note that, to construct the simulated single electrocorticographic electrode signal, we averaged the signals over all low-noise microelectrode array channels before computing the SNR. For participant T6, the HFC frequency band did not emerge in any of the sessions. Therefore, we used only LFC and IFC features to calibrate and use the average signal decoder for T6. B: bar plots show decoding accuracy, as measured by normalized mutual information (C_YX), of the historical and average signal decoder for each participant on individual session days and the mean across all sessions (overall). For all participants, the historical decoder outperformed the simulated average signal decoder (P < 0.05, bootstrap test). The same amount of data was used to calibrate and validate both decoders. HFC, high-frequency component; IFC, intermediate-frequency component; LFC, low-frequency component; SNR, signal-to-noise ratio.

DISCUSSION

Our results demonstrate stable, long-term LFP-based communication with an unchanged decoder in two people with severe paralysis. The unchanged decoder was effective for as long as 138 days. Our participants included one person with long-standing LIS (T2), who was unable to move or verbally communicate, and one person with progressive tetraplegia secondary to ALS (T6). Importantly, over the duration of the study, spelling performance did not change significantly. Both participants used the BCI to communicate with family members, caretakers and researchers by typing full sentences. T2 also used the BCI to write and send emails—the participant first selected the email recipient and title using a caretaker-assisted spelling board, then wrote the email body using FlashSpeller. When writing emails a “send” button was present at the end of each FlashSpeller cycle, which could be selected by T2. T2 requested the use of a send button, because he enjoyed using the FlashSpeller BCI system for personal email communication. T6 used other electronic communication systems for email and preferred to use the FlashSpeller to communicate messages to the research community. Decoders calibrated on the same day did not achieve higher performance when compared with the unchanged historical decoder in double-blind comparisons. During these comparisons, participants scored the performance of historical and same-day decoders equally well. Together, these results show that an LFP-based communication BCI could soon provide people with LIS with a slow but reliable ability to communicate with minimum technical supervision or caretaker intervention. This LFP-based decoding could also be used to enhance higher-throughput, intracortical spike-based communication interfaces (Bacher et al. 2015; Gilja et al. 2015; Jarosiewicz et al. 2015; Pandarinath et al. 2017).

While severely paralyzed, our participants were still able to move their eyes and head and neck muscles. Since eye movements and head and neck muscle activity were not controlled for, artifacts originating from eye, head, and neck movements could have been introduced into our recordings. Nonetheless, such artifacts were likely to be common across signals recorded from a small patch of motor cortex covered by one microelectrode array. Therefore, we minimized their influence by using a common-average referencing step in our signal processing.

Stability of LFP signals.

We show that LFPs recorded from a chronically implanted microelectrode array can serve as the basis for stable long-term control of a switch-based communication interface. A major concern with chronic multielectrode recordings has been their suitability for long-term prosthetic control. It is difficult to reliably detect and isolate activity of single units within a day or from one day to the next (Dickey et al. 2009; Perge et al. 2013). Furthermore, changes to the recording environment can lead to a long-term decline in the number of detectable single units (Chestek et al. 2011; Flint et al. 2012b; Moran 2010). However, when available, isolated single units appear to have a predictable long-term relationship to behavior, suggesting a stable underlying map (Chestek et al. 2007). As such, two signal types have been studied as potentially more stable than isolated single units: multiunit activity derived from threshold crossings of a high-frequency band (0.5–7.5 kHz) signal (Chestek et al. 2011; Flint et al. 2013) and LFPs (Flint et al. 2012b; Perge et al. 2014). At present, the evidence suggests that while threshold crossings are more stable neuroprosthetic control signal than was expected a decade ago, LFPs are still more stable from day to day (Flint et al. 2016) and may persist when threshold crossings are lost (Flint et al. 2012b; Simeral et al. 2011). Therefore, for studies of long-term reliability of neuroprosthetic control, LFPs continue to be a signal of interest.

In our study, it was important to distinguish between contributions to stable communication performance from signal processing, signal selection, and changes in participant control strategy. We found that LFP signals recorded over the whole microelectrode array showed long-term stability only for participant T2, suggesting that T2’s stable communication rates resulted from signal processing. For participant T6, we found significant changes of LFP modulations over time across the entire array (Fig. 6B). This result shows that the underlying signals, i.e., the processed LFPs, could not account alone for T6’s stable communication rates. Either decoder design (that is, feature selection and feature weighting) or T6’s control strategy might account for this discrepancy. We next tested decoding performance on open-loop blocks, during which participants could not adapt control strategies due to lack of feedback (Fig. 7). We found that open-loop decoding performance did not degrade over time for T2 or T6, leading us to conclude that signal selection and feature weights, rather than a change in control strategy, accounted for T6’s stable communication rates. Therefore, population-level approaches can continue to decode a stable behavior-to-neuroprosthetic-command map even when the fidelity of the signal has declined on average.

Selection of LFP features to achieve BCI control.

One objective of this study was to develop an LFP feature selection method that readily generalized across participants. Following the data-driven process to identify informative LFP frequency bands, we derived three features on each channel (288 in total). We considered the case where features in a given frequency band were separately informative but highly cross-correlated, while separately less informative features from other frequency bands might provide independent novel information. One way to capture the diversity of information over all available features is to use all features for decoding. However, this approach increases the risk of overfitting the decoder to the training data. Furthermore, we were constrained by the need to perform all signal processing and decoding computations online, which made it necessary to perform feature selection.

Previous studies in decoding movement direction from motor cortical spiking activity and LFPs in monkeys and humans showed that the marginal benefit of added features diminishes substantially in the 30–50 range (Bansal et al. 2011, 2012; Gilja et al. 2015; Wessberg et al. 2000). Therefore, we made a conservative estimate that 50 features will still contain the majority of movement attempt information. We scored the “informativeness” of each feature by its SNR. To capture diverse information potentially available in different frequency bands, we first selected five features with the highest SNR from each frequency band. We supplemented those 15 features with additional 35 features with the highest SNR irrespective of the frequency band to select the total of 50 features. We further mitigated the risk of overfitting the decoder to the training data by using regularization.

Utility of switch-based BCIs.

Although the primary finding of this study relates to the stability of neurophysiological signals, the efficacy of simple switch-based communication BCIs is also worth underscoring. ALS at moderate and severe stages may result in cortical and/or other changes that could disrupt BCI function, including attention deficits (Pinkhardt et al. 2008; Schreiber et al. 2005; Vieregge et al. 1999) and other cognitive impairments (Cipresso et al. 2012). Furthermore, people with ALS can experience problems with vision due to poor conjugate gaze control and corneal injury (Murguialday et al. 2011). People with LIS secondary to brain stem stroke can also develop oculomotor and visual impairments (Caplan 1980; Sellers et al. 2014). Lastly, people with communication deficits related to neurological disease or injury (for instance, related to traumatic brain injury) may have a range of sensory and cognitive abilities but could still benefit from a reliable method of communication for basic needs. The switch-based communication method described here could, for example, also be used with an auditory interface. For all these reasons, for some people with LIS, ALS, or other neurological disorders simple switch-based communication BCIs may be better suited than more complex communication BCIs designed for high communication throughput, including those previously reported by our group. Additionally, approaches and algorithms that can reliably decode state information from intracortical recordings may point toward more expansive use of BCIs for nontyping communication goals such as control of household technology and emergency alarms, for which reliability is paramount (Kageyama et al. 2014).

The development of a fully implantable amplification and telemetry system is a likely precursor to the widespread adoption of BCI technologies based on intracortical recordings. Safety and efficacy of such devices are currently being studied in animal models (Borton et al. 2013; Capogrosso et al. 2016; Yin et al. 2014), and translation for use in people is ongoing. Future work should also aim to implement the signal normalization block as an ongoing, self-updating process.

Comparison with other communication BCI studies.

Other approaches using different signal modalities have investigated communication BCIs (Chaudhary et al. 2016; Wolpaw and Wolpaw 2012). Nevertheless, recent studies have demonstrated the superior performance of intracortical communication BCIs in people with tetraplegia compared with prior studies (Gilja et al. 2015; Jarosiewicz et al. 2015; Pandarinath et al. 2017). However, for successful clinical applications, these systems have to provide stable performance with minimal technical oversight, despite the instability of intracortical signal recordings (Jarosiewicz et al. 2015; Perge et al. 2014). An approach taken by a recent study from our group used an adaptive decoder that conformed to the signal changes to generate stable performance, which allowed the participant with tetraplegia to use a communication BCI without a loss in performance and without reinitializing the decoder over a period of 42 days (Jarosiewicz et al. 2015). In the current study, we followed an alternate approach of harnessing intracortical LFP signals, which are thought to be more stable than the neuronal spiking signals. Despite changes in LFP signals recorded across the entire array in one participant, the online decoding performance remained stable for both participants throughout the duration of the study, over a period of 76 days for one and 138 days for the other.

In a recent study, a person with LIS used a BCI based on electrocorticographic (ECoG) signals to spell text at a rate of 1.82 CCPM over 70 days with an unchanged decoder and without a drop in performance (Vansteensel et al. 2016). In our current study, a person with LIS used an intracortical LFP-based BCI with an unchanged decoder and without a drop of performance over a similar period (76 days) but achieved 69% higher performance (3.07 CCPM). A previous study from our group has shown that people with LIS can use a communication BCI based on intracortical spiking signals, but it used daily calibration of decoders to preserve performance (Bacher et al. 2015). Another study from our group demonstrated stable spelling performance using an intracortical BCI in an individual with tetraplegia who could still speak (Jarosiewicz et al. 2015). This study thus complements recent intracortical BCI communication studies. While a previous study (Jarosiewicz et al. 2015) adapted the decoder based on data collected on the current day, in this study the historical decoder was calibrated with data pooled across about a month. Therefore, our calibration strategy inherently favored features that remained stable over longer periods. This strategy might also benefit spike-based control. The adaptation strategy used in the previous study relies on reasonable neural control in a point-and-click task after a days-long period of disuse and would likely benefit from the methods developed here. LFP signals and the associated data collection, decoder calibration, signal selection, and signal processing strategies developed in our study could contribute not only to point-and-click BCI communication but also to more general neuroprosthetic control.

Other studies have investigated the use of communication BCIs based on other signal modalities as a communication tool for people with LIS (Bacher et al. 2015; Birbaumer et al. 1999; Kübler et al. 1999, 2005; Nijboer et al. 2008; Sellers et al. 2014). A prior study showed that two individuals with LIS secondary to ALS could communicate using an EEG-based BCI, with one of the participants typing 510 characters for over 16 h (Birbaumer et al. 1999). Another recent study has shown that an individual with LIS secondary to brain stem stroke (state similar to that of T2) can communicate using an EEG-based BCI with a performance of below 1 CCPM (Sellers et al. 2014). In our study, all participants, including a person with LIS, typed at a higher rate.

Our system provided higher performance than reported by any EEG- or ECoG-based spellers tested on a similar population of subjects, i.e., people with ALS and people with LIS (Arya et al. 2013; Bai et al. 2010; Brunner et al. 2010; Chen et al. 2015; Fazel-Rezai et al. 2012; Hamer et al. 2002; Hedegärd et al. 2014; Hill et al. 2006; Hinterberger et al. 2005; Kübler and Birbaumer 2008; Mainsah et al. 2015; Marchetti and Priftis 2015; McCane et al. 2014; Miner et al. 1998; Nijboer et al. 2008; Pires et al. 2012; Vansteensel et al. 2016) (see Table 3 for detailed list of spelling rates reported in other studies). In the two above studies in which the communication rates were similar (McCane et al. 2014; Pires et al. 2012), our BCI communication system was used to type considerably longer and more complex messages, demonstrating that people with paralysis can use our BCI system for spelling and that intracortical recording may be an effective solution.

Table 3.

Comparison of spelling performance between this study and other spelling BCI studies

	Able-Bodied Participants		Participants with ALS Without LIS		Participants with LIS
Speller type	Switch	P300/SSVEP	Switch	P300/SSVEP	Switch	P300/SSVEP
Scalp EEG	0–0.86* (Bai et al. 2010)	2.6 (McCane et al. 2015);	0 (Hill et al. 2006);	<2.1 (Nijboer et al. 2008);	0.5 (Birbaumer et al. 1999);	2.1 (McCane et al. 2014);
		5.48 (Pires et al. 2012);		4.97 (Pires et al. 2012);
		5.83 (Chen et al. 2015)	1.2* (Miner et al. 1998)	1.16 (Mainsah et al. 2015)	<2 (Kübler et al. 2001)	<1 (Sellers et al. 2014)
ECoG	0.51 (Hinterberger et al. 2005)	17 (Brunner et al. 2011)			0 (Hill et al. 2006);	0 (Murguialday et al. 2011)
					1.82 (Vansteensel et al. 2016)
Intracortical LFP-based (this study)			6.88		3.07
Intracortical spike-based using other speller types			10.44–17.40 (Jarosiewicz et al. 2015);		9.4 (Bacher et al. 2015);
			13.5–31.6 (Pandarinath et al. 2017)		1.95–11.75 (Jarosiewicz et al. 2015)

Bold values represent the present study. Other studies include all studies listed in three recent reviews (Fazel-Rezai et al. 2012; Kübler and Birbaumer 2008; Marchetti and Priftis 2015). Performance of BCI-based spellers in able-bodied people, people with ALS who retain the ability to speak, and people with LIS resulting from ALS or stroke, as measured in correct characters per minute. When only selections per minute were reported, we include this value as an upper bound. When only binary selections were allowed we estimated a typing rate based on a simple switch keyboard design, with five selections required per character (marked with *). ALS, amyotrophic lateral sclerosis; BCI, brain-computer interface; ECoG, electrocorticographic signals; LIS, locked-in syndrome; LFP, local field potential; SSVEP, steady state visually evoked potential.

Two recent studies showed that near-infrared spectroscopy (NIRS) can be used to detect recognition of the veracity of a Boolean choice in people with complete LIS (Chaudhary et al. 2017; Gallegos-Ayala et al. 2014), which suggests that some people with complete LIS might remain capable of communication with adequate technology. However, the rate of yes/no detections was slow (around 20–25 s per response, responses ~75% correct), possibly reflecting the low signal-to-noise ratio of NIRS responses. Other attempts to provide individuals with complete LIS secondary to ALS with the ability to communicate using electrocorticography-based BCIs were not yet successful (Hill et al. 2006; Marchetti and Priftis 2015). It remains to be seen whether a LFP-based communication BCI could be used by someone with complete LIS, but our study with participants who have ALS or classic LIS suggests that this approach could be effective.

GRANTS

The research was supported by the National Institutes of Health: National Institute on Deafness and Other Communication Disorders (R01DC009899); Rehabilitation Research and Development Service, Department of Veterans Affairs (B6453R, N9228C, and B6459L); Massachusetts General Hospital Deane Institute for Integrated Research on Atrial Fibrillation and Stroke; Joseph Martin Prize for Basic Research; The Executive Committee on Research of Massachusetts General Hospital; and Doris Duke Charitable Foundation and the Swiss National Science Foundation Ambizione program (PZOOP2_168103/1).

DISCLAIMERS

The content of this paper is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health, the Department of Veterans Affairs, or the United States Government.

DISCLOSURES

No conflicts of interest, financial or otherwise, are declared by the authors.

AUTHOR CONTRIBUTIONS

T.M., A.A.S., D.B., S.S.C., J.P.D., and L.R.H. conceived and designed research; T.M., A.A.S., B.L.S., C.B., E.M.O., K.R.T., and E.E. performed experiments; T.M. and A.A.S. analyzed data; T.M. and A.A.S. interpreted results of experiments; T.M. prepared figures; T.M. and A.A.S. drafted manuscript; T.M., A.A.S., D.B., J.D.S., J.S., C.P., S.S.C., J.M.H., K.V.S., J.P.D., and L.R.H. edited and revised manuscript; T.M., A.A.S., D.B., J.D.S., J.S., C.P., B.L.S., C.B., E.M.O., K.R.T., E.E., S.S.C., J.M.H., K.V.S., J.P.D., and L.R.H. approved final version of manuscript.

APPENDIX

Adaptive feature normalization.

To account for the drifts of the neuronal features, Φ_f, we used a recursive algorithm to estimate the their mean, μ_f, and variance, σ_f, using a time constant τ of 240 s (Bruce 1969).

$${\mu }_f\left(t\right)=\frac{\tau -\Delta t}{\tau }\left[{\mu }_f\left(t-\Delta t\right)+{\Phi }_f\left(t\right)\right]$$

$${\sigma }_f\left(t\right)=\sqrt{\frac{\tau -\Delta t}{\tau }\left\{{\sigma }_f^2\left(t-\Delta t\right)+{\left[{\Phi }_f\left(t\right)-{\mu }_f\left(t\right)\right]}^2\right\}}$$

where Δt was the temporal resolution of our online data processing set to 20 ms. Estimated mean of each feature was subtracted from the current feature value and the result was divided by the estimated standard deviation, thereby normalizing all features to zero-mean and unit-variance. To prevent division by zero, a value of 10⁻⁶ was added to the variance estimate.

$${\widehat{\mathrm{\Phi }}}_f\left(t\right)=\frac{{\Phi }_f\left(t\right)-{\mu }_f\left(t\right)}{{\sigma }_f\left(t\right)+{10}^{-6}}$$

SNR used for selection of neural features.

To reduce the time needed for algorithm calibration and to reduce the effect of excessive dimensionality (Hastie et al. 2009), we selected a subset of neural features that were used for decoder calibration. Selection was done by ranking the features according to the SNR between peri-action and baseline feature values (see Milekovic et al. 2012 for similar use of SNR):

$$\begin{array}{l}{\mu }_{{}_{SQUEEZE}}\left(t\right)=E\left[{\widehat{\Phi }}_{{}_{SQUEEZE}}\left(t-t{c}_i\right)\right]\\ {\sigma }_{{}_{SQUEEZE}}\left(t\right)=std\left[{\widehat{\Phi }}_{{}_{SQUEEZE}}\left(t-t{c}_i\right)\right]\\ {\mu }_{{}_{BASELINE}}=E\left[{\widehat{\Phi }}_{{}_{BASELINE}}\left(t\right)\right]\\ {\sigma }_{{}_{BASELINE}}=std\left[{\widehat{\Phi }}_{{}_{BASELINE}}\left(t\right)\right]\end{array}$$

$$SNR\left(t\right)=\frac{\left|{\mu }_{{}_{SQUEEZE}}\left(t\right)-{\mu }_{{}_{BASELINE}}\left(t\right)\right|}{{\sigma }_{{}_{SQUEEZE}}\left(t\right)+{\sigma }_{{}_{BASELINE}}\left(t\right)}$$

$${SNR}_{\begin{array}{c}SQUEEZE\\ VS.\\ BASELINE\end{array}}=SNR\left(t_{\max }\right)t_{\max }=\underset{t}{\max \hspace{0.17em}\arg }\left[SNR\left(t\right)\right],t_S\le t\le t_E$$

where tc_i is the time of the trial i, E is the expectation operator, and std is the standard deviation operator, both acting over trials, t_S (300 ms) and t_E (1.8 s for T2 and 1.5 s for T6) are the start and the end of the investigated epoch.

Calibration of decoding algorithms.

The rLDA classifier was calibrated in a following way. First, we calibrated a set of classifiers that captured the neuronal features specific to the participant’s selection actions, recorded in the selected set of open-loop calibration blocks. In these blocks, the participant was cued to perform selection click action at times tc_i (see Task section for details). Given a set of normalized features (Φ_f1, …, Φ_fm), the peri-selection feature vector F_i was defined as

$$F_i=\left[\begin{array}{c}{\widehat{\Phi }}_{f1}\left(t{c}_i+t_1\right)\\ ⋮\\ {\widehat{\Phi }}_{f1}\left(t{c}_i+t_n\right)\\ {\widehat{\Phi }}_{f2}\left(t{c}_i+t_1\right)\\ ⋮\\ {\widehat{\Phi }}_{f2}\left(t{c}_i+t_n\right)\\ ⋮\\ {\widehat{\Phi }}_{fm}\left(t{c}_i+t_n\right)\end{array}\right]$$

where t₁, ..., t_n are the selected time points relative to the time of the cue. Therefore, the feature vector contained n·m features. The time points t₁, ..., t_n were always equidistant and defined by a set of parameters: 1) the time of the first feature in relation to the time of the cue, t₁, 2) the number of time points, n and 3) the temporal distance between the first and the last feature, t_n−t₁.

Each classifier was built using two classes of feature vectors: the “select” class, which contained peri-select feature vectors (S_class), and the baseline class, which contained feature vectors between selections (B_class).

$$\begin{array}{l}{S}_{class}=\left\{F_i,i\in \left\{1,\mathrm{...},N\right\}\right\}\\ B_{class}=\left\{\begin{array}{cc}\left[\begin{array}{c}{\widehat{\Phi }}_{f1}\left(k\cdot \delta t+t_1\right)\\ ⋮\\ {\widehat{\Phi }}_{f1}\left(k\cdot \delta t+t_n\right)\\ {\widehat{\Phi }}_{f2}\left(k\cdot \delta t+t_1\right)\\ ⋮\\ {\widehat{\Phi }}_{f2}\left(k\cdot \delta t+t_n\right)\\ ⋮\\ {\widehat{\Phi }}_{fm}\left(k\cdot \delta t+t_n\right)\end{array}\right],& \begin{array}{l}k\in \mathbb{N}\\ \forall i,k|\left|t{c}_i-k\cdot \delta t\right|>0.5s\end{array}\end{array}\right\}\end{array}$$

where δt is the time resolution of baseline feature vectors, set to 80 ms, and N is the number of selection cues present in the data used to train the classifier.

These classes were used to build a rLDA classifier (Friedman 1989). Regularization was implemented in a same way as in Milekovic et al. (Milekovic et al. 2013). The probability of every feature vector F(t) in the test data set to belong to the “select” class was then calculated using the classifier.

$$F\left(t\right)=\left[{\widehat{\Phi }}_{f1}\left(t+t_1\right),\mathrm{...},{\widehat{\Phi }}_{f1}\left(t+t_n\right),{\widehat{\Phi }}_{f2}\left(t+t_1\right),\mathrm{...},{\widehat{\Phi }}_{fm}\left(t+t_n\right)\right]$$

Due to the internal latency of the decoder, feature vector F(t) could only lead to a selection detection at time t+t_n. Based on the probability of the selection, refractory period and the detection hysteresis, we assigned “detected selection” (ds) labels to all times in which the selection was detected and assigned “detected as nonselection” (dns) labels to all remaining time points.

To validate each of the classifiers, we divided the set of blocks used to calibrate the classifiers into two equally long subsets, one containing the first halves and another containing the second halves of each block. First, we chose a set of parameter values consisting of 1) a time of the first feature in relation to the selection cue, t₁; 2) a number of features, n; 3) a time distance between the first and the last feature, t_n − t₁; and 4) regularization coefficient, γ. An rLDA classifier was then calibrated to the first part of the data set and used to detect the events on the second part of the data set and vice versa. Detection performance was estimated by calculating the normalized mutual information C_YX (see Performance measures for details) for both combinations and then averaging between the two. Values of the parameters were then changed and the process was repeated, until all parameter values from the parameter grid were tested. The parameter values that were used to calibrate the decoder that gave the maximum C_YX were then used to calibrate a new decoder to the whole data set. This decoder was then used to detect selections during the copy-phrase and free-spelling blocks of the comparison and communication sessions (online) or to detect selections in the open-loop calibration blocks of all completed sessions in a posterior analysis (offline).

Performing the selection of parameter values that give the highest performance is a computationally demanding process. The historical decoder used during the participants’ comparison and communication sessions and all decoders used for offline analysis were calibrated using a parameter grid with numerous values. This was possible since there were no restrictions on the amount of time needed for calibration. However, same-day decoder used during comparison sessions had to be calibrated during the session itself, within several minutes. Therefore, instead of selecting the parameter values from a grid of values, online same-day decoders were calibrated using the parameter values selected through the calibration of the historical decoder.

Measuring cross-session stability of LFP responses.

r_f(s1,s2) was calculated as follows:

$$r_f\left(s1,s2\right)=CC\left[\frac{1}{N_1}{\displaystyle \sum _{n=1}^{N_{s1}}{}^{s1}\widehat{\Phi }{}_f\left(t\right),\frac{1}{N_2}{\displaystyle \sum _{n=1}^{N_{s2}}{}^{s2}\widehat{\Phi }{}_f\left(t\right)}}\right]$$

where CC is the correlation coefficient operator calculated over time t running from 0 to 1.5 s after the cue, and N_s1, N_s2, ${}^{s1}{\widehat{\Phi }}_f$, and ${}^{s2}{\widehat{\Phi }}_f$ are the numbers of responses and the features for sessions s1 and s2, respectively. r_f(s1,s2) was calculated for all 288 possible features. We used only the recordings from the open-loop calibration blocks for the calculation. Since the calibration was standardized for each participant, the same amount of data from each session—three equally long OLC blocks containing 17 cues each—was used to calculate CC.

To estimate the M_r(s1,s2) values for no evoked LFP responses, we randomly drew as many “random” cues from a given block as there were active cues. The draws were made from a uniform distribution. These random cues were then used to calculate M_r(s1,s2). This procedure was repeated 1,000 times to calculate the mean M_r(s1,s2) for random cues, termed RM_r(s1,s2).

Measuring spelling and decoding performance.

Accuracy of a decoding algorithm is often described by its positive and negative predictive values (Fletcher et al. 2014). However, using these measures to obtain a single metric that can be used to optimize a decoder requires establishing an arbitrary tradeoff between these two values. Another way to measure decoding performance, which takes both positive and negative predictive values into account, is to use a metric based on information theory. We used the normalized mutual information C_YX (Coombs et al. 1970), which was calculated using the assumed intended actions of the participant, “real selection” (rs) and “real nonselection” (rns), and the actions detected by BrainGate system, “detected selection” (ds) and “detected nonselection” (dns). We first divided the corresponding blocks into epochs of the same length as the presentation time that we labeled as rs and rns. During the open-loop calibration blocks, we assumed that the participant always responded to the cues. Therefore, every epoch after the selection cue was labeled as rs and all other epochs were labeled as rns. According to the selection detections, we labeled each epoch with one of the four labels: true positive—(rs,ds); false positive—(rns,ds); false negative—(rs,dns); and true negative—(rns,dns). We calculated C_YX by computing the mutual information between detections and real events, divided by the entropy of real events, as described previously (Milekovic et al. 2013).

The spelling performance of our text-entry interface was calculated by counting the total number of correctly typed characters (N_CC) and dividing it by the time that participant spent from the start of the spelling until s/he finished the phrase (value given in correct characters per minute, CCPM). To calculate the confidence intervals of the measure, we split the spelling period into epochs that the participant spent to type every single character, τ_i(char). Occasionally, even though the participants could erase the incorrectly typed character, they would continue spelling. For the purpose of calculating spelling performance, this character was not counted as having been typed, and the time taken to type it was added to the time it took to type the next character. Due to the design of the text-entry interface, there is a minimum time necessary to reach each of the characters, τ_MIN(char). Note that τ_MIN(char) can be different for different characters. Once the character is reached, the time it takes for the participant to type it, κ_i, is independent of the character, even if the character has been missed. The spelling performance (SP) can then be written as

$$SP=\frac{N_{CC}}{{\displaystyle \sum _{i=1}^{N_{CC}}{\tau }_i\left(char\right)}}=\frac{N_{CC}}{{\displaystyle \sum _{i=1}^{N_{CC}}\left[{\tau }_{MIN}\left(char\right)+{\kappa }_i\right]}}=\frac{N_{CC}}{{\tau }_{MIN}\left(phrase\right)+{\displaystyle \sum _{i=1}^{N_{CC}}{\kappa }_i}}$$

We estimated the standard deviation of the spelling performance calculated from a given data set by bootstrapping with 100,000 resamples (Moore et al. 2009). We assumed the normal distribution of the bootstrapped spelling performance values and used the t-distribution to calculate the 95% confidence intervals (Bohm and Zech 2010). During the free-typing blocks, by instantly completing the word through word completion, the participant was able to select several characters at once. To calculate the confidence intervals in these blocks, we bootstrapped the pairs, consisting of the number of the characters spelled in this particular step n_CCi and the corresponding κ_i. When comparing the performance of two different detection algorithms, we again used bootstrapping with 100,000 resamples on a SP_A − SP_B statistic and used the t-distribution to calculate the 95% confidence intervals.