Restoring rapid natural bimanual typing with a neuroprosthesis after paralysis

Justin J Jude; Hadar Levi-Aharoni; Alexander J Acosta; Shane B Allcroft; Claire Nicolas; Bayardo E Lacayo; Nicholas S Card; Maitreyee Wairagkar; Alisa D Levin; David M Brandman; Sergey D Stavisky; Francis R Willett; Ziv M Williams; John D Simeral; Leigh R Hochberg; Daniel B Rubin

doi:10.1038/s41593-026-02218-y

. 2026 Mar 16;29(10):2613–2622. doi: 10.1038/s41593-026-02218-y

Restoring rapid natural bimanual typing with a neuroprosthesis after paralysis

Justin J Jude ^1,^2,^3,^✉, Hadar Levi-Aharoni ^1,^2,³, Alexander J Acosta ¹, Shane B Allcroft ^3,⁴, Claire Nicolas ^1,³, Bayardo E Lacayo ¹, Nicholas S Card ⁵, Maitreyee Wairagkar ⁵, Alisa D Levin ^6,⁷, David M Brandman ⁵, Sergey D Stavisky ⁵, Francis R Willett ⁸, Ziv M Williams ^9,^10,¹¹, John D Simeral ^3,^4,¹², Leigh R Hochberg ^1,^2,^3,^4,¹², Daniel B Rubin ^1,²

PMCID: PMC13633126 PMID: 41840138

Abstract

Here, recognizing keyboard typing as a familiar, high information rate communication paradigm, we developed an intracortical brain–computer interface (iBCI) typing neuroprosthesis providing bimanual QWERTY keyboard functionality for people with paralysis. Typing with this iBCI involves only attempted finger movements, which are decoded accurately with as few as 30 calibration sentences. Sentence decoding is improved using a 5-gram language model. This typing neuroprosthesis performed well for two iBCI clinical trial participants with tetraplegia—one with amyotrophic lateral sclerosis and one with spinal cord injury. Typing speed is user-regulated, reaching 110 characters per minute, resulting in 22 words per minute with a word error rate of 1.6%. This resembles able-bodied typing accuracy and provides higher throughput than current state-of-the-art hand motor iBCI decoding. In summary, a typing neuroprosthesis decoding finger movements, provides an intuitive, familiar and easy-to-learn paradigm for individuals with impaired communication due to paralysis.

Subject terms: Brain-machine interface, Amyotrophic lateral sclerosis, Neural decoding

Jude et al. present a communication intracortical brain–computer interface typing neuroprosthesis enabling familiar, high-speed, bimanual QWERTY keyboard functionality for people with paralysis through the decoding of attempted finger movements.

Main

For individuals with advanced amyotrophic lateral sclerosis (ALS), brainstem stroke and other conditions that cause paralysis, loss of communication is often cited as one of the most devastating symptoms^1,2. People with paralysis affecting both speech and dexterous control of the hands often rely on augmentative and alternative communication devices, such as eye-gaze tracking systems, to maintain communication. These devices, however, are often described as being slow, error-prone and requiring frequent recalibration assisted by care partners, leading to high rates of abandonment^3,4.

iBCIs have been used to restore communication for people with paralysis by providing a point-and-click interface^5–7, in which a computer cursor can be used to select letters from a virtual on-screen keyboard. Although these systems have the advantage that they can be rapidly calibrated^8,9, communication rates are limited by the speed at which the user can point-and-click on individual letters on a computer screen.

More recently, iBCIs have been shown to be effective in enabling higher throughput communication, by means of articulator-based speech decoding^10–15 and character-based handwriting decoding¹⁶. In these approaches, neural activity pertaining to intended movement, in the absence of actual movement of speech articulators or hand effectors, can be used to decode sequences of phonemes or characters to construct language. High accuracy is maintained with language models¹⁷ that take sequences of predicted phonemes or characters and select the most likely intended output sequence based on statistics of large language datasets¹⁸. Although iBCI handwriting decoding¹⁶ may provide high-throughput communication, many individuals may prefer a keyboard-based communication interface due to its well-established integration with digital communication interfaces and their extensive experience with ten-finger typing.

Individual finger movement decoding from single-unit recordings in nonhuman primates has shown to be possible in real time¹⁹, and with high accuracy even without prior knowledge of movement onset using nonlinear decoders²⁰. Several recent human iBCI studies show accurate neural decoding of five-dimensional²¹ and ten-dimensional²² finger movements and several hand-based gestures^23–25 from single-unit recordings in motor cortex, suggesting that higher-dimensional finger movement decoding necessary to enable QWERTY keyboard typing may be feasible. Ten-digit finger decoding has additionally been shown with recordings from the posterior parietal cortex²⁶, demonstrating finger movement encoding in diverse brain regions. Bimanual virtual arm control with nonhuman primates²⁷ and bimanual cursor control²⁸ with human participants has also been demonstrated. Indeed, recent work²⁹ shows fast and accurate character decoding from neural data pertaining to fine bimanual finger movements, albeit at the single-character level. Typing decoding has also recently been performed in able-bodied users with a surface electromyography bracelet³⁰ (although this may be less effective in individuals with severe paralysis due to loss of volitional peripheral nerve activity) and magnetoencephalography-based recordings³¹. While both of these recording methods do not require implanted sensors, performance has not yet reached levels required for accurate and reliable communication.

In this work, we introduce a touch-typing neuroprosthesis used by two BrainGate2 clinical trial participants to communicate rapidly and accurately. The iBCI typing interface only requires attempted finger movements to decode characters resulting in sentences. The requisite finger flexion and extension intentions closely mimic QWERTY keyboard use in the able-bodied population, taking advantage of familiar finger movement imagery and memory of relative key positions to allow for faster learning of the system. In addition, typing is completely self-paced; therefore, the speed of communication is potentially far greater than augmentative and alternative communication devices and previous iBCI typing interfaces that rely on fixed decoding intervals²⁹.

iBCI decoders can have difficulty distinguishing between similar attempted movements, such as similarly shaped characters for handwriting¹⁶, similarly articulated phonemes for speech^10–15 and nearby finger movements for bimanual typing. However, it may be the case that language models are less sensitive to some patterns of confusion when considering final word-level accuracy. We hypothesize that QWERTY may be more advantageous than handwriting for a given character-level accuracy, as adjacent keys may be less likely to be contiguous or interchangeable with each other in the English language^32,33, as opposed to similarly shaped characters.

Taken together, this typing neuroprosthesis has the potential to provide a rapid, highly accurate and intuitive means of communication for people with paralysis.

Results

Neural representation of attempted typing-like finger movements

As an initial assessment, we asked both participants (Fig. 1a,b) to attempt a series of isolated finger movements. In an instructed delay task (Fig. 1c), each participant was presented with a single finger movement cue per trial. Three varying extension and flexion movements (Fig. 1d) were instructed for each finger to ascertain the distinguishability of neural activity pertaining to a total of 30 finger movements required for QWERTY keyboard typing. Participants were instructed to attempt to perform the finger movement cued in text on the screen in front of them when the cue rectangle on screen changed from red to green.

For both participants, there was clear tuning for these discrete finger movements across many electrodes (Extended Data Fig. 1). The trial-averaged multiunit neural activity (change in threshold crossing firing rates) for a well-tuned example electrode in T18 (Fig. 1i), showed prominent responses to each of the three finger movements on all ten fingers. For T17, we observed electrodes in both cortical areas 6d (hand motor) and 6v (speech motor) with responses to almost all finger movements on each finger (Extended Data Fig. 2).

Extended Data Fig. 2 — a, Each panel shows the mean responses of an example electrode in area 6d of motor cortex (electrode 382, placement in left dorsal motor cortex) from T17 tuned to flexion and extension positions for every finger, where each line shows mean change in threshold crossing rate averaged across all trials of a given movement condition. b, Each panel shows the mean responses of an example electrode in speech motor cortex (electrode 73, placement in left ventral premotor cortex, area 6v) from T17, weakly tuned to flexion and extension positions for every finger, where each line shows mean change in threshold crossing rate averaged across all trials of a given movement condition. Shaded regions indicate a 95% confidence interval.

Source data

For T18, we found that tuning to finger movements on both hands was intermixed at the single-electrode level in left precentral gyrus arrays (Fig. 1j), whereas single electrodes in right precentral gyrus arrays tended to have stronger tuning to finger movements on the left hand. For each hand, there was a higher number of tuned electrodes contralaterally than ipsilaterally (Extended Data Fig. 1). In T17 we observed strong tuning to finger movements on both hands in 6d electrodes, with some electrodes in 6v, particularly in the more dorsal 6v array, exhibiting similar tuning to finger movements on both hands (Extended Data Fig. 3).

Extended Data Fig. 3 — Tuning heatmaps for all 6 T17 arrays, array backgrounds shaded by brain area implanted (all in the left hemisphere). Circles indicate that threshold crossing features on a given electrode varied significantly across all 15 unique finger movements on each hand (p < 1 × 10⁻⁵ assessed with one-way analysis of variance). Shading indicates the fraction of variance accounted for (FVAF) by cross movement differences in threshold crossing firing rate.

Source data

To visualize activity at the population level, we performed dimensionality reduction through trial-averaged principal component analysis (PCA) of neural trajectories during trials of the isolated finger movement task (Fig. 1e–h). For both participants, we observed anatomically segmented neural trajectories in several principal components, primarily separating based on hand and finger position. Furthermore, we observed distinguishable trajectories across time for unique finger movements, particularly in T17.

For example, the first two principal components extracted from 6v and 6d electrodes (a total of 256 electrodes; Fig. 1e,f) showed separation between all 30 finger movements in T17. The first principal component for T17 (Fig. 1e) showed hand-dependent activity, with right-hand cues evoking larger firing rate modulation over time compared to the left hand. Furthermore, within each hand, more medial fingers (pinky and ring) tended to have higher principal component values than more lateral fingers (thumb and index). The second PC for T17 (Fig. 1f) showed overlaid finger ordering in each hand, from thumb to pinky.

In T18, the first principal component of trial-averaged PCA extracted from all 384 electrodes (Extended Data Fig. 4) is predominantly modulated by movement onset. The second principal component (Fig. 1g) shows highly diverging trajectories pertaining to hand-dependent activity (distinct separation between right-hand movements in pink, blue and purple and left-hand movements in green, yellow and red colors). We determined that the neural basis for differences in principal component trajectories between medial and lateral fingers does not arise from array placement in either participant, but rather due to distinct neural activity patterns intermixed at the single-electrode level pertaining to individual finger movements (Extended Data Fig. 5). The third principal component for T18 shows high separability in ‘Extend upward’ movements across all ten fingers (Fig. 1h).

Extended Data Fig. 5 — a,c, Tuning heatmaps for a, all 6 T17 arrays, array backgrounds shaded by brain area implanted (all in the left hemisphere) and c, all 6 T18 arrays implanted in both hemispheres. Circles indicate that threshold crossing features on a given electrode varied significantly across all 12 unique finger movements for the 2 medial (pinky and ring) fingers on each hand (left) and the 2 lateral (thumb and index) fingers on each hand (right) (p < 1 × 10⁻⁵ assessed with one-way analysis of variance). Shading indicates the fraction of variance accounted for (FVAF) by cross movement differences in threshold crossing firing rate. b,d, Number of strongly tuned electrodes where threshold crossing features on a given electrode varied significantly across all 12 unique finger movements for the 2 medial (pinky and ring) fingers on each hand (left) and the 2 lateral (thumb and index) fingers on each hand (right) (p < 1 × 10⁻⁵ assessed with one-way analysis of variance) for b, T17 and d, T18.

Source data

Single-trial decoding performance

Encouraged by separability in the trial-averaged low-dimensional representations between finger movement conditions at the population level (Fig. 1e–h), we next applied a nonlinear dimensionality reduction technique to single trials. Two-dimensional (2D) t-distributed stochastic neighbor embedding (t-SNE) spaces consisting of all individual trials, formed using threshold crossing rates recorded during the Go period, reveal a well-separated, low-dimensional space for each participant, with several tight clusters forming for each condition (Fig. 2a,b)

For both participants, trials appeared distinguishable both by finger and by movement type (Fig. 2a,b), with ‘Extend upward’ trials (denoted by circles in Fig. 2a,b) generally more distinct than trials of ‘Straight downward’ or ‘In to palm’ on the same finger. This may correspond to the prominently distinct representation of ‘Extend upward’ seen in the low-dimensional trajectories discussed above (Fig. 1h) for T18 trials. Particularly clear separation of representations of each hand was evident for T18 (Fig. 2b), which again likely corresponds to the clear divergence of neural activity observed in the low-dimensional state-space representations of neural activity described above (Fig. 1g).

To assess our ability to accurately decode individual finger movements, we trained a Gaussian Naive Bayes classifier to distinguish between single trials of each of the 30 discrete movements explored above. For T17 and T18, offline decoding accuracy was 81.3% (95% confidence interval (CI) = [78.8%, 83.6%]) and 95.2% (95% CI = [93.5%, 96.8%]), respectively (Fig. 2c,d). For participant T17 (who only has arrays in the left precentral gyrus), decoding of right-hand movements was not significantly more accurate at 86.1% (95% CI = [83.0%, 88.8%]), than that of left-hand movements at 84.3% (95% CI = [81.2%, 87.5%]) (overlapping CIs), indicating a comparable contribution of the left precentral gyrus electrodes to both contralateral and ipsilateral finger movement encoding in this participant. Errors on the left hand were predominantly between movement types within the same finger and between the same movement across fingers, whereas on the right hand there was a higher rate of across-hand errors (Fig. 2c).

In comparison to T17, T18 exhibited reduced within-hand and within-finger decoding errors (Fig. 2d). This improved performance could be due to both the bilateral placement of electrodes as well as the higher number of electrodes in the hand motor cortex overall. To understand the relative contribution of bilateral array placement on overall decoding performance, we separately examined ipsilateral and contralateral classification accuracy. Unsurprisingly, for participant T18, unilateral Gaussian Naive Bayes decoding accuracy was lower than bilateral, at 88.7% (95% CI = [86.4%, 90.9%]) with just left-hemisphere arrays and 89.0% (95% CI = [86.7%, 91.2%]) with just right hemisphere arrays. Unilateral decoding also revealed cross-hand decoding errors (Extended Data Fig. 6), most notably with left-hemisphere arrays. Therefore, bilateral array placement with T18 may reduce across-hand decoding errors, consistent with the representation in Fig. 1g.

Extended Data Fig. 6 — Classification using a Gaussian Naive Bayes (GNB) classifier for T18 with a, 256 electrodes placed in left precentral gyrus, resulting in a decoding accuracy of 88.7% (95% CI=[86.4, 90.9]), and b, 128 electrodes placed in right precentral gyrus, resulting in a decoding accuracy of 89.0% (95% CI=[86.7, 91.2]).

Source data

The iBCI bimanual QWERTY keyboard

For the neurally controlled keyboard, we considered all 26 letters, space (Sp) and punctuation (question mark, comma, period), for a total of 30 unique tokens, arranged according to the QWERTY keyboard layout (Fig. 3a). The keyboard consists of three adjacent rows of 10 keys per row. Three movements on each finger, corresponding to the movements performed in the isolated movement task (Fig. 1d), allow the user to select any of the 30 keys at any time. Mapped to our keyboard, these movements now correspond to: (1) upward extension from the resting position for an upper-row key, (2) straight downward flexion for a middle row key, and (3) curling into the palm for a bottom-row key. Typing speed is controlled by the participant.

Fig. 3 — a, The keyboard layout closely mimics a QWERTY keyboard. Finger and text guides on the user interface aid in matching finger movements to keyboard characters. Sentences are constructed by the participant performing a series of finger movements corresponding to the keyboard layout shown. Character probabilities are predicted by a RNN from neural features. Character probabilities are continuously considered by the language model, with the currently predicted sentence updated on screen. b,c, T17 (b) and T18 (c). Left, Mean raw character, character post-language model and word post-language model error rates shown across all sessions. Numbered x-axis ticks indicate trial days (relative to array placement surgery on day 0), followed by blocks per day. Each circular marker indicates a block. Right, Mean characters per minute typed by each participant across all sessions per block. Shaded regions indicate a 95% CI. d,e, T17 (d) and T18 (e) real-time example sentence decoding during free typing. Spaces are indicated with the greater than sign (‘>’). Red characters indicate errors, and blue characters below incorrect characters indicate the second most probable character predicted by the RNN.

Source data

Character decoding was performed by a recurrent neural network (RNN), which inferred character probabilities in sequence as the user performed a series of finger movements (Fig. 3a). RNN training (Methods) minimizes a connectionist temporal classification (CTC)^34–36 loss function, which aligns binned neural features and sequences of characters. During sentence typing (Fig. 3a), the key corresponding to the most probable character inferred by the RNN is highlighted in green on the virtual keyboard, providing real-time feedback to the user. A 5-gram language model^17,37,38 predicts likely sentences given character probabilities inferred by the RNN.

In each session, we evaluated system performance by first jointly training the RNN on calibration blocks of sentences from the current day along with all blocks from all previous sessions to improve generalization during online decoding (see Methods for a listing of trial days). The pipeline outlined above (Fig. 3a) was used to decode sentence typing in real time, either in a sentence-copy task in which the participant was asked to copy a sentence shown on screen (Supplementary Videos 1 and 2) or in a free-typing task in which the participant could type any sentence they wished (Supplementary Videos 3 and 4). For both the sentence-copy and free-typing tasks, sentences used for performance evaluation had never been previously seen by the decoder. Sentence origination and conclusion were self-induced in the free-typing task, and intended sentences were confirmed with the participant.

For T17, mean raw character error rates fluctuated depending on the session, with average word error rates (Fig. 3b) predominantly below 15%. For T18, raw character error rate was consistently low, remaining below 12% after the first trial day. Resulting word error rates for T18 were below 7% across all blocks across all sessions, with several blocks with a word error rate of 0% (Fig. 3c). For both participants, communication rates generally improved over subsequent sessions, with T17 reaching a peak of 47 characters per minute and T18 reaching 110 characters per minute. T18’s speed was faster than the previous state-of-the-art in hand motor iBCI-based language decoding using a handwriting strategy¹⁶. Encouragingly, this consistent increase over time in typing speed for both participants did not impact character decoding error rate (Extended Data Fig. 7). We note that the average output latency per initial keystroke relative to the Go cue for the start of a sentence was significantly lower for T18 than T17 (P < 4.51 × 10⁻⁴⁴, one-sided Welch’s t-test across all keys that appear as a starting sentence character for both participants) during trial days with peak typing speed for both participants (Extended Data Fig. 8). This difference in average latency may explain the contrast in typing speed between the participants. Neither participant was a touch typist before disease or injury onset and, as such, both were acclimatizing to the keyboard layout through typing experience as sessions progressed.

Extended Data Fig. 7 — a,b, T17 and T18 character error rate resulting from increasing typing speed for all trials. Each circle marker represents a single sentence trial.

Source data

Extended Data Fig. 8 — Latency from the Go cue to the first character decoded for each of the sentences on trial days with the highest typing speeds for each participant. For each character on the x-axis, we plot a boxplot and strip plot across all sentences where that character is the first character in each sentence for each participant, indicating latency in milliseconds until that character is predicted as part of a decoded sentence (instead of a blank token). 200 total sentences across 4 sessions were considered for T17 and 540 total sentences across 4 sessions were considered for T18, in each case the ground truth initial character is known. Box plots are median centered. The bounds of each box is the inter-quartile range between the 25th and 75th percentile. The minimum and maximum value of the whisker of each box represent 1.5 times the inter-quartile range (data points outside of this range are deemed outliers). Outlier values are represented by point plot extremes below or above whiskers.

Source data

Review of the decoded sentences from the free-typing task highlights the robustness of the pipeline (Fig. 3d,e). In many instances, the sequence of raw character output (that is, the most probable characters predicted by the RNN) was recognizable, even before the language model step. In other cases, the correct character was the second most probable character predicted by the RNN (Fig. 3d,e).

To understand the relative contribution of the language model to word error rate, we examined the pattern of errors produced by the RNN when decoding typing and quantified for each character the frequency of specific RNN substitutions required to produce a correct sentence (Fig. 4a,b). For both T17 and T18, we find that decoding errors were most frequent within the same finger or across adjacent fingers.

Fig. 4 — a–c, Number of substitutions indicates the number of instances of a character on the x-axis requiring substitution with a character on the y-axis to generate a correct sentence. Colored boxes indicate characters belonging to the same finger in the typing task, with characters and boxes in darker shades belonging to the right hand (upper diagonal), and lighter shades belonging to the left hand (lower diagonal). a,b, T17 (a) and T18 (b) offline character substitution RNN decoding errors during the typing task. c, T5¹⁶ offline character substitution RNN decoding errors during a handwriting task. Character order and colored boxes belonging to fingers are not relevant to the handwriting task but are shown for comparison. d, Mean word error rates resulting from increasing character error rates across participants and tasks. Shaded regions indicate a 95% CI. e, Mean word and raw character error rates resulting from an increasing number of RNN training sentences from initialization (no previous RNN training) across participants and tasks. Shaded regions indicate a 95% CI. f, Mean word and raw character error rate when training an RNN on day 0 and evaluating decoding performance on held-out sentences whose neural data was recorded on an increasing number of days into the future without RNN recalibration. Bars indicate a 95% CI. x-axis scale is logarithmic. For participant T18, each evaluation session consisted of 70 trials. For participant T17, each evaluation session consisted of all available trials for that session (100 evaluation trials on day 1, 60 on day 2, 40 on day 10, 60 on day 37, 20 on day 38 and 60 on day 50). g, Word error rate when training an RNN with an increasing number of randomly subsampled electrodes across typing and handwriting with T18. For each electrode number, the circle marker represents the mean word error rate and the shaded region indicates a 95% CI.

Source data

We compared the effect of these QWERTY error patterns against iBCI handwriting error patterns resulting from offline decoding (using the pipeline in Fig. 3a) of a previously published dataset¹⁶ (participant T5). In contrast to QWERTY keyboard errors, which were more likely to occur between nearby keyboard keys, in the handwriting dataset we found that characters that have similar shapes (for example, {a, e, o, u} and {r, n}) were more likely to be confused (Fig. 4c). This can be detrimental when paired with a language model because these letters are often interchangeable when constructing likely sentences that occur in the English language. A similar phenomenon also occurs in speech decoders^11–13, where phonemes that require similar orofacial movements often produce overlapping neural signatures¹²; these phonemes (especially vowels) are often interchangeable when a phoneme-based language model considers probable sentences.

We further examined the implication of QWERTY-based typing versus handwriting error patterns on word error rate. Word error rates resulting from increasing character error rates for T18 typing and T5 handwriting reveal that at low character error rates (less than 25%), both participants have similar word error rates; however, at higher character error rates the QWERTY character errors were less likely to result in word errors (Fig. 4d).

We also examined calibration data requirements for simulated novel participants, wherein an RNN was trained with only trials from a single session and evaluated on the character and word decoding accuracy of the remainder of that session’s trials. Notably, for typing with T18, as few as 30 calibration trials were required to obtain a usable 7% word error rate from scratch, approaching 4% at 50 trials (Fig. 4e). In contrast, for handwriting data collected with T18, 70 calibration trials were required to reach a 7% word error rate. For typing with T17, the minimum number of trials required for viable communication was higher, at 100 trials for a 5% word error rate.

To evaluate the stability of the 30-state decoder and look more closely at the rate of change in neural data, we trained an RNN on trials from an early session for both participants and evaluated its decoding performance on future sessions without any recalibration. We found that neural representations in both participants drifted gradually over time (Fig. 4f), with the original decoder for T17 maintaining a word error rate of 15% after 2 days and T18 maintaining a word error rate of 12% after 7 days. The promisingly low word error rate of both participants after 1 day (11% for T17 and 7% for T18) indicates the potential for day-to-day online decoder training on decoded sentences without explicit recalibration^13,39.

Looking more closely at the effect of electrode count on decoding performance, we examined both iBCI handwriting¹⁶ and typing decoding accuracy with T18 as a function of electrode count. We show that for a comparable number of randomly sampled electrodes, our QWERTY typing approach has a lower word error rate (Fig. 4g), especially for low electrode numbers below 200 electrodes (for example, average randomly subsampled 140 electrodes from T18 produces a 9.5% word error rate with our typing approach, in contrast to 27% word error rate with the handwriting approach at this electrode number).

Discussion

We introduce a touch-typing neuroprosthesis to restore communication for people with paralysis. To our knowledge, this system provides the fastest iBCI communication method reported to date based on decoding from hand motor cortex; participant T18 used this system to communicate at 110 characters per minute, which is 20 characters per minute faster than the previous state-of-the-art method¹⁶. This typing rate of 22 words per minute is 81% the speed of able-bodied smartphone typing of 27.3 words per minute for T18’s age group⁴⁰. High speed is achieved while maintaining comparable accuracy to state-of-the-art iBCI character-based decoding^16,29. In T17, we demonstrate faster iBCI communication with an unrestricted vocabulary in a person with longstanding anarthria and incomplete locked-in syndrome than previous methods⁴¹.

The QWERTY iBCI keyboard uses familiar intended movements, representing an intuitive method of communication. Although less common keyboard layouts such as DVORAK may improve typing speed⁴² and optimized keyboard layouts may improve resulting word-level accuracy^43,44, future iBCI users with touch-typing experience before injury or disease may experience a faster rate of acclimation to the ubiquitous QWERTY keyboard layout, reducing overall time to rapid communication. Once learned, the system may also be used without visual acuity or direct gaze at the screen; as opposed to cursor-based character selection⁵, which requires knowledge of current cursor position.

While recent work²⁹ shows neural typing control at up to 90 characters per minute with 90% accuracy, this is through individually cued character selection with a participant-specific keyboard layout that was not learned by the participant in the study; thus, the reported 90 characters per minute is only an upper bound on communication speed. Finger movement animations are shown to the participant instead of a word or sentence to type as demonstrated in our work, indicating the complexity of character selection²⁹. This participant-specific keyboard in which more frequently used characters are mapped to better decoded movements, may not generalize well to new users; as such, a keyboard layout would need to be remapped based on finger movement tuning in a novel participant. In contrast, our participant-agnostic QWERTY-based approach is intuitive with a ubiquitous keyboard layout. Furthermore, our 30-way decoding strategy underlies a higher, self-paced typing speed.

Differences between the two participants, including disease and injury etiologies, number and location of recording sites, and other variables may have influenced these initial results. In particular, T18 attained higher online typing speed and accuracy than T17. All six of T18’s Utah arrays were placed in area 6d (containing the anatomical ‘hand knob’ of cortex⁴⁵), on both the ipsilateral and contralateral hemispheres, for a total of 384 hand motor electrodes (Fig. 1b)—the most reported that we are aware of in the literature. In contrast, T17 has only two arrays in hand motor areas for a total of 128 presumptive hand motor electrodes (Fig. 1a). Despite this unilateral representation, we were able to decode the intended movements of both hands. This is consistent with recent studies demonstrating bilateral representation of intended movement within the left motor cortex^28,46–49, and it is reassuring that this decoding approach can be used by an iBCI user with unilaterally placed recording electrodes. Indeed, decoding solely unilaterally demonstrated redundant bi-hemispheric representation of bimanual finger movements in T18 (Extended Data Fig. 9).

Extended Data Fig. 9 — a, RNN trained and evaluated offline on 50 held-out typed sentence trials using each T18 array individually. b, RNN trained and evaluated offline on 50 held-out typed sentence trials using arrays on each T18 hemisphere individually. Chance level character error rate results from always predicting the character ‘e’. Box plots within violin plot interiors are median centered. The bounds of each box is the inter-quartile range between the 25th and 75th percentile. The minimum and maximum value of the whisker of each box represent 1.5 times the inter-quartile range (data points outside of this range are deemed outliers). Outlier values are represented by violin plot extremes.

Source data

We believe that our iBCI typing approach may provide a preferable alternative to other means of digital access for communication commonly used by people with paralysis. For example, speech-to-text systems are now frequently used as a primary digital access interface for people with tetraplegia caused by cervical spinal cord injury. However, because many people with spinal cord injury have diminished respiratory capacity, their use for long-form text is often quite fatiguing. Moreover, speech-to-text systems also preclude the privacy afforded by keyboard typing. Additionally, while participant T18 does not use a mechanical ventilator, many individuals with cervical spinal cord injury do rely on mechanical ventilators^50–52, and a fast and facile iBCI-based text entry system may provide an improvement in their quality of life.

Similarly, for people with advanced ALS, eye-gaze tracking systems are commonly used for communication. However, there are many individuals with advanced ALS who are unable to use eye-gaze trackers; some of these individuals are able to slowly communicate using letterboards or other simple means (often relying on some binary switch to make selections), whereas others unfortunately have no means of communication at all. For these individuals, the typing iBCI system described herein, even at the slower communication rate of 46 characters per minute demonstrated by participant T17, would be vastly faster than the viable alternatives.

Future work will focus on integrating the typing neuroprosthesis into communication systems to support people with paralysis, in which typed sentences are either vocalized through a text-to-speech system or kept on the screen for interaction with a digital communication interface (that is, email, texting). The use of the language model can be toggled on/off as its inclusion does not accommodate arbitrary sequences of characters that do not conform to character sequences often seen in English (for example, two-factor authentication passwords). We further plan to integrate wrist gestures to provide a switch between different sets of keys (such as numbers or a different alphabet) or access special keys such as Shift and Backspace.

Additional future work will focus on developing independent participant use through online training of the RNN decoder^13,39. A strategy that could be used would entail the participant starting a day of use by freely typing sentences using an RNN decoder that was trained on previously recorded free-typing data from a recent day. As the word error rate is still reasonably low even after several days of non-use (Fig. 4f), as long as the participant has a means of indicating whether the sentences are being accurately decoded, the data can then be used to update our RNN decoder online during free typing on the current day in the background^13,39 and maintain performance without the need for explicit sentence-copy task recalibration on the current day.

To mitigate neural non-stationarities at longer timescales, other approaches such as unsupervised latent variable modeling techniques to create stable manifolds may substantially reduce the number of sentence-copy recalibration sentences required on a novel decoding day^53–55. Recently, the use of large foundational models pretrained on a large magnitude of data across tasks and subjects has shown to aid generalization to new tasks and unseen sessions of neural data^56–59. Use of these models for online typing decoding may not only reduce recalibration data requirements but also reduce the initial number of calibration sentences required to reach high performance. Similarly, highly parameterized transformer models⁶⁰ have shown to vastly improve stabilization through training on a vast amount of neural speech data, while simultaneously improving overall accuracy.

Methods

Clinical trial

Permission for this study was granted by the US Food and Drug Administration and the Institutional Review Boards of Massachusetts General Hospital (MGH), Brown University and the VA Providence Healthcare System. Research sessions were conducted with two participants, T17 and T18, and data used, which were collected previously from a third participant, T5, all of whom were enrolled in the BrainGate2 clinical trial (ClinicalTrials.gov registration: NCT00912041). This pilot clinical trial was approved under an Investigational Device Exemption (IDE) by the US Food and Drug Administration (IDE G090003) (Caution-Investigational Device, Limited by Federal Law to Investigational Use). All research sessions were performed at each participant’s place of residence. This paper does not report any clinical trial-related outcomes; instead, it describes scientific and engineering discoveries that were made using data collected in the context of the ongoing clinical trial. Trial day numbers are relative to clinical trial enrollment on day 0, wherein the respective participants underwent surgical Utah array implantation.

Participant T17

T17 is a 33-year-old right-handed man with ALS. He was diagnosed with ALS 3 years before the start of this study. He has tetraplegia, anarthria and ventilator dependence; his only remaining volitional motor control is over his extraocular muscles. Following enrollment, T17 had preoperative structural magnetic resonance imaging (MRI), resting-state functional magnetic resonance imaging (fMRI) and task-based fMRI to identify appropriate anatomical targets for microelectrode array placement. Resting-state fMRI was used to generate estimated parcellations of the relevant brain areas using a custom instantiation of the Human Connectome Project^61,62 analysis pipeline that was modified for deployment on clinical MRI scanners able to accommodate a mechanical ventilator. Subsequently, T17 underwent placement of six 64-electrode microelectrode arrays (Blackrock Microsystems; 1.5-mm electrode length) in the left precentral gyrus; two arrays were placed in the dorsal precentral gyrus (area 6d), two arrays were placed in the ventral precentral gyrus (area 6v), and two arrays were placed in middle precentral gyrus (area 55b) (Fig. 1a). T17’s palms were resting on his bed at either side of his torso during all tasks. T17 consented to publication of photographs and videos containing his likeness.

Participant T18

T18 is a 48-year-old right-handed man with tetraplegia resulting from a cervical spinal cord injury (C4 ASIA A) that occurred approximately 18 months before trial enrollment. T18 has no residual volitional movement below the C4 spinal level. He had preoperative structural MRI, task-based functional MRI and resting-state fMRI to allow for functional parcellation using the Human Connectome Project^61,62 pipeline to identify putative targets for microelectrode array placement. He subsequently had four 64-electrode microelectrode arrays (Blackrock Microsystems; 1.5-mm electrode length) placed in the dorsal precentral gyrus (area 6d) of the left (motor dominant) hemisphere and two 64-electrode microelectrode arrays placed in the dorsal precentral gyrus (area 6d) of the right hemisphere (Fig. 1b). T18’s palms were resting on the armrests of his power wheelchair during all tasks. T18 consented to publication of photographs and videos containing his likeness.

Participant T5

T5 is a right-handed man, 65 years old at the time of data collection, with a C4 AIS C (ASIA Impairment Scale C - Motor Incomplete) spinal cord injury that occurred approximately 9 years before study enrollment. In August 2016, two 96-electrode microelectrode arrays (Blackrock Microsystems; 1.5-mm electrode length), were placed in the hand ‘knob’ area of T5’s left-hemisphere (dominant) precentral gyrus. Data are reported from post-implant days 994 to 1,246. T5 handwriting data are publicly available at https://doi.org/10.5061/dryad.wh70rxwmv ref. ⁶³.

Neural signal processing

Neural voltage time-series signals were recorded and digitized (30 kHz, 16 bits per sample) using the NeuroPort system (Blackrock Neurotech) attached to three percutaneous connectors on each participant’s head, and transmitted via three mini-HDMI cables (one to each NeuroPlex headstage), attached to two Gemini hubs (Blackrock Neurotech), before final processing via a neural signal processor (Blackrock Neurotech). Packets of neural data were streamed from the neural signal processor to our processing pipeline. Signals were analog filtered (4th-order Butterworth with corners at 250 Hz to 5 kHz) using the Scipy Python library (scipy.signal.filtfilt).

Linear regression referencing⁶⁴ (LRR) is used to reduce cortically recorded artifacts in extracted neural features arising from environmental electrical noise, through the generation of electrode-specific reference signals composed of weighted sums of other electrodes. Intuitively, this ensures robustness to systematic noise at the electrode level, as individual signals are proportionally pegged to all other electrodes in a given array. LRR filter coefficients and subsequent electrode-specific thresholds were determined using the filtered digitized data recorded in an initial reference block at the beginning of each session. LRR coefficients are computed by solving Y = WX where Y is the signal from a given electrode we require and X is the signal from all other electrodes. We solve for the LRR weight matrix through least-squares calculation W: W = inv(X^TX)X^TY where ‘inv’ is matrix inversion⁶⁵. Electrode-specific thresholds were calculated using filtered 30-kHz data once these calculated references had been applied to identify spike events. Thresholds were set at −3.5 times the standard deviation of the voltage signal per electrode. The number of noncausal threshold crossings⁶⁶ was computed by counting the number of times the filtered neural time series crossed these calculated thresholds (threshold crossing rates). Additionally, spike band power was computed by taking the sum of squared voltages observed during each 10-ms time bin. During closed-loop decoding blocks, feature normalization was used to account for neural non-stationarities (drifts in mean firing rate) that could arise during a block. Within each electrode, threshold crossing rates and spike band power were z-scored (mean subtracted and divided by standard deviation per electrode). Feature extraction (threshold crossings and spike band power), binning, decoding and task phase control were performed through the Python-based, modular BRAND⁶⁷ framework, where each process is instantiated as a self-contained node-based Python program. Messaging between these nodes is performed using a Redis database.

Isolated finger movements

In this instructed delay paradigm, each trial’s preparatory time varied between 3.5 s and 4.5 s for T17 (due to the participant’s preference to accommodate poorer baseline visual acuity) and between 1.5 s and 2.5 s for T18. Both participants had a 2.5-s movement time (including a hold) and 2.5-s rest time. Each participant was instructed to attempt to maintain the rest position (Fig. 1d) until the Go cue (rectangle on screen turns green accompanied by a beep tone), whereupon they were instructed to attempt and hold one of the three active movements (Fig. 1d) for a given finger until ‘Return’ is displayed on screen, accompanied by a click sound. Additionally, a ‘Do nothing’ condition (indicated by ‘None’ in Figs. 1e–h and 2) instructed the participant to remain in the rest position and perform no (intended) movement throughout the Go period. For the analyses in Figs. 1 and 2, the isolated finger movements from trial day 55 was used for T17 (16 repetitions of each finger movement condition) and from trial day 43 for T18 (12 repetitions of each finger movement condition). A further 6 repetitions per finger movement condition were recorded on trial day 95 for T17 and a further 4 repetitions per finger movement condition were recorded on trial day 44 for T18; however, these were not used in the above analyses.

PCA trajectories

To visualize the neural geometry of 30 isolated finger movements, we plotted the first and second (T17) and second and third (T18) principal components by Gaussian smoothing (standard deviation of 5) all trials over time, then averaging all trials pertaining to each movement condition. In the case of T18 (where each trial was 7,500 ms in total), a three-dimensional matrix of shape (31, 750, 384) was formed where 31 is the total number of isolated movement conditions, 750 is the number of 10-ms time bins, and 384 is the number of threshold crossing features used. The time dimension was preserved and stacked in order to form a 2D matrix of shape (31 × 750, 384) before applying the Scikit-learn⁶⁸ implementation of PCA, which after reshaping, produces a three-dimensional matrix of shape (31, 750, 3). We then plotted each of 31 conditions over 750 time bins for the second and third principal components. Averaged PCA trajectories were Gaussian smoothed with a kernel standard deviation of 20. For T18, all 384 electrodes were used, whereas for T17 only the 128 6d electrodes were used.

Peristimulus time histograms

To visualize individual neuronal responses to stimuli, we demonstrate firing rate changes for a representative electrode in T18 upon presentation of the Go cue. Threshold crossing rates for each condition for each finger (Up, Down and In) were mean averaged and Gaussian smoothed (standard deviation of 8) across the time axis. The resulting time series shown in Fig. 1i depicts the average threshold crossing rates across time for each stimulus (finger movement condition). Similarly, representative T17 electrodes are shown in Extended Data Fig. 2.

FVAF

To calculate electrode tuning heat maps in Fig. 1j and Extended Data Figs. 3 and 5, we used threshold crossing rates that were z-scored (mean subtracted and standard deviation divided) and time-averaged in a 1,000-ms window after the Go cue. The significance of tuning was then assessed via a one-way ANOVA applied per electrode, where each ANOVA group corresponded to a different finger movement condition, and each observation was a scalar firing rate for a single trial. Pvalues from each ANOVA were used to define tuning significance (P < 1 × 10⁻⁵) for the tuning heat maps. The FVAF by finger movement tuning on a single electrode was defined as: $FVAF = 1 - \frac{SSERR}{SSTOT}$ , where SSTOT is the total sum of squared average firing rates over all trials. For computing SSTOT, squaring was performed after the mean across all trials was subtracted from each trial first, so that the overall mean firing rate did not contribute to the variance. SSERR is the sum of squared predictions over all trials. Prediction error was assessed with a cross-validated (fivefold) model that predicts the firing rate of each trial based only on the mean of the finger movement condition it belongs to. Condition-specific means were estimated on the training set by taking the sample means across training trials, and were then applied to the held-out test set. If there are large differences in mean firing rate between movement conditions (that is, strong movement tuning), then SSERR will be small relative to SSTOT.

t-SNE

t-SNE is a nonlinear dimensionality reduction technique that preserves local pairwise similarities between samples. We visualize the high-dimensional neural data pertaining to isolated finger movements via the low-dimensional trial-by-trial similarity space rendered using t-SNE, which is performed on time-averaged neural activity across all trials. Time-averaging produces a 2D matrix of shape (number of trials, 384), where 384 is the number of threshold crossing features for T18. We use the Scikit-learn⁶⁸ implementation of t-SNE with a perplexity of 10 (favoring local versus global structure) and PCA initialization. For T18, all 384 electrodes were used, whereas for T17 only the 128 6v electrodes and the 128 6d electrodes were used (for a total of 256 threshold crossing features).

Gaussian Naive Bayes

The Naive Bayes classifier is a simple probabilistic predictor that assumes conditional independence between features given the class. Gaussian Naive Bayes takes this a step further for continuous data and makes the additional assumption that values associated with each class are normally distributed. Isolated finger movement classification results (reported in Fig. 2c,d and Extended Data Fig. 6a,b) were generated using a cross-validated (leave-one-out) Gaussian Naive Bayes classifier.

Threshold crossing features from all trials of the isolated movement task outlined in Fig. 1c from trial day 55 for participant T17 and trial day 43 for participant T18 (Supplementary Tables 3 and 4) were used for this analysis. Trials were z-scored (mean subtracted and standard deviation divided) with block-specific means and standard deviations of threshold crossing features during the Go period of within-block trials of the isolated finger movement task (to account for non-stationarities in the threshold crossing features that accrue over time across blocks). Trials were then time-averaged in the range of 80 ms before the Go cue to 200 ms after the Go cue for both participants. For every trial, the classifier was trained on all other trials and evaluated on the held-out trial. We used the Scikit-learn⁶⁸ implementation of Gaussian Naive Bayes. For T18, all 384 electrodes were used, whereas for T17 only the 128 6v electrodes and 128 6d electrodes were used (for a total of 256 electrodes). Only T18 trials from left-hand finger movements were used for Extended Data Fig. 6a, and only trials from right-hand finger movements were used for Extended Data Fig. 6b.

RNN

This decoding pipeline has its origins in automatic speech recognition research⁶⁹, and is utilized in several other recent works, particularly iBCI and electrocorticography-based speech decoding^11–13. The RNN used for character sequence decoding is implemented in Tensorflow2 (ref. ⁷⁰) as a five-layer gated recurrent unit network, each with 512 units. Training using backpropagation through time minimizes the CTC loss function^34–36, which aligns binned neural features and sequences of characters. Various regularization and data augmentation techniques are utilized: Dropout, Gaussian white noise, L2 weight norm. Hyperparameter details can be found in Supplementary Table 1. For T17, electrodes from areas 6v and 6d were used for sentence typing decoding, for a total of 256 electrodes (electrodes from area 55b were omitted). For T18, all electrodes were used, for a total of 384 electrodes. RNN inference is performed using threshold crossing and spike band power features for a total of 768 features for T18 and 512 features for T17, which were extracted from raw neural data in real time. These features have been binned into 20-ms bins. In the case of T17, character probabilities were emitted by the RNN every 180 ms on a preceding window of 320 ms of neural features. For T18, character probabilities were inferred every 120 ms on a preceding window of 300 ms of neural features, owing to the faster typing speed of this participant (see ‘RNN hyperparameters’ in Supplementary Table 1).

A character is shown highlighted in green on the keyboard on screen every 180 ms for T17 and every 120 ms for T18. The probability for a character does not need to cross a probability threshold for this to occur; however, the most probable class cannot be the blank class (in which case the previously highlighted key will stay highlighted). The blank class is predicted by the RNN if a participant is at momentary rest. The RNN decoder will generate character probabilities every 120 ms or 180 ms until the sentence is completed; however, if the most probable class decoded is a blank token (which is the case when no movement occurs) or the same character as the previously decoded character, the previously highlighted key will stay highlighted in green on the keyboard. The blank class is ignored by the language model and thus allows the participants to type at any speed they choose.

RNN training

Much recent work focuses on reducing or eliminating the adverse decoding effects of neural non-stationarities^{39,53–55,71,72}. Here, we utilized training data across multiple days; for each trial day, our decoding model is jointly trained on all previously recorded sentence-copy data and sentence-copy calibration blocks from the current day to mitigate the effects of neural non-stationarities on online typing decoding.

For T17, each calibration block usually consisted of 20 sentences, with each session varying in the number of calibration blocks depending on how recent the preceding session was (Supplementary Table 3). Each of these calibration blocks of 20 sentences for T17 had an average duration of 1,105.9 s ± 25.6 s (s.d.) on trial day 144.

For T18, each calibration block consisted of 10 sentences (except trial day 44 where calibration blocks were 20 sentences), with most sessions consisting of 3 calibration blocks before online decoding of sentence-copy sentences or free-typing sentences (Supplementary Table 4). Each of these calibration blocks of 10 sentences for T18 had an average duration of 296.9s ± 18.8 s (s.d.) on trial day 107.

For participant T17, we trained our initial RNN model (tested on trial day 95) on 80 sentence-copy calibration sentences (4 blocks of 20 sentences) from trial day 56 in addition to 120 calibration sentences (6 blocks of 20 sentences) from day 95 (this number of additional initial calibration sentences was required on trial day 95 due to non-stationarities accruing in the large time interval between trial days 56 and 95).

For participant T18, we trained our initial RNN model (tested on trial day 45) on 60 sentence-copy calibration sentences (3 blocks of 20 sentences) from trial day 44 in addition to 30 calibration sentences (3 blocks of 10 sentences) from trial day 45 to ensure optimal performance in online decoding. Isolated finger movements were not used to train the sentence decoding RNN for either participant.

A day-specific nonlinear input layer was added per session to account for cross-session neural variability, as was used in recent work^12,13,39, where using day-specific input layers outperformed the alternative of a shared input layer across all session days. Each nonlinear layer contains the same number of units as the dimensionality of the neural features (768 for T18, 512 for T17).

The day-specific input layers consisted of an affine transformation applied to the input neural feature vector followed by a softsign activation function: $\bar{x}$ = softsign (Wx_t + b), where $\bar{x}$ is the day-transformed input vector at time step t, W_i is a 768 × 768 matrix for T18 and a 512 × 512 matrix for T17, and b_i is a 768 × 1 bias vector for T18 and a 512 × 1 bias vector for T17. The resultant $\bar{x}$ is then used as input to the RNN model.

During training, batches of trials only from a given session were selected at random, such that the corresponding input layer is trained along with the five-layer RNN. This resulted in a cross-session ensemble decoder that was relatively robust to non-stationarities in the short to medium term (resulting in the reported online error rates in Fig. 3b,c).

Weighted finite state transducer language model

We use a 5-gram language model implemented using a weighted finite state transducer, built on the Kaldi system¹⁷. This probabilistic model was trained using the WeNet framework³⁷ and utilized the large OpenWebText2 (ref. ¹⁸) corpus to compute conditional probabilities. When given lattices of RNN output probability vectors, the WeNet framework initiates several Viterbi^73,74 searches through the most recent lattice, incorporating character-by-character transition probabilities when inferring the most likely typed sentence. Language model parameter details can be found in Supplementary Table 2. Punctuation not contained in the keyboard, such as apostrophes, are inferred by the language model based on decoded sentence context. As the user typed a given sentence, the entire sentence history of character probabilities inferred by the RNN thus far was used by the 5-gram language model^17,37 to ascertain the most likely sentence typed given the series of RNN probabilities and sequences of characters likely seen in sequence in English. The most likely sentence deduced by the language model based on RNN output up to that time was continuously displayed on screen.

Character and word error rate definitions

Raw character error rate was calculated as the normalized number of character insertions, deletions and substitutions required to produce a correct sentence based on the top-1 RNN predictions at each time step. Similarly, character error rate post-language model was calculated on language model output. Word error rate was calculated as the number of word insertions, deletions and substitutions required to produce a correct sentence, based on language model output. In each case, the total number of errors was normalized by the number of characters or words in a given sentence.

Self-paced typing

Due to RNN training using the CTC loss, the alignment between unaligned neural features and character sequences can be learned by the decoder. This allows for sequence decoding of characters, where whole sentences are cued or self-generated, instead of fixed intervals for each and every character. This means that key presses can occur and be decoded at any time and at any speed during online decoding. When sentence typing is occurring, the RNN is decoding character probabilities on windows of neural data (180 ms for T17 and 120 ms for T18). The participants are indeed not typing keys at this speed (that is, there is not a 1-to-1 mapping of keyboard characters to decoding windows), so if no key press is detected in a given decoding window, a blank class token is predicted. As a participant increases their attempted typing speed, fewer blank tokens are predicted between keyboard characters. If a participant slows down their typing, more blank tokens are predicted between keyboard characters. As blank tokens are ultimately ignored by the language model, the desired sentence is inferred regardless of typing speed. Typing in the sentence-copy task and the free tying task are both self-paced and the above paradigm applies to both tasks.

Free typing

This conversational mode of typing is not cued. Instead, the RNN decoder begins decoding as soon as the block commences. As a component of the CTC loss function used for RNN training, a blank class is used (in addition to the other 30 keyboard key classes) to account for windows of neural features that do not fit well to any of the 30 keys, and instead represent momentary rest periods between key presses. This is also the case during the sentence-copy task. The blank class is ignored by the language model and thus allows the participants to type at any speed they choose in either task, as any of the time windows of 120 ms (T18) or 180 ms (T17) can be predicted as blank if there is no keyboard character predicted.

Thus, while the participant is not typing actively, this blank class is predicted by the RNN. During free typing, this corresponds to no output on screen. Once a non blank class is predicted by the RNN, sentence decoding proceeds as with the sentence-copy task, with RNN output probabilities sent to the language model before the most likely typed sentence is output on screen. The end of a sentence is signaled either with the prediction of a period (‘.’), a question mark (‘?’) or four consecutive seconds of blank class decodes.

Sentence evaluation

Ground-truth calibration sentences in the sentence-copy task were sourced from transcriptions of the Switchboard Corpus⁷⁵, chosen due to their conversational nature. For T17, each calibration block usually consisted of 20 sentences, with each session varying in the number of calibration blocks depending on how recent the preceding session was (Supplementary Table 3). For T18, each calibration block consisted of 10 sentences (except trial day 44 where calibration blocks were 20 sentences), with most sessions consisting of 3 calibration blocks before online decoding of sentence-copy sentences or free-typing sentences (Supplementary Table 4). For all sessions with both participants, the RNN model was trained jointly with calibration sentences from a given session along with all previous calibration and sentence-copy sentences. The number of evaluation sentences in a given block varied from 10–20 sentences in the sentence-copy task to 4–20 in the free-typing task.

Handwriting versus typing substitution errors

Training of the RNN for this analysis (as opposed to usual RNN training for sentence decoding where training continues until the error rate stops improving) is stopped early once a character error rate (top-1) of 30% is reached in the confusion matrices in Fig. 4a–c for T17 typing, T18 typing and T5 handwriting. The RNN is trained and evaluated on the following trial day’s data for each participant/task as follows:

T17 typing: Trained on all calibration and sentence-copy trials from trial days 56, 96, 97, 105 and 40 trials from trial day 95, with evaluation on 100 held-out trials from trial day 95.

T18 typing: Trained on all calibration and sentence-copy trials from trial days 86, 92, 106, 107 and 30 trials from trial day 93, with evaluation on 100 held-out trials from trial day 93.

T5 handwriting: Trained on all trials from sessions 2019.12.09, 2019.12.11, 2019.12.18, 2019.12.20, 2020.01.06, 2020.01.13 and 2020.01.15 and 24 trials from session 2020.01.08, with evaluation on 60 held-out trials from session 2020.01.08.

Evaluation was performed on 100 held-out trials from a single session in the case of T17 and T18 and 60 trials from a single session in the case of T5. For Fig. 4d, the RNN was trained on data from all sessions for each task and participant, with early stopping of RNN training at each of the indicated character error rates on the x axis. Evaluation was again performed on 100 held-out trials from a single session in the case of T18 and 60 trials from a single session in the case of T5.

Average keystroke output latency

To analyze the latency from the Go cue to the first character decoded for each of the sentences on trial days with the highest typing speeds for each participant (trial days 105, 131, 132 and 144 for T17 and trial days 92, 93, 106 and 107 for T18), we first trained an RNN decoder on sentences from the above trial days. We then performed RNN inference on these sentences every 120 ms (T18) and 180 ms (T17) and counted the number of window slides before the first character of the sentence is decoded instead of a blank token. This is then multiplied by the window slide size in milliseconds and summed with the window size for each participant. We captured average output latency for 21 unique characters for T17 and 22 unique characters for T18, owing to the conversational sentences typed only starting with these characters.

Effect of electrode count on performance

To determine the effect of electrode count on decoding performance (Fig. 4g), 40 seeds of each electrode count up to 380 electrodes were run. For each of the 40 seeds for each electrode count, electrodes were randomly selected without replacement. For this comparison, we used T18 typing data from trial day 107, where typing speed was fastest at 110 characters per minute, demonstrating high decoding accuracy when typing speed is high with comparatively few electrodes. Sixty training sentences for each task were used, with 40 test sentences for handwriting and 100 test sentences for typing.

Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.

Online content

Any methods, additional references, Nature Portfolio reporting summaries, source data, extended data, supplementary information, acknowledgements, peer review information; details of author contributions and competing interests; and statements of data and code availability are available at https://doi.org/10.1038/s41593-026-02218-y.

Supplementary information

Supplementary Information^{(306.3KB, pdf)}

Supplementary Tables 1–5 and Descriptions of Supplementary Videos 1–4.

Reporting Summary^{(58.5KB, pdf)}

Supplementary Video 1^{(29.5MB, mp4)}

Supplementary Video 1 - T17 Sentence Copy Task T17 typing each sentence cued on screen at his own pace. Researcher presses button to end decoding of each sentence. T17 consented to publication of photographs and videos containing his likeness.

Supplementary Video 2^{(57.8MB, mp4)}

Supplementary Video 2 - T18 Sentence Copy Task T18 typing each sentence cued on screen at his own pace. Researcher presses button to end decoding of each sentence. Sentence cue ‘we should go different way’ decoded grammatically correctly as ‘we should go different ways’ due to the language model. T18 consented to publication of photographs and videos containing his likeness.

Supplementary Video 3^{(101.5MB, mp4)}

Supplementary Video 3 - T17 Free Typing T17 typing self-generated sentences to communicate with research team at his own pace. Sentence decoding begins when finger movement is detected and ends after several consecutive no movement decodes. T17 consented to publication of photographs and videos containing his likeness.

Supplementary Video 4^{(49MB, mp4)}

Supplementary Video 4 - T18 Free Typing T18 typing self-generated sentences to answer questions asked by research team at his own pace. Sentence decoding begins when finger movement is detected and ends after several consecutive no movement decodes. T18 consented to publication of photographs and videos containing his likeness.

Source data

Source Data^{(13.1MB, zip)}

Statistical source data for Figs. 1–4 and Extended Data Figs. 1–9.

Acknowledgements

We thank T17, T18, T5, their families and care partners for the time and effort they contributed to this research. We thank S. Mernoff for clinical monitoring of trial participants. We thank G. Hill for illustrations in Figs. 1 and 3. We thank M. Masood, D. Rosler and B. Travers for their regulatory and team management efforts supporting this work. This work was supported by: Office of Research and Development, Department of Veterans Affairs (A2295R, to L.R.H. and J.D.S.; A4820R, to L.R.H. and J.D.S.; N2864C, to L.R.H.; A3803R, to J.D.S.), National Institutes of Health (NIH) NIDCD (U01DC017844, to L.R.H. and J.D.S.), K23DC021297 (to D.B.R.), NIH NINDS (U01NS123101, to L.R.H. and J.D.S.), AHA (23SCEFIA1156586, to D.B.R.), CDMRP (HT94252310153, to S.D.S.), a Pilot Award from the Simons Collaboration for the Global Brain (872146SPI, to S.D.S.), A.P. Giannini Postdoctoral Fellowship (NSC) and a Career Award at the Scientific Interface from BWF (to S.D.S.). The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH, or the Department of Veterans Affairs or the US Government. Several analyses were performed using code from GitHub (https://github.com/fwillett/speechBCI/), and T5 handwriting data were acquired from Dryad (https://doi.org/10.5061/dryad.wh70rxwmv)⁶³.

Extended data

Author contributions

J.J.J. conceived the study, built the real-time typing decoder, wrote the manuscript and led the development, analysis and interpretation of all experiments. H.L.-A. and D.B.R. reviewed analysis and interpretation of experimental data. F.R.W. designed the real-time sequence decoder paradigm, originally incorporated the n-gram language model into sequence decoding and produced offline analysis tools. N.S.C., M.W., D.M.B. and S.D.S. built the real-time BRAND feature extraction system, implemented the neural brain-to-text decoding pipeline in BRAND, built the task control interface and offline analysis tools. S.B.A., M.W. and N.S.C. wrote software for neural signal processing. S.B.A., J.D.S. and J.J.J. installed hardware for neural recording with T17 and T18. J.J.J. and B.E.L. collected T18 typing session data. J.J.J., A.D.L. and B.E.L. collected T18 handwriting session data. A.J.A., H.L.-A., C.N. and J.J.J. collected T17 typing session data. F.R.W. collected T5 handwriting session data. J.D.S. managed integration and deployment of BCI system software and hardware. L.R.H. is the sponsor-investigator of the multisite BrainGate2 pilot clinical trial. Z.M.W., D.B.R. and L.R.H. planned T17 and T18 array placement surgeries, and Z.M.W. performed T17 and T18 array placement surgeries. L.R.H. was responsible for all clinical trial-related activity at VA Providence and MGH. L.R.H. and D.B.R. supervised and guided all research activity with T17 and T18. The study was supervised and guided by D.B.R. All authors reviewed and edited the manuscript.

Peer review

Peer review information

Nature Neuroscience thanks Iahn Cajigas, Blaise Yvert and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Data availability

Raw data relevant to this study can be found on Dryad via https://doi.org/10.5061/dryad.cz8w9gjjk (ref. ⁷⁶). Source data are provided with this paper.

Code availability

Code to replicate the main findings of this study is available in a public GitHub repository at https://github.com/justin-jude/typingBCI/.

Competing interests

The MGH Translational Research Center has a clinical research support agreement with Axoft, Neuralink, Neurobionics, Precision Neuro, Synchron and Reach Neuro, for which L.R.H. provides consultative input. L.R.H. is a co-investigator on an NIH SBIR grant with Paradromics and is a non-compensated member of the Board of Directors of a nonprofit assistive communication device technology foundation (Speak Your Mind Foundation). The MGH Translational Research Center has a clinical research support agreement with Paradromics, for which DBR provides consultative input. Mass General Brigham is convening the Implantable Brain-Computer Interface Collaborative Community (iBCI-CC); charitable gift agreements to Mass General Brigham, including those received to date from Paradromics, Synchron, Precision Neuro, Neuralink and Blackrock Neurotech, support the iBCI-CC, for which L.R.H. provides effort. S.D.S. and F.R.W. are inventors on intellectual property owned by Stanford University that has been licensed to Blackrock Neurotech and Neuralink. M.W., S.D.S. and D.M.B. have patent applications related to speech BCI owned by the Regents of the University of California including intellectual property, which has been licensed to a neurotechnology startup. D.M.B. was a surgical consultant for Paradromics. S.D.S. is an advisor to Sonera and a consultant to Neuralink. N.S.C. is a visiting neuroengineer for Neuralink. The other authors declare no competing interests.

Footnotes

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data

is available for this paper at https://doi.org/10.1038/s41593-026-02218-y.

Supplementary information

The online version contains supplementary material available at https://doi.org/10.1038/s41593-026-02218-y.

References

1.Hochberg, L. R. & Cudkowicz, M. E. Locked in, but not out? Neurology82, 1852–1853(2014). [DOI] [PubMed]
2.Lulé, D. et al. Life can be worth living in locked-in syndrome. Prog. Brain Res.177, 339–351 (2009). [DOI] [PubMed]
3.Smidt, A. & Pebdani, R. N. Rethinking device abandonment: a capability approach focused model. Augment. Altern. Commun.39, 198–206 (2023). [DOI] [PubMed]
4.Waller, A. Telling tales: unlocking the potential of AAC technologies. Int. J. Lang. Commun. Disord.54, 159–169 (2019). [DOI] [PMC free article] [PubMed]
5.Pandarinath, C. et al. High performance communication by people with paralysis using an intracortical brain-computer interface. Elife6, e18554 (2017). [DOI] [PMC free article] [PubMed] [Google Scholar]
6.Bacher, D. et al. Neural point-and-click communication by a person with incomplete locked-in syndrome. Neurorehabil. Neural Repair29 462–471 (2015). [DOI] [PMC free article] [PubMed]
7.Jarosiewicz, B. et al. Virtual typing by people with tetraplegia using a self-calibrating intracortical brain-computer interface. Sci. Transl. Med.7, 313ra179 (2015). [DOI] [PMC free article] [PubMed]
8.Brandman, D. M. et al. Rapid calibration of an intracortical brain-computer interface for people with tetraplegia. J. Neural Eng.15, 026007 (2018). [DOI] [PMC free article] [PubMed]
9.Singer-Clark, T. et al. Speech motor cortex enables BCI cursor control and click. J. Neural Eng.22, 036015 (2024). [DOI] [PMC free article] [PubMed]
10.Chang, E. F. & Anumanchipalli, G. K. Toward a speech neuroprosthesis. JAMA323, 413–414 (2020). [DOI] [PMC free article] [PubMed]
11.Metzger, S. L. et al. A high-performance neuroprosthesis for speech decoding and avatar control. Nature620, 1037–1046 (2023). [DOI] [PMC free article] [PubMed]
12.Willett, F. R. et al. A high-performance speech neuroprosthesis. Nature620, 1031–1036 (2023). [DOI] [PMC free article] [PubMed]
13.Card, N. S. et al. An accurate and rapidly calibrating speech neuroprosthesis. N. Engl. J. Med.391, 609–618 (2024). [DOI] [PMC free article] [PubMed] [Google Scholar]
14.Wairagkar, M. et al. An instantaneous voice-synthesis neuroprosthesis. Nature 10.1038/s41586-025-09127-3 (2025). [DOI] [PMC free article] [PubMed]
15.Kunz, E. M. et al. Inner speech in motor cortex and implications for speech neuroprostheses. Cell 10.1016/j.cell.2025.06.015 (2025). [DOI] [PMC free article] [PubMed]
16.Willett, F. R., Avansino, D. T., Hochberg, L. R., Henderson, J. M. & Shenoy, K. V. High-performance brain-to-text communication via handwriting. Nature593, 249–254 (2021). [DOI] [PMC free article] [PubMed]
17.Povey, D. et al. The Kaldi Speech Recognition Toolkit. In IEEE 2011 Workshop on Automatic Speech Recognition and Understanding (IEEE Signal Processing Society, 2011).
18.Gao, L. et al. The Pile: an 800 GB dataset of diverse text for language modeling. Preprint at https://arxiv.org/abs/2101.00027 (2020).
19.Nason, S. R. et al. Real-time linear prediction of simultaneous and independent movements of two finger groups using an intracortical brain-machine interface. Neuron109, 3164–3177 (2021). [DOI] [PMC free article] [PubMed]
20.Aggarwal, V. et al. Asynchronous decoding of dexterous finger movements using M1 neurons. IEEE Trans. Neural Sys. Rehabil. Eng.16, 3–14 (2008). [DOI] [PMC free article] [PubMed] [Google Scholar]
21.Jorge, A., Royston, D. A., Tyler-Kabara, E. C., Boninger, M. L. & Collinger, J. L. Classification of individual finger movements using intracortical recordings in human motor cortex. Neurosurgery87, 630–638 (2020). [DOI] [PubMed]
22.Wodlinger, B. et al. Ten-dimensional anthropomorphic arm control in a human brain-machine interface: difficulties, solutions, and limitations. J. Neural Eng.12, 016011 (2015). [DOI] [PubMed]
23.Shah, N. P. et al. Pseudo-linear summation explains neural geometry of multi-finger movements in human premotor cortex. Nat Commun.16, 5008 (2023). [DOI] [PMC free article] [PubMed]
24.Willsey, M. S. et al. A high-performance brain–computer interface for finger decoding and quadcopter game control in an individual with paralysis. Nat. Med.31, 96–104 (2025). [DOI] [PMC free article] [PubMed] [Google Scholar]
25.Vargas-Irwin, C. E. et al. Gesture encoding in human left precentral gyrus neuronal ensembles. Commun. Biol. 10.1038/s42003-025-08557-z (2025). [DOI] [PMC free article] [PubMed]
26.Guan, C. et al. Decoding and geometry of ten finger movements in human posterior parietal cortex and motor cortex. J. Neural Eng. 10.1088/1741-2552/acd3b1 (2023). [DOI] [PMC free article] [PubMed]
27.Ifft, P. J., Shokur, S., Li, Z., Lebedev, M. A. & Nicolelis, M. A. L. A brain-machine interface enables bimanual arm movements in monkeys. Sci. Transl. Med.5, 154–210 (2013). [DOI] [PMC free article] [PubMed] [Google Scholar]
28.Deo, D. R. et al. Brain control of bimanual movement enabled by recurrent neural networks. Sci. Rep.14, 1598 (2024). [DOI] [PMC free article] [PubMed]
29.Shah, N. P. et al. A flexible intracortical brain-computer interface for typing using finger movements. Preprint at bioRxiv 10.1101/2024.04.22.590630 (2024). [DOI]
30.Sivakumar, V. et al. emg2qwerty: A large dataset with baselines for touch typing using surface electromyography. In 38th Conference on Neural Information Processing Systems (NeurIPS 2024) Track on Datasets and Benchmarkshttps://proceedings.neurips.cc/paper_files/paper/2024/file/a64d53074d011e49af1dfc72c332fe4b-Paper-Datasets_and_Benchmarks_Track.pdf (2024).
31.Lévy, J. et al. Brain-to-text decoding: a non-invasive approach via typing. Preprint at https://arxiv.org/abs/2502.17480 (2025).
32.Noyes, J. The QWERTY keyboard: a review. Int. J. Man Mach. Stud.18, 265–281 (1983).
33.Kinkead, R. Typing speed, keying rates, and optimal keyboard layouts. In Proc. Human Factors and Ergonomics Society Annual Meetinghttps://api.semanticscholar.org/CorpusID:62715400 (1975).
34.Graves, A., Fernández, S., Gomez, F. & Schmidhuber, J. Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks. In ACM International Conference Proceeding Series 10.1145/1143844.1143891 (2006). [DOI]
35.Graves, A. et al. A novel connectionist system for unconstrained handwriting recognition. IEEE Trans. Pattern Anal. Mach. Intell.31, 855–868 (2009). [DOI] [PubMed]
36.Graves, A. et al. Speech recognition with deep recurrent neural networks. In 2013 IEEE International Conference on Acoustics, Speech and Signal Processing 10.1109/ICASSP.2013.6638947 (IEEE, 2013). [DOI]
37.Yao, Z. et al. WeNet: production oriented streaming and non-streaming end-to-end speech recognition toolkit. In Proc. Interspeech 2021 10.21437/Interspeech.2021-1983 (2021). [DOI]
38.Goodman, J. et al. Language modeling for soft keyboards. In Proc. 7th International Conference on Intelligent User Interfaces 10.1145/502716.502753 (2002). [DOI]
39.Fan, C. et al. Plug-and-play stability for intracortical brain-computer interfaces: a one-year demonstration of seamless brain-to-text communication. Adv. Neural Inf. Process. Syst.36, 42258–42270 (2023). [PMC free article] [PubMed]
40.Palin, K. et al. How do people type on mobile devices? Observations from a study with 37,000 volunteers. In Proc. 21st International Conference on Human-Computer Interaction with Mobile Devices and Services 10.1145/3338286.3340120 (2019). [DOI]
41.Metzger, S. L. et al. Generalizable spelling using a speech neuroprosthesis in an individual with severe limb and vocal paralysis. Nat. Commun.13, 6510 (2022). [DOI] [PMC free article] [PubMed]
42.West, L. J. The standard and Dvorak keyboards revisited: direct measures of speed. Paper 98-05-041. Santa Fe Institutehttps://www.santafe.edu/research/results/working-papers/the-standard-and-dvorak-keyboards-revisited-direct (1998).
43.Kirk, A. Improving the Accuracy of Mobile Touchscreen QWERTY Keyboards. M.S. thesis, Univ. of Denver (2018).
44.Smith, B. A., Bi, X. & Zhai, S. Optimizing touchscreen keyboards for gesture typing. In Proc. 33rd Annual ACMConference on Human Factors in Computing Systems 3365–3374 (2015).
45.Yousry, T. A. et al. Localization of the motor hand area to a knob on the precentral gyrus. A new landmark. Brain120, 141–157 (1997). [DOI] [PubMed]
46.Deo, D. R. et al. A mosaic of whole-body representations in human motor cortex. Preprint at bioRxiv 10.1101/2024.09.14.613041 (2024). [DOI]
47.Bundy, D. T., Szrama, N., Pahwa, M. & Leuthardt, E. C. Unilateral, 3D arm movement kinematics are encoded in ipsilateral human cortex. J. Neurosci.38, 10042–10056 (2018). [DOI] [PMC free article] [PubMed]
48.Heming, E. A., Cross, K. P., Takei, T., Cook, D. J. & Scott, S. H. Independent representations of ipsilateral and contralateral limbs in primary motor cortex. Elife8, e48190 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]
49.Ames, K. C. & Churchland, M. M. Motor cortex signals for each arm are mixed across hemispheres and neurons yet partitioned within the population response. Elife8, e46159 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]
50.Como, J. J. et al. Characterizing the need for mechanical ventilation following cervical spinal cord injury with neurologic deficit. J. Trauma59, 912–916 (2005). [DOI] [PubMed]
51.Schreiber, A. F. et al. Separation from mechanical ventilation and survival after spinal cord injury: a systematic review and meta-analysis. Ann. Intensive Care11, 149 (2021). [DOI] [PMC free article] [PubMed]
52.Devivo, M. J. & Chen, Y. Trends in new injuries, prevalent cases, and aging with spinal cord injury. Arch. Phys. Med. Rehabil.92, 332–338 (2011). [DOI] [PubMed]
53.Karpowicz, B. M. et al. Stabilizing brain-computer interfaces through alignment of latent dynamics. Nat. Commun.16, 4662 (2025). [DOI] [PMC free article] [PubMed] [Google Scholar]
54.Jude, J. et al. Robust alignment of cross-session recordings of neural population activity by behaviour via unsupervised domain adaptation. In Proc. 39th International Conference on Machine Learninghttps://proceedings.mlr.press/v162/jude22a.html (PMLR, 2022).
55.Farshchian, A. et al. Adversarial domain adaptation for stable brain-machine interfaces. In Proc.7th International Conference on Learning Representations (ICLR)https://openreview.net/pdf/42b1af648df0cfebe9b454256f895b390736f949.pdf (2019).
56.Azabou, M. et al. A unified, scalable framework for neural population decoding. In 37th Conference on Neural Information Processing Systems (NeurIPS 2023)https://proceedings.neurips.cc/paper_files/paper/2023/file/8ca113d122584f12a6727341aaf58887-Paper-Conference.pdf (2023).
57.Ye, J., Collinger, J., Wehbe, L., & Gaunt, R. Neural Data Transformer 2: multi-context pretraining for neural spiking activity. In 37th Conference on Neural Information Processing Systems (NeurIPS 2023)https://papers.neurips.cc/paper_files/paper/2023/file/fe51de4e7baf52e743b679e3bdba7905-Paper-Conference.pdf (2023).
58.Ye, J. et al. A generalist intracortical motor decoder. In The Thirty-Ninth Annual Conference on Neural Information Processing Systemshttps://openreview.net/forum?id=utXSSdD9mt (2025).
59.Ryoo, A. H.-W. et al. Generalizable, real-time neural decoding with hybrid state-space models. In The Thirty-Ninth Annual Conference on Neural Information Processing Systemshttps://openreview.net/forum?id=1i4wNFgHDd (2025).
60.Card, N. S. et al. Long-term independent use of an intracortical brain-computer interface for speech and cursor control. Preprint at bioRxiv 10.1101/2025.06.26.661591 (2025). [DOI] [PMC free article] [PubMed]
61.Glasser, M. F. et al. The minimal preprocessing pipelines for the Human Connectome Project. Neuroimage80, 105–124 (2013). [DOI] [PMC free article] [PubMed]
62.Glasser, M. F. et al. A multi-modal parcellation of human cerebral cortex. Nature 171–178, 536 (2016). [DOI] [PMC free article] [PubMed]
63.Willett, F. et al. Data from: high-performance brain-to-text communication via handwriting. Dryad 10.5061/dryad.wh70rxwmv (2021). [DOI] [PMC free article] [PubMed]
64.Young, D. et al. Signal processing methods for reducing artifacts in microelectrode brain recordings caused by functional electrical stimulation. J. Neural Eng.15, 026014 (2018). [DOI] [PMC free article] [PubMed]
65.Penrose, R. A generalized inverse for matrices. Math. Proc. Cambridge Philos. Soc.51, 406–413 (1955).
66.Masse, N. Y. et al. Non-causal spike filtering improves decoding of movement intention for intracortical BCIs. J. Neurosci. Methods236, 58–67 (2014). [DOI] [PMC free article] [PubMed]
67.Ali, Y. H. et al. BRAND: a platform for closed-loop experiments with deep network models. J. Neural Eng.21, 026046 (2024). [DOI] [PMC free article] [PubMed]
68.Pedregosa, F. et al. Scikit-learn: machine learning in Python. J. Mach. Learn. Res.12, 2825–2830 (2011).
69.Miao, Y. et al. EESEN: end-to-end speech recognition using deep RNN models and WFST-based decoding. In 2015 IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) 10.1109/ASRU.2015.7404790 (2015). [DOI]
70.Abadi, M. et al. TensorFlow: a system for large-scale machine learning. In Proc. 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI) 265–283 (2016).
71.Jude, J., Perich, M. G., Miller, L. E. & Hennig, M. H. Capturing cross-session neural population variability through self-supervised identification of consistent neuron ensembles. In Proc. 1st NeurIPS Workshop on Symmetry and Geometry in Neural Representations 234–257 (PMLR, 2023).
72.Pun, T. K. et al. Measuring instability in chronic human intracortical neural recordings towards stable, long-term brain-computer interfaces. Commun. Biol.7, 1363 (2024). [DOI] [PMC free article] [PubMed] [Google Scholar]
73.Hunt, A. J. & Black, A. W. Unit selection in a concatenative speech synthesis system using a large speech database. In Proc. 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Vol. 1 10.1109/ICASSP.1996.541110 (1996). [DOI]
74.Eddy, S. R. Accelerated profile HMM searches. PLoS Comput. Biol.7, e1002195 (2011). [DOI] [PMC free article] [PubMed]
75.Godfrey, J. J. et al. SWITCHBOARD: telephone speech corpus for research and development. In Proc. ICASSP-92: 1992 IEEE International Conference on Acoustics, Speech, and Signal Processing Vol. 1 10.1109/ICASSP.1992.225858 (1992). [DOI]
76.Jude, J. et al. Restoring rapid natural bimanual typing with a neuroprosthesis after paralysis. Dryad 10.5061/dryad.cz8w9gjjk (2026). [DOI] [PMC free article] [PubMed]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary Information^{(306.3KB, pdf)}

Supplementary Tables 1–5 and Descriptions of Supplementary Videos 1–4.

Reporting Summary^{(58.5KB, pdf)}

Supplementary Video 1^{(29.5MB, mp4)}

Supplementary Video 2^{(57.8MB, mp4)}

Supplementary Video 3^{(101.5MB, mp4)}

Supplementary Video 4^{(49MB, mp4)}

Source Data^{(13.1MB, zip)}

Statistical source data for Figs. 1–4 and Extended Data Figs. 1–9.

Data Availability Statement

Raw data relevant to this study can be found on Dryad via https://doi.org/10.5061/dryad.cz8w9gjjk (ref. ⁷⁶). Source data are provided with this paper.

Code to replicate the main findings of this study is available in a public GitHub repository at https://github.com/justin-jude/typingBCI/.

[CR1] 1.Hochberg, L. R. & Cudkowicz, M. E. Locked in, but not out? Neurology82, 1852–1853(2014). [DOI] [PubMed]

[CR2] 2.Lulé, D. et al. Life can be worth living in locked-in syndrome. Prog. Brain Res.177, 339–351 (2009). [DOI] [PubMed]

[CR3] 3.Smidt, A. & Pebdani, R. N. Rethinking device abandonment: a capability approach focused model. Augment. Altern. Commun.39, 198–206 (2023). [DOI] [PubMed]

[CR4] 4.Waller, A. Telling tales: unlocking the potential of AAC technologies. Int. J. Lang. Commun. Disord.54, 159–169 (2019). [DOI] [PMC free article] [PubMed]

[CR5] 5.Pandarinath, C. et al. High performance communication by people with paralysis using an intracortical brain-computer interface. Elife6, e18554 (2017). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR6] 6.Bacher, D. et al. Neural point-and-click communication by a person with incomplete locked-in syndrome. Neurorehabil. Neural Repair29 462–471 (2015). [DOI] [PMC free article] [PubMed]

[CR7] 7.Jarosiewicz, B. et al. Virtual typing by people with tetraplegia using a self-calibrating intracortical brain-computer interface. Sci. Transl. Med.7, 313ra179 (2015). [DOI] [PMC free article] [PubMed]

[CR8] 8.Brandman, D. M. et al. Rapid calibration of an intracortical brain-computer interface for people with tetraplegia. J. Neural Eng.15, 026007 (2018). [DOI] [PMC free article] [PubMed]

[CR9] 9.Singer-Clark, T. et al. Speech motor cortex enables BCI cursor control and click. J. Neural Eng.22, 036015 (2024). [DOI] [PMC free article] [PubMed]

[CR10] 10.Chang, E. F. & Anumanchipalli, G. K. Toward a speech neuroprosthesis. JAMA323, 413–414 (2020). [DOI] [PMC free article] [PubMed]

[CR11] 11.Metzger, S. L. et al. A high-performance neuroprosthesis for speech decoding and avatar control. Nature620, 1037–1046 (2023). [DOI] [PMC free article] [PubMed]

[CR12] 12.Willett, F. R. et al. A high-performance speech neuroprosthesis. Nature620, 1031–1036 (2023). [DOI] [PMC free article] [PubMed]

[CR13] 13.Card, N. S. et al. An accurate and rapidly calibrating speech neuroprosthesis. N. Engl. J. Med.391, 609–618 (2024). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR14] 14.Wairagkar, M. et al. An instantaneous voice-synthesis neuroprosthesis. Nature 10.1038/s41586-025-09127-3 (2025). [DOI] [PMC free article] [PubMed]

[CR15] 15.Kunz, E. M. et al. Inner speech in motor cortex and implications for speech neuroprostheses. Cell 10.1016/j.cell.2025.06.015 (2025). [DOI] [PMC free article] [PubMed]

[CR16] 16.Willett, F. R., Avansino, D. T., Hochberg, L. R., Henderson, J. M. & Shenoy, K. V. High-performance brain-to-text communication via handwriting. Nature593, 249–254 (2021). [DOI] [PMC free article] [PubMed]

[CR17] 17.Povey, D. et al. The Kaldi Speech Recognition Toolkit. In IEEE 2011 Workshop on Automatic Speech Recognition and Understanding (IEEE Signal Processing Society, 2011).

[CR18] 18.Gao, L. et al. The Pile: an 800 GB dataset of diverse text for language modeling. Preprint at https://arxiv.org/abs/2101.00027 (2020).

[CR19] 19.Nason, S. R. et al. Real-time linear prediction of simultaneous and independent movements of two finger groups using an intracortical brain-machine interface. Neuron109, 3164–3177 (2021). [DOI] [PMC free article] [PubMed]

[CR20] 20.Aggarwal, V. et al. Asynchronous decoding of dexterous finger movements using M1 neurons. IEEE Trans. Neural Sys. Rehabil. Eng.16, 3–14 (2008). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR21] 21.Jorge, A., Royston, D. A., Tyler-Kabara, E. C., Boninger, M. L. & Collinger, J. L. Classification of individual finger movements using intracortical recordings in human motor cortex. Neurosurgery87, 630–638 (2020). [DOI] [PubMed]

[CR22] 22.Wodlinger, B. et al. Ten-dimensional anthropomorphic arm control in a human brain-machine interface: difficulties, solutions, and limitations. J. Neural Eng.12, 016011 (2015). [DOI] [PubMed]

[CR23] 23.Shah, N. P. et al. Pseudo-linear summation explains neural geometry of multi-finger movements in human premotor cortex. Nat Commun.16, 5008 (2023). [DOI] [PMC free article] [PubMed]

[CR24] 24.Willsey, M. S. et al. A high-performance brain–computer interface for finger decoding and quadcopter game control in an individual with paralysis. Nat. Med.31, 96–104 (2025). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR25] 25.Vargas-Irwin, C. E. et al. Gesture encoding in human left precentral gyrus neuronal ensembles. Commun. Biol. 10.1038/s42003-025-08557-z (2025). [DOI] [PMC free article] [PubMed]

[CR26] 26.Guan, C. et al. Decoding and geometry of ten finger movements in human posterior parietal cortex and motor cortex. J. Neural Eng. 10.1088/1741-2552/acd3b1 (2023). [DOI] [PMC free article] [PubMed]

[CR27] 27.Ifft, P. J., Shokur, S., Li, Z., Lebedev, M. A. & Nicolelis, M. A. L. A brain-machine interface enables bimanual arm movements in monkeys. Sci. Transl. Med.5, 154–210 (2013). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR28] 28.Deo, D. R. et al. Brain control of bimanual movement enabled by recurrent neural networks. Sci. Rep.14, 1598 (2024). [DOI] [PMC free article] [PubMed]

[CR29] 29.Shah, N. P. et al. A flexible intracortical brain-computer interface for typing using finger movements. Preprint at bioRxiv 10.1101/2024.04.22.590630 (2024). [DOI]

[CR30] 30.Sivakumar, V. et al. emg2qwerty: A large dataset with baselines for touch typing using surface electromyography. In 38th Conference on Neural Information Processing Systems (NeurIPS 2024) Track on Datasets and Benchmarkshttps://proceedings.neurips.cc/paper_files/paper/2024/file/a64d53074d011e49af1dfc72c332fe4b-Paper-Datasets_and_Benchmarks_Track.pdf (2024).

[CR31] 31.Lévy, J. et al. Brain-to-text decoding: a non-invasive approach via typing. Preprint at https://arxiv.org/abs/2502.17480 (2025).

[CR32] 32.Noyes, J. The QWERTY keyboard: a review. Int. J. Man Mach. Stud.18, 265–281 (1983).

[CR33] 33.Kinkead, R. Typing speed, keying rates, and optimal keyboard layouts. In Proc. Human Factors and Ergonomics Society Annual Meetinghttps://api.semanticscholar.org/CorpusID:62715400 (1975).

[CR34] 34.Graves, A., Fernández, S., Gomez, F. & Schmidhuber, J. Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks. In ACM International Conference Proceeding Series 10.1145/1143844.1143891 (2006). [DOI]

[CR35] 35.Graves, A. et al. A novel connectionist system for unconstrained handwriting recognition. IEEE Trans. Pattern Anal. Mach. Intell.31, 855–868 (2009). [DOI] [PubMed]

[CR36] 36.Graves, A. et al. Speech recognition with deep recurrent neural networks. In 2013 IEEE International Conference on Acoustics, Speech and Signal Processing 10.1109/ICASSP.2013.6638947 (IEEE, 2013). [DOI]

[CR37] 37.Yao, Z. et al. WeNet: production oriented streaming and non-streaming end-to-end speech recognition toolkit. In Proc. Interspeech 2021 10.21437/Interspeech.2021-1983 (2021). [DOI]

[CR38] 38.Goodman, J. et al. Language modeling for soft keyboards. In Proc. 7th International Conference on Intelligent User Interfaces 10.1145/502716.502753 (2002). [DOI]

[CR39] 39.Fan, C. et al. Plug-and-play stability for intracortical brain-computer interfaces: a one-year demonstration of seamless brain-to-text communication. Adv. Neural Inf. Process. Syst.36, 42258–42270 (2023). [PMC free article] [PubMed]

[CR40] 40.Palin, K. et al. How do people type on mobile devices? Observations from a study with 37,000 volunteers. In Proc. 21st International Conference on Human-Computer Interaction with Mobile Devices and Services 10.1145/3338286.3340120 (2019). [DOI]

[CR41] 41.Metzger, S. L. et al. Generalizable spelling using a speech neuroprosthesis in an individual with severe limb and vocal paralysis. Nat. Commun.13, 6510 (2022). [DOI] [PMC free article] [PubMed]

[CR42] 42.West, L. J. The standard and Dvorak keyboards revisited: direct measures of speed. Paper 98-05-041. Santa Fe Institutehttps://www.santafe.edu/research/results/working-papers/the-standard-and-dvorak-keyboards-revisited-direct (1998).

[CR43] 43.Kirk, A. Improving the Accuracy of Mobile Touchscreen QWERTY Keyboards. M.S. thesis, Univ. of Denver (2018).

[CR44] 44.Smith, B. A., Bi, X. & Zhai, S. Optimizing touchscreen keyboards for gesture typing. In Proc. 33rd Annual ACMConference on Human Factors in Computing Systems 3365–3374 (2015).

[CR45] 45.Yousry, T. A. et al. Localization of the motor hand area to a knob on the precentral gyrus. A new landmark. Brain120, 141–157 (1997). [DOI] [PubMed]

[CR46] 46.Deo, D. R. et al. A mosaic of whole-body representations in human motor cortex. Preprint at bioRxiv 10.1101/2024.09.14.613041 (2024). [DOI]

[CR47] 47.Bundy, D. T., Szrama, N., Pahwa, M. & Leuthardt, E. C. Unilateral, 3D arm movement kinematics are encoded in ipsilateral human cortex. J. Neurosci.38, 10042–10056 (2018). [DOI] [PMC free article] [PubMed]

[CR48] 48.Heming, E. A., Cross, K. P., Takei, T., Cook, D. J. & Scott, S. H. Independent representations of ipsilateral and contralateral limbs in primary motor cortex. Elife8, e48190 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR49] 49.Ames, K. C. & Churchland, M. M. Motor cortex signals for each arm are mixed across hemispheres and neurons yet partitioned within the population response. Elife8, e46159 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR50] 50.Como, J. J. et al. Characterizing the need for mechanical ventilation following cervical spinal cord injury with neurologic deficit. J. Trauma59, 912–916 (2005). [DOI] [PubMed]

[CR51] 51.Schreiber, A. F. et al. Separation from mechanical ventilation and survival after spinal cord injury: a systematic review and meta-analysis. Ann. Intensive Care11, 149 (2021). [DOI] [PMC free article] [PubMed]

[CR52] 52.Devivo, M. J. & Chen, Y. Trends in new injuries, prevalent cases, and aging with spinal cord injury. Arch. Phys. Med. Rehabil.92, 332–338 (2011). [DOI] [PubMed]

[CR53] 53.Karpowicz, B. M. et al. Stabilizing brain-computer interfaces through alignment of latent dynamics. Nat. Commun.16, 4662 (2025). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR54] 54.Jude, J. et al. Robust alignment of cross-session recordings of neural population activity by behaviour via unsupervised domain adaptation. In Proc. 39th International Conference on Machine Learninghttps://proceedings.mlr.press/v162/jude22a.html (PMLR, 2022).

[CR55] 55.Farshchian, A. et al. Adversarial domain adaptation for stable brain-machine interfaces. In Proc.7th International Conference on Learning Representations (ICLR)https://openreview.net/pdf/42b1af648df0cfebe9b454256f895b390736f949.pdf (2019).

[CR56] 56.Azabou, M. et al. A unified, scalable framework for neural population decoding. In 37th Conference on Neural Information Processing Systems (NeurIPS 2023)https://proceedings.neurips.cc/paper_files/paper/2023/file/8ca113d122584f12a6727341aaf58887-Paper-Conference.pdf (2023).

[CR57] 57.Ye, J., Collinger, J., Wehbe, L., & Gaunt, R. Neural Data Transformer 2: multi-context pretraining for neural spiking activity. In 37th Conference on Neural Information Processing Systems (NeurIPS 2023)https://papers.neurips.cc/paper_files/paper/2023/file/fe51de4e7baf52e743b679e3bdba7905-Paper-Conference.pdf (2023).

[CR58] 58.Ye, J. et al. A generalist intracortical motor decoder. In The Thirty-Ninth Annual Conference on Neural Information Processing Systemshttps://openreview.net/forum?id=utXSSdD9mt (2025).

[CR59] 59.Ryoo, A. H.-W. et al. Generalizable, real-time neural decoding with hybrid state-space models. In The Thirty-Ninth Annual Conference on Neural Information Processing Systemshttps://openreview.net/forum?id=1i4wNFgHDd (2025).

[CR60] 60.Card, N. S. et al. Long-term independent use of an intracortical brain-computer interface for speech and cursor control. Preprint at bioRxiv 10.1101/2025.06.26.661591 (2025). [DOI] [PMC free article] [PubMed]

[CR61] 61.Glasser, M. F. et al. The minimal preprocessing pipelines for the Human Connectome Project. Neuroimage80, 105–124 (2013). [DOI] [PMC free article] [PubMed]

[CR62] 62.Glasser, M. F. et al. A multi-modal parcellation of human cerebral cortex. Nature 171–178, 536 (2016). [DOI] [PMC free article] [PubMed]

[CR63] 63.Willett, F. et al. Data from: high-performance brain-to-text communication via handwriting. Dryad 10.5061/dryad.wh70rxwmv (2021). [DOI] [PMC free article] [PubMed]

[CR64] 64.Young, D. et al. Signal processing methods for reducing artifacts in microelectrode brain recordings caused by functional electrical stimulation. J. Neural Eng.15, 026014 (2018). [DOI] [PMC free article] [PubMed]

[CR65] 65.Penrose, R. A generalized inverse for matrices. Math. Proc. Cambridge Philos. Soc.51, 406–413 (1955).

[CR66] 66.Masse, N. Y. et al. Non-causal spike filtering improves decoding of movement intention for intracortical BCIs. J. Neurosci. Methods236, 58–67 (2014). [DOI] [PMC free article] [PubMed]

[CR67] 67.Ali, Y. H. et al. BRAND: a platform for closed-loop experiments with deep network models. J. Neural Eng.21, 026046 (2024). [DOI] [PMC free article] [PubMed]

[CR68] 68.Pedregosa, F. et al. Scikit-learn: machine learning in Python. J. Mach. Learn. Res.12, 2825–2830 (2011).

[CR69] 69.Miao, Y. et al. EESEN: end-to-end speech recognition using deep RNN models and WFST-based decoding. In 2015 IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) 10.1109/ASRU.2015.7404790 (2015). [DOI]

[CR70] 70.Abadi, M. et al. TensorFlow: a system for large-scale machine learning. In Proc. 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI) 265–283 (2016).

[CR71] 71.Jude, J., Perich, M. G., Miller, L. E. & Hennig, M. H. Capturing cross-session neural population variability through self-supervised identification of consistent neuron ensembles. In Proc. 1st NeurIPS Workshop on Symmetry and Geometry in Neural Representations 234–257 (PMLR, 2023).

[CR72] 72.Pun, T. K. et al. Measuring instability in chronic human intracortical neural recordings towards stable, long-term brain-computer interfaces. Commun. Biol.7, 1363 (2024). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR73] 73.Hunt, A. J. & Black, A. W. Unit selection in a concatenative speech synthesis system using a large speech database. In Proc. 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Vol. 1 10.1109/ICASSP.1996.541110 (1996). [DOI]

[CR74] 74.Eddy, S. R. Accelerated profile HMM searches. PLoS Comput. Biol.7, e1002195 (2011). [DOI] [PMC free article] [PubMed]

[CR75] 75.Godfrey, J. J. et al. SWITCHBOARD: telephone speech corpus for research and development. In Proc. ICASSP-92: 1992 IEEE International Conference on Acoustics, Speech, and Signal Processing Vol. 1 10.1109/ICASSP.1992.225858 (1992). [DOI]

[CR76] 76.Jude, J. et al. Restoring rapid natural bimanual typing with a neuroprosthesis after paralysis. Dryad 10.5061/dryad.cz8w9gjjk (2026). [DOI] [PMC free article] [PubMed]

PERMALINK

Restoring rapid natural bimanual typing with a neuroprosthesis after paralysis

Justin J Jude

Hadar Levi-Aharoni

Alexander J Acosta

Shane B Allcroft

Claire Nicolas

Bayardo E Lacayo

Nicholas S Card

Maitreyee Wairagkar

Alisa D Levin

David M Brandman

Sergey D Stavisky

Francis R Willett

Ziv M Williams

John D Simeral

Leigh R Hochberg

Daniel B Rubin

Abstract

Main

Results

Neural representation of attempted typing-like finger movements

Fig. 1. Neural tuning for contralateral and ipsilateral finger movements is present in precentral gyrus.

Extended Data Fig. 1. Number of electrodes tuned to 30-way finger movements by participant and array.

Extended Data Fig. 2. T17 Peristimulus time histograms during 30-way finger movements.

Extended Data Fig. 3. T17 Electrode tuning to 30-way finger movements.

Extended Data Fig. 4. T18 PC1 of trial-averaged spiking features during 30-way finger movements.

Extended Data Fig. 5. T17 and T18 Electrode tuning to medial vs. lateral finger movements on each hand.

Single-trial decoding performance

Fig. 2. Thirty distinct finger movements can be decoded from the precentral gyrus.

Extended Data Fig. 6. Gaussian Naïve Bayes decoding of 30-way finger movements from individual hemispheres.

The iBCI bimanual QWERTY keyboard

Fig. 3. The iBCI bimanual QWERTY keyboard affords high-throughput communication.

Extended Data Fig. 7. T17 and T18 typing speed did not affect accuracy.

Extended Data Fig. 8. T17 and T18 typing latency for the first character in a sentence.

Fig. 4. Word-level decoding is robust to QWERTY error patterns and the typing neuroprosthesis can be rapidly calibrated.

Discussion

Extended Data Fig. 9. T18 individual array and individual hemisphere RNN decoding.

Methods

Clinical trial

Participant T17

Participant T18

Participant T5

Neural signal processing

Isolated finger movements

PCA trajectories

Peristimulus time histograms

FVAF

t-SNE

Gaussian Naive Bayes

RNN

RNN training

Weighted finite state transducer language model

Character and word error rate definitions

Self-paced typing

Free typing

Sentence evaluation

Handwriting versus typing substitution errors

Average keystroke output latency

Effect of electrode count on performance

Reporting summary

Online content

Supplementary information

Source data

Acknowledgements

Extended data

Author contributions

Peer review

Peer review information

Data availability

Code availability

Competing interests

Footnotes

Extended data

Supplementary information

References

Associated Data

Supplementary Materials

Data Availability Statement

ACTIONS