A benchtop system to assess the feasibility of a fully independent and implantable brain-machine interface

Abstract

Objective.

State-of-the-art invasive brain-machine interfaces (BMIs) have shown significant promise, but rely on external electronics and wired connections between the brain and these external components. This configuration presents health risks and limits practical use. These limitations can be addressed by designing a fully implantable BMI similar to existing FDA-approved implantable devices. Here, a prototype BMI system whose size and power consumption are comparable to those of fully implantable medical devices was designed and implemented, and its performance was tested at the benchtop and bedside.

Approach.

A prototype of a fully implantable BMI system was designed and implemented as a miniaturized embedded system. This benchtop analogue was tested in its ability to acquire signals, train a decoder, perform online decoding, wirelessly control external devices, and operate independently on battery. Furthermore, performance metrics such as power consumption were benchmarked.

Main results.

An analogue of a fully implantable BMI was fabricated with a miniaturized form factor. A patient undergoing epilepsy surgery evaluation with an electrocorticogram (ECoG) grid implanted over the primary motor cortex was recruited to operate the system. Seven online runs were performed with an average binary state decoding accuracy of 87.0% (lag optimized, or 85.0% at fixed latency). The system was powered by a wirelessly rechargeable battery, consumed ~150 mW, and operated for >60 h on a single battery cycle.

Significance.

The BMI analogue achieved immediate and accurate decoding of ECoG signals underlying hand movements. A wirelessly rechargeable battery and other supporting functions allowed the system to function independently. In addition to the small footprint and acceptable power and heat dissipation, these results suggest that fully implantable BMI systems are feasible.

1. Introduction

Invasive brain-machine interfaces (BMIs) that are based on intracortical microelectrode arrays (MEAs) or subdural electrocortigram (ECoG) have shown significant promise in restoring motor function in those with severe paralysis due to neurological injuries. For example, such invasive BMIs have enabled accurate brain-control of multi-degree-of-freedom (DOF) robotic arms [1–6] or functional electrical stimulation (FES) devices [7, 8] after stroke or spinal cord injury. However, these systems typically require externally powered amplifier systems and full-size computers to acquire and analyze the signals. As a result, the current generation of implantable BMI systems are typically limited to stationary applications within an indoor or lab setting. Furthermore, the intracortical microelectrode arrays have skull protruding components which can potentially act as a conduit for infection, and may not be viewed by most people and potential users as aesthetically pleasing or socially acceptable. These factors may make it difficult for such systems to find safe and practical clinical application outside of the lab and may lead to poor adoption within the community.

Some researchers proposed to address these problems by wirelessly transmitting the brain signals to a computer for analysis and processing [9–13]. However, such a solution has several drawbacks. For example, the continuous wireless transmission of high-bandwidth neural data will most likely incur high power consumption, leading to faster battery depletion and increased thermal injury risk. In addition, chronic exposure of the brain and skull to wireless signals has unknown effects and may potentially carry unwanted long term risks [14, 15]. In an attempt to avoid this issue, some researchers proposed to shift the site of wireless transmission away from the head to other areas of the body [16–18]. However, both of these approaches still rely on an external computer to decode brain signals, which in turn limits system operation within the wireless range of the base station.

The authors propose that a fully independent and implantable BMI, whereby brain signal acquisition and analysis are all performed within an invasively implanted system, may be one way to overcome the aforementioned problems and achieve clinical practicality (see figure 1). Such a system would avoid the need to have skull protruding components or reliance on external computing platforms. Consequently, such a system would be conspicuous and therefore aesthetically and socially acceptable to potential users, family, and other onlookers. Furthermore, a fully implantable BMI carries the convenience of being highly mobile and always available to the user. Such a system is required to be implemented in a manner where the hardware power and heat dissipation are low and its footprint is small enough to be safely implanted in a person. In addition, the system must be able to accurately perform real-time brain signal decoding. Finally, support mechanisms necessary for operation while isolated inside the human body would also need to be included, such as battery-based power management, and wireless communication to external base station and end-effectors. To the best of the author’s knowledge, such a fully implantable BMI system has not been achieved.

Figure 1.

Envisioned fully implantable BMI system. This illustrates a hypothetical scenario where the system is implanted in a patient with cervical spinal cord injury. ECoG electrodes are implanted over the area of interest and signals are amplified, multiplexed, and digitized by the skull unit (SU) and passed to the chest wall unit (CWU) for processing. The CWU will decode the ECoG signals and send wireless commands to the end-effector. A base station can be used for wireless set up and configuration of the system, including processes such as training data acquisition, offline training and calibration of the BMI decoder.

1.1. Concept design of a fully implantable BMI

A fully implantable BMI system is envisioned to independently perform signal acquisition, offline and online data analysis, and control end-effectors. The authors envision such a fully implantable BMI to be ECoG-based due to ECoG’s proven long-term stability and safety [19, 20]. Such a BMI would consist of two major subsystems: a skull unit (SU) and a chest-wall unit (CWU) (see figure 1). First, The SU houses the amplifier array and multiplexor (MUX), is envisioned to be embedded in the user’s skull while connected to ECoG electrodes, and will be responsible for acquiring, amplifying, serializing and digitizing the ECoG signals. The multiplexed and digitized signals will be sent through a subcutaneous tunneling cable to the chest wall unit (CWU), similar the approach used in deep brain stimulators. Note that signal multiplexing minimizes the number of wires within this cable. The CWU houses all of the computing components and will be responsible for storing and analyzing ECoG signals in real time as well as controlling end-effector systems and communicating with an external base station or end-effectors. The external base station system is used to wirelessly configure the entire system as needed. Note that constraining the SU to only perform signal acquisition reduces its overall size and power consumption, thereby minimizing the risk of thermal injury to the brain as well as the potential invasiveness of the prospective implant in the head. Also note that placing the wireless transceiver (TRX) within the CWU spares the brain and skull from potential long term exposure to radio frequency signals.

This paper describes the design of a prototype based on this envisioned BMI system that is capable of decoding brain signals without sacrificing performance and while occupying a footprint sufficiently small to allow for safe implantation. The BMI functions were validated at the bedside by testing its ability to accurately decode human ECoG signals underlying hand movement in real time while the support functions were benchmarked with a variety of benchtop tests.

2. Methods

2.1. Overview

To test the feasibility of the concept of a fully implantable BMI, a benchtop prototype of the system envisioned in figure 1 was designed and fabricated (see figure 2). Similar to the envisioned system above, this prototype comprises an analogue of the SU and CWU. A development board was also designed and fabricated to dock and connect these two subsystems as well as facilitate programming and testing of the integrated system. The system was programmed to perform the necessary BMI and supporting functions (described in section 2.3). Finally, the entire system was tested at the bedside in its ability to perform online decoding of ECoG signals and its supporting functions were benchmarked.

Figure 2.

Block diagram of the implantable BMI analogue. Dashed lines indicate wireless connections. Similar to the envisioned fully implantable BMI system, the SU analogue acquires ECoG signals and passes them to the CWU analogue. The CWU, composed of 2 microcontroller (MCU) cores and supporting components, performs all necessary processing of the ECoG signals. Decoded states are wirelessly sent to an end-effector. The base station is used to wirelessly configure the implantable BMI analogue. The BMI system is powered by a wirelessly rechargeable battery. The development board connects all components together and enables initial programming of the BMI via a USB port. A programmer (which can either be a desktop computer or a dedicated programming device), is used to deploy the initial image of the BMI program onto the MCU cores.

2.2. Hardware design

2.2.1. Skull unit analogue.

The SU analogue comprises a 32-channel commercial bioamplifier integrated circuit (IC) with integrated MUX and 16-bit analog-to-digital converter (ADC) (Intan Technology, Santa Monica, CA). The amplifier IC was mounted on a custom printed circuit board (PCB) which connected the amplifier IC’s serial peripheral interface (SPI) to the CWU analogue and its input channels to ECoG electrodes via an interface on the development board.

2.2.2. Chest wall unit analogue.

To facilitate on-board processing, the CWU analogue was designed as a special purpose miniaturized computer on a custom PCB. The CWU comprises low-power central processing units (two 48 MHz ARM Cortex-M0+ microcontrollers; Microchip, Chandler, AZ), storage modules (512-MiB NAND flash memory; Micron, Boise, ID), and memory (two 512-KiB FRAM; Cypress Semiconductor, San Jose, CA). The rationale for having two separate microcontroller cores was to divide the computing burden necessary for real-time BMI operation, as well as to provide a means for future self-programming of new software update deployments. Due to the isolated nature of a future fully implantable BMI, wireless communication between the CWU analogue and base station or end-effectors must be established and implemented in a manner that is compliant with the Federal Communications Commission designated Medical Device Radiocommunications Service (FCC MedRadio [21]) for implantable medical devices. This was facilitated by a LoRA radio TRX (HOPE Microelectronics, Xili, Shenzhen, China), set to operate at 406 MHz using on-off-keying (OOK) modulation. Also, since all implantable medical devices must be independently powered upon implantation, the system’s power was provided by a 2500 mAh rechargeable 3.7 V lithium-polymer battery. A linear voltage regulator was used to maintain the system voltage at 3.3 V. Battery level status was measured by a battery monitor IC (Texas Instruments, Dallas, TX). A wireless charging IC (Texas Instruments, Dallas, TX; compliant with Qi wireless power transfer standards v1.2) with accompanying 47 μH induction coil (Wurth Electronik, Waldenburg, Germany) enabled battery recharging by induction.

2.2.3. Development board.

In order to facilitate testing and programming of the SU and CWU analogues, a development board was designed and fabricated to provide the necessary connections and interfaces for the two subsystems. More specifically, the development board utilized a stackable board-to-board connector interface that provided breakouts for critical connections for the SU and the CWU analogues. This included 32 industry standard touch-proof jacks that were used to plug in ECoG connectors and route ECoG signals to the SU analogue. Also, USB interfaces facilitated programming and debugging the microcontrollers from a desktop computer. Note that this development board is not necessary in the future implantable system (figure 1).

2.2.4. Base station.

During the testing of the system, a desktop computer with a MedRadio band radio TRX played the role of a base station in order to wirelessly control the implantable BMI analogue. The base station’s MedRadio TRX was realized by interfacing a LoRA radio TRX with an ARM-based microcontroller. This wireless module was connected to the desktop computer by a USB cable.

2.3. Software design

2.3.1. BMI software overview.

The overarching goal of the implantable BMI analogue’s software is to perform basic online decoding of brain signals. More specifically, the system was designed to classify ECoG data into either move or idle states in real time, similar to the state decoding scheme in existing BMI systems [22, 23]. To this end, the BMI system’s main functions are to perform BMI training data collection procedures, generate an ECoG decoding model, and subsequently utilize the ECoG decoding model in online BMI operations. Outside of these main functions, the BMI software also has a number of supporting functions necessary for proper operation, such as power management and wireless communication with external devices. This BMI software was implemented as a custom C++ program, compiled on a desktop computer using Visual Studio (Microsoft Corp, Redmond WA), and deployed onto the two microcontrollers of the CWU analogue (referred to as Core 1 and Core 2) via USB connections on the development board (note that the desktop computer plays the role of the programmer in figure 2). Cores 1 and 2 were programmed to divide the computing burden in a manner necessary to perform all of these functions in real time. Once the BMI software was deployed, the BMI analogue was detached from the desktop computer and subsequently controlled wirelessly through the base station computer. To this end, the base station software (running on the base station computer) was designed to run in conjunction with the CWU software. The base station software enabled all of the functions and settings of the BMI analogue to be controlled wirelessly.

2.3.2. Base station software.

The base station software was designed to provide the experimenter with wireless access to all the settings and functions of the BMI. It was written in Visual C# and implemented with a Windows Forms graphical user interface to facilitate ease of use. The base station software also provided a means to download and visualize ECoG signals, upload data and settings, present training cues to subjects during training data collection, and set the BMI to run in online mode. Ultimately, the base station and this its software are not required to operate the CWU analogue after the system settings are appropriately configured and the BMI decoder is trained (described further below).

2.3.3. Signal acquisition and data management.

ECoG signals from 32 electrodes were acquired at 500 Hz sample rate per channel after going through a configurable 7.5–200 Hz analog band-pass filter. This rate was deemed adequate to acquire the ECoG high-γ band (80–160 Hz), which is known to be most informative of the execution of motor tasks [24]. The ECoG data was temporarily stored inside the FRAM modules during acquisition. During training data collection, the data was transferred to NAND flash for permanent storage. On the other hand, the data was decoded in real time during online BMI operation. A file system was implemented to facilitate the organization and future retrieval of all data stored on the NAND flash.

2.4. Assessment

2.4.1. BMI decoder training.

The BMI decoder utilized a supervised learning approach to distinguish between idle and move states based on ECoG signals. This required ECoG data underlying idle and move states to be collected. To this end, upon command from the base station software, the BMI initiated training data collection. It then wirelessly sent cues to the base station directing the base station software to display alternating idle and move cues to the subject. Nominally, each idle or move epoch was 5 s long, and the process was repeated for a total of 60 s. The BMI acquired ECoG data underlying each state and stored the data in the NAND flash.

The BMI software then used the collected ECoG data to train the decoder. The average band-specific powers were calculated in the α-to-β band (8–35 Hz) as well as in the high-γ band (80–160 Hz), as follows. This band combination was chosen as it was shown to yield the highest state decoding accuracy for elementary upper extremity movements [24]. The first 500 ms out of the 5 s data were discarded to accommodate human reaction delay after a cue change. The subsequent 4.5 s ECoG time series were divided into 750 ms non-overlapping segments, re-referenced to the common average reference (CAR), and passed through a bank of software biquadratic filters in the αβ and high-γ bands (filter design details and characteristics are provided in the supplementary material (stacks.iop.org/JNE/16/066043/mmedia)). The filtered signals were squared and averaged in time to calculate a scalar value representing the band power in each channel and segment, i.e. $E_{c,s}^{\alpha \beta }$ and $E_{c,s}^{\gamma }$ for channel c and segment s. These band power averages were downloaded for visualization. The channels with high-contrast between idle and move states were empirically chosen for subsequent analysis. The rationale for this channel selection is that: (1) not all ECoG electrodes placed over brain areas are relevant and demonstrate robust modulation during movement; (2) training with all 32 channels would have required much longer training time to acquire adequate number of samples for the increased dimension size; (3) a longer training also increases analysis time, which may be impractical in a hospital testing environment.

The band powers $E_{c,s}^{\alpha \beta }$ and $E_{c,s}^{\gamma }$ were concatenated and processed by classwise principal component analysis (CPCA) [25] to reduce the dimension of input data. Specifically, the algorithm was set to retain principal components accounting for at least 92% of variance in each class. The class separability was further enhanced by running linear discriminant analysis (LDA) on the CPCA-transformed data. Mathematically, f = T_LDAΦ_CPCA(d), where $d\in {\mathbb{R}}^{2c}$ are the band powers, Φ_CPCA is the piecewise linear CPCA transformation, T_LDA is the LDA transformation matrix, and $f\in \mathbb{R}$ are 1D spatio-spectral features. Subsequently, Bayes rule was used to calculate the posterior probabilities of each state given the observed feature. Due to the piecewise nature of the feature extraction transformation, the following four posteriors were calculated: P_I(I|f*), P_I(M|f*), P_M(I|f*), and P_M(M|f*), where f* is the observation, and P_I(·|·) and P_M(·|·) are the posterior probabilities calculated in the idle and move state subspaces, respectively. The subspace with the most convincing evidence (the highest posterior probability) was then selected as the winning subspace and its posteriors P(I|f*) and P(M|f*) were logged. Note that P(I|f*) = 1 − P(M|f*). This decoding methodology has been successfully tested and validated in many real-time BMI operations [22, 23, 26, 27], and was utilized in this study so that focus can be placed on design and validation of the device prototype.

The decoder training function was benchmarked to determine how the time necessary to train the decoder scales with the number of channels. To test the worst-case scenario, the decoder was trained on indistinguishable sets of training data. Specifically, the BMI analogue recorded 60 s of environmental noise (all channels unplugged) and treated the data as if they were alternating idle and move epochs. Note that this 60 s length matched the duration of training data. The time required to complete the decoder training process was measured for 4, 8, 16, 24, and 32 selected channels.

2.4.2. Online BMI operation.

During online operation $E_{c,s}^{\alpha \beta }$ and $E_{c,s}^{\gamma }$ were calculated for non-overlapping 750 ms ECoG signal windows across all the selected channels. They were subsequently processed as described in section 2.4.1 to determine the winning posterior probability P(M|f*). To avoid uncertain state transitions and minimize the subject’s mental workload, a binary state machine approach was used. Specifically, if T_I < P(M|f*) < T_M, the system remained in the current state, where T_I and T_M (T_M > T_I) are appropriately chosen state transition thresholds. If P(M|f*) ⩾ T_M, the system remained in the move state or transitioned from the idle to move state, as the case may be. On the other hand, if P(M|f*) ⩽ T_I, the system either remained in the idle state or transitioned from the move to idle state. Note that this approach has been successfully used in prior high-performance BMIs [22, 23, 26–28]. The BMI analogue system utilized a short dedicated time window (nominally ~50 ms long) to wirelessly transmit the decoded state and the corresponding P(M|f*) for the most recent 750 ms window to the base station software and end-effector. Note that the base station acted as the end-effector in this study, since the feedback was provided as an on-screen icon that indicated the decoded state.

Prior to online operation and performance assessment, a calibration procedure was used to determine the values of T_I and T_M. The BMI system was set to run in the online mode but with no control of end-effectors. More specifically, the BMI system sent cues to the base station directing the base station software to display alternating idle and move instructions to the subject. Each idle and move epoch was 10 s long, and the calibration ran for 1–2 min. The resulting P(M|f*) were logged by the base station software. Subsequently, P(M|f*) during idle and move cues were plotted on a histogram, and the experimenter empirically determined suitable values for T_I and T_M.

2.4.3. Online BMI performance assessment.

Ethical approval for testing of the implantable BMI analogue was obtained from the Institutional Review Boards (IRB) of the University of California, Irvine and the Rancho Los Amigos National Rehabilitation Center (RLANRC). Subjects undergoing epilepsy surgery evaluation with ECOG electrodes implanted over the hand primary motor cortex (M1) were recruited. Subjects were asked to perform alternating epochs of repetitive hand grasp/release on the side contralateral to the ECoG electrodes as directed by the computerized cues displayed on screen by the base station software. The decoding model was generated and calibrated using methods described in sections 2.4.1 and 2.4.2. Once the values of T_I and T_M were found, the subject operated the BMI in the online mode. Subjects were asked to follow visual cues to alternate between idling and repetitive hand grasp/release while the BMI decoded their ECoG signals in real time (section 2.4.2). The decoded state was also displayed to the subjects as a form of feedback. This was repeated for ~five 8.5 s-long alternations of idle/move (nominally 85 s for each online run). Multiple runs were performed as tolerated by the subjects or as time permitted. Since analgesic medications with significant sedating effects are often given to this patient population, experimental compliance may be suboptimal. Therefore, a video camera recorded subjects’ physical movements as the ground truth (acquired at 30 Hz, and synchronized with the development board). The BMI performance was assessed by comparing the decoded states to the ground truth and calculating the rate of correctly decoded states. In instances when the subject’s hand was occluded at that time instance, the online accuracy calculation was not performed since the ground truth could not be established for that time point. In addition, due the causal nature of the decoder, the online BMI accuracy were also lag optimized. Namely, the above decoding accuracy calculations were repeated by delaying the ground truth state by up to 2 s, and the set with the highest overall accuracy was reported for each run. This lag optimization helps to account for various factors such as delay between ECoG signal modulation and actual movements, uncertainties in determining exact start and stop times of movement, as well as any potential processing delays. On the other hand, since the system nominally updates every 800 ms, the above performance measures were also calculated at a fixed latency of 800 ms.

2.4.4. Power management and other benchmarks.

The BMI analogue was benchmarked to determine the power consumption during its various functions, including standby (running no operations other than periodically transmitting its status to the base station), training data collection, decoder training, online decoding, and wireless data transfer (continuous wireless data transmission to base station). Power consumption was derived from the average current measurements on an ammeter connected in series with the battery while the BMI analogue performed each of the above functions over a 2 min period (nominal battery voltage was assumed to be 3.7 V for this calculation). All 32 channels were active during these benchmark tests.

For any implantable medical device, it is critical for battery state to be monitored so as to know when it is necessary to recharge the system or when the battery is nearing failure. The CWU software was designed to monitor the battery status for voltage and remaining capacity. The microcontroller cores obtained this data from the battery monitor IC on the CWU analogue, and then wirelessly transmitted the data to the base station. To characterize the system’s typical battery life, the battery was first charged to full capacity (defined as when no current entered the battery) by means of a wireless power emitter (IDT Technologies, using Qi protocol). Then, the system was set to run continuously in the online BMI mode as the battery voltage was self-monitored by the CWU, until the system shut down due to battery depletion. The same procedure was repeated when the system was in standby mode. Then, to characterize the typical recharge profile, the wireless emitter was used to recharge the depleted battery until full while the CWU self-monitored the battery voltage.

The wireless file transfer function was benchmarked for speed. The wireless transfer speed was determined by measuring the time required to successfully send a training data file (which consisted of ECoG data underlying a single idle or move epoch) to the base station (includes time required for any error correction). This was performed for all data files from the training run above (n = 12 files) and the entire process was repeated for a total of three times (total of 36 file transfers).

3. Results

3.1. Hardware and software

The SU and CWU analogues were designed and fabricated as multilayer PCBs. Figure 3 shows them mounted on a 100 mm × 100 mm development board. The SU and CWU analogue PCBs occupy a 17 mm × 19 mm and 33 mm × 33 mm footprint areas, respectively. The BMI software was successfully deployed onto the CWU analogue MCU cores via USB programming ports. Once the BMI software was uploaded, the system was detached from the computer and operated independently with power from a rechargeable lithium-polymer battery. The base station software was successfully compiled as a standalone Windows executable program with a graphical user interface. The base station software subsequently performed all communication with the BMI analogue via the USB-controlled LoRA dongle over a 406 MHz band wireless link.

Figure 3.

SU and CWU analogues mounted on the development board. CWU analogue comprises the CWU board, antenna, charging coil, and battery. SU analogue comprises the amplifier/ADC chip. ECoG electrodes are plugged into standard touchproof jacks and the USB port enables initial programming. Note that a foot print for a micro-SD card adapter was included in the lower left hand corner, but the adapter was not used in this study.

3.2. Online BMI performance

A patient (40 year old, male) undergoing epilepsy surgery evaluation with ECoG electrode implantation over the left M1 area (see figure 4) was recruited and provided his informed consent to participate in this study. The study was performed after all seizures necessary to identify the seizure foci for epilepsy surgery were captured and all anti-epileptic medications were restarted. During this time, the subject was disconnected from the hospital monitoring system while awaiting explantation of ECoG electrodes. The subject remained seizure free during the entire study time. The grid locations were identified by MRI-CT image fusion and electrode clustering using methods described in [29]. Figure 4 shows the axial top view of the electrodes connected to the CWU analogue. Note that electrode placement was dictated by clinical needs.

Figure 4.

Subject’s ECoG grid placements. The left central convexity grid was connected to the CWU analogue. Electrodes containing salient features of the hand grasp movement are circled in red. In addition, two electrodes on the right interhemispheric (IH) grid were used as signal reference and circuit ground.

During the training phase, the subject performed a repetitive right hand grasp/release task as per section 2.4.1. An example of the ECoG signals acquired can be seen in figure 5. The signals were visually inspected by the experimenter. A total of seven salient channels were empirically selected as they were deemed to have the highest feature contrast between idle and move states (circled in red on figure 4). Subsequently, these channels were used for decoding model generation and calibration. Due to the subject’s fatigue and somnolence from medications, the calibration and online testing was done 2 d after the training data acquisition and decoding database generation. Since P(M|f*) during idle and move from the calibration run were highly separated, T_I and T_M were set to 5% and 95%, respectively. The subject was able to perform a total of 8 online runs. The video files were reviewed to determine the ground truth. The percentage of correctly decoded states (P_Correct) across seven runs averaged 87.0% in the lag optimized approach, and 85.0% in the fixed latency approach. The performance metrics for all runs are shown in table 1. A representative example of the decoding timeline from an online run is shown in figure 6. Run #2 was discarded from analysis due to camera obstruction, preventing the recovery of the ground truth. For Runs #1 and #6, the camera’s view of the subject’s hand was obstructed temporarily, leading to three and one time decoded states being excluded from the online performance calculation, respectively. Since the BMI was manually terminated at the end of each online run, run durations were not exactly identical. In Run #1, the BMI experiment was discontinued earlier than expected while in Runs #5 and #6, the BMI system was allowed to operate for longer than the typical run duration.

Figure 5.

Representative signals from electrode #50 on the left central grid (figure 4) during training data collection. The training run began with a 5 s idle epoch and alternated with 5 s move epochs where the subject performed repetitive hand grasp/release throughout the epoch. White/green shades denote idle/move epochs, as defined by system generated cues during the training data collection process.

Table 1.

Online performance from all runs. P_Correct is the overall decoding accuracy. P(I|I) is defined as the percentage of correctly decoded idle states and P(M|M) is defined as the percentage of correctly decoded move states. Time indicates how long each experimental run lasted. Performances are reported at optimized latencies (lags) and at the 800 ms nominal update rate.

		Lag optimized				Lag fixed to 800 ms
Run	Time (s)	PCorrect (%)	P(I|I) (%)	P(M|M) (%)	Lag (ms)	PCorrect (%)	P(I|I) (%)	P(M|M) (%)
1	75.7	85.5	92.6	72.1	869	83.9	92.0	78.4
3	86.1	81.7	87.1	74.4	636	80.0	83.9	76.9
4	88.0	84.7	93.8	66.7	896	83.3	93.5	74.3
5	104.3	82.6	100.0	64.7	430	77.9	94.7	64.6
6	124.2	89.6	91.2	87.2	825	91.4	91.2	91.5
7	86.6	90.5	90.9	88.1	451	84.7	80.0	89.2
8	87.4	93.1	100.0	84.1	397	89.9	90.6	89.2
Average	93.2	87.0%	93.6%	77.6%	655	85.0%	89.5%	81.3%

Figure 6.

Timeline of online Run #8. The BMI decoded the ECoG signals into idle or move state. Black trace: posterior probabilities P(M|f*). Dashed lines: state machine transition thresholds, T_I and T_M. Decoded: BMI states from the decoder. Red = Idle. Green = Move. Actual: ground truth states based on video analysis. Gaps between red and green lines represent uncertainties due to the subject momentarily placing his arm outside of the camera’s field of view. Note that since these gaps did not coincide with the time point of any BMI analysis window, no decoded windows were excluded from the decoding accuracy analysis. Cue: instructional visual cue shown to the subject.

3.3. Computing and power management benchmarks

A number of a computing benchmarks were measured during the process of testing the BMI system. The time that it took for the CWU analogue to calculate P(M|f*) for any 750 ms window was ~300 ms. A more detailed breakdown of the online benchmark by processes and Cores is provided in the supplementary data. For the decoder training benchmark, it was found that the time to complete the decoder training was directly proportional to number of channels (R² = 0.964, see figure A1). The wireless transfer speed averaged 975.5 ± 31.0 bytes s⁻¹ across 36 file transfers. Anecdotally, it was observed that interference to line-of-sight between the BMI and the base station could lead to reduction in the transfer speed.

Power consumption of the BMI analogue as it was performing various functions is shown in table 2. When set to run in the online decoding mode, the BMI could operate continuously for over 60 h on a single battery charge (figure 7). The system was able to reliably operate over 3.2–4.2V supply-voltage range. It required ~18 h to recharge the battery from depletion to full capacity (see figure A2).

Table 2.

Average current and power consumption of the BMI analogue during various BMI functions. Power consumption is derived using 3.7 V as the nominal battery voltage.

Task	Current (mA)	Power (mW)
Standby	37.5	138.8
Training data collection	42.1	155.8
Decoder training	41.4	153.2
Online decoding operation	40.6	150.2
Wireless file transfer	43.8	162.1

Figure 7.

Battery discharge plots during standby and online decoding operation. The ADC ceased functioning below 3.2 V, therefore stopping the online decoding. Files can still be wirelessly transferred until the NAND cutoff voltage. The battery circuit cut off all power around 2.5 V.

4. Discussion

4.1. Primary findings

In this study, an benchtop analogue of a fully independent and implantable BMI system was successfully implemented, tested on the benchtop and at bedside, and achieved a high level of online performance. This represents an important step in demonstrating that fully implantable BMI systems are feasible and may overcome the issues that prevent clinical translation of current-generation invasive BMI systems. The author’s prior work in [30] demonstrated that BMIs could be reduced into hand held size (13 × 9 ×3 cm³), but the system only had the capability of operating on 4 EEG channels and was not designed for implantability due to excessive size and power consumption. The current work advances this significantly further by achieving a miniaturized form factor with a size comparable to existing implantable medical devices, e.g. deep brain stimulators or responsive neural stimulators, while also being mindful of power and heat dissipation. This implies that a future implantable BMI can include all necessary components of a miniaturized form factor, thereby eliminating the need for any skull-protruding electronics, bulky external amplifiers, or computers. The ability to execute all data analysis steps and online decoding eliminates the need for fully implantable BMIs that rely on power-hungry wireless tethering to an external system for signal processing.

This miniaturization was achieved without sacrificing performance as the system’s online decoding accuracy is comparable to or surpasses that of conventional desktop-based binary classifiers [22, 30, 31]. Notably, the acquired signals demonstrate the expected ECoG features underlying motor behavior, including α and β band desynchronization [32], and high-γ synchronization [33–36]. Furthermore, accurate online decoding was achieved with only a short duration of training data and without any significant user practice. Although some brief false transitions were observed (see figure 6), such noisy state transitions may be minimized in the future by averaging P(M|f*) across multiple decoding windows, albeit with slower response. The subject’s performance also increased with each successive online run, indicating that if time permitted, additional user training would have likely resulted in even higher online performance. Finally, the fully implantable BMI analogue’s ability to operate independently on battery power, be recharged wirelessly, and wirelessly communicate with the base station and end-effectors indicates that a future BMI device can be successfully implanted and isolated inside of a human body where no direct access is possible. The features that make the BMI analogue system potentially translatable into a practical fully implantable device will be discussed in further detail.

The current BMI analogue system can act as a basis for future implantable BMI systems. More specifically, the current system can readily undergo additional translational steps to make it into a fully implantable form that complies with regulatory requirements. First, the SU analogue can be integrated with ECoG electrodes, and both analogues can be placed within hermetically-sealed titanium enclosures. Given that they have a size comparable to modern-day implants and given the known biocompatibility of titanium, these subsystems can be readily implanted in humans. The tunneling cable that connects the SU and CWU involves a surgical process similar to what is employed in deep-brain stimulator implantation and hence should not pose any additional problems. The foot print of these subsystems can potentially be reduced even further by utilizing smaller versions of each discrete component, e.g. bare dies (as in [37]) or even integrating them into an ASIC form. Second, although the base station is a desktop computer in the current study, this can eventually be replaced by a handheld device to facilitate better mobility and ease-of-use for future users and clinicians. It should also be noted that the base station does not need to be constantly present once all the BMI settings are configured. Third, while the analogue system’s 406 MHz band radio already complies with the FCC MedRadio standard, additional protocols for co-existance with other implantable wireless devices (as in [38, 39]) and cybersecurity (e.g. encryption/authentication and device specific addressing) can be implemented in the future to satisfy FDA requirements. Finally, upon integration into titanium enclosures, it can be estimated that the system’s power consumption of ~150 mW will fall within an acceptable and safe range of heat dissipation within the human body [40].

The BMI analogue system’s battery can provide over 2 d of constant online BMI use before requiring a recharge via wireless induction. Although it nominally requires up to 18 hrs to completely recharge the battery (figure A2), it is reasonable to charge the battery for ~6–8 hrs every 48 hrs to maintain continuous operation. More specifically, it can be seen in figure 7 that 48 hrs of operation will discharge the battery to between 3.7–3.8 V. Subsequently, 8 hrs of charging from this voltage level will bring the battery back up to >4.15 V (figure A2). A future scenario can be envisioned where the BMI user undergoes wireless charging as they sleep (typically 6–8 hrs for average adult). With typical batteries having as many as 1000 recharge cycles, the current design could last up to 6–7 years before requiring replacement. The implementation of additional power saving features in software can minimize power consumption. Namely, this includes shutting down components or placing them into sleep mode when not in use to further reduce power consumption. A custom ASIC implementation can also help reduce power consumption by minimizing the number of redundant or unnecessary subcomponents from the ICs currently used in the system. Finally, other process improvements such as implementing direct memory access (DMA) or even simplifying the decoding algorithm can also help optimize power consumption. Such measures are important to minimize the need for battery replacement surgeries and their associated costs and complication risk. Power consumption optimization can also help to ultimately reduce the battery and implant size.

4.2. Limitations

The process of generating a decoding model lengthens by ~0.7 min with an each additional channels involved, and can take excessively long if a large number of channels are required in the future. This may lead to faster battery depletion, especially if more training data is collected. However, the decoding model may not need to be generated frequently. More specifically, it is notable that the training data was collected on a separate day from when the online runs were performed, yet it was still able to yield a high accuracy online performance. Although not explicitly tested here, this suggests that ECoG signals may be stable enough such that a decoding model can be used over extended periods of time.

The fully implantable BMI analogue system has only been programmed to perform binary-state classification. Although this is just a single decoding scheme, it can be readily applied to high-impact clinical applications. This includes BMIs to control gait prostheses in paraplegic SCI patients [26], or for virtual keyboard control in patients with locked-in syndrome [41], or for control of prostheses for hand grasp and release for stroke patients with chronic distal upper extremity weakness [6]. Furthermore, the BMI software can be altered to decode more classes or perform other decoding schemes, such as estimating upper or lower extremity movement trajectories from ECoG signals [33, 42–44]. As an implant, any such change or update to the BMI software would require the ability to perform wireless reprogramming. Although this feature was not explicitly demonstrated here, the fully implantable BMI analogue hardware has the capacity to accommodate this feature. Specifically, both microcontroller cores can be programmed with a function that would allow them to act as an in-system programmer (ISP) for each other. Subsequently, any new compiled image for the microcontroller cores can be wirelessly transmitted to the CWU and stored in the NAND flash storage. The cores would take turns deploying the new image onto one another to complete the update. It should be noted that adding more classes may require more training data collection and involve longer decoder training. Although not explicitly benchmarked, it is expected that faster processors may be required to maintain real time processing beyond a certain number of classes. Similarly, the addition of another memory module can help enable the implementation of a sliding window during decoding for faster response time.

Although an extended stability test was not explicitly performed in this study, it can be seen that the system and its software did not ‘crash’ for the entire duration of the battery depletion test (figure 7).

4.3. Future directions

Future work will involve the translation into a fully implantable form. In addition to integration of all electronic components into hermetically sealed enclosures and software improvements discussed above, the system will be properly sterilized and packaged followed by industrial standard testing (i.e. helium leak testing, sterility validation, animal model testing, etc) to demonstrate safety and satisfy regulatory requirements prior to human implantation. Also, given that sensation is critical for all motor functions, electrical stimulators will be added in future iterations to deliver direct cortical stimulation to the sensory cortex for artificial sensory feedback. Artificial sensation can potentially help further improve the user’s performance at operating the system and make the system more compelling to potential users and clinicians.

5. Conclusion

An analogue of a fully implantable BMI system with a small form factor was successfully designed, fabricated and tested both at the benchtop and at the bedside. This system is capable of performing all BMI functions without reliance on an external computer. The system can operate independently within an isolated environment given a wirelessly rechargeable battery and wireless communication capability with a base station and end-effectors. Despite these design constraints, decoding performance was not sacrificed and the system’s power and heat dissipation were within safe and acceptable ranges. These findings suggest that the fully implantable and independent BMI systems are feasible. Future work will include translation of the analogue device into an implantable form and adding features such as electrical stimulation capability for artificial sensation.