High-density neural recording system design

Abstract

Implantable medical devices capable of monitoring hundreds to thousands of electrodes have received great attention in biomedical applications for understanding of the brain function and to treat brain diseases such as epilepsy, dystonia, and Parkinson’s disease. Non-invasive neural recording modalities such as fMRI and EEGs were widely used since the 1960s, but to acquire better information, invasive modalities gained popularity. Since such invasive neural recording system requires high efficiency and low power operation, they have been implemented as integrated circuits. Many techniques have been developed and applied when designing integrated high-density neural recording architecture for better performance, higher efficiency, and lower power consumption. This paper covers general knowledge of neural signals and frequently used neural recording architectures for monitoring neural activity. For neural recording architecture, various neural recording amplifier structures are covered. In addition, several neural processing techniques, which can optimize the neural recording system, are also discussed.

Introduction

Implantable medical devices (IMDs) such as pacemaker and cochlear implants are widely used since 1960 to treat diseases such as irregular heart rhythm and hearing loss, respectively. Next-generation IMDs also aim to treat various brain diseases such as epilepsy [1], Parkinson’s disease [2], and dystonia [3] or to establish a brain-computer interface. To accomplish such targets, it is necessary to know the exact function of the brain. For this purpose, recent neural recording technologies have advanced through multi-disciplinary works.

To study dynamics and connectivity of the brain, various methods from invasive microelectrodes array to non-invasive functional magnetic resonance imaging (fMRI) are widely used. Non-invasive neural recording technology such as fMRI or electroencephalogram (EEG) has been widely used for both animal and human subjects. Since the non-invasive methods do not require clinical surgery, it has been popular than the invasive method when performing brain experiments. However, the major drawback of the non-invasive method is its limitation in temporal and spatial resolution.

Figure 1a shows various methods for studying the temporal and spatial coverage of the brain with an estimated number of neurons [4–6]. The non-invasive neural recording modalities such as fMRI can cover the whole-brain level spatial range. However, its temporal resolution is very limited to detect neural activity in real-time because fMRI detects the blood concentration of the brain based on blood-oxygen-level-dependent (BOLD) signal [7]. Furthermore, fMRI requires a large and expansive medical device, so it cannot be utilized as wearable devices. Optical imaging such as calcium imaging can offer large-scale neural network activity in better resolution than fMRI, but its temporal resolution is still inferior to catch the single neuron level activity. It also requires stationary bulky hardware. The EEG signal, which is the electrical activity from scalp skin recorded with surface electrodes, has been widely used because of ease of experiment and safety. However, its temporal and spatial resolutions are severely impaired as the neural signals travel through the scalp.

Fig. 1.

Spatial and temporal resolution of various brain monitoring modalities [4–6]

Invasive neural recording methods such as patch-clamp and penetrating microelectrodes require clinical surgery for implantation and suffer from risk of cell damage during insertion [8–10]. However, compared to non-invasive neural recording methods, its temporal and spatial resolution is far superior to catch the single neuron cell activity. The patch clamping can provide very high temporal and spatial resolution of neural activity, but by its monitoring nature, which is penetrating neuron cell membrane, only a few days of monitoring can be available before cells are deactivated. The depth electrodes such as a multi-electrodes array (MEA) or silicon probes can provide high spatial and temporal resolution compared to non-invasive methods without cell death, so many researchers have studied brain activity with using depth electrodes [11–25]. The depth electrode, however, poses a serious challenge in long-term chronic usage because of immune responses against electrodes [26]. Recently, electrocorticography (ECoG) has gained popularity as an alternative solution between non-invasive and invasive methods. Surface electrodes are placed on brain dura (epidural ECoG) or arachnoid (subdural ECoG) for signal measurements without penetrating the brain cortex. Since it is placed under the skull, signal quality is still superior compared to EEG [6].

In the past, measured neural signals from electrodes such as patch-clamp or depth electrodes were processed with a wall-mounted hardware system that was heavy and bulky. Thus, experimental subjects were mainly cell level subjects like single-cell neurons or anesthetized animals. However, information from those experiments was limited to study neural activities on large-scale networks or living task-performing animals. To address these issues, advanced hardware systems with integrated circuits have been developed to implement a portable system that does not hinder animal movements. A ground-breaking integrated circuit for neural recording was introduced by Harrison et al. [11], and many energy-efficient and high-performance systems were invented for high-resolution neural interface [27–42].

While previous multi-disciplinary works were done with a few integrated neural recording channels, now recent studies aim to research large-scale neural activity with a large number of recording channels [43–57]. Since neuron does not act independently, studying inter-relationship between neurons is critical for revealing brain functions. Figure 2a shows the yearly research trend of simultaneously recorded neurons with using electrophysiological modalities (black squares) or imaging modalities (red triangles) [58]. It shows that the number of simultaneously recorded neurons is rising exponentially, while recent advances of imaging techniques grant 10–100 times more monitored neurons. Recent study [59] also showed that about 10⁶ of neurons can be simultaneously recorded in a non-invasive way. However, even with non-invasive neural recording, there are still five order of magnitude difference compared with neurons of human brain. Even worse, the number of neural recording channels that is implemented with integrated circuits is quite limited compared to non-invasive methods.

Fig. 2.

Yearly trend of a simultaneously recorded neurons and b a number of channels of the neural recording system implemented in integrated circuits

Figure 2b shows the yearly research trend of the implemented neural recording systems with active probes (blue triangles) or passive probes (red squares). It can be observed that the number of neural recording channel implemented in integrated circuits was slowly rising from single channel to thousands of channels [59–74, 75]. The state-of-the-art implantable neural recording system was proposed by Neuralink [66], but the number of neural recording channels is still limited to 1024. Even considering in-vitro silicon probe implementation [67], still 19,584 electrodes and recording channels were implemented, which is smaller than imaging methods. From this, the need for high-density neural recording system that can monitor thousands of channels in real-time in fully implantable device is still highly required.

This paper provides a general review of the high-density neural recording system and its requirements. To understand the specifications of the neural recording system, general knowledge about neural signals is given in Sect. 2. Then, the overall neural recording system structure and its figure-of-merit are explained in Sect. 3. In Sect. 4, conventional neural processing circuits with state-of-the-art processing algorithms are presented, and Sect. 5 concludes this article.

Neural signal

The action potential (AP) signal generated by the ion movement at a neuron cell is the basis of the neural signal. In the resting state, transmembrane proteins in the neuronal membrane keep the concentration of potassium (K⁺) and chloride (Cl⁻) ions inside the cell high relative to outside the cell, while the concentration of sodium (Na⁺) ions low [76]. In this state, diffusion force and electrical force of individual ions reach an equilibrium state, and the membrane potential can be calculated by Goldman–Hodgkin–Katz equation as −70 mV. When excitatory synaptic input is delivered to the neuron, the depolarization of neuron membrane leads to influx of Na⁺ and outflux of K⁺. This ion movement causes sharp transient potential differences and can be measured with electrodes such as patch-clamp or depth electrodes.

Measuring this transient voltage potential can be performed in two ways: Outside the cell membrane or inside the cell membrane. Measurement of the potential inside the cell with a sharp electrode, which penetrates the neuron cell membrane, is referred to as intracellular recording, and measurement of the potential outside the cell is referred to as extracellular recording. Intracellular AP recording typically has voltage amplitude of 10–70 mV_pp and a frequency range of 100 Hz–10 kHz. However, penetrating cell membrane results in cell death in several hours, so most of in-vivo experiments are done in an extracellular way. Extracellular AP (also known as neural spike) recording provides a relatively safe recording environment, but its measured signal is significantly degraded compared to intracellular AP due to filtering properties through the extracellular medium and cell membrane. Its voltage amplitude falls about two to three orders of magnitude compared to the intracellular AP to be 50–500 uV_pp, and its frequency response becomes slightly high-passed filtered.

A single-neuron level recording with a close-spaced electrode can measure neural spikes more accurately. However, if multiple neurons are present near the electrode, the low-frequency component of each neural spike is measured as a slow drift like a transient signal because cell fluid between neuron and cell membrane acts as a low-pass filter. This signal is called the local field potential (LFP) and also can be used in various applications. LFP has a typical value of 0.5–5 mV_PP and a frequency band of 1 mHz–200 Hz. Figure 3 shows the waveforms of the typical neural signal and its filtered signal. It can be observed that the spike-vague raw neural signals can be divided into fast-moving neural spikes and slow-moving LFP with proper frequency filtering.

Fig. 3.

Recorded waveforms of a typical raw neural signals and b its filtered signals

Measuring neural spikes and sorting their timing are important in many applications. However, for large-population neural activities, measuring every single neuron is not possible because the number of neurons is too high. Therefore, another neural signal that can represent the large-population activity of nearby neurons can be further considered. This signal is called multi-unit activity (MUA) and proved useful in many studies [44, 77–79]. Figure 4 shows how MUA is calculated along with its transient waveform compared to the raw neural signal. Since the MUA signal is band-passed in neural spike frequency of 300–6000 Hz and calculated in root-mean-square, it is also called as spike band power (SBP). Compared to LFP, MUA provides more local activity of neurons. The LFP measured at any electrode typically has sizable contributions from neurons that are located several hundred micrometers away. In [80], when measuring the neuron population at a point 500 μm away, LFP decayed 5 times compared to the population center value, but MUA decayed 30 times which showed that distance-decay of the MUA is much greater than LFP. Thus, the local activity of the neuron network can be measured and estimated with MUA. Table 1 summarized neural signal modalities and their characteristics.

Fig. 4.

Raw neural signal and its calculated multi-unit activity (MUA) signal

Table 1.

Summary of neural signals and their characteristics

Neural signal	Signal amplitude	Signal frequency	Spatial extent (mm)	Simultaneously recordable neurons
Action potential (patch clamp	10–70 mVPP	100 Hz–10 kHz	–	1 individual neurons
Neural spike (depth electrode)	50–500 μVPP	100 Hz–10 kHz	− 0.1	10–100 individual neurons
Local field potential (depth electrode)	0.5–5 mVPP	1 mHz–200 Hz	− 3	Medium population of neurons
Multi unit activity (depth electrode)	10–400 μVPP	300 Hz–6 kHz	− 0.2	Small population of neurons
ECoG (Dura Surface Elec.)	10–1 mVPP	1 mHz–500 Hz	–	Large population of neurons
EEG (Scalp Surface Elec.)	10–100 μVPP	1–20 Hz	–	Very large population of neurons

High-density neural recording system

Requirements for implantable system

As explained in Sect. 2, most of the neural signals (e.g., AP, LFP, MUA, ECoG, etc.) have very weak amplitude and audio level frequency range. Furthermore, since it is implanted inside the living body, the design of a neural recording integrated circuit requires unique specifications.

Power consumption of the implantable system is the most important consideration because it is directly related to heat generation of the device [81]. A recent study showed that implanted systems dissipating more than 40 mW of power can cause a temperature increase of more than 2 °C and lead to cell death in days [82]. The input-referred noise (IRN) is also an important parameter because neural signals suffer from very weak voltage amplitude. High IRN directly leads to a drastic drop of signal-to-noise ratio (SNR). To measure LFP or ECoG, a neural recording system should be able to handle a very low-frequency band for neural signal integrity. The variable gain control is another important consideration against PVT variations that can affect the system performance. Common-mode rejection ratio (CMRR) and power supply rejection ratio (PSRR) should be also considered because the common-mode signal is fed through electrodes and power supply noise from wall-mounted 50/60 Hz or switching noise from the wireless power supply can pose SNR degradation. In addition, electrode-tissue interface tends to have up to several MΩ ranges and a few hundred DC offsets, so input impedance and DC rejection of neural recording channels should be carefully checked for signal integrity. Finally, for high-density neural recording more than hundreds of channels, the silicon area of each recording channel should be minimized. Table 2 shows the design specifications and primary considerations in integrated neural recording systems.

Table 2.

Design specification and primary consideration in integrated neural recording systems

Specification	Primary consideration		Typical range	References
Power consumption	Heat	Pchip < 40 mW for Δt < 2 °C	2.72–33 μW	[58, 71]
	IRN	Input transconductance(gm, in) ∝ ID
Input referred noise(IRN)	Input signal amplitude	Low IRN to achieve high SNR for signals as small as 50 μVPP	3.46–8.98 μVrms	[65, 71]
3-dB bandwidth	Input signal frequency range	Bandpass ability for frequency band of input neural signal	0.5 Hz–10 kHz	[64, 71]
Variable gain	Input signal amplitude	Adjustable gain to achieve highest dynamic range	52–68 dB	[71]
CMRR, PSRR	Power, common noise rejection	Reject common-mode signal at 50/60 Hz and reject power supply noise	75–92 dB	[64, 71]
Input impedance	Electrode-tissue interface	Higher input impedance than electrode-tissue interface for signal integrity	100 MΩ–10 TΩ	[6, 18]
DC rejection	Electrode-tissue interface	Block DC offset at electrode-tissue interface to prevent saturation	0.1–1 Hz	[17, 18, 74]
Area	Multi-channel implementation	Consume little silicon area as possible for multi-channel implementation	0.03–20 mm2	[68, 71]

_:To be maximized; _: Depend on signal; _: To be minimized

Neural recording channel

As explained in Sect. 3.1, there are many required specifications when designing a neural recording system. To satisfy those specifications, researchers have proposed advanced architectures and elements in the neural recording system. Figure 5 shows the conventional neural recording system which consists of a low-noise amplifier (LNA), analog-to-digital converter (ADC), and wireless telemetry circuit. Typically, multiple LNAs share the single ADC to save silicon area and power consumption as shown in Fig. 5. For that, an analog multiplexer can be used before ADC to time-multiplex the neural signals from several LNAs. A multiplexed neural signal which has a frequency band up to 10 M kHz is fed to ADC, where M is the number of LNAs connected to one ADC, so the sampling speed of ADC should be at least 20 M kHz to satisfy the Nyquist criterion. Then, for N-bit resolution ADC, the digitized neural signal will be 20 k M N/ s. The wireless telemetry circuit can be either an inductive link or an RF link depending on data rate to communicate outside the body through the skin [83–85].

Fig. 5.

Conventional neural recording systems that consist of electrodes, LNA, multiplexor, ADC, and wireless telemetry

When the electrode provides a weak neural signal, LNA amplifies the neural signal which is then digitized through ADC. As explained in Sect. 3.1, there are several considerations for designing LNA, and the LNA structure can be classified according to how the DC offset of the electrode-tissue interface is removed. The AC-coupled structure uses capacitors to remove DC offset, while the DC-coupled structure uses a low-pass filter in the feedback structure. A well-organized review about neural recording amplifiers can be found in [86].

Among various amplifier structures used in the LNA, the current-reuse structure has been widely used because of its high input transconductance [24]. Since the amplifier inside LNA usually dominates the overall IRN of the recording channel, minimizing the amplifier noise is crucial to increase SNR. As explained in [87], the input transconductance of an amplifier (g_{m, IN}) is the dominant factor for decreasing the amplifier noise. Figure 6 shows the current-reuse amplifier structure proposed in [37]. Compared to conventional amplifiers with only NMOS or PMOS transistors as their input stage, the current-reuse amplifier employs both NMOS and PMOS in an inverter structure to share the same drain current.

Fig. 6.

Conventional differential input stage and current-reuse differential input stage in neural amplifiers

Considering only thermal noise of transistors, the current noise power spectral density is modeled as:

$$\overline{{\text{i}}_{\text{n}}^2}=4·\text{k}·\text{T}·\mathrm{\gamma }·{\text{g}}_{\text{m}}$$ 1

where k is the Boltzmann constant, T is the absolute temperature in kelvin, γ is the thermal noise coefficient, and g_m is the transconductance of the input transistor.

For the conventional amplifier with NMOS only input stage, input-referred noise can be calculated as follows:

$$\overline{v_{n,i}^2}=\frac{2·\overline{i_{n,N}^2}}{(g_{m,N}{)}^2}.$$ 2

For the current-reuse amplifier, input-referred noise is calculated as follows:

$$\overline{v_{n,i}^2}=\frac{2·(\overline{i_{n,N}^2}+\overline{i_{n,P}^2})}{(g_{m,N}+g_{m,P}{)}^2}.$$ 3

It can be observed that the effective input transconductance (g_{m, IN}) is the sum of transconductance of NMOS (g_{m, N}) and PMOS (g_{m, P}). When the same current noise density and transconductance for NMOS and PMOS are assumed, input-referred noise of the current-reuse amplifier is:

$$\overline{v_{n,i}^2}=\frac{2·(\overline{i_{n,N}^2}+\overline{i_{n,P}^2})}{(g_{m,N}+g_{m,P}{)}^2}=\frac{2·(2·\overline{i_{n,N}^2})}{(2·g_{m,N}{)}^2}=\frac{\overline{i_{n,N}^2}}{(g_{m,N}{)}^2}$$ 4

which is half of the INR from the conventional amplifier [37]. A small increase of PMOS transistor area can lead to a drastic reduction of IRN, while consuming the same static current.

In addition, there are several types of ADCs that can be employed in neural recording systems. Figure 7 shows the general trade-off relationship between sampling rate and resolution and the candidates for neural ADC [88]. In case the neural recording ADC samples neural spikes which have a frequency range up to 10 kHz, the minimum sampling speed should be 20 kHz. For power and area efficient ADC, a successive approximation register (SAR) ADC is the most general choice among them.

Fig. 7.

ADC structures for neural recording systems depending on resolution and sampling rate

Figure of merit for neural recording

To compare the performance between various neural recording channels, a figure-of-merit (FoM) has been widely adopted. There are two types of FoM: (1) noise efficiency factor (NEF) and power efficiency factor (PEF), and (2) channel-FoM (Ch. FoM) and energy-area FoM (E-A FoM).

NEF represents the relationship of the noise-current trade-off in the amplifier and has the lower limit of 1, which means the thermal noise of a single BJT amplifier. While NEF only considers the current from the supply, PEF considers the supply voltage as well. NEF and PEF are defined as following [27, 89]:

$$NEF=\overline{V_{\mathit{IRN}}}·\sqrt{\frac{2·I_{\mathit{Tot}}}{\pi ·V_t·4·k·T·BW}}$$ 5

$$PEF=NE{F}^2·V_{\mathit{DD}}$$ 6

where V_IRN is the total input-referred noise of amplifier, I_tot the total current from the supply, V_t the thermal voltage, which is about 26 mV, BW the −3 dB bandwidth of the system, and V_DD is the supply voltage of the amplifier. Therefore, NEF is generally used for neural recording channel and PEF is often used to emphasize the power consumption of neural recording channel. Even though NEF and PEF provide good insight into the trade-off between power consumption and noise, they are not the complete FoM for neural recording channels because they only cover the power and noise trade-off. The other important aspects, such as silicon area, dynamic range, input impedance, etc., are not considered in NEF, PEF. Recent yearly NEF trends can be found in [6].

To include the silicon area as comparison factor, which is an important parameter for high-density neural recording systems, channel FoM (Ch. FoM) and energy-area FoM (E-A FoM) are proposed in [69] and defined as follows:

$$Ch.FoM=\frac{P_{\mathit{Ch}}}{2^{ENOB,Ch}·f_s}$$ 7

$$E-A.FoM=Ch.FoM·A_{\mathit{Ch}}$$ 8

where P_Ch is the averaged power consumption of a single recording channel, ENOB_Ch is the measured maximum effective number of bits, f_s is the sampling frequency, and A_Ch is averaged area per channel which includes the amplifier, buffer, and ADC. Recent yearly research trends for Ch. FoM and E–A FoM, which consider the area, power, and noise trade-off, can be found in [5].

Neural signal processing system

Neural spike detection algorithm

As explained in Sect. 3.2, the wireless telemetry circuit transmits digitized neural signals outside the body through the wireless link [71]. However, in high-density neural recording, the high data rate become a bottleneck. For example, a 256-channel neural recording system with 10-bit ADC resolution requires high data rate of 51.2 Mbps. Furthermore, the power consumption of the wireless telemetry circuit depends on the data rate as well. Assuming the data rate of 200 kbps per channel and energy consumption of 50 pJ/bit, the wireless telemetry requires 10 μW/channel, which exceeds the power consumption of the neural recording channel itself [43]. To address this issue, an on-chip neural signal processing such as a spike detection algorithm can be used to reduce the data rate.

There are mainly two types of spike detection algorithms that can easily be implemented in neural system IC [90]; (1) threshold-crossing detection and (2) nonlinear energy operator (NEO), as shown in Fig. 8. The threshold-crossing detection was proposed in the late 1990s [91] with using simple analog comparators. When the neural signal crosses a pre-defined value (threshold), the output of the comparator shows the binary value of one. Then, the neural spike can be captured with using the comparator outputs. This method is very simple and easy to implement, but the main downfall is how to determine the threshold value (V_TH), which may varies depending on environments. Conventionally, V_TH is determined as follows [92]:

$$V_{\mathit{TH}}=4·{\sigma }_n$$ 9

$${\sigma }_n=median\left(\frac{\left|x_{\mathit{bp}}\right|}{0.6745}\right)$$ 10

where x_bp is bandpass-filtered signal and σ_n is a standard deviation of the background noise. If V_TH is universally applied to all recording channels, variations such as distance to nearby neurons, type of neurons, and PVT variation of the chip can cause false detection of spikes or missing neural spikes.

Fig. 8.

Spike detection algorithm: a threshold crossing and b nonlinear energy operator (NEO)

To overcome such disadvantages of the threshold-crossing detection method, researchers have used a nonlinear energy operator (NEO) that takes into consideration of the energy-level difference between transient points rather than the absolute transient value itself [93–96]. For the continuous-time function x(n), NEO(n) is defined as follows:

$$NEO\left(n\right)=x^2\left(n\right)-x\left(n-1\right)\times x\left(n+1\right)$$ 11

Equation 9 implies that NEO(n) can be maximized when both amplitude and frequency of x(n) are large. After detection, the wireless telemetry can send only selected data such as neural spike signals, which significantly reduces the required data rate.

High-density neural recording system with signal processing

The limitation of the conventional neural spike detection algorithm is that it tries to find neural spikes after measuring raw neural signals from all electrodes, which already wastes power and area for recording channels. Since the number of neurons in the brain is very large, the number of neurons firing simultaneously is limited. In [97], based on the energy consumption of the brain and single neural spike, the author concluded that under 1% of neurons are active simultaneously. If the signals from non-active brain regions are recorded and processed through the system, then power and area consumption are wasted. Thus, the advanced electrode-selection algorithm is proposed based on the relative neuron activation which is evaluated based on MUA [63].

As explained in Sect. 2, MUA represents the local neural population’s activation. Therefore, relative local activity can be compared by observing MUA, and the most activated areas of the brain can be selected. Then, the neural spikes from those selected area (electrodes) can be recorded with a fewer number of recording channels. For example, the activated electrodes among 32 electrodes can be automatically connected and recorded with 12 recording channels by using 8-channel neural spike scanning circuits as shown in Fig. 9. The neural scanning channel consists of a power-efficient amplifier and a squaring circuit to calculate MUA. Then, the controller compares the MUA values from each scanning channel, and three most activated electrodes groups are selected and connected to 12 neural recording channels. With using the neural scanning method, the neural recording system can save the power and silicon area while still measuring the neural spikes from activated electrodes.

Fig. 9.

Adaptive electrode-selection method with neural scanning for high-density neural recording systems

Conclusion

In this paper, requirements and implementation of the high-density neural recording system are covered. The difference between non-invasive and invasive neural signals is explained, while several types of neural signals including MUA are described. The paper also discussed about the figure-of-merits and issues of the conventional recording system. In the end, the neural recording systems with signal processing functions are introduced which can provide a promising solution for high-density neural recording while using the limited hardware resources.

Funding

This work was supported by national research foundation of Korea (NRF-2020R1F1A1074719 and 2020M3F3A2A01081240) funded by ministry of science & ICT.

Declarations

Conflict of interest

Han-Sol Lee declares that he has no conflict of interest. Kyeongho Eom declares that he has no conflict of interest. Minju Park declares that she has no conflict of interest. Seung-Beom Ku declares that he has no conflict of interest. Kwonhong Lee declares that he has no conflict of interest. Hyung-Min declares that he has no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.