Plasticity and Adaptation in Neuromorphic Biohybrid Systems

Summary

Neuromorphic systems take inspiration from the principles of biological information processing to form hardware platforms that enable the large-scale implementation of neural networks. The recent years have seen both advances in the theoretical aspects of spiking neural networks for their use in classification and control tasks and a progress in electrophysiological methods that is pushing the frontiers of intelligent neural interfacing and signal processing technologies. At the forefront of these new technologies, artificial and biological neural networks are tightly coupled, offering a novel “biohybrid” experimental framework for engineers and neurophysiologists. Indeed, biohybrid systems can constitute a new class of neuroprostheses opening important perspectives in the treatment of neurological disorders. Moreover, the use of biologically plausible learning rules allows forming an overall fault-tolerant system of co-developing subsystems. To identify opportunities and challenges in neuromorphic biohybrid systems, we discuss the field from the perspectives of neurobiology, computational neuroscience, and neuromorphic engineering.

Graphical Abstract

Neuroscience; Bioengineering; Computer Hardware; Computing Methodology

Introduction

Disorders such as stroke, and neurodegenerative diseases such as multiple sclerosis and Parkinson disease, disrupt connections among different brain regions, causing a severe loss of motor, sensory, and cognitive functions (Semprini et al., 2018). With the final aim of developing innovative technologies to such neurological disorders, about 20 years ago researchers started to explore the possibility to create “biohybrid” systems, based on the functional interaction between biological (e.g., neural) systems and artificial devices (Vassanelli and Mahmud, 2016). The concept of biohybrid technology is seeing increasing attention as a potential direction for the creation of novel brain machine/computer interfaces (BCI) and neuroprostheses, which allow the human nervous system of disabled patients to communicate with electronic and/or robotic devices. With a primary focus on data acquisition, BCI represent approaches mainly designed to establish a direct communication between the central nervous system (CNS) of a patient with an effector (i.e., a robotic arm, a functional electrical stimulator device) (Bouton et al., 2016; Collinger et al., 2013; Silva, 2018). In contrast, neuroprostheses and neuromodulators aim at functional electrical stimulation of the CNS, e.g., to block epileptic seizures (Bergey et al., 2015; Jobst and Thomas, 2015), and reduce Parkinsonian symptoms (Priori, 2015; Rosin et al., 2011). More recently, neurorehabilitative applications have also shifted into focus, where the close coupling of artificial systems and nervous tissue promises approaches to treat brain (Guggenmos et al., 2013) and spinal cord injury (Wagner et al., 2018). However, a major obstacle in developing novel neuroprostheses based on bidirectional communication with and within the brain is the complex nature of interactions among different brain areas, which in turn presents a challenge for the development of appropriate stimulation protocols as well as for testing those devices using in vivo models (Buccelli et al., 2019). As suggested by Steve Potter (Potter and DeMarse, 2001), a possible alternative methodology consists of exploiting biohybrid systems composed by simpler experimental preparations such as in vitro neuronal cultures, which can be easily controlled, manipulated, and monitored, thus reducing the experimental variability and facilitating the interpretation of results. In vitro biohybrid systems, being at the interface between neuroscience and robotics, are able to provide an excellent testbed for modulation of neuronal tissue and forming the basis of future closed-loop neural interfaces (Potter, 2010). We will first introduce the concept of biological neural networks (BNNs) and their use as a subsystem of the overall biohybrid; then we will focus on its artificial counterpart, formed by artificial neural networks (ANNs). The term ANN here references to both rate-based and spike-based neural networks; our particular focus is, however, on those ANNs that borrow computational principles from biology as is common in neuromorphic approaches.

Figure 1 illustrates the generalized setup of a biohybrid system. The physical interface to the BNN is implemented via stimulation and recording electrodes. Acquired signals are amplified and digitized in the acquisition frontend, before they are further processed to extract informative features that serve as input to the ANN. ANN output is likewise used to determine a stimulation protocol, which controlled the current- or voltage drivers connected to the stimulation electrodes to close the loop.

Figure 1.

The Basic Concept of ANN and BNN Interfacing

Throughout this review, we intend to outline the design choices made within the ANN and BNN subsystems and discuss protocols and implementations of plasticity and adaptation in dedicated sections. We will conclude with highlighting techniques of adaptation and plasticity employed in the overall biohybrid system, providing a perspective on each topic from the viewpoint of neurobiology, neuroengineering, computational neuroscience, and neuromorphic engineering research.

The Biological Subsystem

Since the beginning of research in biohybrid systems, elementary biological preparations were considered in vitro as a starting ground for the study of learning and adaptation. As already mentioned, the ease of physicochemical control and pharmacological manipulation makes in vitro experimental models highly versatile and ethically acceptable, as non-sentient systems. In the following, we briefly introduce the dissociated and the organotypic culturing techniques, which in recent years enjoyed a renewed interest due to the latest development of stem cell technologies as well as the access to human brain tissue collected during resective surgery. Table 1 summarizes the most commonly used preparations, together with their corresponding signal characteristics and extractable features, taken from key works.

Table 1.

Exemplary Overview of In Vitro Preparations Used in Biohybrid Systems

	Dissociated Cortical Neuron Culture (Keren and Marom, 2014)	Organotypic Hippocampal Slice (Berger et al., 2012)	Spinal Cord In Vitro (Joucla et al., 2016)	Brainstem In Vitro (Kositsky et al., 2009)	Whole Brain In Vitro (Bonifazi et al., 2013)
Amplitude	100 μV	100 μV	50 μV	10 μV	100 mV intracellular; 500 μV LFP; 50 μV (spikes)
Spectrum	150−3,000 Hz	400−4,000 Hz	0.08–3,000 Hz	200−5,000 Hz	150−8,000 Hz (spikes); 0.7–300 Hz (LFP)
Features	Action potentials; multi-unit activity	Action potentials; multi-unit activity	Burst activity; action potentials	Action potentials; LFP	Action potentials; LFP
Interface	MEA	MEA	Shank	Suction glass pipette	Suction glass pipette; MEA
Animal model	Newborn rat	Rat	Rat	Sea lamprey	Guinea pig

Dissociated Cultures

The simplest and most successful biological model for biohybrids as well as for studies on plasticity and ex vivo learning is represented by in vitro cultures of dissociated neurons (Bi and Poo, 1998; Marom and Shahaf, 2002; Tateno and Jimbo, 1999). Living cells can be harvested from the (early postnatal or embryonic) nervous tissue of rodents, typically the hippocampus, the cortex, or the spinal cord, by mechanical dissociation and enzymatic digestion (Fedoroff and Richardson, 1992). The dissociation process exploits the property of enzymes like trypsin or papain to disrupt the connections between neighboring cells in the tissue and is followed by mechanical separation. In the preparation of dissociated hippocampal neuron cultures for their use as BNN, it is fundamental to cut off the dentate gyrus from each dissected hippocampus, as it contains neuronal precursors that are able to differentiate in culture, thus modifying the density of neurons in the network.

Once separated from the extracellular matrix, cells can be maintained alive for weeks to several months (Potter, 2010), bathed with a culture milieu supplemented by growth factors. Neurons require a coating protein that mimics the extracellular matrix and helps neurons to adhere. The first steps of neuron dissociation and pre-plating are performed in the presence of bovine fetal serum and antibiotics to prevent the neuronal culture from bacterial contamination. Serum concentration is successively reduced and substituted with serum-free supplements to remove catabolites and add fresh nutrients while maintaining trophic factors. Cells are also incubated at physiological temperature and atmosphere while plated on a Petri glass dish or on a substrate-integrated array of microdevices (Chao et al., 2008; Keller and Frega, 2019; Rutten et al., 2001). Once dissociated, the cells lose all their existing cellular processes. Hours after plating, they start extending neurites again for up to 1 mm while progressively reconnecting by functional synaptic connections, over the course of 3–4 weeks in vitro. As glial cells within the cell suspension tend to form an insulating layer on multielectrode arrays (MEAs) that compromises recording as well as stimulation, compounds like cytosine arabinoside are added to the culture medium to block the glial cells' mitotic cycle. Alternatively, incubating the cell suspension on a standard Petri dish allows only the glial cells to adhere, whereas the neurons remain in suspension and can be subsequently collected (Vassanelli and Fromherz, 1997). The resulting system approximates a two-dimensional monolayer of randomly interconnected neurons, usually cultured at a density of several thousands of cells per square millimeter, with cellular composition reminiscent of those of the initial tissue (Nakanishi and Kukita, 1998; Neale, 1989). Although both control conditions and pathological conditions have been modeled by dissociated cell cultures in vitro, obtained from rodents (Moskalyuk et al., 2019), the use of human-derived pluripotent stem cells has also gained substantial momentum. There, cells are derived from patients and are afterward reprogrammed biochemically to differentiate ex vivo into functional neurons and astrocytes (Frega et al., 2019; Liu et al., 2017). Intriguingly, embryonic stem cells can also be induced to differentiate into GABAergic neurons (Ban et al., 2007) that exhibit spontaneous and evoked activity that is spread throughout the culture through excitatory and inhibitory synaptic connections.

The saline solutions used during experiments on MEA guarantee the stability of osmolarity conditions. In case of long-term recordings, perfusion of extracellular solution must be continued throughout the experiment. Cultured vertebrate cells are known to reorganize in vitro without recreating any of the original in vivo microcircuitry, typical of the intact neural tissue. Nonetheless, the random character of the synaptic connectivity in cultures, the homogeneity of cell types, together with the improved observability of synaptic transmission and plasticity, make dissociated neuronal cultures a very versatile and highly accessible preparation for studying those key universals that are the most relevant for biohybrids: excitability, learning, and memory (Keren and Marom, 2014; Marom and Shahaf, 2002).

Organotypic Cultures

When larger portions of the in vivo cytoarchitecture and synaptic connectivity need to be retained, or when a heterogeneous composition of neuronal types is of interest, it is possible to slice (juvenile) brain tissue and keep the individual samples alive in culture for long periods of time (van Bergen et al., 2003). This technique is called organotypic slice culturing as the environment individual cells are embedded in is by definition organlike (Gahwiler, 1988; Humpel, 2015). Oxygenation is then the most important limiting factor, adding complexity to the maintenance procedure. As opposed to a cellular monolayer, oxygen hardly diffuses within a slice of brain tissue, when blood vessels and circulation have been interrupted. Then, a proper exposure to gas, nutrients, and chemicals must be ensured e.g., by slowly rotating the plastic tubes where the slices are stored during incubation. Alternatively slices are placed on a semipermeable membrane, at the boundary between the culture medium and air. The slicing process is known to sever a large portion of neuronal connections; maintaining a 200- to 300-mm-thick slice alive in an artificial cerebrospinal fluid is known to be associated to a reactive synaptogenesis (Humpel, 2015) leading to the formation of novel synapses.

Acute brain slices can be placed on top of MEAs and organotypic brain slices cultured on the MEA surface with protocols adapted from glass coverslips. In both cases, and particularly with acute slices, the recorded signals differ from dissociated cultures as they comprise, and in fact are dominated in amplitude by, low-frequency local field potentials (LFPs), elicited by synaptic and neuronal activity.

Organotypic slice cultures have often been employed while coupled to substrate arrays of microdevices and used to advance our understanding of emerging electrophysiological behaviors (Beggs and Plenz, 2003; Plenz and Thiagarajan, 2007) as well as the development and plasticity of activity states (Buonomano, 2003; Johnson and Buonomano, 2007). Conventionally, MEA are structured as regular grids of electrodes on either a polymer or silicon substrate. Conformal MEAs (cMEAs) represent an interesting alternative to the planar MEAs for in vitro biohybrid systems, especially for organotypic cultures and slices. These use a more complex arrangement of electrodes that follows the organ's structure, achieving a higher channel density while avoiding an overhead in signal acquisition hardware (Gholmieh et al., 2006). Here continuous perfusion of the slice (Panuccio et al., 2018) ensures osmolarity conditions. Typically, slices are held in place by a nylon mesh attached to a platinum Ω-wire, which improves tissue adhesion and mechanical stability (Mapelli and D'Angelo, 2007; Obien et al., 2019), further aided by inducing coagulation with thrombin (Hutzler et al., 2006). This arrangement is compatible with combined patch-clamp recordings where viable cells are identified for their smoothness and clear outline (Gibb and Edwards, 1994).

Portions of (adult) human brain tissue collected as a side product of resective surgical interventions in patients suffering from epilepsy or brain cancer have been used as acute preparations (Goriounova et al., 2018; Testa-Silva et al., 2014) or even cultured successfully in vitro for long periods of time (Eugene et al., 2014). As the availability of physiological human nervous tissue is highly limited, this approach opens important possibilities in gaining insight into the dynamics and anatomy of the human cortex on a microscopic scale.

Artificial Drivers of BNN Dynamics

Dissociated neurons are spontaneously active, exhibiting alternate periods of high activity (i.e., network bursts) with periods of quiescence, in a dynamic state that resembles the fluctuation of UP and DOWN states in vivo (Wagenaar et al., 2006). This activity is typical of the mammalian nervous system, where it can be found at different levels of cortical organization and function. Indeed, in vivo cortical circuits spontaneously generate slow oscillatory activity in the absence of external inputs (e.g., during sleep or anesthesia; Chauvette et al., 2011) or during quiet wakefulness. In vitro, the activity dynamics are an indicator of network formation (Chiappalone et al., 2006; Wagenaar et al., 2006): In early developmental stages, mainly irregular and asynchronous spiking activity is exhibited. Following the second week in vitro, spikes start to cluster into bursts, a persistent feature of mature networks (Marom and Shahaf, 2002), found in both hippocampal (Bonifazi, 2005) and cortical (Chiappalone et al., 2007) cultures. Moreover, studies involving organotypic and acute brain slices (Beggs and Plenz, 2003), cultures of dissociated neurons (Pasquale et al., 2008), in vivo brains (Petermann et al., 2009), and in humans (Dehghani et al., 2012) suggest that such periods of intense firing may be modeled as “neuronal avalanches,” following scale-invariant statistical distributions in space and time. These distributions have been found to follow power laws and are conserved across species and experimental preparations, independent of the recording technique employed (e.g., MEA, intracortical recordings, magnetoencephalography, functional magnetic resonance imaging; Massobrio et al., 2015a). Activity in dissociated cultures can be artificially driven by electrical stimulation, delivered through one or more electrodes of the MEA. Interestingly Pasquale et al. (2008) report that by applying a simple, low-frequency (<1 Hz) electrical stimulation to different network locations, the rank orders of electrodes during evoked and spontaneous events are remarkably similar, independent of the stimulation source. This study provides the first evidence that similarity between spontaneous and evoked activity patterns, a basic and important feature of cortical function in vivo (Luczak et al., 2007; Mao et al., 2001; Stringer et al., 2019), can also be observed in generic (unstructured) cultures of dissociated cortical neurons (Pasquale et al., 2017). This provides an important indication that simple in vitro systems retain fundamental properties of more complex in vivo structures, including plasticity and learning (Marom and Shahaf, 2002). In general, low-frequency, sustained electrical stimulation locks the phase of periodic bursts to the applied stimuli (Baljon et al., 2009; Chiappalone et al., 2007; Maeda et al., 1995). Moreover, the low-frequency stimulation pattern can induce long-lasting alterations in spontaneous (Bologna et al., 2010; le Feber et al., 2010; Vajda et al., 2008) as well as evoked activity (Ide et al., 2010) of cortical networks, modulating the spatiotemporal dynamics of network bursting. Higher rates of stimulation induce a transition from synchronized bursting activity into a sparser spiking behavior, more similar to the in vivo awake cortical dynamics (Wagenaar et al., 2005). Conversely, an electrical stimulation pattern tailored to the network's endogenous activity is able to efficiently induce modifications in the network synchronization, and it affects the network bursting properties in particular, by increasing both firing and bursting rates (Zullo et al., 2012). The reason behind this is the intrinsic variability of the neuronal dynamics, which spans multiple spatial (i.e., from single neuron to an entire network) and temporal (from seconds to hours) scales (Gal and Marom, 2013; Mainen and Sejnowski, 1995). Thus, by using more “natural” distributions of the stimuli it was possible to better entrain the intrinsic dynamics of single, isolated neurons as well as entire networks (Scarsi et al., 2017).

Hebbian plasticity, in the form of long-term potentiation (LTP) and long-term depression (LTD), has been successfully induced by MEA-based electrical stimulation, as reported in different neural preparations in vitro (see Massobrio et al., 2015b, for a complete review). The first demonstration in cultured mammalian networks was provided by the pioneering work of Maeda et al. (1998). They reported the ability to use tetanic stimulation (burst trains with intra-burst frequency >10 Hz and inter-burst frequency <1 Hz) through one or more electrodes to induce long-term changes in both firing probability and latency of response to the stimulus provided. Starting from this result, other groups tested modified versions of tetanic stimulation, either increasing or decreasing the stimulation frequencies, resulting in medium/long-lasting changes in terms of probability of evoking spikes at each electrode location (Chiappalone et al., 2008; Jimbo et al., 1998; Madhavan et al., 2007; Ruaro et al., 2005; Tateno and Jimbo, 1999). The ability of manipulating synaptic efficacy, even in the context of in vivo settings has been, moreover, shown in primates (Jackson et al., 2006) where the authors successfully employed feedback stimulation to reorganize the representation of movement in the wrist area of primary motor cortex through the use of a chronically implanted feedback stimulator. Here, stimuli delivered within a time window of 50 ms from a recorded spike in motor cortex induced Hebbian plasticity, changing the preferential firing of neurons at the recording site to correlate with the output effect observed in the neurons of the stimulation site. Even without the presence of precise timing, tetanic stimulation has been proved to be an effective technique for the implementation of direct reward-based training protocols to induce synaptic depression or potentiation at the population level, resulting in an increase or decrease of evoked activity, as shown in dissociated cortical neurons using MEA (Chiappalone et al., 2008). The same paradigm was adopted for improving the learning capability of a biohybrid system in vitro composed by a network of neurons interfaced with an MEA and a small robotic agent (Novellino et al., 2007) or its virtual simulation (Tessadori et al., 2012). In terms of learning, a seminal article by S. Marom's group first demonstrated conditioned in vitro learning capabilities of dissociated cultures (Marom and Shahaf, 2002). To achieve this goal, the authors designed a closed-loop protocol with the purpose of driving network activity toward a specific state: Electrical stimulation was used to provide a reinforcement signal whenever the network spontaneously displayed the desired behavior. The results were replicated by other groups (le Feber et al., 2010; Li et al., 2007; Pimashkin et al., 2013; Sinapayen et al., 2017), and some constraints on the learning and its relations to spontaneous activity were defined. Upon learning, the profiles of spontaneous bursts were changed and spiking synchrony increased (Li et al., 2007; Stegenga, 2009).

The ability to influence BNN dynamics with the facilitation of long-lasting effects on its network topology and function underlines the applicability of advanced feedback-oriented electrophysiological methods (mostly based on electrical stimulation) as an experimental paradigm in neuroscientific research, with the final aim of creating novel therapeutic devices. Here, neuromorphic biohybrid systems promise to provide the opportunity of implementing control schemes that re-establish physiological connectivity and neuronal dynamics. Likewise, the ability to make comparisons between closed-loop/trained and open-loop/untrained conditions allows the verification and the assessment of hypotheses related to intra-BNN plasticity.

Spiking and Non-spiking Neural Networks as the Artificial Subsystem

Various approaches to the design of spiking and non-spiking neuronal networks in hardware exist and have made their way from conventional hardware into the field of neuromorphic engineering, which aims at the simulation or emulation of biologically plausible neurons in hardware. While the degree of biological realism in hardware implementations of spiking neural network (SNN) is higher than in rate-based neural networks in that the internal communication is typically asynchronous and event based, certain trade-offs are made to accommodate for the differences between silicon and living tissue. Cell morphology or nonlinear propagation of potentials along the dendritic tree are typically not represented. Rather, the common approach is to implement neuronal behavior as point processes without a spatial dimension, where synaptic inputs are processed similarly in every synapse, and without respect to distance to the soma. This allows a greater degree of flexibility in engineering networks and massively reduces the silicon area required.

Despite these discrepancies, a particular advantage of neuromorphic implementations over software simulations based on frameworks such as Neuron, Nest, or Brian (Gewaltig and Diesmann, 2007; Goodman and Brette, 2009; Hines and Carnevale, 2001) is the fact that biological real-time and even faster-than-biology simulations can be achieved here, making hardware implementations particularly suited for biohybrid experiments that operate under latency constraints.

Neuromorphic Processors as a Computational Substrate

Neuromorphic implementations of neural networks feature a lower power consumption resulting in a lowered heat dissipation that enables the use of more computational resources within a biocompatible thermal budget. This factor is particularly promising in the design of implantable neuroprosthetic devices. Since the early 1990s, neuromorphic systems have become an active field of research, with the pioneer chips developed by Jung et al. (2001), Mahowald and Douglas (1991), Mead (1990). Neuromorphic integrated circuits (ICs) (Indiveri et al., 2011) can be divided into two major categories: analog/mixed-signal application-specific integrated circuits (ASICs) on the one hand and purely digital implementations (field-programmable gate arrays [FPGAs], microprocessors, or ASICs) on the other. A wide variety of neuron models have been implemented on several platforms: threshold models can be found in Ambroise et al. (2013), Indiveri and Fusi (2007), Kohno et al. (2016), Liu and Douglas (2004), Mayr et al. (2016), Qiao et al. (2015), Schemmel et al., 2007, Vogelstein et al., 2004 and conductance-based model are used by works such as Binczak et al. (2006), George et al., 2015, Hasler et al., 2007, Natarajan and Hasler (2018), Partzsch et al. (2020), Renaud et al., 2007, Sorensen (2004). Most analog and mixed-signal designs rely on macros that emulate neuronal behavior directly but use digital techniques to communicate action potentials and establish a network topology. Likewise, implementations of plasticity rules are usually mixed, with analog synaptic weight computation and digital weight storage. The BrainScaleS system, for instance, (Rast et al., 2013; Aamir et al., 2018) follows this approach to integrate neuromorphic chips and a routing system (Hartmann et al., 2010) on a full wafer scale. It simulates large-scale neural networks hundreds of times faster than biological time, which, however, complicates the interaction with BNN if a close integration of ANN and BNN neurons is desired. In the digital domain, SNN implementations for predominantly bioinspired applications can be found in works by Levi et al. (2018), Nanami and Kohno (2016), Rice et al., 2009, Sabarad et al., 2012, Wang et al. (2013). Here, multiprocessor architectures such as LOIHI (Davies et al., 2018) or SpiNNaker (van Albada et al., 2018; Furber et al., 2013) can be found. In addition, there exists a wide field of Graphics Processing Units (GPUs) and dedicated accelerators for purely rate-based networks as found in Aimar et al. (2019), for example, that are intended for general artificial intelligence applications. These digital systems usually offer a large degree of flexibility in their use, compared with analog sub-threshold neuromorphic processors in particular. Implementations on FPGA platforms are becoming more numerous, as available resources in FPGA drastically increased in the recent years and HDL synthesis toolchains are made more widely available. As a result of this trend, Cassidy et al., 2011 were able to present an implementation on 1 million Leaky Integrate-and-Fire (LIF) neurons in a single FPGA platform, which makes this approach a viable alternative to ASICs such as TrueNorth (Merolla et al., 2014) by IBM, which features 1 million neurons and 256 million synapses. In comparison (Arthur et al., 2012) describe the implementation of 256 IF neurons and 1,024 × 256 synapses; Wang et al. (2013), the implementation of 4,000 neurons and 1.15 million synapses; and Khoyratee et al. (2019), the implementation of 15,000 Hodgkin-Huxley neurons. Various crossovers also exist, e.g., analog neuromorphic chips enhanced by plasticity carried out in FPGAs (Qiao et al., 2015). Despite the ability to provide large real-time networks, neuromorphic processors have still not fully made their way into the research on biohybrid systems, presumably due to the only recent emergence of large-scale platforms and their limited availability, as Figure 2 indicates. The trend toward larger-scale neuromorphic processors and the usage of bioinspired computational principles such as massive parallelism and asynchronous communication opens novel perspectives in this field.

Figure 2.

Number of Artificial Neurons (Implemented in Hard- or Software) Used in Various Biohybrid Systems, Compared with the Increase in Neurons Integrated in Neuromorphic Processors in General

The groundwork in ANN/BNN interaction was provided by works like Le Masson et al. (1995) where interactions between single artificial and living neurons were established for the first time.

Since this point, initially small neural networks like central pattern generators have been used for biohybrid systems to study locomotion (Joucla et al., 2016, Jung et al., 2001; Sorensen, 2004). Recently, more complex systems have been designed using real-time, spiking ANN. A study by Chou et al. implemented a bidirectional interface between an ANN and a retinal slice obtained from an adult rat and recorded by an MEA (Chou et al., 2015). Here, however, interactions were suffering from latencies beyond 100 ms. In the projects Coronet, RAMP, and Brainbow, real-time ANN (analog/mixed signal chips in Coronet and RAMP, FPGA in the Brainbow project) have been interfaced with BNN for closed-loop applications (Bonifazi et al., 2013; Keren et al., 2019; Serb et al., 2020) (for reference see Figure 5). The RAMP project achieves an open-loop setup via internet-based remote communication, where signals from biological neurons are preprocessed by memristors and then act as input to the analog ANN (Serb et al., 2020). The Coronet project shows activity propagation in alternating BNN and ANN populations in a closed loop. In contrast, the Brainbow project aims at the replacement of BNN areas by ANN (Buccelli et al., 2019). As an alternative application to this neuroprosthetic orientation, ANNs are also used to perform biomimetic stimulations for driving BNN dynamics. As one among many works with this aim Mosbacher et al. (2020) (see section Artificial Drivers of BNN Dynamics) show that BNN can synchronize with ANN using optogenetic stimulation controlled by ANN dynamics. This study highlights benefits of optogenetic stimulation over electrophysiological approaches when it comes to improving specificity and spatial resolution of stimulation.

Figure 5.

Exemplary Approaches Taken in the Realization of Biohybrid Systems

(A) Experimental setup created within the context of CORONET (Keren et al., 2019).

(B) The Brainbow setup (Buccelli et al., 2019).

(C) The approach taken within the RAMP project (Serb et al., 2020).

A variety of different experimental setups expand the technological side of biohybrid systems beyond the ANN. These make use of robotic platforms that allow the interaction between the BNN and the environment through platform sensors and actuators. In its role as the artificial “body” for the BNN, the robotic platforms' kinematics can be limited to a controllable level of complexity, which poses a benefit over the use of the animals' motor system in neuroprosthetic approaches (Kudoh et al., 2011; Mussa-Ivaldi, 2010; Warwick et al., 2010). The exchange of information between BNN and the robotic platform is naturally bidirectional. Sensory feedback is established by translating the platform's sensory data to stimulus protocols, whereas the platform's actuators are driven by the activity observed in the BNN. This concept has been explored, e.g., by DeMarse et al. (2001), where bursts of activity that propagate through the population was successfully produced. Artificially attenuating the feedback on some sensors caused the bidirectionally interfaced robotic platform to exhibit a new and repeatable movement trajectory (Reger et al., 2000).

Particularly the large number of neuromorphic sensors in the literature, such as auditory sensors (Liu and Delbruck, 2010), visual sensors (König et al., 2002; Moeys et al., 2017), and tactile sensors (Lee et al., 2015), could provide such a natural interface to BNNs, thus forming future experimental paradigms for the study of network-scale information processing in vitro.

Biologically Plausible Learning Rules and Their Implementation in ANN

The key paradigms after which many neuromorphic processors are designed directly influence which type of learning rules are most resource efficient and scalable. A high degree of parallelism and asynchronous communication demands that plasticity mechanisms are localized to the particular pre- or post-synaptic neuron. Numerous biologically plausible models for the adjustment of synaptic weights fit these requirements, as shown in Table 2, reproduced with permission from a work by Mayr et al. (see reference Mayr (2010)). The displayed mechanisms for the change in synaptic efficacy are here shown together with the protocols used to evoke them within the BNN.

Table 2.

Mechanisms of Synaptic Plasticity and Adaptation Observed in Biology, and Approaches of Modeling Them

Short Description and Reference	Experimental Characteristics	Pre-synaptic Protocol	Post-synaptic Protocol
Conventional STDP (Bi and Poo, 1998)	Glutamatergic synapses onto cultured disassociated rat hippocampal neurons, embryonic rat	Sub-threshold EPSC evoked via a 100-mV, 1-msdepolarization step at 1 Hz, 60 repetitions	AP evoked by current pulse 2 nA, 2 ms, same timing protocol as pre-syn. Δt to pre-syn. spikes: sweep −90…+90 ms
Frequency-dependent STDP, Figure 8A of Sjöström et al. (2001)	Slices of visual cortex, synapses in apical dendrites in thick tufted L5 neurons 12 to 21-day Long-Evans rats	Extracellular stimulation, 50 single pairings at 0.1-Hz repetition, pairings at 10, 20, 40, 50 Hz: in groups of 5, with 15 repetitions at 0.1 Hz	Single AP by 0.8–1.5 nA, 5-ms current injection, Δtto presyn. spikes: ±10 ms
Triplet pulse patterns (Froemke and Dan, 2002)	Slices of visual cortex, glutamatergic synapses onto L2/3 pyramidal neurons, 2- to 5-week Sprague-Dawley rats	60–80 triplets at 0.2 Hz, 5−150 μA, 0.1- to 1-ms single-pulse extracellular stimulation	AP evoked by 1 nA, 2−3 ms postsyn. current injection, triplets with one or two presyn. pulses, Δt's of triplet spikes with respect to each other: sweep −100…+100 ms
Pre-synaptic burst patterns, Figure 4 of Froemke et al. (2006)	Slices of visual cortex L2/3 pyramidal cells, 10- to 35-day Sprague-Dawley rats	5−150 μA, 0.1−1 ms extracellular stimulation, 2–5 EPSPs at 100 Hz, 30–40 repetitions at 0.2 Hz	0.5–2.5 nA, 1.5−5 ms current injection, single AP Δt≤6 ms before/after presyn. Burst
Standard rate (Dudek and Bear, 1992)	Slices of hippocampal area CA1, Schaffer collateral fibers onto pyramidal cells, adult male albino rats	Pre-synaptic tetanus, 900 pulses, single repetition, pulse frequencies 1−50 Hz, excited with 10−30 μA, 0.2 ms current injection	No control/recording of post-synaptic cell activity mentioned, Three assumptions tested: (1) uncorrelated 10 Hz Poisson, (2) postsyn. APs evoked by EPSCs in LIAF neuron with 5% threshold noise, (3) no postsyn. APs, only sub-threshold EPSC influence
Voltage control (Artola et al., 1990), additional similar experiments in Ngezahayo, 2000	Slices of adult rat visual cortex, L2/3 regular spiking neurons	Extracellular stimulation 50 Hz tetanus, five 2-s pulse trains spaced at 10-s intervals, four times EPSC test intensity	Intracellular current injection to target membrane voltage

Reproduced with permission from Mayr (2010). AP, Action Potential; EPSC, Excitatory Postsynaptic Current; LIAF, Leaky Integrate and Fire.

In the selection of appropriate rules within the context of biohybrid systems, closed-loop real-time interaction with BNN imposes that synaptic weight updates need to be made during network operation, without the necessity of a separation of network-wide training and operation phases, which would interrupt the interaction. The classical model that features these aspects is Hebbian learning as used in works like Hogri et al. (2015), commonly involving timing information as STDP. Here, the weight update is a function of the relative timing between the pre- and post-synaptic spikes, Δt = t_post−t_pre, as a correlation metric (see Figure 3A for reference). The weight update Δw is now following an exponential dependence scaled by A⁺ and A⁻ as well as a time constant of τ (see Equation 1) to faithfully describe long-term plasticity found in cortical neurons as described in (Bi and Poo, 1998)

$$\text{Δ}w=\{\begin{array}{cc}{A}^{+}\text{∗}{e}^{-\text{Δ}t/r^{+}}0,& \text{if}\text{Δ}t>0\\ A^{-}\text{∗}{e}^{-\text{Δ}t/r^{-}}0,& \text{if}\text{Δ}t>0\end{array}$$ (Equation 1)

Figure 3.

Illustration of Plasticity Rules

(A) Weight update as a function of timing between pre- and post-synaptic timing in STDP.

(B) Weight update as a function of post-synaptic firing rate in BCM.

Particularly for the use with novel memory technologies such as memristive devices, STDP brings the benefit of a straightforward implementability, as Zamarreño-Ramos et al. (2011) show. Here, both terminals of the device are actively driven with specifically shaped waveforms. The potential over the memristor terminals is dependent on the correlation of the two waveforms and results in changing conductivity. As shown in Indiveri et al. (2006), however, this form of plasticity is less well suited to hardware implementation in terms of area and complexity than learning rules that rely on post-synaptic voltages instead of post-synaptic spikes. Due to this reason, besides strict STDP rules, an alternative approach to updating weights is found in formulations such as the one proposed by Brader et al. (2007). In contrast to STDP, this rule evaluates a weight update function every time a pre-synaptic spike is produced, but the update Δw is dependent only on the internal state variables of the post-synaptic neuron at the time of the pre-synaptic spike arrival. These internal state variables are the membrane voltage itself, V_mem, and V_Ca, which represents the calcium concentration in the synapse. In an abstract sense, V_Ca represents an integrated version of the neurons past firing history and decays with a slow time constant in the absence of activity. The calcium variable is updated by a fixed increment, when the neuron spikes. Both V_mem and V_Ca in combination determine one of three plasticity states: potentiation (LTP), depression (LTD), and neutral (no update). Besides its use in conventional synapses (Qiao et al., 2015), this rule has been successfully used with memristor-based synapses (Jo et al., 2010), where at the arrival of a spike, a biphasic waveform is applied to the pre-synaptic terminal of the device. The post-synaptic terminal is driven with a waveform that is determined by the two state variables, to either cause a positive or negative voltage across the devices terminals, with a resulting depressing/potentiating effect (Covi et al., 2018; Mostafa et al., 2016). If the neuron is in a neutral state, instead, it will apply a low-voltage pulse, thus inducing no weight update but still allowing to derive excitatory/inhibitory post-synaptic currents. Intriguingly, memristive behavior to implement synaptic functionality can be achieved through biomolecular means as works by Hasan et al., 2018 and Najem et al. (2018) show. Here the authors create conductive pathways by voltage-driven insertion of alamethicin peptides into an insulating lipid bilayer, a process that is used to induce Hebbian and rate-based forms of plasticity. Depending on the chosen protocol for establishing connectivity with the ANN side of the biohybrid, latency can pose a limiting factor in system design. This is particularly true in the case of unknown and variable latency as it presents setups that use the internet for interconnecting the two subsystems, e.g., via the user datagram protocol (Serb et al., 2017). In such cases, it is crucial to select a model of plasticity that is robust against both lossy transmission of spikes (due to packet drops) and variable transmission delays. Here, the applicability of spike-timing based models is limited and the use of more abstract, rate-based models such as the Bienenstock-Cooper-Munro (BCM) rule presents a robust alternative.

Supported by experimental paradigms such as (Dudek and Bear, 1992) (see Table 2 for reference) in BCM, potentiation or depression of a synaptic weight is triggered by pre-synaptic activity x and depending on the post-synaptic activity

$$y=\sum _i{w}_i{x}_i$$

with the synaptic weights w_i. In its original formulation, the weight update is described as

$$\frac{d{w}_i}{dt}=y\left(y-{\theta }_m\right)x_i-\tau w_i$$

Here, τw_i implements a constant decay (Bienenstock et al., 1982). The threshold θ_m determines at what rate the transition from LTD to LTP occurs and is itself modulated by the firing rate such that, e.g., further potentiation of a strong firing becomes gradually harder:

$${\theta }_m={\left(\frac{\overline{c}}{c_0}\right)}^p\overline{c}$$

where $\overline{c}$ denotes the temporal average of the post-synaptic firing and c₀ and p are fixed constants. Figure 3B illustrates the weight updates dependency on post-synaptic activity y and the threshold θ. In their work, Serb et al. (2020) make use of a modified BCM rule where θ_m is an experimentally chosen constant instead of using a variable threshold as originally proposed.

Beyond such methods of LTP that focus on the manipulation of weights in a predefined network topology, the optimization of synaptic resources through structural plasticity can be achieved in neuromorphic circuits as illustrated in George et al., 2017 and Yan et al. (2019). In George et al., 2017 the authors expand an STDP-based learning rule for the simulation of synaptogenesis (here the routing of a new pre-synaptic partner to the synapse circuit connected to the post-synaptic neuron) and for synapse degeneration (the removal of the pre-synaptic partner connection). In the proposed model, the removal of a synapse is triggered whenever the LTP rule approaches a minimum weight value w_ij+Δw<θ. Instead of direct pruning, the synapse is evaluated for the duration of a time period. During this time window, the synapse has the chance to recover to w_ij+Δw>θ; however, it is no longer subject to STDP. Instead, it recovers through coactivation with another synapse onto the same post-synaptic neuron. Whenever an old synapse is pruned, a weak new synapse is formed on the same post-synaptic neuron connected to a randomly chosen source. Its low initial weight increases the chance of removal. In case the synapse connects a completely silent source, e.g., an electrode channel that does not record any spikes, the main driver to its removal is homeosynaptic normalization, which guarantees

$$\sum _{j=0}^N{w}_{ij}=const$$

for all post-synaptic weights w_ij;j = 0,…,N. This is implemented by adding to the smallest weights (LTD) and subtracting from the largest weights (LTP) on the post-synaptic neuron in question. Notably, this work illustrates the combined use of FPGA for flexible reconfigurability and analog processors for their advantages in energy efficiency and speed.

Among other implementable rules mentioned above, the Maple-IC (Mayr et al., 2013) also supports more complex models of plasticity, such as triplet plasticity (Froemke and Dan, 2002) or frequency-dependent STDP (Sjöström et al., 2001). This flexibility is achieved through analog synapse circuits that are interfaced with pre- and post-synaptic waveforms generated by the IC neurons. These allow adaptation mechanisms to impact the weight update dependent on post-synaptic activity as necessary for the implementation of triplet or BCM behavior. The analog weight is periodically sampled and stored in memory for long-term stability. Likewise, the value in memory is converted to an analog post-synaptic current to achieve weight-dependent stimulation of the post-synaptic neuron.

Compared with real-time adaptation in settings that take inspiration from biological mechanisms of plasticity, offline training methods are predominantly utilizing classical methods such as indirect optimal linear estimation using ridge regression (Collinger et al., 2013), e.g., to map the acquired BNN firing rates to the kinematics of an agent, here in the form of a robotic arm. Notably, the strategy employed for the acquisition of training data involves the subject observing and imagining the activity of the autonomously moving actor as it completes the task without being interfaced with the subject. This process evokes a neuronal correlate in the subject's motor cortex, which is then used to form the training set for parameter estimation.

The use of neuromorphic ANNs that support biologically plausible learning rules makes it possible to observe the evolution of BNN and ANN dynamics side by side, forming an important tool for model verification in future research on plasticity and adaptation.

The Overall Biohybrid System

Experimental electrophysiology aims at the creation of interfaces that allow information exchange with BNN through recording and stimulation technologies. As a large and active field of research in itself, it has seen much attention in recent years, resulting in a variety of strategies for the implementation of closed-loop biohybrid systems. The currently most adapted technology for acquisition of biosignals in the field of biohybrid systems is MEAs, due to their compatibility with standard protocols for in vitro preparations, including primary neuronal cell cultures and brain slices. Certain factors beyond the conventional use of MEA need to be considered in this context: Usually evaporation during incubation of the BNN and the resulting destabilization of osmolarity conditions is prevented by ensuring a sufficient volume of solution. This implies a minimum chamber height, which can complicate the combined use of MEA and path clamp approaches for single-cell recording and stimulation (Serb et al., 2020). Similarly, MEA operation and simultaneous optical imaging through water immersion objectives imposes a minimum chamber diameter. Besides cylindrical chamber geometries, funnel-shaped chambers have been employed to limit the culture area to an inner circle centered on the MEA while maintaining a good tolerance to medium evaporation (Giacomello et al., 2011; Schmidtner and Fromherz, 2006). These are particularly beneficial when the number of cells available for plating is a limiting factor. When it comes to the electrode material selection, metal microelectrodes typically used in conventional MEAs, e.g., platinum black, iridium oxide, or gold, are mechanically sensitive and deteriorate over time, which particularly impacts their charge injection capacity in stimulation. A good alternative that is especially common in active Complementary Metal–Oxide–Semiconductor (CMOS) MEA is the use of oxide thin-film insulated capacitive microelectrodes (Serb et al., 2017), which exhibit an increased mechanical stability. These capacitive electrodes are employed in the recording from slices of neuronal tissue where they have been shown to allow the observation of LFP propagation across the recording area (Stangl and Fromherz, 2008). Besides featuring a high temporal resolution, recent CMOS MEAs enable spatial resolutions down to 20 μm or below (Frey et al., 2010; Hutzler et al., 2006; Viswam et al., 2019) and can feature three-dimensional electrodes that are a suitable choice to record from within the slice, bypassing its surface, which has a high likelihood of damage from the cutting procedure (Mapelli and D'Angelo, 2007).

For the application in vitro, silicon-based active neural probes such as NeuroPixels and the Utah array are seeing increased adaptation in the recent years as these technologies have seen broader attention and several systematic reviews; we here point to Soucy et al. (2019) for further reference.

To give a visual overview of the advances in biohybrid systems, and to compare them to developments in neural recording in general, Figure 4 illustrates the primary sources cited in this review. Empty circles describe advances in neural recording, reproduced from Stevenson (2020) with permission. The symbol “⋆” denotes the channels used in stimulation and “•” denotes the channels used in recording in a biohybrid context. The exponential fits (dashed, full line) illustrate the trend in resource increase over the past decade.

Figure 4.

Advancements in the Count of Recording Channels in Biohybrid Systems and in Experimental Electrophysiology in General

Although an increase of both recording and stimulation channels in the past years is visible, the particular challenge in scaling up channel counts in the context of closed-loop biohybrids lies in the scalability of both analog and digital computational resources involved in signal conditioning and processing, necessary for maintaining low latencies in the ANN BNN interaction. Comparing stimulation and recording channel count, it becomes visible that stimulation technologies still lack behind today's recording approaches in terms of number of channels. It can be assumed that this is due to a limitation in selectivity that reduces effective spatial resolution for functional excitation and induces recording artifacts.

Information Processing at the ANN/BNN Interface

Electrophysiologically recorded biosignals can be generally categorized along a scale of invasiveness that is proportional to the spatial and temporal resolution of the acquired signal. On the highest level of invasiveness, intracellular recordings can provide the ANN with detailed information on ion channel dynamics, as Broccard et al. highlight in their review on the topic of neural interfaces (Broccard et al., 2017). Here, the dynamic clamp approach can be used to directly provide the biological neuron's membrane voltage to the artificial side, which feeds back a current resembling excitatory/inhibitory post-synaptic currents of synapse activation, as shown in works like Masson et al. (2002). Due to its use of patch clamp as a means to acquire intracellular voltages, this approach is limited in scalability and restricted to few neurons at most, as it involves the manual application of glass suction pipettes. As a more scalable alternative to single intracellular recordings, particularly in in vitro experiments, MEAs provide a widely adopted interface to establish communication between BNN and ANN through the recording of extracellular single neuron activity.

In MEA, extracellular recordings close to the cell membrane are referenced to an electrically neutral point and allow the observation of localized changes in ion concentration as a result of neuronal activity, referenced to a remote electrically neutral point. An interesting approach that aims at combining the ability to precisely observe changes in potential through intracellular recording with extracellular recordings' superior spatial resolution and usability is found in nanopillar electrodes. These three-dimensional structures are initially enveloped in the cellular membrane without penetrating it and form recording sites that can be used in large arrays. Through the use of electroporation, even a breach of the cellular membrane and subsequent intracellular recording is feasible, as demonstrated by Xie et al. (2012).

Within BNN, action potentials communicate information between numerous computationally limited neurons that operate in a highly parallel and asynchronous fashion. In contrast, conventional von-Neumann computing architectures are operating in a sequential and synchronous manner, commonly at high clock rates in the order of gigahertz and a much smaller degree of parallelism. The available high clock frequencies in conventional hardware influenced the design of electrophysiological signal acquisition systems in that these are typically designed around a limited number of analog-to-digital converters (ADCs) with large sampling frequencies, which are connected to a larger number of pre-amplifiers via analog multiplexers to periodically sample electrode potentials. The resulting multiplexed time-series of raw data is well suited to be processed further by the aforementioned platforms; when, however, the BNN is to be interfaced with neuromorphic ANNs that operate in a more biologically plausible fashion, a translation back into an event-based representation of data needs to be performed. Resulting streams of events can then be used to elicit excitatory or inhibitory post-synaptic potentials in the input layer neurons of the ANN through event-based synapse activation, similar to asynchronous protocols frequently used in the internal communication among ANN neurons (Corradi and Indiveri, 2015). To implement the necessary translation from a multiplexed time-series into digital event streams, spike detection and sorting approaches are utilized, in which recorded action potentials are clustered by similarity of waveform features.

When used for the detection of action potentials, extracellular recordings can be analyzed in real time with dedicated hardware accelerators for spiking detection and sorting (Do et al., 2019; Zamani et al., 2018; Zeinolabedin et al., 2016). In implantable platforms, these allow the extraction of spikes close to the analog recording frontend, which yields the benefit of greatly reducing data rates for downstream interaction with an ANN, as background activity such as local field potentials and subthreshold dynamics are rejected. Under the aspect of energy efficiency and detection accuracy, integer coefficient detectors that convolve the incoming data with an experimentally determined filtering kernel and threshold the result provide valuable alternatives to hard threshold or nonlinear energy operator (NEO) based approaches as authors like (Zeinolabedin et al., 2016) have shown.Where the above-mentioned family of sorting and detection algorithms operates on data acquired through conventional analog-to-digital converters, an intriguing alternative approach is to avoid the intermediate representation as a discrete time-series altogether and directly convert the acquired potentials to an asynchronous event stream with the help of a delta modulator ADC that outputs “UP” and “DOWN” spikes. In this scheme, each of the modulator's output spikes represents the waveform's positive or negative crossing of a user-defined threshold. Compared with sophisticated spike detection schemes, this approach is a more generalized one that directly allows representing arbitrary waveforms as a stream of events. Reconstruction into a time-series is achieved by addition of the known positive or negative spiking thresholds at the point in time when the corresponding event occurs (Corradi and Indiveri, 2015). Although this approach benefits from general applicability to various signals, from LFP to single unit activity, its analog sub-threshold CMOS implementation comes with a susceptibility to device mismatch and nonlinearity in the conversion. Moreover, when compared with conventional signal acquisition, pre-processing steps such as feature extraction and noise rejection have to be achieved in an event-based fashion by the ANN.

As hinted above, electrophysiology allows to extract a number of features beyond action potentials that indicate neuronal activity and can serve as an input to ANN. The chosen feature in biohybrid systems is strongly dependent on the experimental setting. Particularly on the lowest level of invasiveness and spatial detail considered here, potentials recorded through paddle electrodes in the peripheral nervous system and on the spinal cord or the surface of cortex as part of electrocorticograms provide lower-frequency components that are indicative of group activity (Joucla et al., 2016).

These signals express important biomarkers symptomatic of various neurodegenerative diseases, e.g., Parkinson disease (Lehmkuhle et al., 2009), and provide great opportunities for clinical applications. Moreover, especially in the interaction with biological central pattern generators, biohybrid systems profit from this type of recording, as illustrated by Buccelli et al. (2019), where the authors use bursts to trigger spiking activity in the ANN side of their biohybrid. In situations where either spatial or temporal resolution prohibits the focus on single unit activity, signals undergo extensive signal conditioning before feature extraction. Besides computationally expensive Fourier-based or wavelet-based methods of spectral analysis, simple passband filtering has been shown sufficient to detect group activity in some applications (Tayeb et al., 2017). Further equally lightweight methods involve full wave rectification and the filtering with moving averages to obtain a marker for population activity and for Direct Current (DC) offset removal (Joucla et al., 2016; Jung et al., 2001).

Apart from these methods of signal conditioning, dimensionality reduction is predominantly achieved through PCA or independent component analysis (Cunningham and Yu, 2014), with the aim of overcoming bandwidth limitations and noise cancellation. In terms of the technical implementation of interfaces bridging ANN and BNN, bandwidth and scalability are concerns that guide feature design. On the one hand, the computational complexity in the feature extraction needs to be low enough to support a low overhead on latency. On the other hand, a reduction of data at this early stage of processing aids the reduction of bandwidth problems. For the translation from an arbitrary input like above measures of multiunit activity or principal signal components into events, the signal can be encoded as the rate of a Poisson source, which generates the corresponding stochastic stream of events. This step removes the informative content of single spikes and their relative timing but is shown to be well suited to stochastic processing approaches. In an extension of this approach, Schmuker et al. use “receptor neurons” that are modeled by gamma point processes, which take a normalized amplitude as their rate input (Schmuker et al., 2014). To prevent synchronization of the subsequent neuronal circuit, the generated event stream is shifted in time by an initial time to first spike, drawn from a gamma distribution. This approach poses a viable alternative to the classically used Poisson process (Nawrot et al., 2008) due to its more accurate spike statistics and their increased regularity, which is beneficial to subsequent rate estimations, e.g., in rate-based plasticity. In applications wherein timing constraints are relaxed, the representation of time-series as two-dimensional epochs and their processing by non-neuromorphic convolutional neural networks is an approach that has been adopted, e.g., in Nurse et al., 2016.

As a prerequisite for a meaningful interaction between biological networks and ANN in a hybrid system, the ANN has to be reactive to the continuous stream of input from the biological side, but it also has to exhibit sufficient and self-sustained dynamics to retain information on past input, to integrate the biological inputs over time. In ANNs this is achieved through the use of recurrent connections. In this framework, the ANN acts as the reservoir from which information on the input signal can be derived via a linear, trained readout (Lukoševičius et al., 2012). Various methods exist for adapting and optimizing the reservoir for containing maximum information, as discussed by Lukoševičius and Jaeger (2009). Mean-field approaches are another tool for understanding and controlling ANN dynamics and were already applied to biohybrids (Partzsch et al., 2020). Mean-field analysis of a neuronal population assumes that all neurons of the population receive statistically independent input and share the same parameters, at least statistically. Under these assumptions, the mean behavior of a neuron and its variation over neurons can be derived analytically for various neuron models (Renart et al., 2003). This allows to form a quantitative connection from neuron and network parameters to expected population dynamics. Moreover, it produces a qualitative understanding on how to influence those dynamics by targeted parameter changes. This knowledge has successfully been used to control network dynamics in neuromorphic hardware in the presence of significant mismatch effects (Giulioni et al., 2012). Although a wide variety of dynamical regimes can be generated (Brunel, 2000; Mattia and Giudice, 2004), which of these regimes would be most suited for an ANN within a biohybrid system remains an open question. In Partzsch et al. (2020), the proposition was made that the ANN should resemble the dynamics of the biological side, which in this case was the population-burst dynamics of an in vitro cultured neuron population. This approach has been used successfully in the biohybrid of the Coronet project (https://cordis.europa.eu/project/id/284772/) (see Figure 5 for reference).

The translation of ANN activity to appropriate stimulation protocols is as varied as the methods of feature extraction and processing from recordings. The most straightforward approach here is the triggering of single stimulus waveforms whenever an event occurs in the output population of the ANN. This, however, requires the experimental selection of parameters in the stimulation waveform (Buccelli et al., 2019; Jung et al., 2001). With the argument that high stimulation frequencies have adverse fatigue effects Eftekhar et al., 2007 focus on a modulation of the stimulation current amplitude, rather than a direct translation of spike to stimulus. This is, however, only possible in tight boundaries where the stimulus amplitude does not cause harm to the tissue and electrode corrosion is avoidable. The system described by the authors uses a stream of events to generate a stimulus pulse whose current amplitude is linearly related to the number of events. A middle path between a one-to-one translation of events to stimulation pulses and amplitude modulation is formed by letting every event trigger a burst of stimuli as in Joucla et al. (2016). Here, a one-to-one mapping between electrode and artificial spiking neuron is chosen to form the closed-loop biohybrid.

The implementation of the discussed feature-extraction and stimulus-forming methods in hardware enables the scaling of biohybrid systems and promises a level of integration on the ANN side that supports future implantable devices for novel therapeutic applications.

Plasticity in the Closed-Loop Interaction of ANN and BNN

Several of the present works focus on evoking plastic adaptations in exclusively either ANN or BNN, whereas the ability of neuromorphic biohybrid systems to tightly couple these components allows to observe their joint activity on a holistic scale, as ANN and BNN codevelop through mutual influence (Chiolerio et al., 2017). On a temporal scale, the mechanisms employed in adaptive interaction range from short-term plasticity mechanisms in the substitution of central pattern generators in the spinal cord with ANN controllers to the implementation of robust long-term plasticity as shown in the interaction of neuronal cultures with robotic agents that provide sensory input to the BNN, which in turn acts as the driver for the agent actors. Short-term plasticity here comprises, e.g., synaptic depression algorithms such as the activity-dependent depression rule proposed by Tabak et al. (2000). This rule is specifically modeled after developing neuronal networks in the spinal cord, to reflect the rhythmic discharge in recurrent networks of excitatory neurons to establish a cyclic pattern of firing episodes. The population's mean firing rate here determines the fraction of synapses that are not subject to synaptic depression. Likewise, dependent on the mean firing rate, the neuron's firing threshold gradually rises over the duration of the bursting episode, making it gradually harder to fire. Both parameters recover throughout the silent episodes between bursts (Tabak et al., 2000). This particular rule operates on a global metric of activity and is implemented to establish rhythmic firing patterns in biohybrid systems like those published by Ambroise et al. (2013) and Joucla et al. (2016) where it is used to provide cyclic activity patterns in the interaction of FPGA-based ANN with neonatal rat spinal cord.

Several authors make use of other short-term plasticity mechanisms (Buccelli et al., 2019; Keren et al., 2019; Mosbacher et al., 2020), highlighting the benefit that these rules tend to be dependent on pre-synaptic activity alone, which makes them implementable with little computational resources. A further example for models of this class is found in Izhikevich and Edelman (2008) where the synaptic weight is scaled by a factor x that approaches x = 1 with a time constant of τ_x, but can amplify or attenuate the synaptic activation dependent on the choice of a parameter p as

$$\dot{x}=\left(1-x\right)/{\tau }_x\text{if}{S}_{pre}:x\leftarrow px$$

with the pre-synaptic spike S_pre. Buccelli et al. (2019) use this mechanism in combination with the implementation of synaptic noise and the simulation of axonal delays to establish an ANN that is capable of substituting lesioned sub-populations of neurons in vitro. Here, the experimental paradigm is based on a culture of cortical neurons that underwent lesion-induced separation. Bidirectional activity-dependent stimulation is first used to verify the ability to cause correlated activity in the two sub-populations. Subsequently, an FPGA-based ANN implementing 100 Izhikevich neuron models and exhibiting the plasticity mechanisms mentioned above replaces one of the two sub-populations. Statistical analysis of the observed firing in the interfaced sub-population showed correlated activity with both its biological and artificial counterpart, demonstrating a successful bidirectional communication.

An alternative formulation of Short-Term Plasticity (STP) optimized for implementation in digital hardware is found in Noack et al. (2015), where two independent terms for facilitation u and depression R use inter-spike intervals Δt_n as their input and adjust the amplitude of the post-synaptic current:

$$PS{C}_n=A\cdot \left(u_n-R_n\right)$$

$$u_{n+1}=u_n\cdot \left(1-U\right)\cdot {\text{e}}^{\frac{\text{Δ}{t}_n}{{\tau }_u}}+U$$

$$R_{n+1}=\left(\left(1-\alpha \right)\cdot R_n+\alpha \cdot u_n\right)\cdot {\text{e}}^{\frac{\text{Δ}{t}_n}{{\tau }_R}}$$

This approach is first formulated in Markram et al. (1998) and emphasizes biological plausibility in capturing vesicle depletion.

Beyond these short-term adaptive effects that aim at establishing the desired ANN-BNN dynamics, works that explore long-term plasticity are aimed at replacing the function of, e.g., damaged cortical tissue with artificial devices (i.e., cognitive prostheses).

In the use of biohybrids to re-establish the functional connection between a source area in the BNN and the target area it projects to, the pre-lesion acquisition of activity from the target area allows to learn the functional relation between the source area in the physiological case. Post-lesion, the ANN is tasked to re-establish physiological interaction between the areas. To do so, a series of works involve parametric models (multi-input-multi-output-ANN) (Berger et al., 2011; Hampson et al., 2018, 2012). The ANN is composed of individual neuron-like circuits that implement synaptic functionality through a Volterra series of kernels to perform nonlinear system identification and relate the spike-based input to the resulting post-synaptic potential. The neurons output spike-train is captured in a feedback term that likewise affects the neurons firing behavior. Here the kernels used to transform input and output spikes into continuous hidden variables are described by coefficients that are trained with the aim of reproducing the observed input-output relationship. For this purpose, pre-recorded input and output activities are used to fit the kernel parameters following an iterative reweighted least-squares method that yields maximum likelihood estimates of the model coefficients that relate to the interaction of input and output neurons in the case of successful task performance. Although this approach presents a rather abstract neuron model in terms of biological plausibility, the added benefit here is the degree of flexibility in shaping synaptic functionality and the model's strength in predicting output activity.

The performance of this approach is demonstrated in a visual Delayed-Match-to-Sample (DMS) task in the hippocampus of both rodents (Berger et al., 2011) and in primates (Hampson et al., 2012), where prefrontal cortex (PFC) layer 2/3 recordings served as input to the aforementioned algorithm and the ANN output was used to stimulate PFC layer 5. In this intriguing demonstration of a successful employment of neuromorphic biohybrid systems in the field of neuroprosthetics, learning is performed offline on the above-mentioned physiological activity in both neuronal populations pre-lesion. The ANN memory is subsequently loaded with the extracted model parameters to restore task performance in the pharmacologically impaired BNN. Here the fact that modern biologically plausible ANNs as described in the section Spiking and Non-spiking Neural Networks as the Artificial Subsystem allow online learning opens the perspective of extending approaches as those of Berger and Hampson et al. in a real-time adaptive fashion, to increase their robustness against long-term changes in the neuronal firing behavior, e.g., as a response to plasticity induced by continuous functional stimulation and to tissue reactions as a result of electrode implantation.

With the aim of reproducing BNN computation to a higher degree of biological plausibility, authors who are part of the Brainbow (https://cordis.europa.eu/project/id/284772/) project consortium realized a prototype of a neuroprosthetic device able to interact with dissociated cortical neurons from rats plated over a 60-channel MEA (Buccelli et al., 2019). Neurons were plated according to a bimodular layout by means of custom-made polydimethylsiloxane (PDMS) masks, which allowed to create two interconnected sub-populations of cells. During the in vitro development, the biological connections between the two neuronal assemblies give rise to several almost synchronous network bursts. During experiments, a laser ablation of the connections between the two neuronal sub-populations was applied, as an experimental model of focal lesion. The pre-lesion correlation of spiking activity between the two populations was observed to be strong and stable, indicating a functional communication. Following the injury it collapsed to zero, however, thus proving both an anatomical and a functional disconnection. The re-establishment of connectivity was here performed through a neuromorphic ANN using an FPGA, able to perform a real-time burst detection and stimulus triggering as activity-dependent stimulation. Through this bidirectional interfacing, it was possible to partially recover the cross-correlation features without imposing any preferred direction of causation. In a second experiment, one of the BNN sub-populations was entirely replaced by the ANN. By exploiting bursts as the main feature to control the stimulation, it was possible to obtain a partial recovery of the cross-correlation without affecting the firing of the remaining BNN population (see Figure 5 for reference).

The use of long-term plasticity in both BNN and ANN is illustrated in Hogri et al. (2015), where rodent cerebellar microcircuits are used as the BNN and the ANN is modeled after the physiological learning mechanisms involved in learning the timing of discrete movements. The experimental framework consists of an eyeblink conditioning paradigm in rodent that pairs a conditioned stimulus with a delayed unconditioned one. As the animal learns the association of the two stimuli, long-term depression leads to a gradually earlier disinhibition in neurons of the deep cerebellar nuclei, leading to an anticipatory motor response. The hypothesized depression, alongside LTP, was implemented in a mixed-signal ASIC to create a form of Hebbian plasticity that informed the latency and rate of the motor response through functional stimulation. Another example of biologically plausible learning in neuromorphic ANN is given by the project RAMP (https://cordis.europa.eu/project/id/612058), where the consortium relied on remote connections for their interfacing of various systems components. This allowed to create a distributed setup that greatly simplified meeting laboratory conditions necessary for the work with both experimental electronic devices (memristors in this case) and in vitro electrophysiology. The memristor-based synapse setup was here used to interconnect ANN and BNN, and to implement an unsupervised learning to adjust the connection weights between the two subsystems dependent on the firing activity of both sides (Serb et al., 2020) (see Figure 5 for reference).

The works mentioned in this section highlight the applicability of neuromorphic systems in future therapeutic devices to reestablish connectivity after lesions and replace tissue affected by stroke and neurodegenerative diseases.

Discussion

The ability to offer a platform that supports biologically plausible plasticity is one of the features that distinguish neuromorphic biohybrids from conventional neuroprostheses. Neuroprostheses usually are goal oriented in terms of feature extraction from recorded neural signals, implementing a signal processing that is focused on extracting only features relevant for the task at hand. For stimulation, neuroprostheses usually rely on plasticity on the biological side, e.g., retinal implants evoke phosphenes and leave it to human brain plasticity to interpret the signals. In contrast, for both recording and stimulation, biohybrids try to couple significantly deeper into the natural dynamics of the neural tissue. This is illustrated, e.g., by Berger et al. (2011), where the natural signal transformation from a tissue area to its projection is mapped into an artificial device to enhance the communication between both areas. Further examples would be, e.g., unlocking hidden temporal dimensions in the readout (Coronet, Keren et al. (2019)), adapting via long-term plasticity to signal features (Brainbow, Buccelli et al. (2019)) or trying to couple both into short-term signal features via memristors, and simulating structural plasticity to explore the search space for useful biological signals (Ramp, Serb et al. (2020)).

At present, closed-loop biohybrid systems are implemented on various levels of abstraction from dynamic clamp methods in vitro to studies with human subjects. We categorize these works by their focus on different aspects of learning. On the one hand, ANN-centric learning as discussed in the section Biologically Plausible Learning Rules and Their Implementation in ANN is predominantly used to aid the interpretation of BNN signals, e.g., to detect pathological network dynamics and to find application in, e.g., the detection of epileptic seizures, without the focus on inducing changes in the BNN. BNN-centric approaches (see the section The Overall Biohybrid System), on the other hand, aim at training the BNN through feedback stimulation, with the goal of using it as the computational substrate in, e.g., robot navigation tasks and for therapeutic applications, re-establishing physiological neuronal dynamics.

As Table 3 illustrates, at the current state, MEA-based signal acquisition technologies dominate as the chosen frontend to interface ANN and BNN, as they provide a high degree of control and come with high spatial and temporal resolutions in recording, as well as increased selectivity in stimulation, due to the planar nature of dissociated or slice-based BNN. Works like Berger et al. (2012, 2011) and Hampson et al. (2018) that primarily focus on neuroprosthetic applications, however, indicate a general trend toward more complex experimental settings and illustrate a progression from in vitro to in vivo such as intracortical recordings.

Table 3.

Overview of the Different BNN Preparations Cited in This Review

Dissociated Culture, cortical	Bonifazi et al., 2013, Buccelli et al., 2019; Keren et al., 2019; Serb et al., 2020; Chou et al., 2015; Warwick et al., 2010; Mosbacher et al., 2020; DeMarse et al., 2001; Keren and Marom, 2014; Novellino et al., 2007 Marom and Shahaf, 2002; Tessadori et al., 2012
Cortex, in vivo	Bergey et al., 2015, Bouton et al., 2016; Collinger et al., 2013; Nurse et al., 2016; Guggenmos et al., 2013; Hampson et al., 2012
Spinal cord, in vitro	Ambroise et al., 2013, Joucla et al., 2016; Jung et al., 2001
Brainstem	Kositsky et al., 2009; Mussa-Ivaldi, 2010; Reger et al., 2000
Cerebellum, in vivo	Hogri et al., 2015
Globus pallidum, in vivo	Rosin et al., 2011
Hippocampus, in vivo	Hampson et al., 2018
Isolated whole brain	Bonifazi et al., 2013
Slice, hippocampal	Berger et al., 2005
Slice, LGNd	Masson et al., 2002
Dissociated culture, heart neuron	Sorensen, 2004

Along this line, Cottone and colleagues, recently designed a noninvasive stimulation protocol in healthy humans (Cottone et al., 2017), called “transcranial individual neurodynamics stimulation” (iIDS), based on the endogenous dynamics of a target neuronal population. The tIDS effectively changed the excitability of the target pools, thus opening a new avenue for high-efficacy personalized neuromodulation strategies based on individual local neurodynamics.

Like the choice of preparations used as BNN in the context of biohybrid systems, the computational substrate for the implementation of ANN also takes various forms ranging from simulations on general-purpose computers to highly application-specific ICs, as our overview suggests (see Table 4). Although general-purpose platforms are still seeing frequent use, we identify a trend toward application-specific hardware and neuromorphic systems in general. Neuromorphic ASIC ANNs in particular promise low latencies and the ability to process high data-rate recordings for the establishment of truly real-time closed-loop biohybrid systems. Moreover, their energy and area efficiency increases their thermal and mechanical biocompatibility in future implantable devices.

Table 4.

Overview of ANNs and Their Use of Plasticity in the Works Cited in This Review

	Plasticity Rule	Hardware Platform
Bouton et al., 2016	∗	¶
Collinger et al., 2013	†	¶
Hampson et al., 2018	†	¶
Keren and Marom, 2014	†	¶
DeMarse et al., 2001	†	¶
Tessadori et al., 2012	†	¶
Joucla et al., 2016	∗	ǁ
Buccelli et al., 2019	∗	ǁ
Mosbacher et al., 2020	∗	ǁ
Ambroise et al., 2013	∗	ǁ
Khoyratee et al., 2019	∗	ǁ
Levi et al., 2018	‡	ǁ
Keren et al., 2019	∗	∗∗
Nurse et al., 2016	†	∗∗
Hogri et al., 2015	‡	††
Corradi and Indiveri, 2015	‡	††
Serb et al., 2020	§	††
Berger et al., 2012	†	††
Hampson et al., 2012	†	††

Learning Rules: ^∗Spike based STP; ^†Traditional Optimization Methods; ^‡Hebbian Learning; ^§Rate-Based; Hardware Platforms: ^¶Conventional Computer; ^ǁFPGA; ^∗∗Digital ASIC; ^††Analog/Mixed-Signal ASIC.

The plasticity and adaptation mechanisms used in the presented works are strongly dependent on the ANNs' level of abstraction in modeling neuronal tissue, which dictates latency and whether timing or rate-based plasticity is chosen. Besides considerations regarding the ANN, however, neuromorphic biohybrids that aim at faithfully reproducing biological paradigms need to employ those adaptive mechanisms that are also observed in the BNN used, as in the case of biohybrid re-establishment of circular population dynamics in the spinal cord, where STP plays an important role.

As neuromorphic platforms increase the number of available ANN neurons (see Figure 2), a trend in biohybrid systems toward neuromorphic FPGA- and ASIC-based ANN is also visible. These platforms hold great potential of enabling higher throughput and reduced latencies for implementing real-time closed-loop experimental settings. In their role as biohybrid subsystems, neuromorphic ANNs provide a valuable testbed for the re-creation of observed mechanisms of plasticity and learning in BNN, which allows a direct comparison of the dynamics in both subsystems to explore novel hypotheses in neuroscientific research. In terms of biological plausibility it has to be noted, however, that there are features in BNN that are at the current point not captured by neuromorphic ANN. Circuit-based implementations of spiking neurons usually make use of “point-neurons” that do not capture dendritic tree morphology and nonlinear propagation of post-synaptic potentials. This discrepancy forms perspectives particularly for platforms such as SpiNNaker Furber et al. (2013), which through their multiprocessor approach support, e.g., compartment models of neurons.

In terms of meaningful information rate, both neuroprostheses and biohybrids today are limited to at most 10−100 Bits/s and by latencies that limit real-time feedback. As the comparison graphs in this review show, both the capacity of neuromorphic processors to support larger networks and likewise the electrode count in neurotechnology are increasing. This is also aided by the development of interfacing circuitry that does not rely on the traditional analog-digital and digital-analog converters, but directly uses pulses to extract information from biological tissue (Corradi and Indiveri, 2015) and/or pulses for stimulation (Serb et al., 2017). This achieves the triple aim of extracting more relevant information, making interfacing sparser/more information dense, as well as providing a natural exchange language to the neuromorphic component in biohybrids. Collectively, the above opens the pathway to move from today's proofs of concept to next-generation biohybrids that achieve data rates that are comparable with those within BNN. Biohybrids on this scale could finally unlock techniques for meaningful exchange of information with the brain, as a substitute for, e.g., lesioned motor or sensory areas of cortex, and for applications in cognitive enhancement technologies.

As a corollary to that, we would advocate for future biohybrid designs to operate on neuromorphic sensory data, embedding the biohybrid in an interactive environment that provides natural stimuli. Already present work on robotic biohybrids (Kudoh et al., 2011; Mussa-Ivaldi, 2010; Novellino et al., 2007; Warwick et al., 2010) could be extended upon, with a theoretical background that captures how the bidirectional information transfer in biohybrid systems can be progressively enhanced.

Another intriguing perspective is the use of novel devices such as memristive elements. Here, neuromorphic biohybrids promise an application where inherent characteristics such as stochasticity in both weight updates and synapse activation can be exploited. Due to the nanoscale nature of these devices, an unprecedented large number of these stochastic channels can be integrated and form a technology that could impact the implementation of learning for neuromorphic systems in general.

Like in the domains of interfacing and processing in the ANN, biological preparations have seen advances in the past years as well, with the advent of organoids such as those described in Kawada et al. (2017), Kirihara et al. (2019), and Lancaster and Knoblich (2014) featuring three-dimensional network morphologies. These works show a promising step toward BNNs that exhibit an increased similarity to in vivo network morphology and at the same point allow rapid and precise signal acquisition in future biohybrid systems. Likewise, pluripotent stem cells (Frega et al., 2019) hold the potential to increase the availability of human-derived tissue for use in the context of biohybrids, ultimately making these systems a valuable tool in studying human neuropathology and electrophysiological approaches to rehabilitation and treatment.