Plasticity in memristive devices for spiking neural networks

Abstract

Memristive devices present a new device technology allowing for the realization of compact non-volatile memories. Some of them are already in the process of industrialization. Additionally, they exhibit complex multilevel and plastic behaviors, which make them good candidates for the implementation of artificial synapses in neuromorphic engineering. However, memristive effects rely on diverse physical mechanisms, and their plastic behaviors differ strongly from one technology to another. Here, we present measurements performed on different memristive devices and the opportunities that they provide. We show that they can be used to implement different learning rules whose properties emerge directly from device physics: real time or accelerated operation, deterministic or stochastic behavior, long term or short term plasticity. We then discuss how such devices might be integrated into a complete architecture. These results highlight that there is no unique way to exploit memristive devices in neuromorphic systems. Understanding and embracing device physics is the key for their optimal use.

Introduction

In 1971, Leon Chua indicated the possible existence of a fourth basic electrical component (Chua, 1971). This component, the memristor, would complement those already known namely resistance, capacitor, and inductor, and offer new opportunities for system design (Chua and Kang, 1976). In particular, Chua proposed to use memristors or similar memristive devices to fabricate synapses and neurons following the Hodgkin–Huxley formalism. From this theoretical work, several publications have cited the memristive phenomenon without naming it as such and without linking it to Chua's theory (Upadhyaya and Chandra, 1995; Lau et al., 2004; Waser and Aono, 2007; Wu et al., 2007; Pershin and Di Ventra, 2008). HP labs were the first to recognize a device as a memristor in 2008 (Strukov et al., 2008), and they highlighted both the technology and its possible applications.

In parallel, the designers of the neuromorphic community worked hard on achieving CMOS neurons to reach electrical energy consumption of the order of picojoule per spike (Wijekoon and Dudek, 2008; Livi and Indiveri, 2009; Rangan et al., 2010; Merolla et al., 2011; Joubert et al., 2012). However, if the neuron implementation still have to face important challenges to match the neurons density and functionality required for neuromorphic circuits, the most abundant element in a neural network is the synapse. Consequently, most of the efforts have been concentrated on achieving high density memories with embedded synaptic functionalities (i.e., synaptic plasticity) in a single component. To become functional, the realization of a plastic synapse requires three parts: (i) synaptic weight storage, (ii) circuit for updating this weight depending on the network activity, and (iii) circuit for information transmission between two neurons. The neuromorphic community has developed a strong interest in memristive devices because these nanodevices and the associated integration strategies offer potential solutions to realize these three functions.

Resistive Random Access Memory (ReRAM) technologies in its broad sense have been developed for pure memory applications but can fall into the memristive system classification (Baek et al., 2004; Lee et al., 2008; Wong et al., 2012). These different technologies are mostly used in binary mode and are at the stage of industrialization and commercialization (e.g., ReRAM from Panasonic and Samsung) with high endurance, low energy, and high integration capability performances (Kawahara et al., 2012; Liu et al., 2013). Such performances can be an interesting platform for the implementation of synaptic weight storage (even in binary mode) if integration strategies and specific architectures are developed in order to offer a suitable solution to the large access required between neuron (i.e., computing node) and synapses (memory) inherent to parallel computing in neuromorphic circuits (and unsolved by Von Neumann architectures and associated bottleneck). In addition, their use in analog mode (or multilevel), is the subject of great attention and could be an effective solution for the implementation of synaptic functions.

Defining a memristor itself (see Figure 1A) can be debatable. Leon Chua now defines a memristor as any element that has an I(V) curve pinched at 0 V (Figure 1B) (Chua, 2014). This definition is widely used in the literature for characterizing devices, and in this paper we synonymously use the historic word memristor or the more generic “memristive device.” A general feature of memristive devices is to offer a non-volatile modification of its resistance (or conductance) as a function of the current (charge) or voltage (flux) driving the device. In particular, neuromorphic circuit designers prefer to think of memristors as resistive components that have the following properties: (i) the greater the electrical charge that has passed through the component, the more the resistance value decreases, (ii) the resistance value is stored in the element even after it is turned off. Moreover, this modification appears if the charge through the memristor goes over a “threshold” (Figure 2D).

Figure 1.

(A) Symbol of memristor; (B) characteristic transport features of memristors: pinched iv loops for different values of the maximum injected current.

Figure 2.

(A) Experimentally measured STDP function ξ_i(ΔT) on biological synapses (data from Bi and Poo, 1998, 2001). (B) Ideal STDP update function used in computational models of STDP synaptic learning. (C) Anti-STDP learning function for inhibitory STDP synapses. (D) Shape of memristor weight update function f(v_MR). (E) Spike shape waveform.

Memristors can be realized using several technologies and we can categorize these technologies in four large families. The first includes anionic and cationic Red-Ox devices operating on Oxidation–Reduction principles. The second is phase-change memories (PCM), where resistive switching is connected with a physical phase change. Organic elements represent the third family. The fourth family finally comprises elements using purely electronic effects such as ferroelectric tunnel and spintronic memristors. These technologies possess different behaviors and therefore different fields of application. As part of this paper about synaptic plasticity, we also point out that these technologies will lead to different plastic behaviors and learning rules. These differences enrich the palette of possibilities for neuromorphic design. As Jeong et al. (2013), the purpose of this paper is not to present an exhaustive list of memristive technology and of their associated behavior, but rather to present the different forms of learning that have been observed. In our paper, all data about memristive devices have been measured by at least one of the co-authors.

If computing and memorization principles in neural networks are not completely understood, it is now widely recognized that learning in such systems is associated to synaptic weight modification that tends to reinforce or depress the strength of the connection between two neurons and grouped into the wide class of synaptic plasticity. The most popular description of learning was proposed by Hebb with the postulate “who fire together, wire together” (Hebb, 1949). In other words, two neurons presenting a correlated activity will tend to reinforce their synaptic connection. A first requirement is to define what we call neuron activity: two different approaches are commonly used, (i) rate coding strategies correspond to the definition of neuron activity as the mean firing rate estimated on a chosen time window while (ii) temporal coding corresponds to the assignment of neuron activity to a single spike event with a given time stamp with respect to the other spiking neurons considered in the network. Based on this different coding strategies, variations of Hebbian learning have been proposed such has Spike Rate Dependent Plasticity (SRDP) or the very popular Spike Timing Dependent Plasticity (STDP). In particular, STDP has attracted a large interest in the memristive device community because of its practical implementation based on overlapping pulses coming from the pre and post neurons. We present in Section STDP Learning Thanks to Overlapping Events theoretical elements that allow the understanding of the application of this basic learning algorithm. Starting from this ideal case, we present practical implementations of STDP in solid state devices and show how material constraint (i.e., switching mechanism, operating conditions, …) can be used to realize various form of STDP. Then we present two cases of “ferroelectric” memristors based on thin film semiconductor-metal-metaloxide compounds. These compounds were some of the first materials to be used as memristive synapses (see Kuzum et al., 2013 for a review). The first of our ferroelectric memristors is based on several 100 nm thick BiFeO₃ films experiencing resistive switching in the Schottky barrier formed with one of the contacts. Specifically, the memristive effect in these devices is effected by a change of the depletion layer of the Schottky diode due to a non-volatile charge transfer similar to the “moving barrier” of TiO₂. The second consists of ferroelectric tunnel junctions of very thin (~1 nm) BiFeO₃ films in which tunneling resistance is linked to the polarization of the barrier. They differ radically by the time scales on which they operate and thus by the contexts in which they could be used. A third case based on spin-transfer torque magnetic tunnel junction is also presented in Section Spin-Transfer Torque Magnetic Tunnel Junction as a Stochastic Synapse. It presents a stochastic behavior in learning which is in some ways reminiscent of biological neural networks. In Section SRDP with Memristive Devices, we present different form of SRDP observed in biological synapses and of interest for spike rate coding strategies. We first show how Short Term Plasticity, corresponding to a temporary modification of the weight that tends to relax toward a resting state, can be used to implement rate dependent modification of the weight. A second example describes how Short Term/Long Term plasticity transitions can be reproduced by taking advantage of device stability characteristics. Before the conclusion, Section Toward Memristor-CMOS Architectures and Circuits opens the discussion on the characteristics of circuit architectures that will drive memristors following their electrical behavior.

STDP learning thanks to overlapping events

Theoretical principles

STDP is the ability of natural or artificial synapses to change their strength according to the precise timing of individual pre- and/or post-synaptic spikes (Gerstner et al., 1993, 1996; Markram et al., 1997; Bi and Poo, 1998, 2001; Zhang et al., 1998; Feldman, 2000; Mu and Poo, 2006; Cassenaer and Laurent, 2007; Jacob et al., 2007; Young, 2007; Finelli et al., 2008; Masquelier et al., 2008, 2009). A comprehensive overview of STDP and of its history can be found elsewhere (Sjöström and Gerstner, 2010). STDP learning in biology is inherently asynchronous and on-line, meaning that synaptic incremental update occurs while neurons and synapses transmit spikes and perform computations in parallel. Early proposals of this used artificial time-multiplexing to alternate continuously and synchronously between “performing” and “weight update” phases (Snider, 2008), thus requiring global system-wide synchronization. This can become a severe handicap when scaling up systems. Another option is a fully asynchronous implementation for memristor-based STDP where “performing” and “weight update” phases happen simultaneously in a natural manner, as in biology (Linares-Barranco and Serrano-Gotarredona, 2009a,b; Zamarreño-Ramos et al., 2011; Bichler et al., 2012b; Kuzum et al., 2012), and where there is no need for any global synchronization.

Figure 2A shows the change of synaptic strength (in percent) measured experimentally from biological synapses as function of relative timing ΔT = t_pos − t_pre between the arrival time t_pre of a pre-synaptic spike and the time t_pos of the generation of a post-synaptic spike. Although the data shows stochasticity, we can infer an underlying interpolated function ξ(ΔT) as shown in Figure 2B.

$$\xi \left(\Delta T\right)=\{\begin{array}{c}{a}^{+}{e}^{-\frac{\Delta T}{{\tau }^{+}}}if\Delta T>0\\ -a^{-}{e}^{-\frac{\Delta T}{{\tau }^{-}}}if\Delta T<0\end{array}$$ (1)

For a causal pre- to post-spike timing relation (ΔT > 0) the strength of the synapse is increased, while for an anti-causal relation (ΔT < 0) it is decreased. In the case of synapses with negative synaptic strength (as in some artificial realizations), the reversed version shown in Figure 2C can be used. Microchip CMOS circuit implementations of STDP rules that follow the description of Equation (1) have been reported (Indiveri et al., 2006), which result in about 30 transistors per plastic synapse, and thus may lead to high costs for their hardware realization. There is, overall, general thinking that STDP is very expensive to implement in conventional CMOS microchips (Fieres et al., 2008; Khan et al., 2008). However, it can be implemented with just one memristor per synapse if appropriate peripheral signal conditioning neurons are used in hybrid CMOS/memristor realizations.

For our purpose, we will consider a particular type of memristors, named voltage/flux driven memristor, which can be mathematically defined by.

$$\begin{array}{c}{i}_{MR}=G\left(w,v_{MR}\right)v_{MR}\\ \dot{w}=f\left(v_{MR}\right)\end{array}$$ (2)

Memristor current and voltage are in general related through a non-linear conductance G (in the i_MR vs. v_MR plane), whose shape is tuned by parameter w. Most of the times, however, we may approximate the conductance as being totally linear i_MR = G(w)v_MR, where the value of w is dependent on the history of v_MR. Parameter w represents some structural property of the memristor. This parameter changes non-linearly as a function f( ) of the evolution of the memristor voltage v_MR, so that the derivative of w is governed by the second equation in (Equation 2). A typical shape of this function is shown in Figure 2D, where a “dead zone” between two threshold voltages is present. While the memristor voltage is kept within this dead zone, parameter w will remain constant, and G will not change. But if the memristor voltage goes out of the dead zone, the (linear or non-linear conductance G) will change.

The STDP learning rule (as modeled by Equation 1) can, in theory, be implemented by (i) using a particular type of voltage/flux driven memristor (Jo et al., 2010), while (ii) providing appropriately shaped pre- and post-synaptic spikes available at both synapse (memristor) electrodes (Zamarreño-Ramos et al., 2011). For example, we can consider a pair of identical pre- and post-synaptic spikes with a shape resembling that of biological spikes (see Figure 2E), with an on-set duration |t⁺_ail| and a tail of duration |t⁻_ail|,

$$spk\left(t\right)=\{\begin{array}{cc}{A}_{mp}^{+}\frac{e^{\frac{t}{{\tau }^{+}}}-e^{-\frac{t_{ail}^{+}}{{\tau }^{+}}}}{1-e^{-\frac{t_{ail}^{+}}{{\tau }^{+}}}}& if-t_{ail}^{+}<t<0\\ -A_{mp}^{-}\frac{e^{-\frac{t}{{\tau }^{-}}}-e^{-\frac{t_{ail}^{-}}{{\tau }^{-}}}}{1-e^{-\frac{t_{ail}^{-}}{{\tau }^{-}}}}& if0<t<t_{ail}^{-}\\ 0& ifotherwise\end{array}$$ (3)

Under these circumstances, memristor voltage is v_MR(t, Δt) = α_pos spk(t) – α_pre spk(t + Δt) and synaptic strength change can be computed as.

$$\Delta w\left(\Delta T\right)={\displaystyle \int f}\left(v_{MR}\left(t,\Delta T\right)\right)dt=\xi \left(\Delta T\right)$$ (4)

which has been shown to result in the same shape illustrated in Figure 2B (Zamarreño-Ramos et al., 2011). Furthermore, by reshaping the spike waveform, one can fine tune or completely alter the STDP learning function ξ(ΔT), as illustrated in Figure 3. This way, by building neurons with a given degree of shape programmability, it is possible to change the STDP learning function at will, depending on the application, or make it evolve in time as learning progresses.

Figure 3.

Illustration of influence of action potential shapes on the resulting STDP memristor weight update function ξ(ΔT). Memristor upper and lower thresholds are normalized to amplitudes ±1.0. From (A1,A2) to (E1,E2) the same spike waveform travels forward and backward. In (F1,F2) the forward and backward waveforms are the same but have opposite polarity. In (G1,G2) to (H1,H2) the forward and backward waveforms are different. In (G1,G2), the positive pulse of the backward waveform exceeds amplitude +1.0, thus producing negative STDP update whenever there is a post-synaptic spike alone (G2); otherwise if pre- and post-synaptic spikes happen within a given time window, there will be positive STDP update.

Figure 4A shows a way of interconnecting memristors and CMOS neurons for STDP learning. Triangles represent the neuron soma, the flat side indicating its input (dendrites) and the sharp side its output (axon). Dark rectangles are memristors, each representing one synaptic junction. Every neuron controls the voltage at its input (V_post in Figure 4B) and output (V_pre in Figure 4B) nodes. When the neuron is not spiking it forces a constant voltage at both nodes, while collecting through its input node the sum of input synaptic spike currents coming from the memristors, which contribute to changing the neuron internal state. When the neuron spikes, it sets a one-spike waveform at both input and output nodes. This way, they send their output spikes forward as pre-synaptic spikes for the destination synaptic memristors, but also backward to preceding synaptic memristors as post-synaptic spikes. Zamarreño et al. showed extensive simulations on these concepts, and how one can change from STDP to anti-STDP by switching polarities of spikes or memristors (Zamarreño-Ramos et al., 2011). For example, (Figures 3F1,F2) illustrate the case where forward and backward spikes have opposite polarities, resulting in a symmetric STDP update function ξ(ΔT). Figures 3G1,G2 illustrate an example where forward and backward spikes are different, with the backward spike such that its positive part exceeds the positive memristor threshold (v_th = 1.0). This produces LTD (long term depression) or negative STDP update whenever there is a post-synaptic spike sufficiently apart from a pre-synaptic one; and produces LTP (long term potentiation) if pre- and post-synaptic spikes happen within a given time window (Bichler et al., 2012a,b). Figures 2H1,H2 illustrate a similar STDP update behavior, except that the update (whether positive or negative) is restricted to a limited time window.

Figure 4.

(A) Example of Memristors and CMOS neuron circuits arrangement for achieving STDP learning: feed-forward neural system with 3 layers of neurons and two fully connecting synapse crossbars. (B) Details of parts around one post-synaptic neuron. While a neuron is silent, it sets a constant DC voltage at its input (V_post) and output (V_pre) nodes. When a neuron is sending a spike, it sets a voltage spike at both nodes.

If the system is structured into neural layers (for example, Figure 4A shows a 3-neuron-layer system) with memristive synapses in between, then for each layer all pre-synaptic neurons should have the same forward spike shape and all post-synaptic neurons should have the same backward shape. This way, all memristive synapses between these two neural layers will have the same STDP function ξ(ΔT).

Waveform-defined plasticity in ferroelectric resistive switching memristors

In this section, we concentrate on an analysis of resistive switching BiFeO3 (BFO). Our BFO memristors are grown by pulsed laser deposition on Pt/Ti/SiO2/Si substrate with a circular Au top contact (Shuai et al., 2013), see Figure 5A. The BFO films have a thickness of some 100 nm. The top contact forms a Schottky diode, causing the created devices to show resistive switching with a rectifying behavior (Shuai et al., 2011). The devices exhibit a combination of voltage- and charge-driven behavior, and are consistent with the requirements of Section Theoretical Principles. When stepping DC voltages across the device, the resistance will follow an exponential curve (Mayr et al., 2012). The voltage level defines the converged resistance value, while the charge passed through the device defines the time frame until this converged value is achieved.

Figure 5.

(A) Layout/processing of BiFeO3 devices used (Shuai et al., 2013); (B) driving voltage waveforms (from top to bottom): pre-synaptic waveform, post-synaptic waveform, resulting differential voltage across memristor and resulting memristance change shown as percentage change in current through the memristor for a fixed 2 V measurement voltage (Cederstroem et al., 2013); (C) measured STDP curves for two different STDP time window settings; time windows are adjusted via the time constants of the exponentials slopes of pre- and post-synaptic waveform, which changes the LTP respectively the LTD part of the STDP window; Weight change as change in current through the memristor; (D) measured spike triplet curves (Froemke and Dan, 2002), weight change as change in current through the memristor (Mayr et al., 2012).

Resistive switching in BFO shows a number of characteristics which make it well-suited for use as a synapse. For instance, the dependence between voltage level and converged resistance makes the BFO devices conform closely to the ideal waveform-driven plasticity postulated in Figure 3, as plastic changes in the memristor closely follow the overlapping pre- and post-synaptic waveforms. Up to 8 bit analog resolution can be reliably programmed in the device (Shuai et al., 2013). Due to the Schottky diode, there is also high-ohmic region up to 1 V. Similar to the paradigm of Linn et al. (2010), this can be used in an array of BFO devices to define a voltage readout-region where only a single device in the array is active, eliminating the multiple sneak current paths that would otherwise severely limit practical array size (Flocke and Noll, 2007). While this characteristic potentially enables large crossbar arrays of BFO devices, defect density is on the order of 30% for an “open circuit” type failure, so a placement algorithm (Mayr et al., 2007) would have to be used in a memristive array to map around defect memristors.

The devices also experience a modification threshold at ca. 2 V, i.e., starting from the Schottky diode threshold at 1 V up to 2 V, the memristance can be measured by the current flow, but the charge inherent in this current does not change the memristance. If appropriate waveforms are chosen, the 2 V threshold extracts pre- and post-synaptic activity correlation as memristance change, as postulated in Section Theoretical Principles. All these voltages are broadly compatible with CMOS logic processes, in contrast to other material choices that need significantly higher voltages (Kuzum et al., 2013).

The waveforms in the upper two curves of Figure 5B are used as pre- respectively post-synaptic voltage. Those curves have not been shown in Figure 3; however their asymmetry is in the spirit of Figures 3G1,H1. These waveforms implement the plasticity model of Mayr et al. (2010), which allows for both rate- and spike-based plastic behavior. In the third curve of Figure 5B, which shows the resulting differential voltage across the memristor, the modification thresholds at about 2 V are marked. As can be seen, these are crucial in permitting modification only for true pre-post coincidences (such as at 30 ms), filtering out single pre- or post-synaptic events (such as at 20 ms). The resulting synaptic modification is shown in the last curve of Figure 5B, exhibiting a close match with the theoretical model (Mayr et al., 2010).

Measured STDP curves using this paradigm are shown in Figure 5C. With their exact reproduction of the waveform-defined exponential time window, they showcase the capability of BFO synapses for fine-grained analog weights. In most current memristive materials, the STDP curves deviate significantly more, and their time windows are primarily defined by the physical device characteristics, not the driving waveform (Alibart et al., 2012; Kuzum et al., 2013). In contrast, the voltage-memristance relationship of the BFO synapses lets them conform nicely to the waveform-defines-plasticity paradigm postulated in theory (Zamarreño-Ramos et al., 2011). Through this direct translation of the driving voltage waveforms into the plasticity shape, different time windows can be easily configured via the pre- and post-synaptic waveforms, as can be seen from the two sample curves in Figure 5C.

By introducing adaptation into the post-synaptic waveform, specifically an exponential dependence of the post-synaptic action potential duration on the inter-spike interval, the plasticity rule of Mayr and Partzsch (2010) is also able to reproduce triplet and rate plasticity (Froemke and Dan, 2002). When exploring the triplet paradigm with memristors, a faithful reproduction of biological triplet data can be seen (Figure 5D), due again to the excellent correlation between driving waveform and evoked memristive plasticity. The post-synaptic adaptation introduced for triplet plasticity can be observed in the different pulse widths in the second curve in Figure 5B (Noack et al., 2010).

Defining the plasticity entirely through the waveform can also be used to substantially speed up synapse behavior in BFO up to a 50 μs time scale (You et al., 2014). A switched capacitor system such as (Mayr et al., 2014b), if equipped with a scalable time base (Eisenreich et al., 2009), also offers the intriguing possibility of operating a high-density, CMOS-memristor hybrid neuromorphic system at varying timescales to accommodate different tasks, such as real-time interoperation with a visual sensor vs. offline, high-speed classification tasks where an accelerated timescale leads to faster classification.

High-speed plasticity in ferroelectric tunnel memristors

“Purely electronic” memristors are nanodevices in which the resistance changes are obtained through electron mediated phenomena at interfaces. These memristors promise an increased endurance and reliability, since the material structure is preserved, as well as a faster switching speed.

The “ferroelectric tunnel memristor” (Bibes et al., 2010) is based on an emerging digital memory concept, subject of intense academic and industrial developments, the ferroelectric resistive RAM (International Technology Roadmap For Semiconductors, 2011). Its base is the ferroelectric tunnel junction (FTJ): an insulating ultrathin (several nanometers) ferroelectric barrier sandwiched between two metallic electrodes (Figure 6A). Strain from the substrate assures that the ferroelectric polarization points to one of the electrodes. The polarization can be switched upon application of short voltage pulses and results in resistance changes of up to several orders of magnitude (Garcia et al., 2009; Chanthbouala et al., 2012a). This resistance contrast is linked to different polarization screening in the electrodes: the effective tunneling barrier height dependents on the direction of the ferroelectric layer's polarization and therefore strongly influences the tunneling current. Additionally, the strong non-linearity of the ferroelectric tunnel junction allows for a non-destructive resistance reading at low DC voltage.

Figure 6.

(A) Optical microscope image of the chip after patterning showing 5×10 ferroelectric tunnel junctions (FTJ); (B) 3D representation of a zoomed area containing a few FTJs. The three parallel bars are the ground-signal-ground contact pads; (C) 3D sketch of one FTJ (Boyn et al., 2014); (D) schema of the voltages applied to the memristor. The reading pulse V_read is lower than the threshold (V_read = 200 mV). Writing is performed by the application of 100 ns voltage pulses of different amplitudes. The writing voltages increase from −2 V to V_max by a step of 0.1 V. Then, the amplitude of the writing pulses decreases to −V_max; (E) dependence of the resistance of the ferroelectric tunnel memristor measured at V_read on the applied writing cycles. The different curves correspond to different consecutive measurements with varying V_max.

By designing the devices in such way that the switching occurs through non-uniform ferroelectric domain configurations, quasi-analog resistance variations can be obtained (Chanthbouala et al., 2012b). A direct link between these intermediate resistance states and the ferroelectric domain configuration allows the description of its dynamic behavior through models of domain nucleation and growth in ferroelectric films. Furthermore, the cumulative behavior upon application of trains of voltage pulses has already been demonstrated. As the polarization reversal process in the ferroelectric film depends on pulse amplitude and duration, these parameters can be adapted to achieve the desired resistance change in the memristive device—a very promising feature for the implementation of STDP-based learning with ferroelectric tunnel memristors (Chanthbouala et al., 2012b).

It has recently been demonstrated that fully-patterned solid-state ferroelectric tunnel memristors based on BiFeO₃ (fully patterned submicron Co/BiFeO₃/Ca_0.96Ce_0.04MnO₃ tunnel junctions) can be produced with high yield and with low device-to-device variations. They show resistance contrasts of more than 3 orders of magnitude, can be commuted with pulses of 100 ns and amplitudes of about 2 V, and have a large endurance of over 4 × 10⁶ cycles (Boyn et al., 2014).

In Figure 6, we plot as in Yamada et al. (2013) the multilevel behavior of a ferroelectric tunnel memristor depending on applied voltages. The curves in Figure 6B show the DC resistance value of the device after writing pulses of different amplitudes. To use this memristor as a plastic synapse we consider −V_MR to represent the time difference ΔT = t_post − t_pre. Then ΔT > 0, i.e., V_MR < 0 in Figure 6B, implies increasing conductance that corresponds to Hebb's rule. Conversely, ΔT < 0 results in a decrease of the synaptic weight.

Choosing the waveform of Figure 3B1 for pre- and post-synaptic voltage neurons, the width of the positive square pulse can be as low as 100 ns in the case of the ferroelectric tunnel memristor. Accordingly, the ramp phase of the waveform will be a few times larger than this. As a result, the time difference between spikes for the STDP shown in Figure 3B2 can be less than 1 μs.

Spin-transfer torque magnetic tunnel junction as a stochastic synapse

Spin-Transfer Torque Magnetic Tunnel Junctions (STT-MTJs) constitute another choice to implement plastic non-volatile synapses. They rely on a different operating mechanism than the devices presented in the rest of the paper, and for this reason are not always thought as memristive devices. Their specific stochastic behavior, however, can be particularly interesting for synaptic applications. And as they constitute the basic cell of the second generation of Spin Transfer Torque Magnetic RAM (STT-MRAM)—which is currently reaching the market—, they present a high level of CMOS compatibility and of maturity.

The basic structure of a STT-MTJ is presented in Figure 7A and is constituted by an ensemble of layers of different materials. The magnetic “fixed” layer is a small magnet whose magnetization is pinned in one direction. The magnetic “free” layer is a thinner magnet whose magnetization can be either parallel (P) or antiparallel (AP) to the one of the fixed layer. Due to the Tunnel Magnetoresistance effect, the electrical resistance of the P and AP state is different. And due to the Spin Transfer Torque effect, a positive current can switch the device from AP to P state, and a negative current can switch the device from P to AP state. This leads to the I–V curve seen in Figure 7B, which is reminiscent of a memristive device. However, MTJs are truly binary device: AP and P states are the only possible states. Some proposals exist to increase the number of states (Lou et al., 2008) or to include another physical effect (domain wall motion) in the MTJ to reach multilevel behavior (Wang et al., 2009; Chanthbouala et al., 2011). However, these variations do not exhibit the same degree of maturity as binary STT-MTJs. In comparison with traditional memristive devices, STT-MTJs are fast to write (programming can be as fast as 1.5 ns) and possess outstanding endurance (switching the free layer magnet is not associating with an aging mechanism). Their main drawback is a relatively high fabrication cost and a low R_OFF/R_ON ratio. STT-MTJs should be associated with different CMOS circuits than other memristive devices for this reason (Zhang et al., 2014).

Figure 7.

From Vincent et al. (2014, 2015). (A) Cartoon of a Spin Transfer Torque Magnetic Tunnel Junction (STT-MTJ). (B) Typical I–V curve of the STT-MTJ. (C) Experimental measurements of stochastic switching. (D) Model of mean switching time as a function of programming current. (E) Our simplified STDP rule and PRE and POST overlapping pulses which implement it naturally with STT-MTJs.

Additionally, a specificity of STT-MTJs, of special interest for synaptic applications, is that switching is stochastic. When one applies a programming pulse, a STT-MTJ has only a probability to switch state, which is independent of the STT-MTJ's history: every time the programming pulse is applied, the STT-MTJ has the same probability to switch state. This is well-seen on the experimental measurements of Figure 7C on devices of Devolder et al. (2008), Marins de Castro et al. (2012). The switching probability can be controlled by programming voltage and pulse duration. The basic physics behind this effect is well-understood (Diao et al., 2007; Devolder et al., 2008) and we have recently developed a comprehensive analytical model of it for circuits and systems designers (Vincent et al., 2015). As seen in Figure 7D, a striking feature is that the mean switching time of the STT-MTJ can be adjusted over many orders of magnitude by choosing the programming current. It has also been proven that STT-MTJ stochastic switching can be used to generate high quality random numbers that pass standardized statistical tests qualifying true random number generators (Fukushima et al., 2014). Stochastic switching can also be adjusted by layout of the junctions (size and eccentricity).

STT-MTJs are suitable for implementing a stochastic version of STDP that has been studied in several recent works (Kavehei, 2013; Suri et al., 2013; Yu et al., 2013; Vincent et al., 2014). They exploit, at the system level, a functional equivalence (Goldberg et al., 2001) that exists between multi-level deterministic synapses and binary probabilistic synapses. When a long term potentiation or depression occurs, instead of changing the conductance of the synapse partially, stochastic STDP has a small probability of changing it totally. And if several STT-MTJs are connected in parallel, a multibit synapse can be emulated. Since STT-MTJs have no internal dynamic besides stochastic switching, stochastic STDP can be implemented using similar strategies to the one used for ferroelectric devices. Only the behavior at the system level will be different.

In our works, we have been working with a stochastic version of the simplified version of STDP which is theorized in Nessler et al. (2013) and also used in Bichler et al. (2012a), Suri et al. (2013), Querlioz et al. (2013), and similar to the one of Figures 3G1,G2. A possible implementation with STT-MTJs is summarized on Figure 7E. It relies on overlapping pulses, but with clear separation of transmission and programming operation (Suri et al., 2013; Vincent et al., 2014). Although very simple, this STDP rule can lead to complex machine learning tasks like learning to detect cars on a video (Vincent et al., 2014). Additionally, we have observed that it is surprisingly robust to STT-MTJ variability (Vincent et al., 2014). However, this is just an example and other forms of STDP may be implemented with STT-MTJs if one accepts their stochastic nature.

SRDP with memristive devices

The learning process described in the previous section has been implemented in a large variety of solid state memory devices with non-volatile characteristics. However, if we consider the synaptic plasticity mechanisms observed in biological computing systems, modification of the synaptic efficiency (evaluated by measuring the transmission of a single spike and equivalent to the synaptic weight) can be either permanent (i.e., lasting for months to years) or temporary (i.e., relaxing to its initial state with a characteristic time constant in the milliseconds to hours range). This observation leads to the definition of Long Term Plasticity (LTP) and Short Term Plasticity (STP), respectively. We can notice that the boundary classification into Long Term (LT) and Short Term (ST) effects is not well-defined and should be considered with respect to the required task. Both STP and LTP can correspond to an increase or decrease of the synaptic efficiency thus leading to the definition of Short Term (Long Term) potentiation and depression, respectively. In biology, synaptic plasticity can be attributed to various mechanisms involved in the transmission of the signal between a pre- and post-neuron, such as neurotransmitter release modification, neurotransmitter recovery in the pre-synaptic connection, receptors sensitivity modification or even structural modification of the synaptic connection (see Bliss and Collingridge, 1993), for a description of the different mechanisms involved in STP and LTP). Based on this observation, two important points need to be stressed. First, STP and LTP processes are not restricted to a particular learning strategy (i.e., STDP and SRDP, for example). In this section, we present examples of STP and LTP processes based on a particular case of rate coding strategy but these considerations are still valid for other coding strategies (see Alibart et al., 2012, for STDP with STP devices). Secondly, if plasticity is intimately linked to the notion of learning, it is important to notice that there is no one-to-one equivalence between the concepts of STP, LTP and the notion of Short Term Memory (STM) and Long Term Memory (LTM). Indeed, even if a direct parallel has been proposed based on the particular concept of memory consolidation (Lamprecht and Ledoux, 2004), which corresponds to accumulation of Short Term effect leading to Long Term memory, there are still very important questions to be answered about how learning (and the associated synaptic plasticity) is related to the memorization of information that can also present different time scale from milliseconds to years.

Short term plasticity (STP)

Implementation of STP has been proposed in a variety of nanoscale memory devices. The first proposition of STP was realized in a nanoparticles/organic memory transistor (NOMFET)—Figure 8 (Alibart et al., 2010). The basic principle of this device is equivalent to a floating gate transistor. Charges are stored in the nanoparticles and modify the channel conductivity via Coulombic repulsion between the carriers (holes) and the charged nanoparticles. The particularity of this device is to present a leaky memory behavior: charges stored in the nanoparticles tend to relax with a characteristic time constant in the 1–100 s range. When the NOMFET is connected in a diode like configuration (Figure 8A), each input spike (with a negative voltage value) charges the nanoparticles and decreases the NOMFET conductivity. Between pulses, charges escape from the nanoparticles and the conductivity relaxes toward its resting value. By analogy with biology, this device mimics the STP observed in depressing synapses (Figures 8B,C) and described by Abbott et al. (1997). As a matter of comparison, this synaptic functionality is realized with a single memory transistor when its implementation in Si based technologies (i.e., CMOS) required 7 transistors (Boegerhausen et al., 2003). STP has been also demonstrated in two-terminal devices that would ensure higher device density when integrated into complex systems. Equivalently, STP in two terminals devices is implemented by taking advantage of the volatility of the different memory technologies (i.e., low retention of the state that is often a drawback in pure memory applications). Cationic redox systems based on Electro-Chemical Memory cells (ECM) (Ohno et al., 2011) or anionic Valence Change Memory (Chang et al., 2011; Yang et al., 2013) have demonstrated STP with a facilitating behavior. In such devices, Short Term Plasticity is ensured by the low stability of the conducting filaments that tend to dissolve, thus relaxing the device toward the insulating state. TiO2 VCM cells have been reported with both facilitating and depressing behavior (Lim et al., 2013) with relaxation related to oxidation-reduction counter reaction. Protonic devices have demonstrated STP with depressing functionality due to proton recovery latency from atmosphere required to restore the proton concentration and conductivity (Josberger et al., 2014).

Figure 8.

(A) Schematic of the NOMFET in a diode-like configuration. This leaky memory transistor was proposed to implement the Short Term Plasticity (B) STP measured in biological synapses (Varela et al., 1997) and (C) STP implement in solid state device.

In the case of rate dependent plasticity, STP can be of depressing type (i.e., decrease of the synaptic efficiency when synaptic activity increase) or facilitating type (i.e., increase of synaptic efficiency when synaptic activity increase). In terms of functionality, Abbott et al. (1997) has demonstrated that depressing synapses with STP act as a gain control device (at high frequency, i.e., high synaptic activity, the synaptic weight is decreased, thus leading to a reduction of the signal when activity becomes too important). More generally, STP (both depressing and facilitating) provides a very important frequency coding property (as depicted in Figures 8B,C) that could play a major role in the processing of spike-rate coded information. Indeed, if a simple Integrate and Fire neuron (I&F) is associated with static weight (with no dependence with spike frequency), the computing node (i.e., neuron and synapses) is only a linear filter (linear combination of the different input) while STP turns the node to non-linear. This property can be used to implement reservoir computing approaches as proposed by Maass (Buonomano and Maass, 2009) with the Liquid State Machine and could be an important property of biological systems for computation.

Co-existence of STP and LTP in the same device

If the contribution of ST and LT processes to computing is not completely understood in biological systems, we should consider that both STP and LTP effects in synaptic connections are required in neuro-inspired computing systems. A first approach is to consider that repetition of short term effects should lead to Long Term modification in the synaptic connections. This behavior would explain the important hypothesis of memory consolidation in the sense of psychology (Lamprecht and Ledoux, 2004). Ohno et al. (2011) reported the coexistence of Long Term and Short Term Potentiation in atomic bridge technology (Figure 9). Depending on pre-synaptic activity (associated to spike rate in this case), the synaptic conductivity is increased due to the formation of a Ag filament across the insulating gap. While for low frequency, the bridge tends to relax between pulses, higher frequencies lead to a strong filament that maintains the device in the ON state. These results suggest a critical size of the bridging filament in order to maintain the conductive state (i.e., providing a LTP of the synaptic connection). Similar results have been obtained in a variety of memory devices where filamentary switching displayed two regimes of volatility. Chang et al. (2011) have evidenced a continuous evolution of the volatility as a function of the conductivity level of the device in WO3 oxide cells attributed to the competition between oxygen vacancies drift (creation of conductive path across the device) and lateral diffusion (disruption of the conducting filaments). Another description of these two regimes of volatility could be associated to a competition between surface and volume energies in the conductive filament.

Figure 9.

Adapted from Ohno et al. (2011). (A) Schematic of atomic bridge devices that was proposed for Short Term Plasticity, Long Term Plasticty (STP/LTP) transition demonstration. Depending on the spiking activity, (B) the metallic filament do not bridge the two electrodes and tends to relax toward the OFF state while it remains (C) in the ON state once it bridges the two electrodes.

If this transition between Short Term Plasticity and Long Term Plasticity is intuitively well-associated to the concept of STM to LTM learning in psychology, we can note that it induces some restriction in term of network functionality. Indeed, in biology, the facilitating process observed at short time scale and associated to an increase of neurotransmitter release probability during a burst of spike (i.e., corresponding to an increase of synaptic efficiency at high frequency spiking rate) is additive with LTP (Bliss and Collingridge, 1993). In this case the node (neuron and synapses) maintains its rate coding property (associated to short term process and described previously as a non-linear node) and can also display long term modification of the synaptic weight. Alternative approaches are still needed as proposed by Cantley et al. (2011) where Short Term processes and Long Term Processes are realized by two different devices (leaky floating gate transistor and non-volatile two-terminal devices) in order to match the complexity of biological synapses. One fundamental issue that needs to be explored is the balance between the device functionality required for proper operation of computing systems (i.e., performances) and optimal integration in order to match synaptic density required for computing.

Toward memristor-CMOS architectures and circuits

In order to exploit the plasticity of memristor-based artificial synapses, specific circuit architecture needs to be developed. Indeed, depending on the polarity and electrical characteristics of investigated devices, two types of circuits have been identified which are described in the following paragraphs.

Circuits for bipolar memristors

Most of the works on memristive devices that have been published over the last couple of years focus on bipolar resistive switching devices (Waser and Aono, 2007; Snider, 2008; Strukov et al., 2008; Jo et al., 2010). This is the case for all the devices presented in Section STDP Learning Thanks to Overlapping Events. These devices exhibit characteristics close to the original Memristor predicted by Chua. Their resistance can be increased or decreased with opposite polarity voltage pulses and the resistance change is cumulative with the previous state of the device, which makes them particularly suitable to implement synaptic-like functionality.

A biologically-inspired spiking NN-based computing paradigm which exploits the specific physics of those devices is presented in Querlioz et al. (2011, 2013). In this approach, CMOS input and output neurons are connected by bipolar memristive devices used as synapses. It is natural to lay out the nanodevices in the widely studied crossbar as illustrated on Figure 10. Learning is competitive thanks to lateral inhibition and fully unsupervised using a simplified form of STDP.

Figure 10.

Basic crossbar circuit topology. Wires originate from CMOS input layer (horizontal black wires) and from the CMOS output layer (vertical gray wires). Memristive nanodevices are located at the cross points of the horizontal and vertical wires.

Using this topology, performance comparable to traditional supervised networks has been measured (Querlioz et al., 2013) for the textbook case of character recognition, despite extreme variations of various memristive device parameters. With the same approach, unsupervised learning of temporally correlated patterns from a spiking silicon retina has also been demonstrated. When tested with real-life data, the system is able to extract complex and overlapping temporally correlated features such as car trajectories on a freeway (Bichler et al., 2012a).

Circuits for unipolar memristors

All that we have discussed in this work can be adapted to another class of memristive devices—the unipolar devices where all applied voltages to increase or decrease the resistance value are positive. Among them, in particular, Phase-Change Memory (PCM) has good maturity, scaling capability, high endurance, and good reliability (Fantini et al., 2010). PCM resistance can be modified by applying a temporal temperature gradient modifying the material organization between an amorphous and a crystalline phase. The amorphous region inside the phase change layer can be crystallized by applying set pulses, thus increasing device conductance. It was shown that the magnitude of the relative increase in conductance can be controlled by the pulse amplitude and by the equivalent pulse width (Kuzum et al., 2012). Amorphization, on the other hand, is a more power-hungry process and is not progressive with identical pulses. The current required for amorphization is typically 5–10 times higher than for crystallization, even for state-of-the art devices.

To overcome these issues, a novel low-power architecture “2-PCM Synapse” was introduced in Bichler et al. (2012b). The idea is to emulate synaptic functions in large scale neural networks using two PCM devices constituting one synapse as shown in Figure 11. These two devices have an opposite contribution to the neuron's integration. When the synapse needs to be potentiated, the Long Term Potentiation (LTP) PCM device undergoes a partial crystallization, increasing the equivalent weight of the synapse. Similarly, when the synapse must be depressed, the Long Term Depression (LTD) PCM device is crystallized. As the LTD device has a negative contribution to the neuron's integration, the equivalent weight of the synapse is reduced. Furthermore, because gradual crystallization is achieved with successive identical voltage pulses, the pulse generation is greatly simplified. Note however that such synaptic circuit will require a slightly more complex post-synaptic neuron circuit in order to deal with pulse integration and generation. This should have a limited impact on the overall neuromorphic circuit given the lower number of neurons vs. synapses.

Figure 11.

Left: Experimental LTP characteristics of the unipolar PCM device. Right: Principle of an equivalent bipolar synapse realized with a 2-PCM circuit. Note that the neuron circuit is not represented on the schematic.

Discussion

Memristive devices are an appealing solution to implement plastic synapses, if we develop the specific driving signals to emulate different learning rules. The most popular synaptic plasticity implementation is based on the realization of Hebbian learning, and in particular of STDP. We shall however note that other plasticity mechanisms exist that have been studied and modeled as suggested in a recent work (Kornijcuk et al., 2014). In this paper, we focused on different implementations of STDP, by taking advantage of the device physics of different memristive devices. The functional differences in the behaviors of the devices directly translate into differences in the learning rules (real time or accelerated, deterministic or stochastic). Using other devices, we also presented other synaptic ideas, such as short term plasticity, or those which exploit interactions between short term and long term plasticity. Finally, we proposed some implementation ideas, offering a large overview of the different possibilities in several material systems.

As memristors are primarily targeted toward future high-density nanoscale arrays, CMOS driver circuits need to be scaled to these dimensions as well. That is to say, the required neuromorphic driver circuits need to be moved to deep submicron technologies. One recently presented method to achieve this is the use of switched-capacitor neuromorphic circuits, which are able to implement the required analog waveforms in high density technologies as small as 28 nm (Mayr et al., 2014b). Coupled with deep submicron CMOS sensors (Henker et al., 2007), they offer the possibility of a full image processing pyramid based on memristive computation in a nanoscale CMOS-memristor hybrid. However, developing appropriate and highly scaled driver circuits for memristive synapses which do not bring large overheads is a significant goal for today's research. This is especially true for proposals that exploit passive crossbar integration. Such circuit topology is particularly appealing for neuromorphic engineers as it offers a direct equivalent for the neuron/synapse circuit with high parallelism and high integration density in which a single device is associated to a single synapse between two neurons (input line and output column). However, it brings circuit challenges (crosstalk, sneak path, impedance mismatch,…) that need to be overcome.

From a more systems' perspective, the most interesting applications for nanoscale memristors will be those that require a large number of learned or programmed synaptic weights. It is important to already consider such applications, to understand the true impact of memristive technology. One of these applications is the Neural Engineering Framework (Eliasmith and Anderson, 2004), which can be used to implement straightforward signal computation, sensor fusion (Mayr et al., 2014a), and recognition (Bichler et al., 2012a), but also models of cognition (Eliasmith et al., 2012). The large number of synapses offered by nanoscale memristive arrays makes the implementation of complex cognitive processing of such large-scale models (Eliasmith et al., 2012) on a single CMOS-memristor hybrid IC a real possibility.

Finally, it is important to understand that there are no absolute optimal memristive devices for the implementation of plasticity in hardware neural networks. The variety of behaviors observed in today's research will be an advantage for neuromorphic chip designers and computational neuroscientists since it opens new paths of implementation of neural computations. In this respect, the plastic behaviors measured on memristive devices and presented in this paper provide the primitive for future neuromorphic breakthroughs.

Conflict of interest statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.