Spike-driven threshold-based learning with memristive synapses and neuromorphic silicon neurons

E Covi; R George; J Frascaroli; S Brivio; C Mayr; H Mostafa; G Indiveri; S Spiga

doi:10.1088/1361-6463/aad361

1. Introduction

Neural networks and deep learning algorithms are currently achieving impressive state-of-the-art results in a wide range of application areas [1–3]. At the same time, computational neuroscience has made tremendous progress in uncovering key principles of computation used by the brain [3–5]. However, despite the remarkable progress, today's artificial electronic systems are still not able to compete with biological ones in real-time sensory and cognitive tasks carried out in complex and uncertain settings. Understanding how to use the brain's computational principles to build compact ultra-low power cognitive computing systems based on unreliable and inhomogeneous nanoscale components has become a goal of utmost importance for the information and communication technology (ICT) industry. This is especially true now that Moore's scaling law, the main driving force behind the tremendous evolution of all electronic devices for the last 50 years, is predicted to come to an end at the horizon of 2020–2025 [6].

Mixed memristive–neuromorphic electronic systems offer a technology that can pave the road towards this objective [3, 7–10]. These systems combine in an optimal way the properties of standard complementary metal–oxide–semiconductor (CMOS) technologies and those of emerging resistive memory technologies to emulate the biological neuron-synapse systems, where synapses can facilitate (potentiation) or inhibit (depression) the connection between neurons. On the neuronal side, asynchronous analogue sub-threshold CMOS circuits are used to emulate the ion diffusion observed in biological neurons whilst keeping the currents in the pA to nA range [1, 11]. On the synaptic side, oxide-based resistive switching (RS) memristive devices are promising candidates as artificial synapses because they allow on-line update and non-volatile storage of synaptic weights, which map to the device conductance [8, 12–16]. Moreover, RS device programming voltages are comparable with the power supply of typical neuromorphic chips, which makes the on-chip integration of RS memristive devices less problematic with respect to other non-volatile devices such as floating gate transistors.

Efficient bio-inspired learning protocols are being developed [17–19], but their implementation still poses a significant challenge [20, 21]. One of the most widely used learning rules in current neuromorphic architectures is spike-timing dependent plasticity (STDP). STDP was first observed in biological neurons [22] and establishes a relationship between the timing of the relative firing activity between two neurons (within a ms-time window), namely the pre-synaptic and post-synaptic neuron, and the weight modification of the synapse that connects them. Accordingly, STDP is able to catch temporal correlations among events that occur within a time window of few ms. Usually proposed implementations of STDP rely on the overlap of pre- and post-spikes that are as long as the correlation time the learning rule is expected to be sensitive to [23, 24]. This expedience, though, poses serious issues related to synaptic matrix overload and constraints in the throughput of a network. Moreover, biological evidences and computational neuroscience studies have shown that the basic pair-wise STDP rule has critical limitations in both biological plausibility and computational power [25–27]. Indeed, STDP in computational models [17, 18] and in hardware implementations [23, 28–30] is carried out considering combinations of few pre- and post-spikes, whereas biological experiments demonstrate that the overall spiking activity within a certain time frame (e.g. including spike rates) plays also an important role in determining the occurrence and nature (potentiation or depression) of the plasticity event [31]. New learning schemes involving spike rate [32, 33] and new plasticity models taking the rate of the synaptic activity into account [34–36] have therefore been proposed.

In this work, we pursue the same approach of going beyond basic STDP mechanisms, and demonstrate in memristor-CMOS hardware a spike-driven plasticity rule belonging to the class of models where plasticity occurs depending on the post-synaptic neuron state at the time of the pre-synaptic spike [34]. Indeed, the information about the temporal correlation of incoming events is coded in the post-synaptic neuron state variables that are evaluated in real-time by an analogue and sub-threshold CMOS circuitry. According to the state variable values at the time of a pre-synaptic spike, the conductance of a HfO₂-based memristor [37] synapse is changed either between two separated states (binary operation) or throughout many adjacent values (analogue or multilevel operation) by plasticity circuits. The proposed implementation features a number of positive advantages. (i) The learning rule is implemented with pulses that are much shorter than the time correlation among incoming events that the network is expected to be sensitive to because the temporal information is stored in the neurons. This ensures an easier management of large memristive arrays with respect to using long overlapping pulses responsible for the coding of the temporal correlations. (ii) The plasticity circuits interfacing neuron and synapses allow the management of write and read phases in an asynchronous manner and, in particular, avoids the integration of the synaptic current during the programming phase, which would result in an improper evaluation of the synaptic efficacy because of current overshoot during the memristor switching. (iii) The temporal information over ms-time windows is provided by relatively small capacitor thanks to the sub-threshold CMOS design which reduces the charging-discharging currents down to the pico- to nano-Ampére range.

In summary, the present work opens the way towards the implementation of several stimulation protocols for life-long on-line learning.

2. Methods

The memristive synapse is fabricated at laboratory level and at micron scale area and it is a (bottom to top) 40 nm TiN/5.5 nm HfO₂/50 nm Pt $40 \times 40$ μm² structure. The details of the fabrication process are reported in [38]. In an initial phase, the device is in its pristine high resistance state (∼1 MΩ) and needs to be formed to initiate the switching. Indeed, the forming process creates for the first time a conductive filament shorting the two electrodes which can be partially dissolved and re-instated with depression and potentiation operations [39–41], respectively. The forming operation is carried out with a current sweep up to 1 mA. The device characterisations shown in section 3.1 are carried out using a standard automatic test equipment (ATE), more specifically a B1500A semiconductor parameter analyser by Keysight, and a custom printed circuit board (PCB) interfacing the sample with the source pulse generator unit (SPGU) and the source measuring units (SMUs) of the Keysight B1500A, as described in [42]. Stimuli applied to the device as described in section 3.1 are trains of rectangular pulses with rise and fall time of 100 ns, time width of either 50 μs or 100 μs and different voltage amplitudes.

In all the experiments of section 3, time widths in the order of tens to hundreds of μs are used to program the device due to their compatibility with real-time applications.

The CMOS chip is a full-custom analogue/digital neuromorphic VLSI chip fabricated using a standard 0.35 μm CMOS process and it includes special pads in the back-end to allow external wire connection of memristive devices. The parameters which determine the neuron behaviour (refractory period, firing threshold, time constant of the calcium variable...) as well as the thresholds used in the plasticity rule (see section 3.3) and the parameters which shape the pre- and post-spike (voltage amplitude and time width) can be set externally by the user. The chip and the interfacing circuits have been presented and fully characterised in [4, 11, 43].

To configure the chip parameters, and acquire data for analysis, we interface the chip to a host PC using a field programmable gate array (FPGA), and a custom digital to analogue (DAC) board. Details on the setup built around the chip can be found in the supplementary information (stacks.iop.org/JPhysD/51/344003/mmedia), section S1.

All the experiments from section 3.2 are carried out by stimulating one silicon neuron through the host PC with a constant injected current, thus making the neuron fire spikes at a constant rate (e.g. of 100 Hz in our experiments).

The memristive device is wired-connected to the neuromorphic chip, but for testing purposes we need the possibility of monitoring the device resistive state and of setting the device resistance at a determined state. As these operations cannot be carried out by the chip, we need to connect the ATE to the device as well. However, the chip keeps the terminals of the synaptic device at 1.65 V, whereas the ATE channels are initially at 0 V. Therefore, manually switching the device between chip and ATE would result in damaging the device. Hence, we designed a custom switch PCB using reed relays to allow an automatic and protected transition of the memristive devices terminals between the ATE and the neuromorphic chip. Further details on the switch PCB can be found in the supplementary information, section S2.

3. Results

To demonstrate the spike-driven plasticity rule, we configured the hardware system to use two silicon neurons connected by a memristive synapse. In the following, we first analyze the properties of each stand-alone element, then we describe the overall system behaviour.

3.1. Synaptic plasticity in memristive devices

The role of the synapse in the system is played by the memristive device, which exhibits plastic behaviour when properly stimulated. Figures 1(a) and (b) show the device behaviour when stimulated with trains of depressing and potentiating pulses, respectively. The trains of pulses have different voltage amplitudes and the same time width of 100 μs. In these experiments, the device was first brought to its low resistance state, LRS (high resistance state, HRS), before being depressed (potentiated) by DC sweeps. It can be noted that the pulse voltage amplitude determines different device behaviours. As an example, in figure 1(a), when the voltage amplitude is low, the stimulation does not affect the device state (stimulation at −0.50 V). The increase of the voltage amplitude determines a change in the device resistance, which can be gradual, featuring several resistance levels (stimulation at −0.95 V), or abrupt, as in the case of −1.40 V stimulation, where only two states are distinguishable. Moreover, it should be noted that higher voltages correspond also to wider resistance windows. The same behaviour is observable during potentiation (figure 1(b)): the pulse voltage determines whether the resistance state remains unchanged (stimulation at 0.50 V) or whether the device shows analogue (stimulation at 0.80 V) or digital (stimulation at 1.15 V) behaviour. Plasticity is therefore closely related to the choice of the spike amplitude and it is a trade-off between the number of different synaptic levels and the obtainable resistance window. Indeed, with high depressing and potentiating voltages (–1.5 V and 1.1 V, respectively), the device shows a digital behaviour, as in the case of figure 1(c), where potentiation/depression cycles carried out with alternating pulses are shown. The LRS is stable, whereas the HRS shows higher variability. Despite HRS variability, the resistance window of about one order of magnitude is constant throughout the cycles. The reduction of the stimulation voltage with respect to the one used in digital operation results in a gradual modulation of the synaptic weight. At the beginning, a preliminary characterisation of the synaptic device is needed to set the parameters during the learning experiments. Figure 1(d) shows analogue potentiation/depression cycles carried out with 300 pulses of 100 μs and with amplitude of 0.8 V and −1 V, respectively. The HRS to LRS ratio is lower than in figure 1(c), but the cycles are repeatable and the synaptic weight is gradually modulated in both operations.

**Figure 1.** (a) Depression operation with trains of 100 spikes with time width of 100 μs and amplitudes (lower to higher) of −0.50 V, −0.95 V, and −1.40 V. (b) Potentiation operation with trains of 100 spikes of 100 μs and amplitudes (lower to higher) 0.50 V, 0.80 V, and 1.15 V. (c) Cycles of digital depression (red squares) and digital potentiation (blue circles) carried out using the automatic test equipment. Depression: voltage amplitude −1.5 V, time width 100 μs. Potentiation: voltage amplitude 1.1 V, time width 50 μs. (d) Cycles of analogue depression (red squares) and potentiation (blue circles) carried out using the automatic test equipment. Depression: 300 pulses, voltage amplitude −1 V, time width 100 μs. Potentiation: 300 pulses, voltage amplitude 0.8 V, time width 100 μs. Prior to operation of the neuromorphic chip, it is possible to set the voltage levels of the applied waveforms according to experimentally derived values, to achieve both depression and potentiation, with similar dynamic ranges.
Download figure:
Standard image High-resolution image

3.2. Silicon implementation of neurons

The silicon neuron (neuron element, NE) is a CMOS implementation that faithfully reproduces the properties of an adaptive exponential integrate and fire model [44]. The model reproduces the biological channels and ions which take part in the normal operation of biological neurons, as detailed in the supplementary information, section S3. Figure 2 shows the analogies between a biological neuron and its silicon counterpart. In biology, when an action potential reaches the end terminals of a pre-synaptic neuron, it causes the release of neurotransmitters, which cross the synapse and bind to the neurosynaptic receptors of the post-synaptic neuron dendrites (biochemical signal transmission in the figure). The biochemical signal is translated into an electric one, the excitatory post-synaptic current (EPSC, I_EPSC in the figure), which then propagates to the neuron's soma. All the EPSC signals coming from each dendrite sum in the neuron soma, thus altering the neuron's membrane potential ( $V_{mem}$ in the figure) in the axon hillock. When the membrane potential overcomes a threshold, the neuron produces an action potential (i.e. a rapid increase and decrease of membrane voltage $V_{mem}$ ) and propagates this 'spike' along the axon. The same mechanism is reproduced in the silicon neuron. Here, when a spike is fired by the pre-synaptic neuron, a current proportional to the synaptic weight flows through the synapse and is sensed by the post-synaptic neuron. The sensed current is low-pass filtered, thus modeling the EPSC, whose integral is the membrane potential $V_{mem}$ . When an action potential is generated, the post-synaptic neuron fires a bi-phasic spike, i.e. a spike with a higher voltage phase followed by a lower voltage phase, at its output terminal.

**Figure 2.** Comparison between biological neurons (top) and implemented silicon neurons (bottom).
Download figure:
Standard image High-resolution image

Regarding plasticity mechanism, the change of the synaptic strength in biology is typically proportional to a variation in the number of neurotransmitter receptors in the post-synaptic neuron. The different spike rate and timing between pre- and post-synaptic neurons cause an alteration of the calcium intra-cellular concentration, which is believed to be responsible for the increase/decrease of the number of neurotransmitter receptors (and, consequently, of the potentiation/depression of the synapse). To implement the same phenomenon in hardware, a dedicated circuit integrates the neuron's spiking activity to produce a calcium-concentration variable which keeps track of the recent post-neural activity. To enable synaptic weight updates, the system produces overlapping spikes, one from the pre-synaptic neuron and one from the post-synaptic neuron, on the memristive device. Specifically, the post-synaptic neuron back-propagates a spike towards the memristive synapse at the onset of the bi-phasic pulse generated by the pre-synaptic neuron. Two interface circuits, namely the pre- and post-interface, enable the neuron to generate spikes able to drive the memristive synapse at both its terminals (i.e. to properly shape the pre- and the post-spike). The nature of the plasticity events (potentiating or depressing the synapse) is determined by the additional spike-driven learning circuits that sense the post-synaptic membrane potential and calcium concentration variable (see also [11] for details). Further details on the neuron model can be found in the supplementary information, section S3.

3.3. Spike-driven threshold-based plasticity rule

A synaptic plasticity event is triggered every time a pre-synaptic spike is produced. The magnitude and direction of the weight change depend only on the state of the post-synaptic neuron at the time of the pre-synaptic spike arrival. The state variables playing a major role in the plasticity rule are post-synaptic neuron membrane potential $V_{mem}$ and calcium variable $V_{\rm Ca}(t)$ . Figure 3(a) shows a measurement of the membrane potential of one of the silicon neurons on the chip, while figure 3(b) shows a measurement of its calcium variable V_Ca. In figure 3(b), each spiking event from the neuron is indicated with an arrow. V_Ca naturally decays over time with slow time constant, unless a spiking event occurs to determine an increase of the calcium variable.

**Figure 3.** Example of (a) membrane potential and (b) calcium variable. The thresholds used to determine the neuron state (LTP, i.e. potentiation, LTD, i.e. depression, or none, i.e. neutral) are indicated with the Greek letter θ. The arrows in (b) indicate the occurrence of a pre-synaptic spike.
Download figure:
Standard image High-resolution image

Depending on the values of these two internal variables, the neuron is in one of three plasticity states: potentiation (LTP), depression (LTD), or neutral. More specifically, upon the arrival of a pre-spike, if the neuron is in its potentiation (depression) state, it will carry out a potentiating (depression) operation, i.e. it will fire a potentiation (depressing) spike, thus inducing a weight update in the synapse. If the neuron is in a neutral state, instead, it will fire a low voltage spike (i.e. a neutral spike), thus inducing no weight update. The change in weight $\Delta w$ is:

$\begin{align} \newcommand{\e}{{\rm e}} \displaystyle \mathrm{\Delta }{w} = \left\{\begin{array}{r@{\quad }lr} \hfill a & \mathrm{if}\; V_{mem}(t|_{t_{pre}})>\theta_I \quad \mathrm{and} \quad \theta_{LTP}^{ low} <V_{\rm Ca}(t|_{t_{pre}})<\theta_{LTP}^{ high} \nonumber \\ \hfill -b & \mathrm{if}\; V_{mem}(t|_{t_{pre}}) \leqslant \theta_I \quad \mathrm{and} \quad \theta_{LTD}^{ low} <V_{\rm Ca}(t|_{t_{pre}})<\theta_{LTD}^{ high} \nonumber \\ 0 & \mathrm{otherwise} \end{array}\right. \label{eq:fusi} \nonumber \end{align} \tag{ 1 }$

where a and b are parameters which may in general depend on the programming requirements of the synaptic device; $\theta_I$ is the threshold on $V_{mem}$ ; $\theta_{LTP/LTD}^{high/low}$ define ranges, possibly overlapping, for the calcium variable, to enable potentiation, depression or neutral plasticity operations.

The thresholds are set externally, therefore they can be adjusted so as to force the neuron to be in a determined state in certain circumstances, e.g. during system calibration.

3.4. Mixed memristive–neuromorphic electronic system

Figure 4(a) shows the system used to carry out plasticity experiments, which consists of a memristive device placed between two silicon neurons, namely a pre-synaptic (NE1) and a post-synaptic neuron (NE2). When NE1 is stimulated, it fires a bi-phasic spike at its output terminal. The spikes from NE1 are sensed by NE2, which fires back a potentiating, a depressing or a neutral spike at its input terminal, depending on its state variables. Figures 4(b) and (c) show (top to bottom) some representative experimental curves of the membrane potential of NE1 (V_mem), of the pre-spike from NE1 (NE1_out), of the post-spike from NE2 (NE2_in) during a depression (figure 4(b)) and a potentiation (figure 4(c)), and of the voltage ΔV across the synaptic device, where ΔV is defined as the difference between the voltage at NE2_in minus the one at NE1_out. When V_mem overcomes a threshold, NE1 fires a bi-phasic pre-spike at its output terminal. The bi-phasic spike triggers a response at NE2_in according to (1), i.e. a neutral spike (low voltage spike which does not affect the device state, not shown), a depressing spike (figure 4(b)), or a potentiating spike (figure 4(c)). Measurements of membrane potential and calcium concentration in NE2 can be seen in [43], figure 8. The pre- and post-spikes overlap and the memristive device changes its resistance once the parameters of the neuron spike generators are set to provide the correct voltage drop on the device. As evident in figures 4(b) and (c), the pulse width of the post-spike is half the pulse width of the pre-spike. Therefore, the post-spike overlaps only with one phase of the bi-phasic pre-spike, and this phase is named writing phase. In the other phase, the synaptic weight is read, hence the name of read phase. The memristor conductance is read only in the read phase by measuring the current flowing through it. The read phase follows the write phase when NE2 is in potentiation or neutral state, whereas it precedes the writing phase when NE2 is in depression state.

**Figure 4.** (a) Illustration of the experimental configuration used in this work: a memristive device connects the pre-synaptic terminal NE1_out of neuron NE1 to the post-synaptic terminal NE2_in of neuron NE2. The neuron symbols are the ones used on neuroscience to represent biological pyramidal cells, with input on the left of the triangle and output on the right. External stimuli are used to elicit spikes on neuron NE1; the propagated spikes cause neuron NE2 to fire in response, observable on terminal NE2_out. (b), (c) Membrane potential of NE1 (first panel) and bi-phasic spikes fired by NE1 (second panel) and NE2 (third panel) during a depression and a potentiation, respectively. Fourth panel: voltage drop across the device. Vertical dashed lines evidence the simultaneity of $V_{mem}$ overtaking the firing threshold (top panels in (b) and (c)), the bi-phasic spike generation by NE1 (second panels in (b) and (c)) and the response of NE2 (third panels in (b) and (c)). All the curves are acquired via oscilloscope.
Download figure:
Standard image High-resolution image

3.5. Synaptic weight evaluation

During the read phase, NE2 integrates the current flowing through the memristive synapse, which is proportional to the memristor conductance, and generates an excitatory post-synaptic current, which is then used to increase its membrane potential. When the membrane potential overcomes its firing threshold, an action potential is generated, which results in NE2 firing a bi-phasic pulse at its output. The firing rate at NE2_out is proportional to the current injected in NE2 and, consequently, to the synaptic device resistance. We therefore can use the neural activity at NE2_out, monitored under specific and controlled system conditions, as a way to measure the synaptic weight without using the ATE. To this end, we explore the response at NE2_out under different synaptic weight conditions and relate it with the resistive state of the device in order to be able to measure the device resistance through the monitoring of the firing rate at NE2_out. Firstly, we use the ATE to set the synaptic device in a known resistive state. Then, we connect the device to the neuromorphic chip via the switch PCB. Post-synaptic neural activity on the NE2_out terminal is measured as the pre-synaptic neuron NE1 is forced to fire for 1 s length at an inter-stimulus-interval of 2.5 ms. Meanwhile, we force the post-synaptic neuron to stay in neutral mode in order not to affect the device resistance while measuring. The procedure to estimate the synaptic weight is repeated for a series of known resistive states (read through the ATE).

Figure 5(a) shows an example of neuronal activity during the stimulation protocol when the memristive synapse is in its potentiated or depressed state. The pre-synaptic neuron NE1 (top panel) is stimulated at a constant rate. In turn, the post-synaptic neuron NE2 both responds at NE2_in with a neutral pulse, thus generating the voltage difference ΔV shown in the second panel of figure 5(a) on the device, and fires bi-phasic spikes at NE2_out (third and fourth panels) with a rate proportional to the synaptic weight. When the synapse is in its potentiated (depressed) state, as in the third (fourth) panel of figure 5(a), the connection between the neurons is facilitated (inhibited) and the frequency of the activity at NE2_out is about 100 Hz (50 Hz). The potentials applied during the read phases of the neutral pulses lie below 400 mV, as shown in the second panel of figure 5(a), which was set as the safe operation range in which the reading voltage (see figure 1) should lie.

In order to show that the neutral pulses do not alter the resistive state of the memristive device, a control with a series of resistors, replacing the memristive device in the experiment, was performed. Figure 5(b) displays the firing rates obtained with control resistors and with the memristive device. Moreover, an exponential-like calibration curve is drawn to fit the non-linear dependence between device resistance and neuronal firing activity. The close overlap of the fit to the acquired data suggests that indeed the measurement protocol does not affect the memristive device. Furthermore, the obtained fitting equation is used in the experiments to directly acquire a resistance estimate through the observation of post-synaptic firing.

3.6. Synaptic weight update

After establishing the relationship between the synaptic weight and the firing rate at NE2_out in a quantitative manner, we can trigger subsequent potentiating/depressing events in the memristive device with the neuromorphic chip to demonstrate the plasticity rule.

Potentiation (depression) occurs when the overlapping of pre- and post-spikes results in a voltage drop across the device high enough to allow for a change in the device resistance.

Figure 6 shows the plasticity experiments resulting from the stimulation of the memristive synapse by the CMOS neurons using the spike-driven threshold-based plasticity rule. Figure 6(a) shows few cycles of potentiation/depression where the device is operated in digital fashion. After each plasticity event, the memristive device state is read by setting the system in neutral mode and measuring the firing rate at NE2_out terminal. The left y-axis of figure 6(a) indicates the resistance of the device extracted from the characterization curve (see figure 5(b)), whereas the right y-axis indicates the measured firing rate of NE2_out. After a potentiating event, the device is in its LRS (corresponding to a firing rate of about 110 Hz), whereas after depression it is in its HRS (corresponding to a firing rate of about 50 Hz). The results are in line with previous characterizations of the device (figures 1(c) and 5(b)). In the following, the amplitude of both the bi-phasic spikes and of the potentiating and depressing spikes are selected according to the results in figure 1 in order to obtain resistance ranges compatible in both the plasticity operations.

Figures 6(b) and (c) demonstrate analogue depression and potentiation, respectively. Figure 6(b) shows two depression operations carried out with spikes of 100 μs and −1 V. The experiments were carried out stimulating NE1 for several seconds and then reading the resistance of the synapse with the ATE. The device gradually switches from its LRS to its HRS and several intermediate states are detectable. The comparison between the two curves in the figure evidences some variability in the weight update steps. This behaviour is nonetheless in line with previous characterisations shown in figure 1(d), where the HRS is shown to suffer from higher variability than LRS, and is in agreement with the inherent variability of filamentary memristive devices [5, 9, 12, 38, 45, 46]. However, it has been demonstrated [47] that variability is not expected to be an issue in larger systems because of the intrinsic parallel and fault tolerant nature of spike-based neuromorphic networks. Figure 6(c) demonstrate analogue plasticity with spikes of 100 μs and different amplitudes. Here as well, the resistance change is gradual and the voltage amplitude of the spike affects the speed of the resistance decrease. Indeed, after 10 s stimulation, we observe a higher resistance change when the device is stimulated with higher voltage amplitudes. Moreover, it can be noted that in both figures 6(b) and (c), the evolution of the resistance is faster in the first seconds of the stimulation, in accordance with figures 1(a) and (b).

4. Discussion

In this work, we provide a hardware demonstration of the spike-driven threshold-based learning rule proposed in [34]. The learning rule is hebbian in that the plasticity event occurs depending on the firing activity of the pre-synaptic neuron. The direction of the weight update (increasing or decreasing) depends on the post-synaptic neuron state, which is defined on the basis of the value of two state variables, the membrane potential and the calcium intra-cellular concentration. The latter includes the information on the neuron's recent firing activity. The rule is asynchronous and behaves as the STDP, but the timing dependence between the pre- and post-synaptic neuron's firing activities is not explicit because there is no direct coding of the temporal relationship between the two spikes. The implicit relationship with time allows the system to fire a plasticity event even in presence of a single pre-spike. Unlike the conventional pair-wise STDP [23, 24, 48, 49], the spike-driven threshold-based plasticity rule exploits the calcium variable to regulate the direction of the synaptic weight. The calcium variable tracks the recent neuronal activity at the post-synaptic neurons and has, therefore, a fundamental role in the evolution of learning dynamics. Moreover, since the decay time constant of this variable is slow, the present implementation can support the sensitivity to temporal correlation in the real-time (ms) domain through μs-long pulses with the synchronisation provided by the neuron/synapse interface as described with reference to figures 4(b) and (c). Conversely, many literature proposals for STDP implementation involve overlapping pulses whose lengths provide the time window of the sensitivity to temporal correlation between incoming events [23, 24]. Despite many literature reports in this direction aim only at providing a proof of principle, the use of long overlapping pulses poses fundamental practical problems, e.g. in case of (the likely) occurrence of two simultaneous plasticity events on the same neuron. Indeed, since a long spike pulse would activate an entire row/column of synapses, the shared spike address line cannot be re-used for new plasticity events until the previous spike/plasticity operation has concluded.

The neuron/synapse interface, whose functionality we demonstrate in this paper, provides a solution for an additional aspect often neglected in memristor-based STDP implementations, e.g. the accurate reading of the memristive conductance. Indeed, one neuron must integrate current contributions that are proportional to the conductances of each afferent synapse. This requires that the current flowing through the memristive device is always driven by the same voltage drop on it and for the same time and, therefore, it is necessary to define a precise read voltage an read time, which must not overlap with the programming operation of the device. This functionality is well demonstrated by our system as discussed with reference to figures 4(b) and (c). The achievement of sensitivity to real-time temporal correlations through short pulses is made possible at the expenses of neuron complication, i.e. by the use of additional electronics for spike integration into the calcium variable (see figure 3(b)). However, this is realised with a compact log-domain integrator employing CMOS transistors in the sub-threshold regime [1], which allows the use of relatively small VLSI-compatible capacitors [50].

To summarise, the present work demonstrates, in a simplified case of a two neurons-one memristor system, the possibility to use HfO_x based memristors to implement a spike-driven threshold based learning rule compatible with asynchronous CMOS circuits able to work also in a biological compatible time scale, and pushes for future applications directed to efficient circuits for online learning. The scaling up of the present system to a real application-oriented product requires the arrangement of synapses in an array. While passive and selector free crossbars have been proposed and demonstrated for some applications [51, 52], the co-integration of selectors and memristive devices might be required for very high density systems. In terms of selectors, despite two-terminal bipolar selector devices constituting the optimum solution in terms of area occupancy, the level maturity of their technology is still not fully compatible with industrial exploitation and co-integration with CMOS circuits and memristors. In this respect, it is worth noticing that the neuron/synapse interface of the CMOS chip used in this work is already equipped with digital asynchronous control signals [11]. These signals can be possibly exploited, through a suitable re-design of the chip, to drive also select transistors in active crossbar, which can be foreseen as a short/mid-term solution for spiking neural networks.

To conclude, the present work, despite its small scale, provides the solution for a certain number of issues raising from the attempts of interfacing memristive devices with an event-based neuromorphic processor. The addressed technical issues take on a trailblazing role in view of the VLSI of memristive devices with CMOS neuromorphic processors.

5. Conclusion

In this work, we presented the hardware demonstration of a spike-driven threshold-based learning rule in a system centered around a neuromorphic processor that uses the firing activity of analogue neuron circuits to trigger plasticity events in memristive synapses interfaced to silicon neurons. We demonstrate that the rule allows asynchronous plasticity events and weight dependent post-synaptic activity, thus improving the conventional STDP learning rule and paving the road towards the development of integrated mixed neuromorphic/memristive systems for real-time applications.

Acknowledgments

This work is partly supported by the project 'NeuRAM3' (EU Horizon2020, Grant No. 687299), by the project 'RAMP' (EU FP7, Grant No. 612058), and by the EPSRC project 'CHIST-ERA' ERA-Net, EPSRC EP/J00801X/1, EP/K017829/1. EC was partially funded by COST Action IC1401—MemoCiS (Memristors—Devices, Models, Cir- cuits, Systems and Applications) in the framework of the Short Term Scientific Mission (STSM) program.

Spike-driven threshold-based learning with memristive synapses and neuromorphic silicon neurons

Article metrics

Submit

Permissions

Author e-mails

Author affiliations

Author notes

ORCID iDs

Dates

Peer review information

Abstract

1. Introduction

2. Methods

3. Results