# Learning and discrimination through STDP

in a top-down modulated associative memory ^{1}^{1}1Published in Proceedings of 14 European Symposium on Artifical Neural Networks (ESANN 2006), M. Verleysen (ed.), p 611-616, Bruges, Belgium

###### Abstract

This article underlines the learning and discrimination capabilities of a model of associative memory based on artificial networks of spiking neurons. Inspired from neuropsychology and neurobiology, the model implements top-down modulations, as in neocortical layer V pyramidal neurons, with a learning rule based on synaptic plasticity (STDP), for performing a multimodal association learning task. A temporal correlation method of analysis proves the ability of the model to associate specific activity patterns to different samples of stimulation. Even in the absence of initial learning and with continuously varying weights, the activity patterns become stable enough for discrimination.

## 1 Introduction

In the scope of pattern recognition and machine learning, artificial neural networks have proved to be computationally efficient tools. However classical learning methods and connectionist models have shown several limitations, e.g. fast adaptation to changing environment, or modelling temporal binding and synchronization phenomena for multimodal integration [1, 2].

Introduced in [3] and extensively described later [4], spiking neurons use precise timing of spike emissions as relevant neural code, in opposition to the rate coding of the first two generations of neural networks [5]. Spike time coding increases the speed of image processing [6] and the computational power of neural networks [7]. However learning in spiking neuron networks (SNNs) is not yet controlled for performing general purpose discrimination tasks efficiently. A first track is to exploit neuroscientist knowledge on synaptic plasticity mechanisms, like Spike-Timing-Dependent Plasticity (STDP) [8, 9], that can be easily implemented in SNNs as unsupervised dynamic hebbian learning rule for on-line adaptation of weights.

A second idea is to incorporate attentional mechanisms in the model, starting from an architecture of bidirectional associative memory already proven efficient for pattern recognition [10]. It is largely approved that top-down processing is involved in attention [11], which is fundamental for efficient learning. From physiological studies [12], we derive that neocortical layer V pyramidal neurons, able to integrate bottom-up and top-down signals, are well suited for modelling top-down influences in the network.

Only few work try to associate SNNs and STDP in pattern recognition tasks, and none of them study the influence of top-down modulations in such tasks. In this article we present a three layer multimodal associative memory coupling models of spiking pyramidal neurons and interneurons in a STDP driven learning and discrimination task. We show that the model is able to dynamically associate specific patterns of activity to bimodal stimulations and we study the influence of top-down modulations on learning speed and discrimination.

## 2 Multimodal bidirectional associative memory

### Neuron model and network architecture

Neurobiological studies have shown that neocortical layer V pyramidal neurons have specific abilities to integrate distal (layers I and II) top-down inputs and proximal (thalamic, layers V and VI) bottom-up inputs thanks to a dendritic and axonal action potential (AP) initiation sites [13, 12]. Hence we retain neocortical layer V pyramidal neurons as basic model of neuron (Figure 1a) for our implementation of bimodal bidirectional associative memory.

Based on a classical BAM architecture (Bidirectional Associative Memory), adapted for multimodal association [10], and also inspired from the architecture in [14], we define a network (Figure 1b) of three layers with 100 excitatory and inhibitory neurons each. Sensory stimulations are received on the two perceptive layers, one for each modality. The associative layer plays a role of data fusion and gives the output pattern.

### Neural dynamics and interconnections

Excitatory neocortical layer V pyramidal neurons are coupled with inhibitory interneurons for dynamically modulate the amount of activity in the network (for safe of clarity, inhibitory interneurons are not represented on Figure 1b). The neurons are modeled in spike time coding, on the basis of the Spike Response Model [4]. At each time , the membrane potential of a postsynaptic neuron depends on the spike emission times of every presynaptic neurons , weighted by , and on its own past emission times , whith denoting the firing of neuron , according to the equation

(1) |

where is the hyperpolarisation kernel of neuron and is the postsynaptic response kernel. For excitatory pyramidal neurons only, if top-down influences are activated, an extra term is added to the righthand side of equation (1), where is the response kernel to calcium action potentials CaAPs emitted at from the distal tufted dendrites emission site towards the somatic initiation site [12] (Figure 1a).

The emissions of CaAPs (spikes) are function of both the depolarization of distal dendritic site of neuron via backpropagated sodium action potentials NaAPs and distal dendritic excitation above a threshold via pre-synaptic spikes [13]. The coincidence of backpropagated NaAPs and crossing gives rise to CaAPs, thus facilitating a spike emission for neuron .

The two perceptive layers (Figure 1b) receive sensory inputs on the basal dendritic site of the excitatory pyramidal neurons. Each pyramidal neuron makes a synapse to the basal site of the associative layer neurons, which in turn are connected to distal dendritic sites of the perceptive neurons, without layer distinction. Hence, perceptive pyramidal neurons compute the integration of top-down (associative) and bottom-up (sensory) inputs. At each pyramidal neuron in each layer we associate one strongly connected interneuron which makes inhibitory synapses with every other pyramidal neurons of the same layer.

### Model of synaptic plasticity

The weight from a presynaptic neuron to a postsynaptic neuron is modified at each spike emission time or . Let be the time difference between the last post and presynaptic spikes and the weight variation of synapse , subject to the bounds and , or for inhibitory synapses. We chose the following multiplicative rule for synaptic plasticity

(2) |

where and represent the linear weight variation functions of Figure 1c.

## 3 Bimodal association task

### Bimodal stimulation and output

A set of bimodal stimulations is a set of pairs where is the input vector of perceptive layer . In the present article, the network has been tested on a set of pairs of characters represented by matrices of black or white pixels. Associations to be learned are (‘A’,‘1’), (‘B’,‘2’) and so on. An input pattern is presented repetitively times to the network, during every . Each time slot, a black pixel causes a spike emission of the pyramidal neuron on perceptive layer . No spikes are caused by white pixels. The output of the network is the spiking activity pattern of the associative layer recorded along the presentation of a bimodal stimulation . Output can be represented by a matrix of (# associative neurons) (# elementary time slots) binary values: for an output spike emission, otherwise.

### Protocol

The objective is to evaluate the ability of the model, first to reproduce a learned activity for each bimodal stimulation , second to discriminate pairs of stimuli via different output activities for . The experimental protocol consists in two successive phases of stimulation. Starting from random weights, in phase 1 the network adapts all the connections by STDP through repetitive presentations of a sequence of all the input patterns, in a given order (Figure 2). Phase 1 lasts and is followed immediately by phase 2 where each pattern is again presented times to the network ( for each presentation), but the order of the is random and varies from one sequence to the next. All along the protocol, the weights of the network still continue varying through STDP learning. However phase 2 is mainly designed for testing the ability of the model to recall and discriminate all the patterns, independently of the sequence order imposed for learning stabilisation in phase 1. At each step of presentation of , is the output activity pattern of the network for stimulation at step ( to in phase 1, to in phase 2). The learned activity is defined as the activity pattern for at the end of phase 1. The same experimental protocol is applied in two conditions: A network without top-down modulations and a network with CaAPs top-down modulations.

### Analysis

The matricial representations of the network outputs are compared by the method of template correlation analysis, as defined in [15]. The method produces a template correlation coefficient, ranging between and , which indicates the overlapping of two matrices, and then the similarity between two activity patterns. First we have studied, for all the stimulations , the correlation of the successive outputs in phase 1 to the learned activity , in both conditions (Figure 3 left). Second we have computed the average correlations between the learned activity and all the outputs in phase 2, either for the same pattern, , or for all the other stimulations, (Figure 3 right, “same” or “different”).

## 4 Results and conclusion

The experimental protocol has been applied to five networks with different initial weights. Figure 3 presents the mean values of correlation coefficients, computed on all the stimulations and all the networks, for learning (Student test) and discrimination (ANOVA), without and with top-down modulations. The values that are globally significant w.r.t the others are marked by a star.

Figure 3 (left) shows that the correlation coefficients quickly increase in the first steps of learning phase 1 and reach values close to . That proves a good stability of the network output for every stimulations learned by STDP, even in the absence of initial or supervised learning. The influence of top-down modulations is clearly positive: The outputs are better correlated with the learned activities, both in strength and time.

In discrimination (Figure 3 right), the mean correlation coefficients measure the global ability of the model to discriminate patterns during the recall phase 2. Although the weights continue varying by STDP adaptation, learned activities are efficiently reproducible as specific outputs for the learned stimulations, even presented in random order. In both conditions, the correlation with learned activities is significantly higher for the same pattern (case same) than for all the other ones (case different). Top-down influences improve the correlation (close to ) but reduces the difference between the same and different cases.

The results show a stable activity obtained for each bimodal stimulation , reproducible during task and significantly different for each stimulation. We also show that top-down modulations can significantly increase the stability and reproducibility, with the consequence of reducing the differences between stimuli specific responses. Our observations are coherent with the behaviour of general learning systems: High learnability power (e.g. VC-dimension) is not always suitable for good generalisation. However, in the framework of our experiments, our model of top-down modulated associative memory has both the abilities of fast learning ( or steps of STDP are sufficient for convergence) and good discrimination (correlation with different learned activities is significantly lower than correlation with the stimulus specific response).

## References

- [1] J.H. Reynolds and R. Desimone. The role of neural mechanisms of attention in solving the binding problem. Neuron, 24:19–29, 1999.
- [2] F. Crick and C. Koch. A framework for consciousness. Nature neuroscience, 6(2):119–126, 2003.
- [3] W. Gerstner and J.L. Van Hemmen. Associative memory in a network of ’spiking’ neurons. Network, 3:139–164, 1992.
- [4] W. Gerstner and W. Kistler. Spiking Neuron Models: Single Neurons, Populations, Plasticity. Cambridge University Press, 2002.
- [5] W. Maass. Networks of spiking neurons: The third generation of neural network models. Neural Networks, 10(9):1659–1671, 1997.
- [6] S. Thorpe, A. Delorme, and R. Van Rullen. Spike-based strategies for rapid processing. Neural networks, 14:715–725, 2001.
- [7] W. Maass and H. Markram. On the computational power of circuits of spiking neurons. Journal of computer and system sciences, 69(4):593–616, December 2004.
- [8] L.F. Abbott and S.B. Nelson. Synaptic plasticity : taming the beast. Nature neuroscience, 3:1178–1183, 2000.
- [9] R.P.N. Rao and T.J. Sejnowski. Spike-timing-dependent plasticity as temporal difference learning. Neural computation, 13:2221–2237, 2001.
- [10] E. Reynaud and H. Paugam-Moisy. A multiple bam for hetero-association and multisensory integration modelling. In In Proceedings of IJCNN’2005, pages 2117–2122. IEEE, 2005.
- [11] J.B. Hopfinger, M.H. Buonocore, and G.R. Mangun. The neural mechanisms of top-down attentional control. Nature neuroscience, 13(3):284–291, 2000.
- [12] M.E. Larkum, J.J. Zhu, and B. Sakmann. Dendritic mechanisms underlying the coupling of the dendritic with the axonal action potential initiation zone of adult rat layer V pyramidal neurons. Journal of physiology, 533(2):447–466, 2001.
- [13] M.E. Larkum and J.J. Zhu. Signaling of layer I and whisker-evoked and action potentials in distal and terminal dendrites of rat neocortical pyramidal neurons in vitro and in vivo. Journal of neuroscience, 22(16):6991–7005, 2002.
- [14] M. Siegel, K. Körding, and P. König. Integrating top-down sensory processing by somato-dendritic interactions. Journal of computational neuroscience, 8:161–173, 1999.
- [15] K. Louie and M.A. Wilson. Temporally structured replay of awake hippocampal ensemble activity during rapide eye movement sleep. Neuron, 29:145–156, 2001.