Unsupervised Learning of Persistent and Sequential Activity — TEI-tagged view

PereirapersName, BrunelpersName (2020date). “Unsupervised Learning of Persistent and Sequential Activity.” Frontiers in Computational Neurosciencetitle·j.

Object of study — whether persistent activityterm and sequential activityterm need different learning rulesterm, or are one rule at two settings of the input statisticsterm

Research question — what conditions a plasticityterm rule and an input statisticterm must meet to learn either type of dynamics stably

Contribution — one temporally asymmetric Hebbianterm rule learns both, with slow stimuli giving persistent activityterm and fast stimuli sequencesterm - provided a stabilisation mechanism masks the recurrentterm connections during learning

Full text — tagged

The reference list is not part of the text we hold for this paper. Our copy was extracted from the publisher's page and the bibliography did not survive that extraction, so the article below ends before its references. Nothing has been removed by us.

Unsupervisedterm Learning of Persistent and Sequential Activityterm Ulises PereirapersName Ulises PereirapersName 1 Department of Statisticsterm, The University of Chicago, Chicago, IL, United States 1 , Nicolas BrunelpersName Nicolas BrunelpersName 1 Department of Statisticsterm, The University of Chicago, Chicago, IL, United States 2 Department of Neurobiology, The University of Chicago, Chicago, IL, United States 3 Department of Neurobiology, Duke University, Durham, NC, United States 4 Department of Physics, Duke University, Durham, NC, United States 1, 2, 3, 4, * 1 Department of Statisticsterm, The University of Chicago, Chicago, IL, United States 2 Department of Neurobiology, The University of Chicago, Chicago, IL, United States 3 Department of Neurobiology, Duke University, Durham, NC, United States 4 Department of Physics, Duke University, Durham, NC, United States Edited by: Wulfram Gerstner, École Polytechnique Fédérale de Lausanne, Switzerland Reviewed by: Maurizio Mattia, Istituto Superiore di Sanit (ISS), Italy; Paolo Del Giudice, Istituto Superiore di Sanit (ISS), Italy ✉ *Correspondence: Nicolas BrunelpersName nicolas.brunelpersName@duke.edu Received 2018 Oct 3; Accepted 2019 Dec 23; Collection date 2019.

Copyright © 2020 PereirapersName and BrunelpersName.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

PMCID: PMC6978734 PMID: 32009924 Abstract Two strikingly distinct types of activity have been observed in various brain structures during delay periods of delayed response tasks: Persistent activity (PA)term·def, in which a sub-population of neurons maintains an elevated firing rateterm throughout an entire delay period; and Sequential activity (SA)term·def, in which sub-populations of neurons are activated sequentially in time. It has been hypothesized that both types of dynamics can be “learned” by the relevant networksterm from the statisticsterm of their inputs, thanks to mechanisms of synaptic plasticityterm. However, the necessary conditions for a synaptic plasticityterm rule and input statisticsterm to learn these two types of dynamics in a stable fashion are still unclear. In particular, it is unclear whether a single learning ruleterm is able to learn both types of activity patterns, depending on the statisticsterm of the inputs driving the networkterm. Here, we first characterize the complete bifurcation diagramterm·stat of a firing rateterm model of multiple excitatory populationsterm with an inhibitoryterm mechanism, as a function of the parameters characterizing its connectivityterm. We then investigate how an unsupervisedterm temporally asymmetric Hebbianterm plasticityterm rule shapes the dynamics of the networkterm. Consistent with previous studies, we find that for stable learning of PAterm and SAterm, an additional stabilizationterm mechanism is necessary. We show that a generalized version of the standard multiplicative homeostaticterm plasticityterm (Renart et al., 2003 ; Toyoizumi et al., 2014 ) stabilizes learning by effectively masking excitatory connections during stimulationterm and unmasking those connections during retrievalterm. Using the bifurcation diagramterm·stat derived for fixed connectivityterm, we study analytically the temporal evolution and the steady stateterm of the learned recurrentterm architecture as a function of parameters characterizing the external inputs. Slow changing stimuli lead to PAterm, while fast changing stimuli lead to SAterm. Our networkterm model shows how a networkterm with plastic synapses can stably and flexibly learn PAterm and SAterm in an unsupervisedterm manner.

Keywords: unsupervisedterm learning, persistent activityterm, sequential activityterm, synaptic plasticityterm, Hebbianterm plasticityterm, homeostaticterm plasticityterm Introduction Selective persistent activity (PA)term·def has been observed in many neurophysiological experiments in primates performing delayed response tasks, in which the identity or spatial location of a stimulus must be maintained in working memory, in multiple cortical areas, including areas in the temporal lobe (Fuster et al., 1982 ; Miyashita, 1988 ; Miyashita and Chang, 1988 ; Sakai and Miyashita, 1991 ; Nakamura and Kubota, 1995 ; Miller et al., 1996 ; Naya et al., 1996 ; Erickson and Desimone, 1999 ), parietal cortex (Koch and Fuster, 1989 ; Chafee and Goldman-Rakic, 1998 ), and prefrontal cortex (Fuster and Alexander, 1971 ; Funahashi et al., 1989 , 1990 , 1991 ; Miller et al., 1996 ). More recently, selective persistent activityterm has also been observed in mice (Guo et al., 2014 ; Liu et al., 2014 ; Inagaki et al., 2019 ) as well as flies (Kim et al., 2017 ). It has been hypothesized that PAterm represents the mechanism at a networkterm level of the ability to hold an item in working ( active ) memory for several seconds for behavioral demands. Theoretical studies support the hypothesis that persistent activityterm is caused by recurrentterm excitatory connections in networksterm of heavily interconnected populationsterm of neurons (Amit et al., 1994 ; Durstewitz et al., 2000 ; Wang, 2001 ; BrunelpersName, 2005 ). In these models, PAterm is represented as a fixed point attractorterm of the dynamics of a networkterm that has multiple stable fixed points. The connectivityterm matrix in such models has a strong degree of symmetry, with strong recurrentterm connections between sub-groups of neurons which are activated by the same stimulus. This connectivityterm matrix can be learned by modifying recurrentterm connections in a networkterm according to an unsupervisedterm Hebbianterm learning ruleterm (Mongillo et al., 2005 ; Litwin-Kumar and Doiron, 2014 ; Zenke et al., 2015 ).

Sequential activity (SA)term·def has been also observed across multiples species in a number of behaviors such as spatial navigation (Foster and Wilson, 2006 ; Harvey et al., 2012 ; Grosmark and Buzsáki, 2016 ) and bird song generation (Hahnloser et al., 2002 ; Amador et al., 2013 ; Okubo et al., 2015 ). Furthermore, a large body of experimental evidence shows that SAterm can be learned throughout experience (Okubo et al., 2015 ; Grosmark and Buzsáki, 2016 ). Several theoretical networkterm models have been able to produce SAterm (Amari, 1972 ; Kleinfeld and Sompolinsky, 1988 ; Abeles, 1991 ; Diesmann et al., 1999 ; Izhikevich, 2006 ; Liu and Buonomano, 2009 ; Fiete et al., 2010 ; Waddington et al., 2012 ; Cannon et al., 2015 ). In these models, the connectivityterm contains a feedforwardterm structure—neurons active at a given time in the sequenceterm project in a feedforwardterm manner to the group of neurons which are active next. From a theoretical stand point, the mechanism to generate SAterm is fundamentally different from the one that generates PAterm. While SAterm usually corresponds to a path in the state space of the networkterm, PAterm is identified as a fixed point attractorterm. Thus, SAterm has an inherent transient nature while PAterm is at least linearly stable in a dynamical system sense.

The question of how sequential activityterm can be learned in networksterm with plastic synapses has received increased interest in recent years. The models investigated can be roughly divided in two categories: models with supervisedterm and unsupervisedterm plasticityterm rules. In models with supervisedterm plasticityterm rules, the synapses are updated according the activity of the networkterm and an error signal that carries information about the difference between the current networkterm dynamics and the one that it is expected to learn by the networkterm (Sussillo and Abbott, 2009 ; Laje and Buonomano, 2013 ; Memmesheimer et al., 2014 ; Rajan et al., 2016 ). In models with unsupervisedterm plasticityterm rules, sequential dynamics is shaped by external stimulationterm without an error signal (Jun and Jin, 2007 ; Liu and Buonomano, 2009 ; Fiete et al., 2010 ; Waddington et al., 2012 ; Okubo et al., 2015 ; Veliz-Cuba et al., 2015 ). In those models SAterm is generated spontaneously, and the temporal statisticsterm of the stimulationterm shapes the specific timingterm of the sequencesterm.

Both experimental and theoretical work therefore suggest that neural networksterm in the brain are capable to learn PAterm and SAterm. One unresolved issue is whether the learning rulesterm used by brain networksterm to learn PAterm are fundamentally different than the ones used to learn SAterm, or whether the same learning ruleterm can produce both, depending on the statisticsterm of the inputs to the networkterm. Learning rulesterm employed in theoretical studies to learn PAterm typically do not contain any temporal asymmetry, while rules used to learn SAterm need to contain such a temporal asymmetry.

Here, we hypothesize that a single learning ruleterm is able to learn both, depending on the statisticsterm of the inputs. We investigate what are the conditions for the plasticityterm mechanisms and external stimulationterm to learn PAterm or SAterm using unsupervisedterm plasticityterm rules. We consider a model composed of multiple populationsterm of excitatory neurons, each activated by a distinct stimulus. We consider a sequential stimulationterm protocol in which each populationterm of neurons is stimulated one at a time, one after the other. This protocol is characterized by two parameters, the duration of stimulus presentations and the time interval between stimulationsterm. This simple setting allows us to explore between the extremes of isolated stimulationsterm with short or large duration and sequential stimulationsterm close or far apart temporally. We use a rate modelterm·mth to describe the activity of populationsterm of neurons (Wilson and Cowan, 1972 ). The connectivityterm in this model represents the average of the synaptic connections between populationsterm of neurons, allowing to investigate at a mesoscopic level the learning mechanisms of PAterm and SAterm. This model has the advantage of analytical tractability.

This paper is organized as follows: We first characterize the types of possible dynamics observed in networkterm with both feedforwardterm and recurrentterm connections, in the space of possible (fixed) connectivities. We then show that a networkterm with plastic connections described by a unsupervisedterm temporally asymmetric Hebbianterm plasticityterm rule stimulated sequentially does not stably learn PAterm and SAterm. We then explore two types of stabilizationterm mechanisms: (1) synaptic normalizationterm; (2) a generalized version of the standard multiplicative homeostaticterm plasticityterm (Renart et al., 2003 ; Toyoizumi et al., 2014 ). We show that when a synaptic normalizationterm mechanism is included, PAterm and SAterm cannot be learned stably during sequential stimulationterm. However, the addition of a generalized homeostaticterm learning ruleterm leads to successful learning of PAterm or SAterm by weakening the excitatory synaptic weightsterm during stimulationterm, effectively masking the connectivityterm shaped by Hebbianterm learning. After stimulationterm, the learned connectivityterm structure is unmasked by the strengthening of the synaptic connections in a multiplicative fashion. PAterm or SAterm is learned depending on the temporal parameters of external inputs, and the learning can be characterized analytically as a dynamical system in the space of fixed connectivities parametrized by the stimulus parameters.

1. Methods 1.1. Networksterm With Fixed Connectivityterm We first consider two different n populationterm rate modelsterm·mth that share in common two connectivityterm motifs that have been classically considered a distinctive feature of PAterm and SAterm respectively: recurrentterm and feedforwardterm connections. The two networkterm models considered are: (1) n excitatory populationsterm; (2) n excitatory populationsterm with shared inhibition. The strength of the recurrentterm and feedforwardterm connections are w and s , respectively. We used the current based version of the widely used firing rateterm model, which is equivalent to its rate based version (Miller and Fumarola, 2012 ) with three different non-linear transfer functionsterm.

1.1.1. Networkterm of Excitatory Neurons The networkterm consists in n excitatory populationsterm connected by recurrentterm and feedforwardterm connections with strength w and s , respectively as it is shown in Figure 1A I. The dynamics is given by: Figure 1.

Open in a new tab Networkterm models with fixed connectivityterm. (A) Two models of recurrentterm and feedforwardterm connected populationsterm: (I) pure excitatory; (II) excitatory with shared inhibition. (B) Transfer Functionsterm: piecewise linear (PL), sigmoidal (S), and piecewise non-linear (PNL).

τ d u 1 d t = I 1 − u 1 + w ϕ ( u 1 ) τ d u i d t = I i − u i + w ϕ ( u i ) + s ϕ ( u i − 1 ) i = 2 , … , n (1) where u i and I i correspond to the synaptic and the external input to populationterm i respectively, τ is the characteristic time scale for excitatory populationsterm, and ϕ( u ) is the current to average firing rateterm transfer functionterm (or f-I curve). The resulting average firing ratesterm are denoted by r i ≡ ϕ( u i ).

1.1.2. Networkterm of Excitatory Neurons With Shared Inhibition The networkterm consists in n excitatory populationsterm connected as in section 1.1.1, and a single inhibitoryterm populationterm fully connected with the excitatory populationsterm. A schematic of the networkterm architecture is shown in Figure 1A II. Assuming a linear inhibitoryterm transfer functionterm, the dynamics of the networkterm is given by: τ d u 1 d t = I 1 − u 1 + w ϕ ( u 1 ) − w E I u I τ d u i d t = I i − u i + w ϕ ( u i ) + s ϕ ( u i − 1 ) − w E I u I i = 2 , … , n τ I d u I d t = − u I + w I E ∑ j = 1 n ϕ ( u j ) , (2) where w EI is the average inhibitoryterm synaptic strength from inhibitoryterm to excitatory populationsterm, w IE the average inhibitoryterm synaptic strength from excitatory to inhibitoryterm populationsterm and τ I the characteristic time scale of the inhibitoryterm populationterm. When τ I ≪ τ, then u I ≈ w I E ∑ i = 1 N ϕ ( u i ) and Equation (2) becomes τ d u 1 d t = I 1 − u 1 + w ϕ ( u 1 ) − w I n ∑ j = 1 n ϕ ( u j ) τ d u i d t = I i − u i + w ϕ ( u i ) + s ϕ ( u i − 1 ) − w I n ∑ j = 1 n ϕ ( u j ) i = 2 , … , n , (3) where w I ≡ nW EI W IE . See Figure S1 for the agreement between the full model described in Equation (2) and its approximation in Equation (3).

1.2. Transfer Functionsterm For the fixed connectivityterm part of this study we used three different families of transfer functionsterm. The sigmoidal transfer functionterm is described by ϕ ( u ) = 1 2 ( 1 + tanh [ a ( u + b ) ] ) . (4) This is a saturating monotonic function of the total input, and represents a normalized firing rateterm. This transfer functionterm has been widely used in many theoretical studies in neuroscience (Ermentrout and Terman, 2010 ; Gerstner et al., 2014 ), and has the advantage to be smooth. Furthermore, we have recently shown that such transfer functionsterm provide good fits to in vivo data (PereirapersName and BrunelpersName, 2018 ).

The second transfer functionterm considered is piecewise linear: ϕ ( u ) = { 0 if θ > u ν ( u − θ ) if θ ≤ u ≤ u c ν ( u c − θ ) u c < u . (5) This is a piecewise linear approximation of the sigmoidal transfer functionterm. The main advantage of this transfer functionterm is its analytical tractability—the non-linear dynamics of a networkterm with such a transfer functionterm can be analyzed as a piecewise linear dynamical system.

The third transfer functionterm used in this work is piecewise non-linear (BrunelpersName, 2003 ) ϕ ( u ) = { 0 if θ ˜ > u ν ˜ ( u − θ ˜ u ˜ c − θ ˜ ) 2 if θ ˜ ≤ u ≤ u ˜ c 2 ν ˜ u − θ ˜ u ˜ c − θ ˜ − 3 4 u ˜ c < u . (6) This transfer functionterm combines several features that are present in more realistic spiking neuron models and/or real neurons: a supralinear region at low rates, described by a power law (Roxin et al., 2011 ), and a square root behavior at higher rates, as expected in neurons that exhibit a saddle-node bifurcationterm·mth to periodic firing (Ermentrout and Terman, 2010 ). Examples of these three transfer functionsterm are shown in Figure 1B .

1.3. Sequential Stimulationterm During the learning protocol excitatory populationsterm are stimulated sequentially once at a time for a period T and a time delayterm·var Δ. The stimulationterm can be implemented as a sequenceterm of vectors presented to the entire the networkterm (i.e., I e → 1 , I e → 2 , … , I e → n ), each vector corresponds to the canonical base in ℝ n scaled by a stimulation amplitudeterm·var I . This sequenceterm of stimulationterm is repeated k times. To prevent a concatenation between the first and the last populationterm stimulated, the period between each repetition k is much longer than T and Δ and any time constant of the networkterm. Each stimulus in the sequenceterm has the same magnitude, that is larger than the learning thresholdterm (i.e., r w < I ). A schematic diagram of the stimulationterm protocol is described in section 2.2.

1.4. Temporally Asymmetric Hebbianterm Plasticityterm Rule When a temporally asymmetric Hebbianterm plasticityterm rule is included (see sections 2.2–2.5 in Results), the dynamics of excitatory-to-excitatory connectivityterm obeys d W i , j d t = w m a x f [ r i ( t ) ] g [ r j ( t − D ) ] − W i , j τ w [ r i ( t ) , r j ( t − D ) ] , (7) where f ( r ) and g ( r ) are sigmoidal functions given by g ( r ) = 1 2 ( 1 + tanh [ a pre ( r − b pre ) ] ) (8) f ( r ) = 1 2 ( 1 + tanh [ a post ( r − b post ) ] ) . (9) They describe the dependence of the learning ruleterm on post- and pre-synaptic firing ratesterm, respectively [i.e., their dependence on ϕ( u i ) and ϕ( u j )], and are bounded by zero for small or negative values of the populationterm synaptic current, and by one for large values (see Figures 4A,B ). Here w max is the maximal synaptic efficacy; D is a temporal delay; and τ w is an activity-dependent time constant of the plasticityterm rule. The learning time scale is given by Figure 4.

Open in a new tab Unsupervisedterm Hebbianterm learning ruleterm. (A) Piecewise linear transfer functionterm. The dashed gray horizontal line indicates the plasticityterm thresholdterm r w . (B) Post-synaptic dependence on the rates of the stationary connectivityterm function, f ( r ). The vertical dashed gray line indicates the plasticityterm thresholdterm. (C) Contour plot of the stationary connectivityterm function, 𝕎 max f ( r i ) g ( r j ). The dashed gray box indicates the plasticityterm thresholdterm. For W i,j = 0.5 w max the red region below the dashed green line corresponds to LTD while the yellow region above it to LTP. Parameters can be found in Table S3 .

τ w [ r i ( t ) , r j ( t − D ) ] = { ∞ if r i < r w and r j < r w T w if r w ≤ r i or r w ≤ r j . (10) (see dashed line in Figures 4A–C ) and time scale respectively. The time scale T w is chosen to be several orderterm of magnitude slower than the populationterm dynamics (see Table S3 ). When pre- and post-synaptic firing ratesterm are below a plasticityterm thresholdterm r w , the activity-dependent time constant τ w becomes infinite, and therefore no plasticityterm occurs. When pre and/or post-synaptic firing ratesterm are above r w , then the activity-dependent time constant τ w is equal to T w , and plasticityterm is ongoing. Thus, with this rule strong, long (i.e., large T ) and/or frequent enough (i.e., short Δ) stimuli produce lasting modifications in the synaptic weightsterm. When w max f [ r i ( t )] g [ r j ( t − D )] < 𝕎 i, j , the corresponding synaptic weightterm tend to decrease, and therefore LTD takes place. For a fixed high pre-synaptic firing rateterm, LTD occurs when post-synaptic firing ratesterm are low, similar to learning rulesterm such as the BCM rule (Bienenstock et al., 1982 ) and Hebbianterm rules used in recurrentterm networksterm (Mongillo et al., 2003 ). Likewise, for a fixed high post-synaptic firing rateterm, LTD also occurs in an intermediate and low region of pre-synaptic firing ratesterm. Conversely, when 𝕎 i, j < w max f [ r i ( t )] g [ r j ( t − D )], the corresponding synaptic weightterm tend to increase, and LTP takes place.

In our model, we assume the activity of each populationterm represents the average activity over many neurons (of the orderterm of tens of thousands). Likewise, synaptic couplings are assumed to represent averages over very large numbers of synapses connecting two populationsterm (or one populationterm to itself). Therefore, the learning parameters (e.g., D and T w ) represent average quantities over a large number of synapses. We expect the variability of these parameters to disappear in the limit of large number synaptic couplings.

1.5. Synaptic Normalizationterm When a synaptic normalizationterm mechanism is included (see section 2.3 in Results), in addition to the Hebbianterm plasticityterm rule described in section 1.4, in our networkterm simulationsterm·mth, at each time step we subtracted the average synaptic change to each incoming synapse to a given populationterm. This average is taken over all the incoming synapses to a particular neuron. This simulationterm·mth scheme ensures that the sum of the incoming synaptic weightsterm to each populationterm remains constant, i.e., ∑ j = 1 n W i , j = C i = 1 , 2 , … , n . (11) 1.6. Generalized Homeostatic Plasticity Ruleterm·def We implement a modified version of the multiplicative homeostaticterm rule proposed in Renart et al. ( 2003 ) and Toyoizumi et al. ( 2014 ) (see sections 2.4 and 2.5 in Results). The rule is implemented in addition to the Hebbianterm plasticityterm rule described in the section 1.4. In this rule a homeostaticterm variable H i slowly controls the firing rateterm of neuron i by scaling its synaptic weightsterm multiplicatively. The synaptic weightsterm will be given by W i , j ( t ) = H i ( t ) W i , j ( t ) . (12) The variable 𝕎 i, j ( t ) is governed by the Hebbianterm plasticityterm rule described by Equations (7–10). The dynamics for H i is given by τ H H ˙ i = ( 1 − r i ( t ) r 0 ) H i − H i 2 , (13) where r 0 = ϕ( u 0 ) is a parameter that controls the average firing rateterm of populationterm i and τ H is the characteristic time scale of the learning ruleterm. Note that because of the quadratic term in the r.h.s. of Equation (13), this rule does not in general keep the firing ratesterm at a fixed value, and therefore this rule is not strictly speaking homeostaticterm. Therefore, we term this rule generalized homeostatic plasticity ruleterm·def, since this rule is a generalization of the standard multiplicative homeostaticterm plasticityterm rule in Renart et al. ( 2003 ) and Toyoizumi et al. ( 2014 ). As in the standard rule, for strong inputs, the homeostaticterm variables of this rule decrease to values that are smaller than 1, scaling down the excitatory connections and masking synaptic changes learned via Hebbianterm plasticityterm. However, unlike the standard rule, after stimulationterm, the homeostaticterm variables return to values H i ~ O ( 1 ) , and the synaptic changes learned via Hebbianterm plasticityterm are unmasked . See section 2.4 and section 5 in the Supplementary Material for a detailed analysis of this rule.

1.7. Learning Dynamics Under Noisy Stimulationterm In the last section of the Results, we include noise in the populationterm dynamics in orderterm to asses the robustness of the learning process (see section 2.5 in Results). The equations used to describe the dynamics of the networkterm with Hebbianterm and generalized homeostaticterm plasticityterm are given by τ u ˙ i = − u i + σ η i + I i + ∑ j = 1 n H i W i , j r j − W I n ∑ i = 1 n ϕ ( u i ) W . i , j = w m a x f [ r i ( t ) ) ] g [ r j ( t − D ) ] − W i , j τ w ( r i ( t ) , r j ( t − D ) ) τ H H ˙ i = ( 1 − r i ( t ) r 0 ) H i − H i 2 , (14) where r i ( t ) = ϕ( u i ( t )) for i = 1, 2, …, n and η i is a Gaussian white noise.

1.8. Code Simulationsterm·mth were performed using code written in Python. A self-contained version of the code that reproduces all the figures in this paper is available in the GitHub repository: https://github.com/ulisespereira/Unsupervisedterm .

2. Results 2.1. Persistent and Sequential Activityterm in Networksterm With Fixed Connectivityterm To better understand the dependence of PAterm and SAterm generation on networkterm connectivityterm, we consider first a simple n populationterm rate modelterm·mth with fixed feedforwardterm and recurrentterm connectivityterm (see Figure 1A ). This architecture possesses the two connectivityterm motifs that have been classically considered the hallmarks of PAterm and SAterm—recurrentterm and feedforwardterm connections—in a space of parameters that is low dimensional enough to be suitable for full analytical treatment. In this model, the dynamics of the networkterm is characterized by the synaptic inputs u i to each populationterm of the networkterm ( i = 1, …, n ) whose dynamics obey the system of ordinary differential equations in Equation (1). Note that we use here the current based formulation of the firing rateterm equations, that has been shown to be equivalent to the rate based formulation (Miller and Fumarola, 2012 ).

In this model, we identify the regions in the connectivityterm parameter space where SAterm, PAterm, or decaying sequencesterm of activity (dSA) are generated. We start with a piecewise linear transfer functionterm with slope ν, and compute the bifurcation diagramterm·stat that gives the boundaries for qualitatively different dynamics in the parameter space (see Figure 2A and section 2 in the Supplementary Material for mathematical details). We find that robust SAterm can be generated provided recurrentterm connections are smaller than the inverse of the slope ν, and the feedforwardterm connections are strong enough, w < 1/ν < w + s . For large values of w ( w > 1/ν), the dynamics converge to a fixed point where 0 ≤ p ≤ n populationsterm are in a high rate state, where p depends on the initial conditions. When both recurrentterm and feedforwardterm connections are weak enough (i.e., w + s < 1/ν) the activity decays to zero firing rateterm fixed point, after a transient in which different populationsterm are transiently activated—a pattern which we term decaying sequence of activityterm·def or dSA.

Figure 2.

Open in a new tab PAterm and SAterm generation in a networkterm with fixed connectivityterm. (A) Phase diagram for model (I) using a piecewise linear transfer functionterm (top-left plot) and examples of the dynamics corresponding to the three phases. Dashed lines correspond to the dynamics for the same networkterm but using a linear transfer functionterm. (B) SAterm generation for models (I) and (II) using sigmoidal (first row), piecewise non-linear (second row), and piecewise linear (third row) transfer functionsterm. The parameters for the transfer functionsterm are the same as the ones used in Figure 1B . Firing ratesterm of excitatory populationsterm are colored according to their position in the feedforward-recurrent connected excitatory networkterm. Parameters used in (A,B) can be found in Table S1 .

This picture is qualitatively similar when other types of non-linear transfer functionsterm are used (see Methods and Figure 1B for the transfer functionsterm used in this paper). The saturation non-linearity of the transfer functionterm is key to generate long lasting (non-attenuated) SAterm even when the number of populationsterm is large. In a linear networkterm, sequential activityterm would increase without bound for an increasing number of populationsterm participating in the SAterm (see Figure 2A , dashed lines and section 2 in the Supplementary Material for mathematical details). During sequential activityterm, each populationterm is active for a specific time interval. We used the analytical solution of the linearized system (see Equation S3) to show that the duration of this active interval scales as the squared root of the position of the populationterm along the sequenceterm. This implies that for long lasting SAterm the fraction of active populationsterm will increase with time (see Figure 2A ). This feature is not consistent with experimental evidence that shows that the width of the bursts of activity along the sequenceterm is approximately constant in time (Hahnloser et al., 2002 ; Harvey et al., 2012 ). In the model, we can prevent this phenomenon by including negative feedback mechanisms to our networkterm architecture as global inhibition (see Figure 1A II). We found that in both cases the networkterm robustly generates PAterm and SAterm in which the fraction of active populationsterm is approximately constant in time. These results were also qualitatively similar when different saturation non-linearities in the transfer functionterm were considered (see Figure 2B ).

We now turn our attention to the networkterm of excitatory neurons with global inhibition ( Figure 1A II), since inhibition is likely to be the dominant source of negative feedback in local cortical circuits. Inhibitoryterm interneurons are typically faster than excitatory neurons (McCormick et al., 1985 ). For the sake of simplicity we set the inhibitoryterm populationterm dynamics as instantaneous compared with the excitatory timescaleterm. Our numerical simulationsterm·mth confirm that this approximation preserves all the qualitative features of the dynamics with finite inhibitoryterm time constants, up to values of τ I = 0.5τ (see Figure S1 ). Using this approximation, the connectivityterm of the networkterm is equivalent to a recurrentterm and feedforwardterm architecture plus a uniform matrix whose elements are w I ≡ nw EI w IE . We obtained the bifurcation diagramterm·stat for such a networkterm with a piecewise linear transfer functionterm (see section 4 in the Supplementary Material ). This new bifurcation diagramterm·stat shows qualitative differences with the pure excitatory networkterm bifurcation diagramterm·stat (see Figure 3 ). First, a qualitatively different behavior arises, where SAterm ends in persistent activityterm (region SAterm/PAterm). Second, the PAterm region breaks down in n ( n + 1)/2 square regions of size w I / n × w I / n . Each region is characterized by a minimum and maximum number of populationsterm active during PAterm. The lower left corner of each squared region is ( i m i n ( w I n ) , 1 + i m a x ( w I n ) ) with i min , i max = 1, 2, …, n (see Figure 3 , different regions in graded blue), where i min and i max correspond to the minimum and maximum number of populationterm active during PAterm within this squared region when just the first populationterm is initialized in the active state ( Figure 3 top and middle right plots). Therefore, the number of possible patterns of PAterm increases with the strength of the recurrentterm connections and decreases with strength of the feedforwardterm connections. On the other hand, the SAterm/PAterm is divided in n qualitatively different rectangular regions of size ( w I n ) × [ 1 - j S A / P A ( w I n ) ] with j SAterm / PAterm = 1, 2, …, n , where j SAterm / PAterm corresponds to the number of populationsterm that ends in PAterm after SAterm elicited by stimulating the first populationterm in the sequenceterm ( Figure 3 bottom right plot). Then for a given strength of the recurrentterm connectivityterm w * above 1 + ( w I n ) , the critical feedforwardterm strength s c that separates the PAterm and SAterm/PAterm regions is Figure 3.

Open in a new tab Bifurcation diagramterm·stat for feedforward-recurrent connected networkterm of excitatory populationsterm with shared inhibition. Top left plot: Bifurcation diagramterm·stat in the s - w plane, showing qualitatively different regions: dSA (gray), SAterm (red), SAterm/PAterm (green), and PAterm (blue). The PAterm region is divided in sub-regions which are distinguished by the maximum and minimum number of populationsterm active during PAterm (see text). The SAterm/PAterm region is also subdivided into sub-regions characterized by a different number of the maximum number of populationsterm active in PAterm at the end of the sequenceterm. Regions are separated by black lines and sub-regions are separated by gray lines. Five plots encompassing the bifurcation diagramterm·stat show examples of the dynamics observed in its four qualitatively different regions. Initial condition: first populationterm active at the maximum rate, while the rest is silent. The location in the corresponding regions of the parameter space are indicated with the symbols on the top right of the surrounding plots. Parameters can be found in Table S2 . Firing ratesterm of excitatory populationsterm are colored as in Figure 2 .

s c = w I n ⌈ ( w * − 1 − w I n ) n w I ⌉ , (15) where ⌈·⌉ is the ceiling function. Similarly, for a given strength of the feedforwardterm connection s * above w I n , the critical recurrentterm strength separating SAterm/PAterm and PAterm is w c = w I n ⌈ ( s * − w I n ) n w I ⌉ . (16) Lastly, we find that the SAterm region is shrunk compared with the pure excitatory networkterm, and that the dSA region is wider.

2.2. Unsupervisedterm Temporally Asymmetric Hebbianterm Plasticityterm Rule Let us consider now a fully connected networkterm of n excitatory populationsterm with plastic synapses and global fixed inhibition. The plasticityterm rule for the excitatory-to-excitatory connectivityterm is described by Equation (7). Using this learning ruleterm, with fixed pre- and post-activity, the connectivityterm tends asymptotically to a separable function of the pre- and post-synaptic activity. The functions f ( r ) and g ( r ) are bounded by zero for small or negative values of the populationterm synaptic current, and by one for large values (see Figures 4A,B ). This learning ruleterm is a generalization of classic Hebbianterm rules like the covariance rule (Dayan and Abbott, 2001 ), with a non-linear dependence on both pre and post-synaptic firing ratesterm.

The delay D in the learning ruleterm leads to a temporal asymmetry (Blum and Abbott, 1996 ; Gerstner and Abbott, 1997 ; Veliz-Cuba et al., 2015 ). This delay describes the time it takes for calcium influx through NMDA receptors to reach its maximum (Sabatini et al., 2002 ; Graupner and BrunelpersName, 2012 ). When this learning ruleterm operates and the networkterm is externally stimulated, the connectivityterm changes depending on the interaction of the input, the networkterm dynamics and the learning ruleterm. Due to the relaxational nature of Equation (7), for long times without external stimulationterm the connectivityterm matrix will converge to a stationary rank-1 matrix with entries of the form f ( r i * ) g ( r j * ) , where r → * = ϕ ( u → * ) is the stationary firing rateterm vector, independent of all inputs presented in the past. Therefore, stimuli learned in the connectivityterm matrix will be erased by the background activity of the networkterm for long times after stimulationterm. To prevent this inherent forgetting nature of the learning ruleterm we introduce an activity-dependent plasticityterm time scale in Equation (10). Thus, when pre and/or post-synaptic currents are below a plasticityterm thresholdterm r w , the time scale becomes infinite, and therefore no plasticityterm occurs. When both are above r w , then the time constant is given by T w [see Equation (10) and Figure 4 ]. Lastly, the time scale T w of these changes are chosen to be several orderterm of magnitude slower than the populationterm dynamics, consistent with the time it takes (~1 min or more) for plasticityterm to be induced in standard synaptic plasticityterm protocols (see e.g., Markram et al., 1997 ; Bi and Poo, 1998 ; Sjöström et al., 2001 , but see Bittner et al., 2017 ). Importantly, the weight changes are proportional to the difference w max f [ r i ( t )] g [ r j ( t − D )]−𝕎 i, j , which depends on both the pre and post-synaptic firing ratesterm and the current value of the synaptic weightterm (see Equation 7). Typically, when both pre and post-synaptic firing ratesterm are at intermediate values (see the red region in Figure 4C ) the corresponding synapse undergoes LTD (weight strength tend to decrease), and when they are both high (see the yellow region in Figure 4C ) it undergoes LTP (wight strength tend to increase). Therefore, the rule goes from no plasticityterm to LTD and then to LTP when both pre and post-synaptic firing ratesterm increase (see diagonal direction in Figure 4C ), which is reminiscent of results in pairing experiments (Ngezahayo et al., 2000 ).

Our goal is to understand the conditions for a sequential stimulationterm to lead the networkterm dynamics to PAterm or SAterm, depending of the temporal characteristics of the stimulus, when this plasticityterm rule is introduced. Here we consider a simple stimulationterm protocol where each populationterm in the networkterm is stimulated sequentially one populationterm at a time (see Figure 5A ). In this protocol, populationterm 1 is first stimulated for some time T . Then, after an inter-stimulation time Δ, populationterm 2 is stimulated for the same duration T . The other populationsterm are then stimulated one at a time (3, 4, …, n ) using the same protocol. The amplitude of the stimulationterm is fixed such that the maximum of the current elicited in each populationterm leads to a firing rateterm that is greater than the plasticityterm thresholdterm of the learning ruleterm. The time interval between each repetition of the sequenceterm is much longer than T and Δ and any time constant of the networkterm. When the duration of each stimulationterm is larger than the synaptic delay (i.e., D < T ), recurrentterm connections increase, since the Hebbianterm term driving synaptic changes ( f [ r i ( t )] g [ r i ( t − D )], where i is the stimulated populationterm) becomes large after a time D after the onset of the presentation. When the inter-stimulation time is smaller that the synaptic delay (i.e., Δ < D ), then the feedforwardterm connections increase, since the Hebbianterm term driving synaptic changes ( f [ r i +1 ( t )] g [ r i ( t 𢄠 D )]) is large in some initial interval during presentation of stimulus i +1 (see also Herz et al., 1988 ).

Figure 5.

Open in a new tab Sequential stimulationterm and initial synaptic weightsterm dynamics. (A) Schematic diagram showing stimulationterm protocol for two populationsterm. Populationterm 1 is first stimulated for some time T . Then, after an inter-stimulation Δ time, populationterm 2 is stimulated for the same duration T . (B) The weight dynamics for a networkterm of excitatory populationsterm with shared inhibition and a piecewise linear transfer functionterm is shown for four different stimulationterm regimes. Top-left: Δ < D < T ; top-right: D < Δ, T ; bottom-left: T , Δ < D ; bottom-right: T < D < Δ. Cyan: recurrentterm connections; Yellow/Green: feedforwardterm; Blue: all other connections. Parameters can be found in Tables S3, S4 .

As a result, there are four distinct regions of interest depending on the relative values of the Δ and T with respect to the synaptic delay D . When T is larger than the synaptic delay, and Δ is smaller than the synaptic delay, both recurrentterm and feedforwardterm connections increase. When T is larger than the synaptic delay and Δ is much larger than D , only the recurrentterm connections increase. When Δ is smaller than the synaptic delay and T is much smaller, only the feedforwardterm connections increase. Lastly, when Δ is larger and T is smaller than D no changes in the connectivityterm are observed. The initial temporal evolution of both recurrentterm and feedforwardterm weights in representative examples of the four regions is presented in Figure 5B . We chose not to study the region corresponding to 2 T + Δ < D here, which is a region where “feedforwardterm” connections involving non-nearest neighbor populationsterm can also increase during learning.

We found that this learning ruleterm is in general unstable for long sequential stimulationterm when both feedforwardterm and recurrentterm connections increase during the stimulationterm (i.e., Δ < D < T ) to values large enough to produce persistent activityterm states. This is a consequence of the classic instability observed with Hebbianterm plasticityterm rules, where a positive feedback loop between the increase in synaptic connectivityterm and increase in firing ratesterm leads to an explosive increase in both (Dayan and Abbott, 2001 ). Larger feedforwardterm and recurrentterm connections lead to an increase in number of populationsterm active at the same time during stimulationterm (see Figures 6A,D ) which produce an increase of the overall connectivityterm by the synaptic plasticityterm rule ( Figures 6B,C ). This leads to an increase in the overall activity producing longer periods of PAterm during stimulationterm until a fixed point where many populationsterm have high firing ratesterm is reached, and the connectivityterm increases exponentially to its maximum value (see Figures 6B,C ). By increasing the plasticityterm thresholdterm, it is possible to increase the number of stimulationsterm (and consequently the strength of the feedforwardterm and recurrentterm connections) where the networkterm's activity is stable. However, this does not solve the problem, since the instability on the weights eventually occurs but for a larger number of stimulationsterm and stronger synaptic weightsterm. In orderterm to prevent this instability, we investigate in the next sections two different stabilizationterm mechanisms: synaptic normalizationterm and generalized homeostaticterm plasticityterm. Throughout this paper, for testing whether PAterm, SAterm, SAterm/PAterm, or dSA is learned, after sequential stimulationterm we stimulate the first populationterm and then check whether the networkterm recalls the corresponding type of activityterm·var (see Figure 3 ).

Figure 6.

Open in a new tab Runawayterm instability of the unsupervisedterm Hebbianterm learning ruleterm. (A) Populationterm dynamics during 4 s of sequential stimulationterm with T = 19 ms and Δ = 10 ms. After about 25 s, the last six populationsterm stimulated become active at maximal rates. (B) Synaptic weightsterm dynamics during stimulationterm. Cyan: recurrentterm connections; Light Yellow/Green: feedforwardterm; Red: feed-backward. (C) Connectivityterm matrix at different stimulationterm times. From left to right and from top to bottom: 0, 10, 20, and 30 s. (D) Three examples of populationterm dynamics during a single sequential stimulationterm at 0, 20.28, and 27.3 s, respectively. Note the buildup of activity preceding each stimulus presentation because of the build-up in the feedforwardterm connectivityterm at 20.28 s. In (A,D) the black and gray traces indicate a scaled version of the stimulus. Parameters can be found in Tables S3, S4 . Firing ratesterm of excitatory populationsterm are colored according to their position in the sequential stimulationterm.

2.3. Synaptic Normalizationterm The first mechanism we consider is synaptic normalizationterm. This mechanism is motivated by experimental evidence of conservation of total synaptic weightterm in neurons (Royer and Paré, 2003 ; Bourne and Harris, 2011 ). In our model, we enforce that the sum of the incoming synaptic weightsterm to a given populationterm is fixed throughout the dynamics (see Equation 11 in Methods). This constraint prevents the growth of all the synaptic weightsterm to their maximum value during sequential stimulationterm due to the Hebbianterm plasticityterm, as is described in the previous section. This leads to a heterogeneous dynamics in the synaptic weightsterm where they strongly fluctuate in time during the stimulation periodterm·var (see Figure 7B ). We find that the networkterm does not reach a stable connectivityterm structure, and that the connectivityterm after the stimulationterm markedly depends on the specific moment when stimulationterm ended for a large range of stimulationterm parameters.

Figure 7.

Open in a new tab Heterogeneous synaptic dynamics for Hebbianterm plasticityterm and synaptic normalizationterm. (A) Populationterm dynamics during 10 s of sequential stimulationterm with T = 19 ms and Δ = 10 ms. (B) Synaptic weightsterm dynamics during stimulationterm. Cyan: recurrentterm connections; Light Yellow/Green: feedforwardterm; Red: feed-backward; Blue: feed-second-forward; Green: feed-second-backward. (C) Connectivityterm matrix at different stimulationterm times. From left to right and from top to bottom: 0, 13.8, 27.6, and 41.5 s. (D) Two examples of populationterm dynamics during a single sequential stimulationterm at 0 and 15.8 s respectively. In (A,D) the black and gray traces indicate a scaled version of the stimulus. (E) Networkterm dynamics after learning for the initial condition where the first populationterm is active at high rate and the rest silent. Parameters can be found in Tables S3, S4 . Firing ratesterm of excitatory populationsterm are colored according to their position in the sequential stimulationterm.

At the initial stages of the stimulationterm, feedforwardterm and recurrentterm connections grow, while the rest of the synaptic connections decrease at the same rate (see Figure 7B ). When the feedforwardterm and recurrentterm connections are large enough for producing persistent activityterm, co-activation between a populationterm(s) undergoing persistent activityterm and the populationterm active due to the stimulationterm (which are not necessarily adjacent in the stimulationterm sequenceterm, see Figures 7A,D ) produces an increase in feed-back and upper triangular connections that are different than feedforwardterm and recurrentterm (see Figure 7B ). In turn, feedforwardterm and recurrentterm connections decrease due to the synaptic normalizationterm mechanism. This leads to complex dynamics in the synaptic weightsterm, in which the connections sustaining co-active neuronal assemblies (i.e., group of excitatory populationsterm with clustered connectivityterm) learned via Hebbianterm plasticityterm are depressed due to the interplay between synaptic normalizationterm and sequential stimulationterm. This then leads to the formation of new assemblies due to the interplay of Hebbianterm plasticityterm and sequential stimulationterm.

During stimulationterm, the feedforwardterm and recurrentterm connectivityterm studied in the first section increase first, leading then in a second stage to connectivities with strong bi-directional connections (see Figure 7C ). Persistent activityterm can not be learned consistently after long times, and most populationsterm are simultaneously active in retrieved states (see Figure 7E ). For a very few parameter values compared with the explored parameter space sequential activityterm is learned (data not shown). Overall, it is unclear whether neural circuits can use the observed complex synaptic dynamics to store retrievable information about the external stimuli. Thus, we find that synaptic normalizationterm is not sufficient in this case to stabilize learning dynamics and to lead to a consistent retrievalterm of PAterm or SAterm. We checked that this finding is robust to changes in parameters, in particular to changes in the sum of incoming synaptic weightsterm [parameter C in Equation (11)] and the sequential stimulationterm magnitude I , period T , and delay Δ. In the next section we consider a second stabilizationterm mechanism, namely generalized homeostaticterm plasticityterm.

2.4. Generalized Homeostaticterm Plasticityterm Homeostaticterm plasticityterm is another potential stabilizationterm mechanism that has been characterized extensively in experiments (Turrigiano et al., 1998 ; Turrigiano, 2017 ). The interplay between homeostaticterm plasticityterm and Hebbianterm plasticityterm has recently been the focus of multiple theoretical studies (Renart et al., 2003 ; Toyoizumi et al., 2014 ; Keck et al., 2017 ). Here, we study the effect of a generalization of the standard multiplicative homeostaticterm plasticityterm and Hebbianterm plasticityterm for learning SAterm and PAterm. We consider a model for homeostaticterm plasticityterm in which the overall connectivityterm at each time W i, j ( t ) is given by the multiplication of two synaptic variables with different time scales as is shown in Equation (12). In this equation, the fast plastic variable 𝕎 i, j ( t ) (time scale of seconds) is governed by Hebbianterm plasticityterm, see Equation (7). On the other hand, the slow (with a time scale of tens to hundred of seconds) homeostaticterm variable H i ( t ) scales the incoming weights to populationterm i , ensuring that the networkterm maintain low average firing ratesterm on long time scales. The dynamics of the homeostaticterm variable is given by Equation (13). This is a generalization of the standard homeostaticterm learning ruleterm (Renart et al., 2003 ; Toyoizumi et al., 2014 ), that does not include the quadratic term in the r.h.s. of Equation (13). We term this rule generalized homeostaticterm plasticityterm since is a non-linear generalization of the standard multiplicative homeostaticterm plasticityterm rule in Renart et al. ( 2003 ) and Toyoizumi et al. ( 2014 ). The equation proposed in Toyoizumi et al. ( 2014 ) stabilizes the networkterm's activity during stimulationterm, preventing the runawayterm of the firing ratesterm and synaptic weightsterm. Scaling down the overall connectivityterm during stimulationterm prevents co-activation of multiple populationsterm, and lead to stable learning. However, in the networkterm's steady stateterm (i.e., when times longer than the time scale of the homeostaticterm variable have passed without any stimulationterm), if the equation proposed in Toyoizumi et al. ( 2014 ) is used, then each connection will be proportional to the factor ϕ - 1 ( r 0 ) r 0 multiplied by a number of orderterm one (see section 5 in the Supplementary Material for the mathematical details). This implies that the steady stateterm connectivityterm after learning will depend sensitively on the choice of the value of the objective background firing rateterm (i.e., r 0 ) and the specific functional form of the transfer functionterm (i.e., ϕ( u )). Due to the transfer functionterm non-linearity, small changes in r 0 might produce large values for the factor ϕ - 1 ( r 0 ) r 0 and therefore very strong connections for the steady stateterm connectivityterm. This is due to the fact that steady stateterm large values in the homeostaticterm variable H scale up the connectivityterm learned via Hebbianterm plasticityterm in a multiplicative fashion, see Equation (12). In practice, PAterm is retrieved almost always independently of the type of stimulationterm presented during learning, and in the absence of the quadratic term in Equation (13) no temporal attractorterm other than PAterm can be learned. This problem can be prevented by the introduction of a quadratic term in the original homeostaticterm rule (see section 5 in the Supplementary Material ). Note that with this quadratic term, the homeostaticterm plasticityterm rule does not exactly achieve a given target firing rateterm, and therefore is not strictly speaking “homeostaticterm.” Additionally, the timescaleterm of this rule is of the orderterm of minutes (see Figure 8A ), which is an orderterm of magnitude faster of what has been reported experimentally (Turrigiano et al., 1998 ; Turrigiano, 2008 , 2017 ) (see section 3.3 for a discussion).

Figure 8.

Open in a new tab Learning dynamics in a networkterm with Hebbianterm and generalized homeostaticterm plasticityterm. (A) Top: synaptic weightsterm dynamics during and after stimulationterm. Cyan: recurrentterm; Yellow: feedforwardterm; Blue: all other connections. Bottom: homeostaticterm variables colored according to the position of the corresponding excitatory populationterm in the sequential stimulationterm (as in B ). Gray vertical dashed line indicate the end of the sequential stimulationterm. (B) Neuron dynamics during stimulationterm for two different periods of time. (C) Snapshots of the connectivityterm matrix W i,j ( t ) at the end of the sequential stimulationterm (left) and 60 s after the end of the sequential stimulationterm (right). (D) Networkterm dynamics after learning following an initial condition where the first populationterm is active at high rate while all others are silent for two different stimulationterm parameters, one that generates SAterm (left), the other PAterm (right). Parameters can be found in Tables S3, S4 . Firing ratesterm of excitatory populationsterm are colored according to their position in the sequential stimulationterm.

We explore the role of this generalized homeostaticterm learning ruleterm for learning both PAterm and SAterm. During sequential stimulationterm, the average firing rateterm is higher than the background objective firing rateterm r 0 , and the homeostaticterm variables decrease to values that are smaller than 1, see Figures 8A,C . As a result, during sequential stimulationterm the dynamics of the homeostaticterm variable will be dominated by the linear version of the homeostaticterm learning ruleterm proposed in Toyoizumi et al. ( 2014 ), since H i 2 ≪ 1 . Then, the small values that the homeostaticterm variables take during the sequential stimulationterm scale down the increasing values of the recurrentterm and feedforwardterm connections due to Hebbianterm plasticityterm. This produces a weak excitatory connectivityterm during a repeated sequential stimulationterm (see Figure 8C , left), preventing activation of spurious populationsterm during stimulationterm (see Figure 8B ), even though the strength of recurrentterm and feedforwardterm connections learned via Hebbianterm plasticityterm are strong enough to produce PAterm or SAterm, since these connections are masked by the homeostaticterm variable. When the networkterm returns to the steady stateterm after sequential stimulationterm, the homeostaticterm variables return to values H i ~ O ( 1 ) (see section 5 in the Supplementary Material for the mathematical details), and the recurrentterm and feedforwardterm connections learned via Hebbianterm plasticityterm are unmasked . This mechanism stabilizes learning, allowing the networkterm to stably learn strong recurrentterm and feedforwardterm connections, consistent with SAterm or PAterm dynamics (see Figure 8D ).

The weakening of recurrentterm connections during sequential stimulationterm allows us to derive an approximate analytical description of the temporal evolution of the synaptic connectivityterm with learning. Since the net current due to connections between populationsterm is very small, each populationterm dynamics is well-approximated by an exponential rise (decay) toward the stimulationterm current (background current) provided the stimulationterm is strong and inhibition is weak enough (see Figure 9 ). By using this approximation we build a mapping that yields the value of the recurrentterm and feedforwardterm synaptic strengths as a function of stimulationterm number k , stimulation periodterm·var, T , and delay, Δ (see Equations S39, S40 in 6 of Supplementary Material ). This mapping provides a fairly accurate match of both the dynamics of the synaptic weightsterm and the final steady stateterm connectivityterm matrix in the case of weak (see Figure 10A , corresponding to w I = 1) and stronger inhibition (see Figure 10B , w I = 2). The mapping derived for evolution of the synaptic weightsterm during sequential stimulationterm corresponds to a dynamical system in the ( s, w ) phase space that depends on the stimulus parameters (Δ, T ) and the initial connectivityterm. The final connectivityterm corresponds to the fixed point of these dynamics (see Equations S41, S42 in section 6 of Supplementary Material ).

Figure 9.

Open in a new tab Analytical approximation of the dynamics of the networkterm with Hebbianterm and generalized homeostaticterm plasticityterm. First row: Current dynamics for the second and third populationsterm in a networkterm of 20 populationsterm during one presentation of the sequenceterm. The dashed red line shows the analytical approximation for the dynamics during stimulationterm (Equation S32 in section 6 of Supplementary Material ). Second row: Dynamics of the recurrentterm synaptic strength within the second populationterm (cyan), and the “feedforwardterm” synaptic strength from the second to the third populationterm (yellow) during the same presentation of the sequenceterm. The dashed red line shows the analytical approximation for the synaptic weightterm dynamics (Equations S34, S37 in section 6 of Supplementary Materials ). (A,B) correspond to the first and the fifth presentation of the stimulationterm sequenceterm respectively. Parameters can be found in Tables S3, S4 .

Figure 10.

Open in a new tab Changes in recurrentterm and feedforwardterm synaptic strengths with learning, for different sequencesterm with different temporal parameters. Left: Dynamics of recurrentterm and feedforwardterm connections in the ( s, w ) parameter space during sequential stimulationterm for four different values of Δ and T . Black circles (SAterm), plus signs (PAterm), hexagons (dSA), and squares (PAterm/SAterm) show the simulated dynamics. Red traces indicate the approximated dynamics derived in section 6 of Supplementary Material . Right: Rates dynamics after many presentations of the sequenceterm. The first populationterm was initialized at high rates, the others at low rates. (A,B) correspond to w I = 1, w max = 3, and ( T , Δ) = {(7, 14), (50, 40), (5, 15), (22, 8.5)} (in ms) and w I = 2, w max = 3.5, and ( T , Δ) = {(11, 14), (50, 40), (5, 15), (23, 8.5)}, respectively. The rest of the parameters can be found in Tables S3, S4 . Firing ratesterm of excitatory populationsterm are colored according to their position in the sequential stimulationterm.

Figure 10 shows that depending on the temporal characteristics of the input sequenceterm, the networkterm can reach any of the four qualitatively different regions of the phase diagrams in a completely unsupervisedterm fashion. For values of Δ that are smaller than the synaptic delay D and T on the orderterm or larger than D , the networkterm generates SAterm. For values of T approximately larger than D and for Δ small enough, the dynamics lead to SAterm/PAterm. Lastly PAterm is obtained for large enough Δ and T . These observations match with the intuition that stimulationsterm long enough but far delayed in time leads to learning of PAterm and that stimulationsterm contiguous in time but short enough leads to SAterm. Stimulationsterm between these two conditions (long and contiguous) leads to a combination of both dynamics, i.e., SAterm/PAterm, as shown in Figure 10 .

2.5. Learning and Retrievalterm Is Robust to Noise Under in vivo conditions neural systems operate with large amount of variability in their inputs. In orderterm to assess the effect of highly variable synaptic input current during learning and retrievalterm, we add a mean zero uncorrelated white noise to the dynamics when both Hebbianterm learning and generalized homeostaticterm plasticityterm are included in the networkterm, as described in Equation (14). We found that both the synaptic weightsterm dynamics during learning and the retrieved spatiotemporal dynamics after learning are robust to noise (see Figure 11 ), even when the amplitude of the noise is large (i.e., inputs with values equal to the standard deviation of the noise lead to a populationterm to fire at 30% of the maximum firing rateterm). During sequential stimulationterm, the learning dynamics is marginally altered for both weak and strong inhibition (compare Figure 11 with Figure 10 ). Importantly, the synaptic weightsterm reach very similar stationary values compared with the case without noise. After learning, even though the rates stochastically fluctuate in time, the retrieved spatiotemporal attractorsterm (i.e., PAterm, SAterm, dSA, or PAterm/SAterm) are qualitatively similar as in the case without noise (compare Figure 11 with Figure 10 ). One qualitative difference in the case with external noise, is that in both SAterm and PAterm/SAterm dynamical regimes random inputs lead to a repetition of the full or partial learned sequenceterm. Altogether, this simulationsterm·mth show that the networkterm can robustly learn and retrieve qualitatively the same spatiotemporal attractorsterm in the presence of external noise.

Figure 11.

Open in a new tab Learning dynamics under noisy stimulationterm. Same as in Figure 10 , but in the presence of a white noise input current, with mean 0 and standard deviation of 0.3 [i.e., σ = 0.3 in Equation (14)]. Firing ratesterm of excitatory populationsterm are colored according to their position in the sequential stimulationterm.

3. Discussion We have shown that under sequential stimulationterm a networkterm with biologically plausible plasticityterm rules can learn both PAterm or SAterm depending on the stimulus parameters. Two plasticityterm mechanisms are needed: (1) Hebbianterm plasticityterm with temporal asymmetry; (2) a stabilizationterm mechanism which prevents the runawayterm of synaptic weightsterm while learning. When unsupervisedterm Hebbianterm plasticityterm is present alone the networkterm fails to stably learn PAterm or SAterm, while including a generalized homeostaticterm plasticityterm stabilizes learning. For stable learning, we show that the learning process is described by a low dimensional autonomous dynamical system in the space of connectivities, leading to a simplified description of unsupervisedterm learning of PAterm and SAterm by the networkterm from external stimuli. Depending on the stimulus parameters, the networkterm is flexible enough to learn selectively both types of activity by repeated exposure to a sequenceterm of stimuli, without need for supervision. This suggests that cortical circuits endowed with a single learning ruleterm can learn qualitatively different neural dynamics (i.e., persistent vs. sequential activityterm) depending on the stimuli statisticsterm.

Using the full characterization of the bifurcation diagramterm·stat in the space of fixed feedforwardterm and recurrentterm connections developed here, we mapped the evolution of the connectivityterm during stimulationterm in the bifurcation diagramterm·stat. We analytically and numerically showed that the synaptic weightsterm evolve in the feedforwardterm—recurrentterm synaptic connections space until they reach their steady stateterm (when the number of sequential stimulationsterm is large). The specific point of the steady stateterm in the bifurcation diagramterm·stat depends solely on the stimulationterm parameters—stimulation periodterm·var T and time delayterm·var Δ—and the connectivityterm initial conditions. We found that stimulationsterm with long durations and large delays generically leads to the formation of PAterm, whereas stimulationsterm with long enough durations and short delays leads to the formation of SAterm. Thus, persistent stimulationterm leads to persistent activityterm while sequential stimulationterm leads to sequential activityterm.

3.1. Learning of Sequencesterm in Networksterm A growing number of networkterm models have been shown to be able to learn sequential activityterm. Models with supervisedterm learning can reproduce perfectly target sequencesterm through minimization of a suitable error function (Sussillo and Abbott, 2009 ; Laje and Buonomano, 2013 ; Memmesheimer et al., 2014 ; Rajan et al., 2016 ), but the corresponding learning rulesterm are not biophysically realistic.

Other investigators have studied how unsupervisedterm learning rulesterm leads to sequenceterm generation. Early models of networksterm of binary neurons showed how various prescriptions for incorporating input sequencesterm in the connectivityterm matrix can lead to sequenceterm generation (see Kuhn and van Hemmen, 1991 )—or, sometimes, both sequenceterm generation or fixed point attractorsterm depending on the inputs (Herz et al., 1988 ). The drawback of these models is that they separated a learning phase in which recurrentterm dynamics was shut down in orderterm to form the synaptic connectivityterm matrix, and a retrievalterm phase in which the connectivityterm matrix does not change anymore.

Our model removes this artificial separation, since both plasticityterm rule and recurrentterm dynamics operate continuously, both during learning and recall. However, we found that there needs to be a mechanism to attenuate recurrentterm dynamics during learning for it to be stable. The mechanism we propose rely on a modified version of a standard homeostaticterm rule. Other mechanisms have been proposed, such as neuromodulators that would change the balance between recurrentterm and external inputs during presentation of behaviorally relevant stimuli (Hasselmo, 2006 ).

The cost of not having supervision is that the networkterm can only learn the temporal orderterm of the presented stimuli, but not their precise timingterm. Veliz-Cuba et al. ( 2015 ) have recently provided a model which bear strong similarities with our model (rate modelterm·mth with unsupervisedterm temporally asymmetric Hebbianterm plasticityterm rule), but includes in addition a short-term facilitation mechanism that allows the networkterm to learn both orderterm and precise timingterm of a sequenceterm presented in input. However, their mechanisms require parameters to be precisely related.

Models with temporally asymmetric Hebbianterm plasticityterm have also been investigated in the context of the hippocampus (Abbott and Blum, 1996 ; Gerstner and Abbott, 1997 ; Mehta et al., 1997 ; Jahnke et al., 2015 ; Chenkov et al., 2017 ; Theodoni et al., 2018 ). In such models, feedforwardterm connectivityterm is learned through multiple visits of neighboring place fields, and sequential activityterm (“replays”) can be triggered using appropriate inputs mimicking sharp-wave ripples. Other models use unsupervisedterm Hebbianterm plasticityterm but qualitatively distinct mechanisms to generate sequential activityterm. In particular, several studies showed that sequencesterm can be generated spontaneously from unstructured input noise (Fiete et al., 2010 ; Okubo et al., 2015 ). Murray and Escola ( 2017 ) showed that sequencesterm can be generated in networksterm of inhibitoryterm neurons with anti-Hebbian plasticityterm, and proposed that this mechanism is at work in the striatum.

3.2. Learning of Persistent Activityterm With Unsupervisedterm Hebbianterm Plasticityterm In our model, unsupervisedterm temporally asymmetric Hebbianterm plasticityterm is unstable, and after a number of stimulationsterm, all synaptic weightsterm increase to the maximum value. The destabilization occurs during sequential stimulationterm when the populationterm previously stimulated in the sequenceterm presents persistent activityterm at the moment the next populationterm is being stimulated. This leads to the increase of the feedforwardterm synaptic weightterm between these two populationsterm, producing that both present PAterm while the next populationterm in the sequenceterm is being stimulated. This process continues in each repetition of the sequential stimulationterm until all populationsterm in the networkterm present PAterm leading to an increase in the synaptic weightsterm to maximum values. However, previous studies have shown that PAterm could be learned without additional stabilizationterm mechanisms. In Del Giudice and Mattia ( 2001 ), Del Giudice et al. ( 2003 ), and Mongillo et al. ( 2005 ) PAterm is learnt without any stabilizing mechanism except for short-term depression. These studies have a number of differences with ours, however, we believe the key difference for stable learning PAterm is that in these models there is PAterm (in the absence of inputs to the networkterm) below the thresholdterm for LTP induction - something is not currently present in our model since PAterm is always at the maximal rate. Therefore, unlike our model, PAterm after stimulationterm does not lead to effective changes in the synaptic connections. Interestingly, they introduced short term depression to stabilize learning by rapidly shutting down high firing ratesterm during stimulationterm or PAterm, which resembles the effect of the generalized homeostatic plasticity ruleterm·def in our model. In Mongillo et al. ( 2003 ), connectivities with stable recurrentterm, feedforwardterm and feedback connections between pairs of populationsterm emerge from unsupervisedterm Hebbianterm learning without any stabilizationterm mechanism. After learning, when one populationterm in the pair presents PAterm the other transition to PAterm stochastically after a period of time due to spike noise. Importantly, as in the previous models discussed above, the learning thresholdsterm are set in such a way that there is no learning when both populationsterm present PAterm. Additionally, during learning the presentations of stimuli are far apart, and unlike our model learning feedback and feedforwardterm connections are due to persistent activityterm in one neuron, and the stimulationterm in the other, followed by a small increase in the synaptic strengths in each presentation. Lastly, another difference with the models above, and with classical models for persistent delay activity (Amit and BrunelpersName, 1997 ), is that in our model the “background state” is at zero activity. This is an unrealistic feature stemming from the simplified piecewise linear transfer functionterm used in our networkterm (see Equation 5).

3.3. Stabilizationterm Mechanisms Consistent with many previous studies (Dayan and Abbott, 2001 ), we have shown that a networkterm with unsupervisedterm Hebbianterm plasticityterm under sequential stimulationterm leads to a runawayterm of the synaptic weightsterm. This instability is due to a positive feed-back loop generated by the progressive increase of networkterm activity leading to a progressive increase in average synaptic strength when PAterm or SAterm are being learned. One possible solution for this problem was first proposed in the context of attractorterm neural networkterm models (Amit et al., 1985 ; Tsodyks and Feigel'Man, 1988 ; Amit and Fusi, 1994 ). In these models, patterns are learned upon presentation during a learning phase where synapses are plastic but there is no ongoing networkterm dynamics. After the learning phase , the learning of attractorsterm is tested in a retrievalterm phase , where the networkterm dynamics is ongoing but synaptic plasticityterm is not present. Therefore, by compartmentalizing in time dynamics and learning, the networkterm dynamics does not lead to changes in the synaptic weightsterm during retrievalterm, and conversely, changes in synaptic weightsterm do not lead to changes in the dynamics during learning. This separation prevents the observed runawayterm of the synaptic weightsterm due to unsupervisedterm Hebbianterm plasticityterm.

However, it is unclear whether such compartmentalization exists in cortical networksterm. In this work, we explored the alternative scenario, in which both plasticityterm and dynamics happen concurrently during learning and retrievalterm (see also Mongillo et al., 2005 ; Litwin-Kumar and Doiron, 2014 ; Zenke et al., 2015 for a similar approach in networksterm of spiking neurons). First, we found that by adding to unsupervisedterm Hebbianterm plasticityterm an instantaneous synaptic normalizationterm mechanism that maintains constant the sum of incoming synaptic weightsterm to each populationterm, PAterm and SAterm cannot be stably learned. Second, we found that adding a generalized homeostaticterm plasticityterm to unsupervisedterm Hebbianterm plasticityterm leads to stable learning of PAterm and SAterm. During sequential stimulationterm, the increase in co-activation between multiple populationsterm due to recurrentterm and feedforwardterm connections learned via unsupervisedterm Hebbianterm plasticityterm is prevented by suppressing its effect in the networkterm dynamics. Homeostaticterm plasticityterm scales down the overall connectivityterm producing a weakly connected networkterm. PAterm and SAterm is prevented to occur during stimulationterm, which weakens the positive feed-back loop generated by the increase in co-activations of neuronal populationsterm. After learning, the dynamic variables of the Homeostaticterm plasticityterm rule reach a steady stateterm with values similar of what they where before stimulationterm (see Figure 8A ) and the connectivityterm learned via unsupervisedterm Hebbianterm plasticityterm can lead to retrievalterm of PAterm and SAterm upon stimulationterm (see Figure 8C ). The homeostaticterm variable reaches its steady stateterm at a value close to one, and the connectivityterm recovers, unmasking the feedforwardterm and recurrentterm learned architecture. We have also tried other stabilizationterm mechanisms such as inhibitoryterm to excitatory plasticityterm (Vogels et al., 2011 ) instead of homeostaticterm plasticityterm. In this case we found that stable learning of PAterm and SAterm is possible, but for distinct sets of networkterm and stimulationterm parameters (data not shown).

As explained in Zenke and Gerstner ( 2017 ) and Zenke et al. ( 2017 ), in orderterm to prevent the runawayterm of the synaptic weightsterm produced by Hebbianterm plasticityterm, the timescaleterm of any compensatory mechanism should be of the same orderterm or faster than the Hebbianterm timescaleterm. For generalized homeostaticterm plasticityterm, the timescaleterm of the homeostaticterm variable H i is dependent on the firing rateterm of neuron i and the target firing rateterm [i.e., ϕ( u i )/ϕ( u 0 )]. When the networkterm firing rateterm is close to the target firing rateterm the homeostaticterm learning ruleterm is slow, and the homeostaticterm mechanism seldom play a role in the dynamics. On the other hand, for high firing ratesterm the homeostaticterm plasticityterm timescaleterm becomes faster, preventing the runawayterm of the synaptic weightsterm. There is currently an ongoing debate about whether the timescalesterm of compensatory processes used in theoretical studies, as the ones used here, are consistent with experimental evidence (see e.g., Zenke and Gerstner, 2017 ; Zenke et al., 2017 ).

Author Contributions UP and NB designed, performed the research, and wrote the manuscript.

Conflict of Interest The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments UP thanks to the Champaign Public Library for providing the physical space where part of this work has been completed.

Footnotes Funding. This work was supported by NIH R01 MH115555, NIH R01 EB022891, and ONR N00014-16-1-2327.

Supplementary Material The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fncom.2019.00097/full#supplementary-material Click here for additional data file. (279.2KB, PDF)