12: Lecture Hebbian learning and plasticity Experiences change the way we perceive, perform, think and plan. They do so physically by changing the structure of the nervous system, alternating neural circuits that participate in perceiving, performing, thinking and planning. A very simplified view of learning would state that learning modulates (changes) the input-output, or stimulusaction relationship of an organism. Certainly our environment influences how we react to it, and our reactions influence our environment. How learning is achieved in central nervous structures is a major focus of Computational Neuroscience. Learning can easily be observed in behavioral experiments, but there are many examples of changes in neural responses after learning, or after experimental manipulations that change the input-output relationships of a stimulus. Neural firing rates, the temporal precision of their firing, their tuning curves or receptive fields, all of these change with learning and experience. Experience
Stimulus
Organism
Response
Experience
Stimulus
Neuron Organism
Response
At the neural level, many different types of changes can be imagined. For example, recordings from individual neurons in the hippocampus show that these neurons change their "place field" (i.e. responses to location in space) as the animal investigates the experiment. This change could be due to changes in the way visual stimuli affect these neurons (synaptic), or in the way the neurons respond to the same inputs (intrinsic). A very simple example of learning at the organismal level which has been worked out a the neural level is that of sensitization of the gill withdrawal reflex in Aplysia. In the sea mollusk aplysia, a light touch to the animal's siphon results in gill withdrawal.
This reflex response habituates with repeated stimulation, meaning that the reflex response disappears after repetitive stimulation.
If touching the siphon is accompanied by an electrical stimulation to the animal's tail, then the siphon touch elicits a strong withdrawal response again: The noxious stimulus to the tail sensitizes the gill withdrawal reflex.
- Activities of a few neurons can account for the gill withdrawal reflex and its plasticity during sensitization: (1) mechanosensory neurons that innervate the siphon and the tail (SN); (2) motor neurons that innervate muscles of the gill (MN); (3) interneurons that receive inputs from a variety of sensory neurons. SN
MN Touch
SN
MN
Touch
Prior to sensitization, activation of the siphon causes an EPSP to occur in the gill motor neurons (MN). This EPSP decreases when the siphon is repeatedly stimulated (20 and 50 min) (habituation). Activation of the serotonic facilatory interneurons by the tail shock enhances release of transmitter from the sensory neurons onto the motor neurons, increasing the EPSP in the motor neurons even after it has been decreased or habituated.
SN
MN Touch
SN
MN
Touch
Here, the tailshock has provoked a change in the synaptic interaction between the sensory and motor neuron: transmitter release is increased after the tail is shocked. In modeling terms, this would mean that the synaptic weight has been increased, because the action of a presynaptic neuron (SN) onto a postsynaptic neuron (MN) has been increased. (Reminder: the synaptic weight summarizes the effect a single presynaptic action potential has on the postsynaptic voltage. It includes the amount of transmitter release as well as the maximal conductance change seen at the postsynaptic site and the effect of this conductance change onto the postsynaptic membrane voltage). Using the representation presented above:
Experience Tail shock
Stimulus (Siphon touch)
Organism
Response Gill withdrawl
Experience Serotonine release
Stimulus SN action potential
MN Organism
Response MN EPSP
Take another simple example of learning: Pavlov' dog. This "classical" example of classical conditioning was described by the Russian Ivan Pavlov (1849-1936). He trained dogs to associate a tone with a food-reward: (1) the dog initially shows no response to a tone; (2) there is a measurable salivation in response to food; (3) after the tone has been repeatedly presented at the same time than the food, salivation occurs in response to the tone alone in the absence of food. The dog has formed an association between the tone and the food.
Experience Food
Stimulus Tone
Organism
Response Salivation
The response function has been changed: previously the stimulus (tone) evoked no response, now the stimulus evokes a response (salivation). This change has occurred because of an experience (food). In the most simple neural network one could imagine, this function can be described as follows:
Sensory neuron (SN) Food Motor neuron (MN) Salivation Tone Auditory sensory neuron (aSN)
aSN SN MN time
Food
Tone
Before the conditioning, synapses exist between the sensory neurons detecting the presence of food (SN) and the motor neuron driving salivation (MN). No or very weak synapses exist between the sensory neurons detecting the tone (aSN) and the motor neuron (MN). After conditioning, the motor neurons respond to the tone alone, suggesting that synapses have been formed, or strengthened, between the aSN and the MN. In order for this to happen, the tone and the food have to be presented simultaneously. This means that the aSN and the MN have to be active at the same time in order for the synapse to be strengthened. Hebbian learning The idea that connections between neurons that are simultaneously active are strengthened is often referred to as "hebbian learning", and a large number of theoretical rules to achieve such learning in neural networks have been described over the years. Historically, ideas about "hebbian learning" go far back: in 1890, the Harvard philosopher William James formulated the idea that brain activity is regulated by converging inputs onto a given neuron (The amount of activity at any given point in the brain cortex is the sum of the tendencies of all other points to discharge into it, such tendencies being proportionate (1) to the number of times the excitement of each other point may have accompanied that of the point in question; (2) to the intensity of such excitements and (3) to the absence of any rival point functionally disconnected with the first point, into which the discharge might be diverted.). In 1949, Donald Hebb formulated what became the basis of the idea of "hebbian learning" ("When an axon of cell A is near enough to excite a cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place in one or both cells such that As efficiency, as one of the cells firing B, is increased.).
Lets think of this two statements in terms of the formalism we have employed so far: The amount of activity (action potentials) at any given point (postsynaptic neuron) in the brain cortex is the sum of the tendencies of all other points (presynaptic neurons) to discharge into it, such tendencies being proportionate (1) to the number of times the excitement of each other point (presynaptic action potentials) may have accompanied that of the point in question (synchronous pre- and postsynaptic spiking); (2) to the intensity of such excitements (synaptic strengths) and (3) to the absence of any rival point functionally disconnected with the first point, into which the discharge might be diverted (other postsynaptic neurons, inhibition).. And : "When an axon of cell A (presynaptic neuron) is near enough to excite a cell B (postsynaptic neuron) and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place in one or both cells such that As efficiency (synaptic weight), as one of the cells firing B, is increased. Experimentally, a form of hebbian learning has first bee discovered in the hippocampal structure: when a pyramidal cell in a hippocampal brain slice is depolarized (i.e. active) at the same time that one of its incoming inputs is activated (i.e. a presynaptic neuron fires), the synapse that has been activated becomes strengthened.
Stimulation electrode (extracellular)
axons from presynaptic neurons
Recording and injection electrode (intracellular)
Post-recording
post 100%
post-recording post-injection pre 5 sec 2 hours Pairing
Synapses can also be strengthened when high frequency stimulation is used to activate the presynaptic fibers.
Stimulation electrode
Axons from presynaptic neurons
Recording electrode (intracellular)
100%
post
5 sec
pre
... 2 hours
100 Hz tetanus
Tetanus
Baseline
When the presynaptic fibers are activated at high frequencies (typically 100Hz), the postsynaptic neuron is still depolarized from the first pulses when subsequent pulses arrive. In the last 20-30 years, a wealth of data has been accumulated on the properties and mechanisms underlying long-term-potentiation as well as long-term-depression (a decrease of synaptic strength). At the same time, neural network modeling and computational neuroscience research has analyzed the theoretical implications for LTP and learning. Evidence for the involvement of LTP in learning. Evidence that long term potentiation (usually studied in slices) may be involved in learning in the behaving animal comes from a number of observations (by no means a complete list): (1) LTP can be obtained by electrical stimulation in in vivo preparations as well as in behaving animals; (2) animals in which NMDA receptors have been blocked are impaired in certain memory tasks like the radial maze or the water maze; (3) genetically engineered mice which have no NMDA receptors in the hippocampal formation CA1 are impaired on spatial learning tasks AND pyramidal cells in this brain area have less precise spatial receptive fields; (4) in a study using electrical stimulation in the olfactory bulb as cues for olfactory discrimination, an enhancement of the evoked potentials was observed ONLY for stimulations paired with a reward; (5) neuromodulators like acetylcholine, which enhance or enable LTP formation in brain slice experiments impair learning in behavioral situations. Hebbian learning rule The formulation of associative learning that has gathered the most attention for those studying the brain was due to Donald Hebb (see quote above). This proposition has led to a number of mathematical rules, the simplest of which is:
wij = xi xj where wij is the change in the synaptic weight connecting neuron j to neuron i and xi and xj are activities the (firing rates, action potentials) of neurons i and j, and is a scaling parameter often called learning rate.
wij Neuron j Neuron i
xi = F[wij xj] F: linear or non-linear function transforming input into output activity wij = xi xj change in synaptic weight
Reminder: wij stands for the synaptic weight between presynaptic neuron j an postsynaptic neuron i.
One way to interpret this rule is that each time neurons i and j each fire an action potential, the synaptic weight between them is increased. A second interpretation is that the synaptic weight between them is increased proportionally to the average firing rates of both neurons. A third interpretation is that whenever neuron j fires, the synaptic weight wij is increased by a factor proportional to the activity (voltage) in neuron i: wij = vi xj In 1973, Bliss and Lomo first published evidence for a biological mechanism leading to associative change in synaptic strength between neurons.. This learning rule, as well as the experimental observation underlying it, is associative. This refers to the fact that both the pre- and postsynaptic neuron need to be activated at the same time (reminder: at the same time is a relative statement in biology) for the change in synaptic weight (or efficacy) to work. Because of this property, the Hebbian learning rule can serve to form associations between the activity in the pre- and postsynaptic neurons. The associative nature of long term potentiation (LTP) can be due to two important properties of a synaptic receptor called NMDA receptor. When the neurotransmitter glutamate is released from the presynaptic terminal of many synapses in the brain, it binds to (at least) two kinds of postsynaptic receptors. Binding to the first kind of receptor, called AMPA receptor, leads to rapid increase of current (and depolarization) in the postsynaptic cell. possibly contributing to spiking in this cell.
AMPA receptor
1) Action potential in the presynaptic terminal leads to release of glutamate. 2) Glutamate binds to AMPA receptors on the postsynaptic membrane. The binding process changes the confirmation of the protein in such a way as to increase its conductance and 3) Na+ ions can now enter the cell. The resulting current depolarizes the membrane of the postsynaptic neuron (positive ions enter cell -> inside of cell becomes more postive with respect to outside -> depolarization).
The action of glutamate on the second kind of receptor, called NMDA receptor, is more complicated. When the postsynaptic membrane is at potentials close to the resting membrane potential (~ 55-75 mV), the NMDA channel is blocked by magnesium ions (imagine these ions sitting in the receptor and blocking the access for glutamate). This magnesium block is voltage dependent and it is relieved when the postsynaptic neuron is depolarized to near or above firing threshold. Therefore, for current (+ions) to enter the cell through the conductance linked to NMDA receptors, presynaptically released glutamate needs to bind to these receptors, AND the postsynaptic neurons needs to be depolarized in order to unblock the NMDA receptors. As a consequence, both pre (glutamate release) and post (depolarization) neurons need to be active in order for current to pass through NMDA-type conductances.
NMDA receptor when postsynaptic neuron is at rest
1) Presynaptic neuron fires an action potential and releases glutamate. 2) because postsynaptic neurons membrane is near resting potential, Mg2+ ions block the NMDA receptors. Glutamate cannot bind to NMDA receptors and conductance is not changed -> no depolarization of postsynaptic cell.
NMDA receptor when postsynaptic neuron is depolarized
1) Presynaptic neuron fires an action potential and releases glutamate. 2) Because postsynaptic membrane is depolarized, Mg2+ ions are expelled from the NMDA channel. 3) Glutamate binds to NMDA receptor, which leads to an increase in conductance in the channel linked to NMDA. 4) Na+ and Ca2+ ions enter the cell and cell depolarizes.
A second important property of the NMDA channels is that part of the current they pass through is carried by calcium ions. A host of experimental results indicate that calcium then leads to a cascade of cellular events which eventually lead to LTP (strengthening of synapses). If we consider that the opening of the conductance leading to the influx of Ca2+ into the cell necessitates both the presynaptic activation (release of glutamate) and
the postsynaptic activation (release of magnesium block), then the NMDA channel provides the substrate to implement the "Hebbian learning rule". The action of the NMDA channel is the basis for the multiplication in the equation governing the changes in synaptic strength: potentiation of synaptic strength occurs only if the presynaptic activity xj > 0 (glutamate release) and if the postsynaptic activity vi > 0 (depolarization, or xi > 0 action potential). In order to ensure that only depolarization is taken into account, the equation is rewritten as: wij = F[vi] xj, where F is a linear threshold function. Of course, when the wij = xi xj form of the equation is used firing rates can only be positive. So lets apply this learning rule to our example based on Pavlovs experiments with dogs:
Sensory neuron (SN) Food Motor neuron (MN) Salivation Tone Auditory sensory neuron (aSN)
aSN SN MN time
...
Food Tone Food Tone Food Tone Tone
Lets assume that the synapse between the aSN and the MN is weak at the beginning, whereas the synapse between the SN and MN is strong enough to fire the MN. We have: Output (SN) = 1.0 when there is food visible Output (aSN) = 1.0 when there is a tone audible Input (MN) = WMN, SN * Output (SN) + WMN, aSN * Output (aSN) Output (MN) = 1.0 if Input (MN) >= MN Output (MN) = 0.0 if Input (MN) < MN In order to have the MN fire in response to food irrespectively of the tone, WMN, SN * Output (SN) >= MN and since Output (SN) = 1 when there is food, we know that WMN, SN needs to be at least 1.
In order to not have the MN fire in response to the tone (initial state) irrespectively of the food, WMN, aSN * Output (aSN) < MN and since Output (aSN) = 1 when there is a tone, we know that WMN, SN needs to be smaller than 1. So, lets start with WMN, SN = 0.1 and WMN, SN = 1.0. and lets assume for now tha...