Feedback Control of Learning by the Cerebello-Olivary Pathway

Preparing to load PDF file. please wait...

0 of 0
100%
Feedback Control of Learning by the Cerebello-Olivary Pathway

Transcript Of Feedback Control of Learning by the Cerebello-Olivary Pathway

CHAPTER
5 Feedback Control of
Learning by the CerebelloOlivary Pathway
Anders Rasmussen1, Germund Hesslow
Department of Experimental Medical Science, Lund University, Lund, Sweden 1Corresponding author: Tel.: þ46 46 222 0638; Fax: þ46 46 211 04 37, e-mail address: [email protected]
Abstract
The ability to anticipate future events and to modify erroneous anticipatory actions is crucial for the survival of any organism. Both theoretical and empirical lines of evidence implicate the cerebellum in this ability. It is often suggested that the cerebellum acquires “expectations” or “internal models.” However, except in a metaphorical sense, the cerebellum, which consists of a set of interconnected nerve cells, cannot contain “internal models” or “have expectations.” In this chapter, we try to untangle these metaphors by translating them back into neurophysiological cause and effect relationships. We approach this task from within the paradigm of classical conditioning, in which a subject, through repeated presentations of a conditional stimulus, followed by an unconditional stimulus, acquires a conditioned response. Importantly, the conditioned response is timed so that it anticipates the unconditioned response. Available neurophysiological evidence suggests that Purkinje cells, in the cerebellar cortex, generate the conditioned response. In addition, Purkinje cells provide negative feedback to the inferior olive, which is a relay for the unconditional stimulus, via the nucleo-olivary pathway. Purkinje cells can therefore regulate the intensity of the signal derived from the unconditional stimulus, which, in turn, decides subsequent plasticity. Hence, as learning progresses, the olivary signal will become weaker and weaker due to increasing negative feedback from Purkinje cells. Thus, in an important sense, learning-induced changes in Purkinje cell activity constitute an “expectation” or “anticipation” of a future event (the unconditional stimulus), and, consistent with theoretical models, future learning depends on the accuracy of this expectation.
Keywords
cerebellum, feedback, inferior olive, nucleo-olivary inhibition, Purkinje cells, classical conditioning, internal models

Progress in Brain Research, Volume 210, ISSN 0079-6123, http://dx.doi.org/10.1016/B978-0-444-63356-9.00005-4 © 2014 Elsevier B.V. All rights reserved.

103

104 CHAPTER 5 Feedback Control of Learning
1 FEEDBACK IS ESSENTIAL FOR LEARNING
Learning in general involves the acquisition of new behavior or the modification of existing behavior. This requires a changed pattern of muscular contractions, which in turn requires a changed pattern of neuronal signaling. Typically, learning is not of a binary nature. Rather, behaviors change gradually until a desired response in a given situation is acquired, after which learning stops. Such modification must necessarily involve feedback to the brain, signaling whether or not our behavior achieves the intended end state. This is also true of motor learning in the cerebellum. Such learning will be described in detail in other chapters of this book. Here we will focus on a specific aspect of cerebellar learning: how plasticity in a cerebellar microcomplex is subject to feedback control by the nucleo-olivary pathway in the context of eyeblink conditioning.
Theoretical work on the cerebellum has improved our understanding considerably, but often authors stop at a rather abstract level where it is considered sufficient to say that the cerebellum “generates a model” or “expects” sensory outcomes. The cerebellum, which consists of a collection of interconnected cells whose firing pattern influences other parts of the organism, cannot, except metaphorically speaking, generate predictions or contain models.
While metaphors can certainly be a great tool for facilitating comprehension, it is important that we are able to translate a metaphor back into the language of cause and effect. In this chapter, we aim to explain how feedback works at a neuronal level. We will not entirely refrain from the use of metaphors but our focus will be on causal chains of physiological events. How do neurons change their firing during learning? What is the nature of the feedback that prevents further changes when adaptive behavior has been attained? We will argue that some of these mentalistic concepts often used to explain learning, such as “predictions,” “internal models,” or “expectations,” could be interpreted in terms of the physiology of the cerebellar microcomplex.
2 ANTICIPATING CONSEQUENCES
It would be impractical if, to assess the consequences of a certain behavior, one had to wait for feedback on every action. Indeed, almost every behavior involves a complicated series of timed muscle contractions, and if we were to wait for sensory feedback following every single contraction, it would take a very long time to perform even the simplest of actions. For this reason, the brain must be able to anticipate the consequences of a certain action, prior to its execution.
This ability to anticipate feedback can and has been described using a number of different frameworks. Kamin, for instance, suggested that learning depends on the extent to which a certain outcome is “surprising” (Kamin, 1969). As long as outcomes match our conscious or unconscious expectations, no learning occurs. This makes intuitive sense because if all our behaviors result in the desired consequences, then there would be no reason to change our behavior. Subsequently, Rescorla and Wagner tried to formalize this concept, stating that a change in associative strength

3 Classical Conditioning 105
depends on the existing associative strength. Another way of putting it is that we learn when events violate our expectations (Rescorla and Wagner, 1972). Their mathematical framework leads to further predictions about when associations are strengthened and weakened including the subsequently demonstrated overexpectation phenomenon (see below).
More recently, the brain’s capacity to anticipate outcomes has been described within the framework of internal models (Ebner and Pasalar, 2008; Shadmehr et al., 2010; Wolpert et al., 1998). When preparing to execute a certain action, the brain simultaneously generates “internal models” of upcoming sensory and motor events. Learning occurs when these models fail, which is similar to saying that learning occurs when events violate our expectations, or in short that learning occurs when we are surprised.
3 CLASSICAL CONDITIONING
When a neutral conditional stimulus (CS) is repeatedly followed by a reflex eliciting unconditional stimulus (US), a subject will learn to respond to the CS so as to anticipate the US. In the eyeblink conditioning paradigm, a sound is typically presented before a corneal airpuff. Following a certain number of repetitions or “trials,” the subject learns to blink in response to the tone, before the airpuff hits the cornea. At this point, the subject has acquired a conditioned response (CR) to the CS (Kehoe and Macrae, 2002). CRs can be extinguished by repeatedly presenting the CS alone. In eyeblink conditioning, a subject that has previously learned to blink in response to a tone will cease to do so if the tone is repeatedly presented without the airpuff (Kehoe, 2006; Kehoe and Macrae, 2002).
It is possible to condition an animal to respond to more than one CS. For example, subjects can learn to blink in response to a tone and a light stimulus, as long as these are not presented simultaneously, in which case one of the two stimuli will overshadow the other (Gormezano et al., 1983; Kehoe, 1982). Using combined CSs can have interesting and sometimes counterintuitive consequences. For example, a subject that has acquired CRs in response to one CS cannot acquire CRs to a second CS if it is presented together with the first one. For example, if a subject has learned to blink in response to a tone and one then adds a light, thus presenting the tone and the light simultaneously (still followed by the US), the subject will not learn to blink in response to the light. Put another way, the learned association to the first CS blocks association to the second CS. This phenomenon is known as Kamin blocking (Kamin, 1969). A phenomena related to Kamin blocking is overexpectation, which occurs when two CSs, each of which elicits a CR, are presented simultaneously, followed by the US. Initially, the simultaneous presentation results in a stronger CR; however, the strength of the CR will gradually decrease, even though the US is still presented (Kehoe and White, 2004).
Both Kamin blocking and overexpectation can be understood from Rescorla and Wagner’s framework (Rescorla and Wagner, 1972). To understand blocking, imagine that a particular CS is already maximally associated with the US. Adding a second CS will not induce further learning because the subject has learned to

106 CHAPTER 5 Feedback Control of Learning
“expect” the US following the presentation of the first CS. Adding a second CS does not alter the subject’s expectations, and if expectations are not violated, no learning occurs. Similarly, overexpectation occurs because the summed associative strength of the two CSs exceeds the actual strength of the US. Because the actual US is weaker than the expected US strength, the associative strength of the CSs gradually weakens (Kehoe and White, 2004).
The fact that classical conditioning is critically dependent on the cerebellum (see below), together with the fact that CRs are timed so as to anticipate the US, has led several researchers to suggest that the ability to anticipate future outcomes relies on the cerebellum (Ebner and Pasalar, 2008; Herreros and Verschure, 2013; Wolpert et al., 1998). Prior research has resulted in a detailed understanding of classical conditioning, from a behavioral as well as a neuronal perspective. Because we have this knowledge, classical conditioning provides an ideal experimental paradigm in which it is possible to approach the neurophysiological foundation of anticipation. Within this paradigm, we can begin to understand what constitutes an internal model and what it really means to say that the brain holds an expectation.
4 THE CEREBELLAR MICROCOMPLEX
The basic unit of cerebellar function is the microcomplex. Both anatomical and physiological work in the 1960s by Voogd and Oscarsson and their collaborators on the projections from the inferior olive revealed a pattern of sagittal zonation in the cerebellar cortex. Groups of olivary cells project to sagittal bands, typically 1–2 mm wide, of Purkinje cells, which in turn project to distinct cell groups in the cerebellar nuclei. These zones, named A, B, C1, Cx, C2, C3, D, have specific targets in the cerebellar nuclei and are also related to different functions (Ito, 1984; Oscarsson, 1979; Voogd and Glickstein, 1998). More detailed analysis of the climbing fiber projections to the C3 and B zones showed that these could be further subdivided into what was then termed microzones (Oscarsson, 1979). A microzone is a sagittally oriented strip of the cerebellar cortex, in which the Purkinje cells have the same climbing fiber input, that is, input driven by coupled olivary cells receiving identical peripheral inputs.
A cortical microzone, which can be a few mm long and a couple of hundred mm wide, projects to a distinct group of cells in a cerebellar nucleus that controls a single muscle, or perhaps a small group of muscles controlling a simple movement. Thus, stimulating or inhibiting Purkinje cells that receive climbing fiber input from the periorbital area modifies activity in the eyelid (Heiney et al., 2014; Hesslow, 1994a) and can suppress an on-going conditioned blink response (Hesslow, 1994b). Because of its intimate connections with nuclear and olivary cells, the microzone concept has been replaced by that of a microcomplex or microcircuit (Apps and Garwicz, 2005; Dean et al., 2010; Ito, 1984), which includes the nuclear and olivary cells and their connections.
A further reason to regard the microcomplex or microcircuit as the basic cerebellar unit is the fact that some microzones are functionally connected (Apps and Garwicz,

4 The Cerebellar Microcomplex 107

2005; Oscarsson, 1979). For instance, climbing fibers from the dorsal accessory olive branch to innervate microzones in both the C3 and C1 zones. These microzones in turn project to the same cells in the anterior interpositus nucleus. An illustration of this principle (Fig. 1) is the identification of (at least) four distinct areas of the cerebellar cortex that receive climbing fiber input from the periorbital area and that

A

Cerebellar anatomy

IV

C3

V

VI VII

PM

B

Cerebellar circuit

pf

CS pathway

Grc mf

Pc
-
CN

Output

cf
N-O
- IO +

US pathway
FIGURE 1
Localization of eyeblink areas on the cerebellar cortex and cerebellar connectivity. (A) Cerebellar microzones that show eyeblink-related activity. (B) Cells and pathways in the cerebellar circuit involved in eyeblink conditioning. The CS is delivered via mossy fibers (mf), synapsing on granule cells (Grc), which contact Purkinje cells (PC) via parallel fibers (pf). The US is delivered via climbing fibers (cf), originating in the inferior olive (IO). Purkinje cells project to the cerebellar nuclei (CN), which project to motor nuclei that control eye muscles. In addition, the cerebellar nuclei inhibit the inferior olive via the nucleo-olivary pathway (N-O).

108 CHAPTER 5 Feedback Control of Learning
control the orbicularis oculi muscle (Hesslow, 1994a,b). Overall, the evidence suggests that the microcomplexes form independent units, where each microcomplex has its own olivocerebellar connections but it also seems probable that the nucleo-olivary fibers project to those olivary cells that supply the Purkinje cells controlling the corresponding nuclear cells (Andersson and Hesslow, 1987).
5 CLASSICAL CONDITIONING REQUIRES THE CEREBELLUM
It has been known for a couple of decades that classical, or Pavlovian, conditioning, such as eyeblink conditioning, depends on cerebellar mechanisms. Inspired by theoretical ideas by Marr (1969) and Albus (1971), and their own anatomical findings, Yeo et al. (1985) suggested that the CS is transmitted to the cerebellar cortex via the mossy fiber/parallel fiber system whereas information about the US is transmitted by the climbing fibers. The US is assumed to induce synaptic changes in the cerebellar cortex so that the CS, after training, will elicit a suppression of simple spike firing in the Purkinje cells. Because the Purkinje cells are inhibitory, this causes a disinhibition of cells in the cerebellar nuclei, and an excitatory signal downstream through the red nucleus and the motor neurons in the facial nucleus (for the eyelid response) and the accessory abducens nucleus (for the nictitating membrane response) (Hesslow and Yeo, 2002).
This view has received strong support by recordings from Purkinje cells. It has been shown that, during eyeblink conditioning, Purkinje cells in an eyelid controlling area of the C3 zone (Fig. 1A) develop a pause response to the CS, a “Purkinje cell CR” (Hesslow and Ivarsson, 1994; Jirenhed et al., 2007). A similar response develops if the CS is direct stimulation of mossy fibers entering the cerebellum and the US is direct stimulation of climbing fibers.
The Purkinje cell CR mirrors many aspects of the overt response CR. The Purkinje cell CR develops after paired CS–US presentations and is extinguished when the CS is presented alone. It reappears very fast when paired stimulation is reinstated after extinction (Jirenhed et al., 2007). The overt blink CR tends to be timed so that the maximum amplitude coincides with the onset of the US. If the interstimulus interval (ISI) between CS and US is increased, additional training will cause the CR latency to adapt to the new ISI. The Purkinje cell CR is adaptively timed in the same way, and it also changes its temporal properties in response to changes in CS parameters just as the overt CR (Jirenhed and Hesslow, 2011; Svensson et al., 2010). Because it has also been demonstrated that these Purkinje cells control the overt CR, we may assume that the Purkinje cell CR drives the overt CR (Hesslow, 1994a,b).
6 THE NUCLEO-OLIVARY PATHWAY AND NEGATIVE FEEDBACK
Since Purkinje cells are GABAergic, a pause in their intrinsic firing will disinhibit the cerebellar nuclei, the primary target of Purkinje cell axons. The cerebellar nuclei project to other nuclei in the brainstem that control motor output. However,

6 The Nucleo-Olivary Pathway and Negative Feedback 109

importantly the cerebellar nuclei also project to the inferior olive, which is the origin of the climbing fibers that relay the US signal (see above). If the nucleo-olivary pathway, which is also GABAergic (De Zeeuw et al., 1989; Nelson and Mugnaini, 1989), is stimulated prior to the US, the signal that reaches the cerebellar cortex is strongly suppressed (Bengtsson and Hesslow, 2006; Hesslow, 1986; Svensson et al., 2006) (Fig. 2).
Andersson et al. (1988) proposed that the nucleo-olivary pathway provides a negative feedback signal to regulate learning in the cerebellar cortex. When a response has been learned and an excitatory signal is sent to the motor system by the cerebellar nuclei, these will also send an inhibitory signal to the inferior olive. The stronger the response in the nuclei is, the stronger the suppression and the weaker the teaching signal from the olive to the cortex becomes. In accordance with this hypothesis, it has been shown that the climbing fiber response elicited by the US is indeed weaker when a CR has been acquired (Hesslow and Ivarsson, 1996; Rasmussen et al., 2008; Sears and Steinmetz, 1991) (Fig. 3). Furthermore, Bengtsson et al. (2007) trained decerebrate ferrets in an eyeblink conditioning paradigm until they had acquired stable CRs. When they then stimulated the nucleo-olivary pathway about 50 ms before the US in a series of paired CS–US trials, the CRs were extinguished with a time course quite similar to that which occurred during unpaired CS trials. This result supports the idea that activity in the nucleo-olivary pathway can block the US signal and induce extinction.
An unusual but highly interesting feature of the nucleo-olivary pathway is the long delay between activation of the nucleo-olivary pathway and the inhibition of the inferior olive (Fig. 2B). If one stimulates the pathway directly using electrical

A

B

Relative field potential amplitude (%)

100 Control *

32 mA

50

*

100 mA * 0
0 40 ms

400 mV

50

100

150

Stimulus interval (ms)

FIGURE 2

Stimulation of the nucleo-olivary pathway causes a suppression of periorbitally elicited field potentials on the cerebellar cortex. (A) Field potentials elicited by periorbital stimulation. The amplitude of the field potential was significantly reduced when the periorbital stimulation was preceded by stimulation of the nucleo-olivary pathway. (B) The suppression of the periorbital field potential was substantially larger when the stimulation of the nucleo-olivary pathway preceded the periorbital stimulation by at least 40 ms.

Adapted from Hesslow (1986).

110 CHAPTER 5 Feedback Control of Learning

A

US (control)

After conditioning

200 µV

CS + US 10 ms

B

400

Naive

Conditioning

After extinction

After reconditioning

Extinction Reconditioning

Field potential amplitude (µV)

0 0

C 200

Before training

US CS + US Training sessions
Complex spikes Simple spikes

10 After training

1000

Simple spikes (%)

Complex spikes (%)

0 CS 300 ms US

0 CS 300 ms

FIGURE 3

Conditioned responses suppress olivary activity. (A) Sample sweeps demonstrating that after acquisition of conditioned eyeblink responses, the field potential elicited by periorbital stimulation is suppressed when preceded by the conditional stimulus. (B) The average amplitude of the periorbitally elicited field potential during different phases of conditioning. (C) Complex spike activity, which reflects olivary activity, is suppressed when Purkinje cells have acquired a conditioned pause response.

(A, B) adapted from Hesslow and Ivarsson (1996) and (C) adapted from Rasmussen et al. (2008).

7 Reaching Equilibrium 111
stimulation, the main inhibition of the olive occurs with a 25–75 ms delay (Hesslow, 1986; Svensson et al., 2006). This appears to be caused by asynchronous GABA release onto the IO (Best and Regehr, 2009). One important implication of this delay is that the olivary inhibition resulting from the Purkinje cell CRs (Fig. 3) should reach its maximum at about the same time that the US arrives at the inferior olive. If this delay had not existed, the inhibition would arrive too early to have any effect on the US (Lepora et al., 2010). The nucleo-olivary inhibition explains why Purkinje cell activity correlates with subsequent complex spike activity (Miall et al., 1998) and why Purkinje cell CRs (Jirenhed et al., 2007) result in a suppression of olivary activity (Hesslow and Ivarsson, 1996; Rasmussen et al., 2008).
7 REACHING EQUILIBRIUM
When the inferior olive receives input from another part of the brain, it typically releases more than one action potential up through the climbing fibers. It was observed several decades ago that the inferior olive fires in high-frequency bursts (>250 Hz) (Armstrong and Rawson, 1979; Eccles et al., 1966), but the potential implications of this observation have long been overlooked. Indeed, many researchers have implicitly or explicitly assumed that the IO fires in an “all-or-none” fashion (Ito, 2001). However, recently a handful of papers specifically addressing the burst firing nature of the inferior olive and its functional implications have been published (Maruta et al., 2007; Mathy et al., 2009; Najafi and Medina, 2013; Rasmussen et al., 2013). Collectively, these papers demonstrate that the inferior olive fires in bursts containing 1–6 pulses, and that the number of EPSPs elicited in the Purkinje cell dendrite matches the number pulses in the climbing fibers (Fig. 4).
A model with “all-or-none” complex spikes (Ito, 2001) would permit learning, assuming that US elicited complex spikes are suppressed when preceded by a CR. The direction of learning would then depend on the probability that a complex spike is elicited. However, all-or-none complex spikes cannot provide information about the size of an error (Herreros and Verschure, 2013; Najafi and Medina, 2013). The fact that the IO fires in bursts potentially enables the negative feedback from the cerebellar cortex to alter the number of pulses in the IO burst. Such a graded US signal not only results in a more fine-tuned system but is actually a criterion for some theoretical models of cerebellar function (Herreros and Verschure, 2013; Lepora et al., 2010; Najafi and Medina, 2013).
Thus, rather than blocking the teaching signal completely, the negative feedback could potentially alter the number of spikes in the climbing fiber bursts (Fig. 5). If this were the case, the number of pulses in the climbing fiber signal would reflect both the degree of learning and the size of the error (Najafi and Medina, 2013). The number of pulses in the climbing fiber signal may in turn determine which, if any, plastic changes are triggered in the cerebellar cortex. In support of this idea, we recently demonstrated that whereas a US consisting of three or more climbing fiber impulses leads to the acquisition of Purkinje cell CRs, a US consisting of a

A
Periorbital

1 CF

2 CF

3 CF

stim

Spontanous pulse pulses pulses

** *

*

*

*

*

**

*

*

**

**

*

*

*

* ****

*

1 mV

10 ms
B Trials
700

* EPSPs

0

100 SS%

1 pulse I

600

3 pulses III

2 pulses II

500 400 300 200

1 pulse I

2x5 pulses IIIII IIIII

100

0 CS (300 ms)

0

100 SS %

Cell trained with 450 paired trials prior to start

FIGURE 4
Cerebellar learning requires burst of pulses in the climbing fibers. (A) Representative sweeps from intracellular recordings showing that the number of elicited EPSPs in Purkinje cell dendrites corresponds to the number of stimulus pulses applied to the climbing fibers, and that peripheral, periorbital stimulation elicits multiple EPSPs (cf. Mathy et al., 2009). (B) Acquisition of Purkinje cell pause responses only occurs when a burst of pulses is delivered to the climbing fibers. When a single stimulus is applied, the pause response is extinguished.
Adapted from Rasmussen et al. (2013).
ResponsePurkinje CellsHesslowCerebellar CortexFibers