Cerebellar Learning [1st Edition] 9780444634269, 9780444633569

Table of contents :
Content:
Series PagePage ii
CopyrightPage iv
ContributorsPages v-vi
PrefacePages vii-viiiNarender Ramnani
Chapter 1 - Long-Term Depression as a Model of Cerebellar PlasticityPages 1-30Masao Ito, Kazuhiko Yamaguchi, Soichi Nagao, Tadashi Yamazaki
Chapter 2 - The Organization of Plasticity in the Cerebellar Cortex: From Synapses to ControlPages 31-58Egidio D'Angelo
Chapter 3 - Questioning the Cerebellar DoctrinePages 59-77Elisa Galliano, Chris I. De Zeeuw
Chapter 4 - Distribution of Neural Plasticity in Cerebellum-Dependent Motor LearningPages 79-101Michael Longley, Christopher H. Yeo
Chapter 5 - Feedback Control of Learning by the Cerebello-Olivary PathwayPages 103-119Anders Rasmussen, Germund Hesslow
Chapter 6 - Cerebellum-Dependent Motor Learning: Lessons from Adaptation of Eye Movements in PrimatesPages 121-155Suryadeep Dash, Peter Thier
Chapter 7 - Decorrelation Learning in the Cerebellum: Computational Analysis and Experimental QuestionsPages 157-192Paul Dean, John Porrill
Chapter 8 - Modeling the Evolution of the Cerebellum: From Macroevolution to FunctionPages 193-216Jeroen B. Smaers
Chapter 9 - Cerebellar and Prefrontal Cortex Contributions to Adaptation, Strategies, and Reinforcement LearningPages 217-253Jordan A. Taylor, Richard B. Ivry
Chapter 10 - Automatic and Controlled Processing in the Corticocerebellar SystemPages 255-285Narender Ramnani
IndexPages 287-294
Volume in SeriesPages 295-297

Citation preview

Advisory Editors

Stephen G. Waxman

Bridget Marie Flaherty Professor of Neurology Neurobiology, and Pharmacology; Director, Center for Neuroscience & Regeneration/Neurorehabilitation Research Yale University School of Medicine New Haven, Connecticut USA

Donald G. Stein

Asa G. Candler Professor Department of Emergency Medicine Emory University Atlanta, Georgia USA

Dick F. Swaab

Professor of Neurobiology Medical Faculty, University of Amsterdam; Leader Research team Neuropsychiatric Disorders Netherlands Institute for Neuroscience Amsterdam The Netherlands

Howard L. Fields

Professor of Neurology Endowed Chair in Pharmacology of Addiction Director, Wheeler Center for the Neurobiology of Addiction University of California San Francisco, California USA

Elsevier Radarweg 29, PO Box 211, 1000 AE Amsterdam, The Netherlands The Boulevard, Langford Lane, Kidlington, Oxford, OX5 1GB, UK First edition 2014 Copyright # 2014 Elsevier B.V. All rights reserved No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means electronic, mechanical, photocopying, recording or otherwise without the prior written permission of the publisher Permissions may be sought directly from Elsevier’s Science & Technology Rights Department in Oxford, UK: phone (þ44) (0) 1865 843830; fax (þ44) (0) 1865 853333; email: [email protected]. Alternatively you can submit your request online by visiting the Elsevier web site at http://elsevier.com/locate/permissions, and selecting Obtaining permission to use Elsevier material Notice No responsibility is assumed by the publisher for any injury and/or damage to persons or property as a matter of products liability, negligence or otherwise, or from any use or operation of any methods, products, instructions or ideas contained in the material herein. Because of rapid advances in the medical sciences, in particular, independent verification of diagnoses and drug dosages should be made Library of Congress Cataloging-in-Publication Data A catalog record for this book is available from the Library of Congress British Library Cataloguing in Publication Data A catalogue record for this book is available from the British Library ISBN: 978-0-444-63356-9 ISSN: 0079-6123 For information on all Elsevier publications visit our website at store.elsevier.com Printed and bound in Great Britain 14 15 16 11 10 9 8 7

6 5 4 3

2 1

Contributors Egidio D’Angelo Department of Brain and Behavioral Sciences, University of Pavia, and Brain Connectivity Center, C. Mondino National Neurological Institute, Pavia, Italy Suryadeep Dash Robarts Research Institute, Western University, London, Ontario, Canada Chris I. De Zeeuw Department of Neuroscience, Erasmus MC Rotterdam, Rotterdam, and Netherlands Institute for Neuroscience, Royal Netherlands Academy of Arts & Sciences, Amsterdam, The Netherlands Paul Dean Department of Psychology, Sheffield University, Sheffield, United Kingdom Elisa Galliano Department of Neuroscience, Erasmus MC Rotterdam, Rotterdam, The Netherlands Germund Hesslow Department of Experimental Medical Science, Lund University, Lund, Sweden Masao Ito RIKEN Brain Science Institute, Saitama, Japan Richard B. Ivry Department of Psychology, and Helen Wills Neuroscience Institute, University of California, Berkeley, CA, USA Michael Longley Department of Neuroscience, Physiology and Pharmacology, University College London, London, UK Soichi Nagao RIKEN Brain Science Institute, Saitama, Japan John Porrill Department of Psychology, Sheffield University, Sheffield, United Kingdom Narender Ramnani Department of Psychology, Royal Holloway, University of London, Egham, UK Anders Rasmussen Department of Experimental Medical Science, Lund University, Lund, Sweden Jeroen B. Smaers Department of Anthropology, Stony Brook University, Stony Brook, NY, USA Jordan A. Taylor Department of Psychology, Princeton University, Princeton, NJ, USA

v

vi

Contributors

Peter Thier Department of Cognitive Neurology, Hertie Institute for Clinical Brain Research, University of Tuebingen, Tuebingen, Germany Kazuhiko Yamaguchi RIKEN Brain Science Institute, Saitama, Japan Tadashi Yamazaki Graduate School of Informatics and Engineering, The University of Electro-Communications, Tokyo, Japan Christopher H. Yeo Department of Neuroscience, Physiology and Pharmacology, University College London, London, UK

Preface The idea that cerebellar circuitry plays an important role in learning, the subject of this volume, is one that has robustly withstood the test of time at least since the seminal papers of Brindley and Marr in the 1960s. In recent years, there have been important transformations in our understanding of cerebellar biochemistry, anatomy, and physiology that have changed the way that we think about cerebellar mechanisms that support learning and the forms of behavior that the cerebellum can control. Researchers have revised their accounts of cerebellar learning to account for these changes, and this volume brings some of these together. As with most topics in behavioral neuroscience, these accounts need to span multiple levels of analysis, from molecules through to behavior. The organization of the volume reflects this range. Masao Ito contributes the opening chapter to the volume with an account of the cellular mechanisms that support long-term depression as a classical model of cerebellar plasticity. Egidio D’Angelo then discusses the potential for multiple forms of plasticity to exist in the cerebellum. In the third chapter, Elisa Galliano and Chris De Zeeuw continue the narrative that challenges three long-held traditional ideas and propose ways in which these could be revised. There follow three chapters that study cerebellar learning using relatively simple, cerebellar-dependent forms of learning in well-understood models. The benefits of these models include relatively specific questions that can be addressed and high levels of experimental control that can be achieved. In the first of these, Michael Longley and Christopher Yeo discuss the value of using the classically conditioned eyeblink and nictitating membrane responses to study cerebellar mechanisms of learning, and summarize findings from lesion and inactivation experiments. The chapter makes comparisons between learning mechanisms that support eye blink and NMR conditioning with those that support the vestibulo-ocular reflex (VOR). In the next chapter, Anders Rasmussen and Germund Hesslow discuss how work using classical eyeblink conditioning has contributed to an understanding of how learning-related feedback is itself regulated by learning-related cerebellar outputs. In the last of the three chapters that focus on simple behaviors, Suryadeep Dash and Peter Thier discuss the roles played by specific areas of the primate cerebellar cortex in the adaptation of three kinds of eye movement behaviors (the adaptation of the VOR, saccades, and smooth pursuit). In Chapter 7, Paul Dean and John Porrill contribute a computational account of decorrelation learning in the cerebellum, where parallel fiber synapses are weakened if they correlate positively with climbing fiber input but strengthened if they are negatively correlated. The authors discuss the application of this approach to motor control, sensory prediction, and higher cognitive function. The evolution of the corticocerebellar system is an important subject in its own right, but in this literature little attention has been paid to it in the context of cerebellar learning. Jeroen Smaers therefore contributes a chapter that takes a

vii

viii

Preface

comparative approach in which he discusses the evolutionary expansion of the cerebellum in the context of enhanced motor and cognitive skills. The last two chapters focus on cerebellar interactions with frontal lobe areas and the roles that this plays in more complex forms of learning. They acknowledge the fact that such learning must involve interactions between explicit and implicit processes involving both the cerebellum and the neocortex. In the first, Jordan Taylor and Richard Ivry consider the role of the cerebellum in adaptation, strategy, and reinforcement learning. They suggest that cerebellar mechanisms are engaged in the skilled movement execution. However, the authors are skeptical about the involvement of cerebellar circuitry in skilled action selection and suggest that prefrontal and basal ganglia mechanisms are likely to play a prominent role in this. I contribute the final chapter of the volume and focus on similar issues. In contrast to the authors of the previous chapter, I suggest that the cerebellum might play an important role in the acquisition of both motor and cognitive skills and in the formation of habits through instrumental learning. I also suggest that just as cerebellar outputs suppress the processing of error feedback in classical eyeblink conditioning, it might also suppress higher forms of feedback such as reward error in more complex forms of learning. The study of cerebellar learning has a very broad span and this research area is growing fast. It is not possible to do justice to all of the ideas in the field in a single volume, but the hope is that the chapters in this volume reflect current perspectives that range from molecular and cellular mechanisms of learning-related plasticity, through computational accounts and simple forms of learning in animals, to systems-level accounts of complex learning in humans. Narender Ramnani

CHAPTER

Long-Term Depression as a Model of Cerebellar Plasticity

1

Masao Ito*,1, Kazuhiko Yamaguchi*, Soichi Nagao*, Tadashi Yamazaki{ *

RIKEN Brain Science Institute, Saitama, Japan Graduate School of Informatics and Engineering, The University of Electro-Communications, Tokyo, Japan 1 Corresponding author: Tel.: þ81-48467-6984; Fax: þ81-48467-6975, e-mail address: [email protected]

{

Abstract Long-term depression (LTD) here concerned is persistent attenuation of transmission efficiency from a bundle of parallel fibers to a Purkinje cell. Uniquely, LTD is induced by conjunctive activation of the parallel fibers and the climbing fiber that innervates that Purkinje cell. Cellular and molecular processes underlying LTD occur postsynaptically. In the 1960s, LTD was conceived as a theoretical possibility and in the 1980s, substantiated experimentally. Through further investigations using various pharmacological or genetic manipulations of LTD, a concept was formed that LTD plays a major role in learning capability of the cerebellum (referred to as “Marr-Albus-Ito hypothesis”). In this chapter, following a historical overview, recent intensive investigations of LTD are reviewed. Complex signal transduction and receptor recycling processes underlying LTD are analyzed, and roles of LTD in reflexes and voluntary movements are defined. The significance of LTD is considered from viewpoints of neural network modeling. Finally, the controversy arising from the recent finding in a few studies that whereas LTD is blocked pharmacologically or genetically, motor learning in awake behaving animals remains seemingly unchanged is examined. We conjecture how this mismatch arises, either from a methodological problem or from a network nature, and how it might be resolved.

Keywords Albus, climbing fiber, long-term depression, LTD, long-term potentiation, LTP, Marr, motor learning, parallel fiber, perceptron

Progress in Brain Research, Volume 210, ISSN 0079-6123, http://dx.doi.org/10.1016/B978-0-444-63356-9.00001-7 © 2014 Elsevier B.V. All rights reserved.

1

2

CHAPTER 1 Cerebellar LTD and Learning

1 A HISTORICAL OVERVIEW OF LTD STUDIES 1.1 Leading Theories Donald Hebb (1904–1985) early proposed synaptic plasticity as a cellular mechanism of memory and learning (Hebb, 1949). In the 1960s, neuronal connections in the cerebellum composed of Purkinje cells, molecular layer interneurons, Golgi cells, granule cells, mossy fibers, climbing fibers, and nuclear neurons were dissected in terms of excitation/inhibition and divergence/convergence (see Eccles et al., 1967). The beauty of the circuitry diagram suggested that the cerebellum is a “neuronal machine” that is engaged in the processing of some important information, but there was no idea about whether the machine is capable of learning or where a memory element is located (see Ito, 2006). David Marr (1945–1980), James Albus, and a few others assumed certain types of synaptic plasticity as such a memory element purely on a theoretical ground (Albus, 1971; Marr, 1969); this was a monumental assumption in the history of cerebellar research (Strata, 2009). Marr (1969) followed Brindley’s (1964) suggestion that, when both parallel fibers and climbing fibers are activated synchronously, parallel fiber–Purkinje cell synapses are activated both presynaptically and postsynaptically, that is, the type of condition that induces a Hebbian form of plasticity (Hebb, 1949), that is, longterm potentiation (LTP). In Marr’s (1969) neural network model, the pathways from mossy fibers to granule cells (the origin of parallel fibers) to Purkinje cells constitute a three-layered associative learning network. Each climbing fiber conveys a cerebral instruction for an elemental movement, and the receiving Purkinje cell is also exposed via the mossy fiber input to information about the context in which the climbing fiber fired. During rehearsal of an action, each Purkinje cell learns to recognize such contexts, and later, after the action has been learned, the occurrence of the context alone is enough to fire the Purkinje cell, which then initiates the next elemental movement. On the other hand, Albus (1971) assumed that the opposite, that is, synchronous activation of parallel fibers and climbing fibers leads to long-term depression (LTD). Albus’ (1971) neural network model is a close analogy to Rosenblatt’s (1962) simple perceptron, the first man-made learning machine. In this model, climbing fibers act as an outside teacher who changes the intensity of those parallel fiber–Purkinje cell synapses activated at that moment. When the performance of the cerebellum is erroneous, relevant climbing fibers send error signals and thereby depress concurrently activated parallel fiber–Purkinje cell synapses that are responsible for the erroneous performance of the cerebellum. Both the Marr and Albus models were primarily designed for discrimination of spatial patterns and have no capability of discriminating temporal patterns. A decade later, Fujita (1982) proposed an adaptive filter model of the cerebellum that was capable of discriminating temporal patterns. The essential assumption in Fujita’s model is that the neuronal circuits involving mossy fibers, granule cells, and Golgi cells constitute a phase converter, which generates a set of multiphase versions of a mossy fiber input in parallel fibers. The adaptive filter model of the cerebellum has

1 A Historical Overview of LTD Studies

been developed further (Dean and Porrill, 2011) and is discussed further in a later chapter. By amalgamating these models, current neural network models of the cerebellum can deal with spatiotemporal information (e.g., Buonomano and Mauk, 1994; Yamazaki and Tanaka, 2009).

1.2 Experimental Approach to LTD and Motor Learning Early efforts to observe LTP or LTD in Purkinje cells experimentally were hampered by various technical difficulties, but eventually evidence for LTD was obtained by in vivo electrophysiological experiments (Ekerot and Kano, 1985; Ito, 1989; Ito and Kano, 1982; Ito et al., 1982). Then, the successful reproduction of LTD in in vitro cerebellar slices (Crepel and Jaillard, 1991; Karachot et al., 1994; Sakurai, 1987; Schreurs et al., 1996) and tissue-cultured Purkinje cells (Hirano, 1990; Linden, 1991; Shigemoto et al., 1994) greatly facilitated characterization of cerebellar LTD and analyses of molecular processes underlying LTD. Since the 1990s, signal transduction processes that occur in synaptic spines to induce LTD have been analyzed in detail (see Section 2). While LTD was drawing attention, Sakurai (1987) early recognized that, if not accompanied by conjunctive activation of climbing fibers, repetitive stimulation of parallel fibers alone leads to persistent potentiation of the parallel fiber–Purkinje cell transmission. Later studies revealed an important role of this type of LTP as the counterpart of LTD (see Sections 2 and 4). In parallel with these progresses at cellular/molecular levels, system level studies have also progressed. Several types of motor learning have been identified for analyzing the learning mechanism of the cerebellum (Section 3). Examples include adaptation of horizontal vestibulo-ocular reflex (HVOR), horizontal optokinetic eye movement response (HOKR), saccade, eyeblink conditioning, hand reaching, and cursor tracking. Neuronal circuits for each of these functions involve a combination of a small cerebellar cortical area (microzone) and a corresponding small group of vestibular or cerebellar nuclear neurons. This corticonuclear microcomplex receives mossy fiber inputs and converts them to nuclear outputs. Each microcomplex receives climbing fiber inputs from a small group of inferior olivary neurons for learning (see Section 3), and peptidergic or aminergic inputs for neuromodulation (e.g., Nisimaru et al., 2013). A microcomplex constitutes a unit of cerebellar learning machinery. Note that, within a microcomplex, whereas LTD occurs in the microzone, another memory element, LTP, occurs at the mossy fiber–nuclear neuron synapses. With both cortical LTD and nuclear LTP involved, learning proceeds in two steps (see Section 3).

1.3 Memory Mechanisms in the Cerebellum It has been more than 40 years since Marr (1969) and Albus (1971) published their theoretical papers. A central and debated issue that follows up these papers concerns the role of climbing fibers in motor learning. We recall earlier discussions about

3

4

CHAPTER 1 Cerebellar LTD and Learning

whether climbing fiber signals represent errors in the motor performance (Ito, 2001, 2002a). On the basis of accumulating evidence indicating that climbing fiber discharges represent errors in motor learning, we take the view that climbing fiber-driven LTD is a major memory mechanism in the cerebellar cortex (Ito, 2000, 2001, 2013). This and relevant notions may collectively be called the Marr–Albus–Ito hypothesis (see Schonewille et al., 2011). On the other hand, however, the cerebellum is regarded as a control machine without learning (Rokni et al. 2008), or LTD has no role in learning (Welsh et al., 2005). The significance of LTD as an essential element of a learning neuronal network will be discussed in Section 4.

1.4 Recent Issues Very recently, a question has been raised about the Marr–Albus–Ito hypothesis on the basis of the finding that, although many pharmacological or genetic manipulations that block LTD also block motor learning (Ito, 2000, 2001, 2013), a few cases have shown a mismatch. In them, LTD was blocked, but motor learning remained virtually unaffected (Schonewille et al., 2011; Welsh et al., 2005). Various reasons, either technical or fundamental, are presently explored to explain this mismatch, and we will discuss them in Section 5.

2 MOLECULAR MECHANISMS OF LTD The past two decades have been particularly fruitful in the analysis of molecular mechanisms of LTD, as reviewed previously (Daniel et al., 1998; Ito, 2001, 2002b). The availability of specific inhibitors and gene manipulation techniques (especially, Purkinje cell-specific ones) provides a powerful tool to explore the molecules involved in LTD (e.g., Aiba et al., 1994; De Zeeuw et al., 1998). Underlying LTD, complex signal transduction processes are triggered by parallel fiber and climbing fiber inputs, and proceed in dendritic spines of Purkinje cells in three successive major phases (I, II, and III). Eventually, a-amino-3-hydroxy-5-methyl-4isoxazolone propionic acid (AMPA)-type glutamate receptors (AMPA receptors) on the synaptic membrane are destabilized and removed from the synaptic membrane by internalization. This results in a decrease in the number of AMPA receptors on the synaptic membrane, which is represented by a decrease of parallel fiber-stimulation-evoked excitatory postsynaptic current or potential (PF-EPSC or PF-EPSP), that is, LTD.

2.1 Signal Transduction Underlying LTD Figure 1 shows a flowchart of chemical signals for the induction of LTD integrating the results reported from many laboratories. In phase I, climbing fiber signals induce the strong entry of Ca2þ ions through voltage-sensitive Ca2þ channels. Parallel fiber signals also contribute to the Ca2þ responses because parallel fiber-released

2 Molecular Mechanisms of LTD

Ca2+

dep CF

VSCC

I

II

AMPAR

PKC

Ca2+ PF

PGD2/E2 AA

MEK

mGluR1 Gq/G11 PLC

+P

Raf

surge

AMPAR

III

MAPK IP3

IP3 R

COX-2

AMPAR -P

D Ps M m M

PF LTD

i

cPLA2a CaMKII

* PDE1

cGM P

PKG

* PP2A

GC NO

FIGURE 1 Signal transduction and receptor recycling in LTD induction. Chain of reactions induced by conjunctive stimulation of parallel fibers (PFs) and climbing fibers (CFs). I, II, and III indicate the three phases of the signal transduction process. AA, arachidonic acid; dep, depolarization; Gq/G11, subtypes of G-protein; IP3R, IP3 receptor; MEK, MAPK (mitogen-activated protein kinase) kinase; mGluR1, metabotropic glutamate receptor type 1; PGD2/E2, prostaglandin D2 or E2; þP. phosphorylation; P, dephosphorylation; PLC, phospholipase C (PLCb3 or 4 subtypes); Raf, kinase related to MAP kinase; *, depressant action. Other abbreviations are defined in the text.

glutamate activates metabotropic glutamate receptor type 1 (mGluR1), which in turn activates phospholipase C (PLC) to generate inositol-triphosphate (IP3). IP3 induces the release of Ca2þ from intracellular stores. The coincident activation of parallel fibers and a climbing fiber converging onto the test Purkinje cell produces Ca2þ signals much larger than the linear sum of responses to either parallel fiber or climbing fiber activation, leading to an abrupt surge in the Ca2þ concentration of Purkinje cell dendrites (Wang et al., 2000). In phase II, the Ca2þ surge induced in phase I activates Ca-dependent enzymes such as the a subtype of protein kinase C (PKCa), cytosolic phospholipase A2a (cPLA2a), and Ca2þcalmodulin kinase II (CaMKII). The activated PKC then phosphorylates AMPA receptors to induce LTD, whereas the activated cPLA2a produces arachidonic acid, which in turn activates cyclooxygenase-2 (COX-2). The activated COX-2 finally generates prostaglandin D2 or E2, which is required for LTD induction (Le et al., 2010). The PKC and cPLA2a are interconnected by the MAPK (MAP kinase)–cPLA2a–arachidonic acid–PKC–Raf–MEK–MAPK positive-feedback loop pathway (Fig. 1), which may function in LTD persistence (Kuroda et al., 2001; Tanaka and Augustine, 2008). Recently, Kawaguchi and Hirano (2013) reported that the CaMKII activated by the Ca2þ surge suppresses phosphodiesterase 1, which subsequently facilitates the cGMP/protein kinase G (PKG) cascade. The activated PKG

5

6

CHAPTER 1 Cerebellar LTD and Learning

phosphorylates the G-substrate (GS), which inhibits two protein phosphatases, PP1 and PP2A, leading to increased dephosphorylation of AMPA receptors, antagonizing the phosphorylating action of PKC (Endo et al., 1999, 2003). Note that the CaMKII pathway can also be activated by nitric oxide (NO). When parallel fiber impulses activate neuronal nitric oxide synthase (n-NOS) in parallel fibers, the produced NO diffuses to the postsynaptic side of parallel fiber–Purkinje cell synapses (Shibuki and Kimura, 1997). There, via the cGMP–PKG–GS–protein phosphatases pathway, NO eventually dephosphorylates AMPA receptors. GSdeficient mutant mice exhibit LTD that is abolished at postnatal weeks 5–6; before or after these weeks, LTD occurs normally (Endo et al., 2009). This finding suggests that the NO–cGMP–G-substrate cascade is required for LTD for only a few weeks postnatally. This peculiar age dependence of LTD may be relevant to the finding that cultured Purkinje cells (12–17 days in vitro) obtained from n-NOS-null mice exhibited LTD (Linden et al., 1995). Note that NO is also implicated in LTP (see Section 2.3), and it is guessed that CaMKII and NO switch LTD to LTP and vice versa (van Woerden et al., 2009). In phase III, the above-described signal transduction pathways converge onto AMPA receptors to control their trafficking, as discussed below. The AMPA receptor is a heteromeric tetramer protein, and its four subunits (GluA1–4) have been identified (Traynelis et al., 2010). In contrast to the complex subunit composition and composition-dependent trafficking of AMPA receptors in hippocampal pyramidal cells (Shi et al., 2001), AMPA receptors in parallel fiber synapse membrane on Purkinje cells are composed of only GluA2/3 (Douyard et al., 2007).

2.2 Constitutive Trafficking of AMPA Receptors AMPA receptors recycle among different pools to maintain their distribution in the basal state by a synaptic activity-independent (constitutive) trafficking mechanism. The size of the pool located in the synaptic membrane is reflected in the amplitude of parallel fiber-stimulation-induced PF-EPSC or PF-EPSP in Purkinje cells, which would be proportionate to the number of AMPA receptors (Linden, 2001). The number of AMPA receptors in the synaptic membrane should be determined from the balance between constitutive elimination of AMPA receptors via endocytosis and constitutive insertion of AMPA receptors via exocytosis. When endocytosis is blocked, the number of AMPA receptors should increase. In contrast, when exocytosis is blocked, the number of AMPA receptors should decrease. Intracellular infusion of tetanus toxin (TeTx, 150 nM), a blocker of soluble N-ethylmaleimide-sensitive fusion protein (NSF) attachment protein receptor (SNARE)-dependent exocytosis, into a Purkinje cell, decreased the PF-EPSC amplitude to 60%. This reduction was presumed to be maximal because an increase in TeTx concentration to 500 nM or the use of botulinum neurotoxin C (1 mM) instead of TeTx caused a similar decrease to approximately 60% (Tatsukawa et al., 2006). Therefore, the 60% fraction of PF-EPSP amplitude should reflect the size of the “stabilized synaptic pool” of AMPA receptors bound to the synaptic membrane, whereas

2 Molecular Mechanisms of LTD

the remaining 40% fraction represents the “mobile synaptic pool.” In the presence of TeTx that blocks exocytosis, AMPA receptors in the mobile synaptic pool should be removed from the synaptic membrane by endocytosis and internalized to the “internal mobile pool,” as visualized immunocytochemically in cultured Purkinje cells (Tatsukawa et al., 2006). In contrast, dynasore, a membrane-permeable dynamin inhibitor I (Macia et al., 2006), which blocked endocytosis, increased PF-EPSC by the shift of AMPA receptors from the internal mobile pool to the mobile synaptic pool (Yamaguchi and Nagao, 2012). These observations indicate that substantial AMPA receptor recycling proceeds constitutively among the three distinct pools: the stable synaptic pool, mobile synaptic pool, and mobile internal pool (s, m, and i in Fig. 1). Constitutive AMPA receptor recycling may be driven by signal transductions shown in phases II and III in Fig. 1, which may maintain basal activities even without the strong drive from Ca2þ surge. Indeed, Go¨9676, a PKCa/b1 inhibitor, attenuates the TeTx-induced decrease in PF-EPSC amplitude, suggesting that constitutive endocytosis of AMPA receptors depends on the basal activity of PKC (Tatsukawa et al., 2006). PD98059, a specific MEK inhibitor, also induces similar effects, suggesting the involvement of MEK in the basal activity of endocytosis (Tatsukawa et al., 2006). Actin also appears to be involved in constitutive AMPA receptor trafficking because jasplakinolide, an inhibitor of actin depolymerization, attenuates TeTx-induced receptor rundown and increases the size of the stabilized synaptic AMPA receptor pool (Tatsukawa et al., 2006).

2.3 Receptor Recycling in LTP LTP concerned here is the persistent increase in PF-EPSC, reflecting an increase in the number of AMPA receptors in parallel fiber–Purkinje cell synapses. In cerebellar slices, this type of LTP is induced by repetitive stimulation of parallel fibers at 1 Hz via NO release at parallel fiber synapses (Lev-Ram et al., 1997; Shibuki and Okada, 1991). When solely applied to slice preparations, an NO donor causes LTP without stimulation of parallel fibers, and an NOS inhibitor or an NO scavenger blocks LTP completely (Lev-Ram et al., 2002). NO-induced increase in the surface expression of GluA2 has been visualized in cultured cerebral cortical neurons (Huang et al., 2005). Because intracellularly applied botulinum neurotoxin strongly suppresses NO donorinduced LTP, persistent dominance of SNARE-dependent exocytosis over endocytosis would underlie LTP (Kakegawa and Yuzaki, 2005). The balance between exocytosis and endocytosis is differentially controlled by the binding of GluA2 to two binding sites, namely, NSF site, required for exocytosis of AMPA receptors, and the adaptor protein 2 (AP-2) site required for endocytosis of these receptors. These two sites partially overlap in the carboxyl terminus (CT) of GluA2 near the transmembrane domain (Lee et al., 2002). NO may directly S-nitrosylate NSF, and S-nitrosylated NSF may increase the affinity of GluA2 to the NSF-binding site. This postulate is supported by an experiment using an inhibitory peptide (pep-R845A; Lee et al., 2002), which specifically blocks the binding between GluA2 and NSF (Kakegawa and Yuzaki, 2005); as is expected,

7

8

CHAPTER 1 Cerebellar LTD and Learning

infusion of pep-R845A into a Purkinje cell decreased PF-EPSC amplitudes and suppressed NO donor-induced LTP. Other factors required for LTP are protein phosphatases PP1, 2A, and 2B (Belmeguenai and Hansel, 2005). LTP is impaired in Purkinje cell-specific PP2Bnull mice without affecting LTD (Schonewille et al., 2010). These protein phosphatases may enhance GluA2 binding to glutamate receptor-interacting protein (GRIP) and consequently strengthens GluA2 stabilization in LTP. A subtle balance between kinase/phosphatase activities would tune the distribution of AMPA receptors among their three pools. Interestingly, aCaMKII, which is required for LTP in hippocampal neurons, is not essential for LTP in Purkinje cells (Hansel et al., 2006).

2.4 Receptor Recycling in LTD LTD concerned here represents a persistent decrease in PF-EPSC elicited by conjunctive stimulation of parallel fibers and climbing fibers. (Note that climbing fiber stimulation is often replaced by strong depolarizing pulses that induce Ca2þ entry.) In LTD, the number of AMPA receptors in the synaptic membrane may decrease owing to the dominance of endocytosis over exocytosis. The finding that intracellularly infused dynamin–prolin-rich domain peptide or amphiphysin SH3 domain peptide effectively blocked LTD (Wang and Linden, 2000) supports this postulate. This is because either of these peptides blocks interaction between dynamin and amphiphysin, which is required for AP-2–clathrin–dynamin-dependent endocytosis. Since the role of PKC in LTD was discovered (Linden and Connor, 1991), the relationship between phosphorylation of GluA2 and the mechanism of AMPA receptor internalization has extensively been explored. As a result, it was found that phosphorylation of GluA2-CT at Ser880 by PKC is essential for LTD induction (Matsuda et al., 2000; Steinberg et al., 2004; Xia et al., 2000). LTD was absent in cultured Purkinje cells from mutant mice lacking GluA2 subunit. Transfection of GluA2 with a point mutation that eliminates PKC phosphorylation of Ser880 fails to restore LTD. Moreover, transfection with a point mutant that mimics phosphorylation at Ser880 occludes subsequent LTD. Thus, PKC phosphorylation of GluA2 at Ser880 is a critical event for LTD in cultured Purkinje cells (Chung et al., 2003). The phosphorylation disrupts the interaction of GluA2-CT with the PDZ domaincontained protein GRIP/ABP on the cell membrane and consequently destabilizes the GluA2/3 AMPA receptor. Moreover, interaction between GluA2 CT and PICK1 is required for AMPA receptor internalization (Steinberg et al., 2004). A guess is that in parallel fiber–Purkinje cell synapses, the GluA2-CT domain of AMPA receptors binds to GRIP to fix the receptors to the synaptic membranes. When PKC phosphorylates Ser880 of GluA2-CT, it may follow that AMPA receptors are disconnected from GRIP and bind to PICK1 for internalization. A mechanism for maintaining LTD in the long term is shown in Fig. 1. PKC is equipped with a positive-feedback loop that includes MAPK, and this feedback activates PKC for more than 20 min (Tanaka and Augustine, 2008) and is stopped by a substrate of PKC, Raf kinase inhibitory protein (RKIP) (Yamamoto et al., 2012).

3 Roles of LTD in Motor Learning

Other factors that may affect LTD are transmembrane AMPA receptor regulatory protein (TARP) and d2 glutamate receptor (GluD2). Stargazin, a member of TARP, is expressed in Purkinje cells, highly phosphorylated in the basal state, and dephosphorylated by LTD-inducing chemical stimulation via calcineurin. Inclusion of the calcineurin autoinhibitory peptide in the patch pipette solution completely inhibits LTD in slices (Nomura et al., 2012). Stargazin possibly contributes to formation of the stabilized synaptic pool of AMPA receptors, and its calcineurin-dependent dephosphorylation may contribute to LTD induction by destabilization of AMPA receptors at the synaptic region. Kohda et al. (2013) have recently reported that GluA2-CT Tyr876 dephosphorylation is sufficient to restore GluA2-CT Ser880 phosphorylation and LTD induction in GluD2-null Purkinje cells. Thus, GluD2 would gate LTD by coordinating interactions between the two phosphorylation sites of GluA2.

3 ROLES OF LTD IN MOTOR LEARNING We now focus our attention on the roles of LTD in cerebellar-controlled motor learning and review recent efforts to identify the microcomplex responsible for each type of motor learning and confirm error representation by climbing fibers. A number of cases have been investigated, but in particular, HOKR and HVOR have been extensively analyzed in mice, rats, rabbits, cats, and monkeys and provide substantial data for examining relationships between LTD and motor learning.

3.1 Adaptation of HOKR HOKR is evoked by the motion of visual surrounding of an animal, and HVOR by the motion of the animal. HOKR and HVOR are functionally closely associated with each other and operate cooperatively to stabilize a visual image during animal movement. They share the neural circuitry of vestibular nuclei, cerebellar flocculus, and extraocular muscle motor neurons in the brainstem (e.g., Ito, 2011). Gain of HOKR is quantified by comparing the magnitude of evoked eye movements with that of visual surrounding. Importantly, gains of HOKR are not fixed: they are under learning control. As shown in Fig. 2A, when a mouse is placed at the center of a cylindrical dot- or striped-screen with its head fixed and trained to view an oscillating screen continuously for 1 h, HOKR gains increase by 30% (Katoh et al., 1998; Nagao 1983). These gain increases, which recover within 24 h, represent “fast adaptation.” When we repeat such a 1-h training daily for 1 week, HOKR gains further increase up to 100%. This increase can be monitored for 3 weeks and represents “slow adaptation” (Shutoh et al., 2006; Wang et al., 2014). Thus, the HOKR adaptation paradigm provides a good experimental model for analyzing the neural mechanisms underlying fast and slow motor learning. For example, fast HOKR adaptation has been examined to reveal the requirement of lipid signaling through the cPLA2a–COX-2 pathway (Fig. 1) for motor learning (Le et al., 2010).

9

10

CHAPTER 1 Cerebellar LTD and Learning

FIGURE 2 Fast and slow HOKR adaptation. (A) Fast and slow HOKR adaptations in mice. Two pictures in the right-upper quadrant show the TV–camera system for eye movement measurement (left) and HOKR training apparatus (right). Mice were trained by 1 h of 0.16 Hz–15 (peak-to-peak) screen oscillation daily for 5 days. (B) Neural circuitry of HOKR and HVOR, and the location of memory trace of fast and slow adaptations. CF, climbing fiber; EOM, extraocular muscle motor neurons; IO, inferior olive; MF, mossy fiber; MFA; memory of fast adaptation; MSA, memory of slow adaptation; MVN, medial vestibular nucleus; NRTP, nucleus reticularis tegmenti pontis; PF, parallel fiber; VO, vestibular organ. (C) Effects of bilateral floccular microinfusion of lidocaine on the adapted HOKR gains. Hollow column shows the mean initial HOKR gain at the first training day. Gray, striped, and filled columns, respectively, show the mean HOKR gain at the start, after 1 h of training, and 30 min after lidocaine infusions on the fourth day. Bars indicate SE. **p < 0.01; ns, p > 0.05 (paired t-test). Panels (A) and (C) are modified from Shutoh et al. (2006).

3 Roles of LTD in Motor Learning

HOKR adaptation gives an opportunity of testing the hypothesis that climbing fibers convey error signals. Purkinje cells in the flocculus H-zone (Nagao, 1988) receive climbing fiber signals, representing retinal slips via the dorsal cap of the inferior olive. Retinal slips represent errors in compensating for a moment of the visual surrounding (represented often by a patterned screen) with an HOKR-evoked eye movement. When a screen is continuously oscillated around a stationary animal, HOKR adaptively increases its gain toward minimization of the retinal slip.

3.2 Adaptation of HVOR HVOR is evoked by head oscillation in darkness. The combined oscillation of the turntable and the surrounding screen for 1–2 h in either an in-phase or an out-ofphase direction induces fast HVOR adaptation. In-phase combination decreases HVOR gains, whereas out-of-phase combination increases HVOR gains. In primates or cats, training with turntable oscillation wearing 0.5 reducing lenses or left–right reversed (Dove) prisms is often used for fast gain-down adaptation, and training with turntable oscillation wearing 2 magnifying lenses for fast gain-up adaptation. Slow HVOR adaptation is induced by repetition of training for fast HVOR adaptation, while the animals wear lenses or Dove prisms continuously for more than 3 days (Anzai et al., 2010; Gonshor and Mellvill-Jones, 1976; Kassardjian et al. 2005; Miles et al., 1980; Robinson, 1976). Unlike slow HOKR adaptation, gains changed by slow HVOR adaptation recover rather quickly within 1–2 days (Anzai et al., 2010; Miles et al., 1980). During HVOR under illumination, movements of visual surrounding relative to head movements induce retinal slips, which drive climbing fiber signals to flocculus Purkinje cells via the dorsal cap. Adaptive changes of HVOR gain, either increase or decrease, occur toward minimization of retinal slips.

3.3 LTD in Fast HOKR Adaptation Figure 2B shows the neural circuitry of HOKR and HVOR. The optokinetic signals that induce HOKR are projected to the medial vestibular nucleus via the pontine tegmental reticular nucleus (NRTP) through the accessory optic tract (Miyashita et al., 1980). NRTP also projects to the middle part of the flocculus, which is called the H-zone (Nagao et al., 1985). The H-zone Purkinje cells directly inhibit the medial vestibular nuclear neurons relaying HOKR and HVOR (Kawaguchi, 1985). Thus, the H-zone constitutes the inhibitory bypath to the brainstem HOKR and HVOR main path (Ito, 1984, 2011). Many lines of experimental evidence supporting the idea that LTD occurring in the H-zone underlies fast HOKR adaptation have been reported. Lesions of the flocculus abolished not only fast adaptation (rabbit: Nagao, 1983; mouse: Katoh et al., 1998) but also slow adaptation (mouse: Shutoh et al., 2006). H-zone Purkinje cells exhibit simple spike activity that correlates well with fast HOKR adaptation and their complex spike activity that reflects retinal slip signals, which are necessary for

11

12

CHAPTER 1 Cerebellar LTD and Learning

induction of adaptation (Nagao, 1988). Floccular microinfusions of drugs (e.g., L-NMMA) that block LTD depress both the fast and slow HOKR adaptations (Katoh et al., 2000; Shutoh et al., 2006). Gene-knockout mice lacking LTD such as n-NOS-null mice (Katoh et al., 2000; Shutoh et al., 2006), mGuR1-null. mice (Shutoh et al., 2002), cPLA2a-null mice (Le et al., 2010), and type-1 IP3 receptornull mice (Sugawara et al., 2013) exhibit impaired fast HOKR adaptation. The role of LTD in fast HVOR adaptation is assumed to be similar to that revealed by pharmacological and gene-knockout experiments of HOKR (described in this Section). A model study implementing LTD well simulated the activity of H-zone Purkinje cells observed during fast HOKR adaptation (Yamazaki and Nagao, 2012). Moreover, a recent electron microscopy (EM) study combined with the freeze-fracture method and immunochemistry in mice revealed evidence of LTD in fast-HOKRadapted H-zone Purkinje cells. Wang et al. (2014) quantified AMPA receptors (GluA2–4) and d2 glutamate receptors (GluD2) in parallel fiber–Purkinje cell synapses and found that immediately after HOKR training for 1 h, the density of AMPA receptors in H-zone parallel fiber–Purkinje cell synapses decreased by 25%, whereas that of GluD2 remained unchanged. Importantly, the density of AMPA receptors in H-zone parallel fiber–Purkinje cell synapses recovered in the mice sacrificed 24 h after the end of 1-h HOKR training. This EM study thus visualized the AMPA receptor internalization underlying LTD as disappearance and reappearance of AMPA receptors in parallel fiber–Purkinje cell synapses (see Section 2).

3.4 Memory Transfer in Slow HOKR Adaptation As shown in Fig. 2C, when bilateral flocculi were shut down by microinfusion of lidocaine immediately after the end of slow HOKR adaptation training, the fast adaptation that increased the HOKR gains by 1-h training immediately before the microinfusion was cancelled, however, the gain increases accumulated through the prior days of slow adaptation HOKR training, was not affected at all (Shutoh et al. 2006). This observation suggests that whereas the memory trace of fast HOKR adaptation is maintained in the flocculus, that of slow adaptation is not maintained there. The observation that HVOR gains increase after slow HOKR adaptation suggests that the memory trace of the slow adaptation is located downstream either in the medial vestibular nucleus or in the extraocular muscle motor neurons (Fig. 2B). The responsiveness of vestibular nuclear neurons was examined by mapping the field response induced by electrical stimulation of vestibular nerves under chlorase-urethane anesthesia (Shutoh et al., 2006). The amplitudes and slopes of the evoked monosynaptic field potentials in the medial vestibular nucleus of slowHOKR-adapted mice were found to be larger than those of non- and fast-HOKRadapted mice, suggesting that the excitability of medial vestibular neurons is actually enhanced by slow HOKR adaptation. Thus, the memory trace of HOKR adaptation is initially acquired in the H-zone parallel fiber–Purkinje cell synapses by LTD, and later by repetition of training, it is transferred to the flocculus-target neurons in the medial vestibular nucleus to be consolidated in long-term memory (Fig. 2B).

3 Roles of LTD in Motor Learning

Similar memory transfer has been observed in slow HVOR adaptation in cats (Kassardjian et al., 2005) and monkeys (Anzai et al., 2010).

3.5 LTD and Memory Transfer Plasticity at mossy fiber–deep cerebellar nuclear neuron synapses has been reported by Pugh and Raman (2006) and Zhang and Linden (2006), and its role in the memory transfer has now been suggested. Genetic manipulation to block the cGMPdependent biochemical cascade (Endo et al., 2009) and deletion of G-proteincoupled orphan receptor 5B (Sano et al., under submission) impaired specifically memory transfer in slow HOKR adaptation. Moreover, suppression of the electrical activity of floccular Purkinje cells in the posttraining period by muscimol microinfusion impairs memory transfer (Okamoto et al., 2011b). Blockade of de novo local protein synthesis in the flocculus during the training period by anisomycin or actinomycin D microinfusion produced the same effect (Okamoto et al., 2011a). Therefore, both the electrical and metabolic activities of floccular Purkinje cells are required for the induction of plasticity in the medial vestibular nucleus. It appears that certain molecules synthesized in Purkinje cells during fast HOKR adaptation are transported to the medial vestibular nucleus to induce plasticity there (Okamoto et al., 2011a). Two recent EM studies of slow HOKR adaptation (Aziz et al., 2014; Wang et al., 2014) have consistently demonstrated that the number of parallel fiber–Purkinje cell synapses decreased by 30% in the H-zone, in parallel with slow HOKR adaptation. Importantly, the decrease in the number of synapses started 4 h after the completion of memory transfer. These observations suggest two possible new functions of LTD. One is that LTD, which initially downregulates AMPA receptors in parallel fiber–Purkinje cell synapses, may trigger the elimination of some of the downregulated synapses. Another is that the synaptic reorganization of the H-zone caused by the elimination of parallel fiber–Purkinje cell synapses may support the memory trace of slow adaptation maintained in the medial vestibular nucleus.

3.6 Saccade and Other Reflexes Saccadic eye movements exhibit the well-defined saccade adaptation (McLaughlin, 1967), which is exerted by Purkinje cells in the posterior vermis (Soetedjo et al., 2008). These cells receive error signals via the superior colliculus and the subnucleus b of the medial accessory nucleus of the inferior olive (Kojima et al., 2007). The classical conditioning of the eyeblink reflex is another example of cerebellumcontrolled motor learning (Lincoln et al., 1982; Yeo et al., 1985a,b,c). In this conditioning, the air puff to the cornea that implies error in the closure of eye lids to avoid the corneal stimulation evokes climbing fiber responses in Purkinje cells of hemispheric lobule VI. The conditioned eyeblink responses so developed are an adaptation to avoid harmful cornea stimulus.

13

14

CHAPTER 1 Cerebellar LTD and Learning

Besides somatic reflexes, the cerebellum also controls autonomic reflexes. A recent study revealed that the folium-p of the flocculus is involved in redistribution of arterial blood flow between active muscles and visceral organs in defense behavior (Nisimaru et al., 2013). This is to respond to the demand for arterial blood in active muscles during defense reactions, yet maintaining blood pressure at a slightly elevated level. Folium-p Purkinje cells generate climbing fiber responses in response to aortic nerve stimulation signaling excessively high blood pressure, which implies an error in redistribution of arterial blood flow.

3.7 Arm Movement Recently, a neural network model has been proposed for the prism adaptation of throwing arm movement in humans (Nagao et al., 2013). In the study by Martin et al. (1996), the fast adaptation to the new prism occurred in the same manner when fast adaptation occurred to the old prism 6 weeks ago, while maintaining the memory of slow adaptation of 6 weeks of training to the old prism. This observation suggests that the memory of slow adaptation is maintained in the long term. In the neural network model, the memory of fast prism adaptation is formed at parallel fiber–Purkinje cell synapses through LTD, and the memory of slow prism adaptation is formed at mossy fiber–deep cerebellar nuclear neuron synapses through memory transfer (Nagao et al., 2013). The model suggests that these two memories act independently of each other to enable us to similarly learn a novel motor skill at any time, while preserving memories of old motor skills.

4 SIGNIFICANCE OF LTD IN CEREBELLAR NEURAL NETWORK Here, we consider the significance of LTD in the cerebellar neural network from theoretical viewpoints.

4.1 Role of Plasticity: Memory Formation Versus Signal Enhancement Generally, synaptic plasticity has at least two roles: one is to form memory, which is critical for learning, and the other is to enhance signals for robust neurotransmission. The latter, for example, improves signal-to-noise ratio and helps tune for precise spike timing. Although several plasticity mechanisms besides the presently highlighted postsynaptic LTD at PF–Purkinje cell synapses are distributed within the cerebellar cortex (Fig. 3A), including presynaptic LTD, and pre- and postsynaptic LTPs at parallel fiber–Purkinje cell synapses (Lev-Ram et al., 2002; Qiu and Kno¨pfel, 2009; Sakurai, 1987; Belmeguenai et al., 2010; Schonewille et al., 2010), enhancement of GABAergic transmission of inhibitory synapses on Purkinje cells called rebound potentiation (Kano et al., 1992), bidirectional plasticity at parallel fiber–molecular layer interneuron synapses (Jo¨rntell and Ekerot, 2002, 2003),

4 Significance of LTD in Cerebellar Neural Network

FIGURE 3 Simulation of granule cell activity during HOKR. (A) Schematic illustration of cerebellar circuitry and locations of plasticity sites distributed within the cortex. Numbers in square brackets represent the references that report the plasticity: [1] Andreescu et al. (2011), [2] Seja et al. (2012), [3] Schonewille et al. (2010), [4] Schonewille et al. (2011), [5] Jo¨rntell and Ekerot (2002), and [6] Jo¨rntell and Ekerot (2003). (B) Simulated granule cell activity in response to sinusoidally oscillating mossy fiber signals by a spiking network model of the cerebellum (Yamazaki and Nagao, 2012). Top and middle panels represent the spike patterns in two examples of granule cells during 100 cycles of sinusoidal screen oscillation at 0.5 Hz in a simulation of HOKR. In each panel, gray dots represent spikes, and the black line plots the average firing rate (spikes/s). Horizontal axis represents time, and vertical axis the cycle number of screen oscillation. The granule cell in the top panel exhibit in-phase modulation with respect to the modulation of the mossy fiber input signals, whereas the cell in

15

16

CHAPTER 1 Cerebellar LTD and Learning

intrinsic plasticity of granule cells (Seja et al., 2012), and bidirectional plasticity at mossy fiber–granule cell synapses (D’Angelo et al., 1999), some of them may be more important in signal transmission than in memory formation. Indeed, blocking LTP at mossy fiber synapses or intrinsic plasticity of granule cells disrupts phase reversal adaptation of HVOR, whereas baseline performance and gain adaptation remain intact (Andreescu et al., 2011; Seja et al., 2012). For phase reversal of HVOR, a population of granule cells whose activity modulates out of phase with the head rotation would be indispensable. Mossy fibers to a flocculus would convey vestibular signals modulating either in phase or out of phase with head rotation sensed by ipsilateral or contralateral labyrinths. Although the flocculus receives mossy fiber inputs from labyrinths of both sides (Shinoda and Yoshida, 1975), those projections are mainly from the ipsilateral labyrinth (Nagao et al., 1997; Osanai et al., 1999). This observation implies that granule cells receive mainly in-phase modulating signals from mossy fibers. A computational study (Yamazaki and Nagao, 2012) demonstrates that in-phase modulation of mossy fiber activity is converted to out-of-phase modulation of granule cells by the recurrent network with Golgi cells (Fig. 3B). The population of out-of-phase modulating granule cells, however, is relatively small. These observations imply that some mechanisms are necessary to amplify such out-of-phase modulating signals. The intrinsic plasticity of granule cells may be indispensable for this purpose. In addition, computational studies using simulated Pavlovian delay eyeblink conditioning suggest that bidirectional plasticity at mossy fiber–granule cell synapses plays a role in enhancing signal transmission for robust conditioned stimuli (Yamazaki and Tanaka, 2007) and tune spike timing of granule cells on the millisecond order (Garrido et al., 2013).

4.2 Induction Mechanism: Climbing-Fiber Specific Changes Versus Nonspecific Changes LTD alone may not be sufficient for robust learning. LTD, which is triggered by climbing fiber signals and is therefore climbing-fiber-specific (Ito, 2013), should be coupled with climbing-fiber-nonspecific LTP, to counterbalance the parallel fiber–Purkinje cell synaptic weights against spontaneous climbing fiber activity. A computer simulation demonstrates that when simulated LTP is blocked, all parallel fiber–Purkinje cell synapses die out owing to spontaneous climbing fiber activity,

the middle panel out-of-phase modulation. Although all mossy fiber input signals modulate in the same phase, a subset of granule cells (e.g., cell in the middle panel) exhibit out-ofphase modulation due to the recurrent inhibition exerted by Golgi cells. Bottom panel represents the average activity of all granule cells in the model, which is modulated in phase with respect to the modulation of the mossy fiber input signals. The model is composed of 102,400 granule cells, 1024 Golgi cells, 16 Purkinje cells, 16 molecular layer interneurons, 1 vestibular nuclear, and 1 inferior olivary neuron.

4 Significance of LTD in Cerebellar Neural Network

resulting in the failure of timed conditioned responses (Medina et al., 2000). This could be a possible reason for the failure of timed conditioned responses in LTPdeficient mice (Schonewille et al., 2010). It could also lead to the decrease in HVOR gain induced by both gain increase training and gain decrease training in the same mice (Schonewille et al., 2010). HVOR gain depends on the modulation depth of Purkinje cell activity (Nagao, 1989). In an extreme case when all parallel fiber– Purkinje cell synapses die out, Purkinje cells cannot modulate their spike discharges, thereby decreasing the HVOR gain. These observations suggest that LTD has a central role in learning, whereas LTP plays supportive roles in LTD to make the learning robust and to retain the learned memory.

4.3 Learning Principles: Supervised Learning Versus Unsupervised Learning The cerebellum is considered to acquire internal models of given physical or mental objects (Ito, 2008, 2011; Kawato, 1999). An internal model of a given object is a table of input–output relationships of the object or its inverse relationships. Internal models are indispensable for fast, smooth, and coordinated movements based on prediction. Supervised learning is a learning principle for this purpose: a supervised learning machine, using explicit instruction signals, generates the same output signals in response to context signals. In the Marr–Albus–Ito hypothesis, it is proposed that the cerebellar cortex is a supervised learning machine, which is consistent with the view that the cerebellum acquires internal models. Another candidate learning principle in the cerebellar cortex is unsupervised learning, which does not require explicit instructions. Rather, an unsupervised learning machine aims to extract statistical properties hidden in the input signals such as correlation. Parallel fiber– Purkinje cell synapses could undergo LTP as well as LTD without climbing fiber signals (Lev-Ram et al., 2002; Qiu and Kno¨pfel, 2009; Sakurai, 1987). Therefore, it is possible to employ unsupervised learning as the basis of the cerebellar learning principle. However, we still do not have a firm theoretical foundation to acquire internal models by unsupervised learning without explicit instruction signals (except for Anastasio, 2001).

4.4 Ideal Location for Memory: Purkinje Cells Versus Other Cells According to the learning principle, as long as the cerebellar cortex is a supervised learning machine, output cells must receive two inputs, one for context information and the other for instruction. Purkinje cells are an obvious candidate for this view, as are molecular layer interneurons, which show bidirectional plasticity with their parallel fiber inputs (Jo¨rntell and Ekerot, 2002, 2003). They follow the Hebbian-type learning rule in which the direction of learning could be regulated by glutamate secreted from climbing fiber terminals (Szapiro and Barbour, 2007). However, the learning that relies on glutamate spillover may not be suitable for learning information that requires fine temporal precision such as timing of a conditioned response in

17

18

CHAPTER 1 Cerebellar LTD and Learning

delay eyeblink conditioning. Moreover, multiple climbing fibers provide glutamate spillover for the interneurons (Szapiro and Barbour, 2007), which means that the interneurons may receive mixed instruction signals from multiple climbing fibers. Furthermore, molecular layer interneurons may not be good candidates for controlling the activity of cells in the cerebellar or vestibular nuclei that produce the final output of the cerebellum because their contribution to downstream cells is indirect. Purkinje cells, in contrast, can affect these nuclei directly and modulate the activity of downstream cells rapidly and efficiently.

4.5 Memory Formation Process: Acquisition Versus Consolidation The memory formation process has two stages. First, memory is acquired by learning and is then consolidated. Acquisition and consolidation are different processes, and so different mechanisms would be involved. LTD seems to be involved in both acquisition and consolidation, whereas some other mechanisms may be involved in consolidation. For example, selective depletion of GABA receptors on Purkinje cells by genetic manipulation causes failure of consolidation of HVOR gain and phase learning, even though the mutants are not ataxic (Wulff et al., 2009). This implies that molecular layer interneuron–Purkinje cell synapses are indispensable for long-term memory formation. When GABAergic inputs to Purkinje cells are blocked, these Purkinje cells elicit spikes more regularly (Ha¨usser and Clark, 1997). The regular spiking and synchronization of Purkinje cell populations could inhibit cerebellar or vestibular nuclei tonically, thereby preventing rebound firing in the nuclei (Bengtsson et al., 2011; Person and Raman, 2012). This seems necessary for long-term memory formation at mossy fiber–nuclear cell synapses (McElvain et al., 2010; Pugh and Raman, 2006). The failure of long-term memory consolidation thus appears to be due to inadequate modulation of nuclear cell activity caused by Purkinje cells, suggesting that Purkinje cells provide instruction signals to the nuclei (Medina, 2011).

4.6 Time at Impairment: Mature Brain Versus Developing Brain Clinical studies have demonstrated that impairments due to cerebellar agenesis are less severe than those seen in acute cerebellar damage in adults (Boyd, 2010). Remarkable redundancy of a developing brain could, at least partially, compensate for the absent cerebellum (Lemon and Edgley, 2010). These observations imply that gene-manipulated animals born with a nonfunctioning cerebellum could exhibit normal behavior. Going one step further, impairment of LTD may be so severe that other parts of the brain, probably the cerebral cortex, must take over the function of the cerebellum entirely. Another possibility is that mice may have parallel or backup pathways for motor learning (Yuzaki, 2013). For example, in wild-type cerebellar slices, as mentioned in Section 2, inclusion of calcineurin inhibitory peptides in the patch pipette solution blocks LTD (Nomura et al., 2012), whereas Purkinje cell-specificcalcineurin knockout mice exhibit normal LTD induction (Schonewille et al., 2010).

5 LTD Versus Learning Mismatch

5 LTD VERSUS LEARNING MISMATCH Whereas the coincidence of LTD blockade and loss of motor learning is the basis for the Marr–Albus–Ito hypothesis, their mismatch was observed in the experiments by Schonewille et al. (2011) using three types of mutant mice (PICK1 KO, GluR2Delta7 KI, and GluR2K882A KI), in which internalization of AMPA receptors is disturbed. In these mice, LTD induction is blocked, but motor learning seems normal. In another study, a pharmacological agent, T-588, blocked LTD in rats in vivo under anesthesia, but motor learning was preserved in awake rats orally fed with T-588 (Welsh et al., 2005). Motor learning was also exhibited by awake mice intraperitoneally injected with T-588 (Schonewille et al., 2011). These two examples of mismatched LTD induction and motor learning may compel us to reconsider the Marr– Albus–Ito hypothesis. Some researchers try to develop an expanded hypothesis in terms of distributed plasticity (e.g., Boyden et al., 2004; Gao et al., 2012). Others even reject the learning mechanism in the cerebellum (Welsh et al., 2005). However, before doing so, it is important to consider various technical or fundamental problems that might cause such a mismatch.

5.1 Experimental Gaps in Testing of LTD and Motor Learning The most plausible, even likely, possibility is that the cases of mismatch could arise from different experimental conditions for testing LTD induction and motor learning. In most of the above-cited experiments, LTD is tested in in vitro slices by applying artificial electric pulse stimuli. However, motor learning is examined by observing responses of an awake behaving animal to natural stimuli in adaptation in ocular reflexes, eyeblink conditioning, and adaptive locomotion. Despite these grave differences, one often naively assumes that when LTD is blocked in vitro for a certain cause, the same cause invariably blocks in vivo LTD in behaving animals as well. However, we must realize that this type of inference is not more than a weak supposition because in none of these studies LTD has not been examined directly in an awake behaving animal performing the actual process of motor learning. The only exception is the case that the decrease in the number of AMPA receptors in parallel fiber–Purkinje cell synapses was found as a trace of LTD after fast HOKR adaptation (Aziz et al., 2014; Wang et al., 2014). Ambiguity in blocking motor learning would be unavoidable when one tries to apply a pharmacological inhibitor to an animal through the blood–brain barrier. Indeed, whereas T-588 failed to affect motor learning in rats (Welsh et al., 2005) or mice (Schonewille et al., 2011), intraperitoneal administration of T-588 into marmosets effectively blocked fast HVOR adaptation dose dependently (Anzai and Nagao, 2014). There seems to be a significant species difference between rodents and marmosets concerning the ease of T-588 in penetrating the blood–brain barrier or in shifting from the abdominal cavity to blood.

19

20

CHAPTER 1 Cerebellar LTD and Learning

5.2 Compensation Might Restore Motor Learning The cerebellum has a marked capability for functional compensation. A classic experiment demonstrated that lesions of the cerebellum induced postural disturbances, which then gradually diminished. This compensation has been ascribed to the cerebral cortex because once recovered, postural disturbances reappeared after cerebral lesions (Dow and Moruzzi, 1958). When cerebellar tissues are deprived of LTD and consequently unable to learn, other tissues, most likely the cerebral cortex, could take over the lost cerebellar function. When this happens during development in the global gene-manipulated mice, adult motor learning could be performed outside the cerebellum, irrelevant of LTD status. The possibility of such compensation at a system level needs to be examined. Compensation in mutant mice is also expected to occur in cerebellar circuits in terms of distributed plasticity. Schonewille et al. (2011) examined and rejected the following three possibilities that may have compensated for the lack of LTD: enhanced postsynaptic LTP, presynaptic LTP, or age-dependent LTD. However, a possibility still remains for putative compensations that might rescue the behavioral phenotype in the absence of LTD induction (Hirano, 2013). The three types of genetic manipulation used by Schonewille et al. (2011) disturbed the activity-dependent internalization of AMPA receptors, but the parallel fiber–Purkinje cell synapses exhibited seemingly normal basic transmission and presynaptic LTP. This finding would imply that the constitutive recycling of AMPA receptors at parallel fiber– Purkinje cell synapses remains normally functional. The possibility to be examined is that converting a part of the constitutive internalization to activity-dependent compensates for the lack of the activity-dependent internalization required for LTD.

5.3 Limitations in Testing LTD in Slices In vitro brain slices provide an excellent experimental material for studying synaptic transmission and plasticity and have greatly contributed to the recent successes in molecular/cellular neuroscience. Nevertheless, it should be noted that in vitro slices function under fundamentally artificial conditions, such as disconnected electrical and chemical signaling from surrounding tissues and continuous washout of potential plasticity factors, such as corticotropin-releasing factor (Miyata et al., 1999), by perfusion with artificial bath solutions. Furthermore, to induce LTD in slices, artificial stimuli composed of electric pulses must be applied. In an experiment by Schonewille et al. (2011), as is routine for slice experiments on LTD, parallel fibers and climbing fibers were stimulated simultaneously at 1 Hz for 5 min (300 pulses). These stimuli induced a 30–40% decrease in the amplitude of PF-EPSC or the rising slope of PF-EPSP in Purkinje cells. However, the extent of decrease is lessened by half upon choosing either 200 or 400 pulses, or upon shifting the frequency of simultaneous parallel fiber and climbing fiber stimulations to either 0.75 or 1.2 Hz (Karachot et al., 1994). Furthermore, picrotoxin was added to the perfusate to block postsynaptic inhibition, which otherwise interferes with LTD induction (Ekerot and

6 Perspectives

Kano, 1985). The in vitro LTD observed by these researchers and many others is, therefore, highly labile because it is induced with a precise but artificial tuning of electric stimuli under the effect of picrotoxin. Consequently, a disturbance could easily upset in vitro LTD induction. On the other hand, LTD induction by natural stimuli under in vivo conditions might remain robust so that its blockade under similar conditions or perturbations could be relatively difficult. One reason for believing this scenario is the differential efficiency of electric stimulation of parallel fibers between artificial in vitro and natural in vivo conditions. In an awake animal, a single impulse in a mossy fiber tends to induce bursting spikes in a granule cell (Chadderton et al., 2004; Rancz et al., 2007). Hence, at each parallel fiber synapse, natural stimulation via mossy fibers could provide stronger activation than a 1-Hz pulse stimulation of parallel fibers. In the abovementioned mismatch observed by Schonewille et al. (2011), these important differences could potentially account for observations of a lack of concordance between in vitro and in vivo effects.

6 PERSPECTIVES From the review in Section 5, we are uncertain whether under currently practiced methodologies, the failure to observe LTD induction in slices causally ensures the absence of LTD during the process of motor learning. We therefore hypothesize that, under certain highly tuned physiological, pharmacological, or compensated genetic conditions, a cause that effectively blocks artificial stimulus-induced LTD in slices may fail to block natural stimulus-induced in vivo LTD in a behaving animal. Thus, a crucial question is how one could confirm LTD induction in an intact animal actually performing motor learning. The unequivocal answer to this question should be derived by reliably inducing and monitoring LTD in awake behaving animals. To date, LTD-like depression of spike discharges in Purkinje cells has been observed in behaving monkeys during a simple motor learning task (Gilbert and Thach, 1977; Medina and Lisberger, 2008). What is now needed at this juncture is the actual observation of LTD as a synaptic process in behaving animals. Such a process as this should be causally associated with concordant signal transduction and AMPA receptor trafficking in parallel fiber–Purkinje cell synapses. Unfortunately, properly addressing these issues requires experimental approaches that remain to be developed. Nevertheless, very recent EM studies have revealed a trace of LTD; the density of AMPA receptors in parallel fiber synapses on Purkinje cells decreased by 25% without affecting the number and microstructure of synapses during the fast HOKR adaptation, which should have induced LTD (Aziz et al., 2014; Wang et al., 2014). Hence, our position on recent examples of “mismatch” between LTD induction and motor learning reported by Schonewille et al. (2011) is that what must be challenged is modern neuroscience technology before one can adequately challenge the Marr–Albus–Ito hypothesis.

21

22

CHAPTER 1 Cerebellar LTD and Learning

While the causes of LTD versus learning mismatch are being explored, it is important to develop an expanded neuronal network model of the cerebellum by taking a highly distributed nature of synaptic plasticity in the mossy fiber–granule cell– Purkinje cell pathways incorporating molecular layer inhibitory neurons. Several established views are now emerging in favor of learning (see Hansel et al., 2001; Ohtsuki et al., 2009). Even though evidence supports the view that climbing fibers convey error signals and induce LTD, some experiments suggest that climbing fibers are not simply an all-or-none device for the induction of LTP or LTD in the parallel fiber–Purkinje cell synapses. Instead, the number of action potentials in each climbing fiber burst varies and it encodes olivary oscillations that may influence both the timing and learning aspects of cerebellar functions (Jacobson et al., 2008; Mathy et al., 2009). Thus, the two major functions proposed for climbing fibers may be integrated. Peculiarly, signals carried by only climbing fibers or by parallel fibers are sufficient for motor learning with an additive effect when both signals are present (Ke et al., 2009). This observation suggests that motor learning may not be exclusively linked to climbing fiber activity (Ohtsuki et al., 2009). Note also that climbing fiber–Purkinje cell synapses are endowed with a high degree of structural and functional plasticity (Hansel and Linden, 2000; Ohtsuki et al., 2009; Strata and Rossi, 1998). To summarize, the current controversy around the Marr–Albus–Ito hypothesis might be resolved in either of the following two ways. First, a new technology might convincingly reveal whether LTD is invariably induced during motor learning or not. Second, more advanced experimental or theoretical investigations of the cerebellar neural network might enable us to draw precise structural–functional features of distributed plasticity in the cerebellar neural network. We may then be able to find how differently the classic Marr–Albus–Ito type of plasticity and the idea of multiple, distributed type of plasticity account for the operation of the real cerebellar neural network.

Acknowledgments The authors are grateful for the helpful discussions made during this writing by Prof. Takao Hensch (Harvard University, Molecular and Cellular Biology) and Dr. Thomas Launey (RIKEN Brain Science Institute, Molecular Neurocybernetics Unit).

References Aiba, A., Kano, M., Chen, C., Stanton, M.E., Fox, G.D., Herrup, K., Zwingman, T.A., Tonegawa, S., 1994. Deficient cerebellar long-term depression and impaired motor learning in mGluR1 mutant mice. Cell 79, 377–388. Albus, J., 1971. A theory of cerebellar function. Math. Biosci. 10, 25–60. Anastasio, T.J., 2001. Input minimization: a model of cerebellar learning without climbing fiber error signals. NeuroReport 12, 3825–3831.

References

Andreescu, C.E., Prestori, F., Brandalise, F., D’Errico, A., De Jeu, M.T., Rossi, P., Botta, L., Kohr, G., Perin, P., D’Angelo, E., De Zeeuw, C.I., 2011. NR2A subunit of the N-methyl D-aspartate receptors is required for potentiation at the mossy fiber to granule cell synapse and vestibulo-cerebellar motor learning. Neuroscience 176, 274–283. Anzai, M., Kitazawa, H., Nagao, S., 2010. Effects of reversible pharmacological shutdown of cerebellar flocculus on the memory of long-term horizontal vestibulo-ocular reflex adaptation in monkeys. Neurosci. Res. 68, 191–198. Anzai, M., Nagao, S., 2014. Motor learning in marmosets: vestibulo-ocular reflex adaptation and its sensitivity to inhibitors of Purkinje cell long-term depression. Neurosci. Res. 83, 33–42. Aziz, W., Wang, W., Kasaf, S., Alsayed, A.A.M., Fukazawa, Y., Shigemoto, R., 2014. Distinct kinetics of synaptic formation and memory decay in massed and spaced learning. Proc. Natl. Acad. Sci. U.S.A. 111, E195–E202. Belmeguenai, A., Hansel, C., 2005. A role for protein phosphatases 1, 2A, and 2B in cerebellar long-term potentiation. J. Neurosci. 25, 10768–10772. Belmeguenai, A., Hosy, E., Bengtsson, F., Pedroarena, C.M., Piochon, C., Teuling, E., He, Q., Ohtsuki, G., De Jeu, M.T., Elgersma, Y., De Zeeuw, C.I., Jo¨rntell, H., Hansel, C., 2010. Intrinsic plasticity complements long-term potentiation in parallel fiber input gain control in cerebellar Purkinje cells.. J. Neurosci. 30, 13630–13643. Bengtsson, F., Ekerot, C.F., Jo¨rntell, H., 2011. In vivo analysis of inhibitory synaptic inputs and rebounds in deep cerebellar nuclear neurons. PLoS One 6, e18822. Boyden, E.S., Katoh, A., Raymond, J.L., 2004. Cerebellum-dependent learning: the role of multiple plasticity mechanisms. Annu. Rev. Neurosci. 27, 581–609. Boyd, C.A.R., 2010. Cerebellar agenesis revisited. Brain 133, 941–944. Brindley, G.S., 1964. The use made by the cerebellum of the information that it receives from sense organ. IBRO Bull. 3, 80. Buonomano, D.V., Mauk, M.D., 1994. Neural network model of the cerebellum: temporal discrimination and the timing of motor response. Neural Comput. 6, 38–55. Chadderton, P., Margrie, T.W., Ha¨usser, M., Hansel, C., 2004. Integration of quanta in cerebellar granule cells during sensory processing. Nature 428, 856–860. Chung, H.J., Steinberg, J.P., Huganir, R.L., Linden, D.J., 2003. Requirement of AMPA receptor GluR2 phosphorylation for cerebellar long-term depression. Science 300, 1751–1755. Crepel, F., Jaillard, D., 1991. Pairing of pre- and postsynaptic activities in cerebellar Purkinje cells induce long-term synaptic efficacy in vitro. J. Physiol. Lond. 432, 123–141. D’Angelo, E., Rossi, P., Armano, S., Taglietti, V., 1999. Evidence for NMDA and mGlu receptor-dependent long-term potentiation of mossy fiber-granule cell transmission in rat cerebellum. J. Neurophysiol. 81, 277–287. Daniel, H., Levenes, C., Crepel, F., 1998. Cellular mechanisms of cerebellar LTD. Trends Neurosci. 21, 401–407. De Zeeuw, C.I., Hansel, C., Bian, F., Koekkoek, S.K., van Alphen, A.M., Linden, D.J., Oberdick, J., 1998. Expression of a protein kinase C inhibitor in Purkinje cells blocks cerebellar LTD and adaptation of the vestibulo-ocular reflex. Neuron 20, 495–508. Dean, P., Porrill, J., 2011. Evaluating the adaptive filter model of the cerebellum. J. Physiol. Lond. 589 (Pt14), 3459–3470. Douyard, J., Shen, L., Huganir, R.L., Rubio, M.E., 2007. Differential neuronal and glial expression of GluR1 AMPA receptor subunit and the scaffolding proteins SAP97 and 4.1N during rat cerebellar development. J. Comp. Neurol. 502, 141–156.

23

24

CHAPTER 1 Cerebellar LTD and Learning

Dow, R.S., Moruzzi, G., 1958. The Physiology and Pathology of the Cerebellum. The University Minnesota Press, Minneapolis. Eccles, J.C., Ito, M., Szentagothai, J., 1967. The Cerebellum as a Neuronal Machine. Springer Verlag, Berlin, Heidelberg, New York. Ekerot, C.-F., Kano, M., 1985. Long-term depression of parallel fiber synapses following stimulation of climbing fibers. Brain Res. 342, 357–360. Endo, S., Suzuki, M., Sumi, M., Nairn, A.C., Morita, R., Yamakawa, K., Greengard, P., Ito, M., 1999. Molecular identification of human G-substrate: a possible downstream component of cyclic GMP-dependent protein kinase. Proc. Natl. Acad. Sci. U.S.A. 96, 2467–2472. Endo, S., Nairn, A.C., Greengard, P., Ito, M., 2003. Thr123 of rat G-substrate contributes to its action as a protein phophatase inhibitor. Neurosci. Res. 45, 79–89. Endo, S., Shutoh, F., Dinh, T.L., Okamoto, T., Ikeda, T., Suzuki, M., Kawahara, S., Yanagihara, D., Yamada, K., Kirino, Y., Hartell, N.A., Yamaguchi, K., Itohara, S., Nairn, A., Greengard, P., Nagao, S., Ito, M., 2009. Dual involvement of G-substrate in motor learning revealed by gene deletion. Proc. Natl. Acad. Sci. U.S.A. 106, 3525–3530. Fujita, M., 1982. Adaptive filter model of the cerebellum. Biol. Cybern. 45, 195–206. Gao, Z., van Beugen, B.J., De Zeeuw, C.I., 2012. Distributed synergistic plasticity and cerebellar learning. Nat. Rev. Neurosci. 13, 619–635. Garrido, J.A., Ros, E., D’Angelo, E., 2013. Spike timing regulation on the millisecond scale by distributed synaptic plasticity at the cerebellum input stage: a simulation study. Front. Comp. Neurosci. 7, 64. Gilbert, P.F., Thach, W.T., 1977. Purkinje cell activity during motor learning. Brain Res. 128, 309–328. Gonshor, A., Mellvill-Jones, G.M., 1976. Extreme vestibulo-ocular reflex adaptation induced by prolonged optical reversal of vision. J. Physiol. Lond. 256, 381–414. Hansel, C., Linden, D.J., 2000. Long-term depression of cerebellar climbing fiber-Purkinje neuron synapse. Neuron 26, 473–482. Hansel, C., Linden, D.J., D’Angelo, E., 2001. Beyond parallel fiber LTD: the diversity of synaptic and non-synaptic plasticity in the cerebellum. Nat. Neurosci. 4, 467–475. Hansel, C., de Jeu, M., Belmeguenai, A., Houtman, S.H., Builtendijk, G.H., Andrew, D., De Zeeuw, C.I., Elgersma, Y., 2006. aCaMKII is essential for cerebellar LTD and motor learning. Neuron 51, 835–843. Ha¨usser, M., Clark, B.A., 1997. Tonic synaptic inhibition modulates neuronal output pattern and spatiotemporal synaptic integration. Neuron 19, 665–678. Hebb, D., 1949. The Organization of Behavior. Wiley, New York. Hirano, T., 1990. Effects of postsynaptic depolarization on the induction of synaptic depression between a granule cell and a Purkinje cell in rat cerebellar culture. Neurosci. Lett. 119, 145–147. Hirano, T., 2013. Long-term depression and other synaptic plasticity in the cerebellum. Proc. Jpn. Acad. B 89, 183–195. Huang, Y., Man, H.Y., Sekine-Aizawa, Y., Han, Y., Juluri, K., Luo, H., Cheah, J., Lowenstein, C., Huganir, R.L., Snyder, S.H., 2005. S-Nitrosylation of N-ethylmaleimide sensitive factor mediates surface expression of AMPA receptors. Neuron 46, 533–540. Ito, M., 1984. The Cerebellum and Neural Control. Raven, New York. Ito, M., 1989. Long-term depression. Annu. Rev. Neurosci. 12, 85–102. Ito, M., 2000. Mechanisms of motor learning in the cerebellum. Brain Res. 886, 237–245. Ito, M., 2001. Cerebellar long-term depression—characterization, signal transduction and functional roles. Physiol. Rev. 81, 1143–1195.

References

Ito, M., 2002a. Historical review of the significance of the cerebellum and the role of Purkinje cells in motor learning. Ann. N. Y. Acad. Sci. 978, 273–288. Ito, M., 2002b. The molecular organization of cerebellar long-term depression. Nat. Rev. Neurosci. 3, 896–902. Ito, M., 2006. Cerebellar circuitry as a neuronal machine. Prog. Neurobiol. 78, 272–303. Ito, M., 2008. Control of mental activities by internal models in the cerebellum. Nat. Rev. Neurosci. 9, 304–313. Ito, M., 2011. The Cerebellum: Brain for an Implicit Self. FT Press, New York. Ito, M., 2013. Error detection and representation in olivo-cerebellar system. Front. Neural Circuits 7, 1. http://dx.doi.org/10.3389/fncir.2013.0001. Ito, M., Kano, M., 1982. Long-lasting depression of parallel fiber-Purkinje cell transmission by conjunctive stimulation of parallel fibers and climbing fibers in the cerebellar cortex. Neurosci. Lett. 33, 253–258. Ito, M., Sakurai, M., Tongroach, P., 1982. Climbing fibre induced depression of both mossy fibre responsiveness and glutamate sensitivity of cerebellar Purkinje cell. J. Physiol. Lond. 324, 113–134. Jacobson, G.A., Rokni, D., Yarom, Y., 2008. A model of the olivo-cerebellar system as a temporal pattern generator. Trends Neurosci. 31, 617–625. Jo¨rntell, H., Ekerot, C.F., 2002. Reciprocal bidirectional plasticity of parallel fiber receptive fields in cerebellar Purkinje cells and their afferent interneurons. Neuron 34, 797–806. Jo¨rntell, H., Ekerot, C.F., 2003. Receptive field plasticity profoundly alters the cutaneous parallel fiber synaptic input to cerebellar interneurons in vivo. J. Neurosci. 23, 9620–9631. Kakegawa, W., Yuzaki, M., 2005. A mechanism underlying AMPA receptor trafficking during cerebellar long-term potentiation. Proc. Natl. Acad. Sci. U.S.A. 102, 17846–17851. Kano, M., Rexhausen, U., Dreessen, J., Konnerth, A., 1992. Synaptic excitation produces a long-lasting rebound potentiation of inhibitory synaptic signals in cerebellar Purkinje cells. Nature 356, 601–604. Karachot, L., Kado, R.T., Ito, M., 1994. Stimulus parameters for induction of long-term depression in in vitro slices of rat cerebellum. Neurosci. Res. 21, 161–168. Kassardjian, C.D., Tan, Y.F., Chung, J.Y., Heskins, R., Peterson, M.J., Broussard, D.M., 2005. The site of motor memory shifts with consolidation. J. Neurosci. 25, 7979–7985. Katoh, A., Kitazawa, H., Itohara, S., Nagao, S., 1998. Dynamic characteristics and adaptability of mouse vestibulo-ocular and optokinetic reflex eye movements and the role of flocculoolivary system revealed by chemical lesions. Proc. Natl. Acad. Sci. U.S.A. 95, 7705–7710. Katoh, A., Kitazawa, H., Itohara, S., Nagao, S., 2000. Inhibition of nitric oxide synthesis and gene-knockout of neural nitric oxide synthetase impaired adaptation of mouse optokinetic eye movements. Learn. Mem. 7, 220–226. Kawaguchi, Y., 1985. Two groups of secondary vestibular neurons mediating horizontal canal signals, probably to the ipsilateral medial rectus muscle, under inhibitory influences from the cerebellar flocculus in rabbits. Neurosci. Res. 2, 434–446. Kawaguchi, S.Y., Hirano, T., 2013. Gating of long-term depression by Ca2þ/calmodulindependent protein kinase II through enhanced cGMP signaling in cerebellar Purkinje cells. J. Physiol. Lond. 591, 1707–1730. Kawato, M., 1999. Internal models for motor control and trajectory planning. Curr. Opin. Neurobiol. 9, 718–727. Ke, M.C., Guo, C.C., Raymond, J.L., 2009. Elimination of climbing fiber instructive signals during motor learning. Nat. Neurosci. 12, 1171–1179. Kohda, K., Kakegawa, W., Matsuda, S., Yamamoto, T., Hirano, H., Yuzaki, M., 2013. The d2 glutamate receptor gates long-term depression by coordinating interactions

25

26

CHAPTER 1 Cerebellar LTD and Learning

between two AMPA receptor phosphorylation sites. Proc. Natl. Acad. Sci. U.S.A. 110, E948–E957. Kojima, Y., Yoshida, K., Iwamoto, Y., 2007. Microstimulation of the midbrain tegmentum creates learning signals for saccade adaptation. J. Neurosci. 27, 3759–3767. Kuroda, S., Schweighofer, N., Kawato, M., 2001. Exploration of signal transduction pathways in cerebellar long-term depression by kinetic stimulation. J. Neurosci. 21, 5693–5702. Le, T.D., Shirai, Y., Okamoto, T., Tatsukawa, T., Nagao, S., Shimizu, T., Ito, M., 2010. Lipid signaling in cytosolic phopholipase A2a-cyclooxygenae-2 cascade mediates cerebellar long-term depression and motor learning. Proc. Natl. Acad. Sci. U.S.A. 107, 3198–3203. Lee, S.H., Liu, L., Wang, Y.T., Sheng, M., 2002. Clathrin adaptor AP2 and NSF interact with overlapping sites of GluR2 and play distinct roles in AMPA receptor trafficking and hippocampal LTD. Neuron 36, 661–674. Lemon, R.N., Edgley, S.A., 2010. Life without a cerebellum. Brain 133, 652–654. Lev-Ram, V., Jiang, T., Wood, J., Lawrence, D.S., Tsien, R.Y., 1997. Synergies and coincidence requirements between NO, cGMP, and Ca2þ in the induction of cerebellar long-term depression. Neuron 18, 1025–1038. Lev-Ram, V., Metha, S.B., Kleinfeld, D., Tsien, R.Y., 2002. Reversing cerebellar long-term depression. Proc. Natl. Acad. Sci. U.S.A. 100, 15989–15993. Lincoln, J.S., McCormick, D.A., Thompson, R.F., 1982. Ipsilateral cerebellar lesions prevent learning of the classically conditioned nictitating membrane/eyelid response. Brain Res. 242, 190–193. Linden, D.J., 1991. A long-term depression of AMPA currents in cultured cerebellar Purkinje neurons. Neuron 7, 81–89. Linden, D.J., 2001. The expression of cerebellar LTD in culture is not associated with changes in AMPA-receptor kinetics, agonist affinity, or unitary conductance. Proc. Natl. Acad. Sci. U.S.A. 98, 14066–14071. Linden, D.J., Connor, J.A., 1991. Participation of postsynaptic PKC in cerebellar long-term depression in culture. Science 254, 1656–1659. Linden, D.J., Dawson, T.M., Dawson, V.L., 1995. An evaluation of the nitric oxide/cGMP/ cGMP-dependent protein kinase cascade in the induction of cerebellar long-term depression in culture. J. Neurosci. 15, 5098–5105. Macia, E., Ehrlich, M., Massol, R., Boucrot, E., Brunner, C., Kirchhausen, T., 2006. Dynasore, a cell-permeable inhibitor of dynamin. Dev. Cell 20, 839–850. Marr, D., 1969. A theory of cerebellar cortex. J. Physiol. Lond. 202, 437–470. Martin, T.A., Keating, J.G., Goodkin, J., Bastian, H.P., Thach, W.T., 1996. Throwing while looking through prisms. II. Specificity and storage of multiple gaze-throw calibrations. Brain 119, 1199–1211. Mathy, A., Ho, S.S.N., Davie, J.T., Duguid, I.C., Clarke, B.A., Ha¨usser, M., 2009. Encoding of oscillation by axonal bursts in inferior olive neurons. Neuron 62, 388–399. Matsuda, S., Launey, T., Mikawa, S., Hirai, H., 2000. Disruption of AMPA receptor GluR2 clusters following long-term depression induction in cerebellar Purkinje neurons. EMBO J. 19, 2765–2774. McElvain, L.E., Bagnall, M.W., Sakatos, A., du Lac, S., 2010. Bidirectional plasticity gated by hyperpolarization controls the gain of postsynaptic firing responses at central vestibular nerve synapses. Neuron 68, 763–775. McLaughlin, S.C., 1967. Parametric adjustment in saccadic eye movements. Percept. Psychophys. 2, 359–362.

References

Medina, J.F., 2011. The multiple roles of Purkinje cells in sensory-motor calibration: to predict, teach and command. Curr. Opin. Neurobiol. 21, 616–622. Medina, J.F., Lisberger, S.G., 2008. Links from complex spikes to local plasticity and motor learning in the cerebellum awake-behaving monkeys. Nat. Neurosci. 11, 1185–1192. Medina, J.F., Garcia, K.S., Nores, W.L., Taylor, N.M., Mauk, M.D., 2000. Timing mechanisms in the cerebellum: testing predictions of a large-scale computer simulation. J. Neurosci. 20, 5516–5525. Miles, F.A., Braitman, D.J., Dow, B.M., 1980. Long-term adaptive changes in primate vestibuloocular reflex. IV. Electrophysiological observations in flocculus of adapted monkeys. J. Neurophysiol. 43, 1477–1493. Miyashita, Y., Ito, M., Jastreboff, P.J., Maekawa, K., Nagao, S., 1980. Effects upon eye movements of rabbits induced by severance of mossy fiber visual pathway to the cerebellar flocculus. Brain Res. 198, 210–215. Miyata, M., Okada, D., Hashimoto, K., Kano, M., Ito, M., 1999. Corticotropin-releasing factor plays a permissive role in cerebellar long-term depression. Neuron 22, 763–775. Nagao, S., 1983. Effects of vestibulocerebellar lesions upon dynamic characteristics and adaptation of vestibulo-ocular and optokinetic eye movements in pigmented rabbits. Exp. Brain Res. 53, 36–46. Nagao, S., 1988. Behavior of floccular Purkinje cells correlated with adaptation of horizontal optokinetic eye movement response in pigmented rabbits. Exp. Brain Res. 73, 489–497. Nagao, S., 1989. Behavior of floccular Purkinje cells correlated with adaptation of vestibuloocular reflex in pigmented rabbits. Exp. Brain Res. 77, 531–540. Nagao, S., Ito, M., Karachot, L., 1985. Eye field in the cerebellar flocculus of pigmented rabbits determined with local electrical stimulation. Neurosci. Res. 3, 39–51. Nagao, S., Kitamura, T., Nakamura, N., Hiramatsu, T., Yamada, J., 1997. Differences of the flocculus and ventral paraflocculus in the mossy and climbing fiber input organization. J. Comp. Neurol. 382, 480–498. Nagao, S., Honda, T., Yamazaki, T., 2013. Transfer of memory trace of cerebellum-dependent motor learning in human long-term prism adaptation: a model study. Neural Netw. 47, 72–80. Nisimaru, N., Mittal, C., Shirai, Y., Sooksawate, T., Anaddaraj, P., Hashikawa, T., Nagao, S., Arata, A., Sakurai, T., Yamamoto, M., Ito, M., 2013. An orexin-neuromodulated cerebellar circuit. Proc. Natl. Acad. Sci. U.S.A. 110, 14124–14131. Nomura, T., Kakegawa, W., Matsuda, S., Kohda, K., Nishiyama, J., Takahashi, T., Yuzaki, M., 2012. Cerebellar long-term depression requires dephosphorylation of TARP in Purkinje cells. Eur. J. Neurosci. 35, 402–410. Ohtsuki, G., Piochon, C., Hansel, C., 2009. Climbing fiber signal and cerebellar gain control. Front. Cell. Neurosci. 4, http://dx.doi.org/10.3389/neuro.03.004.2009. Okamoto, T., Endo, S., Shirao, T., Nagao, S., 2011a. Cerebellar cortical protein synthesisdependent transfer of memory trace underlies the spacing effect in motor learning. J. Neurosci. 31, 8958–8966. Okamoto, T., Shirao, T., Shutoh, F., Suzuki, T., Nagao, S., 2011b. Post-training cerebellar cortical activity plays an important role for consolidation of memory of cerebellum-dependent motor learning. Neurosci. Lett. 504, 53–56. Osanai, R., Nagao, S., Kitamura, T., Kawabata, I., Yamada, J., 1999. Differences in mossy and climbing afferent sources between flocculus and ventral and dorsal paraflocculus in the rat. Exp. Brain Res. 124, 248–264.

27

28

CHAPTER 1 Cerebellar LTD and Learning

Person, A.L., Raman, I.M., 2012. Purkinje neuron synchrony elicits time-locked spiking in the cerebellar nuclei. Nature 481, 502–506. Pugh, J.R., Raman, I.M., 2006. Potentiation of mossy fiber EPSCs in the cerebellar nuclei by NMDA receptor activation followed by postinhibitory rebound current. Neuron 51, 113–123. Qiu, D.L., Kno¨pfel, T., 2009. Presynaptically expressed long-term depression at cerebellar parallel fiber synapses. Pflugers Arch. 457, 865–875. Rancz, E.A., Ishikawa, T., Diguid, I., Chadderton, P., Mahon, S., Ha¨usser, M., 2007. Highfidelity transmission of sensory information by single mossy fiber boutons. Nature 450, 1245–1248. Robinson, D.A., 1976. Adaptive gain control of vestubulo-ocular reflex by the cerebellum. J. Neurophysiol. 39, 954–969. Rokni, D., Llina´s, R., Yarom, Y., 2008. The morpho/functional discrepancy in the cerebellar cortex: looks alone are deceptive. Front. Neurosci. 2, 192–198. Rosenblatt, F., 1962. Principles of Neurodynamics: Perceptron and the Theory of Brain Mechanisms. Spartan Books, Washington, DC. Sakurai, M., 1987. Synaptic modification of parallel fibre-Purkinje cell transmission in in vitro guinea-pig cerebellar slices. J. Physiol. Lond. 394, 463–480. Schonewille, M., Gao, Z., Boele, H.-J., Veloz, M.F.V., Amerika, W.E., Simek, A.A.M., De Jeu, M.T., Steinber, J.F., Takamiya, K., Hoebeck, F.E., Linden, D.J., Huganir, R.L., De Zeeuw, C.I., 2011. Reevaluating the role of LTD in cerebellum-dependent motor learning. Neuron 70, 43–50. Schonewille, M., Belmeguenai, A., Koekkoek, S.K., Houtman, S.H., Boele, H.J., van Beugen, B.J., Gao, Z., Badura, A., Ohtsuki, G., Amerika, W.E., Hosy, E., Hoebeek, F.E., Elgersma, Y., Hansel, C., De Zeeuw, C.I., 2010. Purkinje cell-specific knockout of the protein phosphatase PP2B impairs potentiation and cerebellar motor learning. Neuron 67, 618–628. Schreurs, B.G., Oh, M.M., Alkon, D.L., 1996. Pairing-specific long-term depression of Purkinje cell excitatory synaptic potentials results from a classical conditioning procedure in rabbit cerebellar cortex. J. Neurophysiol. 75, 1051–1060. Seja, P., Schonewille, M., Spitzmaul, G., Badura, A., Klein, I., Rudhard, Y., Wisden, W., Hu¨bner, C.A., De Zeeuw, C.I., Jentsch, T.J., 2012. Raising cytosolic Cl- in cerebellar granule cells affects their excitability and vestibulo-ocular learning. EMBO J. 31, 1227–1230. Shi, S.H., Hayashi, Y., Esteban, J.A., Malinow, R., 2001. Subunit-specific rules governing AMPA receptor trafficking to synapses in hippocampal pyramidal neurons. Cell 105, 331–343. Shibuki, K., Kimura, S., 1997. Dynamic properties of nitric oxide release from parallel fibers in rat cerebellar slices. J. Physiol. Lond. 498, 443–452. Shibuki, K., Okada, D., 1991. Endogenous nitric oxide release required for long-term synaptic depression in the cerebellum. Nature 365, 50–52. Shigemoto, R., Abe, T., Nomura, S., Nakanishi, S., Hirano, T., 1994. Antibodies inactivating mGluR1 metabotropic glutamate receptor block long-term depression in cultured Purkinje cells. Neuron 12, 1245–1255. Shinoda, Y., Yoshida, K., 1975. Neural pathways from the vestibular labyrinths to the flocculus in the cat. Exp. Brain Res. 22, 97–111. Shutoh, F., Katoh, A., Kitazawa, H., Aiba, A., Itohara, S., Nagao, S., 2002. Loss of adaptability of horizontal optokinetic response eye movements in mGluR1 mutant mice. Neurosci. Res. 42, 141–145.

References

Shutoh, F., Ohki, M., Kitazawa, S., Itohara, S., Nagao, S., 2006. Memory trace of motor learning shifts transsynaptically from cerebellar cortex to nuclei for consolidation. Neuroscience 139, 767–777. Soetedjo, R., Kojima, Y., Fuchs, A.F., 2008. Complex spike activity in the oculomotor vermis of the cerebellum: a vector error signal for saccade motor learning? J. Neurophysiol. 100, 1949–1966. Steinberg, J.P., Huganir, R.L., Linden, D.J., 2004. N-ethylmaleimide-sensitive factor is required for the synaptic incorporation and removal of AMPA receptors during cerebellar long-term depression. Proc. Natl. Acad. Sci. U.S.A. 101, 18212–18216. Strata, P., 2009. David Marr’s theory of cerebellar learning: 40 years later. J. Physiol. Lond. 587, 5519–5520. Strata, P., Rossi, F., 1998. Plasticity of olivocerebellar pathway. Trends Neurosci. 21, 407–413. Sugawara, T., Hisatsune, C., Le, T.D., Hashikawa, T., Hirono, M., Nagao, S., Mikoshiba, K., 2013. IP3R1 regulates cerebellar circuits by maintaining the spine and dendritic morphologies of Purkinje cells in adult mice. J. Neurosci. 33, 12186–12196. Szapiro, G., Barbour, B., 2007. Multiple climbing fibers signal to molecular layer interneurons exclusively via glutamate spillover. Nat. Neurosci. 10, 735–742. Tanaka, K., Augustine, G.J., 2008. A positive feedback signal transduction loop determines timing of cerebellar long-term depression. Neuron 59, 608–620. Tatsukawa, T., Chimura, T., Miyakawa, H., Yamaguchi, K., 2006. Involvement of basal protein kinase C and extracellular signal-regulated kinase 1/2 activities in constitutive internalization of AMPA receptors in cerebellar Purkinje cells. J. Neurosci. 26, 4820–4825. Traynelis, S.F., Wollmuth, L.P., McBain, C.J., Menniti, F.S., Vance, K.M., Ogden, K.K., Hansen, K.B., Yuan, H., Meyers, S.J., Dingledine, R., 2010. Glutamate receptor ion channels: structure, regulation, and function. Pharmacol. Rev. 62, 405–496. van Woerden, G.M., Hoebeck, F.E., Gao, Z., Nagaraja, R.Y., Hoogenraad, C.C., Kushner, S.A., Hansel, C. De, Zeeuw, C.I., Elgersma, Y., 2009. CaMKII controls the direction of plasticity at parallel fiber-Purkinje cell synapses. Nat. Neurosci. 12, 823–825. Wang, Y.T., Linden, D.J., 2000. Expression of cerebellar long-term depression requires postsynaptic clathrin-mediated endocytosis. Neuron 25, 635–647. Wang, S.S., Denk, W., Ha¨usser, M., 2000. Coincidence detection in single dendritic spines mediated by calcium release. Nat. Neurosci. 3, 1266–1273. Wang, W., Nakadate, K., Masugi-Tokita, M., Shutoh, F., Aziz, W., Tarusawa, E., Lorincz, A., Molner, E., Kesaf, S., Li, Y.-Q., Fukazawa, Y., Nagao, S., Shigemoto, R., 2014. Distinct cerebellar engrams in short-term and long-term motor learning. Proc. Natl. Acad. Sci. U.S.A. 111, E189–E193. Welsh, J.P., Yamaguchi, H., Zeng, X.H., Kojo, M., Nakada, Y., Takagi, A., Sugimori, M., Llinas, R.R., 2005. Normal motor learning during pharamacological intervention of Purkinje cell long-term depression. Proc. Natl. Acad. Sci. U.S.A. 102, 17166–17171. Wulff, P., Schonewille, M., Renzi, M., Viltono, L., Sassoe¨-Pognetto, M., Badura, A., Gao, Z., Hoebeek, F.E., van Dorp, S., Wisden, W., Farrant, M., De Zeeuw, C.I., 2009. Synaptic inhibition of Purkinje cells mediates consolidation of vestibulo-cerebellar motor learning. Nat. Neurosci. 12, 1042–1051. Xia, J., Chung, H.J., Wihler, C., Huganir, R.L., Linden, D.J., 2000. Cerebellar long-term depression requires PKC-regulated interactions between GluR2/3 and PDZ domaincontaining proteins. Neuron 28, 499–510.

29

30

CHAPTER 1 Cerebellar LTD and Learning

Yamaguchi, K., Nagao, S., 2012. Change in trafficking kinetics of AMPA-receptor underlying LTD revealed by photolysis of caged inhibitory peptide in cerebellar Purkinje cell. FENS Abstr. 6, p076.25. Yamamoto, Y., Lee, D., Kim, Y., Lee, B., Seo, C., Kawasaki, H., Kuroda, S., Tanaka-Yamamoto, K., 2012. Raf kinase inhibitory protein is required for cerebellar long-term synaptic depression by mediating PKC-dependent MAPK activation. J. Neurosci. 32, 14254–14264. Yamazaki, T., Nagao, S., 2012. A computational mechanism for unified gain and timing control in the cerebellum. PLoS One 7, e33319. Yamazaki, T., Tanaka, S., 2007. A spiking network model for passage-of-time representation in the cerebellum. Eur. J. Neurosci. 26, 2279–2282. Yamazaki, T., Tanaka, S., 2009. Computational models of timing mechanisms in the cerebellar granular layer. Cerebellum 8, 423–432. Yeo, C., Hardiman, M.J., Glickstein, M., 1985a. Classical conditioning of nictitating membrane response of the rabbit. I. Lesions of the cerebellar nuclei. Exp. Brain Res. 60, 87–98. Yeo, C., Hardiman, M.J., Glickstein, M., 1985b. Classical conditioning of nictitating membrane response of the rabbit. II. Lesions of the cerebellar cortex. Exp. Brain Res. 60, 99–113. Yeo, C., Hardiman, M.J., Glickstein, M., 1985c. Classical conditioning of nictitating membrane response of the rabbit. III. Connection of cerebellar lobule HVI. Exp. Brain Res. 60, 114–126. Yuzaki, M., 2013. Cerebellar LTD vs. motor learning—Lessons learned from studying GluD2. Neural Netw. 47, 36–41. Zhang, W., Linden, D.J., 2006. Long-term depression at the mossy fiber-deep cerebellar nucleus synapse. J. Neurosci. 26, 6935–6944.

CHAPTER

The Organization of Plasticity in the Cerebellar Cortex: From Synapses to Control

2

Egidio D’Angelo*,{,1 *

Department of Brain and Behavioral Sciences, University of Pavia, Pavia, Italy Brain Connectivity Center, C. Mondino National Neurological Institute, Pavia, Italy 1 Corresponding author: Tel.: þ39(0)382 987606; Fax: þ39(0)382 987527, e-mail address: [email protected]

{

Abstract The cerebellum is thought to play a critical role in procedural learning, but the relationship between this function and the underlying cellular and synaptic mechanisms remains largely speculative. At present, at least nine forms of long-term synaptic and nonsynaptic plasticity (some of which are bidirectional) have been reported in the cerebellar cortex and deep cerebellar nuclei. These include long-term potentiation (LTP) and long-term depression at the mossy fiber–granule cell synapse, at the synapses formed by parallel fibers, climbing fibers, and molecular layer interneurons on Purkinje cells, and at the synapses formed by mossy fibers and Purkinje cells on deep cerebellar nuclear cells, as well as LTP of intrinsic excitability in granule cells, Purkinje cells, and deep cerebellar nuclear cells. It is suggested that the complex properties of cerebellar learning would emerge from the distribution of plasticity in the network and from its dynamic remodeling during the different phases of learning. Intrinsic and extrinsic factors may hold the key to explain how the different forms of plasticity cooperate to select specific transmission channels and to regulate the signal-to-noise ratio through the cerebellar cortex. These factors include regulation of neuronal excitation by local inhibitory networks, engagement of specific molecular mechanisms by spike bursts and theta-frequency oscillations, and gating by external neuromodulators. Therefore, a new and more complex view of cerebellar plasticity is emerging with respect to that predicted by the original “Motor Learning Theory,” opening issues that will require experimental and computational testing.

Keywords long-term synaptic plasticity, cerebellum, motor control, granule cell, Purkinje cell

Progress in Brain Research, Volume 210, ISSN 0079-6123, http://dx.doi.org/10.1016/B978-0-444-63356-9.00002-9 © 2014 Elsevier B.V. All rights reserved.

31

32

CHAPTER 2 Plasticity in the Cerebellum

1 INTRODUCTION Although the cerebellum is known to play a fundamental role in procedural learning, the underlying cellular and circuit mechanisms are still incompletely understood. The “Motor Learning Theory” proposed by Marr and Albus (Albus, 1971; Marr, 1969) predicted that learning had to occur at the parallel fiber– Purkinje cell synapse under climbing fiber control. The simplicity and elegance of this prediction have attracted generations of scientists who have tried to substantiate or invalidate the hypothesis. A fundamental breakthrough has been the discovery of parallel fiber–Purkinje cell long-term depression (LTD) (Ito and Kano, 1982). The importance of this discovery followed that of long-term potentiation (LTP) in the hippocampus (Bliss and Lomo, 1973) so that the two major forms of learning (declarative and nondeclarative) had both their own form of plasticity. Conceptually, it was also relevant that synaptic transmission could be persistently changed in opposite directions, actually generating LTP and LTD at different synapses. However, the discovery of cerebellar LTD did not fully answer previous questions about cerebellar functioning but generated new questions instead. The first issue, very general indeed, is whether the cerebellum is a learning or a timing machine (D’Angelo and De Zeeuw, 2009; Eccles, 1973; Eccles et al., 1967; Ivry et al., 2002). The second issue is whether the cerebellum is the place where learning and memory take place or rather it is instrumental to cause learning in different brain regions (Diedrichsen et al., 2010; Raymond et al., 1996; Shadmehr and Mussa-Ivaldi, 2012). The third issue is whether parallel fiber–Purkinje cell LTD is needed and sufficient for cerebellar learning (Gao et al., 2012; Hansel et al., 2001). In order to answer these questions, various experimental approaches have been used and a constellation of evidences has been reported. Relevant results have been obtained performing accurate experimental investigations at the molecular, cellular, and microcircuit levels in experimental animals as well as using neuroimaging and neurostimulation in humans in vivo. Recently, the field has benefitted of various mathematical models, which helped assessing the consistency of functional hypotheses on the role of plasticity in cerebellar learning. In this review, I focus on how recent evidences help addressing the core of the issue: how does plasticity in the cerebellar circuit contribute to procedural learning and memory? As noticed above, cerebellar long-term synaptic plasticity was initially thought to occur only at the parallel fiber–Purkinje cell synapse, but now synaptic plasticity is known to be distributed and to occur at several sites in the granular layer, molecular layer, and deep cerebellar nuclei (DCN) (Gao et al., 2012; Hansel et al., 2001; Raman and Bean, 1999). The cerebellar circuit long-term modifications affect both excitatory and inhibitory synapses and involve regulation of both synaptic and nonsynaptic mechanisms. Here, the salient properties of plasticity at different network locations are reviewed and discussed in terms of their potential contribution to learning and memory in the cerebellum (Fig. 1).

1 Introduction

FIGURE 1 Schematic drawing of the cerebellar circuit and its forms of plasticity. The principal elements of the cerebellar circuit and associated structures are indicated: mossy fiber (MF), parallel fiber (PF), climbing fiber (CF), granule cell (GrC), Golgi cell (GoC), Purkinje cell (PC), stellate cell/basket cell (SC/BC), deep cerebellar nuclei cell (DCNC), and inferior olive cell (IO-C). The cerebellar cortex is indicated by a shadowed area. The cerebellar circuit expresses at least nine recognized forms of plasticity, some of which bidirectional, included into three main subcircuits: (1) granular layer (green area): mossy fiber–granule cell LTP and LTD (MF–GrC LTP/LTD), granule cell LTP of intrinsic excitability (GrC IE-LTP). (2) Molecular layer circuit (yellow area): presynaptic parallel fiber–Purkinje cell LTP and LTD (presynaptic PF–PC LTP/LTD), postsynaptic parallel fiber–Purkinje cell LTP and LTD (postsynaptic PF–PC LTP/LTD), climbing fiber–Purkinje cell LTD (CF-PC LTD), stellate cell/basket cell inhibitory LTP (SC/BC iLTP), Purkinje cell LTP of intrinsic excitability (PC IE-LTP). (3) Deep cerebellar nuclei (red area): mossy fiber–DCN cell LTP and LTD (MF–DCNC LTP/LTD), Purkinje cell–DCN cell inhibitory LTP and LTD (PC–DCNC iLTP/LTD), and DCN cell LTP of intrinsic excitability (DCNC IE-LTP).

33

34

CHAPTER 2 Plasticity in the Cerebellum

2 PLASTICITY IN THE GRANULAR LAYER Among all forms of cerebellar plasticity, that occurring in the granular layer is the hardest to understand since it has not been considered in previous theories. Actually, mossy fiber–granule cell plasticity was originally denied by stating that, since sooner or later it would saturate, it should not exist at all (Marr, 1969). Clearly, the concepts of LTP/LTD balance and of regulation of the inhibitory circuit were missing (see below). After its discovery, granular layer plasticity has been deeply investigated and has opened a view on how signal entering the cerebellum is processed and retransmitted.

3 MOSSY FIBER–GRANULE CELL LTP AND LTD Long-term synaptic plasticity in the granular layer occurs between mossy fibers and granule cells and consists of a bidirectional change involving LTP and LTD (D’Angelo et al., 1999; D’Errico et al., 2009; Gall et al., 2005; Sola et al., 2004) as well as changes in granule cell intrinsic excitability (Armano et al., 2000; Nieus et al., 2006). The induction depends on NMDA receptors and the subsequent calcium influx, which can be reinforced by activation of voltage-dependent calcium channels (VDCCs), metabotropic glutamate receptors (mGluRs), and calcium-induced calcium release (CICR) from intracellular stores: short lowfrequency bursts and poor membrane depolarization favor LTD, while long high-frequency bursts and strong membrane depolarization favor LTP (D’Errico et al., 2009; Gall et al., 2005). The level of depolarization, in turn, depends on the excitatory/inhibitory balance and therefore on the number of active mossy fibers and on the intensity of Golgi cell inhibition (Mapelli and D’Angelo, 2007). The sensitivity to these factors, and therefore the effectiveness of induction, depends on neuromodulators. In particular, nicotine acting on alfa6 receptors can bias induction in favor of LTP by shifting the calcium/plasticity relationship, thereby causing LTP with short bursts otherwise ineffective or even causing LTD (Prestori et al., 2013). Therefore, the LTP/LTD balance depends on both the input pattern and neuromodulators (extrinsic factors) as well as on local inhibitory network activity (intrinsic factors). Both LTP and LTD expressions depend on changes in release probability, which increases with LTP and decreases with LTD. The communication from the postsynaptic induction site and the presynaptic expression site is not completely clarified but requires nitric oxide (NO) to operate (Maffei et al., 2002, 2003). Finally, LTP is accompanied by a change in intrinsic electroresponsiveness, which enhances granule cell firing (Armano et al., 2000). This change consists in a reduction of spike threshold possibly related to a change in the persistent Na current and to changes in K currents (Nieus et al., 2006).

3 Mossy Fiber–Granule Cell LTP and LTD

3.1 Plasticity in the Granular Layer Inhibitory Circuit Given the strong dependence on GABAergic inhibition, the induction of LTP and LTD is expected to reflect activity in the inhibitory loop, which could in turn be regulated and be subject to plasticity. There is evidence that parallel fiber–Purkinje cell synapse can undergo LTD (Robberechts et al., 2010) and that membrane hyperpolarization can persistently change Golgi cell pacemaking (Hull et al., 2013). Moreover, several mechanisms may regulate inhibitory transmission, involving the tonic GABA level, metabotropic receptor systems, and various ionic channels in the pre- and postsynaptic elements of the granule cell–Golgi cell loops (Brandalise et al., 2012; Mapelli et al., 2009; Rossi et al., 2006). Although plasticity in the inhibitory Golgi cell loop has still to be investigated in detail, it may provide a powerful regulatory mechanism for mossy fiber–granule cell plasticity (Garrido et al., 2013; Mapelli et al., submitted for publication).

3.2 Plasticity in the Granular Layer In Vivo A remarkable advancement in the understanding of granular layer plasticity has come from the demonstration that granular layer LTP and LTD occur in vivo (Roggeri et al., 2008). LTP and LTD can be induced by patterned tactile stimulation and affect both the sensory component and the cerebrocortical component of granular layer response waves (Diwakar et al., 2011). Moreover, LTP in vivo can be markedly enhanced by activation of nicotinic acetylcholine receptors (Prestori et al., 2013), providing evidence for gating mechanisms relating plasticity to the behavioral state of attention and learning (Schweighofer et al., 2001, 2004).

3.3 The Consequences of Granular Layer Plasticity: Geometry, Timing, and Coding In situ experiments have revealed three remarkable properties of granular layer plasticity, which is translated into timing, geometry, and coding of output burst to be conveyed to Purkinje cells for further elaboration. LTP and LTD, thanks to their presynaptic expression mechanism, have a specific impact on the short-term dynamics of facilitation and depression occurring during transmission of mossy fiber bursts to granule cells (Nieus et al., 2006). Actually, LTP accelerates release anticipating emission of the first spike, while LTD does the opposite. Granule cell firing rate is controlled by plasticity of granule cell intrinsic excitability. This effect has two remarkable consequences: to implement the time-window control by regulating collision of the fist spike with inhibition in the feed-forward loop (D’Angelo and De Zeeuw, 2009) and to regulate information transmission (see Section 3.4). LTP and LTD, by modifying the quantal properties of neurotransmission, can change the input/output relationship of the granule cell. The impact of the release

35

36

CHAPTER 2 Plasticity in the Cerebellum

probability has been quantified using mutual information (MI) analysis, showing that LTP is associated with an increase and a decrease in MI. About half of MI is regulated by changes in release probability and the rest by changes in intrinsic excitability (Arleo et al., 2010). LTP and LTD tend to be organized in center-surround exploiting the geometrical arrangement of Golgi cell axons and dendrites with respect to mossy fibers and granule cells (Mapelli and D’Angelo, 2007). The reason of this is that the center is more excited making granule cells more depolarized so that the increased activation of NMDA receptors promotes LTP, while the surround is less excited and the lower calcium influx through NMDA receptors causes LTD. Eventually, the center reinforces its ability to convey spikes at higher frequency and with shorter delays than the surround, implementing time windowing and MI transfer in spatially organized manner (D’Angelo et al., 2013; Solinas et al., 2010).

3.4 Theoretical Implications Mossy fiber–granule cell LTP and LTD are unsupervised and may serve to improve spatiotemporal recoding of mossy fiber input patterns into new granule cell discharges (D’Angelo and De Zeeuw, 2009). In the Motor Learning Theory (Marr, 1969) and in the Adaptive Filter Model theory (Dean and Porrill, 2010, 2011; Dean et al., 2010), this operation has been conceived as an “expansion recoding” subdividing the input bursts conveyed by mossy fibers into multiple time-dependent spike sequences in granule cells. Given the properties explained above, it is expected that recoding will in fact redesign the geometry, timing, and information transfer through the granular layer. A first attempt at modeling the impact of granular layer plasticity was carried out using a firing rate model in which synaptic weights were controlled by optimizing information transfer through the granular layer (Schweighofer et al., 2001). The main observation was that the strength of mossy fiber synapses had to be counterbalanced by that of Golgi cell synapses and complemented by changes in intrinsic excitability. Moreover, mossy fiber–granule cell plasticity needed to be gated, circumventing the absence of teaching lines in this part of the circuit. This model actually anticipated the existence of changes in intrinsic excitability (Armano et al., 2000) and gating (Prestori et al., 2013), which have been subsequently demonstrated, and predicted the role of inhibitory plasticity between Golgi cells and granule cells. Recently, computational models have been used to simulate the impact of multiple distributed synaptic weights in the cerebellar granular layer network (Garrido et al., 2013; Solinas et al., 2010). In response to mossy fiber bursts, synaptic weights at multiple connections played a crucial role to regulate spike number and positioning in granule cells. The weight at mossy fiber to granule cell synapses regulated the delay of the first spike, and the weight at mossy fiber and parallel fiber to Golgi cell synapses regulated the duration of the time window during which the first spike could be emitted. Moreover, the weights of synapses controlling Golgi cell activation regulated the intensity of granule cell inhibition and therefore the number of spikes that could be emitted. First-spike timing was regulated with millisecond precision and the

4 Plasticity in the Molecular Layer

number of emitted spikes ranged from 0 to 3. Interestingly, different combinations of synaptic weights optimized either first-spike timing precision or spike number, efficiently controlling transmission and filtering properties. These results predicted that distributed synaptic plasticity could regulate the emission of quasi-digital spike patterns on the millisecond timescale and allow the cerebellar granular layer to flexibly control burst transmission along the mossy fiber pathway.

4 PLASTICITY IN THE MOLECULAR LAYER Long-term synaptic plasticity in the molecular layer has been predicted by the Motor Learning Theory based on the consideration that learning had to be supervised in order to be efficiently related to motor errors. Moreover, learning had to occur over the largest available set of information lines capable of carrying contextual information. Thus, the solution suggested by anatomy was to locate learning at the parallel fiber–Purkinje cell synapses (carrying contextual information) under supervision of climbing fibers (carrying teaching signals). The sign of the change was hypothesized to conform to either LTD (Marr, 1969) or LTP (Albus, 1971). This logical solution has dominated the field of cerebellar plasticity, suggesting that parallel fiber– Purkinje cell had to be the leading candidate to explain cerebellar learning (Ito, 1984). However, several additional aspects have emerged and the overall picture of cerebellar plasticity is now substantially changed (Gao et al., 2012). The Purkinje cells and their afferent synapses provide a wide set of plastic mechanisms, and different studies have supported the existence of multiple forms of plasticity, which can be summarized in five main groups differing for mechanisms, induction patterns, and functional implications (Boyden et al., 2004; Coesmans et al., 2004; Hansel et al., 2001; Ito, 2001): (1) (2) (3) (4) (5)

postsynaptic parallel fiber LTP and LTD presynaptic parallel fiber LTP and LTD climbing fiber LTD plasticity of Purkinje cell intrinsic excitability plasticity at molecular layer inhibitory synapses

4.1 Mechanisms of Postsynaptic Parallel Fiber LTP and LTD Parallel fiber–Purkinje cell LTD induction requires a complex signal transduction pathway. Parallel fiber stimulation causes glutamate release, which acts on two types of receptors: AMPA receptors and metabotropic receptors (mGluRs). The mGluRs bind the G-protein–GDP complex initiating a local signal transduction pathway activating phospholipase C and the hydrolysis of phosphatidylinositol 4,5-bisphosphate into the diffusible second messenger molecules inositol 1,4,5-trisphosphate (IP3) and diacylglycerol (DAG). The climbing fiber contribution to LTD induction consists of large, widespread calcium transients evoked by complex spikes (Konnerth et al., 1992). The release of glutamate by climbing fiber terminals

37

38

CHAPTER 2 Plasticity in the Cerebellum

activates AMPA receptors causing a strong Purkinje cell depolarization, with consequent Ca2þ increase caused by VDCCs and CICR. Moreover, climbing fibers also activate NMDA receptors further enhancing calcium influx (Piochon et al., 2010). The simultaneous increase of Ca2þ and DAG activates PKC, which acts as a coincidence detector of climbing fiber and parallel fiber activity. PKC phosphorylates the GluR2 subunit of AMPA receptors at the active parallel fiber synapses causing their desensitization and internalization through clathrin-mediated endocytosis, thus causing LTD parallel fiber–Purkinje cell contacts (Wang and Linden, 2000; Xia et al., 2000). The effect of climbing fiber pairing can be substituted by postsynaptic depolarization, suggesting that depolarization normally attributed to climbing fibers could be replaced by simultaneous activation of a few tens of parallel fibers. Indeed, simultaneous activation of several parallel fibers generates a postsynaptic calcium transient confined into the spines, whose amplitude increases with the number of active parallel fibers reaching levels similar to those produced in the same region by climbing fiber activation (Denk et al., 1995; Midtgaard et al., 1993). As a result, 1-Hz parallel fiber stimulation at relatively high stimulus strength (either single pulses or brief 10–50 Hz bursts) produces a gradual summation of the calcium signal leading to LTD (Hartell, 1996). In addition, LTD can be induced by intense parallel fiber stimulation. Therefore, climbing fiber pairing may not be an essential requisite but rather a facilitatory and synchronizing factor (see below). NO proved necessary for the induction of postsynaptic parallel fiber–Purkinje cell LTD evoked by brief parallel fiber bursts (Crepel and Jaillard, 1990; Daniel et al., 1993; Huang et al., 1993; Lev-Ram et al., 1997; Shibuki and Okada, 1991). In Purkinje cells, the activation of an NO-dependent form of guanylate cyclase (GC) triggers the cGMP/ PKG pathway, whose effect is to prevent the dephosphorylation of AMPA receptors by blocking the PP2/PP1/PP2B cascade and therefore unblocking PKC. The NO synthase (NOS) required to produce NO is most likely located in the parallel fibers (Kimura et al., 1998; Southam et al., 1992). It has been proposed that NO production arises from the activation of NR2A-containing NMDARs located on parallel fibers (Bidoret et al., 2009; Casado et al., 2002; Shin and Linden 2005). The specific deactivation kinetics of these NMDAR could determine the high frequencies of activity required for NO-dependent LTD induction. Activation of NMDA receptors in molecular layer interneurons could be another source of NO (Carter and Regehr, 2000). Postsynaptic parallel fiber–Purkinje cell LTD evoked by brief parallel fiber bursts is heterosynaptic and can spread tens of microns from the induction site (Hartell, 1996). This spread could involve NO (Reynolds and Hartell, 2000; Wang et al., 2000) and arachidonic acid (Reynolds and Hartell, 2001), both acting as potent activators of GC. A form of parallel fiber LTP can be induced by a single pulse of parallel fiber stimulation at 1 Hz for 5 min (Belmeguenai and Hansel, 2005; Coesmans et al., 2004; Lev-Ram et al., 2002). Its induction is postsynaptic, involving the activation of PKA, PKC, and CaMKII (van Woerder et al., 2009) and the insertion of GluR2 subunit of AMPA receptors in the spine membrane (Kakegawa and Yuzaki, 2005). This molecular mechanism can be considered opposite to the AMPA receptors internalization that leads to parallel fiber–LTD, and indeed, it has been shown that

4 Plasticity in the Molecular Layer

these forms of LTP and LTD are mutually reversible (Coesmans et al., 2004; Lev-Ram et al., 2003). The postsynaptic Ca2þ transient plays an important role in determining the direction of plasticity at parallel fiber–Purkinje cell synapse. Interestingly, the bidirectional plasticity at this synapse is the mirror image of the one previously unraveled in the hippocampus (Jo¨rntell and Hansel, 2006). The model of different Ca2þ thresholds for bidirectional plasticity was proposed in 1982 (Bienenstock et al., 1982) and termed “BCM rule.” According to the BCM rule, lower and higher Ca2þ transients are associated with the induction of LTD and LTP, respectively. In parallel fiber–Purkinje cell synapse, there is a opposite scenario and lower and higher Ca2þ transients are associated with the induction of LTP and LTD, respectively. Therefore, classical parallel fiber–Purkinje cell LTD may be just a special case of a general mechanism of bidirectional parallel fiber–Purkinje cell plasticity expressed by regulation of AMPA receptor desensitization and membrane expression. These LTP and LTD are driven by two forces, the amount of intracellular calcium and the amount of NO. High calcium can be obtained by high-frequency coherent activation of parallel fibers and by activating the climbing fiber. High NO can be obtained by activation of parallel fiber presynaptic terminals and molecular layer interneurons. Various combinations of these factors would explain the variety of induction conditions for LTD and LTP.

4.2 Mechanisms of Presynaptic Parallel Fiber LTP and LTD In the last decade, it has become apparent that a climbing fiber-independent form of LTP with presynaptic expression can also occur. This form of LTP can be observed after low-frequency (2–8 Hz) parallel fiber stimulation (Crepel and Jaillard, 1990; Hirano, 1991; Sakurai, 1987; Shibuki and Okada, 1992). This LTP at the parallel fiber synapse is triggered by presynaptic calcium influx, which activates Casensitive adenylyl cyclase 1 (AC1), which rises cAMP and activates PKA increasing neurotransmitter release (Kimura et al., 1998; Linden and Ahn, 1999; Salin et al., 1996; Storm et al., 1998). PKA acts at the presynaptic site by phosphorylating vesicle release-related proteins, in particular, the active zone protein Rac1 and vesicular proteins (Castillo et al., 1997; Powell et al., 2004). This LTP, like the one with postsynaptic expression, may also depend on NO production (Hartell, 2002; Qiu and Knopfel, 2007). NO regulates the probability of glutamate release at activated synapses and through transcellular diffusion at heterosynaptic synapses. This mechanism occurs via cGMP- and PKG-dependent pathways. In addition, endocannabinoids can regulate presynaptic LTP (Le Guen and De Zeeuw, 2010). Endocannabinoid release is evoked by high-frequency bursts, depending on the activation of postsynaptic mGluR1 and NMDA receptors in Purkinje cell or molecular layer interneuron. Activation of cannabinoid 1 (CB1) receptors in parallel fiber terminal suppresses AC1 and thereby attenuates cAMP-dependent PKA activity and induction of presynaptic LTP (van Beugen et al., 2006). Therefore, a retrograde regulation mechanisms takes part to presynaptic LTP control. A presynaptically expressed parallel fiber LTD is observed when presynaptically expressed parallel fiber LTP is prevented pharmacologically (Qiu and Kno¨pfel,

39

40

CHAPTER 2 Plasticity in the Cerebellum

2009). This type of plasticity is most effectively induced by 4 Hz parallel fiber stimulation, a protocol similar to that effective for presynaptic LTP, and requires activation of cannabinoid CB1 receptors. In this case, the endocannabinoids are released in an NMDA receptor-dependent, but not mGlu1 receptor-dependent, fashion. Thus, in principle, bidirectional plasticity mechanisms exist for both postsynaptic and presynaptic plasticity at the parallel fiber–Purkinje cell synapse.

4.3 Mechanisms of Climbing Fiber LTD For long time, the climbing fiber to Purkinje cell synapse was considered to fire a large all-or-nothing action potential independent of strength of climbing fiber stimulation. However, the climbing fiber synapse can also undergo plastic changes (Hansel and Linden, 2000). Climbing fiber LTD was induced by a short tetanization of climbing fibers causing a reduction of the slow component of the complex spike. The climbing fiber LTD also results in change in afterhyperpolarization, which can prolong the complex spike pause. Interestingly, the changes induced in calcium transients following climbing fiber LTD can affect the induction of both postsynaptically expressed LTD and LTP at the parallel fiber to Purkinje cell synapse (Coesmans et al., 2004). Thus, the climbing fiber–Purkinje cell synapse can exert a complex regulatory role on long-term synaptic plasticity and dendritic integration changing the spike output of the Purkinje cell (Othsuki et al., 2009).

4.4 Plasticity of Purkinje Cell Intrinsic Excitability Purkinje cell excitability can be enhanced by somatic current injections or parallel fiber stimulation protocols that also induce parallel fiber LTP (Belmeguenai et al., 2010). Signal cascades involved in LTP are shared by intrinsic plasticity, which indeed requires postsynaptic Ca2þ signaling followed by activation of the PP1/PP2A/PP2B cascade. A complex interaction between these phosphatase and PKA and casein kinase 2 ultimately leads to a downregulation of small conductance Ca2þ-activated K channels. Intrinsic plasticity of Purkinje cells is promoted by parallel fiber LTP, but inhibits the subsequent LTP induction. A possible explanation for this reduced LTP induction is that intrinsic plasticity is accompanied by enhanced spine calcium signaling (Belmeguenai et al., 2010) which could promote LTD rather than LTP (Coesmans et al., 2004). Thus, parallel fiber LTP could be accompanied by intrinsic plasticity at activated parallel fiber synapses, while its induction could be reduced through intrinsic plasticity at weaker and neighboring nonpotentiated synapses.

4.5 Plasticity at Molecular Layer Inhibitory Synapses At low frequency, climbing fibers not only excite Purkinje cells but also suppress GABA release from inhibitory interneurons through AMPA receptor activation (Satake et al., 2000, 2006). However, repetitive climbing fiber stimulation can potentiate the amplitude of inhibitory postsynaptic currents in Purkinje cells

4 Plasticity in the Molecular Layer

(Kano, 1996; Kawaguchi and Hirano, 2002). This inhibitory LTP of GABAergic interneuron–Purkinje cell synapse requires a postsynaptic Ca2þ transient in Purkinje cells due to activation of voltage-gated Ca2þ channels and IP3-mediated Ca2þ release from internal store (Hashimito and Kano, 2001). This Ca2þ transient activates CaMKII, which in turn leads to a Ca2þ-dependent upregulation of GABA-A receptor activity (Kano et al., 1992, 1996; Kawaguchi and Hirano, 2007).

4.6 The Neurophysiological Consequences of Molecular Layer Plasticity As a first step to understand the complex set of plasticity mechanisms in the molecular layer, the consequences of the classical form of parallel fiber–Purkinje cell LTD (Ito and Kano, 1982) need to be revisited. This LTD, induced by low-frequency pairing of parallel fiber and climbing fiber activity, causes a decrease of simple spike firing in the Purkinje cell and thus leads to reduced inhibitory input to DCN cells, increased output from the cerebellum, and enhanced execution of movement (Hansel et al., 2001; Ito, 1984, 2001). This LTD regulates the pattern of Purkinje cell discharge (Steuber et al., 2007), and it was recently observed at the parallel fiber–Purkinje cell synapse in alert animals (Ma´rquez-Ruiz and Cheron, 2012). According to the central statement of the Motor Learning Theory, classical parallel fiber–Purkinje cell LTD is supervised and serves to store correlated granular layer patterns under the teaching signal generated by climbing fibers. A critical demonstration of this proposal is that the Purkinje cell may indeed operate as a “perceptron” capable of detecting the huge amount of combinations generated at its parallel fiber synapses, which can be “digitally” switched on or off by LTD or LTP (Brunel et al., 2004). However, in terms of mechanisms controlling the synaptic strength, the picture emerging from latest evidences is much more complex than previously thought. It is well established that the large majority of parallel fiber synapses are silent (Isope and Barbour, 2002; Ito, 2006; Wang et al., 2000), suggesting that LTD is the dominating plasticity process. If the default state of parallel fiber synapses during development is to be silent, LTP would then be the driving process required in order to obtain any kind of learning (Jo¨rntell and Hansel, 2006). This hypothesis is akin with the proposal that, by missing feedback inhibition, the molecular layer may use LTD as a surrogate in order to enhance the signal-to-noise ratio in Purkinje cells (De Schutter, 1995). Importantly, while postsynaptic LTD/LTP are at least partially supervised through climbing fiber activity, presynaptic LTP/LTD appear to depend on ongoing activity patterns in the cerebellar network instead. Presynaptic LTP may be driven by salient patterns selected in the granular layer, for example, by high-frequency bursts or by low-frequency correlated activity. Therefore, learning would result from the interaction of several forms of synaptic plasticity and not just from classical LTD alone. But what is then the role of climbing fibers and complex spikes? The climbing fibers may bias the learning process by regulating the LTP/LTD balance when

41

42

CHAPTER 2 Plasticity in the Cerebellum

learning becomes intense, like when attention is enhanced or errors in motor execution become large and frequent.

4.7 The Behavioral Consequences of Molecular Layer Plasticity A major proof that postsynaptic parallel fiber–Purkinje cell LTD could provide the critical cellular mechanism for cerebellar learning was its analogy with the mechanisms of acquisition of associative eyelid conditioning (Medina and Mauk, 1999). The conditioned stimulus is conveyed to the cerebellar circuitry through the parallel fibers, while the unconditioned stimulus through the climbing fibers. The association of these inputs would generate a depression of the parallel fiber–Purkinje cell synapses, producing a conditioned via disinhibition of the DCN. A new and more complex view on the impact of molecular layer plasticity on cerebellar learning and performance has been opened by blocking specific forms of plasticity and using cell-specific transgenic mice, which have recently shown that motor learning can occur normally in the absence of parallel fiber–Purkinje cell LTD (Schonewille et al., 2011). In turn, investigations on transgenic mice have suggested that postsynaptic LTP at parallel fiber–Purkinje cell synapse may substantially contribute to cerebellar motor learning. CAMK2b / mice are ataxic and show deficit in the acquisition of new motor task (van Woerder et al., 2009), and mutant mice in which LTP induction is blocked show pronounced deficits in motor coordination (Schonewille et al., 2010). Moreover, the Purkinje cell-specific deletion of PP2B (L7-Pp2b), which leads to prominent impairments in motor performance and motor learning, affects not only LTP at parallel fiber–Purkinje cell but also intrinsic excitability (Schonewille et al., 2010). Finally, plasticity in molecular layer interneuron–Purkinje cell synapses may also be relevant for cerebellar learning (Wulff et al., 2009). The selective deletion from Purkinje cells of the two subunits, and thereby GABA-A receptors (Purkinje cell-2 mice), affects Purkinje cell simple spike activity and motor behavior. Although these mutant mice are not ataxic, they show a deficit in both phase reversal learning and gain and phase consolidation of the vestibulo-ocular reflex (VOR). Thus, plasticity at molecular layer interneuron–Purkinje cell synapses might have a role in cerebellar signal coding and memory formation but may not be essential for normal motor performance.

5 AN INTEGRATED VIEW OF CEREBELLAR CORTICAL PLASTICITY It is reasonable to believe that the numerous forms of plasticity of the cerebellar cortex need to be integrated into coherent patterns, although the corresponding regulation mechanisms remain poorly investigated. These forms of plasticity may be coordinated by intrinsic and extrinsic mechanisms. Intrinsic mechanisms include local biochemical cascades and inhibitory circuits, while extrinsic mechanisms include oscillation and resonance and neuromodulatory systems (Fig. 2). This section

5 An Integrated View of Cerebellar Cortical Plasticity

43

44

CHAPTER 2 Plasticity in the Cerebellum

proposes hypothetical theory-independent organizing schemes deserving experimental and modeling evaluation.

5.1 Potentiation of Transmission Channels and Signal-to-Noise Ratio in the Mossy Fiber Pathway 5.1.1 The NO System of Granule Cells The granule cells are neurons producing NO both in the dendrites and in parallel fiber terminals following NMDA receptor activation. NO released in the granular layer is needed for mossy fiber–granule cell LTP (Maffei et al., 2003), while NO released in the molecular layer from parallel fibers causes presynaptic LTP and enhances postsynaptic LTD (Casado et al., 2002). NO is also responsible for heterosynaptic plasticity. Therefore, NO release may exert a coordinated regulatory action causing LTP at both the mossy fiber–granule cell and presynaptic parallel fiber–Purkinje cell synapse, potentiating selected transmission channels.

5.1.2 The Calcium Control System in Purkinje Cells Intracellular calcium in Purkinje cell spines depends on several regulatory mechanisms and controls several forms of plasticity. Basically, all factors causing strong Purkinje cell excitation, including intense parallel fiber and climbing fiber activity, lead to strong calcium elevations depressing AMPA receptors (postsynaptically expressed LTD) and enhancing GABA-A receptors (inhibitory LTP), therefore globally reducing Purkinje cell responsiveness. NO released by activity in parallel fibers and molecular layer interneurons (MLIs) favors postsynaptically expressed LTD. Conversely, weak calcium elevations enhance AMPA receptors (postsynaptically expressed LTP) and enhance Purkinje cell intrinsic excitability, therefore globally raising Purkinje cell responsiveness. FIGURE 2 The major mechanisms of plasticity in cerebellar cortical circuit. The figure highlights two elements: the central position of granule cells in coordinating granular layer and molecular layer plasticity, and the PC pivotal role in coordinating molecular layer plasticity. The granule cells express NMDA receptors and release NO, thus controlling plasticity both at the mossy fiber and at the parallel fiber synapses. The PC coordinates plasticity at the synapses formed with parallel fibers, climbing fibers, and stellate cells/basket cells. Mossy fiber (MF), parallel fiber (PF), climbing fiber (CF), granule cell (GrC), Golgi cell (GoC), Purkinje cell (PC), and stellate cell/basket cell (SC/BC). Various membrane receptors and ionic channels are indicated in the figure. Intracellular elements have their usual meaning and are described in the text. Cyclic AMP, adenylate cyclase, protein kinase A (cAMP/AC/PKA), Cyclic GMP, guanylyl cyclase, protein kinase G (cAMP/AC/PKA), diacylglycerol, IP3, protein kinase C (DAG, OP3, PKC), phospholipase 1, A1, A2 (PLA1/ PL1/PLA2), calcium–-calmodulin kinase II (CAMKII) calcium (Ca2þ), nitric oxide (NO), and nitric oxide synthase (NOS). Depolarization is indicated by yellow stars.

5 An Integrated View of Cerebellar Cortical Plasticity

As a whole, it appears that NO, by coordinating LTP, may potentiate transmission along selected mossy fiber channels, while the calcium control system in Purkinje cells may rescale Purkinje cell responsiveness enhancing the signal-to-noise ratio. It should also be considered that the presynaptic expression of NO-dependent LTP, both at the mossy fiber and at the parallel fiber synapses, effectively enhances short-term facilitation against short-term depression. This would improve transmission of single spikes or short spike busts, further increasing the effectiveness of channels involved.

5.2 Contrast Enhancement and Geometrical Organization of Plasticity The spatial organization of LTP and LTD in the granular layer, which depends on the geometrical arrangement of Golgi cell inhibition (Mapelli and D’Angelo, 2007), favors transmission of spikes at higher frequency and with shorter delays in the center than in the surround (D’Angelo et al., 2013; Gandolfi et al., 2014; Solinas et al., 2010). It would therefore be of interest to understand how long-term synaptic plasticity is organized in the molecular layer. It has been suggested that parallel fiber– Purkinje cell LTD occurs together with parallel fiber–MLI LTP and MLI–Purkinje cell iLTP, while parallel fiber–Purkinje cell LTP occurs together with parallel fiber– MLI LTD and MLI–Purkinje cell iLTD (Gao et al., 2012). This coordinated chain of plastic events would concur not only to reinforce the changes occurring in Purkinje cells but also to regulate the spatial distribution of plasticity. Inhibition generated by stellate cells would preferentially spread along the parallel fiber beams, while that of basket cells orthogonal to it. Therefore, an active granule cell ascending axon bundle may generate spatially organized LTP and LTD also in the molecular layer, an issue that remains to be clarified.

5.3 Coordination of Plasticity During Patterned Circuit Activity In response to the incoming mossy fiber bursts, the granule cells respond with new bursts which reach the parallel fiber terminals and are transmitted to Purkinje cells (Chadderton et al., 2004). Repetition of bursts is essential to generate LTP and LTD: long high-frequency bursts induce LTP, while short low-frequency bursts induce LTD. Likewise, at the parallel fiber–Purkinje cell synapse, NO production requires short bursts in order to effectively regulate presynaptically and postsynaptically induced LTP and LTD. Although the impact of native granular layer patterns on Purkinje cell responses remains to be investigated, certain patterns of activity may facilitate coordinated plastic changes. Particular attention has been given to theta-burst patterns that occur during certain behavioral states like active motion and cognition as well as during sleep. Bursts are transmitted from the thalamocortical system to the cerebellum through corticocerebellar projections (Ros et al., 2009). The granular layer is itself resonant at theta frequency and actively amplifies the incoming theta-burst patterns generating

45

46

CHAPTER 2 Plasticity in the Cerebellum

coherent theta-frequency oscillations through a complex set of mechanisms (Gandolfi et al., 2013; Hartmann and Bower, 1998). The granule cell bursts reach the parallel fiber terminals and are transmitted to Purkinje cells, which are also activated by the climbing fibers conveying low-frequency signals from the inferior olive. The climbing fibers have been shown to play the role of synchronizing subsets of Purkinje cells (Marshall and Lang, 2011), and coincidence of parallel fiber and climbing fiber low-frequency oscillations has been proposed to amplify specific Purkinje cell responses by resonance (D’Angelo, 2010). This mechanism may effectively select subsets of Purkinje cells at the intersection of climbing fiber and parallel fibers oscillating in phase and promote plasticity at their synapses. Notably, presynaptically expressed LTP and LTD are generated by theta patterns. Moreover, presynaptically expressed LTP and LTD are promoted by climbing fibers, whose activity needs to be paired with that of the parallel fibers within a time window of about 200 ms (Wang et al., 2000), a duration coincident with a theta cycle. In this way, plasticity in Purkinje cell synapses could be coordinated by theta-burst cycles over large subsets of synapses and neurons.

5.4 Gating of Plasticity by Neuromodulatory Systems 5.4.1 Neuromodulatory Mechanisms of Gating Various neuromodulators (noradrenaline, serotonin, acetylcholine, dopamine) may play a critical role in gating cerebellar LTP and LTD. Given their wide distribution across the cerebellar cortex and nuclei, these systems have a remarkable potential to control when and how learning has to occur (Schweighofer et al., 2004). This mechanism is required to relate cerebellar learning to general functional state of the brain: serotonin would convey responsibility signals, noradrenaline would convey errorrelated signals, acetylcholine would convey success signals, and dopamine would convey reward signals. Although it has been shown that acetylcholine can modify cerebellar plasticity (Prestori et al., 2013; Rinaldo and Hansel, 2013), the impact of other neuromodulators requires further investigation.

6 CEREBELLAR CORTICAL PLASTICITY AND TIMING The cerebellum has long been proposed to operate as a “timing machine” (Eccles, 1967) and a “learning machine” (Ito, 2006). The cerebellum controls motor behavior with millisecond precision (Osborne et al., 2007; Timmann et al., 1999), so it is expected that its computations are performed on a comparable time scale. There are indications, mostly derived from cellular investigations in rat cerebellar slices, that the granular layer (see Fig. 1) is capable of exerting a close control on spike timing (D’Angelo and De Zeeuw, 2009). The granule cells (GrCs) generate brief spike bursts in the axon initial segment, which are almost instantaneously ( N-O USr > 0

pf

Grc

NO

-

CS

CS

CS

CS

CS

CS

CN

cf

CN

Output

CS pathway

Overexpectation US < N-O USr < 0

PC

PC

mf

Trained US = N-O USr = 0

IO / PC intra.

IO

CS

+

Eyestim

CS

Eyestim

CS

Eyestim

EMG US pathway

CS Eyestim

CS Eyestim

CS Eyestim

FIGURE 5 Predicted activity in Purkinje cells (PC), cerebellar nuclei (CN), inferior olive as reflected by EPSPs in Purkinje cell dendrites (IO/PC intra.), and eyelid muscle (EMG), during different stages of conditioning (naive, trained, and overexpectation). The reinforcing value of the US signal (USr) depends on the balance between the US and the nucleo-olivary inhibition (N-O). In a naive state, the CS does not cause any change in Purkinje cell activity, and because of this, the cerebellar nuclei remain inhibited. Since there is little nucleo-olivary inhibition, eye stimulation results in a burst of EPSPs in the Purkinje cell dendrite, which drives plastic changes in the cerebellar cortex, resulting in gradually increasing nucleo-olivary inhibition. After training, Purkinje cells disinhibit the cerebellar nuclei, resulting in an EMG response as well as increased nucleo-olivary inhibition, which in turn reduce the number of EPSPs elicited by eye stimulation. The circuit has reached equilibrium where the climbing fiber input does not induce further plasticity. When two CSs (both generating CRs) are presented simultaneously, there will be a stronger pause response in the Purkinje cells. This results in more disinhibition of the cerebellar nuclei as well as a stronger overt CR (EMG activity). In addition, more nucleo-olivary inhibition suppresses the burst from the olive below the equilibrium point, driving plasticity in the opposite direction (extinction).

single impulse causes extinction of previously acquired pause responses (Rasmussen et al., 2013) (Fig. 4). Our proposed model, in which learned pauses in Purkinje cell activity can alter the number of spikes in the climbing fiber signal, give rise to a number of predictions, some of which remain untested (Fig. 5). For example, given that complex spike appearance depends on the number of spikes in the climbing fiber signal, the appearance of a peripherally elicited complex spikes ought to depend on whether it is preceded by a Purkinje cell pause response. In extension, we predict that the appearance of complex spikes change gradually as learning progresses (Fig. 5). Though difficult to test we would also predict that in intracellular recordings, it should be

113

114

CHAPTER 5 Feedback Control of Learning

possible to see a reduction in the number of EPSPs elicited by peripheral stimulation if the stimulation is preceded by a CR.

8 BACK TO BEHAVIOR These neurophysiological findings bring us closer to understanding various behavioral phenomena. We hope to have established that in a conditioned animal, the CS results in inhibition of the IO, the strength of which corresponds to the degree of association between the CS and the US. Due to the uniquely long delay in the nucleoolivary pathway, GABAergic input from the cerebellar nuclei to the IO coincides with the arrival of the US signal (if present). Ultimately, this means that the stronger the association between the CS and US, the stronger the inhibition of the US signal will be. Based on this, we can explain both blocking and overexpectation, as well as make some additional predictions. Learning to a second CS, when combined with a CS that is already producing CRs, may be blocked because the olivary inhibition generated by the CR suppresses the teaching signal (Kim et al., 1998). In the light of the studies reviewed here, we suggest that this olivary suppression does not need to prevent the olive from firing entirely. To see the blocking effect, it might be sufficient that the nucleo-olivary inhibition changes the number of spikes in the olivary discharge. Nucleo-olivary feedback can also explain the fact that reducing the US intensity following acquisition of a CR results in partial extinction (Kehoe and White, 2002). Reducing the strength of the teaching signal in a situation where the negative feedback matches the strength of the teaching signal would move the system away from equilibrium which would trigger further plasticity, in this case extinction. Based on their mathematical framework, Rescorla and Wagner (1972) predicted that simultaneous presentation of two CSs, each of which produce CRs, will result in partial extinction, even when followed by the US. That is, following a number of such combined presentations, the response to the individual CSs will decrease. The reasoning was that combined presentations would result in overexpectation of the strength of the US, and since plasticity depends on violation of expectations, the response to the individual CSs should change. This prediction was subsequently tested and confirmed on a behavioral level in rabbits (Kehoe and White, 2004). Overexpectation can potentially be explained by the cerebellar feedback mechanisms discussed in this chapter. It is plausible that if two CSs, each of which inhibit the IO, are presented simultaneously, then the combined IO inhibition will suppress the teaching signal below the equilibrium level, resulting in extinction.

9 FEEDBACK, ANTICIPATION, AND NUCLEO-OLIVARY INHIBITION The idea that Purkinje cells regulate the activity of IO cells projecting back to it, and that interactions in this feedback loop are critical for motor learning has recently received increased attention (Chaumont et al., 2013; Herreros and Verschure, 2013;

10 Broadening the Perspective

Ito, 2008; Koziol et al., 2011; Lepora et al., 2010; Schweighofer et al., 2013). Through our improved understanding of the feedback mechanisms that are active during cerebellar learning, we can begin to understand what it really means to say that the brain is anticipating future events. We have argued that the Purkinje cell CR is the neurophysiological basis of the learned blink response. However, each Purkinje cell CR also results in inhibition of the inferior olive, which is timed so that it coincides with the arrival of the US, if it is present. In essence, the Purkinje cell CR and the resulting inhibition of the inferior olive are an anticipation of the coming US signal. The suppression of the US signal, assuming it is delivered, will be proportional to the amount of learning that has taken place. We suggest that if the anticipated US intensity matches the actual US intensity, there will be no further plasticity in the cerebellar cortex, and in extension, there will be no further behavior modification. In other words, the cerebellar network will be at an equilibrium level where subsequent input, given that it does not deviate from prior input, will not induce further plasticity. However, if the anticipated US intensity deviates from the actual US intensity, then the signal from the inferior olive to the cortex will be above or below the equilibrium level, which will induce plasticity (Fig. 5). For example, suppose that, following a number of paired CS–US presentations, a subject has acquired CRs. This means that Purkinje cells in the subject’s cerebellar cortex pause following presentation of the CS (with a certain delay). This pause response inhibits the US signal. When this stage has been reached, the system is at an equilibrium meaning that additional paired CS–US presentations will not lead to further changes. Now suppose that we change the intensity of the US stimulation. In this case, the inhibition of the inferior olive will match the intensity of the US, and therefore, the signal that reaches the cerebellar cortex will deviate from the equilibrium. In a metaphorical sense, we could say that there is a difference between the US intensity, as predicted by the Purkinje cells, and the actual US intensity. We have now come full circle and should be able to tie everything together. It was shown early on that conditioning occurs when our expectations are violated or our predictions are erroneous. In other words, we must be able to somehow anticipate future events and it is when our anticipations fail that learning occurs. Since the brain cannot really “anticipate” or “predict,” we must try to find the neurophysiological basis of these events. Here we hope to have shown that the Purkinje cell CR is a potential candidate for neural activity that, in a certain sense, anticipates future outcomes and determines subsequent plasticity.

10 BROADENING THE PERSPECTIVE Although there exist a great deal of evidence that more or less directly supports the claims asserted here, many predictions have yet to be rigorously tested in the lab. For example, even though we know that the Purkinje cell CR can inhibit the inferior olive (Hesslow and Ivarsson, 1996), resulting in a reduction of complex spikes (Rasmussen et al., 2008), it remains to be shown if and how the nucleo-olivary inhibition can affect the number of spikes in the climbing fiber signal. Given that this

115

116

CHAPTER 5 Feedback Control of Learning

variable appears to be crucial in determining the direction of learning (Mathy et al., 2009; Najafi and Medina, 2013; Rasmussen et al., 2013), answering these questions ought to be a priority. The extent to which the general principles described here apply to other parts of the brain is also an open question. The conclusions we have drawn have, with a few exceptions, been based on studies on eyeblink conditioning which is thought to rely on a relatively discrete part of the cerebellum. Other parts of the cerebellum are involved in other types of learning. We know for instance that the flocculus is involved in the adaptation of the vestibulo-ocular reflex (Ito, 1998), and still other parts of the cerebellum contribute to other brain functions. It is plausible that other parts of the cerebellum also have a feedback system that shares features with the system that has been described here. However, it is possible, perhaps even likely, that there are differences between the feedback system that controls the acquisition of conditioned eyeblinks and the feedback systems that control other cerebellar functions. Broadening the perspective even further, we may ask how the feedback system described here relates to feedback systems for different types of learning that may or may not rely on the cerebellum. For example, it has been shown that the activity of dopaminergic following a reward is greater if the reward was unexpected based on the history of rewards (Fiorillo et al., 2003). Indeed as recognized by Schultz (2006), the activity of dopamine neurons in response to rewarding stimuli is consistent with the Rescorla Wagner model. Just like the signal from the inferior olive to the cerebellar cortex decreases as learning progresses, and the teaching signal becomes predictable, so the activity of dopamine neurons decreases as the reward becomes increasingly predictable. If the actual teaching signal and the predicted teaching signal in eyeblink conditioning are “compared” in the inferior olive, where is the predicted reward and the actual reward compared that allow dopamine neurons to fire in the way they do? Is there a separate anatomical system filling this function or are other brain structures recruited? It is not inconceivable that dopamine neurons, via the pathways connecting the basal ganglia and the cerebellum (Bostan and Strick, 2010), recruit the cerebellar circuitry to perform a comparison between the expected and actual reward signals. Indeed, it would be more economical if different parts of the brain shared a common neural circuitry to perform comparisons between anticipated outcomes and actual outcomes. Future research should aim to determine to what extent the principles described in this chapter applies to different circumstances and different parts of the brain. In conclusion, there is little doubt that the cerebellum plays an important role in various forms of learning and that there are feedback mechanisms in place to regulate this learning process. Specifically, we suggest that the Purkinje cell CR, apart from generating the overt CR, can push the intensity of the US signal above or below an equilibrium level, which, in turn, determines subsequent plasticity. This means that, in an important sense, learning-induced changes in Purkinje cell activity constitute an “expectation” or “anticipation” of a future event (the US), and, consistent with theoretical models, future learning depends on the accuracy of this expectation.

References

References Albus, J., 1971. A theory of cerebellar function. Math. Biosci. 10, 25–61. Andersson, G., Hesslow, G., 1987. Activity of Purkinje cells and interpositus neurones during and after periods of high frequency climbing fibre activation in the cat. Exp. Brain Res. 67, 533–542. Andersson, G., Garwicz, M., Hesslow, G., 1988. Evidence for a GABA-mediated cerebellar inhibition of the inferior olive in the cat. Exp. Brain Res. 72, 450–456. Apps, R., Garwicz, M., 2005. Anatomical and physiological foundations of cerebellar information processing. Nat. Rev. Neurosci. 6, 297–311. Armstrong, D.M., Rawson, J.A., 1979. Activity patterns of cerebellar cortical neurones and climbing fibre afferents in the awake cat. J. Physiol. 289, 425–448. Bengtsson, F., Hesslow, G., 2006. Cerebellar control of the inferior olive. Cerebellum 5, 7–14. Bengtsson, F., Jirenhed, D., Svensson, P., Hesslow, G., 2007. Extinction of conditioned blink responses by cerebello-olivary pathway stimulation. Neuroreport 18, 1479–1482. Best, A.R., Regehr, W.G.G., 2009. Inhibitory regulation of electrically coupled neurons in the inferior olive is mediated by asynchronous release of GABA. Neuron 62, 555–565. Bostan, A.C., Strick, P.L.L., 2010. The cerebellum and basal ganglia are interconnected. Neuropsychol. Rev. 20, 261–270. Chaumont, J., Guyon, N., Valera, A.M., Dugue´, G.P., Popa, D., Marcaggi, P., Gautheron, V., Reibel-Foisset, S., Dieudonne´, S., Stephan, A., Barrot, M., Cassel, J.-C., Dupont, J.-L., Doussau, F., Poulain, B., Selimi, F., Le´na, C., Isope, P., 2013. Clusters of cerebellar Purkinje cells control their afferent climbing fiber discharge. Proc. Natl. Acad. Sci. U.S.A. 110, 16223–16228. Dean, P., Porrill, J., Ekerot, C.F., Jorntell, H., 2010. The cerebellar microcircuit as an adaptive filter: experimental and computational evidence. Nat. Rev. Neurosci. 11, 30–43. De Zeeuw, C.I., Holstege, J.C., Ruigrok, T.J.H., Voogd, J., 1989. The cerebellar, mesodiencephalic and GABAergic innervation of the glomeruli in the cat inferior olive. A comparison at the ultrastructural level. In: Strata, P. (Ed.), The Olivocerebellar System in Motor Control. Springer, London, pp. 111–116. Ebner, T.J., Pasalar, S., 2008. Cerebellum predicts the future motor state. Cerebellum 7, 583–588. Eccles, J.C.C., Llinas, R., Sasaki, K., 1966. The excitatory synaptic action of climbing fibres on the Purkinje cells of the cerebellum. J. Physiol. 182, 268–296. Fiorillo, C.D., Tobler, P.N., Schultz, W., 2003. Discrete coding of reward probability and uncertainty by dopamine neurons. Science 299, 1898–1902. Gormezano, I., Kehoe, E.J., Marshall, B., 1983. Twenty years of classical conditioning research with the rabbit. Prog. Psychobiol. Physiol. Psychol. 10, 197–275. Heiney, S.A., Kim, J., Augustine, G.J., Medina, J.F., 2014. Precise control of movement kinematics by optogenetic inhibition of Purkinje cell activity. J. Neurosci. 34, 2321–2330. Herreros, I., Verschure, P.F.M.J., 2013. Nucleo-olivary inhibition balances the interaction between the reactive and adaptive layers in motor control. Neural Netw. 47, 64–71. Hesslow, G., 1986. Inhibition of inferior olivary transmission by mesencephalic stimulation in the cat. Neurosci. Lett. 63, 76–80. Hesslow, G., 1994a. Correspondence between climbing fibre input and motor output in eyeblink-related areas in cat cerebellar cortex. J. Physiol. 476, 229–244. Hesslow, G., 1994b. Inhibition of classically conditioned eyeblink responses by stimulation of the cerebellar cortex in the decerebrate cat. J. Physiol. 476, 245–256.

117

118

CHAPTER 5 Feedback Control of Learning

Hesslow, G., Ivarsson, M., 1994. Suppression of cerebellar Purkinje cells during conditioned responses in ferrets. Neuroreport 5, 649–652. Hesslow, G., Ivarsson, M., 1996. Inhibition of the inferior olive during conditioned responses in the decerebrate ferret. Exp. Brain Res. 110, 36–46. Hesslow, G., Yeo, C.H., 2002. The functional anatomy of skeletal conditioning. In: Moore, J.W. (Ed.), A Neuroscientist’s Guide to Classical Conditioning. Springer, London, pp. 86–146. Ito, M., 1984. The modifiable neuronal network of the cerebellum. Jpn. J. Physiol. 34, 781–792. Ito, M., 1998. Cerebellar learning in the vestibulo-ocular reflex. Trends Cogn. Sci. 2, 313–321. Ito, M., 2001. Cerebellar long-term depression: characterization, signal transduction, and functional roles. Physiol. Rev. 81, 1143–1195. Ito, M., 2008. Control of mental activities by internal models in the cerebellum. Nat. Rev. Neurosci. 9, 304–313. Jirenhed, D., Bengtsson, F., Hesslow, G., 2007. Acquisition, extinction, and reacquisition of a cerebellar cortical memory trace. J. Neurosci. 27, 2493–2502. Jirenhed, D., Hesslow, G., 2011. Learning stimulus intervals–adaptive timing of conditioned purkinje cell responses. Cerebellum 10, 523–535. Kamin, L.J., 1969. Predictability, surprise, attention, and conditioning. In: Campbell, B.A., Church, R.M. (Eds.), Punishment and Aversive Behavior. Appleton-Century-Crofts and Fleschner Publishing Company, pp. 279–296. Kehoe, E.J., 1982. Overshadowing and summation in compound stimulus conditioning of the rabbit’s nictitating membrane response. J. Exp. Psychol. Anim. Behav. Process. 8, 313–328. Kehoe, E.J., 2006. Repeated acquisitions and extinctions in classical conditioning of the rabbit nictitating membrane response. Learn. Mem. 13, 366–375. Kehoe, E.J., Macrae, M., 2002. Fundamental behavioral methods and findings in classical conditioning. In: Moore, J.W. (Ed.), A Neuroscientist’s Guide to Classical Conditioning. Springer, London, pp. 171–231. Kehoe, E.J., White, N.E., 2002. Extinction revisited: similarities between extinction and reductions in US intensity in classical conditioning of the rabbit’s nictitating membrane response. Anim. Learn. Behav. 30, 96–111. Kehoe, E.J., White, N.E., 2004. Overexpectation: response loss during sustained stimulus compounding in the rabbit nictitating membrane preparation. Learn. Mem. 11, 476–483. Kim, J.J., Krupa, D.J., Thompson, R.F., 1998. Inhibitory cerebello-olivary projections and blocking effect in classical conditioning. Science 279, 570–573. Koziol, L.F., Budding, D.E., Chidekel, D., 2011. From movement to thought: executive function, embodied cognition, and the cerebellum. Cerebellum 11, 505–525. Lepora, N.F., Porrill, J., Yeo, C.H., Dean, P., 2010. Sensory prediction or motor control? Application of marr-albus type models of cerebellar function to classical conditioning. Front. Comput. Neurosci. 4, 140. Marr, D., 1969. A theory of cerebellar cortex. J. Physiol. 202, 437–470. Maruta, J., Hensbroek, R.A., Simpson, J.I.I., 2007. Intraburst and interburst signaling by climbing fibers. J. Neurosci. 27, 11263–11270. Mathy, A., Ho, S.S.N., Davie, J.T., Duguid, I.C., Clark, B.A., Ha¨usser, M., 2009. Encoding of oscillations by axonal bursts in inferior olive neurons. Neuron 62, 388–399. Miall, R.C., Keating, J.G., Malkmus, M., Thach, W.T., 1998. Simple spike activity predicts occurrence of complex spikes in cerebellar Purkinje cells. Nat. Neurosci. 1, 13–15.

References

Najafi, F., Medina, J.F., 2013. Beyond “all-or-nothing” climbing fibers: graded representation of teaching signals in Purkinje cells. Front. Neural Circuits 7, 115. Nelson, B., Mugnaini, E., 1989. Origins of GABA-ergic inputs to the inferior olive. In: Strata, P. (Ed.), The Olivocerebellar System in Motor Control. Springer Verlag, Berlin, pp. 90–102. Oscarsson, O., 1979. Functional units of the cerebellum-sagittal zones and microzones. Trends Neurosci. 2, 143–145. Rasmussen, A., Jirenhed, D., Hesslow, G., 2008. Simple and complex spike firing patterns in Purkinje cells during classical conditioning. Cerebellum 7, 563–566. Rasmussen, A., Jirenhed, D., Zucca, R., Johansson, F., Svensson, P., Hesslow, G., 2013. Number of spikes in climbing fibers determines the direction of cerebellar learning. J. Neurosci. 33, 13436–13440. Rescorla, R.A., Wagner, A.R., 1972. A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement. In: Black, A.H., Prokasy, W.F. (Eds.), Classical Conditioning II: Current Research and Theory. Appleton-Century-Crofts, New York. Schultz, W., 2006. Behavioral theories and the neurophysiology of reward. Annu. Rev. Psychol. 57, 87–115. Schweighofer, N., Lang, E.J., Kawato, M., 2013. Role of the olivo-cerebellar complex in motor learning and control. Front. Neural Circuits 7, 94. Sears, L.L., Steinmetz, J.E., 1991. Dorsal accessory inferior olive activity diminishes during acquisition of the rabbit classically conditioned eyelid response. Brain Res. 545, 114–122. Shadmehr, R., Smith, M.A., Krakauer, J.W., 2010. Error correction, sensory prediction, and adaptation in motor control. Annu. Rev. Neurosci. 33, 89–108. Svensson, P., Bengtsson, F., Hesslow, G., 2006. Cerebellar inhibition of inferior olivary transmission in the decerebrate ferret. Exp. Brain Res. 168, 241–253. Svensson, P., Jirenhed, D., Bengtsson, F., Hesslow, G., 2010. Effect of conditioned stimulus parameters on timing of conditioned Purkinje cell responses. J. Neurophysiol. 103, 1329–1336. Voogd, J., Glickstein, M., 1998. The anatomy of the cerebellum. Trends Cogn. Sci. 2, 307–313. Wolpert, D.M., Miall, R.C., Kawato, M., 1998. Internal models in the cerebellum. Trends Cogn. Sci. 2, 338–347. Yeo, C.H., Hardiman, M.J., Glickstein, M., 1985. Classical conditioning of the nictitating membrane response of the rabbit. III. Connections of cerebellar lobule HVI. Exp. Brain Res. 60, 114–126.

119

CHAPTER

Cerebellum-Dependent Motor Learning: Lessons from Adaptation of Eye Movements in Primates

6

Suryadeep Dash*, Peter Thier{,1 *

Robarts Research Institute, Western University, London, Ontario, Canada Department of Cognitive Neurology, Hertie Institute for Clinical Brain Research, University of Tuebingen, Tuebingen, Germany 1 Corresponding author: Tel.: þ49 7071 2983057; Fax: þ49 7071 295326, e-mail address: [email protected]

{

Abstract In order to ameliorate the consequences of ego motion for vision, human and nonhuman observers generate reflexive, compensatory eye movements based on visual as well as vestibular information, helping to stabilize the images of visual scenes on the retina despite ego motion. And in order to fully exploit the advantages of foveal vision, they make saccades to shift the image of an object onto the fovea and smooth pursuit eye movements to stabilize it there despite continuing object movement relative to the observer. With the exception of slow visually driven eye movements, which can be understood as manifestations of relatively straightforward feedback systems, most eye movements require a direct conversion of sensory input into appropriate motor responses in the absence of immediate sensory feedback. Hence, in order to generate appropriate oculomotor responses, the parameters linking input and output must be chosen suitably. Moreover, as the parameters may change from one manifestation of a movement to the next, for instance because of oculomotor fatigue, the choices should also be quickly modifiable. This chapter will present evidence showing that this fast parametric optimization, understood as a functionally distinct example of motor learning, is an accomplishment of specific parts of the cerebellum devoted to the control of eye movements. It will also discuss recent electrophysiological results suggesting how this specific form of motor learning may emerge from information processing in cerebellar circuits.

Keywords saccades, smooth pursuit, vestibular, vestibuloocular reflex, optokinetic reflex, fatigue, adaptation, fovea, retina, feedback

Progress in Brain Research, Volume 210, ISSN 0079-6123, http://dx.doi.org/10.1016/B978-0-444-63356-9.00006-6 © 2014 Elsevier B.V. All rights reserved.

121

122

CHAPTER 6 Cerebellum-Dependent Motor Learning

1 INTRODUCTION Skilled movements are acquired and improved by learning. Swimming, for example, is an acquired motor skill, which many of us learned in a pool. But, swimming in a pool vis-a`-vis in the ocean or a river is not the same and would require an adaptation of the movements learned in the pool to the constraints imposed by the new environments. An adaptation of motor programs is also demanded by the fact that our body does not stay the same across the life span but undergoes changes due to aging, diseases, or injury. To be able to swim successfully despite these changes, we have to adapt our movements to the changes of the body. Motor learning is an umbrella term for the processes that allow us to acquire new motor skills and to adjust our movements to changes of the physics of the body and the world. Swimming is an example of a complicated movement pattern that requires coordinated efforts of all four extremities and a large number of muscles and joints, continuously improved by learning. In other words, the optimization of complex motor behaviors involves simultaneous adjustments of many parameters, impeding attempts to unravel the underlying physiological principles. This is why research on the physiology of motor learning usually resorts to simpler movements, in which the number of parameters to be considered is comparatively small. Motor learning of simpler movements is often referred to as adaptation and is generally implicit in nature. In this chapter, we will focus on adaptation of different types of eye movements as examples of relatively simple movements characterized by fairly straightforward biomechanics and a small number of degrees of freedom, greatly facilitating the description of their kinematics and dynamics and the analysis of the interdependence of the two. Adaptations of eye movements are generally elicited by introducing a movement error based on an experimental manipulation that results in a mismatch between the desired movement and the actual eye movement. Eye movement adaptation leads to a gradual decrease of the motor error, making the realized movement more akin to the desired movement. Eye movement adaptation is severely impaired by cerebellar lesions, suggesting a key role of the cerebellum. We will try to shed light on the question how the cerebellum contributes to oculomotor learning by taking a closer look at the adaptation of three types of eye movements: 1. The adaptation of the vestibuloocular reflex (VOR) as an example of an important class of reflexive eye movements, generated in order to compensate the consequences of head movements. 2. The adaptation of goal-directed ballistic eye movement called saccades, made to shift the image of an object of interest into the fovea. 3. The adaptation of goal-directed tracking eye movements stabilizing the image on the fovea, referred to as smooth pursuit eye movements.

2 ADAPTATION OF THE VOR VORs are compensatory eye movements that stabilize the image of an object on the retina in the face of rotatory or translatory head movements. The eye moves reflexively in the direction needed to reduce image slip due to the head movement, driven

2 Adaptation of the VOR

by acceleration signals picked up by the vestibular organ (Angelaki, 2004; Crawford and Vilis, 1991). The fact that the driving force is a nonvisual signal explains why a VOR can be observed even in complete darkness. Yet, as the VOR serves vision, visual information is needed in order to optimize the VOR response or to cancel it in case of voluntary head-tracking movements. Visual input also matters in the case of low-velocity, low-frequency head movements, not or only transiently reported by the vestibular organs. In this case, a signal on the retinal movement of the visual background is used to generate a compensatory (optokinetic) eye movement. In other words, this optokinetic reflex, which we will not discuss further in this review (see Ilg, 1997; Masseck and Hoffmann, 2009 for more details), is a visual complement of the VOR, expanding the frequency range of gaze stabilization toward the lowfrequency side. How important the VOR is for vision is probably best documented by the experience of vestibular patients suffering from its loss: A healthy person can effortlessly read street signs while walking, thanks to a VOR that compensates walking-related head movements and the resulting retinal image motion. On the other hand, the patient will have to stop walking in order to read. This is the only way to avoid blurred vision. Adaptation of the VOR is the process that tries to eliminate differences between the head movement and the compensatory eye movement (Blazquez et al., 2004; Boyden et al., 2004). If the eye velocity is equal and opposite to the head velocity, then the VOR gain is said to be unity, leading to perfect image stabilization. In laboratory settings, the VOR gain can be adaptively increased or decreased by using either magnifying/minifying lenses, reversing prism, or by changing the relationship between the head movement and the image movement on a computer screen, causing a destabilization of the retinal background in the face of a head movement. For example, a normal VOR (gain ¼ 1) is elicited when the head movement is not accompanied by any image movement on the computer screen. If, however, the image on the screen moves in the direction of the head, the compensatory eye movement needed will be smaller. Upon many repetitions, the VOR is adaptively decreased to elicit eye movements with lower velocity (gain < 1), ideally able to keep the screen image stable on the retina. On the other hand, if the screen image moves in a direction opposite to that of the head movement, then an adaptive increase in eye velocity ensues (gain > 1). VOR adaptation can be divided into acute adaptation (adaptation in minutes to hours) and chronic adaptation (taking hours to days and weeks). Acute adaptation is typically induced by simple visuo-vestibular mismatch paradigms as described before. On the other hand, chronic adaptation is generally achieved by long-term application of magnifying or minifying lenses associated with vestibular signals generated by natural head movements. Retention of adaptation after chronic (long-term) adaptation is better and longer than after acute adaptation (Kuki et al., 2004). The decay of the adaptive gain change starts immediately and exponentially in the case of acute (short-term) adaptation. Moreover, irrespective of acute or chronic adaptation, gain-increase VOR adaptation shows a faster decay after elimination of the mismatch condition when compared to gain-decrease VOR adaptation (Kuki et al., 2004; Miles and Eighmy, 1980).

123

124

CHAPTER 6 Cerebellum-Dependent Motor Learning

3 SHORT-TERM SACCADIC ADAPTATION We continuously scan the environment with sequences of high-velocity fovealizing saccades and intervening periods of fixation, allowing the subject to scrutinize the fovealized object image. Saccades take only a few tens of milliseconds (ms). In other words, they are too fast to rely on visual feedback to decide when to stop the eye movement and to shape the trajectory to the target. Hence, rather than relying on sensory feedback, the control of saccades deploys internal feedback (Fuchs et al., 1985; Optican, 2005; Optican and Robinson, 1980). The retinal vector pointing to the location of the target of interest is converted into a saccade vector pointing to the desired eye position at the end of the saccade. An efference copy of the motor command is used to predict the actual eye position at significant latency and the saccadic drive reflects the difference between the desired and the predicted actual eye position. If the two are equal, the eyes will come to a stop. This “forward model” will generate saccades that will be accurate as long as the prediction of the actual eye position is accurate. If this is not the case, the saccade will be inaccurate, not able to shift the target image into the fovea, causing a visual error that could be used to adjust the prediction for future saccades. In other words, although visual feedback is too slow to contribute to the online control of saccades, it is indispensable for the post hoc evaluation of saccade performance and recalibration of the efference copy that may be needed. Note that this model is in principle also able to account for changes of saccadic behavior resulting from changes of the motor command. If for instance the motor command should be weakened, for example, because of muscle fatigue or eye muscle palsy due to disease, internal feedback warrants that the eyes will reach the target, albeit later, despite the drop in eye velocity. An inappropriate prediction of eye position for saccades can be easily simulated in the laboratory by an intrasaccadic target jump, moving the target away from the retinal location that caused the saccade in the first place (McLaughlin, 1967). The consequence will be a saccade that misses the target, prompting the need for subsequent corrective saccades. Due to saccadic suppression (Bremmer et al., 2009; Ibbotson and Cloherty, 2009), the visual system lacks knowledge of the intrasaccadic target jump causing the saccade error. Rather than assuming something as unlikely as a high-speed intrasaccadic target jump, the oculomotor system prefers the much more plausible assumption that the insufficient saccade may have been a consequence of resorting to a nonoptimal prediction of actual eye position. Hence, if the same error occurs repetitively because one saccade after the other is subject to stereotypical target jumps, the prediction is changed in a manner allowing the eyes to get to the target in one stroke, despite the intrasaccadic jump. This “short-term adaptation” of saccade metrics requires a few dozen trials in humans and up to several hundred trials in nonhuman primates (Hopp and Fuchs, 2004). In essence, shortterm saccadic adaptation (STSA) leads to a remapping of a constant retinal vector, pointing to a desired visual location, onto a new saccade vector. The most frequently studied variants of STSA are “inward” STSA (¼gain-decrease STSA) that is observed when the intrasaccadic target jump brings the target closer to the fixation point and manifests itself with a gradual decrease in saccade gain. In contrast, “outward”

3 Short-Term Saccadic Adaptation

STSA (¼gain-increase STSA) is induced when the intrasaccadic target jump further increases target eccentricity, leading to a gradual increase in saccade amplitude. A closer look at the kinematics of saccades resulting from these two forms of STSA suggests interesting differences (Fig. 1A). Gain-decrease saccades sport a decline in saccadic peak velocity that corresponds to the decline observed when subjects have to make long sequences of stereotypic saccades of a well-defined amplitude and direction (saccadic resilience experiment). In the case of saccades observed in this saccadic resilience experiment, the velocity decline is compensated by a fully compensatory upregulation of saccade duration (Golla et al., 2008;

A

B

Gain-increase SPA

600 Early

400

Late 200 0 0

20

20

40

80

60

Acceleration (°/s/s)

Velocity (°/s)

Gain-increase STSA 300 200

Late 100

Early 0 -100 200 0 100 Time from pursuit onset (ms)

Time from saccade onset (ms)

Gain-decrease SPA Acceleration (°/s/s)

Velocity (°/s)

Gain-decrease STSA 600

Mean ± std error Early

400 200 Late

0 20

0

20

40

60

200

Early

100 0 -100

Late

-200

80

-100

0

100

Saccadic resilience

600

Early 400 Late

200 0 20

0

20

40

60

Time from saccade onset (ms)

80

Acceleration (°/s/s)

Velocity (°/s)

SPEM resilience

200

Early

200 150

Late

100 50 0 -100 100 0 200 Time from pursuit onset (ms)

FIGURE 1 (A) Changes in the velocity profile of eye saccades as a result of gain-increase STSA (top), gain-decrease STSA (middle), and saccadic resilience (bottom). Data are based on average sessions recorded from two monkeys. Early and late profiles refer to the mean velocity profiles of the first and last few trials. (B) Changes in the acceleration profile of SPEMs during exemplary sessions of gain-increase SPA (top), gain-decrease SPA (middle), and SPEM resilience (bottom). The average eye acceleration during the first quarter (early) and the last quarter (late) of trials with standard error (gray shadow) is represented for gain-increase SPA, gain-decrease SPA, and SPEM resilience.

125

126

CHAPTER 6 Cerebellum-Dependent Motor Learning

Prsa et al., 2010). This is what would be expected based on the forward model of saccades sketched before. Gain-decrease saccades resulting from inward STSA show a similar decrease of peak velocity but lack a comparable adjustment of duration (Catz et al., 2008; Golla et al., 2008). In other words, they build on a natural tendency of saccades to lose velocity, a tendency that reflects changes taking place upstream of the oculomotor plant (Prsa et al., 2010), possibly due to cognitive fatigue (Chen-Harris et al., 2008; Fuchs and Binder, 1983; Straube et al., 1997). On the other hand, gainincrease saccades resulting from outward STSA show a considerable upregulation of saccade duration, overcompensating any changes in saccade amplitude due to changes in saccade velocity (Catz et al., 2008; Ethier et al., 2008; Golla et al., 2008) (Fig. 1A). The standard time course of the buildup of STSA may be accelerated by unconscious memory of previous adaptations, a phenomenon referred to as meta-adaptation (Kojima et al., 2004). Much stronger and longer-lasting saccadic adaptation was achieved by Robinson and colleagues (2006), who exposed monkeys to daily gaindecrease STSA sessions over altogether 19 days, while blindfolding the animals in between to avoid exposure to normal visual stimuli resetting the adapted state. The differences in the amount and the time course of adaptation in these experiments are the reason to distinguish them as long-term saccadic adaptation (LTSA) from standard STSA, two forms of adaptation that may depend on overlapping but not fully congruent functional mechanisms. Mueller and colleagues (2012) recently carried out corresponding experiments inducing long-term gain-increase saccadic adaptation (Mueller et al., 2012). They found that gain-increase LTSA builds up more slowly than gain-decrease LTSA, whereas the absolute gain change was larger (Mueller et al., 2012). Long-term changes in saccade amplitude are also observed in patients with unilateral abducens palsy (Abel et al., 1978; Kommerell et al., 1976) and monkeys with unilateral weakening of horizontal recti by tenectomy (Optican and Robinson, 1980). Due to the weakening, the affected side exhibits hypometric saccades. When the investigators of the aforementioned studies patched the healthy eye forcing the subject to see with the paretic eye, they observed a progressive increase in saccade amplitude in the weakened eye toward orthometric saccades in the course of 3 days, accompanied by the development of progressive hypermetria of the patched eye. The fact that the regression of hypometria of the paretic eye is inevitably linked with the development of hypermetria of the normal eye is a manifestation of Hering’s law of equal innervation (Hering, 1977) that assumes that conjugate eye movements are a consequence of the eyes sharing a common control signal. This signal can be adjusted to offer more drive to the paretic eye, yet with the consequence that the adjusted signal is too large for the normal eye, causing overshoot.

4 SMOOTH PURSUIT ADAPTATION Smooth pursuit eye movements (SPEMs) are tracking eye movements used to stabilize the image of a moving object of interest on the fovea. Simply put, SPEMs can be understood as the product of a feedback circuit that translates information

4 Smooth Pursuit Adaptation

on retinal target motion into an appropriate eye movement response, reducing retinal image slip (Rashbass, 1961; Robinson et al., 1986). However, the first 100–150 ms of the SPEMs are driven by uncompensated retinal target image motion due to the long latencies of visual information processing. As a consequence of the sluggishness of vision, the eye movement response evoked by the moving target starts only 100– 150 ms after target motion onset (SPEM latency). In other words, the 100–150 ms of SPEMs following the onset of the eye movement are an open-loop response (SPEM initiation) whose size depends solely on the visual target motion signal and a gain parameter that specifies the transformation of the target movement into a pursuit command. How is the gain parameter chosen? The study of smooth pursuit adaptation (SPA) (see below) suggests that the expected eye movement gain governing the early closed-loop behavior is used as a reference for the open-loop gain. This seems reasonable as the probability that the movement of a natural pursuit target will substantially change in this brief period is low. As a consequence, there is a good chance that already the initial SPEM has the right velocity, thereby reducing the need for corrective saccades that would otherwise jeopardize the continuous scrutiny of the moving target. SPA refers to the short-term changes in the gain of SPEM initiation brought about by an experimental manipulation that causes a violation of the aforementioned goal to minimize the pursuit error at the time closed-loop behavior kicks in. This is achieved by exposing the observer to a sequence of trials in which the target moves at an initial constant velocity for around 100–200 ms and then steps to a new predictable velocity, stereotypically at the same point in time. The pursuit velocity evoked by the initial target velocity is changed such as to make it more similar to the target velocity after the velocity step, thereby minimizing the retinal errors prevailing at the time the loop is closed (Dash et al., 2010; Fukushima et al., 1996; Kahlon and Lisberger, 1996). If the target steps to a higher velocity, subjects learn to upregulate the pursuit gain evoked by the initial target velocity (gain-increase SPA). Correspondingly, if the target velocity steps to a lower velocity following the initial target ramp, subjects gradually learn to downregulate their initial pursuit gain (gaindecrease SPA). Similar to STSA, also SPA reflects changes in timing. The major difference between the two is that SPA is based on the control of eye acceleration rather than eye velocity as in the case of STSA (Fig. 1B). Specifically, during gain-decrease SPA, velocity decreases due to a decrease in peak acceleration not compensated by an increase in the duration of the initial eye acceleration pulse (Dash and Thier, 2013). On the other hand, during gain-increase SPA the acceleration profile expands (i.e., the eyes are accelerated for a longer time) while peak acceleration may increase, decrease, or stay unchanged (Dash and Thier, 2013). In other words, the kinematic changes associated with gain-increase SPA and gain-decrease SPA are not mirror symmetric, similar to the asymmetry characterizing gain-increase and gain-decrease STSA. Yet another parallel holds for the effects of fatigue. If rhesus monkeys are asked to carry out long sequences of stereotypical step-ramp smooth pursuit eye movements (Dash and Thier, 2013), they are able to maintain a constant SPEM peak velocity despite constantly declining SPEM peak acceleration. The decline in peak

127

128

CHAPTER 6 Cerebellum-Dependent Motor Learning

acceleration is compensated by an expansion of the acceleration profile (i.e., an increase in acceleration duration). These changes are analogous to the compensation of the decline in peak eye velocity by increasing movement duration in the case of a saccade resilience experiment described earlier. The decrease in peak acceleration that is observed during gain-decrease SPA may be taken as a manifestation of fatigue. On the other hand, the ability to expand the acceleration pulse in order to realize gain-increase SPA is the same one that is used to compensate SPEM fatigue (Dash and Thier, 2013) (Fig. 1B).

5 OCULOMOTOR CEREBELLUM—AN OVERVIEW An intact cerebellum is indispensable for the eye movement adaptations described before, ensuring optimal VOR responses for gaze stabilization and optimal gaze shifts to stationary and moving objects of interest (saccades and SPEMs). Many regions of the primate cerebellum are involved in eye movements (see Thier, 2011 for a more detailed overview; Fig. 2). The major cerebellar substrates of gaze stabilization and its context-dependent manipulation are the flocculus and the adjoining ventral paraflocculus (together occasionally referred to as floccular complex (FC)), located in the ventrolateral cerebellum. The flocculus proper is composed of four to five distinct lobuli and is located posterior to the ventral paraflocculus (VPF) which comprises another six lobuli. Actually, the FC does not seem to be confined to controlling gaze holding but to make a contribution to SPEMs as well, a dual role that may reflect the need to coordinate slow eye movements for gaze holding and slow eye movements subserving smooth pursuit (Kahlon and Lisberger, 2000; Medina and Lisberger, 2008, 2009; Rambold et al., 2002; Zee et al., 1981). Also the dorsal paraflocculus (DPF), located dorsal to the FC, does not seem to be confined to one type of eye movement. The evidence available suggests contributions to the control of saccades and SPEMs (Noda and Mikami, 1986). The so-called oculomotor vermis (OMV) is a distinct representation of both saccades and smooth pursuit well separated from the flocculus and the paraflocculus, located in the dorsal midline cerebellum, consisting of large parts of vermal lobuli VI and VII. The more caudal parts of the midline cerebellum comprising vermal lobuli IX and X (the nodulus and uvula, respectively) are adjoining the FC, a topographical proximity which is paralleled by functional proximity. Like the FC, also the caudal vermis contributes foremost to the processing of vestibular and associated optokinetic information. Having access to signals from the otolith organs as well as the semicircular canals, the caudal vermis helps to disambiguate ambiguous vestibular experiences. A case in point is the need to distinguish acceleration signals provided by the otolith organs due to changes in head orientation relative to gravity from those reflecting linear acceleration of the observer, a disambiguation that relies on additional information provided by the semicircular canals (Angelaki and Hess, 1994). Another example of a specific contribution of the caudal vermis is the need to reinterpret a persistent semicircular canal signal in the case of an intervening

5 Oculomotor Cerebellum—An Overview

Rostral IIIa IIIb

Oculomotor vermis

IVa IVb IVb

Dorsal view Va Vb

?

Left

?

Vb VIb

?

?

VIa

VIc VIIAa VIIAa

?

?

VIIAb VIIAc VIII

Hemispheric oculomotor region

VII

?

?

Posterior view

VIII ?

?

IX

?

?

Nodulus Dorsal paraflocculus

Lobulus petrosus

Flocculus Nodulus

Uvula

IX

Ventral view

X

us

cul

Dorsal paraflocculus

c Flo

Lobulus petrosus

Ventral paraflocculus Rostral FIGURE 2

Illustration of eye movement-related areas in the primate cerebellum. Views of the cerebellum modified from Madigan and Carpenter (1971). Adapted from Thier (2011).

129

130

CHAPTER 6 Cerebellum-Dependent Motor Learning

reorientation of the head relative to gravity (Angelaki et al., 2004; Waespe et al., 1985). The final and arguably least understood cerebellar region implicated in the control of eye movements is a large and ill-defined region in the hemispheres, adjoining vermal lobule VII and loosely referred to as HVII for hemispheric (region) VII. HVII seems to contribute to saccades as well as to SPEMs (Ohki et al., 2009). Yet, it remains to be clarified what the specific contributions of hemispheric region VII to these two types of goal-directed eye movements are and if and how they differ from those of the well-studied eye movement-related regions in the vermis and the posterolateral cerebellum. As pertinent information on contributions to oculomotor learning is limited to the FC and the OMV, we will concentrate on these structures in the remainder of this review.

6 FLOCCULAR COMPLEX 6.1 Anatomical Considerations Although the flocculus proper and the VPF clearly differ both in terms of their phylogenetic age as well as their connections, many of the physiological studies, in particular those using awake behaving monkeys, have treated the two as a supposedly functionally homogenous region, referred to as the so-called floccular complex (FC) (summarized in Lisberger, 2009). However, only the flocculus proper receives the majority of its mossy fiber input from vestibular ganglion, floccular projection neurons in the vestibular nuclei (Langer et al., 1985b; Nagao et al., 1997a), and only the flocculus projects in turn exclusively to floccular target neurons in the vestibular nuclei (Langer et al., 1985a; Nagao et al., 1997b). The VPF, on the other hand, seems much more dependent on input from the pontine nuclei (Nagao et al., 1997a). The older anatomical work (Brodal, 1982; Langer et al., 1985b) has often been taken to support the notion of a significant projection from the pontine nuclei to the flocculus, the major brainstem recipient of input from cerebrocortical input. Yet, a recent study, addressing differences in inputs to flocculus and the VPF, found almost no pontine nuclei input to the flocculus in the face of significant pontine nuclei input to the VPF (Nagao et al., 1997a). On the other hand, the above-mentioned studies agree on a significant bilateral input to both components of the FC from the nucleus reticularis tegmenti pontis (NRTP). The output of the flocculus and VPF also differ significantly with the flocculus primarily projecting to the vestibular nuclei while the VPF projecting to parts of the interpositus and dentate nucleus in addition to the vestibular nuclei (Nagao et al., 1997b).

6.2 Role of FC in VOR and VOR Adaptation The VOR is mediated by a three-neuron circuit with a primary vestibular afferent from the vestibular labyrinth impinging on a relay neuron in the vestibular nuclei which in turn controls a motor neuron activating the eye muscle (LorentedeNo, 1933). Based on stimulation experiments, Fukuda and colleagues (1972) established

6 Floccular Complex

an additional influence on relay neurons in the vestibular nuclei, namely a direct inhibitory input originating from the rabbit flocculus (Fukuda et al., 1972). Subsequent lesion studies suggested that this pathway from the flocculus is critical for VOR adaptation in rabbits, cats, and monkeys (Ito et al., 1974; Lisberger et al., 1984; Robinson, 1976; Zee et al., 1981). The aforementioned studies and many more during 1970–1980 led Masao Ito to propose the “flocculus hypothesis” which asserted the flocculus to be the site of VOR adaptation (Ito, 1982). In rabbits, flocculus Purkinje cells (PCs) discharge in-phase as well as 180 out of phase relative to head velocity during visual suppression of a VOR induced by sinusoidal head rotation (Ghelarducci et al., 1975). Dufosse´ and colleagues (1978) showed that differential modulation of these in-phase and out-of-phase floccular PCs could depress or enhance the VOR gain, respectively, in rabbits. The early recording studies in monkeys did not differentiate between the two components of the FC. Although the units reported were usually presented as floccular units, many of them may actually have been recorded from the VPF, which is why we will discuss them as FC units, unless a study to be discussed presented unmistakable anatomical information. Consistent with earlier rabbit studies, Watanabe (1985) identified a zone (the H (horizontal) zone) in the FC, with PCs whose simple spike (SS) discharge, arguably the signal passed on target neurons in the vestibular nuclei, was modulated in conjunction with the horizontal VOR. When these units were tested during acute VOR adaptation (both gain-increase and gain-decrease), they exhibited clear changes in the strength of their modulation. Considering that the PC axons, transporting the SS, are inhibitory (Watanabe, 1985), the direction of the changes seemed to be compatible with the notion that the PV SS signal mediates behavioral adaptation by controlling target neuron discharge. In an earlier study, Miles and colleagues (1980) had reached a different conclusion based on recordings from so-called gaze velocity PC SS units (¼GVP) in the FC, activated by gaze velocity (the sum of eye and head velocity; see the following section on SPA for further details) to the left or the right. Although the discharge of the units studied showed changes in conjunction with chronic VOR adaptation, the direction of the changes was actually opposite to the one to be expected if an adapted PC SS signal, modulating the discharge of vestibular nuclei relay neurons, were responsible for the adapted behavior. Rather, the observations seemed to suggest that the PC responses were secondary reflections of adaptation-related neuronal changes occurring downstream in the brainstem (Miles and Lisberger, 1981). Also Lisberger and colleagues (1994) reported that chronic horizontal VOR adaptation modulated the sensitivity of FC GVP for head velocity in a direction, not able to account for the needs of vestibular nuclei neurons driving the adapted VOR behavior. This conclusion was based on measurement of head velocity sensitivity in a VOR cancellation paradigm. On the other hand, when the same units were studied during VOR in darkness, the discharge modulation was actually found to be changed by adaptation in a direction compatible with the idea that the PC is the major site of learning (Lisberger et al., 1994). Nevertheless, the authors concluded that the site of learning was downstream in the brainstem based on the finding that the response of these PCs to a 300-ms pulse of head

131

132

CHAPTER 6 Cerebellum-Dependent Motor Learning

velocity was too late for a causal relationship. Consideration on the site of learning is further complicated by a recent study by Raymond and Lisberger (1997) that reported that FC SS responses are appropriate for inducing acute VOR adaptation but only at low stimulus frequencies. Also the work on vertical VOR adaptation (both chronic and acute and both gain-increase as well as gain-decrease) does not allow a clear conclusion on the site of learning: it has shown adaptation-related changes in both the firing of PC SS as well as Y-group neurons (a component of the vestibular nuclei complex devoted to the vertical, targeted by FC) (Blazquez et al., 2003; Hirata and Highstein, 2001; Partsalis et al., 1995a). Blazquez and colleagues (2003) demonstrated that following chronic VOR adaptation, PC SS change their eye velocity and eye position sensitivity in addition to head velocity sensitivity. The net change was direction appropriate to guide the adapted behavior (previous studies had assumed that only head velocity sensitivity changes, leading to the conclusion of incorrect direction to modify behavior). However, FC removal after chronic vertical VOR adaptation leads to a partial loss of learned VOR changes, suggesting that after chronic VOR adaptation, information on the adapted VOR is also stored downstream of the FC (Partsalis et al, 1995b). The bottom line is that the various recording studies trying to unravel the neuronal mechanisms underlying VOR adaptation have not been able to delineate a simple picture. The picture emerging seems to be one of multiple sites of learning, at the level of PC as well as downstream of PC. Although not ruling out sites of learning serving VOR adaptation outside the FC, the lesion work clearly supports the notion of an important role of the FC as lesions of the FC lead to a complete inability in learning new VOR gains (Rambold et al., 2002; Zee et al., 1981). The study by Rambold and coworkers, moreover, suggests that the critical part of the FC is the VPF rather than the flocculus proper.

6.3 Role of FC in SPEMs and SPA Nagao (1992) studied the differences in responses of PC SS of flocculus and VPF during SPEMs. He found that PC SS of VPF strongly modulated with target velocity and position while floccular PC SS showed much smaller modulation. He also showed head velocity sensitivity in floccular PC SS during VOR vis-a-vis no head sensitivity in VPF PC SS (Nagao, 1992). In view of the differences in connections and clear differences in the physiological properties of the two parts of the FC, Nagao (1992) concluded that the flocculus and the VPF subserve distinct roles. The differences suggest a role of the flocculus in controlling the VOR as discussed earlier, whereas the VPF might be more closely associated with SPEMs. Other studies of SPEMs and SPA in monkeys have rarely tried to differentiate between the two components of the FC and usually ascribed the observations obtained to the FC at large. Electrical stimulation of the monkey FC is able to elicit slow eye movements at a short latency of only 10 ms, in terms of their kinematics reminiscent of the slow component of vestibular reflexes as well as SPEMs (Lisberger et al., 1994). More specific information on a role of the FC in SPEMs

6 Floccular Complex

comes from single-unit recording studies, which demonstrated pursuit-related FC PC SS responses (Buttner and Waespe, 1984; Lisberger and Fuchs, 1978; Miles and Fuller, 1975; Noda and Suzuki, 1979). There are two classes of FC PC implicated in SPEMs: GVP and eye velocity PC (EVP). GVP exhibits SS responses that have been suggested to reflect gaze velocity, that is, the sum of eye and head velocity in a world-centered frame of reference. Their responses seemed to be best explained by assuming that they sum eye velocity relative to head and head velocity in the world in order to represent gaze velocity in a world-centered frame of reference: this assumption can account for the fact that these neurons are activated by smooth pursuit with the head immobile as well as by VOR cancellation or visuo-vestibular conflict stimuli. During VOR cancellation, a normal VOR response to passive head movement is suppressed by asking subjects to fixate a head stationary target (Miles and Fuller, 1975). During visuo-vestibular conflict stimulation, the VOR is suppressed by rotating the optokinetic drum with the vestibular turntable (Buttner and Waespe, 1984). GVPs show no or little response to head movements that evoke a compensatory VOR stabilizing the eyes in space. These neurons were generally active during SPEMs in downward and ipsilateral directions (Krauzlis and Lisberger, 1994; Stone and Lisberger, 1990). A true GVP should have identical sensitivities to eye and head velocity. This is what Lisberger and Fuchs (1978) indeed reported in a first study of this type of neurons. Yet, a more recent experiment on squirrel monkeys (Belton and McCrea, 2000) is at odd with the assumption of equal sensitivities. Rather, these authors reported that the head velocity sensitivity of the FC recorded was actually only half of their eye velocity sensitivity. Also Fukushima and colleagues (1999) did not find much support for the existence of GVP in the strict sense in a selected group of vertically tuned FC PC SS. The second class of SPEM-related PC that has been distinguished are PCs whose SS discharge is modulated by eye velocity relative to the head. These EVPs respond during SPEMs with the head as well as during passive head movements, causing a vestibularly driven eye movement relative to the head (Lisberger and Fuchs, 1978; Stone and Lisberger, 1990). EVPs do not respond during active head movements (Belton and McCrea, 2000). This holds for smooth headtracking movements as well as for head movements, which are part of saccadic gaze shifts. Saccadic gaze shifts are based on initial targeting eye and head shifts, which are followed by a continuation of the head movement in the direction of the initial shifts, accompanied by a vestibularly mediated rollback of the eyes in the opposite direction. Inactivation experiments based on muscimol injections into the FC carried out by Belton and McCrea found deficits in combined eye and head pursuit (Belton and McCrea, 1999). However, these deficits were limited to the eye movement component of the tracking behavior. Following inactivation, the monkeys could still follow visual targets with pure head movements and cancel the VOR during active head movements, suggesting that the FC may not be involved in active gaze movements. Bilateral lesions of FC and particularly the VPF lead to deficits in the SPEM behavior (Rambold et al., 2002; Zee et al., 1981). In the study conducted by Rambold and colleagues, across five monkeys, two monkeys with bilateral lesions largely

133

134

CHAPTER 6 Cerebellum-Dependent Motor Learning

confined to the flocculus proper did not show SPEM deficits, two monkeys exhibited mild SPEM deficits following unilateral lesions of the FC that included the VPF, and one monkey exhibited severe deficit after bilateral lesion of both flocculus and VPF. FC PC SS also show adaptive changes in their firing rate during SPA (Kahlon and Lisberger, 2000; Medina and Lisberger, 2009). Kahlon and Lisberger (2000) demonstrated changes of the discharge rates in a small sample (n ¼ 10) of FC PC SS during either gain-increase or gain-decrease SPA in their preferred direction (Kahlon and Lisberger, 2000). Recently, Medina and Lisberger dealt with changes of the discharge patterns of FC PC SS during the learned addition of a directional component 250 ms after target onset (Medina and Lisberger, 2008, 2009). Assuming a normal pursuit latency of 100–150 ms, it is safe to infer that the directional adaptation in their case took place not during pursuit initiation, that is, in the open-loop period but later. Since the adaptive changes in behavior happened after the open-loop period, when feedback about the eye movement was already available, adaptation in this case was different from feed-forward predictive motor adaptation in the strict sense. They also reported changed velocity sensitivity when a directional component was adaptively added in the PC’s preferred direction (on direction). Do the changes in the FC PC SS during SPA reflect a causal link? Unfortunately, an answer to this question is not available as there has been no lesion or inactivation experiment to causally link FC in SPA. Following FC lesions, significant SPEM was preserved, even in experiments in which the lesions had encroached on the DPF. As cerebellectomy is known to completely disrupt SPEMs (Westheimer and Blair, 1973), the minor effects of FC/DPF lesions suggest additional representations of SPEMs in other parts of cerebellar cortex. A major representation is the OMV discussed below whose role in SPEMs and SPA is well supported by highly consistent observations of single-unit responses and the behavioral consequences of lesions.

7 OCULOMOTOR VERMIS 7.1 Anatomical Considerations The OMV receives its primary mossy fiber input from the dorsal pontine nuclei (DPN) and the NRTP, which in turn receive input from a number of cortical and subcortical areas involved in visual motion processing and the planning of eye movements (Thielert and Thier, 1993; Thier and Mock, 2006; Yamada and Noda, 1987). The OMV also receives modest mossy fiber inputs from the vestibular complex, the nucleus prepositus hypoglossi, and paramedian pontine reticular formation (PPRF) (Thielert and Thier, 1993; Yamada and Noda, 1987). The climbing fiber (CF) input to the OMV, responsible for the generation of PC complex spikes (CS), arises from subnucleus B of the medial accessory nucleus of the inferior olive (Yamada and Noda, 1987). The OMV projects exclusively to the caudal region of the fastigial nucleus (FOR). The FOR, which contains both saccade- and SPEM-related neurons,

7 Oculomotor Vermis

receives its afferences from the same areas that also target the OMV and in turn projects to the contralateral PPRF, the dorsomedial reticular formation, the rostral interstitial nucleus of the medial longitudinal fasciculus, bilaterally to inferior and lateral vestibular nuclei, to the midline pontine raphe nuclei (Noda et al., 1990), and last but not least heavily to the more rostral parts of the superior colliculus (SC) (May et al., 1990; Noda et al., 1990). Both the DPN and the NRTP contribute to saccades as well as to SPEMs, deploying oculomotor neurons that vary regarding the degree of sensitivity for either type of visually guided eye movements (Crandall and Keller, 1985; Dicke et al., 2004; Suzuki et al., 1999; Thier and Mock, 2006; Thier et al., 1988; Tziridis et al., 2009). In view of this input, it is not surprising that also the OMV sports saccadeas well as pursuit-related PCs—the only output neuron of cerebellar cortex. Actually, our recent work suggests that practically all OMV PCs respond to saccades as well as to SPEMs (Smilgin et al., 2012).

7.2 Role of the OMV in Saccades, STSA, and Saccadic Resilience The earliest suggestion that the posterior vermis and neighboring paravermal regions might be involved in saccades came from lesion and electrical stimulation experiments carried out in the 1970s. Relatively large surgical lesions centering on the posterior vermis of the monkey cerebellum were reported to cause dysmetric saccades (Aschoff and Cohen, 1971; Ritchie, 1976). At about the same time, Ron and Robinson employed electrical stimulation of cerebellar cortex in an attempt to evoke eye movements. Using relatively high current strengths (around 1 mA), they could delineate a region involving the posterior vermis and neighboring paravermal lobules, whose stimulation gave rise to saccades (Ron and Robinson, 1973). This quite extensive saccade representation could be narrowed considerably in later microstimulation experiments by Noda and coworkers. These experiments delineated a low current (i.e., 10 mA) representation of microstimulation-evoked saccades limited to vermal lobuli 6c and 7A, a representation which is usually referred to as the OMV (Fujikado and Noda, 1987; Noda and Fujikado, 1987a,b). Consistent with the stimulation results, the OMV exhibits a high density of saccade-related responses. OMV PCs show changes of their SS discharge rate around the execution of a saccade and do not respond to the visual presentation of the target. Both saccade-related burst PCs as well as pause PCs may be observed (Catz et al., 2008; Ohtsuka and Noda, 1995). In many cases, the responses are directionally selective. Two studies followed the behavior of individual PC SS during STSA (Catz et al., 2008; Kojima et al., 2010). Catz and colleagues (2008) recorded PC SS from a large sample of neurons (128 PC SS during gain-increase STSA and 84 during gaindecrease STSA) during the course of STSA, leading to stable behavioral adaptation. Individual PC SS showed adaptation-related increases in discharge rate during gainincrease STSA (52% of PC SS) and decreases in discharge rates during gain-decrease STSA (64% of PC SS) (Fig. 3A). As adaptation leads to the adoption of a new saccade amplitude, one might expect changes in the discharge rates, simply as a

135

A2

0.79

0.77

0.74

0.71

CS

0.91

0.87

50 ms

30 spikes/s

50 ms

100 ms 0.93

1 spike/s

1 spike/s 50 spikes/s

SS

0.99

1.08

1.06

1.17

1.19

1.23

1.25

1.06

0.84

1.11

375 300

B1

5 10 15 Amplitude (°)

160 140 120 100

spikes/s

Course of adaptation (trial no. #) Course of adaptation (trial no. #)

180

100

80

50

60 0

-100 0 100 Time relative to pursuit onset (ms) 100

150

80 100

60 40

50 20 0

-100 0 100 Time relative to pursuit onset (ms)

70 40

100 5 10 15 Amplitude (°)

B2

Gain-decrease SPA

150

200

10 14 18 Amplitude (°)

Gain-increase SPA Course of adaptation (trial no. #)

10 15 Amplitude (°)

250

Before learning During learning 100

Course of adaptation (trial no. #)

60

1.18 SS peak discharges (spikes/s)

100

spikes/s

SS peak discharges (spikes/s)

0.80 Before learning During learning 450

1.02

100 150

80 60

100 40 20 0

50 -100 0 100 Time relative to pursuit onset (ms)

120

120 100

100

80

80

60

60

40

40

20

20

0

spikes/s

0.85

PC 6

PC 5 0.93

spikes/s

0.84

0.97

Course of adaptation

CS

2 spikes/s 30 spikes/s

50 ms

100 ms

Gain-increase STSA PC 4

PC 3 0.98

2 spike/s

50 ms

PC 2 150 spikes/s

PC 1 Gain = 0.96 SS

Course of adaptation

2 spike/s

100 spikes/s

Gain-decrease STSA

1 spike/s 40 spikes/s

A1

-100 0 100 Time relative to pursuit onset (ms)

Before learning After learning

FIGURE 3 Changes in PC response during STSA and SPA: A1 shows three examples of PC SS and corresponding CS recorded during gain-decrease STSA. A2 shows three examples of PC SS and corresponding CS recorded during gain-increase STSA. For each column, the peristimulus histogram (PSTH) represents the mean saccade-related activity of the cell at

7 Oculomotor Vermis

consequence of shifting to a different position on the amplitude tuning curve of a unit. Yet this simple explanation did not apply as adaptation led to a clear deviation from the prediction based on the preadaptation tuning functions. Many PC SS were recorded simultaneously with concomitant CS, the latter usually showing a trend of changes opposite to that of the SS discharge rate. Kojima and colleagues (2010) also reported adaptation-related changes in OMV PC SS (42% of 61 PC SS), although only during gain-decrease STSA. The kinematics of normal saccades is only poorly reflected by the responses of individual PC SS (Thier et al., 2000, 2002). Also, the aforementioned studies reporting changes at the level of individual PC during STSA found the associations between SS firing of single PCs and changes of behavior to be modest at best (Catz et al., 2008; Kojima et al., 2010). In other words, an observer trying to describe the metric of an unadapted or adapted saccade, relying on individual PC SS, would most certainly fail. However, other than our hypothetical observer, target neurons in the FOR, the major recipient of input from saccade-related PCs, do not depend on an SS input from an individual PC. The reason is that there is a high degree of anatomical convergence of individual PC axons on neurons in the DCN (Palkovits et al., 1977). The functional consequence of the anatomical convergence is that the collective SS activity of the PCs converging on an individual DCN neuron should provide a much better description. Actually, a very precise description of saccade kinematics is provided by the collective activity of a larger group of PC SS. The duration of the population response during orthometric saccades shows an almost perfect correlation with saccade duration, which in turn determines saccade amplitude (Thier et al., 2000, 2002). Moreover, the end of the SS PC population burst coincides precisely with the end of the saccade (Thier et al., 2000, 2002). Catz and colleagues (2008) studied the changes of the OMV PC SS population burst during gain-increase STSA. They could show that the tight relationship between movement end and burst end was maintained (Fig. 4A). As described further below, gain-increase STSA is based on increasing movement duration and correspondingly, movement end is shifted to later points in time. This shift of movement end was accompanied by a corresponding shift of the end of the population burst. Does the tight correlation between discharge end and the end of the saccade actually reflect a causal relation? The answer to this question is in the affirmative as the loss of PCs and others elements of the OMV circuitry

different times during STSA from the beginning (topmost) to the end (bottommost) of adaptation. The number given on top of each PSTH indicates the mean saccade gain for the group of trials underlying the PSTH. For PCs 1, 2, 4, and 5, the bottom row depicts the amplitude tuning of a neuron before (blue curve) and during (red curve) adaptation. B1 shows two exemplary PC SS during the course of gain-decrease SPA. B2 shows two exemplary PC SS during the course of gain-increase SPA. B1 and B2 show raster plots with superimposed spike density functions for the first (black) and the last (grey) quarter of the trials. The solid vertical line indicates SPEM onset. The course of SPA is from bottom to top. (A1, A2) adapted from Catz et al. (2008) and (B1, B2) from Dash et al. (2013).

137

CHAPTER 6 Cerebellum-Dependent Motor Learning

A1

Control: before STSA

A4

70

60

50

100 80 60 40 20

Saccade duration (ms)

-50 0 50 Time from saccade onset (ms)

Saccade duration (ms)

A2

Mean rate (spike/s)

Saccade duration (ms)

n = 212 cells N = 8226 saccades

A3 70

Gain-decrease STSA

Gain-increase STSA

n = 84 cells 55 N =7611 saccades

n = 128 cells N = 13228 saccades

60 45 50

-50

0

-50

50

0

50

A5 After gain-increase adaptation 70 n = 128 cells N = 17682

After gain-decrease adaptation n = 84 cells 55 N = 13354

60 45 50

-50 0 50 Time from saccade onset (ms)

B1

-50 0 50 Time from saccade onset (ms)

B2 1.2

Gain-increase SPA

1.2

spk(t-15ms) = e* pos(t) + f * vel(t) + g * acc(t) + h

1.0

1.0

0.8

0.8 CD

CD

138

0.6

0.6 0.4

0.4 Median 25%–75% Min–Max

0.2 0.0

Gain-decrease SPA spk(t-15ms) = e* pos(t) + f * vel(t) + g * acc(t) + h

bin1 bin2 bin3 bin4

Median 25%–75% Min–Max

0.2 0.0

bin1 bin2 bin3 bin4

FIGURE 4 (A1–A5) Effect of STSA on saccade-related PC SS population burst (PB). A1 shows the PB before the onset STSA. The PB is plotted as a function of saccade time (x-axis) and duration (y-axis), measured in discrete time steps (x-axis: 1 ms, y-axis: 2.5 ms). The PB is the instantaneous firing of a large group of PC SS recorded individually but subjected to the same

7 Oculomotor Vermis

due to OMV pathology eliminates the ability to control saccade duration needed in order to select the saccade amplitude required. Barash and colleagues (1999) carried out bilateral surgical lesions of the OMV and studied the postlesion effects for an extended period of time. They reported saccadic hypometria, that is, saccades whose amplitudes were too short, early after the lesion. This saccadic hypometria became gradually smaller in the course of the first weeks after the lesion until after 3 months to 1 year when saccades were, on average, on target again. Yet, a profound increase of end point variability, observed early on, persisted even after complete recovery of saccadic hypometria (Barash et al., 1999; Ignashchenkova et al., 2009; Takagi et al., 1998). Patients having midline lesions of the cerebellum encompassing the OMV show similar deficits (Golla et al., 2008). A complete lesion of the OMV also led to irreversible loss of the ability to adapt saccade amplitude on a short time scale toward larger amplitudes (gain-increase STSA; Barash et al., 1999; Fig. 5A). Takagi and colleagues (1998) studied gain-decrease STSA and reported some late recovery that could probably be accounted for by the incomplete nature of the lesions made. Recent reports have demonstrated that patients suffering from cerebellar pathology involving the OMV likewise lose the capacity for gain-increase STSA (Golla et al., 2008; Xu-Wilson et al., 2009). Normal human subjects achieve the increase of saccade amplitude required by the gainincrease STSA paradigm by virtue of increasing saccade duration. This ability to increase saccade durations is lost in patients with vermal pathology (Golla et al., 2008), which is why they are unable to increase saccade amplitude. On the other hand, these patients may still sport a certain amount of gain-decrease STSA, that is, they remain able to reduce their saccade amplitude over time. Gain-decrease STSA in these patients is accompanied by a decrease in saccade peak velocity and unaltered duration. As saccade duration does not change, a reduced saccade amplitude must result. A similar decrease in peak saccade velocity is also seen in an experiment of saccadic paradigm. The data underlying the PB for a time roughly midway during gain-increase STSA (A2) and gain-decrease STSA (A3), respectively, were collected when the gain change achieved amounted to 5–10%. A4 shows the PB after stable gain-increase STSA had been achieved and A5 the PB after stable gain-decrease STSA (gain change >15%). Black dashed, thin white, and thick white lines depict, respectively, saccade onset, saccade offset, and PB end. Size of the PC SS sample underlying the PB: 128 for gain-increase STSA and 84 for gain-decrease STSA. (B1, B2) Depict changes in the coefficient of determination (CD ¼ r2; for the equation in the figure obtained after correlating the discharge profiles of individual PC SS and population response with a linear combination of eye position (pos), velocity (vel), and acceleration (acc); e, f, and g are the position, velocity, and acceleration coefficient for the given equation at time t; respectively) of individual PC SS and the population response during gain-increase SPA (B1) and gain-decrease SPA (B2). The distribution of CD for individual PC SS is shown in median/quartile values of CD throughout the course of SPA divided up into four bins for the entire population of PC SS. The CD obtained for instantaneous population response is added as stars. (A1–A5) adapted from Catz et al. (2008) and (B1, B2) from Dash et al. (2013).

139

A2

A1 Monkey 2

Monkey 1

6 1

2

3

4

Monkey 2

Monkey 1

5

Before lesion B

A

19

19

15° Target

14

14 0

350

0

1500

After recovery from dysmetria D + 367 days + 84 days

C

20

Saccade size (deg)

19

Fix

9

250 ms

10 0

1500

0

1500

Saccade number

B (°)

Increasing paradigm (40–80°/s)

Prelesion

Postlesion

10

10 Control

Control 0

0

-10

-10

-20

-20 0

0.2

0.4

0.6

0

0.8

0.2

0.4

0.6

0.8

10

10

Early in adaptation

Early in adaptation 0

0

-10

-10

-20

-20 0

0.2

0.4

0.6

0

0.8

0.2

0.4

0.6

0.8

10

10 Late in adaptation

0

Late in adaptation

0

-10

-10

-20

-20 (s) 0

0.2

0.4

0.6

0.8

0

0.2

0.4

0.6

0.8

FIGURE 5 Consequences of lesions of the OMV for saccades and smooth pursuit eye movements. (A1) Exemplary visually guided saccades made before (1), early (2), and late (3) after an OMV lesion. Note that the hypometria, obvious in the first days and weeks after the lesion (2), completely disappears after a few months (3). However, an increased variability of saccade end points remains. (A2) Plots of saccade amplitude as function of trial number in a typical

7 Oculomotor Vermis

resilience in which a long sequence of stereotypic saccades with short intervals between subsequent saccades is carried out. Healthy subjects compensate this drop in peak velocity, taken as the signature of fatigue, by upregulating saccade duration in a fully compensatory manner, thereby keeping saccade amplitude constant. Vermal patients lack this compensatory upregulation of saccade duration (Golla et al., 2008). A similar inability to keep saccade amplitude stable in a saccade resilience experiment was also observed by Barash et al. (1999) in one of their two lesioned monkeys. In sum, the inability of OMV-lesioned monkeys and the vermal patients to adjust saccade duration clearly supports the view that the end of the PC SS population response determines the end of normal saccades as well as saccades resulting from gain-increase adaptation, thereby ensuring the precision of saccade end point. The prediction that the population burst duration should get longer in the case of fatigue compensation has not been tested yet. However, Catz and colleagues (2008) took a look at the population PC SS behavior for gain-decrease STSA (Fig. 4A). Surprisingly, it turned out that in this particular case, the relationship between saccade end and the end of the population burst was broken. Whereas the movement of the adapted saccades ended at a similar time after adaptation, both the start as well as the end of the population burst moved to considerably earlier points in time. In an attempt to explain this unexpected result, the authors argued that the gain-decrease saccades observed may basically have been default saccades whose duration was fully determined by the brainstem saccade generator without significant modification by cerebellar influences. To release a saccade whose duration is not extended by the gain-increase STSA experiment before and late after the lesion. Gain-increase STSA was achieved by resorting to the classical McLaughlin paradigm (McLaughlin, 1967): The monkeys were presented a visual target at 15 eccentricity which was shifted to 20 during the saccade. Initially, the eyes landed at 15 and a second, corrective saccade had to be added in order to bring the eyes to the final position of the target. However, in the course of the experiment, which involved many stereotypic repetitions of the target jump, the subjects learned to upregulate the amplitude of the first saccade, thereby bringing the eyes closer to the final position of the target and alleviating the need for a second saccade. Whereas the two monkeys showed the expected gain-increase adaptation before the lesion, it was completely lost—probably permanently—after the lesion. (B) SPA before and after OMV lesions. In this particular experiment, initial pursuit velocity was adapted by doubling target velocity at the time the target reached the straight-ahead position. The dashed line describes the target position. Early after the activation of the velocity doubling, the eyes were unable to keep up with the target as the visual delay prevented an immediate reaction and the eye velocity was still determined by the lower target velocity seen at 100 ms earlier. However, in the course of the adaptation experiment, pursuit got better. This was the consequence of the fact that the gain of the conversion of the low target velocity before the velocity step was upregulated. This adaptation was largely lost after the lesion and in general monkeys exhibited a significant decline in eye velocity postlesion. (A1, A2): from Barash et al. (1999) and (B): Takagi et al. (2000).

141

142

CHAPTER 6 Cerebellum-Dependent Motor Learning

cerebellum, the PC population burst should end well before a critical period in which the occurrence of PC input would keep a saccade going beyond the duration set by the brainstem.

7.3 Role of the OMV in SPEMs, SPA, and SPEM Resilience Initially, the OMV was mainly appreciated as a representation of saccades as the early lesion and stimulation work did not seem to suggest a role in the second type of goal-directed eye movements, SPEMs. Electrical stimulation in and around the OMV conducted by Ron and Robinson (1973) did not evoke any slow eye movements and Ritchie (1976) had not reported deficits in SPEMs following vermal lesions. However, already as early as 1979, the first evidence for a role of the OMV in SPEMs had become available as Kase and coworkers reported pursuit-related PC SS in the OMV, interpreted as units encoding the movement of the target guiding the eyes in a world-centered reference frame (Kase et al., 1979). How come that the early microstimulation studies reported only evoked saccades? Actually, as shown by Krauzlis and Miles (1998), this is a consequence of choosing a particular behavioral context, namely fixation of stationary targets or internally driven exploratory saccades. If, on the other hand, the experimental monkey is engaged in SPEMs, stimulating the OMV causes a profound modulation of pursuit velocity, a context dependency of the stimulation effect that is in full accordance with a role in the control of saccades as well as SPEMs (Krauzlis and Miles, 1998). In fact, Suzuki and Keller (1988a) reported that many OMV PC SS can be driven by both saccades and SPEMs (Suzuki and Keller, 1988a). As mentioned earlier, recent work indicates that OMV PC SS confined to only one of the two types of goal-directed eye movements are actually rare exceptions (Smilgin et al., 2012). Many of the recording studies trying to unravel the role of the OMV in SPEMs have used sinusoidally moving targets, useful for the analysis of steady-state SPEMs but arguably of little use in studies of SPEM initiation (Shinmei et al., 2002; Suzuki and Keller, 1988a,b). The aforementioned studies found OMV PC SS to respond to eye velocity, head velocity, and even to the motion of the visual target. In a recent study, Dash and colleagues (2012) used pursuit targets moving according to a stepramp profile to study the responses of OMV PCs and found that the SS responses of SPEM-related PCs were characterized by a formidable diversity of profiles, including SPEM-related phasic bursts, phasic pauses, burst–tonic responses, tonic responses, tonic pause responses, and also bursts followed by pauses. More than 60% of OMV PCs showed phasic burst responses to SPEM onset and almost a third of the PCs exhibited pause responses. Typically, these OMV PC SS were directionally tuned and were much more sensitive to eye velocity than to other kinematic parameters (Dash et al., 2012). A multiple linear regression analysis with a consideration of the three kinematic parameters, eye position, velocity, and acceleration could only vaguely predict the discharge of the individual OMV PC with velocity being the most important kinematic parameter. The diversity of pursuit responses and, correspondingly, the

7 Oculomotor Vermis

relatively unreliable prediction of kinematics provided by individual PC SS are analogous to the poor description of saccades by individual PC SS. Again, similar to saccades, also in the case of SPEMs, the PC population discharge does much better. In the case of SPEMs, it provides a perfect description of the kinematics of the eye movement with eye velocity having a much larger weight than eye position or acceleration (Dash et al., 2012). In a recent study of SPA, the same authors demonstrated that the PC SS population response is also able to account for the kinematic changes resulting from adaptation (Fig. 4B). When they followed the behavior of 163 OMV PC SS during either gain-increase SPA (n ¼ 104) or gain-decrease SPA (n ¼ 59), they found that SS firing increased in 67% of the units during gain-increase SPA and decreased in 68% of the units during gain-decrease SPA (Dash et al., 2013). More specifically, both PC SS showing excitatory pursuit-related activity (burst, burst–tonic, or tonic responses) as well as those with inhibitory pursuit-related activity (tonic or phasic pauses) exhibited increased discharge rates during gain-increase SPA. On the other hand, only excitatory PC SS showed a decrease in their firing rates during gain-decrease SPA (Fig. 3B). As in the case of unadapted SPEMs, also in the case of adapted SPEMs the SS population activity provided a much better description of eye movement kinematics than individual units with eye velocity being the key kinematic variable (Fig. 4B). While eye velocity sensitivity increased during the course of gain-increase SPA, no change was observed for PC SS tested during gain-decrease SPA. These results suggest a functional difference between gain-increase and gain-decrease SPA, analogous to the difference between the two corresponding forms of STSA. Gain-increase adaptation can be taken as the consequence of a specific adjustment of a PC SS population signal, which, in the case of SPA, causes an increased velocity drive. On the other hand, gain-decrease adaptation must be the consequence of changes taking place upstream of OMV, an adjustment that is not further modified by the PC code. As in the case of STSA, the lesion work is in general supportive of a causal link between the discharge of OMV PC SS and SPEMs, although not every aspect of the findings fit. Surgical lesions of the monkey OMV cause a profound impairment of the openloop part of SPEMs evoked by a linear target movement (Takagi et al., 2000). More specifically, the lesions carried out in this study led to a decrease in the gain of the early, still open-loop phase of SPEMs due to a drop in the peak acceleration. There was no influence on the SPEM latency and only a modest effect on the steady-state SPEMs. The magnitude of the SPEM deficit across monkeys was correlated with the size of the saccade deficits observed. Lesions of the OMV also disrupted the ability to parametrically adjust the gain of the SPEMs in the initial open-loop period. Both gain-increase and gain-decrease SPA were impaired after OMV lesions (Takagi et al., 2000; Fig. 5B), the latter not necessarily to be expected in view of the OMV PC SS data discussed in the preceding paragraph. In a recent experiment, we (Dash and Thier, 2013) studied the consequences of an OMV lesion in a SPEM resilience experiment in which healthy monkeys and one lesioned monkey were confronted with a long series of pursuit trials offering highly stereotypic

143

144

CHAPTER 6 Cerebellum-Dependent Motor Learning

step-ramp target movements. Unlike the control case, the monkey with a lesion of the OMV exhibited a continuous drop of the gain of SPEM initiation. This drop in peak SPEM velocity could be attributed to the inability to compensate the fatigue-induced drop in peak acceleration by an expansion of the acceleration duration. This inability to upregulate the duration of acceleration profile to compensate the gradual drop in peak acceleration was permanently abolished, arguably because of the loss of a PC SS population SS signal offering a suitably adjusted description of the kinematics. Unfortunately, pertinent single-unit data allowing us to challenge this speculation are not available yet.

8 COMPLEX SPIKE ACTIVITY DURING STSA AND SPA CS of PCs, induced by CF afferents originating from the inferior olive, have been playing a key role in prevailing concepts of cerebellum-based motor learning (Albus, 1971; Marr, 1969). As first suggested by Marr (1969), CF activity is assumed to reflect an error signal, capturing the deviation of the realized motor behavior from the planned behavior used to adjust future manifestations of the same behavior. The classical theories posit that this error signal modulates the strengths of parallel fiber synapses on PCs, which in turn depend on mossy fiber input to the cerebellum, thought to reflect the behavior (Marr, 1969). If the CF indeed conveyed an error signal to PCs, then the discharge modulation should be expected to be maximal at the time the deficiency is maximal and disappear once the behavior is optimal, showing a monotonic relationship between CS modulation and changes in the size of the performance error. However, recent studies of OMV PC CS firing during STSA as well as SPA are at odds with this simple expectation (Catz et al., 2005; Dash et al., 2010). In contrast to the expected monotonic decline between CS and performance error, CS modulation actually was found to increase in parallel with the progress of STSA/ SPA, reaching its maximum at the end of adaptation, that is, at a time, the performance error had become minimal and at times even zero. For both STSA and SPA, learned gain decreases were accompanied by the buildup of an increased probability of CS occurrence at the time of movement initiation (Fig. 6). Conversely, learned gain increases were accompanied by a buildup of a decreased probability of CS occurrence (Fig. 6). Observations, likewise suggesting that the CS firing might be strongly influenced by the adjusted behavior, were also reported by Kahlon and Lisberger (2000). They studied the behavior of a small sample of FC CS units during SPA and found a learning-related modulation for gain-decrease SPA that appeared toward the end of the learning session. However, they did not investigate if this increased CS probability showed any gradual change in the course of SPA. They did not find any CS modulation during gain-increase SPA. On the other hand, a clear indication of error encoding by FC CS was reported by Medina and Lisberger (2008), who studied CS units during the directional adaptation of SPEMs. They found a CS modulation when a directional component was added in the direction that was opposite (¼off-direction) to the preferred direction characterizing the SS

A

Gain-increase STSA n=98 neurons Adapted

Adaptation

No adaptation -250

0

250

Gain-decrease STSA n=74 neurons Adapted

Mean rate (spike/s)

0.4

0.2

0.05 Adaptation

No adaptation -250

250

0

Time from saccade offset (ms)

B

Gain-increase SPA

Gain-decrease SPA

RS (°/s)

RS (°/s)

C

Spikes/s

Spikes/s

C

Co

Co

FIGURE 6 (A) Changes in the average firing rate of a population of CS units during the course of gainincrease STSA (top; 98 CS units) and gain-decrease STSA (bottom, 74 CS units). (B) The top row shows the mean modulation of retinal slip (RS) in the first 100 ms after SPEM onset as (Continued)

146

CHAPTER 6 Cerebellum-Dependent Motor Learning

discharge of the same PC SS. There was a strong change in the probability of CS firing in a 100-ms window and 100 ms after the direction component was added in the off-direction. This modulation slowly decreased, paralleling the increase in directional adaptation. No CS modulation was observed during adaptation in the on-direction. Hence, this latter study seems in line with the idea that the CS reflects an oculomotor error signal. On the other hand, the fact that OMV CS modulation in the other studies was strongest at the end of the adaptation period seems to suggest a role in stabilizing the adjusted behavior. In the aforementioned studies, consistent error information was used to drive the behavior toward a new state, reflecting the annihilation of the error. This is the major difference between these studies and a study by Soetedjo and colleagues (2006) who recorded OMV PC CS responses evoked by simultaneous adaptation in both horizontal directions based on an error, pointing in the same constant direction, independent of saccade direction (e.g., gainincrease STSA for leftward saccades and gain-decrease STSA for rightward saccades) (Soetedjo and Fuchs, 2006). As the direction of error was switched after blocks of only 200–300 trials (i.e., on average 100–150 trials for each saccade direction), it seems unlikely that stable behavioral adaptation could have developed. Considering a group of 27 CS units that were saccade related at the onset of the experiment, the authors reported significant changes of the CS rate for particular error directions. These changes appeared late, during the time from primary saccade offset to the beginning of the corrective saccade and were interpreted as manifestations of a retinal error driving behavioral corrections. Obviously, this conclusion differs from the assumed role in stabilizing adjusted goal-directed eye movements (STSA: Catz et al., 2005; SPA: Dash et al., 2010 and Kahlon and Lisberger, 2000). Recent work by Junker and colleagues (2013) on single trial STSA may reconcile the seemingly incompatible views on the role of the CS in oculomotor adaptation: these authors tested monkeys on center-out saccades in various directions. On a fraction of the trials, the target was stepped further out or back toward the fixation point in an unpredictable manner. The visual error in trial n  1 had a clear consequence for the amplitude of the saccade carried out in trial n. The amplitude changed in a manner that would have reduced the error if trial n would have experienced the same target shift as trial n  1. The authors could moreover show that the firing of OMV PC CS occurring before the saccade in trial n showed a subtle influence of the error

FIGURE 6—Cont’d function of the course of gain-increase (left) and gain-decrease SPA (right), respectively, pooled across all the sessions of adaptation. The overall number of trials making up an individual adaptation session, which varied between 100 and 250, was divided up into five bins. The bottom row shows the corresponding modulation in mean CS firing. The CS modulation measure reflects the population average of 44 and 51 CS units recorded. CS firing decreases during the course of gain-increase SPA and increases during gain-decrease SPA. (A) adapted from Catz et al. (2005) and (B) from Dash et al. (2010).

8 Complex Spike Activity During STSA and SPA

prevailing on trial n  1. In other words, the CS offers a memory trace of an error in order to shape future behavior. How could this memory trace be realized? The speculative answer may be the same one as the answer to the question, how the much stronger CS modulation in studies in which consistent error information leads to more persistent behavioral modification (Catz et al., 2005; Dash et al., 2010) can be explained? We suggest that both may result from integration of CF activity based on signal feedback from cerebellar cortex to the inferior olive, the latter the source of CFs. In this scenario, feedback from inferior olive neurons is assumed to be influenced by an error signal, arguably derived from the SC (Frankfurter et al., 1976; Harting, 1977; Kaku et al., 2009; Soetedjo et al., 2009). Positive feedback would keep the impact of the error alive for the upcoming trial and if consistent over many trials actually increase its impact on the CF activity. We earlier suggested that the kinematics of unadapted as well as adapted goaldirected eye movements are determined by an SS population response that undergoes appropriate changes. But how are these changes be brought about and is there any role of the CS? In the case of gain-increase STSA, according to Catz et al. (2005), the probability of observing CS decreases around the time of the saccade (Fig. 6A). Conversely, in the case of gain-decrease STSA, leading to smaller amplitude saccades, the probability increases. Maximal long-term depression (LTD) at parallel fiber synapses is observed when a CS occurs within 200 ms after SS (Wang et al., 2000). Hence, the CS “burst” during gain-decrease STSA found by Catz and colleagues (2005) to peak at 23 ms relative to saccade end might induce LTD if the saccade-related SS burst appeared between 220 and 20 ms relative to saccade offset. SS, which would normally have appeared in this period, would be suppressed. Consequently, the SS population signal would stop earlier. On the other hand, the suppression of CS during gain-increase STSA, maximal at 10 ms, might reduce LTD if the SS burst emerged between 210 and 10 ms, unleashing SS in this period and thereby extending the SS population signal. In other words, relative to nonadapted saccades, we would expect to see longer-lasting saccade-related SS activity in the case of gainincrease STSA, but shorter-lasting saccade-related SS activity for gain-decrease STSA. This is exactly what was found in the study of SS activity during STSA (Catz et al., 2008). Hence, specific changes of the CS discharge pattern may be responsible for behaviorally relevant changes in the SS population signal. In other words, we suggest that the PC SS population response develops the temporal structure needed based on CS facilitating the SS responses of individual PC whose timing is appropriate and, conversely, by suppressing those whose timing is inappropriate. Of course, for this mechanism to work requires that the spectrum of occurrences of individual SS responses covers the full temporal range to be covered by the population signal. As SS responses depend on their parallel fiber input, which in turn reflects the mossy fiber input, the expectation is that the temporal dispersion of saccade-related mossy fiber activity is at least as wide as the temporal extent of the SS PC signal. Actually, this expectation is surpassed as saccade-related mossy fiber signals cover a significantly wider range of times relative to the saccade.

147

148

CHAPTER 6 Cerebellum-Dependent Motor Learning

The same holds for the major interneuron, processing mossy fiber/parallel fiber input to P-cells, the Golgi cell. Finally, saccade-related Golgi-cell responses do not change as a consequence of saccadic adaptation (Prsa et al., 2009). This finding is important as it clearly indicates that learning-related changes do not take place before the level of the P-cell (see Fig. 7 for a conceptual model of the role of the OMV during STSA).

Parallel fibers

P-cell

Time

Granule cells

Mossy fibers

Time

Climbing fibers

Time Time Saccade Saccade onset end

Inferior olive

Cerebellar nuclei

Saccade Saccade onset end

FIGURE 7 A conceptual model of the role of the OMV during STSA. (1) Saccade-related responses in the mossy fiber/parallel fiber input to the OMV cover a wide temporal range relative to saccade onset and end. (2) The CF input, responsible for the generation of CS, selects appropriate input by vetoing parallel fiber input in a specific period of time relative to the occurrence of a CS, arguably based on heterosynaptic long-term depression of parallel fiber synapses. (3) The axons of a larger group of PC axons converge on individual target neurons in the caudal fastigial nucleus. Accordingly, the target neuron response depends on the collective SS input (¼the SS population burst, PB) from this group of PCs (assuming that the rare CS do not matter at the level of the deep cerebellar nuclei). (4) The end of the PB releases a nFOR “stop” signal that terminates the saccadic eye movement. (5) Changing the duration and in general the shape of the PB is a consequence of shifting the occurrence of individual CS relative to the saccade depending on information on the behavioral needs reaching the inferior olive, the source of CF. Adapted from Prsa et al. (2009).

References

9 CONCLUSIONS Work on the cerebellar control of eye movements and their adaption has been able to delineate several distinct and largely segregated parts of the cerebellum with specific contributions to specific types of eye movements and oculomotor learning. A posterolateral complex with the flocculus, the paraflocculus, the nodulus, and uvula is devoted to the control of gaze-holding reflexes and their learning-based adjustment and a midline region; the ocular motor vermis is the major substrate of the two forms of goal-directed eye movements and their adaptation. The specific role of a large and poorly delineated hemispheric representation of eye movements remains to be explored. Arguably, it is the work on STSA which has suggested the most stimulating recent vista on the cerebellar basis of oculomotor learning. In a nutshell, it sees PCs as temporal filters, choosing appropriate signal contributions from mossy fiber input spread out in time. This temporal filter is tuned by CF input, ensuring that the activity collective activity of groups of PCs impinging on individual target neurons leads to the right behavioral consequences.

References Abel, L.A., Schmidt, D., Dell’Osso, L.F., Daroff, R.B., 1978. Saccadic system plasticity in humans. Ann. Neurol. 4, 313–318. Albus, J.S., 1971. A theory of cerebellar function. Math. Biosci. 10, 25–61. Angelaki, D.E., 2004. Eyes on target: what neurons must do for the vestibuloocular reflex during linear motion. J. Neurophysiol. 92, 20–35. Angelaki, D.E., Hess, B.J., 1994. The cerebellar nodulus and ventral uvula control the torsional vestibulo-ocular reflex. J. Neurophysiol. 72, 1443–1447. Angelaki, D.E., Shaikh, A.G., Green, A.M., Dickman, J.D., 2004. Neurons compute internal models of the physical laws of motion. Nature 430 (6999), 560–564. Aschoff, J.C., Cohen, B., 1971. Changes in saccadic eye movements produced by cerebellar cortical lesions. Exp. Neurol. 32, 123–133. Barash, S., Melikyan, A., Sivakov, A., Zhang, M., Glickstein, M., Thier, P., 1999. Saccadic dysmetria and adaptation after lesions of the cerebellar cortex. J. Neurosci. 19, 10931–10939. Belton, T., McCrea, R.A., 1999. Contribution of the cerebellar flocculus to gaze control during active head movements. J. Neurophysiol. 81, 3105–3109. Belton, T., McCrea, R.A., 2000. Role of the cerebellar flocculus region in the coordination of eye and head movements during gaze pursuit. J. Neurophysiol. 84, 1614–1626. Blazquez, P.M., Hirata, Y., Heiney, S.A., Green, A.M., Highstein, S.M., 2003. Cerebellar signatures of vestibulo-ocular reflex motor learning. J. Neurosci. 23 (30), 9742–9751. Blazquez, P.M., Hirata, Y., Highstein, S.M., 2004. The vestibulo-ocular reflex as a model system for motor learning: what is the role of the cerebellum? Cerebellum 3, 188–192. Boyden, E.S., Katoh, A., Raymond, J.L., 2004. Cerebellum-dependent learning: the role of multiple plasticity mechanisms. Annu. Rev. Neurosci. 27, 581–609.

149

150

CHAPTER 6 Cerebellum-Dependent Motor Learning

Bremmer, F., Kubischik, M., Hoffmann, K.P., Krekelberg, B., 2009. Neural dynamics of saccadic suppression. J. Neurosci. 29, 12374–12383. Brodal, P., 1982. Further observations on the cerebellar projections from the pontine nuclei and the nucleus reticularis tegmenti pontis in the rhesus monkey. J. Comp. Neurol. 204, 44–55. Buttner, U., Waespe, W., 1984. Purkinje cell activity in the primate flocculus during optokinetic stimulation, smooth pursuit eye movements and VOR-suppression. Exp. Brain Res. 55, 97–104. Catz, N., Dicke, P.W., Thier, P., 2005. Cerebellar complex spike firing is suitable to induce as well as to stabilize motor learning. Curr. Biol. 15, 2179–2189. Catz, N., Dicke, P.W., Thier, P., 2008. Cerebellar-dependent motor learning is based on pruning a Purkinje cell population response. Proc. Natl. Acad. Sci. U.S.A. 105, 7309–7314. Chen-Harris, H., Joiner, W.M., Ethier, V., Zee, D.S., Shadmehr, R., 2008. Adaptive control of saccades via internal feedback. J. Neurosci. 28, 2804–2813. Crandall, W.F., Keller, E.L., 1985. Visual and oculomotor signals in nucleus reticularis tegmenti pontis in alert monkey. J. Neurophysiol. 54, 1326–1345. Crawford, J.D., Vilis, T., 1991. Axes of eye rotation and Listing’s law during rotations of the head. J. Neurophysiol. 65, 407–423. Dash, S., Thier, P., 2013. Smooth pursuit adaptation (SPA) exhibits features useful to compensate changes in the properties of the smooth pursuit eye movement system due to usage. Front. Syst. Neurosci. 7, 67. Dash, S., Catz, N., Dicke, P.W., Thier, P., 2010. Specific vermal complex spike responses build up during the course of smooth-pursuit adaptation, paralleling the decrease of performance error. Exp. Brain Res. 205, 41–55. Dash, S., Catz, N., Dicke, P.W., Thier, P., 2012. Encoding of smooth-pursuit eye movement initiation by a population of vermal Purkinje cells. Cereb. Cortex 22, 877–891. Dash, S., Dicke, P.W., Thier, P., 2013. A vermal Purkinje cell simple spike population response encodes the changes in eye movement kinematics due to smooth pursuit adaptation. Front. Syst. Neurosci. 7, 3. Dicke, P.W., Barash, S., Ilg, U.J., Thier, P., 2004. Single-neuron evidence for a contribution of the dorsal pontine nuclei to both types of target-directed eye movements, saccades and smooth-pursuit. Eur. J. Neurosci. 19, 609–624. Dufosse´, M., Ito, M., Jastreboff, P.J., Miyashita, Y., 1978. A neuronal correlate in rabbit’s cerebellum to adaptive modification of the vestibulo-ocular reflex. Brain Res. 150 (3), 611–616. Ethier, V., Zee, D.S., Shadmehr, R., 2008. Changes in control of saccades during gain adaptation. J. Neurosci. 28, 13929–13937. Frankfurter, A., Weber, J.T., Royce, G.J., Strominger, N.L., Harting, J.K., 1976. An autoradiographic analysis of the tecto-olivary projection in primates. Brain Res. 118, 245–257. Fuchs, A.F., Binder, M.D., 1983. Fatigue resistance of human extraocular muscles. J. Neurophysiol. 49, 28–34. Fuchs, A.F., Kaneko, C.R., Scudder, C.A., 1985. Brainstem control of saccadic eye movements. Annu. Rev. Neurosci. 8, 307–337. Fujikado, T., Noda, H., 1987. Saccadic eye movements evoked by microstimulation of lobule VII of the cerebellar vermis of macaque monkeys. J. Physiol. 394, 573–594. Fukuda, J., Highstein, S.M., Ito, M., 1972. Cerebellar inhibitory control of the vestibulo-ocular reflex investigated in rabbit 3rd nucleus. Exp. Brain Res. 14 (5), 511–526.

References

Fukushima, K., Tanaka, M., Suzuki, Y., Fukushima, J., Yoshida, T., 1996. Adaptive changes in human smooth pursuit eye movement. Neurosci. Res. 25, 391–398. Fukushima, K., Fukushima, J., Kaneko, C.R., Fuchs, A.F., 1999. Vertical Purkinje cells of the monkey floccular lobe: simple-spike activity during pursuit and passive whole body rotation. J. Neurophysiol. 82, 787–803. Ghelarducci, B., Ito, M., Yagi, N., 1975. Impulse discharges from flocculus Purkinje cells of alert rabbits during visual stimulation combined with horizontal head rotation. Brain Res. 87 (1), 66–72. Golla, H., Tziridis, K., Haarmeier, T., Catz, N., Barash, S., Thier, P., 2008. Reduced saccadic resilience and impaired saccadic adaptation due to cerebellar disease. Eur. J. Neurosci. 27, 132–144. Harting, J.K., 1977. Descending pathways from the superior collicullus: an autoradiographic analysis in the rhesus monkey (Macaca mulatta). J. Comp. Neurol. 173, 583–612. Hering, E., 1977. The theory of binocular vision. Plenum Press, New York. Hirata, Y., Highstein, S.M., 2001. Acute adaptation of the vestibuloocular reflex: signal processing by floccular and ventral parafloccular Purkinje cells. J. Neurophysiol. 85 (5), 2267–2288. Hopp, J.J., Fuchs, A.F., 2004. The characteristics and neuronal substrate of saccadic eye movement plasticity. Prog. Neurobiol. 72 (1), 27–53. Ibbotson, M.R., Cloherty, S.L., 2009. Visual perception: saccadic omission—suppression or temporal masking? Curr. Biol. 19, R493–R496. Ignashchenkova, A., Dash, S., Dicke, P.W., Haarmeier, T., Glickstein, M., Thier, P., 2009. Normal spatial attention but impaired saccades and visual motion perception after lesions of the monkey cerebellum. J. Neurophysiol. 102, 3156–3168. Ilg, U.J., 1997. Slow eye movements. Prog. Neurobiol. 53 (3), 293–329. Ito, M., 1982. Cerebellar control of the vestibulo-ocular reflex—around the flocculus hypothesis. Annu. Rev. Neurosci. 5, 275–296. Ito, M., Shiida, T., Yagi, N., Yamamoto, M., 1974. Visual influence on rabbit horizontal vestibulo-ocular reflex presumably effected via the cerebellar flocculus. Brain Res. 65 (1), 170–174. Junker, M.A., Endres, D., Dicke, P.W., Thier, P., 2013. A signal from the past? Modulation in climbing fiber activity during trial-by-trial adaptation in the oculomotor vermis of rhesus monkeys. Program No. 469.12. Neuroscience Meeting Planner, Society for Neuroscience, New Orleans, LA. Kahlon, M., Lisberger, S.G., 1996. Coordinate system for learning in the smooth pursuit eye movements of monkeys. J. Neurosci. 16, 7270–7283. Kahlon, M., Lisberger, S.G., 2000. Changes in the responses of Purkinje cells in the floccular complex of monkeys after motor learning in smooth pursuit eye movements. J. Neurophysiol. 84, 2945–2960. Kaku, Y., Yoshida, K., Iwamoto, Y., 2009. Learning signals from the superior colliculus for adaptation of saccadic eye movements in the monkey. J. Neurosci. 29, 5266–5275. Kase, M., Noda, H., Suzuki, D.A., Miller, D.C., 1979. Target velocity signals of visual tracking in vermal Purkinje cells of the monkey. Science 205, 717–720. Kojima, Y., Iwamoto, Y., Yoshida, K., 2004. Memory of learning facilitates saccadic adaptation in the monkey. J. Neurosci. 24, 7531–7539.

151

152

CHAPTER 6 Cerebellum-Dependent Motor Learning

Kojima, Y., Soetedjo, R., Fuchs, A.F., 2010. Changes in simple spike activity of some Purkinje cells in the oculomotor vermis during saccade adaptation are appropriate to participate in motor learning. J. Neurosci. 30, 3715–3727. Kommerell, G., Olivier, D., Theopold, H., 1976. Adaptive programming of phasic and tonic components in saccadic eye movements. Investigations of patients with abducens palsy. Invest. Ophthalmol. 15, 657–660. Krauzlis, R.J., Lisberger, S.G., 1994. Simple spike responses of gaze velocity Purkinje cells in the floccular lobe of the monkey during the onset and offset of pursuit eye movements. J. Neurophysiol. 72, 2045–2050. Krauzlis, R.J., Miles, F.A., 1998. Role of the oculomotor vermis in generating pursuit and saccades: effects of microstimulation. J. Neurophysiol. 80, 2046–2062. Kuki, Y., Hirata, Y., Blazquez, P.M., Heiney, S.A., Highstein, S.M., 2004. Memory retention of vestibuloocular reflex motor learning in squirrel monkeys. Neuroreport 15, 1007–1011. Langer, T., Fuchs, A.F., Chubb, M.C., Scudder, C.A., Lisberger, S.G., 1985a. Floccular efferents in the rhesus macaque as revealed by autoradiography and horseradish peroxidase. J. Comp. Neurol. 235, 26–37. Langer, T., Fuchs, A.F., Scudder, C.A., Chubb, M.C., 1985b. Afferents to the flocculus of the cerebellum in the rhesus macaque as revealed by retrograde transport of horseradish peroxidase. J. Comp. Neurol. 235, 1–25. Lisberger, S.G., 2009. Internal models of eye movement in the floccular complex of the monkey cerebellum. Neuroscience 162, 763–776. Lisberger, S.G., Fuchs, A.F., 1978. Role of primate flocculus during rapid behavioral modification of vestibuloocular reflex. I. Purkinje cell activity during visually guided horizontal smooth-pursuit eye movements and passive head rotation. J. Neurophysiol. 41, 733–763. Lisberger, S.G., Miles, F.A., Zee, D.S., 1984. Signals used to compute errors in monkey vestibuloocular reflex: possible role of flocculus. J. Neurophysiol. 52 (6), 1140–1153. Lisberger, S.G., Pavelko, T.A., Bronte-Stewart, H.M., Stone, L.S., 1994. Neural basis for motor learning in the vestibuloocular reflex of primates. II. Changes in the responses of horizontal gaze velocity Purkinje cells in the cerebellar flocculus and ventral paraflocculus. J. Neurophysiol. 72 (2), 954–973. LorentedeNo, R., 1933. Vestibulo-ocular reflex arc. Arch. Neurol. Psychiatr. (Chicago) 30, 245–291. Madigan, J.C., Carpenter, M.B., 1971. Cerebellum of the rhesus monkey: Atlas of lobules, laminae, and folia, in sections. University Park Press, Baltimore. Marr, D., 1969. A theory of cerebellar cortex. J. Physiol. 202, 437–470. Masseck, O.A., Hoffmann, K.P., 2009. Comparative neurobiology of the optokinetic reflex. Ann. N.Y. Acad. Sci. 1164, 430–439. May, P.J., Hartwich-Young, R., Nelson, J., Sparks, D.L., Porter, J.D., 1990. Cerebellotectal pathways in the macaque: implications for collicular generation of saccades. Neuroscience 36 (2), 305–324. McLaughlin, S., 1967. Parametric adjustment in saccadic eye movements. Percept. Psychophys. 2 (8), 358–362. Medina, J.F., Lisberger, S.G., 2008. Links from complex spikes to local plasticity and motor learning in the cerebellum of awake-behaving monkeys. Nat. Neurosci. 11, 1185–1192. Medina, J.F., Lisberger, S.G., 2009. Encoding and decoding of learned smooth-pursuit eye movements in the floccular complex of the monkey cerebellum. J. Neurophysiol. 102, 2039–2054.

References

Miles, F.A., Eighmy, B.B., 1980. Long-term adaptive changes in primate vestibular ocular reflex. I. Behavioral observations. J. Neurophysiol. 43, 1406–1425. Miles, F.A., Fuller, J.H., 1975. Visual tracking and the primate flocculus. Science 189, 1000–1002. Miles, F.A., Lisberger, S.G., 1981. Plasticity in the vestibulo-ocular reflex: a new hypothesis. Annu. Rev. Neurosci. 4, 273–299. Miles, F.A., Braitman, D.J., Dow, B.M., 1980. Long-term adaptive changes in primate vestibuloocular reflex. IV. Electrophysiological observations in flocculus of adapted monkeys. J. Neurophysiol. 43 (5), 1477–1493. Mueller, A.L., Davis, A.J., Robinson, F.R., 2012. Long-term size-increasing adaptation of saccades in macaques. Neuroscience 224, 38–47. Nagao, S., 1992. Different roles of flocculus and ventral paraflocculus for oculomotor control in the primate. Neuroreport 3, 13–16. Nagao, S., Kitamura, T., Nakamura, N., Hiramatsu, T., Yamada, J., 1997a. Differences of the primate flocculus and ventral paraflocculus in the mossy and climbing fiber input organization. J. Comp. Neurol. 382, 480–498. Nagao, S., Kitamura, T., Nakamura, N., Hiramatsu, T., Yamada, J., 1997b. Location of efferent terminals of the primate flocculus and ventral paraflocculus revealed by anterograde axonal transport methods. Neurosci. Res. 27, 257–269. Noda, H., Fujikado, T., 1987a. Involvement of Purkinje cells in evoking saccadic eye movements by microstimulation of the posterior cerebellar vermis of monkeys. J. Neurophysiol. 57, 1247–1261. Noda, H., Fujikado, T., 1987b. Topography of the oculomotor area of the cerebellar vermis in macaques as determined by microstimulation. J. Neurophysiol. 58, 359–378. Noda, H., Mikami, A., 1986. Discharges of neurons in the dorsal paraflocculus of monkeys during eye movements and visual stimulation. J. Neurophysiol. 56, 1129–1146. Noda, H., Suzuki, D.A., 1979. Processing of eye movement signals in the flocculus of the monkey. J. Physiol. 294, 349–364. Noda, H., Sugita, S., Ikeda, Y., 1990. Afferent and efferent connections of the oculomotor region of the fastigial nucleus in the macaque monkey. J. Comp. Neurol. 302, 330–348. Ohki, M., Kitazawa, H., Hiramatsu, T., Kaga, K., Kitamura, T., Yamada, J., Nagao, S., 2009. Role of primate cerebellar hemisphere in voluntary eye movement control revealed by lesion effects. J. Neurophysiol. 101 (2), 934–947. Ohtsuka, K., Noda, H., 1995. Discharge properties of Purkinje cells in the oculomotor vermis during visually guided saccades in the macaque monkey. J. Neurophysiol. 74 (5), 1828–1840. Optican, L.M., 2005. Sensorimotor transformation for visually guided saccades. Ann. N.Y. Acad. Sci. 1039, 132–148. Optican, L.M., Robinson, D.A., 1980. Cerebellar-dependent adaptive control of primate saccadic system. J. Neurophysiol. 44, 1058–1076. Palkovits, M., Mezey, E., Hamori, J., Szentagothai, J., 1977. Quantitative histological analysis of the cerebellar nuclei in the cat. I. Numerical data on cells and on synapses. Exp. Brain Res. 28, 189–209. Partsalis, A.M., Zhang, Y., Highstein, S.M., 1995a. Dorsal Y group in the squirrel monkey. I. Neuronal responses during rapid and long-term modifications of the vertical VOR. J. Neurophysiol. 73 (2), 615–631. Partsalis, A.M., Zhang, Y., Highstein, S.M., 1995b. Dorsal Y group in the squirrel monkey. II. Contribution of the cerebellar flocculus to neuronal responses in normal and adapted animals. J. Neurophysiol. 73 (2), 632–650.

153

154

CHAPTER 6 Cerebellum-Dependent Motor Learning

Prsa, M., Dash, S., Catz, N., Dicke, P.W., Thier, P., 2009. Characteristics of responses of Golgi cells and mossy fibers to eye saccades and saccadic adaptation recorded from the posterior vermis of the cerebellum. J. Neurosci. 29, 250–262. Prsa, M., Dicke, P.W., Thier, P., 2010. The absence of eye muscle fatigue indicates that the nervous system compensates for non-motor disturbances of oculomotor function. J. Neurosci. 30, 15834–15842. Rambold, H., Churchland, A., Selig, Y., Jasmin, L., Lisberger, S.G., 2002. Partial ablations of the flocculus and ventral paraflocculus in monkeys cause linked deficits in smooth pursuit eye movements and adaptive modification of the VOR. J. Neurophysiol. 87, 912–924. Rashbass, C., 1961. The relationship between saccadic and smooth tracking eye movements. J. Physiol. 159, 326–338. Raymond, J.L., Lisberger, S.G., 1997. Multiple subclasses of Purkinje cells in the primate floccular complex provide similar signals to guide learning in the vestibulo-ocular reflex. Learn. Mem. 3 (6), 503–518. Ritchie, L., 1976. Effects of cerebellar lesions on saccadic eye movements. J. Neurophysiol. 39, 1246–1256. Robinson, D.A., 1976. Adaptive gain control of vestibulo-ocular reflex by cerebellum. J. Neurophysiol. 39, 954–969. Robinson, D.A., Gordon, J.L., Gordon, S.E., 1986. A model of the smooth pursuit eye movement system. Biol. Cybern. 55, 43–57. Robinson, F.R., Soetedjo, R., Noto, C., 2006. Distinct short-term and long-term adaptation to reduce saccade size in monkey. J. Neurophysiol. 96, 1030–1041. Ron, S., Robinson, D.A., 1973. Eye movements evoked by cerebellar stimulation in the alert monkey. J. Neurophysiol. 36, 1004–1022. Shinmei, Y., Yamanobe, T., Fukushima, J., Fukushima, K., 2002. Purkinje cells of the cerebellar dorsal vermis: simple-spike activity during pursuit and passive whole-body rotation. J. Neurophysiol. 87, 1836–1849. Smilgin, A., Dicke, P.W., Thier, P., 2012. Purkinje cells of the monkey oculomotor vermis respond to saccades as well as to smooth pursuit initiation. Program No. 580.01. Neuroscience Meeting Planner, Society for Neuroscience, New Orleans, LA. Soetedjo, R., Fuchs, A.F., 2006. Complex spike activity of purkinje cells in the oculomotor vermis during behavioral adaptation of monkey saccades. J. Neurosci. 26, 7741–7755. Soetedjo, R., Fuchs, A.F., Kojima, Y., 2009. Subthreshold activation of the superior colliculus drives saccade motor learning. J. Neurosci. 29, 15213–15222. Stone, L.S., Lisberger, S.G., 1990. Visual responses of Purkinje cells in the cerebellar flocculus during smooth-pursuit eye movements in monkeys. I. Simple spikes. J. Neurophysiol. 63, 1241–1261. Straube, A., Robinson, F.R., Fuchs, A.F., 1997. Decrease in saccadic performance after many visually guided saccadic eye movements in monkeys. Invest. Ophthalmol. Vis. Sci. 38, 2810–2816. Suzuki, D.A., Keller, E.L., 1988a. The role of the posterior vermis of monkey cerebellum in smooth-pursuit eye movement control. I. Eye and head movement-related activity. J. Neurophysiol. 59, 1–18. Suzuki, D.A., Keller, E.L., 1988b. The role of the posterior vermis of monkey cerebellum in smooth-pursuit eye movement control. II. Target velocity-related Purkinje cell activity. J. Neurophysiol. 59, 19–40.

References

Suzuki, D.A., Yamada, T., Hoedema, R., Yee, R.D., 1999. Smooth-pursuit eye-movement deficits with chemical lesions in macaque nucleus reticularis tegmenti pontis. J. Neurophysiol. 82, 1178–1186. Takagi, M., Zee, D.S., Tamargo, R.J., 1998. Effects of lesions of the oculomotor vermis on eye movements in primate: saccades. J. Neurophysiol. 80, 1911–1931. Takagi, M., Zee, D.S., Tamargo, R.J., 2000. Effects of lesions of the oculomotor cerebellar vermis on eye movements in primate: smooth pursuit. J. Neurophysiol. 83, 2047–2062. Thielert, C.D., Thier, P., 1993. Patterns of projections from the pontine nuclei and the nucleus reticularis tegmenti pontis to the posterior vermis in the rhesus monkey: a study using retrograde tracers. J. Comp. Neurol. 337, 113–126. Thier, P., 2011. The oculomotor cerebellum. In: Liversedge, S.P., Gilchrist, I., Everling, S. (Eds.), The Oxford Handbook of Eye Movements. Oxford University Press, Oxford. Thier, P., Mock, M., 2006. The oculomotor role of the pontine nuclei and the nucleus reticularis tegmenti pontis. Prog. Brain Res. 151, 293–320. Thier, P., Koehler, W., Buettner, U.W., 1988. Neuronal activity in the dorsolateral pontine nucleus of the alert monkey modulated by visual stimuli and eye movements. Exp. Brain Res. 70, 496–512. Thier, P., Dicke, P.W., Haas, R., Barash, S., 2000. Encoding of movement time by populations of cerebellar Purkinje cells. Nature 405, 72–76. Thier, P., Dicke, P.W., Haas, R., Thielert, C.D., Catz, N., 2002. The role of the oculomotor vermis in the control of saccadic eye movements. Ann. N.Y. Acad. Sci. 978, 50–62. Tziridis, K., Dicke, P.W., Thier, P., 2009. The role of the monkey dorsal pontine nuclei in goaldirected eye and hand movements. J. Neurosci. 29, 6154–6166. Waespe, W., Cohen, B., Raphan, T., 1985. Dynamic modification of the vestibulo-ocular reflex by the nodulus and uvula. Science 228 (4696), 199–202. Wang, S.S., Denk, W., Hausser, M., 2000. Coincidence detection in single dendritic spines mediated by calcium release. Nat. Neurosci. 3, 1266–1273. Watanabe, E., 1985. Role of the primate flocculus in adaptation of the vestibulo-ocular reflex. Neurosci. Res. 3 (1), 20–38. Westheimer, G., Blair, S.M., 1973. Oculomotor defects in cerebellectomized monkeys. Invest. Ophthalmol. 12, 618–621. Xu-Wilson, M., Chen-Harris, H., Zee, D.S., Shadmehr, R., 2009. Cerebellar contributions to adaptive control of saccades in humans. J. Neurosci. 29, 12930–12939. Yamada, J., Noda, H., 1987. Afferent and efferent connections of the oculomotor cerebellar vermis in the macaque monkey. J. Comp. Neurol. 265, 224–241. Zee, D.S., Yamazaki, A., Butler, P.H., Gucer, G., 1981. Effects of ablation of flocculus and paraflocculus of eye movements in primate. J. Neurophysiol. 46, 878–899.

155

CHAPTER

Decorrelation Learning in the Cerebellum: Computational Analysis and Experimental Questions

7

Paul Dean1, John Porrill Department of Psychology, Sheffield University, Sheffield, United Kingdom Corresponding author: Tel.: þ44-114-222-6521; Fax: þ44-114-276-6515, e-mail address: [email protected]

1

Abstract Many cerebellar models use a form of synaptic plasticity that implements decorrelation learning. Parallel fibers carrying signals positively correlated with climbing-fiber input have their synapses weakened (long-term depression), whereas those carrying signals negatively correlated with climbing input have their synapses strengthened (long-term potentiation). Learning therefore ceases when all parallel-fiber signals have been decorrelated from climbing-fiber input. This is a computationally powerful rule for supervised learning and can be cast in a spike-timing dependent plasticity form for comparison with experimental evidence. Decorrelation learning is particularly well suited to sensory prediction, for example, in the reafference problem where external sensory signals are interfered with by reafferent signals from the organism’s own movements, and the required circuit appears similar to the one found to mediate classical eye blink conditioning. However, for certain stimuli, avoidance is a much better option than simple prediction, and decorrelation learning can also be used to acquire appropriate avoidance movements. One example of a stimulus to be avoided is retinal slip that degrades visual processing, and decorrelation learning appears to play a role in the vestibulo-ocular reflex that stabilizes gaze in the face of unpredicted head movements. Decorrelation learning is thus suitable for both sensory prediction and motor control. It may also be well suited for generic spatial and temporal coordination, because of its ability to remove the unwanted side effects of movement. Finally, because it can be used with any kind of time-varying signal, the cerebellum could play a role in cognitive processing.

Keywords Cerebellum, eye blink conditioning, vestibulo-ocular reflex, spike-timing dependent plasticity, avoidance learning, long-term depression, long-term potentiation, supervised learning, reafference, least mean squares

Progress in Brain Research, Volume 210, ISSN 0079-6123, http://dx.doi.org/10.1016/B978-0-444-63356-9.00007-8 © 2014 Elsevier B.V. All rights reserved.

157

158

CHAPTER 7 Decorrelation Learning and the Cerebellum

1 INTRODUCTION A central feature of the models published soon after the seminal description of the anatomy and physiology of the cerebellum (Eccles et al., 1967) is that they can learn (Albus, 1971; Marr, 1969). Whenever cerebellar output (modeled as Purkinje cell simple-spike firing, Fig. 1A) is in error, a climbing-fiber signal produces changes in the efficacy of synapses between parallel fibers and Purkinje cells, which in turn alter Purkinje cell firing. Eventually, according to the models, cerebellar output reaches the desired values. This procedure only works if the synaptic adjustment does in fact reduce output error. A learning rule that achieves this goal can be derived analytically (Appendix, Fig. 1B), and takes the form: dw i ¼ bheðt Þpi ðt Þi

(7.1)

where dwi denotes the change in weight (i.e., efficacy) of the synapse carrying the i-th parallel-fiber signal, e(t) the climbing-fiber or error signal, pi(t) the signal on the i-th parallel fiber (both functions of time), b is the learning rate and the angled brackets denote expected or mean values. The error signal here is the difference between the actual and desired output (actual and desired Purkinje cell firing). The rule means that if the signal on a particular parallel fiber is positively correlated with the error, the efficacy of the corresponding synapse is reduced; conversely, if the correlation is negative, efficacy is increased. It appears that correlation is being taken to imply cause, which in this case it does provided cerebellar output can in fact influence the error signal appropriately. If so, the rule reduces weights on those parallel-fiber signals that increase the error, while increasing those that reduce it (Dean et al., 2002). Albus (1971) used an informal version of this rule in which “[t]he amount of weakening of each synapse is proportional to how strongly that synapses is exciting the Purkinje cell at the time of the error signal” (pp. 44–45). It seems to have been first applied formally by Fujita (1982, equation 24), and versions of it are currently used in almost all cerebellar models that seek to simulate the role of the cerebellum in behavioral tasks (references in Dean et al., 2010). The fact that the rule is widely used in modeling makes it all the more important to find out whether it is correct, and the next section briefly describes some of the questions raised by current experimental evidence. Subsequent sections consider what the computational consequences of this learning rule would be for cerebellar function, assuming that it is in fact correct. The learning rule shown in Eq. (7.1) has a number of names (Appendix). Here, the actual process of learning is referred to as decorrelation learning, since this draws attention to an important feature of the rule, which is that learning only stops when there is no longer a correlation between any parallel-fiber signal and the climbingfiber signal. Thinking of cerebellar learning in decorrelation terms can help intuitions about cerebellar function in complex circuits. Finally we note that, although the decorrelation formulation presented here is very powerful, this does not rule out other complementary modes of operation (discussed further in Dean and Porrill, 2011).

1 Introduction

FIGURE 1 A schematic diagram of basic features of the cerebellar cortical microcircuit. Mossy-fiber inputs y(t) are distributed over many granule cells, whose axons bifurcate to produce parallel fibers carrying signals pi(t). These fibers form synapses with weights wi on Purkinje cells. Each Purkinje cells also receives a climbing-fiber signal e(t), which in Marr–Albus models is assumed to alter the weights wi. In these models, Purkinje cell output z(t) is assumed to be simple-spike firing, with the effects of complex spikes produced by climbing-fiber input usually neglected. Not shown in the diagram: (i) Purkinje cell output is inhibitory and acts via neurons in the deep cerebellar nuclei (and vestibular nuclei); (ii) granule cell axons also form synapses on molecular-layer interneurons (stellate and basket cells), which in turn form inhibitory synapses on Purkinje cells. In this way, granule cells influence Purkinje cells via both an excitatory direct and an inhibitory indirect pathway. (B) Systems-level equivalent of circuit shown in A, which corresponds to an adaptive (analysis–synthesis) filter. Processing of mossy-fiber input y(t) by the granule cell layer is interpreted as analysis by a bank of causal filters Gi so that the parallel fibers carry signals which form an expansion recoding, pi ¼ Gi[y], P of the mossy-fiber input. Purkinje cell output is modeled as a weighted sum z(t) ¼ wipi(t) of P its parallel-fiber inputs so the Purkinje cell implements a linear-in-weights filter C ¼ wiGi. The climbing-fiber input is interpreted as a training signal e(t) that adapts synaptic weights wi using the decorrelation learning rule derived in the Appendix. Panels (A) adapted from fig. 1A of Porrill et al. (2004) and (B) from fig. 1B of Porrill et al. (2004).

159

160

CHAPTER 7 Decorrelation Learning and the Cerebellum

2 IMPLEMENTATION OF LEARNING RULE The derivation of the learning rule in Eq. (7.1) assumes that neuronal firing rates could be treated as continuous variables (Appendix). But in order to relate the rule to experimental evidence, it has to be translated into a form suitable for signals that are carried by neuronal spike trains, that is in a spike-timing dependent plasticity (STDP) form. The next section briefly describes one way in which the translation can be carried out (more detailed discussion of the theoretical issues involved is given in Menzies et al., 2010).

2.1 STDP Version When both parallel-fiber and climbing-fiber signals have higher than normal values in a small time range, Eq. (7.1) requires that the weight decreases; this implies that near simultaneous climbing-fiber and parallel-fiber spikes should reduce (depress) the efficacy of the synapse between the parallel fiber and the Purkinje cell. However, it is clear that this process on its own would lead to eventual silencing of all synapses, so it must be balanced by some process that increases (potentiates) synaptic efficacy. This combination of depression and potentiation can be summarized in the spiketiming dependent plasticity (STDP) profile illustrated in Fig. 2B, in which the strong depression for nearly coincident spikes is surrounded by much weaker potentiating side lobes for parallel-fiber spikes that are not coincident with a climbing-fiber spike. It can be demonstrated that this STDP profile implements the decorrelation learning rule for suitable rate coding schemes (cf. Menzies et al., 2010). Since parallel-fiber and climbing-fiber signals at their tonic rates carry no signal information, these rates in combination should not produce learning. To achieve this, the total amounts of depression and potentiation under the STDP profile must be exactly balanced. A remarkable consequence of such a balance can be deduced by considering climbing-fiber spikes that are “missing” from the tonic background. Parallel-fiber spikes that are close in time to these missing spikes fail to produce the expected depression; instead these climbing-fiber spike “holes” effectively drive an equal and opposite potentiation. Hence, despite its seeming emphasis on depression, the spiking learning rule is completely symmetric in practice: positive signal correlations produce depression while negative signal correlations produce equal and opposite potentiation. This symmetry is very important, for example, it supplies a mechanism via which previously silent synapses can be recruited to perform a task (see below). Although Fig. 2B captures the basic shape of the STDP learning rule, it is misleading in that it assumes the error signal carried by the climbing fibers is instantaneously available for learning at the relevant synapses. However, for some regions of the cerebellum the error signal can be subject to large transmission delays (e.g., 50–100 ms for visual processing), which means that the parallel-fiber and climbingfiber signals will not match exactly in time. This mismatch can be shown to lead to unstable learning at high frequencies (Porrill and Dean, 2007a). Since the

2 Implementation of Learning Rule

FIGURE 2 Spike-timing dependent plasticity (STDP) implementation of decorrelation learning. (A) Incremental changes in the weights of synapses between parallel fibers and Purkinje cells depend on the relative timing T of parallel-fiber and climbing-fiber action potentials. Positive T is chosen to represent climbing-fiber spikes arriving after parallel-fiber spikes, so that the parallel-fiber contribution to Purkinje cell input could have causally affected the component of the teaching signal carried by that climbing-fiber spike. (B) The decorrelation learning rule can be induced by the spike-timing dependent plasticity profile shown in this plot. Nearly coincident parallel-fiber and climbing-fiber spikes produce long-term depression (LTD), whereas widely separated parallel-fiber and climbing-fiber spikes produce a much smaller amount of long-term potentiation (LTP). The total amount of LTD and LTP must balance in order to produce the symmetric behavior required by the learning rule. To obtain an exact correlational rule, the dip must be infinitesimally wide and infinitely high (a delta function) and the surround LTP lobes must be infinitely wide and infinitesimally high. The more realistic smooth profile shown here will restrict learning performance by causing it to fall off at very high and very low frequencies (further details in Menzies et al., 2010). (Continued)

161

162

CHAPTER 7 Decorrelation Learning and the Cerebellum

climbing-fiber signal cannot be advanced, unstable learning has to be prevented by delaying the effect of the parallel-fiber signal. A filter which affects such a delay is called an eligibility trace (Fig. 2C), and it can be incorporated into the STDP profile by situating the depression dip at an inter-spike time corresponding to the transmission delay (Fig. 2D). One further implementation issue is the seeming limitation imposed by the low spike rate of climbing fibers. How can such a firing rate of around 1 Hz carry useful information about movement errors with frequencies up to 5 Hz and above? The answer is that the error signal is used for learning not instantaneous feedback, so that weight changes can be averaged over many training events. Hence, over the relevant training period, we only require that the expected value on the right hand side of the learning rule is properly calculated. The theoretical constraint is that, over multiple trials, climbing-fiber activity should provide an unbiased estimate of the teaching signal at the relevant frequencies. It is clear that an “event-triggered average” of climbing-fiber activity over many trials can contain frequencies higher than 1 Hz and in practice cerebellar learning rates are slow enough to provide substantial effective high-frequency content in the teaching signal.

2.2 Experimental Questions Initial debates about the decorrelation learning rule focused on whether the cerebellum used it in any form (references in, e.g., Jacobson et al., 2008; Llina´s et al., 2004). More recently attention has shifted to understanding what particular form of the rule might be implemented by the cerebellar microcircuit. The STDP profile shown in Fig. 2D suggests that the most prominent form of plasticity at synapses between parallel fibers and Purkinje cells would be depression of synaptic efficacy, produced by conjunctive stimulation of parallel and climbing fibers. This phenomenon, now known as cerebellar long-term-depression or LTD, was demonstrated by Ito et al. (1982) and has since been the subject of extensive experimental investigation (for reviews see, e.g., Ito, 2001, 2012). The less prominent increase in synaptic efficacy (long-term potentiation, LTP) produced by stimulation of parallel fibers alone (Fig. 2B and D) has been described in postsynaptic FIGURE 2—Cont’d (C) The symmetric STDP profile in B does not respect causality as LTD can be produced by parallel-fiber spikes arriving after climbing-fiber spikes. The expected delay between climbing-fiber spike and the parallel-fiber spike which could be responsible (about 100 ms when the teaching signal is, e.g., retinal slip) can be incorporated into the learning rule by incorporating an eligibility trace induced by the parallel-fiber spike which describes the eligibility for learning of subsequent climbing-fiber spikes. Here, this is chosen to be causal and peak at the expected delay. (D) The eligibility trace can be combined with the STDP profile in B to produce a causal STDP profile tuned to the expected delay in the teaching signal. The LTD dip produces maximum learning at the expected delay, and the broad dip limits high-frequency learning, reducing instabilities caused by any inaccuracy in the estimated delay.

2 Implementation of Learning Rule

form by Lev-Ram et al. (2002), and shown to be capable of reversing the effects of LTD (Coesmans et al., 2004; Lev-Ram et al., 2003). Functional consequences of this bidirectional plasticity have been reviewed by Jo¨rntell and Hansel (2006), including its ability to explain how parallel-fiber stimulation without conjunctive climbingfiber stimulation can enormously increase the tactile receptive fields of Purkinje cells in the C3 zone of the cerebellum in vivo (Ekerot and Jorntell, 2003; Jo¨rntell and Ekerot, 2002, 2011). In general terms, it seems that the bidirectional plasticity observed at synapses between parallel fibers and Purkinje cells fits reasonably well with the STDP profile (Fig. 2D) derived from the decorrelation learning rule (Dean et al., 2010). There are, however, a number of findings that show that the detailed mechanisms underlying this fit are not well understood. For example, LTD at synapses between parallel fibers and Purkinje cells is not observed in mice with mutations that specifically prevent the internalization of AMPA receptors at those synapses (Schonewille et al., 2011). Yet these mice do not show classical cerebellar learning impairments on tasks such as eye blink conditioning or adaptation of the vestibular-ocular reflex. Less dramatic but still of concern is the discrepancy between in vitro studies of LTD which typically find some learning when parallel-fiber and climbing-fiber inputs arrive simultaneously (although learning may be faster when the climbing-fiber signal is delayed by 100 ms, cf. Fig. 2D), and in vivo studies of eye blink conditioning that find no learning for simultaneously presented conditioned and unconditioned stimuli (reviewed by Hesslow et al., 2013). One explanation for these discrepancies is that because synapses between parallel fibers and Purkinje cells are extremely complex and can display numerous forms of both LTD and LTP in vitro, at present the actual form used in vivo has yet to be identified. Alternatively, differences between in vitro and in vivo conditions may produce substantial differences in the properties of the same LTD process (both arguments developed further by Ito, 2012). Moreover, it is possible that in vivo new learning often starts with LTP, rather than LTD (Porrill and Dean, 2008). One computational feature of the decorrelation learning rule is that synapses for parallel fibers carrying noise or signals unrelated to climbing-fiber signals will eventually be driven to zero, and experimental evidence indicates that a high proportion (up to 98%) of parallel-fiber synapses are indeed silent (Isope and Barbour, 2002; Jo¨rntell and Ekerot, 2002; Wang et al., 2000). Since the only way that silent synapses can take part in new learning is via LTP, it is possible that the properties of in vitro LTP rather than LTD will be relevant to in vivo learning. In fact, a further implication of relying on LTP is that there needs to be plasticity in the inhibitory pathway from granule cells to Purkinje cells via molecular-layer interneurons (stellate and basket cells), otherwise tasks such as eye blink conditioning that require a reduction in Purkinje cell firing could not be learnt (Dean et al., 2010; Ekerot and Jo¨rntell, 2003; Porrill and Dean, 2008). Such plasticity, predicted by Albus (1971), has been described in vivo (e.g., Jirenhed et al., 2013; Jo¨rntell and Ekerot, 2003, 2011; Jo¨rntell et al., 2010), with general properties consistent with the decorrelation rule, that is, in the inhibitory pathway

163

164

CHAPTER 7 Decorrelation Learning and the Cerebellum

parallel-fiber stimulation on its own gives LTD, but LTP in conjunction with climbing-fiber stimulation, the converse of the STDP required by the excitatory pathway (Dean et al., 2010). Present evidence therefore suggests that bidirectional plasticity of the general form shown in Fig. 2 could underlie in vivo electrophysiological results for synapses between parallel fibers and Purkinje cells, and in converse form could underlie results for synapses between parallel fibers and molecular-layer interneurons. However, important issues remain to be resolved, in particular, the relative roles of LTD and LTP in different cerebellar learning tasks and the precise mechanisms that mediate LTP and LTD in each of the two pathways between granule and Purkinje cells (references in, e.g., Andreescu et al., 2011; Belmeguenai et al., 2010; Boyden et al., 2006; Prestori et al., 2013; Schonewille et al., 2010). Moreover, the efficacy of the proposed learning rule will be influenced by the effects of plasticity at additional sites in cerebellar cortex, vestibular nuclei, and possibly the deep cerebellar nuclei (e.g., D’Angelo and De Zeeuw, 2009; De Zeeuw and Yeo, 2005; Gao et al., 2012; Hesslow et al., 2013; Porrill and Dean, 2007a), though these effects will not be considered further here.

3 PROPERTIES OF LEARNING RULE The general properties of the decorrelation learning rule have been extensively studied in artificial devices, one of which—a form of adaptive filter (Fig. 1B)—has a structure similar to that of the simplified microcircuit shown in Fig. 1A (Fujita, 1982). These studies have first of all demonstrated the computational power of the rule for supervised learning, that is, learning that uses a teaching or error signal (e.g., Widrow and Stearns, 1985) This power derives from the rule’s mathematical foundations (Appendix), which ensure that it guaranteed to converge to the optimal solution given an appropriate error signal (e.g., see Widrow and Stearns, 1985). Secondly, devices that use the rule, including adaptive filters, have been employed in a wide variety of circuits for many different signal-processing and motor-control tasks (e.g., Widrow and Stearns, 1985). This is clearly appropriate for the cerebellum, where a relatively uniform microcircuit is functionally divided into many microcomplexes, each with their unique set of external connections (e.g., Apps and Hawkes, 2009; Ito, 1970, 1997; Porrill et al., 2013). Thirdly, the learning rule is highly suitable for what are generally considered to be typical “cerebellar” tasks. Historically, on the basis of available anatomical, clinical, and lesion evidence, the cerebellum was associated with motor control (e.g., Dow and Moruzzi, 1958; Ito, 1984). For example, clinical observations indicated that cerebellar damage led to inaccurate and uncoordinated movements, with little apparent effect on standard tests of sensory processing (Glickstein et al., 2009). Subsequent investigations however indicated that the cerebellum was involved in active sensing, where the acquisition of sensory information was dependent on the organism’s own activities (e.g., Bower, 1997; Bower and Parsons, 2003). A central feature

4 Sensory Prediction

of active sensing was identified as the ability to predict the sensory consequences of movement (e.g., Bastian, 2011; Imamizu, 2010; Ito, 2012; Medina, 2011; Wolpert et al., 1998). In the next sections, therefore we examine more specifically how the decorrelation learning rule could be used by the cerebellum for both sensory prediction and movement control.

4 SENSORY PREDICTION The predicted sensory consequences of movements can be used to help solve a variety of different sensorimotor problems. One of these is the reafference problem, which occurs when the sensory signals produced by the organism’s own movement interfere with sensory signals coming from the outside world. This problem has long been recognized (for review see, e.g., Cullen, 2004), and is sometimes referred to as the reafference problem because it requires distinguishing between “reafferent” (internally generated) and “exafferent” (externally generated) signals. We focus on the reafference problem as an example of sensory prediction because it is well studied in a biological context, with reasonable evidence of cerebellar involvement.

4.1 The Reafference Problem 4.1.1 Computational Analysis Use of decorrelation learning in artificial systems (Widrow and Stearns, 1985; Widrow et al., 1975) suggests a circuit for predicting the sensory consequences of movement which is effective, and requires only signals that are biologically available (Fig. 3A). The signals are (i) an efference copy of the motor command r(t), and (ii) input from the relevant sensors s(t) þ n(t), which is a mixture of reafferent n(t) and exafferent s(t) signals. The key to the circuit is an adaptive element (here corresponding to a cerebellar microcomplex) that takes the efferent copy as its mossy-fiber input, and the difference between its output nest(t) and the observed sensory signal s(t) þ n(t) as its teaching signal. The central idea is that this signal will train the microcomplex to produce an accurate estimate nest(t) of the sensory effects n(t) generated by motor commands, even though the dynamical links between the commands and the sensory consequences (which include neural circuits, muscle and other tissue properties, and sensor characteristics) are unknown. Since the actual sensory input to the system is the sum s(t) þ n(t), where s(t) is the externally generated sensory signal, subtraction from it of the cerebellar output gives an estimate of s(t) of the form sest ðt Þ ¼ sðt Þ þ n ðt Þ  n est ðt Þ

(7.2)

The critical computational point here is that when nest(t) is incorrect, the sensory estimate sest(t) will still be contaminated by the effects of motor commands, and this residual contamination will reveal itself as a correlation between the sensory estimate and the efference copy. By decorrelating these two quantities, the learning rule therefore produces an accurate estimate of external sensory input. The decorrelation

165

166

CHAPTER 7 Decorrelation Learning and the Cerebellum

FIGURE 3 (A) Generic adaptive architecture for reafference problem. The task is to cancel the “reafferent” noise n(t) produced by the system’s own movements that additively corrupts external (exafferent) signals of interest s(t). Motor commands r(t) produce the noise n(t) acting via an unknown dynamic process (motor plant plus sensory dynamics). An efference copy of the commands is sent as input to an adaptive element, which learns to produce an estimate nest(t) of the noise. This estimate is used to cancel the noise, resulting in a prediction sest(t) of the exafferent signal s(t), which also acts as teaching signal. The adaptive element learns to decorrelate r(t) from the teaching signal sest(t). When learning is complete, the adaptive element transforms its input r(t) into nest(t) exactly as the motor plant and sensor dynamics do, so that nest(t) ¼ n(t). This means that sest(t) ¼ s(t), so the output of the system corresponds to the uncorrupted exafferent signal, and also ceases to be correlated with r(t)— that is, decorrelation learning has taken place. (B) Specific adaptive architecture for detecting novel whisker contacts. In this case, the motor commands r(t) moves the rat’s whiskers back and forth (whisking), a movement that affects whisker sensors so producing a reafferent signal n(t). An efference copy of the whisking commands is sent as mossy-fiber input to cerebellar zone A2, whose output via the dorsolateral protuberance (DLP, a part of the deep cerebellar nuclei) acts as an estimate nest(t) of the reafferent signal. This output is sent to the superior colliculus, where it is subtracted from the observed whisker signal s(t) þ n(t). Collicular output is thus an estimate sest(t) ¼ s(t) þ n(t)  nest(t) of external whisker contacts, which is used both to drive orienting movements of the head (not shown), and as an teaching signal to zone A2 via climbing fibers from the caudal medial accessory olive (cMAO). When learning is complete nest(t) ¼ n(t) and so sest(t) ¼ s(t). Panels (A) adapted from fig. 1 of Anderson et al. (2012) and (B) redrawn from fig. 8 of Anderson et al. (2012).

4 Sensory Prediction

learning rule is thus exquisitely suited to the reafference problem. If the effects of the motor command are substantially delayed by the dynamics of the plant, then nest(t) is a predictive estimate. It is thought the cerebellum can predict over an approximately subsecond range. The potential of the circuit in Fig. 3A for improving sensory processing has been demonstrated in a biomimetic robot that uses artificial whiskers to explore its environment (Anderson et al., 2010, 2012). The original circuit from which Fig. 3A is derived was designed for canceling any form of noise (also termed interference) from a signal of interest (Widrow and Stearns, 1985, fig. 12.1). Thus the input to the system does not have to be a motor command, but could be for example another sensory signal—hence, the neutral term “reference input” in Widrow and Stearns (1985) and the corresponding notation r(t). The circuit in Fig. 3A also illustrates an important theoretical point about the nature of the “error” signal, since in this case it is a copy of the desired system output, which is an estimate of the “real” (exafferent) sensory signal. Unlike a conventional error signal, this would not be expected to decay to zero. Instead, it ceases to change when none of the input signals to the adaptive element are correlated with system output. Thus, decorrelation learning will have ceased even though there is a clear “error” signal. Finally, the adaptive element in the circuit is sometimes said to be learning a “forward model.” Although thinking in terms of forward and inverse internal models can be helpful, it is also sometimes a source of confusion (as discussed by, e.g., Medina, 2011; Porrill et al., 2013) and we have chosen not to adopt it here.

4.1.2 Experimental Questions Early studies suggesting a role for the cerebellum in reafference used imaging data from human subjects (e.g., Blakemore et al., 2000, 2001), and were not primarily concerned with the details of the underlying circuitry. Here, however, we focus on the experimental question of identifying neural equivalents of the structures shown in Fig. 3A. This question has two parts, one concerning which regions of the cerebellum are involved, and the second concerning how those regions connect to other parts of the brain—particularly, in the case of the reafference circuit, the part of the brain that implements the comparator. The decorrelation rule only works if cerebellar output influences climbing-fiber signals appropriately. In the circuit of Fig. 3A, this influence occurs inside the system, in the comparator. Some answers to these questions have been proposed for rat whisker movements, and primate head movements. In addition, we argue that studies of eye blink classical conditioning can be viewed in the context of the reafference circuit shown in Fig. 3A. In the case of rat whisking (Fig. 3B, modified from Anderson et al., 2012), the proposal is that a specific region of cerebellar cortex, zone A2 (Fig. 4), receives an efference copy of whisking commands r(t) as mossy-fiber input, and sends its output nest(t) (via a part of the deep cerebellar nuclei termed the dorsolateral protuberance) to the superior colliculus. This output is compared with whisker sensory input s(t) þ n(t) in the superior colliculus, and the result sest(t) sent both to the inferior olive (caudal medial accessory olive) as teaching signal, and to the motor system to produce orienting movements to novel whisker contacts. The implication is that zone A2

167

168

CHAPTER 7 Decorrelation Learning and the Cerebellum

FIGURE 4 Schematic diagram of flattened cerebellar cortex in rat taken from fig. 11 of Voogd (2011). The left hand side shows the cerebellar lobules I–X (I–V anterior lobe, VI–X posterior lobe) labeled in the vermis, together with the names given to their hemispheric extensions HI–HX lateral to the vermis (details of nomenclature given in, e.g., Glickstein and Voogd, 1995; Glickstein et al., 2011). The lobules run approximately medio-laterally. The right hand side shows the organization of cerebellar zones A to D2, strips of cortex that (again approximately) run rostro-caudally. A given zone receives climbing-fiber input from a particular region of the inferior olive and projects solely to a particular region of the deep cerebellar nuclei. Moreover, these two regions are themselves interconnected, forming what appears to be a tightly organized functional subregion. Diagrams of this kind allow particular areas of cerebellar cortex to be conveniently located by their zone and lobule. For more detailed diagrams of cerebellar zones, including their relation to Purkinje cell zebrin staining, see for example, Sugihara and Shinoda (2004), Voogd and Ruigrok (2004), Apps and Hawkes (2009), and Marzban and Hawkes (2011).

learns to predict the sensory effects of whisking n(t), and thus improve the detection of exafferent vibrissal signals s(t). However, although the basic connectivity shown in Fig. 3B is consistent with experimental evidence, and the role of the superior colliculus in orienting to unexpected whisker deflection is well established (references in Anderson et al., 2012), as yet the nature of the signals conveyed by Purkinje cells in zone A2 or neurons in the dorsolateral protuberance is unknown. In the case of primate head (and body) movements, neurons in the rostral fastigial nucleus appear to code sest(t) explicitly, where s(t) itself corresponds to exafferent

4 Sensory Prediction

vestibular and proprioceptive signals related to movement of the head and body (Brooks and Cullen, 2013). This finding directly supports a role for the cerebellum in the reafference problem, and points to a number of important questions concerning the underlying circuitry. For example, the area of the vermis that projects to these neurons, and the nature of its climbing-fiber input, do not appear to have been identified. If it is the case that the comparator in this particular circuit is the rostral fastigial nucleus itself (Brooks and Cullen, 2013), then Purkinje cells in the corresponding area of vermis should signal predicted reafference nest(t) (cf. Fig. 3A). In addition, it is unclear how the signal from the rostral fastigial nucleus is used by the circuit for suppressing the vestibular-ocular reflex described previously (e.g., Roy and Cullen, 2004). One problem here is the sophistication available for processing head movements, since it appears the system can distinguish between voluntary head movements that are carried out and those that are mechanically prevented, by comparing predicted and actual signals from neck proprioceptors (Brooks and Cullen, 2013; Roy and Cullen, 2004). Such a comparison adds yet another layer of complexity to the underlying circuit. More thoroughly studied than either of the above two neural circuits is the one that mediates delay eye blink conditioning (Fig. 5). Here, the relevant area of cerebellar cortex is located primarily in zone C3 (Fig. 4), confined to the hemispheric part

FIGURE 5 Specific adaptive architecture for sensory prediction in classical conditioning of the eye blink. The conditional stimulus r(t) in effect acts to produce a painful unconditional stimulus n(t) to the eye or surrounding tissue after an unknown delay. A copy of r(t) is sent as mossy-fiber input to zone C3 (and possibly zone D0, Mostofi et al., 2010) of lobule HVI. The output of this eye blink region acts via the anterior interpositus nucleus AIP as a prediction nest(t) of the unconditional stimulus. This prediction is sent by the nucleo-olivary pathway to part of the dorsal accessory olive where it is subtracted from the sensory signal provided by part of the trigeminal nucleus. The output of the olive is thus an estimate sest(t) of any unpredicted painful stimulus s(t) to the eye or surrounding tissue and is used as a teaching signal sent via climbing fibers to the eye blink region of cerebellar cortex. Learning in the eye blink region proceeds until nest(t) ¼ n(t) so there is no longer any correlation between r(t) and sest(t). In typical classical conditioning experiments, s(t) is set to zero, so learning proceeds until the teaching signal sest(t) is also zero.

169

170

CHAPTER 7 Decorrelation Learning and the Cerebellum

of lobule VI (Mostofi et al., 2010, for rabbit). In eye blink conditioning, a conditioned stimulus (often a tone) is presented shortly before an unconditioned stimulus (e.g., mild shock to the skin round the eye) that elicits an eye blink, the unconditioned response. After a number of such pairings, the tone itself elicits a blink (conditioned response) whose peak amplitude occurs approximately at the same time as the unconditioned response is delivered. Very extensive experimental investigation (reviewed in, e.g., De Zeeuw and Yeo, 2005; Hesslow and Yeo, 2002; Thompson and Steinmetz, 2009) has revealed that mossy-fiber input to the eye blink zone conveys information about the conditioned stimulus, climbing-fiber input conveys information about the unconditioned stimulus, and the output of the eye blink zone is related to the conditioned response. Although eye blink conditioning is often treated as an example of associative learning, here we consider it in the framework of sensory prediction and the reafference problem. We do so by comparing three features of the circuits shown in Figs. 3A and 5. First, as far as the adaptive element in Fig. 3A is concerned, the critical computational feature of its input is that it predicts future sensory signals. The circuit remains the same whether this predictive input is an efferent copy (Fig. 3A) or another sensory stimulus (Fig. 5). The predictive task is actually simpler in the case of eye blink conditioning than in usual reafference problems, since the dynamics linking conditioned and unconditioned stimuli are a simple delay between stimulus onsets. In fact, reafference problems in general can be solved by the most suitable mixture of efference copy and relevant sensory signals (Cullen, 2004). Thus, as far as sensory prediction is concerned, the difference between the input signals of Figs. 3A and 5 is not fundamentally important, and reflects the generic character of the original noise-cancelation circuit (Widrow and Stearns, 1985; Widrow et al., 1975). Secondly, although cerebellar output in eye blink conditioning is usually considered in terms of motor commands, it is well known to have a sensory predictive component. For example, in classical conditioning the unconditioned stimulus is delivered regardless of the organism’s response, yet the climbing-fiber signal to cerebellar cortex, which in eye blink conditioning is driven by inescapable periorbital shock, nonetheless diminishes as acquisition proceeds (e.g., Hesslow and Ivarsson, 1996; Rasmussen et al., 2008; Sears and Steinmetz, 1991). The climbing-fiber signal appears to be predicted shock, and models of eye blink conditioning have therefore typically used a comparator in which cerebellar output is compared with the unconditioned stimulus signal, just as in Fig. 5 (e.g., Grossberg and Schmajuk, 1989; Lepora et al., 2010; Medina et al., 2000b; Moore et al., 1989). The training signal then becomes not s(t) þ n(t) but s(t) þ n(t)  nest(t), and since the chances of a second painful stimulus occurring at the same time as the one administered experimentally are low, it can be treated simply as n(t)  nest(t). With this signal, acquisition ceases when nest(t) equals n(t), that is, it does not continue indefinitely as the classical conditioning paradigm suggests it might. Also, as soon as n(t) is omitted after acquisition, a signal for extinction is available. And once acquisition is complete, addition of

4 Sensory Prediction

a second predictive signal (e.g., a light) to the first should not result in further learning for that signal, since the training signal is effectively zero. This phenomenon, known as blocking, has been observed experimentally (references in Kim et al., 1998). In general, comparator models mimic the performance of trial-level models of conditioning such as that of Rescorla and Wagner (1972), which explicitly use the unpredicted unconditioned stimulus as a teaching signal, exactly as in Fig. 5 (Lepora et al., 2010). Thirdly, a number of studies have suggested that in the specific case of eye blink conditioning an excellent candidate for the comparator is the inferior olive itself (e.g., Andersson et al., 1988; Bengtsson and Hesslow, 2006; Bengtsson et al., 2007; Jirenhed et al., 2007; Kim et al., 1998; Medina et al., 2002; Nicholson and Freeman, 2003; Rasmussen et al., 2008; Sears and Steinmetz, 1991). A subset of neurons in the relevant part of the deep cerebellar nuclei (the anterior interpositus nucleus) send inhibitory projections to the relevant region of the inferior olive (dorsal accessory olive), and these nucleo-olivary neurons would provide a natural substrate for the cerebellar signal nest(t) in Fig. 5. The issue is not settled: it is unclear whether other comparators are located elsewhere (e.g., the red nucleus), or whether an olivary comparator function has to be combined with tonic regulation of Purkinje cell simple-spike firing rates (references in Lepora et al., 2010). Nonetheless, experimental studies of eye blink conditioning have made considerable progress in identifying a vital component of the circuit shown in Fig. 3A. The resemblance between the sensory prediction circuitry of eye blink conditioning and the circuit proposed for solving the reafference problem points to the likely importance of the decorrelation learning rule in the latter task. It has been established that paired stimulation of mossy and climbing-fiber inputs to the C3 zone in lobule VI reduces the firing rates of Purkinje cells located there, whereas subsequent stimulation of mossy fibers alone increases them back to baseline (Jirenhed et al., 2007). This is entirely consistent with the STDP version of the decorrelation learning rule (Fig. 2), although as discussed previously important issues concerning details of the underlying mechanisms remain (Rasmussen et al., 2013). Purkinje cell responses during eye blink conditioning have in fact been modeled using an STDP rule apparently quite similar to that shown in Fig. 2 (Medina et al., 2000a). It is also the case that the circuit of Fig. 5 is connected so that decreases in Purkinje cell firing affect climbing-fiber input in the appropriate direction, that is to decrease it. The general importance of this aspect of connection has been demonstrated by Badura et al. (2013), who found that mutant mice in which the projections of the inferior olive were rerouted ipsilaterally displayed much more severe ataxia that seen after simple cerebellar inactivation.

4.1.3 Summary From a computational perspective, the decorrelation rule seems ideally suited to solving the reafference problem. Application of the rule in artificial systems has led to a simple and effective circuit (Fig. 3A), shown to be useful in robotics. However, corresponding neural circuits have yet to be fully characterized. Thus,

171

172

CHAPTER 7 Decorrelation Learning and the Cerebellum

a proposed circuit for rat whisking corresponds to the known anatomy (Fig. 3B), but the electrophysiological evidence is lacking (Anderson et al., 2012). Conversely, electrophysiological evidence implicates the primate rostral fastigial nucleus in head and body reafference, but the circuitry awaits clarification (Brooks and Cullen, 2013). It seems likely that further experimental investigation of these candidate circuits will advance our understanding of the role of the cerebellum (and as argued here the role of decorrelation learning) in dealing with the reafference problem. We have also suggested that the eye blink-conditioning circuit (Fig. 5) is similar in many respects to the theoretical circuit, even though the task it appears to be carrying out is not quite classical reafference cancelation. There are two differences, the first being that cerebellar output here labeled nest(t) is also used to produce a movement. The implications of this difference are considered further in the section on motor control below. The second is that if the inferior olive is indeed the comparator, then no signal corresponding to sest(t) is available to the rest of the system outside the cerebellum, since the inferior olive projects exclusively to cerebellar targets. As mentioned above, this might make good functional sense, because the chances of a second painful stimulus occurring at the same time as the unconditioned stimulus are low, and the important task for the organism is to use information about the predicted painful stimulus, that is, nest(t). But this reinforces the observation that the task in eye blink conditioning may not be identical to the standard reafference problem. Tantalizingly, the clearest evidence at present concerns not the cerebellum itself, but what are termed precerebellar structures. These structures resemble the cerebellar microcircuit in certain respects, and extensive experimental investigation has demonstrated that some do indeed adaptively remove reafferent interference from sensory signals, Moreover they use a form of the decorrelation learning rule to do so. For example, the electrosensory lateral line lobe of the Mormyrid electric fish is able to learn to form a negative image of the electric organ discharges produced during active electrosensing in order to cancel them from the output of its passive electrosensory receptors (e.g., Bell et al., 1997, 2008; Montgomery et al., 2012; Requarth and Sawtell, 2011). In these cells, the efference copy signals m(t) are supplied on parallel-fiber inputs to the apical dendrites, whereas the mixed sensory signal n(t)þs(t) is also supplied as an input on basal dendrites. The comparator stage can thus occur within the cell itself, with neuronal output as the teaching signal, eliminating the need for a climbing fiber (Dean and Porrill, 2010; Porrill et al., 2013). The required learning rule is a homosynaptic and anti-Hebbian version (for more discussion see, e.g., Roberts and Bell, 2000; Sawtell and Williams, 2008) of the decorrelation learning rule. The evidence concerning the role of precerebellar structures in the reafference problem can only be suggestive of a similar role for the cerebellum, but it does indicate that the basic cerebellar microcircuit has the required computational capacity, and that sensory prediction may have been an evolutionarily ancient cerebellar function.

5 Motor Control

4.2 General Sensory Prediction Recent studies have indicated a cerebellar role in sensory prediction for a wide range of active sensing and motor control tasks (e.g., Bastian, 2011; Bhanpuri et al., 2013; Cerminara et al., 2009; Imamizu et al., 2003; Izawa et al., 2012; Knolle et al., 2012; Miall et al., 2007; Roth et al., 2013; Schlerf et al., 2012; Shmuelof et al., 2012; Tseng et al., 2007). In broad terms, this is consistent with the fact that an adaptive filter using the decorrelation learning rule can be connected effectively in many different circuits that use sensory prediction (e.g., Porrill et al., 2013; Widrow and Stearns, 1985) besides the one shown in Fig. 3A. However, as with the early experiments on reafference mentioned earlier, these studies often use imaging techniques, or examine the performance of patients with cerebellar damage. Detailed information concerning circuitry, such as the exact nature of the signals entering and leaving the particular region of cerebellar cortex, or the relative contribution of extracerebellar structures, remain unknown. In particular, we do not know how cerebellar output affects climbing-fiber input for the relevant microcomplex. At this stage, therefore it is not possible to relate these findings in any specific way to the decorrelation learning rule. Hence, the emphasis here on the restricted topic of reafference.

5 MOTOR CONTROL A natural extension of the reafference problem to motor control comes from the observation that some sensory signals are better avoided than predicted, for example painful stimuli that indicate tissue damage. We therefore consider first the computational issues involved in converting the predictive circuit of Fig. 3A to a circuit capable of signaled (conditioned) avoidance learning, that is using a predictive stimulus to trigger a movement of the body that avoids the pain. Secondly we look at an example where the need for avoidance is perhaps less obvious, which is when the “bad” stimulus is movement of the image across the retina. Such movement irreversibly degrades the visual signal, and so also requires its avoidance by gaze stabilization rather than prediction.

5.1 Signaled-Avoidance Learning 5.1.1 Computational Analysis The circuit for signaled-avoidance learning shown in Fig. 6A differs from the sensory prediction circuit of Fig. 3A in two minor ways. The input to the system r(t) is now labeled “predictive stimulus” (cf. Fig. 5), and this can be either an efference copy or an external stimulus, or a mixture of the two. In similar fashion, the “unknown process” can refer to the body’s own dynamics, or to a contingency imposed externally by an experimenter, or the physics of the environment (cf. previous discussion of eye

173

174

CHAPTER 7 Decorrelation Learning and the Cerebellum

FIGURE 6 (A) Generic circuit for signaled avoidance. Here, a stimulus r(t) can be thought of as producing, via an unknown process or contingency, a painful stimulus n(t). A copy of r(t) is sent to the adaptive element that uses it to produce a motor command mest(t). This command acts via the appropriate motor circuitry and plant to produce what is in effect an estimate nest(t) of the painful stimulus n(t). As learning proceeds, the avoidance movement becomes more successful, until eventually n(t) is avoided completely, that is, nest(t) ¼ n(t). In the case of signaled avoidance, the comparison between nest(t) and n(t) takes place in the external world (dotted box). The unavoided pain signal sest(t) is the sum of any unpredicted painful stimulus s(t), and the inaccuracy n(t)  nest(t) of the prediction, and is used as the teaching signal for the adaptive element. It can activate the motor circuitry on its own account, triggering an escape rather than avoidance movement (not shown). (B) Specific circuit for signaled (conditioned) eye blink avoidance. Here, r(t) is the conditioned stimulus, which predicts the arrival of the painful unconditioned stimulus n(t) after an unknown delay. A copy of r(t) is sent to the cerebellar eye blink region (lobule HVI, zone C3 possibly encroaching on D0) which produces via the anterior interpositus nucleus (AIP) a motor command mest(t). This acts via the red nucleus, accessory abducens nucleus, and nictitating membrane (NM) plant dynamics to produce the conditioned nictitatingmembrane movement with temporal profile nest(t). The unavoided pain signal sest(t) ¼s(t) þ n(t)  nest(t) is detected by corneal receptors and relayed to the trigeminal nucleus, where it can be used to drive an unconditioned NM response (not shown) and is also sent to the dorsal accessory olive to be used as a teaching signal.

5 Motor Control

blink circuit in Fig. 5). These two changes mean that the circuit in Fig. 6A is more general than that of Fig. 3A, but do not affect its basic signal-processing capability. The remaining two differences between the circuits of Figs. 6A and 3A are more significant. First, the internal comparator in Fig. 3A has been removed because “comparison” now takes place in the external world (dotted box in Fig. 6A), in the sense that the movement produced by the circuit physically reduces the painful effects that would otherwise have occurred. One potential computational problem with the external comparator is that for certain kinds of avoidance response the system may not be able to tell if the contingency between predictor stimulus and pain has been altered, and so would not be capable of extinction. This problem is discussed in the next section on eye blink conditioning. Secondly, the outcome of the adaptive element is no longer a simple estimate nest(t) of the predicted sensory signal, but a motor command mest(t) that will produce a movement with temporal profile nest(t) to avoid n(t). This command is acted on by the transfer functions B of the premotor circuitry and P of the motor plant, so that PB mest ðt Þ ¼ n est ðt Þ

(7.3)

The complexities introduced by B and P vary in importance from system to system. In simple cases of conditioned avoidance, the cerebellar output may be able to use the reflex circuitry for escape or withdrawal already in place for the painful stimulus itself. However, for more precise movements such as those required for the VOR, this complexity assumes considerable computational importance. It is therefore considered in more detail in the section on the VOR below.

5.1.2 Experimental Questions Figure 6B shows a circuit, based on Fig. 6A, proposed here for the specific case of signaled-avoidance learning using the eye blink response. To ensure that avoidance is physically possible, the unconditioned stimulus needs to be an airpuff or similar stimulus, whose effects on the cornea can in fact be avoided by closure of eyelids or nictitating membrane (NM). This contrasts with the classical conditioning circuit shown in Fig. 5, where the unconditioned stimulus is a periorbital shock that is unaffected by eye blinks. Avoidance learning is in fact usually regarded as a form of instrumental or operant conditioning, as opposed to a form of classical conditioning (e.g., Mackintosh, 1974; Moore and Gormezano, 1961). The circuit shown in Fig. 6B assumes that the relevant regions of cerebellar cortex and the deep cerebellar nuclei used for avoidance learning are the same as those used for sensory prediction in classical conditioning of the eye blink, namely zone C3 of cortical lobule HVI, and the anterior interpositus nucleus (AIP). The latter projects to the red nucleus, which in turn projects to eye blink motoneurons in both the facial nucleus for movements of the external eyelid, and (as shown here) in the accessory abducens nucleus for movements of the NM. Early in acquisition these conditionedresponse movements will be too small and perhaps wrongly timed, so some air from

175

176

CHAPTER 7 Decorrelation Learning and the Cerebellum

the unconditioned stimulus (airpuff) will continue to reach the cornea. This painful stimulus will be registered by the trigeminal nucleus, whose output triggers an unconditioned reflex blink and also modulates the relevant region of the inferior olive, the dorsal accessory olive. The climbing-fiber teaching signal is relayed to the relevant part of cerebellar cortex. Eventually the movements become fully effective in avoiding the airpuff, so there are no painful stimuli to be signaled by the trigeminal nucleus, hence, no modulation of the inferior olive and no further decorrelation learning. This proposed circuit for eye blink avoidance learning raises a number of experimental questions. One is whether avoidance does in fact use the same basic neural machinery as classical conditioning. There is some direct evidence that the response profiles of avoidance and conditioned eyelid responses are very similar (Mauk and Ruiz, 1992), and also, much more indirectly, a general impression that the effects of neural manipulations are the same for both avoidance learning (airpuff and unrestrained eyelid) and classical conditioning (periorbital shock). This general impression is reinforced by cases where an apparent avoidance paradigm (airpuff plus unrestrained eyelids) is not regarded as importantly different from classical conditioning (for recent example see Chettih et al., 2011). However, detailed comparisons of the effects of cerebellar manipulations in the two paradigms have yet to be made. Another experimental question concerns an issue raised in the previous section, about the relation between the motor command mest(t) and its sensory effects nest(t). In the case of the NM, it appears that the temporal profile of the classically conditioned-response command is delayed and lengthened by the (first-order) dynamics of the retrobulbar muscle and the membrane itself (Lepora et al., 2007). However, these effects are not large compared to the variability of both conditioned-response amplitudes and peak timing (references in Lepora et al., 2010). In any case, the durations of conditioned responses (100–800 ms, depending on interval between conditioned and unconditioned stimulus onset) are often substantially greater than those of the unconditioned stimulus (typically 60 ms for periorbital shock), suggesting the system is not concerned in precisely matching the temporal characteristics of conditioned-response and unconditioned stimulus. It may however be concerned with response amplitude, because evidence on the effects of eyelid restraint suggests that the amplitude of unconditioned blinks is under adaptive control by the cerebellum (Chen and Evinger, 2006; Pellegrini and Evinger, 1997). How this adaptive circuitry might be combined with the circuit proposed in Fig. 6B is not understood. The third question is also related to an issue raised in the previous section, that of extinction. How would the circuit on Fig. 6B know if the unconditioned stimulus were omitted? This is a generic problem with signaled avoidance (Mackintosh, 1974). One obvious possibility in the present case is that when the eyes are fully closed, the airpuff can still be detected by sensors on the eyelids (see also Longley and Yeo, this volume). As far as we are aware, there is no experimental evidence either for or against this possibility. Finally, if the same areas of cerebellar circuitry are indeed involved in both sensory prediction and avoidance learning, then the circuits shown in Figs. 5 and 6B

5 Motor Control

have to be combined. However, it is unclear how in practice the internal and external comparators would work together. Could a sufficiently strong nucleo-olivary signal prematurely stop avoidance learning, in effect fooling the system about the external world? The possibility that the comparators do in fact work together is suggested by the observation that the delay introduced by the nucleo-olivary pathway (90 ms, Best and Regehr, 2009; Hesslow, 1986) is similar to that introduced by the dynamics of the NM plant (Lepora et al., 2010). But little is known about the details of how such a collaboration could work.

5.1.3 Summary The fact that painful stimuli are better avoided than just predicted suggests the utility of extending the sensory predictor circuit of Fig. 3A, and a candidate theoretical circuit for signaled avoidance is shown in Fig. 6A. As with sensory prediction itself, it seems that the most investigated candidate neural circuit for signaled avoidance is the one used in eye blink conditioning (Fig. 6B). However, although it seems plausible that the same cerebellar circuitry is used for both prediction and avoidance, this has yet to be firmly established, and the problems of coordinating an internal comparator with actions in the external world are little understood. Moreover, the decorrelation learning rule cannot be simply applied to motor commands in the same way it can be applied to sensory predictions. This point is examined in more detail in the next section. Although we have focused here on eye blink circuitry as an example of signaled avoidance, other possibilities have been suggested recently. A version of the circuit shown in Fig. 6B has been proposed to account for our general ability, apparently acquired in early childhood, to move our limbs without harming ourselves (Dean et al., 2013). This version uses an efference copy of motor commands to the limbs as input, and reflex withdrawal circuits in the spinal cord to organize the output. This circuit is based on the fore- and hindlimb areas of zone C3 in lobules IV and V, and derives from previous suggestions concerning the functions of these areas (Ekerot et al., 1995, 1997; Jo¨rntell and Ekerot, 2003). The role of signaled avoidance in preventing damaging collisions has also been explored in the context of robot locomotion (Herreros and Verschure, 2013), using a cerebellar-based control circuit to explore the possible role of an internal comparator on switching behavior from reactive mode (unconditioned reflex) to adaptive mode (conditioned reflex). Finally, the conditioned eye blink response to visual threat, again thought to be acquired in early childhood, is compromised in patients with cerebellar degeneration (Thieme et al., 2013). Lesion-symptom mapping implicates regions in the posterior lobe, two of which are in HVI.

5.2 Gaze Stabilization Although whole-image movement (retinal slip) is very different from pain, its degradation of visual processing is so undesirable that many species take great lengths to stabilize the direction of gaze. Gaze stabilization is in effect driven by avoidance of

177

178

CHAPTER 7 Decorrelation Learning and the Cerebellum

retinal slip, so it is possible that the pain avoidance circuit shown in Fig. 6A could be adapted for gaze stabilization. Here, we focus on just one aspect of gaze stabilization, namely the vestibulo-ocular reflex (VOR). Figure 7 shows a signaled-avoidance circuit (Fig. 6A) modified for the horizontal VOR (references in, e.g., Boyden et al., 2004). The input r(t) to the system is now a vestibular signal (from the semicircular canals with some processing in the vestibular nuclei), and it signals a movement of the head that on its own would produce a retinal slip signal n(t). A copy of r(t) is sent as mossy-fiber input to the floccular region of the cerebellum (Fig. 4) where it is transformed into a motor command mest(t) that acts on premotor circuitry in the brainstem. This circuitry, which also receives the vestibular signal r(t) itself, produces an eye movement that can be considered an estimate nest(t) of the eye-rotation produced by the head movement n(t). Any inaccuracy in this eye movement gives rise to a retinal slip signal n(t)  nest(t) which is relayed via structures such as the nucleus of the optic tract (NOT) to the inferior olive (dorsal cap of Kooy) as the training signal for decorrelation learning in the flocculus. There may also be “exafferent” retinal slip, induced for example by an experimenter, but if

FIGURE 7 Signaled-avoidance circuit for the vestibulo-ocular reflex (VOR). Here, the input r(t) is a vestibular signal that is related to actual eye disturbance n(t) via a complex and unknown process involving inverse vestibular processing (to reconstruct actual head movement) and the mechanical linkage between head and eye movement. A copy of r(t) is sent to the flocculus, whose output acts in conjunction with r(t) on brainstem premotor circuitry to produce a motor command mest(t). This acts on the oculomotor plant to produce an “avoidance” eye movement with temporal profile nest(t). In the external world (dotted box), this is in effect subtracted from a notional retinal slip signal, combining predictable (n(t)) and unpredictable (s(t)) retinal slip. The result is an actual retinal slip signal sest(t) that combines any unpredictable retinal slip (s(t)) with slip that was not successfully avoided (n(t)  nest(t)). This signal is relayed via the NOT (nucleus of the optic tract) and related structures to the dorsal cap of Kooy in the inferior olive, which sends climbing fibers to the flocculus. To ensure the stable learning of accurate avoidance eye movements nest(t), the flocculus also receives as mossy-fiber input an efference copy of the motor command mest(t) (see text).

5 Motor Control

this is uncorrelated with head movement then decorrelation learning will not be affected. The driving of the avoidance response by r(t) itself is a difference between the circuits for the VOR (Fig. 7) and signaled pain avoidance (Fig. 6A). Although such driving can occur in signaled avoidance (e.g., the alpha reflex where the CS does trigger an escape response on its own account) it is not a central feature. Here, however, the output of the adaptive element mest(t) acts together with r(t), so the function of the adaptive element can be regarded as reflex calibration. In this case, the system is avoiding current not future retinal slip, so is not acting as a predictor (though there appear to be other gaze-stabilization circuits that are predictive: see Brooks and Cullen, 2013). A second difference between the circuits of Figs. 6A and 7 is that retinal slip is a directional signal, so the problem of extinction is avoided. The third and perhaps most important difference is that the eye movements needed for gaze stabilization must be very precise, much more so than for example eye blink CRs, which can be an order of magnitude longer that the unconditioned stimulus. In the VOR eye movement, amplitude and timing must be matched as closely as possible to the temporal profile of the head movement, a requirement that focuses attention on Eq. (7.3). This equation shows that the output of the adaptive element in Fig. 6A is not an estimate nest(t) of painful signal to be avoided, but an estimate mest(t) of the motor command required avoid it. The situation for the VOR is similar, though more complicated because of the influence of the vestibular input r(t) on eye movement. However, in both cases (Figs. 6A and 7) the teaching signal is still what is sometimes termed “sensory error” n(t)  nest(t), when what is needed is “motor error” m(t)  mest(t) where m(t) is the correct motor command. Since these two signals are related (as in Eq. 7.3) by a complex combination of premotor processing in the brainstem and the action of the oculomotor plant it can be seen that the “motor error” m(t)  mest(t) is not directly available to the system. We have glossed over this problem for the slightly simpler case of signaled avoidance by pointing to the relative imprecision of the CR, in effect by assuming that the differences between sensory and motor error can be safely ignored, but for other kinds of movement it is not clear this assumption is warranted. In particular, there is a danger for complex plants that, for certain frequencies, the relation between sensory and motor error will change in sign, so producing disastrously unstable learning (cf. results of Badura et al., 2013, discussed previously). The solution to this problem that is shown in Fig. 7 is to take an efference copy of the eye-movement command as an additional mossy-fiber input to the flocculus. This “recurrent architecture” using sensory error can be shown theoretically to produce stable learning (Porrill and Dean, 2007b), and has been used in simulated and robot VOR adaptation (Dean et al., 2002; Haith and Vijayakumar, 2009; Lenz et al., 2009; Porrill et al., 2004). Informally, it can be understood as an extension of decorrelation learning to the case where the relevant correlation to be removed is that between one’s own motor commands and some undesired sensory outcome: the correlation again is assumed to signify cause, and removing the correlation equivalent to

179

180

CHAPTER 7 Decorrelation Learning and the Cerebellum

learning accurate movements. The recurrent architecture is not the only solution that has been proposed for the motor-error problem; in particular, Kawato and coworkers have argued strongly for feedback–error–learning where sensory error is converted approximately into a motor-error signal than can be used both for online control and adaptive learning (e.g., Gomi and Kawato, 1992). Recent discussion of the comparative merits of the two solutions can be found in Porrill et al. (2013). Although many studies of floccular function in the VOR are consistent with the circuit shown in Fig. 7, its complexities have made it difficult to firmly identify mechanisms of synaptic plasticity (Boyden et al., 2004; Dean and Porrill, 2011; Porrill et al., 2013). Part of the problem is the factor mentioned previously, that there are both direct excitatory and indirect inhibitory pathways from granule cells to Purkinje cells, and evidence suggests that synapses in both pathways are capable of LTD and LTP. But there is an additional site of synaptic plasticity in the vestibular nuclei that, depending on the precise experimental conditions, can transfer learning from cerebellar cortex to the vestibular nuclei, or adapt the VOR on its own with no apparent learning in cerebellar cortex (Boyden et al., 2004; Ke et al., 2009; Mcelvain et al., 2010; Menzies et al., 2010; Porrill and Dean, 2007a). This latter complexity arises in part because of the optokinetic reflex (OKR) driven by retinal slip (not shown in Fig. 7), which corresponds to the unconditioned eye blink reflex. Because of 100 ms delays in retinal slip processing, this reflex operates primarily at low frequencies (less than 0.5 Hz, Paige, 1983). However, its existence can complicate experimental attempts to modify the VOR. For example, one way of driving VOR gain down is to rotate the subject on a turntable, in phase with rotation of the visual surround. But if the rotation is at a frequency within the range of the OKR the role of the VOR is lessened as the OKR itself reduces retinal slip. Moreover, because the OKR is partly mediated by the flocculus itself, the adaptation paradigm described above will ensure a correlation between the output of the flocculus as it drives the OKR and vestibular input. This is the correlation that can drive plasticity in the vestibular nuclei themselves, in the extreme case without plasticity in the flocculus (Boyden et al., 2004; Mcelvain et al., 2010; Menzies et al., 2010; Porrill and Dean, 2007a). Complexities of this kind can prove even more significant when smooth pursuit, also mediated by the flocculus, plays an important role (Ke et al., 2009). An important experimental question, therefore, is how to devise training conditions that would allow different sites of synaptic plasticity to be investigated selectively. One suggestion (Boyden et al., 2004) is to confine training (at least in primates) to high frequencies such as 5 Hz, where it is known Purkinje cell simple-spike output does not modulate and therefore does not produce learning in the vestibular nucleus (Raymond and Lisberger, 1998).

5.3 General Motor Control We have argued that the decorrelation learning rule is ideally suited to the reafference problem, and to sensory noise cancelation in general. Moreover, the basic noisecancelation circuit can be easily extended to produce avoidance of stimuli rather than

6 Future Directions

simple prediction, which indicates how the decorrelation rule can also be used for motor control. The two examples of motor control considered in some detail were signaled-avoidance learning for eye blink, and gaze stabilization by the VOR. A third example would be saccadic adaptation, in which the sensory stimulus to be avoided rather than predicted is the appearance of a target outside the fovea at the end of a saccade (e.g., Hopp and Fuchs, 2004; Soetedjo et al., 2009). However, for both eye blink and VOR (and indeed saccades) the candidate neural implementation of the theoretical circuit shows that external wiring of cerebellar microzones can be more complex than apparently required, and that these complexities can hinder identification of the underlying computational principles. In eye blink conditioning, it appears that the circuitry may use both an internal comparator and external comparison to combine sensory and motor functions, in a way not familiar to artificial systems that also use the decorrelation rule. In gaze stabilization, the OKR and VOR cooperate in complex and subtle ways to ensure reduction of retinal slip, exploiting where helpful an extracerebellar site of plasticity that extends the dynamic range of the VOR (Porrill and Dean, 2007a). Although these complexities continue to make obstacles for the experimental characterization of cerebellar learning rules, they do point to the possibility that detailed understanding of cerebellar circuits will not only throw light on sensorimotor processing in the brain, but may also suggest novel algorithms for such processing in artificial systems, especially in robotics.

6 FUTURE DIRECTIONS Finally, we briefly consider some rather more general consequences of the decorrelation learning rule for future understanding of cerebellar function, in the context first of coordination, and then of “cognitive” processing.

6.1 Coordination A major problem for complex organisms and robots is the undesirable impact that action by one part can have on other parts. An example of this has already been mentioned: clumsy scratching of the face could in principle cause damage to the eye. Other examples include possible effects of body movements on balance or blood pressure. The general theme is that a desired action is likely to have unwanted side effects. Circuits that use the decorrelation learning rule for signaled avoidance appear ideally suited for dealing with the problem of unwanted side effects. For example, in the absence of adjustment by the cerebellum, certain movements will cause undesirable changes in blood pressure. This can be prevented by having a region of the cerebellum that (i) receives as mossy-fiber input efference copies of the relevant command, (ii) receives as climbing-fiber input a signal related to blood pressure, and (iii) has an output connected appropriately to the neural machinery responsible for cardiovascular control (as in, e.g., Figs. 6A and 7). This region of the cerebellum can

181

182

CHAPTER 7 Decorrelation Learning and the Cerebellum

then use decorrelation learning to both predict when a given command would be followed by changes in blood pressure, and act so as to avoid those changes. For this particular example of coordination, a circuit has been identified, centered on part of the flocculus (Fig. 4) and with the kind of external connectivity just described, that does in fact appear to avoid the otherwise excessive increases in blood pressure that would otherwise be produced by defense reactions (Nisimaru et al., 2013). However, in general the detailed circuits that underlie the multiple forms of coordination required by complex organisms are not yet well understood. It is possible that attempts to use cerebellar-inspired control algorithms to ensure coordination in complex robots may provide clues about the sort of connectivity to look for in particular cases. Signaled-avoidance circuits may also be helpful for what might be termed temporal coordination, needed for rapid sequences of movements made by the same part of the body, as in speech or playing a musical instrument. The precise movement required at any point in a rapid sequence will be influenced by the position of the fingers, tongue, and so forth at the end of the previous movement. In this case, a copy of the next motor command together with information related to current position and velocity would be used as mossy-fiber input, and the difference between intended and actual sound (for speech or music) as climbing-fiber input. Again, while it appears that decorrelation learning could in principle be used to ensure temporal coordination, detailed descriptions of candidate circuits do not yet appear to be available, and studies in robots may provide useful clues.

6.2 Cognitive Tasks The signal r(t) in the computational circuits shown in Figs. 3A, 6A, and 7 can be motor, e.g., efference copy, or sensory, e.g., proprioceptive. As far as the circuits themselves are concerned, the key property of r(t) is not whether it is sensory or motor, but how it is correlated with the teaching signal. To the extent that “thoughts” can be represented in the form of a time-varying signal r(t), then they too can be processed by circuits using decorrelation learning. Thus in principle it seems computationally feasible for some regions of the cerebellum to participate in cognitive functions (e.g., Ito, 2008, 2012). Moreover, there is extensive anatomical, imaging and clinical evidence suggestive of such participation (for reviews see, e.g., Fatemi et al., 2012; Fuentes and Bastian, 2007; Gowen and Miall, 2007; Nicolson et al., 2001; Ramnani, 2006; Schmahmann, 2010). The problem however is that this evidence is not yet detailed enough to allow identification of putative circuits that could underlie cerebellar involvement in specific cognitive functions (see, e.g., Frith et al., 2000; Gowen and Miall, 2007; Glickstein et al., 2011). Typically, the exact regions of cerebellum involved, their inputs and outputs, and above all how the outputs affect climbing-fiber input, remain unknown. Here, we briefly outline, in the absence of these circuit details, two ways in which decorrelation learning could in principle contribute to a cerebellar role in cognitive processing.

6 Future Directions

First, the reafference-problem circuit of Fig. 3A can be speculatively applied to internal speech. The input r(t) to the circuit becomes preverbal thoughts, which are translated by an unknown dynamic process into internal speech n(t), perhaps for example to utilize the powers of various forms of verbal memory. In the circuit of Fig. 8, the cerebellum receives a copy of these thoughts, and learns to provide an estimate nest(t) of the resultant inner speech. This estimate is subtracted from the combined signal for inner speech n(t) and external speech s(t), thus producing an estimate sest(t) of the external signal. Learning proceeds exactly as in Fig. 3A until there is no longer any correlation between r(t) and sest(t), that is until nest(t) ¼ n(t) and the effects of inner speech are exactly canceled (cf. Ackermann et al., 2007; Scott, 2013). It can be conjectured that if this circuit were to fail, then internal speech would become confused with external signals, perhaps leading to illusions of voices in the head (e.g., Frith et al., 2000). Secondly, it has been reported that “histoanatomic abnormalities of the cerebellum are one of the most consistent neuroanatomic findings in the brains of autistic individuals . . .it is likely that abnormalities of the cerebellum contribute significantly to many of the clinical features of the disorder” (Fatemi et al., 2012, p. 779). Although the computational underpin of such a contribution is quite unclear, one possibility is suggested by a consistent feature of first-hand accounts of the experience of autism, namely disorders in sensory processing, both hypo- and hypersensitivity (e.g., Cesaroni and Garber, 1991). Both hypo- and hypersensitivity would be consistent with a difficulty in setting a threshold level for attended stimuli, which in turn would be a natural consequence of a failure in noise cancelation (e.g., Fig. 3A). In a whisking robot rat (Anderson et al., 2010, 2012), failure to solve the reafference problem means choosing either a low threshold for stimulus salience that produces unwanted “ghost” orients to predictable self-produced stimuli, or a high threshold that ensures significant external events are missed. The two choices

FIGURE 8 Predicting internal speech. This diagram is a relabeling of Fig. 3A. Preverbal thoughts are converted by a complex unknown process into internal speech, where they appear in combination with external speech sources. If these internal speech sensations are not canceled, they could lead to the illusions of voices in the head.

183

184

CHAPTER 7 Decorrelation Learning and the Cerebellum

lead to hyperactive and quiescent behaviors respectively. After noise cancelation, a single threshold effectively separates the two types of input. As far as we are aware, the possibility that sensory deficits in autism might be mediated by cerebellar–collicular connectivity has not previously been considered; for example a recent review concentrates solely on cerebro-cerebellar connectivity (Fatemi et al., 2012). If this pathway and mechanism were to be implicated in autism, it might lead to the development of convenient new animal models and experimental techniques based on the whisker noise-cancelation paradigm.

APPENDIX DERIVATION OF SUPERVISED-LEARNING RULE This rule for supervised learning has a number of names, for example covariance, delta, Widrow–Hoff, and least mean squares (LMS). It has been proposed not just for the cerebellum, but for artificial devices that use supervised learning, such as adaptive filters (Fig. 1B). If the desired output of the Purkinje cell (Fig. 1A) is zd(t) (Fig. 1B), we can define the neuron output error as the difference between actual and desired neuron output X e ðt Þ ¼ z ðt Þ  z d ðt Þ ¼ w i pi ðt Þ  z d ðt Þ (A1) The mean square output error over time interval T ð  2  1 t 0 þT e ¼ eðt Þ2 dt T t0

(A2)

provides a well-behaved measure of performance over that time (we use angle brackets to express expected or mean values). Hence, we can define a cost function E ðw Þ ¼

1  2 e 2

(A3)

which quantifies the performance of the neuron for any given value of the weight vector w ¼ (w1, w2, . . ., wn). The optimal weight estimate (in the sense of least squares, for these data) minimizes this cost function. While there are many direct techniques available to solve such minimization problems, we are looking specifically for a biologically plausible algorithm that can be implemented as a synaptic learning rule. For a learning rate parameter b, which is chosen positive and small enough, weight changes given by the gradient descent learning rule Dw i ¼ b

@E @w i

(A4)

are guaranteed to reduce the cost function (unless the gradient is zero, in which case we are already at the global minimum for the quadratic cost function we have chosen). The gradient of the cost function is

References

    @E 1 @e2 @e ¼ e ¼ hepi i ¼ @w i 2 @w i @w i

(A5)

hence, the gradient descent learning rule can be written as Dw i ¼ bhepi i

(A6)

Since this change in weight is proportional to the correlation of pi(t) and e(t), learning stops when parallel-fiber activity has zero correlation with climbing-fiber activity, hence, this process is also called decorrelation learning (Dean et al., 2002).

Acknowledgments Preparation of this article was supported by grants from the European Union (REALNET, 270434 FP7) and the EPSRC (EP/1032533/1).

References Ackermann, H., Mathiak, K., Riecker, A., 2007. The contribution of the cerebellum to speech production and speech perception: clinical and functional imaging data. Cerebellum 6, 202–213. Albus, J.S., 1971. A theory of cerebellar function. Math. Biosci. 10, 25–61. Anderson, S.R., Pearson, M.J., Pipe, A., Prescott, T., Dean, P., Porrill, J., 2010. Adaptive cancelation of self-generated sensory signals in a whisking robot. IEEE Trans. Robot. 26, 1065–1076. Anderson, S.R., Porrill, J., Pearson, M.J., Pipe, A., Prescott, T., Dean, P., 2012. An internal model architecture for novelty detection: implications for cerebellar and collicular roles in sensory processing. PLoS ONE 7, e44560. Andersson, G., Garwicz, M., Hesslow, G., 1988. Evidence for a GABA-mediated cerebellar inhibition of the inferior olive in the cat. Exp. Brain Res. 72, 450–456. Andreescu, C.E., Prestori, F., Brandalise, F., D’Errico, A., De Jeu, M.T.G., Rossi, P., Botta, L., Kohr, G., Perin, P., D’Angelo, E., De Zeeuw, C.I., 2011. NR2A subunit of the n-methyl d-aspartate receptors are required for potentiation at the mossy fiber to granule cell synapse and vestibulo-cerebellar motor learning. Neuroscience 176, 274–283. Apps, R., Hawkes, R., 2009. Cerebellar cortical organization: a one-map hypothesis. Nat. Rev. Neurosci. 10, 670–681. Badura, A., Schonewille, M., Voges, K., Galliano, E., Renier, N., Gao, Z.Y., Witter, L., Hoebeek, F.E., Chedotal, A., De Zeeuw, C.I., 2013. Climbing fiber input shapes reciprocity of Purkinje cell firing. Neuron 78, 700–713. Bastian, A.J., 2011. Moving, sensing and learning with cerebellar damage. Curr. Opin. Neurobiol. 21, 596–601. Bell, C., Bodznick, D., Montgomery, J., Bastian, J., 1997. The generation and subtraction of sensory expectations within cerebellum-like structures. Brain Behav. Evol. 50, 17–31. Bell, C.C., Han, V., Sawtell, N.B., 2008. Cerebellum-like structures and their implications for cerebellar function. Annu. Rev. Neurosci. 31, 1–24.

185

186

CHAPTER 7 Decorrelation Learning and the Cerebellum

Belmeguenai, A., Hosy, E., Bengtsson, F., Pedroarena, C.M., Piochon, C., Teuling, E., He, Q., Ohtsuki, G., De Jeu, M.T., Elgersma, Y., De Zeeuw, C.I., Jo¨rntell, H., Hansel, C., 2010. Intrinsic plasticity complements long-term potentiation in parallel fiber input gain control in cerebellar Purkinje cells. J. Neurosci. 30, 13630–13643. Bengtsson, F., Hesslow, G., 2006. Cerebellar control of the inferior olive. Cerebellum 5, 7–14. Bengtsson, F., Jirenhed, D.A., Svensson, P., Hesslow, G., 2007. Extinction of conditioned blink responses by cerebello-olivary pathway stimulation. NeuroReport 18, 1479–1482. Best, A.R., Regehr, W.G., 2009. Inhibitory regulation of electrically coupled neurons in the inferior olive is mediated by asynchronous release of GABA. Neuron 62, 555–565. Bhanpuri, N.H., Okamura, A.M., Bastian, A.J., 2013. Predictive modeling by the cerebellum improves proprioception. J. Neurosci. 33, 14301–14306. Blakemore, S.J., Wolpert, D., Frith, C., 2000. Why can’t you tickle yourself? Neuroreport 11, R11–R16. Blakemore, S.J., Frith, C.D., Wolpert, D.M., 2001. The cerebellum is involved in predicting the sensory consequences of action. NeuroReport 12, 1879–1884. Bower, J.M., 1997. Is the cerebellum sensory for motor’s sake or motor for sensory’s sake: the view from the whiskers of a rat? Prog. Brain Res. 114, 463–496. Bower, J.M., Parsons, L.M., 2003. Rethinking the ’lesser brain’. Sci. Am. 289, 40–47. Boyden, E.S., Katoh, A., Raymond, J.L., 2004. Cerebellum-dependent learning: the role of multiple plasticity mechanisms. Annu. Rev. Neurosci. 27, 581–609. Boyden, E.S., Katoh, A., Pyle, J.L., Chatila, T.A., Tsien, R.W., Raymond, J.L., 2006. Selective engagement of plasticity mechanisms for motor memory storage. Neuron 51, 823–834. Brooks, J.X., Cullen, K.E., 2013. The primate cerebellum selectively encodes unexpected selfmotion. Curr. Biol. 23, 947–955. Cerminara, N.L., Apps, R., Marple-Horvat, D.E., 2009. An internal model of a moving visual target in the lateral cerebellum. J. Physiol. Lond. 587, 429–442. Cesaroni, L., Garber, M., 1991. Exploring the experience of autism through firsthand accounts. J. Autism Dev. Disord. 21, 303–313. Chen, F.P., Evinger, C., 2006. Cerebellar modulation of trigeminal reflex blinks: interpositus neurons. J. Neurosci. 26, 10569–10576. Chettih, S.N., Mcdougle, S.D., Ruffolo, L.I., Medina, J.F., 2011. Adaptive timing of motor output in the mouse: the role of movement oscillations in eyelid conditioning. Front. Integr. Neurosci. 5, 72. Coesmans, M., Weber, J.T., De Zeeuw, C.I., Hansel, C., 2004. Bidirectional parallel fiber plasticity in the cerebellum under climbing fiber control. Neuron 44, 691–700. Cullen, K.E., 2004. Sensory signals during active versus passive movement. Curr. Opin. Neurobiol. 14, 698–706. D’Angelo, E., De Zeeuw, C.I., 2009. Timing and plasticity in the cerebellum: focus on the granular layer. Trends Neurosci. 32, 30–40. Dean, P., Porrill, J., 2010. The cerebellum as an adaptive filter: a general model? Funct. Neurol. 25, 1–8. Dean, P., Porrill, J., 2011. Evaluating the adaptive-filter model of the cerebellum. J. Physiol. Lond. 589, 3459–3470. Dean, P., Porrill, J., Stone, J.V., 2002. Decorrelation control by the cerebellum achieves oculomotor plant compensation in simulated vestibulo-ocular reflex. Pro. Biol. Sci. 269, 1895–1904.

References

Dean, P., Porrill, J., Ekerot, C.F., Jo¨rntell, H., 2010. The cerebellar microcircuit as an adaptive filter: experimental and computational evidence. Nat. Rev. Neurosci. 11, 30–43. Dean, P., Anderson, S.R., Porrill, J., Jo¨rntell, H., 2013. Adaptive-filter model of cerebellar zone C3: possible basis for safe limb control. J. Physiol. 591, 5459–5474. De Zeeuw, C.I., Yeo, C.H., 2005. Time and tide in cerebellar memory formation. Curr. Opin. Neurobiol. 15, 667–674. Dow, R.S., Moruzzi, G., 1958. The Physiology and Pathology of the Cerebellum. University of Minnesota Press, Minneapolis, MN. Eccles, J.C., Ito, M., Szenta´gothai, J., 1967. The Cerebellum as a Neuronal Machine. SpringerVerlag, Berlin. Ekerot, C.F., Jo¨rntell, H., 2003. Parallel fiber receptive fields: a key to understanding cerebellar operation and learning. Cerebellum 2, 101–109. Ekerot, C.-F., Jo¨rntell, H., Garwicz, M., 1995. Functional relation between corticonuclear input and movements evoked on microstimulation in cerebellar nucleus interpositus anterior in the cat. Exp. Brain Res. 106, 365–376. Ekerot, C.F., Garwicz, M., Jo¨rntell, H., 1997. The control of forelimb movements by intermediate cerebellum. Prog. Brain Res. 114, 423–429. Fatemi, S.H., Aldinger, K.A., Ashwood, P., Bauman, M.L., Blaha, C.D., Blatt, G.J., Chauhan, A., Chauhan, V., Dager, S.R., Dickson, P.E., Estes, A.M., Goldowitz, D., Heck, D.H., Kemper, T.L., King, B.H., Martin, L.A., Millen, K.J., Mittleman, G., Mosconi, M.W., Persico, A.M., Sweeney, J.A., Webb, S.J., Welsh, J.P., 2012. Consensus paper: pathological role of the cerebellum in autism. Cerebellum 11, 777–807. Frith, C.D., Blakemore, S., Wolpert, D.M., 2000. Explaining the symptoms of schizophrenia: abnormalities in the awareness of action. Brain Res. Rev. 31, 357–363. Fuentes, C.T., Bastian, A.J., 2007. ’Motor cognition’—what is it and is the cerebellum involved? Cerebellum 6, 232–236. Fujita, M., 1982. Adaptive filter model of the cerebellum. Biol. Cybern. 45, 195–206. Gao, Z.Y., Van Beugen, B.J., De Zeeuw, C.I., 2012. Distributed synergistic plasticity and cerebellar learning. Nat. Rev. Neurosci. 13, 619–635. Glickstein, M., Voogd, J., 1995. Lodewijk Bolk and the comparative anatomy of the cerebellum. Trends Neurosci. 18, 206–211. Glickstein, M., Strata, P., Voogd, J., 2009. Cerebellum: history. Neuroscience 162, 549–559. Glickstein, M., Sultan, F., Voogd, J., 2011. Functional localization in the cerebellum. Cortex 47, 59–80. Gomi, H., Kawato, M., 1992. Adaptive feedback control models of the vestibulocerebellum and spinocerebellum. Biol. Cybern. 68, 105–114. Gowen, E., Miall, R.C., 2007. The cerebellum and motor dysfunction in neuropsychiatric disorders. Cerebellum 6, 268–279. Grossberg, S., Schmajuk, N.A., 1989. Neural dynamics of adaptive timing and temporal discrimination during associative learning. Neural Netw. 2, 79–102. Haith, A., Vijayakumar, S., 2009. Implications of different classes of sensorimotor disturbance for cerebellar-based motor learning models. Biol. Cybern. 100, 81–95. Herreros, I., Verschure, P.F.M.J., 2013. Nucleo-olivary inhibition balances the interaction between the reactive and adaptive layers in motor control. Neural Netw. 47, 64–71. Hesslow, G., 1986. Inhibition of inferior olivary transmission by mesencephalic stimulation in the cat. Neurosci. Lett. 63, 76–80.

187

188

CHAPTER 7 Decorrelation Learning and the Cerebellum

Hesslow, G., Ivarsson, M., 1996. Inhibition of the inferior olive during conditioned responses in the decerebrate ferret. Exp. Brain Res. 110, 36–46. Hesslow, G., Yeo, C.H., 2002. The functional anatomy of skeletal conditioning. In: Moore, J.W. (Ed.), A Neuroscientist’s Guide to Classical Conditioning. Springer, New York, NY. Hesslow, G., Jirenhed, D.A., Rasmussen, A., Johansson, F., 2013. Classical conditioning of motor responses: what is the learning mechanism? Neural Netw. 47, 81–87. Hopp, J.J., Fuchs, A.F., 2004. The characteristics and neuronal substrate of saccadic eye movement plasticity. Prog. Neurobiol. 72, 27–53. Imamizu, H., 2010. Prediction of sensorimotor feedback from the efference copy of motor commands: a review of behavioral and functional neuroimaging studies. Jpn. Psychol. Res. 52, 107–120. Imamizu, H., Kuroda, T., Miyauchi, S., Yoshioka, T., Kawato, M., 2003. Modular organization of internal models of tools in the human cerebellum. Proc. Natl. Acad. Sci. U. S. A. 100, 5461–5466. Isope, P., Barbour, B., 2002. Properties of the cerebellar granule cell-Purkinje cell synapse: an anatomical and functional study. FENS Abstr. 081, 1. Ito, M., 1970. Neurophysiological aspects of the cerebellar motor control system. Int. J. Neurol. (Montevideo) 7, 162–176. Ito, M., 1984. The Cerebellum and Neural Control. Raven Press, New York, NY. Ito, M., 1997. Cerebellar microcomplexes. Int. Rev. Neurobiol. 41, 475–487. Ito, M., 2001. Cerebellar long-term depression: characterization, signal transduction, and functional roles. Physiol. Rev. 81, 1143–1195. Ito, M., 2008. Control of mental activities by internal models in the cerebellum. Nat. Rev. Neurosci. 9, 304–313. Ito, M., 2012. The Cerebellum: Brain for an Implicit Self. FT Press, Upper Saddle River, NJ. Ito, M., Sakurai, M., Tongroach, P., 1982. Climbing fibre induced depression of both mossy fibre responsiveness and glutamate sensitivity of cerebellar Purkinje cells. J. Physiol. Lond. 324, 113–134. Izawa, J., Riscimagna-Hemminger, S.E., Shadmehr, R., 2012. Cerebellar contributions to reach adaptation and learning sensory consequences of action. J. Neurosci. 32, 4230–4239. Jacobson, G.A., Rokni, D., Yarom, Y., 2008. A model of the olivo-cerebellar system as a temporal pattern generator. Trends Neurosci. 31, 617–625. Jirenhed, D.A., Bengtsson, F., Hesslow, G., 2007. Acquisition, extinction, and reacquisition of a cerebellar cortical memory trace. J. Neurosci. 27, 2493–2502. Jirenhed, D.A., Bengtsson, F., Jo¨rntell, H., 2013. Parallel fiber and climbing fiber responses in rat cerebellar cortical neurons in vivo. Front. Syst. Neurosci. 7, Article 16. Jo¨rntell, H., Ekerot, C.F., 2002. Reciprocal bidirectional plasticity of parallel fiber receptive fields in cerebellar Purkinje cells and their afferent interneurons. Neuron 34, 797–806. Jo¨rntell, H., Ekerot, C.F., 2003. Receptive field plasticity profoundly alters the cutaneous parallel fiber synaptic input to cerebellar interneurons in vivo. J. Neurosci. 23, 9620–9631. Jo¨rntell, H., Ekerot, C.F., 2011. Receptive field remodeling induced by skin stimulation in cerebellar neurons in vivo. Front. Neural Circuits 5, Article 3. Jo¨rntell, H., Hansel, C., 2006. Synaptic memories upside down: bidirectional plasticity at cerebellar parallel fiber-Purkinje cell synapses. Neuron 52, 227–238. Jo¨rntell, H., Bengtsson, F., Schonewille, M., De Zeeuw, C.I., 2010. Cerebellar molecular layer interneurons—computational properties and roles in learning. Trends Neurosci. 33, 524–532.

References

Ke, M.C., Guo, C.C., Raymond, J.L., 2009. Elimination of climbing fiber instructive signals during motor learning. Nat. Neurosci. 12, 1171–1179. Kim, J.J., Krupa, D.J., Thompson, R.F., 1998. Inhibitory cerebello-olivary projections and blocking effect in classical conditioning. Science 279, 570–572. Knolle, F., Schroger, E., Baess, P., Kotz, S.A., 2012. The cerebellum generates motor-to-auditory predictions: ERP lesion evidence. J. Cogn. Neurosci. 24, 698–706. Lenz, A., Anderson, S.R., Pipe, A.G., Melhuish, C., Dean, P., Porrill, J., 2009. Cerebellar inspired adaptive control of a compliant robot actuated by pneumatic artificial muscles. IEEE Trans. Syst. Man Cyber. B Cyber. 39, 1420–1433. Lepora, N.F., Mavritsaki, E., Porrill, J., Yeo, C.H., Evinger, C., Dean, P., 2007. Evidence from retractor bulbi EMG for linearised motor control of conditioned nictitating membrane responses. J. Neurophysiol. 98, 2074–2088. Lepora, N., Porrill, J., Yeo, C.H., Dean, P., 2010. Sensory prediction or motor control? Application of Marr-Albus models of cerebellar function to classical conditioning. Front. Comput. Neurosci. 4, 1–16. Lev-Ram, V., Wong, S.T., Storm, D.R., Tsien, R.Y., 2002. A new form of cerebellar long-term potentiation is postsynaptic and depends on nitric oxide but not cAMP. Proc. Natl. Acad. Sci. U. S. A. 99, 8389–8393. Lev-Ram, V., Mehta, S.B., Kleinfeld, D., Tsien, R.Y., 2003. Reversing cerebellar long-term depression. Proc. Natl. Acad. Sci. U. S. A. 100, 15989–15993. Llina´s, R.R., Leznik, E., Makarenko, V.I., 2004. The olivo-cerebellar circuit as a universal motor control system. IEEE J. Oceanic Eng. 29, 631–639. Mackintosh, N.J., 1974. The Psychology of Animal Learning. Academic Press Inc, London. Marr, D., 1969. A theory of cerebellar cortex. J. Physiol. Lond. 202, 437–470. Marzban, H., Hawkes, R., 2011. On the architecture of the posterior zone of the cerebellum. Cerebellum 10, 422–434. Mauk, M.D., Ruiz, B.P., 1992. Learning-dependent timing of Pavlovian eyelid responses— differential conditioning using multiple interstimulus intervals. Behav. Neurosci. 106, 666–681. Mcelvain, L.E., Bagnall, M.W., Sakatos, A., Du Lac, S., 2010. Bidirectional plasticity gated by hyperpolarization controls the gain of postsynaptic firing responses at central vestibular nerve synapses. Neuron 68, 763–775. Medina, J.F., 2011. The multiple roles of Purkinje cells in sensori-motor calibration: to predict, teach and command. Curr. Opin. Neurobiol. 21, 616–622. Medina, J.F., Garcia, K.S., Nores, W.L., Taylor, N.M., Mauk, M.D., 2000a. Timing mechanisms in the cerebellum: testing predictions of a large-scale computer simulation. J. Neurosci. 20, 5516–5525. Medina, J.F., Nores, W.L., Ohyama, T., Mauk, M.D., 2000b. Mechanisms of cerebellar learning suggested by eyelid conditioning. Curr. Opin. Neurobiol. 10, 717–724. Medina, J.F., Nores, W.L., Mauk, M.D., 2002. Inhibition of climbing fibres is a signal for the extinction of conditioned eyelid responses. Nature 416, 330–333. Menzies, J.R.W., Porrill, J., Dutia, M., Dean, P., 2010. Synaptic plasticity in medial vestibular nucleus neurons: comparison with computational requirements of VOR adaptation. PLoS ONE 5, e13182. Miall, R.C., Christensen, L.O., Cain, O., Stanley, J., 2007. Disruption of state estimation in the human lateral cerebellum. PLoS Biol. 5, 2733–2744. Montgomery, J.C., Bodznick, D., Yopak, K.E., 2012. The cerebellum and cerebellum-like structures of cartilaginous fishes. Brain Behav. Evol. 80, 152–165.

189

190

CHAPTER 7 Decorrelation Learning and the Cerebellum

Moore, J.W., Gormezano, I., 1961. Yoked comparisons of instrumental and classical eyelid conditioning. J. Exp. Psychol. 62, 552–559. Moore, J.W., Desmond, J.E., Berthier, N.E., 1989. Adaptively timed conditioned responses and the cerebellum: a neural network approach. Biol. Cybern. 62, 17–28. Mostofi, A., Holtzman, T., Grout, A.S., Yeo, C.H., Edgley, S.A., 2010. Electrophysiological localization of eyeblink-related microzones in rabbit cerebellar cortex. J. Neurosci. 30, 8920–8934. Nicholson, D.A., Freeman, J.H., 2003. Addition of inhibition in the olivocerebellar system and the ontogeny of a motor memory. Nat. Neurosci. 6, 532–537. Nicolson, R.I., Fawcett, A.J., Dean, P., 2001. Developmental dyslexia: the cerebellar hypothesis. Trends Neurosci. 24, 508–511. Nisimaru, N., Mittal, C., Shirai, Y., Sooksawate, T., Anandaraj, P., Hashikawa, T., Nagao, S., Arata, A., Sakurai, T., Yamamoto, M., Ito, M., 2013. Orexin-neuromodulated cerebellar circuit controls redistribution of arterial blood flows for defense behavior in rabbits. Proc. Natl. Acad. Sci. U. S. A. 110, 14124–14131. Paige, G.D., 1983. Vestibulo-ocular reflex and its interaction with visual following mechanisms in the squirrel monkey. I. Response characteristics in normal animals. J. Neurophysiol. 49, 134–151. Pellegrini, J.J., Evinger, C., 1997. Role of cerebellum in adaptive modification of reflex blinks. Learn. Mem. 4, 77–87. Porrill, J., Dean, P., 2007a. Cerebellar motor learning: when is cortical plasticity not enough? PLoS Comput. Biol. 3, 1935–1950. Porrill, J., Dean, P., 2007b. Recurrent cerebellar loops simplify adaptive control of redundant and nonlinear motor systems. Neural Comput. 19, 170–193. Porrill, J., Dean, P., 2008. Silent synapses, LTP and the indirect parallel-fibre pathway: computational consequences of optimal noise processing. PLoS Comput. Biol. 4, e1000085. Porrill, J., Dean, P., Stone, J.V., 2004. Recurrent cerebellar architecture solves the motor error problem. Pro. Biol. Sci. 271, 789–796. Porrill, J., Dean, P., Anderson, S.R., 2013. Adaptive filters and internal models: multilevel description of cerebellar function. Neural Netw. 47, 134–149. Prestori, F., Bonardi, C., Mapelli, L., Lombardo, P., Goselink, R., De Stefano, M.E., Gandolfi, D., Mapelli, J., Bertrand, D., Schonewille, M., De Zeeuw, C., D’Angelo, E., 2013. Gating of long-term potentiation by nicotinic acetylcholine receptors at the cerebellum input stage. PLoS ONE 8, e64828. Ramnani, N., 2006. The primate cortico-cerebellar system: anatomy and function. Nat. Rev. Neurosci. 7, 511–522. Rasmussen, A., Jirenhed, D.A., Hesslow, G., 2008. Simple and complex spike firing patterns in Purkinje cells during classical conditioning. Cerebellum 7, 563–566. Rasmussen, A., Jirenhed, D.A., Zucca, R., Johansson, F., Svensson, P., Hesslow, G., 2013. Number of spikes in climbing fibers determines the direction of cerebellar learning. J. Neurosci. 33, 13436–13440. Raymond, J.L., Lisberger, S.G., 1998. Neural learning rules for the vestibulo-ocular reflex. J. Neurosci. 18, 9112–9129. Requarth, T., Sawtell, N.B., 2011. Neural mechanisms for filtering self-generated sensory signals in cerebellum-like circuits. Curr. Opin. Neurobiol. 21, 602–608. Rescorla, R.A., Wagner, A.R., 1972. A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and non-reinforcement. In: Black, A.H., Prokasy, W.F.

References

(Eds.), Classical conditioning II: current research and theory. Appleton-Century-Crofts, New York, NY. Roberts, P.D., Bell, C.C., 2000. Computational consequences of temporally asymmetric learning rules: II. Sensory image cancellation. J. Comput. Neurosci. 9, 67–83. Roth, M.J., Synofzik, M., Lindner, A., 2013. The cerebellum optimizes perceptual predictions about external sensory events. Curr. Biol. 23, 930–935. Roy, J.E., Cullen, K.E., 2004. Dissociating self-generated from passively applied head motion: neural mechanisms in the vestibular nuclei. J. Neurosci. 24, 2102–2111. Sawtell, N.B., Williams, A., 2008. Transformations of electrosensory encoding associated with an adaptive filter. J. Neurosci. 28, 1598–1612. Schlerf, J., Ivry, R.B., Diedrichsen, J., 2012. Encoding of sensory prediction errors in the human cerebellum. J. Neurosci. 32, 4913–4922. Schmahmann, J.D., 2010. The role of the cerebellum in cognition and emotion: personal reflections since 1982 on the dysmetria of thought hypothesis, and its historical evolution from theory to therapy. Neuropsychol. Rev. 20, 236–260. Schonewille, M., Belmeguenai, A., Koekkoek, S.K., Houtman, S.H., Boele, H.J., Van Beugen, B.J., Gao, Z., Badura, A., Ohtsuki, G., Amerika, W.E., Hosy, E., Hoebeek, F.E., Elgersma, Y., Hansel, C., De Zeeuw, C.I., 2010. Purkinje cell-specific knockout of the protein phosphatase PP2B impairs potentiation and cerebellar motor learning. Neuron 67, 618–628. Schonewille, M., Gao, Z., Boele, H.J., Vinueza Veloz, M.F., Amerika, W.E., Simek, A.A., De Jeu, M.T., Steinberg, J.P., Takamiya., K., Hoebeek, F.E., Linden, D.J., Huganir, R.L., De Zeeuw, C.I., 2011. Reevaluating the role of LTD in cerebellar motor learning. Neuron 70, 43–50. Scott, M., 2013. Corollary discharge provides the sensory content of inner speech. Psychol. Sci. 24, 1824–1830. Sears, L.L., Steinmetz, J.E., 1991. Dorsal accessory olive activity diminishes during acquisition of the rabbit classically conditioned eyelid response. Brain Res. 545, 112–122. Shmuelof, L., Huang, V.S., Haith, A.M., Delnicki, R.J., Mazzoni, P., Krakauer, J.W., 2012. Overcoming motor “forgetting” through reinforcement of learned actions. J. Neurosci. 32, 14617–14622. Soetedjo, R., Fuchs, A.F., Kojima, Y., 2009. Subthreshold activation of the superior colliculus drives saccade motor learning. J. Neurosci. 29, 15213–15222. Sugihara, I., Shinoda, Y., 2004. Molecular, topographic, and functional organization of the cerebellar cortex: a study with combined aldolase C amd olivocerebellar labeling. J. Neurosci. 24, 8771–8785. Thieme, A., Thurling, M., Galuba, J., Burciu, R.G., Goricke, S., Beck, A., Aurich, V., Wondzinski, E., Siebler, M., Gerwig, M., Bracha, V., Timmann, D., 2013. Storage of a naturally acquired conditioned response is impaired in patients with cerebellar degeneration. Brain 136, 2063–2076. Thompson, R.F., Steinmetz, J.E., 2009. The role of the cerebellum in classical conditioning of discrete behavioral responses. Neuroscience 162, 732–755. Tseng, Y.W., Diedrichsen, J., Krakauer, J.W., Shadmehr, R., Bastian, A.J., 2007. Sensory prediction errors drive cerebellum-dependent adaptation of reaching. J. Neurophysiol. 98, 54–62. Voogd, J., 2011. Cerebellar zones: a personal history. Cerebellum 10, 334–350.

191

192

CHAPTER 7 Decorrelation Learning and the Cerebellum

Voogd, J., Ruigrok, T.J.H., 2004. The organization of the corticonuclear and olivocerebellar climbing fiber projections to the rat cerebellar vermis: the congruence of projection zones and the zebrin pattern. J. Neurocytol. 33, 5–21. Wang, S.S., Khiroug, L., Augustine, G.J., 2000. Quantification of spread of cerebellar longterm depression with chemical two-photon uncaging of glutamate. Proc. Natl. Acad. Sci. U. S. A. 97, 8635–8640. Widrow, B., Stearns, S.D., 1985. Adaptive Signal Processing. Prentice-Hall Inc, Engelwood Cliffs, NJ. Widrow, B., Glover, J.R., Mccool, J.M., Kaunitz, J., Williams, C.S., Hearn, R.H., Zeidler, J.R., Dong, E., Goodlin, R.C., 1975. Adaptive noise cancelling—principles and applications. Proc. IEEE 63, 1692–1716. Wolpert, D.M., Miall, R.C., Kawato, M., 1998. Internal models in the cerebellum. Trends Cogn. Sci. 2, 338–347.

CHAPTER

8

Modeling the Evolution of the Cerebellum: From Macroevolution to Function

Jeroen B. Smaers1 Department of Anthropology, Stony Brook University, Stony Brook, NY, USA 1 Corresponding author: Tel.: 631 632 7609; Fax: 631 632 9165, e-mail address: [email protected]

Abstract The purpose of this contribution is to explore how macroevolutionary studies of the cerebellum can contribute to theories on cerebellar function and connectivity. New approaches in modeling the evolution of biological traits have provided new insights in the evolutionary pathways that underlie cerebellar evolution. These approaches reveal patterns of coordinated size changes among brain structures across evolutionary time, demonstrate how particular lineages/species stand out, and what the rate and timing of neuroanatomical changes were in evolutionary history. Using these approaches, recent studies demonstrated that changes in the relative size of the posterior cerebellar cortex and associated cortical areas indicate taxonomic differences in great apes and humans. Considering comparative differences in behavioral capacity, macroevolutionary results are discussed in the context of theories on cerebellar function and learning.

Keywords brain evolution, posterior cerebellum, evolutionary neuroscience, comparative method, phylogenetic mapping, great apes, prefrontal cortex

1 CEREBELLUM, LEARNING, AND HUMAN EVOLUTION Understanding human abilities and adaptations in the context of the vast variability of behaviors observed in the animal kingdom is the subject of many fields in the life sciences. The acquisition and accumulation of motor skills can be viewed as a central feature in characterizing human’s unique abilities. Much of the “culture” that humankind has accumulated is directly or indirectly based on the ability to develop and combine complex motor sequences. From speech, making stone tools, handling complex machinery, to driving cars and navigating computers; all is predicated on the ability to acquire, accumulate and flexibly control, and anticipate motor actions. Progress in Brain Research, Volume 210, ISSN 0079-6123, http://dx.doi.org/10.1016/B978-0-444-63356-9.00008-X © 2014 Elsevier B.V. All rights reserved.

193

194

CHAPTER 8 Modeling the Evolution of the Cerebellum

Because the brain is the substrate of all behavior, neuroscientists are in a prominent position to contribute to discussions on the origin and nature of humans’ unique abilities to manipulate their environment. With its role in motor and cognitive function, the cerebellum plays a key role in this endeavor. Recent work has highlighted the role of the cerebellum in language function (Booth et al., 2007; Leiner et al., 1993; Stoodley and Schmahmann, 2009), working memory (Ding et al., 2012; Vandervert et al., 2007), executive function (Koziol et al., 2012; Strick et al., 2009), and the development of internal control models of behavioral learning (Imamizu et al., 2000, 2003; Koziol et al., 2012). These studies have helped elevate the status of the cerebellum as a “supervised learning machine” (Koziol et al., 2014). A full understanding of how the cerebellum influences human function and adaptation can, however, only come from adding a comparative perspective to the equation. After all, it is only by knowing the other that one knows the self. Comparative studies contextualize brain–behavior associations observed in humans in a broader taxonomic context. By comparing brain–behavior interactions between different species, comparative studies can be used to falsify or specify existing hypotheses or develop novel hypotheses about the nature of human brain–behavior associations. Detailed comparative information can also be used to infer evolution. Considering that the variability observed today is the necessary result of foregone changes, many comparative biologists aim to unravel how extant variation can inform us on how traits have evolved across millions of years. Applied to the study of the brain, such evolutionary studies provide information on how brain–behavior associations have changed across millions of years of evolution. Given the vast anatomical and behavioral variability present in the animal kingdom, it is clear that comparative studies potentially harbor a wealth of useful information. Researchers concerned with expanding the taxonomic and evolutionary scope of neuroscientific studies are often referred to as “evolutionary neuroscientists.”

2 EVOLUTIONARY NEUROSCIENCE AND ITS ADOPTION OF THE CEREBELLUM Evolutionary neuroscientists aim to understand the comparative and evolutionary context of the human brain. To accomplish this goal, they quantify different aspects of relevant brain areas/structures (e.g., volume, neural density, gray level index, cortical thickness) across species. To infer the evolutionary patterns that underlie crossspecies variation in these quantitative aspects, evolutionary neuroscientists have turned to phylogenetic comparative methods. These methods consider the genetic relatedness between the species under study (summarized as a phylogenetic tree or family tree) to infer scaling trends between brain structures, coevolutionary patterns among brain structures and/or detailed evolutionary pathways that underlie present-day variation. The information that results from these studies provides a detailed macroevolutionary context of the human brain that allows answering fundamental questions about its putative uniqueness in the natural world.

2 Evolutionary Neuroscience and Its Adoption of the Cerebellum

Traditionally, neuroscientists have focused on neocortical specializations to characterize what makes the human brain unique. Since the landmark publication of The Cerebellum and Cognition (Schmahmann, 1997), the cerebellum has increasingly been recognized as a fundamental contributor to human behavior and to cognition in general. Recent work has, for example, described the cerebellum as the “missing link” in the neurological underpinnings of many cognitive domains (MacLeod, 2012). With the growing awareness of the role of the cerebellum in human cognition, evolutionary neuroscientists have gradually turned their attention to studying this brain structure from a macroevolutionary perspective. In many ways, the cerebellum is an ideal structure for broad scale comparative analyses. Cerebellar morphology is uniform across all mammals, comprising of the same 10 lobules (Larsell, 1970). Cerebellar lobules are topographically organized with the cerebral cortex in that each lobule connects with specific cortical areas. This topographic organization comprises proportionate dedicated surface areas (Buckner et al., 2011) and number of neurons (Herculano-Houzel, 2010) such that cerebellar organization can be considered to comprise a map of much of the rest of the brain. This property of cortico-cerebellar organization implies that the evolution of the cerebellum indicates much more than the evolution of the cerebellum per se: crossspecies variation in cerebellar lobules also reveals variation in its associated cortical areas. Considering the homologous nature of cerebellar anatomy, function, and development in mammals, putative differences in selective investment across different lineages can validly be attributed to differences in selective investment in behavioral capacities. Being largely constrained to investigating anatomical information across species, (macro)evolutionary neuroscientists have, however, been limited in their contribution to discussions about the function of particular brain areas. To infer function based on variation in quantitative aspects of brain anatomy, evolutionary neuroscientists have principally relied on lesion studies in nonhuman primates (mostly macaque monkeys), functional neuroimaging studies in humans, and relevant principals of brain–behavior interactions. Invasive approaches provide the most detailed anatomical information on particular brain areas, but for ethical reasons they cannot be applied to humans and great apes. Recent advances in neuroimaging have exponentially increased our understanding of the functional organization of the human brain, but for practical reasons, they cannot be applied to most nonhuman primates. Despite their respective limitations, these approaches provide the most detailed account of brain–behavior interactions in humans and their closest relatives and thus provide the most relevant context for evolutionary neuroscientists to produce functional interpretations based on cross-species anatomical variation (Frey et al., 2011; Kelly et al., 2010; Mackey and Petrides, 2010; Petrides et al., 2012). When faced with large comparative datasets across many different species evolutionary neuroscientists have also relied on the “principle of proper mass” (Jerison, 1973). This principle states that there is a strong correlation between behavior and the size of a particular area/structure. In other words, if a brain area is demonstrated to be larger relative to brain size in a particular species, this species may indicate a greater capacity for the behavior supported by that area. This principle has been supported by a wide variety

195

196

CHAPTER 8 Modeling the Evolution of the Cerebellum

of studies across the animal kingdom (DeVoogd et al., 1993; Huber et al., 1997; Iwaniuk and Wylie, 2007; Smith et al., 2010) and has also been found to apply to human interindividual differences (Draganski et al., 2004; Fleming et al., 2010; Maguire et al., 2000). The principle of proper mass is of particular relevance to the comparative study of the cerebellum because the volume of cerebellar lobules is a linear function of its number or neurons (Herculano-Houzel, 2010) and by extension its relative investment in particular cortico-cerebellar information processing loops. With recent advances in comparative methods and renewed efforts to collect neuroanatomical data across a wide range of primate species (Bauernfeind et al., 2013; de Sousa et al., 2013; MacLeod et al., 2003; Raghanti et al., 2011; Sherwood et al., 2005; Smaers et al., 2010, 2011a,b, 2013), increasingly detailed information on the macroevolutionary context of the human brain is now becoming available. By providing detailed evolutionary pathways of brain structures and systems, evolutionary neuroscientists are getting into a position where they can contribute to debates on the putative function of brain structures/areas. With more detailed knowledge of the evolutionary pathways underlying cerebellar variation across species, observed behavioral differences between species can be used as comparative empirical tests of hypotheses on cerebellar function (Bril et al., 2012). Two main aspects have, however, hampered the inclusion of macroevolutionary studies in the broader neuroscientific community. First, the data collected across a wide range of different species necessarily has a lower anatomical and functional resolution. For example, whereas tracer studies in a single species can identify connectivity patterns between precisely defined areas, such detailed anatomical information is not available across many species where it is often limited to more gross-anatomical comparisons. Pertaining to the cerebellum, however, recent studies have greatly improved the anatomical resolution of available data (MacLeod et al., 2003; Smaers et al., 2011b, 2013). Second, macroevolutionary data require a different analytical approach. Because species cannot be regarded as independent data points, standard parametric tests should not be applied (Felsenstein, 1985; Harvey and Pagel, 1991). Specialized comparative methods have been developed to deal with issues related to weighting data points for phylogenetic relatedness. Although these methods have become standard in comparative biology, they often require specific expertise hampering their incorporation into other life sciences. The purpose of this contribution is to narrow the gap between neuroscience and macroevolutionary studies of the brain by exploring how results of macroevolutionary studies of the cerebellum can contribute to recent theories on cerebellar function and connectivity. A summary of the main conclusions from comparative studies of cerebellar connectivity will be presented, followed by an overview of the main macroevolutionary approaches in brain evolution and their results pertaining to the evolution of the cerebellum. Lastly, macroevolutionary results will be discussed in the context of theories on cerebellar function and learning.

3 Comparative Studies of Cerebellar Connectivity

3 COMPARATIVE STUDIES OF CEREBELLAR CONNECTIVITY Although neuroscientists are primarily interested in the anatomy and function of the human brain, they arguable have more detailed anatomical information about the macaque brain. The reason being that invasive techniques that provide a wealth of anatomical knowledge (such as lesion studies, single-cell electrophysiology, and tracer studies) cannot ethically be applied in humans. Recent advances in noninvasive neuroimaging techniques (Amunts et al., 2013; Buckner et al., 2013) have provided new opportunities to assess the congruence between the macaque and human brain and assess which aspects of neural organization have been phylogenetically conserved or derived in humans (Rilling, 2014). Human and nonhuman studies on the cerebellum have primarily focused on mapping the connectivity patterns of the cerebellar cortex with the cerebrum. The argument is that, because the cytoarchitecture of the cerebellar cortex is uniform, the functional topography of the cerebellum can only be highlighted through its connectional topography with the cerebrum (Ramnani, 2012). Early tracer studies in the macaque brain identified dense projections between cortical motor areas and the cerebellum, but found little evidence of prefrontal projections. Considering that the main route through which the cerebral cortex supplies information to the cerebellum is through the pontine nuclei, Brodal (1978) and Glickstein et al. (1985) investigated the density of the projections from cortical motor areas and prefrontal areas to the pontine nuclei. Both concluded that the densest projections were found to arise in cortical motor areas, with few in prefrontal areas. The dominance of cortical motor projections with cerebellar lobules in the macaque brain is now generally agreed upon and considered to primarily affect lobules HIV, HV, HVI, HVIIB, and HVIII. Schmahmann and Pandya (1995, 1997), however, found the presence of terminal anterograde tracer label in prefrontal areas demonstrating that the cerebrocerebellar system also incorporates associative cerebral regions. Strick and colleagues (Dum and Strick, 2003; Kelly and Strick, 2003; Middleton and Strick, 2001) further suggested that the cerebellar output channels via the dentate nucleus comprises a distinction in connectivity between the dorsal (with primary motor cortex) and ventral (with prefrontal areas) parts. Cerebellar connectivity with prefrontal areas was found to primarily involve Crus I and Crus II. Prefrontal contributions to the corticocerebellar system have now become more widely accepted (Ramnani, 2006, 2012). Importantly, the above-described work in nonhuman primates corresponds to work in humans using fMRI. O’Reilly et al. (2010) found that resting state activity patterns in the primary motor cortex were coherent with those in lobules HV, HVI, and HVIII, and those in prefrontal areas with HVIIA, Crus I, and Crus II. Similarly, Krienen and Buckner (2009) found an association between primary motor cortex and HV and HVIIIB, and between three prefrontal areas and lobules VI, VIIB, Crus I, and Crus II. The consistency in the patterns of cortico-cerebellar connectivity observed in human and nonhuman studies suggests that cerebellar anatomical topography is largely conserved among primates. To infer the extent to which humans stand out compared to nonhuman primates in aspects of cortico-cerebellar connectivity, Ramnani et al. (2006) investigated cortical

197

198

CHAPTER 8 Modeling the Evolution of the Cerebellum

contributions to the cortico-ponto-cerebellar system using DT-MRI tractography. Comparing macaque monkeys to humans, they confirmed the dominant contribution of cortical motor areas in the macaque brain, but additionally demonstrated a relatively large prefrontal contribution to the human cortico-ponto-cerebellar system. Balsters and colleagues (Balsters et al., 2010) further pushed the boundaries of comparative cerebellar analysis by also including information for the chimpanzee and capuchin monkey. Rather than using tracer labels to infer connectivity, Balsters and colleagues investigated the relative expansion of prefrontal projecting cerebellar lobules in humans versus chimpanzees and capuchins. The study found that each species constitutes its own grade with increasing relative volumes for Crus I and Crus II in capuchins, chimpanzees, and humans. These results suggest that prefrontal projecting lobules have gradually expanded toward the human lineage. Although limited in comparative scope (number of different species included in the study), these studies have provided a wealth of information for evolutionary neuroscientists (Rilling, 2014). But it is the limited comparative scope that has necessitated macroevolutionary neuroscientists to rely on detailed functional and anatomical information from few species to interpret less functionally and anatomically specific information across many species. This depth versus breadth trade-off in data availability has particularly affected functional interpretations. The evolutionary assumption that comparing a macaque (or chimpanzee) brain to a human provides valid information about the evolution of the human brain is crude at best. With the human–chimpanzee comparison, for example, it is often forgotten that they both share a common ancestor around 6 million years ago and thus that chimpanzees have evolved an equal amount of time as humans have since their last common ancestor. In other words, chimpanzees may indicate evolutionary patterns that are unique to their lineage, confounding the comparison with humans (Sayers et al., 2012). With more phylogenetic distance between humans and macaques (around 30 million years), it is clear that this comparison is even more problematic. It is this gap in comparative neuroanatomy that many evolutionary neuroscientists have sought to remedy by collecting data across many different species. The advantage of these studies is clear, as comparisons across many different species allow inferring what is unique about every individual lineage within a broader evolutionary context. Understandably, such a macroevolutionary context provides a more detailed picture on the evolutionary patterns that underlie observed cross-species variation and thus has the potential to increase the evolutionary resolution of inferences about how the human brain may stand out.

4 MACROEVOLUTIONARY STUDIES OF THE CEREBELLUM 4.1 (Phylogenetic) Scaling: Who Has the Biggest Cerebellum? The first element to tackle when considering a sample of many different species is how to interpret the vast diversity in the size of the brain and its subregions across species. The size of the primate brain can differ by a factor of 750 (humans: 1330 g;

4 Macroevolutionary Studies of the Cerebellum

mouse lemur: 1.78 g), while the size of individual brain structures can differ up to a factor of 1400 (human neocortex: 1007 g; mouse lemur: 0.74 g). In order to deduce patterns of relative size differences between species the traditional approach has been to scale brain structure sizes to each other or to overall brain size in a regression (this approach to studying brain evolution was first employed by Passingham, 1973). Such analyses reveal how particular brain structures keep pace with changes in other brain structures as the brain changes in overall size across different species. Employing this scaling approach to investigating trait differences among species has deep roots in biology (Huxley, 1932) and has become one of its most fundamental approaches to deducing the patterns of change that underlie comparative variation (Brown and West, 2000). Applied to brain structure evolution, it may reveal how different brain structures influence comparative differences in brain architecture. For example, in primates neocortical gray matter is found to scale to brain size with a scaling coefficient (the slope of the regression) roughly equal to 1 (1.03), while neocortical white matter scales with a coefficient higher than 1 (1.21) (Smaers et al., 2010). This indicates that in bigger-brained primates, neocortical gray matter is enlarged by a factor of 1 relative to smaller brained primates, while neocortical white matter is disproportionately enlarged by a factor of 1.2. The animal with the bigger brain thus ends up with a different brain architecture (i.e., disproportionately more neocortical white matter). This scaling approach reveals another useful element that can be tied to speciesspecific selective pressures. If a particular species (or clade) is shown to indicate a brain structure size higher than predicted given its primate scaling relationship (evidenced in a different intercept of the regression), it is inferred that this species has a higher relative size of this brain structure. Such a taxonomic difference in the mean value of a trait has been referred to as a “grade shift” (Pagel and Harvey, 1988). Anthropoid primates, for example, have been shown to have a larger neocortex given their brain size when compared to prosimians (Frahm et al., 1982). Recent studies on more detailed cortical structures have also used this approach to reveal human prefrontal specializations (Smaers et al., 2011a; Smaers, 2013; Sherwood and Smaers, 2013). Combining information on scaling patterns between brain structures and how particular species deviate from clade-general scaling patterns hereby provides a powerful analytical tool to investigating brain structure evolution. MacLeod et al. (2003) employed this technique to investigate the evolution of the cerebellar hemispheres. Scaling the size of the cerebellar hemispheres to the size of the vermis, apes demonstrate larger cerebellar hemispheres relative to the vermis. This grade shift was indicated to be uniform across all apes and thus likely to have evolved either in the common ancestor of all apes, or consistently throughout ape evolution. The enlarged cerebellar hemispheres in apes were subsequently linked to hominoid adaptations to frugivory and suspensory feeding (MacLeod et al., 2003). The analyses by MacLeod did not, however, consider that comparative data should not be analyzed using standard parametric tests. One potential issue with using species as independent data points in standard parametric tests such as a regression is that species are, by definition, not independent entities. Species are linked by the principle of

199

200

CHAPTER 8 Modeling the Evolution of the Cerebellum

common descent resulting in the fact that some species are closer related than others. Differential genetic relatedness implies an expectation of similarity whereby the degree of similarity depends on the level of shared ancestry and on the changes that have accumulated across generations. For example, the comparison between the brain size of a chimpanzee (
400 g) to that of a spider monkey (
90 g) cannot be considered as equivalent to that of a gibbon (
90 g) because chimpanzees share a common ancestor with capuchin monkeys
50 Mya, but only
20 Mya with gibbons. The gibbon is therefore expected to be more similar to a chimpanzee than a spider monkey. Advancements in gene-sequencing techniques have now lead to the availability of reliable phylogenies (Arnold et al., 2010) that can be used in open source statistical software packages to incorporate the expectation of similarity based on phylogenetic relatedness into standard parametric analyses (Orme et al., 2012; Revell, 2012). It is important to note that the inclusion of phylogenetic information does not always lead to improved results (Smaers, 2013). The phylogenetic signal in the data (the degree to which phylogenetic relatedness is expected to influence the result) may be nonexistent in which case a phylogenetic regression collapses into a standard regression (Freckleton et al., 2002). The most recent methods automatically incorporate the degree of phylogenetic dependence into the computation of regression coefficients, collapsing into a standard regression when the phylogenetic signal is zero and adjusting for the level of phylogenetic dependence if the signal is not zero. These methods provide a fundamental improvement in comparative statistical analyses because, if the phylogenetic signal is not zero, not taking into account phylogenetic relatedness results in inaccurate computations of scaling coefficients (Martins et al., 2002). Although MacLeod’s analyses (MacLeod, 2012; MacLeod et al., 2003) do not include phylogenetic information, a phylogenetic reanalysis of the same data confirms that apes indeed have larger cerebellar hemispheres compared to monkeys (Fig. 1A). These analyses confirm that the slope of the two groups is not different but the intercept is (slope for apes: 1.46 with a 95% confidence interval (CI) of 1.08:1.83; slope for monkeys: 1.44 with a 95% CI of 1.09:1.78; intercept for apes: 1.25 with a 95% CI of 0.58:1.92; intercept for monkeys: 0.29 with a 95% CI of 0.02:0.61). More specifically, these results indicate that the cerebellar hemisphere in apes is around two to three times larger relative to the vermis than in monkeys. For example, both the gibbon and the spider monkey have a brain size
90 cm3, but the ratio of cerebellar hemisphere volume to vermis volume is 4.5 for the gibbon (10.3 vs. 2.3 cm3) and 2.4 for the spider monkey (7.3 vs. 3.1 cm3). A similar analysis using data from recent work (Smaers et al., 2013) collecting volumes on the posterior lobules of the cerebellar hemisphere (the part of the cerebellar hemispheres that indicates the most intricate projections to the prefrontal cortex) confirms this ape grade shift (Fig. 1C) and similarly finds that the posterior cerebellar hemisphere in apes is 1.6–2.9 times larger relative to the vermis in apes. In both the Macleod and Smaers datasets, this grade shift is, however, not indicated when comparing the volume of the (posterior) cerebellar hemispheres to the volume of the rest of the brain (Fig. 1B and D), suggesting that the cerebellar increase in apes may be offset by a similar relative increase in another area encapsulated in “rest of brain.”

4 Macroevolutionary Studies of the Cerebellum

Cerebellar hemisphere

A

B 5

5

4

4

3

3

2

2

1

1

0

0 0.0

0.5

1.0

1.5

2.0

2.5

Posterior cerebellar cortex

3

4

5

6

7

D

C

4

4

3

3

2

2

1

1

1.0

1.5

2.0 2.5 3.0 Rest of cerebellum

3.5

4

5 6 Rest of brain

7

FIGURE 1 Scaling of whole (A and B) and posterior (C and D) cerebellar hemisphere volume to the volume of the rest of the cerebellum and the rest of the brain. Phylogenetic generalized least squares analysis (Orme et al., 2012) was used to account for phylogenetic relatedness. Data come from MacLeod et al. (2003) and Smaers et al. (2013).

Considering evidence for functional and anatomical connectivity between the posterior cerebellar hemispheres, prefrontal cortex, and cortical motor areas, these areas are likely candidates for those that offset the cerebellar grade shift when compared to “rest of brain.” If a similar grade shift emerges in these areas, it would provide support for an evolutionary link with the cerebellum. Figure 2A and B presents such analyses demonstrating that, although within the 95% CI of a log–log regression, human prefrontal cortex and underlying white matter is indicated to be 3.3 times larger than predicted relative to the rest of the neocortex and 2.3 times larger relative to the rest of the brain (predictions based on the monkey allometry). Respective values for the chimpanzee are 2.0 and 1.76; and for the gorilla 1.58 and 1.36. Considering the scaling pattern of prefrontal cortex þ posterior cerebellar cortex

201

B 4

5

3

4

2

3

1

2

0

1 3

4 5 Rest of Neocortex

6

C Prefrontal + posterior cerebellar cortex

Prefrontal

A

4

5 Rest of brain

6

7

5

4

3

2

1 3.5

4.0

4.5

5.0 5.5 6.0 Rest of brain

6.5

7.0

FIGURE 2 Scaling of prefrontal cortex volume to the volume of the rest of the neocortex (A) and the rest of the brain (B), and of the sum of the volumes of prefrontal cortex and posterior cerebellar cortex to the volume of the rest of the brain (C). Phylogenetic generalized least squares analysis (Orme et al., 2012) was used to account for phylogenetic relatedness. Prefrontal data come from Smaers et al. (2011a) and cerebellar data come from Smaers et al. (2013).

4 Macroevolutionary Studies of the Cerebellum

relative to the rest of brain (Fig. 2C), humans are indicated to have a prefrontoposterior cerebellar formation that is 1.86 times larger than predicted relative to the rest of the brain. These results thus suggest a marked increase in prefrontal volume in great apes and humans (see also Passingham and Smaers, 2014; Sherwood and Smaers, 2013; Smaers, 2013), aligning with the ape grade shift observed in cerebellar hemispheres and posterior cerebellar cortex.

4.2 Phylogenetic Correlations: Toward Patterns of Evolutionary Connectivity Phylogenetic scaling provides crucial information on how the size of individual brain structures change relative to each other across species, and which species indicate exceptional relative increases or decreases in individual brain structures. However, the brain works as a distributed system and in order to reveal the evolution of the systemic nature of the brain, a focus on individual brain structures is inherently limited. To capture more clearly how size changes in particular brain structures interact relative to size changes in other structures, evolutionary neuroscientists have turned to correlational analyses. Barton and Harvey (2000) demonstrated significant correlations between brain structures that have anatomical and functional links. Whiting and Barton (2003) subsequently focused more on the cortico-cerebellar network and found significant correlations between whole neocortex, whole cerebellar cortex, and the cerebellar nuclei, and between the thalamus and the cerebellar nuclei. More recent work has further subdivided whole neocortex and whole cerebellar cortex volume in more functionally relevant subregions allowing inferring more precise evolutionary patterns. Using data on the frontal cortex and the cerebellar hemispheres, Smaers et al. (2011b) demonstrated that two main correlation patterns underlie cortico-cerebellar evolution: one set including the frontal lobe and the basal ganglia, another including the nonfrontal cortex and the pons. Smaers et al. (2013) increased resolution even further by subdividing the frontal cortex into prefrontal cortex and frontal motor areas and delineating the part of the cerebellar hemispheres that has the closest functional associations with the prefrontal cortex, that is, its posterior lobe. Results from this study revealed a significant correlation between the posterior cerebellar hemispheres and the frontal motor areas, but not prefrontal cortex, across anthropoid primates. Another, perhaps more promising, approach than bivariate correlations is that of principal component analysis. This type of analysis deduces clusters of correlating variables from a multivariate dataset and allows quantifying the contribution of each cluster (or “component”) to explaining the overall variation observed in the data. This approach may provide a better fit for multivariate brain structure data because it better reflects the systemic interaction between different brain structures in which individual structures contribute to different information processing loops to various extents. As an approach commonly used in studies on the evolution of mammalian brain organization (de Winter and Oxnard, 2001; Finlay and Darlington, 1995; Reep et al., 2007; Willemet, 2012), Smaers and Soligo (2013) employed it with the aim of

203

204

CHAPTER 8 Modeling the Evolution of the Cerebellum

quantifying the contribution of various clusters of coevolving brain structures to explaining brain structure variation in anthropoid primates. Including information on 20 brain structures/areas (including the cerebellar hemispheres and hemispherespecific information on the prefrontal cortex and frontal motor areas), this study reveals two principal components relevant to cerebellar evolution: one that reflects the descending motor pathway (spinocerebellum, mesencephalon, and medulla) and one that links the cerebellar hemispheres to frontal motor areas. Together, these two clusters explained up to 10.6% of total brain structure variation, with the majority of overall variation explained by changes in the prefrontal cortex, striatum and hippocampal-entorhinal formation (totalling 46.1%) (Smaers and Soligo, 2013). These results suggest that, although the cerebellum plays a fundamental role in higher cognitive functioning and is integrally associated to the neocortex, the majority of changes in anthropoid brain evolution are telencephalic. Phylogenetic scaling analyses thus suggest that (great) apes and humans may indicate a coordinated relative increase of prefrontal cortex and prefrontal projecting cerebellar lobules. Phylogenetic correlation analysis does not provide a conclusive answer to this particular question. Phylogenetic correlations, however, only indicate patterns across the entire sample under study and do not provide information on the extent to which individual lineages align with the overall correlation pattern. In order to recognize coevolutionary patterns in particular lineages (e.g., in great apes), an approach with a higher evolutionary resolution is needed. For these reasons, Smaers and colleagues developed the approach of “phylogenetic mapping” (Smaers and Soligo, 2013; Smaers et al., 2012, 2013).

4.3 Phylogenetic Mapping: Inferring Detailed Patterns of Evolutionary History Comparative correlations provide useful insight into which brain structures have coevolved across an entire group of species. Correlations are however, per definition, average trends across entire datasets, with little information on the extent to which particular data points diverge from the average trend. Comparative correlations thus have the undesirable attribute that they do not pick up on whether the evolutionary trends in particular species align with or diverge from the more general evolutionary patterns observed in the sample. From a statistical point of view, the average trend may be most important, from an evolutionary point of view that may not be the case. Different species may follow different adaptive directions and the extent to which particular species deviate from clade-general patterns may reveal fundamental aspects of evolution. To reveal the extent to which particular species deviate from clade-general correlation patterns, recent work has developed an approach that comprises the comparison of lineage-specific evolutionary rates (Smaers et al., 2012). This approach infers the rate of change that occurs along each branch in the tree of life and estimates trait values for each ancestral node. This information is then visualized back on to the phylogenetic tree thereby effectively “mapping” the evolution of a trait along the branches of a phylogenetic tree (Smaers and Soligo, 2013; Smaers et al., 2013). This approach hereby provides

4 Macroevolutionary Studies of the Cerebellum

a detailed picture of the evolutionary history of a particular trait that includes rates of change, the evolutionary timing of instances of putative bursts of change, the occurrence of independent evolution (similar patterns of change occurring independently in distantly related species), and the extent to which each lineage aligns with or deviates from clade-general correlation patterns (Smaers et al., 2012). Using this approach, Smaers et al. (2011b) indicated that the ape ancestral branch and the human–chimpanzee clade invested heavily in the relative expansion of the frontal cortico-cerebellar formation. Combining phylogenetic mapping with results from principal components analyses and including information on subdivisions of the frontal lobe (frontal motor areas and prefrontal cortex), Smaers and Soligo (2013) demonstrated that the principal component characterized by high loadings of the cerebellar hemispheres and frontal motor areas is particularly pronounced in the ape and human ancestral branches. Although these components explained only up to 10.6% of overall variation in brain organization across anthropoid primates, phylogenetic mapping thus provided a more nuanced picture by revealing that the cortico-cerebellar formation was particularly increased in great apes and humans. Similarly, Smaers et al. (2013) were able to further refine the clade-general correlation between frontal motor areas, but not prefrontal cortex, and posterior cerebellum. Specifically, phylogenetic mapping revealed a marked trend of disproportionate coordinated increase of the prefrontal and posterior cerebellar hemispheres in apes, particularly in the ape ancestral lineage and the lineage ancestral to chimpanzees and humans. Another approach recently introduced to investigate the contribution of particular brain systems to computational capacity is to consider the evolution of the relative size of particular systems in relation to overall brain size (Sherwood and Smaers, 2013; Smaers and Soligo, 2013). This comparison is useful because as the relative size of particular brain systems increase or decrease over evolutionary time in coordination with overall brain size, the absolute size attributed to each system will change correspondingly. Because brain structure size scales approximately isometrically as a function of cell number (Gabi et al., 2010; Herculano-Houzel et al., 2007), the corresponding change in the absolute size of the brain system will affect its computational power. Following this logic, Fig. 3 presents phylogenetic mapping results from a comparison of evolutionary rates between changes in posterior cerebellar cortex, prefrontal cortex, and prefrontal relative to posterior cerebellar cortex volume with overall brain size. Results indicate disproportionate coordinated increase of posterior cerebellar cortex and prefrontal cortex with brain size in the ancestral lineages leading to great apes, and the chimpanzee–human clade in particular (Fig. 3A and B). The human lineage indicates an additional coordinated increase of prefrontal and overall brain size volume (Fig. 3B). To gauge the relative contribution of the prefrontal cortex to the prefronto-posterior cerebellar system, an additional analysis indicates the rates of change for prefrontal cortex relative to those of posterior cerebellar cortex in association with overall brain size, indicating the largest evolutionary signal in the human lineage and the lineage ancestral to the chimpanzee–human clade (Fig. 3C).

205

Disproportionate coordinated increase

B

A

Mya

40

30

20

10

Posterior cerebellar cortex and brain size

0

C

Pithecia pithecia

Pithecia pithecia

Pithecia pithecia

Cebus albifrons

Cebus albifrons

Cebus albifrons

Lagothrix lagotricha

Lagothrix lagotricha

Lagothrix lagotricha

Ateles geoffroyi

Ateles geoffroyi

Ateles geoffroyi

Alouatta seniculus

Alouatta seniculus

Alouatta seniculus

Pan troglodytes troglodytes

Pan troglodytes troglodytes

Pan troglodytes troglodytes

Homo sapiens

Homo sapiens

Homo sapiens

Gorilla gorilla gorilla

Gorilla gorilla gorilla

Gorilla gorilla gorilla

Hylobates lar

Hylobates lar

Hylobates lar

Nasalis larvatus

Nasalis larvatus

Nasalis larvatus

Piliocolobus badius

Piliocolobus badius

Piliocolobus badius

Papio anubis

Papio anubis

Papio anubis

Miopithecus talapoin

Miopithecus talapoin

Miopithecus talapoin

Erythrocebus patas

Erythrocebus patas

Erythrocebus patas

Cercopithecus ascanius

Cercopithecus ascanius

Cercopithecus ascanius

Cercopithecus mitis

Cercopithecus mitis

Mya

40

30

20

10

Prefrontal and brain size

0

Cercopithecus mitis

Mya

40

30

20

10

0

Prefrontal relative to posterior cerebellar cortex and brain size

FIGURE 3 Phylogenetic mapping of coordinated changes in the relative volumes of posterior cerebellar cortex and brain size (A), prefrontal cortex and brain size (B), and prefrontal cortex relative to posterior cerebellar cortex and brain size (C). Variable rates for individual lineages were computed using the method of independent evolution (Smaers and Soligo, 2013; Smaers et al., 2012). Prefrontal data come from Smaers et al. (2011a,b) and cerebellar data come from Smaers et al. (2013).

5 How Can Macroevolutionary Studies Contribute to Cerebellar Function?

The analyses described above thus tackle the question of putative evolutionary changes in relative cerebellar and/or prefrontal cortex size across individual primate lineages by using different approaches (phylogenetic scaling, phylogenetic correlations, phylogenetic mapping) and different anatomical variables (frontal lobe, prefrontal cortex, frontal motor areas, cerebellar hemispheres, posterior cerebellar cortex). Although the use of different anatomical variables produces slightly different results, a consistent picture emerges across analyses demonstrating grade shifts for the relative size of the prefronto-posterior cerebellar system in great apes and humans. Macroevolutionary results align with neuroanatomical and neuroimaging studies in the macaque and human brain in providing further support for frontal motor- and prefrontal-cerebellar connectivity across a wide range of primate species. The dominance of cortical motor projections with cerebellar lobules in the macaque brain (Brodal, 1978; Glickstein et al., 1985) meshes well with the finding that relative size changes in frontal motor areas and the posterior cerebellar cortex are significantly correlated across anthropoid primates (Smaers et al., 2013). Prefrontal contributions to the cortico-cerebellar system are found to increase more specifically in great apes and humans. Prefrontal and posterior cerebellar size changes indicate similar patterns of coordinate size changes in great apes, separating them from monkeys (Fig. 3A and B). Humans indicate increased prefrontal over posterior cerebellar expansion, separating them from other great apes (Fig. 3C). These results align with results from studies using DT-MRI tractography in macaques and humans (Ramnani et al., 2006) and MRI volumetric comparisons of capuchin monkeys, chimpanzees, and humans (Balsters et al., 2010). Different lines of enquiry thus converge on indicating frontal motor-cerebellar connections in anthropoid primates, and proposing increased prefrontal contributions to the cortico-cerebellar as a characterizing feature of great ape and human brain evolution. Future analyses should also explore whether parietal areas indicate a similar pattern of (great) ape increase comparable to that of prefrontal and posterior cerebellar cortex. Considering evidence of anatomical connectivity between the cerebellum and parietal areas (Clower et al., 2001; Prevosto et al., 2010), such evolutionary analyses are likely to provide valuable information. Lack of available data on parietal areas has, however, hampered this particular research question. Preliminary work based on incomplete data suggests a similar great ape and human grade shift in the parietal cortex (Passingham and Smaers, 2014). The same work, however, also recognizes that more and better data on parietal volumes across different species are needed to resolve this issue.

5 HOW CAN MACROEVOLUTIONARY STUDIES CONTRIBUTE TO OUR UNDERSTANDING OF CEREBELLAR FUNCTION? By combining information on detailed patterns of anatomical evolution across individual lineages of the primate tree of life with behavioral information from the corresponding species, macroevolutionary studies are in a unique position to assess

207

208

CHAPTER 8 Modeling the Evolution of the Cerebellum

brain–behavior interactions across a broad range of environmental conditions. Within a macroevolutionary context, the evolution of any lineage can be perceived as the result of a natural experiment in which species have adapted to different environments. The different environmental conditions in which species have evolved can be considered as the experimental variable that has naturally produced different conditions for different species. The behavioral and anatomical characteristics of present-day living species can then be considered as the evolutionary responses to these different (natural) experimental conditions. This potentially leads to a wealth of information from which brain-behavioral information can be deduced. While the potential contribution of such studies in providing information across a wide range of species across different environmental conditions is far reaching, it should be clear it is also inherently limited. Although the evolutionary resolution of neuroanatomical information has recently increased (see above), information on behavioral capacity across species is sparse. One potential way forward is to focus on behaviors that have been well studied and for which a reasonable amount of information is available across different primate species. Tool use has traditionally been one of the primary foci of human evolutionary studies because it is arguably one of the main factors separating humans from nonhuman primates. Consequently, tool use in primates and humans has been studied from a wide range of different perspectives (archeological, palaeoanthropological, primatological, psychological, neuroscientific). In short, archeological and palaeoanthropological research indicates that tool use has been fundamental to hominin life for up to 3.5 Myr (McPherron et al., 2010; Semaw, 2000). Since the first reports of tool use by chimpanzees by Goodall (1964), there have been numerous reports of tool use by other primate species such as gorillas (Breuer et al., 2005), orang utans (vanSchaik et al., 1996), and capuchin monkeys (Fragaszy et al., 2004). Despite examples of tool use in different nonhuman primates, tool use is widely accepted to be most advanced in chimpanzees (Whiten et al., 1999). Considering the comparative prevalence of tool use, ecological psychologists analyzed the behavioral organization and skill acquisition process involved in different types of tool use (nut-cracking and stone-flaking). For nut-cracking tasks two crucial parameters have been identified that are under control of the actor: the mass of the hammer and the velocity of the hammer at the time of contact (Bril et al., 2012). Both capuchins and chimpanzees have been shown to have a basic understanding of the functional properties of a percussive task by being able to select appropriate tools (Bril et al., 2009). However, this capacity to perceive the affordances of objects as potential tools needs to be learned through experience (Bril et al., 2010). Similarly, the recognition of subtle contrasts in suitability of possible hammers in Indian craftsmen also depends on level of expertise. Both chimpanzees and humans are also able to adjust the velocity of impact based on the constraints of the task (a higher velocity with lighter hammers) (Bril et al., 2009). But also here expertise significantly impacts the behavior with a higher level of expertise associated with a reduced velocity at impact. The behavioral organization and skill acquisition involved in percussive tasks are thus similar in humans and chimpanzees and is crucially dependent on experience.

5 How Can Macroevolutionary Studies Contribute to Cerebellar Function?

The case of stone-flaking provides a much more complex behavioral condition. In addition to decisions with regard to the mass and velocity of the hammer, stoneflaking requires asymmetric bimanual coordination and an understanding of fracture mechanics (Bril et al., 2012). These actions require an adaptable and flexible nesting of differentiated functions, in which movements of two hands are modulated in such a way to meet various functional demands of the situation. Experience is even more important in stone-flaking than in nut-cracking, with much greater differences in performance between experts and novices (Bril et al., 2010). For conchoidal flaking (observed in humans only), the requirements are yet even higher and only very advanced knappers are able to produce the flake they intended (Nonaka et al., 2010). Conchoidal flaking requires a strong rule-based cognitive anticipatory element that includes accurately predicting the consequences of a strike based on an exploration of the properties of the core and of the hammer stone, and setting up an interrelationship among the variables in such a way as to comply with the constraints of the task (Nonaka et al., 2010). In short, expert stone tool flaking requires anticipation, rule-based “dendritic” decision making, experience, and visualization (J.J. Shea, personal communication). The emphasis on the importance of experience and trial-by-trial reduced activity in observations of tool use tasks is congruent with the suggested role of the cerebellum in the transition from controlled to automatic processing of movements toward skillfully executed patterns (Albus, 1971). Indeed, tract-tracing, neuroimaging, and macroevolutionary studies on cerebellar evolution indicate a similar pattern of subsequent neuroanatomical grade shifts in chimpanzees and humans as comparative behavioral studies indicate in capacity for tool use. The cognitive requirements associated to conchoidal flaking (anticipation, rule-based decision making, experience, and visualization) and human’s unique ability to perform this behavior corresponds with the increased prefrontal contribution to the cortico-cerebellar system in humans (Fig. 3C) (see also Balsters et al., 2010; Ramnani et al., 2006) and studies demonstrating that prefrontal projecting lobules (Crus I and Crus II) in the human cerebellar cortex respond to activity related to the execution of abstract rules that govern action, rather than movement itself (Balsters and Ramnani, 2008, 2011). Neuroimaging studies in humans and macaques provide further support for the link between observed species differences in tool use capacity and patterns of cortico-cerebellar activity. Obayashi et al. (2001) demonstrated that macaque tool use activates cortical motor areas (intraparietal cortex, presupplementary motor cortex, premotor cortex), cerebellum and basal ganglia. Human tool use includes a fronto-parietal praxis network as well as the cerebellum and basal ganglia (Lewis, 2006). These studies thus also implicate parietal areas (see also Padberg et al., 2007; Stout and Chaminade, 2007), supporting the importance of visualization in these tasks (Vanduffel et al., 2002) and further emphasizing the need for future macroevolutionary analyses to focus on aspects of parietal cortex evolution (Passingham and Smaers, 2014). Tool use by capuchin monkeys (Cebus species) (Chevalier-Skolnikoff, 1989; Fragaszy et al., 2004) provides an additional test for macroevolutionary inference

209

210

CHAPTER 8 Modeling the Evolution of the Cerebellum

of cerebellar evolution and its link to tool use. Capuchin monkeys share a common ancestor with macaques, chimpanzees, and humans as far back as 45 million years ago and during that time have evolved independently on a separate continent. Capuchin monkeys are unmatched in tool use proficiency in New World primates and thus provide a rigorous test for macroevolutionary studies. Congruent with the role of cortico-cerebellar processing in tool use, Smaers and Soligo (2013) demonstrated that cebus monkeys indicate the highest score among New World monkeys on a principal component that comprises frontal cortical motor areas and cerebellar hemispheres. The posterior cerebellar grade shift in great apes and subsequent prefrontal cortex grade shifts in great apes and humans also provides support for the role of the corticocerebellar system in the evolution of human language and general reasoning. The process by which new sound patterns are learned in order to produce new words in normal adults and children (Baddeley et al., 1998) is remarkably similar to the way stone tool flaking skills are acquired. The trial-by-trial decreasing activity curve of motor learning and its resulting automation of skillfully executed patterns through the development of rule-based relationships in working memory is hereby extended from motor actions that manipulate the external environment (such as tool use) to those that produce sounds for communicative purposes (i.e., phonological patterns) (Koziol et al., 2014; Vandervert, 2011). Leiner et al. (1986, 1989) proposed a theory of cerebellar evolution that relates to general reasoning, proposing that apes and humans evolved a dual hierarchy of cerebellar function. On a lower level, the cerebellum effects the manipulation of muscles through efferent connections with frontal motor areas. On a higher level, the cerebellum effects the manipulation of symbols through efferent connections with the prefrontal cortex. The cerebellum is hereby proposed to function in a similar way on both levels: through enabling learned procedures to be executed optimally. The higher level of cerebellar function (pertaining to the lateral cerebellar hemispheres) is proposed to have given rise to the “skillfull manipulation of ideas” and hereby also to increased language dexterity.

6 SUMMARY In sum, the patterns of coordinated size changes in the cortico-cerebellar system in anthropoid primates observed in macroevolutionary studies align with tract-tracing and neuroimaging studies on cortico-cerebellar connectivity in humans and particular nonhuman primates (macaque, cebus, chimpanzee). This congruence of results provides additional support for anatomical connectivity between cortical motor areas and the cerebellum in primates, and increased prefrontal contributions to the corticocerebellar system in humans. Macroevolutionary studies further demonstrate that the relative size of the cortico-cerebellar system increased significantly in great apes. In addition to increasing the relative size of the posterior cerebellar cortex, great apes indicate increased prefrontal (relative to cortical motor) contributions to the corticocerebellar system, which distinguishes them from monkeys. The human–chimpanzee

References

clade and the human lineage in particular further expanded this great ape trend, eventually distinguishing humans from other great apes. These studies thus suggest that the cortico-cerebellar system has been subject to at least 20 Myr of selective expansion in great apes, and that humans are the extreme of this great ape trend with an additional expansion of the prefrontal contribution to the cortico-cerebellar system. Macroevolutionary studies have the potential to contribute to interpretations on cerebellar function by comparing observed behavioral differences between species with inferred evolutionary pathways of neuroanatomical changes in respective lineages. In this context, comparative observations of tool use proficiency, and the cognitive behavioral requirements needed to perform different types of percussive tasks, align with studies proposing that the principal contribution of the cerebellum to cognition is the automation of cognitive processing. The transition from “controlled” to “automatic” processing, in which movement tasks that require initial attention and problem solving gradually require less attention and become more efficient, is analogous to the cognitive analysis of behavioral organization of percussive tasks. The case of conchoidal flaking, which requires a rule-based cognitive anticipatory element, is of particular relevance as it is unique to humans and, combined with evidence for increased prefrontal contribution to the cortico-cerebellar system in humans, provides further support for the role of the lateral cerebellum in rule-based rather than movement-based processing. The subsequent grade shifts in posterior cerebellar cortex and prefrontal cortex observed in great apes and humans also provide further support for the role of the cerebellum in language “dexterity” and general reasoning.

Acknowledgment I thank John J. Shea for comments on the cognitive requirements of expert stone tool flaking.

References Albus, J.S., 1971. A theory of cerebellar function. Math. Biosci. 10, 25–61. Amunts, K., Lepage, C., Borgeat, L., Mohlberg, H., Dickscheid, T., Rousseau, M.-E., Bludau, S., Bazin, P.-L., Lewis, L.B., Oros-Peusquens, A.-M., Shah, N.J., Lippert, T., Zilles, K., Evans, A.C., 2013. BigBrain: an ultrahigh-resolution 3D human brain model. Science 340, 1472–1475. Arnold, C., Matthews, L.J., Nunn, C.L., 2010. The 10 k trees Website: a new online resource for primate phylogeny. Evol. Anthropol. 19, 114–118. Baddeley, A., Gathercole, S., Papagno, C., 1998. The phonological loop as a language learning device. Psychol. Rev. 105, 158–173. Balsters, J.H., Ramnani, N., 2008. Symbolic representations of action in the human cerebellum. Neuroimage 43, 388–398. Balsters, J.H., Ramnani, N., 2011. Cerebellar plasticity and the automation of first-order rules. J. Neurosci. 31, 2305–2312.

211

212

CHAPTER 8 Modeling the Evolution of the Cerebellum

Balsters, J.H., Cussans, E., Diedrichsen, J., Phillips, K.A., Preuss, T.M., Rilling, J.K., Ramnani, N., 2010. Evolution of the cerebellar cortex: the selective expansion of prefrontal-projecting cerebellar lobules. Neuroimage 49, 2045–2052. Barton, R.A., Harvey, P.H., 2000. Mosaic evolution of brain structure in mammals. Nature 405, 1055–1058. Bauernfeind, A.L., De Sousa, A.A., Avasthi, T., Dobson, S.D., Raghanti, M.A., Lewandowski, A.H., Zilles, K., Semendeferi, K., Allman, J.M., Craig, A.D., Hof, P.R., Sherwood, C.C., 2013. A volumetric comparison of the insular cortex and its subregions in primates. J. Hum. Evol. 64, 263–279. Booth, J.R., Wood, L., Lu, D., Houk, J.C., Bitan, T., 2007. The role of the basal ganglia and cerebellum in language processing. Brain Res. 1133, 136–144. Breuer, T., Ndoundou-Hockemba, M., Fishlock, V., 2005. First observation of tool use in wild gorillas. PLoS Biol. 3, e380. Bril, B., Dietrich, G., Foucart, J., Fuwa, K., Hirata, S., 2009. Tool use as a way to assess cognition: how do captive chimpanzees handle the weight of the hammer when cracking a nut? Anim. Cogn. 12, 217–235. Bril, B., Rein, R., Nonaka, T., Wenban-Smith, F., Dietrich, G., 2010. The role of expertise in tool use: skill differences in functional action adaptations to task constraints. J. Exp. Psychol. Hum. Percept. Perform. 36, 825–839. Bril, B., Smaers, J., Steele, J., Rein, R., Nonaka, T., Dietrich, G., Biryukova, E., Hirata, S., Roux, V., 2012. Functional mastery of percussive technology in nut-cracking and stoneflaking actions: experimental comparison and implications for the evolution of the human brain. Philos. Trans. R. Soc. Lond. B Biol. Sci. 367, 59–74. Brodal, P., 1978. The corticopontine projection in the rhesus monkey. Origin and principles of organization. Brain 101, 251–283. Brown, J.H., West, G.B. (Eds.), 2000. Scaling in Biology. Oxford University Press, New York. Buckner, R.L., Krienen, F.M., Castellanos, A., Diaz, J.C., Yeo, B.T.T., 2011. The organization of the human cerebellum estimated by intrinsic functional connectivity. J. Neurophysiol. 106, 2322–2345. Buckner, R.L., Krienen, F.M., Yeo, B.T.T., 2013. Opportunities and limitations of intrinsic functional connectivity MRI. Nat. Neurosci. 16, 832–837. Chevalier-Skolnikoff, S., 1989. Spontaneous tool use and sensorimotor intelligence in Cebus compared with other monkeys and apes. Behav. Brain Sci. 12, 561–586. Clower, D.M., West, R.A., Lynch, J.C., Strick, P.L., 2001. The inferior parietal lobule is the target of output from the superior colliculus, hippocampus, and cerebellum. J. Neurosci. 21, 6283–6291. De Sousa, A.A., Sherwood, C.C., Hof, P.R., Zilles, K., 2013. Lamination of the lateral geniculate nucleus of catarrhine primates. Brain Behav. Evol. 81, 93–108. De Winter, W., Oxnard, C.E., 2001. Evolutionary radiations and convergences in the structural organization of the mammalian brain. Nature 409, 710–714. Devoogd, T., Krebs, J.R., Healy, S.D., Purvis, A., 1993. Relations between song repertoire size and the volume of brain nuclei related to song: comparative evolutionary analyses among oscine birds. Proc. Biol. Sci. 254, 75–82. Ding, H., Qin, W., Jiang, T., Zhang, Y., Yu, C., 2012. Volumetric variation in subregions of the cerebellum correlates with working memory performance. Neurosci. Lett. 508, 47–51. Draganski, B., Gaser, C., Busch, V., Schuierer, G., Bogdahn, U., May, A., 2004. Changes in grey matter induced by training. Nature 427, 311–312.

References

Dum, R.P., Strick, P.L., 2003. An unfolded map of the cerebellar dentate nucleus and its projections to the cerebral cortex. J. Neurophysiol. 89, 634–639. Felsenstein, J., 1985. Phylogenies and the comparative method. Am. Nat. 125, 1–15. Finlay, B.L., Darlington, R.B., 1995. Linked regularities in the development and evolution of mammalian brains. Science 268, 1578–1584. Fleming, S.M., Weil, R.S., Nagy, Z., Dolan, R.J., Rees, G., 2010. Relating introspective accuracy to individual differences in brain structure. Science 329, 1541–1543. Fragaszy, D., Izar, P., Visalberghi, E., Ottoni, E.B., De Oliveira, M.G., 2004. Wild capuchin monkeys (Cebus libidinosus) use anvils and stone pounding tools. Am. J. Primatol. 64, 359–366. Frahm, H.D., Stephan, H., Stephan, M., 1982. Comparison of brain structure volumes in Insectivora and Primates. 1. Neocortex. J. Hirnforsch. 23, 375–389. Freckleton, R.P., Harvey, P.H., Pagel, M., 2002. Phylogenetic analysis and comparative data: a test and review of evidence. Am. Nat. 160, 712–726. Frey, S., Pandya, D.N., Chakravarty, M.M., Bailey, L., Petrides, M., Collins, D.L., 2011. An MRI based average macaque monkey stereotaxic atlas and space (MNI monkey space). Neuroimage 55, 1435–1442. Gabi, M., Collins, C.E., Wong, P., Torres, L.B., Kaas, J.H., Herculano-Houzel, S., 2010. Cellular scaling rules for the brains of an extended number of primate species. Brain Behav. Evol. 76, 32–44. Glickstein, M., May, J.G., Mercier, B.E., 1985. Corticopontine projection in the macaque: the distribution of labeled cortical cells after large injections of horseradish peroxidase in the pontine nuclei. J. Comp. Neurol. 235, 343–359. Goodall, J., 1964. Tool-using and aimed throwing in a community of free-living chimpanzees. Nature 201, 1264–1266. Harvey, P.H., Pagel, M., 1991. The Comparative Method in Evolutionary Biology. Oxford University Press, New York. Herculano-Houzel, S., 2010. Coordinated scaling of cortical and cerebellar numbers of neurons. Front. Neuroanat. 4. Herculano-Houzel, S., Collins, C.E., Wong, P., Kaas, J.H., 2007. Cellular scaling rules for primate brains. Proc. Natl. Acad. Sci. U. S. A. 104, 3562–3567. Huber, R., Van Staaden, M.J., Kaufman, L.S., Liem, K.F., 1997. Microhabitat use, trophic Patterns, and the evolution of brain structure in African cichlids. Brain Behav. Evol. 50, 167–182. Huxley, J.S., 1932. Problems of Relative Growth. Methuen, London. Imamizu, H., Miyauchi, S., Tamada, T., Sasaki, Y., Takino, R., Putz, B., Yoshioka, T., Kawato, M., 2000. Human cerebellar activity reflecting an acquired internal model of a new tool. Nature 403, 192–195. Imamizu, H., Kuroda, T., Miyauchi, S., Yoshioka, T., Kawato, M., 2003. Modular organization of internal models of tools in the human cerebellum. Proc. Natl. Acad. Sci. U. S. A. 100, 5461–5466. Iwaniuk, A.N., Wylie, D.R.W., 2007. Neural specialization for hovering in hummingbirds: hypertrophy of the pretectal nucleus lentiformis mesencephali. J. Comp. Neurol. 500, 211–221. Jerison, H.J., 1973. Evolution of the Brain and Intelligence. Academic Press, New York. Kelly, R.M., Strick, P.L., 2003. Cerebellar loops with motor cortex and prefrontal cortex of a nonhuman primate. J. Neurosci. 23, 8432–8444.

213

214

CHAPTER 8 Modeling the Evolution of the Cerebellum

Kelly, C., Uddin, L.Q., Shehzad, Z., Margulies, D.S., Castellanos, F.X., Milham, M.P., Petrides, M., 2010. Broca’s region: linking human brain functional connectivity data and non-human primate tracing anatomy studies. Eur. J. Neurosci. 32, 383–398. Koziol, L.F., Budding, D.E., Chidekel, D., 2012. From movement to thought: executive function, embodied cognition, and the cerebellum. Cerebellum 11, 505–525. Koziol, L.F., Budding, D., Andreasen, N., D’Arrigo, S., Bulgheroni, S., Imamizu, H., Ito, M., Manto, M., Marvel, C., Parker, K., Pezzulo, G., Ramnani, N., Riva, D., Schmahmann, J., Vandervert, L., Yamazaki, T., 2014. Consensus paper: the cerebellum’s role in movement and cognition. Cerebellum 13, 151–177. Krienen, F.M., Buckner, R.L., 2009. Segregated fronto-cerebellar circuits revealed by intrinsic functional connectivity. Cereb. Cortex 19, 2485–2497. Larsell, O., 1970. Comparative Anatomy and Histology of the Cerebellum from Monotremes Through Apes. University of Minnesota Press, Minneapolis. Leiner, H.C., Leiner, A.L., Dow, R.S., 1986. Does the cerebellum contribute to mental skills? Behav. Neurosci. 100, 443–454. Leiner, H.C., Leiner, A.L., Dow, R.S., 1989. Reappraising the cerebellum: what does the hindbrain contribute to the forebrain? Behav. Neurosci. 103, 998–1008. Leiner, H.C., Leiner, A.L., Dow, R.S., 1993. Cognitive and language functions of the human cerebellum. Trends Neurosci. 16, 444–447. Lewis, J.W., 2006. Cortical networks related to human use of tools. Neuroscientist 12, 211–231. Mackey, S., Petrides, M., 2010. Quantitative demonstration of comparable architectonic areas within the ventromedial and lateral orbital frontal cortex in the human and the macaque monkey brains. Eur. J. Neurosci. 32, 1940–1950. Macleod, C., 2012. The missing link: evolution of the primate cerebellum. In: Hofman, M.A., Falk, D. (Eds.), Evolution of the Primate Brain: From Neuron to Behavior. Progress in Brain Research, vol. 195. Elsevier Science, Amsterdam, pp. 165–187. Macleod, C.E., Zilles, K., Schleicher, A., Rilling, J.K., Gibson, K.R., 2003. Expansion of the neocerebellum in Hominoidea. J. Hum. Evol. 44, 401–429. Maguire, E.A., Gadian, D.G., Johnsrude, I.S., Good, C.D., Ashburner, J., Frackowiak, R.S.J., Frith, C.D., 2000. Navigation-related structural change in the hippocampi of taxi drivers. Proc. Natl. Acad. Sci. U. S. A. 97, 4398–4403. Martins, E.P., Diniz, J.A.F., Housworth, E.A., 2002. Adaptive constraints and the phylogenetic comparative method: a computer simulation test. Evolution 56, 1–13. Mcpherron, S.P., Alemseged, Z., Marean, C.W., Wynn, J.G., Reed, D., Geraads, D., Bobe, R., Bearat, H.A., 2010. Evidence for stone-tool-assisted consumption of animal tissues before 3.39 million years ago at Dikika, Ethiopia. Nature 466, 857–860. Middleton, F.A., Strick, P.L., 2001. Cerebellar projections to the prefrontal cortex of the primate. J. Neurosci. 21, 700–712. Nonaka, T., Bril, B., Rein, R., 2010. How do stone knappers predict and control the outcome of flaking? Implications for understanding early stone tool technology. J. Hum. Evol. 59, 155–167. Obayashi, S., Suhara, T., Kawabe, K., Okauchi, T., Maeda, J., Akine, Y., Onoe, H., Iriki, A., 2001. Functional brain mapping of monkey tool use. Neuroimage 14, 853–861. O’Reilly, J.X., Beckmann, C.F., Tomassini, V., Ramnani, N., Johansen-Berg, H., 2010. Distinct and overlapping functional zones in the cerebellum defined by resting state functional connectivity. Cereb. Cortex 20, 953–965. Orme, D., Freckleton, R., Thomas, G., Petzoldt, T., Fritz, S., Isaac, N., Pearse, W., 2012. caper: comparative analyses of phylogenetics and evolution in R. R package version 0.5.

References

Padberg, J., Franca, J.G., Cooke, D.F., Soares, J.G.M., Rosa, M.G.P., Fiorani, M., Gattass, R., Krubitzer, L., 2007. Parallel evolution of cortical areas involved in skilled hand use. J. Neurosci. 27, 10106–10115. Pagel, M.D., Harvey, P.H., 1988. The taxon-level problem in the evolution of mammalian brain size—facts and artifacts. Am. Nat. 132, 344–359. Passingham, R.E., 1973. Anatomical differences between the neocortex of man and other primates. Brain Behav. Evol. 7, 337–359. Passingham, R.E., Smaers, J.B., 2014. Is the prefrontal cortex especially enlarged in the human brain? Brain Behav. Evol. In review. Petrides, M., Tomaiuolo, F., Yeterian, E.H., Pandya, D.N., 2012. The prefrontal cortex: comparative architectonic organization in the human and the macaque monkey brains. Cortex 48, 46–57. Prevosto, V., Graf, W., Ugolini, G., 2010. Cerebellar inputs to intraparietal cortex areas LIP and MIP: functional frameworks for adaptive control of eye movements, reaching, and arm/eye/head movement coordination. Cereb. Cortex 20, 214–228. Raghanti, M.A., Simic, G., Watson, S., Stimpson, C.D., Hof, P.R., Sherwood, C.C., 2011. Comparative analysis of the nucleus basalis of Meynert among primates. Neuroscience 184, 1–15. Ramnani, N., 2006. The primate cortico-cerebellar system: anatomy and function. Nat. Rev. Neurosci. 7, 511–522. Ramnani, N., 2012. Frontal lobe and posterior parietal contributions to the cortico-cerebellar system. Cerebellum 11, 366–383. Ramnani, N., Behrens, T.E.J., Johansen-Berg, H., Richter, M.C., Pinsk, M.A., Andersson, J.L.R., Rudebeck, P., Ciccarelli, O., Richter, W., Thompson, A.J., Gross, C.G., Robson, M.D., Kastner, S., Matthews, P.M., 2006. The evolution of prefrontal inputs to the cortico-pontine system: diffusion imaging evidence from macaque monkeys and humans. Cereb. Cortex 16, 811–818. Reep, R.L., Finlay, B.L., Darlington, R.B., 2007. The limbic system in mammalian brain evolution. Brain Behav. Evol. 70, 57–70. Revell, L.J., 2012. phytools: an R package for phylogenetic comparative biology (and other things). Methods Ecol. Evol. 3, 217–223. Rilling, J.K., 2014. Comparative primate neuroimaging: insights into human brain evolution. Trends Cogn. Sci. 18, 45–55. Sayers, K., Raghanti, M.A., Lovejoy, C.O., 2012. Human evolution and the chimpanzee referential doctrine. Annu. Rev. Anthropol. 41. Schmahmann, J.D. (Ed.), 1997. The Cerebellum and Cognition. Academic Press, London. Schmahmann, J.D., Pandya, D.N., 1995. Prefrontal cortex projections to the basilar pons in rhesus-monkey: implications for the cerebellar contribution to higher function. Neurosci. Lett. 199, 175–178. Schmahmann, J.D., Pandya, D.N., 1997. Anatomic organization of the basilar pontine projections from prefrontal cortices in rhesus monkey. J. Neurosci. 17, 438–458. Semaw, S., 2000. The world’s oldest stone artefacts from Gona, Ethiopia: their implications for understanding stone technology and patterns of human evolution between 2.6-1.5 million years ago. J. Archaeol. Sci. 27, 1197–1214. Sherwood, C.C., Smaers, J.B., 2013. What’s the fuss over human frontal lobe evolution? Trends Cogn. Sci. 17, 432–433. Sherwood, C.C., Hof, P.R., Holloway, R.L., Semendeferi, K., Gannon, P.J., Frahm, H.D., Zilles, K., 2005. Evolution of the brainstem orofacial motor system in primates: a comparative study of trigeminal, facial, and hypoglossal nuclei. J. Hum. Evol. 48, 45–84.

215

216

CHAPTER 8 Modeling the Evolution of the Cerebellum

Smaers, J.B., 2013. How humans stand out in frontal lobe scaling. Proc. Natl. Acad. Sci. 110, E3682. Smaers, J.B., Soligo, C., 2013. Brain reorganization, not relative brain size, primarily characterizes anthropoid brain evolution. Proc. Biol. Sci. 280, 20130269. Smaers, J.B., Schleicher, A., Zilles, K., Vinicius, L., 2010. Frontal white matter volume is associated with brain enlargement and higher structural connectivity in haplorrhine primates. PLoS One 5, e9123. Smaers, J.B., Steele, J., Case, C.R., Cowper, A., Amunts, K., Zilles, K., 2011a. Primate prefrontal cortex evolution: human brains are the extreme of a lateralized ape trend. Brain Behav. Evol. 77, 67–78. Smaers, J.B., Steele, J., Zilles, K., 2011b. Modeling the evolution of cortico-cerebellar systems in primates. In: Johnson, J.I., Zeigler, H.P., Hof, P.R. (Eds.), New Perspectives on Neurobehavioral Evolution. Annals of the New York Academy of Sciences, New York. Smaers, J.B., Dechmann, D.K.N., Goswami, A., Soligo, C., Safi, K., 2012. Comparative analyses of evolutionary rates reveal different pathways to encephalization in bats, carnivorans, and primates. Proc. Natl. Acad. Sci. U. S. A. 109, 18006–18011. Smaers, J.B., Steele, J., Case, C.R., Amunts, K., 2013. Laterality and the evolution of the prefronto-cerebellar system in anthropoids. In: Mcgrew, W.C., Marchant, L., Schiefenho¨vel, W. (Eds.), Evolution of Human Handedness. Annals of the New York Academy of Sciences, New York. Smith, A.R., Seid, M.A., Jimenez, L.C., Wcislo, W.T., 2010. Socially induced brain development in a facultatively eusocial sweat bee Megalopta genalis (Halictidae). Proc. Biol. Sci. 277, 2157–2163. Stoodley, C.J., Schmahmann, J.D., 2009. The cerebellum and language: evidence from patients with cerebellar degeneration. Brain Lang. 110, 149–153. Stout, D., Chaminade, T., 2007. The evolutionary neuroscience of tool making. Neuropsychologia 45, 1091–1100. Strick, P.L., Dum, R.P., Fiez, J.A., 2009. Cerebellum and nonmotor function. Annu. Rev. Neurosci. 32, 413–434. Vandervert, L., 2011. The evolution of language: the cerebro-cerebellar blending of visualspatial working memory with vocalizations. J. Mind Behav. 32, 317–331. Vandervert, L.R., Schimpf, P.H., Liu, H., 2007. How working memory and the cerebellum collaborate to produce creativity and innovation. Creativity Res. J. 19, 1–18. Vanduffel, W., Fize, D., Peuskens, H., Denys, K., Sunaert, S., Todd, J.T., Orban, G.A., 2002. Extracting 3D from motion: differences in human and monkey intraparietal cortex. Science 298, 413–415. Vanschaik, C.P., Fox, E.A., Sitompul, A.F., 1996. Manufacture and use of tools in wild Sumatran orangutans—implications for human evolution. Naturwissenschaften 83, 186–188. Whiten, A., Goodall, J., Mcgrew, W.C., Nishida, T., Reynolds, V., Sugiyama, Y., Tutin, C.E.G., Wrangham, R.W., Boesch, C., 1999. Cultures in chimpanzees. Nature 399, 682–685. Whiting, B.A., Barton, R.A., 2003. The evolution of the cortico-cerebellar complex in primates: anatomical connections predict patterns of correlated evolution. J. Hum. Evol. 44, 3–10. Willemet, R., 2012. Understanding the evolution of mammalian brain structures: the need for a (new) cerebrotype approach. Brain Sci. 2, 203–224.

CHAPTER

Cerebellar and Prefrontal Cortex Contributions to Adaptation, Strategies, and Reinforcement Learning

9

Jordan A. Taylor*, Richard B. Ivry{,{,1 *

Department of Psychology, Princeton University, Princeton, NJ, USA Department of Psychology, University of California, Berkeley, CA, USA { Helen Wills Neuroscience Institute, University of California, Berkeley, CA, USA 1 Corresponding author: Tel: þ1 510 642-7146; Fax: þ1-510-642-5293, e-mail address: [email protected] {

Abstract Traditionally, motor learning has been studied as an implicit learning process, one in which movement errors are used to improve performance in a continuous, gradual manner. The cerebellum figures prominently in this literature given well-established ideas about the role of this system in error-based learning and the production of automatized skills. Recent developments have brought into focus the relevance of multiple learning mechanisms for sensorimotor learning. These include processes involving repetition, reinforcement learning, and strategy utilization. We examine these developments, considering their implications for understanding cerebellar function and how this structure interacts with other neural systems to support motor learning. Converging lines of evidence from behavioral, computational, and neuropsychological studies suggest a fundamental distinction between processes that use error information to improve action execution or action selection. While the cerebellum is clearly linked to the former, its role in the latter remains an open question.

Keywords cerebellum, prefrontal cortex, basal ganglia, sensorimotor learning, adaptation, reinforcement learning, systems interaction, error-based learning, ataxia

1 INTRODUCTION The rules of American baseball define the strike zone as a region with a fixed width (1700 ) and variable height based on the distance between the hitter’s chest and knees. The hitter is most vulnerable to low pitches, ones that cross the zone near, or just Progress in Brain Research, Volume 210, ISSN 0079-6123, http://dx.doi.org/10.1016/B978-0-444-63356-9.00009-1 © 2014 Elsevier B.V. All rights reserved.

217

218

CHAPTER 9 Systems Interactions

below, the knees. For such pitches, hitters are successful in reaching base less than 20% of the time, considerably lower than their overall success rate (Encina, 2013). Given these probabilities, the pitcher would be wise to consistently aim for this location. However, this strategy not only takes considerable practice but also entails considerable risk. Pitches that are just a few inches too high end up right where hitters have their best success; a hoped for strike out pitch is suddenly a fan’s home run souvenir. Given these challenges, we can ask, how does the pitcher master this skill? One possibility is that learning centers on updating processes involved in action execution. By this view, to improve accuracy, the pitcher might aim to the same location each time, using the outcome of recent throws to adjust a learned sensorimotor relationship with the goal of reducing variability. However, low variability entails its own cost in baseball. The pitcher must vary the targeted location so that the hitter cannot focus on one region of the strike zone. A successful pitcher has to use outcome information to also improve action selection. Perhaps, the next pitch should be aimed slightly higher or lower in the strike zone, or, depending on the previous outcome, require a shift to a new region of the strike zone. In this review, we focus on recent developments in the motor learning and skill acquisition literature that explore how people use a multiplicity of learning processes to improve action execution and action selection. This theme has been advanced in behavioral, computational, and neuroscientific studies. With respect to the latter, sensorimotor learning has long been assigned to the functional domain of the cerebellum, inspired by models of how this subcortical structure is essential for errorbased learning. However, the multiple learning systems perspective underscores the need to consider the cerebellum within the broader context of a distributed learning network and point to interesting ways in which the cerebellum interacts with other subcortical and cortical systems during skill acquisition.

2 THE CEREBELLUM AND ERROR-BASED LEARNING The role of the cerebellum in coordination and movement regulation took hold in the nineteenth century. Ablation of this structure in a variety of animals produced profound impairments of coordination, even in the absence of weakness (Dalton, 1861; Fine et al., 2002; Flourens, 1824; Marshall and Magoun, 1997). Similarly, early descriptions of individuals with lesions of the cerebellum focused on the decomposition of multijoint movements (Babinski, 1896; 1902) and abnormalities in the regularity, rate, and force of muscle activations (Holmes, 1939), a constellation of symptoms now referred to as cerebellar ataxia. A defining notion of ataxia is that this disorder produces problems in the execution of goal-directed movements, even if the affected individual still retains the intent, or representation of the goal of the desired action (Holmes, 1939). While the early neurology literature focused on the loss of coordination in ataxia, the second half of the twentieth century witnessed a paradigm shift as the study of

2 The Cerebellum and Error-Based Learning

learning came to the forefront. As detailed pictures of the idiosyncratic anatomy and physiology of the cerebellum began to emerge, computational neuroscientists took up the challenge to develop functional hypotheses of the cerebellum. Two highly influential papers, the first published by David Marr in 1969 and the second by James Albus in 1971 (Albus, 1971; Marr, 1969), laid out the core ideas of how the cerebellum subserves an essential role in error-based learning, a hypothesis that remains central in current studies of cerebellar function. While a thorough review of this work is beyond the scope of this chapter, it is instructive to review the key features of the Marr–Albus theory. Both papers sought to explain why the Purkinje cells of cerebellar cortex receive two unique inputs, the parallel fibers and the climbing fibers. Parallel fibers are the axonal extensions of granule cells, with each fiber making single synapses on hundreds of thousands of Purkinje cells. The parallel fibers carry information from the ascending tracts of the spinal cord, many subcortical nuclei, and, via the pontine nuclei, large parts of the cerebral cortex (Jansen and Brodal, 1954). The integration of parallel fiber activity causes the Purkinje cells to generate simple spikes, high-frequency bursts of firing. In contrast, climbing fibers originate in the inferior olive. Each of these fibers targets at most a few Purkinje cells, but through extensive innervation patterns across the Purkinje cell dendritic arbor, the climbing fiber can result in a massive action potential, the complex spike. Marr and Albus recognized that the simple-complex spike arrangement offered an ideal situation for supervised, error-based learning. In this model, the parallel fibers generate a representation of the state of the system, a state that incorporates information about the state of the body as well as a state that has access to current motor commands (e.g., efference copy). The climbing fibers serve as the teacher, generating complex spikes when an unexpected event is encountered. With rather simple, yet elegant, algorithms, this interaction provides the essential ingredients for error-based learning. In the Albus model, this learning was hypothesized to entail a weakening of synaptic strength between the parallel fibers and Purkinje cells, an idea that anticipated long-term depression (Albus, 1989; Ito et al., 1982). The Marr–Albus theory has been elaborated and modified over the past 40 years, but the core idea of the cerebellum as a system for supervised, error-based learning has become established as one of the central tenants of cerebellar theory (Ito, 2006). Early experimental tests of the Marr–Albus model came from studies of the vestibular-ocular reflex, with gain changes in reflex correlated with variation in simple spike activity (Fukuda et al., 1972; Ito, 1974). More causal tests came about with the seminal discoveries of Richard Thompson and colleagues on classical conditioning of the eyeblink response in the rabbit (McCormick and Thompson, 1984a,b). This work provided compelling evidence that the conditioned response was localized to the cerebellum: Focal lesions of either the cerebellar cortex or deep cerebellar nuclei resulted in the abolition of the CR with minimal effect on the UR (Yeo et al., 1985). The eyeblink paradigm also allowed experimenters to conduct strong tests of the Marr–Albus model, replacing the effects of the CS and US by direct stimulation of the parallel fibers or inferior olive, respectively. Eyeblink conditioning and

219

220

CHAPTER 9 Systems Interactions

VOR adaptation continue to be amazingly fruitful tasks, serving as model systems for studying the cellular, molecular, and genetic basis of sensorimotor learning (Boyden et al., 2006; Gao et al., 2012; Raymond et al., 1996; Schonewille et al., 2011). These tasks have also been used in behavioral and imaging studies in humans, with the results providing converging evidence of an essential learning role for the cerebellum (Cheng et al., 2008; Logan and Grafton, 1995; Schubert and Zee, 2010; Timmann et al., 2010). Critical to both eyeblink conditioning and VOR learning is the presence of an error signal. In the former, the airpuff constitutes an error (i.e., unexpected aversive stimulus) and the animal learns to attenuate the negative effects of this US by closing the eyelid in response to predictive conditioning stimulus such as a tone or light. In the latter, the error comes in the form of retinal slip, the difference between the position of the eye and the stimulus being tracked. The notion of error representation in the cerebellum also came from reaching studies in the primate, where climbing fiber discharge was observed when the animal experienced an unexpected sensory event (Gilbert and Thach, 1977). Most pertinent to the current review is the body of literature that has amassed over the past 25 years involving studies of sensorimotor adaptation during volitional action. Here, researchers have employed a range of environmental perturbations, asking how cerebellar pathology affects adaptation. The simplest task, at least experimentally, is to have participants wear prism glasses and make reaching movements to visual targets (Held and Hein, 1958; Helmholtz, 1909/1962). Participants learn to adjust their reaching or throwing movements in a direction that offsets the prismatic distortion. The time course of learning generally follows an exponential function, one in which the error is reduced in a roughly proportional manner across training (Fig. 1). When the prismatic lens are removed there is a prominent aftereffect, such that it takes several reaches or throws to return to original (nonprism) eye-hand calibration. Cerebellar damage, experimentally induced in nonhuman primates or naturally occurring in humans, results in attenuated learning (Baizer and Glickstein, 1974; Martin et al., 1996; Weiner et al., 1983). The subjects continue to produce large errors even after many reaches when wearing the prismatic devices, and, correspondingly, show a smaller aftereffect than control participants. This cerebellar-mediated learning impairment has been confirmed in many sensorimotor adaptation studies in which the participants make reaching movements within virtual reality environments. For example, participants can be asked to reach in a force field, with the hand displaced in a direction orthogonal to the path of motion by a force that is proportional to velocity (Smith and Shadmehr, 2005) or a visuomotor perturbation can be imposed by rotating the position of a feedback cursor relative to true hand position. Across these various types of perturbations, the picture is quite consistent in showing that patients with cerebellar pathology exhibit a marked impairment in adapting to sensorimotor perturbations (CriscimagnaHemminger et al., 2010; Gibo et al., 2013; Izawa et al., 2012; Morton and Bastian, 2004; Rabe et al., 2009; Smith and Shadmehr, 2005).

2 The Cerebellum and Error-Based Learning

FIGURE 1 Hypothetical learning curve during adaptation to an arbitrary visuomotor perturbation. The perturbation is imposed during movements 100–200. Target errors are initially in the direction of the perturbation, but, with training, adaptation occurs. The perturbation is removed on trial 201 and an aftereffect is observed in which target errors are in the direction opposite to the perturbation. The size of the visuomotor perturbation is in arbitrary units (percent).

An appealing feature of virtual reality environments is that they provide the experimenter with control over the perturbation, and, as such, offer the opportunity to manipulate the magnitude and form of the error signal. For example, a force-field or visuomotor perturbation can be introduced abruptly or in a gradual manner. In the former, the participant is aware that the environment has been altered, even though their response to the perturbation may or may not be under strategic control, an issue we return to below. In the latter, the participant is generally completely unaware of the perturbation, at least during the early trials of learning. While one paper indicated that patients with cerebellar pathology showed a much more pronounced deficit in adapting to an abrupt perturbation (Criscimagna-Hemminger et al., 2010), subsequent work indicates that the patients’ deficit is similar to both types of perturbations (Gibo et al., 2013; Schlerf et al., 2013). When analyzed at a group level, cerebellar pathology clearly disrupts learning across a range of adaptation tasks. However, a more fine-grained analysis points to some degree of domain-specificity within the cerebellum. Indeed, one of the first studies of prism adaptation (Martin et al., 1996) pointed to a dissociation between cerebellar contributions to learning and coordination. Ataxia was especially marked in patients with lesions of the superior regions of the cerebellum, lesions that encompassed the classic arm/hand representation in lobule V. However, these patients tended to show modest deficits in adaptation. In contrast, patients with more inferior

221

222

CHAPTER 9 Systems Interactions

lesions showed marked deficits in learning, despite minimal evidence of ataxia. More recent work using sophisticated lesion reconstruction methods has revealed intriguing dissociations within the cerebellum on different adaptation tasks. Force field adaptation deficits are associated with lesions of more superior aspects of the cerebellum relative to visuomotor adaptation deficits (Donchin et al., 2012; Rabe et al., 2009), a pattern that is consistent with anatomical differences in the representation of task-relevant information. Whereas the errors for force-field adaptation are primarily somatosensory, visuomotor adaptation is primarily driven by a visual error signal. There is a crude superior-inferior gradient in terms of the relative projections of somatosensory and visual inputs to the cerebellar cortex.

3 COMPUTATIONAL MODELS OF SENSORIMOTOR ADAPTATION The neuropsychological literature discussed above provides compelling evidence that patients with cerebellar lesions are impaired on tasks requiring sensorimotor adaptation. However, the specific computational role of the cerebellum in such tasks has been the subject of considerable debate. The Marr–Albus model predicts that error signals, arising from the climbing fibers shape parallel fiber-Purkinje cell synapses to modulate the representation of the system’s state, with this modulation producing changes in future responses to similar states. However, most adaptation studies utilize a block design in which the perturbation (e.g., prisms, forces, and rotations) is applied for a fixed period of time. The lack of variance in the perturbation makes it difficult to elucidate a trial-by-trial relationship between error signals and changes in motor output. Through the introduction of randomly varying perturbations, two seminal studies with healthy individuals were able to identify the relationship between error and adaptation on trial-by-trial basis (Scheidt et al., 2001; Thoroughman and Shadmehr, 2000). The results of these studies showed that the amount of change on a single trial was proportional to the size of the perturbation or motor error on the preceding trial. This learning process can be characterized by a linear dynamical system, with various instantiations being realized in state–space models (Thoroughman and Shadmehr, 2000), autoregressive models with exogenous inputs (Scheidt et al., 2001), or hidden Markov models (Schlerf et al., 2013). For a visuomotor rotation task, the dynamical system can be represented as a state–space model as follows: en ¼ r n  ^r n

(1)

^ r nþ1 ¼ A^r n þ Ben

(2)

Equation (1) represents the error on trial (n), which captures the idea that sensorimotor adaptation requires learning an internal model (^r n ). When the system is properly calibrated, the output anticipates the effects of the perturbation (^r n ). Learning in this model is error-driven: In Eq. (2), B reflects the learning rate, or the proportion of the

3 Computational Models of Sensorimotor Adaptation

error that is compensated for from one trial to the next. The value of B tends to be between 0.10 and 0.30, meaning trial-to-trial corrections adjust for approximately 10–30% of the error. While learning would occur more rapidly with higher values of B, such systems tend to be unstable. The other parameter A is considered a memory term, indicating how well the system retains a memory of the internal model from trial-to-trial. Values for this are almost always quite high (A > 0.99), at least for relatively simple tasks such as reaching. Equations (1) and (2) describe the simplest form of a state–space model, capturing the effects of learning in a range of adaptation tasks through the operation of a single learning process. More sophisticated versions have focused on the idea that error information and the updating process may occur over multiple timescales (Smith et al., 2006). For example, a fast system may operate with a large learning rate (B) and a small memory term (A), whereas a slow system may use a smaller learning rate (B) and a large memory term (A). Multirate models have been employed to account for signatures of interference, forgetting, and recall within the linear dynamical system framework. These models have also been used to specify the learning impairments observed in patients with cerebellar degeneration. Tseng et al. (2007) used an adaptation task in which participants learned to reach in the face of a 20 visuomotor rotation (Tseng et al., 2007). They compared two conditions: In one, the participants were provided with continuous online feedback and required to terminate the movement at the target. In the other, the participants were instructed to produce “shooting movements,” attempting to pass through the target until they contacted a virtual pillow. Contrasting these two conditions allowed the authors to evaluate different hypotheses for the patients’ learning deficit. It is possible that learning deficits are secondary to control problems. For example, the patients’ ataxia might make it difficult to use online feedback or control the terminal phase of a movement, with the added control problems placing demands on resources that could otherwise be used for learning. By including the shooting condition, the experiments sought to reduce the control demands on the patients, both by eliminating online corrections and providing an external support to aid movement termination. However, the results showed that the patients were equally impaired in both conditions (Fig. 2A). More important, a model-based analysis revealed a common learning-rate deficit in both tasks. Whereas the learning rate for controls ranged from 0.10 to 0.34, the values for the patients clustered around 0.03 (Fig. 2B). Taken together, the two studies provide strong evidence that the patients’ learning deficit centers on an impairment in trial-by-trial adaptation and is not secondary to problems related to their ataxia. This hypothesis is further reinforced in a study that compared two types of visuomotor rotations, one in which a 20 perturbation was introduced abruptly and another in which the rotation was introduced gradually in 4 increments (Schlerf et al., 2013). In both cases, the patients adapted at a slower rate than the controls, reached lower levels of asymptotic performance, and showed reduced aftereffects. The data were analyzed with a model designed to assess if the performance deficit might reflect a credit assignment problem: Intuitively, one might assume that someone with ataxia

223

224

CHAPTER 9 Systems Interactions

FIGURE 2 Impaired adaptation in patients with cerebellar ataxia. (A) Top: Control participants learned to counter a 20 perturbation (dark shaded area) with either pointing (left) or shooting movements (right) and showed a large aftereffect. Bottom: Patients with ataxia were unable to counter the perturbation and showed a smaller aftereffect. (B) The adaptation rate, as measured by a state–space model, was similar for the two types of movements. The adaptation rates for the patients cluster near the lower values for both tasks. Adapted from Tseng et al. (2007).

may attribute an error in performance to their inability to control their movements rather than attribute the error to a change in the environment (and thus should be incorporated in an internal model of that environment). To address this question, estimates of the participants’ motor variability in the absence of a perturbation were obtained. These values were then used in a probabilistic model based on a Markov-chain process to estimate learning rates. The results indicated that the ataxic individuals exhibited a reduction in learning rate, even when the differences in motor noise were incorporated into the model (see also, Smith and Shadmehr, 2005).

4 MULTIPLE LEARNING MECHANISMS IN SENSORIMOTOR ADAPTATION Linear dynamical systems have provided a simple, yet elegant tool to account for performance on sensorimotor adaptation across a range of tasks. However, a single-process version, such as that described by Eqs. (1) and (2), have proven to be inadequate to account for more complex learning phenomena such as generalization, spontaneous recovery, and asymmetries between the rate of adaptation and the rate at which the aftereffect washes out once the perturbation is removed (Zarahn et al., 2008). As noted above, one solution has been to posit that error-based learning may operate over multiple timescales: both the error and decay parameters can be

4 Multiple Learning Mechanisms in Sensorimotor Adaptation

expanded to influence performance over multiple trials or change as a function of time. For example, the two-rate model of Smith et al. (2006) not only accounts for the rather abrupt shape of many learning functions but, more importantly, can account for patterns of interference observed when participants are successively exposed to conflicting perturbations (Smith et al., 2006). Multiple-rate models fail to capture one phenomenon observed in many studies of human learning: savings in relearning (Zarahn et al., 2008). Savings is defined as faster learning upon reexposure to something that had been previously learned, but then “forgotten.” In classical conditioning studies, faster acquisition of a conditioned response following extinction compared to initial acquisition would constitute savings. The classic account of this phenomenon is that extinction did not really abolish the conditioned association, but rather, induced the animal to learn a second association, one in which the CS is not paired with the US, and thus does not generate a CR. Savings occurs because the repairing of the CS and US invokes the original context, allowing the dormant CS–CR association to be unmasked. Linear dynamical systems are incapable of producing savings since such systems do not retain a memory of previous states: learning in such systems involves recalibrating the state of a single representation, rather than the acquisition of multiple representations. As such, once the aftereffect is washed out, learning would have to begin anew, even if the original perturbation was reintroduced. This prediction, however, is at odds with a number of empirical reports (Huang et al., 2011; Kitago et al., 2013; Zarahn et al., 2008). Adaptation occurs much more rapidly, especially in the initial trials when people are reexposed to a previously learned perturbation. Observations such as these have led motor learning theorists to consider that performance changes in sensorimotor learning tasks involves the operation of multiple learning processes (Huang et al., 2011; Kitago et al., 2013). Indeed, this trend brings the study of motor learning into closer alignment with memory research where theorists have long entertained the idea of multiple learning systems and processes. This issue was, of course, brought to the forefront in Milner’s classic studies with HM (Scoville and Milner, 1957). Not only did this case indicate that the medial temporal lobe was essential for the formation of selective types of memories, but the case also reemphasized the classic observation that severe impairments in learning can exist even when long-term memory is relatively spared. A similar question can be asked with respect to the cerebellum: While the evidence clearly indicates that this structure is essential for sensorimotor adaptation, or learning, is it also essential for storage, or consolidation of the acquired memory? Or is there a partition between learning and consolidation for sensorimotor adaptation, similar to what has been proposed for episodic and declarative memory? This issue has been addressed in the eyeblink conditioning literature. Lesions of either the cerebellar cortex or deep cerebellar nuclei preclude the acquisition of the conditioned response. However, if the lesions are introduced postacquisition, the conditioned response is only abolished following lesions of the DCN (Clark et al., 1984; McCormick and Thompson, 1984a). In contrast, animals with postacquisition lesions restricted to the cerebellar cortex continue to produce conditioned responses

225

226

CHAPTER 9 Systems Interactions

(McCormick and Thompson, 1984b). Interestingly, the adaptive timing of these responses is disrupted (Koekkoek et al., 2003; Perrett et al., 1993). Rather than produce CRs that are maximal at the anticipated time of the US, the animal now produces CRs that occur in immediate response to the CS. Thus, the cerebellar cortex is essential for learning (especially the precise temporal features of the CR), but consolidation of the acquired association may be independent of the cerebellar cortex (Kellet et al, 2010). The question of whether learning and consolidation are functionally distinct in sensorimotor adaptation has received relatively little attention. Recently, this problem has been taken up in studies comparing the relative contribution of the cerebellum and cerebral cortex during sensorimotor learning. Galea and colleagues (2011) evaluated performance changes during visuomotor adaptation following transcranial direct cortical stimulation (tDCS) (Galea et al., 2011). Anodal tDCS has been found to facilitate learning in a range of tasks (Nitsche et al., 2010; Reis et al., 2009), presumably by putting the targeted region into an “up” state by increasing neuronal excitability at baseline. In the Galea study, tDCS of the cerebellar cortex increased learning rates during adaptation compared to tDCS of the motor cortex or sham stimulation, but had no effect on the recovery from the aftereffect when the perturbation was removed. tDCS of the motor cortex had the opposite effect: learning rate during adaptation was unaffected, but the aftereffect persisted for a longer period of time. Although washout only constitutes a very early probe of consolidation, this double dissociation suggests a selective role for the cerebellum in learning, with consolidation being cortically mediated. The latter hypothesis is further supported by evidence showing that consolidation is selectively disrupted when TMS pulses are applied over motor cortex during force-field adaptation (Hadipour-Niktarash et al., 2007). Computationally, the cerebellum and neocortex have been hypothesized to use distinct learning processes. The cerebellum, with its unique physiology, is viewed as the prototypical system for error-based learning, with adaptation driven by the difference between the predicted and actual sensory outcome of a movement. In contrast, learning within the cortex, may be primarily driven by Hebbian processes, with synaptic efficacy strengthened as a function of coactivation. Within the motor learning field, the behavioral signature of Hebbian learning has come to be called use-dependent learning, reflecting the fact that repetition alone can be sufficient to increase the likelihood of a movement (Diedrichsen et al., 2010), or introduce a bias in the execution of other, related movements (Verstynen and Sabes, 2011). A use-dependent process can account for savings (Huang et al., 2011); as such, savings may not arise from facilitation of error-based learning processes associated with the cerebellum, but rather from the reactivation of movement patterns stored in the cerebral cortex. In this view, savings is linked to processes associated with action selection, with the reintroduction of the perturbation serving as a cue for memory recall (Morehead et al., 2013). Models of decision making have focused on yet another learning process, reinforcement learning, to account for how organisms learn to select the optimal response for a given context (Daw et al., 2006; Sutton and Barto, 1998). For the rat in the maze, reinforcement learning processes can explain how the animal chooses

4 Multiple Learning Mechanisms in Sensorimotor Adaptation

to turn toward the baited arm when approaching the branch point in a T-maze. For the human at the casino, a similar process is hypothesized to dictate whether a gambler continues to hammer away at one slot machine in expectation of a jackpot, or switches seats to try her luck on another machine. As with error-based learning, reinforcement learning operates by comparing an expected and realized outcome. However, the expectation here is on anticipated reward (Sutton and Barto, 1998). If an outcome produces a greater than expected reward, the likelihood of repeating that action is increased; if the outcome is less than the expected reward, the likelihood of repeating that action is decreased. Dopamine activity in the basal ganglia, and in particular, the ventral striatum, correlates with the size of these prediction errors (Schultz, 1998). An important difference between standard models of reinforcement learning and error-based learning relates to the information content of the error signal. In errorbased learning (see Eq. 2), the error is vectorial: the sensory prediction error provides information on how the movement should be modified in order to be successful on future actions. For example, if the reach lands to the left of the target, then the internal model has to be recalibrated to reduce this deviation, a form of gradient descent. In reinforcement learning, the error is either categorical (e.g., the rat either obtained the reward or failed to obtain the reward) or, if metrical, indicates the difference in the value of the reward. Receiving a small payoff from one slot machine does not provide information concerning which of the other slot machines is likely to provide a bigger payout. In general, reinforcement learning has been applied to account for how organisms choose which action to take, rather than explain how a selected movement is executed or optimized. In principle, the performance changes observed during sensorimotor learning could come about from reinforcement learning, error-based, or some combination of these processes. In a recent study of visuomotor adaptation, Izawa and Shadmehr (2011) provided a particularly clever comparison of reward-based and error-based learning during a visuomotor adaptation task (Fig. 3). In both conditions, a rotation of 8 was introduced gradually over 320 trials. In the error-based condition, participants were provided with online feedback of the cursor. In the reward-based condition, they received binary feedback, indicating if the reach had intersected the target or missed the target. Participants modified their trajectories in both conditions, effectively counteracting the effects of the rotation. That is, performance improved in a similar manner with either reward- or error-based feedback. A test of sensorimotor remapping, however, indicated that the representational change was quite different in the two conditions (Fig. 3C). For this test, participants were required to reach to a position with the trained limb and then reach to the same position with the untrained limb. Participants trained with error-based feedback showed a discrepancy between the final hand position of the two hands, suggesting a recalibration of the sensorimotor mapping associated with the trained hand. In contrast, participants who received reward-based feedback did not show a difference between the judged position when reaching with either hand, suggesting that no remapping had occurred. Thus, reward-based feedback was sufficient information

227

228

CHAPTER 9 Systems Interactions

FIGURE 3 Compensating for a visuomotor rotation with different types of feedback. (A) Feedback was either provided online or in the form of binary signal indicating success or failure (reward). (B) Performance for two representative participants over the course of 500 trials in which an 8 perturbation was gradually introduced. (C) Sensory remapping, as measured by the localization task, was present for the online feedback group and negligible for the reward feedback group. Adapted from Izawa and Shadmehr (2011).

to counter the gradually introduced rotation, but was insufficient to train an internal model. It is important to ask how participants in the reward condition learned to counter the rotation since they could not see the size or direction of their movement errors. Theoretically, there are two, related possibilities. First, due to simple random variation in reaching performance, reaches that deviate in the opposite direction of the rotation would be positively reinforced. The selective reinforcement of these reaches would induce a systematic shift in reaching direction, one that counters the rotation. Second, success with reward-based feedback could come about through a more exploratory process. That is, the participant might explore different reaching directions,

5 Strategy Use During Sensorimotor Adaptation

with reinforcement-learning mechanisms biasing the system toward actions that result in rewards and away from actions that fail to produce rewards. With this process, the exploratory process would have to be repeated and expanded as the size of the rotation increases. In support of this second idea, participants in the reward-based feedback condition showed an increase in reach variance compared to the error-based feedback condition (Fig. 3C). Indeed, Izawa and colleagues (2011) developed a reinforcement-learning model that learned to counter the rotation through random exploration that resulted in the reinforcement of successful action policies. Learning within this model does not involve the adaptation of an internal model, consistent with the observation that the reward-based participants did not show a change in performance on the intermanual matching task. The study by Izawa and colleagues highlights that the same visuomotor rotation can be solved through two very different forms of learning. With vectorial errors, performance can improve through the adaptation of an internal model; as described in Eqs. (1) and (2), the error provides a supervised signal that informs the system about how an internal model should be changed. This process is not possible with categorical errors. As such, performance improvements come about via reinforcement-learning mechanisms that promote changes in action selection. It is interesting to note that in both conditions, learning was implicit. Whereas adaptation is widely recognized as an implicit process, models of reinforcement learning make transparent that changes in action selection can also result from the operation of implicit processes. Supposing that learning can result from either adaptation or changes in action selection helps resolve some lingering discrepancies in the sensorimotor adaptation literature. A number of studies have reported that the size of the aftereffect, the cleanest probe of adaptation, frequently falls well short of the size of the rotation when participants are not provided with online feedback (Hinder et al., 2008; Peled and Karniel, 2012; Shabbott and Sainburg, 2010). It may well be that, under such conditions, performance changes reflects the combined effects of two (or more) learning processes. Error-based adaptation would be strongest with online feedback (and thus produce larger aftereffects), whereas endpoint feedback might promote changes in action selection. In Section 5, we will turn to recent work on sensorimotor adaptation in which experimenters have developed methods to directly examine the simultaneous operation and interaction of multiple learning processes.

5 STRATEGY USE DURING SENSORIMOTOR ADAPTATION Error-based, use-dependent, and reinforcement learning all entail the operation of continuous mechanisms that gradually guide performance to the correct solution. As noted above, learning rates in adaptation studies tend to be between 0.10 and 0.30, indicating that only a fraction of the error is accounted for when updating

229

230

CHAPTER 9 Systems Interactions

the internal model. The rate of learning in reinforcement-learning studies varies as a function of the task context and number of action choices, but tends to fall in the higher end of this range (Daw et al., 2006; Li and Daw, 2011; O’Doherty et al., 2003; Rutledge et al., 2009; Stoloff et al., 2011; Wittmann et al., 2008). Usedependent learning is likely a slower process, especially if dependent on Hebbian mechanisms where synaptic changes require multiple repetitions. A notable feature of human learning is our ability to rapidly modify our behavior in response to perceived changes in the context. In the classic learning literature, this phenomenon is sometimes referred to as one-shot learning (Carey and Bartlett, 1978; Lake et al., 2011), although such changes need not be achieved in a single trial. For example, the field goal kicker on a football team may note that there is a strong crosswind blowing from left to right and choose to aim his kick to the left of the goal posts. Strategic changes such as this can lead to abrupt changes in performance. Recent studies have begun to ask how such processes impact sensorimotor learning, and, correspondingly, what types of information constrain the utilization of strategies. The idea that motor learning can benefit from strategic and explicit processes has a long history in cognitive psychology. Fitts and Posner (1967), in their classic work, proposed that skill learning could be characterized by three sequential stages (Fitts and Posner, 1967). Learning begins with a cognitive stage in which verbally based, cognitive strategies are used to establish the task goals and general movement features required to achieve these goals. In this stage, the person must determine the appropriate sequence of actions. In a second, associative stage, the sensorimotor space is explored, establishing the mapping between the desired sequence of actions and their associated movements. Finally, an autonomous stage entails the consolidation of the motor commands, allowing actions to be performed in a fluent and flexible manner. While this idea has been around for nearly 50 years, the contribution of explicit strategies to sensorimotor adaptation tasks has largely been ignored in the experimental literature. One reason is that strategies tend to be idiosyncratic and highly variable across individuals. In addition, it can be difficult to assess strategies within the context of the task, and reports at the end of learning are of questionable reliability given that the strategy may change over time. For example, participants have difficulty describing complex force-field perturbations, even if they have a sense that something about the environment has been altered with the onset of the perturbation. For these reasons, studies of explicit processes in sensorimotor adaptation tasks have generally used indirect probes such as changes in reaction time (Fernandez-Ruiz et al., 2011), self-reports (Heuer and Hegele, 2008; Hwang et al., 2006), or susceptibility to dual-task interference (Galea et al., 2010; Taylor and Thoroughman, 2007, 2008). A direct approach was introduced by Mazzoni and Krakauer (2006) in a study of visuomotor rotation. The experimenters described the 45 clockwise perturbation to the participants and instructed them to aim in the counterclockwise direction of the perturbation as a compensatory strategy (Fig. 4). To facilitate strategy use,

5 Strategy Use During Sensorimotor Adaptation

FIGURE 4 Experimental conditions used by Mazzoni and Krakauer (2006) to assess the effect of strategy use in countering a visuomotor rotation. (A) Top left: Before the rotation the participant reaches to a goal target (gray-filled circle) and the visual feedback is veridical. Top right: The participant experiences two trials of the 45 visuomotor rotation. Bottom left: The participant is then instructed to offset this rotation by aiming to a landmark 45 clockwise from the target (unfilled circle). Bottom right: As training continues, target errors (performance) drift in the direction of the aiming target. Note that in all displays, eight circles arranged along an invisible ring, indicating possible target locations were always visible. One turned into the goal target. (B) The top panel shows a standard adaptation with a gradual reduction in directional error over trials. In the bottom panel, the instructions are provided after the first two trials with the rotation. Target error immediately drops to near 0, but then increases (drifts) in the opposite direction. Adapted from Mazzoni and Krakauer (2006).

landmarks were added to the display, positioned 45 apart. Thus, when the target appeared at one location, the participant simply had to aim to the neighboring landmark in order to negate the perturbation. Not surprisingly, the participants were able to use the strategy and immediately succeed in compensating for the rotation. However, as training continued, their performance deteriorated. The reaches drifted in the direction of the strategy (direction was greater than 45 counterclockwise), with the error growing to over 20 . How can we account for this paradoxical result, a situation where performance actually gets worse with practice? Mazzoni and Krakauer (2006) hypothesize that this task design pits two processes against one another, an explicit strategy and implicit sensorimotor adaptation, with the latter winning out. The key here is to

231

232

CHAPTER 9 Systems Interactions

consider the error signal for adaptation. We typically think of the error as the difference between the target location and the feedback location. However, their findings suggest that, the error signal for adaptation is not target error but rather is based on the difference between the predicted and actual outcome, that is, a sensory prediction error. With the instructed strategy, the prediction is no longer at the target location; it is now at the aiming location. Thus, when feedback appears at the target location, the adaptation system is presented with a large error. The internal model is adjusted to reduce this error, resulting in the drift phenomenon where performance error increases across trials. Consistent with this account, participants showed a sizable aftereffect when the rotation was turned off and they were instructed to reach directly to the target. If adaptation were driven by an error based on a comparison of the target location and feedback, neither drift nor an aftereffect should have been observed. These results suggest that adaptation is insensitive to whether or not the movement achieves its goal. Movement goals and sensory predictions are usually well-aligned: In the standard visuomotor adaptation task, participants aim for the target location and expect their movement to terminate at that location. One might suppose that Mazzoni and Krakauer (2006) have created a clever, yet ecologically unrealistic situation. However, there are likely many situations in which the final outcome of an action deviates from the initial planned action. Consider our baseball example again in which a pitcher is attempting to throw a 12–6 curveball, one in which the pitch initially looks to cross the plate near the hitter’s chest and then drops down to the ankles (thus, 12–6 as in the positions on a clock face). For this pitch, the 12 o’clock aiming direction does not match the final 6 o’clock position. Given the results of Mazzoni and Krakauer (2006), we might expect our pitcher to start elevating the pitches as they correct for the difference between the predicted and actual position of the ball (at least if the adaptation system does not have a sense of the effect of a curveball). At present, we can only rely on anecdotal evidence with respect to pitching—certainly there are many regretful pitchers who have watched a mislocated curveball result in a home run. It is also possible that, with practice, skilled actions come to reach a balance between the effects of strategic and adaptation processes. Mazzoni and Krakauer limited practice with the rotation to just 80 trials, observing the endpoint error rise to approximately 25 . By their model, we would expect that this error would, with extended practice, continue to grow up to 45 , at which point the sensory prediction error would be zero. We tested this prediction by using their aiming strategy task in an extended training session (Taylor and Ivry, 2011). Consistent with their results, participants initially drifted in the direction of the strategy. However, with continued training, the target errors began to decrease and, by 200 trials or so, were near zero (Fig. 5A). Interestingly, when the rotation was turned off after 320 trials, an aftereffect of around 20 was observed. To account for this nonmonotonic behavior, we developed a novel state space model in which performance is the result of two processes, each driven by its unique error term (Fig. 5B; Eqs. 3 and 4).

5 Strategy Use During Sensorimotor Adaptation

FIGURE 5 Extended training while using an explicit strategy to counter a 45 rotation. The target error becomes small when the strategy is implemented (Trial 42). The target error drifts in the direction of the strategy for about 80 trials and then reverses with performance eventually becoming asymptotic with little target error. However, a large aftereffect is observed when the rotation is turned off and participants stop using the strategy, revealing the magnitude of implicit adaptation. Circles: Observed data for the group. Solid curve: Fit of the two-process model fit. (B) Implicit adaptation is based on a sensory prediction error (aiming error, gray), defined as the difference between the aiming location and the feedback location. Strategy adjustment is based on target error (black), the difference between the target location and the feedback location. Adapted from Taylor and Ivry (2011).

etarget ¼ ðr n  ^r n Þ þ S n n eaiming n

¼ ðr n  ^r n Þ

^ rn þ r nþ1 ¼ A^

(3) (4)

Beaiming n

(5)

S nþ1 ¼ Esn þ Fetarget n

(6)

Target error is directly influenced by the strategy (s). It can be immediately offset by the introduction of a strategy to offset an external perturbation. Aiming error, in contrast, is not directly influenced by the strategy. Rather, it represents a sensory prediction error, defined as the difference between the aiming location and the feedback location (see Taylor and Ivry, 2011 for derivation). While, the aiming error signal is used by an implicit adaptation system to update an internal model (Eq. 5), the target error is used to update the strategy (Eq. 6). As such, the goal, or performance error is used to adjust the strategy. This two-process, two-error signal model

233

234

CHAPTER 9 Systems Interactions

was capable of capturing the nonmonotonic learning behavior of the participants, as well as more subtle features such as the relationship between the size of error and the size of the aftereffect (Taylor and Ivry, 2011).

6 CEREBELLAR AND NEOCORTICAL CONTRIBUTIONS TO SENSORIMOTOR ADAPTATION The aiming task introduced by Mazzoni and Krakauer (2006) provided a tool for observing the operation of two processes, one based on the use of an explicit strategy and the other driven by implicit adaptation of a forward model. Our modeling work suggests that these two processes operate in a concurrent and, to some degree, independent manner. As a further test of this two-process model, we sought to determine if these processes were associated with distinct neural systems by testing patients with different neurological conditions. As outlined above, many studies have shown that patients with cerebellar pathology are impaired in tasks requiring sensorimotor adaptation. The aiming study offers a novel test of this idea, one in which the patients should actually perform better than controls. Indeed, we set out to test two predictions by comparing the performance of patients with cerebellar ataxia and matched controls on the aiming task. First, we assumed that the patients would have little difficulty using an aiming strategy when given explicit instructions. As such, we expected that, similar to the control participants, they would be able to immediately compensate for a 45 rotation by successfully aiming to a neighboring landmark. Second, and most interesting, we expected that the patients would show attenuated drift given the assumption that the cerebellum is essential for adaptation. As can be seen in Fig. 6, both of these predictions were confirmed (Taylor et al., 2010). After instructed about the strategy, the patients and controls immediately reduced target error. Over the next 80 trials, however, their performance diverged: While the reaches for the control participants showed the characteristic drift pattern (mean maximum drift ¼ 11.3 ), the patients’ movements remained highly accurate in terms of terminating near the target location (mean max. drift ¼ 5.4 ). These results provide compelling evidence, not only that the integrity of the cerebellum is essential for adaptation, but that this process is driven by a sensory prediction error. We assume that these patients have difficulty in generating a prediction of the expected outcome of the movement and are therefore unable to update an internal model. The cerebellar results on the aiming task provide a single dissociation, indicating that the adaption component can be selectively disrupted. Stronger neuropsychological evidence for our two-process model requires showing the reverse, namely, that a different group exhibits a selective disruption of the strategic process. To date, the neural systems associated with a strategic process for sensorimotor learning are unknown. One potential candidate is the prefrontal cortex. Classically, the prefrontal cortex is associated with cognitive control, a catchall phrase to encompass processes such as goal representation, planning, and performance monitoring (Miller and

6 Cerebellar and Neocortical Contributions to Sensorimotor Adaptation

FIGURE 6 Neuropsychological studies of explicit strategy. (A) A 45 counterclockwise rotation was imposed for trials between the dotted vertical lines and participants were instructed to reach to the landmark clockwise to the target. Patients with cerebellar degeneration (light circles) showed less drift than the Control participants (dark), as well as smaller aftereffects, consistent with predicted impairment in adaptation. (B) Patients with unilateral lesions in the prefrontal cortex (light) showed greater drift than their matched control group (dark), although the aftereffects were similar. This pattern is indicative of an impairment in strategy change with intact adaptation. (C) Lesion reconstruction for patients with prefrontal cortex damage. The reconstructions are overlaid and individually colored for each patient. Panel A is adapted from Taylor et al. (2010).

235

236

CHAPTER 9 Systems Interactions

Cohen, 2001; Milner, 1963). While these functions are not typically linked to motor learning, a number of neuroimaging studies have shown that the prefrontal cortex, specifically dorsolateral prefrontal cortex, is consistently activated during the sensorimotor adaptation tasks (Floyer-Lea and Matthews, 2004, 2005; Sakai et al., 1998; Shadmehr and Holcomb, 1997). This activation is especially apparent during the early stages of learning (Seidler and Noll, 2008). Indeed, the rate at which participants compensate for visuomotor perturbations during the early stage of learning correlates strongly with the magnitude of the BOLD response in dorsolateral prefrontal cortex activation (Anguera et al., 2010). One hypothesis offered to account for this pattern is that the large performance errors observed early in learning engage spatial working memory, either as part of a monitoring process or to develop compensatory strategies to respond to the perturbations (Anguera et al., 2009, 2010, 2011, 2012). Indirect evidence supporting a role of prefrontal cortex in sensorimotor learning comes from studies on aging. Older adults consistently show slower learning curves in visuomotor adaptations tasks compared with younger adults (Bock, 2005; Fernandez-Ruiz et al., 2000; Hegele and Heuer, 2010; Hegele and Heuer, 2013; Heuer and Hegele, 2008; Heuer et al., 2011; McNay and Willingham, 1998). Interestingly, these impairments appear to be related to reduced awareness of the perturbation and use of explicit, compensatory strategies (Heuer and Hegele, 2008; McNay and Willingham, 1998). Heuer and Hegele (2008) found that, while younger adults showed less error than older adults when tested on a visuomotor adaptation task, the aftereffects for the two groups were very similar, suggesting that motor adaptation was largely intact in the older adults. To directly test for explicit knowledge of the perturbation, the participants were asked to rotate a ray, originally connecting the start and target position, to an orientation that indicated the direction they should move in order to hit the target. On average, the older adults did not rotate the line as much as the younger adults, suggesting that they had less explicit knowledge of the perturbation. In fact, when this proxy of strategy use was taken into account, the learning curves for the two groups were similar. Finally, lesions of PFC, including naturally occurring lesions and those transiently induced with TMS, can profoundly impair learning on motor learning tasks (Gomez Beldarrain et al., 1999; Ivry et al., 2008; Pascual-Leone et al., 1996; Slachevsky et al., 2001; Slachevsky et al., 2003). In the case of a visuomotor perturbation, patients with PFC lesions have a complete lack of awareness of the perturbation, even when the perturbation is quite large (Slachevsky et al., 2001, 2003). Furthermore, even when the patients are aware of the perturbation, they have difficulty describing it and, perhaps more importantly, have difficulty reporting what action would be required to compensate for the perturbation. Taken together, the neuroimaging, aging, and lesion studies provide evidence not only that the integrity of prefrontal cortex is important for motor learning, but also that it may be specifically related to the employment of strategic processes. Motivated by these findings, we recently tested a group of patients with unilateral lesions of lateral prefrontal cortex (LPFC; eight left-sided lesion, two right-sided lesions), on the strategic aiming task. The lesions for these patients are quite variable, both in size

6 Cerebellar and Neocortical Contributions to Sensorimotor Adaptation

and position, but, as can be seen in Fig. 6C, they all encompass LPFC. We selected patients with minimal receptive language problems so that they could understand the strategy instructions. In addition, the majority of the patients did not suffer significant hemiplegia (at least to the degree that they could make reaching movements). For two of the patients, we had to do the testing with their ipsilesional limb; for the others, the task was performed with the contralesional limb. A deficit in using strategic processes could manifest in different ways. One possibility is that a patient with such an impairment would be unable to implement the strategy or inconsistent in applying the strategy. By this hypothesis, we might expect patients with LPFC damage to show performance functions similar to those observed in typical visuomotor adaptation studies, with a gradual decrease in target error over the course of training. Alternatively, an impaired strategy system might result in amplified drift, with the patients unable to modify their behavior in response to the increasing target error as adaptation builds up. The results were generally consistent with the latter prediction (Fig. 6B). Patients with PFC damage were able to follow the instruction to use the explicit strategy to counter the rotation. However, on average, they showed increased drift (mean maximum drift ¼ 21.8  8.9 ) compared to age-matched controls (11.5  7.4 ), although this difference was only marginally reliable (t(17) ¼ 1.7, p ¼ 0.1). Interestingly, the PFC patients and controls showed similar adaptation as assessed in a final eight trials in which feedback was removed and the participants were asked to stop using the strategy when reaching to the targets. The size of the aftereffect, based on the average of these eight trials was 8.4  2.9 for the PFC group compared to 9.2  3.7 for the controls (t(17) ¼ 0.5, p ¼ 0.6). Although we need to be cautious in interpreting null results, the results suggest that the LPFC group has difficulty modifying an instructed strategy, even though the size of the target error becomes quite pronounced due to adaptation. From visual inspection, only one of the 10 PFC patients showed an abrupt change in target error during training, the clearest signature of strategy change. In contrast, five of the nine age-matched controls showed large trial-by-trial fluctuations in target error. Failure to modify a strategy could be viewed as a form of perseveration, a common problem observed in individuals with PFC lesions on tests of cognitive control (Heaton, 1981; Milner, 1963). Unfortunately, however, we did not include a sufficient number of training trials with the rotation to be confident in our estimates of overt changes in strategy. Taken together, the performance of the cerebellar and PFC group on the strategyaiming task constitutes a double dissociation. By the model outlined in Section 5, the cerebellar group fails to use a sensory prediction error signal to adapt an internal model. In contrast, the PFC fails to use a target error signal to modify a strategy. While these results converge with previous reports using standard sensorimotor tasks, the aiming task offers a cleaner way to isolate these processes, one in which adaptation and strategy change pull the system in opposite directions. Future work will be required to assess the computational role of other neural regions in sensorimotor learning. One obvious candidate is the basal ganglia given its widely discussed role in skill acquisition (Doyon et al., 2009; Shmuelof et al., 2012).

237

238

CHAPTER 9 Systems Interactions

A few studies have examined the performance of patients with basal ganglia pathology on visuomotor adaptation tasks. Individuals with Parkinson’s disease show normal learning rates and sizable aftereffects, suggesting that internal model adaptation is intact (Fernandez-Ruiz et al., 2003; Weiner et al., 1983). However, these patients show reduced savings when retested in a second session, a result interpreted to implicate the basal ganglia in motor consolidation (Bedard and Sanes, 2011; Fernandez-Ruiz et al., 2003; Marinelli et al., 2009). This hypothesis is interesting to consider given the important role of dopamine in reinforcement learning (Fiorillo et al., 2008; Frank et al., 2004; Schultz, 2006; Wise, 2004). If the cerebellum mediates internal model adaptation during learning and conveys this information to cortex (Galea et al., 2011), we can speculate that the basal ganglia helps consolidate this newly formed memory through dopaminergic modulation of cortex (Hosp and Luft, 2013; Koralek et al., 2012). This hypothesis predicts that variation in the level of dopamine, even in neurologically intact individuals, would affect consolidation following motor learning. For example, participants who experience a greater degree of success during initial motor learning, and presumably, have increased reward-related dopamine release, show greater retention compared to participants who learned more slowly, even if the degree of learning during the initial session was comparable between groups (Trempe et al., 2012). Moreover, a recent study found that rewarding motor performance is critical for the retention and expression of the newly acquired motor memories (Pekny et al., 2011). These results suggest that the basal ganglia are not involved in internal model adaptation, but contribute to motor learning through their role in consolidation and, perhaps, the expression (selection) of learned movements. It is also possible that the basal ganglia support processes associated with strategy change, perhaps through reinforcement learning. To date, reinforcement-learning models have been designed to look at classical and instrumental conditioning. In terms of adaptation tasks, it is possible that learning driven by target error is dependent on reinforcement learning, a hypothesis that would suggest a direct role of the basal ganglia in sensorimotor learning. Alternatively, the contribution of the basal ganglia may be more indirect, providing a modulatory input to the cerebral cortex. There is clearly a need to test patients with basal ganglia pathology, or employ dopaminergic manipulations in healthy participants, on tasks that provide probes on adaptation, strategy use, and other learning mechanisms.

7 SYSTEMS INTERACTION IN SENSORIMOTOR LEARNING Our neuropsychological studies provide an example of how cerebellar and cortical learning systems interact to support one form of motor learning. The idea that learning, even for a simple task such as reaching in a perturbed environment, involves the coordinated operation of cortical and subcortical areas is one that has been broadly promoted. For example, neuroimaging studies of skill acquisition consistently show

7 Systems Interaction in Sensorimotor Learning

the engagement of a distributed cortical–subcortical network, with many areas showing similar changes in activation patterns over the course of learning (Doyon and Benali, 2005; Keele et al., 2003; Seidler, 2010). Less clear is what is meant by “interact.” A priori, we tend to assume that these systems operate in an interdependent manner, perhaps with some degree of functional specialization. Our work with the strategy-aiming task, though, requires considering that these different neural systems may operate with considerable independence, reflecting their distinct computational principles. Strategic processes, associated here with the prefrontal cortex, appear to focus on the task goal, using outcome success to evaluate the utility of selected actions. Adaptation processes, associated here with the cerebellum, are concerned with ensuring that an executed movement produces its intended consequences. In standard adaptation studies, these two computations are confounded: the task goal and sensory prediction converge on the same location and we see learning converging in a monotonic fashion toward more accurate movements. From this point of view, it is reasonable to infer that the processes operate in a “coordinated” manner to promote task success. The strategy manipulation allows these signals to be unconfounded, and importantly reveals considerable independence. While there are a number of limitations of our two-process model, it does make explicit a few important points concerning systems interaction. First, the drift phenomenon makes salient that input to the cerebellum is severely constrained. This system does not appear to have access to information about the task goal; in the strategy-aiming task, this means that the cerebellum does not have access to the strategy. Superficially, this would appear to be a very “dumb” system, imposing corrections to sensory prediction errors even if this undermines successful task performance. However, this “dumbness” reveals an appealing simplicity. It may be advantageous to computationally isolate processes designed to handle action selection and action execution. The cerebellum need not consider whether or not the right action has been selected. It is simply given its marching orders to take a motor command and determine if the sensory outcome of the movement matches expectations. Evaluating whether the motor command was appropriate given a particular task context is deferred to noncerebellar systems. A second feature of our two-process model is that both the strategy and adaptation processes operate in a continuous manner. The nonmonotonic function evident in Fig. 5, with an initial rise in target error followed by a reversal, might be viewed as indicative of the successive operation of two processes. However, within the framework of our model, this nonmonotonicity is an emergent property, reflecting the fact that the magnitude of the two-error signals changes over time. With the initial application of the strategy, the prediction error signal is large (45 ), while the target error signal is small (around 0 ). As adaptation occurs, the former becomes smaller and the latter larger. Thus, one need not posit a switch from adaptation to strategy adjustment. Both processes remain operative at all stages of performance. A dynamic tension between the two learning processes is reached, allowing performance to stabilize with reaches successfully terminating near the target location.

239

240

CHAPTER 9 Systems Interactions

Indeed, even when performance becomes asymptotic, the two learning systems continue to operate, pushing performance in opposite directions. As long as sensory feedback appears at a location other than the intended (aiming) location, there will remain a sensory prediction error to drive adaptation. Should this increase target error, an adjustment in the strategy will have the opposite effect. Evidence consistent with this hypothesis is seen in the fact that, with extended training at asymptote, the magnitude of the aftereffect is larger than the magnitude of maximum drift. Drift is constrained by changes in the strategy, whereas the aftereffect provides a probe that is independent of the strategy. One might expect that if participants were trained for an infinite amount of time, adaptation might eventually reach the size of the perturbation. However, even in standard adaptation tasks, performance generally asymptotes at a value less than the perturbation, perhaps because there is some trial-to-trial forgetting of the internal model. Further evidence that overall performance reflects a dynamic tension between two learning processes comes from experiments in which we manipulated the salience of the sensory prediction error (Taylor and Ivry, 2011). In one condition, the aiming landmarks disappeared as soon as the participant initiated each reach. In another condition, the landmarks were only visible during training blocks (no rotation), used to teach participants how to reach 45 away from a target location. During the rotation þ strategy adaptation phase, only the target was visible. In both conditions, participants were successful in compensating for the rotation when given the aiming strategy. However, the manipulations affected adaptation. When the aiming landmarks disappeared, drift was about half the size of that observed with fully visible landmarks. When the landmarks were absent, drift was minimal (see also Benson et al., 2011). Similarly, the aftereffect was reduced in both conditions. These effects were captured by our model by a single parameter that represented a weight assigned to the sensory prediction error signal, the signal driving adaptation. We note that, in theory, sensory prediction error is based on the difference between predicted and actual sensory information, information that is dominated by the visual modality in studies of visuomotor adaptation (Block and Bastian, 2010; Sober and Sabes, 2005). As such, there is no need for the aiming landmarks if the person can generate a representation of the predicted location of the feedback. However, these results indicate that the landmarks serve as a salient proxy of the predicted location, providing a visible point of comparison with the feedback. When the landmark disappears or is absent, the strength of this signal is weakened. Our two-process model provides a computational account of how two learning systems interact during motor learning (Taylor and Ivry, 2011). While results from the strategy-aiming task emphasized the need to consider performance as the composite of two, qualitatively different processes, the basic idea of systems interaction has been advocated in many studies of motor learning (Heuer and Hegele, 2008; Heuer et al., 2011; Hwang et al., 2006; Michel et al., 2007; Redding and Wallace, 1996; Redding et al., 2005; Sulzenbruck and Heuer, 2009; Taylor and Thoroughman, 2007, 2008). Indeed, some researchers have argued that multiple learning processes may best be viewed as competitive; for example, prior explicit

7 Systems Interaction in Sensorimotor Learning

or declarative knowledge has been shown to interfere with statistical learning (Bonatti et al., 2005; Finn and Hudson Kam, 2008) and performing declarative tasks during or subsequently following motor learning can affect recall (Brown and Robertson, 2007; Keisler and Shadmehr, 2010; Taylor and Thoroughman, 2008). However, in these studies, as well as in our strategy-aiming task, the evidence for multiple learning systems has largely been indirect, measured through changes in learning rate, size of aftereffects, or postexperiment tests of knowledge of the perturbation. As noted previously, postexperimental survey data are notoriously unreliable, especially when adaptation becomes complete. Participants may report they were aware that the environment had been perturbed, but after a few hundred trials, fail to recall if they utilized a strategy to facilitate performance. We have recently devised a new task to directly assess systems interactions, focusing on the interplay of strategic aiming and adaptation (Taylor et al., 2014). Participants were provided with a continuous array of visual landmarks surrounding the target and were required to report their aiming direction prior to each movement (Fig. 7). During initial training, the aiming requirement must have seemed odd to the participants: They would report the number “0” prior to each reach and then move directly to the target. Our interest, though, was in their behavior once a 45 rotation was introduced. The participants rapidly reduced their target error, with the data revealing that performance was a combination of a change in the aiming direction and adaptation of an internal model. Interestingly, the time courses for these two processes were quite different. Adaptation was slow and proceeded in a monotonic fashion, with the final state of learning during rotation training approximating the size of the aftereffect. Aiming direction, on the other hand, was highly nonmonotonic, exhibiting large fluctuations early in training, before settling into smaller adjustments late in training. The dynamics here provide further evidence of the interaction between two learning processes. The large changes in the aiming report data provided a quick fix to the perturbation. But the solution must be modified over time because adaptation continued to operate. This result again underscores the inflexible and independent nature of the cerebellum, implementing its learning rule even in the face of effective performance. Less clear is how to consider constraints associated with strategy change. In our original model (Taylor and Ivry, 2011), we applied a state–space model to account for strategy change, with learning designed to monotonically reduce the target error. That is, we used the same gradient descent algorithm to capture strategy change and forward model adaptation, with the former driven by a target error signal and the latter by an aiming error signal. While this formalism seems consistent with cerebellar-based adaptation, it may not be an appropriate characterization of strategy change. The aiming report data are highly variable, at least in the early stages of learning, suggestive of an exploratory process. Similarly, in our original strategy study (Taylor and Ivry, 2011), some of the individual performance functions revealed categorical-like adjustments to the aiming strategy. Strategy change may require an alternative learning process, one that is more amenable to one-trial learning or some sort of exploratory process.

241

242

CHAPTER 9 Systems Interactions

FIGURE 7 Experimental task to directly probe strategy utilization. (A) Prior to reaching, the participant is required to verbally report their aiming location (numbered locations). On the critical trials, the feedback location is rotated 45 counterclockwise from the target. (B) Target error for the Instruction group (black) and a control group that was not required to report their aiming location (light). The rotation was present between 56 and 376 (dashed vertical lines). (C) Angle of aiming location reports for the Instruction group. (D) Estimate of internal model adaptation, calculated by subtraction of the aiming direction (in C) from the target error (in B). Adapted from Taylor et al. (2014).

Exploratory behavior has generally been considered from the perspective of reinforcement learning. Within this general class of models, one distinction has been made between model-free and model-based (Daw et al., 2011; Sutton and Barto, 1998). In model-free reinforcement learning, choices are made by evaluating the

8 Cerebellum and Sensorimotor Learning: Beyond Adaptation

expected values for different actions and selecting the option expected to maximize reward (Daw et al., 2011; Haith and Krakauer, 2013; Sutton and Barto, 1998). This may be sufficient to capture the implicit changes in performance observed by Izawa and colleagues (Izawa and Shadmehr, 2011), where people learned to compensate for a gradual visuomotor rotation with categorical feedback. In model-based reinforcement learning (Daw et al., 2011; Sutton and Barto, 1998), the participant develops a representation of the action space. For sensorimotor adaptation, this might be a representation of the relationship between movements of the hand and the cursor. For a given target location, the participant would use the model to select the action expected to counter the rotation. Alternatively, participants may learn to employ a simpler rule or heuristic of the kind “when the error is to the left, go right; when the error is to the right, go left.” Consistent with this hypothesis, we observed a win-stay/lose-shift pattern in the aiming direction time course data, such that the aim was less likely to change on the next trial if the previous trial was successful. Thus, the application of reinforcement-learning ideas to motor learning tasks may require a hybrid model that incorporates these ideas. Upon first encountering the rotation, rapid learning may be facilitated by an exploratory process where different solutions are tested. Later on, a model-based or lose-shift process might become dominant as the participant makes small changes in the strategy to compensate for ongoing adaptation. While it remains for future work to determine the best formalism of strategy use and strategy change, it is instructive to consider the general applicability of the multiple process idea to sensorimotor adaptation. As a first step toward addressing this question, we tested a group of participants on a standard visuomotor adaptation task (no aiming landmarks, no report) and compared their performance to the group who were provided with landmarks and asked to report their aiming location (Taylor et al., 2014). While forward model adaptation was greater for those tested on the standard task, their aftereffect was considerably less than the actual rotation, despite the fact that performance at the end of the training phase showed minimal error (Fig. 7B). These results suggest that visuomotor adaptation, even in conventional paradigms, entails multiple processes, with adaptation supplemented by an additional aiming “strategy,” even if that strategy may operate at an implicit level (which would suggest that the term “strategy” is a bit of a misnomer).

8 CEREBELLUM AND SENSORIMOTOR LEARNING: BEYOND ADAPTATION To this point, we have emphasized the critical role for the cerebellum in using sensory prediction errors for adapting a forward model. The drift phenomenon observed in the explicit strategy task highlights the modular nature of this mechanism, making clear that this cerebellar process does not have access to the strategy. However, this result does not need to reflect a general feature of the cerebellum. It would be unwise to treat cerebellar computations as reflecting a single process given the extensive

243

244

CHAPTER 9 Systems Interactions

connectivity between this structure and multiple cortical and subcortical regions (Buckner et al., 2011; Krienen and Buckner, 2009; O’Reilly et al., 2010; Strick et al., 2009). Moreover, it may well be that there is an asymmetry in communication between the cerebral cortex and cerebellum. While our work suggests that (some parts of) the cerebellum have little access to cortical representations (e.g., goals/ strategies), cortical representations may be modulated by cerebellar processing. Anatomically, this connectivity has been described as entailing relatively closed loops, with symmetric projections from the neocortex to the cerebellum and vice versa. Functionally, it may be that there is some degree of asymmetry. Our work with the aiming task suggests that (some parts of) the cerebellum have little access to cortical representations; that is, cerebellar adaptation does not appear to be modulated by strategies or goal outcomes. Nonetheless, it may be that cortical representations of these constructs may be modulated by cerebellar processing. This asymmetry hypothesis is motivated, in part, by one intriguing paradox that comes about from the multiple system perspective. As is made transparent in our studies with aiming targets, performance reflects the joint contribution of adaptation and strategy utilization. If the domain of the cerebellum is restricted to the former, we might expect the patients to rely on nonadaptation processes to compensate for a sensory perturbation. For example, they might overcome a visuomotor rotation by making greater use of a strategy. Indeed, this shift should be facilitated when the rotation is large because the target error experienced by the patients is much larger than that experienced by control participants, especially as training proceeds. This prediction is not supported by the data. In both force-field (Gibo et al., 2013; Smith and Shadmehr, 2005) and visuomotor adaptation (Schlerf et al., 2013; VacaPalomares et al., 2013) studies, the patients are similarly affected when presented with small or large perturbations. These results suggest that the requirement for cerebellar integrity is not limited to adaptation, or that the operation of adaptation, in some manner, constrains the operation of other learning processes. For example, the cerebellum, through its connections with prefrontal cortex, may also contribute to the operation of strategic processes. Various functional hypotheses have been put forward to account for processing within a cerebellar–prefrontal network. Perhaps the predictive capability of the cerebellum extends beyond sensory prediction error. For example, the ability to use semantic information to predict the final word of a sentence is disrupted by rTMS of the cerebellum (Lesage et al., 2012), suggesting a more general role of this system in prediction beyond the sensorimotor domain. Or cerebellar–prefrontal loops may be part of a working memory system, helping maintain action plans such as the current state of the strategy (Spencer and Ivry, 2009) or simulating outcomes for different aiming locations (see Strick et al., 2009). In a similar vein, the cerebellum may also be a critical node in a reinforcementlearning network. While studies of reinforcement learning have focused on the basal ganglia and orbitofrontal cortex, the BOLD response in the cerebellum has also been found to be correlated with reward prediction error (O’Doherty et al., 2003; Seymour et al., 2004). Moreover, one study showed that patients with focal cerebellar lesions

9 Conclusions

were impaired in reward-based learning tasks, having difficulty in both learning to associate arbitrary objects with different reward values and in reassigning these values when the stimulus-outcome associations were reversed (Thoma et al., 2008). The computational role for the cerebellum in such tasks remains unclear, but recent studies have identified anatomical projections between the cerebellum and basal ganglia (Bostan et al., 2010; Hoshi et al., 2005). One hypothesis to consider is that this pathway allows the reward prediction system to differentiate between errors in selection and errors in execution. In standard reinforcement-learning models, the failure to obtain an expected reward comes about because the selected object was erroneously over-valued. However, an expected reward will also be missed if the required action is not executed properly. To return to the world of baseball example, consider a pitcher who believes a particular batter cannot hit a curveball. If the pitcher delivers a beautiful curve and the batter hits a home run, the pitcher should use the negative prediction error to update his beliefs, becoming hesitant to throw a curveball when the batter next hits. But what if the home run occurs when a pitched curveball fails to curve? In this context, the pitcher might be wise to ignore the negative prediction error and maintain his beliefs about the batter. We suggest that cerebellar projects to the basal ganglia, and perhaps also to prefrontal cortex, are essential for discriminating between different types of errors. That is, the output from the cerebellum could modulate reward prediction errors, with the occurrence of a sensory prediction error providing a signal to deemphasize a reward prediction error. This hypothesis offers a novel account for the failure of patients with cerebellar pathology to use strategic processes in a compensatory manner in sensorimotor adaptation tasks. A noisy sensory prediction error system not only impairs adaptation but also removes a modulatory input to the reward prediction error system. Even if the patient develops an appropriate strategy, misreaches resulting from poor adaptation or execution would not be discounted and lead to negative reward prediction signals that diminishes the value of that strategy. These ideas remain to be tested, but offer a computational perspective on how interactions might arise between the cerebellum, cortex, and basal ganglia that build on their unique representational capabilities.

9 CONCLUSIONS The work reviewed here makes clear that sensorimotor learning is not the result of a single process, but rather involves a multiplicity of different learning processes. We have emphasized that learning, even in simple perturbation studies, can occur at multiple levels: cerebellar-based internal model adaptation can improve action execution, while prefrontal cortex and basal ganglia based processes may improve action selection. A critical question for future study is to determine the extent to which these systems operate in relative independence, or, more generally, describe the manner in which they interact.

245

246

CHAPTER 9 Systems Interactions

Nonetheless, it is important to keep in mind that their joint operation is advantageous in terms of hastening overall motor learning, at least when learning is defined as improved task performance. The current architecture reflects the operation of distinct evolutionary processes, the adaptation of a host of mechanisms that were selected to solve different problems (pardon the pun). In combination, a multipronged learning system can be flexible and produce performance gains over multiple timescales. While the cerebellum can learn to make predictions about the sensory consequences of a selected action to improve motor execution, this process is relatively slow, limited, by stability considerations, to small trial-by-trial increments. Human competence is greatly enhanced by our ability to employ explicit strategies, utilize social learning, and exploit feedback in evaluating different action options.

Acknowledgments We thank Samuel Brudner and Ludovica Labruna for help with the chapter preparation and Samuel McDougle for helpful comments. J. T. was supported by R01NS084948 from the National Institute of Neurological Disorders and Stroke. R. I. was supported by R01HD060306 from the National Institute of Child Health and Human Development and R01NS074917 from the National Institute of Neurological Disorders and Stroke.

References Albus, J.S., 1971. A theory of cerebellar function. Math. Biosci. 10, 25–61. Albus, J.S., 1989. Marr and Albus theories of the cerebellum: two early models of associative memory. In: Proceedings of IEEE compcon. Anguera, J.A., Seidler, R.D., Gehring, W.J., 2009. Changes in performance monitoring during sensorimotor adaptation. J. Neurophysiol. 102, 1868–1879. Anguera, J.A., Reuter-Lorenz, P.A., Willingham, D.T., Seidler, R.D., 2010. Contributions of spatial working memory to visuomotor learning. J. Cogn. Neurosci. 22, 1917–1930. Anguera, J.A., Reuter-Lorenz, P.A., Willingham, D.T., Seidler, R.D., 2011. Failure to engage spatial working memory contributes to age-related declines in visuomotor learning. J. Cogn. Neurosci. 23, 11–25. Anguera, J.A., Bernard, J.A., Jaeggi, S.M., Buschkuehl, M., Benson, B.L., Jennett, S., Humfleet, J., Reuter-Lorenz, P.A., Jonides, J., Seidler, R.D., 2012. The effects of working memory resource depletion and training on sensorimotor adaptation. Behav. Brain Res. 228, 107–115. Babinski, J., 1896. Sur le reflexe cutane plantaire dans certaines affections organiques du system nerveux central. Compt. Rend. Soc. Biol. 3, 207. Babinski, J., 1902. Sur le role du cervelat dans actes volitionnels necessitant une sucession rapide de mouvements-diadiococnesie. Rev. Neurol. X, 1013–1015. Baizer, J., Glickstein, M., 1974. Role of the cerebellum in prism adaptation. J. Physiol. Lond. 236, 34–35. Bedard, P., Sanes, J.N., 2011. Basal ganglia-dependent processes in recalling learned visualmotor adaptations. Exp. Brain Res. 209, 385–393.

References

Benson, B.L., Anguera, J.A., Seidler, R.D., 2011. A spatial explicit strategy reduces error but interferes with sensorimotor adaptation. J. Neurophysiol. 105, 2843–2851. Block, H.J., Bastian, A.J., 2010. Sensory reweighting in targeted reaching: effects of conscious effort, error history, and target salience. J. Neurophysiol. 103, 206–217. Bock, O., 2005. Components of sensorimotor adaptation in young and elderly subjects. Exp. Brain Res. 160, 259–263. Bonatti, L.L., Pena, M., Nespor, M., Mehler, J., 2005. Linguistic constraints on statistical computations: the role of consonants and vowels in continuous speech processing. Psychol. Sci. 16, 451–459. Bostan, A.C., Dum, R.P., Strick, P.L., 2010. The basal ganglia communicate with the cerebellum. Proc. Natl. Acad. Sci. U.S.A 107, 8452–8456. Boyden, E.S., Katoh, A., Pyle, J.L., Chatila, T.A., Tsien, R.W., Raymond, J.L., 2006. Selective engagement of plasticity mechanisms for motor memory storage. Neuron 51, 823–834. Brown, R.M., Robertson, E.M., 2007. Off-line processing: reciprocal interactions between declarative and procedural memories. J. Neurosci. 27, 10468–10475. Buckner, R.L., Krienen, F.M., Castellanos, A., Diaz, J.C., Yeo, B.T., 2011. The organization of the human cerebellum estimated by intrinsic functional connectivity. J. Neurophysiol. 106, 2322–2345. Carey, S., Bartlett, E., 1978. Acquiring a single new word. Papers and Reports on Child Language Development 15, 17–29. Cheng, D.T., Disterhoft, J.F., Power, J.M., Ellis, D.A., Desmond, J.E., 2008. Neural substrates underlying human delay and trace eyeblink conditioning. Proc. Natl. Acad. Sci. U.S.A. 105, 8108–8113. Clark, G.A., Mccormick, D.A., Lavond, D.G., Thompson, R.F., 1984. Effects of lesions of cerebellar nuclei on conditioned behavioral and hippocampal neuronal responses. Brain Res. 291, 125–136. Criscimagna-Hemminger, S.E., Bastian, A.J., Shadmehr, R., 2010. Size of error affects cerebellar contributions to motor learning. J. Neurophysiol. 103, 2275–2284. Dalton, J.C., 1861. On the cerebellum, as the center of co-ordination of voluntary movement. Am. J. Med. Sci. 61, 83–88. Daw, N.D., O’doherty, J.P., Dayan, P., Seymour, B., Dolan, R.J., 2006. Cortical substrates for exploratory decisions in humans. Nature 441, 876–879. Daw, N.D., Gershman, S.J., Seymour, B., Dayan, P., Dolan, R.J., 2011. Model-based influences on humans’ choices and striatal prediction errors. Neuron 69, 1204–1215. Diedrichsen, J., White, O., Newman, D., Lally, N., 2010. Use-dependent and error-based learning of motor behaviors. J. Neurosci. 30, 5159–5166. Donchin, O., Rabe, K., Diedrichsen, J., Lally, N., Schoch, B., Gizewski, E.R., Timmann, D., 2012. Cerebellar regions involved in adaptation to force field and visuomotor perturbation. J. Neurophysiol. 107, 134–147. Doyon, J., Benali, H., 2005. Reorganization and plasticity in the adult brain during learning of motor skills. Curr. Opin. Neurobiol. 15, 161–167. Doyon, J., Bellec, P., Amsel, R., Penhune, V., Monchi, O., Carrier, J., Lehericy, S., Benali, H., 2009. Contributions of the basal ganglia and functionally related brain structures to motor learning. Behav. Brain Res. 199, 61–75. Encina, E.A., 2013. Orioles Director of Pitching Development Rick Peterson’s “String” Theory of Pitching Down in the Zone. The Baltimore Sun. Fernandez-Ruiz, J., Hall, C., Vergara, P., Diiaz, R., 2000. Prism adaptation in normal aging: slower adaptation rate and larger aftereffect. Brain Res. Cogn. Brain Res. 9, 223–226.

247

248

CHAPTER 9 Systems Interactions

Fernandez-Ruiz, J., Diaz, R., Hall-Haro, C., Vergara, P., Mischner, J., Nunez, L., Drucker-Colin, R., Ochoa, A., Alonso, M.E., 2003. Normal prism adaptation but reduced after-effect in basal ganglia disorders using a throwing task. Eur. J. Neurosci. 18, 689–694. Fernandez-Ruiz, J., Wong, W., Armstrong, I.T., Flanagan, J.R., 2011. Relation between reaction time and reach errors during visuomotor adaptation. Behav. Brain Res. 219, 8–14. Fine, E.J., Ionita, C.C., Lohr, L., 2002. The history of the development of the cerebellar examination. Semin. Neurol. 22, 375–384. Finn, A.S., Hudson Kam, L.C., 2008. The curse of knowledge: first language knowledge impairs adult learners’ use of novel statistics for word segmentation. Cognition 108, 477–499. Fiorillo, C.D., Newsome, W.T., Schultz, W., 2008. The temporal precision of reward prediction in dopamine neurons. Nat. Neurosci. 11, 966–973. Fitts, P.M., Posner, M.I., 1967. Human Performance. Brooks/Cole Pub. Co., Belmont, California. Flourens, P., 1824. Recherches expe´rimentales sur les proprie´te´s et les fonctions du syste`me nerveux, dans les animaux verte´bre´s, first ed. Chez Crevot, Paris. Floyer-Lea, A., Matthews, P.M., 2004. Changing brain networks for visuomotor control with increased movement automaticity. J. Neurophysiol. 92, 2405–2412. Floyer-Lea, A., Matthews, P.M., 2005. Distinguishable brain activation networks for shortand long-term motor skill learning. J. Neurophysiol. 94, 512–518. Frank, M.J., Seeberger, L.C., O’reilly, R.C., 2004. By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science 306, 1940–1943. Fukuda, J., Highstein, S.M., Ito, M., 1972. Cerebellar inhibitory control of the vestibulo-ocular reflex investigated in rabbit 3rd nucleus. Exp. Brain Res. 14, 511–526. Galea, J.M., Sami, S.A., Albert, N.B., Miall, R.C., 2010. Secondary tasks impair adaptation to step- and gradual-visual displacements. Exp. Brain Res. 202, 473–484. Galea, J.M., Vazquez, A., Pasricha, N., De Xivry, J.J., Celnik, P., 2011. Dissociating the roles of the cerebellum and motor cortex during adaptive learning: the motor cortex retains what the cerebellum learns. Cereb. Cortex 21, 1761–1770. Gao, Z., Van Beugen, B.J., De Zeeuw, C.I., 2012. Distributed synergistic plasticity and cerebellar learning. Nat. Rev. Neurosci. 13, 619–635. Gibo, T.L., Criscimagna-Hemminger, S.E., Okamura, A.M., Bastian, A.J., 2013. Cerebellar motor learning: are environment dynamics more important than error size? J. Neurophysiol. 110, 322–333. Gilbert, P.F., Thach, W.T., 1977. Purkinje cell activity during motor learning. Brain Res. 128, 309–328. Gomez Beldarrain, M., Grafman, J., Pascual-Leone, A., Garcia-Monco, J.C., 1999. Procedural learning is impaired in patients with prefrontal lesions. Neurology 52, 1853–1860. Hadipour-Niktarash, A., Lee, C.K., Desmond, J.E., Shadmehr, R., 2007. Impairment of retention but not acquisition of a visuomotor skill through time-dependent disruption of primary motor cortex. J. Neurosci. 27, 13413–13419. Haith, A.M., Krakauer, J.W., 2013. Model-based and model-free mechanisms of human motor learning. In: Richardson, M.J., Riley, M.A., Schockley, K. (Eds.), Progress in Motor Control VII: Neural Computational and Dynamic Approaches. Springer, New York, pp. 1–21. Heaton, R.K., 1981. The Wisconsin Card Sorting Test manual. Psychological Assessment Resources Inc., Odessa, TX. Hegele, M., Heuer, H., 2010. Adaptation to a direction-dependent visuomotor gain in the young and elderly. Psychol. Res. 74, 21–34.

References

Hegele, M., Heuer, H., 2013. Age-related variations of visuomotor adaptation result from both the acquisition and the application of explicit knowledge. Psychol. Aging 28, 333–339. Held, R., Hein, A., 1958. Adaptation of disarranged hand-eye coordination contigent upon afferent stimulation. Percept. Mot. Skills 8, 87–90. Helmholtz, H.E.F., 1909/1962. Treatise on Physiological Optics. Dover, New York. Heuer, H., Hegele, M., 2008. Adaptation to visuomotor rotations in younger and older adults. Psychol. Aging 23, 190–202. Heuer, H., Hegele, M., Sulzenbruck, S., 2011. Implicit and explicit adjustments to extrinsic visuo-motor transformations and their age-related changes. Hum. Mov. Sci. 30, 916–930. Hinder, M.R., Tresilian, J.R., Riek, S., Carson, R.G., 2008. The contribution of visual feedback to visuomotor adaptation: how much and when? Brain Res. 1197, 123–134. Holmes, G., 1939. The cerebellum of man. The Hughlings Jackson memorial lecture. Brain 62, 1–30. Hoshi, E., Tremblay, L., Feger, J., Carras, P.L., Strick, P.L., 2005. The cerebellum communicates with the basal ganglia. Nat. Neurosci. 8, 1491–1493. Hosp, J.A., Luft, A.R., 2013. Dopaminergic meso-cortical projections to M1: role in motor learning and motor cortex plasticity. Front. Neurol. 4, 145. Huang, V.S., Haith, A., Mazzoni, P., Krakauer, J.W., 2011. Rethinking motor learning and savings in adaptation paradigms: model-free memory for successful actions combines with internal models. Neuron 70, 787–801. Hwang, E.J., Smith, M.A., Shadmehr, R., 2006. Dissociable effects of the implicit and explicit memory systems on learning control of reaching. Exp. Brain Res. 173, 425–437. Ito, M., 1974. The control mechanisms of cerebellar motor system. In: Schimidt, F.D., Worden, F.G. (Eds.), The Neuroscience. MIT Press, Cambridge, pp. 293–303. Ito, M., 2006. Cerebellar circuitry as a neuronal machine. Prog. Neurobiol. 78, 272–303. Ito, M., Sakurai, M., Tongroach, P., 1982. Climbing fibre induced depression of both mossy fibre responsiveness and glutamate sensitivity of cerebellar Purkinje cells. J. Physiol. 324, 113–134. Ivry, R.B., Schlerf, J., Xu, J., Klemfuss, N., Griffiths, T., 2008. Strategic and Recalibration Processes During Visumotor Rotation in Cerebellar Ataxia, Society for Neuroscience Abstract, Washinton D.C. Izawa, J., Shadmehr, R., 2011. Learning from sensory and reward prediction errors during motor adaptation. PLoS Comput. Biol. 7, e1002012. Izawa, J., Criscimagna-Hemminger, S.E., Shadmehr, R., 2012. Cerebellar contributions to reach adaptation and learning sensory consequences of action. J. Neurosci. 32, 4230–4239. Jansen, J., Brodal, A., 1954. Aspects of Cerebellar Anatomy. Johan Grundt Tanum, Oslo, Norway. Keele, S.W., Ivry, R., Mayr, U., Hazeltine, E., Heuer, H., 2003. The cognitive and neural architecture of sequence representation. Psychol. Rev. 110, 316–339. Keisler, A., Shadmehr, R., 2010. A shared resource between declarative memory and motor memory. J. Neurosci. 30, 14817–14823. Kellet, D.O., Fukunaga, I., Chen-Kubota, E., Dean, P., Yeo, C.H., 2010. Memory consolidation in the cerebellar cortex. PLoS One 5 (7), e11737. Kitago, T., Ryan, S.L., Mazzoni, P., Krakauer, J.W., Haith, A.M., 2013. Unlearning versus savings in visuomotor adaptation: comparing effects of washout, passage of time, and removal of errors on motor memory. Front. Hum. Neurosci. 7, 307.

249

250

CHAPTER 9 Systems Interactions

Koekkoek, S.K., Hulscher, H.C., Dortland, B.R., Hensbroek, R.A., Elgersma, Y., Ruigrok, T.J., De Zeeuw, C.I., 2003. Cerebellar LTD and learning-dependent timing of conditioned eyelid responses. Science 301, 1736–1739. Koralek, A.C., Jin, X., Long 2nd., J.D., Costa, R.M., Carmena, J.M., 2012. Corticostriatal plasticity is necessary for learning intentional neuroprosthetic skills. Nature 483, 331–335. Krienen, F.M., Buckner, R.L., 2009. Segregated fronto-cerebellar circuits revealed by intrinsic functional connectivity. Cereb. Cortex 19, 2485–2497. Lake, B.M., Salakhutdinov, R., Gross, J., Tenenbaum, J.B., 2011. One shot learning of simple visual concepts. In: Proceedings of the 33rd Annual Conference of the Cognitive Science Society. Lesage, E., Morgan, B.E., Olson, A.C., Meyer, A.S., Miall, R.C., 2012. Cerebellar rTMS disrupts predictive language processing. Curr. Biol. 22, R794–R795. Li, J., Daw, N.D., 2011. Signals in human striatum are appropriate for policy update rather than value prediction. J. Neurosci. 31, 5504–5511. Logan, C.G., Grafton, S.T., 1995. Functional anatomy of human eyeblink conditioning determined with regional cerebral glucose metabolism and positron-emission tomography. Proc. Natl. Acad. Sci. U.S.A. 92, 7500–7504. Marinelli, L., Crupi, D., Di Rocco, A., Bove, M., Eidelberg, D., Abbruzzese, G., Ghilardi, M.F., 2009. Learning and consolidation of visuo-motor adaptation in Parkinson’s disease. Parkinsonism Relat. Disord. 15, 6–11. Marr, D., 1969. A theory of cerebellar cortex. J. Physiol. 202, 437–470. Marshall, L.H., Magoun, H.W., 1997. Discoveries in the Human Brain. Humana Press, NJ. Martin, T.A., Keating, J.G., Goodkin, H.P., Bastian, A.J., Thach, W.T., 1996. Throwing while looking through prisms. I. Focal olivocerebellar lesions impair adaptation. Brain 119 (Pt. 4), 1183–1198. Mazzoni, P., Krakauer, J.W., 2006. An implicit plan overrides an explicit strategy during visuomotor adaptation. J. Neurosci. 26, 3642–3645. Mccormick, D.A., Thompson, R.F., 1984a. Cerebellum: essential involvement in the classically conditioned eyelid response. Science 223, 296–299. Mccormick, D.A., Thompson, R.F., 1984b. Neuronal responses of the rabbit cerebellum during acquisition and performance of a classically conditioned nictitating membrane-eyelid response. J. Neurosci. 4, 2811–2822. Mcnay, E.C., Willingham, D.B., 1998. Deficit in learning of a motor skill requiring strategy, but not of perceptuomotor recalibration, with aging. Learn. Mem. 4, 411–420. Michel, C., Pisella, L., Prablanc, C., Rode, G., Rossetti, Y., 2007. Enhancing visuomotor adaptation by reducing error signals: single-step (aware) versus multiple-step (unaware) exposure to wedge prisms. J. Cogn. Neurosci. 19, 341–350. Miller, E.K., Cohen, J.D., 2001. An integrative theory of prefrontal cortex function. Annu. Rev. Neurosci. 24, 167–202. Milner, B., 1963. Effects of different brain lesions on card sorting: the role of the frontal lobes. Arch. Neurol. 9, 100–110. Morehead, J.R., Crossley, M., Ivry, R.B., 2013. Savings upon re-aiming in visuomotor adaptation. Presented at the Translational and Computational Motor Control Conference, San Diego, CA. Morton, S.M., Bastian, A.J., 2004. Prism adaptation during walking generalizes to reaching and requires the cerebellum. J. Neurophysiol. 92, 2497–2509. Nitsche, M.A., Jakoubkova, M., Thirugnanasambandam, N., Schmalfuss, L., Hullemann, S., Sonka, K., Paulus, W., Trenkwalder, C., Happe, S., 2010. Contribution of the premotor

References

cortex to consolidation of motor sequence learning in humans during sleep. J. Neurophysiol. 104, 2603–2614. O’Reilly, J.X., Beckmann, C.F., Tomassini, V., Ramnani, N., Johansen-Berg, H., 2010. Distinct and overlapping functional zones in the cerebellum defined by resting state functional connectivity. Cereb. Cortex. 20, 953–965. O’doherty, J.P., Dayan, P., Friston, K., Critchley, H., Dolan, R.J., 2003. Temporal difference models and reward-related learning in the human brain. Neuron 38, 329–337. Pascual-Leone, A., Wassermann, E.M., Grafman, J., Hallett, M., 1996. The role of the dorsolateral prefrontal cortex in implicit procedural learning. Exp. Brain Res. 107, 479–485. Pekny, S.E., Criscimagna-Hemminger, S.E., Shadmehr, R., 2011. Protection and expression of human motor memories. J. Neurosci. 31, 13829–13839. Peled, A., Karniel, A., 2012. Knowledge of performance is insufficient for implicit visuomotor rotation adaptation. J. Mot. Behav. 44, 185–194. Perrett, S.P., Ruiz, B.P., Mauk, M.D., 1993. Cerebellar cortex lesions disrupt learningdependent timing of conditioned eyelid responses. J. Neurosci. 13, 1708–1718. Rabe, K., Livne, O., Gizewski, E.R., Aurich, V., Beck, A., Timmann, D., Donchin, O., 2009. Adaptation to visuomotor rotation and force field perturbation is correlated to different brain areas in patients with cerebellar degeneration. J. Neurophysiol. 101, 1961–1971. Raymond, J.L., Lisberger, S.G., Mauk, M.D., 1996. The cerebellum: a neuronal learning machine? Science 272, 1126–1131. Redding, G.M., Wallace, B., 1996. Adaptive spatial alignment and strategic perceptual-motor control. J. Exp. Psychol. Hum. Percept. Perform. 22, 379–394. Redding, G.M., Rossetti, Y., Wallace, B., 2005. Applications of prism adaptation: a tutorial in theory and method. Neurosci. Biobehav. Rev. 29, 431–444. Reis, J., Schambra, H.M., Cohen, L.G., Buch, E.R., Fritsch, B., Zarahn, E., Celnik, P.A., Krakauer, J.W., 2009. Noninvasive cortical stimulation enhances motor skill acquisition over multiple days through an effect on consolidation. Proc. Natl. Acad. Sci. U.S.A. 106, 1590–1595. Rutledge, R.B., Lazzaro, S.C., Lau, B., Myers, C.E., Gluck, M.A., Glimcher, P.W., 2009. Dopaminergic drugs modulate learning rates and perseveration in Parkinson’s patients in a dynamic foraging task. J. Neurosci. 29, 15104–15114. Sakai, K., Hikosaka, O., Miyauchi, S., Takino, R., Sasaki, Y., Putz, B., 1998. Transition of brain activation from frontal to parietal areas in visuomotor sequence learning. J. Neurosci. 18, 1827–1840. Scheidt, R.A., Dingwell, J.B., Mussa-Ivaldi, F.A., 2001. Learning to move amid uncertainty. J. Neurophysiol. 86, 971–985. Schlerf, J.E., Xu, J., Klemfuss, N.M., Griffiths, T.L., Ivry, R.B., 2013. Individuals with cerebellar degeneration show similar adaptation deficits with large and small visuomotor errors. J. Neurophysiol. 109, 1164–1173. Schonewille, M., Gao, Z., Boele, H.J., Veloz, M.F., Amerika, W.E., Simek, A.A., De Jeu, M.T., Steinberg, J.P., Takamiya, K., Hoebeek, F.E., Linden, D.J., Huganir, R.L., De Zeeuw, C.I., 2011. Reevaluating the role of LTD in cerebellar motor learning. Neuron 70, 43–50. Schubert, M.C., Zee, D.S., 2010. Saccade and vestibular ocular motor adaptation. Restor. Neurol. Neurosci. 28, 9–18. Schultz, W., 1998. Predictive reward signal of dopamine neurons. J. Neurophysiol. 80, 1–27. Schultz, W., 2006. Behavioral theories and the neurophysiology of reward. Annu. Rev. Psychol. 57, 87–115.

251

252

CHAPTER 9 Systems Interactions

Scoville, W.B., Milner, B., 1957. Loss of recent memory after bilateral hippocampal lesions. J. Neurol. Neurosurg. Psychiatry 20, 11–21. Seidler, R.D., 2010. Neural correlates of motor learning, transfer of learning, and learning to learn. Exerc. Sport Sci. Rev. 38, 3–9. Seidler, R.D., Noll, D.C., 2008. Neuroanatomical correlates of motor acquisition and motor transfer. J. Neurophysiol. 99, 1836–1845. Seymour, B., O’doherty, J.P., Dayan, P., Koltzenburg, M., Jones, A.K., Dolan, R.J., Friston, K.J., Frackowiak, R.S., 2004. Temporal difference models describe higher-order learning in humans. Nature 429, 664–667. Shabbott, B.A., Sainburg, R.L., 2010. Learning a visuomotor rotation: simultaneous visual and proprioceptive information is crucial for visuomotor remapping. Exp. Brain Res. 203, 75–87. Shadmehr, R., Holcomb, H.H., 1997. Neural correlates of motor memory consolidation. Science 277, 821–825. Shmuelof, L., Huang, V.S., Haith, A.M., Delnicki, R.J., Mazzoni, P., Krakauer, J.W., 2012. Overcoming motor “forgetting” through reinforcement of learned actions. J. Neurosci. 32, 14617–14621. Slachevsky, A., Pillon, B., Fourneret, P., Pradat-Diehl, P., Jeannerod, M., Dubois, B., 2001. Preserved adjustment but impaired awareness in a sensory-motor conflict following prefrontal lesions. J. Cogn. Neurosci. 13, 332–340. Slachevsky, A., Pillon, B., Fourneret, P., Renie, L., Levy, R., Jeannerod, M., Dubois, B., 2003. The prefrontal cortex and conscious monitoring of action: an experimental study. Neuropsychologia 41, 655–665. Smith, M.A., Shadmehr, R., 2005. Intact ability to learn internal models of arm dynamics in Huntington’s disease but not cerebellar degeneration. J. Neurophysiol. 93, 2809–2821. Smith, M.A., Ghazizadeh, A., Shadmehr, R., 2006. Interacting adaptive processes with different timescales underlie short-term motor learning. PLoS Biol. 4, e179. Sober, S.J., Sabes, P.N., 2005. Flexible strategies for sensory integration during motor planning. Nat. Neurosci. 8, 490–497. Spencer, R.M., Ivry, R.B., 2009. Sequence learning is preserved in individuals with cerebellar degeneration when the movements are directly cued. J. Cogn. Neurosci. 21, 1302–1310. Stoloff, R.H., Taylor, J.A., Xu, J., Ridderikhoff, A., Ivry, R.B., 2011. Effect of reinforcement history on hand choice in an unconstrained reaching task. Front. Neurosci. 5, 41. Strick, P.L., Dum, R.P., Fiez, J.A., 2009. Cerebellum and nonmotor function. Annu. Rev. Neurosci. 32, 413–434. Sulzenbruck, S., Heuer, H., 2009. Functional independence of explicit and implicit motor adjustments. Conscious. Cogn. 18, 145–159. Sutton, R.S., Barto, A.G., 1998. Reinforcement Learning: An Introduction. MIT Press, Cambridge, MA. Taylor, J.A., Ivry, R.B., 2011. Flexible cognitive strategies during motor learning. PLoS Comput. Biol. 7, e1001096. Taylor, J.A., Thoroughman, K.A., 2007. Divided attention impairs human motor adaptation but not feedback control. J. Neurophysiol. 98, 317–326. Taylor, J.A., Thoroughman, K.A., 2008. Motor adaptation scaled by the difficulty of a secondary cognitive task. PLoS One 3, e2485. Taylor, J.A., Klemfuss, N.M., Ivry, R.B., 2010. An explicit strategy prevails when the cerebellum fails to compute movement errors. Cerebellum 9, 580–586.

References

Taylor, J.A., Krakauer, J.W., Ivry, R.B., 2014. Explicit and implicit contributions to learning in a sensorimotor adaptation task. J. Neurosci. 34, 3023–3032. Thoma, P., Bellebaum, C., Koch, B., Schwarz, M., Daum, I., 2008. The cerebellum is involved in reward-based reversal learning. Cerebellum 7, 433–443. Thoroughman, K.A., Shadmehr, R., 2000. Learning of action through adaptive combination of motor primitives. Nature 407, 742–747. Timmann, D., Drepper, J., Frings, M., Maschke, M., Richter, S., Gerwig, M., Kolb, F.P., 2010. The human cerebellum contributes to motor, emotional and cognitive associative learning. A review. Cortex 46, 845–857. Trempe, M., Sabourin, M., Proteau, L., 2012. Success modulates consolidation of a visuomotor adaptation task. J. Exp. Psychol. Learn. Mem. Cogn. 38, 52–60. Tseng, Y.W., Diedrichsen, J., Krakauer, J.W., Shadmehr, R., Bastian, A.J., 2007. Sensory prediction errors drive cerebellum-dependent adaptation of reaching. J. Neurophysiol. 98, 54–62. Vaca-Palomares, I., Diaz, R., Rodriguez-Labrada, R., Medrano-Montero, J., Vazquez-Mojena, Y., Velazquez-Perez, L., Fernandez-Ruiz, J., 2013. Spinocerebellar ataxia type 2 neurodegeneration differentially affects error-based and strategic-based visuomotor learning. Cerebellum 12, 848–855. Verstynen, T., Sabes, P.N., 2011. How each movement changes the next: an experimental and theoretical study of fast adaptive priors in reaching. J. Neurosci. 31, 10050–10059. Weiner, M.J., Hallett, M., Funkenstein, H.H., 1983. Adaptation to lateral displacement of vision in patients with lesions of the central nervous system. Neurology 33, 766–772. Wise, R.A., 2004. Dopamine, learning and motivation. Nat. Rev. Neurosci. 5, 483–494. Wittmann, B.C., Daw, N.D., Seymour, B., Dolan, R.J., 2008. Striatal activity underlies novelty-based choice in humans. Neuron 58, 967–973. Yeo, C.H., Hardiman, M.J., Glickstein, M., 1985. Classical conditioning of the nictitating membrane response of the rabbit. II. Lesions of the cerebellar cortex. Exp. Brain Res. 60, 99–113. Zarahn, E., Weston, G.D., Liang, J., Mazzoni, P., Krakauer, J.W., 2008. Explaining savings for visuomotor adaptation: linear time-invariant state-space models are not sufficient. J. Neurophysiol. 100, 2537–2548.

253

CHAPTER

Automatic and Controlled Processing in the Corticocerebellar System

10 Narender Ramnani1

Department of Psychology, Royal Holloway, University of London, Egham, UK Corresponding author: Tel.: +441784443519, e-mail address: [email protected]

1

Abstract During learning, performance changes often involve a transition from controlled processing in which performance is flexible and responsive to ongoing error feedback, but effortful and slow, to a state in which processing becomes swift and automatic. In this state, performance is unencumbered by the requirement to process feedback, but its insensitivity to feedback reduces its flexibility. Many properties of automatic processing are similar to those that one would expect of forward models, and many have suggested that these may be instantiated in cerebellar circuitry. Since hierarchically organized frontal lobe areas can both send and receive commands, I discuss the possibility that they can act both as controllers and controlled objects and that their behaviors can be independently modeled by forward models in cerebellar circuits. Since areas of the prefrontal cortex contribute to this hierarchically organized system and send outputs to the cerebellar cortex, I suggest that the cerebellum is likely to contribute to the automation of cognitive skills, and to the formation of habitual behavior which is resistant to error feedback. An important prerequisite to these ideas is that cerebellar circuitry should have access to higher order error feedback that signals the success or failure of cognitive processing. I have discussed the pathways through which such feedback could arrive via the inferior olive and the dopamine system. Cerebellar outputs inhibit both the inferior olive and the dopamine system. It is possible that learned representations in the cerebellum use this as a mechanism to suppress the processing of feedback in other parts of the nervous system. Thus, cerebellar processes that control automatic performance may be completed without triggering the engagement of controlled processes by prefrontal mechanisms.

Keywords cerebellum, prefrontal cortex, dual systems, forward models, learning, cognitive, skills, automatic, controlled

Progress in Brain Research, Volume 210, ISSN 0079-6123, http://dx.doi.org/10.1016/B978-0-444-63356-9.00010-8 © 2014 Elsevier B.V. All rights reserved.

255

256

CHAPTER 10 Cerebellum and Automatic Processing

1 DUAL SYSTEMS, SKILLS, AND HABITS In the cognitive sciences, dual systems accounts propose that two interacting information-processing systems can account for some properties of human performance (Kahneman, 2011; Schneider and Shiffrin, 1977; Shiffrin and Schneider, 1977, 1984). A brief account of their general properties is useful at this point. Borrowing from the terminology used by Kahneman (2011), “System 2” is concerned with controlled, conscious processing of information at an abstract level. It is capable of flexible problem-solving, particularly when such problems are novel. It can present solutions quickly (even after a single trial), but it is reliant on feedback while performance is ongoing and slow. It is also effortful, dependent on a limited capacity working memory system and prone to disruption by concurrently performed tasks. In contrast, “System 1” engages in “automatic” processing at an unconscious level and has the advantage of being fast, feedforward, accurate, and dependent on prior experience rather than ongoing feedback. Performance is not as prone to disruption by other processes that run concurrently with it, allowing multiple tasks to be performed simultaneously. It is also free from the limitations of working memory. However, this system is highly dependent on the cumulative experience derived through practice, so optimal performance is typically not achieved until many trials have elapsed. Performance is also heavily dependent on the context in which behaviors are learned and degrades if attempted outside the context in which informationprocessing skills are acquired. Although System 1 uses feedback to shape future behavior very effectively, ongoing performance is relatively insensitive to feedback. Some dual systems accounts suggest that System 2 processes that are initially conscious and controlled are adopted by System 1 such that they become unconscious and automatic (Evans, 2008). The two processes have complementary properties and work in tandem. As System 1 learns to execute certain tasks, it frees System 2 to engage with novel problems. A commonly used operational definition of automatic performance is that a primary task should be relatively immune to the distracting effects of a concurrently performed secondary task. In this scenario, the parallel operation of System 1 and System 2 means that the learned primary task can be handled by the former and simultaneously, the secondary task by the latter. Of course, many properties of System 1 are comparable to those related to habits. Ashby et al. (2010) discuss criteria that researchers have used to define habitual behavior. One of these is simply the assumption that training beyond asymptotic performance results in automaticity. Other, perhaps more useful, criteria are that behaviors are repeatedly and reliably evoked by particular contexts, and that performance of the habitual behavior is unimpaired by the simultaneous performance of a secondary task. In the literature, habits are often contrasted with “goal-directed” behavior that is also a hallmark of System 2 behavior. In an instrumental context in which attainment of rewards is contingent on the selection of the right response, behaviors are sensitive to knowledge about the relationships between behaviors and outcomes (Dickinson, 1985). They are driven by the prospect of outcomes. If behaviors are goal-directed, then changing the contingency or the value of the reward

2 Control Theory

should elicit corresponding changes in behavior (Dezfouli and Balleine, 2012). On the other hand, although habits develop from goal-directed behaviors, one of their defining features is that they are not immediately sensitive to either the value of the outcome (Adams, 1982) or the contingency between the outcome and the behavior (Dickinson et al., 1998). Thus, habits have properties that are very similar to those of System 1 in which fast performance is achieved partly because there is little requirement to use feedback for online guidance of ongoing behavior. Later, I suggest that cerebellar mechanisms may make important contributions to the properties of System 1. The fact that the primary task is initially controlled by System 2 and transfers to System 1 means that the two systems must interact. I suggest that control theory provides a framework in which this might happen, and that the cerebellum and its connections with the neocortex provide an anatomical basis for this interaction. Control theory has given us a good basis for understanding motor learning (the automation of motor control) in the context of cerebellar interactions with the motor cortex and the motor apparatus. This framework can be extended so that it can accommodate the automation of cognitive operations. It is well known that the operations of System 2 are supported by information processing in the prefrontal cortex. For example, the rehearsal of information in working memory is known to cause sustained activity in the prefrontal cortex across working memory delays (D’Esposito et al., 2000; Funahashi and Takeda, 2002; Goldman-Rakic, 1999; Ramnani and Miall, 2003). Distractors that are presented during working memory rehearsal and are effective in interrupting the maintenance of information in working memory also disrupt sustained delay-related activity (Sakai et al., 2002). Asking subjects to consciously “attend” to actions activates the prefrontal cortex, even if those actions are well-learned and automatic (Jueptner et al., 1997; Leiner et al., 1986, 1989; Passingham, 1996). Some years ago in a previous edition of this publication, Houk (1997) suggested that the interactions between the prefrontal cortex and the cerebellum might support the “automation of thought processes.” This echoes earlier suggestion that cerebellar circuitry contributes to the acquisition of mental skills (Leiner et al., 1986, 1989). Picking up this thread again, I consider the evidence that automation involves the control of cognitive operations to pass from prefrontal circuitry that handles System 2 to cerebellar circuitry that is handled by System 1. Understanding the relationship between control theory and corticocerebellar information processing may provide us with insights into the mechanisms.

2 CONTROL THEORY Inverse and forward models are two principal forms of internal model which have been used to understand cerebellar contributions to learning, but the discussion here is restricted to forward models (Wolpert and Miall, 1996), which is a particularly useful control theoretic construct for explaining cerebellar learning. The inclusion

257

258

CHAPTER 10 Cerebellum and Automatic Processing

of forward models into information-processing systems allows the system as a whole to operate more efficiently by reducing information-processing load and, over time, adjust performance as external demands change. Most authors are agreed on the fundamental properties of forward models (Kawato and Wolpert, 1998; Wolpert and Miall, 1996). The elements involved consist of controllers that issue commands to plants which execute them (Fig. 1). The primary motor cortex might be considered a controller because it generates motor commands and sends them via the corticospinal tract to the spinal cord and motor apparatus (the plant). An “efference copy” of this command is directed to a forward model. This is used by the forward model to generate a prediction about the likely consequences of the command being executed. If the motor cortex issues commands to limb musculature, for example, the forward model will predict the likely proprioceptive feedback that is generated by the resulting limb movement, and compare those predictions with the actual proprioceptive feedback delivered when the commands are executed (Fig. 2). Error signals are generated by calculating the discrepancies between actual and expected feedback. These are gradually minimized in subsequent trials by the fine-tuning of expectations and responses. One of the qualities of a forward model is that it allows the system as a whole to respond rapidly and accurately to inputs that it has processed in the past, without the need to process ongoing feedback in real time to maintain performance. Eliminating the need to process feedback reduces the information-processing demands on the system and allows it to respond quickly. These properties are similar to those of System 1.

Controller

Command

Plant

Efference copy of command Action Forward model Consequences

FIGURE 1 Schematic diagram of the generic organization of control theoretic elements. Controllers issue commands to plants which generate actions. Efference copies of these commands are read by forward models which use them to predict the consequences of actions and compare predictions with actual consequences.

3 The Cerebellum and Forward Models

Controller

Plant

Primary motor cortex

Spinal cord and motor apparatus

Th Movement Cerebellar Cortex Fwd model Sensory consequences

Inf. olive

FIGURE 2 The primary motor cortex as a controller and the spinal cord and motor apparatus as the plant. This schematic figure depicts the collateralization of pathways from the primary motor cortex to the spinal cord, and its inputs into the cerebellar cortex. Forward models in cerebellar circuitry use this to predict the sensory consequences of the resulting movements. These are conveyed to the cerebellar cortex via the inferior olive, and used by the forward model as an error signal to induce supervised learning. Cerebellar outputs return their outputs to the primary motor cortex via the thalamus.

3 THE CEREBELLUM AND FORWARD MODELS The claim that the cerebellum contains forward models requires at least three assumptions to be supported. First, cerebellar circuitry must be capable of supporting plasticity. There is wide theoretical and empirical support for this view (Boyden et al., 2004), and the evidence is discussed in earlier chapters (Chapters 1–3). Second, the circuits that supply the cerebellum with input must be organized so that information reaching the cerebellum must be copies of that destined for other locations. This requires that information reaching the cerebellum must arrive via collaterals of fibers that convey it to other parts of the nervous system. Third, there must be pathways that are capable of supplying the cerebellar cortex with information that is relevant to performance error. This chapter considers the evidence that the corticocerebellar system has such properties and that they can support cognitive skills.

259

260

CHAPTER 10 Cerebellum and Automatic Processing

Corticocerebellar pathways comprise a set of closed loops with generic architecture. Lamina V cells in neocortical areas send outputs to the cerebellar cortex via the pontine nuclei, and these in turn send their outputs back to the same neocortical areas via the cerebellar nuclei and the thalamus. There is a growing body of work that documents the contributions of various neocortical areas with the corticocerebellar system, and this is well documented elsewhere (Ramnani, 2006, 2012; Strick et al., 2009). In brief, in primates cerebellar cortical lobules HIV–HVI and HVIIIB (Kelly and Strick, 2003) and vermal lobules VB–HVIIIB are interconnected with the medial premotor areas as well as the primary motor cortex (Coffman et al., 2011). The hemispheral and vermal parts of lobule VII are interconnected with area 46 of the prefrontal cortex (Kelly and Strick, 2003). There is support for the view that this is probably also the case in humans (O’Reilly et al., 2010), and that in fact the “prefrontal loop” is likely to play a more important role in humans than it does in other primates (Balsters et al., 2010; Ramnani et al., 2006). There are also a number of studies that investigate corticopontine projections using conventional tracers which are reviewed elsewhere (Ramnani, 2006, 2012), and the details of a few that are relevant to this review are discussed below. Can it be argued on the basis of their anatomical traits that corticopontine projections convey efference copy information? The frontal lobe areas that communicate with the cerebellum through the corticopontine system themselves constitute a system of nodes in a functional network. Together, the prefrontal, premotor, and primary motor areas of the frontal lobe form a hierarchically organized system (Fuster, 1990, 1993; Hoshi, 2008; Koechlin et al., 2003; Ramnani and Owen, 2004). Representations of action goals in prefrontal circuitry are translated in increasingly detailed motor plans by cascading information through successive levels of this network via the premotor cortex and the primary motor cortex. At the lowest level of this hierarchy, the primary motor cortex (area 4) sends its motor commands to the spinal cord via the corticospinal tract (Lemon, 2008). Ugolini and Kuypers (1986) demonstrated that the collaterals of corticospinal fibers from the motor cortex terminate in the pontine nuclei. In the same study, they demonstrated that the pontine targets of these inputs send onward projections to areas of the anterior lobe of the cerebellar cortex (lobules HIV–HVI). Putative forward models in these areas may therefore use copies of commands from the motor cortex to predict the proprioceptive consequences of the resulting movement and compare them with the actual consequences. These computations involve calculating the differential between the actual and predicted proprioceptive consequences of movement. Cerebellar forward models therefore require inputs that convey the actual proprioceptive signals that result from the movements generated by the commands. Such inputs do indeed arrive at the same areas and do so directly through ascending fibers via the spinocerebellar system (Oscarsson, 1973; Oscarsson and Uddenberg, 1964, 1965). In control theoretic terms, the spinal cord and musculoskeletal apparatus are the plant that is controlled by commands from the motor cortex (controller), and the circuitry in cerebellar cortical lobules HIV–HVI can learn and retain a forward model of this plant

3 The Cerebellum and Forward Models

(Fig. 2). Ugolini and Kuypers (1986) have suggested that the collateralization of neocortical efferents to the pontine nuclei is a general principle of organization that probably extends to all such inputs rather than being a special case. As is discussed above, the neocortical control of movement does not begin in the motor cortex but with more abstract representations of action in higher parts of the frontal lobe action control network. The primary motor cortex (area 4) receives inputs from both the dorsal the ventral parts of the premotor cortex (area 6; Dum and Strick, 2005). In previous work, I have introduced the idea that the interactions of higher order areas with the cerebellum might also be described in terms of control theoretic principles (Ramnani, 2006). In this system, a given area in the neocortex may adopt the role of controller in one context, but the role of a plant in another. For example, the primary motor cortex is not only the source of commands to the spinal and musculoskeletal system but also the recipient of commands from higher order areas such as the premotor cortex (Fig. 3). Whereas the projections that convey information from the premotor cortex to the primary motor cortex are well understood (Muakkassa and Strick, 1979; Shimazu et al., 2004), it is not widely appreciated that they form collaterals which terminate in the pontine nuclei. Ueta et al. (In press) have shown that neurons in the secondary motor cortex in rats which project to the primary motor cortex arise in layer 5. They send collaterals to the pontine nuclei. Interestingly, neurons that project in the other direction (from primary to secondary motor cortex) tend not to contribute collaterals to the pontine nuclei. Arguably, in relation to its outputs, the primary motor cortex operates as a controller which exerts Controller Premotor cortex

Controller/plant Primary motor cortex

Plant Spinal cord and motor apparatus

Th

Th

Movement Forward models

Cerebellar cortex

Cerebellar cortex

Sensory consequences

Inf. olive

FIGURE 3 The primary motor cortex in a hierarchically organized neocortical system, as both a plant that receives commands, and as a controller that issues commands (as in Fig. 2). Separate areas of the cerebellar cortex may contain forward models that predict the responses of the spinal cord to commands from the primary motor cortex, and the responses to the primary motor cortex to commands from the premotor cortex.

261

262

CHAPTER 10 Cerebellum and Automatic Processing

its influence on the spinal cord and musculoskeletal system, but in relation to its inputs it operates as a plant under the control of the premotor cortex. The circuitry therefore exists which allows commands from the premotor cortex to the primary motor cortex to be copied to the cerebellum via corticopontine collaterals. Forward models in the cerebellum would use efference copies of these commands to generate predictions about the consequences of operations in the motor cortex. Orioli and Strick (1989) showed that the premotor and primary motor cortices connect with separate regions of the cerebellar dentate nucleus via the corticopontocerebellar system. Although this study investigated cerebellar outputs from the dentate nucleus to these cortical areas rather than their inputs to the cerebellum, it has been argued that cerebellar cortical and nuclear territories are modular and that closed loops are an organizing principle in the corticocerebellar system (Strick et al., 2009). Input and output areas are likely to be one and the same. Cerebellar forward models of the primary motor cortex and the spinal cord/musculoskeletal system are therefore likely to operate relatively independently of each other because their cerebellar territories are anatomically segregated (Fig. 3). The dorsal and ventral parts of the premotor cortex in turn receive their inputs from areas of the prefrontal cortex (Lu et al., 1994; Luppino et al., 2003; Takada et al., 2004), and these contribute to the translation of prefrontal activity that codes action goals into premotor activity that relates to the planning of movement (Hoshi, 2008). Although neurons in the prefrontal cortex are known to send projections to the pontine nuclei (Kelly and Strick, 2003; Schmahmann and Pandya, 1995), no study that I am aware of has systematically tested for the collateralization of corticocortical fibers that originate in the prefrontal cortex. Given the ubiquitous nature of collaterals that contribute to the corticopontine system, it seems likely that prefrontal–premotor projections also collateralize and contribute such projections. I have suggested previously that the cerebellar cortical targets of area 46 in hemispheral (Crus I and Crus II) and vermal parts of lobule VII (Kelly and Strick, 2003) could contain forward models that predict the responses of premotor cortex to commands issued by the prefrontal cortex (Ramnani, 2006). In such a scenario, when the prefrontal cortex plays the role of the controller, the premotor cortex becomes the plant because it is subject to a command from the prefrontal cortex (and the prefrontal cortex sends a copy of that command to those areas of the cerebellar cortex). This is in contrast to the role that the premotor cortex plays in which it acts as a controller. The description above relates to the prefrontal cortex at the apex of a decision and action hierarchy, but Ito (2008) similarly suggests that the prefrontal cortex can act as a controller that sends commands to “mental models” in the temporoparietal cortex—this is effectively the “plant” which is manipulated and controlled by the prefrontal cortex to coordinate perception. It is suggested that when cerebellar internal models replace the prefrontal cortex, processing becomes implicit so that thinking becomes “intuitive.” Area 46 is considered as the apex of a hierarchically organized action control system by virtue of its involvement in higher order contributions to action selection and its connectivity with the premotor cortex (Passingham and Wise, 2012). However, area 46 is itself subject to influence from many other prefrontal areas by virtue of

3 The Cerebellum and Forward Models

its connections (Petrides and Pandya, 1999), and executive function depends on the interactions between parts of the prefrontal cortex. For example, in primates, it is interconnected with the anterior prefrontal cortex (area 10) in the frontal pole (Petrides and Pandya, 2007; Ramnani and Owen, 2004; Tsujimoto et al., 2010, 2011, 2012). Area 46 and area 10 both project to the pontine nuclei (Kelly and Strick, 2003; Schmahmann and Pandya, 1995). The pontine terminations of prefrontal–prefrontal collateral projections have not been investigated as far as I am aware, but if such collaterals do exist, then it is possible that cerebellar circuits contain forward models that predict the consequences of interactions between different prefrontal areas. Such mechanisms might therefore be able to contribute to the automation of cognitive operations quite independently of their contributions to the automation of motor control (Ramnani, 2006). The pattern that emerges from this anatomical organization is one in which a hierarchically organized set of neocortical areas operates in a predominantly serial fashion, but each information-processing node in this hierarchy connects with its own cerebellar module that acquires the ability to model and predict its operations (Fig. 4). One of the implications of such a system is that information processing at each level in the serially organized neocortical system can be automated independently of any other level. For example, information processing in the motor cortex and the prefrontal cortex can be acquired by separate forward models, and the acquisition of each is independent of the other. This property is important because it suggests that during skill acquisition the processing of abstract information can proceed independently of information that is highly contextualized (e.g., strongly dependent on the effectors that execute the movement). Higher order forms of motor learning, such as motor sequence learning, involve the development of cognitive skills related to the rules that govern the movement (Tanji and Hoshi, 2001), as well as the acquisition of motor skills related to the kinematic properties of the movements themselves. It is often assumed that the transition from unskilled to skilled sequence execution involves the transition from abstract to contextualized representations (see Taylor and Ivry (2013) for an excellent review of related issues that I have not been able to cover here). However, under the current proposal, it should be possible for the processing of abstract representations related to the serial order of responses to become skilled independently from the skill acquisition related to movement kinematics. Hence, the execution of a set of rules can become increasingly automatic (e.g., less prone to distractors), but these rules might still be applied using different sets of effectors. Under such circumstances, the application of automated rules should still yield performance savings. As with any serially organized system, a disadvantage of the neocortical system described above is that the entire system is prone to failure if only one of its modules fails. If, for example, the premotor cortex fails to convert information pertaining to action goals into a motor plan, then the desired movement will not be executed. The advantage of linking a forward model to each level of the hierarchy is that the forward model may substitute for the neocortical element that it models in the event of its failure. Of course, there is also a requirement for the serial process to continue to

263

CHAPTER 10 Cerebellum and Automatic Processing

Controllers and plants

Decision

Action planning / movement

System 2 Prefrontal cortex (outputs to area 46)

Prefrontal cortex (area 46)

Premotor cortex

Primary motor cortex

Spinal cord and motor apparatus

Dentate

Forward models

System 1

Feedback

264

Cerebellar cortex

Cerebellar cortex Lobule VII/HVII

Cerebellar cortex

Cerebellar cortex Lobules IV–VI

Dorsal

Ventral

Movement Sensory consequences

VTA Inferior olive Decision consequences

FIGURE 4 Simplified schematic diagram of a proposed system that links areas that process decisions and actions, forward models, and error feedback. Excitatory inputs are shown as arrows, and inhibitory inputs are shown as filled circles. The hierarchically organized neocortical and spinal cord system for translating high level goals into movements is depicted in the top row (prefrontal cortex, premotor cortex, primary motor cortex, spinal cord and motor apparatus). The information cascade of from higher order to lower order areas results in movements. Higher level areas send commands to lower level areas and copy these commands to separate forward models, each in a distinct part of the cerebellar cortex. The prefrontal cortex (including area 46 and its inputs) contributes to controlled processing (System 2). Cerebellar cortical forward models that predict responses of prefrontal commands contribute to automatic processing (System 1). Cerebellar outputs from the dentate return projections back to neocortical areas via the thalamus (not shown). Whereas ventral dentate is connected with the prefrontal cortex, the dorsal dentate is connected with the primary motor cortex. Movements generate both sensory consequences (e.g., proprioceptive signals) and decision consequences (e.g., reward signals). Sensory consequences of movement reach the parts of the cerebellar cortex connected with the motor cortex via the inferior olive. However, parts of the inferior olive also have access to decision-related error feedback such as those related to reward expectation. This is conveyed to the cerebellar cortical areas connected with the prefrontal cortex through both the inferior olive and the dopamine system, principally via the ventral tegmental area (VTA). This feedback system may be an important source of error feedback for shaping forward models that contribute to the acquisition of cognitive skills. Outputs from the cerebellar nuclei are known to inhibit not only the inferior olive but also the VTA.

4 Cognitive Habits

completion despite the failure, so the internal model might even act as a controller by supplying a command to the next node in the neocortical hierarchy. Finally, a further advantage of this modular architecture is that individual forward models are much less prone to interference between representations.

4 COGNITIVE HABITS Graybiel (2008) suggests that habits can comprise cognitive expressions of routine (habits of thought) as well as motor expressions of routine. These characteristics suggest that habits are sequential, repetitive, motor, or cognitive behaviors elicited by external or internal triggers that, once released, can go to completion without constant conscious oversight

In doing so, she asserts that habits are not simply goal-directed actions that become habitual movements. Cognitive processes can also become routine and habitual, and can become so independently of the motor demands associated with achieving goals such as obtaining rewards. In contrast to classical conditioning in which responses are shaped by the occurrence of an unconditioned stimulus (US) over which the subject has no control, a defining feature of instrumental learning is that outcomes are contingent on the responses made by a subject. In other words, associations are formed between responses and outcomes. But these associations might themselves be conditional upon the context in which they occur. Responses may lead to specific outcomes, but only under certain circumstances. If those circumstances change, then the response– outcome contingency no longer holds and the same response no longer yields a given outcome. Learning the associations between the cues that signal these circumstances, the responses that subjects select and the resulting outcomes have been termed “conditional visuomotor learning” (Wise and Murray, 2000). The arbitrary nature of the instruction cue that defines the context in which responses lead to certain outcomes engages fundamentally cognitive, rule-related operations that require considerable flexibility. As Wise and Murray (and Passingham) put it, By liberating the sensory guidance of action from the shackles of spatial information, arbitrary visuomotor mapping allows the selection of any action already within a behavioral repertoire in response to any input. As Passingham has emphasized, this adaptation permits the choice of action based on the prevailing context. Wise and Murray (2000).

The use of conditional visuomotor learning has allowed researchers to systematically investigate the basis of this flexibility in the brains of humans and nonhuman primates under rigorous experimental control. Lesion studies have identified the circuitry that is mandatory for conditional visuomotor learning. These include areas of the prefrontal cortex and its connections with temporal lobe visual areas, dorsal parts of the premotor cortex, and the basal ganglia (Wise and Murray, 2000). What is

265

266

CHAPTER 10 Cerebellum and Automatic Processing

important for the purposes of this discussion is that this is also a good model with which to study the transformation of initially flexible rules into habits and to understand the involvement of cerebellar circuitry in this process. Do cerebellar lesions impair such behavior? Five studies of cerebellar patients have been reported, which suggest that cerebellar circuitry is important for such behavior. Bracke-Tolkmitt et al. (1989) reported that patients with cerebellar pathology (lesions and atrophy) performed worse than controls on a conditional visuomotor task, although they were not impaired in a range of memory tests. Canavan et al. (1994) also reported that patients were particularly impaired on the performance of conditional visuomotor tasks involving color–word associations. Tucker et al. (1996) tested patients with cerebellar ataxia and those with basal ganglia disorders (Parkinson’s disease and Huntington’s disease) on a conditional visuomotor task. Patients were matched for age and IQ with healthy control subjects. Cerebellar patients made significantly more errors than matched controls, unlike Parkinson’s disease patients whose error rates were comparable with those of matched controls. Drepper et al. (1999) tested patients with cerebellar degenerative disease on a conditional visuomotor task and matched them with controls on the basis of factors that included age, sex, handedness, visual memory ability, level of intelligence, and educational background. They too were more impaired on this task compared with controls. Timmann et al. (2002) also studied the ability of cerebellar patients on a conditional visuomotor task and compared them with matched controls. They reported that the time to decision was longer in cerebellar patients in ways that could not be explained by their motor deficits. This might perhaps reflect the reliance of performance on System 2 rather than on System 1. Taken together, these studies show evidence that cerebellar lesions are associated with deficits in the execution of conditional visuomotor tasks, whether in terms of increased time to decision or increased error rates. Their results are consistent with the idea that cerebellar integrity is needed for the information processing that supports these tasks. However, on their own these findings cannot be used to support the conclusion that the cerebellum contains plasticity that supports conditional visuomotor learning. The first is an argument that applies to all lesion studies: the effects of lesions reach well beyond the site of the lesion. Lesions not only impair local information processing at the lesion site, but also alter distant processing in areas that are connected with the lesioned area. Changes in the levels of tonic activity supplied by lesioned areas to connected target areas are likely to alter normal activity in those targets and therefore also impair the ways in which they process information (Luria, 1976; Monakow, 1914). Indeed, crossed cerebral–cerebellar diaschesis is a well-known effect in which in cerebellar lesions result in reduced metabolism in the contralateral frontal lobe (Baron et al., 1981; Boni et al., 1992; Infeld et al., 1995; Meneghetti et al., 1984; Pantano et al., 1987). By themselves, lesion studies are not able to discriminate between local effects of cerebellar lesions on behavior and the behavioral effects of impaired frontal lobe processing that arise from cerebellar lesions. The second problem with the interpretation is that lesions of the cerebellar dentate appear not to affect performance on conditional visuomotor learning in monkeys (Nixon and Passingham, 2000). It is not clear why the results of this

4 Cognitive Habits

single study should differ from the five reported above. It is possible that species differences could account for the differences in performance, and this would be consistent with the finding that the loop which connects the cerebellar cortex with the prefrontal cortex is enlarged in humans compared to monkeys (see above). Given the problems of interpreting lesion studies, it is important to study activity associated with conditional visuomotor tasks in the healthy human brain. Some work in our laboratory has used functional MRI to investigate whether cerebellar cortical areas which are interconnected with the prefrontal cortex have the properties that one would expect if they stored representations required for cognitive skills. A typical trial begins with an instruction cue that signals the appropriateness of a response– outcome contingency for that context. This is followed by a variable delay, and a “Go” signal that prompts subjects to select a response (initially random). The cue is completely arbitrary with respect to the action. The outcome (error feedback) then guides choices that are made on subsequent repetitions of the instruction cue. Subjects are usually presented with a range of different pseudorandomly presented cues. In these studies, we were able to time-lock activity to instruction cues that coded the response rules, such that it could not account for the activity time-locked to the other trial components (“Go” signal, movement, or feedback). We first showed that activity in the parts of the cerebellar cortex that connect with the prefrontal cortex (Crus I and Crus II of lobule HVII) was specifically evoked by instruction cues that arbitrarily specified an action to be performed at the time of a later “Go” trigger (Balsters and Ramnani, 2008). In that study, we used two control conditions in which the trial structure was identical to that of the experimental condition. In one control condition, subjects used direct cues rather than arbitrary rules to guide action (an image of a hand with a highlighted finger instructing which button to press). In the second control condition, we used a symbolic cue which signaled to the subject that although the “Go!” signal would appear, they would not be told which movement to execute until the time of the trigger itself, so no specific response could be selected at the time of the instruction cue. We looked for an interaction between two factors, each with two levels (factor 1: symbolic vs. direct; factor 2: informative vs. uninformative; Fig. 5A). Consistent with our hypothesis, we found that activity in Crus I and Crus II that was evoked by instruction cues that were both symbolic and informative (Fig. 5B–D). While this form of learning begins as goal-directed, repeated presentations of cues not only cause decreases in the error rate but also increase resistance to distractors as measured by dual task methods. One would predict that learning-related activity evolves over the time course of learning. Balsters and Ramnani (2011) manipulated the rate of learning for two sets of instruction cues such that one set was acquired more quickly than the other, and also tested performance under dual task conditions both before and after learning to test for automaticity changes. One set of rules became automatic more quickly than the other. In line with the hypothesis, the results showed that trial-to-trial decreases in activity were present in cerebellar cortical Crus II (as would be predicted by Albus, 1971, and consistent with evidence of from studies of cerebellar long-term depression; Chapter 1). The rate of decrease was faster for the set of cues that became automatic more quickly (Fig. 6).

267

CHAPTER 10 Cerebellum and Automatic Processing

Symbolic

Direct

Informative

Informative GO!

Uninformative

Uninformative “Go!” signal cue Trigger cue (500 ms) (1000 ms)

Instruction cues (500 ms)

Feedback (500 ms)

Instructed delay period

Response period

variable over 2 TRs (6 s)

variable over 2 TRs (6 s)

A

B

C Symbolic uninformative Symbolic informative Direct uninformative Direct informative

0.7 0.6 BOLD amplitude

268

0.5 0.4 0.3 0.2 0.1 0 –0.1 –0.2 2

4

6

8

10

12

14

16

Upper value of peri-stimulus time bins (s).2 s bin

FIGURE 5 Trial structure (upper panel). Instruction cues in four conditions (experimental condition: cues were both symbolic and informative; others were controls). Hemodynamic activity could be time-locked to these cues without a significant contribution from other trial components (“Go” signal, movement, and error feedback) because the interval between them varied from trial to trial. Lower panel shows (A) activations in parts of the cerebellar cortex that are known to be interconnected with the prefrontal cortex; (B) comparable section from Schmahmann et al. (2000); and (C) hemodynamic activity from the peak voxel in (A) in each of the four conditions. From Balsters and Ramnani (2008).

The studies reported above employed first-order rules in which instructions directly specified the response. By virtue of the experimental designs used, activity cannot be easily explained in terms of motor planning and execution. Nevertheless, Balsters et al. (2013) applied a more stringent test of the hypothesis by testing for

4 Cognitive Habits

C

0.25 0.2

A

0.15 0.1 0.05 0 –0.05

B

–0.1 –0.15 –0.2 –0.25 1

3

5

7

9 11 13 15 17 19 21 23 25 27 29 31 33 35 Trial number

Reaction time (ms)

D 500 475 450 425 400 375 350 325 300

LL HL

S2: Initial behavioral training

S3: Dual task

S4: Training during scanning

S5: Dual task

FIGURE 6 (A) Labeled section from Schmahmann et al. (2000) at a comparable plane of section to (B) which shows the location in which there was a differential rate of activity change for instruction cues which were fast (dark gray) and slowly (light gray) changing activity; these corresponded to trials in which cues became automatic with training at different rates (light gray, slow; dark gray, fast). Performance was assessed under dual task conditions. From Balsters and Ramnani (2011).

cerebellar activity that was related to second-order rules. These differ from firstorder rules because instruction cues specify which first-order rule to follow rather than which action to execute (Fig. 7). Subjects had no information about the specific response required to complete the task, but were able to select a rule on the basis of the cue. An advantage of the design was that activity could be identified for both higher and lower order cues, and activity time-locked to both kinds of rule was found in Crus I and Crus II, as predicted. This group of studies, along with others (Balsters et al., In press), add weight to the argument that cerebellar cortical areas which connect with the prefrontal cortex may store representations (perhaps forward models) that allow rules established in System 2 to be executed using System 1.

4.1 Cerebellar Cortex and Higher Level Feedback Research on the neural basis of habit formation and the acquisition of cognitive skills has been dominated by the study of the basal ganglia. The narrative provided by many authors is compelling, and the literature is replete with empirical support (Graybiel, 2008; Yin and Knowlton, 2006). However, the exclusive focus on the basal ganglia does not reflect the reality that other systems are likely to be similarly

269

Parameter estimate

CHAPTER 10 Cerebellum and Automatic Processing

2

2.5 2 1.5 1 0.5 0

1.5

1stR 1stC 2ndR 2ndC 1.5

1 0.5 0 1stR 1stC 2ndR 2ndC

Parameter estimate

Parameter estimate

270

1 0.5 0 1stR

1stC 2ndR 2ndC

–0.5

FIGURE 7 Areas in the cerebellar cortex (Crus I and Crus II) in which activity for both first- (1stR) and second-order (2ndR) rules was significantly different from their corresponding control trials (1stC and 2ndC). Bar graphs reflect parameter estimates for each condition. From Balsters et al. (2013).

involved. These accounts emphasize the closed-loop architecture of corticobasal ganglia circuitry and the plastic properties of striatal medium spiny neurons, but sometimes overlook the fact that the corticocerebellar system has very similar properties (the structure of closed-loop corticobasal ganglia loops mirrors the architecture of corticocerebellar system which also connect the cerebellum and cerebral cortex; the Purkinje cells of the cerebellar cortex also engage in information storage through the experience-dependent modification of their synaptic inputs). In contrast, other accounts acknowledge the contributions of both the basal ganglia and cerebellum to skill learning and attempt to distinguish between their contributions. Doya (2000) and Houk et al. (2007) each provide comparable accounts. In essence, they propose that cerebellar and basal ganglia loops with the neocortex each play a role in learning. Whereas cerebellar circuitry engaged in supervised learning is thought to be governed by error signals from the inferior olive, it is suggested that striatal circuitry receives error feedback from the reward system and is engaged in reinforcement learning. Houk and colleagues suggest, for instance, that the corticobasal ganglia circuits learn to discover “ballpark” actions that are appropriate for specific contexts, and that corticocerebellar circuits then sculpt and refine the details of movements. These accounts suggest that the basal ganglia are therefore engaged in cognitive processes related to action selection, and that the cerebellum is more concerned with the skilled motor control. There are two reasons why these accounts need to be modified. First, they are partly predicated on the assumption that the basal ganglia are interconnected with neocortical areas concerned with action selection (area 46 in the prefrontal cortex), and that the cerebellum is interconnected with areas concerned with motor control. Both of these statements are true, but the picture is of course more complex than this. In fact, prefrontal area 46 connects with both the

4 Cognitive Habits

basal ganglia and the cerebellum (as discussed above). Information required for the selection of action could therefore reach both. The accounts are also based on the assumption that the basal ganglia receive information about reward-related prediction errors, and that the cerebellum receives sensorimotor prediction errors. Again, both points are true but it has been shown that the basal ganglia receive information about both reward and sensorimotor errors (Cody and Rickards, 1995). Importantly for this chapter, there is evidence of multiple pathways through which prediction errors related to higher level outcomes (including rewards) can reach the cerebellum and some of this evidence is discussed below. These points suggest that the role played by the basal ganglia could be extended from one that is restricted to reward-based learning to one which also includes sensorimotor learning (and there are indeed accounts which posit its role in both). The cerebellar cortex may also process error feedback related to reward predictions. The differences between the cerebellar and basal ganglia contributions to sensorimotor and reward-based learning may be less clear than previously thought.

4.2 Could Climbing Fibers Convey Errors Related to Cognitive Processing? The debates around the precise role that climbing fibers play in cerebellar learning have been on ongoing for many years and are by no means settled. However, there is support for the view that climbing fibers signal discrepancies between actual and expected sensory outcomes of movement and drive information storage in cerebellar circuits (Gilbert and Thach, 1977; Greger and Norris, 2005; Medina and Lisberger, 2008; Ojakangas and Ebner, 1994; Rasmussen et al., 2008). How does association cortex influence the inferior olive, and what kind of prediction errors might be required to shape forward models associated with the acquisition of cognitive skills? The information cascade through frontal lobe circuitry mentioned above might eventually result in a movement that causes the attainment of goals. Whereas the movements executed to obtain the goals will generate sensory consequences, in an instrumental setting error feedback is also generated that specifies whether or not a desired outcome has been achieved. The two forms of feedback are quite distinct. I suggest that both forms of feedback are conveyed to separate forward models— one that predicts sensory consequences of the movement involved, and the other which predicts higher level consequences such as the attainment of a reward (Fig. 4). Although the existence of connections between the inferior olive and the cerebral cortex has been known about for quite some time (Walberg, 1956), our knowledge of the details is sparse. There is little detailed information about the topographical organization of these connections that matches that reported in studies of corticopontine pathways. The contributions of the cortical motor areas are well known and terminate in parts of the inferior olive that project to locations in the anterior lobe of the cerebellar cortex (Saint-Cyr, 1983; Sousa-Pinto, 1969). The projections from the association cortex are of more interest here because they might convey higher order error-related information to the olivocerebellar system. Sasaki et al. (1977)

271

272

CHAPTER 10 Cerebellum and Automatic Processing

used eletrophysiological methods to map the influence that a range of neocortical areas have on the inferior olive by stimulating them and recording complex spikes from the cerebellar cortex. This is possibly the only detailed study in primates that sheds light on existence of cortico-olivary influences from the frontal association cortex. They showed both mossy and climbing fiber responses in widespread areas of the cerebellar cortex, but one cannot use their findings to draw detailed conclusions about the anatomical organization of the pathways through which these influences are exerted. Most work on the connectivity of the prefrontal cortex with the inferior olive has been conducted in rats, but the comparisons between the rat prefrontal cortex and that of humans and nonhuman primates are not straightforward and there are disagreements about the cross-species comparability of the prefrontal cortex. Notes of caution are provided elsewhere (Passingham and Wise, 2012; Preuss, 1995; Schoenbaum et al., 2009; Seamans et al., 2008; Uylings et al., 2003). Watson et al. (2009) report that in rats, stimulation of the prelimbic cortex (a part of the medial prefrontal cortex) resulted in complex spikes in vermal parts of lobule VII, confirming the existence of a pathway through which the medial prefrontal cortex can influence the cerebellar cortex via the inferior olive. In that study, it is not clear whether the prefrontal influence on the inferior olive was through a direct or an indirect pathway. However, Swenson et al. (1989) used anterograde tracers to investigate the connections of the inferior olive with two specific cortical areas. Injections of tracer into sensory or motor cortices generally resulted in label in the principal olive (PO), the dorsal accessory olive (DAO), and caudal parts of the medial accessory olive (cMAO). They also injected anterograde tracer into specific parts of the rat medial frontal cortex (infralimbic, prelimbic, and orbital cortex), and reported the presence of tracer in olivary areas that included the rostral pole and ventral portion of the MAO, in the beta subnucleus, and in the ventrolateral outgrowth of the dorsal cap of Kooy. Wise (2008) suggests that these prefrontal areas in rat may be homologous to monkey and human orbital and insular cortex (areas 13a, 14a, Iapm, and Iam), and the anterior and subgenual parts of the cingulate cortex (areas 14c, 32, 25, 24, 32, and 24). Suzuki et al. (2012) used a retrograde transsynaptic tracer (modified rabies virus) in conjunction with a conventional tracer (cholera toxin beta subunit) to map the polysynaptic pathways from the cerebral cortex to the cerebellar cortex. The tracers were injected into parts of the cerebellar cortex in rats. Importantly, they were able to map the paths of the tracers not only through the pontine nuclei but also through the inferior olive. They injected tracer into three locations within hemispheral and vermal parts of lobule VII. Injections into Crus II in the hemispheral parts resulted in first-order label in rostromedial DAO, rostral MAO, the dorsomedial group, and ventral lamella of PO if injected in a medial location. When injected more laterally, tracer was found in the dorsal lamella of PO (including the dorsomedial group). One of the interesting features of this study was that transsynaptic tracer could be localized to areas of the cerebral cortex. Second-order label was found in the motor cortex (M1, M2, and FR3), and judging from published figures, possibly also orbitofrontal and insular cortex. Medial and lateral placements of tracer in vermal parts of lobule VII resulted in broadly similar distributions of

4 Cognitive Habits

terminal label. First-order label in the inferior olive was found in caudal MAO and in the ventral lamella of PO. Areas of the pontine nuclei also contained such label in NRTP and BPN. The authors reported the presence of label in the pyramidal cells of the retrosplenial cortex, the face areas of the somatosensory cortex, and the ventrolateral orbital cortex (Fig. 8). Information from such areas can therefore reach the cerebellar cortex via the inferior olive.

FIGURE 8 Injection sites: cholera toxin beta subunit tracer (top left) and modified rabies virus (top right). Presence of transsynaptic rabies virus (middle left showing label in VO and LO). Magnified in the middle right panel to show label in individual cells in layer V. Lower panel shows three coronal schematic sections. Gray dots show the distribution of labeled cells. Figure courtesy of Tom Ruigrok depicting the results reported in Suzuki et al. (2012) derived from case 1095.

273

274

CHAPTER 10 Cerebellum and Automatic Processing

If the retrosplenial cortex and ventrolateral parts of the orbitofrontal cortex send outputs to the vermal part of lobule VII via the inferior olive, then the finding is important because it has implications for our understanding of how higher order areas influence information processing in the cerebellum. The retrosplenial cortex (Ranganath and Ritchey, 2012) is the most caudal subdivision of the cingulate cortex and in monkeys is heavily interconnected with the hippocampal system (Aggleton et al., 2012; Vann et al., 2009). In rats, lesions of the retrosplenial cortex impair contextual fear conditioning (Corcoran et al., 2011; Keene and Bucci, 2008; Kwapis et al., In press; Lukoyanov and Lukoyanova, 2006) and performance on memoryguided spatial navigation (Pothuizen et al., 2008; Sugar et al., 2011). “Head direction” cells of the retrosplenial cortex are tuned to the locations to which the head is pointing and are important for processing related to self-motion (Cho and Sharp, 2001). In humans, it plays important roles in the retrieval of autobiographical memories and the processing of self-relevant information in social cognition (Sugar et al., 2011; Vann et al., 2009). It is not a structure that appears to be involved in sensorimotor processing. The orbitofrontal cortex is interconnected with systems that are engaged in the processing of reward feedback and lesions of the orbitofrontal cortex in rats cause deficits not in motor control but in a wide variety of higher order behaviors associated with the processing of feedback during goal-directed behavior (Hollerman et al., 2000; Rolls and Grabenhorst, 2008; Schultz et al., 2000; Walton et al., 2011). It is an important part of the reward processing system in both rodents and primates (Murray and Rudebeck, 2013; Rudebeck et al., 2013) and activity in the orbitofrontal cortex reflects errors in reward prediction (O’Doherty et al., 2003; Ramnani et al., 2004; Rogers et al., 2004; Schultz, 2001; Tremblay and Schultz, 2000). It is also worth noting that the inferior olive is known to receive dopaminergic inputs from the mesodiencephalic dopamine system (Oades and Halliday, 1987; although whether they convey reward-related prediction errors is not known). It is perhaps also worth considering that the inferior olive is the target of dopamine neurons (Sladek and Bowman, 1975; Toonen et al., 1998). Of course, the study by Suzuki et al. (2012) does not by itself distinguish the relative contributions of these neocortical areas to the pontine nuclei and the inferior olive. This can only be determined with further studies. However, it is notable that Schmahmann and Pandya (1995) found no projections to the pontine nuclei from either the parahippocampal cortex or the orbitofrontal cortex to the pontine nuclei in primates. This is important because first, in conjunction with Suzuki et al. (2012), it strongly suggests that information from those areas is more likely to be conveyed via the inferior olive. Second, it supports the view that association cortex appears to connect with the cerebellum through two separate systems, each via one of the two major precerebellar nuclei. Passingham and Wise (2012) present the case that dorsal parts of the prefrontal cortex are concerned with organizing the responses required to obtain outcomes. The dorsal parts of the prefrontal cortex (including area 46) represent actions at an abstract level, and as discussed above, these are realized through the connections of area 46 with the premotor cortex. On the basis of the arguments presented above,

4 Cognitive Habits

I suggest that this system contributes information through the pontine nuclei, conveying efference copies of descending commands that are processed by cerebellar forward models in the vermal and hemispheral parts of lobule VII. The retrosplenial cortex, hippocampal system, and orbitofrontal cortex are bound into a system through their connections (Cavada et al., 2000), and it can been argued that they are concerned with the higher level processing of sensory information (Passingham and Wise, 2012). The hippocampal system and retrosplenial cortex are well known to contribute to processing spatial information for the purposes of navigation (see above). Wallis (2012) argues that activity in orbitofrontal neurons in both rats and primates can code a variety of decision-related variables including decision-related outcomes. Activity in the orbitofrontal cortex often reflects the discrepancies between actual and expected rewards and is concerned with whether or not goals such as rewards are attained (see below). This second system may therefore convey information such as reward-related prediction errors from the orbitofrontal cortex to the cerebellar cortex (see below) and this may reach cerebellar cortical lobule VII via the inferior olive. If area 46 is engaged in orchestrating the strategies required to achieve these outcomes, then it is possible that vermal and hemispheral parts of lobule VII use the commands from area 46 to predict reward-related outcomes, and that they use actual feedback from the orbitofrontal cortex via the inferior olive to test and modify such predictions.

4.3 Cerebellar Communication with the Dopamine System Conventional ideas about the circuitry involved in monitoring the outcomes of goaldirected behavior implicate the dopamine system and its targets. These include parts of the cingulate cortex (Apps et al., 2012, 2013, 2014; Bush et al., 2000; Rogers et al., 2004; Schall et al., 2002), the orbitofrontal cortex (see above), and the ventral striatum in the basal ganglia (Schultz, 2013; Schultz and Dickinson, 2000). They are strongly influenced by direct projections from dopamine cells that originate in the ventral tegmental area (VTA). The ability of these neurons to code for reward error (the discrepancies between actual and expected reward outcomes) is well documented (Schultz, 2013). These neurons become active when unexpected rewards appear and become silent at the time that expected rewards fail to appear. Moreover, they also become active in the presence of cues that reliably predict rewards. Although the inferior olive is widely regarded as the principal route through which error signals arrive at the cerebellar cortex, there is some evidence that the dopamine system can also provide information about the outcomes of reward-related predictions directly to the cerebellar cortex. The influence appears to be reciprocal. I suggest below that the dopamine system might impact directly on cerebellar physiology, and that the cerebellum might modulate the influence of dopamine release in other areas, and therefore influence instrumental, goal-directed learning. Like the inferior olive, the VTA is a precerebellar nucleus because it sends direct projections to the cerebellum (Ikai et al., 1992; Simon et al., 1979). Ikai et al. (1992) report that in rats, VTA dopamine neurons terminate in the hemispheral parts of lobule

275

276

CHAPTER 10 Cerebellum and Automatic Processing

VII (Crus I and Crus II)—the same regions that receive corticopontine inputs from area 46. It is worth noting that other parts of the cerebellar cortex are also very likely to receive dopaminergic inputs from alternative sites of origin because dopamine receptors are widely distributed across the cerebellar cortex (Panagopoulos et al., 1991). Dopamine neurons terminate on cerebellar cortical granule cells and on the cell bodies of Purkinje cells. They can therefore modulate the activity of parallel fiber inputs to Purkinje cells by influencing granule cells, and also the outputs of Purkinje cells themselves. Using double-labeling techniques, Ikai et al. (1994) showed that the dopamine pathways from VTA to the cerebellum are collaterals of those pathways which convey information to the prefrontal cortex. The reward-related error signals conveyed to the cerebellum are therefore likely to be copies of error signals that are conveyed to the prefrontal cortex. Prediction errors for rewards are therefore signaled not only to prefrontal areas which are known to monitor the outcomes of cognitive operations, but may also convey these to cerebellar areas that model the prefrontal processing that generates the strategies for obtaining them. There is also evidence which suggests that the cerebellar cortex is capable of exerting an influence on the dopamine system, and this raises the possibility that cerebellar activity may be capable of modulating instrumental forms of learning that depend on reward-related prediction errors. Mittleman et al. (2008) reported that in wild-type mice, stimulation of the cerebellar cortex results in a significant increase in dopamine release in the medial prefrontal cortex, and this effect is markedly reduced in Lurcher mice after the degeneration of Purkinje cells. Stimulation of the cerebellar nuclei also results in such an increase and as one would expect this effect is still present after Purkinje cells have degenerated. Rogers et al. (2011) suggest that this effect depends on multiple pathways which connect cerebellar outputs with the medial prefrontal cortex. However, they reported that increases in prefrontal dopamine release evoked by cerebellar stimulation were profoundly reduced by pharmacological inactivations of the VTA, as compared with much smaller effects when alternative pathways were inactivated. This suggests that the VTA is an important route through which the cerebellum can control the action of dopamine in the prefrontal cortex. Nieoullon and Dusticier (1980) reported that electrical stimulation of the cerebellar nuclei caused changes in dopamine release in the caudate and the substantia nigra (although whether such changes were increases or decreases varied depending on laterality and anatomical location). Earlier in this chapter, it mentioned that an important feature of habit learning is a decreased sensitivity to error feedback, and failures to adapt to changes associated with the predictability of rewards (reinforcer devaluation). One of the performance benefits of this could be that processes governed by feedforward mechanisms are allowed to run through to completion, uninterrupted by the direction of attention to unexpected events (in other words, during learning, processes governed by System 1 suppress and predominate over those governed by System 2). Through what mechanisms might learning contribute to the suppression of feedback-related information as learning becomes maximal? It has been shown that during classical eyeblink conditioning, commands related to conditioned responses from anterior parts of the interpositus nucleus are

5 Conclusion

accompanied by an inhibitory signal to the parts of the inferior olive that are involved in the processing of the US teaching signal (see Rasmussen and Hesslow, Chapter 5). This is consistent with the finding that the amplitudes of unconditioned responses to the US are typically of low amplitude when learning is maximal. Yeo and Hesslow (1998) propose that “when a CR of sufficient amplitude is generated in the AIP, olivary activity, and hence further learning, would be turned off.” In other words, there is a negative feedback loop that effectively reduces the sensitivity of the system to the US when learning is maximal. When considering the anatomical and physiological relationships between the cerebellum and the dopamine system, it becomes evident that cerebellar inhibition might also be imposed on signals that convey information related to reward errors during higher forms of learning. There is evidence that in at least some instances, the influence of the cerebellum on the dopamine system is inhibitory rather than excitatory (Nieoullon and Dusticier, 1980). Anatomical studies show that cerebellar inhibition could act not only on the sources of dopamine neurons in the VTA but also on the substantia nigra. The cerebellar nuclei (including the dentate) return inhibitory projections to the VTA (Ikai et al., 1992; Perciavalle et al., 1989; Simon et al., 1979), just as they do to the inferior olive. Snider et al. (1976) reported that output neurons in the cerebellar nuclei terminate in both the VTA and in the substantia nigra—another major source of dopaminergic outputs. While the dentate and interpositus nuclei send crossed projections to both the substantia nigra and the VTA, the fastigial nuclei send uncrossed projections only to the VTA. Given that cerebellar cortical outputs to the cerebellar nuclei are inhibitory, cerebellar cortical lesions that disinhibit the cerebellar nuclei would also increase nuclear inhibition of the dopamine system. Lesions of the cerebellar cortex would therefore result in decreased dopamine levels in the forebrain. Consistent with this view, Snider and Snider (1977) reported that over the period of a few weeks, lesions of the cerebellar cortex resulted in decreased forebrain dopamine levels. Of course, this is a rather different scenario to one in which the physiology of the cerebellar cortex changes during learning. Under those circumstances, a gradual trial-to-trial reduction in the activity of cerebellar Purkinje cells would result in the lifting of inhibition on cerebellar nuclear cells in those trials. This would result in an increase in inhibitory output to VTA cells and consequently a decrease in dopamine output to VTA targets such as the prefrontal cortex. These pathways may provide mechanisms through which learning-related outputs from the cerebellum can decrease the sensitivity of other brain areas to signals that convey reward-related prediction errors, thereby suppressing signals that might interrupt the feedforward execution of information processing initiated in the cerebellar cortex.

5 CONCLUSION I have used the hierarchical organization of the frontal lobe to illustrate how it is possible for a given area to be both a controller and a controlled object in control theoretic terms. The fact that each controlled object (including areas of the prefrontal cortex) might be associated with an independent forward model means that the cerebellar

277

278

CHAPTER 10 Cerebellum and Automatic Processing

cortex might contain independent forward models of each controlled object, and can automate higher order, rule-related cognitive processes independently of processes concerned with motor control. Such a system would require forward models in the cerebellum to access efference copies of commands from controllers that also send them to the controlled objects that the forward models simulate. In the case of goaldirected learning, forward models that simulate the responses to commands from the prefrontal cortex may predict a high level outcome. For the predicted outcome to be tested against the actual outcome, the system needs access to such feedback. I have suggested that this may arrive via the inferior olive and the dopamine inputs to the cerebellar cortex. One of the properties of habitual behavior is that it is relatively insensitive to error feedback. I have suggested that the inhibitory output from the cerebellum to the dopamine system might provide a mechanism through which learned representations in the cerebellum can suppress the responses of other areas in the nervous system to reward-related prediction errors. Such a mechanism may be important for preserving the automatic character of the learned task by preventing the engagement of attentional mechanisms in the prefrontal cortex.

Acknowledgment This work is supported by a grant from the Biotechnology and Biological Sciences Research Council, UK (BB/J017116/1).

References Apps, M.A.J., Balsters, J.H., Ramnani, N., 2012. The anterior cingulate cortex: monitoring the outcomes of others’ decisions. Soc. Neurosci. 7, 424–435. Apps, M.A., Green, R., Ramnani, N., 2013. Reinforcement learning signals in the anterior cingulate cortex code for others’ false beliefs. Neuroimage 64, 1–9. Apps, M.A.J., Ramnani, N., 2014. The anterior cingulate gyrus signals the net value of others’ rewards. J. Neurosci. 34, 6190–6200. Adams, C.D., 1982. Variations in the sensitivity of instrumental responding to reinforcer devaluation. Q. J. Exp. Psychol. B 34, 77–98. Aggleton, J.P., Wright, N.F., Vann, S.D., Saunders, R.C., 2012. Medial temporal lobe projections to the retrosplenial cortex of the macaque monkey. Hippocampus 22, 1883–1900. Albus, J.S., 1971. A theory of cerebellar function. Math. Biosci. 10, 25–61. Ashby, F.G., Turner, B.O., Horvitz, J.C., 2010. Cortical and basal ganglia contributions to habit learning and automaticity. Trends Cogn. Sci. 14, 208–215. Balsters, J.H., Ramnani, N., 2008. Symbolic representations of action in the human cerebellum. Neuroimage 43, 388–398. Balsters, J.H., Ramnani, N., 2011. Cerebellar plasticity and the automation of first-order rules. J. Neurosci. 31, 2305–2312. Balsters, J.H., Cussans, E., Diedrichsen, J., Phillips, K.A., Preuss, T.M., Rilling, J.K., Ramnani, N., 2010. Evolution of the cerebellar cortex: the selective expansion of prefrontal-projecting cerebellar lobules. Neuroimage 49, 2045–2052.

References

Balsters, J.H., Laird, A.R., Fox, P.T., Eickhoff, S.B., In press. Bridging the gap between functional and anatomical features of cortico-cerebellar circuits using meta-analytic connectivity modeling. Hum. Brain Mapp. Balsters, J.H., Whelan, C.D., Robertson, I.H., Ramnani, N., 2013. Cerebellum and cognition: evidence for the encoding of higher order rules. Cereb. Cortex 23, 1433–1443. Baron, J.C., Bousser, M.G., Comar, D., Castaigne, P., 1981. “Crossed cerebellar diaschisis” in human supratentorial brain infarction. Trans. Am. Neurol. Assoc. 105, 459–461. Boni, S., Valle, G., Cioffi, R.P., Bonetti, M.G., Perrone, E., Tofani, A., Maini, C.L., 1992. Crossed cerebello-cerebral diaschisis: a SPECT study. Nucl. Med. Commun. 13, 824–831. Boyden, E.S., Katoh, A., Raymond, J.L., 2004. Cerebellum-dependent learning: the role of multiple plasticity mechanisms. Annu. Rev. Neurosci. 27, 581–609. Bracke-Tolkmitt, R., Linden, A., Canavan, A.-G., Rockstoh, B., Scholz, E., Wessel, K., Deiner, H.C., 1989. The cerebellum contributes to mental skills. Behav. Neurosci. 103, 442–446. Bush, G., Luu, P., Posner, M.I., 2000. Cognitive and emotional influences in anterior cingulate cortex. Trends Cogn. Sci. 4, 215–222. Canavan, A.G., Sprengelmeyer, R., Diener, H.C., Homberg, V., 1994. Conditional associative learning is impaired in cerebellar disease in humans. Behav. Neurosci. 108, 475–485. Cavada, C., Company, T., Tejedor, J., Cruz-Rizzolo, R.J., Reinoso-Suarez, F., 2000. The anatomical connections of the macaque monkey orbitofrontal cortex. A review. Cereb. Cortex 10, 220–242. Cho, J., Sharp, P.E., 2001. Head direction, place, and movement correlates for cells in the rat retrosplenial cortex. Behav. Neurosci. 115, 3–25. Cody, F.W.J., Rickards, C., 1995. Proprioception in human basal ganglia disorders. In: Alpha and Gamma Motor Systems, 486–490. Coffman, K.A., Dum, R.P., Strick, P.L., 2011. Cerebellar vermis is a target of projections from the motor areas in the cerebral cortex. Proc. Natl. Acad. Sci. U.S.A. 108, 16068–16073. Corcoran, K.A., Donnan, M.D., Tronson, N.C., Guzman, Y.F., Gao, C., Jovasevic, V., Guedea, A.L., Radulovic, J., 2011. NMDA receptors in retrosplenial cortex are necessary for retrieval of recent and remote context fear memory. J. Neurosci. 31, 11655–11659. D’Esposito, M., Postle, B.R., Rypma, B., 2000. Prefrontal cortical contributions to working memory: evidence from event-related fMRI studies. Exp. Brain Res. 133, 3–11. Dezfouli, A., Balleine, B.W., 2012. Habits, action sequences and reinforcement learning. Eur. J. Neurosci. 35, 1036–1051. Dickinson, A., 1985. Actions and habits—the development of behavioral autonomy. Philos. Trans. R. Soc. Lond. B Biol. Sci. 308, 67–78. Dickinson, A., Squire, S., Varga, Z., Smith, J.W., 1998. Omission learning after instrumental pretraining. Q. J. Exp. Psychol. B 51, 271–286. Doya, K., 2000. Complementary roles of basal ganglia and cerebellum in learning and motor control. Curr. Opin. Neurobiol. 10, 732–739. Drepper, J., Timmann, D., Kolb, F.P., Diener, H.C., 1999. Non-motor associative learning in patients with isolated degenerative cerebellar disease. Brain 122 (Pt. 1), 87–97. Dum, R.P., Strick, P.L., 2005. Frontal lobe inputs to the digit representations of the motor areas on the lateral surface of the hemisphere. J. Neurosci. 25, 1375–1386. Evans, J.S., 2008. Dual-processing accounts of reasoning, judgment, and social cognition. Annu. Rev. Psychol. 59, 255–278.

279

280

CHAPTER 10 Cerebellum and Automatic Processing

Funahashi, S., Takeda, K., 2002. Information processes in the primate prefrontal cortex in relation to working memory processes. Rev. Neurosci. 13, 313–345. Fuster, J.M., 1990. Prefrontal cortex and the bridging of temporal gaps in the perception-action cycle. Ann. N.Y. Acad. Sci. 608, 318–329, discussion 330–336. Fuster, J.M., 1993. Frontal lobes. Curr. Opin. Neurobiol. 3, 160–165. Gilbert, P.F., Thach, W.T., 1977. Purkinje cell activity during motor learning. Brain Res. 128, 309–328. Goldman-Rakic, P.S., 1999. The “psychic” neuron of the cerebral cortex. Ann. N.Y. Acad. Sci. 30, 86813–86826. Graybiel, A.M., 2008. Habits, rituals, and the evaluative brain. Annu. Rev. Neurosci. 31, 359–387. Greger, B., Norris, S., 2005. Simple spike firing in the posterior lateral cerebellar cortex of Macaque Mulatta was correlated with success-failure during a visually guided reaching task. Exp. Brain Res. 167, 660–665. Hollerman, J.R., Tremblay, L., Schultz, W., 2000. Involvement of basal ganglia and orbitofrontal cortex in goal-directed behavior. Prog. Brain Res. 126, 193–215. Hoshi, E., 2008. Differential involvement of the prefrontal, premotor, and primary motor cortices in rule-based motor behavior. In: Bunge, S.A., Wallis, J.D. (Eds.), Neuroscience of Rule-Guided Behavior. Oxford University Press, Oxford. Houk, J.C., 1997. On the role of the cerebellum and basal ganglia in cognitive signal processing. In: De Zeeuw, C.I., Strata, P., Voogd, J. (Eds.), Progress in Brain Research, vol. 114. Elsevier, Oxford, pp. 543–552. Houk, J.C., Bastianen, C., Fansler, D., Fishbach, A., Fraser, D., Reber, P.J., Roy, S.A., Simo, L.S., 2007. Action selection and refinement in subcortical loops through basal ganglia and cerebellum. Philos. Trans. R. Soc. Lond. B Biol. Sci. 362, 1573–1583. Ikai, Y., Takada, M., Shinonaga, Y., Mizuno, N., 1992. Dopaminergic and non-dopaminergic neurons in the ventral tegmental area of the rat project, respectively, to the cerebellar cortex and deep cerebellar nuclei. Neuroscience 51, 719–728. Ikai, Y., Takada, M., Mizuno, N., 1994. Single neurons in the ventral tegmental area that project to both the cerebral and cerebellar cortical areas by way of axon collaterals. Neuroscience 61, 925–934. Infeld, B., Davis, S.M., Lichtenstein, M., Mitchell, P.J., Hopper, J.L., 1995. Crossed cerebellar diaschisis and brain recovery after stroke. Stroke 26, 90–95. Ito, M., 2008. Control of mental activities by internal models in the cerebellum. Nat. Rev. Neurosci. 9, 304–313. Jueptner, M., Stephan, K.M., Frith, C.D., Brooks, D.J., Frackowiak, R.S., Passingham, R.E., 1997. Anatomy of motor learning. I. Frontal cortex and attention to action. J. Neurophysiol. 77, 1313–1324. Kahneman, D., 2011. Thinking, Fast and Slow. Farrar, Straus and Giroux, New York. Kawato, M., Wolpert, D., 1998. Internal models for motor control. Novartis Found. Symp. 218, 291–304, discussion 304–307. Keene, C.S., Bucci, D.J., 2008. Neurotoxic lesions of retrosplenial cortex disrupt signaled and unsignaled contextual fear conditioning. Behav. Neurosci. 122, 1070–1077. Kelly, R.M., Strick, P.L., 2003. Cerebellar loops with motor cortex and prefrontal cortex of a nonhuman primate. J. Neurosci. 23, 8432–8444. Koechlin, E., Ody, C., Kouneiher, F., 2003. The architecture of cognitive control in the human prefrontal cortex. Science 302, 1181–1185.

References

Kwapis, J.L., Jarome, T.J., Lee, J.L., Gilmartin, M.R., Helmstetter, F.J., In press. Extinguishing trace fear engages the retrosplenial cortex rather than the amygdala. Neurobiol. Learn. Mem. Leiner, H.C., Leiner, A.L., Dow, R.S., 1986. Does the cerebellum contribute to mental skills? Behav. Neurosci. 100, 443–454. Leiner, H.C., Leiner, A.L., Dow, R.S., 1989. Reappraising the cerebellum: what does the hindbrain contribute to the forebrain? Behav. Neurosci. 103, 998–1008. Lemon, R.N., 2008. Descending pathways in motor control. Annu. Rev. Neurosci. 31, 195–218. Lu, M.T., Preston, J.B., Strick, P.L., 1994. Interconnections between the prefrontal cortex and the premotor areas in the frontal lobe. J. Comp. Neurol. 341, 375–392. Lukoyanov, N.V., Lukoyanova, E.A., 2006. Retrosplenial cortex lesions impair acquisition of active avoidance while sparing fear-based emotional memory. Behav. Brain Res. 173, 229–236. Luppino, G., Rozzi, S., Calzavara, R., Matelli, M., 2003. Prefrontal and agranular cingulate projections to the dorsal premotor areas F2 and F7 in the macaque monkey. Eur. J. Neurosci. 17, 559–578. Luria, A.R., 1976. The Working Brain: An Introduction to Neuropsychology. Basic Books, New York. Medina, J.F., Lisberger, S.G., 2008. Links from complex spikes to local plasticity and motor learning in the cerebellum of awake-behaving monkeys. Nat. Neurosci. 11, 1185–1192. Meneghetti, G., Vorstrup, S., Mickey, B., Lindewald, H., Lassen, N.A., 1984. Crossed cerebellar diaschisis in ischemic stroke: a study of regional cerebral blood flow by 133Xe inhalation and single photon emission computerized tomography. J. Cereb. Blood Flow Metab. 4, 235–240. Mittleman, G., Goldowitz, D., Heck, D.H., Blaha, C.D., 2008. Cerebellar modulation of frontal cortex dopamine efflux in mice: relevance to autism and schizophrenia. Synapse 62, 544–550. Monakow, C.V., 1914. Die Lokalisation im Grosshirn und der Abbau der Funktion durch kortikale Herde. J. F. Bergmann, Wiesbaden. Muakkassa, K.F., Strick, P.L., 1979. Frontal lobe inputs to primate motor cortex: evidence for four somatotopically organized ‘premotor’ areas. Brain Res. 177, 176–182. Murray, E.A., Rudebeck, P.H., 2013. The drive to strive: goal generation based on current needs. Front. Neurosci. 7, 112. Nieoullon, A., Dusticier, N., 1980. Changes in dopamine release in caudate nuclei and substantia nigrae after electrical-stimulation of the posterior interposate nucleus of cat cerebellum. Neurosci. Lett. 17, 167–172. Nixon, P.D., Passingham, R.E., 2000. The cerebellum and cognition: cerebellar lesions impair sequence learning but not conditional visuomotor learning in monkeys. Neuropsychologia 38, 1054–1072. Oades, R.D., Halliday, G.M., 1987. Ventral tegmental (A10) system—neurobiology. 1. Anatomy and connectivity. Brain Res. Rev. 12, 117–165. O’Doherty, J.P., Dayan, P., Friston, K., Critchley, H., Dolan, R.J., 2003. Temporal difference models and reward-related learning in the human brain. Neuron 38, 329–337. Ojakangas, C.L., Ebner, T.J., 1994. Purkinje cell complex spike activity during voluntary motor learning: relationship to kinematics. J. Neurophysiol. 72, 2617–2630. O’Reilly, J.X., Beckmann, C.F., Tomassini, V., Ramnani, N., Johansen-Berg, H., 2010. Distinct and overlapping functional zones in the cerebellum defined by resting state functional connectivity. Cereb. Cortex 20, 953–965.

281

282

CHAPTER 10 Cerebellum and Automatic Processing

Orioli, P.J., Strick, P.L., 1989. Cerebellar connections with the motor cortex and the arcuate premotor area: an analysis employing retrograde transneuronal transport of WGA-HRP. J. Comp. Neurol. 288, 612–626. Oscarsson, O., 1973. Functional organization of spinocerebellar paths. In: Iggo, A. (Ed.), Handbook of Sensory Physiology. Springer Verlag, Berlin, Heidelberg, New York. Oscarsson, O., Uddenberg, N., 1964. Identification of a spinocerebellar tract activated from forelimb afferents in the cat. Acta Physiol. Scand. 62, 125–136. Oscarsson, O., Uddenberg, N., 1965. Properties of afferent connections to the rostral spinocerebellar tract in the cat. Acta Physiol. Scand. 64, 143–153. Panagopoulos, N.T., Papadopoulos, G.C., Matsokis, N.A., 1991. Dopaminergic innervation and binding in the rat cerebellum. Neurosci. Lett. 130, 208–212. Pantano, P., Lenzi, G.L., Guidetti, B., Di Piero, V., Gerundini, P., Savi, A.R., Fazio, F., Fieschi, C., 1987. Crossed cerebellar diaschisis in patients with cerebral ischemia assessed by SPECT and 123I-HIPDM. Eur. Neurol. 27, 142–148. Passingham, R.E., 1996. Attention to action. Philos. Trans. R. Soc. Lond. B Biol. Sci. 351, 1473–1479. Passingham, R.E., Wise, S.P., 2012. The Neurobiology of the Prefrontal Cortex: Anatomy, Evolution, and the Origin of Insight. Oxford University Press, Oxford. Perciavalle, V., Berretta, S., Raffaele, R., 1989. Projections from the intracerebellar nuclei to the ventral midbrain tegmentum in the rat. Neuroscience 29, 109–119. Petrides, M., Pandya, D.N., 1999. Dorsolateral prefrontal cortex: comparative cytoarchitectonic analysis in the human and the macaque brain and corticocortical connection patterns. Eur. J. Neurosci. 11, 1011–1036. Petrides, M., Pandya, D.N., 2007. Efferent association pathways from the rostral prefrontal cortex in the macaque monkey. J. Neurosci. 27, 11573–11586. Pothuizen, H.H., Aggleton, J.P., Vann, S.D., 2008. Do rats with retrosplenial cortex lesions lack direction? Eur. J. Neurosci. 28, 2486–2498. Preuss, T.M., 1995. Do rats have prefrontal cortex? The Rose-Woolsey-Akert program reconsidered. J. Cogn. Neurosci. 7, 1–24. Ramnani, N., 2006. The primate cortico-cerebellar system: anatomy and function. Nat. Rev. Neurosci. 7, 511–522. Ramnani, N., 2012. Frontal lobe and posterior parietal contributions to the cortico-cerebellar system. Cerebellum 11, 366–383. Ramnani, N., Miall, R.C., 2003. Instructed delay activity in the human prefrontal cortex is modulated by monetary reward expectation. Cereb. Cortex 13, 318–327. Ramnani, N., Owen, A.M., 2004. Anterior prefrontal cortex: insights into function from anatomy and neuroimaging. Nat. Rev. Neurosci. 5, 184–194. Ramnani, N., Elliott, R., Athwal, B.S., Passinghm, R.E., 2004. Prediction error for free monetary reward in the human prefrontal cortex. Neuroimage 23, 777–786. Ramnani, N., Behrens, T.E.J., Johansen-Berg, H., Richter, M.C., Pinsk, M.A., Andersson, J.L.R., Rudebeck, P., Ciccarelli, O., Richter, W., Thompson, A.J., Gross, C.G., Robson, M.D., Kastner, S., Matthews, P.M., 2006. The evolution of prefrontal inputs to the cortico-pontine system: diffusion imaging evidence from macaque monkeys and humans. Cereb. Cortex 16, 811–818. Ranganath, C., Ritchey, M., 2012. Two cortical systems for memory-guided behaviour. Nat. Rev. Neurosci. 13, 713–726. Rasmussen, A., Jirenhed, D.A., Hesslow, G., 2008. Simple and complex spike firing patterns in Purkinje cells during classical conditioning. Cerebellum 7, 563–566.

References

Rogers, R.D., Ramnani, N., MacKay, C., Wilson, J.L., Jezzard, P., Carter, C.S., Smith, S.M., 2004. Distinct portions of anterior cingulate cortex and medial prefrontal cortex are activated by reward processing in separable phases of decision-making cognition. Biol. Psychiatry 55, 594–602. Rogers, T.D., Dickson, P.E., Heck, D.H., Goldowitz, D., Mittleman, G., Blaha, C.D., 2011. Connecting the dots of the cerebro-cerebellar role in cognitive function: neuronal pathways for cerebellar modulation of dopamine release in the prefrontal cortex. Synapse 65, 1204–1212. Rolls, E.T., Grabenhorst, F., 2008. The orbitofrontal cortex and beyond: from affect to decision-making. Prog. Neurobiol. 86, 216–244. Rudebeck, P.H., Saunders, R.C., Prescott, A.T., Chau, L.S., Murray, E.A., 2013. Prefrontal mechanisms of behavioral flexibility, emotion regulation and value updating. Nat. Neurosci. 16, 1140–1145. Saint-Cyr, J.A., 1983. The projection from the motor cortex to the inferior olive in the cat. An experimental study using axonal transport techniques. Neuroscience 10, 667–684. Sakai, K., Rowe, J.B., Passingham, R.E., 2002. Active maintenance in prefrontal area 46 creates distractor-resistant memory. Nat. Neurosci. 5, 479–484. Sasaki, K., Oka, H., Kawaguchi, S., Jinnai, K., Yasuda, T., 1977. Mossy fibre and climbing fibre responses produced in the cerebeller cortex by stimulation of the cerebral cortex in monkeys. Exp. Brain Res. 29, 419–428. Schall, J.D., Stuphorn, V., Brown, J.W., 2002. Monitoring and control of action by the frontal lobes. Neuron 36, 309–322. Schmahmann, J.D., Pandya, D.N., 1995. Prefrontal cortex projections to the basilar pons in rhesus monkey: implications for the cerebellar contribution to higher function. Neurosci. Lett. 199, 175–178. Schmahmann, J.D., Doyon, J., Toga, A., Evans, A., Petrides, M., 2000. MRI Atlas of the Human Cerebellum. Academic Press, San Diego, CA. Schneider, W., Shiffrin, R.M., 1977. Controlled and automatic human information-processing. 1. Detection, search, and attention. Psychol. Rev. 84, 1–66. Schoenbaum, G., Roesch, M.R., Stalnaker, T.A., Takahashi, Y.K., 2009. A new perspective on the role of the orbitofrontal cortex in adaptive behaviour. Nat. Rev. Neurosci. 10, 885–892. Schultz, W., 2001. Reward signaling by dopamine neurons. Neuroscientist 7, 293–302. Schultz, W., 2013. Updating dopamine reward signals. Curr. Opin. Neurobiol. 23, 229–238. Schultz, W., Dickinson, A., 2000. Neuronal coding of prediction errors. Annu. Rev. Neurosci. 23473–23500. Schultz, W., Tremblay, L., Hollerman, J.R., 2000. Reward processing in primate orbitofrontal cortex and basal ganglia. Cereb. Cortex 10, 272–284. Seamans, J.K., Lapish, C.C., Durstewitz, D., 2008. Comparing the prefrontal cortex of rats and primates: insights from electrophysiology. Neurotox. Res. 14, 249–262. Shiffrin, R.M., Schneider, W., 1977. Controlled and automatic human information-processing. 2. Perceptual learning, automatic attending, and a general theory. Psychol. Rev. 84, 127–190. Shiffrin, R.M., Schneider, W., 1984. Automatic and controlled processing revisited. Psychol. Rev. 91, 269–276. Shimazu, H., Maier, M.A., Cerri, G., Kirkwood, P.A., Lemon, R.N., 2004. Macaque ventral premotor cortex exerts powerful facilitation of motor cortex outputs to upper limb motoneurons. J. Neurosci. 24, 1200–1211.

283

284

CHAPTER 10 Cerebellum and Automatic Processing

Simon, H., Le Moal, M., Calas, A., 1979. Efferents and afferents of the ventral tegmental-A10 region studied after local injection of [3H]leucine and horseradish peroxidase. Brain Res. 178, 17–40. Sladek Jr., J.R., Bowman, J.P., 1975. The distribution of catecholamines within the inferior olivary complex of the cat and rhesus monkey. J. Comp. Neurol. 163, 203–213. Snider, S.R., Snider, R.S., 1977. Alterations in forebrain catecholamine metabolism produced by cerebellar lesions in the rat. J. Neural Transm. 40, 115–128. Snider, R.S., Maiti, A., Snider, S.R., 1976. Cerebellar pathways to ventral midbrain and nigra. Exp. Neurol. 53, 714–728. Sousa-Pinto, A., 1969. Experimental anatomical demonstration of a cortico-olivary projection from area 6 (supplementary motor area?) in the cat. Brain Res. 16, 73–83. Strick, P.L., Dum, R.P., Fiez, J.A., 2009. Cerebellum and nonmotor function. Annu. Rev. Neurosci. 32, 413–434. Sugar, J., Witter, M.P., Van Strien, N.M., Cappaert, N.L., 2011. The retrosplenial cortex: intrinsic connectivity and connections with the (para)hippocampal region in the rat. An interactive connectome. Front. Neuroinform. 5, 7. Suzuki, L., Coulon, P., Sabel-Goedknegt, E.H., Ruigrok, T.J., 2012. Organization of cerebral projections to identified cerebellar zones in the posterior cerebellum of the rat. J. Neurosci. 32, 10854–10869. Swenson, R.S., Sievert, C.F., Terreberry, R.R., Neafsey, E.J., Castro, A.J., 1989. Organization of cerebral cortico-olivary projections in the rat. Neurosci. Res. 7, 43–54. Takada, M., Nambu, A., Hatanaka, N., Tachibana, Y., Miyachi, S., Taira, M., Inase, M., 2004. Organization of prefrontal outflow toward frontal motor-related areas in macaque monkeys. Eur. J. Neurosci. 19, 3328–3342. Tanji, J., Hoshi, E., 2001. Behavioral planning in the prefrontal cortex. Curr. Opin. Neurobiol. 11, 164–170. Taylor, J.A., Ivry, R.B., 2013. Implicit and explicit processes in motor learning. In: Prinz, W., Beisert, M., Herwig, A. (Eds.), Action Science: Foundations of an Emerging Discipline. MIT Press, Cambridge, MA, pp. 63–87. Timmann, D., Drepper, J., Maschke, M., Kolb, F.P., Boring, D., Thilmann, A.F., Diener, H.C., 2002. Motor deficits cannot explain impaired cognitive associative learning in cerebellar patients. Neuropsychologia 40, 788–800. Toonen, M., Van Dijken, H., Holstege, J.C., Ruigrok, T.J.H., Koekkoek, S.K.E., Hawkins, R.K., Teune, T.M., Van Der Burg, J., De Zeeuw, C.I., 1998. Light microscopic and ultrastructural investigation of the dopaminergic innervation of the ventrolateral outgrowth of the rat inferior olive. Brain Res. 802, 267–273. Tremblay, L., Schultz, W., 2000. Reward-related neuronal activity during go-nogo task performance in primate orbitofrontal cortex. J. Neurophysiol. 83, 1864–1876. Tsujimoto, S., Genovesio, A., Wise, S.P., 2010. Evaluating self-generated decisions in frontal pole cortex of monkeys. Nat. Neurosci. 13, 120–126. Tsujimoto, S., Genovesio, A., Wise, S.P., 2011. Frontal pole cortex: encoding ends at the end of the endbrain. Trends Cogn. Sci. 15, 169–176. Tsujimoto, S., Genovesio, A., Wise, S.P., 2012. Neuronal activity during a cued strategy task: comparison of dorsolateral, orbital, and polar prefrontal cortex. J. Neurosci. 32, 11017–11031. Tucker, J., Harding, A.E., Jahanshahi, M., Nixon, P.D., Rushworth, M., Quinn, N.P., Thompson, P.D., Passingham, R.E., 1996. Associative learning in patients with cerebellar ataxia. Behav. Neurosci. 110, 1229–1234.

References

Ueta, Y., Otsuka, T., Morishima, M., Ushimaru, M., Kawaguchi, Y., In press. Multiple layer 5 pyramidal cell subtypes relay cortical feedback from secondary to primary motor areas in rats. Cereb. Cortex. Ugolini, G., Kuypers, H.G.J.M., 1986. Collaterals of corticospinal and pyramidal fibers to the pontine gray demonstrated by a new application of the fluorescent fiber labeling technique. Brain Res. 365, 211–227. Uylings, H.B., Groenewegen, H.J., Kolb, B., 2003. Do rats have a prefrontal cortex? Behav. Brain Res. 146, 3–17. Vann, S.D., Aggleton, J.P., Maguire, E.A., 2009. What does the retrosplenial cortex do? Nat. Rev. Neurosci. 10, 792–802. Walberg, F., 1956. Descending connections to the inferior olive. An experimental study in the cat. J. Comp. Neurol. 104, 77–173. Watson, T.C., Jones, M.W., Apps, R., 2009. Electrophysiological mapping of novel prefrontal— cerebellar pathways. Front. Integr. Neurosci. 3, 18. Wallis, J.D., 2012. Cross-species studies of orbitofrontal cortex and value-based decisionmaking. Nat. Neurosci. 15, 13–19. Walton, M.E., Behrens, T.E., Noonan, M.P., Rushworth, M.F., 2011. Giving credit where credit is due: orbitofrontal cortex and valuation in an uncertain world. Ann. N.Y. Acad. Sci. 1239, 14–24. Wise, S.P., 2008. Forward frontal fields: phylogeny and fundamental function. Trends Neurosci. 31, 599–608. Wise, S.P., Murray, E.A., 2000. Arbitrary associations between antecedents and actions. Trends Neurosci. 23, 271–276. Wolpert, D.M., Miall, R.C., 1996. Forward models for physiological motor control. Neural Netw. 9, 1265–1279. Yeo, C.H., Hesslow, G., 1998. Cerebellum and conditioned reflexes. Trends Cogn. Sci. 2, 322–330. Yin, H.H., Knowlton, B.J., 2006. The role of the basal ganglia in habit formation. Nat. Rev. Neurosci. 7, 464–476.

285

Index Note: Page numbers followed by f indicate figures and t indicate tables.

A Acquisition vs. consolidation, memory formation, 18 Adaptable gain controller, 49 Adaptation eye movement, 122 FC, 130–134 HOKR (see Horizontal optokinetic eye movement response (HOKR) adaptation) HVOR (see Horizontal vestibulo-ocular reflex (HVOR) adaptation) oculomotor cerebellum, 128–130 OMV, 134–144 sensorimotor (see Sensorimotor adaptation) SPA, 126–128 STSA, 124–126 VOR, 122–123, 179–180 Adaptive filter model, 2–3, 36, 49–50, 70–71, 164, 173, 184 Adoption, evolutionary neuroscience, 194–196 AIP. See Anterior interpositus nucleus (AIP) Albus, 32, 70, 72, 108, 159f, 219–220, 222 hypothesis, 3–4, 17, 19, 21, 22 learning rule, 84 a-Amino-3-hydroxy-5-methyl-4-isoxazolone propionic acid (AMPA) dephosphorylation, 9 endocytosis, 6–7 LTD, 19 synaptic membrane, 4 tetramer protein, 4 AMPA. See a-Amino-3-hydroxy-5-methyl-4isoxazolone propionic acid (AMPA) AMPA receptor antagonist CNQX, 91, 92, 93 AMPA receptors constitutive trafficking, 6–7 PKC and cPLA2a, 5–6 stargazin, 9 Amphiphysin, 8 Anterior interpositus nucleus (AIP), 83–84, 85f, 106–108, 171, 174f, 175–176 and DAO, 83–84 inactivation experiments, 89 inferior olive, 88 lidocaine, 88 Anterior prefrontal cortex, 262–263 Anterograde tracers, 197, 271–273 Anthropoid primates, 199, 207, 210–211

Arm movement, 14 Ataxia, 171, 218–219, 221–222, 223, 234 Automatic, 62, 209, 211, 256–257, 263, 264f, 267, 269f, 277–278

B Basal ganglia, 79–80, 116, 203, 209, 226–227, 237–238, 244–246, 265–267, 269–271, 275 Behavioral consequences, molecular layer plasticity, 42 Biomimetic robot, 165–167 Bivariate correlations, 203–204 Blood–brain barrier, 19 Brain–behavior interactions, 194, 195–196, 207–208 Brain evolution, 196, 198–199, 203–204, 207 Brainstem, 9, 11, 68–69, 80–81, 82–83, 92–94, 108–109, 131–132, 178–179, 178f

C Calcium control system, Purkinje cells, 44–45 CaMKII pathway, 6 Cannabinoid 1 (CB1) receptors, 39–40 Caudal medial accessory olive (cMAO), 165–167, 271–273 Caudal vermis, 128–130 Cellular and molecular mechanisms, NMR, 80–81 Cellular mechanisms, 59–60 Cerebellar agenesis, 18 Cerebellar and neocortical contributions, 234–238 Cerebellar circuit and plasticity, 32, 33f, 43f Cerebellar cognitive affective syndrome, 63–65 Cerebellar connectivity, 70, 107f, 196, 197–198 comparative studies, 197–198 prefrontal areas, 197 Cerebellar cortex, 3–4, 14–16, 17–18, 106–108, 107f, 109, 109f, 111–113, 113f, 115, 116, 167–168, 168f, 180, 197 Crus I and Crus II, 267, 268–269, 268f, 270f cytoarchitecture, 197 DCN, 225–226 eyeblink control regions, 84 and higher level feedback, 269–271 and neocortex, 203 NMR and eyeblink conditioning, 91–92 and nuclei, 47–48 numerous sensory inputs, 68–69

287

288

Index

Cerebellar cortex (Continued) PCs, 69–70 posterior, 205, 207 prefrontal cortex, 267 tDCS, 226 Cerebellar hemispheres, 199, 200, 201–204, 205, 207, 209–210 Cerebellar learning and control, 49–50 Cerebellar lobules, 168f, 195–196, 197, 204, 207 Cerebellar macroscopical anatomy, 60–61 Cerebellar microcomplex, 106–108, 165 Cerebellar microzones, 107f, 181 Cerebellectomy, 134 Cerebellum, 108, 193–194 adoption, 194–196 dependent learning, 81–83 and error-based learning, 218–222 evolutionary neuroscience and adoption, 194–196 and forward models, 259–265 learning and human evolution, 193–194 macroevolutionary studies, 198–210 memory mechanisms, 3–4 oculomotor, 128–130 and sensorimotor learning, 243–245 cGMP–PKG–GS–protein phosphatases pathway, 6 Cholera toxin beta subunit, 271–273, 273f Chronic adaptation, 123 Cingulate, 271–273, 274, 275 Classical conditioning, 13, 79–80, 81–83, 84, 105–106, 108, 169f, 170–171, 175–176, 225, 265 Classic cerebellar doctrine, 72, 72f Climbing fibres (CF), 32, 33f, 37–38, 39, 40–42, 43f, 44, 46, 68–71, 68f, 72, 83–84, 85f, 111–114, 112f, 134–135, 219, 271–275 activity, 162 Ca2þ ions, 4–5 folium-p Purkinje cells, 14 LTD, 162–163 mechanisms, LTD, 40 Purkinje cells, 158, 159f, 171 signal, 170–171 specific vs. nonspecific changes, 16–17 STDP, 160 cMAO. See Caudal medial accessory olive (cMAO) Cognitive (cognition), 45–46, 61–62, 63–68, 66f, 125–126, 181, 182–184, 194, 195, 203–204, 209, 211, 230, 234–236, 237, 256–257, 259, 262–263, 264f, 265–278 Cognitive and emotional behavior, 61–62, 65–67, 66f Cognitive habits, 265–277 Comparative method, 194, 196

Comparative studies, 65–67 cerebellar connectivity, 197–198 Compensation, motor learning, 20 Complex spikes (CS), 11–12, 40, 62, 111, 113–114, 144–148, 271–273 and CF, 144–146 conceptual model, OMV during STSA, 147–148, 148f gain-increase and gain-decrease STSA, 147–148 inferior olive neurons, 146–147 LTD, 147–148 oculomotor adaptation, 146–147 OMV CS modulation, 144–146 simple spike (SS), 147–148 STSA and SPA, 144–146, 145f Computational models, 36–37, 222–224 Computational models of sensorimotor adaptation cerebellar degeneration, 223 Markov-chain process, 223–224 Marr-Albus model, 222 multirate models, 223 patients with cerebellar ataxia, 223, 224f perturbation size, 222 state-space models, 222–223 visuomotor rotations, 223–224 Conditional stimulus (CS), 42, 105–106, 169–170 lidocaine effects, 88 mossy fibers and unconditional stimulus, 85f neural spiking level, 82–83 Conditional visuomotor learning, 265–267 Conditioned responses (CRs), 16–17, 105, 106, 169–170, 176, 219–220, 225–226, 276–277 suppress olivary activity, 109, 110f Constitutive trafficking, AMPA receptors, 6–7 Controlled, 7–8, 13, 34, 35, 36, 81–82, 94–95, 209, 211, 256–257, 260–261, 262, 264f, 277–278 Control theory (control theoretic), 257–258, 260–262 Cortical lobules HIV-HVI and HVIIIB, 260–261 Cortical microcircuit, 158, 159f Cortical microzone, 106 Cortical plasticity, NMR, 94–95 Cortico-cerebellar connectivity, 197–198, 210–211 Cortico-cerebellar evolution, 203 Corticopontine, 260, 261–262, 275–276 Cortico-ponto-cerebellar system, 197–198 Cre-lox technology, 63 Crossed cerebral-cerebellar diaschesis, 265–267 CRs. See Conditioned responses (CRs) CS–US trials, 109, 115 Cytosolic phospholipase A2a (cPLA2a), 5–6

Index

D DAO. See Dorsal accessory olive (DAO) DCN. See Deep cerebellar nuclei (DCN) Decorrelation learning cerebellum adaptive filter, 159f, 164 motor control, 173–181 sensory prediction, 165–173 STDP, 160–162 parallel-fiber signal, 158, 161f, 162–163, 221–222 Deep cerebellar nuclei (DCN), 32, 33f, 48, 137–139, 225–226 Depression, PF–Purkinje cell synapse, 70–71 Descending motor pathway, 203–204 Discovery of cerebellar LTD, 32 Dopamine (and dopaminergic), 46, 116, 226–227, 237–238, 264f, 274, 275–278 Dopamine neurons, 116, 274, 275–277 Dopamine system, cerebellar communication, 275–277 Dorsal accessory olive (DAO), 34, 83–84, 85f, 89–90, 106–108, 171, 173–175, 174f, 271–273 Dorsal paraflocculus (DPF), 128 Dorsal pontine nuclei (DPN), 134–135 Dorsolateral prefrontal cortex (DLPFC), 234–236 Double-labeling techniques, 69–70, 276 Dual systems, 256–257, 267, 269f

E EBC. See Eyeblink conditioning (EBC) Efference copy, 124, 165–168, 166f, 170, 172, 173–175, 177, 178f, 179–180, 181–182, 219, 258f, 260, 261–262, 277–278 Electrical activity, floccular Purkinje cells, 13 Electrical stimulation, 12–13, 69–70, 81–82, 135, 142 Electron microscopy (EM), 12, 13 Electrophysiological evidence, 171–172 Electrophysiology, eyeblink control, 84 Eletrophysiological methods, 271–273 Endocannabinoids, 39–40 EPSPs, 111, 112f, 113–114, 113f PF-EPSC, 4, 6–7, 20–21 Equilibrium, 111–114 Error-based learning ataxia, 218–219, 221–222 climbing fibers, 219 environmental perturbations, 220 eyeblink conditioning and VOR, 219–220 hypothetical learning curve, 220, 221f Marr–Albus theory, 219–220

parallel fibers, 219 PC, 219 perturbation, 220, 221 prism adaptation, 221–222 Errors, 2, 3–4, 9, 11, 13, 14, 22, 37, 41–42, 46, 49–50, 68–69, 257–258, 259, 264f, 265–267, 269–271 CF, 158 climbing fibers, 271–275 decorrelation learning, 167 motor error, 179 motor performance, 3–4 neuron output, 184 Purkinje cell firing, 158 retinal slips, 11 sensory error, 179 Evolutionary neuroscience, 194–196 cerebellum and cognition, 195 macroevolutionary studies, 196 phylogenetic comparative methods, 194 Eye blink conditioning (EBC), 3, 16, 17–18, 19, 48, 62, 82–83, 84, 85f, 87, 95, 104, 105, 107f, 108, 109, 110f, 116, 163–164, 169–171, 172, 175, 177, 181, 219–220, 225–226, 276–277 cerebellar cortex function, 91–92 cerebellar learning, 48 CF signal, 48, 170–171 CS-CR relationship, 82–83 inferior olive function, 89–91 nucleo-olivary pathway, 48, 170–171 Purkinje cell, 171 sensory prediction circuitry, 171 SLRs, 93–94 two-layer model, 92 and VOR, 219–220 Eye movement adaptation, 122, 128 Eye velocity PC (EVP), 132–133

F Fast HOKR adaptation, LTD, 11–12 Fatigue, 124, 125–126, 127–128, 139–142 FC PC SS, 132–133, 134 Feedback control, 114–115 anticipate, 104–105 cerebellar microcomplex, 106–108 classical conditioning, 105–106, 108 equilibrium, 111–114 negative, 108–111 N-O pathway, 108–111 Fiber-sparing techniques, 87 Firing rate model, 36 First-order rules, 268–269, 270f

289

290

Index

Floccular complex (FC), 128–130 definition, 130 SPEMs and SPA, 132–134 VOR and VOR adaptation, 130–132 Flocculus decorrelation learning, 178–179 medial vestibular nucleus, 12–13 Mossy fibers, 16 OMV, 128–130 Purkinje cells, 11 Folium-p Purkinje cells, 14 Force-field adaptation, 221–222, 226 Forward controller loop, 49 Forward models, 124, 125–126, 234, 241, 243–244, 257–258, 258f, 259–265, 259f, 271, 274–275, 277–278 Fovea, 122, 124, 126–127 Frontal lobe, 203, 205, 207, 260–262, 265–267, 271, 277–278 Frontal motor areas, 203–204, 205, 207, 210 Functional MRI, 267

G GABAA antagonists, 92–94 GABAergic, 18, 35, 40–41, 48, 88–89, 94, 108–109, 114 GABAergic inhibition, 35 Gain control operation, 49 Gaze stabilization, motor control cerebellar substrates, 128 flocculus, 178–179 pain signal, 174f, 178–179 synaptic plasticity, 180 VOR, 128 Gaze velocity PC SS units (GVP), 131–133 Gene expression, 82 Gene-knockout mice, 11–12 GluA2 phosphorylation, 8 Golgi cell inhibition, geometrical arrangement, 45 Granular layer plasticity GABAergic inhibition, 35 in vivo, 35 timing, geometry and coding, 35–36 Granule cell activity, HOKR, 15f, 16 Granule cell layer, 158, 159f, 164 dopamine neurons, 22, 34–37, 275–276 intrinsic plasticity, 14–16 mossy fiber, 22, 34–37 Purkinje cells, 22, 34–37, 159f Great apes, 195–196, 201–203, 204, 205, 207, 210–211 Guanylate cyclase (GC), 38

H Habit, 256–257, 265–278 Hebbian process, 226 Hippocampal system, 274, 275 Horizontal optokinetic eye movement response (HOKR) adaptation, 3, 9–13, 15f Horizontal vestibulo-ocular reflex (HVOR) adaptation, 3, 9, 10f, 11–13, 16–17, 18 Human evolution, 193–194

I Inferior olive (IO), 10f, 13, 45–46, 68–70, 91, 106, 107f, 109–111, 115–116, 134–135, 144–147, 148f, 167–168, 168f, 171, 172, 175–176, 178–179, 178f, 219–220, 259f, 264f, 269–273, 274, 275–277 and CS, 144–146 NMR and eyeblink conditioning, 87–91 Inhibition, 2, 15f, 20–21, 69–70, 89–90, 94, 109–111, 113f, 114–115, 276–277 Instrumental learning, 32, 82, 175, 265 Internal models, 17, 104, 105, 106, 222–224, 227, 229, 237–238, 257–258 Interneuronal circuit, 68–69, 68f Interstimulus interval (ISI), 92, 108 In vitro brain slices, 20–21 Inward STSA, 124–126 IO. See Inferior olive (IO) ISI. See Interstimulus interval (ISI)

K Kamin blocking, 90, 105–106

L Lamina V cells, 260 Learning, 32, 36, 37, 41, 42, 46–47, 59–60, 62, 193–194. See also Instrumental learning cerebellum and human evolution, 193–194 cerebellum-dependent motor learning, 121–156 cognitive habits, 265–277 control theory, 257–258 DCN, 48 decorrelation learning (see Decorrelation learning) error-based learning, 218–222 feedback control, 103–120 LTD (see Long-term depression (LTD)) motor (see Motor learning) motor, granule cell computation, 63, 64f NMR (see Nictitating membrane response (NMR))

Index

PF–Purkinje cell synapse, 70–71 sensorimotor (see Sensorimotor learning) Learning mechanisms in sensorimotor adaptation consolidation, 225 decision making, 226–227 internal model, adaptation, 229 movement errors, 228–229 observations, 225 online feedback, 229 postacquisition, 225–226 reinforcement and error-based learning, 227 reinforcement-learning model, 229 remapping, 227–228 reward-based and error-based feedback, 227–228, 228f, 229 savings, 225 single-process version, 224–225 tDCS, 226 Least mean squares (LMS), 184 Lesion, 11–12, 20, 65–67, 70–71, 79–80, 83–88, 89–90, 91, 92–93, 122, 130–131, 132, 133–134, 135, 139–141, 140f, 142, 143–144, 164–165, 177, 195–196, 218, 219–220, 221–222, 225–226, 235f, 236–237, 244–245, 265–267, 274, 276–277 Lobule HVI, C1 and C3 cortical zones, 83–87 Long-term depression (LTD), 64f, 70–71, 91, 147–148, 162–164, 180, 267 CF, 162–163 mechanisms, climbing fiber, 40 memory mechanisms, cerebellum, 3–4 molecular mechanisms, 4–9 mossy fiber–granule cell plasticity, 34–37 and motor learning, 3, 9–14 postsynaptic parallel fiber, 37–39 presynaptic parallel fiber, 39–40 Purkinje cells, 180 STDP, 163–164 Long-term potentiation (LTP), 32, 33f, 40, 41–42, 44, 45, 46, 66f, 161f, 162–164, 180 climbing fiber, 16–17 exocytosis and endocytosis, GluA2, 7–8 mossy fiber, 14–16 mossy fiber–granule cell plasticity, 34–37 nitric oxide (NO), 6 parallel fibers, 7, 162–163 postsynaptic parallel fiber, 37–39 presynaptic parallel fiber, 39–40 protein phosphatases, 8 Purkinje cells, 163, 164 Long-term saccadic adaptation (LTSA), 126 LTD. See Long-term depression (LTD) LTP. See Long-term potentiation (LTP)

M Macaque brain, 197 Macroevolutionary studies, 198–210 brain–behavior interactions, 207–208 capuchin monkeys, 209–210 cerebellar connectivity, 197–198 conchoidal flaking, 209 cortico-cerebellar processing, 209–211 experience and trial-by-trial, 209 great apes and humans, 210 learning and human evolution, 193–194 neuroimaging studies, 209 nut-cracking tasks, 208 phylogenetic correlations, 203–204 phylogenetic mapping, 204–207 phylogenetic scaling, 198–203 stone-flaking, 209 tool use, 208 Markov-chain process, 223–224 Marr, 32, 70, 72, 108, 159f, 219–220, 222 hypothesis, 3–4, 17, 19, 21, 22 learning rule, 84 Marr–Albus theory, 219–220, 222 Memory consolidation, 87–88, 94–95 Memory formation vs. signal enhancement, 14–16 Memory mechanisms, cerebellum, 3–4 Memory transfer, slow HOKR adaptation, 12–13 “Mental models”, 262 Meta-adaptation, 126 Metabotropic glutamate receptor type 1 (mGluR1), 4–5, 39 Metabotropic receptors (mGluRs), 34, 37–38 Microcircuitry, 80–81 Molecular layer, 2, 14–16, 15f, 17–18, 22 climbing fiber LTD, 40 inhibitory synapses, 40–41 postsynaptic parallel fiber LTP and LTD, 37–39 presynaptic parallel fiber LTP and LTD, 39–40 Purkinje cell excitability, 40 Mossy-fiber (MF), 165, 169–170 collateral plasticity, 93 corticonuclear microcomplex, 3 flocculus, 16 granule cell plasticity, 2–3 neural network model, 2 OMV, 134–135 Mossy fiber–granule cell plasticity, 34–37 Motor behavior cognitive and emotional symptoms, 61–62 Cre-lox technology, 63 posture control and muscle tone, 62

291

292

Index

Motor control, cerebellum brainstem, 130–131 cerebellar function, 61–62 gaze stabilization, 177–180 signal avoidance, 173-177 Motor cortex, 226, 257, 260–262, 263, 264f Motor learning, 104 granule cell computation, 63, 64f LTD HOKR adaptation, 9–13 HVOR adaptation, 11 memory transfer, 13 saccade, 13 Motor sequence learning, 263 Muscular contractions, 104

N Negative feedback, 108–111 Neocortex, 198–199, 201–203, 226, 257, 261–262 Neocortical, 198–199 ataxia, 234 basal ganglia, 237–238 dopamine level, 238 LPFC, 236–237 neuropsychological evidence, 234–236, 235f older adults, 236 PFC lesions, 236 PFC patients and controls, 237 sensory prediction error signal, 237 two-process model, 234–236 N-ethylmaleimide-sensitive fusion (NSF) protein, 6–8 Neural implementation, 181 Neuroimaging and neurostimulation, humans in vivo, 32 Neuromodulatory mechanisms of gating, 46 “Neuronal machine”, 2 Neuron density, 65–67 Neurophysiological consequences, molecular layer plasticity, 41–42 Neurotransmission, 14–16, 35–36 Nicotine, 34 Nictitating membrane (NM), 175–176 Nictitating membrane response (NMR) C1 and C3 cortical zones, lobule HVI, 83–87 cerebellar cortex function, 91–92 classical conditioning, 79–80, 81–83 cortical plasticity, 94–95 inferior olive and eyeblink conditioning, 87–91 NM. See Nictitating membrane (NM) NMR. See Nictitating membrane response (NMR)

Noise-cancelation circuit, 170, 180–181, 183–184 Non monotonic function, 239 NO system of granule cells, 44 Nucleo-olivary, 88–90, 104, 106–111, 107f, 113f, 114–116, 169f, 171, 176–177 Nucleo-olivary (N-O) pathway, 88–90, 108–111, 114–115, 169–170, 169f, 176–177 cerebellar microcomplex, 104, 108–111 CF signal, 90 Nucleus reticularis tegmenti pontis (NRTP), 10f, 11, 130, 134–135, 271–273

O OCN. See Olivo-cortico-nuclear (OCN) Oculomotor cerebellum, 128–130 Oculomotor plant, 125–126, 178–179, 178f Oculomotor system, 124–125 Oculomotor vermis (OMV), 128–130, 134–144 PB, 137–139, 138f, 141–142 PC SS, 137–139 saccades, STSA and saccadic resilience, 135–142 smooth pursuit eye movements, 139–141, 140f SPEMs, SPA and SPEM resilience, 142–144 Olivary cells, 106–108 Olivocortical projections, 68–69 Olivo-cortico-nuclear (OCN) AIP inactivation, 89 cerebellar cortex, cerebellar nuclei and inferior olive, 85f, 88–89 CNQX, 91 OMV. See Oculomotor vermis (OMV) Optokinetic reflex (OKR), 62, 63, 64f, 79–80, 122–123, 180 Optokinetic responses (OKR), 62, 64f, 79–80 Orbitofrontal cortex, 244–245, 274, 275 Outward STSA, 124–125

P Parallel fibers (PFs), 158, 160–164, 172, 219 and CF, 2, 4–5 LTP and LTD, 37–40 PF–Purkinje cell synapses, 7, 12–13, 14–17, 19, 20, 32, 35, 36 Paramedian pontine reticular formation (PPRF), 134–135 Parkinson’s disease, 237–238, 265–267 Pavlovian learning system, 82 PC. See Purkinje cells (PC) Perceptron, 2, 41 Perturbation, 220, 221, 221f, 222–225, 226, 230–231, 236, 240, 244 Phylogenetic comparative methods, 194

Index

Phylogenetic correlations, 203–204 Phylogenetic mapping clade-general correlation patterns, 204–205 cortico-cerebellar formation, 205 DT-MRI tractography, 207 evolutionary rates, 205, 206f neuroanatomical and neuroimaging studies, 207 prefrontal and posterior cerebellar size, 207 Phylogenetic scaling brain structure sizes, 198–199 cerebellar hemispheres, 199, 200, 201f gene-sequencing techniques, 200 grade shift, 199 prefrontal cortex þ posterior cerebellar cortex volume, 201–203, 202f Picrotoxin, 85f, 92–94 PKC phosphorylation of Ser880, 8 Plasticity, 32, 33f, 64f, 70–71, 80–81, 83–84, 91, 92, 104, 114, 116, 162–164, 259, 265–267 bidirectional, 14–16, 17–18, 39–40, 162–163, 164 cerebellar circuitry, 259 cerebellar cortex, 42–48 distribution, cerebellar cortical and cerebellar nuclear, 92–94 granular layer, 34, 35–36 LTD (see Long-term depression (LTD)) memory formation vs. signal enhancement, 45 molecular layer, 37–42 mossy fiber–granule cell, 34–37 NMR and eyeblink conditioning, 37–42 synapsis, 180 Pontine nuclei (also called ‘pons’), 68–69, 84, 130, 197, 203, 219, 260–263, 271–273, 274–275 Population burst (PB), 137–139, 138f, 141–142 Posterior cerebellum hemisphere in apes, 200 and prefrontal cortex, 202f, 205, 207 prefronto-posterior cerebellar system, 207 Postsynaptic Ca2þ transients, 38–39 Postsynaptic parallel fiber LTP and LTD, 33f, 37–39 Precerebellar structures, 172 Prefrontal activity, 262 Prefrontal areas, 197, 262–265 Prefrontal cortex, 200, 201–204, 202f, 205, 206f, 207, 210, 211, 234–237, 235f, 239, 244–246, 257, 260, 262–263, 264f, 265–267, 268–269, 268f, 271–273, 274–275, 276, 277–278 infralimbic, 271–273 orbitofrontal, 271–273 prelimbic, 271–273 “Prefrontal loop”, 260 Prefrontal-prefrontal collateral projections, 262–263

Premotor cortex, 209, 260–267, 261f, 274–275 Presynaptic parallel fiber LTP and LTD, 33f, 39–40 Primary motor cortex, 197, 257–258, 259f, 260–262, 261f, 264f Primate cerebellum, 128, 129f Principal olive (PO), 271–273 Principle of proper mass, 195–196 Prism adaptation, 14, 221–222 Protein kinase C a (PKCa), 5–6 Protein phosphatases, 8 Purkinje cell excitability, 40 Purkinje cells (PC), 2, 7, 8, 12, 13, 17–18, 21, 32, 37, 38, 40, 44–45, 84, 85f, 106–111, 107f, 112f, 113–116, 113f, 158, 167–168, 171, 180, 219, 269–271 CR mirrors, 108 double-labeling approach, 69–70 electrical synapses, Golgi cells, 70 interneuronal circuit, 68–69, 68f mossy fibers and climbing fibers, 68–69 Pyramidal cells, 6, 271–273, 273f

R Reafference, 165–172, 173, 183–184 Reciprocal connections, 63–65 Reinforcement learning, 226–227, 228–230, 237–238, 242–243, 244–245 Rescorla–Wagner model, 116 Retina, 122–123 Retinal slip (RS), 11, 145f, 177–179, 180 gaze stabilization, 11, 144–146, 145f, 177–179, 220 Retrosplenial cortex, 274, 275 Reward, 46, 116, 226–229, 228f, 238, 242–243, 244–245, 256–257, 264f, 265, 269–271, 274, 275, 276–278 Robot locomotion, 177

S Saccades, 13, 124–125, 181 Sensorimotor adaptation cerebellar and neocortical contributions, 234–238 complex force-field perturbations, 230 computational models, 222–224 error signal, 231–232 field goal kicker, 230 learning mechanisms, 224–229 movement goals and sensory predictions, 232 reinforcement-learning, 229–230 strategic and explicit process, 230 target errors, 232–234, 233f

293

294

Index

Sensorimotor adaptation (Continued) visuomotor rotation, 230–231, 231f Sensorimotor learning and cerebellum, 243–245 systems interaction, 238–243 Sensory prediction, cerebellum dumb system, 239 imaging techniques, 173 motor control tasks, 173 reafference problem, 165–172 Sexuality, 59–60 Short-latency responses (SLRs), 85f, 92–94 Short-term saccadic adaptation (STSA), 124–126, 125f, 127–128 Signal-to-noise ratio, mossy fiber, 44–45 Signal transduction, LTD, 4–6 Skills, 256–257, 263, 264f, 267, 269–271 SLRs. See Short-latency responses (SLRs) Smooth pursuit adaptation (SPA), 126–128, 142–144 Smooth pursuit eye movements (SPEMs), 126–128, 142–144 Spike-timing dependent plasticity (STDP) climbing-fiber activity, 162 depression and potentiation, 160 expect delay, 160–162, 161f Spinal and musculoskeletal system, 261–262, 261f Standardized motor tests, 62 Stargazin, 9 State–space models, 222–223 STDP. See Spike-timing dependent plasticity (STDP) Stereotypical step-ramp smooth pursuit eye movements, 127–128 Strike zone, 217–218 Substantia nigra, 276–277 Superior colliculus (SC), 13, 134–135, 167–168 Supervised learning, 17–18, 37, 41, 48, 164, 184–185, 194, 219, 259f, 269–271 Supervised learning machine, 194 Supervised learning vs. unsupervised learning, 17 Swimming, 122 Systems interaction in sensorimotor learning

aiming direction, 241, 242f “dumb” system, 239 explicit/declarative knowledge, 240–241 forward model adaptation, 243 landmarks, training blocks, 240 model-free and model-based reinforcement, 242–243 nonmonotonicity, 239 performance, asymptotic, 240 sensory feedback, 240 sensory prediction error, 240 skill acquisition, 238–239 state-space model, 241 strategic and adaptation process, 239

T Tetanus toxin (TeTx), 6–7 Theta-burst patterns, granular layer, 45–46 Time-lock activity, 267, 268–269, 268f Trigeminal nucleus, 84, 85f, 175–176 Two-layer model, 85f, 92

U Unconditioned stimulus (US), 105–106 Use-dependent learning, 226

V Ventral paraflocculus (VPF), 128, 130, 131–132, 133–134 Ventral tegmental area (VTA), 264f, 275–277 Vermal lobules VB-HVIIIB, 260 Vestibular, 122–123, 128–130, 132–133, 168–169 Vestibular nucleus, 10f, 11, 12–13, 180 Vestibulo-ocular reflexes (VOR), 42, 48, 62, 64f, 70–71, 79–81, 82–83, 92, 116, 122–123, 128, 130–134, 168–169, 175, 177–181, 178f, 219–220 Visuo-vestibular conflict stimuli, 132–133

W Working memory, 256–257

Other volumes in PROGRESS IN BRAIN RESEARCH

Volume 161: Neurotrauma: New Insights into Pathology and Treatment, by J.T. Weber and A.I.R. Maas (Eds.) – 2007, ISBN 978-0-444-53017-2. Volume 162: Neurobiology of Hyperthermia, by H.S. Sharma (Ed.) – 2007, ISBN 978-0-444-51926-9. Volume 163: The Dentate Gyrus: A Comprehensive Guide to Structure, Function, and Clinical Implications, by H.E. Scharfman (Ed.) – 2007, ISBN 978-0-444-53015-8. Volume 164: From Action to Cognition, by C. von Hofsten and K. Rosander (Eds.) – 2007, ISBN 978-0-444-53016-5. Volume 165: Computational Neuroscience: Theoretical Insights into Brain Function, by P. Cisek, T. Drew and J.F. Kalaska (Eds.) – 2007, ISBN 978-0-444-52823-0. Volume 166: Tinnitus: Pathophysiology and Treatment, by B. Langguth, G. Hajak, T. Kleinjung, A. Cacace and A.R. Møller (Eds.) – 2007, ISBN 978-0-444-53167-4. Volume 167: Stress Hormones and Post Traumatic Stress Disorder: Basic Studies and Clinical Perspectives, by E.R. de Kloet, M.S. Oitzl and E. Vermetten (Eds.) – 2008, ISBN 978-0-444-53140-7. Volume 168: Models of Brain and Mind: Physical, Computational and Psychological Approaches, by R. Banerjee and B.K. Chakrabarti (Eds.) – 2008, ISBN 978-0-444-53050-9. Volume 169: Essence of Memory, by W.S. Sossin, J.-C. Lacaille, V.F. Castellucci and S. Belleville (Eds.) – 2008, ISBN 978-0-444-53164-3. Volume 170: Advances in Vasopressin and Oxytocin – From Genes to Behaviour to Disease, by I.D. Neumann and R. Landgraf (Eds.) – 2008, ISBN 978-0-444-53201-5. Volume 171: Using Eye Movements as an Experimental Probe of Brain Function—A Symposium in Honor of Jean Bu¨ttner-Ennever, by Christopher Kennard and R. John Leigh (Eds.) – 2008, ISBN 978-0-444-53163-6. Volume 172: Serotonin–Dopamine Interaction: Experimental Evidence and Therapeutic Relevance, by Giuseppe Di Giovanni, Vincenzo Di Matteo and Ennio Esposito (Eds.) – 2008, ISBN 978-0-444-53235-0. Volume 173: Glaucoma: An Open Window to Neurodegeneration and Neuroprotection, by Carlo Nucci, Neville N. Osborne, Giacinto Bagetta and Luciano Cerulli (Eds.) – 2008, ISBN 978-0-444-53256-5. Volume 174: Mind and Motion: The Bidirectional Link Between Thought and Action, by Markus Raab, Joseph G. Johnson and Hauke R. Heekeren (Eds.) – 2009, 978-0-444-53356-2. Volume 175: Neurotherapy: Progress in Restorative Neuroscience and Neurology — Proceedings of the 25th International Summer School of Brain Research, held at the Royal Netherlands Academy of Arts and Sciences, Amsterdam, The Netherlands, August 25–28, 2008, by J. Verhaagen, E.M. Hol, I. Huitinga, J. Wijnholds, A.A. Bergen, G.J. Boer and D.F. Swaab (Eds.) –2009, ISBN 978-0-12-374511-8. Volume 176: Attention, by Narayanan Srinivasan (Ed.) – 2009, ISBN 978-0-444-53426-2. Volume 177: Coma Science: Clinical and Ethical Implications, by Steven Laureys, Nicholas D. Schiff and Adrian M. Owen (Eds.) – 2009, 978-0-444-53432-3. Volume 178: Cultural Neuroscience: Cultural Influences On Brain Function, by Joan Y. Chiao (Ed.) – 2009, 978-0-444-53361-6. Volume 179: Genetic models of schizophrenia, by Akira Sawa (Ed.) – 2009, 978-0-444-53430-9. Volume 180: Nanoneuroscience and Nanoneuropharmacology, by Hari Shanker Sharma (Ed.) – 2009, 978-0-444-53431-6. Volume 181: Neuroendocrinology: The Normal Neuroendocrine System, by Luciano Martini, George P. Chrousos, Fernand Labrie, Karel Pacak and Donald W. Pfaff (Eds.) – 2010, 978-0-444-53617-4. Volume 182: Neuroendocrinology: Pathological Situations and Diseases, by Luciano Martini, George P. Chrousos, Fernand Labrie, Karel Pacak and Donald W. Pfaff (Eds.) – 2010, 978-0-444-53616-7. Volume 183: Recent Advances in Parkinson’s Disease: Basic Research, by Anders Bjo¨rklund and M. Angela Cenci (Eds.) – 2010,978-0-444-53614-3. Volume 184: Recent Advances in Parkinson’s Disease: Translational and Clinical Research, by Anders Bjo¨rklund and M. Angela Cenci (Eds.) – 2010, 978-0-444-53750-8.

296

Other volumes in PROGRESS IN BRAIN RESEARCH

Volume 185: Human Sleep and Cognition Part I: Basic Research, by Gerard A. Kerkhof and Hans P.A. Van Dongen (Eds.) – 2010, 978-0-444-53702-7. Volume 186: Sex Differences in the Human Brain, their Underpinnings and Implications, by Ivanka Savic (Ed.) – 2010, 978-0-444-53630-3. Volume 187: Breathe, Walk and Chew: The Neural Challenge: Part I, by Jean-Pierre Gossard, Re´jean Dubuc and Arlette Kolta (Eds.) – 2010, 978-0-444-53613-6. Volume 188: Breathe, Walk and Chew; The Neural Challenge: Part II, by Jean-Pierre Gossard, Re´jean Dubuc and Arlette Kolta (Eds.) – 2011, 978-0-444-53825-3. Volume 189: Gene Expression to Neurobiology and Behaviour: Human Brain Development and Developmental Disorders, by Oliver Braddick, Janette Atkinson and Giorgio M. Innocenti (Eds.) – 2011, 978-0-444-53884-0. Volume 190: Human Sleep and Cognition Part II: Clinical and Applied Research, by Hans P.A. Van Dongen and Gerard A. Kerkhof (Eds.) – 2011, 978-0-444-53817-8. Volume 191: Enhancing Performance for Action and perception: Multisensory Integration, Neuroplasticity and Neuroprosthetics: Part I, by Andrea M. Green, C. Elaine Chapman, John F. Kalaska and Franco Lepore (Eds.) – 2011, 978-0-444-53752-2. Volume 192: Enhancing Performance for Action and Perception: Multisensory Integration, Neuroplasticity and Neuroprosthetics: Part II, by Andrea M. Green, C. Elaine Chapman, John F. Kalaska and Franco Lepore (Eds.) – 2011, 978-0-444-53355-5. Volume 193: Slow Brain Oscillations of Sleep, Resting State and Vigilance, by Eus J.W. Van Someren, Ysbrand D. Van Der Werf, Pieter R. Roelfsema, Huibert D. Mansvelder and Fernando H. Lopes da Silva (Eds.) – 2011, 978-0-444-53839-0. Volume 194: Brain Machine Interfaces: Implications For Science, Clinical Practice And Society, by Jens Schouenborg, Martin Garwicz and Nils Danielsen (Eds.) – 2011, 978-0-444-53815-4. Volume 195: Evolution of the Primate Brain: From Neuron to Behavior, by Michel A. Hofman and Dean Falk (Eds.) – 2012, 978-0-444-53860-4. Volume 196: Optogenetics: Tools for Controlling and Monitoring Neuronal Activity, by Thomas Kno¨pfel and Edward S. Boyden (Eds.) – 2012, 978-0-444-59426-6. Volume 197: Down Syndrome: From Understanding the Neurobiology to Therapy, by Mara Dierssen and Rafael De La Torre (Eds.) – 2012, 978-0-444-54299-1. Volume 198: Orexin/Hypocretin System, by Anantha Shekhar (Ed.) – 2012, 978-0-444-59489-1. Volume 199: The Neurobiology of Circadian Timing, by Andries Kalsbeek, Martha Merrow, Till Roenneberg and Russell G. Foster (Eds.) – 2012, 978-0-444-59427-3. Volume 200: Functional Neural Transplantation III: Primary and stem cell therapies for brain repair, Part I, by Stephen B. Dunnett and Anders Bjo¨rklund (Eds.) – 2012, 978-0-444-59575-1. Volume 201: Functional Neural Transplantation III: Primary and stem cell therapies for brain repair, Part II, by Stephen B. Dunnett and Anders Bjo¨rklund (Eds.) – 2012, 978-0-444-59544-7. Volume 202: Decision Making: Neural and Behavioural Approaches, by V.S. Chandrasekhar Pammi and Narayanan Srinivasan (Eds.) – 2013, 978-0-444-62604-2. Volume 203: The Fine Arts, Neurology, and Neuroscience: Neuro-Historical Dimensions, by Stanley Finger, Dahlia W. Zaidel, Franc¸ois Boller and Julien Bogousslavsky (Eds.) – 2013, 978-0-444-62730-8. Volume 204: The Fine Arts, Neurology, and Neuroscience: New Discoveries and Changing Landscapes, by Stanley Finger, Dahlia W. Zaidel, Franc¸ois Boller and Julien Bogousslavsky (Eds.) – 2013, 978-0-444-63287-6. Volume 205: Literature, Neurology, and Neuroscience: Historical and Literary Connections, by Anne Stiles, Stanley Finger and Franc¸ois Boller (Eds.) – 2013, 978-0-444-63273-9. Volume 206: Literature, Neurology, and Neuroscience: Neurological and Psychiatric Disorders, by Stanley Finger, Franc¸ois Boller and Anne Stiles (Eds.) – 2013, 978-0-444-63364-4. Volume 207: Changing Brains: Applying Brain Plasticity to Advance and Recover Human Ability, by Michael M. Merzenich, Mor Nahum and Thomas M. Van Vleet (Eds.) – 2013, 978-0-444-63327-9.

Other volumes in PROGRESS IN BRAIN RESEARCH

Volume 208: Odor Memory and Perception, by Edi Barkai and Donald A. Wilson (Eds.) – 2014, 978-0-444-63350-7. Volume 209: The Central Nervous System Control of Respiration, by Gert Holstege, Caroline M. Beers and Hari H. Subramanian (Eds.) – 2014, 978-0-444-63274-6.

297