Current Controversies In Philosophy Of Perception 1138840076, 9781138840072, 1317556720, 9781317556725

Citation preview

Current Controversies in Philosophy of Perception

This book provides an up-to-date and accessible overview of the hottest and most influential contemporary debates in philosophy of perception, written especially for this volume by many of the most important philosophers of the field. The book addresses the following key questions: • • • • •

Can perception be unconscious? What is the relation between perception and attention? What properties can we perceive? Are perceptual states representations? How is vision different from the other sense modalities (like hearing or smell)? • How do these sense modalities interact with one another? Contributors are Ned Block, Berit Brogaard, Alex Byrne, Robert W. Kentridge, John Kulvicki, Heather Logue, Mohan Matthen, Bence Nanay, Matthew Nudds, Casey O’Callaghan, Adam Pautz, Ian Phillips, Susanna Siegel, and Wayne Wu. Bence Nanay is Professor of Philosophy and BOF Research Professor at the University of Antwerp, where he is also co-director of the Centre for Philosophical Psychology and Senior Research Associate at Peterhouse, Cambridge University. He is the author of Between Perception and Action (2013) and Aesthetics as Philosophy of Perception (2016) and the editor of Perceiving the World (2010).

Current Controversies in Philosophy In venerable Socratic fashion, philosophy proceeds best through reasoned conversation. Current Controversies in Philosophy provides short, accessible volumes that cast a spotlight on ongoing central philosophical conversations. In each book, pairs of experts debate four or five key issues of contemporary concern, setting the stage for students, teachers, and researchers to join the discussion. Short chapter descriptions precede each chapter, and an annotated bibliography and suggestions for further reading conclude each controversy. In addition, each volume includes both a general introduction and a supplemental guide to further controversies. Combining timely debates with useful pedagogical aids allows the volumes to serve as clear and detailed snapshots, for all levels of readers, of some of the most exciting work happening in philosophy today. Series Editor John Turri University of Waterloo Volumes in the Series Current Controversies in Philosophy of Perception Edited by Bence Nanay Current Controversies in Philosophy of Film Edited by Katherine Thomson-Jones Current Controversies in Political Philosophy Edited by Thom Brooks Current Controversies in Virtue Theory Edited by Mark Alfano Current Controversies in Epistemology Edited by Ram Neta Current Controversies in Experimental Philosophy Edited by Edouard Machery and Elizabeth O’Neill Current Controversies in Philosophy of Mind Edited by Uriah Kriegel Praise for the Series “This series constitutes a wonderful addition to the literature. The volumes reflect the essentially dialectical nature of philosophy, and are edited by leading figures in the field. They will be an invaluable resource for students and faculty alike.” Duncan Pritchard, University of Edinburgh

Current Controversies in Philosophy of Perception

Edited by Bence Nanay

First published 2017 by Routledge 711 Third Avenue, New York, NY 10017 and by Routledge 2 Park Square, Milton Park, Abingdon, Oxon, OX14 4RN Routledge is an imprint of the Taylor & Francis Group, an informa business © 2017 Taylor & Francis The right of the editor to be identified as the author of the editorial material, and of the authors for their individual chapters, has been asserted in accordance with sections 77 and 78 of the Copyright, Designs and Patents Act 1988. All rights reserved. No part of this book may be reprinted or reproduced or utilised in any form or by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying and recording, or in any information storage or retrieval system, without permission in writing from the publishers. Trademark notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe. Library of Congress Cataloging-in-Publication Data Names: Nanay, Bence, editor. Title: Current controversies in philosophy of perception / edited by Bence Nanay. Description: 1 [edition]. | New York : Routledge-Taylor & Francis, 2016. | Series: Current controversies in philosophy Identifiers: LCCN 2015050847 | ISBN 9781138840072 Subjects: LCSH: Perception (Philosophy) Classification: LCC B828.45 .C87 2016 | DDC 121/.34—dc23 LC record available at http://lccn.loc.gov/2015050847 ISBN: 978-1-138-84007-2 (hbk) ISBN: 978-1-138-84008-9 (pbk) ISBN: 978-1-315-73302-9 (ebk) Typeset in Minion by Apex CoVantage, LLC

Contents

List of Figures

vii

List of Contributors

viii

1 Philosophy of Perception: A Road Map with Lots of Bypass Roads

1

Bence Nanay

Part I Are Perceptual States Representations? 2 Experiences Are Representations: An Empirical Argument

21 23

Adam Pautz

3 Are Perceptual Experiences Just Representations?

43

Heather Logue

Part II Is Perception Thin or Rich?

57

4 Rich or Thin?

59

Susanna Siegel and Alex Byrne

v

vi

•

Contents

Part III Non-Visual Sense Modalities 5 Auditory Perspectives

81 83

John Kulvicki

6 The Non-Visual Senses: The Distinctive Role of Sounds and Smells

95

Matthew Nudds

Part IV The Multimodality of Perception 7 Enhancement Through Coordination

107 109

Casey O’Callaghan

8 Is Perceptual Experience Normally Multimodal?

121

Mohan Matthen

Part V Is Attention Necessary for Perception?

137

9 The Functional Roles of Attention

139

Robert W. Kentridge and Berit Brogaard

10 Attention and Perception: A Necessary Connection?

148

Wayne Wu

Part VI Can Perception Be Unconscious?

163

11 Debate on Unconscious Perception

165

Ian Phillips and Ned Block

Appendix: Other Controversies in Philosophy of Perception

193

Index

194

Figures

2.1 2.2 2.3 2.4 2.5 8.1 8.2 10.1 10.2 11.1 11.2 11.3 11.4 11.5a–c 11.6 11.7

A blue-looking ball Reflectances of grapes, a blue-looking ball, and a leaf. From MacAdam (1985) The first two chemicals smell citrus-like (“citrus 2” and “citrus 1”); the third smells minty. From Margot (2009) From Goldstein (2009) A hypothetical case The modalities as sources for multimodal perceptual processing The colour of the triangle matches the contour. But since colour and contour are separately processed, this needs active processing. Simulation of the response of a neuron in primate visual area MT to stimuli moving in different directions, placed in its receptive field Depiction of a behavior space that delineates four possible visually guided actions SDT analysis of a simple “yes/no” task Trial sequence from Norman et al. (2014: 2824) This diagram is similar to diagrams in Tong et al. (1998) Mirror stereoscope setup for continuous flash suppression From Mudrik et al. (2011) From Jiang et al. (2006) From Jiang et al. (2006)

vii

24 25 26 27 30 127 131 150 153 167 168 170 171 173 177 179

Contributors

Ned Block is Silver Professor at New York University with a primary appointment in the Philosophy Department and secondary appointments in the Psychology Department and Center for Neural Science. Before NYU he taught at MIT for 25 years. He was an undergraduate at MIT and a graduate student at Oxford and Harvard. He is an elected Fellow of the American Academy of Arts and Sciences and the Cognitive Science Society. Berit Oskar Brogaard is a Danish and American philosopher specializing in the areas of cognitive neuroscience, philosophy of mind, and philosophy of language. Her recent work concerns synesthesia, savant syndrome, blindsight, and perceptual reports. She is Professor of Philosophy in the Department of Philosophy and runs a perception lab at the University of Miami in Coral Gables, Florida. She is the President of the Southern Society for Philosophy and Psychology and the first female President of the Central States Philosophical Association. Alex Byrne is Professor of Philosophy at the Massachusetts Institute of Technology and works mostly on philosophy of mind and epistemology. He is co-editor, with Heather Logue, of Disjunctivism (2008) and, with Gideon Rosen, Joshua Cohen, and Seana Shiffrin, of The Norton Introduction to Philosophy (2015). Robert W. Kentridge is Reader in Psychology at the University of Durham in the UK. He has worked in many areas of experimental psychology but specializes in the study of vision, focusing on the perception of material properties such as color, texture, and gloss; on visual attention; and on visual consciousness (and its absence). He uses methods such as human viii

Contributors

•

ix

neuropsychology, neuroimaging, and psychophysics to study these issues. He has a keen interest in the philosophy of perception, both as a means of clarifying our understanding of what it means to perceive and as a source of novel empirical questions. John Kulvicki is Associate Professor of Philosophy at Dartmouth College, Hanover, NH. His work focuses on problems of representation in the arts and in perception. Heather Logue is a lecturer at the University of Leeds. Her research interests lie mostly in philosophy of mind and epistemology, focusing in particular on metaphysical and epistemological questions related to perceptual experience. Mohan Matthen is Professor of Philosophy and Canada Research Chair in the philosophy of perception at the University of Toronto. He works on many aspects of perception, including the perception of objects, of colors, and of speech, and the perceptual representation of space. He edited the Oxford Handbook of the Philosophy of Perception (2015) and is currently working on a book that will treat the fundamentals of human perception, focusing especially on the role of active and multimodal perception in cognitive activities such as those mentioned above. Matthen was elected Fellow of the Royal Society of Canada in 2012. Bence Nanay is Professor of Philosophy and BOF Research Professor at the University of Antwerp, where he is also co-director of the Centre for Philosophical Psychology and Senior Research Associate at Peterhouse, Cambridge University. He is the author of Between Perception and Action (2013) and Aesthetics as Philosophy of Perception (2016) and the editor of Perceiving the World (2010). He has published more than 90 articles on various topics mainly in philosophy of mind, aesthetics, and philosophy of science. Matthew Nudds is Professor of Philosophy at the University of Warwick. He was previously Senior Lecturer in Philosophy at the University of Edinburgh. Casey O’Callaghan is Associate Professor of Philosophy at Washington University in St. Louis. O’Callaghan’s research aims at an empirically informed philosophical understanding of perception that is driven by thinking about non-visual modalities and the relationships among the senses. His publications have focused upon auditory perception, speech perception, and the theoretical import of multimodality, cross-modal perceptual illusions, and synesthesia. O’Callaghan is author of Sounds: A Philosophical Theory (2007) and Beyond Vision: Philosophical Essays (forthcoming). See http:// caseyocallaghan.com. Adam Pautz is a Professor at Brown University. He has worked on consciousness, perception, the sensible properties, and the mind-body problem. He

x

•

Contributors

is currently working on a “consciousness first” approach in the philosophy of mind. Consciousness cannot be reductively explained in other terms (thought, reasons). Rather, it is a starting point from which to explain other things (thought, reasons, value). Ian Phillips is a philosopher of mind and cognitive science, and Associate Professor and Gabriele Taylor Fellow at St. Anne’s College, Oxford University. His recent research on perception, consciousness, and temporal experience is published in Mind & Language, Philosophy & Phenomenological Research, Philosophical Studies, Philosophical Perspectives, and The Philosophical Quarterly. The Philosopher’s Annual selected a paper of his as amongst the ten best of 2013. He is also the recipient of the ASSC William James Prize for Contributions to the Scientific Study of Consciousness, and the Antwerp Centre for Philosophical Psychology Annual Essay Prize for an essay on unconscious perception. Susanna Siegel is Edgar Pierce Professor of Philosophy at Harvard University. She is author of The Contents of Visual Experience (Oxford 2010) and has published numerous papers in the philosophy of perception and epistemology. Her book The Rationality of Perception is forthcoming from Oxford University Press. Wayne Wu is Associate Professor at and Associate Director of the Center for the Neural Basis of Cognition, Carnegie Mellon University. He has published work on attention, agency, visual perception, sensory consciousness, and schizophrenia.

CHAPTER

1

Philosophy of Perception: A Road Map with Lots of Bypass Roads BENCE NANAY

Philosophy of perception is in very good shape: it is an increasingly central discipline within philosophy, and it is at the forefront of combining empirical findings with age-old philosophical questions. But philosophy of perception is also in pretty bad shape: there is plenty of miscommunication, and even philosophers in very similar subfields within philosophy of perception often talk past each other (none of that of course in this volume). So it may be useful to have a roadmap that at least attempts to relate the different approaches to the philosophical study of perception to one another. This road map, just like the volume itself, is of course partial. Others would have picked six different current controversies in philosophy of perception. The six I picked to some degree reflect my own philosophical taste and also my own sociological assessment of what is generally taken to be important in this field in 2016. The reference to bypass roads is more than a pun. I will not give in-depth analyses of the oeuvres of any of the major contributors to contemporary philosophy of perception. Many of these major contributors say what they have to say in the chapters to follow. Rather than honing in on individual philosophers of perception, I want to spot trends, track down shared assumptions, and explore where some new research directions are leading. The result is, no doubt, very broad brushstroke and sometimes oversimplified, but the hope is that I can at least avoid congestions this way.

1

2

•

Bence Nanay

I. Perceptual Processing Versus Perceptual Phenomenology Different philosophers are interested in perception for different reasons. And they often even mean different things by “perception”. Some philosophers are interested in perceptual processing—not just philosophers, also psychologists and cognitive scientists. The general picture is that when you look out of the window and see that it is raining outside, this all starts with light hitting your retina. Light hitting your retina is the sensory stimulation. But light hitting your retina is not perception yet. Nonetheless, it gets the ball rolling. And then lots of complicated things happen in your brain, and at the end of that you may end up forming a belief that it is raining outside or maybe performing a perceptually guided action (which could happen without the involvement of beliefs, in the case of dorsal vision, for example, a topic I will return to in Section II). So perceptual processing takes us from the sensory stimulation to some various ways in which our cognitive system uses perceptual information—to belief formation, to perceptually guided action, and so on. Some other philosophers, not so much psychologists or cognitive scientists, when they say they are interested in perception, what they mean by this is that they are interested in perceptual experience or perceptual phenomenology. They are interested in what it is like to perceive. They are not particularly bothered about what brain mechanisms bring about this experience—they are interested in various features of this experience itself (say, its alleged transparency) or maybe its various epistemological implications. If you are interested in perception as perceptual processing, you are likely to be interested in perception per se, whether or not it is conscious. Consciousness, as well as the distinction between conscious and unconscious perception, is likely to be of secondary importance. In contrast, if you are interested in perception as perceptual phenomenology, then it’s all about conscious perception. Unconscious perception is at best of secondary importance, which may serve as a tool for understanding perceptual phenomenology—or else it is dismissed altogether as a degenerate form of perception (or maybe even demoted to “mere” information processing). If you are interested in perception as perceptual processing, you have a lot to learn from the empirical sciences: they can really do a lot to explain what happens with the sensory stimulation during various phases of perceptual processing and how it may interact with higher-level mental states like beliefs. (This doesn’t mean that the perception as perceptual processing approach has to give a neuroscientific characterization of perceptual processing—in fact most often this characterization is functional, not anatomical.) If you are interested in perception as perceptual phenomenology, the empirical sciences will be less directly relevant. From the point of view of “what it is like to perceive”, it may be helpful to understand what happens in the primary visual cortex, but this is

Philosophy of Perception

•

3

at best considered as a data point that bears a fairly indirect relation to the real explanandum: perceptual phenomenology. In an ideal world, of course, these two approaches would go hand in hand. The dream is to understand what kind of perceptual processing leads to what kind of perceptual experience. And very many empirical findings very actively use the subjective reports of subjects as data points (not as conclusive evidence, but as data points). Many philosophers of perception (and almost all contributors to this volume) are explicitly (or less explicitly) trying to bring together these two ways of approaching perception. But the two approaches can and do often come apart, and this can lead to serious misunderstandings that divide the field of philosophy of perception considerably. It is easy to caricature both of these approaches. One can dismiss the perception as perceptual processing approach as not philosophy. If you are so interested in the interaction between the primary visual cortex and V4, go and change majors and do cognitive neuroscience, not philosophy. Philosophy is about the grand eternal questions, and the minor details of contingent facts about perceptual processing are neither grand nor eternal. And one can also dismiss the perception as perceptual phenomenology approach as mere intuition mongering. If we are looking at the same scene and you report perceptual phenomenology of a certain kind and I report another kind, how can we decide who is right and who is wrong? The methodology of addressing such questions would need to rely on introspection, and introspection is fabulously unreliable. Again, the aim is to resist the temptation to give a caricature version of either approach and to take both of them seriously. I suspect that all philosophers of perception find one of the two approaches easier to relate to than the other one. I certainly do. But the goal should be to take whatever route we can to understand the complex phenomenon that is perception. And making a distinction between the two very different explananda (perceptual phenomenology and perceptual processing) could be the first step toward clearing up some confusions and misunderstandings. Which big picture view one has about perception, processing, or phenomenology has an impact on some of the major controversies in philosophy of perception. The clearest case of this is probably the most influential contemporary debate in philosophy of perception, the debate about whether perceptual states are representations (see the papers in Brogaard 2014).

II. Perceptual Representation Versus Perceptual Relation Representationalists say that perceptual states are representations: they represent individuals as having properties (see Peacocke 1989, Burge 2005, Crane 2006, Tye 2007, Pautz 2010, Schellenberg 2010, Siegel 2010a, 2010b for very different versions of representationalism). When I look out of the window,

4

•

Bence Nanay

I see dark clouds. I perceptually represent the clouds as having the property of being dark. Things may go wrong, of course; I may have an eye condition that makes me see dark clouds, whereas the clouds are in fact very light. In this case, my perceptual state misrepresents. If I see dark clouds and the clouds are in fact dark, my perceptual state represents correctly. Just what kind of representations these perceptual states are is something I will return to in Sections III and IV. Not all philosophers of perception are representationalists. Some are relationalists (or “naïve realists”): they claim that perceptual states are not representations (or, sometimes more modestly, not primarily representations or not essentially representations; see Campbell 2002, Martin 2004, 2006, Travis 2004, Brewer 2006, 2011 for very different versions of relationalism). Perceptual states do not represent the perceived object. Rather, they have the perceived object as one of their actual constitutive parts. Or, to put it differently, relationalists claim that perceptual states are relations between the subject and the perceived object (and maybe some third relatum labeled as “the third relatum” [Campbell 2002, Brewer 2011]). So the perceived object is not something that may or may not be present when you perceive (as some representationalists would say). It has to be there for your perceptual state to be a perceptual state. One implication of this view is that hallucinations are, at least on one straightforward way of understanding hallucinations (see Byrne and Logue 2008 for a nuanced analysis), not perceptual states: their object is missing—so they cannot be a constitutive part of the perceptual state. Many relationalists are happy to bite this bullet: hallucinations may feel like perceptual states, but they are not—they are in fact radically different. Perceptual states are relations to something actual, whereas hallucinations are something different—whatever hallucinations are, they are by definition not relations to something actual. Relationalism is very much formulated within the framework of the perception as perceptual phenomenology approach. Many of the motivations for this view allude explicitly to phenomenology, for example (see, e.g., Martin 2002, 2004, 2006, Brewer 2011). And this is something most proponents of relationalism would be very happy to acknowledge. So all the claims about perceptual states I attributed to the relationalist are really claims about conscious perceptual experiences. But then what can relationalists say about unconscious perception? Relationalists have, on the face of it, three options. First, they can deny that there is such a thing as unconscious perception or at least question whether we have sufficient evidence to posit unconscious perception. This is what Ian Phillips does in this volume (although he has never endorsed relationalism in print). The strategy would be to show that all alleged examples of unconscious perception are either not perception or not unconscious. And then the debate between relationalists and representationalists would be fought about

Philosophy of Perception

•

5

the details of the perceptual processing (and of some experimental findings about them)—as the contribution by Ian Phillips and Ned Block in this volume shows nicely. I am very sympathetic to this way of steering the representationalism versus relationalism debate toward a closer engagement with details of perceptual processing. The second option for the relationalist would be to allow for unconscious perception but deny that the relationalist analysis applies to those. Perceptual experiences are relations to token objects, but unconscious perceptual states are not. Some of them would even be happy to allow for perceptual representations when it comes to unconscious perceptual representations. But this strategy makes one wonder how much conscious and unconscious perception have in common (presumably not a lot; see McDowell 1994, who is by no means a textbook relationalist). Finally, the third option for relationalists would be to allow for unconscious perception and extend the relationalist analysis to unconscious perceptual states. But I am not sure how this would work: if we allow for unconscious perceptual states, we face some disturbing empirical findings about them— namely, that often we have a conscious and an unconscious perceptual state at the same time. And they can be very different. In some cases of perceptually guided actions (often when there is some optical illusion involved), some properties—say, size properties or spatial location properties—of the perceived object show up in our perceptual experience. But they are very different from the size or spatial direction properties that guide our fine-grained actions (as evidenced by, for example, the grip size with which we approach the object or the direction in which we reach) and of the two different sizeproperties; it is the experienced ones that are further from being veridical. The unconscious perceptual process tracks these properties more closely than the conscious perceptual experience (Goodale and Humphrey 1998, Goodale and Milner 2004, Króliczak et al. 2006). The representationalist has no problem accounting for such findings: the perceptual experience represents the object as having such and such size or spatial location properties, whereas the unconscious action-guiding perceptual state (in the dorsal visual subsystem) represents the object as having different size or spatial location properties. But it is difficult to see what the relationalists would say here, if they want to maintain that both perceptual experiences and unconscious mental states are relations to token objects. The token object we are looking at is the same for both states, and the size and spatial location properties of them are the same as well. How can we have these two very different relations to the very same token object (and the very same properties of this object) then? There is some logical space for maneuvering here (maybe by bringing in the third relatum), but not a lot. And, maybe as a result, few, if any, relationalists go down this path. Most relationalists opt to restrict the relationalist analysis to conscious perceptual experiences. Unconscious perception

6

•

Bence Nanay

is dismissed either as not really perception or as something radically different from perceptual experience. This gives us a way of relating the two distinctions I talked about so far to each other: the distinction between taking perception to be perceptual processing and taking it to be perceptual phenomenology and the distinction between representationalism and relationalism. Relationalism, even if it doesn’t presuppose the perception as perceptual phenomenology attitude, at least makes that way of thinking about perception a natural one. Relationalists tend to be interested in perceptual phenomenology, not in perceptual processing. How about representationalists? Here we get more of a split. Many representationalists are very much interested in perceptual phenomenology, but representationalism is also a natural ally of the perception as perceptual processing approach as an important way of characterizing various stages of perceptual processing is in terms of the way they represent the perceived scene. One thing (most) representationalists seem to agree about is that as long as we allow that perceptual representations may be conscious or unconscious, whatever is special about perceptual representations is the common denominator between conscious and unconscious perception.1 Whether perception is conscious or unconscious leaves basic questions about the nature of perceptual representations untouched. Do we then get some alignment between representationalism and the perception as perceptual processing approach on the one hand and relationalism and the perception as perceptual phenomenology approach? I really don’t think so. Some relationalists have a lot to say about perceptual processing (see Campbell 2002, for example). And some representationalists explicitly talk about perceptual experiences—not perceptual states in general (including Adam Pautz’s representationalist contribution to this volume). But the representations they talk about are not specific to perceptual experiences. So we do not get two package deals—a relationalist perception as perceptual phenomenology approach and a representationalist perception as perceptual processing approach. But depending on which general explananda we consider, phenomenology or processing, the representationalism versus relationalism debate may play out very differently. As we have seen, the main explanandum of relationalism is perceptual phenomenology—some relationalists do talk about perceptual processing, but they tend to do so in order to elucidate perceptual phenomenology. And those representationalists who are interested in perceptual processing, maybe following the practice of cognitive science, use the concept of representation to make sense of perceptual phenomenology. For at least these representationalists, consciousness is not something that is the primary topic of philosophy of perception. For most relationalists, in contrast, philosophy of perception is all about consciousness.

Philosophy of Perception

•

7

The 1980s recipe for naturalizing the mind was this: first we should understand how the mind represents the world, and then, once we are done with that, we can start explaining how some of these representations come to be conscious (Millikan 1984, Dretske 1988, Papineau 1993). This naturalistic approach chimes with the perception as perceptual processing approach and also with representationalism. And the 1990s obsession with consciousness chimes with the perception as perceptual phenomenology approach, although it should be acknowledged that it comes in both relationalist and representationalist flavors. But I’m writing this in 2016—by now, both the 1980s and the 1990s count as retro. The hope is that we can move past this dichotomy—and that we can also move past the dichotomy of perception as perceptual processing versus perception as perceptual phenomenology—and also past the dichotomy of representationalism versus relationalism (see Nanay 2015).

III. Perceptual Content I find it difficult to imagine how such compromise could be achieved without talking about perceptual content at least in some minimal sense. But then again, I am a representationalist. But even some (many?) relationalists would be happy to go along with the concept of perceptual content. I will now take advantage of the fact that Heather Logue’s relationalist contribution to this volume explicitly does so (not all on the relationalist side are so permissive). In any case, many of the current controversies in philosophy of perception are about perceptual content. Some of these may be rephrased in fully representationfree language, but I will not attempt to do so here. What I take to be the most rudimentary way of thinking about perceptual content is this: perceptual states represent individuals as having properties. Or, to put it differently, perceptual states attribute properties to individuals. Or, to put it in a way that maybe even relationalists would accept, perceived properties are bound to perceived individuals perceptually. Three questions arise immediately (and many more, further down the line). What are the properties that are attributed in perception? What are the individuals these properties are attributed to? And what is the format of these kinds of representations? I will address these questions in reverse order.

IV. The Structure of Perceptual Content The question is: how do perceptual states represent the world? The analogy often employed in this context is with the way beliefs represent the world. So it would be tempting to describe the way perceptual states represent the world on the analogy of the way beliefs do so.

8

•

Bence Nanay

In the wake of the “linguistic turn” of analytic philosophy, it became fashionable to talk about (some) mental states as propositional attitudes. Propositional attitudes are attitudes toward a proposition. Beliefs are paradigmatic examples: when we have a belief that it is raining outside, we have a belief attitude toward the proposition that it is raining outside. The content of this mental state, in other words, is a proposition: that it is raining outside. When it comes to perceptual content, it is tempting to reach for the same conceptual machinery of propositional attitudes. Beliefs are propositional attitudes, as are, the argument goes, hopes and desires. When we hope that it will be raining outside, we have a different kind of attitude—the hope attitude—toward the same proposition. Our hope has the same propositional content: that it is raining outside. And from here it is only a short step to extend this way of thinking about mental states to perceptual states. So the suggestion would be that when I see that it is raining outside, I have yet another kind of attitude, a seeing kind of attitude, toward the very same proposition: that it is raining outside. This would make perceptual content propositional and behave in general in a very similar way to the content of beliefs (Matthen 2005, Chalmers 2006). Proponents of this approach could and often would allow for some differences between perceptual content and the content of beliefs—perceptual content, for example, is said to have (necessarily) a demonstrative element embedded in the propositional content. But the way perceptual content is structured according to this view is exactly the way the content of beliefs is structured: propositionally. Those who want to resist this view would need to give a positive account of how perceptual content is structured, if not propositionally (a task made even more complicated by the various senses in which “propositional” is used; see, e.g., Stalnaker 1976). And there are many ways of doing so: we can say that perceptual content is analog, whereas the content of beliefs is digital (Dretske 1981; but see also Dretske 1988), or that what is distinctive about perceptual content is that it represents in a unit-free way (Peacocke 1986, p. 2, Peacocke 1989, p. 300). The general strategy for those who insist that perceptual content is nonpropositional is to argue that perceptual content is structured not the way sentences are but the way images are (Crane 2009, Kulvicki 2015). It is not easy to nail down what this means, but here is a standard way of doing so. Perceptual states attribute some set of properties to all points of the physical space in front of us (see Peacocke 1992, 2008 for one way of working out this idea). So, a photograph on your computer assigns a color to every pixel of the picture and perception assigns a shape, color, and spatial location property to every point in the physical space in front of us, or, rather, to the closest non-transparent surface in every direction.

Philosophy of Perception

•

9

But regardless of how exactly we cash out the structure of perceptual representations, we need to ask what kinds of properties are represented perceptually. I have been talking about color, shape, and spatial location—are these all?

V. Perceived Properties One difference between beliefs and perceptual states is that while beliefs can represent pretty much any property, perceptual states can only represent a restricted set of properties. The big question is just how restricted this set is. I can have a belief that my laptop was produced by exploited workers for large profit, but, when I’m looking at my computer, this is unlikely to be a property of the laptop that I attribute perceptually. So this is a property that is attributed to the laptop non-perceptually. The laptop’s shape property, in contrast, is, presumably, attributed perceptually. But then where to draw the line? This is the topic of the debate between Susanna Siegel and Alex Byrne in this volume. Much of the controversy in this area has centered on the following sorts of properties. It has been argued that we perceive objects as trees and tables (Siegel 2006); as being causally efficacious (Siegel 2005, 2009); as edible, climbable, or Q-able in general (Nanay 2011a, 2012a); as agents (Scholl and Tremoulet 2000); as having some kind of normative character or value (Kelly 2010, Matthen 2010); as having dispositional properties (Nanay 2011b); and as having moral value (Kriegel 2007). In light of the previous discussion, we can identify an ambiguity in this debate. Depending on whether we are interested in perceptual processing or perceptual phenomenology, we get two very different disputes. We can ask what properties show up in our perceptual phenomenology—what properties are represented, consciously, in perceptual experiences. Or we can ask what properties the (not necessarily conscious) representations in the perceptual system represent. And the answer is likely to be very different (see Nanay 2012a, 2012b on this disambiguation). Not all properties that (unconscious) representations in the perceptual system attribute will show up in our perceptual phenomenology. When it comes to the debate about perceptual phenomenology, most arguments for and against the claims that such and such a property is part of our perceptual phenomenology are based on what has been called the methodology of contrast cases (see Siewert 2002, Kriegel 2007, Siegel 2007, Bayne 2009, Masrour 2011). It works like this: if we find two token experiences that only differ in that property P shows up in one but not the other, and if the two experiences differ in their perceptual phenomenology, then property P is part of our perceptual phenomenology. The problem with the contrast case methodology is that it is difficult to settle disagreements about phenomenology. If I say that two experiences differ in

10

•

Bence Nanay

their perceptual phenomenology and you deny this, it is not clear how the issue can be decided. Intuitions wildly differ with regard to which aspects of phenomenal character count as perceptual. Suppose that you learn that the meat you’re eating is rat meat. Your overall phenomenology, supposedly, changes. But does your perceptual phenomenology change? If someone were to claim that it is their non-perceptual phenomenology that changes, it is difficult to see how we could settle this disagreement. I do not mean to suggest that the question about what properties show up in our perceptual phenomenology is intractable—there may be ways of strengthening the methodology of contrast cases, and there may also be other ways (besides introspection) of finding out about perceptual phenomenology (see Bayne 2009, Nanay 2012b, Block 2014). But when it comes to the debate about what kind of properties representations in the perceptual system represent objects as having, the methodology is more straightforward. If we can’t explain, say, the fine-grained motor output caused directly by a perceptual state without positing a perceptual representation that represents a certain kind of property (say, the property of the location in one’s egocentric space), then we have good reason to conclude that this kind of property is perceived (in the sense of represented in perceptual processing [not necessarily consciously]). I give some examples of how this would work in Nanay (2013).

VI. Attention and Perception I have been focusing on just one issue concerning perceived properties-— namely, the range of these properties: are they shape properties, color properties, sortal properties? But there are other questions we can ask about perceptually attributed properties. First, suppose we identified the kinds of properties we perceptually represent objects as having. What are these properties exactly? What is their ontology? For example, are they tropes or universals? Two different token entities can have the very same property if we think of properties as universals. But they are logically incapable of having the same property if we think of properties as tropes (see Campbell 1990). Without getting lost in the metaphysical intricacies of this distinction, it can be pointed out that representing an object as having a universal is a very different mental state from representing an object as having a trope (Mulligan et al. 1984). The question is: do perceptual states represent tropes or universals? If we think of properties as tropes, then this nudges an account of perceptual content toward some sort of compromise between representationalism and relationalism—as seeing two different but indistinguishable objects would amount to being in two very different perceptual states that attribute very different tropes to these objects (Nanay 2012c).

Philosophy of Perception

•

11

Second, properties can be of varying degrees of determinacy. Take color properties. Being red is more determinate than being colored, but it is less determinate than being scarlet. This gives us a nice ordering of propertytypes in terms of their determinacy. The question then is: do perceptual states represent more determinate or less determinate (that is, more determinable) properties? It may be tempting to say that perceptual states represent more determinate properties (or maybe even superdeterminate properties, which means determinates without further determinates; see Funkhouser 2006) and maybe contrast perceptual states with beliefs that can only represent less determinate properties. A picture is worth a thousand words, as they say: when I see an apple, I see the exact (superdeterminate) shade of red that it has. But when I form a belief about an apple (maybe because you told me it was red), I can only represent it in a much less determinate way (as red, and not as having the exact shade of color it has)—so the argument would go. We should resist arguments of this kind (see also Dennett 1996) as we know that many of the properties we perceive (for example, in the periphery of our visual field) are not at all determinate. In other words, perceptual states attribute properties of varying degree of determinacy. And given that perceptual content is the sum total of perceptually attributed properties (structured in some way), the degree of determinacy of perceptually attributed properties is part of what determines perceptual content. But then whatever the degree of determinacy of the perceptually attributed properties depends on is also part of what determines perceptual content. And one plausible candidate for what the degree of determinacy of the perceptually attributed properties depends on is attention. The general idea is that attending to a property makes, or tries to make, this property more determinate (Nanay 2010, Stazicker 2011; see also Yeshurun and Carrasco 1998 for empirical support). But if this is so, then we can never specify the content of a perceptual state without specifying perceptual attention. This leads to a very specific view about the relation between attention and perception. Attention is an absolutely necessary feature of perception: perception is the perceptual attribution of properties of varying determinacy, and it is (partly) attention that fixes how determinate these attributed properties are (which ones are more determinate, which ones are less determinate). In other words, attention is part of perception (see Robert W. Kentridge and Berit Brogaard’s paper in this volume, but see also Wayne Wu’s contribution for discussion). The alternative of this view is that attention is something post-perceptual. You see something, and then you can attend to various features of what you see. But the way you attend does not alter your perception itself. Clearly which view one accepts depends not just on how one thinks about perception but also on how one thinks about attention. If attention is taken to be the act of attending, then the post-perceptual view seems more appealing.

12

•

Bence Nanay

And while some acknowledge that the act of attending can be unconscious and that it does not have to be done deliberately (attention can be captured in a non-deliberate manner, but it is still, according to this approach, normally conscious), this way of thinking about attention has some important commonalities with the perception as perceptual phenomenology approach. What evidence can we have for the deployment of attention? Mainly introspective (again, maybe allowing for fringe cases like unconscious attention). But we can also think of attention as an important aspect of perceptual processing that partly determines perceptual content. As perceptual content does not have to be conscious, there is no reason to hold that attention would need to be conscious, or even that it would be typically conscious. And if we think of attention this way, then the way to find out more about it is to look at empirical findings, both neuro-imaging and behavioral results, which could help us find out how attention influences perceptual processing (and it seems that there is very strong evidence that attention modulates already very early perceptual processing with its influence getting more diverse as perceptual processing unfolds [Gandhi et al. 1999, O’Connor et al. 2002]). If there is a certain amount of miscommunication in philosophy of perception in taking perception either to be perceptual processing or to be perceptual phenomenology, there is even more miscommunication along these lines in the philosophy of attention. And here, just as in philosophy of perception, the prospects for some kind of reconciliation are not completely hopeless. If you think of attention as part of perceptual processing and as something that partly determines perceptual content, you have a straightforward way of explaining something important about perceptual phenomenology—namely, that attention has a major influence on perceptual phenomenology (as the inattentional blindness experiments show nicely; see Mack and Rock 1999 for a summary). So the picture would be one where changes in perceptual phenomenology are explained with reference to low-level and often very early perceptual processes and to the role attention plays in these perceptual processes. I am not sure how any reconciliation between the two very general approaches could be achieved if we start with a concept of attention that is understood as the act of attending and that is tied, even if it is tied loosely, to consciousness. But then again, those who think of attention this way may not be interested in perception (or attention) as a perceptual process (or a constituent thereof).

VII. Sensory Individuals I talked a lot about which properties are attributed perceptually. But what are these properties, whatever they may be, perceptually attributed to? What are the entities we perceptually represent as having these properties? The first thing to note is that they may not be entities per se, but rather events: much of what we perceive are events, not unchanging entities. I will use the general term

Philosophy of Perception

•

13

“sensory individual” to refer to the individual (event or entity) that perceptual states attribute properties to perceptually (see Cohen 2004). We have seen that there is a difference between the range of properties perceptually represented and the range of properties represented (perceptually or not perceptually). The property of being made by exploited workers for large profit can be represented (and attributed to my laptop), but it is unlikely to be perceptually represented (that is, perceptually attributed to my laptop). And similarly, we need to make the same distinction between the individuals we attribute properties to perceptually and the entities we attribute properties to (perceptually or not perceptually). The two may not be the same. And I call the former, the individuals we attribute properties to perceptually, “sensory individuals”. Here, again, we get some discrepancies about what methodology we should use when we are trying to figure out what the sensory individuals of perception are. Philosophers often use introspection (and sometimes even linguistic considerations about how we talk about what we see or what we hear) to figure out what sensory individuals are. But this way of going about the problem arguably presupposes the perception as perceptual phenomenology approach. Further, this is a somewhat dubious strategy, not only if we allow for unconscious perception, where we are extremely unlikely to stumble upon the correct sensory individual either by introspecting or by ordinary language analysis. It is a dubious strategy even for conscious perception: it is, as we have seen, very difficult to tell perceptual and non-perceptual phenomenology apart on the basis of introspection only. Introspection delivers some kind of individuals that we take to be attributing properties to perceptually. But of course the question of sensory individuals is not a question of what we take ourselves to perceive. The alternative, again, would be to see how the perceptual system binds properties to individuals and zero in on the individuals the perceptual system binds properties to. If we have reason to postulate an (unconscious) perceptual representation, presumably, to explain the fine-grained behavior of agents, and if this perceptual representation attributes properties to a certain kind of individual, we have good reason to conclude that this individual is the (or, rather, a) sensory individual of perception. But I will not pretend that this approach solves all the problems about sensory individuals, as we shall see in the next section.

VIII. Differences Between Sense Modalities To make the question of sensory individuals even more complicated, it seems that sensory individuals may vary across sense modalities. The main contenders for sensory individuals in vision are ordinary objects and spatiotemporal

14

•

Bence Nanay

regions (Clark 2000, 2004, Cohen 2004, Matthen 2004). The main contenders for sensory individuals in audition are sounds and ordinary objects (Casati and Dokic 1994, Pasnau 1999, Nudds 2001, 2010, O’Callaghan 2007, Kulvicki 2008, Nudds and O’Callaghan 2009, Matthen 2010). And for olfaction, they are odors and ordinary objects (Lycan 2000, Batty 2010, 2011). Do we hear the sound of the coffee machine or the coffee machine itself? Or do we hear one by hearing the other? These are as current and as controversial questions as current controversies in the philosophy of perception get. But they also point in a more general direction where there has been a lot of new work in philosophy of perception: on the similarities and especially differences between different sense modalities. Most philosophers of perception write about vision. And even when they write about perception in general, they tend to use visual examples. I say “they”, but really I should write “we”—almost all the examples in this piece so far were visual examples. And a new and very commendable trend in philosophy of perception is to examine how much we can generalize from the visual sense modality to non-visual sense modalities (see Clark 2011 for a summary). The question of sensory individuals is one of the fields where any such generalization seems like a very bad idea. Even if we manage to settle whether the sensory individuals of vision are ordinary objects or spatiotemporal regions, this says nothing about the possibility of sounds as auditory sensory individuals or the possibility of odors as olfactory sensory individuals. But there are other important questions where acknowledging and examining the differences between different sense modalities can help us to make progress. One of these is the perception of space. Vision seems to represent space, but it is very controversial whether audition or olfaction does so. Olfaction doesn’t even seem to localize its sensory individual, whatever that may be, in space (see the discussion in Matthew Nudds’ contribution to this volume). And we get similar dissimilarities when it comes to the perception of time or of durations—again, audition just seems very different in this respect from vision. It is important to emphasize that this new research direction of taking the non-visual sense modalities seriously does not accept it as a dogma that different sense modalities are just different. In fact, it has been argued that we can restore at least some degree of similarity between, say, vision and audition, as long as we find the right way to think about sounds (see, for example, John Kulvicki’s contribution to this volume). But any such argument about the similarity between some features of different sense modalities would need to be based on a thorough analysis of the two different sense modalities. As a result, philosophy of perception has become more fragmented, but in a good way. Rather than looking for some very general features of perception per se, more and more philosophers of perception are working on one specific sense modality and the odd quirks this sense modality may have that

Philosophy of Perception

•

15

other sense modalities may not have. And I also have the probably quite naïve hope that this narrowing down of the explanandum may help the reconciliation between the perception as perceptual phenomenology and the perception as perceptual processing approach (partly because if one works on, say, the specificities of olfaction, it is difficult to ignore the [empirical] details of how olfactory information gets processed and rely on introspection only).

IX. The Multimodality of Perception All of the preceding discussion of the differences and similarities between different sense modalities seems to presuppose that perceptual processing in one sense modality is independent from perceptual processing in another sense modality. But this is blatantly not so. One of the major boom areas of philosophy of perception is the study of the multimodal nature of perception—the study of the intricate connections between different sense modalities, which starts at very early stages of perceptual processing (Shams et al. 2000, Vroomen et al. 2001, Spence and Driver 2004, Watkins et al. 2006, O’Callaghan 2008). As revealed in the language I used in the previous paragraph, it is easier to talk about the multimodality of perception within the perception as perceptual processing approach. In fact, it is not clear how the perception as perceptual phenomenology can fully engage with the findings on multimodality without engaging with the perception as perceptual processing approach. What happens in, say, the auditory sense modality is influenced by the, say, visual sense modality (a striking example is ventriloquism, where our auditory experience, where we hear the voice coming from, is influenced by the visual information about the ventriloquist’s and the dummy’s mouth; see Bertelson 1999). And this goes for perceptual experience, not just perceptual processing. So the auditory experience changes, but it changes as a result of some important changes in the visual perceptual processing—the very idea of cross-modal effects presupposes the talk of perceptual processes (as regardless of whether these effects influence processing or experience, what causes these effects is not an experience, but perceptual processing in another sense modality).2 Leaving the perception as perceptual processing versus perception as perceptual phenomenology big-picture discussion aside, one important question about the multimodality of perception is just how radically these findings could or should transform philosophy of perception as we know it. One old question about philosophy of perception is about the individuation of the senses—what makes seeing different from hearing, for example. Do the new findings about the intricate connections between sense modalities make this debate obsolete? Does it make sense to bother with what makes different sense modalities different in a multimodal philosophy of perception? Further, to return to the issue of sensory individuals once more, when I presented the debates about what sensory individuals are in different sense

16

•

Bence Nanay

modalities, I was implicitly assuming, with the rest of that literature, that vision has its sensory individuals, which may or may not be different from the sensory individuals of, say, audition. But given the multimodality of perception, one may wonder how these sensory individuals are related to one another. Most of what we perceive we perceive with more than one sense modality. I call individuals that can be perceived with more than one sense modality, with great terminological innovation, multisensory individuals. Note the difference between the term “multisensory” and “multimodal”, as I use them. Multisensory merely means something we can perceive by two or more sense modalities. Multimodality has to do with the interaction of these senses—something not at all presupposed by the concept of multisensory. So the question is this: what happens when we perceive multisensory individuals? This is not an arcane question. Most of what we perceive are multisensory individuals. In fact, it is quite difficult to think of sensory individuals that are not multisensory individuals in this sense. People are multisensory individuals—we can see them, hear them, smell them, touch them, maybe taste them too. And the same goes for most of the objects and events around us. But there may be some sensory individuals that are not multisensory individuals. Rainbows, for example, arguably, are not multisensory individuals—we can perceive them visually only. But then it is tempting to think of multisensory individuals as multimodal sensory individuals. When we perceive multisensory individuals (like people), we encounter not two different sensory individuals in two different sense modalities that happen to coincide. Rather, we encounter one mereologically complex multimodal sensory individual. We attribute visual properties to the visual part of this mereologically complex multimodal sensory individual, and we attribute auditory properties to its auditory part (see Casey O’Callaghan’s and Mohan Matthen’s contributions to this volume). Do we need to represent both parts of this complex multimodal sensory individual simultaneously in audition and in vision? Not necessarily—maybe our attention alternates between the auditory and the visual parts (Spence and Bayne 2014). It is also unclear just how mereologically complex this multimodal sensory individual is—in the case of perceiving people, the multimodal sensory individual has at least visual and auditory parts, but maybe even an olfactory one? And what makes some sensory individuals, like rainbows, unimodal? Does it have to do with our previous encounters (uni- or multimodal) with individuals of that kind? The answers to these questions are not clear. But taking the multimodality of perception seriously changes the questions we ask about perception. A final editorial remark—the book is structured the way other volumes in the Current Controversies series are structured: one topic, two papers, representing different sides of that controversy. I changed this format in the case of two controversies (Part 2 and Part 6), where I used a format much closer to an

Philosophy of Perception

•

17

actual debate: author A writes 1500 words, then author B writes 1500 words in response, then A again with 1000 words, and B again with 1000 words, and then closing remarks by A and closing remarks by B. I think this format works very well in philosophy, and I wish I had figured this out at an earlier stage of editing this book—I hope other philosophers will use this format more in the future.

Notes 1. Those who take perceptual states to have “phenomenal intentionality” would disagree here. 2. The multimodality of perception, more specifically, the phenomenon of crossmodal binding (Vroomen et al. 2001, Bertelson and de Gelder 2004), spells even more specific trouble for the natural ally of the perception as perceptual phenomenology approach—namely, relationalism (see Nanay 2014).

References Batty, C. 2010 Scents and sensibilia. American Philosophical Quarterly 47: 103–118. Batty, C. 2011 Smelling lessons. Philosophical Studies 153: 161–174. Bayne, T. 2009 Perception and the reach of phenomenal content. Philosophical Quarterly 59: 385–404. Bertelson, P. 1999 Ventriloquism: A case of cross-modal perceptual grouping. In: Aschersleben, G., Bachmann, T., and Müsseler, J. (eds): Cognitive Contributions to the Perception of Spatial and Temporal Events, Amsterdam: Elsevier, pp. 347–362. Bertelson, P. and de Gelder, B. 2004 The psychology of multimodal perception. In: Spence, C. and Driver, J. (eds): Crossmodal Space and Crossmodal Attention, Oxford: Oxford University Press, pp. 141–177. Block, N. 2014 Seeing-as in the light of vision science. Philosophy and Phenomenological Research 89: 560–572. Brewer, Bill 2006 Perception and content. European Journal of Philosophy 14: 165–181. Brewer, Bill 2011 Perception and Its Objects, Oxford: Oxford University Press. Brogaard, B. 2014 (ed): Does Perception Have Content? New York: Oxford University Press. Burge, Tyler 2005 Disjunctivism and perceptual psychology. Philosophical Topics 33: 1–78. Byrne, Alex and Logue, Heather 2008 Either/or. In: Haddock, Adrian and MacPherson, Fiona (eds): Disjunctivism: Perception, Action, Knowledge, Oxford: Oxford University Press, pp. 57–94. Campbell, John 2002 Reference and Consciousness, Oxford: Oxford University Press. Campbell, Keith 1990 Abstract Particulars, Oxford: Blackwell. Casati, R. and Dokic, J. 1994 La Philosopie du Son, Paris: Chambon. Chalmers, D.J. 2006 Perception and the fall from Eden. In: Gendler, T. and Hawthorne, J. (eds): Perceptual Experience, New York: Oxford University Press, pp. 49–125. Clark, Austen 2000 A Theory of Sentience, Oxford: Clarendon Press. Clark, Austen 2004 Sensing, objects, and awareness: Reply to commentators. Philosophical Psychology 17: 563–589. Clark, Austen 2011 Vicissitudes of non-visual objects. Philosophical Studies 153: 175–181. Cohen, Jonathan 2004 Objects, places, and perception. Philosophical Psychology 17: 471–495. Crane, Tim 2006 Is there a perceptual relation? In: Gendler, T. and Hawthorne, J. (eds): Perceptual Experience, Oxford: Oxford University Press, pp. 126–146. Crane, Tim 2009 Is perception a propositional attitude? Philosophical Quarterly 59: 452–469. Dennett, D.C. 1996 Seeing is believing: Or is it? In: Akins, K. (ed): Perception, Oxford: Oxford University Press, pp. 158–172. Dretske, Fred 1981 Knowledge and the Flow of Information. Cambridge, MA: MIT Press.

18

•

Bence Nanay

Dretske, Fred 1988 Explaining Behavior. Cambridge, MA: MIT Press. Funkhouser, E. 2006 The determinable-determinate relation. Nous 40: 548–569. Gandhi, Sunil P., Heeger, David J., and Boynton, Geoffrey M. 1999 Spatial attention affects brain activity in human primary visual cortex. Proceedings of the National Academy of Sciences 96: 3314–3319. Goodale, Melvyn A. and Humphrey, G. Keith 1998 The objects of action and perception. Cognition 67: 181–207. Goodale, M.A. and Milner, A.D. 2004 Sights Unseen, Oxford: Oxford University Press. Kelly, Sean 2010 The normative nature of perceptual experience. In: Nanay, Bence (ed): Perceiving the World, New York: Oxford University Press, pp. 146–159. Kriegel, U. (2007) The phenomenologically manifest. Phenomenology and the Cognitive Sciences 6: 115–136. Króliczak, Grzegorz, Heard, Priscilla, Goodale, Melvyn A., and Gregory, Richard L. 2006 Dissociation of perception and action unmasked by the hollow-face illusion. Brain Research 1080: 9–16. Kulvicki, J. 2008 The nature of noise. Philosophers’ Imprint 8: 1–16. Kulvicki, J. 2015 Analog representation and the parts principle. Review of Philosophy and Psychology 6: 165–180. Lycan, W. 2000 The slighting of smell. In: Bhushan, N. and Rosenfeld, S. (eds): Of Minds and Molecules: New Philosophical Perspectives on Chemistry, Oxford: Oxford University Press, pp. 273–289. Mack, A., and Rock, I. 1998 Inattentional Blindness. Cambridge, MA: MIT Press. Martin, M.G.F. 2002 Particular thoughts and singular thought. Royal Institute of Philosophy Supplement 51: 173–214. Martin, M.G.F. 2004 The limits of self-awareness. Philosophical Studies 120: 37–89. Martin, M.G.F. 2006 On being alienated. In: Gendler, T. and Hawthorne, J. (eds): Perceptual Experience, Oxford: Oxford University Press, pp. 354–410. Masrour, F. 2011 Is perceptual phenomenology thin? Philosophy and Phenomenological Research 83: 366–397. Matthen, Mohan 2004 Features, places and things. Philosophical Psychology 17: 497–519. Matthen, Mohan 2005 Seeing, Doing, and Knowing: A Philosophical Theory of Sense-Perception, Oxford: Clarendon Press. Matthen, Mohan 2010 On the diversity of auditory objects. Review of Philosophy and Psychology 1: 63–89. McDowell, J. 1994 The content of perceptual experience. Philosophical Quarterly 44: 190–205. Millikan, Ruth G. 1984 Language, Thought and Other Biological Categories. Cambridge, MA: MIT Press. Mulligan, Kevin, Smith, Barry, and Simons, Peter 1984 Truth makers. Philosophy and Phenomenological Research 44: 278–321. Nanay, Bence 2010 Attention and perceptual content. Analysis 70: 263–270. Nanay, Bence 2011a Do we see apples as edible? Pacific Philosophical Quarterly 92: 305–322. Nanay, Bence 2011b Do we sense modalities with our sense modalities? Ratio 24: 299–310. Nanay, Bence 2012a Action-oriented perception. European Journal of Philosophy 20: 430–446. Nanay, Bence 2012b Perceptual phenomenology. Philosophical Perspectives 26: 235–246. Nanay, Bence 2012c Perceiving tropes. Erkenntnis 77: 1–14. Nanay, Bence 2013 Between Perception and Action. Oxford: Oxford University Press. Nanay, Bence 2014 Empirical problems with anti-representationalism. In: Brogaard, B. (ed): Does Perception Have Content? New York: Oxford University Press, pp. 39–50. Nanay, Bence 2015 The representationalism versus relationalism debate: Explanatory contextualism about perception. European Journal of Philosophy 23: 321–336. Nudds, M. 2001 Experiencing the production of sounds. European Journal of Philosophy 9: 210–229. Nudds, M. 2010 What are auditory objects? Review of Philosophy and Psychology 1: 105–122. Nudds, M. and O’Callaghan, C. 2009 Sounds and Perception, Oxford: Oxford University Press. O’Callaghan, Casey 2007 Sounds, Oxford: Oxford University Press.

Philosophy of Perception

•

19

O’Callaghan, C. 2008 Seeing what you hear: Crossmodal illusions and perception. Philosophical Issues 18: 316–338. O’Connor, D.H., Fukui, M.M., Pinsk, M.A., and Kastner, S. 2002 Attention modulates responses in the human lateral geniculate nucleus. Nature Neuroscience 5(11): 1203–1209. Papineau, David 1993 Philosophical Naturalism, Oxford–Cambridge, MA: Blackwell. Pasnau, R. 1999 What is sound? Philosophical Quarterly 49: 309–324. Pautz, Adam 2010 An argument for the intentional view of visual experience. In: Nanay, Bence (ed): Perceiving the World, New York: Oxford University Press, pp. 254–309. Peacocke, C. 1986 Analogue content. Proceedings of the Aristotelian Society Supplementary Volume 60: 1–17. Peacocke, C. 1989 Perceptual content. In: Almong, J., Perry, J., and Wettstein, H. (eds): Themes from Kaplan, Oxford: Oxford University Press, pp. 297–329. Peacocke, Christopher 1992 A Study of Concepts, Cambridge, MA: The MIT Press. Peacocke, C. 2008 Sensational Properties: Theses to Accept and Theses to Reject. Revue Internationale de Philosophie 62: 7–24. Schellenberg, Susanna 2010 Particularity and phenomenology of perceptual experience. Philosophical Studies 149: 19–48. Scholl, B.J. and Tremoulet, P.D. 2000 Perceptual causality and animacy. Trends in Cognitive Sciences 4: 299–309. Shams, L., Kamitani, Y., and Shimojo, S. 2000 What you see is what you hear. Nature 408: 788. Siegel, Susanna 2005 The phenomenology of efficacy. Philosophical Topics 33: 65–84. Siegel, Susanna 2006 Which properties are represented in perception? In: Gendler, T. and Hawthorne, J. (eds): Perceptual Experience, Oxford: Oxford University Press, pp. 481–503. Siegel, Susanna 2007 How can we discover the contents of experience? Southern Journal of Philosophy (Supp) 45: 127–142. Siegel, Susanna 2009 The visual experience of causation. Philosophical Quarterly 59: 519–540. Siegel, Susanna 2010a Do visual experiences have contents? In: Nanay, Bence (ed): Perceiving the World, Oxford: Oxford University Press, pp. 333–368. Siegel, Susanna 2010b The Contents of Visual Experience, New York: Oxford University Press. Siewert, Charles 2002 Is visual experience rich or poor? Journal of Consciousness Studies 9: 131–40. Spence, C., and Bayne, T. 2014 Is consciousness multisensory? In: D. Stokes, S. Biggs, and M. Matthen (eds), Perception and Its Modalities, New York: Oxford University Press, pp. 95–132. Spence, C. and Driver, J. 2004 (eds): Crossmodal Space and Crossmodal Attention, Oxford: Oxford University Press. Stalnaker, R. 1976 Possible worlds. Nous 10: 65–75. Stazicker, J. 2011 Attention, visual consciousness and indeterminacy. Mind & Language 26: 156–184. Travis, Charles 2004 The silence of the senses. Mind 113: 57–94. Tye, Michael 2007 Intentionalism and the problem of no common content. Philosophical Perspectives 21: 495–541. Vroomen, J., Bertelson, P., and de Gelder, B. 2001 Auditory-visual spatial interactions: Automatic versus intentional components. In: de Gelder, B., de Haan, E., and Heywood, C. (eds): Out of Mind, Oxford: Oxford University Press, pp. 140–150. Watkins, S., Shams, L., Tanaka, S., Haynes, J.D., and Rees, G. 2006 Sound alters activity in human V1 in association with illusory visual perception. NeuroImage 31: 1247–1256. Yeshurun, Y. and Carrasco, M. 1998 Attention improves or impairs visual performance by enhancing spatial resolution. Nature 396: 72–75.

PART

I

Are Perceptual States Representations?

CHAPTER

2

Experiences Are Representations

An Empirical Argument ADAM PAUTZ

To a first approximation, representationalism is the hypothesis that experiences are representational states akin to beliefs. I will sketch an argument for representationalism on the basis of an inference to the best explanation. I will not attempt to show that it is superior to every alternative. Instead I will focus on its main rivals—namely, “naïve realism” and the “inner state view”. My plan is as follows. I will start with contemporary naïve realism, defended by John Campbell, Bill Fish, and Michael Martin, among others. I will argue that it violates internal dependence: the empirically determined role of the internal processing of the brain in shaping phenomenal character. This will bring us to the inner state view, defended by Ned Block, Brian McLaughlin, and David Papineau, among others. I will argue that, while it accommodates “internal-dependence”, it fails to accommodate the essential “externally directedness” of experience. Finally, I will argue that only representationalists can adequately explain both of these features of experience. The upshot is a largely empirical case for representationalism. At the end, I briefly address what I consider to be the most profound objection to representationalism. Representationalism faces an overlooked modal puzzle about what experiences are possible.

1. Naïve Realism Violates Internal-Dependence: An Empirical Refutation I will introduce “naïve realism” by way of what I consider to be the best argument for it (Pautz 2010, pp. 283–295). To illustrate, suppose you see a blue ball (Figure 2.1). The most basic philosophical question about experience is this: what 23

24

•

Adam Pautz

Figure 2.1 A blue-looking ball

grounds the qualitative character of experience? Now there is something to this thought: it is obvious without argument that in such a case the character of your experience is grounded in nothing but your being acquainted with the bluish color and round shape of some concrete item. If so, then there are a couple of live options as to what this item is. It may be a mental image (a “sense datum”) created by the brain (the “sense datum theory”), or it may just be the ball itself. Since there are well-known problems with the first option, we have a case for accepting the second. The result is naïve realism: the character of your experience is grounded in your being acquainted with the color and shape of the physical object. In general, naïve realists hold that, even before sentient creatures evolved, external items possessed multiple objective (mind-independent) sensible properties: color properties, smell properties, loudness levels, and so on. Objects also possessed viewpoint relative but objective shapes, like being elliptical from here. The role of brain is not to construct experience. Rather the brain “opens the window shutter” to reveal objective properties of the items in the world. That is, when the brain responds to these objective sensible properties in the biologically normal way, this enables the mind to “reach out” and become acquainted with them. This long causal process is the superveniencebase of worldly acquaintance. In such veridical cases, the qualitative character of your experience is fully grounded in what external states you are acquainted with. The naïve realist John Campbell (Campbell and Cassam 2014, p. 27) also mentions your “point of view” as a factor, but then he says (p. 28) that this too is just a matter of which external states in the scene you are acquainted with (together perhaps with your own location in space). Naïve realism is externalist. The brain configures qualitative character only to a very limited extent: only to the extent that it selects what objective external states we get to be acquainted with. For instance, pigeons have different color experiences than humans, only because their different sensory systems enable them to be acquainted with different objective colors (constituted by UV light). As Campbell (2010, p. 206) puts it: [Naïve realism holds] that qualitative properties are in fact characteristics of the world we observe; our experiences have the qualitative characters

Experiences Are Representations

•

25

that they do in virtue of the fact that they are relations to those aspects of the world. So looking for the qualitative character of experience in the nature of a brain state is looking for it in the wrong place; we have to be looking rather at the [properties] of the objects experienced. So much for what naïve realism is. Is it right? I think that there are serious problems with the simple argument for it as just outlined, even if I consider it to be the best argument for naïve realism (Pautz 2010, pp. 295–297). I also think that there are much stronger arguments against naïve realism. Many argue that naïve realists cannot adequately explain illusion and hallucination. Here I will set this issue aside and develop a new line of argument: even in “normal” cases, naïve realism is inadequate. Psychophysics has shown that, even in normal cases, qualitative similarity is very poorly correlated with external physical similarity. At the same time, neuroscience has shown that neural similarity is the only accurate predictor of qualitative similarity. In short, the typical situation is that there is “good internal correlation” even while there is “bad external correlation”. Naïve realists like Campbell, Fish, and Martin neglect the scientific facts. They have it the wrong way around. Looking for the basis of qualitative character in the external world is looking for it in the wrong place; we have to be looking rather at the brain. For example, Figure 2.2 shows reflectances typical of purple-looking grapes, a blue-looking ball, and a green-looking leaf. By any natural measure, it is not the case that the reflectance of the ball objectively resembles the reflectance of the grapes more than the reflectance of the patch of grass. (In fact, if anything, the opposite is true.) Nevertheless, the blue appearance of the ball resembles the purple appearance of the grapes much more than the green appearance of the leaf. So there is “bad external correlation”. Since the explanation of similarity in color appearance is not to be found in the external world, there is reason to think it must reside in the brain. And 0.9

0.6

0.6

0.5

0.5

0.4

0.4

0.3

0.3

0.2

0.2

0.1

0.1

0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 400

500 600 Wavelength (nm) grapes

700

400

500 600 Wavelength (nm) ball

700

400

500 600 Wavelength (nm) leaf

700

Figure 2.2 Reflectances of grapes, a blue-looking ball, and a leaf. From MacAdam (1985)

26

•

Adam Pautz

this is exactly what recent neuroscience suggests. Neuroscience demonstrates “good internal correlation”. As Brouwer and Heeger write, The visual system encodes color by means of a distributed [neural] representation [in area V4] . . . similar colors evoke similar patterns of [neural] activity, and neural representations of color [in V4] can be characterized by low-dimensional “neural color spaces” in which the positions of [experienced] colors capture similarities between corresponding patterns of activity. (2013, p. 15454) So, your internal V4 neural representation of the blue-looking ball resembles your V4 neural representation of the purple-looking grapes more than your V4 neural representation of the green-looking leaf. This is the only available explanation of the resemblance-order among your color experiences. Now let’s take a parallel example involving smell. Consider the chemical properties in Figure 2.3 below. It is not the case that the middle chemical-type, R-limonene, resembles citral more than R-carvone. Rather, it resembles R-carvone more closely. Nevertheless, the perceived smell quality of R-limonene resembles the perceived smell quality of citral much more than the perceived smell quality of R-carvone. R-limonene and citral smell different, but their apparent smell qualities can both be described as “citrus-like”. That is why I call them citrus smell “1” and “2”. By contrast, R-carvone smells “minty”. This is another case of “bad external correlation”. At the same time, neuroscience demonstrates “good internal correlation”. Howard and co-workers (2009) found that “spatially distributed ensemble activity in human posterior piriform cortex (PPC) coincides with perceptual ratings of odor quality, such that odorants with more (or less) similar fMRI patterns were perceived as more (or less) alike”. For instance, they found that your PPC neural representation of R-limonene resembles your PPC neural representation of citral more than your PPC neural representation of R-carvone, in perfect agreement with the character of your smell experiences. As Margot (2009) says, “Because the chemical structure of the odors in [the citrus] odor category are very different, this is strong support for the idea that the PPC codes odor quality rather than structural and chemical similarity”.

Figure 2.3 The first two chemicals smell citrus-like (“citrus 2” and “citrus 1”); the third smells minty. From Margot (2009)

Experiences Are Representations “Same”

•

27

“Different”

/da/

Percentage/da/ responses

100

80

60

40

20 /ta/ 0 0

20 40 60 Voice onset time (ms)

80

Figure 2.4 From Goldstein (2009)

Finally, consider auditory experience. As Figure 2.4 illustrates, if you continuously increase the “voice onset time” of a speech signal (the time between opening the lips and the onset of vocal fold vibration), then suddenly at 30 ms there will be a big, categorical change in the audible quality, from /da/ to /ta/. This categorical change in the perceived sensible property corresponds to no categorical change in the objective stimulus. It corresponds only to a categorical change in your neural representation in the brain (Chang et al. 2010). In addition, even under normal conditions, perceived loudness is related in an enormously complex, non-linear fashion to a number of objective physical properties, including intensity, frequency, and “critical bands”. By contrast, it is related in a simple fashion to the total neural activity produced by a sound, according to standard models (Moore 2003). Given bad external correlation and good internal correlation, naïve realists’ externalist approach fails. To show this, I offer two arguments. First, the argument from irregular grounding. Naïve realists hold that sensible properties (color properties, smell and taste properties, loudness levels) are brain-independent properties of physical items. What, in their view, is the relationship between these objective sensible properties of these items, and the underlying ordinary physical properties: reflectances, chemical properties, and acoustic properties? Bad external correlation (illustrated in Figures 2.3 and 2.4) means that sensible properties and the underlying physical properties fall into different

28

•

Adam Pautz

resemblance-orders. This rules out the view that sensible properties are identical with the underlying physical properties (Pautz 2016). However, naïve realists might hold onto their view by claiming that sensible properties are irreducible objective properties of things that are grounded in, but distinct from, the corresponding physical properties. In that case, their resemblance-orders needn’t match (Allen 2015). But this requires irregular grounding: totally unsystematic and arbitrary grounding connections. Here are some examples. (I) As Figure 2.2 shows, the ball’s reflectance resembles the leaf ’s reflectance more than the grape’s reflectance. Still, naïve realists must hold that these reflectances ground objective colors—namely blue, green, and purple, which stand in a totally different, autonomous resemblance-order (blue evidently doesn’t resemble green more than purple). They must hold that this is just a quirk of reality with no explanation. (II) Likewise they must hold that the chemical structures in Figure 2.3 ground objective smell qualities that stand in a totally different resemblanceorder than they do. (III) They must hold that if varying the voice onset time for a speech signal continuously, then at precisely 30 ms there is a big, discontinuous “jump” in its irreducible, objective audible quality, from /da/ to /ta/. Since they take this audible quality to be independent of our neural response, they have no explanation of why the jump takes place precisely there. (IV) Finally, since they hold that loudness is objective and entirely independent of neural response, they have no explanation of why the loudness of a simple tone doubles when there is a tenfold (10db) increase in its intensity, rather than (say) when there is a twenty-fold increase. The broadly internalist approaches we will look at later—the “inner state view” (§2) and “brain-based representationalism” (§3)—deny the claim of naïve realism that the sensible properties (sensible colors, smell properties, audible properties) of items are explained by those items’ objective physical properties alone. Rather, according to them, qualitative facts are always (at least partially) grounded in our neural responses to external items. We know that our neural responses, unlike the physical properties of the items we perceive, nicely line up with qualitative character (“good internal correlation”). So the grounding connections posited by internalist theories are much more systematic and regular than the ones required by naïve realism. Since it is a truism that we should prefer more systematic theories, we should prefer an internalist approach to naive realism. Again, naïve realists like Campbell are just wrong in holding that to explain qualitative character we should ignore the brain and just look at the objective properties of the things we perceive. The naïve realist Bill Brewer suggested a response to me.1 There are, he says, other cases of irregular grounding. For instance, in their non-aesthetic properties, painting a might resemble painting b less than painting c, even though a and b are both beautiful, whereas c is ugly. Or, again, high-fiving someone and slapping someone are physically quite similar, but they are morally very

Experiences Are Representations

•

29

different (example due to Brian Cutter). If we have to accept some cases of irregular grounding, why is it so bad if naive realists excessively multiply them with the likes of (I)–(IV)? But, first, I dispute that these are cases of irregular grounding: if in these cases the grounding-bases include our emotional responses, then similarities and differences in the grounded properties do line up with similarities and differences in the grounding properties. Second, even if we must accept some cases of arbitrary, unsystematic grounding in nature, this doesn’t undermine my basic point: we surely should prefer theories with more systematic grounding connections. So we should prefer internalist theories to naïve realism. My second empirical argument against naïve realism is the argument from hypothetical cases. First, a preliminary point—on naïve realism, when you perceive a ball or a smell, you are acquainted with its color or smell quality, but not its electronic charge. Why? The only answer is that the visual system is causally responsive to its color and smell quality, but not its charge. So, naïve realists must hold that your acquaintance with simple property-instances is grounded in your bearing some causal relation, R, to those property-instances. Now consider an example involving smell. Recall that objective chemical similarity badly fails to predict qualitative similarity among smells; instead, it is only neural similarity that corresponds with smell similarity. So, for instance, if you smell the chemical types shown in Figure 2.3, then your PPC neural representation of R-limonene will resemble your PPC neural representation of the citrus-smelling citral much more than your PPC neural representation of the mint-smelling R-carvone. Your smell experiences are perfectly in line with this: R-limonene produces in you an experience of a citrus smell quality, one resembling the citrus smell of citral much more than the mint smell of R-carvone. Imagine now that you have a “twin” on a “Twin Earth”. We make two stipulations. First, your twin belongs to a human-like species that is just like our own but for one thing: because of differences in their evolutionary history, their postreceptoral wiring for smell differs from our own. Because of this, your twin’s PPC neural representation of R-limonene resembles his PPC neural representation of the minty-smelling R-carvone more than his PPC neural representation of the citrus-smelling citral —the opposite of what we find in a normal human like yourself. Consequently, while you sort R-limonene with the citrus-smelling citral, your twin sorts it with the minty-smelling R-carvone. Second, we stipulate that, when you and your twin smell these odorants, you both bear complex causal relation R to exactly the same complex olfactory properties of those odorants. Given these physical differences, what is the correct verdict on how your twin’s smell experiences would compare to your own? Previously, we noted that the empirical fact that neural similarity is the only good predictor of smell similarity and that objective chemical similarity is a lousy predictor

30

•

Adam Pautz

of smell similarity. So, given this empirical fact, the only reasonable verdict is that, whereas R-limonene smells citrus-like to you, it smells minty to your twin. This also fits with their sorting differences. The case is a hypothetical “twin earth case”, but this verdict about the case is supported by empirical facts, not our “intuitions”. By contrast, naïve realism implies that you and your twin have exactly the same smell experiences. True, you two undergo totally different PPC neural states, falling into different resemblance-orders and leading to different behaviors. But, we stipulated that, in doing so, you both bear causal relation R to the very same complex olfactory properties of those odorants. Therefore, on naïve realism, your and your twin’s neural states, although they fall into different resemblance-orders and lead to different behaviors, “enable” you to be acquainted with the very same objective olfactory states. On naïve realism, this means that there can be no qualitative differences between your and your twin’s experiences, despite all the neural and behavioral differences telling against this verdict Here is a similar case. You and your twin view the grapes, ball, and leaf with reflectances shown in Figure 2.2. Again, we make two stipulations. First, your twin is just like you but for one thing: because of naturally evolved differences in his species’ postreceptoral neural wiring, your twin’s V4 internal neural representation of the ball is more like his V4 neural representation of the green-looking leaf than his V4 neural representation of the purple-looking grapes—the opposite of the actual situation with you. Consequently, whereas you sort the ball with the grapes, your twin sorts the ball with the leaf. Second, you and your twin nevertheless bear the acquaintance-grounding causal relation R to the very same objective chromatic properties of the ball, the grapes, and the leaf (the ones that, on naïve realism, are their colors). This case is illustrated in Figure 2.5. This case is certainly possible. Compare how a mercury thermometer and a thermoelectric thermometer have different internal states but detect the same temperatures and temperature-differences.2

You

Your twin

V4 state

V4 state differently located in similarity-space

sorting with grapes

sorting with leaf etc.

the ball’s chromatic state

Figure 2.5 A hypothetical case

Experiences Are Representations

•

31

We previously noted the empirical fact that neural similarity is the only good predictor of similarity in color experiences. So, given the empirical facts, the only reasonable verdict is that you and your twin have different experiences. Whereas to you the ball looks bluish, to your twin it looks greenish. But, again, naïve realism delivers the empirically implausible verdict that they have exactly the same color experiences, despite the radical neural and behavioral differences between them, since it implies that they are exactly alike in what objective states they are acquainted with. Further, what Campbell (Campbell and Cassam 2014, p. 28) calls your “point of view” on the ball is the same: you both perceive the ball from the same spatial location, and you are acquainted with exactly the same subset of states involving the ball.3 These examples show that, besides requiring “irregular grounding” (my first argument), naïve realism requires giant flukes. Unlike an internalist view, naïve realism implies that similarity in the “enabling” neural representations has nothing to do with qualitative similarity in our experiences—just as similarity among words (e.g., “Ned” and “Fred”) has nothing to do with similarity among the people and things they are about. The cases involving you and your twin show this. The similarity relations holding among your twin’s neural states are totally different from those holding among yours; nevertheless, as we saw, on naïve realism, this makes no difference to character of his experiences—there is no qualitative difference between his experiences and yours. So in him there is a mismatch between neural similarity and qualitative similarity. Therefore, naïve realism has the absurd implication that it is just a fluke that in normal humans like yourself there is quite generally a nice agreement between neural similarity and qualitative similarity (“good internal correlation”)—somewhat as it would be a fluke if it turned out that similar-looking names (e.g., “Ned” and “Fred”) always named similar-looking individuals. Again, Bill Brewer has suggested a response.4 On this response, the externalist approach of naïve realism is correct after all. So, you and your twin do have the same experiences despite the radical neural and behavioral differences— then what about “good internal correlation”, which seems to show that the neural differences should make for qualitative differences? Brewer’s response is that we would expect good internal correlation even if naïve realism’s externalist theory of qualitative character is correct. To illustrate, look at citral and R-limonene in Figure 2.3. These chemical structures are totally different by any objective measure. Nevertheless, as we saw, naïve realists must say that even before we evolved they grounded similar objective citrus-like smell qualities (“irregular grounding”). According to Brewer, we would expect that we should have evolved so that the neural states that enable us to be acquainted with these similar objective qualities are themselves similar. The neural similarities are patterned by the real similarities in the objective smell qualities.

32

•

Adam Pautz

However, this response fails for a simple reason. Even if naïve realists are right that the chemically very different structures citral and R-limonene ground very similar objective smell qualities (despite my objection against such “irregular grounding”), this is no reason to expect that we should evolve to detect them with similar neural states. The similarities among a creature’s neural states simply aren’t determined by similarities in these alleged objective, autonomous qualities. Rather, they are determined by physical similarities and differences (e.g., similarity in chemical structure or in reflectance), by how these interact with the receptor systems, and by postreceptoral writing further downstream, which is itself dependent on the species’ unique ecology together with the vagaries of the evolutionary process. So the argument from hypothetical cases stands. Given their radically externalist approach, naïve realists cannot explain “good internal correlation” and must consider it a giant fluke. And they must accept the empirically implausible view that, in the cases I described, you and your twin have the same experiences despite the vast neural and behavioral differences. This is unacceptable. In short, naïve realism violates internal-dependence. We need a more internalist view.

2. The Inner State View Violates External Directedness I will now consider the simplest internalist view of experience: the inner state view recently defended by Ned Block, Brian McLaughlin, and David Papineau. Naïve realism holds, roughly, that qualitative character is grounded in relations to the environment. The inner state view is the polar opposite. It holds that having an experience with a certain character is necessarily identical with having some intrinsic physical-functional property, such as a complex distributed neural pattern in the visual system or the olfactory system. All aspects of experience can be reductively explained in internal neuro-functional terms. That exhausts the essence of experience. This view evidently accommodates internal-dependence. Still, I think we should reject the inner state view. For experience is also essentially “externally directed”. It is essentially spatial. Consider, for instance, the visual experience of the blue ball in Figure 2.1. Call this specific type of experience “the ball experience”. The following is obvious: External Directedness. Necessarily, if an individual has the ball experience, then that individual has an experience as of a bluish and round thing in a certain viewer-relative place, even in hallucination cases where there is no such thing. So any such individual has an experience that fully “matches the world” only if a bluish and round thing is present in some kind of space.

Experiences Are Representations

•

33

This is pretheoretical; it doesn’t presuppose any controversial “representationalist” ideas (Pautz 2010, pp. 269–270). Here is a related a point, one emphasized by Bertrand Russell in The Problems of Philosophy (chap. 10): Thought-Grounding Role: Necessarily, if you have the general capacity for thought, and if you have the ball experience, then this explains your having the additional capacity to have thoughts (e. g. something is that way) that are true only if something has the property being round. Further, if you have the ball experience together with other such experiences, then you are in a position to know certain truths about shape and color properties, for instance their patterns of similarity. These truths lack nominalistically acceptable paraphrases: they are essentially about general properties. (Pautz 2010, p. 279) In what follows, I will use “external directedness” to cover both of these two claims. To see why the inner state view violates external directedness, consider a brain in the void, or BIV for short. This BIV popped into existence out of the blue in an otherwise empty universe. BIV only has the neural machinery required to have the neuro-computational state, which, on the inner state, view is necessarily identical with the ball experience. Call that neuro-computational state “N”. Let’s also stipulate that it bears no interesting physical relation to the property being round. This property is not instantiated in BIV. And BIV doesn’t (and can’t) track the instantiation of this property in the outside world. Since BIV bears no interesting physical relation to being round, inner state theorists must hold it bears no interesting relation at all to being round (for they accept an austere physicalism). It follows that, if the inner state view is true, BIV cannot be said to have an experience that matches the world only if something is round—that is, has the property being round. It also lacks the capacity to predicate this property of anything in thought. So, in this scenario, the neural pattern N—and hence, on the inner state view, the ball experience—doesn’t count as an experience as of a round thing and doesn’t ground the capacity to have roundness-involving beliefs or knowledge. So the inner state view violates external directedness. In short, considered in itself, the inner neural-computational state N—and hence, on the inner state view, the ball experience—is no more essentially related to the property of being round than it is essentially related to (say) the property of being a pink elephant. In this way, it is like the orthographic sequence “r-o-u-n-d”, or a headache that was in fact caused by getting hit on the head by a round ball. This result is just clearly wrong. Normal visual experiences are essentially spatial; there is some essential connection between visual

34

•

Adam Pautz

experiences and spatial properties and relations, properties that needn’t be instantiated inside the head when we have those experiences.5 We need a theory of experience that accommodates “external directedness” as well as “internal-dependence”.

3. Experience as Representation: How Experience Might Be Externally Directed and Internally Dependent I am now ready to explain how a “representational” approach to experience might accommodate both external directedness and internal-dependence. It is a middle way between the extremes of naïve realism and the inner state view. I will focus on the ball experience, but afterward I will generalize the argument. Take external directedness first. The simplest explanation is there is a single mental relation R such that (I) having the ball experience essentially involves standing in R to the property being bluish and being round, (II) standing in R to properties grounds one’s ability to predicate those properties in thought and to know truths about those properties (Russell 1912), and (III) bearing relation R to some properties implies its seeming to you that they are instantiated out there in the world. This hypothesis explains external directedness. That is, it explains the fact that, necessarily, if you have the ball experience, then you have an experience that matches the world only if something has these properties (rather than some other properties), and you thereby have the capacity to predicate these properties to things in thought and to learn truths about them. If having the blue ball experience didn’t involve standing in such a relation to these specific properties, what else could explain these facts? Now the question arises, what is the nature of the relation with these features? Sense datum theorists like Russell (1912) believed in such a relation to color and shape properties and called it “acquaintance”. They held that what it is to bear such a relation to a property like being bluish and round is to be acquainted with the instantiation of that property by “sense datum” created by the brain. This view accommodates internal-dependence. It also accommodates external directedness, even though sense data are brain-created. For it holds that sense data literally have spatial properties and relations and occupy a kind of space. But the sense datum view faces problems—for instance, problems indeterminate of experiences and experiences of impossible scenes (Pautz 2010, pp. 280–281). The best view is that in non-veridical cases the property being bluish and round is not instantiated by anything in your vicinity at all. Therefore, we can add a thesis to our proto-theory: (IV) you can bear R to some properties even though they are not instantiated by anything in your vicinity. This makes the theory broadly representationalist.

Experiences Are Representations

•

35

Let us call the hypothesized relation R with features (I)–(IV) “the perceptual predication relation” or the “phenomenal representation relation”.6 The inner state view is inconsistent with the thesis here that having the ball experience essentially involves perceptual predicating the spatial property being round, for it holds that the complete essence of this experience can be completely given in purely internal, neural-computational terms. Now so far our minimal representationalist theory is actually consistent with a form of naïve realism. For naïve realists might hold that, in all cases in which we have the ball experience, we phenomenally represent the property being bluish and round, but add that in veridical cases this is somehow “grounded in” our being acquainted with the instantiation of this property by a real blue sphere (Pautz 2010, pp. 295–296). Then phenomenal representation is externally determined. It is only when we add the empirical fact of internal dependence that we get a view that is inconsistent with naïve realism. For, given internal dependence, we must add a clause to our theory: (IV) what sensible properties (sensible colors, smell qualities, audible qualities) we phenomenally represent are entirely fixed by the intrinsic properties of the subject. For instance, going back the hypothetical case discussed in §1, when you and your twin view the ball, then, due to your different V4 processing, you two phenomenally represent distinct sensible colors (being bluish and being greenish) as co-instantiated with being round. This is a “brain-based” representational theory. Now this may seem puzzling. It seems to us that sensible properties are instantiated in external space. How can this depend on internal processing? Elsewhere I have addressed this puzzle (Pautz 2016). I think that the solution involves recognizing that sensible properties are not real, objective properties of external things; rather, they are “projected” properties, or “virtual properties”. (In fact, because of relativity theory and quantum mechanics, Chalmers [2012, p. 333] extends this kind of neo-Galilean view to the spatial properties and relations that we phenomenally represent.) This view is consistent with representationalism but not naïve realism, since naïve realism requires that the sensible properties are real, objective properties of external items. The argument is general. Different experiences involve the seeming-presence of different properties and ground the ability to think about and know about different properties. Russell (1912, chap. X) was right to emphasize this point, but he was wrong to think that the properties must be instantiated by things we perceive (“sense data”). The best view is that having different experiences necessitates perceptual predicating different property-complexes, where the relevant property-complexes needn’t be instantiated. So, for instance, having a certain auditory experience might entail perceptual predicating the propertycomplex λxλy (x is high-pitched and to the left, and y is low-pitched and to the right), and having a certain smell experience might entail perceptual predicating λx (x is citrus-like and in the region before me).

36

•

Adam Pautz

We have arrived at a necessary covariance between experience and phenomenal predication. This leaves open what the “direction of explanation” is. There are three options. (I) Having an experience with a certain character is distinct from, but essentially grounds, standing in the phenomenal predication relation a certain complex property, an abstract object. (Analogy: the concrete fact that John and Mary are in the room grounds the fact that the people in the room stand in the numbering relation to the number two, an abstract object.) (II) Perceptual predicating a certain property-complex is distinct from, and grounds, having an experience with a certain character. (III) Having an experience with a certain character just is perceptual predicating a certain property-complex, allowing no room for an explanatory, grounding connection between them. I think options (I) and (II) raise puzzles. If experience properties and representational properties are distinct, then what explains the necessary connection between them? And what is the non-representational nature of experience that grounds its representational nature? I favor option (III) because it is simple and avoids such questions. This is a strong “identity” form of representationalism (Pautz 2010, Chalmers 2013, Tye 2014).7

4. A Neglected Problem for Representationalists: The Laws of Appearance Finally, having sketched an argument for representationalism, I turn to what I consider to be its most difficult and interesting problem—a problem that has been almost entirely neglected. Roughly, the problem is that if experiences are just representations, then why can’t you have experiences that represent certain extremely bizarre scenarios? This requires explanation. On representationalism, experience essentially involves the phenomenal predication of complex properties. So experiences can be associated with propositional contents. For instance, if you have an experience in which you perceptual predicate the complex property λxλy (x is red, and y is green, and x is to the left of y), then you phenomenally represent the proposition ∃x∃y (x is green, and y is red, and x is to the left of y). If your experience is non-hallucinatory, you also phenomenally represent a singular proposition attributing this complex property to some specific viewed objects. Now the start of our puzzle is that not every content corresponds to a metaphysically possible experience. In particular, at least some of the following modal claims about the limits of experiences are true: I. It’s impossible to have an experience with the content something is pure blue and also greenish blue. So-called pluralists about color (Pautz 2016) believe such contents, but it’s impossible to have an experience with such a content. (Here and in what follows, by “impossible”, I mean metaphysically impossible.)

Experiences Are Representations

•

37

II. It’s impossible to have an experience with the content something is spherical and cubicle.8 III. It’s impossible to have an experience with the content blue is intrinsically overall more like green than purple. IV. It’s impossible to have an experience whose only content is a wildly disjunctive content—for example, that thing is round and green and directly in front of me or it is square and purple and 45 degrees to the left. V. It’s impossible to have an experience whose only content is that thing’s facing surface is round (i.e., without any information about the chromatic or achromatic colors of the thing or its background). VI. It’s impossible to have an experience that only has the content that thing is cubicle, but that doesn’t have any de se content about the thing’s apparent shape from a particular point of view (as it were, a “God’s eye” visual experience of a cubicle object, from no point of view). VII. It’s impossible to have a single experience whose content is a is red all over and b is wholly behind a (Johnston unpublished). VIII. It’s impossible to have a visual experience in which one phenomenally represents a “high-level” content like that is pine tree or that is a Republican, but in which one phenomenally represents no “low-level” content at all. It’s also impossible to have an experience in which one phenomenally represents such a high-level content, but in which one phenomenally represents a totally incongruous low-level content, like that very same thing is a giant round sphere right in front of me.9 You might take the skeptical view that these strong assertions of metaphysical impossibility are false and that we only think they’re true owing to limits on our imagination. But I myself think that at least some such modal claims are true. Call them laws of appearance, since they are metaphysically necessary prohibitions on how things might appear. Now we have general Occamist reasons to keep “brute modal facts” to a minimum. So, when faced with the laws of appearance, we have reason to derive them from some smaller set of more basic facts. Maybe they can be derived from more general metaphysically necessary truths (e.g., the most general, overarching laws of appearance). Or maybe they can be derived from a pre-modal real definition or account of what it is for an experience to have a certain content, or represent a certain state of affairs. Here is an analogy. I think, with many others, that having certain totally bizarre sets of beliefs and desires is metaphysically impossible. For instance, it’s impossible that someone with my same past and current experiences and my same behavioral dispositions should be correctly interpreted as believing and desiring the negations of all the propositions I believe and desire. So there are also constraints on beliefs and desires (though they differ from the constraints

38

•

Adam Pautz

on experiences I–VIII). Now, in my view, such metaphysical impossibilities involving beliefs and desires can be explained. They can be explained by a rationality-based account of belief and desire in the tradition of Davidson and Lewis. So they are not “brute” modal facts but rather follow from the “real definition” of belief and desire. Here, finally, is the problem for representationalists that I have been leading up to. Representationalists apparently can provide no good explanation of why the “laws of appearance” obtain, so they must hold that they are all brute facts. The only potential explanations I can think of face immediate problems. One potential explanation is this. Impossible states of affairs and highly disjunctive states of affairs are acausal and therefore cannot be tracked. So, given a general “causal” or “tracking” account of what it is for an experience to have a certain content (Tye 2014, p. 51), they cannot be phenomenally represented. This might go some way toward explaining at least some laws of appearance, specifically I–IV. But this explanation fails for two reasons. First, even if a tracking account of phenomenal representation were correct, it couldn’t explain all the laws of appearance. In fact, such a theory entails that V–VIII are false, for some possible visual systems could directly track the relevant states of affairs. Second, given internal-dependence, we must in any case reject “tracking representationalism”. For instance, according to what I take to be the correct representationalist account of the cases discussed earlier (§1), you and your twin track the same objective physical properties (reflectance-types, chemical types) but perceptual predicate distinct sensible properties (sensible colors, smell qualities). Another potential explanation has it that the laws of appearance follow from the fact that phenomenal representation is a “quasi-pictorial” form of representation (a suggestion put to me independently by Chris Hill and Michael Tye). But what does this mean? Maybe it just means that phenomenal representation is necessarily a form of representation that obeys laws of appearance like I–VIII. But then the claim doesn’t provide a substantive, non-circular explanation of these laws. Alternatively, maybe the claim here is entirely about the way phenomenal representation is realized in the brain—that is, about the format of phenomenal representation (Kosslyn 1994). In other words, maybe it is about the nature of the neural “content-vehicles”, not in the first instance about the kinds of “contents” we phenomenally represent. The idea is that this claim nevertheless explains why we cannot phenomenally represent the kinds of contents described in I–VIII. But, first, I do not see how any such a claim might logically entail all of the laws of appearance. In fact, there is a fundamental problem with the idea that it could explain any of them. The laws of appearance I–VIII claim that it’s metaphysically necessary that no one has certain bizarre experiences. By contrast, if

Experiences Are Representations

•

39

it is a truth at all, it is bound to be a contingent truth that phenomenally representation is realized in the brain in a quasi-pictorial format. And, as a matter of logic, a contingent truth cannot explain a necessary one. For instance, surely there are possible creatures for whom the realizers of phenomenal representation are sentence-like, rather than quasi-pictorial. (Representationalist theories are typically broadly functionalist theories allowing for “multiple realizability”.) Presumably, in such a creature’s “sensation box”, there might appear “sensation-sentences” that represent the bizarre contents mentioned in I–VIII. So even if in actual humans the format of phenomenal representation is “quasi-pictorial”, that is not enough to explain why the bizarre experiences described in I–VIII cannot occur in other possible creatures. Therefore, it is not enough to explain I–VIII, understood as claims of metaphysical impossibility.10 Let’s suppose, for the sake of argument, that representationalists cannot explain the “laws of appearance” and must take them to be brute axioms. Would this be a strong reason for rejecting representationalism? I don’t think so, because I think that all theories of experience face a parallel explanatory challenge. For instance, consider naïve realism, which I criticized previously on empirical grounds. Even on this view, we can ask exactly parallel explanatory questions. Why couldn’t you have certain very bizarre veridical or nonveridical experiences? For instance, why couldn’t you have a hallucination of a round square? Why couldn’t you be acquainted with the objective cubicle shape of something “neat”, without being ostensibly acquainted with any other fact about it (for instance, its viewer relative shape from here, or its apparent color)? I think it can be shown that contemporary naïve realists aren’t any better placed to adequately answer such questions than representationalists. So the problem of how to explain “the laws of appearance” is everyone’s problem. Yet it is a problem that philosophers of perception have hardly addressed.11

Notes 1. In an “author meets critics” meeting at the 2014 meeting of the Central Division of the American Philosophical Association. 2. If the naïve realist nonetheless asserts that my two stipulated physical conditions are incompatible (Fish 2013, p. 59), then he or she must back this up with a plausible account of the causal relations that ground acquaintance from which this assertion follows. Elsewhere (Pautz 2011) I argue that this cannot be done. 3. Naïve realists might handle my cases by further revising or complicating their view. Logue (2012, pp. 223–225) says that, to handle my hypothetical cases, naïve realists might say that “features of the subject” (presumably, neural features) partly determine qualitative character. Another idea would be to handle the case by building on Campbell’s idea that experience is not simply a “two-place” relation between a subject and the perceived scene but a “threeplace relation” modified by the “ways in which the scene is given” (Campbell and Cassam 2014, pp. 27–28). However, naïve realists must answer a crucial question. To you, the round surface of the ball looks bluish; to your twin, it looks greenish. Given the empirical facts, that is the most plausible verdict. The crucial question is this: what are these distinct sensible colors that appear to you and your twin to fill the round region of the ball? I think that even

40

4. 5.

6.

7.

•

Adam Pautz

modified naïve realist views cannot adequately answer this question. The naïve realist cannot answer that these distinct sensible colors are objective, response-independent features of the ball, since by stipulation you and your twin causally detect the same response-independent chromatic feature of the ball. Maybe Logue would suggest that naïve realists should instead answer that that they are in fact “features of the subjects” (e.g., features of you and your twin’s different inner postreceptoral neural processing), which, however, you and your twin experience as properties of the round ball! But this option would require a bizarre kind of projective error. Yet another suggestion (put to me by Craig French) would be that naive realists adopt a Shoemaker-style view according to which the distinct sensible colors experienced by you and your twin are distinct “appearance properties” or “color-looks” of the ball, where these are neural-relative, response-dependent properties of the ball of the form normally causing internal V4 processing X in the relevant population. Typically, it is representationalists who propose such a view. Elsewhere I develop several objections showing that representationalists cannot accept this type of view (Pautz 2013). My objections there could be used, mutatis mutandis, to argue that naïve realists cannot adopt it either. In the same meeting mentioned in note 1. Keith Allen suggested a similar response in discussion. Elsewhere (e.g., Pautz 2010, pp. 266–272) I develop this argument in more detail. Papineau (2016, sect. 15) is an inner state theorist who agrees with me that inner state theorists must reject external directedness. However, he offers a surrogate claim using similar-looking spatial terminology: necessarily, if an individual (e.g., BIV) has the ball experience (that is, on his inner state view, the intrinsic neural-computational property N), then there is an internal “phenomenal object” that is “in” the individual’s experience and that is round*. Now you might think, “This is very close to external directedness—so isn’t this good enough?” But don’t be deceived. By a “phenomenal object”, Papineau means something like a population of neurons, and by “visual roundness*” he means a neural-computational property of this population of neurons—a property that is nothing like the property of being round (that is, the property having edges roughly equidistant from a common point). So, despite his similarlooking spatial terminology, Papineau’s claim doesn’t come close to accommodating external directedness: the obvious fact that, necessarily, if you have the ball experience, then you are in a state that matches the world only if something before you is round, where “is round” expresses a genuine spatial condition. Michael Tye (2014, pp. 51–52) is a representationalist who claims that we can be said to be aware of the uninstantiated properties we phenomenally represent. But it is important to realize that representationalists are not committed to this claim. I myself reject it because I find it very odd (Pautz 2010, p. 266). The oddness of the claim is especially clear in the case of uninstantiated relations, a case that Tye doesn’t consider. For instance, we phenomenally represent the relation of spatial containment when we represent one thing as being spatially within another thing. By representing this relation in experience, we somehow become directly aware of certain facts about it—for instance, that it is necessarily transitive. But it is odd in the extreme to say that in non-veridical cases one is aware of the abstract relation of spatial containment, even though it is not instantiated by anything before one. While I favor identity representationalism, it should be noted that it faces couple of important objections. (1) One might think that there are possible worlds (“Eden worlds”) where naïve realism is true and representationalism is false, which would undermine the strong thesis of identity representationalism. (This may be Logue’s objection in her contribution to this volume.) For different possible responses, see Pautz (2010, fn. 33) and Chalmers (2013, p. 350). (2) Another objection to identity representationalism is simply that, intuitively, having a blue ball experience just can’t be identical with standing in a representation relation to a complex property, if this is understood as an abstract object of some kind. Intuitively, having an experience must be distinct from standing in a relation to such a peculiar abstract object. I developed and responded to this objection in previous papers (e.g., 2010, pp. 292– 297). More recently, Papineau (2015, sect. 13) has taken it up; in fact, it is his chief objection

Experiences Are Representations

8.

9.

10.

11.

•

41

to representationalism. I think the objection fails for a couple of reasons. First, the objection only tells against “identity representationalism” and doesn’t tell against a “grounding” form of representationalism (along the lines of [ii] in the text) that concedes that having an experience is distinct from standing in a relation to an abstract object. Second, we have reasons to be dubious of the “intuition” here. For if it is an “intuition of distinctness”, then it cuts equally against pretty much all views, including Papineau’s own inner state view (for, intuitively, having the ball experience is also distinct from being in an inner neural-computation state involving soggy grey brain matter). On the other hand, the intuition might be that it is odd or surprising that the account or real definition of what it is to have a certain experience should mention abstract objects like complex properties. But there is a track record of equally surprising accounts being correct in other domains. For instance, there are convincing arguments for supposing that the right accounts of what it is for two people to believe alike, or for two things to be exactly alike, also essentially appeal to “abstract” items—namely, propositions and properties. You might think that the waterfall illusion shows that we can perceptual predicate two incompatible properties of the same thing. But my view is that the content of the waterfall illusion actually involves no real inconsistency at any particular time (Pautz 2010, p. 303). Nanay (2012) argues that cases of unilateral neglect show that it is possible to phenomenally represent a certain kind of “high-level” contents (viz., one attributing what he calls actionoriented properties), while phenomenally representing “no level content”. This is an interesting interpretation of such cases, but it is also open to doubt—see Masrour (2011) and Raftopoulos (2015). Further, the cases Nanay discusses don’t cast doubt on my distinct claim that it is impossible to have an experience in which one phenomenally represents a high-level content like that is a pine tree, but in which one phenomenally represents a totally incongruous lowlevel content, like that very same thing is a giant blue and round sphere right in front of me. Of course, in response, the representationalist might say: “Ok, I think that the experiences described in I–VIII are only nomically impossible. Contrary to what you say, they are not metaphysically impossible. The reason is that they could occur in other possible creatures for the reason you suggest”. I realize this is a possible response. But, as said at the outset, I think that it is just obvious that at least some of I–VIII are true, understood as strong claims of metaphysical impossibility. So my preference is to accept them, even if I cannot now provide any good explanation of them. (Thanks to Bence Nanay for discussion of this point.) My thanks to Craig French and Bence Nanay for some very helpful comments on an earlier version of this paper.

References Allen, K. 2015. “Colour Physicalism, Naïve Realism, and the Argument from Structure.” Minds and Machines 25: 193–212. Brouwer, G. and D. Heeger. 2013. “Categorical Clustering of the Neural Representation of Color.” Journal of Neuroscience 33: 15454–65. Campbell, J. 2010. “Demonstrative Reference, the Relational View of Experience and the Proximality Principle.” In R. Jeshion (ed.) New Essays on Singular Thought, 193–212. Oxford: Oxford University Press. Campbell, J. and Q. Cassam. 2014. Berkeley’s Puzzle. Oxford: Oxford University Press. Chalmers, D. 2012. Constructing the World. Oxford: Oxford University Press. Chalmers, David J. 2013. “The Contents of Consciousness: Reply to Hellie, Peacocke and Siegel.” Analysis 73: 345–68. Chang, E. F., J. W. Rieger, K. Johnson, M. S. Berger, N. M. Barbaro, and R. T. Knight. 2010. “Categorical Speech Representation in Human Superior Temporal Gyrus.” Nature Neuroscience 13: 1428–32. Fish, W. 2013. “Perception, Hallucination, and Illusion: Reply to My Critics.” Philosophical Studies 163: 57–66.

42

•

Adam Pautz

Goldstein, B. 2009. Sensation and Perception. Pacific Grove, CA: Wadsworth. Howard, J. D., J. Plailly, M. Grueschow, J. D. Haynes, and J. A. Gottfried. 2009. “Odor Quality Coding and Categorization in Human Posterior Piriform Cortex.” Nature Neuroscience 12: 932–9. Johnston, M. unpublished ms. The Manifest. Kosslyn, S. 1994. Image and Brain. Cambridge: MIT Press. Logue, H. 2012. “Why Naive Realism?” Proceedings of the Aristotelian Society 112: 111–37. MacAdam, D. L. 1985. “The Physical Basis of Color Specification.” In Color Measurement: Theme and Variations, 1–25. New York: Springer-Verlag. Margot, C. 2009. “A Noseful of Objects.” Nature Neuroscience 12: 813–4. Masrour, F. 2011. “Does Perception Represent Affordances?” https://consciousnessonline.files. wordpress.com/2011/02/comments-on-nanay-farid.pdf Moore, B. 2003. An Introduction to the Psychology of Hearing. San Diego, CA: Academic Press. Nanay, B. 2012. “Perceptual Phenomenology.” Philosophical Perspectives 26: 235–46. Papineau, D. 2016. “Against Representationalism (about Experience).” International Journal of Philosophical Studies 24(3): 324–47. Pautz, A. 2010. “Why Explain Visual Experience in Terms of Content?” In B. Nanay (ed.) Perceiving the World, 254–310. Oxford: Oxford University Press. Pautz, A. 2011. “Can Disjunctivists Explain our Access to the Sensible World?” Philosophical Issues 21: 384–433. Pautz, A. 2013. “The Real Trouble for Phenomenal Externalists: New Evidence for a Brain-Based Theory of Consciousness.” In R. Brown (ed.) Consciousness: Inside and Out, 237–298. Berlin: Springer-Verlag. Pautz, A. 2016. “How Does Color Experience Represent the World?” In D. Brown and F. MacPherson (eds.) The Routledge Handbook to the Philosophy of Color. London: Routledge. Raftopoulos, A. 2015. “What Unilateral Visual Neglect Teaches us About Perceptual Phenomenology.” Erkenntnis 80: 339–58. Russell, B. 1912. The Problems of Philosophy. London: Williams and Norgate. Tye, M. 2014. “Transparency, Qualia Realism and Representationalism.” Philosophical Studies 170: 39–57.

CHAPTER

3

Are Perceptual Experiences Just Representations?

HEATHER LOGUE

One task of the philosopher of perception is to give a metaphysics of perceptual experience: the sort of mental state associated with vision, hearing, touch, smell, and taste. Giving a metaphysics of perceptual experiences is, roughly, to say what they are “made of ”. Plausibly, perceptual experiences involve goingson in the brain. But we may ask: are they just brain states? If so, which ones? Perhaps ones that represent the subject’s environment in a distinctive manner? If they aren’t just brain states, what more is there to them? Perhaps they include bits of the subject’s environment, too? Before attempting to answer such questions, it is important to be clear about what would make it the case that a given answer is correct. I take it that at least one constraint on a metaphysics of perceptual experience concerns accounting for its phenomenal character—“what it is like” to have a given perceptual experience (Nagel 1974), or its distinctive “feel”. That is, at least part of what we’re trying to do in giving such a theory is to explain where the phenomenal character of perceptual experience comes from; in other words, we’re trying to give an account of the facts in virtue of which a given perceptual experience has the phenomenal character it does.1 On what is arguably the orthodox view, perceptual experiences are states of a subject that involve representing her environment as being a certain way. For example, my current visual experience consists in my visually representing that there is something yellow and crescent-shaped before me (namely, the banana on my desk), much as I might doxastically represent the same proposition.2 In previous work (Logue 2014), I have argued that perceptual experiences involve

43

44

•

Heather Logue

representation on the basis of epistemological considerations. But is perceptual representation all there is to perceptual experience? My focus here is whether we can account for perceptual phenomenal character in terms of perceptual representation alone. Even though I think that the best account of the epistemological role of perceptual experience construes it as involving representation, my hunch is that we will fail to capture something important about perceptual phenomenal character if we restrict our metaphysics of perceptual experience to representational states. However, I must admit that I have found it difficult to put my finger on precisely what it is that representational accounts of phenomenal character fail to capture. The goal of this paper is to survey several proposals and to assess their force. But before we turn to those proposals, it will be helpful to further explicate the representational view of perceptual experience alongside its main contemporary rival.

1. The Representational View vs. the Extended View The Representational View holds that perceptual experience fundamentally consists in the subject representing her environment as being a certain way. (This view is also known as Intentionalism.)3 The schema “fundamentally consists in x” is simply shorthand for the idea that the most basic personal-level psychological explanation of the phenomenal facts is in terms of x. According to the Representational View, the most basic personal-level psychological explanation of these facts is in terms of the subject perceptually representing her environment as being a certain way. Plausibly, there are further subpersonal facts (e.g., having to do with “low-level” transformations of information in the causal process leading up to perceptual experience) and non-psychological facts (e.g., having to do with patterns of neural firing and the like) in virtue of which the personal-level psychological facts obtain. But it is not the business of an “armchair”-based metaphysician of perceptual experience to give theories about such empirical matters. The task is just to identify a personal-level psychological state in virtue of which the phenomenal facts obtain, and it is a presupposition of the endeavor that such general truths are accessible from the armchair via philosophical reflection (thought experiments, transcendental arguments, and so forth). So, for example, the representationalist holds that my current visual experience fundamentally consists in (i.e., the phenomenal facts are explained in terms of) my visually representing that there is a yellow, crescent-shaped thing before me. The Representational View’s main contemporary rival goes by many names: usually Naïve Realism (Martin 2004, Fish 2009), the Relational View (Campbell 2002), or the Object View (Brewer 2011). I prefer the label Extended View, for reasons I will explain shortly. On this view, veridical perceptual experience (roughly: a perceptual experience in which the subject perceives things in her environment, and they are as they perceptually appear to be) fundamentally

Perceptual Experiences

•

45

consists in the subject perceiving things in her environment. That is, the most basic personal-level psychological explanation of the phenomenal character of veridical experience is in terms of the subject perceiving things in her environment, rather than representing her environment as being a certain way (which she could do regardless of whether she’s perceiving anything in it). For example, according to the Extended View, the phenomenal character of my current veridical visual experience fundamentally consists in my perceiving the banana on my desk. Note that this view is an instance of Clark and Chalmers’ (1998) extended mind thesis: the metaphysical structure of veridical perceptual experience literally includes mind-independent things in one’s environment (such as bananas). I prefer the “extended” label, rather than the others previously mentioned, as (a) rivals to this view are also relational in some sense (e.g., representational relations to propositions), (b) we’ve arguably left the realm of the naïve once we’ve started doing the metaphysics of perceptual experience, and (c) a proponent of this sort of view might think that entities other than the object of a veridical experience contribute to its phenomenal character (e.g., certain features of the subject’s perceptual system, or the property instances she experiences). Note that the Extended View entails disjunctivism about perceptual experience. Disjunctivism is a comparative claim concerning veridical experiences and total hallucinations, which are experiences that don’t involve perceiving anything at all in one’s environment (e.g., the sort that would be had by a brain in a vat). Disjunctivism claims that veridical experiences and at least total hallucinations are fundamentally different.4 That is, the most basic personal-level psychological explanation of the phenomenal character of total hallucination is different than that of veridical experience. This simply follows from the fact that total hallucinations don’t involve the subject perceiving anything in her environment by definition—if total hallucinations don’t involve perceiving any such things, they cannot fundamentally consist in perceiving such things. Given that total hallucinations cannot be extended, the truth of the Extended View requires the truth of disjunctivism.5 But let us assume for the sake of argument that disjunctivism can be defended.6

2. Perceptual Phenomenal Character and the Explanatory Gap A common worry about physicalism is that there is an “explanatory gap” between physical states of the brain and phenomenal consciousness—including the sort associated with perceptual experiences in particular. Here is Joseph Levine’s expression of this thought: Let’s call the physical story of seeing red ‘R’ and the physical story for seeing green ‘G’. . . . When we consider the qualitative character of our visual experiences when looking at ripe McIntosh apples, as opposed to ripe

46

•

Heather Logue

cucumbers, the difference is not explained by appeal to G and R. For R doesn’t really explain why I have the one kind of qualitative experience . . . and not the other. (1983, 357–358) (It’s clear from the context that the physical stories R and G are supposed to concern neural or functional states of the brain, and I take it that “qualitative character of perceptual experience” refers to what I’ve been calling “phenomenal character”.) The idea is there is an explanatory gap between (say) G and what it’s like to see green, in the sense that G does not seem to answer the question “Why is it like this to see green?” (asked while demonstrating the phenomenal character of one’s experience of greenness). If I want to know why seeing green is like this, as opposed to what it’s like to see red, saying that the phenomenal character of experiences of greenness is grounded in G sheds no light on the matter whatsoever. William Fish has argued that we can get some traction with this question if we broaden our explanatory base. In particular: . . . the reason that Levine cannot find the answer that he is looking for is because he is searching in the wrong place. The difference in what it is like to see a ripe McIntosh apple and what it is like to see a ripe cucumber is not explained by the differences in the underlying processing—instead, it is explained by the different color properties that the two objects possess. When we see a ripe McIntosh apple, the phenomenal character of our experience is its property of acquainting us with the fact of the object’s being red; when we see a ripe cucumber, it is the experience’s property of acquainting us with the fact of the object’s being green. (Fish 2009, 75–76) By “acquaintance”, Fish means “an irreducible mental relation that the subject can only stand in to objects that exist and features that are instantiated in the part of the environment at which the subject is looking” (2009, 14).7 It sounds like the acquaintance relation is basically the relation of perceiving, with the claim that the perceptual relation is unanalysable built in.8 So the claim is that there is an explanatory gap between phenomenal greenness and what’s going on in the brain, but that’s as it should be—phenomenal character isn’t to be explained solely in terms of what’s going on in the brain. However, Fish thinks that we can explain why it is like this to see green by adopting a version of the Extended View: it is like this to see green simply because the phenomenal character of experiences of greenness just is being acquainted with (i.e., perceiving) something’s being green. That’s all there is to it. By contrast, Fish thinks that the Representational View cannot provide an analogous explanation. For there is an explanatory gap between (e.g.) visually

Perceptual Experiences

•

47

representing greenness and phenomenal greenness, just as there is between the physical story G and phenomenal greenness—the claim that one visually represents greenness doesn’t really explain why it is like this to see it (Fish 2009, 78, fn. 12). On the face of it, though, it seems like we can make a similar complaint about Fish’s proposal.9 Couldn’t the phenomenal property that we call phenomenal redness have been associated with perceiving something’s being green instead? Or, for that matter, couldn’t the phenomenal property that we call phenomenal greenness have been associated with perceiving something’s being red instead? As long as these are live possibilities, there is an explanatory gap between (e.g.) phenomenal greenness and perceiving the fact of something’s being green, and hence the identification of the former with the latter will not be transparent. I suspect that there is an implicit idea lurking in the background, to the effect that phenomenal greenness isn’t just a matter of being acquainted with the fact of something’s being green. In particular, the idea is that phenomenal greenness also involves being acquainted with what it is for something to count as green, or (if you’ll allow me a somewhat obscure but handy way of speaking) the nature of greenness.10 This is suggested by Fish’s talk of “. . . what it is like being supplied by the property itself ” (Fish 2009, 78, fn. 12, second set of italics mine). Phenomenal greenness is supposed to be in some sense “supplied by” or given by greenness; it’s natural to elaborate this thought in terms of phenomenal greenness somehow deriving from the nature of greenness itself. And we can make sense of this claim in terms of the Extended View: phenomenal greenness consists in being acquainted with (i.e., perceiving) not just something’s being green but, in so doing, also becoming acquainted with the very nature of greenness.11 On this picture, the natures of the colours (and perceptible properties in general) are revealed to us in our experiences of them (cf. Johnston 1992). This provides us with an answer to the questions raised earlier. Phenomenal redness couldn’t have been associated with perceiving something’s being green, because phenomenal redness just is being acquainted with the nature of redness—and how on earth could you become acquainted with the nature of redness by perceiving an instance of greenness? (The same goes for the question concerning phenomenal greenness, mutatis mutandis.) Moreover, the Representational View doesn’t seem to be in a position to close its explanatory gap by exploiting the claim that phenomenal greenness involves acquaintance with the nature of greenness—for perceptually representing a property isn’t obviously sufficient for being acquainted with its nature. Putting this all together, if we elaborate Fish’s story as outlined above, one could argue that Extended View has some gap-closing potential that the Representational View lacks. Seeing greenness is like this (demonstrating the phenomenal greenness of my experience) because it consists in acquaintance with the nature of greenness, and the nature of greenness is, well, this (demonstrating

48

•

Heather Logue

the nature of greenness, as revealed in my experience). If this line of thought works, then we have a story about why seeing greenness is like what it’s like. But it’s far from clear that this line of thought works. For just as it’s not clear why perceptually representing a property would be sufficient for being acquainted with its nature, it’s also not clear why perceiving an instance of a property would be sufficient for acquaintance with its nature, either. Indeed, debates over the metaphysics of properties traditionally categorised as “secondary” (e.g., colour, sound, taste, smell) give us good reason to doubt that perceiving an instance of a property is sufficient for acquaintance with its nature. Let’s stick with colour for simplicity’s sake. On one view of the metaphysics of colour—physicalism—colours are properties like disjunctions of surface spectral reflectances (see, e.g., Byrne and Hilbert 2003). If physicalism about colour is correct, the phenomenal character of colour experience leaves us completely in the dark about this (so to speak). Phenomenal greenness does not inherit its nature from greenness as a physicalist about colour would construe it. Arguably, the only metaphysics of colour that stands a chance of vindicating the claim that perceiving an instance of a colour acquaints us with its nature is realist primitivism. Primitivism is the view that colours are primitive, sui generis properties—they cannot be reduced to surface spectral reflectances or dispositions to cause experiences of certain kinds. Realist primitivism is the view that such properties are actually instantiated (see, e.g., Campbell 1993); whereas eliminativist primitivism holds that colours are primitive, sui generis properties that nothing actually has (Pautz 2006). One can’t perceive an instance of a colour in the first place unless it’s actually instantiated; so one can’t be acquainted with the nature of a colour by perceiving an instance of it unless colours are actually instantiated. Now, we must ask: how exactly would perceiving an instance of colour acquaint us with its nature? Presumably, such acquaintance would be afforded in virtue of the fact that the nature of colour is reflected in the phenomenal character of colour experience to a significant extent. It seems that only primitivism can satisfy this condition, as rival metaphysics of colour go substantially beyond anything we could glean from the phenomenal character of colour experience.12 Hence, the claim that perceiving an instance of a colour acquaints us with its nature entails realist primitivism. To summarise—the Extended View can explain why (e.g.) it’s like this to see greenness only if perceiving an instance of greenness is sufficient for acquaintance with its nature, and this is so only if realist primitivism is true.13 So this way of making good on the hunch that perceptual experience isn’t just representational depends on the truth of a controversial metaphysics of colour. I’m not inclined to adopt realist primitivism, but getting into that debate would take us too far afield. Let us move on, in hopes of identifying another way of making good on the hunch that doesn’t require such a contentious metaphysical commitment.

Perceptual Experiences

•

49

3. Phenomenal Character as “Revelatory” Although I’m not on board with realist primitivism, I think there’s something to the idea we ended up with in the last section—namely, that perceptual phenomenal character is in some sense revelatory of the things around us. In this section, I will detach this idea from realist primitivism and assess whether it enables us to make good on the hunch that perceptual experience isn’t just representational. The idea, somewhat less roughly stated, is that perceptual phenomenal character reveals at least something about what the things we experience are like in themselves, independently of our experiences of them (Logue 2012; see also Campbell 2002 and Johnston 2006). I will use the phrase “the intrinsic nature of F-ness” to refer to what things with the property of F-ness are like in themselves (at least as far as being F is concerned). So, for example, the phenomenal character associated with experiences of, say, crescent-shapedness gives us some insight into what crescent-shaped things are like in themselves. If the reader finds this idea totally obscure, it may be helpful to compare what we can learn from perceptual experience to what we can learn from testimony (Logue 2012, 227–228). If I have my eyes closed, and you tell me that there is a crescent-shaped thing in front of me, it seems like I’m missing out on something that I would have gotten simply by seeing it for myself. What’s missing isn’t the information that there is a crescent-shaped thing before me— provided that you’re a reliable source, I have that information in both cases. A plausible candidate for what I’m missing out on is some insight into what crescent-shaped things are like, independently of our experiences of them— that is, some insight into the intrinsic nature of crescent-shapedness.14 In the hope that this idea is at least tolerably clear, let us consider why one might think that the Representational View cannot accommodate it. If perceptual phenomenal character reveals the intrinsic natures of things around us, we must ask: how does it do that? In other words, what is the mechanism of this revelation? The only answer I can think of appeals to resemblance: one has perceptual insight into the intrinsic nature of F-ness to the extent that phenomenal F-ness resembles F-ness itself (cf. Campbell 2002, 155–156).15 So, for example, one has perceptual insight into the intrinsic nature of crescentshapedness to the extent that phenomenal crescent-shapedness resembles crescent-shapedness itself. Now, it’s not clear how the Representational View can make sense of such a resemblance. According to that view, phenomenal crescent-shapedness consists in perceptually representing crescent-shapedness; plausibly, such representations are realised in the brain. But representations need not resemble what they represent in general, and neurally realised perceptual representations certainly do not. A neurally realised perceptual representation of crescent-shapedness does not resemble crescent-shapedness in any significant

50

•

Heather Logue

respect.16 A proponent of the Representational View might insist that the format of perceptual representation is such that it does resemble what it represents (perhaps this is what is supposed to make it different from other forms of mental representation). However, this cannot simply be stipulated—the representationalist must explain how a perceptual representation can resemble what it represents, given how the representation is supposed to be realised (presumably neurally). In short, given that the Representational View cannot accommodate the idea that phenomenal F-ness resembles F-ness, it does not have the resources to explain how phenomenal F-ness could reveal the intrinsic nature of F-ness.17 By contrast, the Extended View can make sense of the required resemblance. According to the Extended View, phenomenal F-ness has an instance of F-ness as a constituent. For example, the phenomenal character of my experience of crescent-shapedness consists in my perceiving an instance of crescent-shapedness—the instance of crescent-shapedness is literally part of the phenomenal character. Given this theory, the idea that phenomenal F-ness can resemble an instance of F-ness is just as intelligible as the claim that a whole can resemble a part of it (Campbell 2002, 155–156). Of course, not all wholes resemble their parts. The point is just that the claim that a whole in fact does resemble its parts makes sense. And the claim that there is such a resemblance in the case at issue seems all the more plausible if it turns out that there’s not much more to phenomenal crescent-shapedness than an instance of crescent-shapedness. In this way, the Extended View can make sense of the prima facie unintelligible claim that phenomenal character can resemble mind-independent entities and thereby explain perceptual insight into the intrinsic nature of F-ness via the mechanism of similarity. At this point, one might think that this line of thought cannot avoid a commitment to realist primitivist views of properties like colour. (There’s a reason why I’ve been sticking to a shape example so far in this section.) For basically the same reasons given in the previous section, phenomenal greenness affords insight into the intrinsic nature of greenness only if primitivism about colour is true. Primitivism is the only metaphysics of colour on which phenomenal character of colour experience even remotely resembles the intrinsic nature of colours themselves. Hence, if phenomenal greenness is to afford insight into the intrinsic nature of greenness, and given that resemblance is the mechanism of such insight, then primitivism must be true. While this is correct, a proponent of the Extended View need not say that phenomenal F-ness affords such insight for all possible substitutions of “F”. In particular, she can say that some types of phenomenal character are less revelatory than others (e.g., that colour phenomenology is less revelatory than shape phenomenology). The suggestion earlier was that one has perceptual insight into the intrinsic nature of F-ness to the extent that phenomenal F-ness resembles F-ness itself. It is compatible with this claim that the extent

Perceptual Experiences

•

51

to which phenomenal F-ness resembles F-ness itself is extremely minimal in some cases. So even if primitivism is false, we can still insist on the need to capture the fact that perceptual phenomenal character is partially revelatory (e.g., perhaps it reveals quite a bit of the intrinsic nature of shapes, and little to nothing about intrinsic nature of colours). The variation in resemblance is a function of how much work features of the subject do in determining phenomenal character; roughly, the more work features of the subject do in determining phenomenal F-ness, the less phenomenal F-ness resembles F-ness (see Logue 2012, sec. III). In his contribution to this volume, Pautz gives good reasons to think that features of the subject are doing quite a bit of the work in determining the phenomenal character of experiences of colour, smell, and sound (pp. 25–27).18 However, this modification to the argument for the Extended View seems to concede too much. The significance of the idea that perceptual phenomenal character is revelatory is considerably diminished if it excludes all the properties traditionally thought of as “secondary”. If the phenomenology of colour, smell, taste (etc.) experiences doesn’t afford any insight into the intrinsic natures of colours, smells, tastes (etc.), then it isn’t as revelatory as we might have initially thought. And it’s a slippery slope from this concession to doubting the watered-down version of the idea—if we concede that perceptual phenomenal character often isn’t revelatory, it’s natural to wonder whether it’s ever revelatory. Indeed, there is an illustrious Kantian tradition according to which we cannot have any knowledge of things as they are in themselves, independently of us (Langton 1998). My hunch is that this Kantian line simply presupposes the falsity of the Extended View. Nevertheless, it’s far from clear how to support the opposite starting point—i.e., how to argue that perceptual phenomenal character is at least partially revelatory of things as they are in themselves. So at best, we’re left with an impasse regarding the starting point of the argument for the Extended View over the Representational View.

4. Phenomenal Character as Potentially Revelatory Even if perceptual phenomenal character isn’t actually revelatory of the intrinsic natures of things, it still seems like it could have been. I take it that this is what is going on in a thought experiment due to David Chalmers: In the Garden of Eden, we had unmediated contact with the world. We were directly acquainted with objects in the world and with their properties. Objects were simply presented to us without causal mediation, and properties were revealed to us in their true intrinsic glory. When an apple in Eden looked red to us, the apple was gloriously, perfectly, and primitively red. There was no need for a long causal

52

•

Heather Logue

chain from the microphysics of the surface through air and brain to a contingently connected visual experience. Rather, the perfect redness of the apple was simply revealed to us. The qualitative redness in our experience derived entirely from the presentation of perfect redness in the world. (2006, 49, emphasis mine) Eden is a possible world in which the intrinsic natures of the properties experienced perfectly reflect the phenomenal characters of experiences of them. Accordingly, perfect redness is a property that perfectly resembles phenomenal redness.19 And given that resemblance is the mechanism of revelation (as suggested above) this resemblance enables phenomenal redness to reveal the intrinsic nature of perfect redness. As I argued in the previous section, only the Extended View can make sense of resemblance between mental states (like phenomenal redness) and mind-independent entities (like perfect redness). It seems that Chalmers accepts this point, as he acknowledges that “. . . in Eden, if not in our world, perceptual experience extends outside the head” (Chalmers 2006, 78). But if an extended theory of perceptual phenomenal character is true in Eden, then (arguably) it is true in the actual world. Actual phenomenal redness resembles a (non-actually instantiated) mind-independent property (perfect redness). Again, this resemblance is intelligible only if the Extended View is true: actual phenomenal redness can resemble perfect redness only if it is the sort of entity that could have perfect redness as a part.20 Of course, it doesn’t actually have perfect redness as a part. It has “imperfect” redness as a part instead—for example, as Chalmers suggests, redness as the physicalist about colour would construe it (2006, 72). But the point is just that it must be possible for it to have a mind-independent entity like perfect redness as a part, and this is the case only if the Extended View is true. In other words, the import of the Eden scenario may be greater than Chalmers acknowledges. The Extended View isn’t just true in Eden; if we are to make sense of the “trans-world” resemblance between perfect redness and actual phenomenal redness, we must conclude that even actual phenomenal redness has mindindependent entities as constituents (i.e., that the Extended View is true in the actual world as well). In short, if perceptual phenomenal character is even potentially revelatory of how things are in themselves, then it must be extended, rather than representational. Obviously, this case was presented in brief, broad strokes, and some important details remain to be filled in (see Logue ms. for an attempt to do so).21 But I think some refinement of this argument could eventually vindicate the hunch that representational accounts of perceptual phenomenal character leave out something important: namely, its capacity to reveal what our surroundings are like independently of us.

Perceptual Experiences

•

53

5. Conclusion It may be helpful to conclude by highlighting exactly what Adam Pautz and I are disagreeing about in our contributions to this volume, in light of the fact that we agree about quite a lot. In particular, I couldn’t agree more that we need “a middle way between the extremes of naïve realism and the inner state view” (p. 34). I don’t think that we can account for phenomenal character just by appealing to intrinsic features of the subject of experience (as the inner state view does); but I don’t think that appealing almost entirely to features of the objects of experience (as the more extreme forms of the Extended View do) will yield a satisfactory account, either. Pautz and I disagree about the precise form the middle way should take. He thinks that it will involve accounting for phenomenal character solely in terms of perceptual representation; whereas I’m sceptical about this endeavour, and I suspect that we’ll need to bring the perceptual relation into the story in order to explain everything that needs explaining. In this paper, I’ve attempted to pin down the source of my scepticism and suspicion. Although it’s not yet maximally clear, I hope the reader finds that some progress has been made in that direction.22

Notes 1. Of course, there are other constraints on a metaphysics of perceptual experience (e.g., stemming from the epistemological role of experience, and the relationship between perception and action). But for reasons of space, I will set these aside. 2. Given the differences between experience and belief, no doubt experiential and doxastic representation differ in significant respects. But the idea is that they are similar in that both are propositional attitudes (in a suitably undemanding sense of “propositional”). 3. Sometimes this label is used to refer to specific versions of the view on which the phenomenal character of an experience supervenes on its representational content (see, e.g., Byrne 2001). But note that this is not the only way to account for perceptual phenomenal character in terms of representation. For instance, one might claim that it is a function of both content and the subject’s attitude to it, such that (e.g.) visually representing that p has a different phenomenal character than tactilely representing that p (cf. the “impure representationalism” defended in Chalmers 2004). 4. Proponents of the Extended View disagree about how to account for illusions, which are (as a first pass) perceptual experiences in which the subject perceives things in her environment, but they perceptually appear to be a way they’re not. Some extend the Extended View to illusions (Brewer 2008, Fish 2009, chap. 6), whereas others propose to account for them in the same way as hallucinations (Campbell 2002, 117; Martin 2006, 360). 5. One might reject the assumption that hallucinations cannot be extended (e.g., Johnston 2004), but this is a minority view. 6. For arguments for and against disjunctivism, see the papers in Haddock and Macpherson (2008) and Byrne and Logue (2009). 7. If this characterisation is to be generalised to other determinates of perceiving (hearing, touching, etc.), then “looking” should presumably be replaced with something along the lines of “directing the relevant sense organ(s)”. 8. At least in this context; strictly speaking, perceiving might be a determinate of a determinable acquaintance relation.

54

•

Heather Logue

9. Pautz agrees with Fish that “there is no gap between the mental condition of being directly acquainted with the redness of something and the mental condition having a ‘reddish’ experience” (Pautz 2013, 28). (By “reddish experience”, I take it that Pautz means an experience that has the phenomenal character actually associated with one’s experiences of redness.) But the questions I’m about to pose in the main text suggest that there is such a gap after all. 10. The reader shouldn’t take this talk to presuppose the falsity of property nominalism; as I hope the initial gloss indicates, the notion I have in mind is something that a nominalist could tolerate. The task of clarifying precisely what’s meant by talk of what it is for something count as an F thing/the nature of F-ness is important, but one I don’t have the space to take on properly here. Suffice it to say that it seems that something along the lines of this idea figures in the debate under discussion, and hopefully it’s clear enough for the fairly broadbrush purposes of this paper. 11. I should emphasise that Fish doesn’t explicitly make this move. I’m making it on his behalf in response to the objection just raised in the previous paragraph; although I believe that he would be sympathetic (as I don’t see any other way of cashing out the phrase just quoted). 12. Primitivism arguably goes beyond what we could glean from colour phenomenology, too, and hence isn’t compatible with the claim that the nature of colour is entirely revealed in experience (Allen 2011). But the point here is that it is the only view on which the nature of colour is reflected in colour phenomenology to a significant extent. 13. It should be noted that there is still an “objective” explanatory gap between primitivist colours and, e.g., surface spectral reflectances (Pautz 2013, 28). A related issue is raised by Pautz in his contribution to this volume, regarding the irregularity of the grounding of the primitivist properties in the microphysical ones (see pp. 4–5). 14. This point is in the vicinity of the initial premise of the knowledge argument against physicalism (Jackson 1982), but note that it is a stronger claim. If I get the information that there’s a crescent-shaped thing before me by testimony rather than perception, I’m not just missing out on knowledge of what it’s like to experience crescent-shapedness (cf. Jackson’s claim that Mary the colour scientist is missing out on phenomenal knowledge). It seems (to me, at least) that I’d also be missing out on knowledge concerning the crescent-shaped thing before me—and my hypothesis is that what I’d be missing out on is knowledge of the intrinsic nature of crescent-shapedness. 15. This claim presupposes that a mental property (perceptual phenomenal character) could resemble a non-mental one (e.g., shape). This presupposition is widely regarded as nonsensical, but that’s arguably just because it is unintelligible according to the traditional theories of perceptual phenomenal character. As I will explain shortly, we can make sense of it on the Extended View. 16. A representationalist might appeal to retinotopic representations in early vision in hopes of securing the required resemblance; the idea is that there really is a visual representation of crescent-shapedness that resembles crescent-shapedness after all. (Thanks to Bence Nanay for this suggestion.) However, I understand the Representational View as claiming that perceptual phenomenal character is grounded in personal-level perceptual representations (i.e., states of the subject) rather than subpersonal information processing states of the sort involved in early vision. So even if there are representations that resemble what they represent in early vision, this does not secure phenomenal resemblance (i.e., between the personallevel representation that constitutes phenomenal F-ness and F-ness). 17. One might suggest that while the vehicle of representation doesn’t resemble what is represented, the content of the representation does—indeed, the content of the representation just is what is represented (and hence trivially resembles what is represented in virtue of being identical to it). However, as I understand the Representational View, perceptual phenomenal character is supposed to consist in representing the content, not (just) in the content represented. Even if phenomenal character supervenes on representational content, we don’t have the former unless the subject represents the latter. So it’s resemblance between the vehicle of the representation and what is represented that is at issue here.

Perceptual Experiences

•

55

18. Pautz objects to my way of reconciling this fact with the Extended View (n. 3). However, his objection assumes that the phenomenal character of experience supervenes on the properties things appear to have; but I think that this is an assumption we ought to reject (see Logue 2012, 218–222 for discussion relevant to this point). 19. Well, perfect redness almost perfectly resembles phenomenal redness, in that perfect redness is a property of mind-independent objects, whereas phenomenal redness is a property of mind-dependent entities (i.e., experiences). If the Extended View is correct, then experiences have mind-independent entities as constituents; but experiences as a whole are minddependent (since experiences require subjects). 20. One might suggest that the Representational View can capture the idea that phenomenal redness has perfect redness as a part, by claiming that it consists in representing perfect redness. But the sense in which the representational content of an experience is “part” of the experience is not a sense that can secure revelation by resemblance. The representational content is what is represented, and perceptual phenomenal character is supposed to consist in representing it. And whatever brain activity counts as representing perfect redness, it does not resemble perfect redness. 21. For example, this line of argument assumes that the Eden scenario is possible, which is not beyond question. I can’t address this issue fully here, but suffice it to say the burden is arguably on my opponent to give a reason for thinking that it isn’t possible, and I have yet to hear one that I find compelling. 22. Thanks to Bence Nanay and audiences at Leeds, Manchester, and Oberlin for helpful comments on previous drafts of this paper.

References Allen, Keith. 2011. “Revelation and the Nature of Colour.” Dialectica 65: 153–76. Brewer, Bill. 2008. “How to Account for Illusion.” In Disjunctivism: Perception, Action, Knowledge, edited by A. Haddock and F. Macpherson, 168–80. Oxford: Oxford University Press. ———. 2011. Perception and Its Objects. Oxford: Oxford University Press. Byrne, Alex. 2001. “Intentionalism Defended.” Philosophical Review 110: 119–240. Byrne, Alex, and David Hilbert. 2003. “Color Realism and Color Science.” Behavioral and Brain Sciences 26: 3–21. Byrne, Alex, and Heather Logue. 2009. Disjunctivism: Contemporary Readings. Cambridge, MA: MIT Press. Campbell, John. 1993. “A Simple View of Colour.” In Reality, Representation, and Projection, edited by J. Haldane and C. Wright, 257–68. New York: Oxford University Press. ———. 2002. Reference and Consciousness. Oxford: Oxford University Press. Chalmers, David. 2004. “The Representational Character of Experience.” In The Future for Philosophy, edited by B. Leiter, 153–81. Oxford: Oxford University Press. ———. 2006. “Perception and the Fall from Eden.” In Perceptual Experience, edited by T. Gendler and J. Hawthorne, 49–125. Oxford: Oxford University Press. Clark, Andy, and David Chalmers. 1998. “The Extended Mind.” Analysis 58: 7–19. Fish, William. 2009. Perception, Hallucination, and Illusion. Oxford: Oxford University Press. Haddock, Adrian, and Fiona Macpherson. 2008. Disjunctivism: Perception, Action, Knowledge. Oxford: Oxford University Press. Jackson, Frank. 1982. “Epiphenomenal Qualia.” The Philosophical Quarterly 32: 127–36. Johnston, Mark. 1992. “How to Speak of the Colors.” Philosophical Studies 68: 221–63. ———. 2004. “The Obscure Object of Hallucination.” Philosophical Studies 120: 113–83. ———. 2006. “Better than Mere Knowledge? The Function of Sensory Awareness.” In Perceptual Experience, edited by T. Gendler and J. Hawthorne, 260–90. Oxford: Oxford University Press. Langton, Rae. 1998. Kantian Humility. Oxford: Oxford University Press.

56

•

Heather Logue

Levine, Joseph. 1983. “Materialism and Qualia: The Explanatory Gap.” Pacific Philosophical Quarterly 64: 354–61. Logue, Heather. Unpublished Manuscript. “Extended Phenomenal Character.” ———. 2012. “Why Naive Realism?” Proceedings of the Aristotelian Society 112: 211–37. ———. 2014. “Experiential Content and Naive Realism: A Reconciliation.” In Does Perception Have Content?, edited by Berit Brogaard, 220–41. Oxford: Oxford University Press. Martin, M. G. F. 2004. “The Limits of Self-Awareness.” Philosophical Studies 120: 37–89. ———. 2006. “On Being Alienated.” In Perceptual Experience, edited by T. Gendler and J. Hawthorne, 354–410. Oxford: Oxford University Press. Nagel, Thomas. 1974. “What Is It like to Be a Bat?” Philosophical Review 83: 435–50. Pautz, Adam. 2006. “Can the Physicalist Explain Colour Structure in Terms of Colour Experience?” Australasian Journal of Philosophy 84: 535–64. ———. 2013. “Do the Benefits of Naive Realism Outweigh the Costs? Comments on Fish.” Philosophical Studies 163: 25–36.

PART

II

Is Perception Thin or Rich?

CHAPTER

4

Rich or Thin?*

SUSANNA SIEGEL AND ALEX BYRNE

Part 1: Siegel We see things. When we see things, they look a certain way. The conscious aspects of seeing include things’ looking to us the way they do. These plain observations make it natural to ask how things may look to us in conscious perception, when we see them. Perception is often dense with social information. Headed for the top floor, you wait for the elevator. The door opens. Someone else is in it already. At the moment he sees you, fear and tension grip his body. Your presence makes him uncomfortable. His discomfort is palpable. He clutches his belongings. He won’t meet your eyes. You realize that in his mind, it’s a live possibility that you—a stranger to him in a small public space—might do something awful during the brief elevator ride. What sets off these micro-interactions is visual perception. The man in the elevator first saw you as you stepped in to join him. But when he saw you, what did he see? One way to make this question more exact is to ask which properties can be presented to you in your visual experience. Which properties characterize the ways things visually appear to you, when you see them? We can draw an initial division by putting colors, textures, spatial relations, shapes (including volumes), luminance, and motion on a list. The division helps define two opposing answers to our question. According to the Thin View, the only properties presented in experience are limited to the ones on the list, which we can call “thin properties”. There are 59

60

•

Susanna Siegel and Alex Byrne

no thin properties other than the ones listed. According to the Rich View, the properties presented in experience can include properties that are not on the list. We can call non-thin properties “rich properties”. The list that defines these two positions is somewhat arbitrary. But it is motivated by three related ideas. First, the properties on the list are computed by early vision in both humans and other animals.1 Since the visual system is typically sensitive to these properties, they plausibly end up characterizing at least some of the conscious aspects of vision. Second, if one comes to perceive properties left off the list, plausibly one does so in part by perceiving the properties on the list. For instance, if watching a softball game leads one to think that the batter’s swing sent the ball flying, one attributes causal efficacy to the bat (and to the batter) in part by seeing their position relative to the ball, their shape, and one saw those things in part by seeing the illumination and color contrast of those items. If color and illumination were undifferentiated from their surroundings, one would not be able to see them individually at all. If any properties are presented in experience, then presumably the thin properties are. Finally, the great majority of properties that can figure in instantaneous impressions formed in the course of perception are left off the list. Think of causal properties, kinds, affordances, traits, appraisals. All of these properties figure in your instant impression of the elevator scene, and none of them are on the list. The list therefore lets us probe the relationship between visual experiences and instantaneous impressions. There is another advantage to defining the debate by using a list of properties. When the Thin View is defined by a list, it is kept distinct from the position that the only properties presented in experience are those that are detected by the visual system. (At a finer level of grain, we can distinguish early vision from late vision, and, at a coarser level, we can ask which properties are detected by perception in any combination of sensory modalities). We don’t need to make a list of properties to state these positions. And the properties detected by the visual system could conceivably include rich properties—such as the property of being a face, or the property of expressing fear, or causal properties. It is therefore a different debate. Thin vs. Rich: Can rich properties be presented in experience? Intra- vs. Extra-visual: Can properties that are not detected by the visual system alone be presented in visual experience? These two debates are in turn distinct from a neighboring question: in the human mind, where are the boundaries, if there are any, between perceptual and non-perceptual processing? The current exchange concerns the first debate, between the Thin View and the Rich View. Like any debate, this one rests on starting assumptions. It takes

Rich or Thin?

•

61

as understood the category of visual experience, construes it as a phenomenological category, and goes on to ask which properties can be presented in that type of experience. The two positions in this debate are not the only options. Maybe no properties are presented in experience. Or maybe only some of the thin properties are.2 Or, finally, maybe nothing is denoted by the category of visual experience, so the debate between Thin and Rich Views gets no traction. But the starting assumptions of the debate are plausible: there are conscious aspects of seeing, and, when we see things, they look a certain way. The ways they look are most naturally characterized by talking about the properties that things look to have. According to the Thin View, when the elevator opens and the man inside first sets his eyes on you, the changes in his visual experience are limited to changes in thin properties. And when you, in turn, see the fear and tension grip his body, you are not visually presented with the properties of being afraid or tense. The Thin View entails that in any visual experience, no matter what external scenes one is seeing, one is only ever presented with the highly circumscribed set of properties on the list. Each party in the elevator takes in social complexities of the situation by visual perception. An initial motivation for the Rich View is that it easily respects the central role that visual experience plays in the overall experience of the people in the elevator. The Rich View allows that the man in the elevator, when he sees you waiting for the elevator in the hallway, is presented in his visual experience with other properties, including the property of being a person, a hallway, an entering, or an exit. It allows that visual experience can attribute psychological properties to the people one sees, including traits such as being kind, welcoming, or aggressive; age properties such as being a child, a young adult, or a grownup; kind properties such as being a person, or a bird, or being male; emotions such as sympathy, fear, boredom, pride, or suspicion; appraisals such as disdain or approval; agential (and causal) properties such as walking intentionally, tripping accidentally, or running with effort; interpersonal relations such as understanding what a person is hearing, or failing to comprehend it; and opportunities for action and interaction, such as the affordance of friendly chit-chat, tight-lipped isolation, or cooperative inquiry. When we set out to characterize a perceptual situation from the point of view of the subject of visual experience, rich properties are the ones that come to mind. With only thin properties at our disposal, most experiences would be unrecognizable. For the moment, let’s use the locutions “looks F” and “looks to be G” in a way that doesn’t have to track exactly the properties presented in visual experience. We can leave open exactly what this use of “looks” does track. Zero in on just one rich property: being scared. When “looks” is used in this neutral way, both positions can agree that, to you, the man in the elevator looks scared. The positions will differ in the role played by visual experience of the man’s looking

62

•

Susanna Siegel and Alex Byrne

scared to you. The Rich View has a straightforward analysis of this role: your visual experience can present the property of being scared. (The Rich View doesn’t have to offer this analysis, but it can). The Thin View has to somehow bridge the gap between being the man’s looking scared to you, and being presented with thin properties. There are three strategies the Thin View can use to analyze X’s looking scared to S. First, it can try to identify a set of thin properties that is co-extensive with being scared. Presenting that set of properties (and attributing them to X) is a good thin candidate for the properties presented in S’s experience, when X looks scared to S. Second, it can try to identify a set of thin properties that X can visually appear to have, when X looks scared, and presenting any of those complexes (and attributing them to X) suffices for X to look scared to S. Third, it can hold that there is no such set of properties, and “looking scared” never occurs entirely in visual experience. Here the Thin View must use other resources to analyze what it is for X to look scared to S. The third option, however, runs up against the plausibility of the idea that for the case of looking scared, as for many other rich properties, looking scared seems to occur entirely in visual experience. A second motivation for the Rich View is that for many rich properties F, the Thin View seems to lack resources to analyze cases in which X looks F to S. Consider the property that a loaf of bread has of being sliced by a knife. The knife cuts the bread. Cutting is a causal property. The knife also moves back and forth, as a piece of bread gradually separates from the rest of the loaf. These are (thin) spatio-temporal properties. If only thin properties were presented in the visual experience of seeing the bread get cut by the knife, then for all your visual experience tells you, the bread’s separation may be making the knife move. Pressure is also a causal property. If only thin properties were presented in the visual experience of seeing a knife slicing bread, your visual experience should leave it open whether the knife is exerting pressure on the bread. It is clear that the knife looks to be cutting the bread, and so the experience as a whole is not neutral on whether causation is occurring. Perhaps, someplace in the mind, there’s a representation of the occurrence that is neutral on whether the knife is cutting the bread (or even just exerting pressure on the bread). The Thin View is committed to more than this. It’s committed to finding a conscious experience in the mix that is neutral in just this way. To avoid positing the rich properties of causation and exerting pressure as presented in visual experience, what if the Thin View divides visual perceptual experience from amodal, non-perceptual experience? This move would make room for the idea that we experience causation, without holding that we experience it visually—even when no other sense modality besides vision is involved. But there is no obvious principled division here. We know that there are perceptual experiences, such as conscious vision. Are there also amodal

Rich or Thin?

•

63

experiences, distinct from judgment, with all the phenomenological hallmarks of perceptual experience, yet which aren’t perceptual? This suggestion sounds like a distinction without a difference.

Part 1: Byrne Suppose you are in Susanna’s office and see a Granny Smith on her desk. You form some beliefs: that it is green and round, that it is an apple, that it is edible, and that Susanna likes apples. Suddenly a small piece of the apple, about the size of a pencil eraser, is extruded and falls on the floor, pushed by a caterpillar crawling out of the newly made hole. You form the belief that the caterpillar caused the piece of apple to fall and that the caterpillar is alive. You believe, then, these propositions: (A) (B) (C) (D) (E) (F)

That it (the apple) is green and round. That it is an apple. That it is edible. That Susanna likes apples. That this (the caterpillar) caused that (the piece of apple) to fall. That this (the caterpillar) is alive.

Further, your beliefs are perceptual, in an intuitive sense. You believe propositions (A)–(F) more-or-less immediately because of what you see—if your eyes had been closed you would not have believed any of them. One way of putting the issue before us concerns the content of perceptual experience; another—closely related—way concerns the properties presented in perceptual experience. But what does this jargon mean? The “content” of a perceptual experience is often explained in these terms: it is a proposition (or set of propositions) specifying the conditions under which the experience is veridical.3 A property “presented in” perceptual experience is then a property attributed by the content of that experience, in the sense in which (C) attributes edibility, for example. Although in her opening contribution, Siegel prefers to speak of properties “presented in experience”, she is also happy talking of the “content of experience” (Siegel 2010), so let us stick with that for the next few paragraphs. Siegel and I agree that (A) is part of the content of your experience—if the apple is not green, or not round, then your experience is not veridical. We also agree that (D) isn’t part of the content of your experience—if it turned out that Susanna did not, after all, like apples, the only error or mistake would lie in your perceptual belief. However, everything else is in dispute. According to Siegel, the content of your experience includes (or, at least, could include) the other items on the list: (B), (C), (E), and (F). In general, she thinks that among the contents of

64

•

Susanna Siegel and Alex Byrne

experiences are propositions about kinds like apple, “affordances” like edibility, causation, and animacy. I do not. The crucial notion of the content of experience (and the derivative notion of the presentation of a property) was explained in terms of an experience’s being “veridical”. (Alternatively, we could have spoken of an experience’s being “accurate” or “correct”.)4 But one might well complain that this is a case of obscurum per obscurius: We do not naturally talk about visual experiences as being veridical or non-veridical, or accurate or inaccurate, or correct or incorrect. I very much doubt that the woman in the street would understand what I mean, were I to ask whether her current visual experience is veridical or non-veridical, or accurate or inaccurate, or correct or incorrect, at least not without some guess work. (Breckenridge 2007: 117) Must do better, then. Let us see how Siegel explains the question up for debate. Siegel informally introduces the question using an example of a sinister elevator encounter: The man in the elevator first saw you as you stepped in to join him. But when he saw you, what did he see? (p. 59) She then tries to make this question “more exact”, but it’s worth pausing to note why the extra effort is needed. Confronting the discomfited stranger in the elevator, you (a philosopher, we may presume) are visibly nervous. This is because you are taking the elevator to the ground floor of the depressing APA hotel to go to the book exhibit, where you plan to kill your publisher. The stranger takes note of your twitching and sees that you are nervous. He has no idea that you are going to the book exhibit, so he does not see that you are going to the book exhibit. Nonetheless, because that is where you are going, and he sees you, he sees you going to the book exhibit. (Later, when he testifies in court at your trial, he will say, “Yes, your honor, I did see her going to the book exhibit, although I didn’t realize it at the time”.)5 So the question “What did the man in the elevator see?” has multiple answers: he saw that you were nervous; he saw you going to the book exhibit; he saw a philosopher with murder in her heart; etc. Further, none of these answers is (or need be) controversial. Whatever the question may be, to which Siegel and I give opposing answers, “What did he see?” is not it. As mentioned, Siegel’s preferred way of putting the question du jour is (quite generally, leaving the man in the elevator behind): “which properties can be presented to you in your visual experience?” Because the notion of

Rich or Thin?

•

65

“presentation” requires explanation, this is immediately glossed as “Which properties characterize the ways things visually appear to you, when you see them?” (p. 59). Equivalently, as suggested by her first paragraph: how do objects look? To the man in the elevator, you look scary. Thus, one way objects (humans, for instance) can look is scary. “Way” talk, as in “the way she looks”, is property talk—the ways are specified by adjectives: scary, nervous, middle-aged, wealthy, Scandinavian. So the question “How did you look to the man in the elevator?” (i.e., “What way did you look . . . ?”) has multiple answers: you looked scary, nervous, and (suppose) middle-aged, and Scandinavian. Further, none of these answers is (or need be) controversial. Whatever the question may be, to which Siegel and I give opposing answers, “How did you look to him?” is not it either. It is sometimes suggested that different “senses” of “looks” can come to the rescue here, but standard tests for ambiguity indicate that this is misguided. To the man in the elevator you simply look scary and (say) pale and small; there’s no sense in which you look pale and small but not scary.6 So if ordinary perceptual verbs are all we have to go on, the issue between us remains obscure. Do we have to lapse back into reliance on the jargon of “veridicality”? Well, sort of, but the situation is not quite as bad as Breckenridge, quoted above, makes out: In Philosophy we often talk of sense experiences being either veridical or non-veridical. These are technical terms, but it is clear from the ease with which beginning students can learn to apply them that they answer to a genuine distinction. (Pendlebury 1990: 224) Pendlebury’s point can be strengthened by noting that talk of “visual illusions” is reasonably familiar to the woman in the street. Striking examples of such are internet fodder and demonstrate how the term can readily be applied to new cases. And once the notion of an illusion is found acceptable, the complementary notion of a veridical experience has to be in good order too.7 Why is this folk-category important? Because it is closely linked to a widespread view in the scientific study of perception, that perceptual modalities like vision are modular in the sense of Fodor (1983). The (early) visual system’s operation is mandatory (you see as two unequal lines whether you like it or not), fast, and informationally encapsulated (in particular, believing that the two lines are unequal does not affect the appearance of unequal lines).8 That modular architecture and the phenomenon of illusions fit together nicely: illusions are precisely cases where the output of the visual system fails to match reality. The present dispute is, I suggest, best understood in these terms. Assuming that there is a visual module (perhaps one less encapsulated

66

•

Susanna Siegel and Alex Byrne

that Fodor himself allows), we can locate the dispute at the level of its outputs. Do they concern kinds, affordances, causation, or animacy? And since the outputs are pieces of information (or misinformation), we may take them to be propositions. To return to the opening example, can the outputs of the visual module include propositions like (B), (C), (E), or (F)? As desired, there is plenty of fruitful controversy. Fodor, for example, aligns himself partly with Siegel, claiming that the outputs of the visual module include “basic perceptual categories” like dog and apple (94–7). But he disagrees with her in excluding subordinate categories like French poodle and Granny Smith. Siegel and I may agree on this much: if the phenomena can be explained without imputing a certain kind of rich content to the output of the visual module, then there is no reason to impute it. The debate turns, then, on how much work thin outputs can do.

Part 2: Siegel If you asked someone on the street what kind of life promotes human flourishing, she would probably not use the Greek word eudaimonia in her answer. Few English speakers possess this concept. It would be absurd to conclude from this observation that when Aristotle discussed this topic, he was on the wrong track. Philosophical horizons aren’t set by English. They’re set by the problems that life and abstracta (such as numbers) can pose. If you asked the man in the elevator to describe his perceptual experience, he almost certainly wouldn’t use the terms “contents”, “properties presented in experience”, “veridical”, “accuracy condition”, or even “visual experience” in his answer. He might even wonder what you’re asking. Most people are unfamiliar with the philosophical discussions of perception that use these terms. They’re technical terms. Byrne warns us to define these bits of philosophical jargon. Then he goes farther and suggests that it isn’t obvious how to define them—and so the issue between the Rich and the Thin View should be reframed as a question about what the deliverances are of modular vision. Not all uses of philosophical jargon are gratuitous. The problems of perception and intentionality are cases in point. Ponce de Leon is looking for the Fountain of Youth. How could his thoughts be coherent, given that there is no Fountain of Youth? Anscombe (1965) pointed out that an analogous question arises in perception. When the elevator man takes you to be threatening upon believing his eyes, he hallucinates or illudes a threatening demeanor. If we want to characterize his visual consciousness, we need to advert to something other than you and the properties you actually have. We cannot read off his perspective from the objects and properties in the world. A partial exception might be the relational property you have of producing

Rich or Thin?

•

67

his experience. But his experience is part of his perspective, and the relational properties take that perspective as understood, rather than providing a way to characterize it. Anscombe saw that theoretical resources beyond ordinary uses of English (and probably any other language) were needed to describe the subject’s perspective in these cases. She thought the notion of an intentional object would help, because those objects had no reality beyond the subject’s perspective. And it does help, up to a point. It gives us a vocabulary to indicate when we’re describing the contours of consciousness. But because of its orientation to objects—one sees an F, one looks for the Fountain of Youth—obscurities remain. If you’re hallucinating a pink elephant and I am too, do our visual experiences have the same intentional objects? What if your elephant is sitting but mine is standing? If you think the Fountain of Youth is in Florida but I think it’s in Alabama, are we looking for the same thing? Intentional objects are supposed to characterize how things are from the subject’s perspective in thought or perception, but objectual locutions barely scratch the surface. It helps to focus on properties rather than objects. It allows more specificity. And it allows us to label properties that don’t have labels in ordinary English. Think of the elevator man’s briefcase. There’s a configuration of its protruding surfaces that you can see from where you happen to be standing. Beyond those facing surfaces, it continues out of view. English doesn’t have a name of the shape formed by those facing surfaces, but that fact does not stop that shape from being presented to you when you look at the man’s briefcase—even if you are an English speaker. Another thing we need, if we want to characterize the perspective of perceivers, is a way to zero in on the mental state we’re asking about. “How things look to you when you see them” is just a placeholder, because the English word “look” can denote too many things. “Properties presented in experience” is better. It gives us a label for the mental state. And it gives us a tool for describing its character. The label might not be useful to Breckenridge’s friend right off the bat, when he asks her if her experience is accurate. But that’s because Breckenridge’s friend, hit with this question out of the blue, presumably isn’t already trying to zero in on the same thing as us. Once she understood that the mental phenomenon concerns a subject’s perspective that can come apart from the way the world is, it would be natural to ask whether things are the way they’re presented as being in her experience. Byrne suggests that the notion of visual illusion is tied to the thesis that early vision operates in a way that is mandatory given specific stimuli, and insulated from other psychological states, such as what the subject wants or believes. But visual illusion is a much broader category. Vision goes beyond early vision. And visual experience goes beyond visual illusions. If we worked with a definition of visual experience as an output (meeting specific conditions) of modular

68

•

Susanna Siegel and Alex Byrne

processing, it would be impossible to formulate a debate about whether every visual experience results from modular processes or not. It would also be impossible to ask whether rich properties can be presented in experience as a result of non-modular processes. It is better to work with notions of visual illusion and visual experience that leave these crucial questions open. By refining the questions “What does the elevator man see?” and “How do you look to him?” we can ask what perceptual experience contributes to our overall conscious state. There is no doubt that such contributions exist. When the Necker cube shifts, this isn’t a change in how things smell to us. It’s a change in how things look to us. There plainly is such a thing as the contours of visual consciousness. There is no need to appeal to modular processes to pin down the visual phenomena about which the Rich and the Thin Views disagree. Whether rich properties can be presented in experience is orthogonal to whether the processes that produced the experiences are modular or not. Consider perceptual learning—long-term changes in perception that result from gaining or practicing a skill. Learning to speak and understand Russian will change how the spoken language sounds to you. Learning to recognize pine trees can change how a forest containing pine trees mixed among others look to you. For cases in which a single stimulus produces different perceptual experiences before and after perceptual learning, we have a phenomenal contrast between two experiences. The phenomenal contrast gives us something to be explained. We can study the contrast by asking a series of questions. Is it a difference in which properties are presented in experience? If so, which properties differ? How these questions are answered depends on the phenomenal contrast one starts out with, and it depends which of the possible answers survive challenges of plausibility. Not every phenomenal contrast is best explained by the Rich View. Many will be explained by changes in thin properties. But to argue successfully for the Rich View, all it takes is one phenomenal contrast that the Rich View best explains. In The Contents of Visual Experience, I formulate a series of phenomenal contrasts and argue that each of them is best explained by a different instance of the Rich View: kind properties, causation, personal efficacy, and a type of mind-independence each play a role in explaining a different type of phenomenal contrast. The contrast method doesn’t care which processes produce the phenomenal contrasts. Controversies abound among psychologists over the extent to which what we want, suspect, or know (other than by perceptual learning) can alter the properties presented in perceptual experience. But these controversies usually concern the boundaries of perception rather than the boundaries of perceptual experience. We can use the method of phenomenal contrast to test the hypothesis that rich properties are presented in experience, no matter how they come to be so represented. Nothing in the method stacks the deck in favor of the Rich View.

Rich or Thin?

•

69

Part 2: Byrne In Siegel’s preferred terms, the dispute between us concerns which properties are “presented in experience”; specifically, are the presented properties rich or thin? On the negative side, I complained that explanations of the terminology of “presented property” in terms of “sees” or “looks” are inadequate, and that more was needed to clarify the dispute.9 (This was not an application of the dubious principle that “philosophical horizons are set by English” [p. 66]. Neither was it an application of the equally dubious injunction to define one’s terms.) On the positive side, I suggested that a better route to clarification is via the notion of a visual illusion, a stock category on the internet (and in vision science) that is readily illustrated by examples, and generalized to new cases. Once the notion of a visual illusion is in hand, so is the complementary notion of a veridical perception, and the content of visual experience is not far behind. And granted that experience has content, we can use that to explain the notion of a property being “presented in” experience. I also suggested that the dispute makes contact with vision science via perceptual modules, conceived along Fodorian lines. Modules (if there are such) plausibly align with illusions: an organism suffers a perceptual illusion just in case the output of a module, a piece of purported information about the organism’s environment, is misinformation. Put in other terms, an organism suffers a perceptual illusion just in case the propositional content of its experience is false. If the modularity thesis is true, then this helps us to pin down the empirical considerations relevant to assessing the Rich View. In reply, Siegel makes two points. First, she claims that “visual illusion is a much broader category” (p. 67), implying that some visual illusions are not cases where the output of the visual module is false. I disagree. Of course one cannot discover the alignment between visual illusions and false visual module output from the armchair, but successes in explaining illusions in broadly modularist terms suggest that they do not come apart. Siegel’s second point is that if “we worked with a definition of visual [illusion]” in terms of visual modules it would be “impossible to formulate a debate about whether every visual [illusion] results from modular processes or not” (p. 68),10 presumably because the answer would simply drop out from our stipulative definition of “visual illusion”. But the proposal was not to define that phrase; rather, it was to link something in the manifest image with something in the scientific image, as rusting is linked with oxidation or the tides with gravity. Once we realize that there is an intimate connection between the tides and gravity, we can bring our theory of gravity to bear on answering questions about the tides; similarly, granted an intimate connection between the visual module and illusions, we can bring our theory of the visual module to bear on the issue of presented properties (themselves explained in terms of illusion).

70

•

Susanna Siegel and Alex Byrne

Of course, although one can hardly deny that we have a visual system in some useful sense, one might well deny that there is anything like a Fodorian visual module. A large amount of research (most of it recent) seems to show that vision is highly cognitively penetrable—belief and other attitudes can influence how things look.11 (This is the return of the old “New Look” movement in psychology, the “New New Look”, as Machery [2015] calls it.) Knowledge that lips are red makes a lip-shaped cutout look redder (Delk and Fillenbaum 1965; see also Macpherson 2012). Faces with stereotypically black features look darker than faces with stereotypically white features, equated for mean luminance (Levin and Banaji 2006). Holding a long rod horizontally, thus altering one’s beliefs about how easy it is to pass through apertures, makes doorways look narrower (Stefanucci and Geuss 2009). And so on. However, this appears to be more confirmation that most published research findings are false (Ioannidis 2005).12 Gross et al. (2014) failed to replicate the Delk and Fillenbaum experiment.13 The perceptual difference between black and white faces, although perfectly genuine, turns out to be due to low-level features, not racial categories (Firestone and Scholl 2015a and b). There is no change in how apertures look, and the contrary indication is likely a task compliance effect (Firestone and Scholl 2014). The Fodorian hypothesis is holding up pretty well.14 It is now time to consider particular examples. Imagine learning to visually recognize teacups. Before, faced with a teacup, you would exclaim, “What is that?” Now you see teacups as teacups, and immediately classify them as such. Let us grant that teacups now palpably look different. (This is not obvious, but plausibly it sometimes occurs in acquired perceptual expertise.)15 There is what Siegel calls a “phenomenal contrast” between one’s perception of teacups before learning to recognize them, and one’s perception of teacups after learning to recognize them. One explanation of the phenomenal contrast is that, as a result of the learning period, the property teacuphood is presented in your experience—in other words, teacuphood figures in the content of your experience. As Siegel says, this is the explanation that she endorses for cases for this sort (Siegel 2010: ch. 4).16 A rival explanation is that a thin or “lower-level” property is presented, a complex shape/size/. . . property or a “teacup gestalt” (110–2).17 This complex property, which we can call teacuphood G, is not teacuphood. Teacups are only found embedded in the social practice of tea-drinking; a teacupG may be found in the natural environment. A rock that by chance has the shape and size of a stereotypical teacup (and so is a teacupG) is not a teacup; conversely, there may be unorthodox or damaged teacups that do not have that distinctive teacup shape. But teacuphoodG is highly correlated with teacuphood; more precisely, teacups that are visibly recognizable as such invariably have teacuphoodG, and teacuphoodG is rarely instantiated in non-teacups. In Block’s terminology, the two are recognitionally coextensive (Block 2014: 562–3).

Rich or Thin?

•

71

Agreeing with Burge, Block thinks it improbable that teacuphood is perceptually presented (560). Why does Siegel think that the rich explanation is to be preferred? Her discussion in The Contents of Visual Experience does not purport to be conclusive. In the absence of a “knock-down argument” (Siegel 2010: 111), she offers two considerations. First, “shape-gestalts that are abstract enough to remain invariant across [teacups] will be invariant across other objects as well” (111)—perhaps coffee mugs and coffee cups (cf. 112). But this seems quite doubtful, because thin properties that differentiate between teacups and coffee mugs or cups are not hard to find. (For instance, teacups have small bases, unlike mugs, and are lower and wider than coffee cups.) Second, she suggests that “the strategy of invoking the representation-invariant colorshape complexes to underpin phenomenological changes does not seem generally available”, and she gives the following example in support. One might learn to recognize a doubtful facial expression, coming to believe that a certain facial contortion “is an expression of doubt only after repeated sightings of it and interaction with the person. This change in interpretation seems to be one that could be accompanied by a phenomenological change as well” (112). Maybe the person’s face would look different, despite no corresponding change in presented thin properties. But what we need at this point is an actual demonstration, which Siegel does not supply. In favor of the alternative thin explanation, consider greebles, those vaguely elvish invented stimuli used to study object recognition.18 One can learn to recognize greebles as such; after the learning period, greebles look different, or so we may suppose. (One can learn to recognize individual greebles, as one may recognize individual faces, but to keep the parallel with teacups let us concentrate on recognizing instances of the kind.) What explains the “phenomenal contrast”? Could it be that greeblehood is not presented in the experience of the novice and is presented in the experience of the expert? That is surely a plausible explanation, but notice that greeblehood is a complex lowlevel property. To be a greeble is simply to have certain shaped parts spatially arranged in a certain way: greeblehood = greeblehoodG. By design, there is no competing candidate explanation in terms of a rich property. There is nothing to threaten the Thin View here. Now imagine that the greebles we typically encounter are in fact fungi, groobles. GrooblehoodG = greeblehood. But, just as teacuphood ≠ teacuphoodG, so grooblehood ≠ grooblehoodG (i.e., greeblehood). Some groobles are not greebles, and some greebles are not groobles. Suppose, starting as a perceptual novice (in particular, not having a recognitional ability for either greebles or groobles) you learn to recognize groobles as such; after the learning period, groobles look different. What explains the phenomenal contrast? Granted the explanation given in the previous case, the most parsimonious hypothesis is that after the learning period, greeblehood (i.e., grooblehoodG) is presented in your experience.

72

•

Susanna Siegel and Alex Byrne

Siegel might counter by adapting her example of recognizing the doubtful facial expression, mentioned three paragraphs back. As immediately above, (typical) greebles are groobles. However, you are ignorant of this fact—as far as you know, there is no high-level unity to greebles, any more than there is to cubes. You learn to recognize greebles, and so greeblehood comes to be presented in your experience. Then, you discover that (typical) greebles are fungi and gain the ability to visually recognize groobles: you effortlessly come to believe that this is a grooble, on seeing one oriented properly in good viewing conditions. If “this change in interpretation” is “accompanied by a phenomenological change as well”, then obviously the presentation of greeblehood can’t explain it, since that has remained constant throughout. So the explanation must be that grooblehood has come to be presented in your experience, vindicating the Rich View. It is here that the link with modularity is relevant. Notice that the rich explanation requires cognitive penetration. It is not the fact that you have perceptually encountered groobles that explains why grooblehood is presented; it is the fact that you know that these visually distinctive items are groobles. And since there is little reason to suppose that vision is penetrable (more cautiously, penetrable in this sort of way), the rich explanation should be rejected, along with the explanandum it was supposed to explain. The “change in interpretation” will not, after all, be accompanied by a change in how groobles look. Even if the Rich View is not true for kinds, there are other candidates. Affordances are not especially promising, because they are typically too local and transient to be attributed by the outputs of encapsulated modules.19 The visual signatures of edibility, for example, vary from place to place and season to season. Causation and animacy are much better, and will be discussed in part 3.

Part 3: Siegel Byrne thinks there’s a “stock category” of visual illusion that can help define the Rich/Thin debate. In the illusion where a straight stick looks bent, the illusion concerns a thin property, bentness. Where there is visual illusion, there is visual experience. And where there is visual experience, there are (Byrne assumes) conditions under which that experience would be veridical. If the stick immersed in water really were bent, then the experience of it as bent would be veridical. If the veridicality conditions of illusions ever include the instantiation of a rich property, then the Rich View is true. But what if the veridicality conditions of illusions only ever include thin properties? Should we then conclude that the Thin View is true? No. Examining the contents of visual illusions found on the internet or in vision science textbooks (Byrne’s sources for stock illusions) would be a poor method for deciding between Rich and Thin, because the visual experiences

Rich or Thin?

•

73

reach far beyond visual illusions and their veridical counterparts. We shouldn’t expect to resolve the debate by checking all the stock examples of visual illusions. Once we know which mental states are the visual experiences, then we’ll know which states of misinformation are visual illusions. But identifying the standard visual illusions isn’t a way to carve out the class of visual experiences. Even by Byrne’s lights, the stock category of visual illusion isn’t needed for the issue to make contact with vision science—contrary to his suggestion that the notion of illusion provides a better route to clarification than mine. Vision science, Byrne suggests, lays it down that there are “perceptual modules, conceived along Fodorian lines”, and the informational outputs of those modules delimit the contents of visual experience. Byrne thinks that “modularity helps us pin down the empirical considerations relevant to assessing the Rich View” (p. 69). But the standard Rich/Thin debate cannot be assimilated to the debate about whether any outputs of perceptual modules are rich. Perceptual experience is a phenomenological category. Perceptual modules, in contrast, are defined functionally: they are, roughly, mandatory processes that always respond in the same way to a proprietary set of inputs. Nothing in the functional characterization of a module can speak to whether its outputs exhaust the conscious representations that constitute perceptual experience. (That characterization does not even speak to whether the outputs are conscious. If rich properties were unconsciously perceived via a module, that would not show that the Rich View is true.) Even if all the outputs of perceptual modules were thin, that conclusion would not entail the Thin View. And in any case, Byrne’s confidence in modularism relies on a scattered list of rebuttals of putative counterexamples, ignoring many productive strands in vision research that do not rely on the presumption of modularity and are oriented around the prima-facie anti-modularist idea that visual experience depends heavily on stored memory and past experience—and so is unlikely to be as insensitive to the rest of the mind as modularism predicts.20 The coherence of the Rich/Thin debate does not depend on whether modularism or the anti-modularism ultimately wins the day. A different strategy would try to assimilate the debate to one about the outputs of exclusively perceptual processes—whether those processes are modular or not. This assimilation strategy falters in the same place as Byrne’s modularist strategy: it does not bypass the need for assumptions about which states are perceptual experiences. We need to make such assumptions to bring the science of perception to bear on the Rich/Thin debate. Consider the interaction between shape and color found in one version of the memory-color phenomenon: a banana-shaped image appears more yellow than something of the same color that isn’t banana-shaped.21 Let’s suppose that this effect is evidence that shape information can influence color information. Color processing does not then look to have its own entirely encapsulated module.

74

•

Susanna Siegel and Alex Byrne

Which color you end up experiencing will depend on which shape information you take in. Presumably this observation shouldn’t lead us to conclude that colors are not presented in experience. The fact that we have color experiences is a fixed point. If it turned out that some color processing wasn’t modular, or that it resulted from processes that didn’t fit our antecedent assumption about which processes are perceptual, we shouldn’t conclude that colors are not presented to us in visual experience. It’s a phenomenological datum that they are. My phenomenal contrast method provides a framework in which we can locate considerations from vision science. The method allows us to distinguish between different possible explanations of carefully chosen phenomenal contrasts. Block’s (2014) case that the rich property of being a facial expression is presented in visual experience fits into this framework. He is rebutting the same alternative to the Rich View that Byrne focuses on: the hypothesis that the representation of thin properties is enough to explain the contrast. Block’s discussion of a series of results in face-perception is designed to show that it isn’t.22 The difference between Byrne’s approach to the issue and mine is this. For Byrne, the only way to identify the perceptual experiences is as the conscious upshot of processes that we can identify independently of their role in producing perceptual experiences. In contrast, I think it is impossible to identify the processes that generate perceptual experiences, without taking as already understood what perceptual experiences are. We have an understanding of perceptual experiences that comes from our familiarity with them, and that understanding is robust enough to identify the experiences, but not detailed enough to settle the Rich/Thin debate. The perceptual experiences are states (or perhaps events) with the phenomenal features that typically characterize perception in the sense-modalities and combinations of them. Carey’s discussion of contact causation gives us a domain where familiarity with perceptual experience and results from experimental sciences combine to support the Rich View. Her book The Origins of Concepts (2009) synthesizes copious evidence that impressions of contact causation are insulated from many types of knowledge of what can cause what. A feather cannot launch a brick, and a shadow cannot launch or be launched by anything. But we can get impressions of launching by these things, if their movements meet certain parameters. Michotte (1963) argued that the parameters are purely spatiotemporal, but Carey argues that he was wrong, because representations of causation are sensitive to agential properties, such as being able to initiate movement. There’s a concept of causation that’s active in infancy, Carey argues, that gets applied to the outputs of modules for detecting objects. Applying the causal concept can result in purely visual representations that have all the hallmarks of perceptual experience—such as the impression of seeing a brick cause a shadow to move.

Rich or Thin?

•

75

Why think that the visual experience ever represents causation, as opposed to only the spatial-temporal movement of object A and object B, when A appears to launch B? Here’s an argument that moves from premises about what properties a person becomes sensitive to, to a conclusion about which properties are presented in experience: if a perceptual experience is produced by a process that is activating its sensitivity to property F (in this case, causation), and presenting F-ness is not precluded by the phenomenal character of the experience, then the experience presents F. A different type of argument moves from premises about changes in a person’s overall perspective, which may not consist in sensitivities to new properties, to conclusions about which properties are presented in experience. For instance, Siewert (1998) presumes that how a neighborhood looks the first time you saw it differs from how it looks once it becomes your neighborhood. Siewert’s plausible-sounding phenomenological presumption doesn’t establish the Rich View. But the structure of the cases he describes makes it plausible that if the phenomenal differences go with differences in which properties the experiences present, those properties are Rich. Its plausibility indirectly supports my contention about the expression of doubt. Learning that a contorted face is an expression of doubt need not always change how it looks, but it is plausible to suppose that it could. It is sometimes said that the experiential changes in these cases are changes in attention.23 You attend to different parts of the house-façade, your neighborhood, or the contorted face. Attending to something is fundamentally a relation to it: one attends to an object, a part of an object, a spatial region, or a property F that something you see instantiated. The observation that attentional changes accompany the phenomenal contrast does not tell against or in favor of the Rich View. If one attends to an object, part of it, or a spatial region, the issue is which properties the experience presents them as having. If we attend to a property, then as before, the issue is whether the property is rich or thin. Rich social properties are strong candidates for coming to be presented in experience due to acculturation. Consider the property of being a white man, which (let’s suppose) you attribute to the man in the elevator. If the Thin View is right, then what’s presented is instead a cluster of thin properties (or a disjunction thereof) that people acculturated to this category use. But how plausible is it that, when someone’s racial whiteness is perceptually salient, its salience consists entirely in the salience of thin properties? It seems more plausible that what’s salient is the possibilities of action surrounding the cultural category. Whether rich social properties illustrate the Rich View or not, our understanding of both perception and culture could benefit by illuminating where they enter in the interface between the perceiver and the world.

76

•

Susanna Siegel and Alex Byrne

Part 3: Byrne As this debate has brought out, one important disagreement between us concerns the notion of a “presented property”. Siegel takes it to be explanatorily primitive, and I do not. We will have to leave it there, but clarifications may help resolve some other issues. First, I was taking illusions to be illustrated by paradigm examples, not to be exhausted by them. If the modularity hypothesis is true, the output of the visual module marks a natural joint in the mind. Granted modularity, false output is what illusions are, and we can bring experimental work to bear on some disputed examples. Suppose an expert is fooled by a fake Royal Albert teacup, confidently pronouncing that it is genuine. Is the expert suffering from a perceptual illusion? Yes, if his or her visual module is delivering the misinformation that it is a Royal Albert teacup; no, if the error is located further downstream. Second, although I have a beef with the way many philosophers talk of “perceptual experiences” (Byrne 2009), my appeal to illusions shows that I do not think that the only route to understanding is via “processes that we can identify independently of their role in producing perceptual experiences” (p. 74). Siegel and I do agree, though, that the Rich/Thin debate cannot be settled without appeal to vision science. Third, I didn’t mean to imply that the “coherence” (p. 73) of the debate turns on the truth of the modularity hypothesis any more than the coherence of certain questions about the tides turns on whether gravity is responsible for them. But—and I think Siegel would concede this much—the hypothesis has some significant support. And if it is true, then we may use it to argue, as I did in part 2, that kind properties like teacuphood are not presented in experience. A more serious bone of contention concerns that last claim. Siegel thinks that even if there is a visual module, the Rich/Thin debate does not concern its outputs. “Even if all the outputs of perceptual modules were thin, that conclusion would not entail the Thin View” (p. 73). Against Siegel, suppose that the modularity hypothesis is true, and that the outputs of the visual module are thin. We can then argue for the Thin View as follows. First premise: there is no difference in how things look without a difference in the outputs of the visual module. Second premise: there is no difference in presented properties without a difference in how things look. Third premise: if the Rich View is true, then there can be a difference in rich presented properties with no difference in thin presented properties. By the first and second premises, any difference in presented properties will be accompanied by a difference in outputs. So, by the third premise and the supposition of thin outputs, the Rich View is false; hence, the Thin View is true. Siegel suggests at various points that the relevant category of “visual experiences” goes beyond the outputs of any visual module. That is, there can be

Rich or Thin?

•

77

different visual experiences, and so presumably differences in how things look, with no difference in output. Thus, she will presumably resist by denying the first premise. Be that as it may, my proposal to connect the Rich/Thin debate with modularity is not idiosyncratic. According to Rips, for example, “The issue of causal perception comes down to the question . . . ‘Do [people] have perceptual modules that respond . . . with an output indicating whether a causal event is taking place?’ ” (Rips 2011: 84; see also Scholl and Tremoulet 2000: 305–6).24 Although Siegel will be unfazed if the outputs are all thin, I will throw in the towel if some of them are rich, which brings us to the perception of causation and animacy (Scholl and Tremoulet 2000). Let us quickly review two examples. First, Michotte’s “launching effect” (mentioned by Siegel)—in effect a Siegelean phenomenal contrast. Ball B is at rest in the middle of a display; ball A enters from the left, moving toward B. When A reaches B, the latter moves off to the right. The introduction of a short delay before B moves off can make a big difference: with no delay, there seems to be exactly the thing Hume was denying, an “impression” of causation; with a delay, two events simply seem to occur successively. Second, the “wolfpack effect” (Gao et al. 2010). Here the two contrast cases involve orientated shapes and a disc, all moving randomly. If the oriented shapes point toward (“look at”) the disc, there is a strong impression of animacy (specifically, the orientated shapes seem to be pursuing the disc). That impression is absent if the shapes are oriented orthogonally to the disc. There is a lot to be said for the view that these effects are genuinely perceptual. For instance, Rolfs et al. (2013) found perceptual adaptation to Michottean launches, and moreover adaption specific to retinal location. However, the perceptual nature of the launching effect does not discriminate between the thin hypothesis on which the output merely concerns a complex thin relation, causationG, rather than the rich relation of causation (cf. Rips 2011: 83; Hilbert and Nishimura 2015). As to what causationG is, experimental work examining the spatiotemporal cues that produce the launching effect has already gone some way to uncovering it. Similar points go for animacy, and for Block’s (2014) use of perceptual adaptation to faces,25 although obviously much more discussion is needed. I do not think Block is as committal as Siegel makes him out to be (see especially 563), but in any case the thin hypothesis is particularly salient here, since “face” has a natural thin interpretation. (Similarly for terms for facial expressions like “happy face”.) A rich-face is something only an animal can have; in contrast, busts and some natural rock formations have thin-faces but not richfaces. What is presented in experience: the property of being a rich-face or the “recognitionally coextensive” property of being a thin-face? The work cited by Block does not settle the issue.26

78

•

Susanna Siegel and Alex Byrne

The Thin View is not irresistible, but it does have the advantage of parsimony. We already know that the visual system has a propensity for grouping stimuli into thin gestalt categories. If that propensity can account for the data, there is no need to take it to be capable of anything more.

Notes * Thanks to Ned Block, David Hilbert, Zoe Jenkin, Eric Mandelbaum, Matt McGrath, Bence Nanay, Sarah Paul, and Bas Rokers. 1. See Pylyshyn (1999). 2. For instance, Thau (2002) denies that color is presented in experience. 3. It is controversial that perceptual experiences have content (see, e.g., Brogaard 2014). Moreover, some of those who agree perceptual experiences have content deny that it is propositional (see, e.g., Burge 2010: 36). However, the fundamental issue between Siegel and me arises for most philosophers of perception, even if they would deprecate the way we put it. (It is also less than clear what “perceptual experiences” are supposed to be, but that can of worms will be left closed.) 4. Cf. Siegel (2010: 30–3). 5. Cf. Barwise (1981). 6. See Thau (2002: 230) and Byrne (2016). Although “looks [adjective]” is unhelpful, “looks the same/different” is indispensable, as the rest of this debate illustrates. 7. Hallucinations are a vexed category that can be ignored for present purposes. 8. For some of the usual qualifications to encapsulation (to accommodate, e.g., cross-modal effects), see Burge (2010: 101–2). 9. Siegel doesn’t entirely disagree with this, but she thinks the problem is that “the English word ‘look’ can denote too many things” (p. 67). However, as I see it, the problem is that “look” (at least as used in a perceptual way) only denotes one thing. 10. The actual quotation has “visual experience” rather than “visual illusion”, but it is clear from the context that Siegel would accept the substitution. 11. For a more careful statement of cognitive (im)penetrability, see Macpherson (2012: 28–9). 12. Although principally not for the reasons given by Ioannidis. 13. The replication was conceptual, not direct. There is other (more recent and much better) work on memory color than the flawed Delk and Fillenbaum study—e.g. Olkkonen et al. (2012), Witzel et al. (2011). Olkkonen et al. found a memory color effect for realistic photographs of fruit, but not for outline fruit shapes. Using a similar methodology, Witzel et al. found a memory color effect for artificial objects with characteristic colors (e.g., smurfs, mailboxes, Nivea cream tins). (Interestingly, they found no effect for fire extinguishers [43].) Assuming that the effect is perceptual (for an alternative explanation, see Zeimbekis 2013), what seems to be important is that the stimulus is visually similar to instances of the kind that have been previously seen, which suggests that “high-level” knowledge that bananas are yellow and smurfs are blue is not what is driving the effect. (See further Gross et al. 2014 and also Olkkonen et al. 2012.) 14. A highly controversial claim, admittedly. For a brief skeptical review of the literature on cognitive penetration, see Machery (2015) (in the useful collection Zeimbekis and Raftopoulos 2015). For a lengthy systematic critique, see Firestone and Scholl 2015a and b. 15. See, e.g., Schyns et al. (1998: 2–4). There is no conflict with modularity here, since a diet of certain inputs may cause modules to change over time. 16. This is a standard way of arguing for the Rich View; for references, see Bayne (2009: 390). 17. Siegel’s actual example involves pine trees rather than teacups; that example is complicated by the fact that (ordinary) pine trees are visually more diverse than (ordinary) teacups. Even in the case of teacups, there is diversity—between English and Japanese varieties, for

Rich or Thin?

18. 19. 20. 21. 22. 23. 24. 25. 26.

•

79

instance. To avoid an unnecessary complication, restrict attention to English teacups. (A more realistic example would involve a sub-category, say Royal Albert teacups, since perceptual expertise is more pronounced at these levels.) For illustrations, see https://en.wikipedia.org/wiki/Greeble_(psychology). However, for arguments that affordances are perceptually presented, see Nanay (2011) and Siegel (2014). See Triesch et al. (2003), Greene and Oliva (2009), and the papers by Feldman-Barrett and by Bar in Bar (2011). See Olkkonen et al. (2012). Another example is Di Bona (2016), who argues that being a female/male voice can be presented in auditory experience. See, e.g., Nanay (2011). Rips also has a useful discussion of Carey’s work (cited by Siegel), not examined here for reasons of space. Fish (2013) also appeals to adaption to (inter alia) faces to argue for the Rich View; for a reply, see Briscoe (2015). It’s also worth noting that greeble recognition, an entirely thin matter, exhibits some of the distinctive characteristics of face recognition (Gauthier and Tarr 1997).

References Anscombe, G. E. M. 1965. The intentionality of sensation: a grammatical feature. Analytical Philosophy, second series, 3–20. ed. R. J. Butler. Oxford: Blackwell. Bar, M. (ed.) 2011. Predictions in the Brain: Using Our Past to Generate a Future. Oxford: Oxford University Press. Barwise, J. 1981. Scenes and other situations. Journal of Philosophy 78: 369–97. Bayne, T. 2009. Perception and the reach of phenomenal content. Philosophical Quarterly 59: 385–404. Block, N. 2014. Seeing‐as in the light of vision science. Philosophy and Phenomenological Research 89: 560–72. Breckenridge, W. 2007. Against one reason for thinking that visual experiences have representational content. Philosophical Perspectives 21: 117–23. Briscoe, R. 2015. Cognitive penetration and the reach of phenomenal content. The Cognitive Penetrability of Perception: New Philosophical Perspectives, ed. J. Zeimbekis and A. Raftopoulos, 174–199. Oxford: Oxford University Press. Brogaard, B. (ed.) 2014. Does Perception Have Content? Oxford: Oxford University Press. Burge, T. 2010. Origins of Objectivity. Oxford: Oxford University Press. Byrne, A. 2009. Experience and content. Philosophical Quarterly 59: 429–51. Byrne, A. 2016. Hill on mind. Philosophical Studies 173: 831–39. Carey, S. 2009. The Origin of Concepts. Oxford: Oxford University Press. Delk, J. L., and S. Fillenbaum. 1965. Differences in perceived color as a function of characteristic color. American Journal of Psychology 78: 290–3. Di Bona, E. 2016. The thickness of auditory perception. MS (Unpublished paper). Feldman-Barrett, L. & Bar, M. (2009). See it with feeling: affective predictions during object perception. Predictions in the Brain: Using our past to generate a future, ed. M. Bar. Oxford: Oxford University Press. Firestone, C., and B. J. Scholl. 2014. “Top-down” effects where none should be found: the El Greco fallacy in perception research. Psychological Science 25: 38–46. ———. 2015a. Can you experience ‘top-down’ effects on perception?: the case of race categories and perceived lightness. Psychonomic Bulletin & Review 22: 694–700. ———. 2015b. Cognition does not affect perception: evaluating the evidence for ‘top-down’ effects. Behavioral and Brain Sciences, FirstView: 1–72. http://dx.doi.org/10.1017/S01405 25X15000965.

80

•

Susanna Siegel and Alex Byrne

Fish, W. 2013. High-level properties and visual experience. Philosophical Studies 162: 43–55. Fodor, J. 1983. The Modularity of Mind. Cambridge, MA: MIT Press. Gao, T., G. McCarthy, and B. J. Scholl. 2010. The wolfpack effect: perception of animacy iresistibly influences interactive behavior. Psychological Science 21: 1845–53. Gauthier, I., and M. J. Tarr. 1997. Becoming a “Greeble” expert: exploring mechanisms for face recognition. Vision Research 37: 1673–82. Greene, M. R., and A. Oliva. 2009. The briefest of glances: the time course of natural scene understanding. Psychological Science 20: 464–72. Gross, S., T. Chaisilprungraung, E. Kaplan, J. A. Menendez, and J. Flombaum. 2014. Problems for the purported cognitive penetration of perceptual color experience and Macpherson’s proposed mechanism. Baltic International Yearbook of Cognition, Logic and Communication Vol. 9, ed. E. Machery and J. Prinz, 1–30. Manhattan, KS: New Prairie Press. Hilbert, D. R., and S. Nishimura. 2015. Visual perception of causation. MS. Ioannidis, J. 2005. Why most published research findings are false. PLoS Medicine 2: 696–701. Levin, D. T., and M. R. Banaji. 2006. Distortions in the perceived lightness of faces: the role of race categories. Journal of Experimental Psychology: General 135: 501–12. Machery, E. 2015. Cognitive penetrability: a no-progress report. The Cognitive Penetrability of Perception: New Philosophical Perspectives, ed. J. Zeimbekis and A. Raftopoulos, 59–74. Oxford: Oxford University Press. Macpherson, F. 2012. Cognitive penetration of colour experience: rethinking the issue in light of an indirect mechanism. Philosophy and Phenomenological Research 84: 24–62. Michotte, A. 1963. The Perception of Causality. Translated by T. R. Miles and E. Miles. New York: Basic Books. Nanay, B. 2011. Do we see apples as edible? Pacific Philosophical Quarterly 92: 305–22. Olkkonen, M., T. Hansen, and K. R. Gegenfurtner. 2008. Color appearance of familiar objects: effects of object shape, texture, and illumination changes. Journal of Vision 8: 1–16. Olkkonen, M., T. Hansen, and K. Gegenfurtner. 2012. High-level perceptual influences on color appearance. Visual Experience: Sensation, Cognition, and Constancy, ed. G. Hatfield and S. Allred, 179–98. Oxford: Oxford University Press. Pendlebury, M. J. 1990. Sense experiences and their contents: a defense of the propositional account. Inquiry 33: 215–30. Pylyshyn, Z. 1999. Is vision continuous with cognition? The case for cognitive impenetrability of visual perception. Behavioral and Brain Sciences 22: 341–65. Rips, L. J. 2011. Causation from perception. Perspectives on Psychological Science 6: 77–97. Rolfs, M., M. Dambacher, and P. Cavanagh. 2013. Visual adaptation of the perception of causality. Current Biology 23: 250–4. Scholl, B. J., and P. D. Tremoulet. 2000. Perceptual causality and animacy. Trends in Cognitive Sciences 4: 299–309. Schyns, P. G., R. L. Goldstone, and J.-P. Thibaut. 1998. The development of features in object concepts. Behavioral and Brain Sciences 21: 1–17. Siegel, S. 2010. The Contents of Visual Experience. Oxford: Oxford University Press. Siegel, S. 2014. Affordances and the contents of perception. Does Perception Have Content?, ed. B. Brogaard, 51–75. Oxford: Oxford University Press. Siewert, C. 1998. The Significance of Consciousness. Princeton: Princeton University Press. Stefanucci, J. K., and M. N. Geuss. 2009. Big people, little world: the body influences size perception. Perception 38: 1782–95. Thau, M. 2002. Consciousness and Cognition. Oxford: Oxford University Press. Triesch, J., D. H. Ballard, M. M. Hayhoe, and B. T. Sullivan. 2003. What you see is what you need. Journal of Vision 3: 86–94. Witzel, C., H. Valkova, T. Hansen, and K. R. Gegenfurtner. 2011. Object knowledge modulates colour appearance. i-Perception 2: 13–49. Zeimbekis, J. 2013. Color and cognitive penetrability. Philosophical Studies 165: 167–75. Zeimbekis, J., and A. Raftopoulos. 2015. The Cognitive Penetrability of Perception: New Philosophical Perspectives. Oxford: Oxford University Press.

PART

III

Non-Visual Sense Modalities

CHAPTER

5

Auditory Perspectives

JOHN KULVICKI

1. Introduction A veil of sound stands between subjects and what they care about hearing. The whole town hears the train rumble over the bridge down by the river, but the train, the bridge, and the crossing are only heard because of the sounds they make. Sounds are heard, and anything else heard thereby. By contrast, there is no veil of sights. Colors, shapes, and the individuals that have them are the objects of typical visual experiences. Those close to the bridge see the train cross, but not by seeing anything else. Hearing is mediated in a manner that vision is not, or so the story goes. There once was a veil of vision, too. From the early modern period through the sense datum theorists of the early twentieth century, philosophers cast sight as mediated by visibles, sense data, or otherwise. Epistemology and commonsense phenomenology both supported the account. Not only might I be hallucinating instead of seeing the coin, but what I see seems elliptical even though the coin seems round. Perhaps, then, the coin is seen in virtue of seeing something else, something elliptical. A compelling retort to this object-splitting proposal—something round, something else elliptical—suggests splitting qualities. The coin is intrinsically round but cuts an elliptical figure, from here (Armstrong 1961). Such relational shape properties, which Thomas Reid (1764) called visible figures, comfortably coexist with objects’ intrinsic shapes. Though sense data are unpopular, object-splitting proposals are the norm for those working on auditory perception. We hear the train but only by hearing the sounds it makes. The goal of this paper is to offer an alternative. Section 2 83

84

•

John Kulvicki

unpacks the dialectic over object- and quality-splitting proposals in vision, then section 3 shows why object splitting is so appealing in audition. Section 4 formulates and defends a quality-splitting proposal for audition. As section 5 shows, this proposal yields two new and appealing theories of sounds. Accepting the proposal means that both vision and audition are (un)mediated in the same way.

2. Visible Figures GE Moore (1922, 185) spies two coins resting on a table. Seen obliquely, their round faces somehow seem elliptical. The half-crown is so distant that it looks, in a way, smaller than the nearby florin, even though it’s also obviously bigger. Moore’s eyes are good, and conditions are hospitable, so it won’t do to say he misperceives. But no coins are both round and elliptical. Nothing is both bigger than, and smaller than, something else. He is either aware of two sets of objects, each with its own set of qualities, or two sets of qualities that characterize one set of objects. Neither option is ideal, because he has one set of names for objects (florin, half-crown) and one set for the qualities (circular, elliptical, large, and small). Something must give, and for Moore it was the objects. He suggested, tentatively, that he was aware of elliptical things—sense data—one bigger than the other, which correspond, somehow, to the differently sized circular coins. Their status as data was significant. It was in virtue of being acquainted with them that Moore thought he became aware of the coins with their own sizes, shapes, and orientations. Others, notably Bertrand Russell (1914) and H. H. Price (1932) ran with the idea. One could always split the properties, and this has been the path taken by those unsympathetic with sense data (Armstrong 1961). The coin is round, and also has another shape, which explains the sense of its ellipticity. The florin’s round top presents various visible figures from different points of view. Lines projecting from a point coincident with one of Moore’s eyes to the edges of the coin’s top form a solid angle specifiable in two dimensions. Such solid angles are not just features the coin happens to have from Moore’s vantage point. They are perceptible. And the florin’s solid angle from a given point is only perceptible when one occupies the location in question. Visible figures earn that name not because they are subjective but because they are only accessible from specific points of view.1 From many points in space, the coin occupies a solid angle just like that occupied by an ellipse seen head-on. The name ellipse is apt, then, even though such contexts aren’t its home ground. Seen head-on, the coin occupies a solid angle just like a circle seen head-on. In such cases, the experience involves two circular spatial qualities, one perspectival, one not. Visible figures do not constitute an independent dimension of experience. Fix an object’s shape, determine a point from which it is viewed, and its visible

Auditory Perspectives

•

85

figure is fixed too. Reid thus cast them as abstractions over intrinsic shapes and orientations. One and the same visible figure is compatible with many shape-orientation pairs. Pictures exploit this fact. By mimicking things’ visible figures, a flat surface can call any number of scenes to mind.2 Correlatively, visible figures cannot be detached from experiences of shapes and orientation. There are no views from nowhere. But it is possible to experience a visible figure without also being aware of intrinsic shape and orientation. In James Turrell’s Twilight Epiphany: Skyspace (2012), a concrete expanse frames a quadrilateral patch of sky. Viewers have no feel for the orientation of the frame, so the whole experience is of a patterned 2-D expanse. Some abstract paintings have a similar effect. Because of this, even though visible figures as described here are not the epistemological fundament the sense datum theorists sought, they are unavoidable and significant aspects of the perception of things in space. Many abstractions over otherwise perceptible properties exist, but only some of them are noticed. Visible figure is presumably available because perceptual systems must parse a signal coming into the eyes that varies in two dimensions across each retina into one that speaks to the facts of a 3-D world. It’s conceivable, but not quite imaginable, that a creature could perceive the intrinsic shapes of objects, as well as their orientations with respect to the creature’s standpoint, without any access to visible figure. We are not such creatures, though it would move beyond the scope of this paper to speculate as to why. As will become clear, a similar point applies to the auditory analogs of visible figures. The sense-datum skeptic has other options for explaining the apparently elliptical aspects of the coins. For example, patterns of light between the perceiver and things perceived have shapes. So, perhaps Moore is aware of the shapes of light patterns, and only thereby the coins and table. Moore sees a sight, one might say, and awareness of it, and its features lead him to awareness of the florin and the half-crown. Such object-splitting worldward of sense data carries all of the costs and none of the benefits of doing it Moore’s way. Epistemological advantages are forfeit, since light patterns are unfit as objects of acquaintance. In addition, there seems little motivation to multiply objects perceived when there is a perfectly coherent sense in which visible figures are abstractions over shape and orientation. The alternative suggestion is not senseless as much as deeply unmotivated from the standpoint of visual experience.

3. Auditory Object Splitting Surprisingly, a version of the visually implausible suggestion dominates thinking about auditory experience and how it relates hearers to the world. We hear

86

•

John Kulvicki

sounds, and only thereby come to know about the objects and events that cause them. It is phenomenologically apt as a description of auditory experience to say that we hear things like weedwackers and mosquitos by or thanks to hearing the sounds they make. However, no straightforward analog to this apparent indirectness is phenomenologically apt as a description of visual awareness of material objects, which involves no perceptual intermediaries. (O’Callaghan 2011, 149) The object-splitting proposal is poorly motivated in vision, but almost obvious in audition. Moore’s florin falls from the table to the concrete floor. It bounces, spins, and settles down. The half-crown follows, and rolls until a baseboard stops it. Auditory experience reveals many features of these objects, events, and environs. Coins fell, not forks or salt shakers. The smaller coin sounded brighter than the bigger one. They hit a hard floor, likely concrete or tile, then bounced, rolled, and spun. The half-crown struck the wall with less oomph than when it hit the floor, and the baseboard sounded hollow, wooden. The room was sparsely furnished, judging from the reverb, so not quite what one would expect of a Cambridge don. Events like collisions, qualities like being metallic, and features of the contexts in which these events take place are all somehow aspects of what Moore hears. Vision is complex, too, of course. Moore can see the coins fall, hit the floor, and catch the sunlight as they spin. The color of the floor, the coins’ shiny surfaces, and the path they traced through space are all perceptually obvious. Though they are similarly complex, object splitting is appealing in the auditory case, but not visually, for three reasons. First, Moore hears the coin collide with the floor; he hears the coin, the floor, and even features of the room, in an amalgamated mess that privileges none of these as the primary object heard. It’s not obvious that Moore hears the event by hearing the coin and its features over time. He might just as well hear aspects of the coin by hearing the event in which it participates. This makes it plausible to locate both particulars of interest—the coin, the event— at one remove from what is heard immediately. Visually, it’s easier to claim that Moore sees the coin and its features, and thereby any events in which it participates. Second, visual events of interest are readily separated from those that allow seeing in the first place. Moore sees the coin, its shape and color, and its descent to the floor. Another event entirely—the light reflecting off the coin—allows Moore to see all of this, but this event per se is not at all perceptually salient. Moore can still see the coins after they stop doing much, as long as the lights

Auditory Perspectives

•

87

stay on. By contrast, Moore learns of the coin, the floor, their qualities, and the collision in a blast that quickly dies out. Everything involved in the episode is still there, the coins at rest on the floor, but they are no longer heard.3 The event that makes hearing possible is more closely related to the event perceived than the objects that participate in it. One might think this speaks in favor of identifying sounds with the events like coins striking the floor. But, and this is the third reason, such events seem possible without any sounds being produced. Coins falling on the moon make no waves, and could never be heard.4 Some collisions could happen between objects that are prevented from vibrating. So it seems, as O’Callaghan suggests, that we can separate sounds from objects and events of interest. We learn of the latter by hearing the sounds they make. Most accounts of sound identify them with particular individuals somehow between hearers and the objects and events that interest them.5

4. Audible Profiles Auditory experience makes object splitting appealing. This section traces an alternative, quality-splitting proposal. Moore hears the coins, the collision, the floor, the room, and many of their qualities in addition to perspectivedependent audible features that abstract over them. Just as he sees the coin’s shape and orientation along with its visible figure, Moore hears features of the coins, floor, and baseboard, along with the situation’s audible profile. Audible profiles are collections of perfectly objective features of the perceived world, but these perspectival qualities are abstractions over features of objects, events, and environs. Moore is aware of them in virtue of how his auditory system must parse a signal that varies in relatively few dimensions into something that is informative along relatively many. Like visible figures, they do not constitute an independent dimension along which auditory experiences can vary. And because they are abstractions over features of ordinary objects, events, and environs, this proposal requires no extra objects to fill the role of sounds.6 Moore shows Anscombe a sketch of the coins he saw. It is a good visual surrogate for the original scene. The picture part corresponding to the florin’s top is elliptical and takes up a larger portion of the picture than the ellipse depicting the farther-off half crown. Anscombe is perfectly aware that she sees, headon, elliptical parts of a flat surface. But she also notices that the scene is very much like that presented by two coins atop a table. The scenes are alike with respect to visible figure, but not with respect to either the intrinsic shapes of things seen or their orientations. In that sense, visible figure is an abstraction over intrinsic shape and orientation, and such surrogacy cases render visible figure quite salient. Looking back at the coins on the table, Moore and Anscombe also notice that they seem to retain their shapes across many viewpoints. Seen from above, the elliptical visible figures collapse to a circle, and, as the angle of viewing

88

•

John Kulvicki

becomes more oblique, the ellipses become more eccentric. Shape constancy suggests that Moore and Anscombe are perceptually aware of the coin’s intrinsic shape, even though, from their different perspectives, each of them notices a different visible figure. There is no visible figure constancy because precise changes in both intrinsic shape and orientation are required for two scenes to be alike in that respect. There is an art to depiction, even if it is just copying appearances. Auditory counterparts to surrogacy and constancy help make a case for understanding audible profiles on the model of visible figures. Anscombe, no fan of sense data, pushes a hesitant Moore off the diving platform. He screams as he falls. Later, in the pub, she mimics the incident. The cry is a full-throated loud vowel that diminishes to a quiet consonant before the splash. Anscombe’s audience is under no misapprehension about what they hear. She’s a stationary source that gets quieter, lower-pitched, and airier over time. Those party to the original event heard a moving source that did not seem to get quieter or lower pitched or washed out over time. If Anscombe had brought a parrot or a smartphone with her to the ambush, she could have produced an even better imitation. But this, too, would have been distinguishable from the original. Like the picture of the coins, the reproduction is like the original in certain perspectival respects, but different otherwise. Those who were present poolside mimic the fall differently. They do this because from their perspectives, the objects and events in question had different audible profiles. Notice the tradeoffs. Something nearby, diminishing in loudness, sounds a lot like a uniformly loud thing receding. Something with little in the high end of the frequency spectrum sounds like something muffled, heard through a wall, or over some distance. The principal players in these affairs—Moore falling, Anscombe imitating—are very different from one another, yet experiences of them are partly alike because they share a number of perspective-dependent audible features that together constitute the scene’s audible profile. The profiles of these contexts are quite similar, though the situations with the profiles are different. Fix a scene—objects, events, environs—and you fix an audible profile, but one audible profile is compatible with different scenes. From the pool, Moore utters some unmentionable. Atop the platform, Anscombe hears the retort and recognizes its source, even though Moore sounds different in these circumstances than he does in the classroom. Moore from a distance, outdoors, sounds different than Moore up close, indoors, but it’s Moore all the same. Kinds of objects, as well as individuals, are readily recognized across contexts in which they are heard, even though the context is something that, in its own way, is heard too. In these constancy cases the audible profiles are different, though at least one of the players in the scene, Moore, is the same. As in vision, constancy and surrogacy make perspectival features salient.

Auditory Perspectives

•

89

Anscombe can tell that Moore screams with constant volume as he falls, but she also has access to the sense in which the loudness decreases, at least from her perch. The situation is different for the spectators below. The sense in which loudness seems to decrease is bound up with her sense of him moving away. While it is tempting to say that Anscombe has access to a sound that decreases in intensity as Moore falls, it’s also quite apt to say she has access to a perspectival feature of the scene. That’s why her imitation can work so well. The audience recognizes a similar audible profile, even though the volume and distance of the source both seem different than they seemed to Anscombe during the original event. Absent the audible profile, it would be hard to explain the effectiveness of her imitation. “Loudness” has two uses, much as “ellipse” does. On the one hand, it identifies the intensity of a perceived event. On the other, it is an abstraction over intensity of the source and its distance. The more abstract sense of the word is perspectival because it depends on the auditory context in which perceivers find themselves. Similarly, auditory brightness can pick out how richly something vibrates at high frequencies or an abstraction over that sense of brightness and intervening media. Through a door, or at some distance, things sound muffled, and even lower in pitch. But so do some things heard up close, albeit in a different sense. Brightness is just one component of what is usually called timbre, and it seems as though most aspects of timbre admit of this duality. Likewise, we can distinguish things heard in large, reverberant rooms from those heard in well-paneled close quarters. In such circumstances, one can readily identify the context as well as the thing heard. One can generate a convincing surrogate of something heard in a reverberant room, even in the smallest study, making use of nothing but a pair of loudspeakers and the appropriate recording. It’s common to describe sounds as “those objects of experience that can be described in terms of their loudness, pitch, and timbre” (Nudds 2010, 106; and see O’Callaghan 2007, 17). But each of these terms applies to at least a couple of things in any given experience. We have seen that the term loudness applies to both source and something else. In some sense, muffled things sound lower in pitch, but, in another sense, they do not. In one sense, timbre applies to objects that we hear, like guitars, but in another it applies to something else— not the guitar, but what one might call the sound of the guitar. This duality, I suggest, is due to the fact that we are aware of audible profiles, in addition to other features of things. Some visual experiences, like those from within Turrell’s installation, remain fairly abstract, revealing nothing but patterns in two dimensions. Socalled disembodied sounds are the analogs of such cases. What makes an auditory experience disembodied is that what one hears is not readily parsed into features of objects and the context in which they are heard. All that remains are experiences that abstract over many possibilities of sources, events, and

90

•

John Kulvicki

environments. It does not do justice to auditory experience to suggest that it is always like this, but it tells in favor of this proposal about audible profiles that some of it is. Like visible figures, these perspectival audible features seem to be artifacts of how the auditory system must parse a signal that varies in relatively few dimensions into one that tracks features of a multi-dimensional environment. The pressure waves that affect the ears vary in intensity, for example, but an intense event, far away, can produce waves that are very much like those produced by subtler nearby ones. Similarly, a nearby event can produce pressure waves that are relatively weak in high frequencies. Faraway events deliver signals light on high frequencies because they diminish in intensity with distance faster than lower frequencies do. Light and close or intense and far? Albert Bregman (1990, 3–6) likens the problem of auditory scene analysis to figuring out what happens on the surface of a lake when all one has to go on are the ripples along two narrow channels leading from it.7 The two ear canals, and the way they are wired up, determine auditory perspectives on scenes. None of this requires, however, that pressure waves are the objects of auditory experience. Instead, it is because of how pressure waves mediate auditory contact with the environment that perspectival audible features are perceived. The features themselves are abstractions over intrinsic and relational features much as visible figures abstract over shape and orientation. Hearing is complex, and the foregoing amounts to just a sketch. To fill out the picture, one would need to explore the full range of auditory surrogacy and constancy. That range is quite wide. Loudspeakers are effective surrogates for all manner of auditory situations, for example. Similar thoughts apply to vision. Spatial layouts constitute just one aspect of visual appearance, and it is an open question whether related claims can be made about color vision (Kulvicki forthcoming). The way to investigate the visual cases is likewise through investigating the range of visual surrogacy and constancy.

5. Whither Sounds? Audible profiles constitute a quality-splitting proposal for hearing. Adopting the proposal leads directly to two new accounts of sounds. First, perhaps sounds just are the components of audible profiles. That is, they are perspectival qualities, which are abstractions over intrinsic and relational features of ordinary objects, environments, and the events in which they participate. This explains how sounds seem to be distinct from, yet intimately tied to, objects, events, and environs. Born of a quality-splitting proposal, this account suggests that no particular individuals stand between hearers and the ordinary things they hear. This contrasts with those that identify sounds with

Auditory Perspectives

•

91

events, or pressure waves.8 Despite that, there is a phenomenological sense in which audible profiles mediate perceptual contact with other worldly states of affairs. Ordinary objects and events are heard by hearing sounds, which are qualities that abstract over such details. According to this view, as Moore falls, the sound decreases in loudness, at least from the height of the diving platform, even though Moore’s scream does not get softer. By hearing the sound that diminishes in loudness while getting a little washed out, one comes to know, somehow, that Moore is moving away, rather than staying nearby. Anscombe’s imitation captures the decreasing volume and the way in which the sound gets washed out, but it is not exactly the same sound as heard when Moore falls. An audiophile’s recording of the event might, upon playback, imitate the sound so well as to fool blindfolded listeners. This trompe l’oreille reproduces the sound, on this view, and this explains why people are fooled. By this account’s lights, auditory perception is a lot like spatial vision if one were only aware of visible figures. An enduring early modern line of thought understands vision in just that way. Awareness of the coins’ intrinsic shapes is at one remove from acquaintance with their visible figures. This view’s popularity faded along with the plausibility of sense-datum epistemology. Moore is visually aware of visible figure, along with the coin’s intrinsic shape and orientation, even when he looks with only one eye. Similar considerations challenge the account of sounds as audible profiles. Yes, perceivers are aware of them, but typical auditory experience makes one aware of much more. The second approach to sounds suggested by audible profiles, then, is that perceivers are aware of both perspectival and non-perspectival qualities at once. This view is, by my lights at least, much more plausible and a much more radical break with existing accounts of sounds. It explicitly rejects the claim that “[w]hatever else we hear, such as ordinary objects or happenings in the environment, we hear by way of or in virtue of hearing the sounds it makes” (O’Callaghan 2007, 13). Sounds are, according to the second approach, features of ordinary objects and happenings, in addition to perspectival features that abstract over them. Since audible profiles are abstractions over features of objects, events, and environs, those features are also natural candidates for being heard. These include dispositions to vibrate in response to being struck, aspects of the events that cause vibrations, such as intensity, and properties of the spaces in which one hears. The cymbal sounds differently when struck with force than when lightly tapped, for example, in a large room, or close-quarters. In some contexts, it is natural to identify what is heard with a feature of the source (the bell has a nice sound), while in others the audible profile is a better candidate—the bell’s sound is quiet and washed out from so far off. Such a catholic view of sounds renders them theoretically uninteresting. Or, to put it differently, it renders them just as interesting as sights are to vision.

92

•

John Kulvicki

The visual world is a complex mix of features that characterize surfaces, media, lights, and shapes. Philosophically interesting questions concern the perception of colors, shapes, transparencies, and the like, not sights per se. Similarly, on this view of sounds, the auditory world is quite complex, and philosophers would do well to focus on aspects of what we can hear, rather than on the nature of sounds per se. This second proposal is spiritually in line with other accounts of sounds as properties (Pasnau 1999, Kulvicki 2008), though these earlier efforts are deaf to perspective. They emphasize the intuition that hearing puts perceivers in touch with non-perspectival features of ordinary objects, such as their dispositions to vibrate in response to being mechanically stimulated. Adding perspectival properties to the mix makes such accounts much more plausible. One worry about identifying sounds with vibratory dispositions is that it doesn’t seem to capture the full range of auditory experience (Kulvicki 2008, §3–4). While auditory constancies strongly suggest we hear stable features of ordinary objects, it seems we also hear aspects of events and the contexts in which those things are heard, too. Audible profiles add exactly what such accounts lack. In fact, Kulvicki (2008), following Pasnau (1999), suggested that sounds are very much like colors. One important piece of that analogy is that sounds are properties of ordinary objects. But another is that sounds are not the only things heard. Color, shape, and all manner of other features figure in visual experience. Auditory experience is similarly complex, or so I suggested, and we should not think that it is limited to sounds. This is perhaps an infelicitous use of the term. It’s just odd to suggest that much of what we hear is not sounds, but something else. Instead, it’s better to recognize that the category of sounds is a mongrel and to focus our efforts at understanding the many kinds of things heard. Auditory experience puts us in touch, primarily, with qualities of ordinary objects, some of which are perspectival, some of which are not. Nudds suggests that “auditory perception, like vision, functions to tell us about objects in our environment” (2010, 109), though he thinks it does this by first letting us know about sounds, which are features of particulars between hearers and ordinary objects. The present suggestion offers a way to understand audition as performing its function without such mediation.

6. Conclusion In some intuitive sense, audition is mediated in a manner that vision is not. This intuition is, upon reflection, unstable. Perspectival properties like visible figures were important to the development of the philosophy of visual perception, and they ought to play a role in the development of our surprisingly nascent thoughts about hearing. Once they are in the mix, it becomes quite plausible that audition is (un)mediated in very much the same way vision is.9

Auditory Perspectives

•

93

Notes 1. Quality splitting becomes appealing once one treats sensations as intentional (Anscombe 1965). D. M. Armstrong (1961, 12–13), Robert Hopkins (1998, ch3), Alva Noë (2004, 83), John Hyman (2006, 75ff), Kulvicki (2006, ch8; 2014a, ch3), and Susanna Schellenberg (2008) all develop Reid’s notion of visible figure in related ways. 2. Some make this phenomenon central to depiction, including Alberti (1435), Leonardo (Kemp 1989), Hopkins (1998), and Hyman (2006). 3. These issues relate to another point of contention in the audition literature. Are sounds the kinds of things that occupy time? See Kulvicki (2014b) for an in-depth discussion of that issue. 4. See, e.g., O’Callaghan (2010, 33–36), who thinks that in such circumstances no sound exists; compare Casati and Dokic (2010). 5. See Casati and Dokic (1994, 2010), O’Callaghan (2007), Sorensen (2008), Pasnau (2009), and Nudds (2010). 6. Schellenberg (2008) suggests that perspectival qualities are at work in audition as well as vision and perhaps other senses, so what follows is a way of fleshing out that suggestion. Berit Brogaard (2010) offers an alternative way of thinking about perspective dependence. O’Callaghan (2007, 147–148) also suggests that audition is perspectival, but he does not seem to think the point is significant for his theory of sounds. 7. See O’Callaghan (2007, 17–18) and Nudds (2010, 72 n3). 8. See, e.g., O’Callaghan (2007) and Casati and Dokic (2010) for the event view and Sorensen (2008) and Nudds (2010, 2014) for the wave view. 9. Thanks to Bence Nanay, audiences at NYU, and the annual Dartmouth-UVM conference, and especially Matt Weiner, for helpful feedback.

References Alberti, L. 1435/1991. On Painting. C. Grayson, trans. London: Penguin. Anscombe, G.E.M. 1965. The intentionality of sensation: a grammatical feature. In R.J. Butler, ed. Analytical Philosophy. Oxford: Blackwell: 158–180. Armstrong, D. 1961. Perception and the Physical World. London: Routledge. Bregman, A. 1990. Auditory Scene Analysis. Cambridge, MA: MIT Press. Brogaard, B. 2010. Strong representationalism and centered content. Philosophical Studies 151: 373–392. Casati, R. and J. Dokic. 1994. La Philosophie du Son. Nîmes: Chambon. Casati, R. and J. Dokic. 2010. Sounds. Stanford Encyclopedia of Philosophy (original entry 2005, major revision 2010). URL = Hopkins, R. 1998. Picture, Image, and Experience. Cambridge: CUP. Hyman, J. 2006. The Objective Eye. Chicago: U. of Chicago Press. Kemp, M. 1989. Leonardo on Painting. New Haven: Yale U. Press. Kulvicki, J. 2006. On Images: Their Structure and Content. Oxford: Clarendon. Kulvicki, J. 2008. The nature of noise. Philosophers’ Imprint 8(11): 1–16. Kulvicki, J. 2014a. Images. London: Routledge. Kulvicki, J. 2014b. Sound stimulants: defending the stable disposition view. In S. Biggs, M. Matthen, and D. Stokes, eds. Perception and Its Modalities. Oxford: OUP: 205–221. Kulvicki, J. forthcoming. Chromatic perspectives. In D. Brown and F. MacPherson, eds. Routledge Companion to the Philosophy of Color. London: Routledge. Moore, G.E. 1922. Philosophical Studies. London: Routledge. Noë, Alva. 2004. Action in Perception. Cambridge, MA: MIT Press. Nudds, M. 2010. What are auditory objects? Review of Philosophy and Psychology 1: 105–122. Nudds, M. 2014. Auditory appearances. Ratio 27(4): 462–482. O’Callaghan, C. 2007. Sounds: A Philosophical Theory. Oxford: OUP.

94

•

John Kulvicki

O’Callaghan, C. 2010. Sounds and events. In M. Nudds and C. O’Callaghan, eds. Sounds and Perception. Oxford: OUP: 26–49. O’Callaghan, C. 2011. Lessons from beyond vision (sounds and audition). Philosophical Studies 153(1): 143–160. Pasnau, R. 1999. What is sound? Philosophical Quarterly 49(196): 309–324. Pasnau, R. 2009. The event of color. Philosophical Studies 142(3): 353–369. Price, H.H. 1932. Perception. London: Methuen. Reid, Thomas. 1764/1801. Inquiry into the Human Mind. 5th ed. Edinburgh: Bell and Bradfute. Russell, B. 1914. Our Knowledge of the External World. London: Routledge. Schellenberg, S. 2008. The situation-dependency of perception. Journal of Philosophy 105(2): 55–84. Sorensen, R. 2008. Seeing Dark Things. Oxford: OUP. Turrell, James. 2012. Twilight Epiphany: Skyspace. Rice University, Houston, TX.

CHAPTER

6

The Non-Visual Senses The Distinctive Role of Sounds and Smells MATTHEW NUDDS

We can contrast two approaches to the non-visual senses. The first approach takes vision to be the paradigm sense, the sense on which our understanding of perception and perceptual experience in general can be modelled. Call this the vision-as-paradigm (VAP) approach. According to VAP, the account we give of visual perception can be generalised to the non-visual senses. The alternative (ALT) approach rejects that idea. According to this approach, there are features of the non-visual senses that cannot be accommodated within an account of perception modelled on vision In this chapter I will suggest that the VAP approach should be rejected. I will not, however, provide a knock-down argument against it. Rather, I will suggest that we must reject it if we are to give a satisfactory account of the non-visual senses. There is not space here for a thoroughgoing discussion of the nonvisual senses. I will focus instead on two senses—audition and olfaction— and describe a feature of both that cannot be accommodated within the VAP approach. The feature in question highlights an important difference between vision and the non-visual senses of olfaction and audition—a difference in the relation between what we perceive and what we are perceptually aware of. The senses in general—and vision, olfaction, and audition in particular—enable the perception of aspects of the material world, but they do so in different ways. Although in the case of both olfaction and audition the objects of our perception can be material objects, or aspects of them, our perception of these objects is mediated by the perceptual awareness of things distinct from them. That is not true of vision. The objects of visual awareness are the concrete 95

96

•

Matthew Nudds

material objects that we take ourselves to see. It is those objects and their properties that are present to us in visual experience. We do not, as Martin says, conceive of the visible world as providing us with objects of visual awareness and attention distinct from the concrete objects we see.1 When I see the colour of an object, I see the object and see it as colored in that way. There is nothing distinct from the object, of which I am aware, which is red. The senses of olfaction and audition are not like this. These senses do provide us with “objects” of perceptual awareness that are distinct from the material objects we take ourselves to perceive. In what follows I will attempt to justify this claim by describing olfaction and audition in more detail, and I will then explain why this matters for our understanding of the non-visual senses. My conclusion will be that the VAP approach should be rejected. A good place to begin consideration of the non-visual senses is with the sense of smell. I can perceive by smell many of the same things that I can see; I can recognise things by smelling them; and I can come to know about the states of things by smelling them. For example, by sniffing at the open door of my fridge I can smell the piece of Camembert that’s in there; I can recognise that that’s what it is; and I can tell that it’s ripe for eating. But in smelling the Camembert, it is not a piece of cheese—the concrete thing itself—that is an object of my olfactory awareness; it is the smell produced by the cheese. If asked to describe how the cheese smells, I might say that it smells rank. What I say is best understood not as characterising the cheese itself, but as characterising the smell of the cheese. Saying that the cheese smells rank doesn’t imply that the cheese is rank. Quite the opposite. A good Camembert produces a smell that has that quality. In a similar way, if I say that a perfume smells sweet what I say doesn’t imply that the perfume is sweet. In general, in characterising how things smell we characterise the smells they produce rather than the things themselves. Although we can smell individual things—a piece of cheese—we don’t treat smells as individuals. We think of the identity of smells as a matter of qualitative resemblance in olfactory properties. If two things produce smells that are absolutely indiscernible, we consider them to have the same smell. Suppose I also have a piece of vegan “cheese” in my fridge and that it has a smell that is indistinguishable from the real thing. In smelling the Camembert and the vegan “cheese”, I don’t encounter two distinct smells. Rather, in smelling the vegan “cheese”, I encounter the very same smell again, produced by some quite different thing. Smells, unlike the things we smell, are not individuals.2 We conceive of smells as distinct from what produces them. In walking into a room, I might notice a smell. I might have no idea what is producing it. Maybe nothing in the room is—it is the lingering smell of a ripe Camembert that was there earlier but has now been eaten. The cheese is gone, but the smell remains. I can attend to the smell, noting its rankness. If I know what produced it, I might say that I can still smell the cheese (that was in the room,

Non-Visual Senses

•

97

now non-existent). In saying that, I mean something to the effect that I can smell a smell that was produced by the cheese. Smelling such lingering smells is commonplace. It follows from the fact that smells can linger in this way; that they are distinct from the things that produce them. In smelling the lingering smell of the cheese, I am not perceptually aware of the cheese itself. The cheese itself isn’t something (couldn’t be, in the circumstances) I can attend to in my experience. The sense of smell, like vision, enables the perception of concrete material things like the cheese. That, it seems plausible to say, is what the sense of smell is for. But the sense of smell enables the perception of concrete material things in a very different way to vision. Unlike vision, the “objects” of olfactory experience are not the concrete material things that we take ourselves to smell. Rather, what is present in olfactory experience—the things of which we are olfactorily aware and to which we can attend—are the smells or odours associated with them. The olfactory world offers us objects of olfactory awareness and attention distinct from the concrete objects we smell. Although smells are what is immediately present in olfactory experience, perceiving smells is not what the sense of smell is for: perceiving smells is merely the means to an end; it is the way in which the sense of smell informs us about the world. To see why this is so requires some understanding of how the sense of smell enables the perception of material things, and so of why we smell the smells we do. Things that produce smells do so because they are composed of numerous volatile chemical components which can escape into the air.3 When we breathe or sniff, and during eating and drinking, air containing these volatile chemicals is drawn over the olfactory epithelium, which contains olfactory receptor cells. Humans are thought to have around 300 different types of receptors, each receptive to a different range of chemicals. Pure chemicals activate several different receptor types; a complex mixture of chemicals activates many more. The sensory signal that is produced by this activation is both complex—involving many different receptor types—and has a temporal profile that reflects how quickly different chemicals interact with the receptors. The resultant sensory signal is processed within the olfactory bulb to produce an output that has both a temporal profile and a two-dimensional spatial organisation. We can think of this as the pattern of activation produced by the smell. Although the odours that we ordinarily experience—the smell of coffee, or of cheese—are chemically complex, and although the air we breathe typically contains a complex soup of volatile chemicals given off by many different things, we are able to discriminate and identify the smells of individual things and substances in our environment, and we experience these smells as unitary and not as a mixture of the individual chemical components that make them up. We do so because olfactory processing identifies the complex signal that corresponds to the odour of familiar substances and distinguishes it from a background of competing odours and chemical noise. Central to

98

•

Matthew Nudds

this is a process of pattern recognition: the olfactory system stores previously encountered patterns of activation so that the pattern of activation produced by smelling an odour can be matched against them. If there is a match, we have an experience of a unified odour discriminable from other odours and distinct from the background. If there is no match our olfactory experience is rather indeterminate and we are unable to discriminate any odour from the background. The result of the olfactory system “recognising” a pattern is that we experience a distinct and unitary smell. This doesn’t mean that we can identify what produced the smell. We may “recognise” a smell in that we experience it as unitary and perhaps find it familiar,4 and yet fail to recognise it in that we are unable to able name it or to identify what produced it. The learning required for discriminating smells—for experiencing them as a distinct and unitary and familiar—is not the kind semantic or conceptual learning required for identification. The olfactory system is able to quickly learn the patterns associated with novel odours so that when we re-encounter them we experience them as distinct and unitary.5 The patterns we learn normally correspond to the mixture of chemicals given off by individual things and substances in our environment. It is our repeated encounters with those things—and so with the mixture of chemicals they give off—that enables the olfactory system to learn the pattern. As a consequence, the unitary smells that we experience—the smells that we can discriminate from the background—normally correspond to things and substances in our environment. It is the establishment of these correspondences that enables the olfactory perception of concrete material things and substances in our environment. Because the capacities to identify discriminate smells are acquired through experience and learning, there are differences across individuals. Oenophiles can discriminate the smells of different wines far more finely than I can. We might think of them as having a better sense of smell than I do: as a result of wide experience and careful practice they can make discriminations where I cannot; they smell more smells than me. But not only do some individuals smell more smells than others: different individuals experience different smells altogether. Here is an example. People who are exposed to certain odor mixtures (for example, cherry and mushroom) come to experience smells that combine the two. For them, mushrooms come to smell cherry-like. As a result, they experience a greater similarity between the smell of mushrooms and of cherries, and have a reduced ability to discriminate the smell of mushrooms from cherries. It seems that once an odour mixture has been encoded, an element that is part of the mixture will activate the encoding for the mixture as a whole. A consequence is that two components of a mixture come to share odour qualities and to be similar to each other. This shows both that past experience determines

Non-Visual Senses

•

99

the qualities or character of the smells we experience, and that the qualities of a smell can change as a consequence of past experience.6 It follows that two individuals who have been consistently exposed to different odour mixtures will come to have different odour experiences, and have experiences of odours that stand in different relationships of similarity. These differences emerge across cultures. For example, in an experiment in which participants were asked to describe the qualities of different odours, Japanese participants described aniseed as “disinfectant-like” and Indian ink as “medicinal”, whereas German participants described them quite differently. Differences in the past experiences of the two groups leads to differences in the qualities they experience odours as having.7 The sense of hearing is in some ways like the sense of smell. I can hear many of the same kinds of things that I can see. I can hear the book that I knocked of the desk hitting the floor; I can hear the car passing by in the street; I can hear a bird singing in the tree outside. Unlike vision, but like smell, I hear something (a material object or event) only by hearing something else—the sound it produces. I hear the book by hearing the sound it makes striking the floor; I hear the car by the hearing the sound made by its engine; I hear the bird by hearing the sound of its singing. Sounds have a characteristic appearance that is determined by their acoustic properties—properties that include loudness, timbre, and sometimes pitch—and by the way they unfold and change over time. Unlike smells, we treat sounds as individuals. Two sounds can be qualitatively identical and yet distinct. If I tap twice on my desk, I hear two sounds that are identical in their acoustic properties, and not simply the same sound again. In hearing an everyday sound my attention is often directed to the source of the sound. In saying how the dropped book sounds, I may, for example, say that it sounded heavy. In doing so I am saying something about the book—that it is heavy—and characterising the sound indirectly, in terms of what produced it. In the case of everyday sounds, it can be difficult to describe the character of the sound itself in anything other than this indirect way. That’s not true in the case of musical sounds where my attention is usually directed to the sounds themselves, rather than to their sources. In listening to music, I attend to the melody or to the rhythm constituted by the sounds and may ignore what produced them, and it is easier to characterise these sounds directly, in terms of their acoustic properties. Even in the case of musical sounds, it is usually possible to attend to what produced them. I can, for example, listen to the way the musician is playing, or listen in order to discover what instrument it is that is playing the melody. Sounds, like smells, are distinct from the things that produce them. When an object is struck, it produces a sound. The striking causes the object to vibrate. The object’s vibration is complex—made up of different frequency components that correspond to different modes of the object’s vibration—and

100

•

Matthew Nudds

lasts until the energy of the striking has dissipated and the object returns to a state of equilibrium. The object’s vibration causes a pressure wave that propagates through the surrounding medium—typically air—and which can be detected by any suitably placed perceiver. The perceiver will hear the striking of the object only if she hears the sound it produces. The striking and the sound are distinct things—distinct events. Some philosophers have argued that sounds are events instantiated by the objects that produce sounds8—that is, that sounds are the vibratory events instantiated by objects when they are caused to vibrate (such vibratory events are distinct from the events—such as the striking—that cause them). I think that they are wrong to do so, but the dispute between the VAP and ALT approaches is independent of that.9 The sense of hearing, like vision, enables the perception of concrete material things such as the book hitting the floor. That is what hearing is for. But the “objects” of auditory experience are not the concrete material things that we take ourselves to hear. Rather, what is present in auditory experience—the things of which I am auditorily aware—are the sounds produced by them. I hear concrete material things by hearing the sounds they produce. We hear sounds when the pressure wave caused by an object’s vibration reaches our ears. The ears function to capture the pressure wave and funnel it to the basilar membrane, which detects the different frequency components that make it up. The output of the basilar membrane is a signal that encodes time-varying information about each frequency component detected in the pressure wave. This signal serves as input to further auditory processing. Our experience is of individual sounds, but the pressure wave that reaches our ears is typically the result of many different vibratory events occurring simultaneously in our environment. To get from the complex mixture of frequency components to individual sounds requires the auditory system to organise and group the frequency components that make up the pressure wave. It does so in such a way that frequency components that are likely to have a common source—likely to have been produced by the same event— are grouped together. This grouping process is both simultaneous (grouping frequency components at a time) and sequential (grouping together simultaneously grouped frequency components over time). The process is both bottom-up (drawing on relationships between individual frequency components) and top-down (drawing on information from previously encountered sequences).10 The consequence of this grouping process is that the sounds we experience normally correspond to the events that produced them. It’s in virtue of this that we can hear those events, and so the objects, that produce sounds. Although the sounds we experience normally correspond to what produced them, this is not always the case. In some circumstances the auditory system groups together frequency components that were not in fact produced by the

Non-Visual Senses

•

101

same event, and when that happens we have an experience of a sound that does not correspond to any event that produced it. During the evolution of the auditory system the circumstances in which this occurred are likely to have been rare. However, with the advent of artificially produced and reproduced sound, they have become commonplace (stereo loudspeakers, for example, produce experiences of sounds that don’t correspond to events that produced them). To hear a sound that doesn’t correspond to what produced it is to experience a kind of auditory illusion: that of merely seeming to hear sound-producing events. It is possible to have the auditory experience of someone coughing over on your left when in fact there’s no one there. The way an object vibrates carries information about that object and about the event that caused it to vibrate: information, for example, about the size, shape, and material composition of the object—and about the force and nature of the event. That information is preserved in the pattern of frequency components grouped together by the auditory system as having been produced by the same event. These groups of frequency components serve as input to auditory processes that recover this information. As a result we are able to perceive the sources of sounds and their features.11 We might be tempted to assimilate auditory perception of sound sources to olfactory perception of the sources of smells, and so to think that our awareness of the character of the sound grounds our recognition of what produced the sound. As a general account of auditory perception, that cannot be right. We can hear where a sound comes from (we can hear where the source of the sound is located), but we don’t do so on the basis of being aware of the character of the sound. Two qualitatively identical sounds can appear to come from different places, so we hear their sources as having different features (having different locations), without there being any difference in the character of the sounds that could ground our recognition of those different features. Whatever account we give of the perception of the location of sound sources on the basis of hearing sounds should, I suggest, be extended to hearing other features of sound sources. That’s true even in those cases where the character of the sound does vary with changes in features of the source. Whilst it’s true that sounds produced by striking something forcefully have a different character to those produce by a less forceful striking, it doesn’t follow that our awareness of the difference in the sounds explains our perception of the difference in the features of the source events. These sketched accounts of olfactory and auditory perception are sufficient to show that they differ from vision in a fundamental way. Whereas in vision it is the concrete material objects that we take ourselves to see that are the objects of perceptual awareness, in audition and olfaction the “objects” of our perceptual awareness are not the material objects we take ourselves to hear and smell; they are the distinct “objects”—sounds and smells—that mediate our auditory and olfactory perception of material objects.

102

•

Matthew Nudds

Proponents of the VAP account will reject this suggestion. They might accept that the “objects” of audition and olfaction are sounds and smells but will insist that sounds and smells are themselves elements of the material world so that, in being perceptually aware of them, we are perceptually aware of the aspects of the material world. Suppose, for example, that we take sounds to be material events of a certain kind—events occurring to material objects. We can then think of audition as involving the perceptual awareness of material events, in just the same way that we can think of vision as (in some cases at least) involving the perceptual awareness of material events. Understood in this way, audition is like vision, and the only difference between audition and vision is a difference in which events we are perceptually aware of and what properties we are aware of those events as having. Similarly, suppose we take smells to be material properties (properties of material things, if not of objects). We can then think of olfaction as involving the perceptual awareness of material properties, just as we can think of vision as involving the perceptual awareness of material properties, such as color or texture. Understood this way, olfaction is like vision. The ALT account emphasises the fact that we don’t just perceive sounds and smells; we perceive their sources too. But proponents of the VAP account won’t deny that.12 They will claim that we can perceive sources in virtue of their standing in the right kind of—typically causal—relation to the sounds and smells of which we are perpetually aware. This kind of indirect perception is not unique to audition and olfaction but can occur in vision too.13 The proponent of VAP can argue, therefore, that I have only shown that vision differs from the other senses with respect to the things we are perceptually aware of. The non-visual senses may differ from vision in their “objects”, but the fundamental nature of the perceptual relation, and of perceptual experience, is the same for these senses as it is for vision. It might seem, then, that the only thing that could be at issue between the VAP approach and the ALT approach is the metaphysical status of sounds and smells: the only grounds the ALT approach could have for rejecting VAP is that it rejects the claim that sounds and smells are aspects of the material world. That would mean the ALT approach is committed to the claim that sounds and smells are non-material. This response mistakes what is at issue between the VAP and ALT account. What is at issue is not the metaphysical status of sounds and smells but how we should view the purpose or function of the non-visual senses. According to the ALT approach, sounds and smells are essentially mediators. We perceive them only in virtue of the fact they play a role in our perception of their sources. The function or purpose of auditory and olfactory perception is not the perception of sounds and smells per se; it is the perception of the sources of sounds and smells, and we experience sounds and smells only as a consequence of the way audition and olfaction carry out this function. The process that leads to our experience of sounds is one that groups together frequency components that

Non-Visual Senses

•

103

are likely to have been produced by the same event in order to recover information about that event. The process that leads to our experience of unitary smells is one that generates templates for chemical mixtures that are likely to originate in a single material substance, in order that we can identify and track that substance. If we assimilate sounds and smells to the objects of perception, if we treat them as analogous to the things we see, then we ignore their nature as mediators. If we accept VAP and say that audition and olfaction are for the perception of sounds and smells, then we miss something important about what those senses do for us—that their most fundamental function is to tell us about things other than sounds and smells. The dispute between the VAP and the ALT approach is not, therefore, a dispute about the status of sounds and smells. It is a dispute about the purpose or function of audition and olfaction. According to VAP, auditory and olfactory perception are for the perception of sounds and smells; sounds and smells are aspects of the material world; so auditory and olfactory perception are for the perception of these distinctive aspects of the material world. Therefore, the fundamental task of an account of olfaction and audition is to understand the perception of sounds and smells. It may be that we can perceive the sources of sounds and smells, but we can understand our perception of sounds and smells independently of understanding our perception of their sources. According to ALT, auditory and olfactory perception are for the perception of the sources of sounds and smells. The sources of sounds and smells are material objects, events, and stuffs. So auditory and olfactory perception are for the perception of the very same aspects of the world that we perceive visually. Therefore, the fundamental task of an account of olfaction and audition is to understand the perception of the sources of sounds and smells. We perceive sounds and smells as a consequence of our perception of their sources, and we can’t understand our perception of sounds and smells independently of understanding the role they play in our perception of their sources. Of course, the things we see can also play a mediating role, but it is not essential to their nature, or the nature of visual perception, that they do so. I can see the mouse under the rug in virtue of seeing the bulge it makes in the rug, so I see the mouse by being perceptually aware of something distinct from it, the bulge in the rug. My perceptual awareness of the bulge mediates my perception of the mouse—the bulge plays a mediating role—but it’s not essential to my awareness of the bulge in the rug or to the bulge itself—that it plays this role.14 We can’t explain the nature of auditory and olfactory perception independently of understanding their mediated nature, and that means we can only understand the perception of sounds and smells (and so the nature of sounds and smells) if we understand their role in enabling the perception of their sources—of material objects, events, and stuffs. I have described above

104

•

Matthew Nudds

how what smells someone perceives and what character they perceive them to have is a function of the way their olfactory system responds to the mixture of chemicals it detects. This response is partly determined by the perceiver’s past experiences, in such a way that two perceivers exposed to the same mixture of volatile chemicals might perceive different smells. That means we don’t have any way to determine what smells there are in the mixture independently of the experiences of suitably placed perceivers. Suppose we ask why perceivers perceive the smells they do. There are many different ways the olfactory system could group a complex chemical mixture. Why does it group one way rather than another? We can answer that by appealing to the role these chemical groupings have in mediating the perception of substances in the perceiver’s environment. If we abstract away from that role, then the grouping is arbitrary, and we have no explanation of why we experience the smells we do. So we can’t explain why perceivers experience the smells they do—or determine what smells there are—other than in terms of their role as mediators in an olfactory process whose function is to track the sources of smells. Something similar is true of sounds. There are different ways the auditory system could group the complex mixture of frequency components that reaches the ears. Why does it group one way rather than another? We can answer that by appealing to the role groups of frequency components have in mediating our perception of the sources of sounds. The auditory system groups together frequency components that are likely to have been produced by the same source event. If we abstract away from that role, then the grouping is arbitrary, and we have no explanation of why we experience the sounds that we do. It might be suggested that what vibratory events an object instantiates is not an arbitrary matter, and that we can explain why we perceive the sounds we do in terms of our perception of those vibratory events. But what vibratory events an object instantiates is only non-arbitrary relative to the events that caused the object to vibrate.15 So if we ask why perceivers perceive the sounds they do, the answer cannot be because those vibratory events are occurring in their environment; they perceive those sounds because they correspond to the events that caused them. We can hear sounds even in the absence of any causal events relative to which we could individuate vibratory events. Suppose we ask why we experience the sounds we do when we listen to a recording played on loudspeakers. The answer cannot be that we perceive the vibratory events instantiated by the loudspeaker cones—there is simply no way to individuate any such events. We experience the sounds we do because the auditory system groups the pattern of frequency components that constitutes the cones’ vibration in such a way that they correspond to the source events whose occurrence would best explain that pattern. There is no non-arbitrary way to individuate vibratory events in abstraction from those supposed source events. The only way to divide up the complex vibration produced by the loudspeaker is in terms of the events that

Non-Visual Senses

•

105

would normally have caused that complex pattern of vibrations. So we can’t explain why perceivers experience the sounds they do other than in terms of a process whose function is to track the events that are the (supposed) sources of those sounds. What this shows is that we cannot simply assimilate sounds and smells to the objects of perception, and hence that the proponent of VAP cannot respond that vision differs from the other senses only with respect to the things we are perceptually aware of. In both audition and olfaction what we experience is the result of a process that structures or organises the perceptual medium, and does so in order to track material objects and events. That means that an account of the non-visual senses must explain their nature as mediated senses: we need to understand the role of the medium itself and give an account of how our perceptual awareness of sounds and smells enables our perception of things other than sounds and smells. Any attempt to give an account of our perceptual experience simply in terms of the material objects and events that we perceive will simply miss what is characteristic of the nonvisual senses.

Notes 1. Martin (2010, 188). There are objects of visual awareness that are not concrete material objects—viz., purely visible phenomena such as shadows, lights, highlights, holograms. Unlike sounds and smells, these visibilia do not play a mediating role—we are aware of them together with or alongside concrete material objects. 2. Martin (2010, 187) suggests that we should think of them as pure universals or qualitative stuffs. 3. For example, Parmigiano-Reggiano cheese has around 160 volatile components, roughly 30 of which are detectable by smell. Aged Rioja red wines have over 800 volatile components, roughly 60 of which are detectable by smell. See Stevenson and Wilson (2007, 1822). 4. The experience of not being able to identify a smell that is familiar is very common. A quick sniff of the contents of jars in your spice cupboard should convince you of this—there will be spices that have a very familiar smell, but which you will struggle to identify without looking at the label. 5. We can learn new smells in many ways, including passive exposure as well as careful attention and observation. 6. See Stevenson and Boakes (2003), Stevenson and Wilson (2007). In a similar way, odours can acquire the quality of “sweetness”. Participants who were presented with novel odours together with a sucrose solution that they sipped by mouth later described the odour as smelling sweeter when sniffed than odours that were presented together with citric acid or water. The odour had acquired a “sweet” quality in virtue of being encoded as a multisensory representation with sweetness (Stevenson and Boakes 2004). 7. Stevenson and Boakes (2004), Stevenson and Wilson (2007, 1827) remark that “our experience of odour quality—what we perceive—is strongly influenced by experience”. 8. See, for example, Casati and Dokic (1994) and O’Callaghan (2007). 9. They are wrong to do so because sounds are independent of the objects to which these vibratory events occur (see further Nudds 2015). Those who insist that sounds are events occurring to objects often do so in effect because they want to accommodate audition within the VAP account. The reasons I outline for rejecting the VAP account don’t require that we reject that view.

106

•

Matthew Nudds

10. For details of the empirical support for this claim, see Bregman (1997, esp. ch.3). There is a more detailed discussion of this aspect of auditory perception in Nudds (2015). 11. For surveys of relevant empirical literature see McAdams (1993) and Lufti (2008). 12. They don’t deny it. See, e.g., Batty (2010), O’Callaghan (2011), and Lycan (2014). O’Callaghan argues that in the case of audition the relation is constitutive rather than causal. 13. Putative examples include seeing something under a carpet by seeing the bulge it makes (see below), seeing the person in the distance by seeing the flag the person is waving, and seeing that the tank is empty by seeing the gauge. It may be that these cases cannot all be explained in the same way. 14. Note that this is not a case of “seeing that”. I can see the mouse under the rug without seeing that there is a mouse under the rug: I might have no idea that it’s a mouse I see. 15. See Nudds (2015) for further discussion of this under-appreciated point.

References Batty, C. (2010). ‘A Representational Account of Olfactory Experience’. Canadian Journal of Philosophy, 40: 511–538. Bregman, A. (1997). Auditory Scene Analysis: The Perceptual Organization of Sound. Cambridge, MA: MIT Press. Casati, R. and Dokic, J. (1994). La philosophie du son. Nımes: Chambon. Lufti, R. (2008). ‘Human Sound Source Identification’. In W. Yost, A. Popper, and R. Fay, eds., Auditory Perception of Sound Sources. New York: Springer: 13–42. Lycan, W. (2014). ‘The Intentionality of Smell’. Frontiers in Psychology, 5: 1–8. Martin, M.G.F. (2010). ‘What’s in a Look’. In Bence Nanay, ed., Perceiving the World. Oxford: Oxford University Press: 160–225. McAdams, S. (1993). ‘Recognition of Sound Sources and Events’. In S. McAdams and E. Bigands, eds., Thinking in Sound. Oxford: Oxford University Press: 146–198. Nudds, M. (2015). ‘Audition’. In Mohan Matthen, ed., The Oxford Handbook of the Philosophy of Perception. Oxford: Oxford University Press: 274–293. O’Callaghan, C. (2007). Sounds. Oxford: Oxford University Press. O’Callaghan, C. (2011). ‘Hearing Properties, Effects, or Parts’. Proceedings of the Aristotelian Society, 111: 375–405. Stevenson, R.J. and Boakes, R.A. (2003). ‘A Mnemonic Theory of Odour Perception’. Psychological Review, 110: 340–364. Stevenson, R.J. and Boakes, R.A. (2004). ‘Sweet and Sour Smells: Learned Synesthesia between the Senses of Taste and Smell’. In G. Calvert, C. Spence, and B. Stein, eds., The Handbook of Multisensory Processes. Cambridge, MA: MIT Press: 69–83. Stevenson, R.J. and Wilson, D.A. (2007). ‘Odour Perception: An Object Recognition Approach’. Perception, 36: 1821–1833.

PART

IV

The Multimodality of Perception

CHAPTER

7

Enhancement Through Coordination

CASEY O’CALLAGHAN

1 Recent work in perceptual psychology, neuroscience, and philosophy challenges the independence of our senses. Cross-modal effects, such as illusions triggered using the intersensory discrepancy paradigm, demonstrate that stimulation to one sensory system can affect another and reshape perceptual experience. Neural processes associated with one sense interact extensively with those associated with others, and integrating information across senses recruits ample cognitive resources. Matthen (this volume) describes this twenty-first-century boom in experimental research on multisensory perception in his introduction (see also O’Callaghan 2012). Beyond perceptual mechanisms and processes, a current controversy among philosophers concerns whether, how, and to what extent perceptual consciousness is multisensory. On one hand, one sense might causally but not constitutively affect experience associated with another. Some maintain that experimental evidence is compatible with the claim that perceptual experience at each time is unisensory or minimally multisensory (see Spence and Bayne 2015). On the other hand, perceptual experience might reflect extensive subpersonal coordination and be richly or deeply multisensory. I defend the latter. Section 2 distinguishes three ways in which perceptual experience may be multisensory, and it presents evidence to support the claim that mature species-typical perceptual experience in humans is richly and deeply multisensory. Section 3 explains why this matters. Section 4 raises two outstanding issues concerning psychological taxonomy for multisensory perception: how to 109

110

•

Casey O’Callaghan

type richly multisensory experiences by modality according to their character, and why multi-sensory effects involve perception rather than extra-perceptual cognition.

2 If you could use only one sense at a time, then your perceptual consciousness at each moment would be unisensory. However, humans typically use more than one sense. You can smell beer while seeing a doorknob or watch an airplane while listening to conversation. If you can perceive with more than one sense at a time, then unless consciousness requires attention and attention is restricted to one modality at a time, in principle you could perceive consciously using more than one sense. Whenever perceptual consciousness is associated with more than one sense modality at a time, say that it is minimally multisensory (or minimally multimodal—when restricted to exteroceptive modalities of sensory perception, here I’ll treat “multimodal” as interchangeable with “multisensory”). Sensory processes might interact extensively while every conscious aspect of a perceptual experience remains associated with some specific modality or another. Therefore, since a subject’s consciousness at a time typically is unified, human perceptual consciousness might be exhausted by features that are associated with each respective sense modality along with whatever accrues thanks to simple co-consciousness. I have called this claim The Thesis of Minimal Multimodality (O’Callaghan 2015; see also Matthen, this volume, Chapter 8, sections II and III). Is perceptual consciousness more than minimally multisensory? We may distinguish two additional ways in which perceptual experience may be constitutively multisensory. First, a perceptual experience that is associated with a given modality on a specific occasion may depend constitutively on another sense modality. For instance, an auditory episode with a certain character might be cross-modally parasitic in that it requires either prior or concurrent visual processing without which it would not be possible. So, distinguish an auditory experience that is presently and historically exclusively auditory—a diachronically purely auditory experience—from an auditory experience that occurs within a rich mix of past and presently visual, tactual, olfactory, and gustatory experiences. For a given pattern of sensory stimulation, the auditory experience of a subject whose perceptual experience over time is purely or exclusively auditory may differ in character from the auditory experience of a subject who enjoys the typical range of visual, tactual, olfactory, and gustatory episodes. Even if the other senses are blocked or anesthetized, making an otherwise typical subject’s perceptual experience at a time merely and wholly auditory, that synchronically auditory experience may differ from the auditory experience of a subject with a historically and presently

Enhancement Through Coordination

•

111

(diachronically) purely auditory experience. If you only ever could hear but not see, touch, taste, or smell, then your auditory experience could differ from how it actually now is (see, e.g., Shams et al. 2011). For example, an intermodal variety of so-called amodal completion may affect how you auditorily experience an event that is visible but unseen, just as seeing a region to be part of a surface that continues behind an occluder may affect how you visually experience it. Thus, hearing a spoken utterance to be the sort of thing that has visible features may impact one’s auditory experience of it (see also Matthen’s discussion of space and place, this volume, Chapter 8, I.2). Or, the capacity to perceptually experience a given feature through one modality may be cross-modally parasitic on the capacity to perceive that feature through another modality, perhaps by perceptual learning. Matthen’s discussion (this volume, Chapter 8, I.1) of hearing a more precise or determinate spatial location thanks to vision provides one example. Seeing solidity thanks to touch may provide another. Or, vision might enable concepts or thoughts that through cognitive penetration affect the character of auditory experience. Or touch might make possible actions that alter the structure of conscious vision. If so, then an experience that is associated with one modality on a given occasion can depend upon past or present sensory episodes that are associated with another modality. Any such cross-modal dependence implies that a presently and historically purely unisensory visual experience, say, could differ in character from the current visual experience of a typical human subject who has a rich background of perceiving using all of the senses. Thus, if modality-specific features include only those that do not require other senses, not every aspect of a perceptual experience is modality specific. Whenever an episode inherits character from another modality, this is a noteworthy respect in which conscious perceptual experience is constitutively multisensory. Perceptual experience at each time nevertheless might be exhausted by that which is associated with a given modality. A visual experience that depends on past or current tactual awareness is still a visual experience (even if it has features drawn from other modalities). Accordingly, it could be that each feature of a conscious perceptual episode on a given occasion is associated with some modality or another, even while some such features depend on other senses. This preserves a weaker version of The Thesis of Minimal Multimodality. However, consider a second way in which perceptual experience may be constitutively multisensory. It may be more than a sum of modality-specific parts. Since I think atomism fails in part for the reasons Matthen describes (this volume, Chapter 8, especially III and IV), I prefer to state the argument neutrally in terms of features, by which I mean attributes or parts, of conscious perceptual episodes. So, the conscious features of a perceptual episode may not be exhausted by those that are associated with a given modality on that occasion plus whatever accrues thanks to simple co-consciousness. For instance, one’s overall perceptual experience at a time may have features that no

112

•

Casey O’Callaghan

corresponding merely visual, merely auditory, merely tactual, merely olfactory, or merely gustatory experience could have independently, even if we make an allowance for cross-modal dependence and for whatever accrues thanks to simple co-consciousness. For example, intermodal feature binding awareness involves perceptually experiencing something common to bear features perceived using different senses (see O’Callaghan 2014). So, you may visuo-tactually experience as of something’s being both red and rough, or audio-visually experience as of something’s being both loud and bright. By contrast, it is possible to see something’s being red while feeling something’s being rough, or to hear something’s being loud while seeing something’s being bright, without perceptually identifying what is seen and felt, or heard and seen. Such cases differ phenomenologically from the perceptual experience of intermodal feature binding. Thus, in experiencing binding, the perceptually apparent numerical sameness of what is multisensorily perceived does not accrue thanks to simple co-consciousness. Therefore, not every phenomenal feature of a conscious perceptual episode is associated with a specific modality or accrues thanks to mere co-consciousness. Moreover, there could be novel feature instances that are perceptible only through the coordinated use of multiple senses. For instance, there is evidence that certain relational features are perceptible multisensorily. Simple temporal and spatial relations—such as simultaneity, order, and relative direction and distance—may be perceptible not only when the relata are perceived through one sense but also when they are perceived through differing senses (see, e.g., Bertelson 1999; Stone et al. 2001; Spence and Squire 2003; cf., Spence and Bayne 2015). In addition, it has been demonstrated experimentally that a novel audio-tactile musical meter is perceptible when subjects are presented with distinct audible and tactual metrical patterns (Huang et al. 2012). The same has been claimed for both intermodal rhythms and intermodal motion (Guttman et al. 2005; Harrar et al. 2008; cf., Huddleston et al. 2008). As a final example, causal relations may be perceptually apparent through a single modality, such as vision, and nothing obvious prevents a causal relation’s being perceptually apparent intermodally, as in perceptually experiencing a visible event to cause a sound (Nudds 2001). This seems especially evident when combined with a case involving apparent binding. Suppose you audio-visually perceive one event as such and then audio-visually perceive another event as such. And suppose you can visually perceptually experience the first event to cause the second and auditorily perceptually experience the first to cause the second. Given that you perceptually identify the visible and audible event at each time, it also is plausible that you could perceptually experience the visible event to cause the audible event. A case in which you multimodally perceptually experience an instance of a relational feature whose relata are accessible through different modalities differs consciously from a case in which you perceptually experience the relata but

Enhancement Through Coordination

•

113

not the relation. Thus, some conscious multisensory perceptual experiences have features beyond those that could be instantiated independently by any corresponding unimodal experience under equivalent stimulation and that do not accrue thanks to mere co-consciousness. Therefore, such a multisensory perceptual experience is not a simple co-conscious sum of modality-specific components. There also could be features of a novel type only perceptible multisensorily. Flavor is a good example because flavors are not fully perceptible thanks to any sense working on its own (see Smith 2015). Perceiving flavor involves at least taste, smell, and somatosensation working together to enable awareness of complex attributes such as the distinctive mintiness of mint, the sweet heat of a roasted chili pepper, or the buttery oak of a California Chardonnay. Flavor experiences plausibly involve unique qualitative characteristics beyond those that could be revealed by any corresponding unimodal experience. If so, the character of experiencing the distinctive mintiness of mint is not merely that of co-consciously tasting, smelling, and feeling at once. A multisensory flavor experience may have features beyond those associated on that occasion with each of the respective modalities. If it is possible to multimodally perceptually experience intermodal feature binding, novel feature instances, or novel feature types, then not every conscious multisensory perceptual episode is a simple co-conscious fusion of what is associated on that occasion with each of the respective modalities. A perceptual experience may be constitutively multisensory in that it cannot be factored without remainder into experiences associated with each of the individual sense modalities plus whatever accrues thanks to mere co-consciousness. Thus, I concur with Matthen’s (this volume, Chapter 8, III) rejection of what he calls “Empiricist Atomism”. Since mere co-consciousness suffices only for composite experiences, there is some constitutively multisensory experience that is not merely a composite of simple modality-specific experiences. So, a perceptual experience may be more than minimally multisensory in at least two ways. First, an experience that is associated with one modality may not have been possible if not for the other senses. That is, it may have a character that no corresponding presently and historically unisensory perceptual experience could have. Second, a multisensory perceptual experience may have features beyond those that are associated on that occasion with each of the respective modalities and that do not accrue thanks to simple co-consciousness. Perceptual experience thus may be multisensory in more than the minimal respect.

3 Suppose that conscious perceptual awareness is richly multisensory in these ways. This has two noteworthy sorts of consequences.

114

•

Casey O’Callaghan

First, projects that aim to focus on and theorize about one or another modality in abstraction or isolation from the others face serious methodological problems. Philosophers of perception typically choose a modality—usually vision—and talk about it alone. The objects of visual awareness, the representational content and phenomenological character of visual experiences, and the epistemic status of visual episodes have dominated philosophical discussions of perception during the past five decades. However, perception is richly multisensory. This means that there is no studying vision itself entirely or wholly in isolation from the other modalities. Seeing happens in the context of other modalities, and even a presently wholly visual episode is shaped by a history of thoroughly multisensory awareness. A visual episode may depend, causally or constitutively, historically or presently, upon other senses. Such dependence may explain, alter, or confound claims about the nature of visual processes, visual objects, visual content, visual phenomenology, and the epistemic status of apparently visual episodes. For instance, theorists debate the nature of visual objects. Does visual awareness target material bodies as such, or does vision fail to distinguish material bodies from ephemera such as shadows, rainbows, holograms, and holes? If visually experiencing a material body as such depends constitutively on tactual awareness of solid objects, then the debate cannot be settled by focusing on vision alone. It requires tackling vision’s relationship with touch. To take another recent example, philosophers have appealed to phenomenological contrasts between pairs of visual experiences in debating whether visual experience itself represents high-level properties. Intermodal interactions and mutual dependence can affect phenomenology in ways that confound both the claim that the relevant contrast is visual and the conclusion that visual content explains an experiential difference. Thus, no perceptual modality—not even vision itself—can be understood exhaustively without comprehending the range of its relationships and the manner of its interactions with other senses. Moreover, perception’s richly multisensory character also means that a full account of perceptual awareness requires more than collected independent treatments of the several senses. Philosophers until recently have presumed that once we have explained visual experience, auditory experience, tactual experience, olfactory experience, and gustatory experience, along with the unity of consciousness, then we will have explained perceptual experience. However, some conscious perceptual episodes have features beyond those that are associated with each of the respective modalities and that are not the result of simple co-consciousness. Thus, no complete account of perceptual awareness ends with assembling autonomous accounts of the senses. Its richly multisensory character means that current theorizing about perception just gets interesting once it has addressed each of the senses.

Enhancement Through Coordination

•

115

The second consequence is more constructive. We gain new perceptual capacities through the coordinated use of multiple senses. It could have been that the senses functioned independently, and perceptual awareness might have been structured as a simple co-conscious fusion of experiences associated with each of the modalities. In fact, however, the richly multisensory nature of perception serves to improve and to extend it. For instance, some cross-modal processes enhance the overall reliability of perception. Suppose two senses respond to a common feature but differ in accuracy or reliability, such as when vision and audition detect the location of an event. Under such conditions, if a conflict arises, perceptual processes tend to weight the perceptual result in deference to the more reliable modality (Welch and Warren 1980; see also Wozny et al. 2010). So, vision tends to dominate hearing when they disagree about location. This yields the ventriloquist effect. When vision’s reliability is poor, as with dim lighting, the effect is reversed and audition biases visual location. Concerning temporal features, vision tends to defer to audition. When audition’s reliability is poor, as with white noise, the reverse holds (cf., Stokes and Biggs 2015, especially section 6). This strategy of weighting and biasing improves the accuracy and reliability of the overall perceptual result. In fact, Shams and Kim (2010, 280) say, “This strategy is statistically optimal as it leads to minimizing the average (squared) error of perceptual estimates; however, it results in errors in some conditions, which manifest themselves as illusions”. Each of the senses does better, and collectively they yield a better product, thanks to the manner in which they coordinate (see also Matthen, this volume, Chapter 8, I.1). The cost is predictable errors. Furthermore, richly multisensory perception enables us to perceive new features and to undergo novel varieties of perceptual experience. In the previous section, I described perceiving the coinstantiation of features perceptible through different senses, perceiving otherwise imperceptible instances of relations that hold between things perceived using different senses, and perceiving wholly novel types of features not perceptible with any mere collection of independent senses. Each involves extending our perceptual capacities and expanding perceptual experience through the coordinated use of multiple senses. Multisensory bootstrapping thereby augments perception and awareness.

4 The claim that perception is richly multisensory raises two further questions. Each concerns psychological taxonomy for richly multisensory awareness. The first deals with how to type richly multisensory experiences by modality. The second deals with why the multimodal phenomena I have discussed belong to perception and not extra-perceptual cognition.

116

•

Casey O’Callaghan

First, do perceptual experiences come in modalities at all? The more we learn about perception’s multisensory character, the more tempting it is to scrap the idea that experiences belong to modalities corresponding to something like the traditional senses. Doing so would have a big impact on theorizing about perception. For instance, many intentionalists appeal to distinct visual and tactual modes of representing content, understood by analogy with propositional attitudes such as believing and desiring. If we scrap modalities, this would have no place in theorizing about perception. Perhaps there is just perceptually representing (see Speaks 2015). This would reshape debates about the adequacy of intentionalism, especially intramodal intentionalism. So, whither the modalities? Rejecting modalities is too quick. Experiences can be typed by modality even in the face of perception’s constitutively multisensory character. However, a closely related commitment should go. To start, perceptual experiences are conscious perceptual episodes or events. We may type such episodes in any number of ways (see Macpherson 2011; cf., Matthen 2015; and Matthen, this volume, Chapter 8, II.2). We can appeal to their objects or contents, the pathways or processes they involve, or their phenomenology. In typing perceptual experiences by their phenomenology, for instance, we may appeal to the distinctive qualitative character associated with the experience as of some proper sensible, such as a hue, a pitch, a temperature, a taste, or a smell. The proper sensibles might be identified by ostension, defeasibly and subject to trial and error, or they might be identified by appealing to some other criterion for individuating senses. In either case, a conscious episode involving awareness as of hue is a visual episode, and so on. You may think this approach has important limitations. For instance, humans sometimes visually experience common sensibles such as shapes. But, it may seem according to this criterion that a shape experience as such cannot be ascribed to a modality. (Or, perhaps spatial experiences are, in Matthen’s terms, “pre-modal”; this volume, Chapter 8, I.2.) So, not every experience that belongs to a modality is ascribed to that modality. In addition, it may seem according to this criterion that some apparently multisensory perceptual experiences belong to no modality at all. For instance, consciously perceiving a novel intermodal relation instance as such belongs to no modality even while it involves vision and touch. However, recall that we are typing perceptual episodes, rather than features (properties or parts) of such episodes, by modality. Features, including phenomenal features, may be associated with a given modality on an occasion, but on this approach a feature’s being associated with a modality on an occasion does not amount to that feature’s belonging to a modality-specific type of experience. Since we are typing episodes rather than features by their phenomenology, a case of experiencing a common sensible may belong to a given modality if that episode also involves experiencing one of that modality’s

Enhancement Through Coordination

•

117

proper sensibles. In addition, every perceptual episode may belong to some modality or another even while that episode’s features are not exhausted by those that are associated on that occasion with some modality or another. Thus, even richly multisensory experiences belong to the modalities they involve. So, the apparent limitations are features not bugs. The key to this approach is that a given conscious perceptual episode’s belonging to one modality does not preclude its also belonging to another modality. The modalities, understood as classes of perceptual episodes typed by phenomenology, are not exclusive. A perceptual experience that is visual also may be auditory and tactual (see also Matthen, this volume, Chapter 8, note 7). Consider a richly multisensory experience, such as experiencing intermodal feature binding. Take a case in which a subject experiences an explosion’s being both loud and bright. Loudness and brightness are proper sensibles associated with hearing and sight, so the episode is identifiable by virtue of its phenomenology as an auditory and as a visual experience (even while its features are not exhausted by those that are associated on that occasion with either audition or with vision). Being an auditory experience is compatible with being a visual experience, and being auditory does not entail being exclusively or exhaustively auditory (see also Matthen’s discussion, this volume, Chapter 8, especially III). Richly multisensory conscious episodes therefore present no trouble for this approach to classifying and investigating experiences by modality. What does pose trouble and what I think must go is the idea that perceptual experiences can be carved, with or without remainder, into modality-specific parts. We should give up on neatly apportioning experiences by modality and theorizing about each in abstraction from the rest. Second, why are multimodal effects perceptual rather extra-perceptual cognition? This is a fertile question, and it opens a lot of doors. I’ll outline part of my approach here. I am sympathetic to the idea that no sharp or principled boundary exists between perception and cognition and that a continuum stretches between the clear cases of each. Even so, the evidence supports situating central multisensory phenomena nearer to paradigms of perception than to paradigms of cognition. To argue for this claim requires specifying the features that tend to distinguish and to diagnose central cases of perception and cognition. And here I think it is useful to recognize that a distinction between perception and extra-perceptual cognition may play differing explanatory roles. For instance, it may figure in empirical or experimental psychological explanation, which aims to explain behavior on the basis of causally efficacious internal mental states. Or, it may figure in personal, rational, or philosophical psychological explanation, which aims to make sense of and to evaluate an agent’s beliefs and actions on the basis of that subject’s reasons. Or, it may figure in characterizing a subject’s consciousness, which aims to capture what it is like subjectively for an individual to undergo a course of experiences. My view is that these

118

•

Casey O’Callaghan

differing explanatory projects mark the distinction between perception and cognition with criteria that yield differing diagnostic features. It is an important question how closely the verdicts line up (cf. Matthen’s discussion of how scientists and philosophers distinguish the senses, in this volume, Chapter 8, II). Rather than work through each in detail, let me sketch one piece of the argument for multisensory perception. Consider experimental psychological explanation (see Matthen, this volume, Chapter 8, especially II.1). Being a perceptual state is a matter of occupying a certain causal functional role. In particular, it roughly involves being responsive to stimulation to the various sensory surfaces and affecting attention, recognition, learning, memory, and action. Researchers operationalize the perceptual role experimentally in a number of ways. For instance, they measure response time, accuracy, and salience, and they attempt to control for confounds such as response strategies adopted to deal with ambiguous stimuli, memory and learning effects, and quick inferences. They introduce conflicts and masks, load attention, alter contextual cues, and test for Stroopability. Lately, they investigate neural activity. The results of several decades’ work leaves little doubt that paradigmatic multisensory effects indeed are perceptual and not simply a matter of higher cognition, conscious inference, or memory (see Stein 2012). Things are complicated by the fact that cognition influences perception in ways previously not recognized or not appreciated. For instance, attention, categorical perception, and perceptual learning structure perceptual processing and perceptual awareness, at a time and over time, in ways traditionally associated with cognition. And this may affect how we understand not just perception’s functional role but also both perception’s rational role and whether some aspect of conscious awareness is perceptual or extra-perceptual. Resolving these issues is future work. Nevertheless, central examples of multisensory phenomena are on par in these respects with accepted, relatively uncontentious examples of conscious intramodal perceptual awareness, such as awareness of objects and feature binding, of events and causes, and of spatial and temporal relations (Matthen, this volume, Chapter 8, especially I and III.3, elaborates this point). Moreover, the example of novel or emergent qualitative features, such as apparent flavors, presents a serious obstacle to skeptics about richly multisensory perceptual consciousness. Still, this question reveals plenty of opportunity for future debate.

5 In summary, section 2 argued that a conscious perceptual episode may be richly or constitutively multisensory in at least two respects. First, a perceptual experience that is associated with a given modality at a time may not have been possible without past or present conscious perceptual episodes associated with another modality.

Enhancement Through Coordination

•

119

Second, perceptual experience is more than a consciously unified sum of visual, auditory, tactual, gustatory, and olfactory experiences. That is, it is not the case that perceptual experience at each time is exhausted by a mere co-conscious fusion of that which is associated on that occasion with each of the respective modalities. Thus, perceptual experience is more than minimally multimodal. Section 3 extracted two main lessons from this. First, there are serious methodological problems with projects that claim to focus on and theorize about one modality in isolation or abstraction from the others, as philosophers of perception commonly have done. An apparently visual phenomenon might best be explained by appeal to non-visual or multisensory phenomena. Moreover, a full account of perception requires more than independently addressing each individual modality. Second, more constructively, we gain new perceptual capacities through the coordinated use of multiple senses. Multisensory coordination enhances perception by improving accuracy and reliability. Moreover, we perceive novel features and undergo new varieties of experience thanks to multisensory bootstrapping. Section 4 addressed two remaining questions concerning psychological taxonomy for richly multisensory perception. First, it described how to type experiences by modality according to phenomenology despite their constitutively multisensory character. They key points are that we type experiential episodes rather than features by modality and that a given episode’s being of one modality does not preclude its being of another modality—the modalities of experience are not exclusive. Second, it sketched a defense of why multisensory phenomena belong to perception rather than extra-perceptual cognition. It argued that multisensory effects fit comfortably among accepted examples of perception relative to the requirements of contemporary psychological science. Nevertheless, a full defense awaits considering the differing explanatory purposes for distinguishing perception from cognition—empirical and philosophical—along with their accompanying criteria.

Acknowledgements Thanks to Mohan Matthen for valuable correspondence and helpful comments during our exchange, and to Bence Nanay for constructive editorial suggestions. My gratitude to Marty, Linda, and the Koehn family for companionship, laughs, and views of Lake Hamilton while I wrote this paper.

References Bertelson, P. (1999). Ventriloquism: A case of cross-modal perceptual grouping. In Aschersleben, G., Bachmann, T., and Müsseler, J., editors, Cognitive Contributions to the Perception of Spatial and Temporal Events, pages 347–362. Elsevier, Amsterdam.

120

•

Casey O’Callaghan

Guttman, S. E., Gilroy, L. A., and Blake, R. (2005). Hearing what the eyes see: Auditory encoding of visual temporal sequences. Psychological Science, 16(3):228–235. Harrar, V., Winter, R., and Harris, L. R. (2008). Visuotactile apparent motion. Perception and Psychophysics, 70(5):807–817. Huang, J., Gamble, D., Sarnlertsophon, K., Wang, X., and Hsiao, S. (2012). Feeling music: Integration of auditory and tactile inputs in musical meter. PLoS ONE, 7(10):e48496. Huddleston, W. E., Lewis, J. W., Phinney, R. E., and DeYoe, E. A. (2008). Auditory and visual attention-based apparent motion share functional parallels. Perception and Psychophysics, 70(7):1207–1216. Macpherson, F. (2011). Taxonomising the senses. Philosophical Studies, 153(1):123–142. Matthen, M. (2015). The individuation of the senses. In Matthen, M., editor, The Oxford Handbook of Philosophy of Perception, pages 567–586. Oxford University Press, Oxford. Nudds, M. (2001). Experiencing the production of sounds. European Journal of Philosophy, 9:210–229. O’Callaghan, C. (2012). Perception and multimodality. In Margolis, E., Samuels, R., and Stich, S., editors, Oxford Handbook of Philosophy of Cognitive Science, pages 92–117. Oxford University Press, Oxford. O’Callaghan, C. (2014). Intermodal binding awareness. In Bennett, D. and Hill, C., editors, Sensory Integration and the Unity of Consciousness, pages 73–103. MIT Press, Cambridge, MA. O’Callaghan, C. (2015). The multisensory character of perception. The Journal of Philosophy, 112(10):551–569. Shams, L. and Kim, R. (2010). Crossmodal influences on visual perception. Physics of Life Reviews, 7(3):269–284. Shams, L., Wozny, D. R., Kim, R. S., and Seitz, A. (2011). Influences of multisensory experience on subsequent unisensory processing. Frontiers in Psychology, 2(264):1–9. Smith, B. C. (2015). The chemical senses. In Matthen, M., editor, The Oxford Handbook of Philosophy of Perception, pages 314–352. Oxford University Press, Oxford. Speaks, J. (2015). The Phenomenal and the Representational. Oxford University Press, Oxford. Spence, C. and Bayne, T. (2015). Is consciousness multisensory? In Stokes, D., Matthen, M., and Biggs, S., editors, Perception and Its Modalities, chapter 4, pages 95–132. Oxford University Press, New York. Spence, C. and Squire, S. (2003). Multisensory integration: Maintaining the perception of synchrony. Current Biology, 13(13):R519–R521. Stein, B. E. (2012). The New Handbook of Multisensory Processing. MIT Press, Cambridge, MA. Stokes, D. and Biggs, S. (2015). The dominance of the visual. In Stokes, D., Matthen, M., and Biggs, S., editors, Perception and Its Modalities, chapter 14, pages 350–378. Oxford University Press, New York. Stone, J. V., Hunkin, N. M., Porrill, J., Wood, R., Keeler, V., Beanland, M., Port, M., and Porter, N. R. (2001). When is now? perception of simultaneity. Proceedings of the Royal Society of London. Series B: Biological Sciences, 268(1462):31–38. Welch, R. B. and Warren, D. H. (1980). Immediate perceptual response to intersensory discrepancy. Psychological Bulletin, 88(3):638–667. Wozny, D. R., Beierholm, U. R., and Shams, L. (2010). Probability matching as a computational strategy used in perception. PLoS Computational Biology, 6(8):e1000871.

CHAPTER

8

Is Perceptual Experience Normally Multimodal?1

MOHAN MATTHEN

The study of multimodal perception is relatively new: according to Stein (2012b), the rate of publication on multisensory integration grew tenfold in the first decade of this century. This is somewhat misleading. It was not so much that the phenomena were unknown or unstudied, but that they were not brought under the methodologies that cognitive scientists employ today to investigate multimodality. (I make no distinction in what follows between “multisensory” and “multimodal”.) Consider speech perception. Vatikiotis-Bateson and Munhall (2012) are not saying anything startling when they write: The acoustics of speech are the signature property of spoken communication, but the movements of the individual producing the speech provide extraordinary information about the talker’s message. Head and face movements, gestures from the hands and arms, and bodily posture adjustments are part and parcel of the sequence of spoken words. (421) W. H. Sumby and Irwin Pollack found as long ago as 1954 (in a threepage paper published in the Journal of the Acoustical Society of America) that visual observation of a speaker’s facial and lip movements contributes to the intelligibility of speech, especially at low speech-to-noise ratios. The gap in the scientific literature was not so much that the visual contribution to speech communication was unknown. Rather, it was that this interaction was not connected to indicators of integrative sensory processing such 121

122

•

Mohan Matthen

as “detection and localization behaviours, speeded reaction times, enhanced/ degraded speech perception, and cross-modal illusions” (Meredith 2012; see also Casey O’Callaghan’s list in section 4, this volume). The required change of outlook came relatively slowly, for as Bernstein (2012) relates, Sumby and Pollack’s article was very little cited for a considerable period of time. (It has now been cited nearly 1800 times, but mostly in this century.) The same was true of the perceptual findings of McGurk and MacDonald (1975), now cited nearly 5000 times, and the neurological findings of Sams et al. (1991) that visual perception of people talking activates the auditory cortex (this still has fewer than 400 citations). This body of convergent evidence was raised as support for the idea that speech consists not of sounds but of certain “gestures” of the vocal tract. But, as O’Callaghan points out in private communication, the seemingly obvious corollary that speech perception integrates auditory perception of speech sounds and of visual perception of gestures of the mouth and tongue—and that it is therefore multimodal—went almost unnoticed for quite a while.2 Bernstein says that change came with Stein and Meredith’s (1993) development of theoretical models to deal with an unrelated multisensory phenomenon, the representation of space in the superior colliculus. It was in the wake of this breakthrough that speech and other phenomena began to be treated as multisensory. Thus, Bernstein writes: “The present era of focussed research on audiovisual speech perception and its underlying neural processing arguably began in the 1990s”. She points to a special session of the Acoustical Society of America in Washington, D.C., and a NATO Advanced Study Institute in France, both in 1995. The situation in philosophy today is similar to that in cognitive psychology pre-1990. There is a host of familiar phenomena—in particular, speech perception and the perception of spatial phenomena—that philosophers simply don’t think of as multimodal. In part, this is based on a difference of focus; philosophers are concerned with perceptual experience, while psychologists are concerned with the interaction of sensory processes.3 Nevertheless, it is fair to say that philosophers’ attitudes too are founded on the absence of an appropriate theory. Many philosophers still cling to an introspectionist methodology, as well as a remnant of Empiricist Atomism that militates against the very idea of integrating distinct sources of information in perception. My main aim in this paper is to show how sensory integration is the norm in perception, even within single modalities such as vision. I contend, in short, that the alleged problems of cross-modal integration, which arise for example in discussions of Molyneux’s Question, actually have nothing to do with the participation of multiple modalities. But—and this is worth saying explicitly—my concern here will be with experience, not sensory systems, structures, or processes. I am sceptical about how much one can know on the basis of introspection. Nonetheless, it is experience that I will be talking

Is Perceptual Experience Multimodal?

•

123

about—my claims about the content of experience find support in its effect on action-capacities and does not rely on personal introspection, as I will document in the appropriate places. The atomistic foundations of philosophical resistance to sensory integration must be re-evaluated. When they are, we will come to see perception as “richly multisensory”. The term is O’Callaghan’s, and I’ll offer a definition distilled from his writing: A multisensory experience is richly multisensory if (a) it is not coconsciousness of separate unisensory states, and (b) its content is not the mere conjunction of the content of unisensory states.4 I will concentrate on showing that our experience of spatial relations is richly multisensory. O’Callaghan, this volume, considers other perceptibles, such as flavour.

I. Place 1. Think about these examples: (a) There is a stale smell in the laundry room. You sniff, moving about the space. Ah, you’ve found it. It’s the wet dishtowel that fell into the crack between the washer and dryer (cp. Von Békésy 1964). (b) You are listening to an orchestra. A single woodwind enters. Who is it? Scanning the orchestra, you see the clarinettist fingering her instrument and moving in time. Now you hear the music as coming from her place (cp. Allais and Burr 2004). In both scenarios, an imprecisely located sense-feature acquires precise location by the participation of another modality—the sense of self-motion (which in turn employs the vestibular and proprioceptive systems and motor feedback) as well as vision in (a), and vision as well as proprioception (to scan the orchestra) in (b). The modalities work together by first detecting causation; the clarinettist is (perceptually, not cognitively) identified as the source of the sound because her visual movements coordinate with the sound; the dishtowel is identified as the source of the odour because smell intensity-gradients radiate from it. Once an object evident to one modality is identified as the cause of an event evident to another, the spatial location of the two modalities is coordinated. Once the visible movements of the clarinettist are identified as the source of the clarinet’s audible melody, the latter is localized in coordination with the former. So we have an earlier (supposedly) unimodal experience and a later transformed experience that contains source and spatial information provided by several modalities working together. The transformed experience is richly multisensory: the earlier unisensory experience did not contain

124

•

Mohan Matthen

source or detailed spatial information. The earlier experience has disappeared and is not a co-conscious component of the transformed experience. In normal perception, we are presented with a perceptual image that consists of material objects, sounds, smells, and various tactile qualities. I hear the banging of pots in that brightly lit restaurant kitchen from which all those delicious smells and radiant heat emanate. This is the kind of image that perceptual experience delivers. In it, all of the perceptual features are bound together and connected by spatial and causal relations. Perhaps some (though clearly not all) of these features arise from single modalities. However that might be, their causal and spatial relations are delivered multimodally. The spatially unified and causally connected image of our surroundings arises from richly multisensory experience. Vision has special responsibility for spatial representation because it provides the finest grained representations; in (b) visual spatial representations localize sounds with a precision that surpasses audition. O’Callaghan writes that in cases like these, cross-modal processes “improve the accuracy and reliability of the overall perceptual result” (cp. O’Callaghan 2014, 148, esp. n 15). I agree, but I want to emphasize an additional point—that these processes enhance the fineness of spatial grain of the perceptual image regardless of accuracy and reliability. As well, there are the causal connections. As O’Callaghan observes, there is a difference between noticing that one and the same object is red and rough and noticing that a visually identified object is red and that a tactile object is rough. There is also a difference between seeing a woman and hearing a clarinet, and hearing a woman you see playing the clarinet. The latter is a richly multisensory experience that is additionally constitutively multimodal, not merely causally so. (See Connolly 2014 for discussion of constitutive multimodality.) The perceptual image is a multimodal melding of information derived from the senses. It is an organic spatial whole, not a mere conjunction of unimodal features. 2. One point should be emphasized at the outset: the perceptual representation of space is not specifically visual or specifically bodily. It is a template or matrix into which all the modalities place features. The same object is perceived as possessing many features from several perceptual modalities. To deliver such a presentation, vision and touch must have access to the same spatial matrix.5 It follows, first, that as I have argued elsewhere (Matthen 2014): The representation of space is pre-modal; it is, in Kant’s words, an a priori intuition. In light of this, I want to urge, second: The total perceptual image is multimodally coordinated.

Is Perceptual Experience Multimodal?

•

125

(For the sake of clarity, it is the second proposition that is more important in the present context, though the first provides important background.) The representation of space underlying conscious perception of the external world is objective and allocentric. Perceptual consciousness seems like a photograph or a movie: a view from a certain perspective with no objective indication of where in the world the point of view is from. Am I looking up Church Street or Spadina Avenue? Am I looking north or west? The intuition based on phenomenology is that perception cannot answer such questions; perceptual phenomenology is egocentric, or so it seems. This intuition has been shown to be wrong: we now know that the perceptual image is linked to something more like the map display on your GPS—an objective map that has your position and heading marked. There is awareness of absolute direction built in. Moreover, when you are in a familiar setting, you are implicitly capable of relating the objects you perceive to landmarks beyond the perceptual horizon (Epstein and Vass 2014). Your experience of the room in which you now stand is, for example, pregnant with the location of the unseen corridor outside.6 I said earlier that the perceptual representation of the spatial matrix is pre-modal. The brain uses this matrix, which happens to be located in the hippocampal formation, to make a map of the locales that you inhabit by correlating movement with perceptual features. The matrix is non-empirical and a priori, and it is filled by the collaboration between the sense of self-movement and other perceptual modalities. (Historical note. E. C. Tolman 1948 was the pioneer of cognitive maps. O’Keefe and Nadel 1978 is the now canonical scientific treatment, with interesting allusions to Kant. The Karolinska Institute’s announcement of the Nobel Prize for Physiology, 2014, awarded to John O’Keefe and Edvard and May-Britt Moser was explicit: the hippocampal coding is “a comprehensive positioning system, an inner GPS”. It also invokes the Kantian echo. It is instructive to think about the perceptual infrastructure needed to maintain this kind of cognitive structure.)

II. Philosophical Resistance 1. Many philosophers are sceptical about multimodal experience. Macpherson (2011) describes their attitude in the following way: A common assumption was that the sensory modalities were perceptual systems isolated from each other. Each sensory system produced, unaffected by the others, a perceptual experience characteristic of that sensory modality (a ‘uni-modal’ experience) and perhaps other uni-sensory, non-conscious, sub-personal representational informational states characteristic of the modality. What is the basis of this “common assumption”?

126

•

Mohan Matthen

One kind of approach, following Grice (1962), is based on the idea that each modality produces experiences of a characteristic phenomenal character. For example, although hearing somebody say “bat” is phenomenally different from hearing somebody splash about in the ocean, the two experiences share a generic auditory phenomenology that marks both off from seeing somebody say “bat” or seeing somebody splash about in the ocean. Analogously, the claim about (a) and (b) is that the transformed experiences are phenomenally like the earlier ones in the crucial modality-identifying way. The stale smell feels smelly in both the first and the transformed experience of (a); both (b) experiences feel auditory.7 It is fair to say that this approach runs against the scientific current. In perceptual science, the modalities are thought of as sources or processes, not products. Perception is a process that results in an image of the organism’s surroundings. In order to construct such an image, animals sample energy. Since there are many kinds of energy—electromagnetic, acoustic, thermal, mechanical, chemical—many different sampling mechanisms have evolved. Generally speaking, there are different transducers for different kinds of energy—the cells that convert light into neural signals are different in kind from those that do the same for sound, chemical reactions, etc. The processes that extract ecological content from these neural signals are also specialized—at least in the early stages of processing, which are energy specific. These specialized and separate processes are the cognitive scientist’s modalities. Later in the perceptual process, representing the environment begins to be the focus (rather than analysing patterns in the specialized receptors); the different modalities contribute to a common conversation. This gives rise to the simplified schema of Figure 8.1, which assumes that the data-streams stay separate in the early stages of processing, and that there is no multimodal influence early on. The current recognition of multisensory processing rests on the dawning realization that these data-streams inter-communicate from the very first stages. Of course, the idea of a “source” is context dependent, and the scientific concept of modality is correspondingly flexible. Touch, for example, is often said to be multisensory, on the grounds that it has transducers for different kinds of energy—stretch receptors, pain receptors, thermal receptors. But it can also be treated as a single modality on the grounds that it integrates the information coming from these diverse receptors at a very early stage. (Fulkerson 2013 has an excellent discussion.) Flavour perception, mainly driven by receptors in the tongue and the nose, is the opposite sort of case. Both sets of receptors respond to chemical signals, but since the nasal receptors operate independently of those in the tongue, it is common to distinguish (retronasal) olfaction from taste and to say that these come together in flavor perception, which is usually held to be multisensory (Smith 2015). In perceptual science, the question posed in my title is about the depth and relative isolation of the supposedly separate processes in Figure 8.1. Are the streams well integrated right from the start, or are they integrated only much later?

Is Perceptual Experience Multimodal?

•

127

Figure 8.1 The modalities as sources for multimodal perceptual processing. It is worth noting that in fact visual information about speech enters the auditory stream very early. (Sams et al. 1991)

2. As I have described the dialectic, philosophers distinguish the modalities by differences of phenomenal character, while perception scientists do so by differences in the way that informative content is extracted from different energy sources. Who is right? Does it even make sense to ask this question? One approach to the question is criterial. The idea is that any consistent criterion marks off a legitimate category. Here’s an extreme example. One

128

•

Mohan Matthen

criterion associated with the word “bank” makes it true of financial institutions; another comprehends the shores of rivers. These are different, but neither is wrong—similarly, some say, with modality. Philosophers use phenomenal character to demarcate the senses; they operate with a phenomenological concept. Scientists use process; they operate with a different concept. These are different, but both are legitimate; because each is consistent, each defines a possibly non-empty class. This is the basis of the criterial approach. Two philosophers who align themselves with the criterial approach, though in different ways, are Macpherson (2011, 2014) and Richardson (2013). Neither adopts quite as extreme a position as the one just outlined—both think there is some overlap of meaning among different criteria—but both hold that there is no conflict among various criteria that define the modalities. Richardson holds that there is an “everyday” conception of flavour on which it is a single modality. Since this concept is defined by different criteria, it is not challenged by the scientific concept, which (as noted previously) treats flavour perception as multisensory. These are simply different ways of carving the roast, she suggests. Macpherson (2011) also thinks that the phenomenological and process criteria are different and orthogonal—she reserves the term “multimodal” for the first and “multisensory” for the second. (As a reminder: I use these terms interchangeably.) She follows Grice in delineating four ways of assigning a given perceptual state to a modality and insists that “we should not choose between these criteria for individuating the senses but instead use them all”. Macpherson (2014) combines the pre-existing criteria into a single similarity space for modalities. A second approach to defining the modalities is analytic. The assumption here is that modalities are real psychological processes and that the task is to discover what differentiates them. (Matthew Nudds 2004 explicitly denies that the distinction between modalities is based on “nature”.) On such an approach, one might begin the consideration of cases (a) and (b) with the idea that vision, audition, olfaction, and the sense of bodily self-awareness do different jobs, and try, as a first step, to figure out how these jobs differ. Then, one might try to formulate a general partitioning of perception in ways that capture relevant differences. The schema of Figure 8.1 is an example of what might result from this procedure. On this reckoning, the phenomenological and the process-oriented descriptions are different ways of partitioning perceptual processing into its natural parts. They might turn out to be incompatible, or complementary, or incoherent—this depends on other features of the general scheme. This was Grice’s approach to the question (and Aristotle’s, for that matter). Analytic inquiries are open-ended. They could have revealed that there are no modalities in anything like Richardson’s everyday sense, or that the distinction depends on explanatory context, as Nudds suggests. These would be significant insights. In the event, though, they came up with something like Figure 8.1. (See, for example, Stein and Meredith 1993, Introduction.)

Is Perceptual Experience Multimodal?

•

129

In my view, the criterial approach makes too many questionable assumptions about the domain of investigation. For example, Macpherson (2014) proposes that the criteria are independently conceived axes for a combined similarity metric, from which Principal Component Analysis (PCA) can derive a taxonomy. There is no reason to accept this. Suppose you charted foods by sweetness, softness, colour, “height” of cuisine, distance between towns of origin, and price per ounce. Would PCA yield up a useful categorization of food? Macpherson hopes that PCA will reveal “an extra deep layer of information” “hidden” within the similarity space of the modalities. This overestimates the efficacy of statistical methods. PCA is designed to pick out statistically independent axes of variation. Sometimes such axes correspond to, or reveal, significant underlying measures; equally often, PCA yields meaningless results. Properly understood, PCA serves as a preliminary to analytic investigation. It is not a panacea, and its mere existence is not a good reason to accept even the partial legitimacy of multiple criteria. Nor, pace Richardson, is the mere existence of an everyday conception of flavour any reason to believe in its coherence or utility. As must be obvious, I favour an analytic approach. I can’t argue for this here, but let me simply observe that it is hard to see how the phenomenological criterion can yield anything useful by way of determining whether the transformed experiences in (a) and (b) are multimodal. This criterion directs us simply to introspect these experiences and decide whether they “feel” purely auditory or purely olfactory or mixed. Such a procedure is far too poorly specified to be useful in contested cases. Does the transformed experience of the clarinet feel auditory? Perhaps, but would we say it also felt visual when we come to notice that it depends on keeping our eyes open? It is at least possible that we would. We need a far more nuanced approach.

III. Integration 1. Locke’s negative answer to Molyneux’s question is motivated by empiricist atomism: the newly sighted man “has not yet attained the experience, that what affects his touch so and so, must affect his sight so and so”. For Locke, the “so and so” stands for what we would call phenomenal character. To say that a sphere is known by touch is for him to say that we know that a sphere causes the phenomenal character produced by touching a sphere. (Locke’s position is implicitly accepted by contemporary philosophers, like Grice, who say that the modalities are defined by phenomenal character.) In quasi-Fregean terms (“mode of presentation”), Locke’s reasoning can be represented as follows:8

130

•

Mohan Matthen

Atomism A simple perceptual experience is, in virtue of its phenomenal character, a mode of presentation of a single perceptual feature. No Synonymy Perceptual experiences that have different phenomenal character are different modes of presentation. Cross-Modal Difference Any two perceptual experiences from different modalities have different phenomenal characters. Conclusion 1 (Unimodality) Since multimodal perceptual experience must incorporate multiple phenomenal characters (by Cross-Modal Difference), it must be composite. Conclusion 2 (Negative Response to Molyneux) Since experiences of different modalities are different modes of presentation (by No Synonymy), it is contingent whether they present the same or different features. Conclusion 3 (Decomposing the Multimodal) Since visual and tactile sphericity are different (by Atomism and No Synonymy), looking at a sphere you hold in your hand creates a conjunction of separate experiences: you are aware of visible sphericity; you are aware of tactile sphericity; you are co-conscious of both. Similarly, when you notice that it is red and rough. Locke’s argument makes it pointless to pursue questions about multimodality: it is a definitional stop. This is one reason we urgently need to reassess it. Now, it is unclear how an atomist should interpret the experience of space. As Kant realized, every perceptual experience (i.e., every “outer” perception) is spatial: for example, every perception of red is experience of red in a place. How should this be understood? We cannot say that experience of feature-in-place is composite—it isn’t experience of feature plus experience of place. This separatist account would posit awareness of red-not-in-place as prior to awareness of red-in-place. Perceptual experience of red-not-in-a-place seems impossible, especially as a norm. (Possibly you could have a fleeting non-perceptual experience of red-not-in-a-place, for instance when you press down on your eyeball.) Moreover, the account does not tell us how to tie feature and place together: if perception of place is separate, it cannot be the “binding parameter”. Atomism is on shaky ground. It cannot account for the integration of features in a spatial field. Examples (a) and (b) suggest that the modalities cooperate to localize features, possibly with vision dominating (Millar 2008, Stokes and Biggs 2014). Such cooperation would not be possible if there were not a shared “vocabulary” of space among the modalities. This shows that even the simplest experiences have a locational component they share with experiences of other modalities. Nor is this sharing learned. Even newborns turn toward a sound and track a moving object with their eyes.9 Moreover, they turn their heads away from odour sources.10 No Synonymy is questionable with respect to our experience of location.

Is Perceptual Experience Multimodal?

•

131

2. The same goes for direction. Evans (1985) wrote: “Disembodied perceivers would have no notion of up, down, left, right, forwards, and backwards”. It is important to temper this observation with the realization that perspective is integrated with allocentric representation. If you stand on your head, lower parts of the environment project to the chin-ward end of the retina; nevertheless, you continue to experience the up-direction as away from the Earth. Up is always defined by awareness of gravity, even visually. (Imogen Dickie tells me that rock climbers always understand “up”—as in “Look up”, “Reach up”—as away from the Earth.) Our sense of up is bodily, even in visual presentations. A newly sighted person should be able to tell which way looks up. Even more importantly, the hippocampal “GPS” incorporates direction cells. These indicate the perceiver’s orientation in a familiar scene; there are cells, for example, that will fire when the perceiver is pointed north, and others that fire when the perceiver is headed south. These cells subserve a sense of direction informed by past experience in all modalities. As O’Callaghan (this volume) writes: “an experience that is associated with one modality on a given occasion can depend upon past or present sensory episodes that are associated with another modality”. 3. The phenomena we have been discussing illustrate a more general principle of perception that applies both within and across modalities: Integration Experience of any given sense feature is modulated by extracted content regarding other sense features. Thus, experience of any one feature contains information about other features. For a within-modality example, take the experience of colour. You see coloured shapes: the colours are tightly confined within the contours. For instance, in Figure 8.2, the colour you see exactly matches the triangular contour. Now, as Livingstone (2002) writes, “cells that code color have larger receptive fields, and there are fewer of them than the cells used for resolving contours”

Figure 8.2 The colour of the triangle matches the contour. But since colour and contour are separately processed, this needs active processing.

132

•

Mohan Matthen

(95). The visual system renders the spatial location and extent of the colour that you see in Figure 8.2 far less precisely than the triangular contour (for which it uses brightness, a component, but not the whole of colour). How then does the colour cling so exactly to the form? The visual system makes colour boundaries in two steps. The first step is described in Anne Treisman’s feature integration theory (Treisman and Gelade 1980). Treisman realized that colour and shape are processed separately in the brain; these processes proceed in parallel, one resulting in a spatial map of colours, the other of forms. In order to co-locate shapes and colours, the two maps have to be collated. Treisman and her co-workers were able to demonstrate that this requires a third process, which she identified as attention. In less attended parts of a scene, perceivers cannot reliably co-locate separately processed features. Attentive co-location is followed by a second step that Livingstone articulates: the system “codes the color of an object by signalling [the object’s] edges, . . . letting the color ‘fill in’ perceptually” (99). In this respect, colour-form perception is like the audio-visual location of the clarinet’s melody. In both cases, a finer determination of place replaces a rough one. Perception performs these integrative operations because it is adapted to certain environmental regularities. It confines colour within object contours because it is extremely rare to find colour that spreads from one object to another. Thus, visual experience is not simply of exactly coterminous colour and shape, but that of a coloured shape. Experience of a coloured surface is different from that of a colour and a surface; the integrated experience is different from its separate atoms. (This contradicts the theory that David Lewis 1966 attributes to “British empiricism and introspectionist psychology”: “the content of visual experience is a sensuously given mosaic of color spots, together with a mass of interpretive judgements injected by the subject”.) Atomism asserts that one feature carries no information about another. But here we find that shape carries information about the extent of colour.

IV. Conclusions Cognitive scientists now accept the existence of multimodal perception, but philosophers are still reluctant to do so. On occasion, this difference rests on the use of different criteria. Take touch. As mentioned earlier, most psychologists are inclined to say that it is multimodal, on the grounds that it involves a variety of receptor types sensitive to different types of incident energy. Philosophers are more reluctant to allow this, because they use other criteria. For example, Fulkerson (2013) has argued that touch is a single modality because it co-assigns its features to the same objects. This is an example of a fruitful controversy about criteria.

Is Perceptual Experience Multimodal?

•

133

In this paper, I have probed a less productive argument that some philosophers use to cast doubt on multimodality. It is the idea that the features presented to us in experience are logically independent of one another (Atomism) combined with the stipulation that any single non-composite experience has indivisible phenomenal character. The conclusions drawn from these two premises are first that the phenomenal character of a non-composite experience can only belong to one modality, and, second, that the feature presented by such an experience cannot depend on any other feature. My challenge to this line of thought has been twofold. First, I have claimed that it cannot accommodate the essentially spatial character of external position, because the representation of space takes input from several modalities. Second, I have argued that Atomism is misguided. The representation of sensory features generally takes content from the representation of other sensory features, particularly with regard to space (and also time, though I did not discuss this here)—and this happens across modalities just as often as it does within modalities. In the end, some philosophers may stick with Atomism, preferring to reformulate the kinds of integration that I have described in terms that make them compatible with Atomism. For a philosopher who chooses this option, two questions remain. The first concerns feature-in-place: how can this be made compatible with Atomism? Second, how will the criterion of multimodality be adjusted so as not to rule it out as incoherent right from the start? One way or another, it would be healthy to set the issue up in such a way that permits a conversation with cognitive scientists. I have not discussed some interesting phenomena that come up in philosophical discussions of multimodality, in particular speech and flavour. I have stuck with space. This is both because multisensory integration is easier to establish in the spatial case, and because I believe that the principles of multisensory integration in spatial perception will find a natural extension to these other domains. In psychology, as we noted in the beginning, investigation of spatial maps was a gateway to the discussion of other multisensory phenomena. Perhaps the same can happen in philosophy.11

Notes 1. My title plays on Casey O’Callaghan (2014): “Not All Perceptual Experience Is Modality Specific.” 2. Initially, I found O’Callaghan’s observation incredible: how is it possible not to remark on multisensory perception in the context of the Motor Theory of Speech Perception? Then I found that I had cited Sumby and Pollack, and McGurk and MacDonald without a single mention of multimodal perception (Matthen 2005, 220). Thanks, Casey! 3. Stein and Meredith (1993, 1) say: “Of primary concern are observations that the use of one sensory system can influence (e.g., enhance, degrade, change) perception via another sensory system”.

134

•

Mohan Matthen

4. Cp. O’Callaghan, this volume, section 2, end: “a multisensory perceptual experience may have features beyond those that are associated on that occasion with each of the respective modalities and that do not accrue thanks to simple co-consciousness”. 5. Touch and proprioception also operate with an egocentric “body schema”. This is, among other things, an independent spatial framework that has to be cross-calibrated with the allocentric spatial matrix that vision contributes to. 6. Cp. Charles Siewert (1998, 258): “To recognize one’s surroundings is to see them as the gateway to somewhere else. . . . The temporarily disoriented person may see where things are around him accurately, but not recognize his surroundings—and when he does, the character of his experience changes”. 7. The account I favour is related but different in a crucial place: an experience is visual if it results from characteristic activities involving the eyes; it is auditory if we use the ears in characteristic ways, etc. (Matthen 2015). This has something in common with the phenomenological account; I can feel that I am using my eyes, so the resulting experiences are felt to be visual. But unlike that account, particularly Grice’s version, mine does not rule out the possibility that visual perception might also be auditory. 8. Richardson (2014) offers a somewhat similar reconstruction of Locke’s thinking. 9. Richardson (2014) argues that “whilst the spatial structure of perceptual experience differs across the senses, its temporal structure does not”. She contrasts shape with rhythm: it is evident to the senses that seen rhythm is the same as felt rhythm, but it is not evident to the senses that seen shape is the same as felt shape. What would she say about place? Isn’t it immediately evident that the place a sound comes from is the same as the place you see a synchronous flash of light? The evidence of infants and of the newly sighted suggests that cross-modal identification of place is immediate. 10. Rieser, Yonas, and Wikner (1976): “The odor source was placed near the nose slightly to the left or right of midline, with its position randomized over repeated trails. . . . It was found that as a group, the newborns turned away from the odor source more frequently than they turned toward it” (856). I’d like to say this implies a positive answer to MolyNOSE problem. 11. Warm thanks to Casey O’Callaghan and Kevin Connolly, both of whom helped me understand the basic issues.

References Allais, David and Burr, David (2004) “The Ventriloquist Effect Results from Near-Optimal Bimodal Integration,” Current Biology 14: 257–262. DOI: 10.1016/j.cub.2004.01.029 Bernstein, Lynne E. (2012) “A Historical Perspective,” in Stein 2012a: 397–405. Connolly, Kevin (2014) “Making Sense of Multiple Senses,” in R. Brown (ed.) Consciousness Inside and Out: Phenomenology, Neuroscience, and the Nature of Experience New York: Springer: 351–364. Epstein, Russell A. and Vass, Lindsay K. (2014) “Neural Systems for Landmark-Based Wayfinding in Humans,” Philosophical Transactions of the Royal Society B 369: 20120533. DOI: 10.1098/ rstb.2012.0533 Evans, Gareth (1985) “Molyneux’s Question,” in A. Phillips (ed.), Gareth Evans, Collected Papers. Oxford: Clarendon Press: 364–399. Fulkerson, Matthew (2013) The First Sense Cambridge MA: MIT Press. Grice, H. P. (1962) “Some Remarks About the Senses,” in R. J. Butler (ed.) Analytic Philosophy, First Series. Oxford: Basil Blackwell: 15–53. Lewis, David K. (1966) “Percepts and Color Mosaics in Visual Experience,” Philosophical Review 75 (3): 357–368. Livingstone, Margaret (2002) Vision and Art: The Biology of Seeing New York: Abrams. Macpherson, Fiona (2011) “Cross-Modal Experiences,” Proceedings of the Aristotelian Society CXI: 429–468.

Is Perceptual Experience Multimodal?

•

135

Macpherson, Fiona (2014) “The Space of Sensory Modalities,” in Stokes, Matthen, and Biggs 2014: 432–461. Matthen, Mohan (2005) Seeing, Doing, and Knowing: A Philosophical Theory of Sense Perception Oxford: Clarendon Press. Matthen, Mohan (2014) “Active Perception and the Representation of Space,” in Stokes, Matthen, and Biggs 2014: 44–72. Matthen, Mohan (2015) “Individuating the Senses,” in M. Matthen (ed.) The Oxford Handbook of the Philosophy of Perception Oxford: Oxford University Press: 567–586. McGurk, Harry and MacDonald, John (1976) “Hearing Lips and Seeing Voices,” Nature 264: 746–748. Meredith, M. Alex (2012) “Multisensory Convergence: Where It All Begins,” in Stein 2012a: 3–11. Millar, Susanna (2008) Space and Sense Hove, England: The Psychology Press. Nudds, Matthew (2004) “The Significance of the Senses,” Proceedings of the Aristotelian Society CIV: 31–51. O’Callaghan, Casey (2014) “Not All Perceptual Experience is Modality Specific,” in Stokes, Matthen, and Biggs: 133–165. O’Keefe, John and Nadel, Lynn (1978) The Hippocampus as Cognitive Map Oxford: Oxford University Press. Richardson, Louise (2013) “Flavour, Taste, and Smell,” Mind and Language 28 (3): 322–341. Richardson, Louise (2014) “Space, Time, and Molyneux’s Question,” Ratio (new series) XXVII (4): 483–505. Rieser, John, Yonas, Albert, and Wikner, Karin (1976) “Radial Localization of Odors by Human Newborns,” Child Development 47: 856–859. Sams, M., Aulanko, R., Hämäläinen, M., Hari, R., Lounasmaa, O. V., Lu, S. T., and Simola, J. (1991) “Seeing Speech: Visual Information from Lip Movements Modifies Activity in the Human Auditory Cortex,” Neuroscience Letters 127 (1): 141–145. Siewert, Charles (1998) The Significance of Consciousness Princeton: Princeton University Press. Smith, Barry (2015) “The Chemical Senses,” in M. Matthen (ed.) The Oxford Handbook of the Philosophy of Perception Oxford: Oxford University Press: 314–352. Stein, Barry E. (ed.) (2012a) The New Handbook of Multisensory Processing Cambridge MA: MIT Press. Stein, Barry E. (2012b) “Introduction,” in Stein 2012a: xi–xvi. Stein, B. E. and Meredith, M. A. (1993) The Merging of the Senses Cambridge MA: MIT Press. Stokes, Dustin and Biggs, Stephen (2014) “The Dominance of the Visual,” in Stokes, Matthen, and Biggs 2014: 350–378. Stokes, Dustin, Matthen, Mohan, and Biggs, Stephen (eds.) (2014) Perception and Its Modalities New York: Oxford University Press. Sumby, W. H. and Pollack, I. (1954) “Visual Contributions to Speech Intelligibility in Noise,” Journal of the Acoustical Society of America 26 (2): 212–215. Tolman, E. C. (1948) “Cognitive Maps in Rats and Men,” Psychological Review 55 (4): 189–208. Treisman, Anne M. and Gelade, Garry (1980) “A Feature-Integration Theory of Attention,” Cognitive Psychology 12: 97–136. Vatikiotis-Bateson, Eric and Munhall, Kevin G. (2012) “Time Varying Coordination in Multisensory Speech Processing,” in Stein 2012a: 421–433. Von Békésy, Georg (1964) “Olfactory Analogue to Directional Hearing,” Journal of Applied Physiology 19: 369–373.

PART

V

Is Attention Necessary for Perception?

CHAPTER

9

The Functional Roles of Attention

ROBERT W. KENTRIDGE AND BERIT BROGAARD

1. Neural Architecture and Neural Coding of Information When confronted with a new visual scene, we need to interpret it in terms of objects in the world, not the retinal image. How might attention help us achieve this? Let us consider some implications of broad principles of the neural architecture and neural coding of information in the visual system. The architectural principle we want to stress concerns systematic changes in the receptive field properties of cells as we move from primary visual cortex along the ventral visual stream moving into the temporal lobe toward the temporal pole (see, e.g., Gross, Rocha-Miranda, & Bender, 1972). In primary visual cortex, receptive field sizes are small, and the properties that receptive fields are tuned for are simple. As we move along the ventral stream, receptive fields become larger and tuning properties more complex. Toward the rostral end of the temporal lobe we might find cells that are tuned to respond selectively to specific body parts that have receptive fields spanning nearly an entire hemifield. Consider the size of a hand, a face, or indeed any other object that these cells might be tuned for. These objects almost always subtend visual angles much smaller than a hemifield. How then can a cell with a much larger receptive field respond selectively to an object when it seems it should be driven, directly or indirectly, not only by signals from the object but also by signals from a large area of space surrounding the object? We can only achieve this if, at levels of the visual system where receptive fields are small, signals from the space surrounding the object are suppressed. Is it possible that this is what attention is for? 139

140

•

Robert W. Kentridge and Berit Brogaard

Consider what we know about the effects of attention on the response properties of cells in the visual system. In the earliest and best-known experiment on the effects of attention on the receptive field properties of cells in the visual system (Moran & Desimone, 1985), the effect of attending to a particular object is to effectively shrink an initially large receptive field around the attended object so another object nearby in space no longer affects the cell’s responses. Changes induced by attention on the effective size, shape, and position of receptive fields have been demonstrated repeatedly since. Reynolds and Desimone (1999) suggest that attentional shrinkage of receptive fields around attended objects serves to solve the binding problem. As only features in the region of space specifically occupied by the object of attention can induce responses in receptive fields shrunk around the object, this means that errors attributing features belonging to other objects in nearby space to the object of attention (illusory conjunctions) are no longer possible. The problem of illusory conjunctions is essentially the same as the problem of ensuring that a cell with a large receptive field and complex tuning is not subject to interference from features within its receptive field other than those belonging to the object of attention. What then should we make of the consequences of attending for the sensitivity of cells that appear to have been demonstrated behaviourally (e.g., Carrasco, Ling, & Read, 2004) or electrophysiologically (e.g., McAdams & Maunsell, 1999)? There is little doubt that attention alters the gain (but probably not sharpness of tuning) of cellular responses to stimuli. One might anticipate that the perceptual consequence of this would be a change in apparent properties of an attended object. This need not, however, be the case. The broad principle of neural coding we want to stress is that representations are generally encoded in a distributed fashion. It is the relative strength (or even relative timing) of responses across multiple cells that encodes information, not the response of individual cells. So, if we increase the gain of all cells within a selected region of space, then we do not necessarily affect the property coded across those cells. The relative activities elicited by a stimulus should remain similar even if the absolute amounts of activity all increase. It is possible that the increase in gains might improve the signal-to-noise ratio of responses to the attended object. The most likely consequence, however, is, that by virtue of inhibitory interactions, the response of cells whose gain is not enhanced by attention will be suppressed by a “winner take all” mechanism (see, e.g., Desimone & Duncan, 1995; Lee, Itti, Koch & Braun, 1999). What then might be happening in behavioural tasks that claim to show that the appearance of attended items is altered? One possibility is that the receptive fields of cells responding to the non-attended items remain large, and so their responses continue to be influenced by aspects of the stimulus other than the object of interest. With a uniform surrounding background, the effect is to “dilute” the effect of matches between the properties of the unattended object and cells’ tuning. The appearance of attended and unattended items may therefore truly differ, but there is

The Functional Roles of Attention

•

141

no ambiguity about the veridical properties of stimuli. Ned Block (2010) suggested that we adopt a view of phenomenal experience dubbed “mental paint” in order to deal with the uncertainty over what constitutes a veridical phenomenal experience that Carrasco’s experiments on the alteration of experience by attention might imply. If, as we suggest here, attention serves to increase the veridicality of experience, then we can discard mental paint. In the account above, attention serves to optimise the veridicality of responses of cells involved in coding complex properties of stimuli. Such optimised responses might be used to engage action, but there is no necessity for this. We might need to know about an object for many reasons that do not involve directing actions toward it (e.g., in order to remember it, something it is hard to credit as being “action”). Similarly, once the identity of an object has been established as best it can, it may be that it is deemed irrelevant and is not subject to any further processing in terms of memory, planning, or action. If consciousness is associated with this deeper processing (as global workspace and allied theories hold), then it is clear that it might be necessary to attend to an object for it to enter consciousness, but attention is not sufficient—not all attended items need enter consciousness.

2. Further Evidence for Selection for Precision Further evidence for the idea that a function of attention is to facilitate enhanced perceptual precision comes from the case of type-2 blindsight. Blindsight is a kind of residual vision found in people with lesions to V1. Subjects with blindsight typically report no visual awareness, but they are nonetheless able to make above-chance guesses about the shape, location, color, and movement of visual stimuli presented to them in their blind field. Type-2 blindsight is a kind of residual vision found in patients with V1 lesions in the presence of some residual awareness (see, e.g., Foley & Kentridge, 2015). Studies indicate that the attributes experienced in type-2 blindsight are less determinate than the properties experienced in ordinary vision. When shown different letters, blindsight subject DB would sometimes report being aware of the direction of the stimulus and having a feeling of whether the stimulus was “smooth” (the O) or “jagged” (the X) (Weiskrantz et al., 1974). When strongly encouraged to provide an answer, blindsight subject KP described experiencing “a very faint flash” in response to stimuli (Weiskrantz, 1980), and blindsight subject JP reported being aware of ill-defined and poorly formed “blobs” when words were flashed in her blind field (Shefrin, Goodin, & Aminoff, 1988). Several subjects have reported having feelings of “something” or “something happening” (Weiskrantz, 1986). These reports testify to the hypothesis that the properties that subjects are aware of in their blind field in type-2 blindsight are determinables (as opposed to determinate properties, such as crimson and square), sometimes of the most general kind (“something”, “something happening”).

142

•

Robert W. Kentridge and Berit Brogaard

The reason type-2 blindsight differs from ordinary visual experience in this respect is likely that it is generated by an alternative visual pathway that bypasses V1. Morland et al. (1999) investigated blindsight subject GY’s ability to make luminance matches in his hemianopic field and between both hemifields. They found that GY was able to make matches when the stimuli were presented in the blind field but was unable to establish matches based on luminance when the stimuli were presented in opposing fields. A plausible explanation of this observation is that the perceived luminance of the stimuli in his blind field (perception of brightness) is derived from direct projections from subcortical areas to extrastriate areas bypassing V1, whereas the perceived luminance of the stimuli in his intact field originate in the normal visual pathway that includes V1. This would make it possible for him to compare stimuli on the basis of luminance when both are presented in the hemianopic field, but, when the stimuli are presented to opposing fields, the distinct pathways would yield different kinds of percepts, making lawful matching difficult. This indicates that V1 plays a crucial role in generating brightness perception. And if GY’s type-2 blindsight vision fails to be fine-grained, this further suggests that brightness perception is required for generating conscious awareness of determinate properties. The latter observation allows us to draw some connections between enhanced perceptual precision and attention (see also Nanay, 2010). Neurophysiological studies have shown that attentional modulation and changes in the luminance of a stimulus can create identical modulations of firing rates (Carrasco et al., 2004). It has also been found that attentional modulation and the intensity of the brightness of a stimulus may have a shared underlying mechanism (Treue, 2001). This points to a neural mechanism according to which attention modulates the strength of a stimulus by altering its perceived luminance, or brightness. But if attentional modulation and changes in luminance have a shared neural mechanism, then we can provide an argument for the view that attentional modulation yields a difference in the determinacy of the perceived attributes. Neurophysiological evidence points to a defect in brightness perception in type-2 blindsight. Brightness perception is likely compromised because type-2 blindsight takes place via a visual pathway that bypasses V1, a region that may be associated with generating brightness percepts. It also appears that only highly determinable (as opposed to determinate) properties are consciously represented in type-2 blindsight. This then suggests that when the mechanism for generating brightness is compromised, then visual experience cannot consciously represent very determinate properties. These considerations suggest that a possible function of attention may be to modulate the strength of a stimulus by altering its perceived luminance, or brightness, and thereby making the perceptual experience more precise.

The Functional Roles of Attention

•

143

3. Prevention of Informational Overload Modulating the strength of a stimulus by altering its perceived brightness is only one function of attention. Another function appears to be to prevent informational overload by selecting relevant information and filtering out irrelevant information from crowded visual scenes. For the case of vision, it has been found that attention can modulate responses in extrastriate and striate visual cortex and even LGN (O’Connor et al., 2002) and in that way affect the nature of our visual experiences. It has been reported to do this in three different ways. Attention enhances neural responses to attended stimuli, attenuates responses to ignored stimuli, and increases the baseline activity in the anticipation of visual stimulation (Koch & Tsuchiya, 2007). Independent evidence that a role of attention is to prevent informational overload comes from the case of hallucinogens (psilocybin, LSD, mescaline). The mushroom-derived hallucinogen psilocybin—one of the most selective hallucinogenic drugs studied—has been shown to significantly reduce subjects’ attentional tracking ability, although it has no significant effect on spatial working memory (Carter et al., 2006). A possible mechanism for how psilocybin affects attentional tracking can be identified by looking closer at how this chemical generates drug-induced hallucinations. It is by now fairly well established that psilocybin is a potent partial agonist at serotonin 5-HT1A/2A/2C receptors, with serotonin 5-HT2A receptor activation directly correlated with hallucinogenic activity (Glennon, 1990; Vollenweider et al., 1997; Nichols, 2004; Presti & Nichols, 2004). Though the mechanism of action varies for different hallucinogens, it is believed that 5-HT2A receptor activation of cortical neurons is responsible for mediating the signaling pattern and behavioral response to hallucinogens (Presti & Nichols, 2004; González-Maeso et al., 2007). The activation of the cortical serotonergic system does not fully explain the perceptual effects of psychedelic drugs, as not all 5-HT2A agonists have an excitatory mechanism of action and not all 5-HT2A agonists have psychedelic effects (e.g., methysergide). So, this raises the question of what other factors need to be present for drug-induced hallucinations to occur when subjects are under the influence of psilocybin. A promising suggestion for how psilocybin and other similar hallucinogenic drugs are associated with visual hallucinatory effects is that it activates layer V pyramidal neurons in the visual cortex, which engage in gating functions in communication between the cortex and the thalamus (Barkai & Hasselmo, 1994; Brogaard, 2013). When a hallucinogen binds to 5-HT2A serotonin receptor on layer V pyramidal neurons, this gives rise to an excitatory response via increased release of the excitatory neurotransmitter glutamate (Scruggs, Schmidt, & Deutch, 2003; Ceglia et al., 2004; Nichols, 2004; Torres-Escalante et al., 2004).

144

•

Robert W. Kentridge and Berit Brogaard

Now, 5-HT2A serotonin receptors are found on both layer V glutamatergic neurons and GABAergic interneurons that connect the visual cortex to the thalamus (Lee & Roth, 2012). There is also a direct activation of GABAergic interneurons through the synapses of pyramidal cells onto the interneurons (Markram et al., 2004). So, a large excitatory response in a pyramidal neuron will lead to a large inhibitory response in the interneuron. As GABAergic interneurons inhibit the brain regions they project to, this will inhibit the normal gating mechanisms of the thalamus, allowing too much information to enter the visual cortex. This points to a mechanism where psilocybin affects attentional tracking by allowing an informational overload of neural information to enter the visual cortex. The additional neural information available to the visual cortex includes both irrelevant information that is normally filtered out as well as random neural activity generated by the thalamus. Drug-induced hallucinations are generated when the brain attempts to make sense of random or irrelevant activity from the thalamus. If this mechanism for how hallucinations may occur under the influence of psilocybin is correct, then that suggests a possible role of selective attention as a modulator of informational input. A function of attention may be to prevent informational overload from the thalamus to sensory cortical areas. This sort of overload would lead to a loss in the tracking abilities of perceptual representations, as the perceptual representations would be a partial result of the brain’s attempt to make sense of random and irrelevant information. So, another way of formulating this role of attention is in terms of veridicality. We might say that attention serves to optimise the veridicality of conscious perceptual representations.

4. Attention for Action? Wu (2014) has argued that attention is for action. More precisely: attention is selection that guides task performance. According to Wu, for some modality M (perceptual, cognitive): If a subject S M-selects X to guide performance of task T, then S M-attends to X. Wu refers to this as an “empirically sufficient condition”. Action suffices for attention. Although Wu also argues that action is necessary for attention, this condition by itself is perfectly consistent with the view that attention has more than one function. One may think of “attention” as semantically on a par with “tool”. To a first approximation, the meaning of “tool” is “a device held in the hand used to carry out a particular function”’. But there are many different functions tools can play. The same goes for attention. As far as we are concerned, one function/purpose of attention could, in principle, be “for action”

The Functional Roles of Attention

•

145

(task-related). This does not rule out that attention can also be selection for precision or modulation of informational input. Increased precision and modulation of informational input may be used to engage action or improve action performance, but there is no requirement that action is engaged as a result of the optimised responses. A precise perceptual representation of an object may be generated only to be judged to be irrelevant and as a result never undergo further processing for memory, thought, planning, or action. Here, of course, it is important that we don’t read the suggestion that attention is selection for precision and modulation of informational input as the claim that enhanced perceptual precision and modulation of visual input require attention. If your spouse turns on the light in the room, this will increase the precision of your visual representation. But this sort of increase in precision does not require any additional attentional selection on your part. Likewise, if your spouse replaces most items in the room with uni-colored patches, she will in some sense have modulated how much visual information enters into your visual system. But this type of modulation does not require any additional attentional modulation on your part. How then are we to understand the claim that attention functions to select for precision and modulate informational input? For the case of vision, the claim is best understood in terms of top-down influences on neural processes in the visual cortex and on neural regions to which the visual cortex projects or from which it receives neural input. Without attentional top-down influences on the visual processes, brightness perception will be remarkably reduced and any conscious visual representation will be notably less precise. Likewise, without the top-down attentional selection of input from the thalamus to the visual cortex, the conscious visual representation that is generated in the visual cortex will be intermingled with hallucinatory elements that do not track reality. It may be argued that one could, in principle, artificially mirror the effects of selective attention by directly manipulating the computation of brightness and the informational flow from the thalamus. This is no doubt correct, but it doesn’t affect the functional claim we are making. A functional role of a mental process is a set of causes and effects that surround a mental process. Specifying a functional role of a mental process thus does not require that we identify a set of mental processes for which the target mental process is required. The aim is to identify a set of causes and effects in a normal brain in normal conditions. As such, it can be a functional role of attention to increase precision and modulate informational input, even if not every conceivable enhancement of perceptual precision or modulation of informational input requires attention. Now, it is plausible that a functional role of attention could be given, where attention simply is understood as the underlying processes that satisfy the specified functional roles. This is a stronger claim, however. It commits us to a conception of radically different underlying processes that satisfy the functional roles as attentional selectional processes. For example, we can imagine

146

•

Robert W. Kentridge and Berit Brogaard

that a microchip could be inserted into the visual system of people whose neural systems don’t prevent informational overflow and play the role that attention plays in neurotypical individuals. On a functional analysis of attention, this microchip would then not simply play the role of attention. The microchip would be (part of) the attentional system. Whether we ought to make this further and stronger claim is beyond the scope of the debate in this volume.

References Barkai, E., and Hasselmo, M. E. (1994). Modulation of the input/output function of rat piriform cortex pyramidal cells. Journal of Neurophysiology 72, 644–658. Block, N. (2010). Attention and mental paint. Philosophical Issues 20, 23–63. Brogaard, B. (2013). Serotonergic hyperactivity as a potential factor in developmental, acquired and drug-induced synesthesia. Frontiers in Human Neuroscience 7, 657. Carrasco, M., Ling, S., and Read, S. (2004). Attention alters appearance. Nature Neuroscience 7(3), 308–313. Carter, L.P., Richards, B.D., Mintzer, M.Z., and Griffiths, R.R. (2006). Relative abuse liability of GHB in humans: A comparison of psychomotor, subjective, and cognitive effects of supratherapeutic doses of triazolam, pentobarbital, and GHB. Neuropsychopharmacology 31, 2537–2551. Ceglia, I., Carli, M., Baviera, M., Renoldi, G., Calcagno, E., and Invernizzi, R. W. (2004). The 5-HT receptor antagonist M100, 907 prevents extracellular glutamate rising in response to NMDA receptor blockade in the mPFC. Journal of Neurochemistry 91, 189–199. Desimone, R., and Duncan, J. (1995). Neural mechanisms of selective visual attention. Annual Review of Neuroscience 18, 193–222. Foley, R., and Kentridge, R. W. (2015). Type-2 blindsight: empirical and philosophical perspectives. Consciousness & Cognition 32, 1–5. Glennon, R. A. (1990). Do classical hallucinogens act as 5-HT2 agonists or antagonists? Neuropsychopharmacology 3, 509–517. González-Maeso, J., Ang, R. L., Yuen, T., Chan, P., Weisstaub, N. V., López-Giménez, J. F., et al. (2008). Identification of a serotonin/glutamate receptor complex implicated in psychosis. Nature 452, 93–97. Gross, C. G., Rocha-Miranda, C. E., and Bender, D. B. (1972). Visual properties of neurons in inferotemporal cortex of the Macaque. Journal of Neurophysiology 35, 96–111. Koch, C., and Tsuchiya, N. (2007). Attention and consciousness: two distinct brain processes. Trends in Cognitive Science 11(1), 16–22. Lee, D.K., Itti, L., Koch, C., and Braun, J. (1999). Attention activates winner-take-all competition among visual filters. Nature Neuroscience 2(4), 375–381. Lee, H.-M., and Roth, B. L. (2012). Hallucinogen actions on human brain revealed. Proceedings of the National Academy of Sciences of the United States of America 109, 1820–1821. doi: 10.1073/pnas.1121358109 Markram, H., Toledo-Rodriguez, M., Wang, Y., Gupta, A., Silberberg, G., and Wu, C. (2004). Interneurons of the neocortical inhibitory system. Nature Reviews Neuroscience 5, 793–807. McAdams, C. J., and Maunsell, J. H. R. (1999). Effects of attention on the orientation tuning functions of single neurons in macaque area V4. Journal of Neuroscience 19, 431–441. Moran, J., and Desimone, R. (1985). Selective attention gates visual processing in the extrastriate cortex. Science 229, 782–784. Morland, A. B., Jones, S. R., Finlay, A. L., Deyzac, E., Le, S., and Kemp, S. (1999). Visual perception of motion, luminance and colour in a human hemianope. Brain 122(Pt 6), 1183–1198. Nanay, B. (2010). Attention and perceptual content. Analysis 70(2), 263–270. Nichols, D. E. (2004). Hallucinogens. Pharmacology Therapeutics 101, 131–181.

The Functional Roles of Attention

•

147

O’Connor, D. H., Fukui, M. M., Pinsk, M. A., and Kastner, S. (2002). Attention modulates responses in the human lateral geniculate nucleus. Nature Neuroscience 5, 1203–1209. Presti, D. E., and Nichols, D. E. (2004). Biochemistry and neuropharmacology of psilocybin mushrooms. Teonanacatl: Sacred Mushrooms of Visions, ed. R. Metzner, 89–108. Sonoma, CA: Green Earth Foundation. Reynolds J.H. and Desimone R. (1999). The role of neural mechanisms of attention in solving the binding problem. Neuron 24, 19–29, 111–125 Scruggs, J. L., Schmidt, D., and Deutch, A. Y. (2003). The hallucinogen 1-[2,5-dimethoxy4-iodophenyl]-2-aminopropane (DOI) increases cortical extracellular glutamate levels in rats. Neuroscience Letters 346, 137–140. Shefrin, S. L., Goodin, D. S., and Aminoff, M. J. (1988). Visual evoked potentials in the investigation of “blindsight”. Neurology 38(1), 104–109. Torres-Escalante, J. L., Barral, J. A., Ibarra-Villa, M. D., Perez-Burgos, A., Gongora-Alfaro, J. L., and Pineda, J. C. (2004). 5-HT1A, 5-HT2, and GABAB receptors interact to modulate neurotransmitter release probability in layer 2/3 somatosensory rat cortex as evaluated by the paired pulse protocol. Journal of Neuroscience Research 78, 268–278. Vollenweider, F. X., Leenders, K. L., Scharfetter, C., Maguire, P., Stadelmann, O., and Angst, J. (1997). Positron emission tomography and fluorodeoxyglucose studies of metabolic hyperfrontality and psychopathology in the psilocybin model of psychosis. Neuropsychopharmacology 16, 357–372. Weiskrantz, L. (1980). Varieties of residual experience. Quarterly Journal of Experimental Psychology 32, 365–386. Weiskrantz, L. (1986). Blindsight: A Case Study and Implications. Oxford: Oxford University Press. Weiskrantz, L., Warrington, E. K., Sanders, M. D., and Marshall, J. (1974). Visual capacity in the hemianopic field following a restricted occipital ablation. Brain 97, 709–728. Wu, W. (2014). Attention. London: Routledge.

CHAPTER

10

Attention and Perception

A Necessary Connection? WAYNE WU

Is attention necessary for perception? Can we perceive only what we attend to? If we think of perception as occurring in two modes—namely, as conscious or unconscious—then our opening question divides into two. (1) Is attention to some target necessary for conscious perception of it? (2) Is attention to some target necessary for unconscious perception of it? As it turns out, (1) has garnered most (all?) of the attention and will be the main target of our discussion. The question of attention as necessary for unconscious perception has, to my knowledge, been neglected, but we shall briefly consider it as well. In this essay, I begin by clarifying attention and consciousness, then argue that the more discussed issue of attention as gatekeeper should be set aside. The central question should not be whether attention allows you to be conscious of some X but rather, given that you are conscious of X, how does attending to it modulate your awareness of it? I then make some remarks about attention as necessary for unconscious perception. The upshot is that the central questions remain open, despite the opinion of many theoreticians that (1) clearly receives a positive answer.

Attention? Consciousness? It is remarkable how obscure the relevant notions deployed in question (1) are despite rigorous debate over the past century (for some references regarding the older debate, see Watzl 2011). An important clarification is Block’s distinction between phenomenal and access consciousness (Block 1995). When theorists express a claim about the relation between attention and 148

Attention and Perception

•

149

consciousness, they refer to phenomenal consciousness, specifically phenomenal perceptual consciousness (in practice, typically visual consciousness of X where X can be a visible property, object, or location). Thus, the focus is on what it is like to have a perceptual experience of some X, on the qualitative or phenomenal character associated with experiencing it. Access consciousness is often invoked as a condition on phenomenal consciousness—namely, that one is phenomenally conscious of X only if one is access conscious of it. While access was originally given a more restricted reading by Ned Block, we can take a system Y’s access of X to involve Y’s selecting X in order to perform some function of Y such as guiding and control of behavior (this allows for reflex, which Block wished to exclude). We can also speak of a subject’s access of X in that the subject selects X to perform subject level functions. Once the relation of access is clear, in terms of selection, one can then add further restrictions on access (e.g., rational access) to refine the notion of access consciousness. It is the job of the access theorist to flesh out the details of these relations. I focus first on phenomenal consciousness and return to access consciousness later.1 Attention remains surprisingly obscure. Cognitive scientists affirm that attention is selection for information processing, and I suspect many think this suffices for a theoretical analysis. Unfortunately, it does not. It simply fails to identify only attention. In vision, selection for information processing is present the moment visual neurons are activated, for their response typically exhibits a form of selectivity or tuning (e.g., their preferred stimulus, that which generates the strongest response relative to other stimuli). This already occurs at the retina when photoreceptors select specific wavelengths of light, but tuning is seen throughout the visual system. Where tuned neurons synapse onto others, then there is selection for further processing throughout the visual system. Yet such selection doesn’t suffice for attention, for the subject’s attending to a stimulus. Thus, if attention is selection, one must be more specific about what type of selection it is. I have argued that scientists endorse a basic conception of attention as selection that guides task performance (Wu 2014). The methodology of the cognitive science of attention, in psychophysics and in neuroscience, presupposes what I have called an empirical sufficient condition: for some modality M (perceptual, cognitive): If a subject S M-selects X to guide performance of task T, then S M-attends to X. Thus, if a subject visually selects a spatial property to guide performance in a reaching task, then S visually attends to that property. Consider a case where a scientist wants to ensure that the subject keeps track of (i.e., attends to) four moving circles out of eight on a screen. How can the experimenter ensure that the subject attends to just the targets at issue? Every experimenter understands

150

•

Wayne Wu

how to accomplish this—namely, by designing a task such that selection of the relevant items is necessary for adequate task performance. Thus, Zenon Pylyshyn constructed the multiple object tracking (MOT) paradigm where the subject is given a set of targets to track (for an overview, see Scholl 2009). In our envisioned experiment, four circles are flashed at the beginning of a trial and then all circles move. At the end of the trial, subjects are asked to identify which circles were the original targets. If the subject correctly identifies the original targets, the subject must have tracked the objects on the basis of selecting some relevant feature (errors introduce complications, but we can restrict sufficiency to some notion of adequate task performance where selection is necessary for adequacy). Accordingly, the subject attended to that feature and by association the objects individuated by the feature. More examples will be given below (for a discussion of other paradigms in light of the condition, see Wu 2014). The empirical sufficient condition informatively makes precise the claim that attention is selection for further processing, where further processing concerns the guidance of task performance. Accordingly, it is natural to take selection for task to not merely be correlated with attentional selection but to, in these cases, constitute or realize attention. This leaves it open that attention could be other things such as selection for precisification of representations or selection to prevent information overload as Kentridge and Brogaard contend (see this volume, Chapter 9). However, in the end, I think that the selection for task, and more broadly the selection for action, conception of attention

Figure 10.1 Simulation of the response of a neuron in primate visual area MT to stimuli moving in different directions, placed in its receptive field. The neuron shows a preference for movement in the “zero degree” direction which yields the strongest response in terms of spikes/second. The solid curve is the unattended condition when the animal is directing attention outside the receptive field; the dotted curve is the attended condition when the animal is directing attention into the receptive field. Figure is from J. Lee and J. H. R. Maunsell (2009), A Normalization Model of Attentional Modulation of Single Unit Responses. PLOS ONE 4(2): e4651. doi:10.1371/journal.pone.0004651

Attention and Perception

•

151

is conceptually prior (on attention as selection for action, see Allport 1987; Neumann 1987; Wu 2014). To see this, note that the empirical sufficient condition is critical for the interpretation of neural data. Consider Figure 10.1, which shows two different responses of the same neuron to a set of stimuli. The two curves illustrate what is known as gain modification (here, multiplicative gain), the amplification of a signal, exhibited as the multiplication of each point in the lower curve by a constant factor (this generates the higher curve). Note first that the neuron is selective in that it has a tuning function, the curves indicating the neuron’s preferred stimulus at the peak of the response function. Such selectivity embodied in an individual curve is not sufficient for attention. Rather, it is the effect of gain modulation, the difference between the two curves, that reflects attention. Attention is said to boost response gain. Even then, the difference between the two curves does not entail attention. After all, one could generate the higher curve simply by modifying the stimulus properties independent of attention, say increasing the intensity of the stimulus to drive for each stimulus the measured firing rate in the top curve (indeed, Kentridge and Brogaard note evidence that the same mechanism might support both processes, Section 2, pp. 141–2). Why are we warranted to see attention as operative in this case? The answer is the empirical sufficient condition, for in the experiment, the researchers controlled the subject’s attention by making appropriate targets task relevant. The idea is that when experimenters collecting the neural data set a task-relevant target in the neuron’s receptive field (the area of space to which the neuron will respond to appropriate stimuli), subjects have to select some relevant X at that location in order to perform a task. Under these conditions of selection for task (hence attention), gain modulation occurs. Furthermore, the data for the bottom curve is generated by directing the subject’s attention outside the receptive field, again by defining a different target as task relevant. Thus, the interpretation of a difference in neural response as “attentional” presupposes the empirical sufficient condition. The same is true of the experiment that Kentridge and Brogaard discuss that effectively demonstrates a contraction of the receptive field around a task relevant object (Chelazzi et al. 1998; Chelazzi et al. 2001). Again, this conclusion is warranted because of the experimental context: the experimenters shift the object that is task relevant to the subject, and then observe changes in neural activity. So, an empirical conception of attention should agree that selection for task constitutes a central form of attention, if we conceive of attention as selection of some sort as cognitive scientists tend to do. The question then is whether we should think of attention as anything else distinct from selection for task. For example, does attention make representations more precise? There is some debate about how to understand precision (Stazicker 2011; Block 2014a), but let us grant it. How should we understand the relation between this effect and

152

•

Wayne Wu

attention? Is the point of attention, in certain cases, to make representations more precise, and does it do this independently of serving action? Kentridge and Brogaard’s contention is that the point of attention can be to increase precision independently of action: “such optimised responses might be used to engage action, but there is no necessity for this” (p. 141). Consider how one might engineer a system that must be selective for action. One can imagine that in certain cases, generating actions relies on more precise representations, and attention that serves performance does so by increasing precision. Part of creating this system would be to endow it with a mechanism for doing just that when it is needed for action (say, when perception of fine details are critical for execution), but, once the machinery is in place, it also seems possible that the mechanism could act independently of any action context. That is, one can trigger perceptual precisification independently of the subject’s doing anything. Do we in such cases have attentional selection for precision but not for action? It seems that we can increase precision without engaging attention. Consider an optometrist who changes lenses while you look at an “E” projected on the wall. You might be preoccupied and not paying attention to the diagram but thinking of other matters (cognitive attention distracts you), or your visual attention is directed elsewhere, even as the doctor effectively sharpens your representation of the “E” by shifting the power of the lenses (say, subtly so as not to capture your attention).2 Here, increased precision is not linked to action, but it also doesn’t seem linked to attention. We can imagine more exotic cases where precision of representations of the “E” is increased by directly manipulating neural response even if attention is directed elsewhere. So, it seems that increasing precision can be disconnected from attention just like gain modulation can be.3 What then has to be added to increasing precision to secure a clear role for attention? Where precision seems clearly tied to attention is where it is serving task performance (I will consider a specific case where increasing precision is its own end below, something perhaps Kentridge and Brogaard have in mind). If so, attention’s connection to task (action) is prior in the order of analysis. The point is that increasing precision is tied to attention when it is tied to action.4 A similar point arises for taking attention as tied to elimination of informational overload given capacity limitations. Again, this can be disconnected from attention. Neumann (1987) noted that irrespective of limitations on capacity, attention is required for agency. Consider the simple case of an individual who sees a ball that she or he can kick with either foot. The depiction of possible behaviors (Figure 10.2) identifies two potential input-output links where the input is understood as a perceptual state directed at the ball and the outputs are understood as kicking responses. To kick the ball, the subject must select relevant visible properties from the ball to program the appropriate kick, and this is selection for task. So, our empirical sufficient condition

Attention and Perception

•

153

Figure 10.2 Depiction of a behavior space that delineates four possible visually guided actions. The left side identifies two inputs, the visual experience as of two different balls, that can then inform two possible kicking movements. A specific action of kicking one of the balls with a specific foot is then depicted as the connection between input and output, say the visually guided left foot kick of the soccer ball.

gives us attention in any context where the subject kicks the ball guided on the basis of seeing it. There is, however, no capacity limitation, or so it seems.5 The informational load is minimal (let us assume). Yet both kicks cannot be performed simultaneously as one foot must be planted for the other foot to kick. To do anything, the agent must choose a specific path where the input state selectively informs task response, and this requires selective attention for task as different parts of the ball are relevant. Attention is needed for agency even without capacity limits. This is Neumann’s crucial insight. Of course, agents are capacity limited if only because of bioenergetic limitations of the brain. It follows that for there to be effective agency, capacity cannot be exceeded (just think of trying to multitask when you obviously cannot). In that respect, attention allows you to deal with capacity limitations as part of its allowing you to be an agent. Dealing with capacity limitations is a secondary effect of dealing with being an agent in a complicated world where not all possibilities can be performed simultaneously. In general, I agree with Kentridge and Brogaard on attention as having certain effects (altering information processing) but disagree that they have found a way of conceiving of attention in a task-independent way. The concept of attention is essentially of

154

•

Wayne Wu

a task-relative capacity that, of course, can have lots of interesting main and side effects on processing such as precisification and informational control.

Filtering (Out) Gatekeeping Gatekeeping is the claim that attention is necessary for consciousness, specifically that one is phenomenally conscious of X only if one is attending to X. Attention can be understood either as perceptual attention or as cognitive access (some sort of cognitive attention). I am going to argue that (1) there is no empirical evidence for gatekeeping in the perceptual attention formulation, (2) our starting empirical hypothesis should be the falsity of gatekeeping, namely what Ned Block calls overflow, and (3) consequently, gatekeeping in the cognitive access version should also be empirically set aside and not treated as a default hypothesis (for more detailed discussion, see chapters 5 and 6 of Wu 2014; the following argument against access replaces my hedging in the latter chapter). Consider the visual attention version of gatekeeping: If S is visually conscious of X, then S is visually attending to X. Where attention is absent, we get induced inattentional blindness where “blindness” means the absence of visual perception. Against this, consider the following version of overflow: If S is not visually attending to X, S is a visual agnosic with respect to X. I use “visual agnosia” with some trepidation given its specific connotations, but the idea is that one can be visually agnosic with respect to X while still seeing X. The agnosia would prevent the subject from seeing the X as an X. Thus, the second conditional identifies a case of visual consciousness without visual attention, and hence the negation of gatekeeping. Think here of an associative agnosic who can see a gorilla but fails to categorize it as a gorilla and hence fails to see the object as a gorilla. Nevertheless, the subject sees the gorilla. We can think of agnosia as a kind of blindness to certain visible properties, a more local visual defect as opposed to the total absence of vision. Gatekeeping is thought to be well supported due to inattentional blindness and change blindness paradigms, work on the attentional blink, and hemispatial neglect. What is common in all these cases is that the absence of attention is taken to suffice for literal blindness (if it is not literal, then the use of “blindness” is misleading). In hemispatial neglect, on one theory, the subject is unable to shift attention to the left hemifield and so is blind to stimuli in that hemifield. In the other cases, experimenters induce blindness by ensuring that attention is not,

Attention and Perception

•

155

cannot be, directed at X. So, the experimenters guarantee (1) that attention is not directed at (X) and thereby claim to induce (2) blindness with respect to X. Take the well-known experiment by Daniel Simons and colleagues to induce inattentional blindness to a gorilla (Simons and Chabris 1999). How did Simons ensure that (1) obtained? He used the empirical sufficient condition by having his subjects track one of two basketballs. Let us grant that Simons was able to “distract” his subjects by having them lock attention on to a ball in order to perform a counting task. A gorilla then appears halfway through the display, with a large number of subjects failing to report the gorilla. Simons concludes that subjects are inattentionally blind to the gorilla on the basis of (3) failure of reporting of X (in this case, the gorilla). So the logic of this experiment is clear. Ensure (1), and, on the basis of (3), infer (2). This logic is present in all the other paradigms or conditions mentioned. Accordingly, (2), blindness to X, best explains (3), the absence of report on X, and it is plausible to then assume that (1) is the cause of (2) as per gatekeeping. The subject is blind because the subject fails to attend. Surprisingly, no one seems to have noticed that (1) on its own is sufficient for (3) irrespective of the truth of (2). For let us assume that experimental conditions guarantee that tracking the ball consumed all the subject’s attentional resources. That is the logic of the experiment—namely, to ensure that no attention remains to direct at the gorilla. But if there is no attention left for the gorilla, one can predict that no subject will report the gorilla so long as attention isn’t captured by the gorilla (where the gorilla captures attention, subjects will report it as observed in the experiment). To report the gorilla, one must select it to inform report, but, by the empirical sufficient condition, such selection is attention. Yet if (1) holds, by hypothesis there is no such capacity left for report. So, (1) predicts (3) even if the subject consciously sees the gorilla. Given (1), the subject will not notice the gorilla, or the subjects will notice because the gorilla captures the subject’s attention. This is precisely the pattern of observed results. Thus, the phenomena noted do not speak for or against gatekeeping. This undercuts the standard evidence for gatekeeping. Perceptual gatekeeping has been accorded a status it has not earned. This is not to say that it is false, but it should not be the starting hypothesis about the relation between attention and consciousness. To my mind, the gatekeeping thesis is theoretically more committed than overflow, which might come as a surprise. If gatekeeping is stronger or more theoretically committed than overflow, then one might choose to begin with the simpler theory—namely, awareness outside of attention. Issues about theoretical strength are tricky, so let me make a historical point. As Jensen et al. point out (Jensen et al. 2011), the progenitor for inattentional blindness work began with Ulric Neisser, who constructed similar videos to provide a visual analog of the dichotic listening paradigm that drove much early attention work (Cherry

156

•

Wayne Wu

1953). In dichotic listening, subjects are presented with two verbal streams, typically one in each ear, and tasked to shadow (verbally parrot) one of the two streams (cf. selection for task). Authors focused on what could be distinguished in the unattended channel—surprisingly, not much. For example, subjects could not report what was said in the unattended channel, and this led to a long debate about whether attention acted as a filter early or late in processing. Broadbent argued for early selection since the subject was largely not aware of the semantics of the unattended channel even though subjects could tell if the voice was a male or female (for discussion see Pashler 1998; Wu 2014). No one, however, was inclined to speak of inattentional deafness to the unattended channel. All allowed that the subject remained aware of certain features in that channel, with the debated issue being how much. The assumption was not inattentional blindness/deafness but, as I have put it, inattentional agnosia. In this context, a shift from agnosia to blindness, from an impoverished awareness to the complete absence of awareness, is a strong change in theoretical perspective that requires experimental evidence. In the visual domain, the received view is that the inattentional blindness paradigms (Mack and Rock 1998) and their ilk that we have discussed provided that shift, and historically that is true. But if my critique is correct, the shift was not theoretically warranted, and indeed a mistake. It goes beyond what earlier attention researchers were inclined to endorse. This point rebounds to the cognitive access version of gatekeeping, for the following inference seems plausible: 1. If S is perceptually conscious of X, then S is access conscious of X. 2. S is access conscious of X only if S perceptually attends to X. 3. Therefore: If S is perceptually conscious of X, then S perceptually attends to X. In this way, the access version of gatekeeping entails the perceptual attention version of gatekeeping. What should our empirical attitude be toward access gatekeeping? I have suggested that our starting position should be inattentional agnosia, and this means accepting provisionally the falsity of (3). Number (2) is plausible, and various theorists hold that perceptual attention makes things accessible (Dehaene, Kerszberg, and Changeux 1998; Prinz 2012). Since (3) is false, by hypothesis, but the argument is valid and (2) is independently plausible, (1) must be false. The bottom line: the appropriate empirical orientation is to deny both versions of gatekeeping. There can be consciousness outside of attention. This is admittedly a controversial claim, but I hope to have shown that there are prima facie grounds to question what has become the received view about the relation between attention and consciousness.

Attention and Perception

•

157

Put Up or . . . ? I have not provided evidence that one view (gatekeeping) is false or that another (overflow) is true. Rather, my point is that (a) we need to be clearer on what attention is so as to (b) assess the claims made regarding attention and conscious perception. To its detriment, the field has proceeded too long with a willingness to give only vague characterizations of attention. I have extracted a widely accepted (even if implicitly) empirical conception of attention and on that basis argued against the widely accepted evidence for gatekeeping. Accordingly, I suggested that we fall back to inattentional agnosia as the default hypothesis, something consistent with the theoretical orientation toward attention in the early days of attention work. This leaves open this empirical question: is there experimental evidence for or against either view? There is a substantial methodological challenge (Block 2007). What we want is a situation where attention is fully directed away from X and a way to assess whether the subject is conscious or not of X in that context. Yet evidence for the latter requires report of some sort, but report plausibly requires attention. The conditions that guarantee attention away from X undercut obtaining the evidence we need regarding consciousness. We have largely failed to confront this challenge, hindered by unclear conceptions of attention. To see one pitfall, consider a recent claim of Ned Block’s (Block 2014b) regarding empirical evidence for overflow (see Block’s discussion of other more compelling cases in that paper). The experiment (Bronfman et al. 2014) involves an adaptation of the Sperling paradigm that involves two tasks with respect to an array of letters of different colors, either high color diversity (where the letters take many different colors) or low color diversity (where the letters are colored in a narrower range, say only greens and blues). The tasks are to identify the letters in a cued row as well as to identify the level of color diversity, and the experimenters purport to show that there is awareness of color outside of attention. Using a cue, the experimenters direct the subject’s “focal attention” to a specific row and show that, despite this, subjects are accurate in judgments of color diversity. “Focal” here is an all too common fudge factor, for the fact that the subjects are instructed to perform two tasks shows that they will deploy attention accordingly.6 The color distribution of the entire array is task relevant, and, by the empirical sufficient condition, the subjects are attending both to the identity of the letters on the cued row and to the distribution of colors across the array (surely, that would be one’s strategy). There is no evidence here of awareness outside of attention but of two ways of deploying attention (attention is split to perform two tasks).7 I am inclined to tentatively side with Block on overflow, but a significant hurdle remains: can we obtain evidence for it? The key here will be coming up with insightful experiments that can only come into view once we have a clearer conception of the constraints and central concepts, such as attention.

158

•

Wayne Wu

Advances have been stymied by a failure to see that attention is not a mysterious notion that is undefinable but, in fact, a simple capacity that is central to the performance of any task. Seeing through the haze opens up new possibilities for innovative empirical work. Perhaps the possibility of such innovative work is a dream. Dennett and Cohen have argued that overflow is untestable (Cohen and Dennett 2011). Given the ingenuity of experimenters, I do not think we have yet exhausted the experimental possibilities to test for overflow. Yet even an endorser of gatekeeping should hope for new, ingenious experiments. The claim of gatekeeping is that whatever serves as gate is a necessary condition on consciousness. The experimental evidence that would support this claim would be the demonstration that when the necessary gate is removed, consciousness disappears as well. The removal of access and accessibility must be conjoined with evidence that consciousness is absent. Yet the problem seen in interpreting inattentional blindness experiments arises here as well. If one eliminates access or accessibility, one eliminates the means for report. The subject is not in a position to say whether conscious is present (overflow) or absent (gatekeeping). Dennett and Cohen consider the perfect experiment where a conscious state is isolated from the mechanisms of access. “How would we know that the state is present?” they pointedly ask the overflow theorist. Sure, but we can direct a similar question at gatekeepers. How would we know that the state is absent? This is what gatekeeping predicts, but we must remember that the absence of information is not information about absence. So, if the window of access to consciousness is closed by removing the mechanisms of access, then we can neither affirm nor deny that consciousness is present. The window onto the relevant state is closed, so the relevant information is, by hypothesis, not accessible. It then seems that gatekeeping is subject to the same challenge of evidential support as overflow. Both theories, then, need ingenious experiments to try to shift the dial in favor of one over the other. The two views are, in many ways, on the same footing.

Attention as Necessary for Unconscious Perception? We return briefly to question (2): is attention necessary for unconscious perception? This question has gotten very little attention, so we might reflect on the relevant issues with the hopes that new work might be informed by the clarifications presented earlier. The paradigm cases of unconscious perception are arguably blindsight (Weiskrantz 1986) and vision served by the dorsal stream (e.g., patient D.F., see Milner and Goodale 2006; Wu 2013). There are two questions we can pose to make headway here. (A) Is there unconscious seeing without attention (a version of overflow)? (B) Is there evidence that attention limits unconscious perception or how could we acquire evidence for this limit? These questions are difficult to assess, for the primary access experimentalists have to unconscious visual states is via the behavior of individuals with

Attention and Perception

•

159

brain lesions (neuropsychological patients). In these contexts, the evidence of unconscious visual states (often, denial that one sees X conjoined with a response that is selective with respect to X) is also evidence of visual attention, selection of the relevant parameter to inform response. This suggests that attention as tied to unconscious vision, but it does not settle the question whether attention is necessary for unconscious vision. Indeed, the likely results obtained when using an inattentional blindness paradigm to assess gatekeeping for unconscious vision, setting aside the methodological worries above, would not be surprising given that such patients already have significantly defective vision. Similarly, the methodological worries about testing overflow accrue in the case of unconscious vision, given that selection for task is the primary way we can experimentally locate unconscious visual states. One possibility relies on the fact that once we talk of unconscious vision, we move from what Chalmers calls the hard problem of consciousness to the easy (easier) problems of access. The recipe for the approach goes as follows. Unconscious visual states will be realized by certain visual brain states in a given context. Using behavioral criteria, we can isolate the presence of unconscious visual states with a view to identify their neural correlate N of a subject level unconscious state. In principle, for any given N, we can then assess whether it is activated in contexts where attention is plausibly directed elsewhere. So, we could assess the presence of an unconscious visual state that represents one object while attention is directed at another, using the selection for task criterion. The central challenge for this task, beyond the technical ones, is to understand what it takes for a neural state to realize a non-conscious mental state. It is not enough, for example, to identify the visual system’s carrying information as sufficient on its own for a mental state whose content is a function of that information. After all, a rod in the retina will carry information, yet no one, I take it, is willing to ascribe mentality solely on that basis. So there are old questions about the relation between the mind (unconscious mental states) and the brain that still need to be addressed. Yet unlike assessing consciousness outside of attention, it might seem more plausible to theorists that we might, someday, assess for unconscious perceptual states outside of attention. One interesting question would be the following: if overflow turns out to be true for unconscious vision (i.e., there are unconscious visual states that are not in the zone of attention), why isn’t the same true for conscious vision? The possibility of a striking bifurcation in the role of attention in vision is tantalizing.

To Conclude It is attention and conscious vision, however, that grabs our theoretical focus. I agree with Kentridge and Brogaard on shifting emphasis to attention as modulating, not gating, consciousness. Marisa Carrasco’s work has been at the center of these discussions (Carrasco, Ling, and Read 2004), but, if inattentional

160

•

Wayne Wu

agnosia is correct, all the experiments on inattentional blindness address the same issue: how does my consciousness of X alter (as opposed to disappear) depending on whether I am attending to X or not? In the gorilla case, it might be that when I shift from inattention to attention to the gorilla, there is some increase in processing that leads to a shift from seeing the gorilla as a black blob to seeing it as a gorilla. Attention might release one from agnosia. In Carrasco’s case, she has argued that attention shifts how things are perceived to be, say, the apparent contrast of a Gabor patch. How to understand such shifts is a matter of active research. Let me close with William James, who, for the most part, got matters right: Everyone knows what attention is. It is the taking possession by the mind, in clear and vivid form, of one out of what seem several simultaneously possible objects or trains of thought. Focalization, concentration, of consciousness are of its essence. It implies withdrawal from some things in order to deal effectively with others. James characterized attention as the mind’s taking possession of something relative to others so as to deal effectively with it, and this echoes the sufficient condition deployed in this paper, a condition I have argued as central to the experimental methodology of the science of attention. At the same time, James was also certain that consciousness is of attention’s essence, but the link between attention and perceptual consciousness remains very much in question. It is very much a live question. On this, Kentridge’s work has been seminal to showing us attention without consciousness (Kentridge 2011). But how attention yields or modulates consciousness is a hard problem to which we have yet to fully engage, in part because of the confusions I noted earlier about gatekeeping and attention. I am optimistic that if we approach these issues with a fresh eye, we can make substantial progress.

Notes 1. Matters are more complex depending on whether the issue is access or accessibility. I will not delve into that distinction here. Block’s restriction involved invocation of “rational” in talk of control, and this would rule out stimulus-driven reflexive responses. Readers can insert their own ways of restricting the notion of access, likely with little ill-effect on what follows in the text. 2. You might note that I am assuming overflow here, but I will undercut gatekeeping in the next section. 3. Some arguments against representationalist attempts to capture attention (actually conscious attention, but the same arguments can be applied here) are given in Wu 2011b. 4. A way to bring our two positions into harmony would be to allow that so long as such precisification could serve action, because it engages machinery that is action-oriented, then precisification on its own could count as attentional. But if this position is endorsed, it still holds to the “action first” account of attention.

Attention and Perception

•

161

5. Claims about capacity limitations should be accompanied by computations that show that information does literally exceed capacity, a task for information-theoretic calculations that the early researchers on attention took pains to undertake but which these days are replaced by more general, non-numerical assertions (cf. Broadbent [1958] who introduced Shannon information theory to psychology). If you think kicking balls overloads the system, I invite you to provide the computations to demonstrate this, but it will be easy to find examples where informational overload is not an issue. 6. I’ll say something stronger in a footnote. Empirical work on attention is plagued with overuse of metaphors whose empirical cash value is rarely demonstrated. The idea of focal attention is confused as is the idea that attention comes in degrees, so more or less attention. When one talks about spotlights or attention as a limited resource, such metaphors breed more metaphors. After over a half-century of modern empirical work on attention, surely we can do better. Note that all researchers should accept the selection for task view given its centrality to methodology and interpretation in the science of attention. But talk of focal selection for task or more or less selection for task is confused or means something rather innocuous. 7. Koch in recent work has tried to argue that the absence of dual task interference shows that attention is not divided and that one task can be performed without attention. This is a complicated matter for it does not simply follow from the absence of task interference that attention is not divided, especially given the methodological centrality of the empirical sufficient condition. But these are complicated matters that I must defer for elsewhere (Wu, in preparation; for more discussion of this experiment, see Wu 2014, section 5.7.1). Block’s work on crowding is another ingenious attempt to circumvent some of the methodological challenges that he has done much work to articulate (Block 2012).

References Allport, A. 1987. “Selection for Action: Some Behavioral and Neurophysiological Considerations of Attention and Action.” In Perspectives on Perception and Action, edited by Herbert Heuer and Andries Frans Sanders, 395–419. Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers. Block, Ned. 1995. “On a Confusion about the Function of Consciousness.” Behavioral and Brain Sciences 18: 227–47. ———. 2007. “Consciousness, Accessibility, and the Mesh between Psychology and Neuroscience.” The Behavioral and Brain Sciences 30 (5–6): 481–99; discussion 499–548. doi:10.1017/ S0140525X07002786. ———. 2012. “The Grain of Vision and the Grain of Attention.” Thought: A Journal of Philosophy 1 (3): 170–84. doi:10.1002/tht3.28. ———. 2014a. The Puzzle of Perceptual Precision, edited by T. Metzinger & J. M. Windt. Open MIND. Frankfurt am Main: MIND Group. http://open-mind.net/papers/the-puzzle-ofperceptual-precision/getAbstract. ———. 2014b. “Rich Conscious Perception Outside Focal Attention.” Trends in Cognitive Sciences 18 (9): 445–47. doi:10.1016/j.tics.2014.05.007. Broadbent, Donald Eric. 1958. Perception and Communication. New York: Pergamon Press. Bronfman, Z. Z., N. Brezis, H. Jacobson, and M. Usher. 2014. “We See More than We Can Report: ‘Cost Free’ Color Phenomenality Outside Focal Attention.” Psychological Science 25 (7): 1394–403. doi:10.1177/0956797614532656. Carrasco, Marisa, S. Ling, and S. Read. 2004. “Attention Alters Appearance.” Nature Neuroscience 7 (3): 308–13. Chelazzi, L., J. Duncan, E. K. Miller, and R. Desimone. 1998. “Responses of Neurons in Inferior Temporal Cortex during Memory-Guided Visual Search.” Journal of Neurophysiology 80 (6): 2918–40. Chelazzi, L., E. K. Miller, J. Duncan, and R. Desimone. 2001. “Responses of Neurons in Macaque Area V4 during Memory-Guided Visual Search.” Cerebral Cortex 11 (8): 761–72.

162

•

Wayne Wu

Cherry, E. C. 1953. “Some Experiments on the Recognition of Speech, with One and with Two Ears.” Journal of the Acoustical Society of America 25 (5): 975–79. Cohen, Michael A., and Daniel C. Dennett. 2011. “Consciousness Cannot Be Separated from Function.” Trends in Cognitive Sciences 15 (8): 358–64. doi:10.1016/j.tics.2011.06.008. Dehaene, Stanislas, M. Kerszberg, and J. P. Changeux. 1998. “A Neuronal Model of a Global Workspace in Effortful Cognitive Tasks.” Proceedings of the National Academy of Sciences of the United States of America 95 (24): 14529–34. Jensen, Melinda S., Richard Yao, Whitney N. Street, and Daniel J. Simons. 2011. “Change Blindness and Inattentional Blindness.” Wiley Interdisciplinary Reviews: Cognitive Science 2 (5): 529–46. doi:10.1002/wcs.130. Kentridge, R. W. 2011. “Attention Without Awareness: A Brief Review.” In Attention: Philosophical and Psychological Essays, edited by Christopher Mole, Declan Smithies, and Wayne Wu, 228–46. New York: Oxford University Press. Mack, Arien, and Irvin Rock. 1998. Inattentional Blindness. Cambridge, MA: MIT Press. Milner, A. David, and Melvyn A. Goodale. 2006. The Visual Brain in Action. 2nd ed. Oxford: Oxford University Press. Neumann, O. 1987. “Beyond Capacity: A Functional View of Attention.” In Perspectives on Perception and Action, edited by Herbert Heuer and Andries Frans Sanders, 361–94. Hillsdale: Lawrence Erlbaum Associates, Publishers. Pashler, H. E. 1998. The Psychology of Attention. Cambridge, MA: MIT Press. Prinz, Jesse. 2012. The Conscious Brain. Oxford: Oxford University Press. Scholl, Brian J. 2009. “What Have We Learned about Attention from Multiple Object Tracking (and Vice Versa)?” In Computation, Cognition, and Pylyshyn, edited by D. Dedrick and L. Trick, 49–78. Cambridge, MA: MIT Press. Simons, D. J., and C. F. Chabris. 1999. “Gorillas in Our Midst: Sustained Inattentional Blindness for Dynamic Events.” Perception 28 (9): 1059–74. Stazicker, J. 2011. “Attention, Visual Consciousness and Indeterminacy.” Mind & Language 26 (2): 156–84. Watzl, Sebastian. 2011. “Attention as Structuring of the Stream of Consciousness.” In Attention: Philosophical and Psychological Essays, edited by Christopher Mole, Declan Smithies, and Wayne Wu, 145–73. New York: Oxford University Press. Weiskrantz, L. 1986. Blindsight: A Case Study and Implications. Oxford: Clarendon Press. Wu, Wayne. 2011b. “What Is Conscious Attention?” Philosophy and Phenomenological Research 82 (1): 93–120. ———. 2013. “The Case for Zombie Agency.” Mind 122 (485): 217–30. ———. 2014. Attention. Abingdon, UK: Routledge.

PART

VI

Can Perception Be Unconscious?

CHAPTER

11

Debate on Unconscious Perception IAN PHILLIPS AND NED BLOCK

Part 1: Phillips To the untutored ear, the idea that perception does not require consciousness may sound as absurd as the idea that thunderstorms do not require anything to be happening in the sky (cf. Wiggins 2001: 218), or that being red does not require being coloured (cf. Moore 1925: 46–7). What then persuades contemporary theorists that unconscious perception is hard, scientific fact? To avoid “getting bogged down in theories of perception”, Prinz (2015: 372–3) stipulates that “unconscious perception” is to mean the “unconscious transduction of information . . . useable by the organism that transduces it”. But this will not suffice if our interest is in defending the philosophically substantive thesis that episodes of the same fundamental kind as episodes of conscious perception can occur unconsciously (Burge 2010: 374–5; Block 2012: 11–12; cf. Prinz 2010: 310). A traditional idea is that perception essentially involves occupying a subjective perspective on an objective world. Pursuing this idea, Burge characterizes perception as constitutively a matter of “objective sensory representation by the individual” (2010: 368). So characterized, not all useable transduced information constitutes perception. For an informational state to constitute perception it must (a) have objective content, representing how particulars are arranged in the subject’s external environment; and (b) be attributable to the individual, not merely, say, to their visuomotor system. If consciousness were a requirement either for objectivity (e.g., Eilan, 2015) or for individual attributability, Burge’s characterization of perception would rule out unconscious perception. However, Burge rejects both 165

166

•

Ian Phillips and Ned Block

suggestions. For Burge, perceptual objectivity is achieved by exercise of the perceptual constancies (399): “capacities to represent environmental attributes, or environmental particulars, as the same, despite radically different proximal stimulations” (114). Further, a representation’s being attributable to the individual turns, paradigmatically, on whether the representation’s content is available to central, coordinating agency (333). Does a commitment to unconscious perception ineluctably follow? Only if centrally available, constancy-implicating representations can occur unconsciously. Here I argue that the empirical evidence which Burge and many others cite fails to establish this contention. Thus, even granting Burge’s controversial claims about perception, the existence of unconscious perception remains an open question. (For much more on these issues, see Phillips forthcoming; and Phillips in press.) Cases of two types ground much contemporary belief in unconscious perception. First, clinical cases in which perception appears preserved despite loss of consciousness. Second, paradigms in which a stimulus continues to influence responding despite apparently being suppressed from conscious awareness. Consider (type-1) blindsight, the striking phenomenon of preserved visual function (standardly evinced by successful forced-choice responding to a narrow range of stimuli) despite destruction of V1 and in the absence of acknowledged awareness (Weiskrantz 1986/2009, Cowey 2010). According to Burge: “blindsight patients perceive environmental conditions. The perception involves perceptual constancies—including motion, location, and size constancies. The perception guides action. There is strong reason to believe that some of these patients lack phenomenal consciousness in the relevant perceptions” (374). Does blindsight involve the perceptual constancies though? Early work on blindsight neglected this issue. Thus, Weiskrantz (2002: 572) notes “that size constancy, or in fact any of the visual constancies, has never been addressed in any blindsight studies of which I am aware”. More recent work supports a negative answer. For example, Alexander and Cowey (2010) provide evidence that whilst their two patients (MS and GY) retain a capacity to locate and detect stimuli, this capacity is exclusively based on the ability to detect sharp luminance contours and stimulus transients. Neither ability implicates the perceptual constancies, instead being interpretable purely in terms of a sensitivity to proximal stimulation (cf. Burge 2010: 352). In keeping with this, Alexander and Cowey conclude that MS and GY have only the ability to detect “events” varying in “subjective salience” (532). Similarly, Azzopardi and Hock (2011) show that motion detection in GY is limited to detection of “objectless” first-order motion energy (i.e., spatiotemporal changes in luminance) as opposed to detection of changes in position or shape. And Kentridge, Heywood, and Weiskrantz (2007) show that their patient DB matches coloured stimuli purely by wavelength and so lacks even rudimentary colour constancy. Such evidence suggests that the preserved visual functions of blindsight do not constitute perception proper.

Debate on Unconscious Perception

•

167

Another issue raised by many clinical conditions including blindsight, neglect, and prosopagnosia is how we can be sure that a failure to report awareness reflects a genuinely complete absence of awareness. Signal detection theory (SDT) provides a helpful framework (Green and Swets 1966). The core insight of SDT is that responses in a perceptual task are the joint upshot of two factors: discriminative capacity (d′) and response criterion (c). In so-called forced-choice tasks in which subjects must select which of two (e.g., spatial or temporal) intervals a target stimulus is presented in, subjects naturally adopt unbiased criteria, simply picking whichever interval produces the largest sensory response. Such tasks provide a direct guide to perceptual sensitivity. However, in many other tasks, the decision space is not symmetric and subjects often exhibit strong biases toward a particular response. For example, in “yes/no” detection tasks in which subjects are asked whether a stimulus is presented or seen, the threshold which a sensory response must meet to elicit a “yes” may be highly variable, exhibiting either “liberal” or “conservative” bias (see Figure 11.1). Many hard-to-control factors encourage bias in ways which are (very plausibly) independent of awareness. For instance, a subject may exhibit conservative bias because they are naturally under-confident, in a low mood state, or have certain pre-conceptions about their own capacities or the experiment’s purpose. Furthermore, subjects with acquired field defects may have standing problems adjusting their criterion from that formerly appropriate, or across their differentially sensitive fields. Such difficulties will yield conservative bias (Azzopardi and Cowey 2001). The upshot is that a failure to report stimulus presence or awareness in the presence of preserved discriminative capacity cannot be assumed to reflect unconscious perception. Instead, we must always take seriously the possibility that it simply reflects conscious perception combined with conservative response bias. This “problem of the criterion” was used to cast doubt on much early psychological work on unconscious

Figure 11.1 SDT analysis of a simple “yes/no” task. Only sensory responses above the subject’s variable response criterion elicit a positive, “yes” response. Two possible criteria are shown: the first (c1) moderately liberal, the second (c2) highly conservative.

168

•

Ian Phillips and Ned Block

perception (e.g., Eriksen 1960 and Holender 1986). It continues to plague studies of perception in clinical populations as well as many other putative cases of unconscious perception—for example, inattentional blindness (e.g., Mack and Rock 1998; see Dulany 2001) and attentional blink (e.g., Luck, Vogel, and Shapiro 1996). One way of avoiding the problem is to turn to studies in which subjects display no preserved discriminative capacity in respect of some feature (i.e., d′ = 0) and yet that feature continues to exert a perceptual influence. The classic paradigm is masked priming in which masking renders a prime stimulus indiscriminable (d′ = 0), and yet the prime still facilitates subsequent responses to congruent, supraliminal targets. However, such paradigms do not provide unproblematic evidence of unconscious perception. First, establishing that task relevant perceptual sensitivity is completely absent, as opposed merely to very low, is a methodological minefield. Second, the assumption that d′ is a fully adequate measure of phenomenal consciousness is far more controversial than usually supposed (e.g., Schmidt 2007). Third, studies rarely concern themselves with showing that the priming effect implicates objective representation, and so perception proper. Here I highlight a fourth broad concern—namely, whether priming effects suffice to evidence individually attributable representations. Consider an exemplary recent study by Norman, et al. (2014) in which subjects were asked to identify the colour of a mask. A preceding prime matched the mask either in surface colour or reflected colour (i.e., wavelength) but, because of a shift in illumination between prime and mask, not both (Figure 11.2). A c.25ms response advantage accrued under

Figure 11.2 Trial sequence from Norman et al. (2014: 2824). Reproduced with permission. Copyright © 2014 Elsevier Ltd. All rights reserved. (Note that original version is, of course, coloured.)

Debate on Unconscious Perception

•

169

surface-matching conditions indicating constancy-based objective colour representation. Moreover, the effect remained even when subjects could not detect the prime (d′ = 0). This is arguably good evidence of unconscious objective representation. However, the natural understanding of the priming effect provides no reason to think such representations are available to central, coordinating agency. Consequently, the paradigm provides no evidence of individual-level perception. The natural understanding of the effect is that the prime activates objective colour representations in the visual system. This activation results in more fluent processing of subsequently presented congruent colours. As a result, the surface-matched mask is seen more quickly and easily, and so responded to faster. Responses are thus facilitated even though representation of the prime itself is entirely restricted to the visual system and so not individually attributable. This opening discussion barely scratches the surface of decades of empirical work. Its purpose is simply to challenge the common conviction that unconscious perception is an incontrovertible empirical datum. To the contrary, the ambition of establishing the existence of individually attributable, objective representations completely outside of phenomenal awareness is fraught with difficulties. We should not assume that it will be achieved.

Part 2: Block Is There Unconscious Seeing, and Why Care? Seeing is a single fundamental natural kind, of which conscious and unconscious seeing are sub-kinds (Block 2010; Burge 2010). This fact provides difficulties for some of the major theories of perception. For example, naïve realism posits that there are no perceptual representations and that the phenomenology of perception is a matter of direct awareness of things and properties in the world (Travis 2004). But if there is unconscious perception, it must be a matter of perceptual representation that is not available to awareness. And if unconscious seeing is unconsciously representing, there is a strong case that conscious seeing is consciously representing. Introduction There is no well-confirmed theory of the scientific nature of consciousness. Consequently there is no scientific proof of any particular type of state that it is unconscious. And given the lack of any “criterion” for consciousness, any particular experiment will depend on assumptions that themselves have a questionable scientific status. But what holds for unconscious states also applies to conscious states—there is no scientific proof that you are consciously reading these words right now, and this point should put the first one into perspective. With unconscious states as with conscious states we have good reasons for attribution in particular cases—even if they fall short of the status of other scientific truths.

170

•

Ian Phillips and Ned Block

Ian Phillips and I have disagreements about whether there is such a thing as unconscious perception. Instead of focusing on the details of those disagreements, what I will do here is describe my favourite case of unconscious seeing. Continuous Flash Suppression (CFS) To understand what continuous flash suppression (CFS) is, let us start with binocular rivalry, a phenomenon diagrammed in Figure 11.3. “A” shows the bottom of the brain of a subject who is wearing red/green glasses and is viewing

Figure 11.3 This diagram is similar to diagrams in Tong et al. (1998). I am grateful to Frank Tong for this diagram and permission to use it.

Debate on Unconscious Perception

•

171

a red house superimposed on a green face. One eye receives a face stimulus, and the other a house stimulus. The conscious perception—diagrammed in “B”—is not a combined image but rather conscious alternation, with the whole visual field filled first by one image, then the other, then the first again, and so on, every few seconds for as long as the subject is perceiving the stimulus. Subjects report that when they are aware of a face, they are not at all aware of the house (except for occasional brief transitional images). Much of the early visual processing does not change with the changing conscious percept. But when the face fills the whole visual field, the face processing system in the brain (part of which is indicated by the blue dot [farthest left]) is much more active while the areas that process the house (red dots [the two right-most dots]) are suppressed. And the reverse happens when the subject is conscious of the house. The reports suggest that the suppressed representation is not conscious. Nao Tsuchiya discovered that a high-contrast rapidly changing (at 10 Hz) coloured image—known as a “Mondrian”—such as the one on the right in Figure 11.4 could make the perception of the stimulus in the other “suppressed” eye unconscious very reliably for up to 10 seconds and pretty reliably unconscious for several minutes. When this CFS process is working properly, 1. Subjects are at chance in making a choice between alternative pictures that have been projected to the suppressed (non-Mondrian) eye. 2. Subjects give the lowest confidence rating on almost all trials. 3. Subjects often insist they are seeing nothing other than the Mondrian. I have used a low-tech version of this CFS procedure (using red/green

Figure 11.4 Mirror stereoscope setup for continuous flash suppression. I am grateful to David Carmel for supplying this figure and for permission to use it.

172

•

Ian Phillips and Ned Block

glasses) in many classes and talks. People often comment that nothing is being projected other than the Mondrian. 4. There is no difference in confidence between correct and incorrect choices of the input to the suppressed eye (Raio et al. 2012)—that is, confidence does not predict accuracy. And this suggests that even when subjects think they might have consciously seen something, they did not. In addition, there are often differences in kind between processing under CFS and conscious seeing. Raio et al. (2012) compared conscious and unconscious fear conditioning. The stimuli in the suppressed eye was either a male or a female face, one of which was paired with a shock. Fear conditioning was measured by changes in skin conductance in response to the picture paired with the shock. Fear conditioning in the unconscious case ramped up quickly and died off quickly as compared with the conscious case. More interestingly, fear conditioning in conscious—but not unconscious—perception involved suppression of the response to the face that did not predict shock. These facts certainly show unconscious processing, but does that processing involve perception? Mudrik et al. (2011) used a version of CFS (diagrammed in Figure 11.5a) in which the non-Mondrian eye received a picture that slowly ramped up in contrast. The other eye was shown a Mondrian that decreased in contrast when the first picture got to full contrast. The photos included food being put in the oven, a basketball game, and an archery scene (depicted in Figure 11.5c). Each of the photos had an anomalous twin in which, for example, the basketball was replaced by a watermelon, the arrow was replaced by a tennis racket, or the food was replaced by a chessboard. Subjects were asked to press the right or left key as soon as they saw any indication of a scene on the left or the right. What the experimenters were interested in was whether the anomalous photos would break through the “cloak of invisibility” faster. And that was what they found, revealing perceptual integration with context in unconscious perception. As always with purported unconscious perception, we must consider whether CFS allowed fragmentary conscious perception. If so, the appreciation of anomaly might only occur after partial awareness of the stimulus, with faster conscious processing of anomalous pictures being responsible for the result. Mudrik tested this possibility (Figure 11.5b) by integrating the pictures with the Mondrians. In the condition depicted in B, the blended images that were ramped up slowly were presented to both eyes so that the perception of the pictures—both anomalous and ordinary—was conscious. In this condition—in which there was real manufactured partial awareness of the sort postulated by the alternative hypothesis—there was no difference between the anomalous and ordinary pictures, thus disconfirming the alternative hypothesis. Another experiment with similar methodology shows that fearful faces break through the cloak of invisibility faster than neutral or happy faces (Yang,

Debate on Unconscious Perception

•

173

Figure 11.5a–c From Mudrik et al. (2011). I am grateful to Liad Mudrik for this figure.

Zald, and Blake 2007). In another, the subject’s attention was drawn or repelled from unconsciously perceived female or male nudes roughly in accordance with gender preferences (Y. V. Jiang et al. 2006). Thus, unconscious perception can involve high-level perceptual categorization that is relevant to personal-level concerns. Unconscious perception must be both unconscious and perception, but there is a potential conflict between these desiderata. The best evidence for lack of consciousness would be if there was absolutely no effect on the visual system—but that would not be perception. And any effect on vision could be used by opponents to argue that the visual registration was not really unconscious.

174

•

Ian Phillips and Ned Block

Everything depends on the details, and I believe that the details cited show that CFS experiments can thread this needle. I find the CFS form of unconscious perception more convincing than those involved in blindsight or visuo-spatial neglect, syndromes that involve brain damage. Brain damage creates uncertainty about how to understand the response. Another advantage of CFS is that the unconscious perceptions last many seconds, making subjects’ insistence on having no awareness of any stimulus more convincing.

Part 3: Phillips Ned Block’s opening statement describes his “favourite case of unconscious seeing”: continuous flash suppression. Here I argue that CFS confronts the same objections which I raised in relation to subliminal priming. As such, it does not convincingly demonstrate that perception of the same fundamental kind as ordinary conscious perception occurs unconsciously. As with subliminal priming, CFS faces two broad concerns. First, is awareness completely abolished (cf. Yang et al. 2014)? Second, do demonstrable effects establish genuine perception by the individual? Here I focus on this second issue. Block takes the alleged fact that “high-level perceptual categorization that is relevant to personal-level concerns” occurs under CFS to establish individual-level perception. In doing so, Block apparently assumes that whether a representation is individually attributable turns on its content. An alternative view is that it turns, not on its content, but on its role. This is the natural understanding of Burge’s suggestion that individual-level representations are paradigmatically those available to central agency. I appealed to this requirement to argue that evidence of objective representation in subliminal priming fails to establish individual-level perception. Various CFS paradigms merit a similar reply. For example, the acquired skin conductance responses evoked by unconsciously presented faces in Raio et al. (2012) do not demonstrate individual-level perception, since such responses are manifestations of the autonomic nervous system, not of central agency. This case seems to me clear-cut; more often it is a delicate question whether a given response-type constitutes an exercise of central agency. Consider the differential orientating responses made to gendered nudes in Jiang et al. (2006). Do these implicate central agency? Not obviously, if the effect is due to the automatic attraction of saccades (as Prinz 2010: 326 suggests; cf. Burge 2010: 333). Even if the effect is attentional, it is controversial whether all attentional effects involve central agency. Setting individual-attributability aside, is Block right that high-level unconscious perceptual representation occurs under CFS? Block cites Yang et al. (2007) and Mudrik et al. (2011), paradigms in which certain stimuli (fearful faces and anomalous scenes) break free from CFS faster than others (neutral

Debate on Unconscious Perception

•

175

faces and familiar scenes). However, the fact that certain stimuli are consciously perceived faster than others may simply indicate that some stimuli are easier (consciously) to detect than others. This no more demonstrates unconscious perception than does the fact that brightly coloured objects are easier to spot than dully coloured ones. To rule out this simple, differential detectability explanation, recent studies ran a control condition wherein stimuli are presented binocularly and made slowly more visible (see Figure 11.5b for Mudrik et al.’s version). Following Mudrik et al., Block argues that since subjects are equally quick to detect both types of stimuli in this control, the faster breakthrough of one type of stimulus from CFS implies specific high-level unconscious perception under CFS. However, as Stein, Hebart, and Sterzer (2011) (also Stein and Sterzer 2014) forcefully argue in relation to the structurally similar study of Jiang, Costello, and He (2007), the control condition used is inadequate. To see why, notice that, whereas in the CFS condition the time after which a stimulus breaks suppression is highly variable, in the control condition the steady ramp in contrast means that the timing of initial awareness is highly predictable. Since CFS and control conditions are studied in separate trial blocks, this creates a crucial difference in temporal uncertainty between conditions. Stein et al. (2011) show that when CFS and control trials are intermixed within blocks, the stimuli which break suppression faster do exhibit a corresponding detection advantage in control trials. This suggests that faster breakthrough is due to differential detectability. Nothing follows regarding unconscious perception. Jiang et al.’s (2006) paradigm in which differential orientating responses are made following the presentation of female or male nudes under CFS avoids these concerns. However, it does not establish “high-level perceptual categorization” since coarse-grained, low-level features statistically associated with gender could equally mediate the effect. Block (2014) argues that perceptual aftereffects indicate genuine perceptual representation. We can test for gender-specific aftereffects by investigating whether an unambiguously female face presented under CFS biases the classification of a subsequently presented gender-ambiguous face. Recent evidence suggests that perceptual aftereffects from stimuli presented under CFS are absent with respect to gender (Amihai, Deouell, and Bentin 2011) and other high-level features including race (ibid.), holistic face processing (Axelrod and Rees 2014), binocular, higher-level components of face shape (Stein and Sterzer 2011), and (to a very large extent) facial expression (Yang, Hong, and Blake 2010). Pace Block, CFS appears to abolish high-level perceptual categorization. These issues have intrinsic interest. Block suggests a wider significance: unconscious perception threatens naïve realism. Block’s view rests on two contentions: (a) that conscious and unconscious seeing are of the same fundamental kind, and (b) that unconscious seeing “must be a matter of perceptual representation”. Naïve realists will likely reject (a). Must they accept (b)? Naive realism’s core tenet is that perceptual episodes involve (non-representational) relations

176

•

Ian Phillips and Ned Block

to mind-independent objects, and so have such objects as constituents (Martin 2006). Could a hypothetical naïve realist think of unconscious perception as involving just such relations? It might be objected that such relations are intended to explain phenomenal character. Yet consider two imperfect analogies. Suppose a good life constitutively involves personal relationships, relationships which partly explain why that life is good. It does not follow that personal relationships of the same kind cannot occur within a bad life. Suppose (with Russellians) that true propositions have mind-independent objects amongst their constituents, objects whose identities partly explain such propositions’ truth-values. It does not follow that false propositions lack mind-independent objects as constituents. The naïve realist does face trouble if she insists that sameness and difference in phenomenal character exclusively turns on sameness and difference in perceptual objects. However, naïve realists standardly reject this claim (e.g., Campbell 2009, Brewer 2011). Questions concerning unconscious perception may nonetheless have wider import. Belief in unconscious perception succours scepticism about the significance of consciousness. If perception is essentially conscious, consciousness may partly be important because seeing is.

Part 4: Block Phillips is right that the controls in the “breaking-Continuous Flash Suppression” (b-CFS) study I cited were inadequate. I had not read the papers by Timo Stein and Phillip Sterzer and their colleagues that he cites. In b-CFS, differences in breaking through the CFS “cloak of invisibility” are compared to differences in detection of “comparable” visible (i.e., without CFS) stimuli. Some stimuli may be easier to detect than others because of salient low-level features. Salient low-level features—rather than high-level features—could explain breaking CFS, and that possibility must be ruled out by controls. In addition, salient low-level features could trigger a non-perceptual response bias or tendency to respond faster to one of the options. However, this control issue probably does not apply to the study I cited, Mudrik et al. (2011). They showed that the anomalous pictures do not differ from the non-anomalous pictures in standard measures of low-level saliency, and they independently controlled chromaticity and spatial frequency, thus making low-level confounds unlikely. Further, not even Stein and Sterzer think that a response bias is a real option in this case: It is important to note that the possible impact of such non-perceptual factors may be limited to the comparison of upright and inverted faces and does not necessarily apply to other b-CFS studies using different stimuli. For example, it is difficult to imagine that observers would have different response criteria for images of complex scenes that differed only in their semantic content (Mudrik et al. 2011). (Stein et al. 2011, 7)

Debate on Unconscious Perception

•

177

Thus, if it is easier to spot a visible picture of someone shaving with a fork rather than with a razor, that can be explained by the same unconscious perception of anomaly underlying both conscious detection and breaking CFS. Summarizing many studies, Phillips says, “Pace Block, CFS appears to abolish high-level perceptual categorization.” On the contrary, there is strong evidence for weak high-level perceptual categorization, including in one of the studies he cites (Yang et al. 2010) so long as there is spatial attention to the location of the stimulus. This point is emphasized in Stein, Thoma, and Sterzer (2015). And in a review of neuroimaging data, Stein and Sterzer (2014) conclude: In summary, neuroimaging studies investigating the processing of visual information during interocular suppression have shown repeatedly that object- or category-specific neural activity in high-level visual areas of the ventral stream is strongly reduced, but can be retrieved when sufficiently sensitive methods of data analysis are used, such as multi-voxel pattern analysis of fMRI data. In sum, there is substantial evidence for high-level unconscious perception. As Phillips notes, the Jiang et al. (2006) CFS study does not use breaking CFS and so is immune to his criticisms of the b-CFS paradigm. In the Jiang et al. (2006) study, the subject sees a fixation point; then each eye gets a pair of stimuli separated by a fixation point. One eye gets a pair of Mondrians while the other eye receives a nude (male or female) on one side with a texture of fragments of nudes on the other side. In Figure 11.6, the top and bottom in the second panel indicate what is presented to each eye. (Do not make the mistake of supposing that the subject sees a nude with one eye and nude texture with the other eye.) The pair of Mondrians separated by the plus sign presented to one eye suppresses conscious perception of both the nude and nude texture in the other eye. Then if the subject’s attention is attracted or repelled by the unconscious perception of the nude, that is due to unconscious perception of the nude. The attraction or repulsion to one side or

Figure 11.6 From Jiang et al. (2006). Reproduced with permission of Proceedings of the National Academy of Sciences.

178

•

Ian Phillips and Ned Block

the other is measured by a very brief presentation of a stripy noise patch that can be slightly tilted one way or the other, and the subjects have to say which. Attention is known to increase accuracy in this judgment so the direction of attention can be assessed by measuring the accuracy in the judgment of tilt. Subjects are asked whether they saw any difference between the right and the left. If they report any difference, their data are excluded—on the assumption that some aspect of the nude must have leaked into consciousness since only the nude/texture provides any asymmetry in the stimuli. The conclusion of the experiment is that subjects’ attention is attracted or repelled in a way that conforms to their gender preferences. There have been some reports that high-level unconscious perception under CFS may involve low-level conscious perception, for example of a cloud of colour (Hong and Blake 2009; Zadbood, Lee, and Blake 2011; Mudrik et al. 2013; Gelbard-Sagiv et al. 2016). But the control in this study provides evidence that whatever low-level conscious perception there might be does not differentiate a nude from a nude texture, so the gender of the stimulus is perceived unconsciously. Phillips says unconscious perception of low-level features associated with gender might explain the result. But even if that is right, the point of appeal to the high level is to justify the conclusion that the perception is by the individual rather than a reaction by a subsystem. And that is accomplished here by the relevance to personal-level gender preferences whether the perception is high or low. Some of these effects are diagrammed in Figure 11.7 where the top graph shows bars of attraction (pointing up) and repulsion (down) for 10 heterosexual males (top) and 10 heterosexual females (bottom). Homosexual males resembled heterosexual females. As you can see, attention in heterosexual males was usually repelled by nude men, but heterosexual females and homosexual males tended to be positive or neutral about both nudes. The upshot is that whether high level or not, this is personal-level unconscious perception. Phillips also says that the Jiang effect may not “implicate central agency” if it is due to “automatic” eye movements, as allegedly alleged by Prinz. I can’t imagine how attraction or repulsion keyed to one’s gender preferences could be automatic in any relevant sense of the term. Prinz claims that the unconsciously perceived nudes attract or repel eye movements rather than attention, but that is not to deny unconscious perception. Contrary to Phillips, Burge and I both deny that involvement in central agency is a necessary condition of unconscious perception. The “paradigmatic” quoted by Phillips is a generic, not a universal (Burge 2010, 370). In any case the attraction and repulsion of attention is an individual-level matter and is not unrelated to agency. In conclusion, though Phillips is right that the controls in the b-CFS studies were inadequate, the overall upshot of the studies described is that there is substantial evidence of unconscious perception.

Debate on Unconscious Perception

•

179

Same gender

0.4

Opposite gender

Attentional effect

0.2

0.0

-0.2

-0.4 1

2

3

4

5

6

7

8

9

10

Average

7

8

9

10

Average

Same gender

0.4

Opposite gender

Attentional effect

0.2

0.0

-0.2

-0.4 1

2

3

4

5

6

Figure 11.7 From Jiang et al. (2006). Reproduced with permission of Proceedings of the National Academy of Sciences. Each pair of black and grey bars represents attraction and repulsion in one individual except the last pair of bars. This pair represents the averages.

Part 5: Phillips I previously argued that CFS paradigms fail to demonstrate genuine unconscious perception by the individual. Here I reply to Block’s objections before focusing on what I take to be our more fundamental disagreement. In itself, the differential breakthrough of stimuli from CFS does not establish unconscious perception since it may simply reflect differing conscious

180

•

Ian Phillips and Ned Block

detection thresholds. Block accepts Stein et al.’s (2011) critique of extant control conditions designed to rule this out. Nonetheless, he suggests that such concerns probably do not apply to Mudrik et al. (2011). In support, Block quotes Stein et al. expressing scepticism that subjects would adopt differing response criteria in relation to Mudrik et al.’s stimuli. In my earlier reply, I did not mention response criteria partly for this reason and partly because Stein et al. provide evidence that the differential breakthrough of upright versus inverted faces in Jiang et al. (2007) also “cannot be ascribed simply to the influence of differential response criteria”. Instead, they suggest that faster breakthrough results from “a lower detection threshold for upright faces”—crucially one not specific to CFS. Block objects to a differential detectability explanation regarding Mudrik et al.’s stimuli on the grounds that they were matched in respect of various low-level features. However, stimuli matched in the relevant ways may still differ in conscious detectability in a non-CFS specific manner. That is all the objection requires, and why Stein et al., whilst well aware of Mudrik et al.’s results, can reasonably claim that their criticisms are “relevant for and extend to all applications of the b-CFS paradigm” (2011: 4; cf. Stein and Sterzer 2014; and Gayet, Van Der Stigchel, and Paffen 2014). Certainly, Mudrik et al.’s findings may be due to unconscious perception of anomaly. Yet given what else we know about flash suppression, and can reasonably extrapolate from studies of binocular rivalry (e.g., Zimba and Blake 1983; see Breitmeyer 2014 for a review), our “default stance should . . . be not to expect much high-level unconscious processing during CFS” (Hesselmann and Moors 2015: 3). Block contests my assessment of the relevant literature, finding in it “substantial evidence for high-level unconscious perception” under CFS. Block is right that Yang et al. (2010) only provide evidence that facial expression “is virtually abolished” (as the authors put it) or abolished “to a very large extent” (as I wrote). However, we should be cautious in relying on Yang et al. as positive evidence of weak high-level categorization outside of consciousness. To ensure unawareness, Yang et al. instructed subjects immediately to press a key if “they perceived anything other than the CFS display”. This terminated the trial. If observers pressed the key on more than 15% of trials, they were excluded from analysis. Yang et al. report that 70% of observers completed the session and that these all “later confirmed that they had not perceived any faces during the CFS period upon questioning” (3–4). This methodology is doubly problematic. First, both key pressing and post-session questioning are subject to obvious response biases and so may easily underestimate true awareness. Second, the practice of post-hoc discarding trials/subjects who show awareness introduces a notorious statistical artifact which Newell and Shanks argue “renders the apparent evidence of unconscious processing almost meaningless” (2014: 50, commenting on Sklar et al. 2012; Carmel 2014 provides a clear exposition of the issue).

Debate on Unconscious Perception

•

181

Concerns with ensuring unawareness aside, imaging data raise a further issue—namely, that differential cortical activation does not guarantee the presence of representations which can influence task performance (Williams, Dang, and Kanwisher 2007). Here, I suggest, we arrive at my more fundamental disagreement with Block. In my opening statement I cited Norman et al. (2014) as providing evidence of genuinely perceptual (constancy-involving) representation outside of consciousness. What I disputed was whether this constituted perception by the individual. Whether high-level feature representation occurs outside of consciousness is a separate issue. I thus agree with Block that whether Jiang et al.’s (2006) results reveal individual-level perception does not turn on whether the representations mediating their effect are high or low level. What matters is whether those representations are constancy-involving and individual-level. Here I press this second issue. Let us grant that Jiang et al.’s results show unconscious attraction and repulsion of attention. Block’s case that the mediating representations constitute perception by the individual “is accomplished . . . by [their] relevance to personal-level gender preferences”. He adds: “I can’t imagine how attraction or repulsion keyed to one’s gender preferences could be automatic in any relevant sense of the term”. However, personal-level gender preferences correlate closely with many reflexive, autonomic responses. For example, Rieger and Savin-Williams (2012) examine the differential pupillary responses elicited by gendered erotic stimuli. Such responses are naturally thought of as automatic. Furthermore, it is doubtful that we must think of the representations mediating them as individual-level. Perhaps Block’s talk of responses being “keyed” to preferences requires the direct involvement of preferences in mediating responses from occasion to occasion. However, gender preference data cannot evidence responses “keyed” in this sense, since gender preferences cannot be manipulated on a trial by trial basis. Block denies that “involvement in central agency” is a necessary condition of personal-level attribution. I did not propose such involvement as a necessary condition. My suggestion was only that when a representation is unavailable to central agency, we lack a positive ground for attribution. Nonetheless, Block and I agree that certain kinds of response may indicate personal-level attribution. However, Block claims that “attraction and repulsion of attention is an individual-level matter and is not unrelated to agency” (cf. Burge 2010: 372). On this we disagree. Where attentional responses are completely stimulus-driven reflexes, operating entirely outside of voluntary control (e.g., Schoeberl et al. 2015), and possibly mediated by subcortical pathways (e.g., Mulckhuyse and Theeuwes 2010), I am unpersuaded that we must think of them as exercises of individual-level agency. If they are not, we lack positive reason for thinking of the perceptual representations implicated by Jiang et al.’s data as constituting individual-level perception.

182

•

Ian Phillips and Ned Block

Part 6: Block We agree that there are unconscious representations in CFS but disagree on whether they are personal or sub-personal. I have been arguing that they reflect personal-level values and understanding. Values: Phillips says attraction and repulsion in Jiang et al. (2006) may be involuntary stimulus-driven reflexes that are subcortically mediated. However, the only actual evidence he presents for this is that pupillary responses are affected by gendered erotic stimuli. He claims, “Such responses are naturally thought of as automatic”. This is outdated. A recent review on this topic says: “The pupillary light response has long been considered an elementary reflex. However, evidence now shows that it integrates information from such complex phenomena as attention, contextual processing, and imagery” (Binda and Murray 2015, 1). This is a review, not an opinion by a fellow-traveler. Further, though exogenous spatial attention such as orienting to a loud noise is stimulus-driven and reflex-like, I know of no evidence for any reflex-like feature-based attention. Moving from values to personal-level understanding—I quoted a review that concludes (p. 8): “In summary, neuroimaging studies investigating the processing of visual information during interocular suppression have shown repeatedly” weak high level activations (Sterzer et al. 2014). Thus, it is very likely that there is unconscious high-level representation. I think whether unconscious representation constitutes unconscious perception turns on both content and role. On content: low-level properties like edge and texture register in early vision but are not normally part of personal-level cognition unlike our awareness of faces and emotions—hence the focus on high-level activation. On role: in the article just cited, Sterzer et al. note that although there have been many studies showing behavioral effects of CFS, no studies as yet have measured behavioral effects simultaneously with neuroimaging evidence of high-level perception. However, I know of no case of a high-level brain activation that does not have the potential to affect some kind of processing, if only on the temporal course of the processing (for example in priming). Phillips appeals to a claim by Hesselmann and Moors (2015) based on work by Randolph Blake and Bruno Breitmeyer on binocular rivalry that the default should “be not to expect much high-level unconscious processing during CFS” (emphasis added). This appeal is doubly flawed. First, the strong evidence for high-level CFS activations I referred to earlier is evidence for weak activations—as are the pupillary effects. If the default is to expect not “much” activation, that actually supports my position. Second, Phillips’ claim depends on a dubious inference from binocular rivalry to CFS. He says, “Yet given what else we know about flash suppression, and can reasonably extrapolate from studies of binocular rivalry (e.g. Zimba and Blake 1983; see Breitmeyer 2014 for a review) . . .”. However, Breitmeyer

Debate on Unconscious Perception

•

183

(2015) argues that binocular rivalry blocks off processing at the earliest stages of vision, whereas CFS operates at a mid-level. Even in the earlier (2014) Breitmeyer article, Breitmeyer places CFS above binocular rivalry in his hierarchy. If a stimulus (say a disk) is followed quickly by another stimulus (say a ring) which shares boundaries with the first stimulus (e.g., the disk sits just inside the ring), then conscious perception of the first stimulus can be reduced or eliminated. This “metacontrast masking” is strongest with the stimulus in one eye and the mask in the other especially when the stimulus and mask are presented nearly simultaneously, suggesting a combination of binocular suppression and metacontrast masking (Schiller and Smith 1968). CFS flickers at 10 hz suggesting that it also combines metacontrast masking with binocular suppression and that combined effect puts it higher on the hierarchy. A further item of evidence: Sklar et al. (2012) showed unconscious “semantic” priming in CFS. Sklar et al. presented three-digit subtraction problems to subjects under CFS (e.g., 9–3–4). Subjects then had to pronounce a single consciously presented digit that could be the result (e.g., 2). Results were faster than non-results. Subjects were asked to report the parity of the first digit in the subtraction problem, and those who got it right were excluded. And in a debriefing afterward, they excluded four subjects who said they had seen the primes. Phillips says excluding subjects who report more than the lowest visibility introduces a “notorious statistical artefact” (Shanks and Berry 2012). I don’t think the conditions for this artifact are met, but I don’t have the space for a discussion. There are four good reasons for thinking the effect was unconscious. First, Sklar et al. used Anthony Greenwald’s respected regression method that is designed to be used with a variety of visibilities (Greenwald, Klinger, and Schuh 1995). This method allows an extrapolation from higher visibilities to zero visibility. (See Kouider and Dehaene 2007, for further explanation.) Greenwald’s method showed a significant unconscious effect. Second, performance on the objective test was negatively correlated with the unconscious effect, suggesting that the effect is unrelated to conscious perception. Third, as Ran Hassin and Asael Sklar have emphasized in correspondence, the effect size for conscious priming is the same or at most twice the size of an unconscious effect. So in order for the effect in this study to be due to conscious perception, more than half the subjects would have had to be conscious of the stimuli—even after the elimination of all who scored above chance on the objective task. Fourth, the priming worked for subtraction but not addition. If subjects were indeed conscious of the stimuli, they should have been just as conscious of the addition as subtraction stimuli. (They speculate as to what the difference in unconscious processing of addition and subtraction and devise a procedure that shows effects for addition.) In sum, though there are plenty of loose ends in a rapidly moving field, there is a good case for personal-level unconscious representation.

184

•

Ian Phillips and Ned Block

Part 7: Phillips I have been arguing that the existence of unconscious perception (construed as objective sensory representation by the individual) remains an open question. Where Block sees a few loose ends, I see unravelling tangled threads. Here I pull further on certain threads before offering some brief closing remarks. Block argues that unconscious representations in CFS are individual-level since they reflect personal-level values and understanding. I cannot see how Jiang et al.’s data establish Block’s values claim (nor related claims about role). Block grants that “exogenous spatial attention” may be “stimulus-driven and reflex-like” but denies that the “feature-based attention” involved in Jiang et al. could be. But what does “feature-based” mean here? In Jiang et al., attention is not directed to gender as a feature but by gendered erotic stimuli to a spatial location. It is well-attested that fearful emotional stimuli can differentially draw reflexive spatial attention (e.g., Phelps, Ling, and Carrasco 2006, experiment 2). And an increasing body of work, including Jiang et al., indicates that “biological relevance, and not exclusively fear, produces an automatic spatial orienting toward the location of a stimulus” (Brosch et al. 2008: 362). Block draws attention to Breitmeyer (2015). What he does not draw attention to is that Breitmeyer there supports my suggestion that Jiang et al.’s effect may be subcortically mediated. Breitmeyer writes: “a suppressed erotically charged image presented in the left visual hemifield could, via retino-subcortical routes, activate the contralateral (right) pulvinar/amygdala, which, in turn, would activate their ipsilateral neocortex and thus bias attentive processing of stimuli in the left visual field” (243, fn. 6). Block is right that Breitmeyer does not think that CFS operates at the same level as binocular rivalry (BR). This does not mean that there is nothing that we can “reasonably extrapolate” from BR. After all, Breitmeyer holds that CFS partly relies on BR suppressive mechanisms. Moreover, whereas Block wishes to place CFS above metacontrast masking in the functional hierarchy, Breitmeyer places CFS “relatively low . . . in the functional hierarchy, somewhere between binocular-rivalry suppression and suppression by backward pattern or metacontrast masking” (2015: 243, fn. 7, my emphasis; cf. 2014: Fig. 5.4). Breitmeyer justifies his (avowedly speculative and tentative) placement of CFS above BR by appeal to Sklar et al. (2012), which Block also focuses on. This striking study reports the priming of responses to targets (e.g., 2) by equations with those targets as answers (e.g., 9–3–4 =). My earlier complaint that perceptual priming cannot directly reveal individual-level representation applies here. But this point aside, does Sklar et al. provide good evidence of sophisticated unconscious processing under CFS? As mentioned, a major issue here is the statistical artefact potentially introduced by the post hoc exclusion of subjects who performed above chance on either objective or subjective measures of awareness. Block denies that the conditions for this artefact

Debate on Unconscious Perception

•

185

are met (he does not say why), and finds it implausible that at least half of non-excluded subjects could have been conscious of the prime (something he suggests would be necessary to explain the relevant effect-size). I disagree. Sklar et al. excluded 60% of subjects. This suggests that the significant majority of their original group may have been conscious of the primes. This surely does raise serious concerns about truncation artefacts. It also appears consistent with half of the remaining subjects having some minimal awareness of the primes (cf. Hesselmann et al. 2015: §4.2). Block offers three further reasons for thinking that Sklar et al.’s effect was unconscious. First, their use of Greenwald’s respected regression method. However, Greenwald’s method is highly controversial given the large assumptions it requires, and great care is needed in its application (Dosher 1998, Merikle and Reingold 1998). Lacking space for a full discussion, let me note one salient point from Dosher which connects to Block’s second argument in favour of unconscious perception—namely, that performance on the objective task was negatively correlated with facilitation effects. This negative correlation indicates a non-linear relationship between direct and indirect measures. However, given such a relationship, facilitation may reduce to zero with or before the direct measure (indicating no unconscious perception), and yet the best-fitted linear regression misleadingly yield precisely the kind of non-zero intercept which Sklar et al. report as evidence of unconscious perception. At a minimum then, more sophisticated analysis is required for this method to be probative. Block finally argues that, in the relevant experiment, priming occurred for subtraction but not addition. But why think that this supports thinking of the effect as unconscious? Conscious or unconscious, the absence of an addition effect needs explaining. Sklar et al. suggest “that participants may have been less strategic in the [easier] addition equations”, providing evidence for this in relation to conscious arithmetic (Experiment 8). Thus, a strategic explanation is demonstratively available in relation to conscious perception. Where does this leave us? Throughout our exchange, Block has proposed various CFS studies as persuasive evidence for unconscious perception (e.g., Jiang et al. 2006; Mudrik et al. 2011; Sklar et al. 2012). However, as Block says, “everything depends on the details”. And, upon scrutiny, the proffered interpretation of these—and structurally similar—studies unravels. Arguably, breaking flash suppression studies only reveal differences in conscious detectability; attentional paradigms only reveal sub-individual-level perceptual representation; and the widespread practice of truncating data leads to the artefactual appearance of unconscious perception where none exists. Such unravelling is not unique to CFS. As briefly discussed in my opening remarks, neither traditional perceptual priming studies nor clinical conditions such as blindsight and neglect convincingly establish unconscious perception (see further Phillips forthcoming; and Phillips in press). Furthermore, whilst it has been convenient here to adopt a broadly Burgean conception of perception,

186

•

Ian Phillips and Ned Block

that conception is hardly beyond dispute (e.g., Campbell 2011). And obviously answers to questions about unconscious perception turn crucially on our conception of perception. The upshot is that for all the resurgent field’s excitement about new techniques and findings, the current consensus in favour of unconscious perception remains significantly grounded in faith as opposed to fact.

Part 8: Block Anna Karenina I endorse the “Anna Karenina” view of unconscious perception (Block 2011) according to which all conscious perceptions are alike, but each unconscious perception is unconscious in its own way. Successful conscious perception is a dance of oscillating feed-forward-and-back loops. Unsurprisingly, there are many substantially different methods of producing unconscious perception that interfere with the dance in different ways. Breitmeyer (2015) describes 24 substantially different ways in which unconscious visual processing can be produced, of which we have here discussed only a few. Given this variety of mechanisms, it is not surprising that Phillips’ criticisms of experimental paradigms have no real unity (other than the allegation of not-perception or not-unconscious). Here is a list of some of the experimental paradigms he discusses with a shorthand description of his criticisms: • Blindsight: failure of constancies and decision-theoretic criterion issues. • Unconscious colour registration: representations are not available to central agency. • Breaking CFS (continuous flash suppression): control trials did not rule out CFS-specific effects, Breitmeyer hierarchy suggests high-level effects should not occur, post-hoc discarding of aware trials illegitimate, cortical activations may be epiphenomenal. • Gender-CFS: low-level confounds, reflexes. Personal Level Here are some areas of agreement between Phillips and me. First, we agree that there are unconscious representations that are involved in perception. (I say those representations often constitute perception, and he says not.) Second we agree that there are unconscious representations in perception that are objective. Phillips notes he “cited Norman et al. 2014 as providing evidence of genuinely perceptual (constancy-involving) representation outside of consciousness”. Phillips first complains that these colour representations are nonetheless sub-personal because they “are not available to central coordinating agency”. Burge and I think central availability is not necessary for

Debate on Unconscious Perception

•

187

unconscious perception, and Phillips agrees, saying that the real point is that without such central availability we have no positive reason for ascribing the personal level. But when a sensory registration reflects personal-level understanding (Mudrik et al. 2011) or values (Jiang et al. 2006), that is a reason to think it is a personal-level perception. Value Phillips objects to Jiang et al. (2006) by alleging that it can be explained by unconscious perception of low-level features associated with gender. However, the connection to personal-level gender preferences is what is at issue, not high versus low level. Phillips also claims that the Jiang effect might be reflexive and subcortical, appealing to supposed reflexive effects of gender on pupil size. But as I noted, pupillary effects often reflect high-level processes. He also references Prinz’s view that the result is due to attraction of eye movements. But Prinz (2012) argues that unconscious recognition of the stimulus is what attracts the eye movement (p. 116). One caution about Jiang et al. (2006): this result is the only one I know of in which personal-level preferences are so strongly revealed in unconscious perception. Understanding I highlighted CFS as a method of producing unconscious perception because the episodes of unconscious perception last seconds (or even minutes) instead of milliseconds, and it can be experienced firsthand by anyone with a computer and a 10-cent pair of red/green glasses. When I started, I was ignorant of two issues concerning CFS, though I don’t see either of them as problems for the studies I cited (and neither apply to the Jiang et al. [2006] study just mentioned). One of the problems—the one emphasized by Phillips—seems to me a red herring. Many of the CFS experiments compared effects under CFS with comparable tasks without CFS to show there were no “CFS-specific” effects. This way of conceiving of the controls is a mistake. All should agree that unconscious processing underlies all conscious perception—though Phillips and I disagree about whether those underlying unconscious processes themselves constitute perception. So it would not be surprising if—without CFS—subjects recognize shaving with a fork faster than shaving with a razor. The real point of the controls should be to rule out low-level confounds and decision effects. On low-level confounds: Mudrik et al. use two batteries of measures to equate for low-level features. This gives their study evidential weight. A second problem with CFS is that as I mentioned there are brief periods of partial awareness—of low-level properties such as colour (Mudrik et al. 2013). However, since anomaly is not such a low-level property, this is not a substantial problem. Note that rationale given for the controls is to avoid CFS-specific effects on behavior. However, if we move from a behavioral

188

•

Ian Phillips and Ned Block

experiment to brain imaging, there is no need for such controls. I quoted Sterzer et al. (2014) on unconscious representation in CFS: In summary, neuroimaging studies investigating the processing of visual information during interocular suppression have shown repeatedly that object- or category-specific neural activity in high-level visual areas of the ventral stream is strongly reduced, but can be retrieved when sufficiently sensitive methods of data analysis are used, such as multi-voxel pattern analysis of fMRI data. Note that this comes from a review by the team that Phillips relies on and does not use the methodology that he objects to. These studies don’t test behavioral effects of these activations in the same experiments, but any neural activation can affect the temporal course of responding (“priming”) in an appropriately chosen task. Philosophers may be thinking of the colour of wires in a computer that do not affect its operation, but this kind of causal isolation does not happen in the brain. In sum, there is strong evidence that unconscious sensory registration often reflects person-level values and understanding. I turn now to a different paradigm. Sandwich-Masking In Draine and Greenwald (1998), subjects were presented with a “sandwichmasked prime”, in this case, a word is preceded and succeeded by “masks”, noisy stimuli known to make the sandwiched item harder to see. Immediately after that they were given a speeded task: classify a word presented without masks—the “target”—as pleasant or unpleasant. Immediately after that they had to decide whether the prime was a word or a series of Xs and Gs. (This tests how visible the prime was.) Both the primes and the targets were chosen from negative words like “vomit”, “kill”, and “bomb”, and positive words like “honor”, “happy” and “kiss”. The result was that if the prime and target were in the same evaluative category, subjects were faster in classifying the target and made fewer errors. Values for unconscious perception were obtained by Greenwald’s regression technique (mentioned in my last segment) in which responses under various levels of visibility are extrapolated back to zero visibility. The classifications of the primes and targets in this study engage both personal-level values and cognition. The experiment just described was criticized by many, including the authors, because the same words were repeated as primes and targets, and it was found that even single consonants from the repeated words worked as primes—suggesting that the result was due to associations and that unconscious understanding of the evaluative was not required. Using accumulated wisdom of many years of inquiry using this sandwich-masked priming

Debate on Unconscious Perception

•

189

technique, Klauer et al. (2007) give a new and convincing version of the experiment showing unconscious priming of novel evaluative and gender-related stimuli similar to the ones just described. The degree of priming was the same regardless of visibility of the prime, strongly suggesting that the effect does not depend on conscious perception. And they got the same results even when the visible targets were smiley and grumpy faces and the primes were the evaluative words, again suggesting that unconscious evaluative categorization was involved. A similar congruency priming experiment was used with stimuli like the anomalous pictures illustrated earlier—for example, a person drinking from a football rather than from a bottle (Mudrik and Koch 2013). The primes were low in contrast, presented briefly (33 ms) and sandwich-masked. (In the Mudrik 2011 CFS experiment described earlier, similar pictures were presented at full contrast for 2.5 seconds.) Subjects were shown a sandwich-masked prime that could be anomalous or not, then a consciously presented target that could also be anomalous or not. Subjects had to press a button indicating whether the target was “weird”, then rate the prime visibility, then whether the prime was “weird”, being instructed to guess if they did not know. Results were reported only for subjects whose rating was “saw nothing”. (I don’t have the space to explain why this procedure is legitimate.) The result was that subjects were slower to judge that a consciously presented picture was congruent (i.e., not “weird”) if it was preceded by an incongruent prime than if it was preceded by a congruent prime. The authors suggest that the unconscious processing of an incongruent prime may have attracted attention, depriving the subsequent task of attention. Again we have unconscious sensory registration that engages personal-level cognition. In conclusion, there are many experimental paradigms that support personal-level unconscious visual perception. Criticisms form an ad hoc list.

References Alexander, I., & Cowey, A. (2010). Edges, colour and awareness in blindsight. Consciousness and Cognition, 19, 520–533. Amihai, I., Deouell, L., & Bentin, S. (2011). Conscious awareness is necessary for processing race and gender information from faces. Consciousness & Cognition, 20, 269–279. Axelrod, V., & Rees, G. (2014). Conscious awareness is required for holistic face processing. Consciousness & Cognition, 27, 233–245. Azzopardi, P., & Cowey, A. (2001). Why is blindsight blind? In B. De Gelder, E. De Haan, & C. Heywood (Eds.), Out of Mind: Varieties of Unconscious Processes, 3–19. Oxford: Oxford University Press. Azzopardi, P., & Hock, H. (2011). Illusory motion perception in blindsight. Proceedings of the National Academy of Sciences of the United States of America, 108, 876–881. Binda, P., & Murray, S. (2015). Keeping a large-pupilled eye on high-level visual processing. Trends in Cognitve Sciences, 19(1), 1–3. Block, N. (2010). Attention and mental paint. Philosophical Issues: A Supplement to Nous, 20, 23–63.

190

•

Ian Phillips and Ned Block

Block, N. (2011). The Anna Karenina theory of the unconscious. Neuropsychoanalysis, 13(1), 34–37. Block, N. (2012). The grain of vision and the grain of attention. Thought, 1(3), 170–184. Block, N. (2014). Seeing-as in the light of vision science. Philosophy and Phenomenological Research, 89(3), 560–573. Breitmeyer, B. (2014). Functional hierarchy of unconscious object processing. In B. Breitmeyer (Ed.), The Visual (un)Conscious and Its (dis)Contents, 89–102. Oxford: Oxford University Press. Breitmeyer, B. (2015). Psychophysical “blinding” methods reveal a functional hierarchy of unconscious visual processing. Consciousness and Cognition. doi: http://dx.doi.org/10.1016/j. concog.2015.01.012 Brewer, B. (2011). Perception and Its Objects. Oxford: Oxford University Press. Brosch, T., Sander, D., Pourtois, G., & Scherer, K. (2008). Beyond fear rapid spatial orienting toward positive emotional stimuli. Psychological Science, 19(4), 362–370. Burge, T. (2010). Origins of Objectivity. Oxford: Oxford University Press. Campbell, J. (2009). Consciousness and reference. In B. McLaughlin & A. Beckermann (Eds.), The Oxford Handbook of Philosophy of Mind, 648–662. Oxford: Oxford University Press. Campbell, J. (2011). Tyler Burge: Origins of objectivity. The Journal of Philosophy, 108(5), 269– 285. Carmel, D. (2014). Some things we need to think about when investigating unconscious perception. Paper presented at the ASSC18. https://www.youtube.com/watch?v=UoCZ-pg10mo Cowey, A. (2010). The blindsight saga. Experimental Brain Research, 200, 3–23. Dosher, B. (1998). The response-window regression method—Some problematic assumptions: Comment on Draine and Greenwald (1998). Journal of Experimental Psychology: General, 127(3), 311–317. Draine, S. C., & Greenwald, A. G. (1998). Replicable unconscious semantic priming. Journal of Experimental Psychology: General, 127(3), 286–303. Dulany, D. (2001). Inattentional awareness. Psyche: An Interdisciplinary Journal of Research on Consciousness, 7(5). http://www.theassc.org/files/assc/2499.pdf. Eilan, N. (2015). Perceptual objectivity and consciousness: A relational response to Burge’s challenge. Topoi, 1–12. doi: 10.1007/s11245–015–9325–4 Eriksen, C. (1960). Discrimination and learning without awareness: A methodological survey and evaluation. Psychological Review, 67, 279–300. Gayet, S., Van Der Stigchel, S., & Paffen, C. L. (2014). Breaking continuous flash suppression: Competing for consciousness on the pre-semantic battlefield. Frontiers in Psychology, 5, 460. Gelbard-Sagiv, H., Faivre, N., Mudrik, L., & Koch, C. (2016). Low-level awareness accompanies ‘‘unconscious’’ high-level processing during continuous flash suppression. Journal of Vision, 16(1:3), 1–16. Green, D. M., & Swets, J. A. (1966). Signal Detection Theory and Psychophysics. Oxford: John Wiley. Greenwald, A., Klinger, M., & Schuh, E. (1995). Activation by marginally perceptible (“subliminal”) stimuli: Dissociation of unconscious from conscious cognition. Journal of Experimental Psychology: General, 124, 22–42. Hesselmann, G., Darcy, N., Sterzer, P., & Knops, A. (2015). Exploring the boundary conditions of unconscious numerical priming effects with continuous flash suppression. Consciousness & Cognition, 31, 60–72. Hesselmann, G., & Moors, P. (2015). Definitely maybe: Can unconscious processes perform the same functions as conscious processes? Frontiers in Psychology, 6(584), 1–5. Holender, D. (1986). Semantic activation without conscious identification in dichotic listening, parafoveal vision, and visual masking: A survey and appraisal. Behavioral and Brain Sciences, 9, 1–66. Hong, S., & Blake, R. (2009). Interocular suppression differentially affects achromatic and chromatic mechanisms. Attention, Perception & Psychophysics, 71(2), 403–411. Jiang, Y., Costello, P., & He, S. (2007). Processing of invisible stimuli: Advantage of upright faces and recognizable words in overcoming interocular suppression. Psychological Science, 18, 349–355.

Debate on Unconscious Perception

•

191

Jiang, Y. V., Costello, P., Fang, F., Huang, M., & He, S. (2006). A gender- and sexual orientationdependent spatial attentional effect of invisible images. Proceedings of the National Academy of Sciences, 103(45), 17048–17052. Kentridge, R., Heywood, C., & Weiskrantz, L. (2007). Colour-contrast processing in human striate cortex. Proceedings of the National Academy of Sciences of the United States of America, 104, 15129–15139. Klauer, K., Eder, A., Greenwald, A., & Abrams, R. (2007). Priming of semantic classifications by novel subliminal prime words. Consciousness and Cognition, 16, 63–83. Kouider, S., & Dehaene, S. (2007). Levels of processing during non-conscious perception: A critical review of visual masking. Philosophical Transactions of the Royal Society B, 362, 857–875. Luck, S., Vogel, E., & Shapiro, K. (1996). Word meanings can be accessed but not reported during the attentional blink. Nature Scientific Reports, 383, 616–618. Mack, A., & Rock, I. (1998). Inattentional Blindness. Cambridge, MA: MIT Press. Martin, M. G. F. (2006). On being alienated. In T. Gendler & J. Hawthorne (Eds.), Perceptual Experience, 354–410. Oxford: Oxford University Press. Merikle, P., & Reingold, E. (1998). On demonstrating unconscious perception: Comment on Draine and Greenwald (1998). Journal of Experimental Psychology: General, 127(3), 304–310. Moore, G. E. (1925). A defence of common-sense. In J. Muirhead (Ed.), Contemporary British Philosophy (2nd series), 193–223. London: Allen and Unwin. (Reprinted from: Philosophical Papers). Mudrik, L., Breska, A., Lamy, D., & Deouell, L. (2011). Integration without awareness: Expanding the limits of unconscious processing. Psychological Science, 22(6), 764–770. Mudrik, L., Gelbard-Sagiv, H., Faivre, N., & Koch, C. (2013). Knowing where without knowing what: Partial awareness and high level processing in continuous flash suppression. Paper presented at the Vision Science Society conference, Naples, Florida. Mudrik, L., & Koch, C. (2013). Differential processing of invisible congruent and incongruent scenes: A case for unconscious integration. Journal of Vision, 13(13), 1–14. Mulckhuyse, M., & Theeuwes, J. (2010). Unconscious attentional orienting to exogenous cues: A review of the literature. Acta Psychologica, 134, 299–309. Newell, B., & Shanks, D. (2014). Unconscious influences on decision making: A critical review. Behavioral and Brain Sciences, 37(1), 1–19. Norman, L., Akins, K., Heywood, C., & Kentridge, R. (2014). Color constancy for an unseen surface. Current Biology, 24(23), 2822–2826. Phelps, E., Ling, S., & Carrasco, M. (2006). Emotion facilitates perception and potentiates the perceptual benefits of attention. Psychological Science, 17(4), 292–299. Phillips, I. (in press). Consciousness and criterion: On Block’s case for unconscious seeing. Philosophy and Phenomenological Research. Phillips, I. (forthcoming). Unconscious perception reconsidered. Prinz, J. (2010). When is perception conscious? In B. Nanay (Ed.), Perceiving the World, 310–332. Oxford: Oxford University Press. Prinz, J. (2015). Unconscious perception. In M. Matthen (Ed.), Oxford Handbook of Philosophy of Perception, 371–389. Oxford: Oxford University Press. Prinz, J. J. (2012). The Conscious Brain. New York: Oxford University Press. Raio, C., Carmel, D., Carrasco, M., & Phelps, E. A. (2012). Nonconscious fear is quickly acquired but swiftly forgotten. Current Biology, 22(12), 477–479. Rieger, G., & Savin-Williams, R. (2012). The eyes have it: Sex and sexual orientation differences in pupil dilation patterns. PLoS One, 7(8), 1–10. Schiller, P., & Smith, M. (1968). Monoptic and dichoptic metacontrast. Perception & Psychophysics, 3, 237–239. Schmidt, T. (2007). Measuring unconscious cognition: Beyond the zero-awareness criterion. Advances in Cognitive Psychology, 31(1–2), 275–287. Schoeberl, T., Fuchs, I., Theeuwes, J., & Ansorge, U. (2015). Stimulus-driven attentional capture by subliminal onset cues. Attention, Perception & Psychophysics, 77(3), 737–748.

192

•

Ian Phillips and Ned Block

Shanks, D., & Berry, C. (2012). Are there multiple memory systems? Tests of models of implicit and explicit memory. The Quarterly Journal of Experimental Psychology, 65(8), 1449–1474. Sklar, A., Levy, N., Goldstein, A., Mandel, R., Maril, A., & Hassin, R. (2012). Reading and doing arithmetic nonconsciously. Proceedings of the National Academy of Sciences, 109, 19614– 19619. Stein, T., Hebart, M., & Sterzer, P. (2011). Breaking continuous flash suppression: A new measure of unconscious processing during interocular suppression? Frontiers in Human Neuroscience, 5(167), 1–17. Stein, T., & Sterzer, P. (2011). High-level face shape adaptation depends on visual awareness: Evidence from continuous flash suppression. Journal of Vision, 11(8), 1–14. Stein, T., & Sterzer, P. (2014). Unconscious processing under interocular suppression: Getting the right measure. Frontiers in Psychology, 5(387), 1–5. Stein, T., Thoma, V., & Sterzer, P. (2015). Priming of object detection under continuous flash suppression depends on attention but not on part-whole configuration. Journal of Vision, 15(3), 1–11. Sterzer, P., Stein, T., Ludwig, K., Rothkirch, M., & Hesselman, G. (2014). Neural processing of visual information under interocular suppression: A critical review. Frontiers in Psychology, 5(453), 1–12. Tong, F., Nakayama, K., Vaughan, J. T., & Kanwisher, N. (1998). Binocular rivalry and visual awareness in human extrastriate cortex. Neuron, 21(4), 753–759. Travis, C. (2004). The silence of the senses. Mind, 113(449), 57–94. Weiskrantz, L. (1986/2009). Blindsight: A Case Study Spanning 35 Years and New Developments. Oxford: Oxford University Press. Weiskrantz, L. (2002). Prime-sight and blindsight. Consciousness & Cognition, 11(4), 568–581. Wiggins, D. (2001). Sameness and Substance Renewed. Oxford: Oxford University Press. Williams, M., Dang, S., & Kanwisher, N. (2007). Only some spatial patterns of fMRI response are read out in task performance. Nature Neuroscience, 10, 685–686. Yang, E., Brascamp, J., Kang, M.-S., & Blake, R. (2014). On the use of continuous flash suppression for the study of visual processing outside of awareness. Frontiers in Psychology, 5(724), 1–17. Yang, E., Hong, S., & Blake, R. (2010). Adaptation aftereffects to facial expressions suppressed from visual awareness. Journal of Vision, 10(12), 1–13. Yang, E., Zald, D., & Blake, R. (2007). Fearful expressions gain preferential access to awareness during continuous flash suppression. Emotion, 7, 882–886. Zadbood, A., Lee, S., & Blake, R. (2011). Stimulus fractionated by interocular suppression. Frontiers in Human Neuroscience, 5(135), 1–9. Zimba, L., & Blake, R. (1983). Binocular rivalry and semantic processing: Out of sight, out of mind. Journal of Experimental Psychology: Human Perception and Performance, 9, 807–815.

Appendix: Other Controversies in Philosophy of Perception Top-down influences on perception: Can our beliefs or maybe desires influence our perceptual processes? How much informational encapsulation is there in our perceptual system? Perception and action: Does perception carve up the world according to actionpossibilities? What mental states mediate between perception and action? Perceptual content: Does perception have analog/imagistic content? Or does it have propositional content like beliefs? Perception and time: How do we perceive time and durations? Perception and space: How do we perceive space? Is the representation of space a precondition of perception per se? Is the perception of space always of egocentric space? Perception and mental imagery: Do they have similar or different phenomenology? Do they have similar or different content? Is perception in some ways dependent on mental imagery? Are there mixed perception/mental imagery cases? Perceptual justification: Some of our perceptual states can justify our beliefs. Does this put some constraints on what perception is or on how perception represents the world? Picture perception: How does our perceptual system work differently when we see an apple face to face and when we see an apple depicted in a picture?

193

Index

accurate perception 64, 66, 67, 115, 118, 119, 124; and attention 157, 172, 178 acquaintance: as grounding the perceptual relation 24, 29–31, 34, 35, 39, 46–8, 51; with sense data 84, 85, 91 action: and attention 144–6, 152–3; perceiving opportunities for 61; perceptually guided 2, 5 aesthetic properties 28 affordance (perception of) 60–1, 64 agency: as requiring attention; and unconscious perception 166, 169, 174, 178, 181, 186 age properties (perception of) 61 agential properties (perception of) 61, 74 agnosia 154, 156, 157, 160, 167 Akins, K. 168 Alberti, L. 93n2 Alexander, I 166 Allais, D. 123 Allen, K. 28, 40n3, 54n12 Allport, A. 150 Amihai, I. 175 amodal completion 111 amygdala 184 animacy (perception of) 77 Anscombe, G.E.M. 66, 87, 88, 89, 91, 93n1

appraisal and perception 60–1 ‘argument from hypothetical cases’ 29–32 ‘argument from irregular grounding’ 27–9 Armstrong, D. 83, 84, 93n1 atomism see ‘empiricist atomism’ attention 10–12; and overflow 146, 154–9; and selection 141, 143, 145, 149, 150–6; and unconscious perception 173, 174, 177–80, 182, 184–5, 186, 189; as affecting determinacy of perceived properties 11–12, 141–2; as affecting perceived luminance 142; as enhancing neural response 143, 151; as enhancing perceptual precision 141–2, 145, 151–2; as gatekeeping 154–9; for action 144–6, 152–3 attentional blink 154, 168 audition see auditory perception auditory perception 14–16, 27, 48, 51, 83–93, 99–105; function of 103, 105 Axelrod, V. 175 Azzopardi, P. 166, 167 Banaji, M.R. 70 Bar, M. 79n20 Barkai, E. 143 Barwise, J. 78n5

194

Index basilar membrane 100 Batty, C. 14, 106n12 Bayne, T. 9, 10, 16, 78n16, 109, 112 beliefs 7–9, 11, 33, 37–8, 63, 70 Bender, D. B. 139 Bentin, S. 175 Bernstein, L. E. 122 Berry, C. 183 Bertelson, P. 15, 17n2, 112 Biggs, S. 115, 130 Binda, P. 182 binding 13; intermodal 112, 113, 117, 118; see also properties, attribution of binding problem 140 binocular rivalry 170, 175, 180, 182–3, 184 Blake, R. 172–3, 174, 175, 178, 180, 182 blindsight 141–2, 158, 166, 167, 174, 185, 186; and determinable properties 141 Block, N. 5, 10, 23, 32, 70, 71, 74, 77, 141, 148, 149, 151, 154, 157, 160n1, 161n7 Boakes, R. A. 105n6, 105n7 brain in the void (BIV), 33 Braun, J 140 Breckenridge, W. 64, 67 Bregman, A. 90, 106n10 Breitmeyer, B. 180, 182, 183, 184, 186 Brewer, B. 4, 28, 31, 44, 53n4, 176 brightness: as modulated by attention 142, 145; auditory property 89; visual property 117, 132, 142 Briscoe, R. 79n25 Broadbent, D. E. 156, 161n5 Brogaard, B. 3, 11, 78n3, 93n6, 143, 150, 151, 152, 153, 159 Bronfman, Z. Z. 157 Brosch, T. 184 Brouwer, G. 26 Burge, T. 3, 71, 78n3, 78n8, 165, 166, 169, 174, 178, 181, 186 Burr, D. 123 Byrne, A. 4, 9, 48, 53n3, 53n6, 72, 73, 76, 78n6 Campbell, J. 4, 6, 10, 23, 24, 25, 28, 31, 39n3, 44, 48, 49, 50, 53n4, 176, 186 Carey, S. 74, 79n24

•

195

Carmel, D. 180 Carrasco, M. 11, 140, 141, 142, 159, 160, 184 Carter 143 Casati, R. 14, 93n4, 93n5, 93n8, 105n8 Cassam, Q. 24, 31, 39n3 categorical perception 27, 118; see also categorization categorization 27, 66, 118, 154, 180, 189; and gestalt perception 70, 71, 78; ‘highlevel perceptual categorization’ 173–5, 177 causal accounts of content 38 causal grounding of perception see acquaintance, as grounding the perceptual relation causation (perception of) 9, 60–4, 68, 74–5, 77, 112, 118, 123–4 Ceglia, I. 143 Chabris, C. F. 155 Chalmers, D. 8, 35, 36, 40n7, 45, 51–2, 53n3, 159 Chang E. F. 27 Changeux, J. P. 156 Chelazzi, L. et al. 151 Cherry, E. C. 155–6 Clark, A. (Austin) 14 Clark, A. (Andy) 45 co-consciousness see multimodal perception, vs. simple co-consciousness cognitive penetration see top-down influences on perception Cohen, J. 13, 14, 158 color 30–5, 46, 48, 60, 131–2, 157; and determinacy 11; and sense data theories of perception 34; and unconscious perception 166, 186–7; as mind independent 24–8; co-location with shape 131–2; memory-color phenomenon 73–4; pluralist accounts 36–7; primitivism 48–51 common sensibles 116 concepts 74, 111 Connolly, K. 124 consciousness: access 148–9, 156; and attention 12, 110, 141, 148–9, 154–9;

196

•

Index

and relationalism 6; hard problem 159; phenomenal 45, 148–9, 166, 168 content, perceptual 7–10, 36–9, 114; analog vs. digital 8; and attention 11–12; and multimodality 114, 116, 123, 131, 133; and veridicality 44, 64, 66, 69; as imagelike 8; as properties presented 11, 63; causal accounts 38; de se 37; high vs. low see rich vs. thin; propositional 8, 36, 63, 69; rich vs. thin 37, 41n9, 59–78; tracking accounts see causal accounts; vehicles of 38, 54n17 Continuous Flash Suppression (CFS) 170–2, 174–80, 182–9 contrast cases see phenomenal contrast Costello, P. 175 Cowey, A. 166, 167 Crane, T. 3, 8 cross-modal dependence 111, 112 cross-modal effects see cross-modal illusions, cross-modal dependence, cross-modal integration cross-modal illusions 15, 109, 122 cross-modal integration 122, 129–33; as enhancing reliability 115, 123, 130 Dang, S. 181 Davidson, D. 38 de Gelder, B. 17n2 Dehaene, S. 156, 183 Delk, J. L. 70, 78n13 demonstrative element of perception 8 Dennett, D. 11, 158 Deouell, L. 175 depiction 87–8 Desimone, R. 140 desires 8, 37–8 determinacy see attention, as affecting determinacy of perceived properties; color, determinacy; properties, determinacy Di Bona, E. 79n22 dichotic listening 155–6 direction (perception of) 112, 131

disjunctivism 45 distinction between perception and cognition 117–18 Dokic, J. 14, 93n4, 93n5, 93n8, 105n8 dorsal visual subsystem 2, 5, 158 Dosher, B. 185 doxastic representation 43, 53, see also extra-perceptual cognition Draine, S. C. 188 Dretske, F. 7, 8 Driver, J. 15 Dulany, D. 168 Duncan, J. 140 early vision 54n16, 60, 67, 182 edibility (perception of) 63, 64, 72 Eilan, N. 165 emotions (perception of) 61, 182 ‘empirical sufficient condition’ 149–52, 155, 157 ‘empiricist atomism’ 113, 122 Epstein, R. A. 125 Eriksen, C. 168 Evans, G. 131 event perception 11, 77, 86–92, 102–5; see also causation (perception of) explanatory gap 45–7 extended mind thesis 45 extended view see relationalism externalism: and relationalism 24, 31 extra-perceptual cognition 110, 115, 117, 119 extrastriate and striate visual cortex 143 face perception 60, 71, 74, 75, 77 fear conditioning 172 Feldman-Barrett, L. 79n20 Fillenbaum, S. 70, 78n13 Firestone, C 70, 78n14 Fish, W. 23, 25, 39n2, 44, 46, 47, 53n4, 54n9, 54n11, 79n25 flavor 113, 118, 126 Fodor, J. 65 Foley, R. 141

Index French, C. 40n3 frequency components, grouping of 99–100 Fulkerson, M. 126, 132 functionalism 39 Funkhouser, E. 11 GABAergic interneurons 144 Gandhi, S. P. 12 Gao, T. 77 Gauthier, I. 79n26 Gayet, S. 180 Gelade, G. 132 Gelbard-Sagiv, H. 178 Geuss, M. N. 70 Glennon, R. A. 143 global workspace theory 141 Goldstein, A. 27 González-Maeso, J. 143 Goodale, M. A. 5, 158 ‘good internal correlation’ 26–8, 31 greebles 71–2 Green, D. M. 166 Greene, M. R. 79n20 Greenwald, A. 183, 184, 188 Grice, P. 126, 128, 129, 134n7 Gross, C. G. 70, 78n13, 139 grounding see acquaintance, as grounding the perceptual relation Guttman, S. E. 112 Haddock, A. 53n6 hallucination 4–5, 32, 39, 45, 143 hallucinogens 143 hard problem of consciousness see consciousness, hard problem hemispatial neglect 154 high level content see content, perceptual, thin vs. rich Hopkins, R. 93n1, 93n2 Huang, J. 112 Huddleston, W. E. 112 Hume, D. 77 Humphrey, K. G 5 Hyman, J. 93n1, 93n2

•

197

illusions 53n4, 65, 67, 68, 69, 72, 73, 101, 109, 115, 122; see also cross modal illusions illusory conjunctions 140 inattentional blindness 12, 154–9, 168 individual level perception see perception, as attributable to individuals individuation of the senses see modalities, individuation of informational overload 143–4, 150–1 ‘inner state view’, the 32–5 intentionalism see representationalism intermodal: feature binding see multimodal perception, and intermodal feature binding; relations see multimodal perception, and intermodal relations ‘internal dependence’ 23, 35 internalism 28; see also ‘inner state view’, the and ‘representationalism, brain-based’ interpersonal relations (perception of) 61 ‘intersensory discrepancy paradigm’ 109 introspection 3, 12, 13, 15, 122–3, 129, 132 Ioannidis, J. 70, 78n12 Itti, L 140 Jackson, F. 54n14 James, W. 160 Jensen, M. S. 155 Jiang, Y. V. 173–5, 177, 180–2, 184–6 Johnston, M. 47, 49, 53n5 judgement see extra-perceptual cognition; doxastic representation Kanwisher, N. 181 Kelly, S. 9 Kemp, M. 93n2 Kentridge, R. W. 11, 141, 150–3, 159, 160, 166, 168 Kerszberg, M. 156 Kim, R. 115 kinds (perception of), 60 Klauer, K. 189

198

•

Index

Klinger, M. 183 Koch, C. 140, 142, 161n7, 189 Kosslyn, S. 38 Kouider, S. 183 Kriegel, U. 9 Króliczak, G. 5 Kulvicki, J. 8, 14, 92, 93n1, 93n3 Langton, R. 51 ‘laws of appearance’ 36–9 layer V glutamatergic neurons 144 layer V pyramidal neurons 143 Lee, H. M. 140, 144, 178 Levin, D. T. 70 Levine, J. 45–6 Lewis, D. 38, 132 Ling, S. 159, 184 Livingstone, M. 131, 132 Locke, J. 129 Logue, H. 4, 7, 39n3, 40n7, 43, 49, 51, 52, 53n6, 55n18 loudness 24, 27, 28, 88–90, 99, 117 low-level content see content, rich vs. thin Luck, S. 168 Lufti, R. 106n11 luminance see brightness; attention, as affecting perceived luminance Lycan, W. 14, 106n12 MacDonald, J. 122, 133n2 Machery, E. 70, 78n14 Mack, A. 12, 156, 168 Macpherson, F. 53n6, 70, 78n11, 116, 125, 128, 129 Margot, C. 26 Markram, H. et al. 144 Martin, M. G. F. 4, 23, 25, 44, 53n4, 105n1, 105n2, 176 Masrour, F. 9, 41n9 material objects (perception of) 86, 95–6, 101–5 Matthen, M. 8, 9, 14, 16, 109, 110, 111, 113, 115–18, 124, 133n2, 134n7 Maunsell, J. H. R 140 McAdams, C. J. 140

McAdams, S. 106n11 McDowell, J. 5 McGurk, H. 122, 133n2 McLaughlin, B. 23, 32 memory see perception, as affected by past experience memory color phenomenon 73, 78n13 mental paint 141 mental states 43; and relationalism 5, 52; as propositional attitudes 8; unconscious 5, 159 Meredith, M. A. 122, 128, 133n3 Merikle, P. 185 metaphysics of perceptual experience 43–5 Michotte, A. 74, 77 Millar, S. 130 Millikan, R. G. 7 Milner, A. D. 5, 158 modalities, individuation of 116, 125–9 modular processing 65–9, 72–4, 76–7 Molyneux’s question 122, 129–30 Mondrian 171–2, 177 Moore, G.E. 27, 84–9, 91, 165 Moors, P. 180, 182 Moran, J. 140 Morland, A. B. 142 motion (perception of) 59, 112, 166 motor feedback see self-movement, sense of Mudrik, L. et al. 172, 174–6, 180, 185–6, 189 Mulckhuyse, M. 181 Mulligan, K.10 multimodal perception: amodal completion 111; and intermodal feature binding 17n2, 112, 117, 118; and intermodal relations 112–13 116, 124; as dependent on prior or concurrent experience 110, 111, 114, 131; constitutively multisensory experience 111, 113, 118; novel features 112, 115–16, 118; vs. simple co-consciousness 110–15, 119 multiple object tracking paradigm 150

Index multiple realizability see functionalism multisensory perception, see multimodal perception Munhall, K. G 121 Murray, S. 182 music 99, 112, 123 Nadel, L. 125 Nagel, T. 43 naïve realism see relationalism Nanay, B. 7, 9, 10, 11, 17n2, 41n9, 79n19, 79n23, 142 Neisser, U. 155 Neumann, O. 151, 152, 153 Newell, B. 180 Nichols, D. E. 143 Nishimura, S. 77 Noë, A. 93n1 Norman, L. 168, 181 Nudds, M. 14, 89, 92, 93n5, 93n7, 93n8, 105n9, 106n10, 106n15, 112, 128 objective properties see properties, objective objective representation see representation, of the external environment object perception, see material objects (perception of) objects of experience 53, 83, 95, 105; auditory 89, 90, 95; olfactory 95, 97 object view the see relationalism O’Callaghan, C. 14, 15, 16, 86–7, 89, 91, 93n4, 93n5, 93n6, 93n7, 93n8, 105n8, 106n12, 109–11, 122–4, 131, 133n1, 133n2, 134n4 O’Connor, D. H. 12, 143 odours 24–5, 95–9, 105n6, 123, 130; see also olfaction O’Keefe, J. 125 olfaction 14–15, 24–5, 95–6, 101–3, 105, 126; function of 103 olfactory bulb 97 olfactory epithelium 97 Oliva, A. 79n20 Olkkonen, M. 78n13, 79n21

•

199

opportunities for action and interaction (perception of) 61 overflow see attention and overflow pain receptors 126 Papineau, D. 7, 23, 32, 40n5, 40n7 Pashler, H. E. 156 Pasnau, R. 14, 92, 93n5 past experience see perception, as affected by past experience Pautz, A. 4, 6, 23, 25, 28, 33–6, 40n3, 40n5, 40n6, 40n7, 41n8, 48, 51, 53, 54n9, 54n13, 55n18 Peacocke, C. 3, 8 Pendlebury, M. J. 65 perception; as affected by past experience 73, 110–11, 118; as attributable to individuals 165, 166, 174, 178, 179, 181, 184; as ‘externally-directed’ see representation, of the external environment; as different from cognition 60–1, 68, 117–18; conscious vs. unconscious 2, 4–6, 12, 13, 73, 148, 158–9, 165–89; mediate 83, 90–2, 95, 101–5; see also personal level representation perceptual adaptation 77 perceptual constancies 88, 92, 166, 169, 181, 186 perceptual image, the 124–6 perceptual intermediaries see perception, mediate perceptual learning 68, 70–1, 98, 111, 118 perceptual precision see attention, as enhancing perceptual precision perceptual predication relation 35; see also properties, attribution of perceptual relation 46, 102; see also acquaintance, as grounding the perceptual relation personal level representation 44–5, 54n16, 165, 173, 174, 178, 181–4, 187–9 perspectival properties see properties, relational Phelps, E. 184

200

•

Index

phenomenal character 2, 23, 43–53, 75, 126–30, 133, 149, 176; as revelatory 49–52 phenomenal consciousness see consciousness, phenomenal phenomenal contrast 9–10, 68, 70, 71, 74, 75, 77 phenomenal predication 36; see also property attribution Phillips, I. 5, 170, 176, 177, 178, 182, 186 physicalism 33, 45, 48, 52 pluralism see color, pluralist accounts Pollack, I. 121, 122, 133n2 predication of properties see properties, attribution of Presti, D. E. 143 Price, H. H. 84 primary visual cortex 139 priming 168–9, 174, 182–5, 188–9 primitivism see color, primitivism Principal Component Analysis (PCA) 129 Prinz, J. 156, 165, 174, 186 ‘problem of the criterion’ 167 proper sensibles 116–17 properties: as ‘projected’ 35, 40n3; attribution of 7, 9, 11, 34–8; complexes of 35, 36, 70, 141; determinacy 11, 98, 141–2; high-level vs. low level see content, perceptual, rich vs. thin; objective 24, 28, 35, 49; relational 67, 83–4, 87–93; secondary 48, 51; sensible properties 24, 27–8, 35, 38; tropes vs. universals 10 propositional attitudes 8, 53n2, 116 propositional content see content propositional propositions 37, 45, 63–4, 66, 176 proprioceptive systems 123 prosopagnosia 167 pulvinar 186 Pylyshyn, Z. 78n1, 150 Raftopoulos, A. 41n9 Raio, C. 172, 174 Read, S. 159

recognition 68, 70, 71, 72, 88–9 Rees, G. 175 reflexive responses 181, 184, 187 regression method 183 Reid, T. 83 Reingold, E. 185 relationalism 3–7, 10, 23–5, 28–39, 40n7, 44, 45, 46, 47, 48, 50–3, 169, 175 relational view see relationalism representation: of the external environment 23, 32, 34–5, 40n5, 126, 165, 168–9, 174, 184; of space 59, 124–5, 130; neural 26–35, 38, 49, 50, 139, 140 (see also attention, as enhancing neural response); unconscious 182–4, 186, 188; see also attention, and unconscious perception; color and unconscious perception; mental states, unconscious; perception, conscious vs. unconscious representationalism 3–7, 23, 35–6, 39–40, 41n7, 44, 46, 49, 50, 116; brain-based 28; identity 40–1n7; ‘tracking’ 38 representational view see representationalism Richardson, L. 128, 129, 134n8, 134n9 Rieger, G. 181 Rieser, J. 134n10 Rips, L. J. 77, 79n24 Rock, I. 12, 156, 168 Rocha-Miranda, C. E. 139 Rolfs, M. 77 Roth, B. L. 144 Russell, B. 33, 34, 35, 84 Sams, M. 122, 127 Savin-Williams, R. 181 Schellenberg, S. 3, 93n1, 93n6 Schiller, P. 183 Schmidt, T. 168 Schoeberl, T. 181 Scholl, B. J. 9, 70, 77, 78n14, 150 Schuh, E. 183 Schyns, P. G. 78n15 Scruggs, J. L. et al. 143 self-movement, sense of 123, 125

Index sense data theory of perception 34–5, 83–5 sensory individuals 12–16 sensory integration see cross-modal integration serotonin 5-HT1A/2A/2C 143 Shams, L 15, 111, 115 Shanks, D. 180, 183 Shapiro, K. 168 Shefrin, S. L. et al. 141 Shoemaker, S. 40n3 Siegel, S. 3, 9, 63, 64, 66, 69, 70, 71, 72, 76, 77, 78n4, 78n9, 78n10, 78n17, 79n19 Siewert, C. 9, 75, 134n6 signal detection theory (SDT) 167 Simons, D. 155 Sklar, A. 180, 183, 184, 185 smells see odours Smith, B. C. 113, 126 Smith, M. 183 social properties (perception of) 75 somatosensation 113 Sorensen, R. 93n5, 93n8 sounds 14, 83, 87–91, 99–101, 112, 123, 130 see also auditory perception Speaks, J. 116 speech perception 27–8, 121–2 Spence, C. 15, 109, 112 Sperling paradigm 157 Squire, S. 112 Stalnaker, R. 8 Stazicker, J. 11, 151 Stefanucci, J. K. 70 Stein, B. E. 118, 121, 122, 128, 133n3, 175, 176, 177, 180 Sterzer, P. 175, 176, 177, 180, 188 Sterzer, P. E. 182 Stevenson, R. J. 105n3, 105n6, 105n7 Stokes, D. 115, 130 Stone, J. V. 112 Sumby, W. H. 121, 122, 133n2 Swets, J. A. 167 Tarr, M. J 79n26 temporal lobe 139 temporal pole 139

•

201

thalamus 144 Thau, M. 78n2, 78n6 Theeuwes, J. 181 Thoma, V. 177 Tolman, E. C. 125 Tong, F. 170 top-down influences on perception 68, 70, 72, 98–9, 100, 111 Torres-Escalante, J. L. et al. 143 touch 111, 114, 124, 126, 132, 134n5 tracking account of content see content, causal accounts traits (perception of) 60 Travis, C. 4, 169 Treisman, A. M. 132 Tremoulet, P. D. 9, 77 Treue 142 Triesch, J. 79n20 ‘trompe l’oreille’ 91 Tsuchiya, N. 143, 171 Turrell, J. 85, 89 twin earth 29–38 Tye, M. 3, 36, 38, 40n6 unisensory perception 109–11, 113, 123; see also multimodal perception, vs. simple co-consciousness V1 19, 141–2, 166 V4 26, 30, 35 Vass, L. K. 125 Vatikiotis-Bateson, E. 121 ventriloquism 15 veridical perception see content, perceptual, and veridicality vestibular systems 123 visual illusions see illusions visuo-spatial neglect 167, 174 Vogel, E. 168 Vollenweider, F. X. 143 Von Békésy, G. 123 Vroomen, J. 15, 17n2 Warren, D. H. 115 Watkins, S. 15

202

•

Index

Watzl, S. 148 Weiskrantz, L. 141, 158, 166 Welch, R. B. 115 ‘what it is like’ see phenomenal character Wiggins, D. 165 Wikner, K. 134n10 Williams, M. 181 Wilson, D. A. 105n3, 105n6, 105n7 Witzel, C. 78n13 ‘wolfpack effect’ 77 Wozny, D. R. 115

Wu, W. 144, 149, 150, 151, 154, 156, 158, 160n3, 161n7 Yang, E. 172–3, 174, 177, 180 Yeshurun, Y. 11 Yonas, A. 134n10 Zadbood, A. 178 Zald, D. 172–3, 174 Zeimbekis, J. 78n13 Zimba, L. 180, 182